Task 12606624

Name	famous_ua5l_1799_200_006673336_6
Workunit	6876589
Created	20 Feb 2011, 17:03:16 UTC
Sent	20 Feb 2011, 17:03:18 UTC
Report deadline	23 May 2011, 0:30:29 UTC
Received	26 Feb 2011, 21:50:21 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1126940
Run time	18 hours 24 min 21 sec
CPU time	14 hours 2 min 17 sec
Validate state	Invalid
Credit	988.30
Device peak FLOPS	2.87 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Mon Feb 21 20:10:48 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x989e146b 0x989e146b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc33c edx: 0x9897b0fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc378 esp: 0xbfffc33c ss: 0x0000001f efl: 0x00000206 eip: 0x9897b0fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libirc.dylib 0x9639d000 - 0x963a0fff /usr/lib/system/libmathCommon.A.dylib 0x9897a000 - 0x98b21fff /usr/lib/libSystem.B.dylib 0x99a48000 - 0x99a56fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=47873, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Wed Feb 23 03:19:16 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x989e146b 0x989e146b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc33c edx: 0x9897b0fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc378 esp: 0xbfffc33c ss: 0x0000001f efl: 0x00000206 eip: 0x9897b0fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libirc.dylib 0x9639d000 - 0x963a0fff /usr/lib/system/libmathCommon.A.dylib 0x9897a000 - 0x98b21fff /usr/lib/libSystem.B.dylib 0x99a48000 - 0x99a56fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4429, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (13243): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13290): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13331): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13374): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13415): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13462): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13503): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (13568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (13688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13761): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13806): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13895): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13937): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13986): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14025): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14073): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14135): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14227): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14251): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14337): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14378): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14425): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14509): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14550): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14597): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (14638): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14681): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14722): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14814): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14859): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14898): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Sun Feb 27 04:24:43 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x989e146b 0x989e146b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc33c edx: 0x9897b0fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc378 esp: 0xbfffc33c ss: 0x0000001f efl: 0x00000206 eip: 0x9897b0fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libirc.dylib 0x9639d000 - 0x963a0fff /usr/lib/system/libmathCommon.A.dylib 0x9897a000 - 0x98b21fff /usr/lib/libSystem.B.dylib 0x99a48000 - 0x99a56fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14944, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... (14956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14997): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15038): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15127): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15174): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15215): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Sun Feb 27 04:41:20 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x989e146b 0x989e146b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x003f5826 um_submodel_init (in famous_um_6.11_i686-apple-darwin) + 406 7 0x00450bd8 um_shell (in famous_um_6.11_i686-apple-darwin) + 3520 8 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 9 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 10 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd3cc edx: 0x9897b0fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd408 esp: 0xbfffd3cc ss: 0x0000001f efl: 0x00000206 eip: 0x9897b0fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libirc.dylib 0x9639d000 - 0x963a0fff /usr/lib/system/libmathCommon.A.dylib 0x9897a000 - 0x98b21fff /usr/lib/libSystem.B.dylib 0x99a48000 - 0x99a56fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15260, iMonCtr=1 Model crash detected, will try to restart... (15260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15305): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15350): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15391): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Sun Feb 27 04:50:50 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x989e146b 0x989e146b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc33c edx: 0x9897b0fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc378 esp: 0xbfffc33c ss: 0x0000001f efl: 0x00000206 eip: 0x9897b0fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libirc.dylib 0x9639d000 - 0x963a0fff /usr/lib/system/libmathCommon.A.dylib 0x9897a000 - 0x98b21fff /usr/lib/libSystem.B.dylib 0x99a48000 - 0x99a56fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15436, iMonCtr=1 Model crash detected, will try to restart... (15436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15481): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15567): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15613): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15654): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15701): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15785): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15826): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Sun Feb 27 05:14:35 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x989e146b 0x989e146b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc33c edx: 0x9897b0fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc378 esp: 0xbfffc33c ss: 0x0000001f efl: 0x00000206 eip: 0x9897b0fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libirc.dylib 0x9639d000 - 0x963a0fff /usr/lib/system/libmathCommon.A.dylib 0x9897a000 - 0x98b21fff /usr/lib/libSystem.B.dylib 0x99a48000 - 0x99a56fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15877, iMonCtr=1 Model crash detected, will try to restart... (15877): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15922): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15965): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16006): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16053): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16139): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (16183): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16233): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16275): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16359): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16405): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16447): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16533): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Sun Feb 27 05:52:34 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x989e146b 0x989e146b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc33c edx: 0x9897b0fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc378 esp: 0xbfffc33c ss: 0x0000001f efl: 0x00000206 eip: 0x9897b0fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_ua5l_1799_200_006673336/lib/libirc.dylib 0x9639d000 - 0x963a0fff /usr/lib/system/libmathCommon.A.dylib 0x9897a000 - 0x98b21fff /usr/lib/libSystem.B.dylib 0x99a48000 - 0x99a56fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16576, iMonCtr=1 Model crash detected, will try to restart... (16576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16623): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16709): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16754): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16795): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16881): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16926): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17015): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17101): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17143): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17191): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (17230): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17277): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17322): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17409): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17581): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17665): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17753): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Feb 2011 21:50:59	1126940	12606624	famous_ua5l_1799_200_006673336_6	299,546	50,543	0.1687
26 Feb 2011 21:50:59	1126940	12606624	famous_ua5l_1799_200_006673336_6	290,186	50,544	0.1742
26 Feb 2011 21:50:59	1126940	12606624	famous_ua5l_1799_200_006673336_6	280,826	50,543	0.1800
22 Feb 2011 23:00:15	1126940	12606624	famous_ua5l_1799_200_006673336_6	271,466	49,455	0.1822
22 Feb 2011 20:10:24	1126940	12606624	famous_ua5l_1799_200_006673336_6	262,106	47,707	0.1820
22 Feb 2011 20:09:14	1126940	12606624	famous_ua5l_1799_200_006673336_6	252,746	45,985	0.1819
22 Feb 2011 20:04:43	1126940	12606624	famous_ua5l_1799_200_006673336_6	243,386	44,258	0.1818
22 Feb 2011 16:10:11	1126940	12606624	famous_ua5l_1799_200_006673336_6	234,026	42,503	0.1816
22 Feb 2011 15:29:28	1126940	12606624	famous_ua5l_1799_200_006673336_6	224,666	40,757	0.1814
21 Feb 2011 20:27:46	1126940	12606624	famous_ua5l_1799_200_006673336_6	215,306	39,027	0.1813
21 Feb 2011 19:52:33	1126940	12606624	famous_ua5l_1799_200_006673336_6	205,946	37,357	0.1814
21 Feb 2011 19:21:37	1126940	12606624	famous_ua5l_1799_200_006673336_6	196,586	35,690	0.1815
21 Feb 2011 19:20:17	1126940	12606624	famous_ua5l_1799_200_006673336_6	187,226	34,026	0.1817
21 Feb 2011 18:53:56	1126940	12606624	famous_ua5l_1799_200_006673336_6	177,866	32,361	0.1819
21 Feb 2011 18:51:03	1126940	12606624	famous_ua5l_1799_200_006673336_6	168,506	30,694	0.1822
21 Feb 2011 17:12:50	1126940	12606624	famous_ua5l_1799_200_006673336_6	159,146	28,967	0.1820
21 Feb 2011 17:12:50	1126940	12606624	famous_ua5l_1799_200_006673336_6	149,786	27,221	0.1817
21 Feb 2011 17:12:50	1126940	12606624	famous_ua5l_1799_200_006673336_6	140,426	25,479	0.1814
21 Feb 2011 17:12:50	1126940	12606624	famous_ua5l_1799_200_006673336_6	131,066	23,749	0.1812
21 Feb 2011 17:12:50	1126940	12606624	famous_ua5l_1799_200_006673336_6	121,706	22,087	0.1815