Task 14850487

Name	hadcm3n_o5aj_2020_40_008026743_1
Workunit	8181857
Created	30 Jun 2012, 15:30:01 UTC
Sent	30 Jun 2012, 15:30:04 UTC
Report deadline	29 Sep 2012, 22:57:15 UTC
Received	3 Jul 2012, 23:30:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1202614
Run time	2 days 19 hours 54 min 8 sec
CPU time	2 days 3 hours 50 min 24 sec
Validate state	Invalid
Credit	1,555.20
Device peak FLOPS	3.86 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.35</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:33:36 (17800): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:37:52 (32662): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:37:54 (32662): No heartbeat from core client for 30 sec - exiting 22:37:55 (32662): No heartbeat from core client for 30 sec - exiting 22:37:56 (32662): No heartbeat from core client for 30 sec - exiting 22:37:57 (32662): No heartbeat from core client for 30 sec - exiting 22:37:58 (32662): No heartbeat from core client for 30 sec - exiting 22:37:59 (32662): No heartbeat from core client for 30 sec - exiting 22:38:00 (32662): No heartbeat from core client for 30 sec - exiting 22:38:01 (32662): No heartbeat from core client for 30 sec - exiting 22:38:02 (32662): No heartbeat from core client for 30 sec - exiting 22:38:03 (32662): No heartbeat from core client for 30 sec - exiting 22:38:04 (32662): No heartbeat from core client for 30 sec - exiting 22:38:05 (32662): No heartbeat from core client for 30 sec - exiting 22:38:06 (32662): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 Tue Jul 3 22:45:58 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9074b000 - 0x9077afff /usr/lib/system/libsystem_info.dylib 0x91069000 - 0x9106afff /usr/lib/system/libremovefile.dylib 0x91231000 - 0x91238fff /usr/lib/system/libsystem_dnssd.dylib 0x91eba000 - 0x91ebbfff /usr/lib/system/libsystem_sandbox.dylib 0x92be9000 - 0x92bf1fff /usr/lib/system/liblaunch.dylib 0x938ca000 - 0x938cbfff /usr/lib/system/libsystem_blocks.dylib 0x93ca2000 - 0x93ca2fff /usr/lib/system/libkeymgr.dylib 0x93d2b000 - 0x93d6efff /usr/lib/system/libcommonCrypto.dylib 0x940ac000 - 0x940b4fff /usr/lib/system/libunwind.dylib 0x940b5000 - 0x940d3fff /usr/lib/system/libsystem_kernel.dylib 0x94264000 - 0x9427afff /usr/lib/system/libxpc.dylib 0x94587000 - 0x94588fff /usr/lib/system/libquarantine.dylib 0x94589000 - 0x9458cfff /usr/lib/system/libmathCommon.A.dylib 0x94c24000 - 0x94c28fff /usr/lib/system/libcache.dylib 0x95883000 - 0x95883fff /usr/lib/system/libdnsinfo.dylib 0x95e21000 - 0x95e23fff /usr/lib/system/libdyld.dylib 0x961f3000 - 0x96201fff /usr/lib/system/libdispatch.dylib 0x96d35000 - 0x96d3dfff /usr/lib/system/libcopyfile.dylib 0x9720d000 - 0x972d8fff /usr/lib/system/libsystem_c.dylib 0x98eff000 - 0x98f02fff /usr/lib/system/libcompiler_rt.dylib 0x99b77000 - 0x99b7bfff /usr/lib/system/libsystem_network.dylib 0x9a675000 - 0x9a6a3fff /usr/lib/libSystem.B.dylib 0x9b8a5000 - 0x9b8a6fff /usr/lib/system/libunc.dylib 0x9bf95000 - 0x9bfa3fff /usr/lib/libz.1.dylib 0x9bfa4000 - 0x9bfabfff /usr/lib/system/libsystem_notify.dylib 0x9c077000 - 0x9c07cfff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=33624, iMonCtr=1 Model crash detected, will try to restart... 22:47:32 (33624): No heartbeat from core client for 30 sec - exiting 22:47:33 (33624): No heartbeat from core client for 30 sec - exiting 22:47:34 (33624): No heartbeat from core client for 30 sec - exiting 22:47:35 (33624): No heartbeat from core client for 30 sec - exiting 22:47:36 (33624): No heartbeat from core client for 30 sec - exiting 22:47:37 (33624): No heartbeat from core client for 30 sec - exiting 22:47:38 (33624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 Tue Jul 3 22:53:45 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9074b000 - 0x9077afff /usr/lib/system/libsystem_info.dylib 0x91069000 - 0x9106afff /usr/lib/system/libremovefile.dylib 0x91231000 - 0x91238fff /usr/lib/system/libsystem_dnssd.dylib 0x91eba000 - 0x91ebbfff /usr/lib/system/libsystem_sandbox.dylib 0x92be9000 - 0x92bf1fff /usr/lib/system/liblaunch.dylib 0x938ca000 - 0x938cbfff /usr/lib/system/libsystem_blocks.dylib 0x93ca2000 - 0x93ca2fff /usr/lib/system/libkeymgr.dylib 0x93d2b000 - 0x93d6efff /usr/lib/system/libcommonCrypto.dylib 0x940ac000 - 0x940b4fff /usr/lib/system/libunwind.dylib 0x940b5000 - 0x940d3fff /usr/lib/system/libsystem_kernel.dylib 0x94264000 - 0x9427afff /usr/lib/system/libxpc.dylib 0x94587000 - 0x94588fff /usr/lib/system/libquarantine.dylib 0x94589000 - 0x9458cfff /usr/lib/system/libmathCommon.A.dylib 0x94c24000 - 0x94c28fff /usr/lib/system/libcache.dylib 0x95883000 - 0x95883fff /usr/lib/system/libdnsinfo.dylib 0x95e21000 - 0x95e23fff /usr/lib/system/libdyld.dylib 0x961f3000 - 0x96201fff /usr/lib/system/libdispatch.dylib 0x96d35000 - 0x96d3dfff /usr/lib/system/libcopyfile.dylib 0x9720d000 - 0x972d8fff /usr/lib/system/libsystem_c.dylib 0x98eff000 - 0x98f02fff /usr/lib/system/libcompiler_rt.dylib 0x99b77000 - 0x99b7bfff /usr/lib/system/libsystem_network.dylib 0x9a675000 - 0x9a6a3fff /usr/lib/libSystem.B.dylib 0x9b8a5000 - 0x9b8a6fff /usr/lib/system/libunc.dylib 0x9bf95000 - 0x9bfa3fff /usr/lib/libz.1.dylib 0x9bfa4000 - 0x9bfabfff /usr/lib/system/libsystem_notify.dylib 0x9c077000 - 0x9c07cfff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=33753, iMonCtr=1 Model crash detected, will try to restart... 22:56:22 (33753): No heartbeat from core client for 30 sec - exiting 22:56:26 (33753): No heartbeat from core client for 30 sec - exiting 22:56:27 (33753): No heartbeat from core client for 30 sec - exiting 22:56:28 (33753): No heartbeat from core client for 30 sec - exiting 22:56:29 (33753): No heartbeat from core client for 30 sec - exiting 22:56:30 (33753): No heartbeat from core client for 30 sec - exiting 22:56:31 (33753): No heartbeat from core client for 30 sec - exiting 22:56:32 (33753): No heartbeat from core client for 30 sec - exiting 22:56:33 (33753): No heartbeat from core client for 30 sec - exiting 22:56:34 (33753): No heartbeat from core client for 30 sec - exiting 22:56:35 (33753): No heartbeat from core client for 30 sec - exiting 22:56:36 (33753): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 Tue Jul 3 23:00:40 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9074b000 - 0x9077afff /usr/lib/system/libsystem_info.dylib 0x91069000 - 0x9106afff /usr/lib/system/libremovefile.dylib 0x91231000 - 0x91238fff /usr/lib/system/libsystem_dnssd.dylib 0x91eba000 - 0x91ebbfff /usr/lib/system/libsystem_sandbox.dylib 0x92be9000 - 0x92bf1fff /usr/lib/system/liblaunch.dylib 0x938ca000 - 0x938cbfff /usr/lib/system/libsystem_blocks.dylib 0x93ca2000 - 0x93ca2fff /usr/lib/system/libkeymgr.dylib 0x93d2b000 - 0x93d6efff /usr/lib/system/libcommonCrypto.dylib 0x940ac000 - 0x940b4fff /usr/lib/system/libunwind.dylib 0x940b5000 - 0x940d3fff /usr/lib/system/libsystem_kernel.dylib 0x94264000 - 0x9427afff /usr/lib/system/libxpc.dylib 0x94587000 - 0x94588fff /usr/lib/system/libquarantine.dylib 0x94589000 - 0x9458cfff /usr/lib/system/libmathCommon.A.dylib 0x94c24000 - 0x94c28fff /usr/lib/system/libcache.dylib 0x95883000 - 0x95883fff /usr/lib/system/libdnsinfo.dylib 0x95e21000 - 0x95e23fff /usr/lib/system/libdyld.dylib 0x961f3000 - 0x96201fff /usr/lib/system/libdispatch.dylib 0x96d35000 - 0x96d3dfff /usr/lib/system/libcopyfile.dylib 0x9720d000 - 0x972d8fff /usr/lib/system/libsystem_c.dylib 0x98eff000 - 0x98f02fff /usr/lib/system/libcompiler_rt.dylib 0x99b77000 - 0x99b7bfff /usr/lib/system/libsystem_network.dylib 0x9a675000 - 0x9a6a3fff /usr/lib/libSystem.B.dylib 0x9b8a5000 - 0x9b8a6fff /usr/lib/system/libunc.dylib 0x9bf95000 - 0x9bfa3fff /usr/lib/libz.1.dylib 0x9bfa4000 - 0x9bfabfff /usr/lib/system/libsystem_notify.dylib 0x9c077000 - 0x9c07cfff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=33856, iMonCtr=1 Model crash detected, will try to restart... 23:08:31 (33856): No heartbeat from core client for 30 sec - exiting 23:08:34 (33856): No heartbeat from core client for 30 sec - exiting 23:08:35 (33856): No heartbeat from core client for 30 sec - exiting 23:08:36 (33856): No heartbeat from core client for 30 sec - exiting 23:08:37 (33856): No heartbeat from core client for 30 sec - exiting 23:08:38 (33856): No heartbeat from core client for 30 sec - exiting 23:08:39 (33856): No heartbeat from core client for 30 sec - exiting 23:08:40 (33856): No heartbeat from core client for 30 sec - exiting 23:08:41 (33856): No heartbeat from core client for 30 sec - exiting 23:08:42 (33856): No heartbeat from core client for 30 sec - exiting 23:08:43 (33856): No heartbeat from core client for 30 sec - exiting 23:08:44 (33856): No heartbeat from core client for 30 sec - exiting 23:08:45 (33856): No heartbeat from core client for 30 sec - exiting 23:08:46 (33856): No heartbeat from core client for 30 sec - exiting 23:08:47 (33856): No heartbeat from core client for 30 sec - exiting 23:08:48 (33856): No heartbeat from core client for 30 sec - exiting 23:08:49 (33856): No heartbeat from core client for 30 sec - exiting 23:08:50 (33856): No heartbeat from core client for 30 sec - exiting 23:08:51 (33856): No heartbeat from core client for 30 sec - exiting 23:08:52 (33856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 Tue Jul 3 23:13:04 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9074b000 - 0x9077afff /usr/lib/system/libsystem_info.dylib 0x91069000 - 0x9106afff /usr/lib/system/libremovefile.dylib 0x91231000 - 0x91238fff /usr/lib/system/libsystem_dnssd.dylib 0x91eba000 - 0x91ebbfff /usr/lib/system/libsystem_sandbox.dylib 0x92be9000 - 0x92bf1fff /usr/lib/system/liblaunch.dylib 0x938ca000 - 0x938cbfff /usr/lib/system/libsystem_blocks.dylib 0x93ca2000 - 0x93ca2fff /usr/lib/system/libkeymgr.dylib 0x93d2b000 - 0x93d6efff /usr/lib/system/libcommonCrypto.dylib 0x940ac000 - 0x940b4fff /usr/lib/system/libunwind.dylib 0x940b5000 - 0x940d3fff /usr/lib/system/libsystem_kernel.dylib 0x94264000 - 0x9427afff /usr/lib/system/libxpc.dylib 0x94587000 - 0x94588fff /usr/lib/system/libquarantine.dylib 0x94589000 - 0x9458cfff /usr/lib/system/libmathCommon.A.dylib 0x94c24000 - 0x94c28fff /usr/lib/system/libcache.dylib 0x95883000 - 0x95883fff /usr/lib/system/libdnsinfo.dylib 0x95e21000 - 0x95e23fff /usr/lib/system/libdyld.dylib 0x961f3000 - 0x96201fff /usr/lib/system/libdispatch.dylib 0x96d35000 - 0x96d3dfff /usr/lib/system/libcopyfile.dylib 0x9720d000 - 0x972d8fff /usr/lib/system/libsystem_c.dylib 0x98eff000 - 0x98f02fff /usr/lib/system/libcompiler_rt.dylib 0x99b77000 - 0x99b7bfff /usr/lib/system/libsystem_network.dylib 0x9a675000 - 0x9a6a3fff /usr/lib/libSystem.B.dylib 0x9b8a5000 - 0x9b8a6fff /usr/lib/system/libunc.dylib 0x9bf95000 - 0x9bfa3fff /usr/lib/libz.1.dylib 0x9bfa4000 - 0x9bfabfff /usr/lib/system/libsystem_notify.dylib 0x9c077000 - 0x9c07cfff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=33986, iMonCtr=1 Model crash detected, will try to restart... 23:16:44 (33986): No heartbeat from core client for 30 sec - exiting 23:16:45 (33986): No heartbeat from core client for 30 sec - exiting 23:16:46 (33986): No heartbeat from core client for 30 sec - exiting 23:16:47 (33986): No heartbeat from core client for 30 sec - exiting 23:16:48 (33986): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 Tue Jul 3 23:19:50 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9074b000 - 0x9077afff /usr/lib/system/libsystem_info.dylib 0x91069000 - 0x9106afff /usr/lib/system/libremovefile.dylib 0x91231000 - 0x91238fff /usr/lib/system/libsystem_dnssd.dylib 0x91eba000 - 0x91ebbfff /usr/lib/system/libsystem_sandbox.dylib 0x92be9000 - 0x92bf1fff /usr/lib/system/liblaunch.dylib 0x938ca000 - 0x938cbfff /usr/lib/system/libsystem_blocks.dylib 0x93ca2000 - 0x93ca2fff /usr/lib/system/libkeymgr.dylib 0x93d2b000 - 0x93d6efff /usr/lib/system/libcommonCrypto.dylib 0x940ac000 - 0x940b4fff /usr/lib/system/libunwind.dylib 0x940b5000 - 0x940d3fff /usr/lib/system/libsystem_kernel.dylib 0x94264000 - 0x9427afff /usr/lib/system/libxpc.dylib 0x94587000 - 0x94588fff /usr/lib/system/libquarantine.dylib 0x94589000 - 0x9458cfff /usr/lib/system/libmathCommon.A.dylib 0x94c24000 - 0x94c28fff /usr/lib/system/libcache.dylib 0x95883000 - 0x95883fff /usr/lib/system/libdnsinfo.dylib 0x95e21000 - 0x95e23fff /usr/lib/system/libdyld.dylib 0x961f3000 - 0x96201fff /usr/lib/system/libdispatch.dylib 0x96d35000 - 0x96d3dfff /usr/lib/system/libcopyfile.dylib 0x9720d000 - 0x972d8fff /usr/lib/system/libsystem_c.dylib 0x98eff000 - 0x98f02fff /usr/lib/system/libcompiler_rt.dylib 0x99b77000 - 0x99b7bfff /usr/lib/system/libsystem_network.dylib 0x9a675000 - 0x9a6a3fff /usr/lib/libSystem.B.dylib 0x9b8a5000 - 0x9b8a6fff /usr/lib/system/libunc.dylib 0x9bf95000 - 0x9bfa3fff /usr/lib/libz.1.dylib 0x9bfa4000 - 0x9bfabfff /usr/lib/system/libsystem_notify.dylib 0x9c077000 - 0x9c07cfff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34075, iMonCtr=1 Model crash detected, will try to restart... 23:22:34 (34075): No heartbeat from core client for 30 sec - exiting 23:22:37 (34075): No heartbeat from core client for 30 sec - exiting 23:22:38 (34075): No heartbeat from core client for 30 sec - exiting 23:22:39 (34075): No heartbeat from core client for 30 sec - exiting 23:22:40 (34075): No heartbeat from core client for 30 sec - exiting 23:22:41 (34075): No heartbeat from core client for 30 sec - exiting 23:22:42 (34075): No heartbeat from core client for 30 sec - exiting 23:22:43 (34075): No heartbeat from core client for 30 sec - exiting 23:22:44 (34075): No heartbeat from core client for 30 sec - exiting 23:22:45 (34075): No heartbeat from core client for 30 sec - exiting 23:22:46 (34075): No heartbeat from core client for 30 sec - exiting 23:22:47 (34075): No heartbeat from core client for 30 sec - exiting 23:22:48 (34075): No heartbeat from core client for 30 sec - exiting 23:22:49 (34075): No heartbeat from core client for 30 sec - exiting 23:22:50 (34075): No heartbeat from core client for 30 sec - exiting 23:22:51 (34075): No heartbeat from core client for 30 sec - exiting 23:22:52 (34075): No heartbeat from core client for 30 sec - exiting 23:22:53 (34075): No heartbeat from core client for 30 sec - exiting 23:22:54 (34075): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 Tue Jul 3 23:27:40 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcd08 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9074b000 - 0x9077afff /usr/lib/system/libsystem_info.dylib 0x91069000 - 0x9106afff /usr/lib/system/libremovefile.dylib 0x91231000 - 0x91238fff /usr/lib/system/libsystem_dnssd.dylib 0x91eba000 - 0x91ebbfff /usr/lib/system/libsystem_sandbox.dylib 0x92be9000 - 0x92bf1fff /usr/lib/system/liblaunch.dylib 0x938ca000 - 0x938cbfff /usr/lib/system/libsystem_blocks.dylib 0x93ca2000 - 0x93ca2fff /usr/lib/system/libkeymgr.dylib 0x93d2b000 - 0x93d6efff /usr/lib/system/libcommonCrypto.dylib 0x940ac000 - 0x940b4fff /usr/lib/system/libunwind.dylib 0x940b5000 - 0x940d3fff /usr/lib/system/libsystem_kernel.dylib 0x94264000 - 0x9427afff /usr/lib/system/libxpc.dylib 0x94587000 - 0x94588fff /usr/lib/system/libquarantine.dylib 0x94589000 - 0x9458cfff /usr/lib/system/libmathCommon.A.dylib 0x94c24000 - 0x94c28fff /usr/lib/system/libcache.dylib 0x95883000 - 0x95883fff /usr/lib/system/libdnsinfo.dylib 0x95e21000 - 0x95e23fff /usr/lib/system/libdyld.dylib 0x961f3000 - 0x96201fff /usr/lib/system/libdispatch.dylib 0x96d35000 - 0x96d3dfff /usr/lib/system/libcopyfile.dylib 0x9720d000 - 0x972d8fff /usr/lib/system/libsystem_c.dylib 0x98eff000 - 0x98f02fff /usr/lib/system/libcompiler_rt.dylib 0x99b77000 - 0x99b7bfff /usr/lib/system/libsystem_network.dylib 0x9a675000 - 0x9a6a3fff /usr/lib/libSystem.B.dylib 0x9b8a5000 - 0x9b8a6fff /usr/lib/system/libunc.dylib 0x9bf95000 - 0x9bfa3fff /usr/lib/libz.1.dylib 0x9bfa4000 - 0x9bfabfff /usr/lib/system/libsystem_notify.dylib 0x9c077000 - 0x9c07cfff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34161, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Jul 2012 10:18:20	1202614	14850487	hadcm3n_o5aj_2020_40_008026743_1	129,600	158,832	1.2256
02 Jul 2012 17:16:41	1202614	14850487	hadcm3n_o5aj_2020_40_008026743_1	103,680	125,654	1.2119
02 Jul 2012 12:01:04	1202614	14850487	hadcm3n_o5aj_2020_40_008026743_1	77,760	94,173	1.2111
02 Jul 2012 12:01:04	1202614	14850487	hadcm3n_o5aj_2020_40_008026743_1	51,840	63,185	1.2188
02 Jul 2012 12:01:04	1202614	14850487	hadcm3n_o5aj_2020_40_008026743_1	25,920	32,069	1.2372