Task 15457688

Name	hadcm3n_z9jv_1880_40_008252507_2
Workunit	8407631
Created	23 Nov 2012, 15:13:42 UTC
Sent	23 Nov 2012, 15:13:45 UTC
Report deadline	22 Feb 2013, 22:40:56 UTC
Received	30 Nov 2012, 23:00:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1110299
Run time	7 days 4 hours 49 min 22 sec
CPU time	4 days 20 hours 3 min
Validate state	Invalid
Credit	2,177.28
Device peak FLOPS	2.93 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:18:56 (6409): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:18:58 (6409): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:06:27 (8888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:06:28 (8888): No heartbeat from core client for 30 sec - exiting 04:06:29 (8888): No heartbeat from core client for 30 sec - exiting 12:02:38 (16645): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:04:56 (25071): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:04:58 (25071): No heartbeat from core client for 30 sec - exiting 14:04:59 (25071): No heartbeat from core client for 30 sec - exiting 14:05:00 (25071): No heartbeat from core client for 30 sec - exiting 14:05:01 (25071): No heartbeat from core client for 30 sec - exiting 14:05:02 (25071): No heartbeat from core client for 30 sec - exiting 14:05:03 (25071): No heartbeat from core client for 30 sec - exiting 14:05:04 (25071): No heartbeat from core client for 30 sec - exiting 18:28:31 (27106): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:16:32 (31415): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:42:08 (36147): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:36:17 (45686): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:03:15 (290): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:57 (12201): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:25:35 (14976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:20:51 (18175): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:55:56 (26132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:55:59 (26132): No heartbeat from core client for 30 sec - exiting 14:56:00 (26132): No heartbeat from core client for 30 sec - exiting 14:56:01 (26132): No heartbeat from core client for 30 sec - exiting 14:56:02 (26132): No heartbeat from core client for 30 sec - exiting 14:56:03 (26132): No heartbeat from core client for 30 sec - exiting 14:56:04 (26132): No heartbeat from core client for 30 sec - exiting 14:56:05 (26132): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:47:44 (67650): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:36:28 (11152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:26:56 (21495): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Fri Nov 30 22:27:39 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90da2000 - 0x90db0fff /usr/lib/libz.1.dylib 0x91f4c000 - 0x91f50fff /usr/lib/system/libsystem_network.dylib 0x923b1000 - 0x923b2fff /usr/lib/system/libremovefile.dylib 0x92880000 - 0x92881fff /usr/lib/system/libunc.dylib 0x93188000 - 0x93253fff /usr/lib/system/libsystem_c.dylib 0x93917000 - 0x9391afff /usr/lib/system/libcompiler_rt.dylib 0x94391000 - 0x94391fff /usr/lib/system/libdnsinfo.dylib 0x94c98000 - 0x94cc7fff /usr/lib/system/libsystem_info.dylib 0x94cc9000 - 0x94d0cfff /usr/lib/system/libcommonCrypto.dylib 0x96597000 - 0x9659ffff /usr/lib/system/liblaunch.dylib 0x96650000 - 0x96666fff /usr/lib/system/libxpc.dylib 0x9740b000 - 0x97412fff /usr/lib/system/libsystem_dnssd.dylib 0x9744f000 - 0x97451fff /usr/lib/system/libdyld.dylib 0x984cb000 - 0x984ccfff /usr/lib/system/libquarantine.dylib 0x98e73000 - 0x98e91fff /usr/lib/system/libsystem_kernel.dylib 0x9a7b0000 - 0x9a7b4fff /usr/lib/system/libcache.dylib 0x9a908000 - 0x9a909fff /usr/lib/system/libsystem_sandbox.dylib 0x9a912000 - 0x9a915fff /usr/lib/system/libmathCommon.A.dylib 0x9ac38000 - 0x9ac40fff /usr/lib/system/libunwind.dylib 0x9ade4000 - 0x9ae12fff /usr/lib/libSystem.B.dylib 0x9b10b000 - 0x9b10cfff /usr/lib/system/libsystem_blocks.dylib 0x9c2f9000 - 0x9c301fff /usr/lib/system/libcopyfile.dylib 0x9c33e000 - 0x9c345fff /usr/lib/system/libsystem_notify.dylib 0x9c796000 - 0x9c796fff /usr/lib/system/libkeymgr.dylib 0x9ceff000 - 0x9cf0dfff /usr/lib/system/libdispatch.dylib 0x9cf2c000 - 0x9cf31fff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28500, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Fri Nov 30 22:27:44 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90da2000 - 0x90db0fff /usr/lib/libz.1.dylib 0x91f4c000 - 0x91f50fff /usr/lib/system/libsystem_network.dylib 0x923b1000 - 0x923b2fff /usr/lib/system/libremovefile.dylib 0x92880000 - 0x92881fff /usr/lib/system/libunc.dylib 0x93188000 - 0x93253fff /usr/lib/system/libsystem_c.dylib 0x93917000 - 0x9391afff /usr/lib/system/libcompiler_rt.dylib 0x94391000 - 0x94391fff /usr/lib/system/libdnsinfo.dylib 0x94c98000 - 0x94cc7fff /usr/lib/system/libsystem_info.dylib 0x94cc9000 - 0x94d0cfff /usr/lib/system/libcommonCrypto.dylib 0x96597000 - 0x9659ffff /usr/lib/system/liblaunch.dylib 0x96650000 - 0x96666fff /usr/lib/system/libxpc.dylib 0x9740b000 - 0x97412fff /usr/lib/system/libsystem_dnssd.dylib 0x9744f000 - 0x97451fff /usr/lib/system/libdyld.dylib 0x984cb000 - 0x984ccfff /usr/lib/system/libquarantine.dylib 0x98e73000 - 0x98e91fff /usr/lib/system/libsystem_kernel.dylib 0x9a7b0000 - 0x9a7b4fff /usr/lib/system/libcache.dylib 0x9a908000 - 0x9a909fff /usr/lib/system/libsystem_sandbox.dylib 0x9a912000 - 0x9a915fff /usr/lib/system/libmathCommon.A.dylib 0x9ac38000 - 0x9ac40fff /usr/lib/system/libunwind.dylib 0x9ade4000 - 0x9ae12fff /usr/lib/libSystem.B.dylib 0x9b10b000 - 0x9b10cfff /usr/lib/system/libsystem_blocks.dylib 0x9c2f9000 - 0x9c301fff /usr/lib/system/libcopyfile.dylib 0x9c33e000 - 0x9c345fff /usr/lib/system/libsystem_notify.dylib 0x9c796000 - 0x9c796fff /usr/lib/system/libkeymgr.dylib 0x9ceff000 - 0x9cf0dfff /usr/lib/system/libdispatch.dylib 0x9cf2c000 - 0x9cf31fff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28500, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Fri Nov 30 22:27:47 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90da2000 - 0x90db0fff /usr/lib/libz.1.dylib 0x91f4c000 - 0x91f50fff /usr/lib/system/libsystem_network.dylib 0x923b1000 - 0x923b2fff /usr/lib/system/libremovefile.dylib 0x92880000 - 0x92881fff /usr/lib/system/libunc.dylib 0x93188000 - 0x93253fff /usr/lib/system/libsystem_c.dylib 0x93917000 - 0x9391afff /usr/lib/system/libcompiler_rt.dylib 0x94391000 - 0x94391fff /usr/lib/system/libdnsinfo.dylib 0x94c98000 - 0x94cc7fff /usr/lib/system/libsystem_info.dylib 0x94cc9000 - 0x94d0cfff /usr/lib/system/libcommonCrypto.dylib 0x96597000 - 0x9659ffff /usr/lib/system/liblaunch.dylib 0x96650000 - 0x96666fff /usr/lib/system/libxpc.dylib 0x9740b000 - 0x97412fff /usr/lib/system/libsystem_dnssd.dylib 0x9744f000 - 0x97451fff /usr/lib/system/libdyld.dylib 0x984cb000 - 0x984ccfff /usr/lib/system/libquarantine.dylib 0x98e73000 - 0x98e91fff /usr/lib/system/libsystem_kernel.dylib 0x9a7b0000 - 0x9a7b4fff /usr/lib/system/libcache.dylib 0x9a908000 - 0x9a909fff /usr/lib/system/libsystem_sandbox.dylib 0x9a912000 - 0x9a915fff /usr/lib/system/libmathCommon.A.dylib 0x9ac38000 - 0x9ac40fff /usr/lib/system/libunwind.dylib 0x9ade4000 - 0x9ae12fff /usr/lib/libSystem.B.dylib 0x9b10b000 - 0x9b10cfff /usr/lib/system/libsystem_blocks.dylib 0x9c2f9000 - 0x9c301fff /usr/lib/system/libcopyfile.dylib 0x9c33e000 - 0x9c345fff /usr/lib/system/libsystem_notify.dylib 0x9c796000 - 0x9c796fff /usr/lib/system/libkeymgr.dylib 0x9ceff000 - 0x9cf0dfff /usr/lib/system/libdispatch.dylib 0x9cf2c000 - 0x9cf31fff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28500, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Fri Nov 30 22:27:50 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90da2000 - 0x90db0fff /usr/lib/libz.1.dylib 0x91f4c000 - 0x91f50fff /usr/lib/system/libsystem_network.dylib 0x923b1000 - 0x923b2fff /usr/lib/system/libremovefile.dylib 0x92880000 - 0x92881fff /usr/lib/system/libunc.dylib 0x93188000 - 0x93253fff /usr/lib/system/libsystem_c.dylib 0x93917000 - 0x9391afff /usr/lib/system/libcompiler_rt.dylib 0x94391000 - 0x94391fff /usr/lib/system/libdnsinfo.dylib 0x94c98000 - 0x94cc7fff /usr/lib/system/libsystem_info.dylib 0x94cc9000 - 0x94d0cfff /usr/lib/system/libcommonCrypto.dylib 0x96597000 - 0x9659ffff /usr/lib/system/liblaunch.dylib 0x96650000 - 0x96666fff /usr/lib/system/libxpc.dylib 0x9740b000 - 0x97412fff /usr/lib/system/libsystem_dnssd.dylib 0x9744f000 - 0x97451fff /usr/lib/system/libdyld.dylib 0x984cb000 - 0x984ccfff /usr/lib/system/libquarantine.dylib 0x98e73000 - 0x98e91fff /usr/lib/system/libsystem_kernel.dylib 0x9a7b0000 - 0x9a7b4fff /usr/lib/system/libcache.dylib 0x9a908000 - 0x9a909fff /usr/lib/system/libsystem_sandbox.dylib 0x9a912000 - 0x9a915fff /usr/lib/system/libmathCommon.A.dylib 0x9ac38000 - 0x9ac40fff /usr/lib/system/libunwind.dylib 0x9ade4000 - 0x9ae12fff /usr/lib/libSystem.B.dylib 0x9b10b000 - 0x9b10cfff /usr/lib/system/libsystem_blocks.dylib 0x9c2f9000 - 0x9c301fff /usr/lib/system/libcopyfile.dylib 0x9c33e000 - 0x9c345fff /usr/lib/system/libsystem_notify.dylib 0x9c796000 - 0x9c796fff /usr/lib/system/libkeymgr.dylib 0x9ceff000 - 0x9cf0dfff /usr/lib/system/libdispatch.dylib 0x9cf2c000 - 0x9cf31fff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28500, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Fri Nov 30 22:27:52 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90da2000 - 0x90db0fff /usr/lib/libz.1.dylib 0x91f4c000 - 0x91f50fff /usr/lib/system/libsystem_network.dylib 0x923b1000 - 0x923b2fff /usr/lib/system/libremovefile.dylib 0x92880000 - 0x92881fff /usr/lib/system/libunc.dylib 0x93188000 - 0x93253fff /usr/lib/system/libsystem_c.dylib 0x93917000 - 0x9391afff /usr/lib/system/libcompiler_rt.dylib 0x94391000 - 0x94391fff /usr/lib/system/libdnsinfo.dylib 0x94c98000 - 0x94cc7fff /usr/lib/system/libsystem_info.dylib 0x94cc9000 - 0x94d0cfff /usr/lib/system/libcommonCrypto.dylib 0x96597000 - 0x9659ffff /usr/lib/system/liblaunch.dylib 0x96650000 - 0x96666fff /usr/lib/system/libxpc.dylib 0x9740b000 - 0x97412fff /usr/lib/system/libsystem_dnssd.dylib 0x9744f000 - 0x97451fff /usr/lib/system/libdyld.dylib 0x984cb000 - 0x984ccfff /usr/lib/system/libquarantine.dylib 0x98e73000 - 0x98e91fff /usr/lib/system/libsystem_kernel.dylib 0x9a7b0000 - 0x9a7b4fff /usr/lib/system/libcache.dylib 0x9a908000 - 0x9a909fff /usr/lib/system/libsystem_sandbox.dylib 0x9a912000 - 0x9a915fff /usr/lib/system/libmathCommon.A.dylib 0x9ac38000 - 0x9ac40fff /usr/lib/system/libunwind.dylib 0x9ade4000 - 0x9ae12fff /usr/lib/libSystem.B.dylib 0x9b10b000 - 0x9b10cfff /usr/lib/system/libsystem_blocks.dylib 0x9c2f9000 - 0x9c301fff /usr/lib/system/libcopyfile.dylib 0x9c33e000 - 0x9c345fff /usr/lib/system/libsystem_notify.dylib 0x9c796000 - 0x9c796fff /usr/lib/system/libkeymgr.dylib 0x9ceff000 - 0x9cf0dfff /usr/lib/system/libdispatch.dylib 0x9cf2c000 - 0x9cf31fff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28500, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Fri Nov 30 22:27:55 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90da2000 - 0x90db0fff /usr/lib/libz.1.dylib 0x91f4c000 - 0x91f50fff /usr/lib/system/libsystem_network.dylib 0x923b1000 - 0x923b2fff /usr/lib/system/libremovefile.dylib 0x92880000 - 0x92881fff /usr/lib/system/libunc.dylib 0x93188000 - 0x93253fff /usr/lib/system/libsystem_c.dylib 0x93917000 - 0x9391afff /usr/lib/system/libcompiler_rt.dylib 0x94391000 - 0x94391fff /usr/lib/system/libdnsinfo.dylib 0x94c98000 - 0x94cc7fff /usr/lib/system/libsystem_info.dylib 0x94cc9000 - 0x94d0cfff /usr/lib/system/libcommonCrypto.dylib 0x96597000 - 0x9659ffff /usr/lib/system/liblaunch.dylib 0x96650000 - 0x96666fff /usr/lib/system/libxpc.dylib 0x9740b000 - 0x97412fff /usr/lib/system/libsystem_dnssd.dylib 0x9744f000 - 0x97451fff /usr/lib/system/libdyld.dylib 0x984cb000 - 0x984ccfff /usr/lib/system/libquarantine.dylib 0x98e73000 - 0x98e91fff /usr/lib/system/libsystem_kernel.dylib 0x9a7b0000 - 0x9a7b4fff /usr/lib/system/libcache.dylib 0x9a908000 - 0x9a909fff /usr/lib/system/libsystem_sandbox.dylib 0x9a912000 - 0x9a915fff /usr/lib/system/libmathCommon.A.dylib 0x9ac38000 - 0x9ac40fff /usr/lib/system/libunwind.dylib 0x9ade4000 - 0x9ae12fff /usr/lib/libSystem.B.dylib 0x9b10b000 - 0x9b10cfff /usr/lib/system/libsystem_blocks.dylib 0x9c2f9000 - 0x9c301fff /usr/lib/system/libcopyfile.dylib 0x9c33e000 - 0x9c345fff /usr/lib/system/libsystem_notify.dylib 0x9c796000 - 0x9c796fff /usr/lib/system/libkeymgr.dylib 0x9ceff000 - 0x9cf0dfff /usr/lib/system/libdispatch.dylib 0x9cf2c000 - 0x9cf31fff /usr/lib/system/libmacho.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28500, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 Nov 2012 04:20:51	1110299	15457688	hadcm3n_z9jv_1880_40_008252507_2	181,440	369,078	2.0342
29 Nov 2012 06:27:01	1110299	15457688	hadcm3n_z9jv_1880_40_008252507_2	155,520	319,582	2.0549
28 Nov 2012 08:11:42	1110299	15457688	hadcm3n_z9jv_1880_40_008252507_2	129,600	271,000	2.0910
27 Nov 2012 10:54:05	1110299	15457688	hadcm3n_z9jv_1880_40_008252507_2	103,680	222,460	2.1456
26 Nov 2012 13:26:44	1110299	15457688	hadcm3n_z9jv_1880_40_008252507_2	77,760	174,317	2.2417
25 Nov 2012 16:01:54	1110299	15457688	hadcm3n_z9jv_1880_40_008252507_2	51,840	116,735	2.2518
24 Nov 2012 16:10:07	1110299	15457688	hadcm3n_z9jv_1880_40_008252507_2	25,920	58,137	2.2429