Task 15439197

Name	hadcm3n_z7r6_1920_40_008245233_0
Workunit	8400357
Created	19 Nov 2012, 17:12:13 UTC
Sent	19 Nov 2012, 17:12:17 UTC
Report deadline	19 Feb 2013, 0:39:28 UTC
Received	25 Nov 2012, 0:16:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1036321
Run time	2 days 7 hours 42 min 28 sec
CPU time	2 days 7 hours 42 min 28 sec
Validate state	Invalid
Credit	1,244.16
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:04:55 (3251): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:56 (3251): No heartbeat from core client for 30 sec - exiting 09:04:57 (3251): No heartbeat from core client for 30 sec - exiting 09:04:59 (3251): No heartbeat from core client for 30 sec - exiting 09:05:00 (3251): No heartbeat from core client for 30 sec - exiting 09:05:01 (3251): No heartbeat from core client for 30 sec - exiting 09:05:02 (3251): No heartbeat from core client for 30 sec - exiting 09:05:03 (3251): No heartbeat from core client for 30 sec - exiting 09:05:04 (3251): No heartbeat from core client for 30 sec - exiting 09:05:05 (3251): No heartbeat from core client for 30 sec - exiting 09:05:06 (3251): No heartbeat from core client for 30 sec - exiting 09:05:07 (3251): No heartbeat from core client for 30 sec - exiting 09:05:08 (3251): No heartbeat from core client for 30 sec - exiting 09:05:09 (3251): No heartbeat from core client for 30 sec - exiting 09:05:10 (3251): No heartbeat from core client for 30 sec - exiting 09:05:11 (3251): No heartbeat from core client for 30 sec - exiting 09:05:12 (3251): No heartbeat from core client for 30 sec - exiting 09:05:13 (3251): No heartbeat from core client for 30 sec - exiting 09:05:14 (3251): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:16:10 (3206): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:16:11 (3206): No heartbeat from core client for 30 sec - exiting 11:16:12 (3206): No heartbeat from core client for 30 sec - exiting 11:16:13 (3206): No heartbeat from core client for 30 sec - exiting 11:16:14 (3206): No heartbeat from core client for 30 sec - exiting 11:16:15 (3206): No heartbeat from core client for 30 sec - exiting 11:16:16 (3206): No heartbeat from core client for 30 sec - exiting 11:16:17 (3206): No heartbeat from core client for 30 sec - exiting 11:16:18 (3206): No heartbeat from core client for 30 sec - exiting 11:16:19 (3206): No heartbeat from core client for 30 sec - exiting 11:16:20 (3206): No heartbeat from core client for 30 sec - exiting 11:16:21 (3206): No heartbeat from core client for 30 sec - exiting 11:16:22 (3206): No heartbeat from core client for 30 sec - exiting 11:16:23 (3206): No heartbeat from core client for 30 sec - exiting 11:16:24 (3206): No heartbeat from core client for 30 sec - exiting 11:16:25 (3206): No heartbeat from core client for 30 sec - exiting 11:16:26 (3206): No heartbeat from core client for 30 sec - exiting 11:16:27 (3206): No heartbeat from core client for 30 sec - exiting 11:16:28 (3206): No heartbeat from core client for 30 sec - exiting 11:16:29 (3206): No heartbeat from core client for 30 sec - exiting 11:16:30 (3206): No heartbeat from core client for 30 sec - exiting 11:16:31 (3206): No heartbeat from core client for 30 sec - exiting 11:16:32 (3206): No heartbeat from core client for 30 sec - exiting 11:16:33 (3206): No heartbeat from core client for 30 sec - exiting 11:16:34 (3206): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:13:31 (7292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:32 (7292): No heartbeat from core client for 30 sec - exiting 17:13:33 (7292): No heartbeat from core client for 30 sec - exiting 17:13:34 (7292): No heartbeat from core client for 30 sec - exiting 17:13:35 (7292): No heartbeat from core client for 30 sec - exiting 17:13:36 (7292): No heartbeat from core client for 30 sec - exiting 17:13:37 (7292): No heartbeat from core client for 30 sec - exiting 17:13:38 (7292): No heartbeat from core client for 30 sec - exiting 17:13:39 (7292): No heartbeat from core client for 30 sec - exiting 17:13:40 (7292): No heartbeat from core client for 30 sec - exiting 17:13:41 (7292): No heartbeat from core client for 30 sec - exiting 17:13:42 (7292): No heartbeat from core client for 30 sec - exiting 17:13:43 (7292): No heartbeat from core client for 30 sec - exiting 17:13:44 (7292): No heartbeat from core client for 30 sec - exiting 17:13:45 (7292): No heartbeat from core client for 30 sec - exiting 17:13:46 (7292): No heartbeat from core client for 30 sec - exiting 17:13:47 (7292): No heartbeat from core client for 30 sec - exiting 17:13:48 (7292): No heartbeat from core client for 30 sec - exiting 17:13:49 (7292): No heartbeat from core client for 30 sec - exiting 17:13:50 (7292): No heartbeat from core client for 30 sec - exiting 17:13:51 (7292): No heartbeat from core client for 30 sec - exiting 17:13:53 (7292): No heartbeat from core client for 30 sec - exiting 17:13:54 (7292): No heartbeat from core client for 30 sec - exiting 17:13:55 (7292): No heartbeat from core client for 30 sec - exiting 17:13:56 (7292): No heartbeat from core client for 30 sec - exiting 17:13:57 (7292): No heartbeat from core client for 30 sec - exiting 17:13:58 (7292): No heartbeat from core client for 30 sec - exiting 17:13:59 (7292): No heartbeat from core client for 30 sec - exiting 17:14:00 (7292): No heartbeat from core client for 30 sec - exiting 17:14:01 (7292): No heartbeat from core client for 30 sec - exiting 17:14:02 (7292): No heartbeat from core client for 30 sec - exiting 17:14:03 (7292): No heartbeat from core client for 30 sec - exiting 17:14:04 (7292): No heartbeat from core client for 30 sec - exiting 17:14:05 (7292): No heartbeat from core client for 30 sec - exiting 17:14:06 (7292): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Nov 24 17:14:55 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c44000 - 0x90c48fff /usr/lib/system/libsystem_network.dylib 0x92288000 - 0x92290fff /usr/lib/system/liblaunch.dylib 0x9229e000 - 0x922ccfff /usr/lib/libSystem.B.dylib 0x92402000 - 0x92418fff /usr/lib/system/libxpc.dylib 0x929aa000 - 0x929adfff /usr/lib/system/libcompiler_rt.dylib 0x929bd000 - 0x929bdfff /usr/lib/system/libdnsinfo.dylib 0x92d0c000 - 0x92d10fff /usr/lib/system/libcache.dylib 0x9323a000 - 0x93242fff /usr/lib/system/libunwind.dylib 0x93756000 - 0x93757fff /usr/lib/system/libunc.dylib 0x93ce7000 - 0x93ceffff /usr/lib/system/libcopyfile.dylib 0x9412a000 - 0x9412dfff /usr/lib/system/libmathCommon.A.dylib 0x94ecc000 - 0x94eeafff /usr/lib/system/libsystem_kernel.dylib 0x965e1000 - 0x96624fff /usr/lib/system/libcommonCrypto.dylib 0x971be000 - 0x971edfff /usr/lib/system/libsystem_info.dylib 0x97cf6000 - 0x97cf7fff /usr/lib/system/libsystem_sandbox.dylib 0x98802000 - 0x98807fff /usr/lib/system/libmacho.dylib 0x98d01000 - 0x98d02fff /usr/lib/system/libquarantine.dylib 0x99b74000 - 0x99b74fff /usr/lib/system/libkeymgr.dylib 0x9a47c000 - 0x9a47efff /usr/lib/system/libdyld.dylib 0x9c3f3000 - 0x9c3f4fff /usr/lib/system/libremovefile.dylib 0x9c44c000 - 0x9c44dfff /usr/lib/system/libsystem_blocks.dylib 0x9cb74000 - 0x9cb82fff /usr/lib/system/libdispatch.dylib 0x9cb9e000 - 0x9cc69fff /usr/lib/system/libsystem_c.dylib 0x9cce1000 - 0x9cce8fff /usr/lib/system/libsystem_dnssd.dylib 0x9cce9000 - 0x9ccf0fff /usr/lib/system/libsystem_notify.dylib 0x9cd40000 - 0x9cd4efff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7407, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Nov 24 17:14:57 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c44000 - 0x90c48fff /usr/lib/system/libsystem_network.dylib 0x92288000 - 0x92290fff /usr/lib/system/liblaunch.dylib 0x9229e000 - 0x922ccfff /usr/lib/libSystem.B.dylib 0x92402000 - 0x92418fff /usr/lib/system/libxpc.dylib 0x929aa000 - 0x929adfff /usr/lib/system/libcompiler_rt.dylib 0x929bd000 - 0x929bdfff /usr/lib/system/libdnsinfo.dylib 0x92d0c000 - 0x92d10fff /usr/lib/system/libcache.dylib 0x9323a000 - 0x93242fff /usr/lib/system/libunwind.dylib 0x93756000 - 0x93757fff /usr/lib/system/libunc.dylib 0x93ce7000 - 0x93ceffff /usr/lib/system/libcopyfile.dylib 0x9412a000 - 0x9412dfff /usr/lib/system/libmathCommon.A.dylib 0x94ecc000 - 0x94eeafff /usr/lib/system/libsystem_kernel.dylib 0x965e1000 - 0x96624fff /usr/lib/system/libcommonCrypto.dylib 0x971be000 - 0x971edfff /usr/lib/system/libsystem_info.dylib 0x97cf6000 - 0x97cf7fff /usr/lib/system/libsystem_sandbox.dylib 0x98802000 - 0x98807fff /usr/lib/system/libmacho.dylib 0x98d01000 - 0x98d02fff /usr/lib/system/libquarantine.dylib 0x99b74000 - 0x99b74fff /usr/lib/system/libkeymgr.dylib 0x9a47c000 - 0x9a47efff /usr/lib/system/libdyld.dylib 0x9c3f3000 - 0x9c3f4fff /usr/lib/system/libremovefile.dylib 0x9c44c000 - 0x9c44dfff /usr/lib/system/libsystem_blocks.dylib 0x9cb74000 - 0x9cb82fff /usr/lib/system/libdispatch.dylib 0x9cb9e000 - 0x9cc69fff /usr/lib/system/libsystem_c.dylib 0x9cce1000 - 0x9cce8fff /usr/lib/system/libsystem_dnssd.dylib 0x9cce9000 - 0x9ccf0fff /usr/lib/system/libsystem_notify.dylib 0x9cd40000 - 0x9cd4efff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7407, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Nov 24 17:15:01 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c44000 - 0x90c48fff /usr/lib/system/libsystem_network.dylib 0x92288000 - 0x92290fff /usr/lib/system/liblaunch.dylib 0x9229e000 - 0x922ccfff /usr/lib/libSystem.B.dylib 0x92402000 - 0x92418fff /usr/lib/system/libxpc.dylib 0x929aa000 - 0x929adfff /usr/lib/system/libcompiler_rt.dylib 0x929bd000 - 0x929bdfff /usr/lib/system/libdnsinfo.dylib 0x92d0c000 - 0x92d10fff /usr/lib/system/libcache.dylib 0x9323a000 - 0x93242fff /usr/lib/system/libunwind.dylib 0x93756000 - 0x93757fff /usr/lib/system/libunc.dylib 0x93ce7000 - 0x93ceffff /usr/lib/system/libcopyfile.dylib 0x9412a000 - 0x9412dfff /usr/lib/system/libmathCommon.A.dylib 0x94ecc000 - 0x94eeafff /usr/lib/system/libsystem_kernel.dylib 0x965e1000 - 0x96624fff /usr/lib/system/libcommonCrypto.dylib 0x971be000 - 0x971edfff /usr/lib/system/libsystem_info.dylib 0x97cf6000 - 0x97cf7fff /usr/lib/system/libsystem_sandbox.dylib 0x98802000 - 0x98807fff /usr/lib/system/libmacho.dylib 0x98d01000 - 0x98d02fff /usr/lib/system/libquarantine.dylib 0x99b74000 - 0x99b74fff /usr/lib/system/libkeymgr.dylib 0x9a47c000 - 0x9a47efff /usr/lib/system/libdyld.dylib 0x9c3f3000 - 0x9c3f4fff /usr/lib/system/libremovefile.dylib 0x9c44c000 - 0x9c44dfff /usr/lib/system/libsystem_blocks.dylib 0x9cb74000 - 0x9cb82fff /usr/lib/system/libdispatch.dylib 0x9cb9e000 - 0x9cc69fff /usr/lib/system/libsystem_c.dylib 0x9cce1000 - 0x9cce8fff /usr/lib/system/libsystem_dnssd.dylib 0x9cce9000 - 0x9ccf0fff /usr/lib/system/libsystem_notify.dylib 0x9cd40000 - 0x9cd4efff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7407, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Nov 24 17:15:04 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c44000 - 0x90c48fff /usr/lib/system/libsystem_network.dylib 0x92288000 - 0x92290fff /usr/lib/system/liblaunch.dylib 0x9229e000 - 0x922ccfff /usr/lib/libSystem.B.dylib 0x92402000 - 0x92418fff /usr/lib/system/libxpc.dylib 0x929aa000 - 0x929adfff /usr/lib/system/libcompiler_rt.dylib 0x929bd000 - 0x929bdfff /usr/lib/system/libdnsinfo.dylib 0x92d0c000 - 0x92d10fff /usr/lib/system/libcache.dylib 0x9323a000 - 0x93242fff /usr/lib/system/libunwind.dylib 0x93756000 - 0x93757fff /usr/lib/system/libunc.dylib 0x93ce7000 - 0x93ceffff /usr/lib/system/libcopyfile.dylib 0x9412a000 - 0x9412dfff /usr/lib/system/libmathCommon.A.dylib 0x94ecc000 - 0x94eeafff /usr/lib/system/libsystem_kernel.dylib 0x965e1000 - 0x96624fff /usr/lib/system/libcommonCrypto.dylib 0x971be000 - 0x971edfff /usr/lib/system/libsystem_info.dylib 0x97cf6000 - 0x97cf7fff /usr/lib/system/libsystem_sandbox.dylib 0x98802000 - 0x98807fff /usr/lib/system/libmacho.dylib 0x98d01000 - 0x98d02fff /usr/lib/system/libquarantine.dylib 0x99b74000 - 0x99b74fff /usr/lib/system/libkeymgr.dylib 0x9a47c000 - 0x9a47efff /usr/lib/system/libdyld.dylib 0x9c3f3000 - 0x9c3f4fff /usr/lib/system/libremovefile.dylib 0x9c44c000 - 0x9c44dfff /usr/lib/system/libsystem_blocks.dylib 0x9cb74000 - 0x9cb82fff /usr/lib/system/libdispatch.dylib 0x9cb9e000 - 0x9cc69fff /usr/lib/system/libsystem_c.dylib 0x9cce1000 - 0x9cce8fff /usr/lib/system/libsystem_dnssd.dylib 0x9cce9000 - 0x9ccf0fff /usr/lib/system/libsystem_notify.dylib 0x9cd40000 - 0x9cd4efff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7407, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Nov 24 17:15:07 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c44000 - 0x90c48fff /usr/lib/system/libsystem_network.dylib 0x92288000 - 0x92290fff /usr/lib/system/liblaunch.dylib 0x9229e000 - 0x922ccfff /usr/lib/libSystem.B.dylib 0x92402000 - 0x92418fff /usr/lib/system/libxpc.dylib 0x929aa000 - 0x929adfff /usr/lib/system/libcompiler_rt.dylib 0x929bd000 - 0x929bdfff /usr/lib/system/libdnsinfo.dylib 0x92d0c000 - 0x92d10fff /usr/lib/system/libcache.dylib 0x9323a000 - 0x93242fff /usr/lib/system/libunwind.dylib 0x93756000 - 0x93757fff /usr/lib/system/libunc.dylib 0x93ce7000 - 0x93ceffff /usr/lib/system/libcopyfile.dylib 0x9412a000 - 0x9412dfff /usr/lib/system/libmathCommon.A.dylib 0x94ecc000 - 0x94eeafff /usr/lib/system/libsystem_kernel.dylib 0x965e1000 - 0x96624fff /usr/lib/system/libcommonCrypto.dylib 0x971be000 - 0x971edfff /usr/lib/system/libsystem_info.dylib 0x97cf6000 - 0x97cf7fff /usr/lib/system/libsystem_sandbox.dylib 0x98802000 - 0x98807fff /usr/lib/system/libmacho.dylib 0x98d01000 - 0x98d02fff /usr/lib/system/libquarantine.dylib 0x99b74000 - 0x99b74fff /usr/lib/system/libkeymgr.dylib 0x9a47c000 - 0x9a47efff /usr/lib/system/libdyld.dylib 0x9c3f3000 - 0x9c3f4fff /usr/lib/system/libremovefile.dylib 0x9c44c000 - 0x9c44dfff /usr/lib/system/libsystem_blocks.dylib 0x9cb74000 - 0x9cb82fff /usr/lib/system/libdispatch.dylib 0x9cb9e000 - 0x9cc69fff /usr/lib/system/libsystem_c.dylib 0x9cce1000 - 0x9cce8fff /usr/lib/system/libsystem_dnssd.dylib 0x9cce9000 - 0x9ccf0fff /usr/lib/system/libsystem_notify.dylib 0x9cd40000 - 0x9cd4efff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7407, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Nov 24 17:15:11 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c44000 - 0x90c48fff /usr/lib/system/libsystem_network.dylib 0x92288000 - 0x92290fff /usr/lib/system/liblaunch.dylib 0x9229e000 - 0x922ccfff /usr/lib/libSystem.B.dylib 0x92402000 - 0x92418fff /usr/lib/system/libxpc.dylib 0x929aa000 - 0x929adfff /usr/lib/system/libcompiler_rt.dylib 0x929bd000 - 0x929bdfff /usr/lib/system/libdnsinfo.dylib 0x92d0c000 - 0x92d10fff /usr/lib/system/libcache.dylib 0x9323a000 - 0x93242fff /usr/lib/system/libunwind.dylib 0x93756000 - 0x93757fff /usr/lib/system/libunc.dylib 0x93ce7000 - 0x93ceffff /usr/lib/system/libcopyfile.dylib 0x9412a000 - 0x9412dfff /usr/lib/system/libmathCommon.A.dylib 0x94ecc000 - 0x94eeafff /usr/lib/system/libsystem_kernel.dylib 0x965e1000 - 0x96624fff /usr/lib/system/libcommonCrypto.dylib 0x971be000 - 0x971edfff /usr/lib/system/libsystem_info.dylib 0x97cf6000 - 0x97cf7fff /usr/lib/system/libsystem_sandbox.dylib 0x98802000 - 0x98807fff /usr/lib/system/libmacho.dylib 0x98d01000 - 0x98d02fff /usr/lib/system/libquarantine.dylib 0x99b74000 - 0x99b74fff /usr/lib/system/libkeymgr.dylib 0x9a47c000 - 0x9a47efff /usr/lib/system/libdyld.dylib 0x9c3f3000 - 0x9c3f4fff /usr/lib/system/libremovefile.dylib 0x9c44c000 - 0x9c44dfff /usr/lib/system/libsystem_blocks.dylib 0x9cb74000 - 0x9cb82fff /usr/lib/system/libdispatch.dylib 0x9cb9e000 - 0x9cc69fff /usr/lib/system/libsystem_c.dylib 0x9cce1000 - 0x9cce8fff /usr/lib/system/libsystem_dnssd.dylib 0x9cce9000 - 0x9ccf0fff /usr/lib/system/libsystem_notify.dylib 0x9cd40000 - 0x9cd4efff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7407, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Nov 2012 01:26:44	1036321	15439197	hadcm3n_z7r6_1920_40_008245233_0	103,680	163,442	1.5764
22 Nov 2012 17:41:38	1036321	15439197	hadcm3n_z7r6_1920_40_008245233_0	77,760	122,655	1.5774
21 Nov 2012 17:29:47	1036321	15439197	hadcm3n_z7r6_1920_40_008245233_0	51,840	82,005	1.5819
20 Nov 2012 17:32:39	1036321	15439197	hadcm3n_z7r6_1920_40_008245233_0	25,920	40,939	1.5794