Task 18699426

Name	hadcm3n_sdfp_1940_40_010020338_0
Workunit	10018409
Created	14 Jul 2015, 13:24:02 UTC
Sent	14 Jul 2015, 13:53:26 UTC
Report deadline	13 Oct 2015, 21:20:37 UTC
Received	20 Jul 2015, 4:37:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1368471
Run time	4 days 17 hours 16 min 50 sec
CPU time	4 days 14 hours 9 min 9 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.36 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:15:35 (341): No heartbeat from core client for 30 sec - exiting 23:16:00 (341): No heartbeat from core client for 30 sec - exiting 23:16:01 (341): No heartbeat from core client for 30 sec - exiting 23:16:02 (341):CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 05:08:11 (335): No heartbeat from core client for 30 sec - exiting 05:08:52 (335): No heartbeat from core client for 30 sec - exiting 05:08:53 (335): No heartbeat from core client for 30 sec - exiting 05:08:54 (335): No heartbeat from core client for 30 sec - exiting 05:08:55 (335): No heartbeat from core client for 30 sec - exiting 05:08:56 (335): No heartbeat from core client for 30 sec - exiting 05:08:57 (335): No heartbeat from core client for 30 sec - exiting 05:08:58 (335): No heartbeat from core client for 30 sec - exiting 05:08:59 (335): No heartbeat from core client for 30 sec - exiting 05:09:00 (335): No heartbeat from core client for 30 sec - exiting 05:09:01 (335): No heartbeat from core client for 30 sec - exiting 05:09:02 (335): No heartbeat from core client for 30 sec - exiting 05:09:03 (335): No heartbeat from core client for 30 sec - exiting 05:09:04 (335): No heartbeat from core client for 30 sec - exiting 05:09:05 (335): No heartbeat from core client for 30 sec - exiting 05:09:06 (335): No heartbeat from core client for 30 sec - exiting 05:09:07 (335): No heartbeat from core client for 30 sec - exiting 05:09:08 (335): No heartbeat from core client for 30 sec - exiting 05:09:09 (335): No heartbeat from core client for 30 sec - exiting 05:09:10 (335): No heartbeat from core client for 30 sec - exiting 05:09:11 (335): No heartbeat from core client for 30 sec - exiting 05:09:12 (335): No heartbeat from core client for 30 sec - exiting 05:09:13 (335): No heartbeat from core client for 30 sec - exiting 05:09:14 (335): No heartbeat from core client for 30 sec - exiting 05:09:15 (335): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(358,0xa113c1a8) malloc: * error for object 0x391ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.9.5 build 13F1096 Sun Jul 19 20:52:42 2015 Thread 0 Crashed: Thread 1: -- Warning: /usr/bin/atos is moving and will be removed from a future OS X release. It is now available in the Xcode developer tools to be invoked via: `xcrun atos` To silence this warning, pass the '-d' command-line flag to this tool. -- atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fae8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/7/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x906b9000 - 0x906bafff /usr/lib/system/libremovefile.dylib 0x906c3000 - 0x90713fff /usr/lib/system/libcorecrypto.dylib 0x909a4000 - 0x909a5fff /usr/lib/system/libunc.dylib 0x90a42000 - 0x90a66fff /usr/lib/system/libxpc.dylib 0x90fb9000 - 0x90fbdfff /usr/lib/system/libmacho.dylib 0x913f8000 - 0x913f9fff /usr/lib/libSystem.B.dylib 0x91413000 - 0x91421fff /usr/lib/libz.1.dylib 0x9162b000 - 0x916bdfff /usr/lib/system/libsystem_c.dylib 0x916be000 - 0x916c9fff /usr/lib/system/libcommonCrypto.dylib 0x91bc5000 - 0x91be2fff /usr/lib/system/libsystem_kernel.dylib 0x92cd2000 - 0x92cd4fff /usr/lib/system/libquarantine.dylib 0x92fa6000 - 0x92facfff /usr/lib/system/libunwind.dylib 0x930e0000 - 0x930e8fff /usr/lib/system/libsystem_dnssd.dylib 0x9319a000 - 0x931ebfff /usr/lib/libstdc++.6.dylib 0x9346d000 - 0x9346efff /usr/lib/system/libsystem_sandbox.dylib 0x9346f000 - 0x93477fff /usr/lib/system/libcopyfile.dylib 0x93d1f000 - 0x93d21fff /usr/lib/system/libsystem_configuration.dylib 0x945a4000 - 0x945a4fff /usr/lib/system/libkeymgr.dylib 0x954df000 - 0x954e4fff /usr/lib/system/libcompiler_rt.dylib 0x95b8f000 - 0x95bbafff /usr/lib/system/libsystem_network.dylib 0x95f30000 - 0x95f33fff /usr/lib/system/libdyld.dylib 0x96103000 - 0x9610afff /usr/lib/system/libsystem_pthread.dylib 0x96c5a000 - 0x96c8bfff /usr/lib/system/libsystem_m.dylib 0x971f4000 - 0x9721cfff /usr/lib/system/libsystem_info.dylib 0x973eb000 - 0x973f0fff /usr/lib/system/libsystem_platform.dylib 0x973f1000 - 0x97409fff /usr/lib/system/libdispatch.dylib 0x97b69000 - 0x97b7bfff /usr/lib/system/libsystem_asl.dylib 0x983bb000 - 0x983d3fff /usr/lib/system/libsystem_malloc.dylib 0x98672000 - 0x9867afff /usr/lib/system/liblaunch.dylib 0x9901a000 - 0x99023fff /usr/lib/system/libsystem_notify.dylib 0x99024000 - 0x99028fff /usr/lib/system/libcache.dylib 0x9b212000 - 0x9b236fff /usr/lib/libc++abi.dylib 0x9b859000 - 0x9b85afff /usr/lib/system/libsystem_blocks.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Jul 2015 04:39:27	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	259,200	396,546	1.5299
19 Jul 2015 19:14:04	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	233,280	362,240	1.5528
19 Jul 2015 04:49:19	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	207,360	327,001	1.5770
18 Jul 2015 17:35:04	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	181,440	286,594	1.5796
18 Jul 2015 03:33:45	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	155,520	245,490	1.5785
17 Jul 2015 16:09:07	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	129,600	204,996	1.5818
16 Jul 2015 23:43:43	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	103,680	163,567	1.5776
16 Jul 2015 12:09:20	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	77,760	122,785	1.5790
15 Jul 2015 23:56:21	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	51,840	81,466	1.5715
15 Jul 2015 12:38:28	1368471	18699426	hadcm3n_sdfp_1940_40_010020338_0	25,920	41,034	1.5831