Task 15478218

Name	hadcm3n_o5gf_2100_40_008254261_2
Workunit	8409385
Created	14 Dec 2012, 13:50:26 UTC
Sent	14 Dec 2012, 13:50:27 UTC
Report deadline	15 Mar 2013, 21:17:38 UTC
Received	6 Jan 2013, 0:18:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1155438
Run time	21 days 6 hours 21 min 5 sec
CPU time	19 days 1 hours 44 min 48 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.91 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:35:40 (8178): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:41 (8178): No heartbeat from core client for 30 sec - exiting 11:23:29 (81123): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:06:16 (98196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:38:33 (13334): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:16:14 (53239): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:33:46 (76574): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:33:15 (26131): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:22:36 (31006): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:27:57 (31088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0x4a32a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0x4a32a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0xb262804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0xb262800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0xb262804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0xb262800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0xb262804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0xb262800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(82147,0xac1912c0) malloc: * error for object 0xb262800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(24946,0xac1912c0) malloc: * error for object 0x2331e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24946,0xac1912c0) malloc: * error for object 0x2331e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24946,0xac1912c0) malloc: * error for object 0x1329e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Jan 5 13:49:17 2013 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fd88 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/25/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x902b7000 - 0x902b8fff /usr/lib/system/libquarantine.dylib 0x90732000 - 0x90739fff /usr/lib/system/libsystem_notify.dylib 0x9073a000 - 0x9073dfff /usr/lib/system/libmathCommon.A.dylib 0x908fe000 - 0x908fefff /usr/lib/system/libkeymgr.dylib 0x9148d000 - 0x91495fff /usr/lib/system/liblaunch.dylib 0x9157f000 - 0x9164afff /usr/lib/system/libsystem_c.dylib 0x92827000 - 0x92835fff /usr/lib/libz.1.dylib 0x9294b000 - 0x92961fff /usr/lib/system/libxpc.dylib 0x93237000 - 0x9323bfff /usr/lib/system/libsystem_network.dylib 0x93291000 - 0x9329ffff /usr/lib/system/libdispatch.dylib 0x932a0000 - 0x932cffff /usr/lib/system/libsystem_info.dylib 0x932d0000 - 0x932d2fff /usr/lib/system/libdyld.dylib 0x93ac6000 - 0x93ae4fff /usr/lib/system/libsystem_kernel.dylib 0x93ae6000 - 0x93aedfff /usr/lib/system/libsystem_dnssd.dylib 0x93d9c000 - 0x93dfefff /usr/lib/libstdc++.6.dylib 0x94d2c000 - 0x94d2ffff /usr/lib/system/libcompiler_rt.dylib 0x972cd000 - 0x972cefff /usr/lib/system/libunc.dylib 0x97b6e000 - 0x97b72fff /usr/lib/system/libcache.dylib 0x97f34000 - 0x97f62fff /usr/lib/libSystem.B.dylib 0x9863f000 - 0x98640fff /usr/lib/system/libsystem_sandbox.dylib 0x98f9b000 - 0x98fa3fff /usr/lib/system/libcopyfile.dylib 0x992b2000 - 0x992f5fff /usr/lib/system/libcommonCrypto.dylib 0x9a612000 - 0x9a613fff /usr/lib/system/libremovefile.dylib 0x9b6f5000 - 0x9b6f6fff /usr/lib/system/libsystem_blocks.dylib 0x9b6fb000 - 0x9b703fff /usr/lib/system/libunwind.dylib 0x9b7b9000 - 0x9b7b9fff /usr/lib/system/libdnsinfo.dylib 0x9cb5d000 - 0x9cb62fff /usr/lib/system/libmacho.dylib 0x9cf28000 - 0x9cf31fff /usr/lib/libc++abi.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Jan 2013 11:26:22	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	777,600	1,631,621	2.0983
04 Jan 2013 17:21:38	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	751,680	1,578,483	2.0999
03 Jan 2013 23:53:05	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	725,760	1,524,756	2.1009
03 Jan 2013 05:15:11	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	699,840	1,466,653	2.0957
02 Jan 2013 11:55:04	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	673,920	1,408,902	2.0906
01 Jan 2013 14:25:43	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	648,000	1,352,658	2.0874
31 Dec 2012 20:57:51	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	622,080	1,297,603	2.0859
31 Dec 2012 03:32:22	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	596,160	1,242,288	2.0838
30 Dec 2012 05:28:36	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	570,240	1,186,223	2.0802
29 Dec 2012 08:45:21	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	544,320	1,128,394	2.0730
28 Dec 2012 14:30:47	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	518,400	1,072,042	2.0680
27 Dec 2012 21:21:14	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	492,480	1,024,769	2.0808
27 Dec 2012 02:23:02	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	466,560	965,316	2.0690
26 Dec 2012 08:13:49	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	440,640	909,496	2.0640
25 Dec 2012 14:00:34	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	414,720	853,513	2.0580
24 Dec 2012 20:02:45	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	388,800	796,534	2.0487
24 Dec 2012 01:54:42	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	362,880	738,455	2.0350
23 Dec 2012 06:07:30	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	336,960	683,591	2.0287
22 Dec 2012 10:14:44	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	311,040	628,126	2.0194
21 Dec 2012 17:51:30	1155438	15478218	hadcm3n_o5gf_2100_40_008254261_2	285,120	575,202	2.0174