Task 16018795

Name	hadcm3n_o5et_1940_40_008411133_1
Workunit	8561989
Created	16 Sep 2013, 3:49:10 UTC
Sent	16 Sep 2013, 4:23:34 UTC
Report deadline	16 Dec 2013, 11:50:45 UTC
Received	25 Oct 2013, 5:50:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1289096
Run time	23 days 6 hours 48 min 9 sec
CPU time	20 days 6 hours 45 min 3 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x2bdc404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x2bdc400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x43f1804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x440ce04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x440ce00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x440ce04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x440ce00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x13f0804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4659,0xacbef2c0) malloc: * error for object 0x13f0800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x308a404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x2867404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x4066404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x2063404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x308a404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x304fc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x3050a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x847c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13032,0xacbef2c0) malloc: * error for object 0x847c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(19209,0xacbef2c0) malloc: * error for object 0x1185604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19209,0xacbef2c0) malloc: * error for object 0x1185600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Thu Oct 24 22:07:08 2013 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fa18 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/8/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90207000 - 0x90209fff /usr/lib/system/libdyld.dylib 0x902e3000 - 0x90326fff /usr/lib/system/libcommonCrypto.dylib 0x90fad000 - 0x90faefff /usr/lib/system/libunc.dylib 0x910e7000 - 0x910e8fff /usr/lib/system/libremovefile.dylib 0x913d3000 - 0x913d7fff /usr/lib/system/libsystem_network.dylib 0x9165e000 - 0x9167cfff /usr/lib/system/libsystem_kernel.dylib 0x9375e000 - 0x9375efff /usr/lib/system/libdnsinfo.dylib 0x93bcc000 - 0x93bcdfff /usr/lib/system/libsystem_blocks.dylib 0x93bcf000 - 0x93bfefff /usr/lib/system/libsystem_info.dylib 0x954e9000 - 0x9554bfff /usr/lib/libstdc++.6.dylib 0x96760000 - 0x96765fff /usr/lib/system/libmacho.dylib 0x97181000 - 0x9724cfff /usr/lib/system/libsystem_c.dylib 0x97e86000 - 0x97e87fff /usr/lib/system/libquarantine.dylib 0x989b8000 - 0x989bffff /usr/lib/system/libsystem_notify.dylib 0x98e7e000 - 0x98e7efff /usr/lib/system/libkeymgr.dylib 0x98fac000 - 0x98fb4fff /usr/lib/system/libunwind.dylib 0x99325000 - 0x9932efff /usr/lib/libc++abi.dylib 0x9a01b000 - 0x9a023fff /usr/lib/system/liblaunch.dylib 0x9a85e000 - 0x9a86cfff /usr/lib/libz.1.dylib 0x9a87a000 - 0x9a882fff /usr/lib/system/libcopyfile.dylib 0x9a883000 - 0x9a88afff /usr/lib/system/libsystem_dnssd.dylib 0x9b6d8000 - 0x9b6dbfff /usr/lib/system/libmathCommon.A.dylib 0x9b7aa000 - 0x9b7aefff /usr/lib/system/libcache.dylib 0x9c0e6000 - 0x9c0f4fff /usr/lib/system/libdispatch.dylib 0x9c101000 - 0x9c117fff /usr/lib/system/libxpc.dylib 0x9c9f1000 - 0x9ca1ffff /usr/lib/libSystem.B.dylib 0x9ca20000 - 0x9ca23fff /usr/lib/system/libcompiler_rt.dylib 0x9ca5e000 - 0x9ca5ffff /usr/lib/system/libsystem_sandbox.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Oct 2013 05:51:49	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	1,036,800	1,752,298	1.6901
24 Oct 2013 06:28:01	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	1,010,880	1,713,614	1.6952
23 Oct 2013 16:43:43	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	984,960	1,673,632	1.6992
22 Oct 2013 07:27:32	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	959,040	1,630,388	1.7000
21 Oct 2013 10:13:53	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	933,120	1,589,497	1.7034
20 Oct 2013 19:53:39	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	907,200	1,547,659	1.7060
20 Oct 2013 07:28:50	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	881,280	1,508,132	1.7113
19 Oct 2013 16:01:00	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	855,360	1,463,609	1.7111
19 Oct 2013 16:01:00	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	829,440	1,417,607	1.7091
18 Oct 2013 06:24:42	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	803,520	1,376,230	1.7128
17 Oct 2013 10:19:12	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	777,600	1,335,432	1.7174
16 Oct 2013 16:29:49	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	751,680	1,289,806	1.7159
16 Oct 2013 03:32:51	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	725,760	1,249,975	1.7223
15 Oct 2013 03:33:06	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	699,840	1,207,679	1.7257
14 Oct 2013 01:28:34	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	673,920	1,166,220	1.7305
13 Oct 2013 10:24:48	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	648,000	1,122,155	1.7317
12 Oct 2013 18:23:07	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	622,080	1,075,063	1.7282
12 Oct 2013 04:13:18	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	596,160	1,033,061	1.7329
09 Oct 2013 18:32:29	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	570,240	990,079	1.7362
09 Oct 2013 04:40:56	1289096	16018795	hadcm3n_o5et_1940_40_008411133_1	544,320	946,865	1.7395