Task 15678034

Name	hadcm3n_39hz_1940_40_008266576_1
Workunit	8421700
Created	23 Mar 2013, 0:15:36 UTC
Sent	23 Mar 2013, 5:40:15 UTC
Report deadline	22 Jun 2013, 13:07:26 UTC
Received	30 Mar 2013, 4:17:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1208255
Run time	6 days 6 hours 11 min 27 sec
CPU time	5 days 15 hours 4 min 55 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:29:47 (25853): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x1068004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x1068000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x2847c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x2847c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x2847c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x2847c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x83b404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x282c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x2847c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x2847c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x2847c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x2847c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x282c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x104ca04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x1068004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x1068000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x1068004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x1068000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x83b404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25956,0xac6482c0) malloc: * error for object 0x80c200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1a54e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1264e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0xa74a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x2a53e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1264e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1a54e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x1256600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65665,0xac6482c0) malloc: * error for object 0x2a41604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x2be1604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x2be1600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x2be1604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x2be1600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0xbeb604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0xbeb600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0xbeb604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0xbeb600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x2be1604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x2be1600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x2be1604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x2be1600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Fri Mar 29 20:26:41 2013 Thread 0 Crashed: Thread 1: hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x13c9a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(73415,0xac6482c0) malloc: * error for object 0x13c9a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fa88 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/7/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90007000 - 0x9000ffff /usr/lib/system/libunwind.dylib 0x90ac2000 - 0x90ac9fff /usr/lib/system/libsystem_notify.dylib 0x90ba2000 - 0x90ba2fff /usr/lib/system/libkeymgr.dylib 0x919e4000 - 0x919e5fff /usr/lib/system/libquarantine.dylib 0x9242b000 - 0x9242dfff /usr/lib/system/libdyld.dylib 0x92709000 - 0x9270afff /usr/lib/system/libremovefile.dylib 0x92be8000 - 0x92bf0fff /usr/lib/system/liblaunch.dylib 0x93678000 - 0x9367bfff /usr/lib/system/libcompiler_rt.dylib 0x94232000 - 0x9423bfff /usr/lib/libc++abi.dylib 0x945f3000 - 0x94611fff /usr/lib/system/libsystem_kernel.dylib 0x960d1000 - 0x9619cfff /usr/lib/system/libsystem_c.dylib 0x96700000 - 0x96701fff /usr/lib/system/libsystem_sandbox.dylib 0x96bf7000 - 0x96bf8fff /usr/lib/system/libsystem_blocks.dylib 0x96c53000 - 0x96c82fff /usr/lib/system/libsystem_info.dylib 0x97314000 - 0x97317fff /usr/lib/system/libmathCommon.A.dylib 0x9734a000 - 0x97378fff /usr/lib/libSystem.B.dylib 0x987f3000 - 0x987f3fff /usr/lib/system/libdnsinfo.dylib 0x9a21e000 - 0x9a222fff /usr/lib/system/libsystem_network.dylib 0x9ac55000 - 0x9ac5cfff /usr/lib/system/libsystem_dnssd.dylib 0x9ad86000 - 0x9ad87fff /usr/lib/system/libunc.dylib 0x9ad88000 - 0x9ad90fff /usr/lib/system/libcopyfile.dylib 0x9b7b3000 - 0x9b7b7fff /usr/lib/system/libcache.dylib 0x9b896000 - 0x9b8d9fff /usr/lib/system/libcommonCrypto.dylib 0x9bbc5000 - 0x9bc27fff /usr/lib/libstdc++.6.dylib 0x9bc28000 - 0x9bc36fff /usr/lib/system/libdispatch.dylib 0x9bc37000 - 0x9bc45fff /usr/lib/libz.1.dylib 0x9ceb2000 - 0x9cec8fff /usr/lib/system/libxpc.dylib 0x9cedf000 - 0x9cee4fff /usr/lib/system/libmacho.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 Mar 2013 00:31:37	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	518,400	486,295	0.9381
29 Mar 2013 12:47:25	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	492,480	462,262	0.9386
29 Mar 2013 05:24:24	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	466,560	438,229	0.9393
28 Mar 2013 21:44:06	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	440,640	414,025	0.9396
28 Mar 2013 13:10:50	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	414,720	389,735	0.9398
28 Mar 2013 04:52:20	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	388,800	365,646	0.9404
27 Mar 2013 20:41:48	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	362,880	341,481	0.9410
27 Mar 2013 12:23:45	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	336,960	316,583	0.9395
27 Mar 2013 05:00:58	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	311,040	292,366	0.9400
26 Mar 2013 21:25:20	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	285,120	268,103	0.9403
26 Mar 2013 12:07:16	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	259,200	244,249	0.9423
26 Mar 2013 04:44:33	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	233,280	219,953	0.9429
25 Mar 2013 21:30:59	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	207,360	196,183	0.9461
25 Mar 2013 13:19:47	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	181,440	171,933	0.9476
25 Mar 2013 05:56:32	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	155,520	147,545	0.9487
24 Mar 2013 21:43:48	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	129,600	123,181	0.9505
24 Mar 2013 13:18:28	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	103,680	99,265	0.9574
24 Mar 2013 05:00:54	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	77,760	74,873	0.9629
24 Mar 2013 05:00:54	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	51,840	50,273	0.9698
23 Mar 2013 14:30:33	1208255	15678034	hadcm3n_39hz_1940_40_008266576_1	25,920	25,041	0.9661