Task 15455485

Name	hadcm3n_zbvm_1880_40_008251935_0
Workunit	8407059
Created	22 Nov 2012, 20:25:21 UTC
Sent	22 Nov 2012, 20:26:18 UTC
Report deadline	22 Feb 2013, 3:53:29 UTC
Received	21 Dec 2012, 12:16:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1101528
Run time	18 days 0 hours 41 min 29 sec
CPU time	15 days 21 hours 32 min 24 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.13 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x380dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(89857,0xa0032540) malloc: * error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(15501,0xa0032540) malloc: * error for object 0x3805c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15501,0xa0032540) malloc: * error for object 0x3805c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15501,0xa0032540) malloc: * error for object 0x3805c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15501,0xa0032540) malloc: * error for object 0x3805c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15501,0xa0032540) malloc: * error for object 0x3805c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(15501,0xa0032540) malloc: * error for object 0x804e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15501,0xa0032540) malloc: * error for object 0x804e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Fri Dec 21 07:14:56 2012 Thread 0 Crashed: 0 libSystem.B.dylib 0x902fbb03 _small_free_list_remove_ptr + 234 1 libSystem.B.dylib 0x902f85cc _szone_free_definite_size + 3457 2 libSystem.B.dylib 0x902f75e8 _free + 244 3 hadcm3n_6.07_i686-apple-darwin 0x0000ba58 __Z12annual_cyclePKSt6vectorISsSaISsEEPKcii + 3482 4 hadcm3n_6.07_i686-apple-darwin 0x0000d36b __Z12decadalMeansiPKc + 957 5 hadcm3n_6.07_i686-apple-darwin 0x000067ff __Z9doCM3Procv + 185 6 hadcm3n_6.07_i686-apple-darwin 0x0000791c __Z8mainLoopv + 410 7 hadcm3n_6.07_i686-apple-darwin 0x000087c7 __Z6workerv + 2989 8 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 _main + 491 9 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 1: 0 libSystem.B.dylib 0x902f0c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x90378429 _nanosleep + 345 2 libSystem.B.dylib 0x903782ca _usleep + 61 3 hadcm3n_6.07_i686-apple-darwin 0x00071a7c __Z11boinc_sleepd + 188 4 hadcm3n_6.07_i686-apple-darwin 0x00067282 __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9031e259 __pthread_start + 345 6 libSystem.B.dylib 0x9031e0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8f6a8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/10/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90003000 - 0x9006dfff /usr/lib/libstdc++.6.dylib 0x902f0000 - 0x90497fff /usr/lib/libSystem.B.dylib 0x9156a000 - 0x9156dfff /usr/lib/system/libmathCommon.A.dylib 0x9a0e4000 - 0x9a0f2fff /usr/lib/libz.1.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Dec 2012 12:21:27	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	777,600	1,373,539	1.7664
20 Dec 2012 06:35:39	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	751,680	1,328,076	1.7668
19 Dec 2012 09:57:12	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	725,760	1,283,566	1.7686
18 Dec 2012 09:31:46	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	699,840	1,239,144	1.7706
17 Dec 2012 08:29:38	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	673,920	1,194,665	1.7727
16 Dec 2012 04:08:41	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	648,000	1,150,322	1.7752
15 Dec 2012 05:07:52	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	622,080	1,104,342	1.7752
14 Dec 2012 00:38:41	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	596,160	1,059,512	1.7772
13 Dec 2012 19:38:36	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	570,240	1,013,684	1.7776
13 Dec 2012 19:38:36	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	544,320	968,134	1.7786
13 Dec 2012 19:38:36	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	518,400	922,673	1.7798
13 Dec 2012 19:38:36	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	492,480	877,202	1.7812
13 Dec 2012 19:38:36	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	466,560	832,052	1.7834
13 Dec 2012 19:38:36	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	440,640	786,506	1.7849
13 Dec 2012 19:38:36	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	414,720	740,791	1.7862
04 Dec 2012 08:27:04	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	388,800	693,964	1.7849
03 Dec 2012 14:55:43	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	362,880	647,872	1.7854
02 Dec 2012 14:59:46	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	336,960	601,514	1.7851
01 Dec 2012 20:03:38	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	311,040	555,117	1.7847
30 Nov 2012 22:38:58	1101528	15455485	hadcm3n_zbvm_1880_40_008251935_0	285,120	508,637	1.7839