Task 16235398

Name	hadcm3n_7nx0_1980_40_008442279_1
Workunit	8593135
Created	14 Jan 2014, 9:30:51 UTC
Sent	14 Jan 2014, 9:38:14 UTC
Report deadline	15 Apr 2014, 17:05:25 UTC
Received	12 Feb 2014, 16:30:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1251762
Run time	22 days 22 hours 24 min 41 sec
CPU time	20 days 15 hours 34 min 8 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.88 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(38326,0xa084c540) malloc: * error for object 0x2006204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(38326,0xa084c540) malloc: * error for object 0x2006200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(38326,0xa084c540) malloc: * error for object 0x1011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(38326,0xa084c540) malloc: * error for object 0x1011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(38326,0xa084c540) malloc: * error for object 0x4800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(38326,0xa084c540) malloc: * error for object 0x4800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(38326,0xa084c540) malloc: * error for object 0x4800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(38326,0xa084c540) malloc: * error for object 0x4800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Mon Feb 10 20:10:29 2014 Thread 0 Crashed: hadcm3n_6.07_i686-apple-darwin(55303,0xa084c540) malloc: * error for object 0x302de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(56688,0xa084c540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56688,0xa084c540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56688,0xa084c540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56688,0xa084c540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56688,0xa084c540) malloc: * error for object 0x5000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56688,0xa084c540) malloc: * error for object 0x5001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Feb 12 06:21:40 2014 Thread 0 Crashed: 0 libSystem.B.dylib 0x97456b0f _small_free_list_remove_ptr + 246 1 libSystem.B.dylib 0x974535cc _szone_free_definite_size + 3457 2 libSystem.B.dylib 0x974525e8 _free + 244 3 hadcm3n_6.07_i686-apple-darwin 0x0000ba58 __Z12annual_cyclePKSt6vectorISsSaISsEEPKcii + 3482 4 hadcm3n_6.07_i686-apple-darwin 0x0000d36b __Z12decadalMeansiPKc + 957 5 hadcm3n_6.07_i686-apple-darwin 0x000067ff __Z9doCM3Procv + 185 6 hadcm3n_6.07_i686-apple-darwin 0x0000876a __Z6workerv + 2896 7 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 _main + 491 8 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 1: 0 libSystem.B.dylib 0x9744bc0e _mach_wait_until + 10 1 libSystem.B.dylib 0x974d3429 _nanosleep + 345 2 libSystem.B.dylib 0x974d32ca _usleep + 61 3 hadcm3n_6.07_i686-apple-darwin 0x00071a7c __Z11boinc_sleepd + 188 4 hadcm3n_6.07_i686-apple-darwin 0x00067282 __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97479259 __pthread_start + 345 6 libSystem.B.dylib 0x974790de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8f998 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/9/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x948ba000 - 0x948bdfff /usr/lib/system/libmathCommon.A.dylib 0x95872000 - 0x958dcfff /usr/lib/libstdc++.6.dylib 0x9744b000 - 0x975f2fff /usr/lib/libSystem.B.dylib 0x99232000 - 0x99240fff /usr/lib/libz.1.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Feb 2014 04:55:14	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	1,036,800	1,783,387	1.7201
10 Feb 2014 06:26:46	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	1,010,880	1,741,693	1.7229
09 Feb 2014 20:54:49	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	984,960	1,701,290	1.7273
09 Feb 2014 04:39:50	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	959,040	1,659,916	1.7308
08 Feb 2014 13:11:42	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	933,120	1,615,377	1.7312
07 Feb 2014 23:38:50	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	907,200	1,572,104	1.7329
06 Feb 2014 22:18:24	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	881,280	1,524,107	1.7294
06 Feb 2014 14:47:12	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	855,360	1,478,699	1.7287
05 Feb 2014 19:07:32	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	829,440	1,438,172	1.7339
05 Feb 2014 06:20:30	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	803,520	1,396,043	1.7374
04 Feb 2014 16:47:01	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	777,600	1,354,795	1.7423
04 Feb 2014 02:32:55	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	751,680	1,310,247	1.7431
04 Feb 2014 02:32:55	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	725,760	1,265,563	1.7438
02 Feb 2014 07:28:16	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	699,840	1,220,924	1.7446
01 Feb 2014 15:54:30	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	673,920	1,176,483	1.7457
01 Feb 2014 15:54:30	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	648,000	1,130,963	1.7453
31 Jan 2014 13:20:27	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	622,080	1,083,799	1.7422
31 Jan 2014 13:20:27	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	596,160	1,034,657	1.7355
31 Jan 2014 13:20:27	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	570,240	988,581	1.7336
28 Jan 2014 01:59:55	1251762	16235398	hadcm3n_7nx0_1980_40_008442279_1	544,320	942,688	1.7319