Task 12831789

Name	hadcm3n_p6fv_1900_40_007225931_0
Workunit	7424171
Created	26 Apr 2011, 15:36:12 UTC
Sent	27 Apr 2011, 11:34:59 UTC
Report deadline	27 Jul 2011, 19:02:10 UTC
Received	11 May 2011, 17:47:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1143614
Run time	8 days 11 hours 13 min 36 sec
CPU time	6 days 7 hours 43 min 8 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.18 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:48:55 (39452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:30:49 (2530): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:17:41 (39097): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:14:53 (50916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:06:21 (81468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:06:22 (81468): No heartbeat from core client for 30 sec - exiting 05:06:23 (81468): No heartbeat from core client for 30 sec - exiting 05:06:24 (81468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x1081fe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x1081fe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x5820604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x5820600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x481f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x481f600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x2032204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x2032200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x1030604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x1030600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x11006604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x11006600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x11006604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x11006600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x11006604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x11006600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41030,0xa0577540) malloc: * error for object 0x11006600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=41031, selfPID=41031, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(11635,0xa0577540) malloc: * error for object 0xa000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(11635,0xa0577540) malloc: * error for object 0xa000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(11635,0xa0577540) malloc: * error for object 0x3809c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Wed May 11 13:46:24 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x93abf0af _small_free_list_remove_ptr + 246 1 libSystem.B.dylib 0x93abbb6c _szone_free_definite_size + 3457 2 libSystem.B.dylib 0x93abab88 _free + 244 3 hadcm3n_6.07_i686-apple-darwin 0x0000ba58 __Z12annual_cyclePKSt6vectorISsSaISsEEPKcii + 3482 4 hadcm3n_6.07_i686-apple-darwin 0x0000d36b __Z12decadalMeansiPKc + 957 5 hadcm3n_6.07_i686-apple-darwin 0x000067ff __Z9doCM3Procv + 185 6 hadcm3n_6.07_i686-apple-darwin 0x0000876a __Z6workerv + 2896 7 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 _main + 491 8 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 1: 0 libSystem.B.dylib 0x93ab41ae _mach_wait_until + 10 1 libSystem.B.dylib 0x93b3b809 _nanosleep + 345 2 libSystem.B.dylib 0x93b3b6aa _usleep + 61 3 hadcm3n_6.07_i686-apple-darwin 0x00071a7c __Z11boinc_sleepd + 188 4 hadcm3n_6.07_i686-apple-darwin 0x00067282 __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x93ae17fd __pthread_start + 345 6 libSystem.B.dylib 0x93ae1682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fae8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/10/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x93ab3000 - 0x93c5afff /usr/lib/libSystem.B.dylib 0x93ccb000 - 0x93cd9fff /usr/lib/libz.1.dylib 0x93f57000 - 0x93f5afff /usr/lib/system/libmathCommon.A.dylib 0x96b62000 - 0x96bccfff /usr/lib/libstdc++.6.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 May 2011 14:00:22	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	259,200	541,310	2.0884
09 May 2011 17:09:32	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	233,280	494,948	2.1217
08 May 2011 21:27:34	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	207,360	444,834	2.1452
08 May 2011 01:09:21	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	181,440	395,603	2.1804
05 May 2011 20:31:22	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	155,520	347,694	2.2357
04 May 2011 21:22:42	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	129,600	289,277	2.2321
03 May 2011 20:52:17	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	103,680	226,441	2.1840
02 May 2011 20:52:11	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	77,760	166,941	2.1469
01 May 2011 22:39:09	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	51,840	108,646	2.0958
30 Apr 2011 22:33:18	1143614	12831789	hadcm3n_p6fv_1900_40_007225931_0	25,920	48,242	1.8612