Task 13305336

Name	hadcm3n_yjr6_1940_40_007426826_0
Workunit	7624329
Created	27 Aug 2011, 6:58:30 UTC
Sent	27 Aug 2011, 7:45:08 UTC
Report deadline	26 Nov 2011, 15:12:19 UTC
Received	27 Oct 2011, 14:26:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	939545
Run time	18 days 15 hours 27 min 25 sec
CPU time	17 days 7 hours 33 min 37 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	2.48 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.6.36</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(68577,0xa004e540) malloc: * error for object 0x840c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(68577,0xa004e540) malloc: * error for object 0x840c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(68577,0xa004e540) malloc: * error for object 0x840c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(68577,0xa004e540) malloc: * error for object 0x840c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(68577,0xa004e540) malloc: * error for object 0x840c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=68597, selfPID=68597, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x83de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x83de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x83de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x83de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x83de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x83de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71586,0xa004e540) malloc: * error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:50:47 (16807): No heartbeat from core client for 30 sec - exiting 11:50:49 (16807): No heartbeat from core client for 30 sec - exiting 11:50:50 (16807): No heartbeat from core client for 30 sec - exiting 11:50:51 (16807): No heartbeat from core client for 30 sec - exiting 11:50:52 (16807): No heartbeat from core client for 30 sec - exiting 11:50:53 (16807): No heartbeat from core client for 30 sec - exiting 11:50:54 (16807): No heartbeat from core client for 30 sec - exiting 11:50:55 (16807): No heartbeat from core client for 30 sec - exiting 11:50:56 (16807): No heartbeat from core client for 30 sec - exiting 11:50:57 (16807): No heartbeat from core client for 30 sec - exiting 11:50:58 (16807): No heartbeat from core client for 30 sec - exiting 11:50:59 (16807): No heartbeat from core client for 30 sec - exiting 11:51:00 (16807): No heartbeat from core client for 30 sec - exiting 11:51:01 (16807): No heartbeat from core client for 30 sec - exiting 11:51:02 (16807): No heartbeat from core client for 30 sec - exiting 11:51:03 (16807): No heartbeat from core client for 30 sec - exiting 11:51:04 (16807): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:03:43 (16815): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:03:44 (16815): No heartbeat from core client for 30 sec - exiting 12:03:45 (16815): No heartbeat from core client for 30 sec - exiting 12:03:46 (16815): No heartbeat from core client for 30 sec - exiting 12:03:47 (16815): No heartbeat from core client for 30 sec - exiting 12:04:20 (16815): No heartbeat from core client for 30 sec - exiting 12:04:21 (16815): No heartbeat from core client for 30 sec - exiting 12:04:22 (16815): No heartbeat from core client for 30 sec - exiting 12:04:23 (16815): No heartbeat from core client for 30 sec - exiting 12:04:24 (16815): No heartbeat from core client for 30 sec - exiting 12:04:25 (16815): No heartbeat from core client for 30 sec - exiting 12:04:26 (16815): No heartbeat from core client for 30 sec - exiting 12:04:27 (16815): No heartbeat from core client for 30 sec - exiting 12:04:28 (16815): No heartbeat from core client for 30 sec - exiting 12:04:29 (16815): No heartbeat from core client for 30 sec - exiting 12:04:30 (16815): No heartbeat from core client for 30 sec - exiting 12:04:31 (16815): No heartbeat from core client for 30 sec - exiting 12:04:32 (16815): No heartbeat from core client for 30 sec - exiting 12:04:33 (16815): No heartbeat from core client for 30 sec - exiting 12:04:34 (16815): No heartbeat from core client for 30 sec - exiting 12:04:35 (16815): No heartbeat from core client for 30 sec - exiting 12:04:36 (16815): No heartbeat from core client for 30 sec - exiting 12:04:37 (16815): No heartbeat from core client for 30 sec - exiting 12:04:38 (16815): No heartbeat from core client for 30 sec - exiting 12:04:39 (16815): No heartbeat from core client for 30 sec - exiting 12:04:40 (16815): No heartbeat from core client for 30 sec - exiting 12:04:41 (16815): No heartbeat from core client for 30 sec - exiting 12:04:42 (16815): No heartbeat from core client for 30 sec - exiting 12:04:43 (16815): No heartbeat from core client for 30 sec - exiting 12:04:44 (16815): No heartbeat from core client for 30 sec - exiting 12:04:45 (16815): No heartbeat from core client for 30 sec - exiting 12:04:46 (16815): No heartbeat from core client for 30 sec - exiting 12:04:47 (16815): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 3 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Oct 2011 03:44:56	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	855,360	1,488,487	1.7402
04 Oct 2011 08:26:14	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	829,440	1,443,735	1.7406
01 Oct 2011 22:52:19	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	803,520	1,397,614	1.7394
30 Sep 2011 21:47:57	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	777,600	1,351,538	1.7381
30 Sep 2011 08:00:10	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	751,680	1,305,553	1.7368
29 Sep 2011 09:33:39	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	725,760	1,260,347	1.7366
28 Sep 2011 11:21:20	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	699,840	1,215,785	1.7372
26 Sep 2011 17:49:49	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	673,920	1,170,938	1.7375
26 Sep 2011 01:39:54	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	648,000	1,125,052	1.7362
25 Sep 2011 09:36:10	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	622,080	1,079,697	1.7356
24 Sep 2011 06:09:59	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	596,160	1,034,480	1.7352
23 Sep 2011 05:03:44	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	570,240	989,513	1.7353
21 Sep 2011 22:05:53	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	544,320	943,830	1.7340
20 Sep 2011 23:35:27	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	518,400	898,416	1.7331
19 Sep 2011 20:28:18	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	492,480	852,784	1.7316
16 Sep 2011 17:28:53	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	466,560	807,621	1.7310
13 Sep 2011 07:01:18	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	440,640	762,130	1.7296
11 Sep 2011 11:25:40	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	414,720	717,194	1.7293
10 Sep 2011 13:58:57	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	388,800	672,604	1.7299
09 Sep 2011 19:49:42	939545	13305336	hadcm3n_yjr6_1940_40_007426826_0	362,880	628,057	1.7308