Task 13540381

Name	hadcm3n_yffb_1900_40_007517352_0
Workunit	7714827
Created	28 Oct 2011, 12:52:01 UTC
Sent	22 Nov 2011, 0:35:58 UTC
Report deadline	21 Feb 2012, 8:03:09 UTC
Received	25 Feb 2012, 10:16:46 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	779304
Run time	27 days 0 hours 49 min 41 sec
CPU time	23 days 23 hours 50 min 54 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	3.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x3004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x814404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x814400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x814404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x814400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x1005404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x1013a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41377,0xa005e540) malloc: * error for object 0x1013a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:11:02 (23867): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(97611,0xa005e540) malloc: * error for object 0x380a204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(97611,0xa005e540) malloc: * error for object 0x380a200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(97611,0xa005e540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(97611,0xa005e540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(97611,0xa005e540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(97611,0xa005e540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(97611,0xa005e540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=97615, selfPID=97615, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x3015c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x3015c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x2806204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x2806200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x3802204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x3810804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x381f204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(30514,0xa005e540) malloc: * error for object 0x381f200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:53:27 (34659): No heartbeat from core client for 30 sec - exiting 01:53:29 (34659): No heartbeat from core client for 30 sec - exiting 01:53:30 (34659): No heartbeat from core client for 30 sec - exiting 01:53:31 (34659): No heartbeat from core client for 30 sec - exiting 01:53:32 (34659): No heartbeat from core client for 30 sec - exiting Signal 3 received, exiting... Called boinc_finish 01:56:20 (44027): No heartbeat from core client for 30 sec - exiting 01:56:22 (44027): No heartbeat from core client for 30 sec - exiting 01:56:23 (44027): No heartbeat from core client for 30 sec - exiting 01:56:24 (44027): No heartbeat from core client for 30 sec - exiting 01:56:25 (44027): No heartbeat from core client for 30 sec - exiting 01:56:26 (44027): No heartbeat from core client for 30 sec - exiting 01:56:27 (44027): No heartbeat from core client for 30 sec - exiting 01:56:28 (44027): No heartbeat from core client for 30 sec - exiting 01:56:29 (44027): No heartbeat from core client for 30 sec - exiting 01:56:30 (44027): No heartbeat from core client for 30 sec - exiting 01:56:31 (44027): No heartbeat from core client for 30 sec - exiting 01:56:32 (44027): No heartbeat from core client for 30 sec - exiting 01:56:33 (44027): No heartbeat from core client for 30 sec - exiting 01:56:34 (44027): No heartbeat from core client for 30 sec - exiting 01:56:35 (44027): No heartbeat from core client for 30 sec - exiting 01:56:36 (44027): No heartbeat from core client for 30 sec - exiting 01:56:37 (44027): No heartbeat from core client for 30 sec - exiting 01:56:38 (44027): No heartbeat from core client for 30 sec - exiting 01:56:39 (44027): No heartbeat from core client for 30 sec - exiting 01:56:40 (44027): No heartbeat from core client for 30 sec - exiting 01:56:41 (44027): No heartbeat from core client for 30 sec - exiting 01:56:42 (44027): No heartbeat from core client for 30 sec - exiting 01:56:44 (44027): No heartbeat from core client for 30 sec - exiting 01:56:45 (44027): No heartbeat from core client for 30 sec - exiting 01:56:46 (44027): No heartbeat from core client for 30 sec - exiting 01:56:47 (44027): No heartbeat from core client for 30 sec - exiting 01:56:48 (44027): No heartbeat from core client for 30 sec - exiting 01:56:49 (44027): No heartbeat from core client for 30 sec - exiting 01:56:50 (44027): No heartbeat from core client for 30 sec - exiting 01:56:51 (44027): No heartbeat from core client for 30 sec - exiting 01:56:52 (44027): No heartbeat from core client for 30 sec - exiting 01:56:53 (44027): No heartbeat from core client for 30 sec - exiting 01:56:54 (44027): No heartbeat from core client for 30 sec - exiting 01:56:55 (44027): No heartbeat from core client for 30 sec - exiting 01:56:56 (44027): No heartbeat from core client for 30 sec - exiting 01:56:57 (44027): No heartbeat from core client for 30 sec - exiting 01:56:58 (44027): No heartbeat from core client for 30 sec - exiting 01:56:59 (44027): No heartbeat from core client for 30 sec - exiting 01:57:00 (44027): No heartbeat from core client for 30 sec - exiting 01:57:01 (44027): No heartbeat from core client for 30 sec - exiting 01:57:02 (44027): No heartbeat from core client for 30 sec - exiting 01:57:03 (44027): No heartbeat from core client for 30 sec - exiting 01:57:05 (44027): No heartbeat from core client for 30 sec - exiting 01:57:06 (44027): No heartbeat from core client for 30 sec - exiting 01:57:07 (44027): No heartbeat from core client for 30 sec - exiting 01:57:08 (44027): No heartbeat from core client for 30 sec - exiting 01:57:09 (44027): No heartbeat from core client for 30 sec - exiting 01:57:10 (44027): No heartbeat from core client for 30 sec - exiting 01:57:11 (44027): No heartbeat from core client for 30 sec - exiting 01:57:12 (44027): No heartbeat from core client for 30 sec - exiting 01:57:13 (44027): No heartbeat from core client for 30 sec - exiting 01:57:14 (44027): No heartbeat from core client for 30 sec - exiting 01:57:15 (44027): No heartbeat from core client for 30 sec - exiting 01:57:16 (44027): No heartbeat from core client for 30 sec - exiting 01:57:17 (44027): No heartbeat from core client for 30 sec - exiting 01:57:18 (44027): No heartbeat from core client for 30 sec - exiting 01:57:19 (44027): No heartbeat from core client for 30 sec - exiting 01:57:20 (44027): No heartbeat from core client for 30 sec - exiting 01:57:21 (44027): No heartbeat from core client for 30 sec - exiting 01:57:22 (44027): No heartbeat from core client for 30 sec - exiting 01:57:23 (44027): No heartbeat from core client for 30 sec - exiting 01:57:24 (44027): No heartbeat from core client for 30 sec - exiting 01:57:25 (44027): No heartbeat from core client for 30 sec - exiting 01:57:26 (44027): No heartbeat from core client for 30 sec - exiting 01:57:27 (44027): No heartbeat from core client for 30 sec - exiting 01:57:28 (44027): No heartbeat from core client for 30 sec - exiting 01:57:29 (44027): No heartbeat from core client for 30 sec - exiting 01:57:30 (44027): No heartbeat from core client for 30 sec - exiting 01:57:31 (44027): No heartbeat from core client for 30 sec - exiting 01:57:32 (44027): No heartbeat from core client for 30 sec - exiting 01:57:33 (44027): No heartbeat from core client for 30 sec - exiting 01:57:34 (44027): No heartbeat from core client for 30 sec - exiting 01:57:35 (44027): No heartbeat from core client for 30 sec - exiting 01:57:36 (44027): No heartbeat from core client for 30 sec - exiting 01:57:37 (44027): No heartbeat from core client for 30 sec - exiting 01:57:38 (44027): No heartbeat from core client for 30 sec - exiting 01:57:39 (44027): No heartbeat from core client for 30 sec - exiting 01:57:40 (44027): No heartbeat from core client for 30 sec - exiting 01:57:41 (44027): No heartbeat from core client for 30 sec - exiting 01:57:42 (44027): No heartbeat from core client for 30 sec - exiting 01:57:44 (44027): No heartbeat from core client for 30 sec - exiting 01:57:45 (44027): No heartbeat from core client for 30 sec - exiting 01:57:46 (44027): No heartbeat from core client for 30 sec - exiting 01:57:47 (44027): No heartbeat from core client for 30 sec - exiting 01:57:48 (44027): No heartbeat from core client for 30 sec - exiting 01:57:49 (44027): No heartbeat from core client for 30 sec - exiting 01:57:50 (44027): No heartbeat from core client for 30 sec - exiting 01:57:51 (44027): No heartbeat from core client for 30 sec - exiting 01:57:52 (44027): No heartbeat from core client for 30 sec - exiting 01:57:53 (44027): No heartbeat from core client for 30 sec - exiting 01:57:54 (44027): No heartbeat from core client for 30 sec - exiting 01:57:55 (44027): No heartbeat from core client for 30 sec - exiting 01:57:56 (44027): No heartbeat from core client for 30 sec - exiting 01:57:57 (44027): No heartbeat from core client for 30 sec - exiting 01:57:58 (44027): No heartbeat from core client for 30 sec - exiting 01:57:59 (44027): No heartbeat from core client for 30 sec - exiting 01:58:00 (44027): No heartbeat from core client for 30 sec - exiting 01:58:01 (44027): No heartbeat from core client for 30 sec - exiting 01:58:02 (44027): No heartbeat from core client for 30 sec - exiting 01:58:03 (44027): No heartbeat from core client for 30 sec - exiting 01:58:04 (44027): No heartbeat from core client for 30 sec - exiting 01:58:05 (44027): No heartbeat from core client for 30 sec - exiting 01:58:06 (44027): No heartbeat from core client for 30 sec - exiting 01:58:08 (44027): No heartbeat from core client for 30 sec - exiting 01:58:09 (44027): No heartbeat from core client for 30 sec - exiting 01:58:10 (44027): No heartbeat from core client for 30 sec - exiting 01:58:11 (44027): No heartbeat from core client for 30 sec - exiting 01:58:12 (44027): No heartbeat from core client for 30 sec - exiting 01:58:13 (44027): No heartbeat from core client for 30 sec - exiting 01:58:14 (44027): No heartbeat from core client for 30 sec - exiting 01:58:15 (44027): No heartbeat from core client for 30 sec - exiting 01:58:16 (44027): No heartbeat from core client for 30 sec - exiting 01:58:17 (44027): No heartbeat from core client for 30 sec - exiting 01:58:18 (44027): No heartbeat from core client for 30 sec - exiting 01:58:19 (44027): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Feb 2012 09:21:06	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	1,036,800	2,072,327	1.9988
24 Feb 2012 08:12:59	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	1,010,880	2,024,748	2.0030
23 Feb 2012 05:10:56	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	984,960	1,975,583	2.0057
22 Feb 2012 04:12:49	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	959,040	1,928,140	2.0105
21 Feb 2012 01:54:07	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	933,120	1,876,881	2.0114
19 Feb 2012 23:57:56	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	907,200	1,825,998	2.0128
19 Feb 2012 00:07:02	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	881,280	1,781,266	2.0212
18 Feb 2012 00:06:59	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	855,360	1,735,760	2.0293
16 Feb 2012 23:35:33	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	829,440	1,689,568	2.0370
15 Feb 2012 23:38:22	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	803,520	1,645,070	2.0473
14 Feb 2012 12:35:47	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	777,600	1,599,138	2.0565
13 Feb 2012 10:34:48	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	751,680	1,549,620	2.0615
12 Feb 2012 08:23:21	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	725,760	1,498,448	2.0647
11 Feb 2012 06:46:43	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	699,840	1,448,854	2.0703
10 Feb 2012 06:15:38	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	673,920	1,402,527	2.0811
09 Feb 2012 05:01:44	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	648,000	1,354,930	2.0909
07 Feb 2012 03:00:19	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	622,080	1,305,382	2.0984
05 Feb 2012 01:32:20	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	596,160	1,257,274	2.1090
03 Feb 2012 23:52:06	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	570,240	1,207,663	2.1178
02 Feb 2012 10:43:55	779304	13540381	hadcm3n_yffb_1900_40_007517352_0	544,320	1,156,086	2.1239