Task 17524464

Name	hadam3p_anz_l0o3_2012_1_009260112_0
Workunit	9353028
Created	1 Dec 2014, 15:06:02 UTC
Sent	7 Dec 2014, 22:28:24 UTC
Report deadline	20 Nov 2015, 3:48:24 UTC
Received	25 Dec 2014, 4:30:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1261095
Run time	6 days 19 hours 33 min 15 sec
CPU time	6 days 18 hours 13 min 44 sec
Validate state	Invalid
Credit	4,484.28
Device peak FLOPS	3.00 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... 21:37:07 (6768): No heartbeat from core client for 30 sec - exiting 21:37:08 (6768): No heartbeat from core client for 30 sec - exiting 21:37:09 (6768): No heartbeat from core client for 30 sec - exiting 21:37:10 (6768): No heartbeat from core client for 30 sec - exiting 21:37:11 (6768): No heartbeat from core client for 30 sec - exiting 21:37:12 (6768): No heartbeat from core client for 30 sec - exiting 21:37:13 (6768): No heartbeat from core client for 30 sec - exiting 21:37:14 (6768): No heartbeat from core client for 30 sec - exiting 21:37:15 (6768): No heartbeat from core client for 30 sec - exiting 21:37:16 (6768): No heartbeat from core client for 30 sec - exiting 21:37:17 (6768): No heartbeat from core client for 30 sec - exiting 21:37:18 (6768): No heartbeat from core client for 30 sec - exiting 21:37:49 (6768): No heartbeat from core client for 30 sec - exiting 21:37:50 (6768): No heartbeat from core client for 30 sec - exiting 21:37:51 (6768): No heartbeat from core client for 30 sec - exiting 21:37:52 (6768): No heartbeat from core client for 30 sec - exiting 21:37:53 (6768): No heartbeat from core client for 30 sec - exiting 21:37:54 (6768): No heartbeat from core client for 30 sec - exiting 21:37:55 (6768): No heartbeat from core client for 30 sec - exiting 21:37:56 (6768): No heartbeat from core client for 30 sec - exiting 21:37:57 (6768): No heartbeat from core client for 30 sec - exiting 21:37:58 (6768): No heartbeat from core client for 30 sec - exiting 21:37:59 (6768): No heartbeat from core client for 30 sec - exiting 21:38:00 (6768): No heartbeat from core client for 30 sec - exiting 21:38:01 (6768): No heartbeat from core client for 30 sec - exiting 21:38:02 (6768): No heartbeat from core client for 30 sec - exiting 21:38:03 (6768): No heartbeat from core client for 30 sec - exiting 21:38:04 (6768): No heartbeat from core client for 30 sec - exiting 21:38:05 (6768): No heartbeat from core client for 30 sec - exiting 21:38:06 (6768): No heartbeat from core client for 30 sec - exiting 21:38:07 (6768): No heartbeat from core client for 30 sec - exiting 21:38:08 (6768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:24:16 (1268): No heartbeat from core client for 30 sec - exiting 23:24:17 (1268): No heartbeat from core client for 30 sec - exiting 23:24:18 (1268): No heartbeat from core client for 30 sec - exiting 23:24:49 (1268): No heartbeat from core client for 30 sec - exiting 23:24:50 (1268): No heartbeat from core client for 30 sec - exiting 23:24:51 (1268): No heartbeat from core client for 30 sec - exiting 23:24:52 (1268): No heartbeat from core client for 30 sec - exiting 23:24:53 (1268): No heartbeat from core client for 30 sec - exiting 23:24:54 (1268): No heartbeat from core client for 30 sec - exiting 23:24:55 (1268): No heartbeat from core client for 30 sec - exiting 23:24:56 (1268): No heartbeat from core client for 30 sec - exiting 23:24:57 (1268): No heartbeat from core client for 30 sec - exiting 23:24:58 (1268): No heartbeat from core client for 30 sec - exiting 23:24:59 (1268): No heartbeat from core client for 30 sec - exiting 23:25:00 (1268): No heartbeat from core client for 30 sec - exiting 23:25:01 (1268): No heartbeat from core client for 30 sec - exiting 23:25:02 (1268): No heartbeat from core client for 30 sec - exiting 23:25:03 (1268): No heartbeat from core client for 30 sec - exiting 23:25:04 (1268): No heartbeat from core client for 30 sec - exiting 23:25:05 (1268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2408, selfPID=2408, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 01:36:23 (1260): No heartbeat from core client for 30 sec - exiting 01:36:24 (1260): No heartbeat from core client for 30 sec - exiting 01:36:25 (1260): No heartbeat from core client for 30 sec - exiting 01:36:26 (1260): No heartbeat from core client for 30 sec - exiting 01:36:27 (1260): No heartbeat from core client for 30 sec - exiting 01:36:28 (1260): No heartbeat from core client for 30 sec - exiting 01:36:29 (1260): No heartbeat from core client for 30 sec - exiting 01:36:30 (1260): No heartbeat from core client for 30 sec - exiting 01:36:31 (1260): No heartbeat from core client for 30 sec - exiting 01:36:32 (1260): No heartbeat from core client for 30 sec - exiting 01:37:03 (1260): No heartbeat from core client for 30 sec - exiting 01:37:04 (1260): No heartbeat from core client for 30 sec - exiting 01:37:05 (1260): No heartbeat from core client for 30 sec - exiting 01:37:06 (1260): No heartbeat from core client for 30 sec - exiting 01:37:07 (1260): No heartbeat from core client for 30 sec - exiting 01:37:08 (1260): No heartbeat from core client for 30 sec - exiting 01:37:09 (1260): No heartbeat from core client for 30 sec - exiting 01:37:10 (1260): No heartbeat from core client for 30 sec - exiting 01:37:11 (1260): No heartbeat from core client for 30 sec - exiting 01:37:12 (1260): No heartbeat from core client for 30 sec - exiting 01:37:13 (1260): No heartbeat from core client for 30 sec - exiting 01:37:14 (1260): No heartbeat from core client for 30 sec - exiting 01:37:15 (1260): No heartbeat from core client for 30 sec - exiting 01:37:16 (1260): No heartbeat from core client for 30 sec - exiting 01:37:17 (1260): No heartbeat from core client for 30 sec - exiting 01:37:18 (1260): No heartbeat from core client for 30 sec - exiting 01:37:19 (1260): No heartbeat from core client for 30 sec - exiting 01:37:20 (1260): No heartbeat from core client for 30 sec - exiting 01:37:21 (1260): No heartbeat from core client for 30 sec - exiting 01:37:22 (1260): No heartbeat from core client for 30 sec - exiting 01:37:23 (1260): No heartbeat from core client for 30 sec - exiting 01:37:24 (1260): No heartbeat from core client for 30 sec - exiting 01:37:25 (1260): No heartbeat from core client for 30 sec - exiting 01:37:26 (1260): No heartbeat from core client for 30 sec - exiting 01:37:27 (1260): No heartbeat from core client for 30 sec - exiting 01:37:28 (1260): No heartbeat from core client for 30 sec - exiting 01:37:29 (1260): No heartbeat from core client for 30 sec - exiting 01:37:30 (1260): No heartbeat from core client for 30 sec - exiting 01:37:31 (1260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:58:10 (4968): No heartbeat from core client for 30 sec - exiting 02:58:11 (4968): No heartbeat from core client for 30 sec - exiting 02:58:12 (4968): No heartbeat from core client for 30 sec - exiting 02:58:13 (4968): No heartbeat from core client for 30 sec - exiting 02:58:14 (4968): No heartbeat from core client for 30 sec - exiting 02:58:15 (4968): No heartbeat from core client for 30 sec - exiting 02:58:16 (4968): No heartbeat from core client for 30 sec - exiting 02:58:17 (4968): No heartbeat from core client for 30 sec - exiting 02:58:18 (4968): No heartbeat from core client for 30 sec - exiting 02:58:19 (4968): No heartbeat from core client for 30 sec - exiting 02:58:20 (4968): No heartbeat from core client for 30 sec - exiting 02:58:21 (4968): No heartbeat from core client for 30 sec - exiting 02:58:22 (4968): No heartbeat from core client for 30 sec - exiting 02:58:23 (4968): No heartbeat from core client for 30 sec - exiting 02:58:24 (4968): No heartbeat from core client for 30 sec - exiting 02:58:25 (4968): No heartbeat from core client for 30 sec - exiting 02:58:26 (4968): No heartbeat from core client for 30 sec - exiting 02:58:27 (4968): No heartbeat from core client for 30 sec - exiting 02:58:28 (4968): No heartbeat from core client for 30 sec - exiting 02:58:29 (4968): No heartbeat from core client for 30 sec - exiting 02:58:30 (4968): No heartbeat from core client for 30 sec - exiting 02:58:31 (4968): No heartbeat from core client for 30 sec - exiting 02:58:32 (4968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:56:07 (5060): No heartbeat from core client for 30 sec - exiting 01:56:09 (5060): No heartbeat from core client for 30 sec - exiting 01:56:10 (5060): No heartbeat from core client for 30 sec - exiting 01:56:11 (5060): No heartbeat from core client for 30 sec - exiting 01:56:12 (5060): No heartbeat from core client for 30 sec - exiting 01:56:13 (5060): No heartbeat from core client for 30 sec - exiting 01:56:14 (5060): No heartbeat from core client for 30 sec - exiting 01:56:15 (5060): No heartbeat from core client for 30 sec - exiting 01:56:16 (5060): No heartbeat from core client for 30 sec - exiting 01:56:17 (5060): No heartbeat from core client for 30 sec - exiting 01:56:18 (5060): No heartbeat from core client for 30 sec - exiting 01:56:19 (5060): No heartbeat from core client for 30 sec - exiting 01:56:20 (5060): No heartbeat from core client for 30 sec - exiting 01:56:21 (5060): No heartbeat from core client for 30 sec - exiting 01:56:22 (5060): No heartbeat from core client for 30 sec - exiting 01:56:23 (5060): No heartbeat from core client for 30 sec - exiting 01:56:24 (5060): No heartbeat from core client for 30 sec - exiting 01:56:25 (5060): No heartbeat from core client for 30 sec - exiting 01:56:26 (5060): No heartbeat from core client for 30 sec - exiting 01:56:27 (5060): No heartbeat from core client for 30 sec - exiting 01:56:28 (5060): No heartbeat from core client for 30 sec - exiting 01:56:29 (5060): No heartbeat from core client for 30 sec - exiting 01:56:30 (5060): No heartbeat from core client for 30 sec - exiting 01:56:31 (5060): No heartbeat from core client for 30 sec - exiting 01:56:32 (5060): No heartbeat from core client for 30 sec - exiting 01:56:33 (5060): No heartbeat from core client for 30 sec - exiting 01:56:34 (5060): No heartbeat from core client for 30 sec - exiting 01:56:35 (5060): No heartbeat from core client for 30 sec - exiting 01:56:36 (5060): No heartbeat from core client for 30 sec - exiting 01:56:37 (5060): No heartbeat from core client for 30 sec - exiting 01:56:38 (5060): No heartbeat from core client for 30 sec - exiting 01:56:39 (5060): No heartbeat from core client for 30 sec - exiting 01:56:40 (5060): No heartbeat from core client for 30 sec - exiting 01:56:41 (5060): No heartbeat from core client for 30 sec - exiting 01:56:42 (5060): No heartbeat from core client for 30 sec - exiting 01:56:43 (5060): No heartbeat from core client for 30 sec - exiting 01:56:44 (5060): No heartbeat from core client for 30 sec - exiting 01:56:45 (5060): No heartbeat from core client for 30 sec - exiting 01:56:46 (5060): No heartbeat from core client for 30 sec - exiting 01:56:47 (5060): No heartbeat from core client for 30 sec - exiting 01:56:48 (5060): No heartbeat from core client for 30 sec - exiting 01:56:49 (5060): No heartbeat from core client for 30 sec - exiting 01:56:50 (5060): No heartbeat from core client for 30 sec - exiting 01:56:51 (5060): No heartbeat from core client for 30 sec - exiting 01:56:52 (5060): No heartbeat from core client for 30 sec - exiting 01:56:53 (5060): No heartbeat from core client for 30 sec - exiting 01:56:54 (5060): No heartbeat from core client for 30 sec - exiting 01:56:55 (5060): No heartbeat from core client for 30 sec - exiting 01:56:56 (5060): No heartbeat from core client for 30 sec - exiting 01:56:57 (5060): No heartbeat from core client for 30 sec - exiting 01:56:58 (5060): No heartbeat from core client for 30 sec - exiting 01:56:59 (5060): No heartbeat from core client for 30 sec - exiting 01:57:00 (5060): No heartbeat from core client for 30 sec - exiting 01:57:01 (5060): No heartbeat from core client for 30 sec - exiting 01:57:02 (5060): No heartbeat from core client for 30 sec - exiting 01:57:03 (5060): No heartbeat from core client for 30 sec - exiting 01:57:04 (5060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:05:17 (8024): No heartbeat from core client for 30 sec - exiting 02:05:18 (8024): No heartbeat from core client for 30 sec - exiting 02:05:19 (8024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:58:41 (6444): No heartbeat from core client for 30 sec - exiting 01:58:42 (6444): No heartbeat from core client for 30 sec - exiting 01:58:43 (6444): No heartbeat from core client for 30 sec - exiting 01:58:44 (6444): No heartbeat from core client for 30 sec - exiting 01:58:45 (6444): No heartbeat from core client for 30 sec - exiting 01:58:46 (6444): No heartbeat from core client for 30 sec - exiting 01:58:47 (6444): No heartbeat from core client for 30 sec - exiting 01:58:48 (6444): No heartbeat from core client for 30 sec - exiting 01:58:49 (6444): No heartbeat from core client for 30 sec - exiting 01:58:50 (6444): No heartbeat from core client for 30 sec - exiting 01:58:51 (6444): No heartbeat from core client for 30 sec - exiting 01:58:52 (6444): No heartbeat from core client for 30 sec - exiting 01:58:53 (6444): No heartbeat from core client for 30 sec - exiting 01:58:54 (6444): No heartbeat from core client for 30 sec - exiting 01:58:55 (6444): No heartbeat from core client for 30 sec - exiting 01:58:56 (6444): No heartbeat from core client for 30 sec - exiting 01:58:57 (6444): No heartbeat from core client for 30 sec - exiting 01:58:58 (6444): No heartbeat from core client for 30 sec - exiting 01:58:59 (6444): No heartbeat from core client for 30 sec - exiting 01:59:00 (6444): No heartbeat from core client for 30 sec - exiting 01:59:01 (6444): No heartbeat from core client for 30 sec - exiting 01:59:02 (6444): No heartbeat from core client for 30 sec - exiting 01:59:03 (6444): No heartbeat from core client for 30 sec - exiting 01:59:04 (6444): No heartbeat from core client for 30 sec - exiting 01:59:05 (6444): No heartbeat from core client for 30 sec - exiting 01:59:06 (6444): No heartbeat from core client for 30 sec - exiting 01:59:07 (6444): No heartbeat from core client for 30 sec - exiting 01:59:08 (6444): No heartbeat from core client for 30 sec - exiting 01:59:09 (6444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:26:14 (5480): No heartbeat from core client for 30 sec - exiting 11:26:15 (5480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:52:25 (7824): No heartbeat from core client for 30 sec - exiting 00:52:26 (7824): No heartbeat from core client for 30 sec - exiting 00:52:27 (7824): No heartbeat from core client for 30 sec - exiting 00:52:28 (7824): No heartbeat from core client for 30 sec - exiting 00:52:29 (7824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:51:27 (2796): No heartbeat from core client for 30 sec - exiting 02:51:28 (2796): No heartbeat from core client for 30 sec - exiting 02:51:29 (2796): No heartbeat from core client for 30 sec - exiting 02:51:30 (2796): No heartbeat from core client for 30 sec - exiting 02:51:31 (2796): No heartbeat from core client for 30 sec - exiting 02:51:32 (2796): No heartbeat from core client for 30 sec - exiting 02:51:33 (2796): No heartbeat from core client for 30 sec - exiting 02:51:34 (2796): No heartbeat from core client for 30 sec - exiting 02:51:35 (2796): No heartbeat from core client for 30 sec - exiting 02:51:36 (2796): No heartbeat from core client for 30 sec - exiting 02:51:37 (2796): No heartbeat from core client for 30 sec - exiting 02:51:38 (2796): No heartbeat from core client for 30 sec - exiting 02:51:39 (2796): No heartbeat from core client for 30 sec - exiting 02:51:40 (2796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:42:45 (2216): No heartbeat from core client for 30 sec - exiting 02:42:46 (2216): No heartbeat from core client for 30 sec - exiting 02:42:47 (2216): No heartbeat from core client for 30 sec - exiting 02:42:48 (2216): No heartbeat from core client for 30 sec - exiting 02:42:49 (2216): No heartbeat from core client for 30 sec - exiting 02:42:50 (2216): No heartbeat from core client for 30 sec - exiting 02:42:51 (2216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:53:16 (5532): No heartbeat from core client for 30 sec - exiting 01:53:17 (5532): No heartbeat from core client for 30 sec - exiting 01:53:18 (5532): No heartbeat from core client for 30 sec - exiting 01:53:19 (5532): No heartbeat from core client for 30 sec - exiting 01:53:20 (5532): No heartbeat from core client for 30 sec - exiting 01:53:21 (5532): No heartbeat from core client for 30 sec - exiting 01:53:22 (5532): No heartbeat from core client for 30 sec - exiting 01:53:23 (5532): No heartbeat from core client for 30 sec - exiting 01:53:24 (5532): No heartbeat from core client for 30 sec - exiting 01:53:25 (5532): No heartbeat from core client for 30 sec - exiting 01:53:26 (5532): No heartbeat from core client for 30 sec - exiting 01:53:27 (5532): No heartbeat from core client for 30 sec - exiting 01:53:28 (5532): No heartbeat from core client for 30 sec - exiting 01:53:29 (5532): No heartbeat from core client for 30 sec - exiting 01:53:30 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3620, selfPID=3620, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3620, selfPID=6192, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_l0o3_2012_1_009260112_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l0o3_2012_1_009260112_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l0o3_2012_1_009260112_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Dec 2014 02:24:45	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	103,979	551,613	5.3050
20 Dec 2014 10:08:48	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	92,459	493,897	5.3418
19 Dec 2014 02:28:37	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	80,939	434,910	5.3733
17 Dec 2014 07:21:42	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	69,419	376,540	5.4242
15 Dec 2014 09:34:40	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	57,899	317,530	5.4842
14 Dec 2014 08:14:40	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	46,379	255,230	5.5031
13 Dec 2014 01:32:54	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	34,859	194,461	5.5785
11 Dec 2014 04:17:21	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	23,339	131,576	5.6376
09 Dec 2014 05:52:56	1261095	17524464	hadam3p_anz_l0o3_2012_1_009260112_0	11,819	67,970	5.7509