Task 18344249

Name	hadam3p_anz_f79n_2012_1_009778784_0
Workunit	9834748
Created	24 Apr 2015, 15:19:31 UTC
Sent	4 May 2015, 11:24:05 UTC
Report deadline	15 Apr 2016, 16:44:05 UTC
Received	21 May 2015, 6:10:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1323381
Run time	3 days 23 hours 9 min 40 sec
CPU time	3 days 11 hours 59 min 13 sec
Validate state	Invalid
Credit	2,993.82
Device peak FLOPS	3.94 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:25:39 (9120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:31:51 (7788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:13 (4176): No heartbeat from core client for 30 sec - exiting 08:35:14 (4176): No heartbeat from core client for 30 sec - exiting 08:35:15 (4176): No heartbeat from core client for 30 sec - exiting 08:35:16 (4176): No heartbeat from core client for 30 sec - exiting 08:35:17 (4176): No heartbeat from core client for 30 sec - exiting 08:35:18 (4176): No heartbeat from core client for 30 sec - exiting 08:35:19 (4176): No heartbeat from core client for 30 sec - exiting 08:35:20 (4176): No heartbeat from core client for 30 sec - exiting 08:35:21 (4176): No heartbeat from core client for 30 sec - exiting 08:35:22 (4176): No heartbeat from core client for 30 sec - exiting 08:35:23 (4176): No heartbeat from core client for 30 sec - exiting 08:35:24 (4176): No heartbeat from core client for 30 sec - exiting 08:35:25 (4176): No heartbeat from core client for 30 sec - exiting 08:35:26 (4176): No heartbeat from core client for 30 sec - exiting 08:35:27 (4176): No heartbeat from core client for 30 sec - exiting 08:35:28 (4176): No heartbeat from core client for 30 sec - exiting 08:35:29 (4176): No heartbeat from core client for 30 sec - exiting 08:35:30 (4176): No heartbeat from core client for 30 sec - exiting 08:35:31 (4176): No heartbeat from core client for 30 sec - exiting 08:35:32 (4176): No heartbeat from core client for 30 sec - exiting 08:35:33 (4176): No heartbeat from core client for 30 sec - exiting 08:35:34 (4176): No heartbeat from core client for 30 sec - exiting 08:35:35 (4176): No heartbeat from core client for 30 sec - exiting 08:35:36 (4176): No heartbeat from core client for 30 sec - exiting 08:35:37 (4176): No heartbeat from core client for 30 sec - exiting 08:35:38 (4176): No heartbeat from core client for 30 sec - exiting 08:35:39 (4176): No heartbeat from core client for 30 sec - exiting 08:35:40 (4176): No heartbeat from core client for 30 sec - exiting 08:35:41 (4176): No heartbeat from core client for 30 sec - exiting 08:35:42 (4176): No heartbeat from core client for 30 sec - exiting 08:35:43 (4176): No heartbeat from core client for 30 sec - exiting 08:35:44 (4176): No heartbeat from core client for 30 sec - exiting 08:35:45 (4176): No heartbeat from core client for 30 sec - exiting 08:35:46 (4176): No heartbeat from core client for 30 sec - exiting 08:35:47 (4176): No heartbeat from core client for 30 sec - exiting 08:35:48 (4176): No heartbeat from core client for 30 sec - exiting 08:35:49 (4176): No heartbeat from core client for 30 sec - exiting 08:35:50 (4176): No heartbeat from core client for 30 sec - exiting 08:35:51 (4176): No heartbeat from core client for 30 sec - exiting 08:35:52 (4176): No heartbeat from core client for 30 sec - exiting 08:35:53 (4176): No heartbeat from core client for 30 sec - exiting 08:35:54 (4176): No heartbeat from core client for 30 sec - exiting 08:35:55 (4176): No heartbeat from core client for 30 sec - exiting 08:35:56 (4176): No heartbeat from core client for 30 sec - exiting 08:35:57 (4176): No heartbeat from core client for 30 sec - exiting 08:35:58 (4176): No heartbeat from core client for 30 sec - exiting 08:35:59 (4176): No heartbeat from core client for 30 sec - exiting 08:36:00 (4176): No heartbeat from core client for 30 sec - exiting 08:36:01 (4176): No heartbeat from core client for 30 sec - exiting 08:36:02 (4176): No heartbeat from core client for 30 sec - exiting 08:36:03 (4176): No heartbeat from core client for 30 sec - exiting 08:36:04 (4176): No heartbeat from core client for 30 sec - exiting 08:36:05 (4176): No heartbeat from core client for 30 sec - exiting 08:36:06 (4176): No heartbeat from core client for 30 sec - exiting 08:36:07 (4176): No heartbeat from core client for 30 sec - exiting 08:36:08 (4176): No heartbeat from core client for 30 sec - exiting 08:36:09 (4176): No heartbeat from core client for 30 sec - exiting 08:36:10 (4176): No heartbeat from core client for 30 sec - exiting 08:36:11 (4176): No heartbeat from core client for 30 sec - exiting 08:36:12 (4176): No heartbeat from core client for 30 sec - exiting 08:36:13 (4176): No heartbeat from core client for 30 sec - exiting 08:36:14 (4176): No heartbeat from core client for 30 sec - exiting 08:36:15 (4176): No heartbeat from core client for 30 sec - exiting 08:36:16 (4176): No heartbeat from core client for 30 sec - exiting 08:36:17 (4176): No heartbeat from core client for 30 sec - exiting 08:36:18 (4176): No heartbeat from core client for 30 sec - exiting 08:36:19 (4176): No heartbeat from core client for 30 sec - exiting 08:36:20 (4176): No heartbeat from core client for 30 sec - exiting 08:36:21 (4176): No heartbeat from core client for 30 sec - exiting 08:36:22 (4176): No heartbeat from core client for 30 sec - exiting 08:36:23 (4176): No heartbeat from core client for 30 sec - exiting 08:36:24 (4176): No heartbeat from core client for 30 sec - exiting 08:36:25 (4176): No heartbeat from core client for 30 sec - exiting 08:36:26 (4176): No heartbeat from core client for 30 sec - exiting 08:36:27 (4176): No heartbeat from core client for 30 sec - exiting 08:36:28 (4176): No heartbeat from core client for 30 sec - exiting 08:36:29 (4176): No heartbeat from core client for 30 sec - exiting 08:36:30 (4176): No heartbeat from core client for 30 sec - exiting 08:36:31 (4176): No heartbeat from core client for 30 sec - exiting 08:36:32 (4176): No heartbeat from core client for 30 sec - exiting 08:36:33 (4176): No heartbeat from core client for 30 sec - exiting 08:36:34 (4176): No heartbeat from core client for 30 sec - exiting 08:36:35 (4176): No heartbeat from core client for 30 sec - exiting 08:36:36 (4176): No heartbeat from core client for 30 sec - exiting 08:36:37 (4176): No heartbeat from core client for 30 sec - exiting 08:36:38 (4176): No heartbeat from core client for 30 sec - exiting 08:36:39 (4176): No heartbeat from core client for 30 sec - exiting 08:36:40 (4176): No heartbeat from core client for 30 sec - exiting 08:36:41 (4176): No heartbeat from core client for 30 sec - exiting 08:36:42 (4176): No heartbeat from core client for 30 sec - exiting 08:36:43 (4176): No heartbeat from core client for 30 sec - exiting 08:36:44 (4176): No heartbeat from core client for 30 sec - exiting 08:36:45 (4176): No heartbeat from core client for 30 sec - exiting 08:36:46 (4176): No heartbeat from core client for 30 sec - exiting 08:36:47 (4176): No heartbeat from core client for 30 sec - exiting 08:36:48 (4176): No heartbeat from core client for 30 sec - exiting 08:36:49 (4176): No heartbeat from core client for 30 sec - exiting 08:36:50 (4176): No heartbeat from core client for 30 sec - exiting 08:36:51 (4176): No heartbeat from core client for 30 sec - exiting 08:36:52 (4176): No heartbeat from core client for 30 sec - exiting 08:36:53 (4176): No heartbeat from core client for 30 sec - exiting 08:36:54 (4176): No heartbeat from core client for 30 sec - exiting 08:36:55 (4176): No heartbeat from core client for 30 sec - exiting 08:36:56 (4176): No heartbeat from core client for 30 sec - exiting 08:36:57 (4176): No heartbeat from core client for 30 sec - exiting 08:36:58 (4176): No heartbeat from core client for 30 sec - exiting 08:36:59 (4176): No heartbeat from core client for 30 sec - exiting 08:37:00 (4176): No heartbeat from core client for 30 sec - exiting 08:37:01 (4176): No heartbeat from core client for 30 sec - exiting 08:37:02 (4176): No heartbeat from core client for 30 sec - exiting 08:37:03 (4176): No heartbeat from core client for 30 sec - exiting 08:37:04 (4176): No heartbeat from core client for 30 sec - exiting 08:37:05 (4176): No heartbeat from core client for 30 sec - exiting 08:37:06 (4176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:38:51 (6120): No heartbeat from core client for 30 sec - exiting 08:38:52 (6120): No heartbeat from core client for 30 sec - exiting 08:38:53 (6120): No heartbeat from core client for 30 sec - exiting 08:38:54 (6120): No heartbeat from core client for 30 sec - exiting 08:38:55 (6120): No heartbeat from core client for 30 sec - exiting 08:38:56 (6120): No heartbeat from core client for 30 sec - exiting 08:38:57 (6120): No heartbeat from core client for 30 sec - exiting 08:38:58 (6120): No heartbeat from core client for 30 sec - exiting 08:38:59 (6120): No heartbeat from core client for 30 sec - exiting 08:39:00 (6120): No heartbeat from core client for 30 sec - exiting 08:39:01 (6120): No heartbeat from core client for 30 sec - exiting 08:39:02 (6120): No heartbeat from core client for 30 sec - exiting 08:39:03 (6120): No heartbeat from core client for 30 sec - exiting 08:39:04 (6120): No heartbeat from core client for 30 sec - exiting 08:39:05 (6120): No heartbeat from core client for 30 sec - exiting 08:39:06 (6120): No heartbeat from core client for 30 sec - exiting 08:39:07 (6120): No heartbeat from core client for 30 sec - exiting 08:39:08 (6120): No heartbeat from core client for 30 sec - exiting 08:39:09 (6120): No heartbeat from core client for 30 sec - exiting 08:39:10 (6120): No heartbeat from core client for 30 sec - exiting 08:39:11 (6120): No heartbeat from core client for 30 sec - exiting 08:39:12 (6120): No heartbeat from core client for 30 sec - exiting 08:39:13 (6120): No heartbeat from core client for 30 sec - exiting 08:39:14 (6120): No heartbeat from core client for 30 sec - exiting 08:39:15 (6120): No heartbeat from core client for 30 sec - exiting 08:39:16 (6120): No heartbeat from core client for 30 sec - exiting 08:39:17 (6120): No heartbeat from core client for 30 sec - exiting 08:39:18 (6120): No heartbeat from core client for 30 sec - exiting 08:39:19 (6120): No heartbeat from core client for 30 sec - exiting 08:39:20 (6120): No heartbeat from core client for 30 sec - exiting 08:39:21 (6120): No heartbeat from core client for 30 sec - exiting 08:39:22 (6120): No heartbeat from core client for 30 sec - exiting 08:39:23 (6120): No heartbeat from core client for 30 sec - exiting 08:39:24 (6120): No heartbeat from core client for 30 sec - exiting 08:39:25 (6120): No heartbeat from core client for 30 sec - exiting 08:39:26 (6120): No heartbeat from core client for 30 sec - exiting 08:39:27 (6120): No heartbeat from core client for 30 sec - exiting 08:39:28 (6120): No heartbeat from core client for 30 sec - exiting 08:39:29 (6120): No heartbeat from core client for 30 sec - exiting 08:39:30 (6120): No heartbeat from core client for 30 sec - exiting 08:39:31 (6120): No heartbeat from core client for 30 sec - exiting 08:39:32 (6120): No heartbeat from core client for 30 sec - exiting 08:39:33 (6120): No heartbeat from core client for 30 sec - exiting 08:39:34 (6120): No heartbeat from core client for 30 sec - exiting 08:39:35 (6120): No heartbeat from core client for 30 sec - exiting 08:39:36 (6120): No heartbeat from core client for 30 sec - exiting 08:39:37 (6120): No heartbeat from core client for 30 sec - exiting 08:39:38 (6120): No heartbeat from core client for 30 sec - exiting 08:39:39 (6120): No heartbeat from core client for 30 sec - exiting 08:39:40 (6120): No heartbeat from core client for 30 sec - exiting 08:39:41 (6120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:42:09 (6824): No heartbeat from core client for 30 sec - exiting 08:42:10 (6824): No heartbeat from core client for 30 sec - exiting 08:42:11 (6824): No heartbeat from core client for 30 sec - exiting 08:42:12 (6824): No heartbeat from core client for 30 sec - exiting 08:42:13 (6824): No heartbeat from core client for 30 sec - exiting 08:42:14 (6824): No heartbeat from core client for 30 sec - exiting 08:42:15 (6824): No heartbeat from core client for 30 sec - exiting 08:42:16 (6824): No heartbeat from core client for 30 sec - exiting 08:42:17 (6824): No heartbeat from core client for 30 sec - exiting 08:42:18 (6824): No heartbeat from core client for 30 sec - exiting 08:42:19 (6824): No heartbeat from core client for 30 sec - exiting 08:42:20 (6824): No heartbeat from core client for 30 sec - exiting 08:42:21 (6824): No heartbeat from core client for 30 sec - exiting 08:42:22 (6824): No heartbeat from core client for 30 sec - exiting 08:42:23 (6824): No heartbeat from core client for 30 sec - exiting 08:42:24 (6824): No heartbeat from core client for 30 sec - exiting 08:42:25 (6824): No heartbeat from core client for 30 sec - exiting 08:42:26 (6824): No heartbeat from core client for 30 sec - exiting 08:42:27 (6824): No heartbeat from core client for 30 sec - exiting 08:42:28 (6824): No heartbeat from core client for 30 sec - exiting 08:42:29 (6824): No heartbeat from core client for 30 sec - exiting 08:42:30 (6824): No heartbeat from core client for 30 sec - exiting 08:42:31 (6824): No heartbeat from core client for 30 sec - exiting 08:42:32 (6824): No heartbeat from core client for 30 sec - exiting 08:42:33 (6824): No heartbeat from core client for 30 sec - exiting 08:42:34 (6824): No heartbeat from core client for 30 sec - exiting 08:42:35 (6824): No heartbeat from core client for 30 sec - exiting 08:42:36 (6824): No heartbeat from core client for 30 sec - exiting 08:42:37 (6824): No heartbeat from core client for 30 sec - exiting 08:42:38 (6824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:16 (5480): No heartbeat from core client for 30 sec - exiting 08:45:17 (5480): No heartbeat from core client for 30 sec - exiting 08:45:18 (5480): No heartbeat from core client for 30 sec - exiting 08:45:19 (5480): No heartbeat from core client for 30 sec - exiting 08:45:20 (5480): No heartbeat from core client for 30 sec - exiting 08:45:21 (5480): No heartbeat from core client for 30 sec - exiting 08:45:22 (5480): No heartbeat from core client for 30 sec - exiting 08:45:23 (5480): No heartbeat from core client for 30 sec - exiting 08:45:24 (5480): No heartbeat from core client for 30 sec - exiting 08:45:25 (5480): No heartbeat from core client for 30 sec - exiting 08:45:26 (5480): No heartbeat from core client for 30 sec - exiting 08:45:27 (5480): No heartbeat from core client for 30 sec - exiting 08:45:28 (5480): No heartbeat from core client for 30 sec - exiting 08:45:29 (5480): No heartbeat from core client for 30 sec - exiting 08:45:30 (5480): No heartbeat from core client for 30 sec - exiting 08:45:31 (5480): No heartbeat from core client for 30 sec - exiting 08:45:32 (5480): No heartbeat from core client for 30 sec - exiting 08:45:33 (5480): No heartbeat from core client for 30 sec - exiting 08:45:34 (5480): No heartbeat from core client for 30 sec - exiting 08:45:35 (5480): No heartbeat from core client for 30 sec - exiting 08:45:36 (5480): No heartbeat from core client for 30 sec - exiting 08:45:37 (5480): No heartbeat from core client for 30 sec - exiting 08:45:38 (5480): No heartbeat from core client for 30 sec - exiting 08:45:39 (5480): No heartbeat from core client for 30 sec - exiting 08:45:40 (5480): No heartbeat from core client for 30 sec - exiting 08:45:41 (5480): No heartbeat from core client for 30 sec - exiting 08:45:42 (5480): No heartbeat from core client for 30 sec - exiting 08:45:43 (5480): No heartbeat from core client for 30 sec - exiting 08:45:44 (5480): No heartbeat from core client for 30 sec - exiting 08:45:45 (5480): No heartbeat from core client for 30 sec - exiting 08:45:46 (5480): No heartbeat from core client for 30 sec - exiting 08:45:47 (5480): No heartbeat from core client for 30 sec - exiting 08:45:48 (5480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:51:18 (6104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=69904, selfPID=68764, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=8620, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8736, selfPID=8736, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8736, selfPID=7584, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_f79n_2012_1_009778784_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f79n_2012_1_009778784_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f79n_2012_1_009778784_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f79n_2012_1_009778784_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f79n_2012_1_009778784_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f79n_2012_1_009778784_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 May 2015 08:17:38	1323381	18344249	hadam3p_anz_f79n_2012_1_009778784_0	69,419	295,635	4.2587
19 May 2015 15:19:49	1323381	18344249	hadam3p_anz_f79n_2012_1_009778784_0	57,899	247,741	4.2788
18 May 2015 13:29:05	1323381	18344249	hadam3p_anz_f79n_2012_1_009778784_0	46,379	199,564	4.3029
18 May 2015 11:31:09	1323381	18344249	hadam3p_anz_f79n_2012_1_009778784_0	34,859	152,383	4.3714
12 May 2015 07:52:23	1323381	18344249	hadam3p_anz_f79n_2012_1_009778784_0	23,339	103,870	4.4505
08 May 2015 19:23:01	1323381	18344249	hadam3p_anz_f79n_2012_1_009778784_0	11,819	54,176	4.5838