Task 18615621

Name	hadam3p_anz_d7m2_2013_1_009941812_0
Workunit	9969174
Created	23 Jun 2015, 12:53:31 UTC
Sent	25 Jun 2015, 17:09:34 UTC
Report deadline	6 Jun 2016, 22:29:34 UTC
Received	22 Jul 2015, 15:38:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1328194
Run time	12 days 9 hours 6 min 38 sec
CPU time	10 days 19 hours 10 min 41 sec
Validate state	Invalid
Credit	4,981.10
Device peak FLOPS	2.68 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:27:20 (2292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:53:00 (7456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:53:01 (7456): No heartbeat from core client for 30 sec - exiting 02:20:34 (7932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:20:35 (7932): No heartbeat from core client for 30 sec - exiting 02:20:36 (7932): No heartbeat from core client for 30 sec - exiting 02:20:37 (7932): No heartbeat from core client for 30 sec - exiting 02:20:38 (7932): No heartbeat from core client for 30 sec - exiting 02:20:39 (7932): No heartbeat from core client for 30 sec - exiting 02:20:40 (7932): No heartbeat from core client for 30 sec - exiting 02:20:42 (7932): No heartbeat from core client for 30 sec - exiting 02:20:43 (7932): No heartbeat from core client for 30 sec - exiting 02:20:44 (7932): No heartbeat from core client for 30 sec - exiting 02:20:45 (7932): No heartbeat from core client for 30 sec - exiting 02:20:46 (7932): No heartbeat from core client for 30 sec - exiting 02:20:47 (7932): No heartbeat from core client for 30 sec - exiting 02:20:48 (7932): No heartbeat from core client for 30 sec - exiting 02:20:49 (7932): No heartbeat from core client for 30 sec - exiting 02:20:59 (4616): Can't acquire lockfile (32) - waiting 35s 03:18:34 (4616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:18:35 (4616): No heartbeat from core client for 30 sec - exiting 03:18:36 (4616): No heartbeat from core client for 30 sec - exiting 03:18:37 (4616): No heartbeat from core client for 30 sec - exiting 03:18:39 (4616): No heartbeat from core client for 30 sec - exiting 03:30:51 (7392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:30:53 (7392): No heartbeat from core client for 30 sec - exiting 03:30:54 (7392): No heartbeat from core client for 30 sec - exiting 03:30:55 (7392): No heartbeat from core client for 30 sec - exiting 03:30:56 (7392): No heartbeat from core client for 30 sec - exiting 03:30:57 (7392): No heartbeat from core client for 30 sec - exiting 04:10:36 (6820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:10:37 (6820): No heartbeat from core client for 30 sec - exiting 04:10:38 (6820): No heartbeat from core client for 30 sec - exiting 05:20:59 (3500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:21:00 (3500): No heartbeat from core client for 30 sec - exiting 05:21:01 (3500): No heartbeat from core client for 30 sec - exiting 05:21:02 (3500): No heartbeat from core client for 30 sec - exiting 05:21:04 (3500): No heartbeat from core client for 30 sec - exiting 05:21:05 (3500): No heartbeat from core client for 30 sec - exiting 05:24:03 (4252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:27:10 (740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7436, selfPID=7436, iMonCtr=2 05:45:33 (2396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:34 (2396): No heartbeat from core client for 30 sec - exiting 05:45:35 (2396): No heartbeat from core client for 30 sec - exiting 05:45:36 (2396): No heartbeat from core client for 30 sec - exiting 05:45:37 (2396): No heartbeat from core client for 30 sec - exiting 05:45:38 (2396): No heartbeat from core client for 30 sec - exiting 05:45:39 (2396): No heartbeat from core client for 30 sec - exiting 05:45:41 (2396): No heartbeat from core client for 30 sec - exiting 06:58:51 (9140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:58:53 (9140): No heartbeat from core client for 30 sec - exiting 06:58:54 (9140): No heartbeat from core client for 30 sec - exiting 06:58:55 (9140): No heartbeat from core client for 30 sec - exiting 06:58:56 (9140): No heartbeat from core client for 30 sec - exiting 06:58:57 (9140): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:24:32 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:24:33 (5652): No heartbeat from core client for 30 sec - exiting 02:24:34 (5652): No heartbeat from core client for 30 sec - exiting 02:24:35 (5652): No heartbeat from core client for 30 sec - exiting 02:24:36 (5652): No heartbeat from core client for 30 sec - exiting 02:24:37 (5652): No heartbeat from core client for 30 sec - exiting 02:24:38 (5652): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1844, iMonCtr=2 Model crash detected, will try to restart... 12:02:16 (6472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:37:02 (1320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:37:04 (1320): No heartbeat from core client for 30 sec - exiting 10:37:05 (1320): No heartbeat from core client for 30 sec - exiting 10:37:06 (1320): No heartbeat from core client for 30 sec - exiting 10:37:07 (1320): No heartbeat from core client for 30 sec - exiting 10:37:08 (1320): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=1316, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8772, selfPID=8564, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_d7m2_2013_1_009941812_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_d7m2_2013_1_009941812_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Jul 2015 05:46:20	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	115,499	912,027	7.8964
20 Jul 2015 17:24:01	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	103,979	820,661	7.8926
18 Jul 2015 18:35:31	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	92,459	728,554	7.8798
16 Jul 2015 05:56:43	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	80,939	633,326	7.8247
14 Jul 2015 18:12:35	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	69,419	538,538	7.7578
13 Jul 2015 14:34:08	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	57,899	449,545	7.7643
12 Jul 2015 12:59:08	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	46,379	363,820	7.8445
11 Jul 2015 06:07:19	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	34,859	278,270	7.9827
08 Jul 2015 02:29:37	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	23,339	190,768	8.1738
03 Jul 2015 08:39:54	1328194	18615621	hadam3p_anz_d7m2_2013_1_009941812_0	11,819	96,608	8.1740