Task 17665827

Name	hadam3p_anz_f2gw_2012_1_009296732_1
Workunit	9380920
Created	3 Jan 2015, 17:07:03 UTC
Sent	3 Jan 2015, 17:43:11 UTC
Report deadline	16 Dec 2015, 23:03:11 UTC
Received	31 Jan 2015, 9:30:15 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1345795
Run time	5 days 12 hours 55 min 27 sec
CPU time	4 days 23 hours 20 min 4 sec
Validate state	Invalid
Credit	3,490.64
Device peak FLOPS	3.61 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.27</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9528, selfPID=14996, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12200, selfPID=12200, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14212, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8044, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6764, selfPID=9792, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:42:26 (5744): No heartbeat from core client for 30 sec - exiting 19:42:27 (5744): No heartbeat from core client for 30 sec - exiting 19:42:28 (5744): No heartbeat from core client for 30 sec - exiting 19:42:29 (5744): No heartbeat from core client for 30 sec - exiting 19:42:30 (5744): No heartbeat from core client for 30 sec - exiting 19:42:31 (5744): No heartbeat from core client for 30 sec - exiting 19:42:32 (5744): No heartbeat from core client for 30 sec - exiting 19:42:33 (5744): No heartbeat from core client for 30 sec - exiting 19:42:34 (5744): No heartbeat from core client for 30 sec - exiting 19:42:35 (5744): No heartbeat from core client for 30 sec - exiting 19:42:37 (5744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:17:52 (2580): No heartbeat from core client for 30 sec - exiting 19:17:54 (2580): No heartbeat from core client for 30 sec - exiting 19:17:55 (2580): No heartbeat from core client for 30 sec - exiting 19:17:56 (2580): No heartbeat from core client for 30 sec - exiting 19:17:57 (2580): No heartbeat from core client for 30 sec - exiting 19:17:58 (2580): No heartbeat from core client for 30 sec - exiting 19:17:59 (2580): No heartbeat from core client for 30 sec - exiting 19:18:00 (2580): No heartbeat from core client for 30 sec - exiting 19:18:01 (2580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:33:57 (8336): No heartbeat from core client for 30 sec - exiting 19:33:58 (8336): No heartbeat from core client for 30 sec - exiting 19:33:59 (8336): No heartbeat from core client for 30 sec - exiting 19:34:00 (8336): No heartbeat from core client for 30 sec - exiting 19:34:01 (8336): No heartbeat from core client for 30 sec - exiting 19:34:03 (8336): No heartbeat from core client for 30 sec - exiting 19:34:04 (8336): No heartbeat from core client for 30 sec - exiting 19:34:05 (8336): No heartbeat from core client for 30 sec - exiting 19:34:06 (8336): No heartbeat from core client for 30 sec - exiting 19:34:07 (8336): No heartbeat from core client for 30 sec - exiting 19:34:08 (8336): No heartbeat from core client for 30 sec - exiting 19:34:09 (8336): No heartbeat from core client for 30 sec - exiting 19:34:10 (8336): No heartbeat from core client for 30 sec - exiting 19:34:11 (8336): No heartbeat from core client for 30 sec - exiting 19:34:12 (8336): No heartbeat from core client for 30 sec - exiting 19:34:13 (8336): No heartbeat from core client for 30 sec - exiting 19:34:15 (8336): No heartbeat from core client for 30 sec - exiting 19:34:16 (8336): No heartbeat from core client for 30 sec - exiting 19:34:17 (8336): No heartbeat from core client for 30 sec - exiting 19:34:18 (8336): No heartbeat from core client for 30 sec - exiting 19:34:19 (8336): No heartbeat from core client for 30 sec - exiting 19:34:20 (8336): No heartbeat from core client for 30 sec - exiting 19:34:21 (8336): No heartbeat from core client for 30 sec - exiting 19:34:22 (8336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:31:47 (9100): No heartbeat from core client for 30 sec - exiting 08:31:48 (9100): No heartbeat from core client for 30 sec - exiting 08:31:49 (9100): No heartbeat from core client for 30 sec - exiting 08:31:50 (9100): No heartbeat from core client for 30 sec - exiting 08:31:51 (9100): No heartbeat from core client for 30 sec - exiting 08:31:53 (9100): No heartbeat from core client for 30 sec - exiting 08:31:54 (9100): No heartbeat from core client for 30 sec - exiting 08:31:55 (9100): No heartbeat from core client for 30 sec - exiting 08:31:56 (9100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8588, selfPID=8884, iMonCtr=1 18:12:21 (7700): No heartbeat from core client for 30 sec - exiting 18:12:22 (7700): No heartbeat from core client for 30 sec - exiting 18:12:23 (7700): No heartbeat from core client for 30 sec - exiting 18:12:24 (7700): No heartbeat from core client for 30 sec - exiting 18:12:26 (7700): No heartbeat from core client for 30 sec - exiting 18:12:27 (7700): No heartbeat from core client for 30 sec - exiting 18:12:28 (7700): No heartbeat from core client for 30 sec - exiting 18:12:29 (7700): No heartbeat from core client for 30 sec - exiting 18:12:30 (7700): No heartbeat from core client for 30 sec - exiting 18:12:31 (7700): No heartbeat from core client for 30 sec - exiting 18:12:32 (7700): No heartbeat from core client for 30 sec - exiting 18:12:33 (7700): No heartbeat from core client for 30 sec - exiting 18:12:34 (7700): No heartbeat from core client for 30 sec - exiting 18:12:35 (7700): No heartbeat from core client for 30 sec - exiting 18:12:36 (7700): No heartbeat from core client for 30 sec - exiting 18:12:38 (7700): No heartbeat from core client for 30 sec - exiting 18:12:39 (7700): No heartbeat from core client for 30 sec - exiting 18:12:40 (7700): No heartbeat from core client for 30 sec - exiting 18:12:41 (7700): No heartbeat from core client for 30 sec - exiting 18:12:42 (7700): No heartbeat from core client for 30 sec - exiting 18:12:43 (7700): No heartbeat from core client for 30 sec - exiting 18:12:44 (7700): No heartbeat from core client for 30 sec - exiting 18:12:45 (7700): No heartbeat from core client for 30 sec - exiting 18:12:46 (7700): No heartbeat from core client for 30 sec - exiting 18:12:47 (7700): No heartbeat from core client for 30 sec - exiting 18:12:48 (7700): No heartbeat from core client for 30 sec - exiting 18:12:49 (7700): No heartbeat from core client for 30 sec - exiting 18:12:51 (7700): No heartbeat from core client for 30 sec - exiting 18:12:52 (7700): No heartbeat from core client for 30 sec - exiting 18:12:53 (7700): No heartbeat from core client for 30 sec - exiting 18:12:54 (7700): No heartbeat from core client for 30 sec - exiting 18:12:55 (7700): No heartbeat from core client for 30 sec - exiting 18:12:56 (7700): No heartbeat from core client for 30 sec - exiting 18:12:57 (7700): No heartbeat from core client for 30 sec - exiting 18:12:58 (7700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7876, selfPID=13052, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7876, selfPID=7876, iMonCtr=1 12:22:35 (8460): No heartbeat from core client for 30 sec - exiting 12:22:37 (8460): No heartbeat from core client for 30 sec - exiting 12:22:38 (8460): No heartbeat from core client for 30 sec - exiting 12:22:39 (8460): No heartbeat from core client for 30 sec - exiting 12:22:40 (8460): No heartbeat from core client for 30 sec - exiting 12:22:41 (8460): No heartbeat from core client for 30 sec - exiting 12:22:42 (8460): No heartbeat from core client for 30 sec - exiting 12:22:43 (8460): No heartbeat from core client for 30 sec - exiting 12:22:44 (8460): No heartbeat from core client for 30 sec - exiting 12:22:45 (8460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:47:23 (6888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4868, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:52:49 (8644): No heartbeat from core client for 30 sec - exiting 10:52:50 (8644): No heartbeat from core client for 30 sec - exiting 10:52:51 (8644): No heartbeat from core client for 30 sec - exiting 10:52:52 (8644): No heartbeat from core client for 30 sec - exiting 10:52:53 (8644): No heartbeat from core client for 30 sec - exiting 10:52:54 (8644): No heartbeat from core client for 30 sec - exiting 10:52:55 (8644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:36:07 (1336): No heartbeat from core client for 30 sec - exiting 13:36:08 (1336): No heartbeat from core client for 30 sec - exiting 13:36:09 (1336): No heartbeat from core client for 30 sec - exiting 13:36:10 (1336): No heartbeat from core client for 30 sec - exiting 13:36:11 (1336): No heartbeat from core client for 30 sec - exiting 13:36:12 (1336): No heartbeat from core client for 30 sec - exiting 13:36:13 (1336): No heartbeat from core client for 30 sec - exiting 13:36:14 (1336): No heartbeat from core client for 30 sec - exiting 13:36:15 (1336): No heartbeat from core client for 30 sec - exiting 13:36:16 (1336): No heartbeat from core client for 30 sec - exiting 13:36:17 (1336): No heartbeat from core client for 30 sec - exiting 13:36:18 (1336): No heartbeat from core client for 30 sec - exiting 13:36:20 (1336): No heartbeat from core client for 30 sec - exiting 13:36:21 (1336): No heartbeat from core client for 30 sec - exiting 13:36:22 (1336): No heartbeat from core client for 30 sec - exiting 13:36:23 (1336): No heartbeat from core client for 30 sec - exiting 13:36:24 (1336): No heartbeat from core client for 30 sec - exiting 13:36:25 (1336): No heartbeat from core client for 30 sec - exiting 13:36:26 (1336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:21:58 (7032): No heartbeat from core client for 30 sec - exiting 21:21:59 (7032): No heartbeat from core client for 30 sec - exiting 21:22:00 (7032): No heartbeat from core client for 30 sec - exiting 21:22:01 (7032): No heartbeat from core client for 30 sec - exiting 21:22:02 (7032): No heartbeat from core client for 30 sec - exiting 21:22:03 (7032): No heartbeat from core client for 30 sec - exiting 21:22:04 (7032): No heartbeat from core client for 30 sec - exiting 21:22:05 (7032): No heartbeat from core client for 30 sec - exiting 21:22:07 (7032): No heartbeat from core client for 30 sec - exiting 21:22:08 (7032): No heartbeat from core client for 30 sec - exiting 21:22:09 (7032): No heartbeat from core client for 30 sec - exiting 21:22:10 (7032): No heartbeat from core client for 30 sec - exiting 21:22:11 (7032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:13:56 (7936): No heartbeat from core client for 30 sec - exiting 11:13:58 (7936): No heartbeat from core client for 30 sec - exiting 11:13:59 (7936): No heartbeat from core client for 30 sec - exiting 11:14:00 (7936): No heartbeat from core client for 30 sec - exiting 11:14:01 (7936): No heartbeat from core client for 30 sec - exiting 11:14:02 (7936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=10628, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8304, selfPID=11068, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_f2gw_2012_1_009296732_1_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f2gw_2012_1_009296732_1_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f2gw_2012_1_009296732_1_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f2gw_2012_1_009296732_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f2gw_2012_1_009296732_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Jan 2015 18:05:01	1345795	17665827	hadam3p_anz_f2gw_2012_1_009296732_1	80,939	382,025	4.7199
26 Jan 2015 20:35:25	1345795	17665827	hadam3p_anz_f2gw_2012_1_009296732_1	69,419	323,389	4.6585
24 Jan 2015 16:56:28	1345795	17665827	hadam3p_anz_f2gw_2012_1_009296732_1	57,899	270,933	4.6794
21 Jan 2015 19:17:23	1345795	17665827	hadam3p_anz_f2gw_2012_1_009296732_1	46,379	215,652	4.6498
17 Jan 2015 22:27:41	1345795	17665827	hadam3p_anz_f2gw_2012_1_009296732_1	34,859	162,897	4.6730
16 Jan 2015 13:33:22	1345795	17665827	hadam3p_anz_f2gw_2012_1_009296732_1	23,339	110,182	4.7209
13 Jan 2015 20:54:21	1345795	17665827	hadam3p_anz_f2gw_2012_1_009296732_1	11,819	57,147	4.8352