Task 13812238

Name	hadam3p_pnw_6x30_2004_1_007591460_1
Workunit	7769590
Created	23 Dec 2011, 7:44:31 UTC
Sent	23 Dec 2011, 7:48:15 UTC
Report deadline	4 Dec 2012, 13:08:15 UTC
Received	17 Jun 2012, 5:45:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1186357
Run time	13 hours 48 min 13 sec
CPU time	12 hours 1 min 18 sec
Validate state	Invalid
Credit	253.03
Device peak FLOPS	3.08 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:50:30 (376): No heartbeat from core client for 30 sec - exiting 19:50:31 (376): No heartbeat from core client for 30 sec - exiting 19:50:32 (376): No heartbeat from core client for 30 sec - exiting 19:50:33 (376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3696, selfPID=3696, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4936, selfPID=4936, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4192, selfPID=4192, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4136, selfPID=2984, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:55:09 (2728): No heartbeat from core client for 30 sec - exiting 08:55:10 (2728): No heartbeat from core client for 30 sec - exiting 08:55:11 (2728): No heartbeat from core client for 30 sec - exiting 08:55:13 (2728): No heartbeat from core client for 30 sec - exiting 08:55:14 (2728): No heartbeat from core client for 30 sec - exiting 08:55:15 (2728): No heartbeat from core client for 30 sec - exiting 08:55:16 (2728): No heartbeat from core client for 30 sec - exiting 08:55:17 (2728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... RegionaCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3836, selfPID=3836, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5164, selfPID=2020, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=25744, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... RegionalController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2408, selfPID=488, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 17:47:26 (2368): No heartbeat from core client for 30 sec - exiting 17:47:27 (2368): No heartbeat from core client for 30 sec - exiting 17:47:28 (2368): No heartbeat from core client for 30 sec - exiting 17:47:29 (2368): No heartbeat from core client for 30 sec - exiting 17:47:30 (2368): No heartbeat from core client for 30 sec - exiting 17:47:31 (2368): No heartbeat from core client for 30 sec - exiting 17:47:33 (2368): No heartbeat from core client for 30 sec - exiting 17:47:34 (2368): No heartbeat from core client for 30 sec - exiting 17:47:35 (2368): No heartbeat from core client for 30 sec - exiting 17:47:36 (2368): No heartbeat from core client for 30 sec - exiting 17:47:37 (2368): No heartbeat from core client for 30 sec - exiting 17:47:38 (2368): No heartbeat from core client for 30 sec - exiting 17:47:39 (2368): No heartbeat from core client for 30 sec - exiting 17:47:40 (2368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4792, selfPID=4792, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4484, selfPID=4484, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4188, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4300, selfPID=1928, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:37:09 (4552): No heartbeat from core client for 30 sec - exiting 13:37:10 (4552): No heartbeat from core client for 30 sec - exiting 13:37:11 (4552): No heartbeat from core client for 30 sec - exiting 13:37:12 (4552): No heartbeat from core client for 30 sec - exiting 13:37:13 (4552): No heartbeat from core client for 30 sec - exiting 13:37:14 (4552): No heartbeat from core client for 30 sec - exiting 13:37:16 (4552): No heartbeat from core client for 30 sec - exiting 13:37:17 (4552): No heartbeat from core client for 30 sec - exiting 13:37:18 (4552): No heartbeat from core client for 30 sec - exiting 13:37:19 (4552): No heartbeat from core client for 30 sec - exiting 13:37:20 (4552): No heartbeat from core client for 30 sec - exiting 13:37:21 (4552): No heartbeat from core client for 30 sec - exiting 13:37:22 (4552): No heartbeat from core client for 30 sec - exiting 13:37:23 (4552): No heartbeat from core client for 30 sec - exiting 13:37:24 (4552): No heartbeat from core client for 30 sec - exiting 13:37:25 (4552): No heartbeat from core client for 30 sec - exiting 13:37:26 (4552): No heartbeat from core client for 30 sec - exiting 13:37:28 (4552): No heartbeat from core client for 30 sec - exiting 13:37:29 (4552): No heartbeat from core client for 30 sec - exiting 13:37:30 (4552): No heartbeat from core client for 30 sec - exiting 13:37:31 (4552): No heartbeat from core client for 30 sec - exiting 13:37:32 (4552): No heartbeat from core client for 30 sec - exiting 13:37:33 (4552): No heartbeat from core client for 30 sec - exiting 13:37:34 (4552): No heartbeat from core client for 30 sec - exiting 13:37:35 (4552): No heartbeat from core client for 30 sec - exiting 13:37:36 (4552): No heartbeat from core client for 30 sec - exiting 13:37:37 (4552): No heartbeat from core client for 30 sec - exiting 13:37:39 (4552): No heartbeat from core client for 30 sec - exiting 13:37:40 (4552): No heartbeat from core client for 30 sec - exiting 13:37:41 (4552): No heartbeat from core client for 30 sec - exiting 13:37:42 (4552): No heartbeat from core client for 30 sec - exiting 13:37:43 (4552): No heartbeat from core client for 30 sec - exiting 13:37:44 (4552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1404, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4920, selfPID=4920, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1952, selfPID=4880, iMonCtr=1 Model crash detected, will try to restart... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1952, selfPID=1952, iMonCtr=2 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 1 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_2.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_3.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_4.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_6x30_2004_1_007591460_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Jun 2012 07:35:55	1186357	13812238	hadam3p_pnw_6x30_2004_1_007591460_1	11,645	39,122	3.3596
17 May 2012 05:55:49	1186357	13812238	hadam3p_pnw_6x30_2004_1_007591460_1	11,639	38,603	3.3167
13 May 2012 08:57:58	1186357	13812238	hadam3p_pnw_6x30_2004_1_007591460_1	11,628	38,105	3.2770
06 May 2012 17:00:38	1186357	13812238	hadam3p_pnw_6x30_2004_1_007591460_1	11,619	37,630	3.2387
06 May 2012 17:00:38	1186357	13812238	hadam3p_pnw_6x30_2004_1_007591460_1	11,616	37,167	3.1996