Task 14658485

Name	hadam3p_pnw_bg4e_1973_1_007907197_1
Workunit	8062309
Created	11 May 2012, 15:42:23 UTC
Sent	11 May 2012, 16:34:06 UTC
Report deadline	23 Apr 2013, 21:54:06 UTC
Received	24 May 2012, 13:57:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1194710
Run time	2 days 22 hours 36 min 59 sec
CPU time	2 days 17 hours 52 min 20 sec
Validate state	Invalid
Credit	1,503.98
Device peak FLOPS	2.50 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... 08:30:03 (1928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:21:50 (3584): No heartbeat from core client for 30 sec - exiting 19:21:51 (3584): No heartbeat from core client for 30 sec - exiting 19:21:52 (3584): No heartbeat from core client for 30 sec - exiting 19:21:53 (3584): No heartbeat from core client for 30 sec - exiting 19:21:54 (3584): No heartbeat from core client for 30 sec - exiting 19:21:55 (3584): No heartbeat from core client for 30 sec - exiting 19:21:56 (3584): No heartbeat from core client for 30 sec - exiting 19:21:57 (3584): No heartbeat from core client for 30 sec - exiting 19:21:58 (3584): No heartbeat from core client for 30 sec - exiting 19:21:59 (3584): No heartbeat from core client for 30 sec - exiting 19:22:00 (3584): No heartbeat from core client for 30 sec - exiting 19:22:01 (3584): No heartbeat from core client for 30 sec - exiting 19:22:02 (3584): No heartbeat from core client for 30 sec - exiting 19:22:03 (3584): No heartbeat from core client for 30 sec - exiting 19:22:04 (3584): No heartbeat from core client for 30 sec - exiting 19:22:05 (3584): No heartbeat from core client for 30 sec - exiting 19:22:06 (3584): No heartbeat from core client for 30 sec - exiting 19:22:07 (3584): No heartbeat from core client for 30 sec - exiting 19:22:08 (3584): No heartbeat from core client for 30 sec - exiting 19:22:09 (3584): No heartbeat from core client for 30 sec - exiting 19:22:10 (3584): No heartbeat from core client for 30 sec - exiting 19:22:11 (3584): No heartbeat from core client for 30 sec - exiting 19:22:12 (3584): No heartbeat from core client for 30 sec - exiting 19:22:13 (3584): No heartbeat from core client for 30 sec - exiting 19:22:14 (3584): No heartbeat from core client for 30 sec - exiting 19:22:15 (3584): No heartbeat from core client for 30 sec - exiting 19:22:16 (3584): No heartbeat from core client for 30 sec - exiting 19:22:17 (3584): No heartbeat from core client for 30 sec - exiting 19:22:18 (3584): No heartbeat from core client for 30 sec - exiting 19:22:19 (3584): No heartbeat from core client for 30 sec - exiting 19:22:20 (3584): No heartbeat from core client for 30 sec - exiting 19:22:21 (3584): No heartbeat from core client for 30 sec - exiting 19:22:22 (3584): No heartbeat from core client for 30 sec - exiting 19:22:23 (3584): No heartbeat from core client for 30 sec - exiting 19:22:24 (3584): No heartbeat from core client for 30 sec - exiting 19:22:25 (3584): No heartbeat from core client for 30 sec - exiting 19:22:26 (3584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:27 (3584): No heartbeat from core client for 30 sec - exiting 19:22:28 (3584): No heartbeat from core client for 30 sec - exiting 19:22:29 (3584): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=920, selfPID=920, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5284, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5688, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 4 07:53:44 (4100): No heartbeat from core client for 30 sec - exiting 07:53:45 (4100): No heartbeat from core client for 30 sec - exiting 07:53:46 (4100): No heartbeat from core client for 30 sec - exiting 07:53:47 (4100): No heartbeat from core client for 30 sec - exiting 07:53:48 (4100): No heartbeat from core client for 30 sec - exiting 07:53:49 (4100): No heartbeat from core client for 30 sec - exiting 07:53:50 (4100): No heartbeat from core client for 30 sec - exiting 07:53:51 (4100): No heartbeat from core client for 30 sec - exiting 07:53:52 (4100): No heartbeat from core client for 30 sec - exiting 07:53:53 (4100): No heartbeat from core client for 30 sec - exiting 07:53:54 (4100): No heartbeat from core client for 30 sec - exiting 07:53:55 (4100): No heartbeat from core client for 30 sec - exiting 07:53:56 (4100): No heartbeat from core client for 30 sec - exiting 07:53:57 (4100): No heartbeat from core client for 30 sec - exiting 07:53:58 (4100): No heartbeat from core client for 30 sec - exiting 07:53:59 (4100): No heartbeat from core client for 30 sec - exiting 07:54:00 (4100): No heartbeat from core client for 30 sec - exiting 07:54:01 (4100): No heartbeat from core client for 30 sec - exiting 07:54:02 (4100): No heartbeat from core client for 30 sec - exiting 07:54:03 (4100): No heartbeat from core client for 30 sec - exiting 07:54:04 (4100): No heartbeat from core client for 30 sec - exiting 07:54:05 (4100): No heartbeat from core client for 30 sec - exiting 07:54:06 (4100): No heartbeat from core client for 30 sec - exiting 07:54:07 (4100): No heartbeat from core client for 30 sec - exiting 07:54:08 (4100): No heartbeat from core client for 30 sec - exiting 07:54:09 (4100): No heartbeat from core client for 30 sec - exiting 07:54:10 (4100): No heartbeat from core client for 30 sec - exiting 07:54:11 (4100): No heartbeat from core client for 30 sec - exiting 07:54:12 (4100): No heartbeat from core client for 30 sec - exiting 07:54:13 (4100): No heartbeat from core client for 30 sec - exiting 07:54:14 (4100): No heartbeat from core client for 30 sec - exiting 07:54:15 (4100): No heartbeat from core client for 30 sec - exiting 07:54:16 (4100): No heartbeat from core client for 30 sec - exiting 07:54:17 (4100): No heartbeat from core client for 30 sec - exiting 07:54:18 (4100): No heartbeat from core client for 30 sec - exiting 07:54:19 (4100): No heartbeat from core client for 30 sec - exiting 07:54:20 (4100): No heartbeat from core client for 30 sec - exiting 07:54:21 (4100): No heartbeat from core client for 30 sec - exiting 07:54:22 (4100): No heartbeat from core client for 30 sec - exiting 07:54:23 (4100): No heartbeat from core client for 30 sec - exiting 07:54:24 (4100): No heartbeat from core client for 30 sec - exiting 07:54:25 (4100): No heartbeat from core client for 30 sec - exiting 07:54:26 (4100): No heartbeat from core client for 30 sec - exiting 07:54:27 (4100): No heartbeat from core client for 30 sec - exiting 07:54:28 (4100): No heartbeat from core client for 30 sec - exiting 07:54:29 (4100): No heartbeat from core client for 30 sec - exiting 07:54:30 (4100): No heartbeat from core client for 30 sec - exiting 07:54:31 (4100): No heartbeat from core client for 30 sec - exiting 07:54:32 (4100): No heartbeat from core client for 30 sec - exiting 07:54:33 (4100): No heartbeat from core client for 30 sec - exiting 07:54:34 (4100): No heartbeat from core client for 30 sec - exiting 07:54:35 (4100): No heartbeat from core client for 30 sec - exiting 07:54:36 (4100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:17:49 (5704): No heartbeat from core client for 30 sec - exiting 14:17:50 (5704): No heartbeat from core client for 30 sec - exiting 14:17:51 (5704): No heartbeat from core client for 30 sec - exiting 14:17:52 (5704): No heartbeat from core client for 30 sec - exiting 14:17:53 (5704): No heartbeat from core client for 30 sec - exiting 14:17:54 (5704): No heartbeat from core client for 30 sec - exiting 14:17:55 (5704): No heartbeat from core client for 30 sec - exiting 14:17:56 (5704): No heartbeat from core client for 30 sec - exiting 14:17:57 (5704): No heartbeat from core client for 30 sec - exiting 14:17:58 (5704): No heartbeat from core client for 30 sec - exiting 14:17:59 (5704): No heartbeat from core client for 30 sec - exiting 14:18:00 (5704): No heartbeat from core client for 30 sec - exiting 14:18:01 (5704): No heartbeat from core client for 30 sec - exiting 14:18:02 (5704): No heartbeat from core client for 30 sec - exiting 14:18:03 (5704): No heartbeat from core client for 30 sec - exiting 14:18:04 (5704): No heartbeat from core client for 30 sec - exiting 14:18:05 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_bg4e_1973_1_007907197_1_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg4e_1973_1_007907197_1_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg4e_1973_1_007907197_1_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg4e_1973_1_007907197_1_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg4e_1973_1_007907197_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg4e_1973_1_007907197_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 May 2012 09:40:57	1194710	14658485	hadam3p_pnw_bg4e_1973_1_007907197_1	69,216	224,431	3.2425
23 May 2012 14:39:17	1194710	14658485	hadam3p_pnw_bg4e_1973_1_007907197_1	57,696	185,323	3.2121
22 May 2012 19:55:15	1194710	14658485	hadam3p_pnw_bg4e_1973_1_007907197_1	46,176	146,916	3.1817
13 May 2012 22:35:00	1194710	14658485	hadam3p_pnw_bg4e_1973_1_007907197_1	34,656	110,387	3.1852
13 May 2012 11:40:55	1194710	14658485	hadam3p_pnw_bg4e_1973_1_007907197_1	23,136	71,728	3.1003
12 May 2012 17:54:35	1194710	14658485	hadam3p_pnw_bg4e_1973_1_007907197_1	11,616	36,884	3.1753