Task 13491775

Name	hadam3p_eu_6omx_2003_1_007499198_0
Workunit	7696673
Created	14 Oct 2011, 19:57:28 UTC
Sent	17 Oct 2011, 0:59:19 UTC
Report deadline	28 Sep 2012, 6:19:19 UTC
Received	4 Nov 2011, 12:09:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1159784
Run time	1 days 17 hours 52 min 2 sec
CPU time	6 hours 12 min 39 sec
Validate state	Invalid
Credit	597.84
Device peak FLOPS	0.75 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <stderr_txt> 08:28:02 (6064): No heartbeat from core client for 30 sec - exiting 08:28:03 (6064): No heartbeat from core client for 30 sec - exiting 08:28:04 (6064): No heartbeat from core client for 30 sec - exiting 08:28:05 (6064): No heartbeat from core client for 30 sec - exiting 08:28:06 (6064): No heartbeat from core client for 30 sec - exiting 08:28:07 (6064): No heartbeat from core client for 30 sec - exiting 08:28:08 (6064): No heartbeat from core client for 30 sec - exiting 08:28:09 (6064): No heartbeat from core client for 30 sec - exiting 08:28:10 (6064): No heartbeat from core client for 30 sec - exiting 08:28:12 (6064): No heartbeat from core client for 30 sec - exiting 08:28:13 (6064): No heartbeat from core client for 30 sec - exiting 08:28:14 (6064): No heartbeat from core client for 30 sec - exiting 08:28:15 (6064): No heartbeat from core client for 30 sec - exiting 08:28:16 (6064): No heartbeat from core client for 30 sec - exiting 08:28:17 (6064): No heartbeat from core client for 30 sec - exiting 08:28:18 (6064): No heartbeat from core client for 30 sec - exiting 08:28:19 (6064): No heartbeat from core client for 30 sec - exiting 08:28:20 (6064): No heartbeat from core client for 30 sec - exiting 08:28:21 (6064): No heartbeat from core client for 30 sec - exiting 08:28:22 (6064): No heartbeat from core client for 30 sec - exiting 08:28:23 (6064): No heartbeat from core client for 30 sec - exiting 08:28:24 (6064): No heartbeat from core client for 30 sec - exiting 08:28:25 (6064): No heartbeat from core client for 30 sec - exiting 08:28:26 (6064): No heartbeat from core client for 30 sec - exiting 08:28:27 (6064): No heartbeat from core client for 30 sec - exiting 08:28:28 (6064): No heartbeat from core client for 30 sec - exiting 08:28:29 (6064): No heartbeat from core client for 30 sec - exiting 08:28:30 (6064): No heartbeat from core client for 30 sec - exiting 08:28:31 (6064): No heartbeat from core client for 30 sec - exiting 08:28:32 (6064): No heartbeat from core client for 30 sec - exiting 08:28:33 (6064): No heartbeat from core client for 30 sec - exiting 08:28:34 (6064): No heartbeat from core client for 30 sec - exiting 08:28:36 (6064): No heartbeat from core client for 30 sec - exiting 08:28:37 (6064): No heartbeat from core client for 30 sec - exiting 08:28:38 (6064): No heartbeat from core client for 30 sec - exiting 08:28:39 (6064): No heartbeat from core client for 30 sec - exiting 08:28:40 (6064): No heartbeat from core client for 30 sec - exiting 08:28:41 (6064): No heartbeat from core client for 30 sec - exiting 08:28:42 (6064): No heartbeat from core client for 30 sec - exiting 08:28:43 (6064): No heartbeat from core client for 30 sec - exiting 08:28:44 (6064): No heartbeat from core client for 30 sec - exiting 08:28:45 (6064): No heartbeat from core client for 30 sec - exiting 08:28:46 (6064): No heartbeat from core client for 30 sec - exiting 08:28:48 (6064): No heartbeat from core client for 30 sec - exiting 08:28:49 (6064): No heartbeat from core client for 30 sec - exiting 08:28:50 (6064): No heartbeat from core client for 30 sec - exiting 08:28:51 (6064): No heartbeat from core client for 30 sec - exiting 08:28:52 (6064): No heartbeat from core client for 30 sec - exiting 08:28:53 (6064): No heartbeat from core client for 30 sec - exiting 08:28:54 (6064): No heartbeat from core client for 30 sec - exiting 08:28:55 (6064): No heartbeat from core client for 30 sec - exiting 08:28:56 (6064): No heartbeat from core client for 30 sec - exiting 08:28:57 (6064): No heartbeat from core client for 30 sec - exiting 08:28:59 (6064): No heartbeat from core client for 30 sec - exiting 08:29:00 (6064): No heartbeat from core client for 30 sec - exiting 08:29:01 (6064): No heartbeat from core client for 30 sec - exiting 08:29:02 (6064): No heartbeat from core client for 30 sec - exiting 08:29:03 (6064): No heartbeat from core client for 30 sec - exiting 08:29:04 (6064): No heartbeat from core client for 30 sec - exiting 08:29:05 (6064): No heartbeat from core client for 30 sec - exiting 08:29:06 (6064): No heartbeat from core client for 30 sec - exiting 08:29:07 (6064): No heartbeat from core client for 30 sec - exiting 08:29:08 (6064): No heartbeat from core client for 30 sec - exiting 08:29:09 (6064): No heartbeat from core client for 30 sec - exiting 08:29:11 (6064): No heartbeat from core client for 30 sec - exiting 08:29:12 (6064): No heartbeat from core client for 30 sec - exiting 08:29:13 (6064): No heartbeat from core client for 30 sec - exiting 08:29:14 (6064): No heartbeat from core client for 30 sec - exiting 08:29:15 (6064): No heartbeat from core client for 30 sec - exiting 08:29:16 (6064): No heartbeat from core client for 30 sec - exiting 08:29:17 (6064): No heartbeat from core client for 30 sec - exiting 08:29:18 (6064): No heartbeat from core client for 30 sec - exiting 08:29:19 (6064): No heartbeat from core client for 30 sec - exiting 08:29:20 (6064): No heartbeat from core client for 30 sec - exiting 08:29:21 (6064): No heartbeat from core client for 30 sec - exiting 08:29:23 (6064): No heartbeat from core client for 30 sec - exiting 08:29:24 (6064): No heartbeat from core client for 30 sec - exiting 08:29:25 (6064): No heartbeat from core client for 30 sec - exiting 08:29:26 (6064): No heartbeat from core client for 30 sec - exiting 08:29:27 (6064): No heartbeat from core client for 30 sec - exiting 08:29:28 (6064): No heartbeat from core client for 30 sec - exiting 08:29:29 (6064): No heartbeat from core client for 30 sec - exiting 08:29:30 (6064): No heartbeat from core client for 30 sec - exiting 08:29:31 (6064): No heartbeat from core client for 30 sec - exiting 08:29:32 (6064): No heartbeat from core client for 30 sec - exiting 08:29:33 (6064): No heartbeat from core client for 30 sec - exiting 08:29:35 (6064): No heartbeat from core client for 30 sec - exiting 08:29:36 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:29:37 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3032, selfPID=3032, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8536, selfPID=9076, iMonCtr=1 Model crash detected, will try to restart... 10:37:39 (7044): No heartbeat from core client for 30 sec - exiting 10:37:40 (7044): No heartbeat from core client for 30 sec - exiting 10:37:41 (7044): No heartbeat from core client for 30 sec - exiting 10:37:42 (7044): No heartbeat from core client for 30 sec - exiting 10:37:43 (7044): No heartbeat from core client for 30 sec - exiting 10:37:44 (7044): No heartbeat from core client for 30 sec - exiting 10:37:45 (7044): No heartbeat from core client for 30 sec - exiting 10:37:47 (7044): No heartbeat from core client for 30 sec - exiting 10:37:48 (7044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6856, selfPID=5748, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6840, selfPID=6976, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7232, selfPID=4160, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 09:31:01 (4576): No heartbeat from core client for 30 sec - exiting 09:31:03 (4576): No heartbeat from core client for 30 sec - exiting 09:31:04 (4576): No heartbeat from core client for 30 sec - exiting 09:31:05 (4576): No heartbeat from core client for 30 sec - exiting 09:31:06 (4576): No heartbeat from core client for 30 sec - exiting 09:31:07 (4576): No heartbeat from core client for 30 sec - exiting 09:31:08 (4576): No heartbeat from core client for 30 sec - exiting 09:31:09 (4576): No heartbeat from core client for 30 sec - exiting 09:31:10 (4576): No heartbeat from core client for 30 sec - exiting 09:31:11 (4576): No heartbeat from core client for 30 sec - exiting 09:31:12 (4576): No heartbeat from core client for 30 sec - exiting 09:31:14 (4576): No heartbeat from core client for 30 sec - exiting 09:31:15 (4576): No heartbeat from core client for 30 sec - exiting 09:31:16 (4576): No heartbeat from core client for 30 sec - exiting 09:31:17 (4576): No heartbeat from core client for 30 sec - exiting 09:31:18 (4576): No heartbeat from core client for 30 sec - exiting 09:31:19 (4576): No heartbeat from core client for 30 sec - exiting 09:31:20 (4576): No heartbeat from core client for 30 sec - exiting 09:31:21 (4576): No heartbeat from core client for 30 sec - exiting 09:31:22 (4576): No heartbeat from core client for 30 sec - exiting 09:31:23 (4576): No heartbeat from core client for 30 sec - exiting 09:31:24 (4576): No heartbeat from core client for 30 sec - exiting 09:31:26 (4576): No heartbeat from core client for 30 sec - exiting 09:31:27 (4576): No heartbeat from core client for 30 sec - exiting 09:31:28 (4576): No heartbeat from core client for 30 sec - exiting 09:31:29 (4576): No heartbeat from core client for 30 sec - exiting 09:31:30 (4576): No heartbeat from core client for 30 sec - exiting 09:31:31 (4576): No heartbeat from core client for 30 sec - exiting 09:31:32 (4576): No heartbeat from core client for 30 sec - exiting 09:31:33 (4576): No heartbeat from core client for 30 sec - exiting 09:31:34 (4576): No heartbeat from core client for 30 sec - exiting 09:31:35 (4576): No heartbeat from core client for 30 sec - exiting 09:31:36 (4576): No heartbeat from core client for 30 sec - exiting 09:31:38 (4576): No heartbeat from core client for 30 sec - exiting 09:31:39 (4576): No heartbeat from core client for 30 sec - exiting 09:31:40 (4576): No heartbeat from core client for 30 sec - exiting 09:31:41 (4576): No heartbeat from core client for 30 sec - exiting 09:31:42 (4576): No heartbeat from core client for 30 sec - exiting 09:31:43 (4576): No heartbeat from core client for 30 sec - exiting 09:31:44 (4576): No heartbeat from core client for 30 sec - exiting 09:31:45 (4576): No heartbeat from core client for 30 sec - exiting 09:31:46 (4576): No heartbeat from core client for 30 sec - exiting 09:31:47 (4576): No heartbeat from core client for 30 sec - exiting 09:31:48 (4576): No heartbeat from core client for 30 sec - exiting 09:31:49 (4576): No heartbeat from core client for 30 sec - exiting 09:31:51 (4576): No heartbeat from core client for 30 sec - exiting 09:31:52 (4576): No heartbeat from core client for 30 sec - exiting 09:31:53 (4576): No heartbeat from core client for 30 sec - exiting 09:31:54 (4576): No heartbeat from core client for 30 sec - exiting 09:31:55 (4576): No heartbeat from core client for 30 sec - exiting 09:31:56 (4576): No heartbeat from core client for 30 sec - exiting 09:31:57 (4576): No heartbeat from core client for 30 sec - exiting 09:31:58 (4576): No heartbeat from core client for 30 sec - exiting 09:31:59 (4576): No heartbeat from core client for 30 sec - exiting 09:32:00 (4576): No heartbeat from core client for 30 sec - exiting 09:32:01 (4576): No heartbeat from core client for 30 sec - exiting 09:32:02 (4576): No heartbeat from core client for 30 sec - exiting 09:32:04 (4576): No heartbeat from core client for 30 sec - exiting 09:32:05 (4576): No heartbeat from core client for 30 sec - exiting 09:32:06 (4576): No heartbeat from core client for 30 sec - exiting 09:32:07 (4576): No heartbeat from core client for 30 sec - exiting 09:32:08 (4576): No heartbeat from core client for 30 sec - exiting 09:32:09 (4576): No heartbeat from core client for 30 sec - exiting 09:32:10 (4576): No heartbeat from core client for 30 sec - exiting 09:32:11 (4576): No heartbeat from core client for 30 sec - exiting 09:32:12 (4576): No heartbeat from core client for 30 sec - exiting 09:32:13 (4576): No heartbeat from core client for 30 sec - exiting 09:32:15 (4576): No heartbeat from core client for 30 sec - exiting 09:32:16 (4576): No heartbeat from core client for 30 sec - exiting 09:32:17 (4576): No heartbeat from core client for 30 sec - exiting 09:32:18 (4576): No heartbeat from core client for 30 sec - exiting 09:32:19 (4576): No heartbeat from core client for 30 sec - exiting 09:32:20 (4576): No heartbeat from core client for 30 sec - exiting 09:32:21 (4576): No heartbeat from core client for 30 sec - exiting 09:32:22 (4576): No heartbeat from core client for 30 sec - exiting 09:32:23 (4576): No heartbeat from core client for 30 sec - exiting 09:32:24 (4576): No heartbeat from core client for 30 sec - exiting 09:32:25 (4576): No heartbeat from core client for 30 sec - exiting 09:32:27 (4576): No heartbeat from core client for 30 sec - exiting 09:32:28 (4576): No heartbeat from core client for 30 sec - exiting 09:32:29 (4576): No heartbeat from core client for 30 sec - exiting 09:32:30 (4576): No heartbeat from core client for 30 sec - exiting 09:32:31 (4576): No heartbeat from core client for 30 sec - exiting 09:32:32 (4576): No heartbeat from core client for 30 sec - exiting 09:32:33 (4576): No heartbeat from core client for 30 sec - exiting 09:32:34 (4576): No heartbeat from core client for 30 sec - exiting 09:32:35 (4576): No heartbeat from core client for 30 sec - exiting 09:32:36 (4576): No heartbeat from core client for 30 sec - exiting 09:32:37 (4576): No heartbeat from core client for 30 sec - exiting 09:32:39 (4576): No heartbeat from core client for 30 sec - exiting 09:32:40 (4576): No heartbeat from core client for 30 sec - exiting 09:32:41 (4576): No heartbeat from core client for 30 sec - exiting 09:32:42 (4576): No heartbeat from core client for 30 sec - exiting 09:32:43 (4576): No heartbeat from core client for 30 sec - exiting 09:32:44 (4576): No heartbeat from core client for 30 sec - exiting 09:32:45 (4576): No heartbeat from core client for 30 sec - exiting 09:32:46 (4576): No heartbeat from core client for 30 sec - exiting 09:32:47 (4576): No heartbeat from core client for 30 sec - exiting 09:32:48 (4576): No heartbeat from core client for 30 sec - exiting 09:32:49 (4576): No heartbeat from core client for 30 sec - exiting 09:32:51 (4576): No heartbeat from core client for 30 sec - exiting 09:32:52 (4576): No heartbeat from core client for 30 sec - exiting 09:32:53 (4576): No heartbeat from core client for 30 sec - exiting 09:32:54 (4576): No heartbeat from core client for 30 sec - exiting 09:32:55 (4576): No heartbeat from core client for 30 sec - exiting 09:32:56 (4576): No heartbeat from core client for 30 sec - exiting 09:32:57 (4576): No heartbeat from core client for 30 sec - exiting 09:32:58 (4576): No heartbeat from core client for 30 sec - exiting 09:32:59 (4576): No heartbeat from core client for 30 sec - exiting 09:33:00 (4576): No heartbeat from core client for 30 sec - exiting 09:33:01 (4576): No heartbeat from core client for 30 sec - exiting 09:33:03 (4576): No heartbeat from core client for 30 sec - exiting 09:33:04 (4576): No heartbeat from core client for 30 sec - exiting 09:33:05 (4576): No heartbeat from core client for 30 sec - exiting 09:33:06 (4576): No heartbeat from core client for 30 sec - exiting 09:33:07 (4576): No heartbeat from core client for 30 sec - exiting 09:33:08 (4576): No heartbeat from core client for 30 sec - exiting 09:33:09 (4576): No heartbeat from core client for 30 sec - exiting 09:33:10 (4576): No heartbeat from core client for 30 sec - exiting 09:33:11 (4576): No heartbeat from core client for 30 sec - exiting 09:33:12 (4576): No heartbeat from core client for 30 sec - exiting 09:33:13 (4576): No heartbeat from core client for 30 sec - exiting 09:33:15 (4576): No heartbeat from core client for 30 sec - exiting 09:33:16 (4576): No heartbeat from core client for 30 sec - exiting 09:33:17 (4576): No heartbeat from core client for 30 sec - exiting 09:33:18 (4576): No heartbeat from core client for 30 sec - exiting 09:33:19 (4576): No heartbeat from core client for 30 sec - exiting 09:33:20 (4576): No heartbeat from core client for 30 sec - exiting 09:33:21 (4576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5180, iMonCtr=2 Mode l crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish 12:18:25 (5180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_4.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6omx_2003_1_007499198_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Oct 2011 16:04:08	1159784	13491775	hadam3p_eu_6omx_2003_1_007499198_0	34,656	93,165	2.6883
18 Oct 2011 18:24:40	1159784	13491775	hadam3p_eu_6omx_2003_1_007499198_0	23,136	62,939	2.7204
17 Oct 2011 16:40:35	1159784	13491775	hadam3p_eu_6omx_2003_1_007499198_0	11,616	32,100	2.7634