Task 15137933

Name	hadam3p_saf_1fgf_1974_1_006947863_2
Workunit	7151179
Created	18 Aug 2012, 1:48:07 UTC
Sent	18 Aug 2012, 1:52:30 UTC
Report deadline	31 Jul 2013, 7:12:30 UTC
Received	2 Sep 2012, 18:01:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1232385
Run time	1 days 18 hours 8 min 24 sec
CPU time	1 days 15 hours 13 min 24 sec
Validate state	Invalid
Credit	1,683.45
Device peak FLOPS	3.31 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> 15:13:51 (4132): No heartbeat from core client for 30 sec - exiting 15:13:52 (4132): No heartbeat from core client for 30 sec - exiting 15:13:53 (4132): No heartbeat from core client for 30 sec - exiting 15:13:54 (4132): No heartbeat from core client for 30 sec - exiting 15:13:55 (4132): No heartbeat from core client for 30 sec - exiting 15:13:57 (4132): No heartbeat from core client for 30 sec - exiting 15:13:58 (4132): No heartbeat from core client for 30 sec - exiting 15:13:59 (4132): No heartbeat from core client for 30 sec - exiting 15:14:00 (4132): No heartbeat from core client for 30 sec - exiting 15:14:01 (4132): No heartbeat from core client for 30 sec - exiting 15:14:02 (4132): No heartbeat from core client for 30 sec - exiting 15:14:03 (4132): No heartbeat from core client for 30 sec - exiting 15:14:04 (4132): No heartbeat from core client for 30 sec - exiting 15:14:05 (4132): No heartbeat from core client for 30 sec - exiting 15:14:06 (4132): No heartbeat from core client for 30 sec - exiting 15:14:07 (4132): No heartbeat from core client for 30 sec - exiting 15:14:09 (4132): No heartbeat from core client for 30 sec - exiting 15:14:10 (4132): No heartbeat from core client for 30 sec - exiting 15:14:11 (4132): No heartbeat from core client for 30 sec - exiting 15:14:12 (4132): No heartbeat from core client for 30 sec - exiting 15:14:13 (4132): No heartbeat from core client for 30 sec - exiting 15:14:14 (4132): No heartbeat from core client for 30 sec - exiting 15:14:15 (4132): No heartbeat from core client for 30 sec - exiting 15:14:16 (4132): No heartbeat from core client for 30 sec - exiting 15:14:17 (4132): No heartbeat from core client for 30 sec - exiting 15:14:18 (4132): No heartbeat from core client for 30 sec - exiting 15:14:19 (4132): No heartbeat from core client for 30 sec - exiting 15:14:21 (4132): No heartbeat from core client for 30 sec - exiting 15:14:22 (4132): No heartbeat from core client for 30 sec - exiting 15:14:23 (4132): No heartbeat from core client for 30 sec - exiting 15:14:24 (4132): No heartbeat from core client for 30 sec - exiting 15:14:25 (4132): No heartbeat from core client for 30 sec - exiting 15:14:26 (4132): No heartbeat from core client for 30 sec - exiting 15:14:27 (4132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7720, selfPID=7372, iMonCtr=1 Model crash detected, will try to restart... 20:15:23 (5964): No heartbeat from core client for 30 sec - exiting 20:15:24 (5964): No heartbeat from core client for 30 sec - exiting 20:15:25 (5964): No heartbeat from core client for 30 sec - exiting 20:15:26 (5964): No heartbeat from core client for 30 sec - exiting 20:15:27 (5964): No heartbeat from core client for 30 sec - exiting 20:15:29 (5964): No heartbeat from core client for 30 sec - exiting 20:15:30 (5964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:00:15 (5464): No heartbeat from core client for 30 sec - exiting 23:00:16 (5464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:48:11 (5160): No heartbeat from core client for 30 sec - exiting 08:48:13 (5160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:19:20 (3552): No heartbeat from core client for 30 sec - exiting 19:19:21 (3552): No heartbeat from core client for 30 sec - exiting 19:19:22 (3552): No heartbeat from core client for 30 sec - exiting 19:19:23 (3552): No heartbeat from core client for 30 sec - exiting 19:19:24 (3552): No heartbeat from core client for 30 sec - exiting 19:19:25 (3552): No heartbeat from core client for 30 sec - exiting 19:19:26 (3552): No heartbeat from core client for 30 sec - exiting 19:19:28 (3552): No heartbeat from core client for 30 sec - exiting 19:19:29 (3552): No heartbeat from core client for 30 sec - exiting 19:19:30 (3552): No heartbeat from core client for 30 sec - exiting 19:19:31 (3552): No heartbeat from core client for 30 sec - exiting 19:19:32 (3552): No heartbeat from core client for 30 sec - exiting 19:19:33 (3552): No heartbeat from core client for 30 sec - exiting 19:19:34 (3552): No heartbeat from core client for 30 sec - exiting 19:19:35 (3552): No heartbeat from core client for 30 sec - exiting 19:19:36 (3552): No heartbeat from core client for 30 sec - exiting 19:19:37 (3552): No heartbeat from core client for 30 sec - exiting 19:19:39 (3552): No heartbeat from core client for 30 sec - exiting 19:19:40 (3552): No heartbeat from core client for 30 sec - exiting 19:19:41 (3552): No heartbeat from core client for 30 sec - exiting 19:19:42 (3552): No heartbeat from core client for 30 sec - exiting 19:19:43 (3552): No heartbeat from core client for 30 sec - exiting 19:19:44 (3552): No heartbeat from core client for 30 sec - exiting 19:19:45 (3552): No heartbeat from core client for 30 sec - exiting 19:19:46 (3552): No heartbeat from core client for 30 sec - exiting 19:19:47 (3552): No heartbeat from core client for 30 sec - exiting 19:19:48 (3552): No heartbeat from core client for 30 sec - exiting 19:19:49 (3552): No heartbeat from core client for 30 sec - exiting 19:19:51 (3552): No heartbeat from core client for 30 sec - exiting 19:19:52 (3552): No heartbeat from core client for 30 sec - exiting 19:19:53 (3552): No heartbeat from core client for 30 sec - exiting 19:19:54 (3552): No heartbeat from core client for 30 sec - exiting 19:19:55 (3552): No heartbeat from core client for 30 sec - exiting 19:19:56 (3552): No heartbeat from core client for 30 sec - exiting 19:19:57 (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8928, selfPID=7444, iMonCtr=1 Model crash detected, will try to restart... 01:56:44 (4760): No heartbeat from core client for 30 sec - exiting 01:56:46 (4760): No heartbeat from core client for 30 sec - exiting 01:56:47 (4760): No heartbeat from core client for 30 sec - exiting 01:56:48 (4760): No heartbeat from core client for 30 sec - exiting 01:56:49 (4760): No heartbeat from core client for 30 sec - exiting 01:56:50 (4760): No heartbeat from core client for 30 sec - exiting 01:56:51 (4760): No heartbeat from core client for 30 sec - exiting 01:56:52 (4760): No heartbeat from core client for 30 sec - exiting 01:56:53 (4760): No heartbeat from core client for 30 sec - exiting 01:56:54 (4760): No heartbeat from core client for 30 sec - exiting 01:56:55 (4760): No heartbeat from core client for 30 sec - exiting 01:56:57 (4760): No heartbeat from core client for 30 sec - exiting 01:56:58 (4760): No heartbeat from core client for 30 sec - exiting 01:56:59 (4760): No heartbeat from core client for 30 sec - exiting 01:57:00 (4760): No heartbeat from core client for 30 sec - exiting 01:57:01 (4760): No heartbeat from core client for 30 sec - exiting 01:57:02 (4760): No heartbeat from core client for 30 sec - exiting 01:57:03 (4760): No heartbeat from core client for 30 sec - exiting 01:57:04 (4760): No heartbeat from core client for 30 sec - exiting 01:57:05 (4760): No heartbeat from core client for 30 sec - exiting 01:57:06 (4760): No heartbeat from core client for 30 sec - exiting 01:57:08 (4760): No heartbeat from core client for 30 sec - exiting 01:57:09 (4760): No heartbeat from core client for 30 sec - exiting 01:57:10 (4760): No heartbeat from core client for 30 sec - exiting 01:57:11 (4760): No heartbeat from core client for 30 sec - exiting 01:57:12 (4760): No heartbeat from core client for 30 sec - exiting 01:57:13 (4760): No heartbeat from core client for 30 sec - exiting 01:57:14 (4760): No heartbeat from core client for 30 sec - exiting 01:57:15 (4760): No heartbeat from core client for 30 sec - exiting 01:57:16 (4760): No heartbeat from core client for 30 sec - exiting 01:57:17 (4760): No heartbeat from core client for 30 sec - exiting 01:57:18 (4760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:25:33 (2088): No heartbeat from core client for 30 sec - exiting 21:25:34 (2088): No heartbeat from core client for 30 sec - exiting 21:25:36 (2088): No heartbeat from core client for 30 sec - exiting 21:25:37 (2088): No heartbeat from core client for 30 sec - exiting 21:25:38 (2088): No heartbeat from core client for 30 sec - exiting 21:25:39 (2088): No heartbeat from core client for 30 sec - exiting 21:25:40 (2088): No heartbeat from core client for 30 sec - exiting 21:25:41 (2088): No heartbeat from core client for 30 sec - exiting 21:25:42 (2088): No heartbeat from core client for 30 sec - exiting 21:25:43 (2088): No heartbeat from core client for 30 sec - exiting 21:25:44 (2088): No heartbeat from core client for 30 sec - exiting 21:25:45 (2088): No heartbeat from core client for 30 sec - exiting 21:25:46 (2088): No heartbeat from core client for 30 sec - exiting 21:25:48 (2088): No heartbeat from core client for 30 sec - exiting 21:25:49 (2088): No heartbeat from core client for 30 sec - exiting 21:25:50 (2088): No heartbeat from core client for 30 sec - exiting 21:25:51 (2088): No heartbeat from core client for 30 sec - exiting 21:25:52 (2088): No heartbeat from core client for 30 sec - exiting 21:25:53 (2088): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=5588, iMonCtr=1 CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:45:33 (2484): No heartbeat from core client for 30 sec - exiting 15:45:34 (2484): No heartbeat from core client for 30 sec - exiting 15:45:35 (2484): No heartbeat from core client for 30 sec - exiting 15:45:36 (2484): No heartbeat from core client for 30 sec - exiting 15:45:37 (2484): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=7844, iMonCtr=1 CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7472, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=4748, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4968, selfPID=3448, iMonCtr=1 Model crash detected, will try to restart... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4968, selfPID=4968, iMonCtr=2 Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_saf_1fgf_1974_1_006947863_2_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1fgf_1974_1_006947863_2_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1fgf_1974_1_006947863_2_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Sep 2012 18:08:53	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	103,776	140,939	1.3581
28 Aug 2012 14:37:20	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	92,256	126,268	1.3687
28 Aug 2012 11:06:29	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	80,736	110,652	1.3705
27 Aug 2012 04:15:21	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	69,216	95,059	1.3734
24 Aug 2012 00:59:26	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	57,696	79,487	1.3777
23 Aug 2012 20:27:39	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	46,176	63,776	1.3812
23 Aug 2012 15:27:52	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	34,656	47,835	1.3803
20 Aug 2012 03:33:25	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	23,136	31,970	1.3818
19 Aug 2012 22:27:19	1232385	15137933	hadam3p_saf_1fgf_1974_1_006947863_2	11,616	15,901	1.3689