Task 13797848

Name	hadam3p_saf_79ve_2000_1_007569731_1
Workunit	7747861
Created	19 Dec 2011, 17:49:56 UTC
Sent	19 Dec 2011, 18:08:46 UTC
Report deadline	30 Nov 2012, 23:28:46 UTC
Received	17 Jun 2012, 5:45:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1186357
Run time	21 hours 49 min 52 sec
CPU time	19 hours 35 min 21 sec
Validate state	Invalid
Credit	375.31
Device peak FLOPS	3.08 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5836, selfPID=5020, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 19:50:30 (3044): No heartbeat from core client for 30 sec - exiting 19:50:31 (3044): No heartbeat from core client for 30 sec - exiting 19:50:32 (3044): No heartbeat from core client for 30 sec - exiting 19:50:33 (3044): No heartbeat from core client for 30 sec - exiting 19:50:34 (3044): No heartbeat from core client for 30 sec - exiting 19:50:35 (3044): No heartbeat from core client for 30 sec - exiting 19:50:37 (3044): No heartbeat from core client for 30 sec - exiting 19:50:38 (3044): No heartbeat from core client for 30 sec - exiting 19:50:39 (3044): No heartbeat from core client for 30 sec - exiting 19:50:40 (3044): No heartbeat from core client for 30 sec - exiting 19:50:41 (3044): No heartbeat from core client for 30 sec - exiting 19:50:42 (3044): No heartbeat from core client for 30 sec - exiting 19:50:43 (3044): No heartbeat from core client for 30 sec - exiting 19:50:44 (3044): No heartbeat from core client for 30 sec - exiting 19:50:45 (3044): No heartbeat from core client for 30 sec - exiting 19:50:46 (3044): No heartbeat from core client for 30 sec - exiting 19:50:47 (3044): No heartbeat from core client for 30 sec - exiting 19:50:49 (3044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3924, selfPID=3924, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5756, selfPID=5756, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4584, selfPID=2996, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4404, selfPID=4404, iMonCtr=2 CPDN Monitor - Quit request from BOINC... RegionCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4384, selfPID=4384, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4396, selfPID=2580, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4336, selfPID=4336, iMonCtr=2 08:55:09 (2712): No heartbeat from core client for 30 sec - exiting 08:55:10 (2712): No heartbeat from core client for 30 sec - exiting 08:55:11 (2712): No heartbeat from core client for 30 sec - exiting 08:55:13 (2712): No heartbeat from core client for 30 sec - exiting 08:55:14 (2712): No heartbeat from core client for 30 sec - exiting 08:55:15 (2712): No heartbeat from core client for 30 sec - exiting 08:55:16 (2712): No heartbeat from core client for 30 sec - exiting 08:55:17 (2712): No heartbeat from core client for 30 sec - exiting 08:55:18 (2712): No heartbeat from core client for 30 sec - exiting 08:55:19 (2712): No heartbeat from core client for 30 sec - exiting 08:55:20 (2712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3652, selfPID=3652, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1768, selfPID=1768, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4060, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3408, selfPID=2772, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4460, selfPID=4460, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3724, selfPID=3724, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4116, selfPID=4116, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4188, selfPID=4188, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4804, selfPID=244, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3996, selfPID=3048, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3164, selfPID=1412, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2924, selfPID=2924, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3092, selfPID=3092, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5172, selfPID=5172, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:37:09 (4476): No heartbeat from core client for 30 sec - exiting 13:37:10 (4476): No heartbeat from core client for 30 sec - exiting 13:37:11 (4476): No heartbeat from core client for 30 sec - exiting 13:37:12 (4476): No heartbeat from core client for 30 sec - exiting 13:37:13 (4476): No heartbeat from core client for 30 sec - exiting 13:37:15 (4476): No heartbeat from core client for 30 sec - exiting 13:37:16 (4476): No heartbeat from core client for 30 sec - exiting 13:37:17 (4476): No heartbeat from core client for 30 sec - exiting 13:37:18 (4476): No heartbeat from core client for 30 sec - exiting 13:37:19 (4476): No heartbeat from core client for 30 sec - exiting 13:37:20 (4476): No heartbeat from core client for 30 sec - exiting 13:37:21 (4476): No heartbeat from core client for 30 sec - exiting 13:37:22 (4476): No heartbeat from core client for 30 sec - exiting 13:37:23 (4476): No heartbeat from core client for 30 sec - exiting 13:37:24 (4476): No heartbeat from core client for 30 sec - exiting 13:37:25 (4476): No heartbeat from core client for 30 sec - exiting 13:37:27 (4476): No heartbeat from core client for 30 sec - exiting 13:37:28 (4476): No heartbeat from core client for 30 sec - exiting 13:37:29 (4476): No heartbeat from core client for 30 sec - exiting 13:37:30 (4476): No heartbeat from core client for 30 sec - exiting 13:37:31 (4476): No heartbeat from core client for 30 sec - exiting 13:37:32 (4476): No heartbeat from core client for 30 sec - exiting 13:37:33 (4476): No heartbeat from core client for 30 sec - exiting 13:37:34 (4476): No heartbeat from core client for 30 sec - exiting 13:37:35 (4476): No heartbeat from core client for 30 sec - exiting 13:37:36 (4476): No heartbeat from core client for 30 sec - exiting 13:37:37 (4476): No heartbeat from core client for 30 sec - exiting 13:37:39 (4476): No heartbeat from core client for 30 sec - exiting 13:37:40 (4476): No heartbeat from core client for 30 sec - exiting 13:37:41 (4476): No heartbeat from core client for 30 sec - exiting 13:37:42 (4476): No heartbeat from core client for 30 sec - exiting 13:37:43 (4476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:37:44 (4476): No heartbeat from core client for 30 sec - exiting 13:37:45 (4476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2300, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5208, selfPID=5208, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3828, selfPID=3828, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4780, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_3.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_4.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_79ve_2000_1_007569731_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Mar 2012 08:15:50	1186357	13797848	hadam3p_saf_79ve_2000_1_007569731_1	23,136	46,464	2.0083
26 Dec 2011 07:54:57	1186357	13797848	hadam3p_saf_79ve_2000_1_007569731_1	11,616	16,369	1.4092