Task 16626856

Name	hadam3p_anz_rajv_2012_1_008743289_0
Workunit	8889267
Created	8 May 2014, 19:55:27 UTC
Sent	11 May 2014, 23:08:15 UTC
Report deadline	24 Apr 2015, 4:28:15 UTC
Received	10 Jul 2014, 5:38:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1143845
Run time	6 days 12 hours 19 min 31 sec
CPU time	4 days 22 hours 26 min 56 sec
Validate state	Invalid
Credit	2,497.00
Device peak FLOPS	2.44 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4244, selfPID=3560, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3256, selfPID=588, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish 21:15:48 (3316): No heartbeat from core client for 30 sec - exiting 21:15:49 (3316): No heartbeat from core client for 30 sec - exiting 21:15:50 (3316): No heartbeat from core client for 30 sec - exiting 21:15:51 (3316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1492, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4532, selfPID=3152, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4168, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... GSuspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4008, selfPID=3392, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:49:46 (1100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:49:47 (1100): No heartbeat from core client for 30 sec - exiting 00:49:48 (1100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3748, selfPID=3748, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:35:39 (4452): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:35:40 (4452): No heartbeat from core client for 30 sec - exiting 03:35:41 (4452): No heartbeat from core client for 30 sec - exiting 03:35:42 (4452): No heartbeat from core client for 30 sec - exiting 03:35:43 (4452): No heartbeat from core client for 30 sec - exiting 03:35:44 (4452): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:31:03 (380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:31:04 (380): No heartbeat from core client for 30 sec - exiting 06:31:05 (380): No heartbeat from core client for 30 sec - exiting 06:31:06 (380): No heartbeat from core client for 30 sec - exiting 06:31:07 (380): No heartbeat from core client for 30 sec - exiting 06:31:08 (380): No heartbeat from core client for 30 sec - exiting 06:31:09 (380): No heartbeat from core client for 30 sec - exiting 06:31:10 (380): No heartbeat from core client for 30 sec - exiting 06:31:11 (380): No heartbeat from core client for 30 sec - exiting 06:31:12 (380): No heartbeat from core client for 30 sec - exiting 06:31:13 (380): No heartbeat from core client for 30 sec - exiting 06:31:14 (380): No heartbeat from core client for 30 sec - exiting 06:31:15 (380): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:52:57 (3476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4776, selfPID=4776, iMonCtr=2 07:29:08 (3320): No heartbeat from core client for 30 sec - exiting 07:29:09 (3320): No heartbeat from core client for 30 sec - exiting 07:29:10 (3320): No heartbeat from core client for 30 sec - exiting 07:29:11 (3320): No heartbeat from core client for 30 sec - exiting 07:29:12 (3320): No heartbeat from core client for 30 sec - exiting 07:29:13 (3320): No heartbeat from core client for 30 sec - exiting 07:29:14 (3320): No heartbeat from core client for 30 sec - exiting 07:29:15 (3320): No heartbeat from core client for 30 sec - exiting 07:29:16 (3320): No heartbeat from core client for 30 sec - exiting 07:29:17 (3320): No heartbeat from core client for 30 sec - exiting 07:29:18 (3320): No heartbeat from core client for 30 sec - exiting 07:29:19 (3320): No heartbeat from core client for 30 sec - exiting 07:29:20 (3320): No heartbeat from core client for 30 sec - exiting 07:29:21 (3320): No heartbeat from core client for 30 sec - exiting 07:29:22 (3320): No heartbeat from core client for 30 sec - exiting 07:29:23 (3320): No heartbeat from core client for 30 sec - exiting 07:29:24 (3320): No heartbeat from core client for 30 sec - exiting 07:29:25 (3320): No heartbeat from core client for 30 sec - exiting 07:29:26 (3320): No heartbeat from core client for 30 sec - exiting 07:29:27 (3320): No heartbeat from core client for 30 sec - exiting 07:29:28 (3320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2832, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1136, selfPID=3404, iMonCtr=1 Model crash detected, will try to restart... CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3884, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4208, selfPID=3204, iMonCtr=1 Model crash detected, will try to restart... GGController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4512, selfPID=3152, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2316, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2796, selfPID=3740, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3880, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4188, selfPID=4012, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4380, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3292, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_rajv_2012_1_008743289_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rajv_2012_1_008743289_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rajv_2012_1_008743289_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rajv_2012_1_008743289_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rajv_2012_1_008743289_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rajv_2012_1_008743289_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rajv_2012_1_008743289_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Jul 2014 20:41:04	1143845	16626856	hadam3p_anz_rajv_2012_1_008743289_0	57,899	422,266	7.2931
15 Jun 2014 19:02:08	1143845	16626856	hadam3p_anz_rajv_2012_1_008743289_0	46,379	338,802	7.3051
10 Jun 2014 09:03:00	1143845	16626856	hadam3p_anz_rajv_2012_1_008743289_0	34,859	251,370	7.2111
29 May 2014 15:56:29	1143845	16626856	hadam3p_anz_rajv_2012_1_008743289_0	23,339	164,018	7.0276
21 May 2014 03:19:44	1143845	16626856	hadam3p_anz_rajv_2012_1_008743289_0	11,819	80,748	6.8321