Task 12338410

Name	hadam3p_eu_xz6x_1968_1_007050817_0
Workunit	7254133
Created	25 Nov 2010, 15:32:37 UTC
Sent	19 Dec 2010, 19:22:08 UTC
Report deadline	2 Dec 2011, 0:42:08 UTC
Received	22 Dec 2010, 23:24:57 UTC
Server state	Over
Outcome	No reply
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1111797
Run time	2 days 17 hours 6 min 15 sec
CPU time	2 days 13 hours 44 min 34 sec
Validate state	Invalid
Credit	1,591.48
Device peak FLOPS	3.07 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.08 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <stderr_txt> 01:34:38 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:34:39 (5520): No heartbeat from core client for 30 sec - exiting 01:34:40 (5520): No heartbeat from core client for 30 sec - exiting 01:34:41 (5520): No heartbeat from core client for 30 sec - exiting 01:34:42 (5520): No heartbeat from core client for 30 sec - exiting 01:34:43 (5520): No heartbeat from core client for 30 sec - exiting 01:34:44 (5520): No heartbeat from core client for 30 sec - exiting 01:34:45 (5520): No heartbeat from core client for 30 sec - exiting 01:34:46 (5520): No heartbeat from core client for 30 sec - exiting 01:34:47 (5520): No heartbeat from core client for 30 sec - exiting 01:34:48 (5520): No heartbeat from core client for 30 sec - exiting 01:34:49 (5520): No heartbeat from core client for 30 sec - exiting 01:34:50 (5520): No heartbeat from core client for 30 sec - exiting 01:34:51 (5520): No heartbeat from core client for 30 sec - exiting 01:34:52 (5520): No heartbeat from core client for 30 sec - exiting 01:34:53 (5520): No heartbeat from core client for 30 sec - exiting 01:34:54 (5520): No heartbeat from core client for 30 sec - exiting 01:34:55 (5520): No heartbeat from core client for 30 sec - exiting 01:34:56 (5520): No heartbeat from core client for 30 sec - exiting 01:34:57 (5520): No heartbeat from core client for 30 sec - exiting 01:34:58 (5520): No heartbeat from core client for 30 sec - exiting 01:34:59 (5520): No heartbeat from core client for 30 sec - exiting 01:35:00 (5520): No heartbeat from core client for 30 sec - exiting 01:35:01 (5520): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2908, selfPID=5896, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2908, selfPID=2908, iMonCtr=1 02:32:48 (6180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:32:49 (6180): No heartbeat from core client for 30 sec - exiting 02:32:50 (6180): No heartbeat from core client for 30 sec - exiting 02:32:51 (6180): No heartbeat from core client for 30 sec - exiting 02:32:52 (6180): No heartbeat from core client for 30 sec - exiting 02:32:53 (6180): No heartbeat from core client for 30 sec - exiting 02:32:54 (6180): No heartbeat from core client for 30 sec - exiting 02:32:55 (6180): No heartbeat from core client for 30 sec - exiting 02:32:56 (6180): No heartbeat from core client for 30 sec - exiting 02:32:58 (6180): No heartbeat from core client for 30 sec - exiting 02:32:59 (6180): No heartbeat from core client for 30 sec - exiting 02:33:00 (6180): No heartbeat from core client for 30 sec - exiting 02:33:01 (6180): No heartbeat from core client for 30 sec - exiting 02:33:02 (6180): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5184, selfPID=5184, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5184, selfPID=3572, iMonCtr=1 02:34:01 (2996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:34:02 (2996): No heartbeat from core client for 30 sec - exiting 02:34:03 (2996): No heartbeat from core client for 30 sec - exiting 02:34:04 (2996): No heartbeat from core client for 30 sec - exiting 02:34:05 (2996): No heartbeat from core client for 30 sec - exiting 02:34:06 (2996): No heartbeat from core client for 30 sec - exiting 02:34:07 (2996): No heartbeat from core client for 30 sec - exiting 02:34:09 (2996): No heartbeat from core client for 30 sec - exiting 02:34:10 (2996): No heartbeat from core client for 30 sec - exiting 02:34:11 (2996): No heartbeat from core client for 30 sec - exiting 02:34:12 (2996): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3024, selfPID=5232, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3024, selfPID=3024, iMonCtr=1 02:42:38 (3104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:42:39 (3104): No heartbeat from core client for 30 sec - exiting 02:42:40 (3104): No heartbeat from core client for 30 sec - exiting 02:42:41 (3104): No heartbeat from core client for 30 sec - exiting 02:42:42 (3104): No heartbeat from core client for 30 sec - exiting 02:42:43 (3104): No heartbeat from core client for 30 sec - exiting 02:42:44 (3104): No heartbeat from core client for 30 sec - exiting 02:42:45 (3104): No heartbeat from core client for 30 sec - exiting 02:42:46 (3104): No heartbeat from core client for 30 sec - exiting 02:42:47 (3104): No heartbeat from core client for 30 sec - exiting 02:42:48 (3104): No heartbeat from core client for 30 sec - exiting 02:42:49 (3104): No heartbeat from core client for 30 sec - exiting 02:42:50 (3104): No heartbeat from core client for 30 sec - exiting 02:42:51 (3104): No heartbeat from core client for 30 sec - exiting 02:42:52 (3104): No heartbeat from core client for 30 sec - exiting 02:42:53 (3104): No heartbeat from core client for 30 sec - exiting 02:42:54 (3104): No heartbeat from core client for 30 sec - exiting 02:42:55 (3104): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5184, selfPID=5184, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5184, selfPID=2668, iMonCtr=1 02:47:08 (4496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:47:09 (4496): No heartbeat from core client for 30 sec - exiting 02:47:10 (4496): No heartbeat from core client for 30 sec - exiting 02:47:11 (4496): No heartbeat from core client for 30 sec - exiting 02:47:12 (4496): No heartbeat from core client for 30 sec - exiting 02:47:13 (4496): No heartbeat from core client for 30 sec - exiting 02:47:14 (4496): No heartbeat from core client for 30 sec - exiting 02:47:15 (4496): No heartbeat from core client for 30 sec - exiting 02:47:16 (4496): No heartbeat from core client for 30 sec - exiting 02:47:17 (4496): No heartbeat from core client for 30 sec - exiting 02:47:18 (4496): No heartbeat from core client for 30 sec - exiting 02:47:19 (4496): No heartbeat from core client for 30 sec - exiting 02:47:20 (4496): No heartbeat from core client for 30 sec - exiting 02:47:23 (4496): No heartbeat from core client for 30 sec - exiting 02:47:24 (4496): No heartbeat from core client for 30 sec - exiting 02:47:25 (4496): No heartbeat from core client for 30 sec - exiting 02:47:26 (4496): No heartbeat from core client for 30 sec - exiting 02:47:27 (4496): No heartbeat from core client for 30 sec - exiting 02:47:28 (4496): No heartbeat from core client for 30 sec - exiting 02:47:29 (4496): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=736, selfPID=4760, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=736, selfPID=736, iMonCtr=1 03:23:46 (5356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:23:48 (5356): No heartbeat from core client for 30 sec - exiting 03:23:49 (5356): No heartbeat from core client for 30 sec - exiting 03:23:50 (5356): No heartbeat from core client for 30 sec - exiting 03:23:51 (5356): No heartbeat from core client for 30 sec - exiting 03:23:52 (5356): No heartbeat from core client for 30 sec - exiting 03:23:53 (5356): No heartbeat from core client for 30 sec - exiting 03:23:54 (5356): No heartbeat from core client for 30 sec - exiting 03:23:55 (5356): No heartbeat from core client for 30 sec - exiting 03:23:56 (5356): No heartbeat from core client for 30 sec - exiting 03:23:57 (5356): No heartbeat from core client for 30 sec - exiting 03:23:58 (5356): No heartbeat from core client for 30 sec - exiting 03:23:59 (5356): No heartbeat from core client for 30 sec - exiting 03:24:00 (5356): No heartbeat from core client for 30 sec - exiting 03:24:01 (5356): No heartbeat from core client for 30 sec - exiting 03:24:02 (5356): No heartbeat from core client for 30 sec - exiting 03:24:03 (5356): No heartbeat from core client for 30 sec - exiting 03:24:04 (5356): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5484, selfPID=5484, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5484, selfPID=6536, iMonCtr=1 03:26:21 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:26:22 (2300): No heartbeat from core client for 30 sec - exiting 03:26:23 (2300): No heartbeat from core client for 30 sec - exiting 03:26:24 (2300): No heartbeat from core client for 30 sec - exiting 03:26:25 (2300): No heartbeat from core client for 30 sec - exiting 03:26:26 (2300): No heartbeat from core client for 30 sec - exiting 03:26:27 (2300): No heartbeat from core client for 30 sec - exiting 03:26:28 (2300): No heartbeat from core client for 30 sec - exiting 03:26:29 (2300): No heartbeat from core client for 30 sec - exiting 03:26:30 (2300): No heartbeat from core client for 30 sec - exiting 03:26:31 (2300): No heartbeat from core client for 30 sec - exiting 03:26:32 (2300): No heartbeat from core client for 30 sec - exiting 03:26:33 (2300): No heartbeat from core client for 30 sec - exiting 03:26:34 (2300): No heartbeat from core client for 30 sec - exiting 03:26:35 (2300): No heartbeat from core client for 30 sec - exiting 03:26:36 (2300): No heartbeat from core client for 30 sec - exiting 03:26:37 (2300): No heartbeat from core client for 30 sec - exiting 03:26:38 (2300): No heartbeat from core client for 30 sec - exiting 03:26:39 (2300): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6396, selfPID=776, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6396, selfPID=6396, iMonCtr=1 04:16:15 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:16:16 (3196): No heartbeat from core client for 30 sec - exiting 04:16:17 (3196): No heartbeat from core client for 30 sec - exiting 04:16:18 (3196): No heartbeat from core client for 30 sec - exiting 04:16:20 (3196): No heartbeat from core client for 30 sec - exiting 04:16:21 (3196): No heartbeat from core client for 30 sec - exiting 04:16:22 (3196): No heartbeat from core client for 30 sec - exiting 04:16:24 (3196): No heartbeat from core client for 30 sec - exiting 04:16:25 (3196): No heartbeat from core client for 30 sec - exiting 04:16:27 (3196): No heartbeat from core client for 30 sec - exiting 04:16:28 (3196): No heartbeat from core client for 30 sec - exiting 04:16:29 (3196): No heartbeat from core client for 30 sec - exiting 04:16:30 (3196): No heartbeat from core client for 30 sec - exiting 04:16:31 (3196): No heartbeat from core client for 30 sec - exiting 04:16:32 (3196): No heartbeat from core client for 30 sec - exiting 04:16:33 (3196): No heartbeat from core client for 30 sec - exiting 04:16:34 (3196): No heartbeat from core client for 30 sec - exiting 04:16:35 (3196): No heartbeat from core client for 30 sec - exiting 04:16:36 (3196): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1320, selfPID=1320, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1320, selfPID=4784, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3344, selfPID=5672, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3344, selfPID=3344, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4696, selfPID=4696, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4696, selfPID=4572, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2764, selfPID=344, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2764, selfPID=2764, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6624, selfPID=6624, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6624, selfPID=7100, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=276, selfPID=4208, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=276, selfPID=276, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4720, selfPID=4720, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4720, selfPID=5204, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... 05:19:23 (5428): called boinc_finish Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3756, selfPID=3756, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3756, selfPID=6128, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 05:19:33 (6128): called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_eu_xz6x_1968_1_007050817_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_xz6x_1968_1_007050817_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_xz6x_1968_1_007050817_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_xz6x_1968_1_007050817_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Dec 2010 20:35:17	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	92,256	212,235	2.3005
22 Dec 2010 14:01:01	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	80,736	187,080	2.3172
22 Dec 2010 12:20:50	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	69,216	161,556	2.3341
21 Dec 2010 22:08:21	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	57,696	136,141	2.3596
21 Dec 2010 13:39:38	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	46,177	110,511	2.3932
21 Dec 2010 12:42:57	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	46,176	110,182	2.3861
21 Dec 2010 10:20:04	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	34,656	83,364	2.4055
20 Dec 2010 18:33:12	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	23,136	56,206	2.4294
20 Dec 2010 10:47:00	1111797	12338410	hadam3p_eu_xz6x_1968_1_007050817_0	11,616	28,353	2.4409