Task 16117191

Name	hadam3p_eu_iyok_2004_1_008490137_0
Workunit	8640950
Created	3 Dec 2013, 20:58:29 UTC
Sent	3 Dec 2013, 20:59:59 UTC
Report deadline	16 Nov 2014, 2:19:59 UTC
Received	3 Jan 2014, 19:39:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1115566
Run time	7 days 8 hours 2 min 9 sec
CPU time	5 days 8 hours 7 min 6 sec
Validate state	Invalid
Credit	1,988.94
Device peak FLOPS	1.25 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:10:36 (5312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:49:59 (6116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:50:00 (6116): No heartbeat from core client for 30 sec - exiting 23:23:36 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:46 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:33:43 (6060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2112, selfPID=2112, iMonCtr=2 C12:19:56 (4180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:54:47 (1228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:29:28 (3308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:04:12 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:59 (5624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:48:25 (3044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:11 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:07:54 (3228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5148, selfPID=5148, iMonCtr=2 18:44:49 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:44:50 (4788): No heartbeat from core client for 30 sec - exiting 20:29:25 (5092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:25:16 (3168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:25:17 (3168): No heartbeat from core client for 30 sec - exiting 23:25:18 (3168): No heartbeat from core client for 30 sec - exiting 23:25:19 (3168): No heartbeat from core client for 30 sec - exiting 23:25:20 (3168): No heartbeat from core client for 30 sec - exiting 19:55:21 (5460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:25:50 (7060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:25:51 (7060): No heartbeat from core client for 30 sec - exiting 22:03:52 (6348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6228, selfPID=3476, iMonCtr=1 Model crash detected, will try to restart... 21:43:13 (5076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4980, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3420, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 11:26:53 (5116): No heartbeat from core client for 30 sec - exiting 11:26:55 (5116): No heartbeat from core client for 30 sec - exiting 11:26:56 (5116): No heartbeat from core client for 30 sec - exiting 11:26:57 (5116): No heartbeat from core client for 30 sec - exiting 11:26:58 (5116): No heartbeat from core client for 30 sec - exiting 11:26:59 (5116): No heartbeat from core client for 30 sec - exiting 11:27:00 (5116): No heartbeat from core client for 30 sec - exiting 11:27:01 (5116): No heartbeat from core client for 30 sec - exiting 11:27:02 (5116): No heartbeat from core client for 30 sec - exiting 11:27:03 (5116): No heartbeat from core client for 30 sec - exiting 11:27:04 (5116): No heartbeat from core client for 30 sec - exiting 11:27:06 (5116): No heartbeat from core client for 30 sec - exiting 11:27:07 (5116): No heartbeat from core client for 30 sec - exiting 11:27:08 (5116): No heartbeat from core client for 30 sec - exiting 11:27:09 (5116): No heartbeat from core client for 30 sec - exiting 11:27:10 (5116): No heartbeat from core client for 30 sec - exiting 11:27:11 (5116): No heartbeat from core client for 30 sec - exiting 11:27:12 (5116): No heartbeat from core client for 30 sec - exiting 11:27:13 (5116): No heartbeat from core client for 30 sec - exiting 11:27:14 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5552, iMonCtr=2 Model crash detected, will try to restart... 20:14:17 (4552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:59:48 (868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:59:49 (868): No heartbeat from core client for 30 sec - exiting 21:59:50 (868): No heartbeat from core client for 30 sec - exiting 21:59:51 (868): No heartbeat from core client for 30 sec - exiting 21:59:52 (868): No heartbeat from core client for 30 sec - exiting 21:59:53 (868): No heartbeat from core client for 30 sec - exiting 21:59:54 (868): No heartbeat from core client for 30 sec - exiting 21:59:55 (868): No heartbeat from core client for 30 sec - exiting 23:35:35 (4140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:18:20 (5096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:20 (4068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:13:43 (3356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:57:21 (4972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:56 (6068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:41 (4836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:35:38 (5924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:56 (1236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:35:00 (7028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:18:36 (6812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:23 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:44:39 (4968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:36:07 (7096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:22 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:15 (5204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:28:10 (6056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1656, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6312, selfPID=5044, iMonCtr=1 Model crash detected, will try to restart... 14:00:06 (5776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:26 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:39:53 (4196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6616, selfPID=6200, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5756, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2628, selfPID=3848, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4428, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4452, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5688, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5700, selfPID=2616, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadam3p_eu_iyok_2004_1_008490137/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadam3p_eu_iyok_2004_1_008490137/dataout/region_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakg.pipe_dummy 2048 </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_eu_iyok_2004_1_008490137_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_iyok_2004_1_008490137_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Jan 2014 00:33:45	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	115,296	461,855	4.0058
01 Jan 2014 23:32:58	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	103,776	417,361	4.0217
01 Jan 2014 10:49:52	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	92,256	375,407	4.0692
31 Dec 2013 12:50:46	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	80,736	331,892	4.1108
29 Dec 2013 17:07:33	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	69,216	288,170	4.1633
27 Dec 2013 22:58:15	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	57,696	245,224	4.2503
24 Dec 2013 14:20:09	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	46,176	200,441	4.3408
14 Dec 2013 19:37:29	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	34,656	147,779	4.2642
09 Dec 2013 20:57:13	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	23,143	100,632	4.3483
09 Dec 2013 20:17:00	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	23,136	99,743	4.3112
07 Dec 2013 18:58:00	1115566	16117191	hadam3p_eu_iyok_2004_1_008490137_0	11,616	47,688	4.1054