Task 12870855

Name	hadam3p_eu_2rb8_2000_1_007250303_0
Workunit	7448543
Created	6 May 2011, 1:11:45 UTC
Sent	7 May 2011, 12:01:34 UTC
Report deadline	18 Apr 2012, 17:21:34 UTC
Received	6 Sep 2011, 22:08:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1137120
Run time	4 days 15 hours 6 min 57 sec
CPU time	3 days 16 hours 36 min 23 sec
Validate state	Invalid
Credit	1,790.21
Device peak FLOPS	2.34 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4160, selfPID=4160, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1216, selfPID=1216, iMonCtr=2 CPDN Monitor - Quit request from BOINC... No ProcCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5416, selfPID=5416, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4612, selfPID=4612, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=804, selfPID=804, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4788, selfPID=4788, iMonCtr=2 17:57:36 (5928): No heartbeat from core client for 30 sec - exiting 17:57:37 (5928): No heartbeat from core client for 30 sec - exiting 17:57:38 (5928): No heartbeat from core client for 30 sec - exiting 17:57:39 (5928): No heartbeat from core client for 30 sec - exiting 17:57:40 (5928): No heartbeat from core client for 30 sec - exiting 17:57:41 (5928): No heartbeat from core client for 30 sec - exiting 17:57:42 (5928): No heartbeat from core client for 30 sec - exiting 17:57:43 (5928): No heartbeat from core client for 30 sec - exiting 17:57:44 (5928): No heartbeat from core client for 30 sec - exiting 17:57:45 (5928): No heartbeat from core client for 30 sec - exiting 17:57:46 (5928): No heartbeat from core client for 30 sec - exiting 17:57:47 (5928): No heartbeat from core client for 30 sec - exiting 17:57:48 (5928): No heartbeat from core client for 30 sec - exiting 17:57:49 (5928): No heartbeat from core client for 30 sec - exiting 17:57:50 (5928): No heartbeat from core client for 30 sec - exiting 17:57:51 (5928): No heartbeat from core client for 30 sec - exiting 17:57:52 (5928): No heartbeat from core client for 30 sec - exiting 17:57:53 (5928): No heartbeat from core client for 30 sec - exiting 17:57:54 (5928): No heartbeat from core client for 30 sec - exiting 17:57:55 (5928): No heartbeat from core client for 30 sec - exiting 17:57:56 (5928): No heartbeat from core client for 30 sec - exiting 17:58:31 (5928): No heartbeat from core client for 30 sec - exiting 17:58:32 (5928): No heartbeat from core client for 30 sec - exiting 17:58:33 (5928): No heartbeat from core client for 30 sec - exiting 17:58:34 (5928): No heartbeat from core client for 30 sec - exiting 17:58:35 (5928): No heartbeat from core client for 30 sec - exiting 17:58:36 (5928): No heartbeat from core client for 30 sec - exiting 17:58:37 (5928): No heartbeat from core client for 30 sec - exiting 17:58:38 (5928): No heartbeat from core client for 30 sec - exiting 17:58:39 (5928): No heartbeat from core client for 30 sec - exiting 17:58:40 (5928): No heartbeat from core client for 30 sec - exiting 17:58:41 (5928): No heartbeat from core client for 30 sec - exiting 17:58:42 (5928): No heartbeat from core client for 30 sec - exiting 17:58:43 (5928): No heartbeat from core client for 30 sec - exiting 17:58:44 (5928): No heartbeat from core client for 30 sec - exiting 17:58:45 (5928): No heartbeat from core client for 30 sec - exiting 17:58:46 (5928): No heartbeat from core client for 30 sec - exiting 17:58:47 (5928): No heartbeat from core client for 30 sec - exiting 17:58:48 (5928): No heartbeat from core client for 30 sec - exiting 17:58:49 (5928): No heartbeat from core client for 30 sec - exiting 17:58:50 (5928): No heartbeat from core client for 30 sec - exiting 17:58:51 (5928): No heartbeat from core client for 30 sec - exiting 17:58:52 (5928): No heartbeat from core client for 30 sec - exiting 17:58:53 (5928): No heartbeat from core client for 30 sec - exiting 17:58:54 (5928): No heartbeat from core client for 30 sec - exiting 17:58:55 (5928): No heartbeat from core client for 30 sec - exiting 17:58:56 (5928): No heartbeat from core client for 30 sec - exiting 17:58:57 (5928): No heartbeat from core client for 30 sec - exiting 17:58:58 (5928): No heartbeat from core client for 30 sec - exiting 17:58:59 (5928): No heartbeat from core client for 30 sec - exiting 17:59:00 (5928): No heartbeat from core client for 30 sec - exiting 17:59:01 (5928): No heartbeat from core client for 30 sec - exiting 17:59:02 (5928): No heartbeat from core client for 30 sec - exiting 17:59:03 (5928): No heartbeat from core client for 30 sec - exiting 17:59:04 (5928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3396, selfPID=3396, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3968, selfPID=6028, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2596, selfPID=2596, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2264, selfPID=2264, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4872, selfPID=4872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5848, selfPID=5848, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=928, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4716, selfPID=4716, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3420, selfPID=3352, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3940, selfPID=3940, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 12:02:11 (4424): No heartbeat from core client for 30 sec - exiting 12:02:12 (4424): No heartbeat from core client for 30 sec - exiting 12:02:13 (4424): No heartbeat from core client for 30 sec - exiting 12:02:14 (4424): No heartbeat from core client for 30 sec - exiting 12:02:15 (4424): No heartbeat from core client for 30 sec - exiting 12:02:16 (4424): No heartbeat from core client for 30 sec - exiting 12:02:17 (4424): No heartbeat from core client for 30 sec - exiting 12:02:18 (4424): No heartbeat from core client for 30 sec - exiting 12:02:19 (4424): No heartbeat from core client for 30 sec - exiting 12:02:20 (4424): No heartbeat from core client for 30 sec - exiting 12:02:21 (4424): No heartbeat from core client for 30 sec - exiting 12:02:22 (4424): No heartbeat from core client for 30 sec - exiting 12:02:23 (4424): No heartbeat from core client for 30 sec - exiting 12:02:24 (4424): No heartbeat from core client for 30 sec - exiting 12:02:25 (4424): No heartbeat from core client for 30 sec - exiting 12:02:27 (4424): No heartbeat from core client for 30 sec - exiting 12:02:28 (4424): No heartbeat from core client for 30 sec - exiting 12:02:29 (4424): No heartbeat from core client for 30 sec - exiting 12:02:30 (4424): No heartbeat from core client for 30 sec - exiting 12:02:31 (4424): No heartbeat from core client for 30 sec - exiting 12:02:32 (4424): No heartbeat from core client for 30 sec - exiting 12:02:33 (4424): No heartbeat from core client for 30 sec - exiting 12:02:34 (4424): No heartbeat from core client for 30 sec - exiting 12:02:35 (4424): No heartbeat from core client for 30 sec - exiting 12:02:36 (4424): No heartbeat from core client for 30 sec - exiting 12:02:37 (4424): No heartbeat from core client for 30 sec - exiting 12:02:38 (4424): No heartbeat from core client for 30 sec - exiting 12:02:39 (4424): No heartbeat from core client for 30 sec - exiting 12:02:40 (4424): No heartbeat from core client for 30 sec - exiting 12:02:41 (4424): No heartbeat from core client for 30 sec - exiting 12:02:42 (4424): No heartbeat from core client for 30 sec - exiting 12:02:43 (4424): No heartbeat from core client for 30 sec - exiting 12:02:44 (4424): No heartbeat from core client for 30 sec - exiting 12:02:45 (4424): No heartbeat from core client for 30 sec - exiting 12:02:46 (4424): No heartbeat from core client for 30 sec - exiting 12:02:47 (4424): No heartbeat from core client for 30 sec - exiting 12:02:48 (4424): No heartbeat from core client for 30 sec - exiting 12:02:49 (4424): No heartbeat from core client for 30 sec - exiting 12:02:50 (4424): No heartbeat from core client for 30 sec - exiting 12:02:51 (4424): No heartbeat from core client for 30 sec - exiting 12:02:52 (4424): No heartbeat from core client for 30 sec - exiting 12:02:53 (4424): No heartbeat from core client for 30 sec - exiting 12:02:54 (4424): No heartbeat from core client for 30 sec - exiting 12:02:55 (4424): No heartbeat from core client for 30 sec - exiting 12:02:56 (4424): No heartbeat from core client for 30 sec - exiting 12:02:57 (4424): No heartbeat from core client for 30 sec - exiting 12:02:58 (4424): No heartbeat from core client for 30 sec - exiting 12:02:59 (4424): No heartbeat from core client for 30 sec - exiting 12:03:00 (4424): No heartbeat from core client for 30 sec - exiting 12:03:01 (4424): No heartbeat from core client for 30 sec - exiting 12:03:02 (4424): No heartbeat from core client for 30 sec - exiting 12:03:03 (4424): No heartbeat from core client for 30 sec - exiting 12:03:04 (4424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5356, selfPID=5356, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4520, selfPID=4520, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5972, selfPID=5972, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4064, selfPID=4064, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=344, selfPID=344, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5392, selfPID=5392, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Sep 2011 13:15:58	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	103,776	312,723	3.0134
30 Aug 2011 08:06:01	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	92,256	275,354	2.9847
17 Aug 2011 20:38:14	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	80,736	240,915	2.9840
15 Aug 2011 21:00:52	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	69,216	208,385	3.0106
14 Aug 2011 22:40:49	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	57,696	170,815	2.9606
14 Aug 2011 10:07:22	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	46,176	138,785	3.0056
07 Aug 2011 19:09:16	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	34,656	104,359	3.0113
25 Jul 2011 15:58:08	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	23,136	69,155	2.9891
01 Jul 2011 23:55:00	1137120	12870855	hadam3p_eu_2rb8_2000_1_007250303_0	11,616	34,520	2.9718