Task 12132156

Name	hadam3p_saf_0se4_1987_1_006860372_0
Workunit	7063688
Created	19 Nov 2010, 10:17:13 UTC
Sent	23 Apr 2011, 11:04:35 UTC
Report deadline	4 Apr 2012, 16:24:35 UTC
Received	19 May 2011, 23:18:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1138333
Run time	2 days 5 hours 28 min 3 sec
CPU time	2 days 1 hours 24 min 6 sec
Validate state	Invalid
Credit	562.19
Device peak FLOPS	2.56 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6344, selfPID=6344, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6684, selfPID=6684, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10708, selfPID=10708, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9188, selfPID=9188, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7744, selfPID=7744, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6716, selfPID=6716, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10728, selfPID=10728, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10852, selfPID=10852, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1616, selfPID=1616, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10720, selfPID=10720, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10272, selfPID=10272, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8312, selfPID=8312, iMonCtr=2 04:40:40 (1020): No heartbeat from core client for 30 sec - exiting 04:40:41 (1020): No heartbeat from core client for 30 sec - exiting 04:40:42 (1020): No heartbeat from core client for 30 sec - exiting 04:40:43 (1020): No heartbeat from core client for 30 sec - exiting 04:40:44 (1020): No heartbeat from core client for 30 sec - exiting 04:40:45 (1020): No heartbeat from core client for 30 sec - exiting 04:40:46 (1020): No heartbeat from core client for 30 sec - exiting 04:40:47 (1020): No heartbeat from core client for 30 sec - exiting 04:40:48 (1020): No heartbeat from core client for 30 sec - exiting 04:40:50 (1020): No heartbeat from core client for 30 sec - exiting 04:40:51 (1020): No heartbeat from core client for 30 sec - exiting 04:40:52 (1020): No heartbeat from core client for 30 sec - exiting 04:40:53 (1020): No heartbeat from core client for 30 sec - exiting 04:40:54 (1020): No heartbeat from core client for 30 sec - exiting 04:40:55 (1020): No heartbeat from core client for 30 sec - exiting 04:40:56 (1020): No heartbeat from core client for 30 sec - exiting 04:40:57 (1020): No heartbeat from core client for 30 sec - exiting 04:40:58 (1020): No heartbeat from core client for 30 sec - exiting 04:40:59 (1020): No heartbeat from core client for 30 sec - exiting 04:41:00 (1020): No heartbeat from core client for 30 sec - exiting 04:41:02 (1020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:56:06 (1740): No heartbeat from core client for 30 sec - exiting 17:56:08 (1740): No heartbeat from core client for 30 sec - exiting 17:56:09 (1740): No heartbeat from core client for 30 sec - exiting 17:56:10 (1740): No heartbeat from core client for 30 sec - exiting 17:56:11 (1740): No heartbeat from core client for 30 sec - exiting 17:56:12 (1740): No heartbeat from core client for 30 sec - exiting 17:56:13 (1740): No heartbeat from core client for 30 sec - exiting 17:56:14 (1740): No heartbeat from core client for 30 sec - exiting 17:56:15 (1740): No heartbeat from core client for 30 sec - exiting 17:56:17 (1740): No heartbeat from core client for 30 sec - exiting 17:56:18 (1740): No heartbeat from core client for 30 sec - exiting 17:56:19 (1740): No heartbeat from core client for 30 sec - exiting 17:56:20 (1740): No heartbeat from core client for 30 sec - exiting 17:56:21 (1740): No heartbeat from core client for 30 sec - exiting 17:56:22 (1740): No heartbeat from core client for 30 sec - exiting 17:56:23 (1740): No heartbeat from core client for 30 sec - exiting 17:56:24 (1740): No heartbeat from core client for 30 sec - exiting 17:56:25 (1740): No heartbeat from core client for 30 sec - exiting 17:56:26 (1740): No heartbeat from core client for 30 sec - exiting 17:56:27 (1740): No heartbeat from core client for 30 sec - exiting 17:56:29 (1740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10920, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=832, selfPID=832, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1400, selfPID=1400, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:13:03 (9516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:13:09 (9516): No heartbeat from core client for 30 sec - exiting 11:13:10 (9516): No heartbeat from core client for 30 sec - exiting 11:13:11 (9516): No heartbeat from core client for 30 sec - exiting 11:13:12 (9516): No heartbeat from core client for 30 sec - exiting 11:13:13 (9516): No heartbeat from core client for 30 sec - exiting 11:13:15 (9516): No heartbeat from core client for 30 sec - exiting 11:13:16 (9516): No heartbeat from core client for 30 sec - exiting 11:13:17 (9516): No heartbeat from core client for 30 sec - exiting 11:13:18 (9516): No heartbeat from core client for 30 sec - exiting 11:13:19 (9516): No heartbeat from core client for 30 sec - exiting 11:13:20 (9516): No heartbeat from core client for 30 sec - exiting 11:13:21 (9516): No heartbeat from core client for 30 sec - exiting 11:13:22 (9516): No heartbeat from core client for 30 sec - exiting 11:13:23 (9516): No heartbeat from core client for 30 sec - exiting 11:13:25 (9516): No heartbeat from core client for 30 sec - exiting 11:13:26 (9516): No heartbeat from core client for 30 sec - exiting 11:13:27 (9516): No heartbeat from core client for 30 sec - exiting 11:13:28 (9516): No heartbeat from core client for 30 sec - exiting 11:13:29 (9516): No heartbeat from core client for 30 sec - exiting 11:18:00 (8656): No heartbeat from core client for 30 sec - exiting 11:18:02 (8656): No heartbeat from core client for 30 sec - exiting 11:18:03 (8656): No heartbeat from core client for 30 sec - exiting 11:18:04 (8656): No heartbeat from core client for 30 sec - exiting 11:18:05 (8656): No heartbeat from core client for 30 sec - exiting 11:18:06 (8656): No heartbeat from core client for 30 sec - exiting 11:18:07 (8656): No heartbeat from core client for 30 sec - exiting 11:18:08 (8656): No heartbeat from core client for 30 sec - exiting 11:18:09 (8656): No heartbeat from core client for 30 sec - exiting 11:18:10 (8656): No heartbeat from core client for 30 sec - exiting 11:18:11 (8656): No heartbeat from core client for 30 sec - exiting 11:18:12 (8656): No heartbeat from core client for 30 sec - exiting 11:18:13 (8656): No heartbeat from core client for 30 sec - exiting 11:18:14 (8656): No heartbeat from core client for 30 sec - exiting 11:18:15 (8656): No heartbeat from core client for 30 sec - exiting 11:18:16 (8656): No heartbeat from core client for 30 sec - exiting 11:18:17 (8656): No heartbeat from core client for 30 sec - exiting 11:18:18 (8656): No heartbeat from core client for 30 sec - exiting 11:18:19 (8656): No heartbeat from core client for 30 sec - exiting 11:18:20 (8656): No heartbeat from core client for 30 sec - exiting 11:18:21 (8656): No heartbeat from core client for 30 sec - exiting 11:18:22 (8656): No heartbeat from core client for 30 sec - exiting 11:18:23 (8656): No heartbeat from core client for 30 sec - exiting 11:18:24 (8656): No heartbeat from core client for 30 sec - exiting 11:18:25 (8656): No heartbeat from core client for 30 sec - exiting 11:18:27 (8656): No heartbeat from core client for 30 sec - exiting 11:18:28 (8656): No heartbeat from core client for 30 sec - exiting 11:18:29 (8656): No heartbeat from core client for 30 sec - exiting 11:18:30 (8656): No heartbeat from core client for 30 sec - exiting 11:18:31 (8656): No heartbeat from core client for 30 sec - exiting 11:18:32 (8656): No heartbeat from core client for 30 sec - exiting 11:18:33 (8656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:05 (6348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:08 (6348): No heartbeat from core client for 30 sec - exiting 11:33:09 (6348): No heartbeat from core client for 30 sec - exiting 11:33:11 (6348): No heartbeat from core client for 30 sec - exiting 11:33:12 (6348): No heartbeat from core client for 30 sec - exiting 11:33:13 (6348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:58:15 (7728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:58:18 (7728): No heartbeat from core client for 30 sec - exiting 15:58:19 (7728): No heartbeat from core client for 30 sec - exiting 15:58:20 (7728): No heartbeat from core client for 30 sec - exiting 15:58:21 (7728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:18:47 (4180): No heartbeat from core client for 30 sec - exiting 00:18:48 (4180): No heartbeat from core client for 30 sec - exiting 00:18:49 (4180): No heartbeat from core client for 30 sec - exiting 00:18:50 (4180): No heartbeat from core client for 30 sec - exiting 00:18:51 (4180): No heartbeat from core client for 30 sec - exiting 00:18:52 (4180): No heartbeat from core client for 30 sec - exiting 00:18:54 (4180): No heartbeat from core client for 30 sec - exiting 00:18:55 (4180): No heartbeat from core client for 30 sec - exiting 00:18:56 (4180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:23:47 (8032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:23:49 (8032): No heartbeat from core client for 30 sec - exiting 00:23:50 (8032): No heartbeat from core client for 30 sec - exiting 00:42:19 (4384): No heartbeat from core client for 30 sec - exiting 00:42:20 (4384): No heartbeat from core client for 30 sec - exiting 00:42:21 (4384): No heartbeat from core client for 30 sec - exiting 00:42:22 (4384): No heartbeat from core client for 30 sec - exiting 00:42:23 (4384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8416, selfPID=8416, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9140, selfPID=9140, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9532, selfPID=9532, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6432, selfPID=6432, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9860, selfPID=9860, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10360, selfPID=10360, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8824, selfPID=8824, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4376, selfPID=4376, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11172, selfPID=11172, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10232, selfPID=10232, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN processCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 May 2011 04:10:02	1138333	12132156	hadam3p_saf_0se4_1987_1_006860372_0	34,656	137,614	3.9709
27 Apr 2011 17:40:36	1138333	12132156	hadam3p_saf_0se4_1987_1_006860372_0	23,136	93,516	4.0420
26 Apr 2011 15:43:29	1138333	12132156	hadam3p_saf_0se4_1987_1_006860372_0	11,616	46,566	4.0088