Task 13821341

Name	hadam3p_pnw_6twj_2008_1_007587339_2
Workunit	7765469
Created	26 Dec 2011, 7:17:47 UTC
Sent	26 Dec 2011, 7:18:54 UTC
Report deadline	7 Dec 2012, 12:38:54 UTC
Received	14 Mar 2012, 6:45:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1052338
Run time	6 days 6 hours 7 min 53 sec
CPU time	5 days 10 hours 48 min 10 sec
Validate state	Invalid
Credit	2,505.24
Device peak FLOPS	1.98 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> 11:20:45 (6384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:20:51 (6384): No heartbeat from core client for 30 sec - exiting 11:20:52 (6384): No heartbeat from core client for 30 sec - exiting 11:20:53 (6384): No heartbeat from core client for 30 sec - exiting 11:20:54 (6384): No heartbeat from core client for 30 sec - exiting 11:58:24 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:58:26 (6064): No heartbeat from core client for 30 sec - exiting 11:58:27 (6064): No heartbeat from core client for 30 sec - exiting 11:58:28 (6064): No heartbeat from core client for 30 sec - exiting 12:16:07 (7256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:16:09 (7256): No heartbeat from core client for 30 sec - exiting 12:16:10 (7256): No heartbeat from core client for 30 sec - exiting 12:16:11 (7256): No heartbeat from core client for 30 sec - exiting 12:16:12 (7256): No heartbeat from core client for 30 sec - exiting 12:16:13 (7256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:10:37 (7296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:10:38 (7296): No heartbeat from core client for 30 sec - exiting 19:10:39 (7296): No heartbeat from core client for 30 sec - exiting 19:10:40 (7296): No heartbeat from core client for 30 sec - exiting 20:19:43 (4848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:19:44 (4848): No heartbeat from core client for 30 sec - exiting 20:19:45 (4848): No heartbeat from core client for 30 sec - exiting 21:08:07 (4724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:09 (4724): No heartbeat from core client for 30 sec - exiting 21:08:10 (4724): No heartbeat from core client for 30 sec - exiting 21:08:11 (4724): No heartbeat from core client for 30 sec - exiting 21:08:12 (4724): No heartbeat from core client for 30 sec - exiting 21:08:13 (4724): No heartbeat from core client for 30 sec - exiting 21:08:14 (4724): No heartbeat from core client for 30 sec - exiting 22:31:32 (5296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:31:33 (5296): No heartbeat from core client for 30 sec - exiting 22:31:34 (5296): No heartbeat from core client for 30 sec - exiting 23:50:52 (6780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:56:41 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:56:43 (4232): No heartbeat from core client for 30 sec - exiting 23:56:44 (4232): No heartbeat from core client for 30 sec - exiting 23:56:45 (4232): No heartbeat from core client for 30 sec - exiting 23:56:46 (4232): No heartbeat from core client for 30 sec - exiting 23:56:47 (4232): No heartbeat from core client for 30 sec - exiting 00:12:29 (8408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:12:32 (8408): No heartbeat from core client for 30 sec - exiting 00:12:33 (8408): No heartbeat from core client for 30 sec - exiting 00:12:34 (8408): No heartbeat from core client for 30 sec - exiting 00:12:35 (8408): No heartbeat from core client for 30 sec - exiting 00:12:36 (8408): No heartbeat from core client for 30 sec - exiting 00:12:37 (8408): No heartbeat from core client for 30 sec - exiting 00:12:38 (8408): No heartbeat from core client for 30 sec - exiting 02:08:17 (3108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:42:38 (7324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:42:40 (7324): No heartbeat from core client for 30 sec - exiting 02:42:41 (7324): No heartbeat from core client for 30 sec - exiting 07:03:23 (6956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:03:25 (6956): No heartbeat from core client for 30 sec - exiting 07:03:26 (6956): No heartbeat from core client for 30 sec - exiting 07:03:28 (6956): No heartbeat from core client for 30 sec - exiting 07:03:29 (6956): No heartbeat from core client for 30 sec - exiting 07:03:30 (6956): No heartbeat from core client for 30 sec - exiting 07:18:41 (5164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:18:43 (5164): No heartbeat from core client for 30 sec - exiting 07:18:44 (5164): No heartbeat from core client for 30 sec - exiting 10:09:58 (8860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:10:02 (8860): No heartbeat from core client for 30 sec - exiting 10:10:03 (8860): No heartbeat from core client for 30 sec - exiting 10:10:04 (8860): No heartbeat from core client for 30 sec - exiting 10:10:05 (8860): No heartbeat from core client for 30 sec - exiting 10:10:06 (8860): No heartbeat from core client for 30 sec - exiting 10:10:07 (8860): No heartbeat from core client for 30 sec - exiting 10:10:08 (8860): No heartbeat from core client for 30 sec - exiting 10:10:09 (8860): No heartbeat from core client for 30 sec - exiting 10:10:10 (8860): No heartbeat from core client for 30 sec - exiting 12:10:32 (7156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:10:34 (7156): No heartbeat from core client for 30 sec - exiting 12:10:35 (7156): No heartbeat from core client for 30 sec - exiting 12:10:36 (7156): No heartbeat from core client for 30 sec - exiting 12:15:48 (7900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:49 (7900): No heartbeat from core client for 30 sec - exiting 12:15:50 (7900): No heartbeat from core client for 30 sec - exiting 12:45:33 (3400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:45:35 (3400): No heartbeat from core client for 30 sec - exiting 12:45:36 (3400): No heartbeat from core client for 30 sec - exiting 15:21:02 (7140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:17:40 (6432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:17:41 (6432): No heartbeat from core client for 30 sec - exiting 20:46:55 (8636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:46:58 (8636): No heartbeat from core client for 30 sec - exiting 20:46:59 (8636): No heartbeat from core client for 30 sec - exiting 20:47:00 (8636): No heartbeat from core client for 30 sec - exiting 20:47:01 (8636): No heartbeat from core client for 30 sec - exiting 20:47:02 (8636): No heartbeat from core client for 30 sec - exiting 20:47:04 (8636): No heartbeat from core client for 30 sec - exiting 20:47:05 (8636): No heartbeat from core client for 30 sec - exiting 21:42:08 (7944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:11 (7944): No heartbeat from core client for 30 sec - exiting 21:42:12 (7944): No heartbeat from core client for 30 sec - exiting 21:42:13 (7944): No heartbeat from core client for 30 sec - exiting 21:42:14 (7944): No heartbeat from core client for 30 sec - exiting 21:43:50 (7620): No heartbeat from core client for 30 sec - exiting 21:43:51 (7620): No heartbeat from core client for 30 sec - exiting 21:43:52 (7620): No heartbeat from core client for 30 sec - exiting 21:43:53 (7620): No heartbeat from core client for 30 sec - exiting 21:43:54 (7620): No heartbeat from core client for 30 sec - exiting 21:43:55 (7620): No heartbeat from core client for 30 sec - exiting 21:43:57 (7620): No heartbeat from core client for 30 sec - exiting 21:43:58 (7620): No heartbeat from core client for 30 sec - exiting 21:43:59 (7620): No heartbeat from core client for 30 sec - exiting 21:44:00 (7620): No heartbeat from core client for 30 sec - exiting 21:44:01 (7620): No heartbeat from core client for 30 sec - exiting 21:44:02 (7620): No heartbeat from core client for 30 sec - exiting 21:44:03 (7620): No heartbeat from core client for 30 sec - exiting 21:44:04 (7620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:05:41 (6660): No heartbeat from core client for 30 sec - exiting 23:05:43 (6660): No heartbeat from core client for 30 sec - exiting 23:05:44 (6660): No heartbeat from core client for 30 sec - exiting 23:05:45 (6660): No heartbeat from core client for 30 sec - exiting 23:05:46 (6660): No heartbeat from core client for 30 sec - exiting 23:05:48 (6660): No heartbeat from core client for 30 sec - exiting 23:05:49 (6660): No heartbeat from core client for 30 sec - exiting 23:05:50 (6660): No heartbeat from core client for 30 sec - exiting 23:05:51 (6660): No heartbeat from core client for 30 sec - exiting 23:05:52 (6660): No heartbeat from core client for 30 sec - exiting 23:05:53 (6660): No heartbeat from core client for 30 sec - exiting 23:05:54 (6660): No heartbeat from core client for 30 sec - exiting 23:05:55 (6660): No heartbeat from core client for 30 sec - exiting 23:05:56 (6660): No heartbeat from core client for 30 sec - exiting 23:05:57 (6660): No heartbeat from core client for 30 sec - exiting 23:05:59 (6660): No heartbeat from core client for 30 sec - exiting 23:06:00 (6660): No heartbeat from core client for 30 sec - exiting 23:06:01 (6660): No heartbeat from core client for 30 sec - exiting 23:06:02 (6660): No heartbeat from core client for 30 sec - exiting 23:06:03 (6660): No heartbeat from core client for 30 sec - exiting 23:06:04 (6660): No heartbeat from core client for 30 sec - exiting 23:06:05 (6660): No heartbeat from core client for 30 sec - exiting 23:06:06 (6660): No heartbeat from core client for 30 sec - exiting 23:06:07 (6660): No heartbeat from core client for 30 sec - exiting 23:06:08 (6660): No heartbeat from core client for 30 sec - exiting 23:06:09 (6660): No heartbeat from core client for 30 sec - exiting 23:06:10 (6660): No heartbeat from core client for 30 sec - exiting 23:06:11 (6660): No heartbeat from core client for 30 sec - exiting 23:06:12 (6660): No heartbeat from core client for 30 sec - exiting 23:06:13 (6660): No heartbeat from core client for 30 sec - exiting 23:06:14 (6660): No heartbeat from core client for 30 sec - exiting 23:06:15 (6660): No heartbeat from core client for 30 sec - exiting 23:06:16 (6660): No heartbeat from core client for 30 sec - exiting 23:06:18 (6660): No heartbeat from core client for 30 sec - exiting 23:06:19 (6660): No heartbeat from core client for 30 sec - exiting 23:06:20 (6660): No heartbeat from core client for 30 sec - exiting 23:06:21 (6660): No heartbeat from core client for 30 sec - exiting 23:06:22 (6660): No heartbeat from core client for 30 sec - exiting 23:06:23 (6660): No heartbeat from core client for 30 sec - exiting 23:06:24 (6660): No heartbeat from core client for 30 sec - exiting 23:06:25 (6660): No heartbeat from core client for 30 sec - exiting 23:06:26 (6660): No heartbeat from core client for 30 sec - exiting 23:06:27 (6660): No heartbeat from core client for 30 sec - exiting 23:06:28 (6660): No heartbeat from core client for 30 sec - exiting 23:06:29 (6660): No heartbeat from core client for 30 sec - exiting 23:06:30 (6660): No heartbeat from core client for 30 sec - exiting 23:06:31 (6660): No heartbeat from core client for 30 sec - exiting 23:06:32 (6660): No heartbeat from core client for 30 sec - exiting 23:06:33 (6660): No heartbeat from core client for 30 sec - exiting 23:06:34 (6660): No heartbeat from core client for 30 sec - exiting 23:06:35 (6660): No heartbeat from core client for 30 sec - exiting 23:06:36 (6660): No heartbeat from core client for 30 sec - exiting 23:06:37 (6660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:06:38 (6660): No heartbeat from core client for 30 sec - exiting 01:59:43 (9076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:45 (9076): No heartbeat from core client for 30 sec - exiting 01:59:46 (9076): No heartbeat from core client for 30 sec - exiting 01:59:47 (9076): No heartbeat from core client for 30 sec - exiting 02:39:57 (8000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:39:59 (8000): No heartbeat from core client for 30 sec - exiting 02:40:00 (8000): No heartbeat from core client for 30 sec - exiting 02:42:14 (4632): No heartbeat from core client for 30 sec - exiting 02:42:15 (4632): No heartbeat from core client for 30 sec - exiting 02:42:16 (4632): No heartbeat from core client for 30 sec - exiting 02:42:17 (4632): No heartbeat from core client for 30 sec - exiting 02:42:18 (4632): No heartbeat from core client for 30 sec - exiting 02:42:19 (4632): No heartbeat from core client for 30 sec - exiting 02:42:20 (4632): No heartbeat from core client for 30 sec - exiting 02:42:22 (4632): No heartbeat from core client for 30 sec - exiting 02:42:23 (4632): No heartbeat from core client for 30 sec - exiting 02:42:24 (4632): No heartbeat from core client for 30 sec - exiting 02:42:25 (4632): No heartbeat from core client for 30 sec - exiting 02:42:26 (4632): No heartbeat from core client for 30 sec - exiting 02:42:27 (4632): No heartbeat from core client for 30 sec - exiting 02:42:28 (4632): No heartbeat from core client for 30 sec - exiting 02:42:29 (4632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:16:44 (8196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:16:46 (8196): No heartbeat from core client for 30 sec - exiting 07:16:47 (8196): No heartbeat from core client for 30 sec - exiting 07:16:48 (8196): No heartbeat from core client for 30 sec - exiting 07:16:49 (8196): No heartbeat from core client for 30 sec - exiting 07:16:51 (8196): No heartbeat from core client for 30 sec - exiting 07:16:52 (8196): No heartbeat from core client for 30 sec - exiting 08:19:49 (6772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:19:51 (6772): No heartbeat from core client for 30 sec - exiting 08:19:52 (6772): No heartbeat from core client for 30 sec - exiting 08:19:53 (6772): No heartbeat from core client for 30 sec - exiting 08:19:54 (6772): No heartbeat from core client for 30 sec - exiting 08:19:55 (6772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:50:38 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:50:42 (5140): No heartbeat from core client for 30 sec - exiting 17:50:43 (5140): No heartbeat from core client for 30 sec - exiting 17:50:44 (5140): No heartbeat from core client for 30 sec - exiting 17:50:45 (5140): No heartbeat from core client for 30 sec - exiting 17:50:46 (5140): No heartbeat from core client for 30 sec - exiting 17:50:47 (5140): No heartbeat from core client for 30 sec - exiting 18:03:20 (8312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:03:22 (8312): No heartbeat from core client for 30 sec - exiting 18:03:23 (8312): No heartbeat from core client for 30 sec - exiting 18:03:24 (8312): No heartbeat from core client for 30 sec - exiting 18:03:25 (8312): No heartbeat from core client for 30 sec - exiting 18:03:26 (8312): No heartbeat from core client for 30 sec - exiting 18:03:27 (8312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:19:20 (4376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:19:21 (4376): No heartbeat from core client for 30 sec - exiting 02:19:22 (4376): No heartbeat from core client for 30 sec - exiting 02:19:23 (4376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 08:14:32 (7384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:14:33 (7384): No heartbeat from core client for 30 sec - exiting 08:14:34 (7384): No heartbeat from core client for 30 sec - exiting 08:14:35 (7384): No heartbeat from core client for 30 sec - exiting 08:22:09 (348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:22:10 (348): No heartbeat from core client for 30 sec - exiting 08:22:11 (348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:25:19 (7324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:25:21 (7324): No heartbeat from core client for 30 sec - exiting 17:25:22 (7324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2156, selfPID=2156, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2924, selfPID=2924, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2748, selfPID=2748, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2712, selfPID=2712, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=204, selfPID=204, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1368, selfPID=1368, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:53:07 (4408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Mar 2012 22:32:50	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	115,296	451,578	3.9167
12 Mar 2012 06:13:16	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	103,776	408,125	3.9327
29 Feb 2012 20:54:28	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	92,256	366,338	3.9709
29 Feb 2012 05:29:34	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	80,736	322,973	4.0004
28 Feb 2012 16:53:32	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	69,216	280,102	4.0468
28 Feb 2012 00:20:30	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	57,696	235,914	4.0889
30 Dec 2011 16:43:24	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	46,176	186,849	4.0465
29 Dec 2011 06:43:19	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	34,656	143,452	4.1393
27 Dec 2011 23:12:27	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	23,153	101,580	4.3873
27 Dec 2011 22:12:09	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	23,136	100,923	4.3622
27 Dec 2011 05:36:51	1052338	13821341	hadam3p_pnw_6twj_2008_1_007587339_2	11,616	55,911	4.8133