Task 15992015

Name	hadcm3n_805l_1980_40_008458140_0
Workunit	8608996
Created	30 Aug 2013, 20:05:17 UTC
Sent	8 Sep 2013, 21:37:01 UTC
Report deadline	9 Dec 2013, 5:04:12 UTC
Received	16 Dec 2013, 15:36:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1289120
Run time	11 days 16 hours 1 min 54 sec
CPU time	11 days 9 hours 16 min 23 sec
Validate state	Invalid
Credit	4,665.60
Device peak FLOPS	2.73 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:00:51 (3100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:11:41 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:27:07 (636): No heartbeat from core client for 30 sec - exiting 09:27:08 (636): No heartbeat from core client for 30 sec - exiting 09:27:09 (636): No heartbeat from core client for 30 sec - exiting 09:27:10 (636): No heartbeat from core client for 30 sec - exiting 09:27:11 (636): No heartbeat from core client for 30 sec - exiting 09:27:12 (636): No heartbeat from core client for 30 sec - exiting 09:27:13 (636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:24:53 (4712): No heartbeat from core client for 30 sec - exiting 22:24:54 (4712): No heartbeat from core client for 30 sec - exiting 22:24:55 (4712): No heartbeat from core client for 30 sec - exiting 22:24:56 (4712): No heartbeat from core client for 30 sec - exiting 22:24:57 (4712): No heartbeat from core client for 30 sec - exiting 22:24:58 (4712): No heartbeat from core client for 30 sec - exiting 22:24:59 (4712): No heartbeat from core client for 30 sec - exiting 22:25:00 (4712): No heartbeat from core client for 30 sec - exiting 22:25:01 (4712): No heartbeat from core client for 30 sec - exiting 22:25:02 (4712): No heartbeat from core client for 30 sec - exiting 22:25:03 (4712): No heartbeat from core client for 30 sec - exiting 22:25:04 (4712): No heartbeat from core client for 30 sec - exiting 22:25:05 (4712): No heartbeat from core client for 30 sec - exiting 22:25:06 (4712): No heartbeat from core client for 30 sec - exiting 22:25:07 (4712): No heartbeat from core client for 30 sec - exiting 22:25:08 (4712): No heartbeat from core client for 30 sec - exiting 22:25:09 (4712): No heartbeat from core client for 30 sec - exiting 22:25:10 (4712): No heartbeat from core client for 30 sec - exiting 22:25:11 (4712): No heartbeat from core client for 30 sec - exiting 22:25:12 (4712): No heartbeat from core client for 30 sec - exiting 22:25:13 (4712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:50:00 (3864): No heartbeat from core client for 30 sec - exiting 23:50:01 (3864): No heartbeat from core client for 30 sec - exiting 23:50:02 (3864): No heartbeat from core client for 30 sec - exiting 23:50:03 (3864): No heartbeat from core client for 30 sec - exiting 23:50:04 (3864): No heartbeat from core client for 30 sec - exiting 23:50:05 (3864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:39:00 (4124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:05:18 (4536): No heartbeat from core client for 30 sec - exiting 10:05:20 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:05:21 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:33:16 (5428): No heartbeat from core client for 30 sec - exiting 12:33:17 (5428): No heartbeat from core client for 30 sec - exiting 12:33:18 (5428): No heartbeat from core client for 30 sec - exiting 12:33:19 (5428): No heartbeat from core client for 30 sec - exiting 12:33:20 (5428): No heartbeat from core client for 30 sec - exiting 12:33:21 (5428): No heartbeat from core client for 30 sec - exiting 12:33:22 (5428): No heartbeat from core client for 30 sec - exiting 12:33:23 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:33:24 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:29:19 (3528): No heartbeat from core client for 30 sec - exiting 14:29:20 (3528): No heartbeat from core client for 30 sec - exiting 14:29:21 (3528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:00:56 (2192): No heartbeat from core client for 30 sec - exiting 20:00:57 (2192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:17:54 (4160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:56:46 (4604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:12:51 (3084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:18:00 (6752): No heartbeat from core client for 30 sec - exiting 14:18:01 (6752): No heartbeat from core client for 30 sec - exiting 14:18:02 (6752): No heartbeat from core client for 30 sec - exiting 14:18:03 (6752): No heartbeat from core client for 30 sec - exiting 14:18:04 (6752): No heartbeat from core client for 30 sec - exiting 14:18:05 (6752): No heartbeat from core client for 30 sec - exiting 14:18:06 (6752): No heartbeat from core client for 30 sec - exiting 14:18:07 (6752): No heartbeat from core client for 30 sec - exiting 14:18:08 (6752): No heartbeat from core client for 30 sec - exiting 14:18:09 (6752): No heartbeat from core client for 30 sec - exiting 14:18:10 (6752): No heartbeat from core client for 30 sec - exiting 14:18:11 (6752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:12:02 (3172): No heartbeat from core client for 30 sec - exiting 09:12:03 (3172): No heartbeat from core client for 30 sec - exiting 09:12:04 (3172): No heartbeat from core client for 30 sec - exiting 09:12:05 (3172): No heartbeat from core client for 30 sec - exiting 09:12:06 (3172): No heartbeat from core client for 30 sec - exiting 09:12:07 (3172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:12:08 (3172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:28:01 (1316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:04:46 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:53:14 (4404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:33:57 (4824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:24:34 (2992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:19:42 (2948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:57:01 (576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:00:31 (2372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:00:33 (2372): No heartbeat from core client for 30 sec - exiting 12:09:02 (4504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:47:50 (3504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:48:03 (3504): No heartbeat from core client for 30 sec - exiting 13:48:04 (3504): No heartbeat from core client for 30 sec - exiting 13:49:35 (3184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:49:38 (3184): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4820, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4820, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4820, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4820, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1712, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1712, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 10:16:39 (4504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:39:36 (4904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2013 16:36:14	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	388,800	575,923	1.4813
04 Nov 2013 14:36:15	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	362,880	537,922	1.4824
03 Nov 2013 12:33:56	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	336,960	499,306	1.4818
30 Oct 2013 07:42:20	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	311,040	461,504	1.4837
23 Oct 2013 10:02:01	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	285,120	423,185	1.4842
22 Oct 2013 19:10:40	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	259,200	384,223	1.4823
22 Oct 2013 07:22:31	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	233,280	346,391	1.4849
20 Oct 2013 11:50:47	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	207,360	308,456	1.4875
18 Oct 2013 12:36:57	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	181,440	270,889	1.4930
16 Oct 2013 14:54:49	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	155,520	232,143	1.4927
07 Oct 2013 17:33:46	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	129,600	193,206	1.4908
25 Sep 2013 09:06:10	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	103,680	154,284	1.4881
18 Sep 2013 14:34:54	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	77,760	114,676	1.4747
17 Sep 2013 14:27:54	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	51,840	75,978	1.4656
13 Sep 2013 14:42:25	1289120	15992015	hadcm3n_805l_1980_40_008458140_0	25,920	37,772	1.4573