Task 15591462

Name	hadcm3n_4l4m_1940_40_008305380_0
Workunit	8456515
Created	7 Feb 2013, 5:58:03 UTC
Sent	7 Feb 2013, 6:00:21 UTC
Report deadline	9 May 2013, 13:27:32 UTC
Received	15 Jun 2013, 8:25:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1204261
Run time	80 days 8 hours 14 min 32 sec
CPU time	70 days 22 hours 33 min 24 sec
Validate state	Invalid
Credit	9,642.24
Device peak FLOPS	1.08 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> 03:55:33 (5860): No heartbeat from core client for 30 sec - exiting 03:55:34 (5860): No heartbeat from core client for 30 sec - exiting 03:55:35 (5860): No heartbeat from core client for 30 sec - exiting 03:55:36 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2648, iMonCtr=1 Model crash detected, will try to restart... 01:51:21 (5576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:21:37 (9856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:34:36 (5956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6020, iMonCtr=1 Model crash detected, will try to restart... 11:01:03 (5184): No heartbeat from core client for 30 sec - exiting 11:01:04 (5184): No heartbeat from core client for 30 sec - exiting 11:01:05 (5184): No heartbeat from core client for 30 sec - exiting 11:01:06 (5184): No heartbeat from core client for 30 sec - exiting 11:01:07 (5184): No heartbeat from core client for 30 sec - exiting 11:01:08 (5184): No heartbeat from core client for 30 sec - exiting 11:01:09 (5184): No heartbeat from core client for 30 sec - exiting 11:01:10 (5184): No heartbeat from core client for 30 sec - exiting 11:01:11 (5184): No heartbeat from core client for 30 sec - exiting 11:01:12 (5184): No heartbeat from core client for 30 sec - exiting 11:01:14 (5184): No heartbeat from core client for 30 sec - exiting 11:01:15 (5184): No heartbeat from core client for 30 sec - exiting 11:01:16 (5184): No heartbeat from core client for 30 sec - exiting 11:01:17 (5184): No heartbeat from core client for 30 sec - exiting 11:01:18 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:45:49 (5060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:32:40 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:41:39 (6884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:25:26 (1368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:55:22 (6840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:53:51 (4068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:37:28 (132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:35:39 (3124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:36:40 (6088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:15 (4632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 02:19:24 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:20:21 (6076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:39:40 (6368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:31:43 (6096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:52:26 (1436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:50 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Jun 2013 01:41:08	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	803,520	6,003,728	7.4718
24 Apr 2013 13:21:36	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	777,600	5,805,859	7.4664
20 Apr 2013 00:26:02	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	751,680	5,473,971	7.2823
15 Apr 2013 17:47:49	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	725,760	5,152,136	7.0990
12 Apr 2013 06:03:25	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	699,840	4,887,448	6.9837
08 Apr 2013 22:41:09	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	673,920	4,638,253	6.8825
05 Apr 2013 19:34:52	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	648,000	4,392,321	6.7783
02 Apr 2013 20:24:27	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	622,080	4,162,914	6.6919
30 Mar 2013 14:10:20	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	596,160	3,937,998	6.6056
27 Mar 2013 22:09:25	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	570,240	3,733,097	6.5465
25 Mar 2013 03:00:49	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	544,320	3,523,575	6.4734
22 Mar 2013 15:30:29	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	518,400	3,326,375	6.4166
20 Mar 2013 16:00:37	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	492,480	3,157,991	6.4124
18 Mar 2013 15:50:34	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	466,560	2,989,369	6.4073
15 Mar 2013 17:36:38	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	440,640	2,803,541	6.3624
13 Mar 2013 04:38:52	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	414,720	2,608,821	6.2906
10 Mar 2013 20:53:03	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	388,800	2,428,392	6.2459
08 Mar 2013 14:50:28	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	362,880	2,251,151	6.2036
06 Mar 2013 10:11:07	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	336,960	2,083,146	6.1822
04 Mar 2013 03:21:57	1204261	15591462	hadcm3n_4l4m_1940_40_008305380_0	311,040	1,911,917	6.1469