Task 16636373

Name	hadcm3n_87xd_1980_40_008719692_1
Workunit	8865670
Created	12 May 2014, 12:23:26 UTC
Sent	12 May 2014, 12:30:17 UTC
Report deadline	11 Aug 2014, 19:57:28 UTC
Received	30 May 2014, 19:12:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1310038
Run time	15 days 22 hours 23 min 26 sec
CPU time	14 days 7 hours 6 min 2 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.06 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:07:42 (796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:19:54 (3416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:38:35 (344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:48:21 (2988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:10:54 (5720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9056, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 06:41:11 (2716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2724, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7000, iMonCtr=1 Model crash detected, will try to restart... 00:10:58 (7000): No heartbeat from core client for 30 sec - exiting 00:10:59 (7000): No heartbeat from core client for 30 sec - exiting 00:11:00 (7000): No heartbeat from core client for 30 sec - exiting 00:11:01 (7000): No heartbeat from core client for 30 sec - exiting 00:11:02 (7000): No heartbeat from core client for 30 sec - exiting 00:11:03 (7000): No heartbeat from core client for 30 sec - exiting 00:11:04 (7000): No heartbeat from core client for 30 sec - exiting 00:11:05 (7000): No heartbeat from core client for 30 sec - exiting 00:11:06 (7000): No heartbeat from core client for 30 sec - exiting 00:11:07 (7000): No heartbeat from core client for 30 sec - exiting 00:11:08 (7000): No heartbeat from core client for 30 sec - exiting 00:11:09 (7000): No heartbeat from core client for 30 sec - exiting 00:11:10 (7000): No heartbeat from core client for 30 sec - exiting 00:11:11 (7000): No heartbeat from core client for 30 sec - exiting 00:11:12 (7000): No heartbeat from core client for 30 sec - exiting 00:11:13 (7000): No heartbeat from core client for 30 sec - exiting 00:11:14 (7000): No heartbeat from core client for 30 sec - exiting 00:11:15 (7000): No heartbeat from core client for 30 sec - exiting 00:11:16 (7000): No heartbeat from core client for 30 sec - exiting 00:11:17 (7000): No heartbeat from core client for 30 sec - exiting 00:11:18 (7000): No heartbeat from core client for 30 sec - exiting 00:11:19 (7000): No heartbeat from core client for 30 sec - exiting 00:11:20 (7000): No heartbeat from core client for 30 sec - exiting 00:11:21 (7000): No heartbeat from core client for 30 sec - exiting 00:11:22 (7000): No heartbeat from core client for 30 sec - exiting 00:11:23 (7000): No heartbeat from core client for 30 sec - exiting 00:11:24 (7000): No heartbeat from core client for 30 sec - exiting 00:11:25 (7000): No heartbeat from core client for 30 sec - exiting 00:11:26 (7000): No heartbeat from core client for 30 sec - exiting 00:11:27 (7000): No heartbeat from core client for 30 sec - exiting 00:11:28 (7000): No heartbeat from core client for 30 sec - exiting 00:11:29 (7000): No heartbeat from core client for 30 sec - exiting 00:11:30 (7000): No heartbeat from core client for 30 sec - exiting 00:11:31 (7000): No heartbeat from core client for 30 sec - exiting 00:11:32 (7000): No heartbeat from core client for 30 sec - exiting 00:11:33 (7000): No heartbeat from core client for 30 sec - exiting 00:11:34 (7000): No heartbeat from core client for 30 sec - exiting 00:11:35 (7000): No heartbeat from core client for 30 sec - exiting 00:11:36 (7000): No heartbeat from core client for 30 sec - exiting 00:11:37 (7000): No heartbeat from core client for 30 sec - exiting 00:11:38 (7000): No heartbeat from core client for 30 sec - exiting 00:11:39 (7000): No heartbeat from core client for 30 sec - exiting 00:11:40 (7000): No heartbeat from core client for 30 sec - exiting 00:11:41 (7000): No heartbeat from core client for 30 sec - exiting 00:11:42 (7000): No heartbeat from core client for 30 sec - exiting 00:11:43 (7000): No heartbeat from core client for 30 sec - exiting 00:11:44 (7000): No heartbeat from core client for 30 sec - exiting 00:11:45 (7000): No heartbeat from core client for 30 sec - exiting 00:11:46 (7000): No heartbeat from core client for 30 sec - exiting 00:11:47 (7000): No heartbeat from core client for 30 sec - exiting 00:11:48 (7000): No heartbeat from core client for 30 sec - exiting 00:11:49 (7000): No heartbeat from core client for 30 sec - exiting 00:11:50 (7000): No heartbeat from core client for 30 sec - exiting 00:11:51 (7000): No heartbeat from core client for 30 sec - exiting 00:11:52 (7000): No heartbeat from core client for 30 sec - exiting 00:11:53 (7000): No heartbeat from core client for 30 sec - exiting 00:11:54 (7000): No heartbeat from core client for 30 sec - exiting 00:11:55 (7000): No heartbeat from core client for 30 sec - exiting 00:11:56 (7000): No heartbeat from core client for 30 sec - exiting 00:11:57 (7000): No heartbeat from core client for 30 sec - exiting 00:11:58 (7000): No heartbeat from core client for 30 sec - exiting 00:11:59 (7000): No heartbeat from core client for 30 sec - exiting 00:12:00 (7000): No heartbeat from core client for 30 sec - exiting 00:12:01 (7000): No heartbeat from core client for 30 sec - exiting 00:12:02 (7000): No heartbeat from core client for 30 sec - exiting 00:12:03 (7000): No heartbeat from core client for 30 sec - exiting 00:12:04 (7000): No heartbeat from core client for 30 sec - exiting 00:12:05 (7000): No heartbeat from core client for 30 sec - exiting 00:12:06 (7000): No heartbeat from core client for 30 sec - exiting 00:12:07 (7000): No heartbeat from core client for 30 sec - exiting 00:12:08 (7000): No heartbeat from core client for 30 sec - exiting 00:12:09 (7000): No heartbeat from core client for 30 sec - exiting 00:12:10 (7000): No heartbeat from core client for 30 sec - exiting 00:12:11 (7000): No heartbeat from core client for 30 sec - exiting 00:12:12 (7000): No heartbeat from core client for 30 sec - exiting 00:12:13 (7000): No heartbeat from core client for 30 sec - exiting 00:12:14 (7000): No heartbeat from core client for 30 sec - exiting 00:12:15 (7000): No heartbeat from core client for 30 sec - exiting 00:12:16 (7000): No heartbeat from core client for 30 sec - exiting 00:12:17 (7000): No heartbeat from core client for 30 sec - exiting 00:12:18 (7000): No heartbeat from core client for 30 sec - exiting 00:12:19 (7000): No heartbeat from core client for 30 sec - exiting 00:12:20 (7000): No heartbeat from core client for 30 sec - exiting 00:12:21 (7000): No heartbeat from core client for 30 sec - exiting 00:12:22 (7000): No heartbeat from core client for 30 sec - exiting 00:12:23 (7000): No heartbeat from core client for 30 sec - exiting 00:12:24 (7000): No heartbeat from core client for 30 sec - exiting 00:12:25 (7000): No heartbeat from core client for 30 sec - exiting 00:12:26 (7000): No heartbeat from core client for 30 sec - exiting 00:12:27 (7000): No heartbeat from core client for 30 sec - exiting 00:12:28 (7000): No heartbeat from core client for 30 sec - exiting 00:12:29 (7000): No heartbeat from core client for 30 sec - exiting 00:12:30 (7000): No heartbeat from core client for 30 sec - exiting 00:12:31 (7000): No heartbeat from core client for 30 sec - exiting 00:12:32 (7000): No heartbeat from core client for 30 sec - exiting 00:12:33 (7000): No heartbeat from core client for 30 sec - exiting 00:12:34 (7000): No heartbeat from core client for 30 sec - exiting 00:12:35 (7000): No heartbeat from core client for 30 sec - exiting 00:12:36 (7000): No heartbeat from core client for 30 sec - exiting 00:12:37 (7000): No heartbeat from core client for 30 sec - exiting 00:12:38 (7000): No heartbeat from core client for 30 sec - exiting 00:12:39 (7000): No heartbeat from core client for 30 sec - exiting 00:12:40 (7000): No heartbeat from core client for 30 sec - exiting 00:12:41 (7000): No heartbeat from core client for 30 sec - exiting 00:12:42 (7000): No heartbeat from core client for 30 sec - exiting 00:12:43 (7000): No heartbeat from core client for 30 sec - exiting 00:12:44 (7000): No heartbeat from core client for 30 sec - exiting 00:12:45 (7000): No heartbeat from core client for 30 sec - exiting 00:12:46 (7000): No heartbeat from core client for 30 sec - exiting 00:12:47 (7000): No heartbeat from core client for 30 sec - exiting 00:12:48 (7000): No heartbeat from core client for 30 sec - exiting 00:12:49 (7000): No heartbeat from core client for 30 sec - exiting 00:12:50 (7000): No heartbeat from core client for 30 sec - exiting 00:12:51 (7000): No heartbeat from core client for 30 sec - exiting 00:12:52 (7000): No heartbeat from core client for 30 sec - exiting 00:12:53 (7000): No heartbeat from core client for 30 sec - exiting 00:12:54 (7000): No heartbeat from core client for 30 sec - exiting 00:12:55 (7000): No heartbeat from core client for 30 sec - exiting 00:12:56 (7000): No heartbeat from core client for 30 sec - exiting 00:12:57 (7000): No heartbeat from core client for 30 sec - exiting 00:12:58 (7000): No heartbeat from core client for 30 sec - exiting 00:12:59 (7000): No heartbeat from core client for 30 sec - exiting 00:13:00 (7000): No heartbeat from core client for 30 sec - exiting 00:13:01 (7000): No heartbeat from core client for 30 sec - exiting 00:13:02 (7000): No heartbeat from core client for 30 sec - exiting 00:13:03 (7000): No heartbeat from core client for 30 sec - exiting 00:13:05 (7000): No heartbeat from core client for 30 sec - exiting 00:13:06 (7000): No heartbeat from core client for 30 sec - exiting 00:13:07 (7000): No heartbeat from core client for 30 sec - exiting 00:13:08 (7000): No heartbeat from core client for 30 sec - exiting 00:13:09 (7000): No heartbeat from core client for 30 sec - exiting 00:13:10 (7000): No heartbeat from core client for 30 sec - exiting 00:13:11 (7000): No heartbeat from core client for 30 sec - exiting 00:13:12 (7000): No heartbeat from core client for 30 sec - exiting 00:13:13 (7000): No heartbeat from core client for 30 sec - exiting 00:13:14 (7000): No heartbeat from core client for 30 sec - exiting 00:13:15 (7000): No heartbeat from core client for 30 sec - exiting 00:13:16 (7000): No heartbeat from core client for 30 sec - exiting 00:13:17 (7000): No heartbeat from core client for 30 sec - exiting 00:13:18 (7000): No heartbeat from core client for 30 sec - exiting 00:13:19 (7000): No heartbeat from core client for 30 sec - exiting 00:13:20 (7000): No heartbeat from core client for 30 sec - exiting 00:13:21 (7000): No heartbeat from core client for 30 sec - exiting 00:13:22 (7000): No heartbeat from core client for 30 sec - exiting 00:13:23 (7000): No heartbeat from core client for 30 sec - exiting 00:13:24 (7000): No heartbeat from core client for 30 sec - exiting 00:13:25 (7000): No heartbeat from core client for 30 sec - exiting 00:13:26 (7000): No heartbeat from core client for 30 sec - exiting 00:13:27 (7000): No heartbeat from core client for 30 sec - exiting 00:13:28 (7000): No heartbeat from core client for 30 sec - exiting 00:13:29 (7000): No heartbeat from core client for 30 sec - exiting 00:13:30 (7000): No heartbeat from core client for 30 sec - exiting 00:13:31 (7000): No heartbeat from core client for 30 sec - exiting 00:13:32 (7000): No heartbeat from core client for 30 sec - exiting 00:13:33 (7000): No heartbeat from core client for 30 sec - exiting 00:13:34 (7000): No heartbeat from core client for 30 sec - exiting 00:13:35 (7000): No heartbeat from core client for 30 sec - exiting 00:13:36 (7000): No heartbeat from core client for 30 sec - exiting 00:13:37 (7000): No heartbeat from core client for 30 sec - exiting 00:13:38 (7000): No heartbeat from core client for 30 sec - exiting 00:13:39 (7000): No heartbeat from core client for 30 sec - exiting 00:13:40 (7000): No heartbeat from core client for 30 sec - exiting 00:13:41 (7000): No heartbeat from core client for 30 sec - exiting 00:13:42 (7000): No heartbeat from core client for 30 sec - exiting 00:13:43 (7000): No heartbeat from core client for 30 sec - exiting 00:13:44 (7000): No heartbeat from core client for 30 sec - exiting 00:13:45 (7000): No heartbeat from core client for 30 sec - exiting 00:13:46 (7000): No heartbeat from core client for 30 sec - exiting 00:13:47 (7000): No heartbeat from core client for 30 sec - exiting 00:13:48 (7000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:42:28 (740): No heartbeat from core client for 30 sec - exiting 11:42:30 (740): No heartbeat from core client for 30 sec - exiting 11:42:31 (740): No heartbeat from core client for 30 sec - exiting 11:42:32 (740): No heartbeat from core client for 30 sec - exiting 11:42:33 (740): No heartbeat from core client for 30 sec - exiting 11:42:34 (740): No heartbeat from core client for 30 sec - exiting 11:42:35 (740): No heartbeat from core client for 30 sec - exiting 11:42:36 (740): No heartbeat from core client for 30 sec - exiting 11:42:37 (740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6692, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:00:35 (7684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:40:13 (8364): No heartbeat from core client for 30 sec - exiting 07:40:15 (8364): No heartbeat from core client for 30 sec - exiting 07:40:16 (8364): No heartbeat from core client for 30 sec - exiting 07:40:17 (8364): No heartbeat from core client for 30 sec - exiting 07:40:18 (8364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:48:42 (7368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:43 (7368): No heartbeat from core client for 30 sec - exiting 08:48:44 (7368): No heartbeat from core client for 30 sec - exiting 08:55:39 (1964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:55:41 (1964): No heartbeat from core client for 30 sec - exiting 09:01:29 (4724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:01:30 (4724): No heartbeat from core client for 30 sec - exiting 09:11:27 (6320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:35:32 (4076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:35:34 (4076): No heartbeat from core client for 30 sec - exiting 09:35:35 (4076): No heartbeat from core client for 30 sec - exiting 09:35:36 (4076): No heartbeat from core client for 30 sec - exiting 09:56:40 (9908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:07:28 (11360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:35:35 (4248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:36:50 (8064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:39:31 (1856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:40:43 (10924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:41:25 (10464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:25 (9712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:27 (9712): No heartbeat from core client for 30 sec - exiting 10:45:28 (9712): No heartbeat from core client for 30 sec - exiting 10:45:29 (9712): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 May 2014 18:12:50	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	518,400	1,235,079	2.3825
29 May 2014 17:48:31	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	492,480	1,173,204	2.3822
28 May 2014 21:50:31	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	466,560	1,110,411	2.3800
28 May 2014 00:19:49	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	440,640	1,047,450	2.3771
27 May 2014 02:55:00	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	414,720	984,664	2.3743
26 May 2014 04:42:16	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	388,800	922,302	2.3722
25 May 2014 09:36:42	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	362,880	859,954	2.3698
24 May 2014 15:15:36	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	336,960	797,157	2.3657
23 May 2014 20:43:53	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	311,040	734,365	2.3610
23 May 2014 01:14:02	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	285,120	670,510	2.3517
22 May 2014 06:29:37	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	259,200	609,442	2.3512
21 May 2014 10:36:29	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	233,280	547,448	2.3467
20 May 2014 15:32:27	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	207,360	484,799	2.3380
19 May 2014 20:53:52	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	181,440	422,598	2.3291
19 May 2014 03:07:07	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	155,520	361,614	2.3252
18 May 2014 10:06:57	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	129,600	305,452	2.3569
17 May 2014 15:43:46	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	103,680	243,610	2.3496
16 May 2014 18:13:50	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	77,760	183,502	2.3599
14 May 2014 16:19:39	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	51,840	127,966	2.4685
13 May 2014 15:55:54	1310038	16636373	hadcm3n_87xd_1980_40_008719692_1	25,920	60,593	2.3377