Task 16648260

Name	hadcm3n_87ym_1980_40_008719737_2
Workunit	8865715
Created	24 May 2014, 16:56:48 UTC
Sent	24 May 2014, 16:57:21 UTC
Report deadline	24 Aug 2014, 0:24:32 UTC
Received	10 Jun 2014, 5:33:51 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1315802
Run time	3 days 16 hours 42 min 38 sec
CPU time	3 days 9 hours 31 min 35 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.18 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> 07:29:47 (4072): No heartbeat from core client for 30 sec - exiting 07:29:48 (4072): No heartbeat from core client for 30 sec - exiting 07:29:49 (4072): No heartbeat from core client for 30 sec - exiting 07:29:50 (4072): No heartbeat from core client for 30 sec - exiting 07:29:51 (4072): No heartbeat from core client for 30 sec - exiting 07:29:52 (4072): No heartbeat from core client for 30 sec - exiting 07:29:53 (4072): No heartbeat from core client for 30 sec - exiting 07:29:54 (4072): No heartbeat from core client for 30 sec - exiting 07:29:55 (4072): No heartbeat from core client for 30 sec - exiting 07:29:56 (4072): No heartbeat from core client for 30 sec - exiting 07:29:57 (4072): No heartbeat from core client for 30 sec - exiting 07:29:58 (4072): No heartbeat from core client for 30 sec - exiting 07:29:59 (4072): No heartbeat from core client for 30 sec - exiting 07:30:00 (4072): No heartbeat from core client for 30 sec - exiting 07:30:01 (4072): No heartbeat from core client for 30 sec - exiting 07:30:02 (4072): No heartbeat from core client for 30 sec - exiting 07:30:03 (4072): No heartbeat from core client for 30 sec - exiting 07:30:04 (4072): No heartbeat from core client for 30 sec - exiting 07:30:05 (4072): No heartbeat from core client for 30 sec - exiting 07:30:06 (4072): No heartbeat from core client for 30 sec - exiting 07:30:07 (4072): No heartbeat from core client for 30 sec - exiting 07:30:08 (4072): No heartbeat from core client for 30 sec - exiting 07:30:09 (4072): No heartbeat from core client for 30 sec - exiting 07:30:10 (4072): No heartbeat from core client for 30 sec - exiting 07:30:11 (4072): No heartbeat from core client for 30 sec - exiting 07:30:12 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:49:48 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:38:36 (4860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:23:25 (4064): No heartbeat from core client for 30 sec - exiting 21:23:26 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... C21:07:49 (4352): No heartbeat from core client for 30 sec - exiting 21:07:50 (4352): No heartbeat from core client for 30 sec - exiting 21:07:51 (4352): No heartbeat from core client for 30 sec - exiting 21:07:52 (4352): No heartbeat from core client for 30 sec - exiting 21:07:53 (4352): No heartbeat from core client for 30 sec - exiting 21:07:54 (4352): No heartbeat from core client for 30 sec - exiting 21:07:55 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:46:50 (1388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4516, iMonCtr=1 Model crash detected, will try to restart... 21:48:23 (3848): No heartbeat from core client for 30 sec - exiting 21:48:24 (3848): No heartbeat from core client for 30 sec - exiting 21:48:25 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:48:26 (3848): No heartbeat from core client for 30 sec - exiting 21:48:27 (3848): No heartbeat from core client for 30 sec - exiting 21:48:28 (3848): No heartbeat from core client for 30 sec - exiting 21:48:29 (3848): No heartbeat from core client for 30 sec - exiting 21:48:30 (3848): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:44:35 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:02:05 (2820): No heartbeat from core client for 30 sec - exiting 08:02:06 (2820): No heartbeat from core client for 30 sec - exiting 08:02:07 (2820): No heartbeat from core client for 30 sec - exiting 08:02:08 (2820): No heartbeat from core client for 30 sec - exiting 08:02:09 (2820): No heartbeat from core client for 30 sec - exiting 08:02:10 (2820): No heartbeat from core client for 30 sec - exiting 08:02:11 (2820): No heartbeat from core client for 30 sec - exiting 08:02:12 (2820): No heartbeat from core client for 30 sec - exiting 08:02:13 (2820): No heartbeat from core client for 30 sec - exiting 08:02:14 (2820): No heartbeat from core client for 30 sec - exiting 08:02:15 (2820): No heartbeat from core client for 30 sec - exiting 08:02:16 (2820): No heartbeat from core client for 30 sec - exiting 08:02:17 (2820): No heartbeat from core client for 30 sec - exiting 08:02:18 (2820): No heartbeat from core client for 30 sec - exiting 08:02:19 (2820): No heartbeat from core client for 30 sec - exiting 08:02:20 (2820): No heartbeat from core client for 30 sec - exiting 08:02:21 (2820): No heartbeat from core client for 30 sec - exiting 08:02:22 (2820): No heartbeat from core client for 30 sec - exiting 08:02:23 (2820): No heartbeat from core client for 30 sec - exiting 08:02:24 (2820): No heartbeat from core client for 30 sec - exiting 08:02:25 (2820): No heartbeat from core client for 30 sec - exiting 08:02:26 (2820): No heartbeat from core client for 30 sec - exiting 08:02:27 (2820): No heartbeat from core client for 30 sec - exiting 08:02:28 (2820): No heartbeat from core client for 30 sec - exiting 08:02:29 (2820): No heartbeat from core client for 30 sec - exiting 08:02:30 (2820): No heartbeat from core client for 30 sec - exiting 08:02:31 (2820): No heartbeat from core client for 30 sec - exiting 08:02:32 (2820): No heartbeat from core client for 30 sec - exiting 08:02:33 (2820): No heartbeat from core client for 30 sec - exiting 08:02:34 (2820): No heartbeat from core client for 30 sec - exiting 08:02:35 (2820): No heartbeat from core client for 30 sec - exiting 08:02:36 (2820): No heartbeat from core client for 30 sec - exiting 08:02:37 (2820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:33:40 (3844): No heartbeat from core client for 30 sec - exiting 06:33:41 (3844): No heartbeat from core client for 30 sec - exiting 06:33:42 (3844): No heartbeat from core client for 30 sec - exiting 06:33:43 (3844): No heartbeat from core client for 30 sec - exiting 06:33:44 (3844): No heartbeat from core client for 30 sec - exiting 06:33:45 (3844): No heartbeat from core client for 30 sec - exiting 06:33:46 (3844): No heartbeat from core client for 30 sec - exiting 06:33:47 (3844): No heartbeat from core client for 30 sec - exiting 06:33:48 (3844): No heartbeat from core client for 30 sec - exiting 06:33:49 (3844): No heartbeat from core client for 30 sec - exiting 06:33:50 (3844): No heartbeat from core client for 30 sec - exiting 06:33:51 (3844): No heartbeat from core client for 30 sec - exiting 06:33:52 (3844): No heartbeat from core client for 30 sec - exiting 06:33:53 (3844): No heartbeat from core client for 30 sec - exiting 06:33:54 (3844): No heartbeat from core client for 30 sec - exiting 06:33:55 (3844): No heartbeat from core client for 30 sec - exiting 06:33:56 (3844): No heartbeat from core client for 30 sec - exiting 06:33:57 (3844): No heartbeat from core client for 30 sec - exiting 06:33:58 (3844): No heartbeat from core client for 30 sec - exiting 06:33:59 (3844): No heartbeat from core client for 30 sec - exiting 06:34:00 (3844): No heartbeat from core client for 30 sec - exiting 06:34:01 (3844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:38:12 (4432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:23:27 (3524): No heartbeat from core client for 30 sec - exiting 17:23:28 (3524): No heartbeat from core client for 30 sec - exiting 17:23:29 (3524): No heartbeat from core client for 30 sec - exiting 17:23:30 (3524): No heartbeat from core client for 30 sec - exiting 17:23:31 (3524): No heartbeat from core client for 30 sec - exiting 17:23:32 (3524): No heartbeat from core client for 30 sec - exiting 17:23:33 (3524): No heartbeat from core client for 30 sec - exiting 17:23:34 (3524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:35 (3524): No heartbeat from core client for 30 sec - exiting 17:23:36 (3524): No heartbeat from core client for 30 sec - exiting 17:23:37 (3524): No heartbeat from core client for 30 sec - exiting 17:23:38 (3524): No heartbeat from core client for 30 sec - exiting 17:23:39 (3524): No heartbeat from core client for 30 sec - exiting 17:23:40 (3524): No heartbeat from core client for 30 sec - exiting 17:23:41 (3524): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4576, iMonCtr=1 Model crash detected, will try to restart... 06:48:05 (4068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:19:41 (3320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:56:24 (4412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:56:25 (4412): No heartbeat from core client for 30 sec - exiting 16:56:26 (4412): No heartbeat from core client for 30 sec - exiting 16:56:27 (4412): No heartbeat from core client for 30 sec - exiting 16:56:28 (4412): No heartbeat from core client for 30 sec - exiting 16:56:29 (4412): No heartbeat from core client for 30 sec - exiting 16:56:30 (4412): No heartbeat from core client for 30 sec - exiting 16:56:31 (4412): No heartbeat from core client for 30 sec - exiting 16:56:32 (4412): No heartbeat from core client for 30 sec - exiting 16:56:33 (4412): No heartbeat from core client for 30 sec - exiting 16:56:34 (4412): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:08:08 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:24:56 (4700): No heartbeat from core client for 30 sec - exiting 18:24:57 (4700): No heartbeat from core client for 30 sec - exiting 18:24:58 (4700): No heartbeat from core client for 30 sec - exiting 18:24:59 (4700): No heartbeat from core client for 30 sec - exiting 18:25:00 (4700): No heartbeat from core client for 30 sec - exiting 18:25:01 (4700): No heartbeat from core client for 30 sec - exiting 18:25:02 (4700): No heartbeat from core client for 30 sec - exiting 18:25:03 (4700): No heartbeat from core client for 30 sec - exiting 18:25:04 (4700): No heartbeat from core client for 30 sec - exiting 18:25:05 (4700): No heartbeat from core client for 30 sec - exiting 18:25:06 (4700): No heartbeat from core client for 30 sec - exiting 18:25:07 (4700): No heartbeat from core client for 30 sec - exiting 18:25:08 (4700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:25:09 (4700): No heartbeat from core client for 30 sec - exiting 18:25:10 (4700): No heartbeat from core client for 30 sec - exiting 18:25:11 (4700): No heartbeat from core client for 30 sec - exiting 18:25:12 (4700): No heartbeat from core client for 30 sec - exiting 18:25:13 (4700): No heartbeat from core client for 30 sec - exiting 18:25:14 (4700): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:59:44 (3844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:36:18 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3588, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Jun 2014 09:03:53	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	259,200	293,493	1.1323
06 Jun 2014 17:00:40	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	233,280	263,893	1.1312
03 Jun 2014 18:58:31	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	207,360	234,354	1.1302
01 Jun 2014 12:49:00	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	181,440	204,988	1.1298
31 May 2014 17:54:43	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	155,520	175,679	1.1296
31 May 2014 09:01:30	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	129,600	146,237	1.1284
30 May 2014 12:17:46	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	103,680	116,687	1.1255
26 May 2014 19:34:50	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	77,760	87,631	1.1269
26 May 2014 10:48:24	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	51,840	57,594	1.1110
25 May 2014 11:32:02	1315802	16648260	hadcm3n_87ym_1980_40_008719737_2	25,920	28,577	1.1025