Task 17356599

Name	hadcm3n_xc50_1940_40_009152026_0
Workunit	9282362
Created	6 Nov 2014, 15:27:38 UTC
Sent	7 Nov 2014, 2:22:33 UTC
Report deadline	6 Feb 2015, 9:49:44 UTC
Received	19 Nov 2014, 20:53:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1308965
Run time	12 days 6 hours 2 min 37 sec
CPU time	11 days 13 hours 51 min 3 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	3.58 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:16:14 (8228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:27:55 (1492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:31 (1492): No heartbeat from core client for 30 sec - exiting 16:29:12 (1492): No heartbeat from core client for 30 sec - exiting 16:29:13 (1492): No heartbeat from core client for 30 sec - exiting 16:29:14 (1492): No heartbeat from core client for 30 sec - exiting 16:29:15 (1492): No heartbeat from core client for 30 sec - exiting 16:29:16 (1492): No heartbeat from core client for 30 sec - exiting 16:29:17 (1492): No heartbeat from core client for 30 sec - exiting 16:29:18 (1492): No heartbeat from core client for 30 sec - exiting 16:29:19 (1492): No heartbeat from core client for 30 sec - exiting 16:29:20 (1492): No heartbeat from core client for 30 sec - exiting 16:29:21 (1492): No heartbeat from core client for 30 sec - exiting 16:29:22 (1492): No heartbeat from core client for 30 sec - exiting 16:29:23 (1492): No heartbeat from core client for 30 sec - exiting 16:29:24 (1492): No heartbeat from core client for 30 sec - exiting 16:29:25 (1492): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:28:09 (7512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:40 (7512): No heartbeat from core client for 30 sec - exiting 16:29:21 (7512): No heartbeat from core client for 30 sec - exiting 16:29:22 (7512): No heartbeat from core client for 30 sec - exiting 16:29:23 (7512): No heartbeat from core client for 30 sec - exiting 16:29:24 (7512): No heartbeat from core client for 30 sec - exiting 16:29:25 (7512): No heartbeat from core client for 30 sec - exiting 16:29:26 (7512): No heartbeat from core client for 30 sec - exiting 16:29:27 (7512): No heartbeat from core client for 30 sec - exiting 16:29:28 (7512): No heartbeat from core client for 30 sec - exiting 16:29:29 (7512): No heartbeat from core client for 30 sec - exiting 16:29:30 (7512): No heartbeat from core client for 30 sec - exiting 16:29:31 (7512): No heartbeat from core client for 30 sec - exiting 16:29:32 (7512): No heartbeat from core client for 30 sec - exiting 16:29:33 (7512): No heartbeat from core client for 30 sec - exiting 16:29:34 (7512): No heartbeat from core client for 30 sec - exiting 16:29:35 (7512): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6496, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6496, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6496, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6496, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6496, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6496, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Nov 2014 12:58:47	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	855,360	984,607	1.1511
19 Nov 2014 03:09:04	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	829,440	953,559	1.1496
18 Nov 2014 18:30:56	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	803,520	921,991	1.1474
18 Nov 2014 09:16:54	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	777,600	890,810	1.1456
18 Nov 2014 00:38:43	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	751,680	860,114	1.1443
17 Nov 2014 15:50:32	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	725,760	829,490	1.1429
17 Nov 2014 06:45:47	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	699,840	798,135	1.1405
16 Nov 2014 21:47:10	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	673,920	767,355	1.1386
16 Nov 2014 12:53:03	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	648,000	736,134	1.1360
16 Nov 2014 03:59:11	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	622,080	705,975	1.1349
15 Nov 2014 19:26:08	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	596,160	677,169	1.1359
15 Nov 2014 10:37:40	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	570,240	647,949	1.1363
15 Nov 2014 01:49:35	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	544,320	617,730	1.1349
14 Nov 2014 17:52:22	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	518,400	587,296	1.1329
14 Nov 2014 07:49:28	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	492,480	555,611	1.1282
13 Nov 2014 21:43:09	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	466,560	523,914	1.1229
13 Nov 2014 11:57:18	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	440,640	493,196	1.1193
13 Nov 2014 03:05:15	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	414,720	463,552	1.1177
12 Nov 2014 17:51:17	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	388,800	433,130	1.1140
12 Nov 2014 08:42:17	1308965	17356599	hadcm3n_xc50_1940_40_009152026_0	362,880	402,708	1.1098