Task 15915654

Name	hadcm3n_4a81_1980_40_008364000_2
Workunit	8514859
Created	14 Aug 2013, 11:40:11 UTC
Sent	14 Aug 2013, 21:03:11 UTC
Report deadline	14 Nov 2013, 4:30:22 UTC
Received	28 Aug 2013, 11:25:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1286666
Run time	9 days 20 hours 24 min 15 sec
CPU time	9 days 7 hours 36 min 54 sec
Validate state	Invalid
Credit	4,354.56
Device peak FLOPS	2.23 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:16:48 (18232): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:16:49 (18232): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:04:42 (1260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11504, selfPID=11504, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:04:53 (7492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:19:34 (1448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:29:25 (9304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:05:23 (6616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9304, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9304, iMonCtr=1 Model crash detected, will try to restart... 03:06:36 (9304): No heartbeat from core client for 30 sec - exiting 03:06:37 (9304): No heartbeat from core client for 30 sec - exiting 03:06:38 (9304): No heartbeat from core client for 30 sec - exiting 03:06:39 (9304): No heartbeat from core client for 30 sec - exiting 03:06:40 (9304): No heartbeat from core client for 30 sec - exiting 03:06:41 (9304): No heartbeat from core client for 30 sec - exiting 03:06:42 (9304): No heartbeat from core client for 30 sec - exiting 03:06:43 (9304): No heartbeat from core client for 30 sec - exiting 03:06:44 (9304): No heartbeat from core client for 30 sec - exiting 03:06:45 (9304): No heartbeat from core client for 30 sec - exiting 03:06:46 (9304): No heartbeat from core client for 30 sec - exiting 03:06:47 (9304): No heartbeat from core client for 30 sec - exiting 03:06:48 (9304): No heartbeat from core client for 30 sec - exiting 03:06:49 (9304): No heartbeat from core client for 30 sec - exiting 03:06:50 (9304): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish 03:06:51 (9304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:06:52 (9304): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12456, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12456, iMonCtr=1 Model crash detected, will try to restart... 03:07:32 (12456): No heartbeat from core client for 30 sec - exiting 03:07:33 (12456): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17232, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17232, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Aug 2013 23:55:19	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	362,880	775,726	2.1377
27 Aug 2013 00:29:01	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	336,960	720,017	2.1368
25 Aug 2013 21:55:49	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	311,040	665,043	2.1381
25 Aug 2013 05:17:25	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	285,120	610,010	2.1395
24 Aug 2013 12:39:08	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	259,200	555,155	2.1418
23 Aug 2013 10:43:30	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	233,280	500,221	2.1443
22 Aug 2013 09:54:53	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	207,360	444,843	2.1453
21 Aug 2013 06:16:11	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	181,440	389,613	2.1473
20 Aug 2013 03:26:50	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	155,520	334,289	2.1495
19 Aug 2013 03:33:37	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	129,600	278,928	2.1522
18 Aug 2013 12:05:32	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	103,680	223,549	2.1561
17 Aug 2013 19:41:48	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	77,760	168,361	2.1651
17 Aug 2013 02:43:35	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	51,840	112,112	2.1627
16 Aug 2013 01:32:51	1286666	15915654	hadcm3n_4a81_1980_40_008364000_2	25,920	56,249	2.1701