Task 15521215

Name	hadcm3n_o0e5_2100_40_008255610_2
Workunit	8410734
Created	2 Jan 2013, 8:24:42 UTC
Sent	2 Jan 2013, 8:24:47 UTC
Report deadline	3 Apr 2013, 15:51:58 UTC
Received	15 Jan 2013, 20:11:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1172912
Run time	7 days 0 hours 40 min 44 sec
CPU time	6 days 14 hours 20 min 39 sec
Validate state	Invalid
Credit	5,909.76
Device peak FLOPS	3.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3664, iMonCtr=1 Model crash detected, will try to restart... 20:24:01 (3740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:48:52 (2332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:07:56 (5688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:10:33 (5344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:44:24 (892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:45:32 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:59:13 (5364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:30:21 (5360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:31:16 (5436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5572, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5572, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5572, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5572, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5572, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5572, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Jan 2013 08:38:45	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	492,480	557,631	1.1323
13 Jan 2013 08:34:24	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	466,560	528,156	1.1320
12 Jan 2013 23:58:00	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	440,640	498,446	1.1312
12 Jan 2013 05:43:47	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	414,720	469,474	1.1320
11 Jan 2013 20:46:02	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	388,800	440,631	1.1333
11 Jan 2013 02:23:13	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	362,880	411,601	1.1343
10 Jan 2013 08:13:33	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	336,960	383,021	1.1367
09 Jan 2013 19:59:25	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	311,040	354,030	1.1382
08 Jan 2013 23:17:59	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	285,120	325,179	1.1405
08 Jan 2013 05:56:26	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	259,200	296,123	1.1424
07 Jan 2013 21:18:49	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	233,280	266,711	1.1433
06 Jan 2013 23:44:04	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	207,360	238,204	1.1487
06 Jan 2013 07:00:45	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	181,440	208,803	1.1508
05 Jan 2013 21:22:23	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	155,520	179,111	1.1517
05 Jan 2013 05:25:07	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	129,600	149,274	1.1518
04 Jan 2013 20:15:05	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	103,680	119,293	1.1506
04 Jan 2013 05:54:43	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	77,760	89,826	1.1552
03 Jan 2013 20:10:33	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	51,840	60,134	1.1600
03 Jan 2013 00:29:06	1172912	15521215	hadcm3n_o0e5_2100_40_008255610_2	25,920	29,948	1.1554