Task 13631926

Name	hadcm3n_u3u9_1980_40_007544947_2
Workunit	7742179
Created	10 Nov 2011, 8:55:30 UTC
Sent	10 Nov 2011, 9:08:41 UTC
Report deadline	9 Feb 2012, 16:35:52 UTC
Received	18 Dec 2011, 0:56:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	936080
Run time	15 days 13 hours 5 min 21 sec
CPU time	15 days 6 hours 32 min 20 sec
Validate state	Invalid
Credit	11,508.48
Device peak FLOPS	3.02 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:01:13 (772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:01:15 (772): No heartbeat from core client for 30 sec - exiting 03:01:16 (772): No heartbeat from core client for 30 sec - exiting 03:01:17 (772): No heartbeat from core client for 30 sec - exiting 03:01:18 (772): No heartbeat from core client for 30 sec - exiting 03:01:19 (772): No heartbeat from core client for 30 sec - exiting 03:01:20 (772): No heartbeat from core client for 30 sec - exiting 03:01:21 (772): No heartbeat from core client for 30 sec - exiting 03:01:22 (772): No heartbeat from core client for 30 sec - exiting 03:01:23 (772): No heartbeat from core client for 30 sec - exiting 03:01:24 (772): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Dec 2011 22:50:27	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	959,040	1,318,185	1.3745
17 Dec 2011 12:29:40	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	933,120	1,281,420	1.3733
17 Dec 2011 02:07:13	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	907,200	1,244,633	1.3719
16 Dec 2011 13:13:00	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	881,280	1,209,760	1.3727
16 Dec 2011 01:54:46	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	855,360	1,173,878	1.3724
15 Dec 2011 14:06:57	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	829,440	1,137,306	1.3712
15 Dec 2011 04:03:04	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	803,520	1,101,605	1.3710
14 Dec 2011 17:25:37	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	777,600	1,064,841	1.3694
14 Dec 2011 07:07:17	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	751,680	1,028,592	1.3684
13 Dec 2011 21:39:38	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	725,760	992,764	1.3679
13 Dec 2011 08:55:54	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	699,840	955,698	1.3656
12 Dec 2011 22:21:58	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	673,920	918,412	1.3628
12 Dec 2011 10:03:43	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	648,000	881,553	1.3604
11 Dec 2011 23:30:49	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	622,080	844,309	1.3572
11 Dec 2011 09:47:01	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	596,160	808,671	1.3565
10 Dec 2011 22:28:57	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	570,240	773,228	1.3560
10 Dec 2011 08:20:59	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	544,320	738,178	1.3561
09 Dec 2011 21:08:00	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	518,400	703,175	1.3564
09 Dec 2011 00:55:45	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	492,480	668,161	1.3567
08 Dec 2011 13:59:31	936080	13631926	hadcm3n_u3u9_1980_40_007544947_2	466,560	632,368	1.3554