Task 13636664

Name	hadcm3n_o05l_1940_40_007544529_3
Workunit	7741761
Created	15 Nov 2011, 22:34:41 UTC
Sent	15 Nov 2011, 22:37:04 UTC
Report deadline	15 Feb 2012, 6:04:15 UTC
Received	14 Dec 2011, 19:57:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1035732
Run time	14 days 19 hours 1 min 19 sec
CPU time	12 days 21 hours 46 min 19 sec
Validate state	Invalid
Credit	8,709.12
Device peak FLOPS	3.14 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:54:45 (1040): Can't acquire lockfile (32) - waiting 35s 20:55:05 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:41:32 (3508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:40:46 (5540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:29:16 (4060): Can't acquire lockfile (32) - waiting 35s 20:29:21 (2352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:29:22 (2352): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5116, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5116, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish 14:49:59 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5396, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5396, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5396, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1056, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Dec 2011 00:30:28	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	725,760	1,090,521	1.5026
13 Dec 2011 10:16:56	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	699,840	1,047,726	1.4971
12 Dec 2011 16:31:00	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	673,920	1,003,669	1.4893
12 Dec 2011 04:32:18	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	648,000	963,800	1.4873
11 Dec 2011 15:28:29	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	622,080	924,301	1.4858
11 Dec 2011 02:20:17	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	596,160	883,706	1.4823
10 Dec 2011 10:11:21	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	570,240	843,696	1.4795
09 Dec 2011 20:27:45	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	544,320	805,424	1.4797
09 Dec 2011 08:52:41	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	518,400	768,007	1.4815
07 Dec 2011 16:54:42	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	492,480	730,479	1.4833
06 Dec 2011 22:01:55	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	466,560	693,177	1.4857
06 Dec 2011 09:55:19	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	440,640	655,231	1.4870
05 Dec 2011 22:06:04	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	414,720	616,819	1.4873
05 Dec 2011 06:50:28	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	388,800	578,781	1.4886
04 Dec 2011 17:32:12	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	362,880	539,384	1.4864
04 Dec 2011 04:18:54	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	336,960	498,510	1.4794
03 Dec 2011 13:17:17	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	311,040	458,110	1.4728
03 Dec 2011 00:54:13	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	285,120	418,285	1.4670
02 Dec 2011 12:58:08	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	259,200	380,826	1.4692
02 Dec 2011 00:09:37	1035732	13636664	hadcm3n_o05l_1940_40_007544529_3	233,280	342,462	1.4680