Task 13012315

Name	hadcm3n_o5ks_1940_40_007267594_2
Workunit	7465834
Created	27 Jun 2011, 18:40:07 UTC
Sent	27 Jun 2011, 18:40:16 UTC
Report deadline	27 Sep 2011, 2:07:27 UTC
Received	14 Aug 2011, 2:37:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1152487
Run time	20 days 2 hours 38 min 43 sec
CPU time	18 days 21 hours 53 min 48 sec
Validate state	Invalid
Credit	9,642.24
Device peak FLOPS	2.28 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4772, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4772, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 11:27:46 (5732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file dataout/ocean_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:22:36 (6276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:55:36 (4628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Aug 2011 16:43:56	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	803,520	1,622,001	2.0186
12 Aug 2011 22:02:23	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	777,600	1,574,132	2.0243
12 Aug 2011 00:04:19	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	751,680	1,524,063	2.0275
10 Aug 2011 19:28:20	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	725,760	1,474,548	2.0317
10 Aug 2011 00:14:26	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	699,840	1,425,784	2.0373
09 Aug 2011 07:59:34	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	673,920	1,369,555	2.0322
08 Aug 2011 14:36:25	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	648,000	1,310,531	2.0224
07 Aug 2011 21:16:55	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	622,080	1,252,122	2.0128
07 Aug 2011 03:26:59	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	596,160	1,190,989	1.9978
26 Jul 2011 15:26:43	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	570,240	1,134,403	1.9893
26 Jul 2011 00:39:18	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	544,320	1,084,147	1.9917
25 Jul 2011 20:23:29	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	518,400	1,030,424	1.9877
25 Jul 2011 19:20:41	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	492,480	982,548	1.9951
25 Jul 2011 19:12:22	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	466,560	934,157	2.0022
25 Jul 2011 18:54:17	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	440,640	884,547	2.0074
25 Jul 2011 18:08:14	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	414,720	834,274	2.0117
25 Jul 2011 17:33:22	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	388,800	767,225	1.9733
25 Jul 2011 16:43:46	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	362,880	717,186	1.9764
25 Jul 2011 15:56:27	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	336,960	667,402	1.9807
25 Jul 2011 15:29:03	1152487	13012315	hadcm3n_o5ks_1940_40_007267594_2	311,040	616,802	1.9830