Task 13373096

Name	hadcm3n_y8fs_1940_40_007453141_0
Workunit	7650644
Created	10 Sep 2011, 15:11:41 UTC
Sent	10 Sep 2011, 17:41:00 UTC
Report deadline	11 Dec 2011, 1:08:11 UTC
Received	22 Sep 2011, 9:53:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1148900
Run time	11 days 6 hours 4 min 17 sec
CPU time	10 days 2 hours 50 min 59 sec
Validate state	Invalid
Credit	5,598.72
Device peak FLOPS	2.48 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 10:31:58 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:31:59 (5652): No heartbeat from core client for 30 sec - exiting 10:32:00 (5652): No heartbeat from core client for 30 sec - exiting 10:32:01 (5652): No heartbeat from core client for 30 sec - exiting 10:32:02 (5652): No heartbeat from core client for 30 sec - exiting 10:32:03 (5652): No heartbeat from core client for 30 sec - exiting 10:32:04 (5652): No heartbeat from core client for 30 sec - exiting 10:32:05 (5652): No heartbeat from core client for 30 sec - exiting 10:32:06 (5652): No heartbeat from core client for 30 sec - exiting 10:32:08 (5652): No heartbeat from core client for 30 sec - exiting 10:32:09 (5652): No heartbeat from core client for 30 sec - exiting 11:16:02 (5192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:52:46 (3016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Sep 2011 23:47:00	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	466,560	888,198	1.9037
21 Sep 2011 10:37:46	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	440,640	842,666	1.9124
20 Sep 2011 20:27:47	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	414,720	797,353	1.9226
20 Sep 2011 07:12:23	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	388,800	751,413	1.9326
19 Sep 2011 17:47:59	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	362,880	705,610	1.9445
19 Sep 2011 01:02:50	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	336,960	656,155	1.9473
18 Sep 2011 08:41:44	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	311,040	600,030	1.9291
17 Sep 2011 16:50:06	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	285,120	543,878	1.9075
17 Sep 2011 00:05:53	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	259,200	487,793	1.8819
16 Sep 2011 10:20:37	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	233,280	440,536	1.8884
15 Sep 2011 23:10:12	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	207,360	401,861	1.9380
15 Sep 2011 12:00:50	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	181,440	362,983	2.0006
14 Sep 2011 22:28:42	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	155,520	317,368	2.0407
14 Sep 2011 09:18:30	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	129,600	271,904	2.0980
13 Sep 2011 16:48:50	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	103,680	217,172	2.0946
12 Sep 2011 20:19:12	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	77,760	161,479	2.0766
12 Sep 2011 01:00:05	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	51,840	107,065	2.0653
11 Sep 2011 08:46:17	1148900	13373096	hadcm3n_y8fs_1940_40_007453141_0	25,920	51,302	1.9792