Task 17477969

Name	hadcm3n_xbf6_1940_40_009151096_2
Workunit	9281432
Created	24 Nov 2014, 13:26:58 UTC
Sent	24 Nov 2014, 13:27:03 UTC
Report deadline	23 Feb 2015, 20:54:14 UTC
Received	2 Feb 2015, 10:56:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1260838
Run time	20 days 3 hours 9 min 45 sec
CPU time	18 days 4 hours 1 min
Validate state	Invalid
Credit	12,130.56
Device peak FLOPS	2.57 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5668, iMonCtr=1 Model crashController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5476, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/xbf6ko.pje9c10 Error converting file to netcdf: dataout/xbf6ko.pie9c10 Error converting file to netcdf: dataout/xbf6ko.pfe9c10 Error converting file to netcdf: dataout/xbf6ka.phe9c10 Error converting file to netcdf: dataout/xbf6ka.pge9c10 Error converting file to netcdf: dataout/xbf6ka.pee9c10 Error converting file to netcdf: dataout/xbf6ka.pde9c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4788, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4372, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4372, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4372, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4440, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2076, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5464, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not runCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5044, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 Jan 2015 20:02:23	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	1,010,880	1,538,910	1.5223
29 Jan 2015 14:46:08	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	984,960	1,500,904	1.5238
27 Jan 2015 18:39:34	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	959,040	1,464,323	1.5269
23 Jan 2015 11:52:47	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	933,120	1,427,266	1.5296
20 Jan 2015 12:17:18	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	907,200	1,390,621	1.5329
18 Jan 2015 12:03:42	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	881,280	1,350,490	1.5324
15 Jan 2015 15:40:09	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	855,360	1,310,562	1.5322
14 Jan 2015 16:54:10	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	829,440	1,270,724	1.5320
11 Jan 2015 21:40:41	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	803,520	1,231,081	1.5321
09 Jan 2015 08:38:21	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	777,600	1,190,204	1.5306
08 Jan 2015 16:23:59	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	751,680	1,148,569	1.5280
08 Jan 2015 16:20:24	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	725,760	1,108,543	1.5274
04 Jan 2015 18:04:49	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	699,840	1,068,042	1.5261
02 Jan 2015 20:42:46	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	673,920	1,027,830	1.5252
01 Jan 2015 19:34:19	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	648,000	988,135	1.5249
31 Dec 2014 20:38:30	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	622,080	947,437	1.5230
30 Dec 2014 10:34:22	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	596,160	907,109	1.5216
29 Dec 2014 13:24:50	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	570,240	866,918	1.5203
28 Dec 2014 15:31:14	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	544,320	826,720	1.5188
22 Dec 2014 14:57:56	1260838	17477969	hadcm3n_xbf6_1940_40_009151096_2	518,400	786,514	1.5172