Task 13798757

Name	hadcm3n_t2td_1940_40_007443186_4
Workunit	7640689
Created	20 Dec 2011, 1:53:32 UTC
Sent	20 Dec 2011, 1:53:52 UTC
Report deadline	20 Mar 2012, 9:21:03 UTC
Received	11 Feb 2012, 18:01:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	429634
Run time	44 days 7 hours 59 min 42 sec
CPU time	43 days 2 hours 25 min 56 sec
Validate state	Invalid
Credit	5,598.72
Device peak FLOPS	0.68 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2404, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t2tdko.pje9c10 Error converting file to netcdf: dataout/t2tdko.pie9c10 Error converting file to netcdf: dataout/t2tdko.pfe9c10 Error converting file to netcdf: dataout/t2tdka.phe9c10 Error converting file to netcdf: dataout/t2tdka.pge9c10 Error converting file to netcdf: dataout/t2tdka.pee9c10 Error converting file to netcdf: dataout/t2tdka.pde9c10 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy 2048 Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2584, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2584, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2864, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2864, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2864, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2864, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Feb 2012 13:46:20	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	466,560	3,582,940	7.6795
07 Feb 2012 02:24:43	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	440,640	3,382,626	7.6766
04 Feb 2012 16:29:00	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	414,720	3,182,084	7.6728
02 Feb 2012 06:29:38	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	388,800	2,981,195	7.6677
30 Jan 2012 21:49:24	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	362,880	2,780,326	7.6618
28 Jan 2012 13:13:40	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	336,960	2,579,615	7.6556
26 Jan 2012 04:27:48	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	311,040	2,379,021	7.6486
23 Jan 2012 19:49:17	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	285,120	2,179,411	7.6438
21 Jan 2012 11:06:15	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	259,200	1,979,040	7.6352
19 Jan 2012 02:10:11	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	233,280	1,778,665	7.6246
16 Jan 2012 16:01:50	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	207,360	1,577,056	7.6054
14 Jan 2012 06:51:46	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	181,440	1,375,573	7.5814
11 Jan 2012 17:22:08	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	155,520	1,176,290	7.5636
09 Jan 2012 09:28:34	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	129,600	980,618	7.5665
07 Jan 2012 01:25:41	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	103,680	784,827	7.5697
04 Jan 2012 17:26:13	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	77,760	588,280	7.5653
02 Jan 2012 00:22:51	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	51,840	391,815	7.5582
29 Dec 2011 22:51:52	429634	13798757	hadcm3n_t2td_1940_40_007443186_4	25,920	195,297	7.5346