Task 13650765

Name	hadcm3n_yekg_1900_40_007517947_2
Workunit	7715422
Created	21 Nov 2011, 7:33:33 UTC
Sent	21 Nov 2011, 7:37:01 UTC
Report deadline	20 Feb 2012, 15:04:12 UTC
Received	22 Mar 2012, 9:00:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1180720
Run time	22 days 18 hours 3 min 8 sec
CPU time	22 days 18 hours 3 min 8 sec
Validate state	Invalid
Credit	7,153.92
Device peak FLOPS	1.49 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.8.16</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yekgko.pja4c10 Error converting file to netcdf: dataout/yekgko.pia4c10 Error converting file to netcdf: dataout/yekgko.pfa4c10 Error converting file to netcdf: dataout/yekgka.pha4c10 Error converting file to netcdf: dataout/yekgka.pga4c10 Error converting file to netcdf: dataout/yekgka.pea4c10 Error converting file to netcdf: dataout/yekgka.pda4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yekgko.pjb4c10 Error converting file to netcdf: dataout/yekgko.pib4c10 Error converting file to netcdf: dataout/yekgko.pfb4c10 Error converting file to netcdf: dataout/yekgka.phb4c10 Error converting file to netcdf: dataout/yekgka.pgb4c10 Error converting file to netcdf: dataout/yekgka.peb4c10 Error converting file to netcdf: dataout/yekgka.pdb4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yekgko.pjb7c10 Error converting file to netcdf: dataout/yekgko.pib7c10 Error converting file to netcdf: dataout/yekgko.pfb7c10 Error converting file to netcdf: dataout/yekgka.phb7c10 Error converting file to netcdf: dataout/yekgka.pgb7c10 Error converting file to netcdf: dataout/yekgka.peb7c10 Error converting file to netcdf: dataout/yekgka.pdb7c10 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yekgko.pjb7c10 Error converting file to netcdf: dataout/yekgko.pib7c10 Error converting file to netcdf: dataout/yekgko.pfb7c10 Error converting file to netcdf: dataout/yekgka.phb7c10 Error converting file to netcdf: dataout/yekgka.pgb7c10 Error converting file to netcdf: dataout/yekgka.peb7c10 Error converting file to netcdf: dataout/yekgka.pdb7c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yekgko.pjb8c10 Error converting file to netcdf: dataout/yekgko.pib8c10 Error converting file to netcdf: dataout/yekgko.pfb8c10 Error converting file to netcdf: dataout/yekgka.phb8c10 Error converting file to netcdf: dataout/yekgka.pgb8c10 Error converting file to netcdf: dataout/yekgka.peb8c10 Error converting file to netcdf: dataout/yekgka.pdb8c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2764, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yekgko.pjc2c10 Error converting file to netcdf: dataout/yekgko.pic2c10 Error converting file to netcdf: dataout/yekgko.pfc2c10 Error converting file to netcdf: dataout/yekgka.phc2c10 Error converting file to netcdf: dataout/yekgka.pgc2c10 Error converting file to netcdf: dataout/yekgka.pec2c10 Error converting file to netcdf: dataout/yekgka.pdc2c10 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yekgko.pjc2c10 Error converting file to netcdf: dataout/yekgko.pic2c10 Error converting file to netcdf: dataout/yekgko.pfc2c10 Error converting file to netcdf: dataout/yekgka.phc2c10 Error converting file to netcdf: dataout/yekgka.pgc2c10 Error converting file to netcdf: dataout/yekgka.pec2c10 Error converting file to netcdf: dataout/yekgka.pdc2c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4004, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4004, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4004, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4004, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4004, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4004, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Mar 2012 12:35:22	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	596,160	1,894,357	3.1776
15 Mar 2012 10:27:55	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	570,240	1,810,834	3.1756
14 Mar 2012 09:15:07	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	544,320	1,727,694	3.1740
12 Mar 2012 12:47:08	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	518,400	1,664,851	3.2115
08 Mar 2012 19:39:49	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	492,480	1,581,413	3.2111
05 Mar 2012 16:19:03	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	466,560	1,497,884	3.2105
28 Feb 2012 10:28:22	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	440,640	1,413,815	3.2085
23 Feb 2012 07:48:47	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	414,720	1,330,650	3.2086
22 Feb 2012 08:12:16	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	388,800	1,247,329	3.2082
20 Feb 2012 16:13:33	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	362,880	1,163,977	3.2076
15 Feb 2012 13:42:57	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	336,960	1,080,727	3.2073
14 Feb 2012 13:57:16	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	311,040	997,421	3.2067
09 Feb 2012 23:33:37	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	285,120	913,936	3.2054
07 Feb 2012 14:17:11	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	259,200	830,640	3.2046
05 Feb 2012 21:47:46	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	233,280	746,991	3.2021
04 Feb 2012 22:13:38	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	207,360	663,630	3.2004
03 Feb 2012 22:45:53	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	181,440	580,655	3.2003
02 Feb 2012 06:09:18	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	155,520	497,841	3.2011
01 Feb 2012 06:32:26	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	129,600	414,754	3.2003
30 Jan 2012 14:14:07	1180720	13650765	hadcm3n_yekg_1900_40_007517947_2	103,680	331,754	3.1998