Task 13629533

Name	hadcm3n_o3d2_1940_40_007543845_2
Workunit	7741077
Created	10 Nov 2011, 5:02:45 UTC
Sent	10 Nov 2011, 5:07:14 UTC
Report deadline	9 Feb 2012, 12:34:25 UTC
Received	29 Nov 2011, 18:49:21 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1179324
Run time	15 days 18 hours 47 min 16 sec
CPU time	14 days 11 hours 18 min 50 sec
Validate state	Invalid
Credit	7,153.92
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o3d2ko.pjf8c10 Error converting file to netcdf: dataout/o3d2ko.pif8c10 Error converting file to netcdf: dataout/o3d2ko.pff8c10 Error converting file to netcdf: dataout/o3d2ka.phf8c10 Error converting file to netcdf: dataout/o3d2ka.pgf8c10 Error converting file to netcdf: dataout/o3d2ka.pef8c10 Error converting file to netcdf: dataout/o3d2ka.pdf8c10 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2080, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Nov 2011 11:41:12	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	596,160	1,239,785	2.0796
28 Nov 2011 16:57:44	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	570,240	1,185,417	2.0788
28 Nov 2011 00:42:56	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	544,320	1,130,415	2.0767
27 Nov 2011 05:24:41	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	518,400	1,075,822	2.0753
26 Nov 2011 08:19:27	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	492,480	1,022,304	2.0758
25 Nov 2011 11:11:36	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	466,560	969,010	2.0769
24 Nov 2011 11:12:17	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	440,640	915,421	2.0775
23 Nov 2011 15:23:53	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	414,720	860,851	2.0757
22 Nov 2011 20:49:10	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	388,800	806,292	2.0738
21 Nov 2011 10:28:57	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	362,880	753,331	2.0760
20 Nov 2011 05:32:08	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	336,960	700,695	2.0795
19 Nov 2011 14:00:10	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	311,040	649,182	2.0871
18 Nov 2011 20:49:53	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	285,120	597,481	2.0955
18 Nov 2011 01:12:10	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	259,200	546,584	2.1087
16 Nov 2011 14:01:27	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	233,280	493,102	2.1138
15 Nov 2011 21:03:50	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	207,360	440,641	2.1250
15 Nov 2011 18:54:43	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	181,440	389,505	2.1467
15 Nov 2011 18:54:43	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	155,520	335,610	2.1580
15 Nov 2011 18:54:43	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	129,600	280,636	2.1654
15 Nov 2011 18:54:43	1179324	13629533	hadcm3n_o3d2_1940_40_007543845_2	103,680	223,846	2.1590