Task 13337538

Name	hadcm3n_o5fb_1900_40_007440348_0
Workunit	7637851
Created	5 Sep 2011, 18:19:48 UTC
Sent	5 Sep 2011, 20:45:38 UTC
Report deadline	6 Dec 2011, 4:12:49 UTC
Received	1 Oct 2011, 12:18:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1163000
Run time	11 days 16 hours 59 min 6 sec
CPU time	11 days 7 hours 1 min 38 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	2.17 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5792, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2236, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o5fbko.pjb5c10 Error converting file to netcdf: dataout/o5fbko.pib5c10 Error converting file to netcdf: dataout/o5fbko.pfb5c10 Error converting file to netcdf: dataout/o5fbka.phb5c10 Error converting file to netcdf: dataout/o5fbka.pgb5c10 Error converting file to netcdf: dataout/o5fbka.peb5c10 Error converting file to netcdf: dataout/o5fbka.pdb5c10 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Sep 2011 01:54:10	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	440,640	967,602	2.1959
17 Sep 2011 21:12:32	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	414,720	911,093	2.1969
17 Sep 2011 04:28:15	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	388,800	855,292	2.1998
16 Sep 2011 12:24:07	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	362,880	799,461	2.2031
15 Sep 2011 20:30:02	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	336,960	743,610	2.2068
15 Sep 2011 04:38:53	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	311,040	687,934	2.2117
14 Sep 2011 11:38:07	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	285,120	628,673	2.2049
13 Sep 2011 03:37:12	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	259,200	572,617	2.2092
12 Sep 2011 11:18:55	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	233,280	515,788	2.2110
11 Sep 2011 18:54:30	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	207,360	459,503	2.2160
11 Sep 2011 02:41:55	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	181,440	403,247	2.2225
10 Sep 2011 09:21:27	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	155,520	346,387	2.2273
09 Sep 2011 15:54:38	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	129,600	289,902	2.2369
08 Sep 2011 23:47:58	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	103,680	233,553	2.2526
08 Sep 2011 06:58:44	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	77,760	174,981	2.2503
07 Sep 2011 15:25:37	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	51,840	117,715	2.2707
06 Sep 2011 16:16:48	1163000	13337538	hadcm3n_o5fb_1900_40_007440348_0	25,920	58,447	2.2549