Task 11886375

Name	famous_vobf_1599_200_006717281_5
Workunit	6920534
Created	15 Sep 2010, 6:15:32 UTC
Sent	15 Sep 2010, 6:17:47 UTC
Report deadline	15 Dec 2010, 13:44:58 UTC
Received	14 Oct 2010, 22:35:15 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	924943
Run time	7 days 7 hours 59 min 18 sec
CPU time	7 days 7 hours 56 min 47 sec
Validate state	Invalid
Credit	2,902.96
Device peak FLOPS	1.81 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5560, selfPID=5560, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5380, selfPID=5380, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5344, selfPID=5344, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2744, selfPID=2744, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5372, selfPID=5372, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5352, selfPID=5352, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3784, selfPID=3784, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2204, selfPID=2204, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3816, selfPID=3816, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3404, selfPID=3404, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4860, selfPID=4860, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2208, selfPID=2208, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4772, selfPID=4772, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4508, selfPID=4508, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2872, selfPID=2872, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2296, selfPID=2296, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2292, selfPID=2292, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=728, selfPID=728, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5016, selfPID=5016, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3808, selfPID=3808, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3656, selfPID=3656, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=796, selfPID=796, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5320, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5320, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5320, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5320, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5320, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5576, selfPID=5576, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3924, selfPID=3924, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5060, selfPID=5060, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3628, selfPID=3628, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2696, selfPID=2696, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4852, selfPID=4852, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1404, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1404, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1404, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1404, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1404, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2528, selfPID=2528, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4332, selfPID=4332, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4736, selfPID=4736, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vobf_1599_200_006717281\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3492, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4416, selfPID=4416, iMonCtr=1 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_vobf_1599_200_006717281/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Sorry, too many model crashes! :-( 15:34:00 (4632): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Oct 2010 22:03:17	924943	11886375	famous_vobf_1599_200_006717281_5	879,866	631,994	0.7183
14 Oct 2010 20:06:04	924943	11886375	famous_vobf_1599_200_006717281_5	870,506	626,147	0.7193
14 Oct 2010 18:33:41	924943	11886375	famous_vobf_1599_200_006717281_5	861,146	620,377	0.7204
14 Oct 2010 05:03:24	924943	11886375	famous_vobf_1599_200_006717281_5	851,786	614,642	0.7216
14 Oct 2010 02:59:36	924943	11886375	famous_vobf_1599_200_006717281_5	842,426	608,616	0.7225
13 Oct 2010 23:04:23	924943	11886375	famous_vobf_1599_200_006717281_5	833,066	602,592	0.7233
13 Oct 2010 17:22:02	924943	11886375	famous_vobf_1599_200_006717281_5	823,706	596,611	0.7243
13 Oct 2010 03:33:26	924943	11886375	famous_vobf_1599_200_006717281_5	814,346	590,671	0.7253
12 Oct 2010 18:40:55	924943	11886375	famous_vobf_1599_200_006717281_5	804,986	584,506	0.7261
12 Oct 2010 16:54:00	924943	11886375	famous_vobf_1599_200_006717281_5	795,626	578,506	0.7271
12 Oct 2010 04:37:17	924943	11886375	famous_vobf_1599_200_006717281_5	786,266	572,713	0.7284
11 Oct 2010 22:43:27	924943	11886375	famous_vobf_1599_200_006717281_5	776,906	566,671	0.7294
10 Oct 2010 23:37:52	924943	11886375	famous_vobf_1599_200_006717281_5	767,546	560,412	0.7301
10 Oct 2010 17:36:40	924943	11886375	famous_vobf_1599_200_006717281_5	758,186	553,582	0.7301
10 Oct 2010 03:48:52	924943	11886375	famous_vobf_1599_200_006717281_5	748,826	546,793	0.7302
09 Oct 2010 23:22:08	924943	11886375	famous_vobf_1599_200_006717281_5	739,466	539,841	0.7300
09 Oct 2010 23:22:08	924943	11886375	famous_vobf_1599_200_006717281_5	730,106	533,045	0.7301
09 Oct 2010 23:22:08	924943	11886375	famous_vobf_1599_200_006717281_5	720,746	526,190	0.7301
09 Oct 2010 23:22:08	924943	11886375	famous_vobf_1599_200_006717281_5	711,386	519,201	0.7298
09 Oct 2010 23:22:08	924943	11886375	famous_vobf_1599_200_006717281_5	702,026	512,270	0.7297