climateprediction.net home page
Task 15667884

Task 15667884

Name hadcm3n_zizq_1880_40_008253498_4
Workunit 8408622
Created 15 Mar 2013, 6:12:22 UTC
Sent 15 Mar 2013, 6:12:24 UTC
Report deadline 14 Jun 2013, 13:39:35 UTC
Received 26 May 2013, 8:24:07 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1084651
Run time 16 days 18 hours 9 min 18 sec
CPU time 15 days 19 hours 1 min 39 sec
Validate state Invalid
Credit 9,331.20
Device peak FLOPS 2.29 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>7.0.28</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/zizqko.pj81c10
Error converting file to netcdf: dataout/zizqko.pi81c10
Error converting file to netcdf: dataout/zizqko.pf81c10
Error converting file to netcdf: dataout/zizqka.ph81c10
Error converting file to netcdf: dataout/zizqka.pg81c10
Error converting file to netcdf: dataout/zizqka.pe81c10
Error converting file to netcdf: dataout/zizqka.pd81c10
CPDN Monitor - Quit request from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/zizqko.pj85c10
Error converting file to netcdf: dataout/zizqko.pi85c10
Error converting file to netcdf: dataout/zizqko.pf85c10
Error converting file to netcdf: dataout/zizqka.ph85c10
Error converting file to netcdf: dataout/zizqka.pg85c10
Error converting file to netcdf: dataout/zizqka.pe85c10
Error converting file to netcdf: dataout/zizqka.pd85c10
CPDN Monitor - Quit request from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
BUFFOUT: C I/O Error - Return code = 32

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
forrtl: There is not enough space on the disk.

CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
04:24:55 (6132): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:15:06 (736): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:18:27 (3972): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:19:05 (6572): No heartbeat from core client for 30 sec - exiting
05:19:06 (6572): No heartbeat from core client for 30 sec - exiting
05:19:07 (6572): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
06:18:37 (1868): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:18:39 (1868): No heartbeat from core client for 30 sec - exiting
07:18:52 (4240): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:18:53 (4240): No heartbeat from core client for 30 sec - exiting
07:18:54 (4240): No heartbeat from core client for 30 sec - exiting
07:39:43 (1088): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:39:45 (1088): No heartbeat from core client for 30 sec - exiting
07:39:46 (1088): No heartbeat from core client for 30 sec - exiting
07:39:47 (1088): No heartbeat from core client for 30 sec - exiting
07:39:48 (1088): No heartbeat from core client for 30 sec - exiting
07:39:49 (1088): No heartbeat from core client for 30 sec - exiting
07:39:50 (1088): No heartbeat from core client for 30 sec - exiting
07:39:51 (1088): No heartbeat from core client for 30 sec - exiting
07:39:52 (1088): No heartbeat from core client for 30 sec - exiting
07:39:53 (1088): No heartbeat from core client for 30 sec - exiting
07:39:54 (1088): No heartbeat from core client for 30 sec - exiting
07:40:53 (588): No heartbeat from core client for 30 sec - exiting
07:40:55 (588): No heartbeat from core client for 30 sec - exiting
07:40:56 (588): No heartbeat from core client for 30 sec - exiting
07:40:57 (588): No heartbeat from core client for 30 sec - exiting
07:40:58 (588): No heartbeat from core client for 30 sec - exiting
07:40:59 (588): No heartbeat from core client for 30 sec - exiting
07:41:00 (588): No heartbeat from core client for 30 sec - exiting
07:41:01 (588): No heartbeat from core client for 30 sec - exiting
07:41:03 (588): No heartbeat from core client for 30 sec - exiting
07:41:05 (588): No heartbeat from core client for 30 sec - exiting
07:41:06 (588): No heartbeat from core client for 30 sec - exiting
07:41:08 (588): No heartbeat from core client for 30 sec - exiting
07:41:09 (588): No heartbeat from core client for 30 sec - exiting
07:41:10 (588): No heartbeat from core client for 30 sec - exiting
07:41:12 (588): No heartbeat from core client for 30 sec - exiting
07:41:13 (588): No heartbeat from core client for 30 sec - exiting
07:41:14 (588): No heartbeat from core client for 30 sec - exiting
07:41:15 (588): No heartbeat from core client for 30 sec - exiting
07:41:16 (588): No heartbeat from core client for 30 sec - exiting
07:41:17 (588): No heartbeat from core client for 30 sec - exiting
07:41:18 (588): No heartbeat from core client for 30 sec - exiting
07:41:19 (588): No heartbeat from core client for 30 sec - exiting
07:41:20 (588): No heartbeat from core client for 30 sec - exiting
07:41:21 (588): No heartbeat from core client for 30 sec - exiting
07:41:22 (588): No heartbeat from core client for 30 sec - exiting
07:41:23 (588): No heartbeat from core client for 30 sec - exiting
07:41:24 (588): No heartbeat from core client for 30 sec - exiting
07:41:25 (588): No heartbeat from core client for 30 sec - exiting
07:41:26 (588): No heartbeat from core client for 30 sec - exiting
07:41:27 (588): No heartbeat from core client for 30 sec - exiting
07:41:28 (588): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
08:00:01 (5080): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:00:03 (5080): No heartbeat from core client for 30 sec - exiting
08:00:04 (5080): No heartbeat from core client for 30 sec - exiting
08:00:05 (5080): No heartbeat from core client for 30 sec - exiting
08:01:19 (7056): No heartbeat from core client for 30 sec - exiting
08:01:20 (7056): No heartbeat from core client for 30 sec - exiting
08:01:21 (7056): No heartbeat from core client for 30 sec - exiting
08:01:22 (7056): No heartbeat from core client for 30 sec - exiting
08:01:23 (7056): No heartbeat from core client for 30 sec - exiting
08:01:24 (7056): No heartbeat from core client for 30 sec - exiting
08:01:25 (7056): No heartbeat from core client for 30 sec - exiting
08:01:26 (7056): No heartbeat from core client for 30 sec - exiting
08:01:27 (7056): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:30:36 (5860): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:30:37 (5860): No heartbeat from core client for 30 sec - exiting
09:00:14 (8032): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:00:15 (8032): No heartbeat from core client for 30 sec - exiting
09:00:16 (8032): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4144, selfPID=4144, iMonCtr=1

zip error: Could not create output file (was replacing the original zip file)
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_zizq_1880_40_008253498/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
26 May 2013 07:24:41 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 777,600 1,364,575 1.7549
24 May 2013 23:18:46 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 751,680 1,320,207 1.7563
23 May 2013 21:48:40 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 725,760 1,273,934 1.7553
22 May 2013 07:22:46 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 699,840 1,226,315 1.7523
21 May 2013 04:46:36 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 673,920 1,178,420 1.7486
10 May 2013 01:29:09 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 648,000 1,130,222 1.7442
08 May 2013 23:18:22 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 622,080 1,081,828 1.7390
07 May 2013 20:39:44 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 596,160 1,033,212 1.7331
05 May 2013 07:44:41 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 570,240 984,471 1.7264
02 May 2013 02:24:00 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 544,320 939,227 1.7255
29 Apr 2013 23:17:36 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 518,400 895,290 1.7270
25 Apr 2013 23:34:56 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 492,480 851,073 1.7281
19 Apr 2013 06:48:00 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 466,560 806,849 1.7294
18 Apr 2013 05:55:07 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 440,640 763,669 1.7331
17 Apr 2013 05:25:47 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 414,720 720,725 1.7379
16 Apr 2013 04:43:25 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 388,800 677,730 1.7431
14 Apr 2013 01:33:59 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 362,880 634,465 1.7484
13 Apr 2013 00:45:49 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 336,960 591,409 1.7551
07 Apr 2013 21:57:54 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 311,040 547,406 1.7599
03 Apr 2013 23:42:03 1084651 15667884 hadcm3n_zizq_1880_40_008253498_4 285,120 504,661 1.7700


©2024 climateprediction.net