climateprediction.net home page
Task 13604792

Task 13604792

Name hadcm3n_ykpk_1980_40_007537744_0
Workunit 7734976
Created 5 Nov 2011, 17:44:14 UTC
Sent 5 Nov 2011, 17:46:54 UTC
Report deadline 5 Feb 2012, 1:14:05 UTC
Received 24 Nov 2011, 18:31:15 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 124
Run time 11 days 12 hours 37 min 5 sec
CPU time 11 days 4 hours 57 min 24 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.19 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:59:10 (2870): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:01:09 (26145): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:04:27 (26172): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:06:29 (26188): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:09:11 (26205): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:11:14 (26221): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:13:51 (26237): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:16:38 (26255): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:16:39 (26255): No heartbeat from core client for 30 sec - exiting
19:16:40 (26255): No heartbeat from core client for 30 sec - exiting
19:16:41 (26255): No heartbeat from core client for 30 sec - exiting
19:19:21 (26290): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:22:00 (26307): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:24:02 (26324): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:26:55 (26340): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:29:23 (26357): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:31:55 (26373): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:34:27 (26409): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:36:29 (26425): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:38:41 (26441): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:40:57 (26458): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:43:50 (26482): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:46:22 (26499): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:48:59 (26534): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:51:11 (26550): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:53:14 (26567): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:55:51 (26584): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:57:53 (26600): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:00:04 (26617): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:02:32 (26657): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:04:33 (26673): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:07:15 (26691): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:09:52 (26707): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:12:14 (26723): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:14:57 (26740): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:17:19 (26776): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:19:46 (26793): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:20:58 (26809): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:21:54 (26825): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:24:40 (26841): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:27:13 (26859): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:30:09 (26875): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:30:10 (26875): No heartbeat from core client for 30 sec - exiting
20:30:11 (26875): No heartbeat from core client for 30 sec - exiting
20:30:12 (26875): No heartbeat from core client for 30 sec - exiting
20:33:02 (26910): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:35:49 (26933): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:35:50 (26933): No heartbeat from core client for 30 sec - exiting
20:35:51 (26933): No heartbeat from core client for 30 sec - exiting
20:35:52 (26933): No heartbeat from core client for 30 sec - exiting
20:38:42 (26947): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:40:50 (26960): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:43:12 (26972): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:46:10 (26986): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:48:27 (27173): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:51:10 (27185): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:54:02 (27198): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:56:55 (27211): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:59:18 (27225): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:01:25 (27239): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:04:23 (27275): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:06:20 (27288): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:08:28 (27300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:10:46 (27312): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:13:14 (27326): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:15:27 (27338): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:19 (27369): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:20:27 (27381): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:23:04 (27393): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:25:45 (27415): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:28:18 (27428): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:30:25 (27440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:33:06 (27471): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:35:39 (27483): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:38:06 (27497): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:40:38 (27510): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:42:45 (27522): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:45:08 (27535): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:47:19 (27565): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:50:11 (27577): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:53:04 (27591): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:55:41 (27603): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:55:42 (27603): No heartbeat from core client for 30 sec - exiting
21:58:33 (27616): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:00:46 (27628): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:03:02 (27664): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:05:44 (27678): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:07:52 (27691): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:10:33 (27703): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:13:21 (27715): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:15:23 (27727): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:18:05 (27760): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:20:57 (27772): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 62 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/ykpkko.pjk0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkko.pjk0c10
Error: Input file: dataout/ykpkko.pik0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkko.pik0c10
Error: Input file: dataout/ykpkko.pfk0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkko.pfk0c10
Error: Input file: dataout/ykpkko.pck0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkko.pck0c10
Error: Input file: dataout/ykpkko.pbk0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkko.pbk0c10
Error: Input file: dataout/ykpkko.pak0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkko.pak0c10
Error: Input file: dataout/ykpkka.phk0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkka.phk0c10
Error: Input file: dataout/ykpkka.pgk0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkka.pgk0c10
Error: Input file: dataout/ykpkka.pek0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkka.pek0c10
Error: Input file: dataout/ykpkka.pdk0c10 is not a valid UM file.
Error converting file to netcdf: dataout/ykpkka.pdk0c10
22:23:15 (27784): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /root/BOINC/projects/climateprediction.net/hadcm3n_ykpk_1980_40_007537744/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
24 Nov 2011 18:39:31 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 518,400 968,260 1.8678
21 Nov 2011 04:37:27 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 492,480 920,782 1.8697
20 Nov 2011 14:14:03 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 466,560 870,265 1.8653
19 Nov 2011 23:55:48 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 440,640 819,695 1.8602
19 Nov 2011 09:38:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 414,720 769,113 1.8545
18 Nov 2011 19:18:54 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 388,800 718,557 1.8481
18 Nov 2011 04:33:04 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 362,880 668,013 1.8409
17 Nov 2011 14:39:53 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 336,960 617,483 1.8325
16 Nov 2011 23:20:53 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 311,040 566,859 1.8225
16 Nov 2011 09:05:07 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 285,120 516,284 1.8108
15 Nov 2011 18:43:53 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 259,200 465,690 1.7966
15 Nov 2011 18:10:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 233,280 419,544 1.7985
15 Nov 2011 18:10:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 207,360 373,690 1.8021
15 Nov 2011 18:10:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 181,440 327,854 1.8070
15 Nov 2011 18:10:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 155,520 282,106 1.8140
15 Nov 2011 18:10:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 129,600 236,895 1.8279
15 Nov 2011 18:10:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 103,680 191,662 1.8486
15 Nov 2011 18:10:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 77,760 144,749 1.8615
15 Nov 2011 18:10:50 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 51,840 97,636 1.8834
15 Nov 2011 18:10:51 124 13604792 hadcm3n_ykpk_1980_40_007537744_0 25,920 49,637 1.9150


©2024 climateprediction.net