climateprediction.net home page
Task 15596121

Task 15596121

Name hadcm3n_486p_1940_40_008309127_0
Workunit 8460262
Created 7 Feb 2013, 19:57:48 UTC
Sent 7 Feb 2013, 20:02:52 UTC
Report deadline 10 May 2013, 3:30:03 UTC
Received 18 Mar 2013, 8:05:14 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1234259
Run time 11 days 3 hours 47 min 4 sec
CPU time 11 days 2 hours 7 min 19 sec
Validate state Invalid
Credit 9,331.20
Device peak FLOPS 3.52 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
17:37:16 (10230): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:39:13 (12205): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:41:34 (12307): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:44:00 (12426): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:45:56 (12571): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:47:42 (12685): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:50:13 (12767): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:52:49 (12923): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:55:10 (13068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:57:21 (13197): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:59:22 (13314): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:02:04 (13460): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:03:50 (13616): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:06:11 (13732): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:08:22 (13850): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:10:13 (13967): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:12:04 (14081): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:14:05 (14185): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:16:11 (14296): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:18:32 (14419): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:20:34 (14545): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:22:55 (14688): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:25:11 (14839): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:27:07 (14954): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:28:58 (15068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:31:24 (15176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:33:10 (15301): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:35:46 (15405): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:38:17 (15528): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:40:03 (15691): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:42:29 (15826): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:44:20 (15947): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:46:17 (16054): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:48:23 (16179): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:50:54 (16294): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:53:00 (16450): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:55:06 (16558): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:57:22 (16676): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:59:23 (16797): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:01:03 (16949): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:02:50 (17055): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:05:11 (17166): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:07:53 (17280): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:09:39 (17437): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:11:35 (17509): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:13:41 (17622): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:15:32 (17736): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 62 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/486pko.pjh0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pko.pjh0c10
Error: Input file: dataout/486pko.pih0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pko.pih0c10
Error: Input file: dataout/486pko.pfh0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pko.pfh0c10
Error: Input file: dataout/486pko.pch0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pko.pch0c10
Error: Input file: dataout/486pko.pbh0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pko.pbh0c10
Error: Input file: dataout/486pko.pah0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pko.pah0c10
Error: Input file: dataout/486pka.phh0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pka.phh0c10
Error: Input file: dataout/486pka.pgh0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pka.pgh0c10
Error: Input file: dataout/486pka.peh0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pka.peh0c10
Error: Input file: dataout/486pka.pdh0c10 is not a valid UM file.
Error converting file to netcdf: dataout/486pka.pdh0c10
19:16:42 (17849): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_486p_1940_40_008309127/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
18 Mar 2013 08:08:32 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 777,600 958,050 1.2321
15 Mar 2013 01:26:27 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 751,680 928,727 1.2355
14 Mar 2013 16:05:43 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 725,760 897,025 1.2360
13 Mar 2013 23:16:58 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 699,840 865,407 1.2366
13 Mar 2013 06:29:20 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 673,920 833,777 1.2372
12 Mar 2013 21:51:42 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 648,000 802,152 1.2379
08 Mar 2013 06:10:42 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 622,080 770,479 1.2386
07 Mar 2013 21:22:01 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 596,160 738,732 1.2392
07 Mar 2013 04:19:07 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 570,240 706,928 1.2397
06 Mar 2013 20:03:31 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 544,320 675,161 1.2404
01 Mar 2013 02:36:00 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 518,400 643,373 1.2411
28 Feb 2013 17:48:30 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 492,480 611,602 1.2419
28 Feb 2013 01:00:40 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 466,560 579,913 1.2430
27 Feb 2013 08:11:41 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 440,640 548,260 1.2442
26 Feb 2013 23:19:29 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 414,720 516,554 1.2455
26 Feb 2013 06:29:35 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 388,800 484,758 1.2468
25 Feb 2013 21:05:42 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 362,880 453,069 1.2485
22 Feb 2013 05:56:25 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 336,960 421,336 1.2504
21 Feb 2013 21:28:54 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 311,040 389,651 1.2527
21 Feb 2013 04:28:44 1234259 15596121 hadcm3n_486p_1940_40_008309127_0 285,120 357,514 1.2539


©2024 cpdn.org