climateprediction.net home page
Task 13619064

Task 13619064

Name hadcm3n_ykys_1900_40_007526570_2
Workunit 7724045
Created 8 Nov 2011, 20:47:09 UTC
Sent 19 Nov 2011, 1:04:00 UTC
Report deadline 18 Feb 2012, 8:31:11 UTC
Received 23 Dec 2011, 9:42:01 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1157390
Run time 33 days 22 hours 24 min 21 sec
CPU time 33 days 20 hours 50 min 57 sec
Validate state Invalid
Credit 12,441.60
Device peak FLOPS 1.37 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:10:16 (2338): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
04:54:14 (2416): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
04:55:36 (4329): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
04:58:38 (4377): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:01:58 (4443): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
05:51:31 (4502): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:46:39 (4549): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:47:29 (4714): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:48:20 (4774): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:49:24 (4799): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:51:07 (4866): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:51:54 (4931): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
07:52:51 (5079): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:54:33 (5196): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:56:18 (5255): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:58:02 (5310): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:59:45 (5333): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:00:35 (5392): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
08:01:29 (5451): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:03:13 (5510): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:04:57 (5569): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:06:41 (5624): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:08:24 (5648): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:09:15 (5708): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
08:10:08 (5768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:11:54 (5826): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:13:37 (5886): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:15:21 (5940): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:17:07 (5964): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:17:58 (6027): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
08:18:51 (6085): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:20:36 (6144): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:22:19 (6202): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:24:02 (6257): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:25:45 (6281): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:26:36 (6340): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
08:27:29 (6399): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:29:14 (6457): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:30:57 (6515): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:32:41 (6569): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:34:24 (6592): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:35:15 (6651): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
08:36:08 (6709): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:37:51 (6767): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:39:34 (6828): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:41:18 (6885): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:43:02 (6906): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:43:53 (6966): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
08:44:47 (7024): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:46:31 (7082): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:48:15 (7140): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:49:58 (7195): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:51:41 (7218): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:52:31 (7277): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
08:53:26 (7336): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:55:11 (7394): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:56:55 (7453): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:42:40 (7469): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:42:44 (7469): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:31:12 (14209): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:31:54 (21778): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:25:29 (21822): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:56:15 (24425): No heartbeat from core client for 30 sec - exiting
23:56:17 (24425): No heartbeat from core client for 30 sec - exiting
23:56:18 (24425): No heartbeat from core client for 30 sec - exiting
23:56:19 (24425): No heartbeat from core client for 30 sec - exiting
23:56:20 (24425): No heartbeat from core client for 30 sec - exiting
23:56:21 (24425): No heartbeat from core client for 30 sec - exiting
23:56:23 (24425): No heartbeat from core client for 30 sec - exiting
23:56:24 (24425): No heartbeat from core client for 30 sec - exiting
23:56:25 (24425): No heartbeat from core client for 30 sec - exiting
23:56:26 (24425): No heartbeat from core client for 30 sec - exiting
23:56:27 (24425): No heartbeat from core client for 30 sec - exiting
23:56:28 (24425): No heartbeat from core client for 30 sec - exiting
23:56:29 (24425): No heartbeat from core client for 30 sec - exiting
23:56:30 (24425): No heartbeat from core client for 30 sec - exiting
23:56:31 (24425): No heartbeat from core client for 30 sec - exiting
23:56:34 (24425): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_ykys_1900_40_007526570/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
23 Dec 2011 06:00:35 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 1,036,800 2,926,319 2.8225
22 Dec 2011 09:23:49 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 1,010,880 2,852,757 2.8221
21 Dec 2011 12:49:01 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 984,960 2,778,986 2.8214
20 Dec 2011 16:06:02 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 959,040 2,704,637 2.8202
19 Dec 2011 19:17:34 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 933,120 2,630,339 2.8189
18 Dec 2011 22:55:22 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 907,200 2,557,077 2.8186
18 Dec 2011 02:36:21 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 881,280 2,484,156 2.8188
17 Dec 2011 06:13:19 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 855,360 2,411,169 2.8189
16 Dec 2011 09:46:45 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 829,440 2,337,892 2.8186
15 Dec 2011 14:22:04 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 803,520 2,264,976 2.8188
14 Dec 2011 17:00:27 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 777,600 2,191,720 2.8186
13 Dec 2011 20:39:15 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 751,680 2,118,584 2.8185
13 Dec 2011 00:12:46 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 725,760 2,045,242 2.8181
12 Dec 2011 03:47:07 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 699,840 1,972,253 2.8181
11 Dec 2011 07:16:29 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 673,920 1,898,787 2.8175
10 Dec 2011 10:51:31 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 648,000 1,825,354 2.8169
09 Dec 2011 14:24:00 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 622,080 1,751,912 2.8162
08 Dec 2011 17:55:37 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 596,160 1,678,499 2.8155
07 Dec 2011 21:24:04 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 570,240 1,604,940 2.8145
07 Dec 2011 00:53:05 1157390 13619064 hadcm3n_ykys_1900_40_007526570_2 544,320 1,531,089 2.8128


©2024 climateprediction.net