climateprediction.net home page
Task 15502620

Task 15502620

Name hadcm3n_o14r_2140_40_008269738_0
Workunit 8424862
Created 23 Dec 2012, 23:58:41 UTC
Sent 24 Dec 2012, 7:54:09 UTC
Report deadline 25 Mar 2013, 15:21:20 UTC
Received 30 Jan 2013, 9:26:21 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1157390
Run time 25 days 11 hours 25 min 32 sec
CPU time 25 days 10 hours 23 min 10 sec
Validate state Invalid
Credit 9,331.20
Device peak FLOPS 1.39 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
09:06:49 (20627): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:22:53 (20667): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
11:22:54 (20667): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
02:20:06 (21418): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:35:13 (21517): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:03:23 (21627): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:25:52 (22002): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:19:29 (3210): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:47:04 (3327): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:47:05 (3327): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:48:30 (3538): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:48:32 (3538): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:12:07 (3824): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
20:52:03 (5092): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:07:54 (5161): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
06:33:43 (5618): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
00:48:01 (6448): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:49:25 (6564): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:56:20 (6646): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
07:56:21 (6646): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
14:01:56 (7233): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
23:29:56 (7823): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:11:18 (7884): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
05:12:46 (7954): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:54:54 (8798): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
06:35:21 (9163): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
06:35:23 (9163): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
17:36:44 (9421): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:52:31 (10434): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
13:28:05 (11796): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:30:08 (11899): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:13:37 (11974): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:23:28 (12157): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:24:52 (12229): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:27:22 (12305): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
02:28:45 (12394): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
14:27:00 (13182): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:27:22 (13182): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:48:50 (13319): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
13:10:02 (14352): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:11:21 (14405): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:27:57 (14469): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
01:51:53 (2281): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	11:12:06 AM	No files match the supplied pattern.
MainError:	11:12:06 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	09:35:33 AM	No files match the supplied pattern.
MainError:	09:35:33 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	08:10:07 AM	No files match the supplied pattern.
MainError:	08:10:07 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
16:16:23 (6265): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
16:16:24 (6265): No heartbeat from core client for 30 sec - exiting
16:16:25 (6265): No heartbeat from core client for 30 sec - exiting
16:16:26 (6265): No heartbeat from core client for 30 sec - exiting
16:16:27 (6265): No heartbeat from core client for 30 sec - exiting
16:16:28 (6265): No heartbeat from core client for 30 sec - exiting
16:16:29 (6265): No heartbeat from core client for 30 sec - exiting
16:16:30 (6265): No heartbeat from core client for 30 sec - exiting
16:16:31 (6265): No heartbeat from core client for 30 sec - exiting
16:16:32 (6265): No heartbeat from core client for 30 sec - exiting
16:16:33 (6265): No heartbeat from core client for 30 sec - exiting
16:16:34 (6265): No heartbeat from core client for 30 sec - exiting
16:16:35 (6265): No heartbeat from core client for 30 sec - exiting
16:16:36 (6265): No heartbeat from core client for 30 sec - exiting
16:16:37 (6265): No heartbeat from core client for 30 sec - exiting
MainError:	09:48:38 AM	No files match the supplied pattern.
MainError:	09:48:38 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	08:08:12 AM	No files match the supplied pattern.
MainError:	08:08:12 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	05:24:33 AM	No files match the supplied pattern.
MainError:	05:24:33 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	03:05:16 AM	No files match the supplied pattern.
MainError:	03:05:16 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	04:22:08 AM	No files match the supplied pattern.
MainError:	04:22:08 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	02:50:57 AM	No files match the supplied pattern.
MainError:	02:50:57 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	02:36:16 AM	No files match the supplied pattern.
MainError:	02:36:16 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
Error converting file to netcdf: dataout/o14rka.ph11c10
Error converting file to netcdf: dataout/o14rka.pg11c10
Error converting file to netcdf: dataout/o14rka.pe11c10
MainError:	01:42:39 AM	No files match the supplied pattern.
MainError:	01:42:39 AM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
21:17:19 (18803): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
21:17:20 (18803): No heartbeat from core client for 30 sec - exiting
21:17:21 (18803): No heartbeat from core client for 30 sec - exiting
21:17:22 (18803): No heartbeat from core client for 30 sec - exiting
21:17:23 (18803): No heartbeat from core client for 30 sec - exiting
21:17:24 (18803): No heartbeat from core client for 30 sec - exiting
21:17:25 (18803): No heartbeat from core client for 30 sec - exiting
21:17:26 (18803): No heartbeat from core client for 30 sec - exiting
21:17:27 (18803): No heartbeat from core client for 30 sec - exiting
21:17:28 (18803): No heartbeat from core client for 30 sec - exiting
21:17:29 (18803): No heartbeat from core client for 30 sec - exiting
21:17:30 (18803): No heartbeat from core client for 30 sec - exiting

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
30 Jan 2013 01:47:14 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 777,600 2,191,660 2.8185
29 Jan 2013 02:38:33 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 751,680 2,118,554 2.8184
28 Jan 2013 03:28:41 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 725,760 2,045,673 2.8187
27 Jan 2013 04:25:09 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 699,840 1,972,685 2.8188
26 Jan 2013 03:59:10 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 673,920 1,899,651 2.8188
25 Jan 2013 05:24:45 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 648,000 1,827,010 2.8195
24 Jan 2013 08:11:01 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 622,080 1,754,386 2.8202
23 Jan 2013 10:38:32 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 596,160 1,681,654 2.8208
22 Jan 2013 08:45:29 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 570,240 1,607,201 2.8185
21 Jan 2013 09:36:29 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 544,320 1,534,173 2.8185
20 Jan 2013 11:15:05 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 518,400 1,461,868 2.8200
19 Jan 2013 12:50:11 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 492,480 1,388,991 2.8204
18 Jan 2013 14:15:35 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 466,560 1,316,178 2.8210
17 Jan 2013 14:31:08 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 440,640 1,243,436 2.8219
16 Jan 2013 15:56:09 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 414,720 1,170,876 2.8233
15 Jan 2013 15:35:55 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 388,800 1,098,107 2.8243
14 Jan 2013 19:43:15 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 362,880 1,025,848 2.8270
13 Jan 2013 19:47:17 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 336,960 953,096 2.8285
12 Jan 2013 22:37:28 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 311,040 880,508 2.8309
11 Jan 2013 18:56:02 1157390 15502620 hadcm3n_o14r_2140_40_008269738_0 285,120 807,498 2.8321


©2024 cpdn.org