climateprediction.net home page
Task 13626512

Task 13626512

Name hadcm3n_t2sj_1940_40_007543350_1
Workunit 7740582
Created 10 Nov 2011, 1:22:29 UTC
Sent 16 Nov 2011, 9:25:59 UTC
Report deadline 15 Feb 2012, 16:53:10 UTC
Received 24 Nov 2011, 20:10:21 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1132282
Run time 6 days 4 hours 2 min 6 sec
CPU time 5 days 11 hours 3 min 56 sec
Validate state Invalid
Credit 933.12
Device peak FLOPS 0.68 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.34</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
21:33:19 (11564): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:22:46 (12812): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:25:02 (12812): No heartbeat from core client for 30 sec - exiting
03:34:38 (13237): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:46:14 (13263): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:59:23 (13289): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:09:03 (13297): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:23:14 (13324): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:38:32 (13350): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:05:52 (13378): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:32:48 (13420): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:41:35 (13464): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:26:01 (13472): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:04:59 (14107): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:19:37 (14168): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:30:40 (14264): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:40:44 (14290): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:50:18 (14299): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:00:53 (14325): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:38:47 (14423): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:56:35 (18362): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:41:50 (18389): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:41:51 (18389): No heartbeat from core client for 30 sec - exiting
05:41:52 (18389): No heartbeat from core client for 30 sec - exiting
06:01:13 (18451): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:31:50 (18495): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:40:40 (18612): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:40:41 (18612): No heartbeat from core client for 30 sec - exiting
07:40:42 (18612): No heartbeat from core client for 30 sec - exiting
07:40:43 (18612): No heartbeat from core client for 30 sec - exiting
07:40:44 (18612): No heartbeat from core client for 30 sec - exiting
07:40:45 (18612): No heartbeat from core client for 30 sec - exiting
07:40:46 (18612): No heartbeat from core client for 30 sec - exiting
07:40:47 (18612): No heartbeat from core client for 30 sec - exiting
07:40:48 (18612): No heartbeat from core client for 30 sec - exiting
07:40:49 (18612): No heartbeat from core client for 30 sec - exiting
07:40:50 (18612): No heartbeat from core client for 30 sec - exiting
07:40:51 (18612): No heartbeat from core client for 30 sec - exiting
07:40:52 (18612): No heartbeat from core client for 30 sec - exiting
08:08:03 (18620): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:26:14 (18665): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:26:15 (18665): No heartbeat from core client for 30 sec - exiting
08:26:16 (18665): No heartbeat from core client for 30 sec - exiting
08:26:17 (18665): No heartbeat from core client for 30 sec - exiting
08:26:18 (18665): No heartbeat from core client for 30 sec - exiting
08:26:19 (18665): No heartbeat from core client for 30 sec - exiting
08:26:20 (18665): No heartbeat from core client for 30 sec - exiting
08:26:21 (18665): No heartbeat from core client for 30 sec - exiting
08:26:22 (18665): No heartbeat from core client for 30 sec - exiting
08:26:23 (18665): No heartbeat from core client for 30 sec - exiting
08:26:24 (18665): No heartbeat from core client for 30 sec - exiting
08:26:25 (18665): No heartbeat from core client for 30 sec - exiting
08:26:26 (18665): No heartbeat from core client for 30 sec - exiting
08:26:27 (18665): No heartbeat from core client for 30 sec - exiting
08:26:28 (18665): No heartbeat from core client for 30 sec - exiting
08:26:29 (18665): No heartbeat from core client for 30 sec - exiting
08:26:30 (18665): No heartbeat from core client for 30 sec - exiting
08:26:31 (18665): No heartbeat from core client for 30 sec - exiting
08:26:32 (18665): No heartbeat from core client for 30 sec - exiting
08:26:33 (18665): No heartbeat from core client for 30 sec - exiting
08:26:34 (18665): No heartbeat from core client for 30 sec - exiting
08:26:35 (18665): No heartbeat from core client for 30 sec - exiting
08:26:36 (18665): No heartbeat from core client for 30 sec - exiting
08:26:37 (18665): No heartbeat from core client for 30 sec - exiting
08:26:38 (18665): No heartbeat from core client for 30 sec - exiting
08:26:39 (18665): No heartbeat from core client for 30 sec - exiting
08:26:40 (18665): No heartbeat from core client for 30 sec - exiting
08:26:41 (18665): No heartbeat from core client for 30 sec - exiting
08:26:42 (18665): No heartbeat from core client for 30 sec - exiting
08:26:43 (18665): No heartbeat from core client for 30 sec - exiting
08:44:27 (18691): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:12:28 (19001): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:39:31 (19045): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:16:40 (19093): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 15 received, exiting...
Called boinc_finish
12:53:06 (2675): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:28:46 (2874): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:37:55 (2919): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:22:47 (2945): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:24:52 (3009): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:51:18 (4535): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:26:29 (4580): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:52:33 (4698): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:10:13 (4742): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:45:30 (5193): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:58:46 (5255): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:58:47 (5255): No heartbeat from core client for 30 sec - exiting
14:58:48 (5255): No heartbeat from core client for 30 sec - exiting
14:58:49 (5255): No heartbeat from core client for 30 sec - exiting
14:58:50 (5255): No heartbeat from core client for 30 sec - exiting
14:58:51 (5255): No heartbeat from core client for 30 sec - exiting
14:58:52 (5255): No heartbeat from core client for 30 sec - exiting
14:58:53 (5255): No heartbeat from core client for 30 sec - exiting
14:58:54 (5255): No heartbeat from core client for 30 sec - exiting
14:58:55 (5255): No heartbeat from core client for 30 sec - exiting
14:58:56 (5255): No heartbeat from core client for 30 sec - exiting
14:58:57 (5255): No heartbeat from core client for 30 sec - exiting
15:16:24 (5554): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:51:09 (5598): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:51:10 (5598): No heartbeat from core client for 30 sec - exiting
15:51:11 (5598): No heartbeat from core client for 30 sec - exiting
15:51:12 (5598): No heartbeat from core client for 30 sec - exiting
15:51:13 (5598): No heartbeat from core client for 30 sec - exiting
15:51:14 (5598): No heartbeat from core client for 30 sec - exiting
15:51:15 (5598): No heartbeat from core client for 30 sec - exiting
15:51:16 (5598): No heartbeat from core client for 30 sec - exiting
16:00:07 (5642): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:53:30 (5668): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:53:31 (5668): No heartbeat from core client for 30 sec - exiting
16:53:32 (5668): No heartbeat from core client for 30 sec - exiting
16:53:33 (5668): No heartbeat from core client for 30 sec - exiting
21:16:01 (5731): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:00:13 (6069): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:00:14 (6069): No heartbeat from core client for 30 sec - exiting
22:00:15 (6069): No heartbeat from core client for 30 sec - exiting
22:00:16 (6069): No heartbeat from core client for 30 sec - exiting
22:00:17 (6069): No heartbeat from core client for 30 sec - exiting
22:00:18 (6069): No heartbeat from core client for 30 sec - exiting
22:00:19 (6069): No heartbeat from core client for 30 sec - exiting
22:00:20 (6069): No heartbeat from core client for 30 sec - exiting
22:00:21 (6069): No heartbeat from core client for 30 sec - exiting
22:00:22 (6069): No heartbeat from core client for 30 sec - exiting
22:00:23 (6069): No heartbeat from core client for 30 sec - exiting
22:00:24 (6069): No heartbeat from core client for 30 sec - exiting
22:00:25 (6069): No heartbeat from core client for 30 sec - exiting
22:00:26 (6069): No heartbeat from core client for 30 sec - exiting
22:18:03 (6415): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:31:10 (6441): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:57:17 (10578): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:14:46 (10604): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:58:15 (10630): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:15:43 (10693): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:33:59 (10737): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
19:51:16 (10908): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
20:34:20 (10934): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
21:09:41 (10997): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
23 Nov 2011 18:50:26 1132282 13626512 hadcm3n_t2sj_1940_40_007543350_1 77,760 467,924 6.0175
21 Nov 2011 16:30:35 1132282 13626512 hadcm3n_t2sj_1940_40_007543350_1 51,840 309,642 5.9730
18 Nov 2011 13:15:23 1132282 13626512 hadcm3n_t2sj_1940_40_007543350_1 25,920 157,035 6.0584


©2024 climateprediction.net