climateprediction.net home page
Task 15559777

Task 15559777

Name hadcm3n_o53h_2140_40_008270456_3
Workunit 8425580
Created 28 Jan 2013, 11:48:23 UTC
Sent 28 Jan 2013, 11:48:30 UTC
Report deadline 29 Apr 2013, 19:15:41 UTC
Received 8 Feb 2013, 2:06:34 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1148818
Run time 9 days 0 hours 7 min 27 sec
CPU time 5 days 23 hours 49 min 47 sec
Validate state Invalid
Credit 3,110.40
Device peak FLOPS 2.29 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
eat from core client for 30 sec - exiting
22:22:37 (19732): No heartbeat from core client for 30 sec - exiting
22:22:38 (19732): No heartbeat from core client for 30 sec - exiting
22:22:39 (19732): No heartbeat from core client for 30 sec - exiting
22:22:40 (19732): No heartbeat from core client for 30 sec - exiting
22:22:41 (19732): No heartbeat from core client for 30 sec - exiting
22:22:42 (19732): No heartbeat from core client for 30 sec - exiting
22:22:43 (19732): No heartbeat from core client for 30 sec - exiting
22:22:44 (19732): No heartbeat from core client for 30 sec - exiting
22:22:45 (19732): No heartbeat from core client for 30 sec - exiting
22:22:46 (19732): No heartbeat from core client for 30 sec - exiting
22:22:47 (19732): No heartbeat from core client for 30 sec - exiting
22:22:48 (19732): No heartbeat from core client for 30 sec - exiting
22:22:49 (19732): No heartbeat from core client for 30 sec - exiting
22:22:50 (19732): No heartbeat from core client for 30 sec - exiting
22:22:51 (19732): No heartbeat from core client for 30 sec - exiting
22:22:52 (19732): No heartbeat from core client for 30 sec - exiting
22:22:53 (19732): No heartbeat from core client for 30 sec - exiting
22:22:54 (19732): No heartbeat from core client for 30 sec - exiting
22:22:55 (19732): No heartbeat from core client for 30 sec - exiting
22:22:56 (19732): No heartbeat from core client for 30 sec - exiting
22:22:57 (19732): No heartbeat from core client for 30 sec - exiting
22:22:58 (19732): No heartbeat from core client for 30 sec - exiting
22:22:59 (19732): No heartbeat from core client for 30 sec - exiting
22:23:00 (19732): No heartbeat from core client for 30 sec - exiting
22:23:01 (19732): No heartbeat from core client for 30 sec - exiting
22:23:02 (19732): No heartbeat from core client for 30 sec - exiting
22:23:03 (19732): No heartbeat from core client for 30 sec - exiting
22:23:04 (19732): No heartbeat from core client for 30 sec - exiting
22:23:05 (19732): No heartbeat from core client for 30 sec - exiting
22:23:06 (19732): No heartbeat from core client for 30 sec - exiting
22:23:07 (19732): No heartbeat from core client for 30 sec - exiting
22:23:08 (19732): No heartbeat from core client for 30 sec - exiting
22:23:09 (19732): No heartbeat from core client for 30 sec - exiting
22:23:10 (19732): No heartbeat from core client for 30 sec - exiting
22:23:11 (19732): No heartbeat from core client for 30 sec - exiting
22:23:12 (19732): No heartbeat from core client for 30 sec - exiting
22:23:13 (19732): No heartbeat from core client for 30 sec - exiting
22:23:14 (19732): No heartbeat from core client for 30 sec - exiting
22:23:15 (19732): No heartbeat from core client for 30 sec - exiting
22:23:16 (19732): No heartbeat from core client for 30 sec - exiting
22:23:18 (19732): No heartbeat from core client for 30 sec - exiting
22:23:19 (19732): No heartbeat from core client for 30 sec - exiting
22:23:20 (19732): No heartbeat from core client for 30 sec - exiting
22:23:21 (19732): No heartbeat from core client for 30 sec - exiting
22:23:22 (19732): No heartbeat from core client for 30 sec - exiting
22:23:23 (19732): No heartbeat from core client for 30 sec - exiting
22:23:24 (19732): No heartbeat from core client for 30 sec - exiting
22:23:25 (19732): No heartbeat from core client for 30 sec - exiting
22:23:26 (19732): No heartbeat from core client for 30 sec - exiting
22:23:27 (19732): No heartbeat from core client for 30 sec - exiting
22:23:28 (19732): No heartbeat from core client for 30 sec - exiting
22:23:29 (19732): No heartbeat from core client for 30 sec - exiting
22:23:30 (19732): No heartbeat from core client for 30 sec - exiting
22:23:31 (19732): No heartbeat from core client for 30 sec - exiting
22:23:32 (19732): No heartbeat from core client for 30 sec - exiting
22:23:33 (19732): No heartbeat from core client for 30 sec - exiting
22:23:34 (19732): No heartbeat from core client for 30 sec - exiting
22:23:35 (19732): No heartbeat from core client for 30 sec - exiting
22:23:36 (19732): No heartbeat from core client for 30 sec - exiting
22:23:37 (19732): No heartbeat from core client for 30 sec - exiting
22:23:38 (19732): No heartbeat from core client for 30 sec - exiting
22:23:39 (19732): No heartbeat from core client for 30 sec - exiting
22:23:40 (19732): No heartbeat from core client for 30 sec - exiting
22:23:41 (19732): No heartbeat from core client for 30 sec - exiting
22:23:42 (19732): No heartbeat from core client for 30 sec - exiting
22:23:43 (19732): No heartbeat from core client for 30 sec - exiting
22:23:44 (19732): No heartbeat from core client for 30 sec - exiting
22:23:45 (19732): No heartbeat from core client for 30 sec - exiting
22:23:46 (19732): No heartbeat from core client for 30 sec - exiting
22:23:47 (19732): No heartbeat from core client for 30 sec - exiting
22:23:49 (19732): No heartbeat from core client for 30 sec - exiting
22:23:50 (19732): No heartbeat from core client for 30 sec - exiting
22:23:51 (19732): No heartbeat from core client for 30 sec - exiting
22:23:52 (19732): No heartbeat from core client for 30 sec - exiting
22:23:53 (19732): No heartbeat from core client for 30 sec - exiting
22:23:54 (19732): No heartbeat from core client for 30 sec - exiting
22:23:55 (19732): No heartbeat from core client for 30 sec - exiting
22:23:56 (19732): No heartbeat from core client for 30 sec - exiting
22:23:57 (19732): No heartbeat from core client for 30 sec - exiting
22:23:58 (19732): No heartbeat from core client for 30 sec - exiting
22:23:59 (19732): No heartbeat from core client for 30 sec - exiting
22:24:00 (19732): No heartbeat from core client for 30 sec - exiting
22:24:01 (19732): No heartbeat from core client for 30 sec - exiting
22:24:02 (19732): No heartbeat from core client for 30 sec - exiting
22:24:03 (19732): No heartbeat from core client for 30 sec - exiting
22:24:04 (19732): No heartbeat from core client for 30 sec - exiting
22:24:05 (19732): No heartbeat from core client for 30 sec - exiting
22:24:06 (19732): No heartbeat from core client for 30 sec - exiting
22:24:07 (19732): No heartbeat from core client for 30 sec - exiting
22:24:08 (19732): No heartbeat from core client for 30 sec - exiting
22:24:09 (19732): No heartbeat from core client for 30 sec - exiting
22:24:10 (19732): No heartbeat from core client for 30 sec - exiting
22:24:11 (19732): No heartbeat from core client for 30 sec - exiting
22:24:12 (19732): No heartbeat from core client for 30 sec - exiting
22:24:13 (19732): No heartbeat from core client for 30 sec - exiting
22:24:14 (19732): No heartbeat from core client for 30 sec - exiting
22:24:15 (19732): No heartbeat from core client for 30 sec - exiting
22:24:16 (19732): No heartbeat from core client for 30 sec - exiting
22:24:17 (19732): No heartbeat from core client for 30 sec - exiting
22:24:18 (19732): No heartbeat from core client for 30 sec - exiting
22:24:19 (19732): No heartbeat from core client for 30 sec - exiting
22:24:20 (19732): No heartbeat from core client for 30 sec - exiting
22:24:21 (19732): No heartbeat from core client for 30 sec - exiting
22:24:22 (19732): No heartbeat from core client for 30 sec - exiting
22:24:23 (19732): No heartbeat from core client for 30 sec - exiting
22:24:24 (19732): No heartbeat from core client for 30 sec - exiting
22:24:25 (19732): No heartbeat from core client for 30 sec - exiting
22:24:26 (19732): No heartbeat from core client for 30 sec - exiting
22:24:27 (19732): No heartbeat from core client for 30 sec - exiting
22:24:28 (19732): No heartbeat from core client for 30 sec - exiting
22:24:29 (19732): No heartbeat from core client for 30 sec - exiting
22:24:30 (19732): No heartbeat from core client for 30 sec - exiting
22:24:31 (19732): No heartbeat from core client for 30 sec - exiting
22:24:32 (19732): No heartbeat from core client for 30 sec - exiting
22:24:33 (19732): No heartbeat from core client for 30 sec - exiting
22:24:34 (19732): No heartbeat from core client for 30 sec - exiting
22:24:35 (19732): No heartbeat from core client for 30 sec - exiting
22:24:36 (19732): No heartbeat from core client for 30 sec - exiting
22:24:37 (19732): No heartbeat from core client for 30 sec - exiting
22:24:38 (19732): No heartbeat from core client for 30 sec - exiting
22:24:39 (19732): No heartbeat from core client for 30 sec - exiting
22:24:40 (19732): No heartbeat from core client for 30 sec - exiting
22:24:41 (19732): No heartbeat from core client for 30 sec - exiting
22:24:42 (19732): No heartbeat from core client for 30 sec - exiting
22:24:43 (19732): No heartbeat from core client for 30 sec - exiting
22:24:44 (19732): No heartbeat from core client for 30 sec - exiting
22:24:45 (19732): No heartbeat from core client for 30 sec - exiting
22:24:46 (19732): No heartbeat from core client for 30 sec - exiting
22:24:47 (19732): No heartbeat from core client for 30 sec - exiting
22:24:48 (19732): No heartbeat from core client for 30 sec - exiting
22:24:49 (19732): No heartbeat from core client for 30 sec - exiting
22:24:50 (19732): No heartbeat from core client for 30 sec - exiting
22:24:51 (19732): No heartbeat from core client for 30 sec - exiting
22:24:52 (19732): No heartbeat from core client for 30 sec - exiting
22:24:53 (19732): No heartbeat from core client for 30 sec - exiting
22:24:54 (19732): No heartbeat from core client for 30 sec - exiting
22:24:55 (19732): No heartbeat from core client for 30 sec - exiting
22:24:56 (19732): No heartbeat from core client for 30 sec - exiting
22:24:57 (19732): No heartbeat from core client for 30 sec - exiting
22:24:58 (19732): No heartbeat from core client for 30 sec - exiting
22:24:59 (19732): No heartbeat from core client for 30 sec - exiting
22:25:00 (19732): No heartbeat from core client for 30 sec - exiting
22:25:01 (19732): No heartbeat from core client for 30 sec - exiting
22:25:02 (19732): No heartbeat from core client for 30 sec - exiting
22:25:03 (19732): No heartbeat from core client for 30 sec - exiting
22:25:04 (19732): No heartbeat from core client for 30 sec - exiting
22:25:05 (19732): No heartbeat from core client for 30 sec - exiting
22:25:06 (19732): No heartbeat from core client for 30 sec - exiting
22:25:07 (19732): No heartbeat from core client for 30 sec - exiting
22:25:08 (19732): No heartbeat from core client for 30 sec - exiting
22:25:09 (19732): No heartbeat from core client for 30 sec - exiting
22:25:10 (19732): No heartbeat from core client for 30 sec - exiting
22:25:11 (19732): No heartbeat from core client for 30 sec - exiting
22:25:12 (19732): No heartbeat from core client for 30 sec - exiting
22:25:13 (19732): No heartbeat from core client for 30 sec - exiting
22:25:14 (19732): No heartbeat from core client for 30 sec - exiting
22:25:15 (19732): No heartbeat from core client for 30 sec - exiting
22:25:16 (19732): No heartbeat from core client for 30 sec - exiting
22:25:17 (19732): No heartbeat from core client for 30 sec - exiting
22:25:18 (19732): No heartbeat from core client for 30 sec - exiting
22:25:19 (19732): No heartbeat from core client for 30 sec - exiting
22:25:20 (19732): No heartbeat from core client for 30 sec - exiting
22:25:21 (19732): No heartbeat from core client for 30 sec - exiting
22:25:22 (19732): No heartbeat from core client for 30 sec - exiting
22:25:23 (19732): No heartbeat from core client for 30 sec - exiting
22:25:24 (19732): No heartbeat from core client for 30 sec - exiting
22:25:25 (19732): No heartbeat from core client for 30 sec - exiting
22:25:26 (19732): No heartbeat from core client for 30 sec - exiting
22:25:27 (19732): No heartbeat from core client for 30 sec - exiting
22:25:28 (19732): No heartbeat from core client for 30 sec - exiting
22:25:29 (19732): No heartbeat from core client for 30 sec - exiting
22:25:30 (19732): No heartbeat from core client for 30 sec - exiting
22:25:31 (19732): No heartbeat from core client for 30 sec - exiting
22:25:32 (19732): No heartbeat from core client for 30 sec - exiting
22:25:33 (19732): No heartbeat from core client for 30 sec - exiting
22:25:34 (19732): No heartbeat from core client for 30 sec - exiting
22:25:35 (19732): No heartbeat from core client for 30 sec - exiting
22:25:36 (19732): No heartbeat from core client for 30 sec - exiting
22:25:37 (19732): No heartbeat from core client for 30 sec - exiting
22:25:38 (19732): No heartbeat from core client for 30 sec - exiting
22:25:39 (19732): No heartbeat from core client for 30 sec - exiting
22:25:40 (19732): No heartbeat from core client for 30 sec - exiting
22:25:41 (19732): No heartbeat from core client for 30 sec - exiting
22:25:42 (19732): No heartbeat from core client for 30 sec - exiting
22:25:43 (19732): No heartbeat from core client for 30 sec - exiting
22:25:45 (19732): No heartbeat from core client for 30 sec - exiting
22:25:46 (19732): No heartbeat from core client for 30 sec - exiting
22:25:47 (19732): No heartbeat from core client for 30 sec - exiting
22:25:48 (19732): No heartbeat from core client for 30 sec - exiting
22:25:49 (19732): No heartbeat from core client for 30 sec - exiting
22:25:50 (19732): No heartbeat from core client for 30 sec - exiting
22:25:51 (19732): No heartbeat from core client for 30 sec - exiting
22:25:52 (19732): No heartbeat from core client for 30 sec - exiting
22:25:53 (19732): No heartbeat from core client for 30 sec - exiting
22:25:54 (19732): No heartbeat from core client for 30 sec - exiting
22:25:55 (19732): No heartbeat from core client for 30 sec - exiting
22:25:56 (19732): No heartbeat from core client for 30 sec - exiting
22:25:57 (19732): No heartbeat from core client for 30 sec - exiting
22:25:58 (19732): No heartbeat from core client for 30 sec - exiting
22:25:59 (19732): No heartbeat from core client for 30 sec - exiting
22:26:00 (19732): No heartbeat from core client for 30 sec - exiting
22:26:01 (19732): No heartbeat from core client for 30 sec - exiting
22:26:02 (19732): No heartbeat from core client for 30 sec - exiting
22:26:03 (19732): No heartbeat from core client for 30 sec - exiting
22:26:04 (19732): No heartbeat from core client for 30 sec - exiting
22:26:05 (19732): No heartbeat from core client for 30 sec - exiting
22:26:06 (19732): No heartbeat from core client for 30 sec - exiting
22:26:07 (19732): No heartbeat from core client for 30 sec - exiting
22:26:08 (19732): No heartbeat from core client for 30 sec - exiting
22:26:09 (19732): No heartbeat from core client for 30 sec - exiting
22:26:10 (19732): No heartbeat from core client for 30 sec - exiting
22:26:11 (19732): No heartbeat from core client for 30 sec - exiting
22:26:12 (19732): No heartbeat from core client for 30 sec - exiting
22:26:13 (19732): No heartbeat from core client for 30 sec - exiting
22:26:14 (19732): No heartbeat from core client for 30 sec - exiting
22:26:15 (19732): No heartbeat from core client for 30 sec - exiting
22:26:16 (19732): No heartbeat from core client for 30 sec - exiting
22:26:17 (19732): No heartbeat from core client for 30 sec - exiting
22:26:18 (19732): No heartbeat from core client for 30 sec - exiting
22:26:19 (19732): No heartbeat from core client for 30 sec - exiting
22:26:20 (19732): No heartbeat from core client for 30 sec - exiting
22:26:21 (19732): No heartbeat from core client for 30 sec - exiting
22:26:22 (19732): No heartbeat from core client for 30 sec - exiting
22:26:23 (19732): No heartbeat from core client for 30 sec - exiting
22:26:24 (19732): No heartbeat from core client for 30 sec - exiting
22:26:25 (19732): No heartbeat from core client for 30 sec - exiting
22:26:26 (19732): No heartbeat from core client for 30 sec - exiting
22:26:27 (19732): No heartbeat from core client for 30 sec - exiting
22:26:28 (19732): No heartbeat from core client for 30 sec - exiting
22:26:29 (19732): No heartbeat from core client for 30 sec - exiting
22:26:30 (19732): No heartbeat from core client for 30 sec - exiting
22:26:31 (19732): No heartbeat from core client for 30 sec - exiting
22:26:32 (19732): No heartbeat from core client for 30 sec - exiting
22:26:33 (19732): No heartbeat from core client for 30 sec - exiting
22:26:34 (19732): No heartbeat from core client for 30 sec - exiting
22:26:35 (19732): No heartbeat from core client for 30 sec - exiting
22:26:36 (19732): No heartbeat from core client for 30 sec - exiting
22:26:37 (19732): No heartbeat from core client for 30 sec - exiting
22:26:38 (19732): No heartbeat from core client for 30 sec - exiting
22:26:39 (19732): No heartbeat from core client for 30 sec - exiting
22:26:40 (19732): No heartbeat from core client for 30 sec - exiting
22:26:41 (19732): No heartbeat from core client for 30 sec - exiting
22:26:42 (19732): No heartbeat from core client for 30 sec - exiting
22:26:43 (19732): No heartbeat from core client for 30 sec - exiting
22:26:44 (19732): No heartbeat from core client for 30 sec - exiting
22:26:45 (19732): No heartbeat from core client for 30 sec - exiting
22:26:46 (19732): No heartbeat from core client for 30 sec - exiting
22:26:47 (19732): No heartbeat from core client for 30 sec - exiting
22:26:48 (19732): No heartbeat from core client for 30 sec - exiting
22:26:49 (19732): No heartbeat from core client for 30 sec - exiting
22:26:50 (19732): No heartbeat from core client for 30 sec - exiting
22:26:51 (19732): No heartbeat from core client for 30 sec - exiting
22:26:52 (19732): No heartbeat from core client for 30 sec - exiting
22:26:53 (19732): No heartbeat from core client for 30 sec - exiting
22:26:54 (19732): No heartbeat from core client for 30 sec - exiting
22:26:55 (19732): No heartbeat from core client for 30 sec - exiting
22:26:56 (19732): No heartbeat from core client for 30 sec - exiting
22:26:57 (19732): No heartbeat from core client for 30 sec - exiting
22:26:58 (19732): No heartbeat from core client for 30 sec - exiting
22:26:59 (19732): No heartbeat from core client for 30 sec - exiting
22:27:00 (19732): No heartbeat from core client for 30 sec - exiting
22:27:01 (19732): No heartbeat from core client for 30 sec - exiting
22:27:02 (19732): No heartbeat from core client for 30 sec - exiting
22:27:03 (19732): No heartbeat from core client for 30 sec - exiting
22:27:04 (19732): No heartbeat from core client for 30 sec - exiting
22:27:05 (19732): No heartbeat from core client for 30 sec - exiting
22:27:06 (19732): No heartbeat from core client for 30 sec - exiting
22:27:07 (19732): No heartbeat from core client for 30 sec - exiting
22:27:08 (19732): No heartbeat from core client for 30 sec - exiting
22:27:09 (19732): No heartbeat from core client for 30 sec - exiting
22:27:10 (19732): No heartbeat from core client for 30 sec - exiting
22:27:11 (19732): No heartbeat from core client for 30 sec - exiting
22:27:12 (19732): No heartbeat from core client for 30 sec - exiting
22:27:13 (19732): No heartbeat from core client for 30 sec - exiting
22:27:14 (19732): No heartbeat from core client for 30 sec - exiting
22:27:15 (19732): No heartbeat from core client for 30 sec - exiting
22:27:16 (19732): No heartbeat from core client for 30 sec - exiting
22:27:17 (19732): No heartbeat from core client for 30 sec - exiting
22:27:18 (19732): No heartbeat from core client for 30 sec - exiting
22:27:19 (19732): No heartbeat from core client for 30 sec - exiting
22:27:20 (19732): No heartbeat from core client for 30 sec - exiting
22:27:21 (19732): No heartbeat from core client for 30 sec - exiting
22:27:22 (19732): No heartbeat from core client for 30 sec - exiting
22:27:23 (19732): No heartbeat from core client for 30 sec - exiting
22:27:24 (19732): No heartbeat from core client for 30 sec - exiting
22:27:26 (19732): No heartbeat from core client for 30 sec - exiting
22:27:27 (19732): No heartbeat from core client for 30 sec - exiting
22:27:28 (19732): No heartbeat from core client for 30 sec - exiting
22:27:29 (19732): No heartbeat from core client for 30 sec - exiting
22:27:30 (19732): No heartbeat from core client for 30 sec - exiting
22:27:31 (19732): No heartbeat from core client for 30 sec - exiting
22:27:32 (19732): No heartbeat from core client for 30 sec - exiting
22:27:33 (19732): No heartbeat from core client for 30 sec - exiting
22:27:34 (19732): No heartbeat from core client for 30 sec - exiting
22:27:35 (19732): No heartbeat from core client for 30 sec - exiting
22:27:36 (19732): No heartbeat from core client for 30 sec - exiting
22:27:37 (19732): No heartbeat from core client for 30 sec - exiting
22:27:38 (19732): No heartbeat from core client for 30 sec - exiting
22:27:39 (19732): No heartbeat from core client for 30 sec - exiting
22:27:40 (19732): No heartbeat from core client for 30 sec - exiting
22:27:41 (19732): No heartbeat from core client for 30 sec - exiting
22:27:42 (19732): No heartbeat from core client for 30 sec - exiting
22:27:43 (19732): No heartbeat from core client for 30 sec - exiting
22:27:44 (19732): No heartbeat from core client for 30 sec - exiting
22:27:45 (19732): No heartbeat from core client for 30 sec - exiting
22:27:46 (19732): No heartbeat from core client for 30 sec - exiting
22:27:47 (19732): No heartbeat from core client for 30 sec - exiting
22:27:48 (19732): No heartbeat from core client for 30 sec - exiting
22:27:49 (19732): No heartbeat from core client for 30 sec - exiting
22:27:50 (19732): No heartbeat from core client for 30 sec - exiting
22:27:51 (19732): No heartbeat from core client for 30 sec - exiting
22:27:52 (19732): No heartbeat from core client for 30 sec - exiting
22:27:53 (19732): No heartbeat from core client for 30 sec - exiting
22:27:54 (19732): No heartbeat from core client for 30 sec - exiting
22:27:55 (19732): No heartbeat from core client for 30 sec - exiting
22:27:56 (19732): No heartbeat from core client for 30 sec - exiting
22:27:57 (19732): No heartbeat from core client for 30 sec - exiting
22:27:58 (19732): No heartbeat from core client for 30 sec - exiting
22:27:59 (19732): No heartbeat from core client for 30 sec - exiting
22:28:00 (19732): No heartbeat from core client for 30 sec - exiting
22:28:01 (19732): No heartbeat from core client for 30 sec - exiting
22:28:02 (19732): No heartbeat from core client for 30 sec - exiting
22:28:03 (19732): No heartbeat from core client for 30 sec - exiting
22:28:04 (19732): No heartbeat from core client for 30 sec - exiting
22:28:05 (19732): No heartbeat from core client for 30 sec - exiting
22:28:06 (19732): No heartbeat from core client for 30 sec - exiting
22:28:07 (19732): No heartbeat from core client for 30 sec - exiting
22:28:08 (19732): No heartbeat from core client for 30 sec - exiting
22:28:09 (19732): No heartbeat from core client for 30 sec - exiting
22:28:10 (19732): No heartbeat from core client for 30 sec - exiting
22:28:11 (19732): No heartbeat from core client for 30 sec - exiting
22:28:12 (19732): No heartbeat from core client for 30 sec - exiting
22:28:13 (19732): No heartbeat from core client for 30 sec - exiting
22:28:14 (19732): No heartbeat from core client for 30 sec - exiting
22:28:15 (19732): No heartbeat from core client for 30 sec - exiting
22:28:16 (19732): No heartbeat from core client for 30 sec - exiting
22:28:17 (19732): No heartbeat from core client for 30 sec - exiting
22:28:18 (19732): No heartbeat from core client for 30 sec - exiting
22:28:19 (19732): No heartbeat from core client for 30 sec - exiting
22:28:20 (19732): No heartbeat from core client for 30 sec - exiting
22:28:21 (19732): No heartbeat from core client for 30 sec - exiting
22:28:22 (19732): No heartbeat from core client for 30 sec - exiting
22:28:23 (19732): No heartbeat from core client for 30 sec - exiting
22:28:24 (19732): No heartbeat from core client for 30 sec - exiting
22:28:25 (19732): No heartbeat from core client for 30 sec - exiting
22:28:26 (19732): No heartbeat from core client for 30 sec - exiting
22:28:27 (19732): No heartbeat from core client for 30 sec - exiting
22:28:28 (19732): No heartbeat from core client for 30 sec - exiting
22:28:29 (19732): No heartbeat from core client for 30 sec - exiting
22:28:30 (19732): No heartbeat from core client for 30 sec - exiting
22:28:31 (19732): No heartbeat from core client for 30 sec - exiting
22:28:32 (19732): No heartbeat from core client for 30 sec - exiting
22:28:33 (19732): No heartbeat from core client for 30 sec - exiting
22:28:34 (19732): No heartbeat from core client for 30 sec - exiting
22:28:35 (19732): No heartbeat from core client for 30 sec - exiting
22:28:36 (19732): No heartbeat from core client for 30 sec - exiting
22:28:37 (19732): No heartbeat from core client for 30 sec - exiting
22:28:38 (19732): No heartbeat from core client for 30 sec - exiting
22:28:39 (19732): No heartbeat from core client for 30 sec - exiting
22:28:40 (19732): No heartbeat from core client for 30 sec - exiting
22:28:41 (19732): No heartbeat from core client for 30 sec - exiting
22:28:42 (19732): No heartbeat from core client for 30 sec - exiting
22:28:43 (19732): No heartbeat from core client for 30 sec - exiting
22:28:44 (19732): No heartbeat from core client for 30 sec - exiting
22:28:45 (19732): No heartbeat from core client for 30 sec - exiting
22:28:47 (19732): No heartbeat from core client for 30 sec - exiting
22:28:48 (19732): No heartbeat from core client for 30 sec - exiting
22:28:49 (19732): No heartbeat from core client for 30 sec - exiting
22:28:50 (19732): No heartbeat from core client for 30 sec - exiting
22:28:51 (19732): No heartbeat from core client for 30 sec - exiting
22:28:52 (19732): No heartbeat from core client for 30 sec - exiting
22:28:53 (19732): No heartbeat from core client for 30 sec - exiting
22:28:54 (19732): No heartbeat from core client for 30 sec - exiting
22:28:55 (19732): No heartbeat from core client for 30 sec - exiting
22:28:56 (19732): No heartbeat from core client for 30 sec - exiting
22:28:57 (19732): No heartbeat from core client for 30 sec - exiting
22:28:58 (19732): No heartbeat from core client for 30 sec - exiting
22:28:59 (19732): No heartbeat from core client for 30 sec - exiting
22:29:00 (19732): No heartbeat from core client for 30 sec - exiting
22:29:01 (19732): No heartbeat from core client for 30 sec - exiting
22:29:02 (19732): No heartbeat from core client for 30 sec - exiting
22:29:03 (19732): No heartbeat from core client for 30 sec - exiting
22:29:04 (19732): No heartbeat from core client for 30 sec - exiting
22:29:05 (19732): No heartbeat from core client for 30 sec - exiting
22:29:06 (19732): No heartbeat from core client for 30 sec - exiting
22:29:07 (19732): No heartbeat from core client for 30 sec - exiting
22:29:08 (19732): No heartbeat from core client for 30 sec - exiting
22:29:09 (19732): No heartbeat from core client for 30 sec - exiting
22:29:10 (19732): No heartbeat from core client for 30 sec - exiting
22:29:11 (19732): No heartbeat from core client for 30 sec - exiting
22:29:12 (19732): No heartbeat from core client for 30 sec - exiting
22:29:13 (19732): No heartbeat from core client for 30 sec - exiting
22:29:14 (19732): No heartbeat from core client for 30 sec - exiting
22:29:15 (19732): No heartbeat from core client for 30 sec - exiting
22:29:16 (19732): No heartbeat from core client for 30 sec - exiting
22:29:17 (19732): No heartbeat from core client for 30 sec - exiting
22:29:18 (19732): No heartbeat from core client for 30 sec - exiting
22:29:19 (19732): No heartbeat from core client for 30 sec - exiting
22:29:20 (19732): No heartbeat from core client for 30 sec - exiting
22:29:21 (19732): No heartbeat from core client for 30 sec - exiting
22:29:22 (19732): No heartbeat from core client for 30 sec - exiting
22:29:23 (19732): No heartbeat from core client for 30 sec - exiting
22:29:24 (19732): No heartbeat from core client for 30 sec - exiting
22:29:25 (19732): No heartbeat from core client for 30 sec - exiting
22:29:27 (19732): No heartbeat from core client for 30 sec - exiting
22:29:28 (19732): No heartbeat from core client for 30 sec - exiting
22:29:29 (19732): No heartbeat from core client for 30 sec - exiting
22:29:30 (19732): No heartbeat from core client for 30 sec - exiting
22:29:31 (19732): No heartbeat from core client for 30 sec - exiting
22:29:32 (19732): No heartbeat from core client for 30 sec - exiting
22:29:33 (19732): No heartbeat from core client for 30 sec - exiting
22:29:34 (19732): No heartbeat from core client for 30 sec - exiting
22:29:35 (19732): No heartbeat from core client for 30 sec - exiting
22:29:36 (19732): No heartbeat from core client for 30 sec - exiting
22:29:37 (19732): No heartbeat from core client for 30 sec - exiting
22:29:38 (19732): No heartbeat from core client for 30 sec - exiting
22:29:39 (19732): No heartbeat from core client for 30 sec - exiting
22:29:40 (19732): No heartbeat from core client for 30 sec - exiting
22:29:41 (19732): No heartbeat from core client for 30 sec - exiting
22:29:42 (19732): No heartbeat from core client for 30 sec - exiting
22:29:43 (19732): No heartbeat from core client for 30 sec - exiting
22:29:44 (19732): No heartbeat from core client for 30 sec - exiting
22:29:45 (19732): No heartbeat from core client for 30 sec - exiting
22:29:46 (19732): No heartbeat from core client for 30 sec - exiting
22:29:47 (19732): No heartbeat from core client for 30 sec - exiting
22:29:48 (19732): No heartbeat from core client for 30 sec - exiting
22:29:49 (19732): No heartbeat from core client for 30 sec - exiting
22:29:50 (19732): No heartbeat from core client for 30 sec - exiting
22:29:51 (19732): No heartbeat from core client for 30 sec - exiting
22:29:52 (19732): No heartbeat from core client for 30 sec - exiting
22:29:53 (19732): No heartbeat from core client for 30 sec - exiting
22:29:54 (19732): No heartbeat from core client for 30 sec - exiting
22:29:55 (19732): No heartbeat from core client for 30 sec - exiting
22:29:56 (19732): No heartbeat from core client for 30 sec - exiting
22:29:57 (19732): No heartbeat from core client for 30 sec - exiting
22:29:58 (19732): No heartbeat from core client for 30 sec - exiting
22:29:59 (19732): No heartbeat from core client for 30 sec - exiting
22:30:00 (19732): No heartbeat from core client for 30 sec - exiting
22:30:01 (19732): No heartbeat from core client for 30 sec - exiting
22:30:02 (19732): No heartbeat from core client for 30 sec - exiting
22:30:03 (19732): No heartbeat from core client for 30 sec - exiting
22:30:04 (19732): No heartbeat from core client for 30 sec - exiting
22:30:05 (19732): No heartbeat from core client for 30 sec - exiting
22:30:06 (19732): No heartbeat from core client for 30 sec - exiting
22:30:07 (19732): No heartbeat from core client for 30 sec - exiting
22:30:08 (19732): No heartbeat from core client for 30 sec - exiting
22:30:09 (19732): No heartbeat from core client for 30 sec - exiting
22:30:10 (19732): No heartbeat from core client for 30 sec - exiting
22:30:11 (19732): No heartbeat from core client for 30 sec - exiting
22:30:12 (19732): No heartbeat from core client for 30 sec - exiting
22:30:13 (19732): No heartbeat from core client for 30 sec - exiting
22:30:14 (19732): No heartbeat from core client for 30 sec - exiting
22:30:15 (19732): No heartbeat from core client for 30 sec - exiting
22:30:16 (19732): No heartbeat from core client for 30 sec - exiting
22:30:17 (19732): No heartbeat from core client for 30 sec - exiting
22:30:18 (19732): No heartbeat from core client for 30 sec - exiting
22:30:19 (19732): No heartbeat from core client for 30 sec - exiting
22:30:20 (19732): No heartbeat from core client for 30 sec - exiting
22:30:21 (19732): No heartbeat from core client for 30 sec - exiting
22:30:22 (19732): No heartbeat from core client for 30 sec - exiting
22:30:23 (19732): No heartbeat from core client for 30 sec - exiting
22:30:24 (19732): No heartbeat from core client for 30 sec - exiting
22:30:25 (19732): No heartbeat from core client for 30 sec - exiting
22:30:26 (19732): No heartbeat from core client for 30 sec - exiting
22:30:27 (19732): No heartbeat from core client for 30 sec - exiting
22:30:28 (19732): No heartbeat from core client for 30 sec - exiting
22:30:29 (19732): No heartbeat from core client for 30 sec - exiting
22:30:30 (19732): No heartbeat from core client for 30 sec - exiting
22:30:31 (19732): No heartbeat from core client for 30 sec - exiting
22:30:32 (19732): No heartbeat from core client for 30 sec - exiting
22:30:33 (19732): No heartbeat from core client for 30 sec - exiting
22:30:34 (19732): No heartbeat from core client for 30 sec - exiting
22:30:35 (19732): No heartbeat from core client for 30 sec - exiting
22:30:36 (19732): No heartbeat from core client for 30 sec - exiting
22:30:37 (19732): No heartbeat from core client for 30 sec - exiting
22:30:38 (19732): No heartbeat from core client for 30 sec - exiting
22:30:39 (19732): No heartbeat from core client for 30 sec - exiting
22:30:40 (19732): No heartbeat from core client for 30 sec - exiting
22:30:41 (19732): No heartbeat from core client for 30 sec - exiting
22:30:42 (19732): No heartbeat from core client for 30 sec - exiting
22:30:43 (19732): No heartbeat from core client for 30 sec - exiting
22:30:44 (19732): No heartbeat from core client for 30 sec - exiting
22:30:45 (19732): No heartbeat from core client for 30 sec - exiting
22:30:46 (19732): No heartbeat from core client for 30 sec - exiting
22:30:47 (19732): No heartbeat from core client for 30 sec - exiting
22:30:48 (19732): No heartbeat from core client for 30 sec - exiting
22:30:49 (19732): No heartbeat from core client for 30 sec - exiting
22:30:50 (19732): No heartbeat from core client for 30 sec - exiting
22:30:51 (19732): No heartbeat from core client for 30 sec - exiting
22:30:52 (19732): No heartbeat from core client for 30 sec - exiting
22:30:53 (19732): No heartbeat from core client for 30 sec - exiting
22:30:54 (19732): No heartbeat from core client for 30 sec - exiting
22:30:55 (19732): No heartbeat from core client for 30 sec - exiting
22:30:56 (19732): No heartbeat from core client for 30 sec - exiting
22:30:57 (19732): No heartbeat from core client for 30 sec - exiting
22:30:58 (19732): No heartbeat from core client for 30 sec - exiting
22:30:59 (19732): No heartbeat from core client for 30 sec - exiting
22:31:00 (19732): No heartbeat from core client for 30 sec - exiting
22:31:01 (19732): No heartbeat from core client for 30 sec - exiting
22:31:02 (19732): No heartbeat from core client for 30 sec - exiting
22:31:03 (19732): No heartbeat from core client for 30 sec - exiting
22:31:04 (19732): No heartbeat from core client for 30 sec - exiting
22:31:05 (19732): No heartbeat from core client for 30 sec - exiting
22:31:06 (19732): No heartbeat from core client for 30 sec - exiting
22:31:07 (19732): No heartbeat from core client for 30 sec - exiting
22:31:08 (19732): No heartbeat from core client for 30 sec - exiting
22:31:09 (19732): No heartbeat from core client for 30 sec - exiting
22:31:10 (19732): No heartbeat from core client for 30 sec - exiting
22:31:11 (19732): No heartbeat from core client for 30 sec - exiting
22:31:12 (19732): No heartbeat from core client for 30 sec - exiting
22:31:13 (19732): No heartbeat from core client for 30 sec - exiting
22:31:14 (19732): No heartbeat from core client for 30 sec - exiting
22:31:15 (19732): No heartbeat from core client for 30 sec - exiting
22:31:16 (19732): No heartbeat from core client for 30 sec - exiting
22:31:17 (19732): No heartbeat from core client for 30 sec - exiting
22:31:18 (19732): No heartbeat from core client for 30 sec - exiting
22:31:19 (19732): No heartbeat from core client for 30 sec - exiting
22:31:20 (19732): No heartbeat from core client for 30 sec - exiting
22:31:21 (19732): No heartbeat from core client for 30 sec - exiting
22:31:22 (19732): No heartbeat from core client for 30 sec - exiting
22:31:23 (19732): No heartbeat from core client for 30 sec - exiting
22:31:24 (19732): No heartbeat from core client for 30 sec - exiting
22:31:25 (19732): No heartbeat from core client for 30 sec - exiting
22:31:26 (19732): No heartbeat from core client for 30 sec - exiting
22:31:27 (19732): No heartbeat from core client for 30 sec - exiting
22:31:30 (19732): No heartbeat from core client for 30 sec - exiting
22:31:31 (19732): No heartbeat from core client for 30 sec - exiting
22:31:32 (19732): No heartbeat from core client for 30 sec - exiting
22:31:33 (19732): No heartbeat from core client for 30 sec - exiting
22:31:34 (19732): No heartbeat from core client for 30 sec - exiting
22:31:35 (19732): No heartbeat from core client for 30 sec - exiting
22:31:36 (19732): No heartbeat from core client for 30 sec - exiting
22:31:37 (19732): No heartbeat from core client for 30 sec - exiting
22:31:38 (19732): No heartbeat from core client for 30 sec - exiting
22:31:39 (19732): No heartbeat from core client for 30 sec - exiting
22:31:40 (19732): No heartbeat from core client for 30 sec - exiting
22:31:41 (19732): No heartbeat from core client for 30 sec - exiting
22:31:42 (19732): No heartbeat from core client for 30 sec - exiting
22:31:43 (19732): No heartbeat from core client for 30 sec - exiting
22:31:44 (19732): No heartbeat from core client for 30 sec - exiting
22:31:45 (19732): No heartbeat from core client for 30 sec - exiting
22:31:46 (19732): No heartbeat from core client for 30 sec - exiting
22:31:47 (19732): No heartbeat from core client for 30 sec - exiting
22:31:48 (19732): No heartbeat from core client for 30 sec - exiting
22:31:49 (19732): No heartbeat from core client for 30 sec - exiting
22:31:50 (19732): No heartbeat from core client for 30 sec - exiting
22:31:51 (19732): No heartbeat from core client for 30 sec - exiting
22:31:52 (19732): No heartbeat from core client for 30 sec - exiting
22:31:53 (19732): No heartbeat from core client for 30 sec - exiting
22:31:54 (19732): No heartbeat from core client for 30 sec - exiting
22:31:55 (19732): No heartbeat from core client for 30 sec - exiting
22:31:56 (19732): No heartbeat from core client for 30 sec - exiting
22:31:57 (19732): No heartbeat from core client for 30 sec - exiting
22:31:58 (19732): No heartbeat from core client for 30 sec - exiting
22:31:59 (19732): No heartbeat from core client for 30 sec - exiting
22:32:00 (19732): No heartbeat from core client for 30 sec - exiting
22:32:01 (19732): No heartbeat from core client for 30 sec - exiting
22:32:02 (19732): No heartbeat from core client for 30 sec - exiting
22:32:03 (19732): No heartbeat from core client for 30 sec - exiting
22:32:04 (19732): No heartbeat from core client for 30 sec - exiting
22:32:05 (19732): No heartbeat from core client for 30 sec - exiting
22:32:06 (19732): No heartbeat from core client for 30 sec - exiting
22:32:07 (19732): No heartbeat from core client for 30 sec - exiting
22:32:08 (19732): No heartbeat from core client for 30 sec - exiting
22:32:09 (19732): No heartbeat from core client for 30 sec - exiting
22:32:10 (19732): No heartbeat from core client for 30 sec - exiting
22:32:11 (19732): No heartbeat from core client for 30 sec - exiting
22:32:12 (19732): No heartbeat from core client for 30 sec - exiting
22:32:13 (19732): No heartbeat from core client for 30 sec - exiting
22:32:14 (19732): No heartbeat from core client for 30 sec - exiting
22:32:15 (19732): No heartbeat from core client for 30 sec - exiting
22:32:16 (19732): No heartbeat from core client for 30 sec - exiting
22:32:17 (19732): No heartbeat from core client for 30 sec - exiting
22:32:18 (19732): No heartbeat from core client for 30 sec - exiting
22:32:19 (19732): No heartbeat from core client for 30 sec - exiting
22:32:20 (19732): No heartbeat from core client for 30 sec - exiting
22:32:21 (19732): No heartbeat from core client for 30 sec - exiting
22:32:22 (19732): No heartbeat from core client for 30 sec - exiting
22:32:23 (19732): No heartbeat from core client for 30 sec - exiting
22:32:24 (19732): No heartbeat from core client for 30 sec - exiting
22:32:25 (19732): No heartbeat from core client for 30 sec - exiting
22:32:26 (19732): No heartbeat from core client for 30 sec - exiting
22:32:27 (19732): No heartbeat from core client for 30 sec - exiting
22:32:28 (19732): No heartbeat from core client for 30 sec - exiting
22:32:29 (19732): No heartbeat from core client for 30 sec - exiting
22:32:30 (19732): No heartbeat from core client for 30 sec - exiting
22:32:31 (19732): No heartbeat from core client for 30 sec - exiting
22:32:32 (19732): No heartbeat from core client for 30 sec - exiting
22:32:33 (19732): No heartbeat from core client for 30 sec - exiting
22:32:34 (19732): No heartbeat from core client for 30 sec - exiting
22:32:35 (19732): No heartbeat from core client for 30 sec - exiting
22:32:36 (19732): No heartbeat from core client for 30 sec - exiting
22:32:37 (19732): No heartbeat from core client for 30 sec - exiting
22:32:38 (19732): No heartbeat from core client for 30 sec - exiting
22:32:39 (19732): No heartbeat from core client for 30 sec - exiting
22:32:40 (19732): No heartbeat from core client for 30 sec - exiting
22:32:41 (19732): No heartbeat from core client for 30 sec - exiting
22:32:42 (19732): No heartbeat from core client for 30 sec - exiting
22:32:43 (19732): No heartbeat from core client for 30 sec - exiting
22:32:44 (19732): No heartbeat from core client for 30 sec - exiting
22:32:45 (19732): No heartbeat from core client for 30 sec - exiting
22:32:46 (19732): No heartbeat from core client for 30 sec - exiting
22:32:47 (19732): No heartbeat from core client for 30 sec - exiting
22:32:48 (19732): No heartbeat from core client for 30 sec - exiting
22:32:49 (19732): No heartbeat from core client for 30 sec - exiting
22:32:50 (19732): No heartbeat from core client for 30 sec - exiting
22:32:51 (19732): No heartbeat from core client for 30 sec - exiting
22:32:52 (19732): No heartbeat from core client for 30 sec - exiting
22:32:53 (19732): No heartbeat from core client for 30 sec - exiting
22:32:54 (19732): No heartbeat from core client for 30 sec - exiting
22:32:55 (19732): No heartbeat from core client for 30 sec - exiting
22:32:56 (19732): No heartbeat from core client for 30 sec - exiting
22:32:57 (19732): No heartbeat from core client for 30 sec - exiting
22:32:58 (19732): No heartbeat from core client for 30 sec - exiting
22:32:59 (19732): No heartbeat from core client for 30 sec - exiting
22:33:00 (19732): No heartbeat from core client for 30 sec - exiting
22:33:01 (19732): No heartbeat from core client for 30 sec - exiting
22:33:02 (19732): No heartbeat from core client for 30 sec - exiting
22:33:03 (19732): No heartbeat from core client for 30 sec - exiting
22:33:04 (19732): No heartbeat from core client for 30 sec - exiting
22:33:05 (19732): No heartbeat from core client for 30 sec - exiting
22:33:06 (19732): No heartbeat from core client for 30 sec - exiting
22:33:07 (19732): No heartbeat from core client for 30 sec - exiting
22:33:08 (19732): No heartbeat from core client for 30 sec - exiting
22:33:09 (19732): No heartbeat from core client for 30 sec - exiting
22:33:10 (19732): No heartbeat from core client for 30 sec - exiting
22:33:11 (19732): No heartbeat from core client for 30 sec - exiting
22:33:12 (19732): No heartbeat from core client for 30 sec - exiting
22:33:13 (19732): No heartbeat from core client for 30 sec - exiting
22:33:14 (19732): No heartbeat from core client for 30 sec - exiting
22:33:15 (19732): No heartbeat from core client for 30 sec - exiting
22:33:16 (19732): No heartbeat from core client for 30 sec - exiting
22:33:17 (19732): No heartbeat from core client for 30 sec - exiting
22:33:18 (19732): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 42 to Word Address 14178304 Failed with Error Code -1

Model crashed: SETPOS: Unit 42 to Word Address 14178304 Failed with Error Code -1
forrtl: No space left on device
forrtl: severe (38): error during write, unit -1, file stdout
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08450E2C  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08131127  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0822CA65  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818A767  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818D749  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08391957  Unknown               UnController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13568, iMonCtr=1
Model crash detected, will try to restart...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 42 to Word Address 198656 Failed with Error Code -1

Model crashed: SETPOS: Unit 42 to Word Address 198656 Failed with Error Code -1

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 42 to Word Address 13940736 Failed with Error Code -1

Model crashed: SETPOS: Unit 42 to Word Address 13940736 Failed with Error Code -1

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 22 to Word Address 448512 Failed with Error Code -1

Model crashed: SETPOS: Unit 22 to Word Address 448512 Failed with Error Code -1

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: STWORK  : Error in PP_FILE                                                                                                                                                                                                                                      tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /data/boinc-client/projects/climateprediction.net/hadcm3n_o53h_2140_40_008270456/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08450E2C  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  081DD74F  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0808D513  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08299901  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0821C545  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08219DA1  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838720B  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  083995C6  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838F8B7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0839BDF8  Unknown               Unknown  Unknown
libc.so.6          F759BBD6  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0804CB11  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13568, iMonCtr=1
Model crash detected, will try to restart...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /data/boinc-client/projects/climateprediction.net/hadcm3n_o53h_2140_40_008270456/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08450E2C  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0810D3C3  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  081826E6  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818CDAC  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08391957  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838F8B7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0839BDF8  Unknown               Unknown  Unknown
libc.so.6          F75A8BD6  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0804CB11  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15864, iMonCtr=1
Model crash detected, will try to restart...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /data/boinc-client/projects/climateprediction.net/hadcm3n_o53h_2140_40_008270456/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  084554BF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  083721F7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0837FE35  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0822B1DE  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818A767  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818D749  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08391957  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838F8B7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0839BDF8  Unknown               Unknown  Unknown
libc.so.6          F7550BD6  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0804CB11  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15864, iMonCtr=1
Model crash detected, will try to restart...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                               
SETPOS: Seek Failed: No space left on device
SETPOS: Unit 22 to Word Address 243712 Failed with Error Code -1

Model crashed: SETPOS: Unit 22 to Word Address 243712 Failed with Error Code -1

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 22 to Word Address 243712 Failed with Error Code -1

Model crashed: SETPOS: Unit 22 to Word Address 243712 Failed with Error Code -1
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation
Stack trace (15 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf7785400]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x806c0d5]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x806e5f2]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8072509]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8077f47]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x80781a3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e1b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf74bcbd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Feb 2013 02:09:30 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 259,200 517,784 1.9976
07 Feb 2013 06:52:13 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 233,280 455,397 1.9521
06 Feb 2013 11:03:34 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 207,360 392,184 1.8913
05 Feb 2013 14:35:32 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 181,440 327,768 1.8065
04 Feb 2013 19:02:56 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 155,520 265,015 1.7041
03 Feb 2013 22:25:04 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 129,600 341,645 2.6361
03 Feb 2013 00:55:50 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 103,680 270,987 2.6137
02 Feb 2013 02:34:13 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 77,760 198,395 2.5514
31 Jan 2013 04:14:53 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 51,840 153,633 2.9636
29 Jan 2013 14:30:24 1148818 15559777 hadcm3n_o53h_2140_40_008270456_3 25,920 75,019 2.8943


©2024 climateprediction.net