climateprediction.net home page
Task 13412971

Task 13412971

Name hadcm3n_u7c9_1980_40_007460821_0
Workunit 7658324
Created 23 Sep 2011, 1:04:23 UTC
Sent 23 Sep 2011, 19:20:45 UTC
Report deadline 24 Dec 2011, 2:47:56 UTC
Received 30 Oct 2011, 18:25:58 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1157390
Run time 32 days 4 hours 47 min 38 sec
CPU time 32 days 3 hours 14 min 6 sec
Validate state Invalid
Credit 11,508.48
Device peak FLOPS 1.38 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
8:04 (16916): No heartbeat from core client for 30 sec - exiting
13:18:05 (16916): No heartbeat from core client for 30 sec - exiting
13:18:06 (16916): No heartbeat from core client for 30 sec - exiting
13:18:07 (16916): No heartbeat from core client for 30 sec - exiting
13:18:08 (16916): No heartbeat from core client for 30 sec - exiting
13:18:09 (16916): No heartbeat from core client for 30 sec - exiting
13:18:10 (16916): No heartbeat from core client for 30 sec - exiting
13:18:11 (16916): No heartbeat from core client for 30 sec - exiting
13:18:12 (16916): No heartbeat from core client for 30 sec - exiting
13:18:13 (16916): No heartbeat from core client for 30 sec - exiting
13:18:14 (16916): No heartbeat from core client for 30 sec - exiting
13:18:15 (16916): No heartbeat from core client for 30 sec - exiting
13:18:16 (16916): No heartbeat from core client for 30 sec - exiting
13:18:17 (16916): No heartbeat from core client for 30 sec - exiting
13:18:18 (16916): No heartbeat from core client for 30 sec - exiting
13:18:19 (16916): No heartbeat from core client for 30 sec - exiting
13:18:20 (16916): No heartbeat from core client for 30 sec - exiting
13:18:21 (16916): No heartbeat from core client for 30 sec - exiting
13:18:22 (16916): No heartbeat from core client for 30 sec - exiting
13:18:23 (16916): No heartbeat from core client for 30 sec - exiting
13:18:24 (16916): No heartbeat from core client for 30 sec - exiting
13:18:25 (16916): No heartbeat from core client for 30 sec - exiting
13:18:26 (16916): No heartbeat from core client for 30 sec - exiting
13:18:28 (16916): No heartbeat from core client for 30 sec - exiting
13:18:29 (16916): No heartbeat from core client for 30 sec - exiting
13:18:30 (16916): No heartbeat from core client for 30 sec - exiting
13:18:31 (16916): No heartbeat from core client for 30 sec - exiting
13:18:32 (16916): No heartbeat from core client for 30 sec - exiting
13:18:33 (16916): No heartbeat from core client for 30 sec - exiting
13:18:34 (16916): No heartbeat from core client for 30 sec - exiting
13:18:35 (16916): No heartbeat from core client for 30 sec - exiting
13:18:36 (16916): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:20:11 (16967): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:21:54 (16998): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
13:33:52 (17048): No heartbeat from core client for 30 sec - exiting
13:33:55 (17048): No heartbeat from core client for 30 sec - exiting
13:33:56 (17048): No heartbeat from core client for 30 sec - exiting
13:33:57 (17048): No heartbeat from core client for 30 sec - exiting
13:33:58 (17048): No heartbeat from core client for 30 sec - exiting
13:33:59 (17048): No heartbeat from core client for 30 sec - exiting
13:34:00 (17048): No heartbeat from core client for 30 sec - exiting
13:34:01 (17048): No heartbeat from core client for 30 sec - exiting
13:34:02 (17048): No heartbeat from core client for 30 sec - exiting
13:34:03 (17048): No heartbeat from core client for 30 sec - exiting
13:34:04 (17048): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:37:05 (17119): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:37:53 (17175): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:39:39 (17224): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:41:34 (17227): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:43:23 (17280): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
13:45:00 (17328): No heartbeat from core client for 30 sec - exiting
13:45:03 (17328): No heartbeat from core client for 30 sec - exiting
13:45:04 (17328): No heartbeat from core client for 30 sec - exiting
13:45:05 (17328): No heartbeat from core client for 30 sec - exiting
13:45:06 (17328): No heartbeat from core client for 30 sec - exiting
13:45:07 (17328): No heartbeat from core client for 30 sec - exiting
13:45:08 (17328): No heartbeat from core client for 30 sec - exiting
13:45:09 (17328): No heartbeat from core client for 30 sec - exiting
13:45:10 (17328): No heartbeat from core client for 30 sec - exiting
13:45:11 (17328): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:47:43 (17386): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:49:26 (17440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:52:23 (17498): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
14:03:16 (17551): No heartbeat from core client for 30 sec - exiting
14:03:17 (17551): No heartbeat from core client for 30 sec - exiting
14:03:19 (17551): No heartbeat from core client for 30 sec - exiting
14:03:20 (17551): No heartbeat from core client for 30 sec - exiting
14:03:21 (17551): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:04:31 (17619): No heartbeat from core client for 30 sec - exiting
14:04:32 (17619): No heartbeat from core client for 30 sec - exiting
14:04:33 (17619): No heartbeat from core client for 30 sec - exiting
14:04:34 (17619): No heartbeat from core client for 30 sec - exiting
14:04:35 (17619): No heartbeat from core client for 30 sec - exiting
14:04:36 (17619): No heartbeat from core client for 30 sec - exiting
14:04:37 (17619): No heartbeat from core client for 30 sec - exiting
14:04:38 (17619): No heartbeat from core client for 30 sec - exiting
14:04:39 (17619): No heartbeat from core client for 30 sec - exiting
14:04:40 (17619): No heartbeat from core client for 30 sec - exiting
14:04:42 (17619): No heartbeat from core client for 30 sec - exiting
14:04:43 (17619): No heartbeat from core client for 30 sec - exiting
14:04:44 (17619): No heartbeat from core client for 30 sec - exiting
14:04:45 (17619): No heartbeat from core client for 30 sec - exiting
14:04:46 (17619): No heartbeat from core client for 30 sec - exiting
14:04:47 (17619): No heartbeat from core client for 30 sec - exiting
14:04:48 (17619): No heartbeat from core client for 30 sec - exiting
14:04:49 (17619): No heartbeat from core client for 30 sec - exiting
14:04:50 (17619): No heartbeat from core client for 30 sec - exiting
14:04:51 (17619): No heartbeat from core client for 30 sec - exiting
14:04:52 (17619): No heartbeat from core client for 30 sec - exiting
14:04:53 (17619): No heartbeat from core client for 30 sec - exiting
14:04:54 (17619): No heartbeat from core client for 30 sec - exiting
14:04:55 (17619): No heartbeat from core client for 30 sec - exiting
14:04:56 (17619): No heartbeat from core client for 30 sec - exiting
14:04:57 (17619): No heartbeat from core client for 30 sec - exiting
14:04:58 (17619): No heartbeat from core client for 30 sec - exiting
14:04:59 (17619): No heartbeat from core client for 30 sec - exiting
14:05:00 (17619): No heartbeat from core client for 30 sec - exiting
14:05:01 (17619): No heartbeat from core client for 30 sec - exiting
14:05:02 (17619): No heartbeat from core client for 30 sec - exiting
14:05:03 (17619): No heartbeat from core client for 30 sec - exiting
14:05:04 (17619): No heartbeat from core client for 30 sec - exiting
14:05:05 (17619): No heartbeat from core client for 30 sec - exiting
14:05:06 (17619): No heartbeat from core client for 30 sec - exiting
14:05:07 (17619): No heartbeat from core client for 30 sec - exiting
14:05:08 (17619): No heartbeat from core client for 30 sec - exiting
14:05:09 (17619): No heartbeat from core client for 30 sec - exiting
14:05:10 (17619): No heartbeat from core client for 30 sec - exiting
14:05:11 (17619): No heartbeat from core client for 30 sec - exiting
14:05:12 (17619): No heartbeat from core client for 30 sec - exiting
14:05:13 (17619): No heartbeat from core client for 30 sec - exiting
14:05:14 (17619): No heartbeat from core client for 30 sec - exiting
14:05:15 (17619): No heartbeat from core client for 30 sec - exiting
14:05:16 (17619): No heartbeat from core client for 30 sec - exiting
14:05:17 (17619): No heartbeat from core client for 30 sec - exiting
14:05:18 (17619): No heartbeat from core client for 30 sec - exiting
14:05:19 (17619): No heartbeat from core client for 30 sec - exiting
14:05:20 (17619): No heartbeat from core client for 30 sec - exiting
14:05:21 (17619): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:07:59 (17699): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
14:15:54 (17736): No heartbeat from core client for 30 sec - exiting
14:15:56 (17736): No heartbeat from core client for 30 sec - exiting
14:15:57 (17736): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:18:10 (17799): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:44:08 (17862): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:47:30 (17930): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:52:31 (17991): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:11:12 (18057): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:17:33 (18121): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:27:33 (18190): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:37:34 (18260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:45:44 (18323): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:46:51 (27603): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:48:39 (27667): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:50:27 (27729): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:52:14 (27790): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:54:26 (27854): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:56:55 (27914): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
04:13:48 (27976): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
04:26:10 (28034): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:41:12 (28099): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
04:45:25 (28161): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
05:01:15 (28223): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
05:56:21 (28291): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
06:41:24 (28368): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:00:51 (28479): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:18:46 (28543): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:51:39 (28611): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:51:42 (28847): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
10:41:56 (28918): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
10:46:56 (28999): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:11:58 (29062): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:22:00 (29126): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:31:16 (29195): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:57:02 (29257): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
12:45:47 (29322): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:17:10 (29391): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:57:41 (29461): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:59:22 (29528): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:01:12 (29590): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:02:56 (29594): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
14:04:43 (29657): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:07:01 (29719): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
14:34:01 (29784): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:36:25 (29847): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:42:59 (29905): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:08:00 (29967): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:13:01 (30032): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:28:02 (30094): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:43:04 (30163): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:53:06 (30225): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:03:07 (30290): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:18:07 (30352): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:33:09 (30419): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:38:10 (30483): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:53:11 (30545): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:08:12 (30610): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:13:13 (30672): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:13:15 (30672): No heartbeat from core client for 30 sec - exiting
17:13:16 (30672): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:23:13 (30734): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:43:15 (30799): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:43:19 (30799): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:58:15 (30864): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:03:17 (30931): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:03:20 (30931): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:18:18 (30993): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:33:21 (31056): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:48:21 (31122): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:53:22 (31185): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:13:24 (31249): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:23:25 (31311): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:38:27 (31380): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:48:27 (31444): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:58:28 (31507): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:13:29 (31571): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:23:30 (31635): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:33:31 (31704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:48:33 (31768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:58:34 (31831): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
21:03:34 (31895): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:03:36 (31895): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
21:23:36 (31957): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
21:38:37 (32026): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:38:38 (32026): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
21:58:38 (32088): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:13:39 (32153): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:28:40 (32215): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:43:41 (32286): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:48:42 (32344): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:13:42 (32407): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:38:46 (32471): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:38:47 (32471): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:58:48 (32582): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:03:33 (32636): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
00:09:07 (32703): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:09:13 (32703): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
00:33:26 (32735): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:35:43 (32735): No heartbeat from core client for 30 sec - exiting
00:36:24 (336): No heartbeat from core client for 30 sec - exiting
00:36:25 (336): No heartbeat from core client for 30 sec - exiting
00:36:26 (336): No heartbeat from core client for 30 sec - exiting
00:36:27 (336): No heartbeat from core client for 30 sec - exiting
00:36:28 (336): No heartbeat from core client for 30 sec - exiting
00:36:29 (336): No heartbeat from core client for 30 sec - exiting
00:36:30 (336): No heartbeat from core client for 30 sec - exiting
00:36:31 (336): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
00:39:55 (409): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:40:04 (409): No heartbeat from core client for 30 sec - exiting
00:40:49 (456): No heartbeat from core client for 30 sec - exiting
00:40:50 (456): No heartbeat from core client for 30 sec - exiting
00:40:51 (456): No heartbeat from core client for 30 sec - exiting
00:40:52 (456): No heartbeat from core client for 30 sec - exiting
00:40:53 (456): No heartbeat from core client for 30 sec - exiting
00:40:54 (456): No heartbeat from core client for 30 sec - exiting
00:40:55 (456): No heartbeat from core client for 30 sec - exiting
00:40:56 (456): No heartbeat from core client for 30 sec - exiting
00:40:57 (456): No heartbeat from core client for 30 sec - exiting
00:40:58 (456): No heartbeat from core client for 30 sec - exiting
00:41:00 (456): No heartbeat from core client for 30 sec - exiting
00:41:01 (456): No heartbeat from core client for 30 sec - exiting
00:41:02 (456): No heartbeat from core client for 30 sec - exiting
00:41:03 (456): No heartbeat from core client for 30 sec - exiting
00:41:04 (456): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:42:46 (496): No heartbeat from core client for 30 sec - exiting
00:42:47 (496): No heartbeat from core client for 30 sec - exiting
00:42:48 (496): No heartbeat from core client for 30 sec - exiting
00:42:50 (496): No heartbeat from core client for 30 sec - exiting
00:42:51 (496): No heartbeat from core client for 30 sec - exiting
00:42:52 (496): No heartbeat from core client for 30 sec - exiting
00:42:53 (496): No heartbeat from core client for 30 sec - exiting
00:42:54 (496): No heartbeat from core client for 30 sec - exiting
00:42:55 (496): No heartbeat from core client for 30 sec - exiting
00:42:56 (496): No heartbeat from core client for 30 sec - exiting
00:42:57 (496): No heartbeat from core client for 30 sec - exiting
00:42:58 (496): No heartbeat from core client for 30 sec - exiting
00:42:59 (496): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:47:22 (551): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:47:29 (551): No heartbeat from core client for 30 sec - exiting
00:47:30 (551): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=618, iMonCtr=1
Model crash detected, will try to restart...
00:52:23 (618): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:52:24 (618): No heartbeat from core client for 30 sec - exiting
00:56:04 (650): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:57:28 (712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:57:37 (712): No heartbeat from core client for 30 sec - exiting
01:01:14 (786): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:02:23 (846): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
SIGABRT: abort called
Stack trace (15 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7798400]
[0xf7798425]
/lib32/libc.so.6(gsignal+0x51)[0xf7622a21]
/lib32/libc.so.6(abort+0x182)[0xf7625e42]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8401b90]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7af5]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7b32]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7c5a]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f825e]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f829d]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839dfbf]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf3]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf760ec76]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=904, iMonCtr=1
Model crash detected, will try to restart...
01:05:19 (904): No heartbeat from core client for 30 sec - exiting
01:05:34 (904): No heartbeat from core client for 30 sec - exiting
01:05:36 (904): No heartbeat from core client for 30 sec - exiting
01:05:37 (904): No heartbeat from core client for 30 sec - exiting
01:05:38 (904): No heartbeat from core client for 30 sec - exiting
01:05:39 (904): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:05:41 (904): No heartbeat from core client for 30 sec - exiting
01:05:42 (904): No heartbeat from core client for 30 sec - exiting
01:05:43 (904): No heartbeat from core client for 30 sec - exiting
01:05:45 (904): No heartbeat from core client for 30 sec - exiting
01:05:46 (904): No heartbeat from core client for 30 sec - exiting
01:05:48 (904): No heartbeat from core client for 30 sec - exiting
01:05:50 (904): No heartbeat from core client for 30 sec - exiting
01:05:51 (904): No heartbeat from core client for 30 sec - exiting
01:05:52 (904): No heartbeat from core client for 30 sec - exiting
01:05:53 (904): No heartbeat from core client for 30 sec - exiting
01:07:24 (960): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:07:26 (960): No heartbeat from core client for 30 sec - exiting
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
SIGABRT: abort called
Stack trace (15 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77c6400]
[0xf77c6425]
/lib32/libc.so.6(gsignal+0x51)[0xf7650a21]
/lib32/libc.so.6(abort+0x182)[0xf7653e42]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8401b90]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7af5]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7b32]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7c5a]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f825e]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f829d]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839dfbf]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf3]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf763cc76]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1022, iMonCtr=1
Model crash detected, will try to restart...
01:10:21 (1022): No heartbeat from core client for 30 sec - exiting
01:10:40 (1022): No heartbeat from core client for 30 sec - exiting
01:10:41 (1022): No heartbeat from core client for 30 sec - exiting
01:10:42 (1022): No heartbeat from core client for 30 sec - exiting
01:10:43 (1022): No heartbeat from core client for 30 sec - exiting
01:10:44 (1022): No heartbeat from core client for 30 sec - exiting
01:10:45 (1022): No heartbeat from core client for 30 sec - exiting
01:10:46 (1022): No heartbeat from core client for 30 sec - exiting
01:10:47 (1022): No heartbeat from core client for 30 sec - exiting
01:10:48 (1022): No heartbeat from core client for 30 sec - exiting
01:10:49 (1022): No heartbeat from core client for 30 sec - exiting
01:10:50 (1022): No heartbeat from core client for 30 sec - exiting
01:10:51 (1022): No heartbeat from core client for 30 sec - exiting
01:10:52 (1022): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:12:06 (1085): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:14:30 (1139): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:16:52 (1192): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:19:16 (1260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:20:40 (1330): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:22:02 (1388): No heartbeat from core client for 30 sec - exiting
01:22:04 (1388): No heartbeat from core client for 30 sec - exiting
01:22:05 (1388): No heartbeat from core client for 30 sec - exiting
01:22:06 (1388): No heartbeat from core client for 30 sec - exiting
01:22:08 (1388): No heartbeat from core client for 30 sec - exiting
01:22:09 (1388): No heartbeat from core client for 30 sec - exiting
01:22:10 (1388): No heartbeat from core client for 30 sec - exiting
01:22:11 (1388): No heartbeat from core client for 30 sec - exiting
01:22:12 (1388): No heartbeat from core client for 30 sec - exiting
01:22:13 (1388): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
SIGABRT: abort called
Stack trace (15 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf775b400]
[0xf775b425]
/lib32/libc.so.6(gsignal+0x51)[0xf75e5a21]
/lib32/libc.so.6(abort+0x182)[0xf75e8e42]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8401b90]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7af5]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7b32]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f7c5a]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f825e]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f829d]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839dfbf]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf3]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75d1c76]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1443, iMonCtr=1
Model crash detected, will try to restart...
01:26:49 (1443): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:28:25 (1491): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:31:25 (1593): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:35:12 (1656): No heartbeat from core client for 30 sec - exiting
01:35:14 (1656): No heartbeat from core client for 30 sec - exiting
01:35:15 (1656): No heartbeat from core client for 30 sec - exiting
01:35:16 (1656): No heartbeat from core client for 30 sec - exiting
01:35:17 (1656): No heartbeat from core client for 30 sec - exiting
01:35:18 (1656): No heartbeat from core client for 30 sec - exiting
01:35:19 (1656): No heartbeat from core client for 30 sec - exiting
01:35:20 (1656): No heartbeat from core client for 30 sec - exiting
01:35:21 (1656): No heartbeat from core client for 30 sec - exiting
01:35:22 (1656): No heartbeat from core client for 30 sec - exiting
01:35:23 (1656): No heartbeat from core client for 30 sec - exiting
01:35:24 (1656): No heartbeat from core client for 30 sec - exiting
01:35:25 (1656): No heartbeat from core client for 30 sec - exiting
01:35:26 (1656): No heartbeat from core client for 30 sec - exiting
01:35:27 (1656): No heartbeat from core client for 30 sec - exiting
01:35:28 (1656): No heartbeat from core client for 30 sec - exiting
01:35:29 (1656): No heartbeat from core client for 30 sec - exiting
01:35:30 (1656): No heartbeat from core client for 30 sec - exiting
01:35:31 (1656): No heartbeat from core client for 30 sec - exiting
01:35:32 (1656): No heartbeat from core client for 30 sec - exiting
01:35:33 (1656): No heartbeat from core client for 30 sec - exiting
01:35:34 (1656): No heartbeat from core client for 30 sec - exiting
01:35:35 (1656): No heartbeat from core client for 30 sec - exiting
01:35:36 (1656): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:38:45 (1692): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:38:48 (1692): No heartbeat from core client for 30 sec - exiting
01:40:19 (1740): No heartbeat from core client for 30 sec - exiting
01:40:22 (1740): No heartbeat from core client for 30 sec - exiting
01:40:23 (1740): No heartbeat from core client for 30 sec - exiting
01:40:24 (1740): No heartbeat from core client for 30 sec - exiting
01:40:25 (1740): No heartbeat from core client for 30 sec - exiting
01:40:26 (1740): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:40:27 (1740): No heartbeat from core client for 30 sec - exiting
01:40:28 (1740): No heartbeat from core client for 30 sec - exiting
01:40:29 (1740): No heartbeat from core client for 30 sec - exiting
01:41:37 (1779): No heartbeat from core client for 30 sec - exiting
01:41:40 (1779): No heartbeat from core client for 30 sec - exiting
01:41:41 (1779): No heartbeat from core client for 30 sec - exiting
01:41:42 (1779): No heartbeat from core client for 30 sec - exiting
01:41:43 (1779): No heartbeat from core client for 30 sec - exiting
01:41:44 (1779): No heartbeat from core client for 30 sec - exiting
01:41:45 (1779): No heartbeat from core client for 30 sec - exiting
01:41:46 (1779): No heartbeat from core client for 30 sec - exiting
01:41:47 (1779): No heartbeat from core client for 30 sec - exiting
01:41:48 (1779): No heartbeat from core client for 30 sec - exiting
01:41:49 (1779): No heartbeat from core client for 30 sec - exiting
01:41:50 (1779): No heartbeat from core client for 30 sec - exiting
01:41:51 (1779): No heartbeat from core client for 30 sec - exiting
01:41:52 (1779): No heartbeat from core client for 30 sec - exiting
01:41:53 (1779): No heartbeat from core client for 30 sec - exiting
01:41:54 (1779): No heartbeat from core client for 30 sec - exiting
01:41:55 (1779): No heartbeat from core client for 30 sec - exiting
01:41:56 (1779): No heartbeat from core client for 30 sec - exiting
01:41:57 (1779): No heartbeat from core client for 30 sec - exiting
01:41:58 (1779): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:43:47 (1818): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:43:49 (1818): No heartbeat from core client for 30 sec - exiting
01:44:51 (1854): No heartbeat from core client for 30 sec - exiting
01:44:52 (1854): No heartbeat from core client for 30 sec - exiting
01:44:53 (1854): No heartbeat from core client for 30 sec - exiting
01:44:54 (1854): No heartbeat from core client for 30 sec - exiting
01:44:55 (1854): No heartbeat from core client for 30 sec - exiting
01:44:56 (1854): No heartbeat from core client for 30 sec - exiting
01:44:57 (1854): No heartbeat from core client for 30 sec - exiting
01:44:58 (1854): No heartbeat from core client for 30 sec - exiting
01:44:59 (1854): No heartbeat from core client for 30 sec - exiting
01:45:00 (1854): No heartbeat from core client for 30 sec - exiting
01:45:01 (1854): No heartbeat from core client for 30 sec - exiting
01:45:02 (1854): No heartbeat from core client for 30 sec - exiting
01:45:03 (1854): No heartbeat from core client for 30 sec - exiting
01:45:04 (1854): No heartbeat from core client for 30 sec - exiting
01:45:05 (1854): No heartbeat from core client for 30 sec - exiting
01:45:06 (1854): No heartbeat from core client for 30 sec - exiting
01:45:07 (1854): No heartbeat from core client for 30 sec - exiting
01:45:08 (1854): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:47:06 (1889): No heartbeat from core client for 30 sec - exiting
01:47:08 (1889): No heartbeat from core client for 30 sec - exiting
01:47:09 (1889): No heartbeat from core client for 30 sec - exiting
01:47:10 (1889): No heartbeat from core client for 30 sec - exiting
01:47:11 (1889): No heartbeat from core client for 30 sec - exiting
01:47:12 (1889): No heartbeat from core client for 30 sec - exiting
01:47:13 (1889): No heartbeat from core client for 30 sec - exiting
01:47:14 (1889): No heartbeat from core client for 30 sec - exiting
01:47:15 (1889): No heartbeat from core client for 30 sec - exiting
01:47:16 (1889): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:49:04 (1927): No heartbeat from core client for 30 sec - exiting
01:49:06 (1927): No heartbeat from core client for 30 sec - exiting
01:49:07 (1927): No heartbeat from core client for 30 sec - exiting
01:49:08 (1927): No heartbeat from core client for 30 sec - exiting
01:49:09 (1927): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:52:22 (1972): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:54:03 (2011): No heartbeat from core client for 30 sec - exiting
01:54:06 (2011): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:54:07 (2011): No heartbeat from core client for 30 sec - exiting
01:58:57 (2047): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:04:25 (2088): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:04:53 (2088): No heartbeat from core client for 30 sec - exiting
02:10:23 (2126): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:10:27 (2126): No heartbeat from core client for 30 sec - exiting
02:15:28 (2164): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:15:32 (2164): No heartbeat from core client for 30 sec - exiting
02:20:26 (2206): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:20:30 (2206): No heartbeat from core client for 30 sec - exiting
02:20:31 (2206): No heartbeat from core client for 30 sec - exiting
02:20:32 (2206): No heartbeat from core client for 30 sec - exiting
02:25:27 (2256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:25:30 (2256): No heartbeat from core client for 30 sec - exiting
02:30:29 (2296): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:30:33 (2296): No heartbeat from core client for 30 sec - exiting
02:30:34 (2296): No heartbeat from core client for 30 sec - exiting
02:35:27 (2334): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:35:30 (2334): No heartbeat from core client for 30 sec - exiting
02:40:30 (2372): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:40:34 (2372): No heartbeat from core client for 30 sec - exiting
02:45:29 (2410): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:45:33 (2410): No heartbeat from core client for 30 sec - exiting
02:45:34 (2410): No heartbeat from core client for 30 sec - exiting
02:45:35 (2410): No heartbeat from core client for 30 sec - exiting
02:50:32 (2449): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:50:34 (2449): No heartbeat from core client for 30 sec - exiting
02:55:30 (2489): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:55:36 (2489): No heartbeat from core client for 30 sec - exiting
02:55:37 (2489): No heartbeat from core client for 30 sec - exiting
03:00:32 (2528): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:05:32 (2566): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:05:36 (2566): No heartbeat from core client for 30 sec - exiting
03:10:31 (2604): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:10:36 (2604): No heartbeat from core client for 30 sec - exiting
03:10:37 (2604): No heartbeat from core client for 30 sec - exiting
03:15:30 (2642): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:15:32 (2642): No heartbeat from core client for 30 sec - exiting
03:15:33 (2642): No heartbeat from core client for 30 sec - exiting
03:20:32 (2680): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:20:37 (2680): No heartbeat from core client for 30 sec - exiting
03:20:38 (2680): No heartbeat from core client for 30 sec - exiting
03:25:33 (2724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:25:36 (2724): No heartbeat from core client for 30 sec - exiting
03:25:37 (2724): No heartbeat from core client for 30 sec - exiting
03:30:35 (2764): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:30:42 (2764): No heartbeat from core client for 30 sec - exiting
03:35:35 (2802): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:35:38 (2802): No heartbeat from core client for 30 sec - exiting
03:40:34 (2840): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:40:39 (2840): No heartbeat from core client for 30 sec - exiting
03:40:40 (2840): No heartbeat from core client for 30 sec - exiting
03:45:35 (2878): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:50:37 (2916): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:50:41 (2916): No heartbeat from core client for 30 sec - exiting
03:50:42 (2916): No heartbeat from core client for 30 sec - exiting
03:55:39 (2956): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:55:43 (2956): No heartbeat from core client for 30 sec - exiting
04:00:38 (2995): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:00:40 (2995): No heartbeat from core client for 30 sec - exiting
04:00:41 (2995): No heartbeat from core client for 30 sec - exiting
04:05:37 (3034): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:05:40 (3034): No heartbeat from core client for 30 sec - exiting
04:05:41 (3034): No heartbeat from core client for 30 sec - exiting
04:10:38 (3072): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:10:40 (3072): No heartbeat from core client for 30 sec - exiting
04:15:38 (3110): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:15:49 (3110): No heartbeat from core client for 30 sec - exiting
04:20:39 (3148): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:20:45 (3148): No heartbeat from core client for 30 sec - exiting
04:25:40 (3193): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:25:43 (3193): No heartbeat from core client for 30 sec - exiting
04:30:43 (3232): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:30:49 (3232): No heartbeat from core client for 30 sec - exiting
04:35:43 (3270): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:35:46 (3270): No heartbeat from core client for 30 sec - exiting
04:35:47 (3270): No heartbeat from core client for 30 sec - exiting
04:40:43 (3308): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:40:45 (3308): No heartbeat from core client for 30 sec - exiting
04:45:45 (3346): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:50:45 (3385): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:50:52 (3385): No heartbeat from core client for 30 sec - exiting
04:50:53 (3385): No heartbeat from core client for 30 sec - exiting
04:55:45 (3425): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:55:48 (3425): No heartbeat from core client for 30 sec - exiting
05:00:53 (3464): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3464, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - No 'heartbeat' from BOINC...
05:05:46 (3503): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:05:50 (3503): No heartbeat from core client for 30 sec - exiting
05:10:46 (3541): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:10:49 (3541): No heartbeat from core client for 30 sec - exiting
05:10:50 (3541): No heartbeat from core client for 30 sec - exiting
05:10:51 (3541): No heartbeat from core client for 30 sec - exiting
05:15:46 (3580): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:15:49 (3580): No heartbeat from core client for 30 sec - exiting
05:15:50 (3580): No heartbeat from core client for 30 sec - exiting
05:20:50 (3618): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:20:53 (3618): No heartbeat from core client for 30 sec - exiting
05:25:52 (3662): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:25:55 (3662): No heartbeat from core client for 30 sec - exiting
05:30:50 (3701): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:30:57 (3701): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3739, iMonCtr=1
Model crash detected, will try to restart...
05:35:55 (3739): No heartbeat from core client for 30 sec - exiting
05:35:56 (3739): No heartbeat from core client for 30 sec - exiting
05:35:57 (3739): No heartbeat from core client for 30 sec - exiting
05:35:58 (3739): No heartbeat from core client for 30 sec - exiting
05:35:59 (3739): No heartbeat from core client for 30 sec - exiting
05:36:00 (3739): No heartbeat from core client for 30 sec - exiting
05:36:01 (3739): No heartbeat from core client for 30 sec - exiting
05:36:02 (3739): No heartbeat from core client for 30 sec - exiting
05:36:03 (3739): No heartbeat from core client for 30 sec - exiting
05:36:04 (3739): No heartbeat from core client for 30 sec - exiting
05:36:05 (3739): No heartbeat from core client for 30 sec - exiting
05:36:06 (3739): No heartbeat from core client for 30 sec - exiting
05:36:07 (3739): No heartbeat from core client for 30 sec - exiting
05:36:08 (3739): No heartbeat from core client for 30 sec - exiting
05:36:09 (3739): No heartbeat from core client for 30 sec - exiting
05:36:10 (3739): No heartbeat from core client for 30 sec - exiting
05:36:11 (3739): No heartbeat from core client for 30 sec - exiting
05:36:12 (3739): No heartbeat from core client for 30 sec - exiting
05:36:13 (3739): No heartbeat from core client for 30 sec - exiting
05:36:14 (3739): No heartbeat from core client for 30 sec - exiting
05:36:15 (3739): No heartbeat from core client for 30 sec - exiting
05:36:16 (3739): No heartbeat from core client for 30 sec - exiting
05:36:17 (3739): No heartbeat from core client for 30 sec - exiting
05:36:18 (3739): No heartbeat from core client for 30 sec - exiting
05:36:19 (3739): No heartbeat from core client for 30 sec - exiting
05:36:20 (3739): No heartbeat from core client for 30 sec - exiting
05:36:21 (3739): No heartbeat from core client for 30 sec - exiting
05:36:22 (3739): No heartbeat from core client for 30 sec - exiting
05:36:23 (3739): No heartbeat from core client for 30 sec - exiting
05:36:24 (3739): No heartbeat from core client for 30 sec - exiting
05:36:25 (3739): No heartbeat from core client for 30 sec - exiting
05:36:26 (3739): No heartbeat from core client for 30 sec - exiting
05:36:27 (3739): No heartbeat from core client for 30 sec - exiting
05:36:28 (3739): No heartbeat from core client for 30 sec - exiting
05:36:29 (3739): No heartbeat from core client for 30 sec - exiting
05:36:30 (3739): No heartbeat from core client for 30 sec - exiting
05:36:31 (3739): No heartbeat from core client for 30 sec - exiting
05:36:32 (3739): No heartbeat from core client for 30 sec - exiting
05:36:33 (3739): No heartbeat from core client for 30 sec - exiting
05:36:34 (3739): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:36:36 (3739): No heartbeat from core client for 30 sec - exiting
05:36:37 (3739): No heartbeat from core client for 30 sec - exiting
05:40:51 (3780): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:40:53 (3780): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3818, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish
SIGSEGV: segmentation violation
Stack trace (10 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf778a400]
/lib32/libc.so.6(getenv+0x64)[0xf7505cd4]
/lib32/libc.so.6(+0x8ad70)[0xf7561d70]
/lib32/libc.so.6(+0x8af21)[0xf7561f21]
/lib32/libc.so.6(localtime_r+0x2c)[0xf756065c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x80b0d9c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x80b2dc4]
/lib32/libpthread.so.0(+0x5955)[0xf7761955]
/lib32/libc.so.6(clone+0x5e)[0xf75a4f1e]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
31 Oct 2011 18:25:46 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 959,040 2,738,423 2.8554
31 Oct 2011 17:33:49 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 933,120 2,664,818 2.8558
31 Oct 2011 17:09:16 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 907,200 2,591,330 2.8564
31 Oct 2011 16:33:21 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 881,280 2,517,688 2.8569
31 Oct 2011 15:11:51 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 855,360 2,443,987 2.8573
31 Oct 2011 13:00:04 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 829,440 2,370,360 2.8578
31 Oct 2011 13:00:04 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 803,520 2,296,676 2.8583
31 Oct 2011 12:59:57 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 777,600 2,223,105 2.8589
31 Oct 2011 13:00:00 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 751,680 2,149,567 2.8597
31 Oct 2011 12:59:59 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 725,760 2,075,991 2.8604
31 Oct 2011 12:59:57 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 699,840 2,002,986 2.8621
31 Oct 2011 13:00:03 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 673,920 1,928,974 2.8623
18 Oct 2011 17:11:01 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 648,000 1,854,852 2.8624
17 Oct 2011 19:19:10 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 622,080 1,780,642 2.8624
16 Oct 2011 23:07:09 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 596,160 1,706,650 2.8627
16 Oct 2011 02:12:03 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 570,240 1,632,733 2.8632
15 Oct 2011 05:54:50 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 544,320 1,558,766 2.8637
14 Oct 2011 08:23:47 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 518,400 1,484,097 2.8628
13 Oct 2011 12:44:34 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 492,480 1,410,002 2.8631
12 Oct 2011 15:21:09 1157390 13412971 hadcm3n_u7c9_1980_40_007460821_0 466,560 1,336,443 2.8645


©2024 cpdn.org