Task 14358214

Name	hadcm3n_o634_2020_40_007856625_1
Workunit	8011737
Created	4 Apr 2012, 18:03:41 UTC
Sent	4 Apr 2012, 18:05:06 UTC
Report deadline	5 Jul 2012, 1:32:17 UTC
Received	10 May 2012, 7:02:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1204790
Run time	16 days 19 hours 36 min 9 sec
CPU time	16 days 4 hours 23 min 6 sec
Validate state	Invalid
Credit	10,886.40
Device peak FLOPS	3.27 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.24</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:36:38 (2138): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:11:49 (1904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:11:50 (1904): No heartbeat from core client for 30 sec - exiting 08:11:51 (1904): No heartbeat from core client for 30 sec - exiting 08:11:52 (1904): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:19:19 (1865): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:36 (1865): No heartbeat from core client for 30 sec - exiting 17:19:41 (1865): No heartbeat from core client for 30 sec - exiting 17:19:42 (1865): No heartbeat from core client for 30 sec - exiting 17:19:43 (1865): No heartbeat from core client for 30 sec - exiting 17:19:44 (1865): No heartbeat from core client for 30 sec - exiting 17:19:45 (1865): No heartbeat from core client for 30 sec - exiting 17:19:46 (1865): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 17:21:36 (5795): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:23:28 (5980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:45 (6193): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:46 (6193): No heartbeat from core client for 30 sec - exiting 17:24:47 (6193): No heartbeat from core client for 30 sec - exiting 17:24:48 (6193): No heartbeat from core client for 30 sec - exiting 17:24:49 (6193): No heartbeat from core client for 30 sec - exiting 17:24:50 (6193): No heartbeat from core client for 30 sec - exiting 17:24:51 (6193): No heartbeat from core client for 30 sec - exiting 17:24:52 (6193): No heartbeat from core client for 30 sec - exiting 17:24:53 (6193): No heartbeat from core client for 30 sec - exiting 17:24:54 (6193): No heartbeat from core client for 30 sec - exiting 17:24:55 (6193): No heartbeat from core client for 30 sec - exiting 17:24:56 (6193): No heartbeat from core client for 30 sec - exiting 17:24:57 (6193): No heartbeat from core client for 30 sec - exiting 17:24:58 (6193): No heartbeat from core client for 30 sec - exiting 17:24:59 (6193): No heartbeat from core client for 30 sec - exiting 17:25:00 (6193): No heartbeat from core client for 30 sec - exiting 17:25:01 (6193): No heartbeat from core client for 30 sec - exiting 17:25:02 (6193): No heartbeat from core client for 30 sec - exiting 17:25:03 (6193): No heartbeat from core client for 30 sec - exiting 17:31:21 (6858): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:37:51 (7658): No heartbeat from core client for 30 sec - exiting 17:37:52 (7658): No heartbeat from core client for 30 sec - exiting 17:37:53 (7658): No heartbeat from core client for 30 sec - exiting 17:37:54 (7658): No heartbeat from core client for 30 sec - exiting 17:37:55 (7658): No heartbeat from core client for 30 sec - exiting 17:37:56 (7658): No heartbeat from core client for 30 sec - exiting 17:37:57 (7658): No heartbeat from core client for 30 sec - exiting 17:37:58 (7658): No heartbeat from core client for 30 sec - exiting 17:37:59 (7658): No heartbeat from core client for 30 sec - exiting 17:38:00 (7658): No heartbeat from core client for 30 sec - exiting 17:38:01 (7658): No heartbeat from core client for 30 sec - exiting 17:38:02 (7658): No heartbeat from core client for 30 sec - exiting 17:38:03 (7658): No heartbeat from core client for 30 sec - exiting 17:38:04 (7658): No heartbeat from core client for 30 sec - exiting 17:38:05 (7658): No heartbeat from core client for 30 sec - exiting 17:38:06 (7658): No heartbeat from core client for 30 sec - exiting 17:38:07 (7658): No heartbeat from core client for 30 sec - exiting 17:38:08 (7658): No heartbeat from core client for 30 sec - exiting 17:38:09 (7658): No heartbeat from core client for 30 sec - exiting 17:38:10 (7658): No heartbeat from core client for 30 sec - exiting 17:38:11 (7658): No heartbeat from core client for 30 sec - exiting 17:38:12 (7658): No heartbeat from core client for 30 sec - exiting 17:38:13 (7658): No heartbeat from core client for 30 sec - exiting 17:38:22 (7658): No heartbeat from core client for 30 sec - exiting 17:38:23 (7658): No heartbeat from core client for 30 sec - exiting 17:38:24 (7658): No heartbeat from core client for 30 sec - exiting 17:38:25 (7658): No heartbeat from core client for 30 sec - exiting 17:38:26 (7658): No heartbeat from core client for 30 sec - exiting 17:38:27 (7658): No heartbeat from core client for 30 sec - exiting 17:38:28 (7658): No heartbeat from core client for 30 sec - exiting 17:38:29 (7658): No heartbeat from core client for 30 sec - exiting 17:38:30 (7658): No heartbeat from core client for 30 sec - exiting 17:38:31 (7658): No heartbeat from core client for 30 sec - exiting 17:38:32 (7658): No heartbeat from core client for 30 sec - exiting 17:38:33 (7658): No heartbeat from core client for 30 sec - exiting 17:38:34 (7658): No heartbeat from core client for 30 sec - exiting 17:38:35 (7658): No heartbeat from core client for 30 sec - exiting 17:38:36 (7658): No heartbeat from core client for 30 sec - exiting 17:38:37 (7658): No heartbeat from core client for 30 sec - exiting 17:38:38 (7658): No heartbeat from core client for 30 sec - exiting 17:38:39 (7658): No heartbeat from core client for 30 sec - exiting 17:38:40 (7658): No heartbeat from core client for 30 sec - exiting 17:38:41 (7658): No heartbeat from core client for 30 sec - exiting 17:38:42 (7658): No heartbeat from core client for 30 sec - exiting 17:38:43 (7658): No heartbeat from core client for 30 sec - exiting 17:38:44 (7658): No heartbeat from core client for 30 sec - exiting 17:38:45 (7658): No heartbeat from core client for 30 sec - exiting 17:38:46 (7658): No heartbeat from core client for 30 sec - exiting 17:38:47 (7658): No heartbeat from core client for 30 sec - exiting 17:38:48 (7658): No heartbeat from core client for 30 sec - exiting 17:38:49 (7658): No heartbeat from core client for 30 sec - exiting 17:38:50 (7658): No heartbeat from core client for 30 sec - exiting 17:38:51 (7658): No heartbeat from core client for 30 sec - exiting 17:38:52 (7658): No heartbeat from core client for 30 sec - exiting 17:38:53 (7658): No heartbeat from core client for 30 sec - exiting 17:38:54 (7658): No heartbeat from core client for 30 sec - exiting 17:38:55 (7658): No heartbeat from core client for 30 sec - exiting 17:38:56 (7658): No heartbeat from core client for 30 sec - exiting 17:38:57 (7658): No heartbeat from core client for 30 sec - exiting 17:38:58 (7658): No heartbeat from core client for 30 sec - exiting 17:38:59 (7658): No heartbeat from core client for 30 sec - exiting 17:39:00 (7658): No heartbeat from core client for 30 sec - exiting 17:39:01 (7658): No heartbeat from core client for 30 sec - exiting 17:39:02 (7658): No heartbeat from core client for 30 sec - exiting 17:39:03 (7658): No heartbeat from core client for 30 sec - exiting 17:39:04 (7658): No heartbeat from core client for 30 sec - exiting 17:39:05 (7658): No heartbeat from core client for 30 sec - exiting 17:39:06 (7658): No heartbeat from core client for 30 sec - exiting 17:39:07 (7658): No heartbeat from core client for 30 sec - exiting 17:39:08 (7658): No heartbeat from core client for 30 sec - exiting 17:39:09 (7658): No heartbeat from core client for 30 sec - exiting 17:39:10 (7658): No heartbeat from core client for 30 sec - exiting 17:39:11 (7658): No heartbeat from core client for 30 sec - exiting 17:39:12 (7658): No heartbeat from core client for 30 sec - exiting 17:39:13 (7658): No heartbeat from core client for 30 sec - exiting 17:39:14 (7658): No heartbeat from core client for 30 sec - exiting 17:39:15 (7658): No heartbeat from core client for 30 sec - exiting 17:39:16 (7658): No heartbeat from core client for 30 sec - exiting 17:39:17 (7658): No heartbeat from core client for 30 sec - exiting 17:39:18 (7658): No heartbeat from core client for 30 sec - exiting 17:39:19 (7658): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:36:32 (2251): No heartbeat from core client for 30 sec - exiting 00:36:34 (2251): No heartbeat from core client for 30 sec - exiting 00:36:35 (2251): No heartbeat from core client for 30 sec - exiting 00:36:36 (2251): No heartbeat from core client for 30 sec - exiting 00:36:37 (2251): No heartbeat from core client for 30 sec - exiting 00:36:38 (2251): No heartbeat from core client for 30 sec - exiting 00:36:39 (2251): No heartbeat from core client for 30 sec - exiting 00:36:40 (2251): No heartbeat from core client for 30 sec - exiting 00:36:41 (2251): No heartbeat from core client for 30 sec - exiting 00:36:42 (2251): No heartbeat from core client for 30 sec - exiting 00:36:43 (2251): No heartbeat from core client for 30 sec - exiting 00:36:44 (2251): No heartbeat from core client for 30 sec - exiting 00:36:45 (2251): No heartbeat from core client for 30 sec - exiting 00:36:46 (2251): No heartbeat from core client for 30 sec - exiting 00:36:47 (2251): No heartbeat from core client for 30 sec - exiting 00:36:48 (2251): No heartbeat from core client for 30 sec - exiting 00:36:49 (2251): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:14:31 (1996): No heartbeat from core client for 30 sec - exiting 22:14:36 (1996): No heartbeat from core client for 30 sec - exiting 22:14:37 (1996): No heartbeat from core client for 30 sec - exiting 22:14:38 (1996): No heartbeat from core client for 30 sec - exiting 22:14:39 (1996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:57:43 (2040): No heartbeat from core client for 30 sec - exiting 23:57:47 (2040): No heartbeat from core client for 30 sec - exiting 23:57:48 (2040): No heartbeat from core client for 30 sec - exiting 23:57:49 (2040): No heartbeat from core client for 30 sec - exiting 23:57:50 (2040): No heartbeat from core client for 30 sec - exiting 23:57:51 (2040): No heartbeat from core client for 30 sec - exiting 23:57:52 (2040): No heartbeat from core client for 30 sec - exiting 23:57:53 (2040): No heartbeat from core client for 30 sec - exiting 23:57:54 (2040): No heartbeat from core client for 30 sec - exiting 23:57:55 (2040): No heartbeat from core client for 30 sec - exiting 23:57:56 (2040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:03:13 (2494): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:03:14 (2494): No heartbeat from core client for 30 sec - exiting 02:03:15 (2494): No heartbeat from core client for 30 sec - exiting 02:03:16 (2494): No heartbeat from core client for 30 sec - exiting 02:03:17 (2494): No heartbeat from core client for 30 sec - exiting 02:03:18 (2494): No heartbeat from core client for 30 sec - exiting 02:03:19 (2494): No heartbeat from core client for 30 sec - exiting 02:03:20 (2494): No heartbeat from core client for 30 sec - exiting 02:03:21 (2494): No heartbeat from core client for 30 sec - exiting 02:03:22 (2494): No heartbeat from core client for 30 sec - exiting 02:03:23 (2494): No heartbeat from core client for 30 sec - exiting 02:03:55 (2494): No heartbeat from core client for 30 sec - exiting 02:03:56 (2494): No heartbeat from core client for 30 sec - exiting 02:03:57 (2494): No heartbeat from core client for 30 sec - exiting 02:03:58 (2494): No heartbeat from core client for 30 sec - exiting 02:03:59 (2494): No heartbeat from core client for 30 sec - exiting 02:04:00 (2494): No heartbeat from core client for 30 sec - exiting 02:04:01 (2494): No heartbeat from core client for 30 sec - exiting 02:04:02 (2494): No heartbeat from core client for 30 sec - exiting 02:04:03 (2494): No heartbeat from core client for 30 sec - exiting 02:04:04 (2494): No heartbeat from core client for 30 sec - exiting 02:04:05 (2494): No heartbeat from core client for 30 sec - exiting 02:04:06 (2494): No heartbeat from core client for 30 sec - exiting 02:04:07 (2494): No heartbeat from core client for 30 sec - exiting 02:04:08 (2494): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:57:16 (3598): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf7769400] [0xf7769430] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf757e1ef] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7581835] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf775e400] [0xf775e430] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75731ef] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7576835] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf755e4d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1881, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf77a8400] [0xf77a8430] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75bd1ef] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75c0835] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75a84d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1881, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf76ec400] [0xf76ec430] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75011ef] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7504835] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf74ec4d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1881, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 May 2012 23:48:48	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	907,200	1,371,196	1.5115
05 May 2012 02:00:09	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	881,280	1,333,755	1.5134
01 May 2012 04:21:37	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	855,360	1,296,999	1.5163
29 Apr 2012 12:31:21	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	829,440	1,259,063	1.5180
29 Apr 2012 02:59:03	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	803,520	1,222,333	1.5212
28 Apr 2012 15:15:08	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	777,600	1,185,263	1.5243
28 Apr 2012 02:39:16	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	751,680	1,148,486	1.5279
27 Apr 2012 16:11:33	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	725,760	1,112,071	1.5323
27 Apr 2012 05:48:22	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	699,840	1,075,893	1.5373
25 Apr 2012 18:39:08	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	673,920	1,038,994	1.5417
25 Apr 2012 03:36:18	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	648,000	998,675	1.5412
24 Apr 2012 15:39:53	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	622,080	957,835	1.5397
24 Apr 2012 02:59:04	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	596,160	918,262	1.5403
23 Apr 2012 14:36:09	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	570,240	878,285	1.5402
23 Apr 2012 03:46:24	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	544,320	838,395	1.5403
18 Apr 2012 21:03:58	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	518,400	798,908	1.5411
18 Apr 2012 09:18:44	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	492,480	759,053	1.5413
17 Apr 2012 20:06:12	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	466,560	719,100	1.5413
17 Apr 2012 02:31:37	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	440,640	679,065	1.5411
16 Apr 2012 15:07:22	1204790	14358214	hadcm3n_o634_2020_40_007856625_1	414,720	638,995	1.5408