Task 15280457

Name	hadcm3n_z8rg_1880_40_008200795_1
Workunit	8355919
Created	13 Sep 2012, 9:18:13 UTC
Sent	13 Sep 2012, 22:27:46 UTC
Report deadline	14 Dec 2012, 5:54:57 UTC
Received	2 Nov 2012, 11:12:21 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	958227
Run time	26 days 10 hours 22 min 14 sec
CPU time	25 days 15 hours 59 min 41 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	1.72 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:09:08 (32559): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:12:13 (8838): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation 17:32:41 (19569): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:32:48 (19569): No heartbeat from core client for 30 sec - exiting 17:32:49 (19569): No heartbeat from core client for 30 sec - exiting 17:32:50 (19569): No heartbeat from core client for 30 sec - exiting 17:32:51 (19569): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:18:37 (27002): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:30:09 (29425): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:30:11 (29425): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:22:30 (25383): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:26:02 (23106): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:44:11 (13712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:44:14 (13712): No heartbeat from core client for 30 sec - exiting 07:44:16 (13712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:19:23 (4238): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:19:29 (4238): No heartbeat from core client for 30 sec - exiting 12:19:30 (4238): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:59:34 (11258): No heartbeat from core client for 30 sec - exiting 16:00:09 (11258): No heartbeat from core client for 30 sec - exiting 16:00:10 (11258): No heartbeat from core client for 30 sec - exiting 16:00:11 (11258): No heartbeat from core client for 30 sec - exiting 16:00:12 (11258): No heartbeat from core client for 30 sec - exiting 16:00:13 (11258): No heartbeat from core client for 30 sec - exiting 16:00:14 (11258): No heartbeat from core client for 30 sec - exiting 16:00:15 (11258): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:17 (11258): No heartbeat from core client for 30 sec - exiting 16:00:19 (11258): No heartbeat from core client for 30 sec - exiting 16:00:20 (11258): No heartbeat from core client for 30 sec - exiting 16:00:23 (11258): No heartbeat from core client for 30 sec - exiting 16:00:24 (11258): No heartbeat from core client for 30 sec - exiting 16:00:27 (11258): No heartbeat from core client for 30 sec - exiting 16:00:28 (11258): No heartbeat from core client for 30 sec - exiting 16:00:31 (11258): No heartbeat from core client for 30 sec - exiting 16:00:32 (11258): No heartbeat from core client for 30 sec - exiting 16:00:33 (11258): No heartbeat from core client for 30 sec - exiting 16:00:36 (11258): No heartbeat from core client for 30 sec - exiting 16:00:37 (11258): No heartbeat from core client for 30 sec - exiting 16:00:38 (11258): No heartbeat from core client for 30 sec - exiting 16:00:39 (11258): No heartbeat from core client for 30 sec - exiting 16:00:40 (11258): No heartbeat from core client for 30 sec - exiting 16:00:43 (11258): No heartbeat from core client for 30 sec - exiting 16:00:44 (11258): No heartbeat from core client for 30 sec - exiting 16:00:45 (11258): No heartbeat from core client for 30 sec - exiting 16:00:46 (11258): No heartbeat from core client for 30 sec - exiting 16:00:48 (11258): No heartbeat from core client for 30 sec - exiting 16:00:49 (11258): No heartbeat from core client for 30 sec - exiting 16:00:50 (11258): No heartbeat from core client for 30 sec - exiting 16:00:51 (11258): No heartbeat from core client for 30 sec - exiting 16:00:53 (11258): No heartbeat from core client for 30 sec - exiting 16:00:54 (11258): No heartbeat from core client for 30 sec - exiting 16:00:55 (11258): No heartbeat from core client for 30 sec - exiting 16:00:56 (11258): No heartbeat from core client for 30 sec - exiting 16:00:57 (11258): No heartbeat from core client for 30 sec - exiting 16:01:03 (11258): No heartbeat from core client for 30 sec - exiting 16:01:05 (11258): No heartbeat from core client for 30 sec - exiting 16:01:06 (11258):17:03:00 (13055): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:48:13 (16265): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:48:16 (16265): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 11:50:17 (18177): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:04:58 (5890): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5914, iMonCtr=1 Model crash detected, will try to restart... 16:03:02 (5914): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:06:02 (7651): No heartbeat from core client for 30 sec - exiting 16:06:03 (7651): No heartbeat from core client for 30 sec - exiting 16:06:04 (7651): No heartbeat from core client for 30 sec - exiting 16:06:05 (7651): No heartbeat from core client for 30 sec - exiting 16:06:06 (7651): No heartbeat from core client for 30 sec - exiting 16:06:07 (7651): No heartbeat from core client for 30 sec - exiting 16:06:08 (7651): No heartbeat from core client for 30 sec - exiting 16:06:10 (7651): No heartbeat from core client for 30 sec - exiting 16:06:11 (7651): No heartbeat from core client for 30 sec - exiting 16:06:12 (7651): No heartbeat from core client for 30 sec - exiting 16:06:13 (7651): No heartbeat from core client for 30 sec - exiting 16:06:14 (7651): No heartbeat from core client for 30 sec - exiting 16:06:15 (7651): No heartbeat from core client for 30 sec - exiting 16:06:16 (7651): No heartbeat from core client for 30 sec - exiting 16:06:17 (7651): No heartbeat from core client for 30 sec - exiting 16:06:18 (7651): No heartbeat from core client for 30 sec - exiting 16:06:20 (7651): No heartbeat from core client for 30 sec - exiting 16:06:21 (7651): No heartbeat from core client for 30 sec - exiting 16:06:22 (7651): No heartbeat from core client for 30 sec - exiting 16:06:23 (7651): No heartbeat from core client for 30 sec - exiting 16:06:24 (7651): No heartbeat from core client for 30 sec - exiting 16:06:25 (7651): No heartbeat from core client for 30 sec - exiting 16:06:26 (7651): No heartbeat from core client for 30 sec - exiting 16:06:27 (7651): No heartbeat from core client for 30 sec - exiting 16:06:28 (7651): No heartbeat from core client for 30 sec - exiting 16:06:29 (7651): No heartbeat from core client for 30 sec - exiting 16:06:31 (7651): No heartbeat from core client for 30 sec - exiting 16:06:32 (7651): No heartbeat from core client for 30 sec - exiting 16:06:33 (7651): No heartbeat from core client for 30 sec - exiting 16:06:34 (7651): No heartbeat from core client for 30 sec - exiting 16:06:35 (7651): No heartbeat from core client for 30 sec - exiting 16:06:36 (7651): No heartbeat from core client for 30 sec - exiting 16:06:37 (7651): No heartbeat from core client for 30 sec - exiting 16:06:38 (7651): No heartbeat from core client for 30 sec - exiting 16:06:39 (7651): No heartbeat from core client for 30 sec - exiting 16:06:40 (7651): No heartbeat from core client for 30 sec - exiting 16:06:42 (7651): No heartbeat from core client for 30 sec - exiting 16:06:43 (7651): No heartbeat from core client for 30 sec - exiting 16:06:44 (7651): No heartbeat from core client for 30 sec - exiting 16:06:45 (7651): No heartbeat from core client for 30 sec - exiting 16:06:46 (7651): No heartbeat from core client for 30 sec - exiting 16:06:50 (7651): No heartbeat from core client for 30 sec - exiting 16:06:51 (7651): No heartbeat from core client for 30 sec - exiting 16:06:52 (7651): No heartbeat from core client for 30 sec - exiting 16:06:56 (7651): No heartbeat from core client for 30 sec - exiting 16:06:57 (7651): No heartbeat from core client for 30 sec - exiting 16:06:58 (7651): No heartbeat from core client for 30 sec - exiting 16:06:59 (7651): No heartbeat from core client for 30 sec - exiting 16:07:00 (7651): No heartbeat from core client for 30 sec - exiting 16:07:01 (7651): No heartbeat from core client for 30 sec - exiting 16:07:05 (7651): No heartbeat from core client for 30 sec - exiting 16:07:06 (7651): No heartbeat from core client for 30 sec - exiting 16:07:07 (7651): No heartbeat from core client for 30 sec - exiting 16:07:08 (7651): No heartbeat from core client for 30 sec - exiting 16:07:09 (7651): No heartbeat from core client for 30 sec - exiting 16:07:10 (7651): No heartbeat from core client for 30 sec - exiting 16:07:11 (7651): No heartbeat from core client for 30 sec - exiting 16:07:13 (7651): No heartbeat from core client for 30 sec - exiting 16:07:14 (7651): No heartbeat from core client for 30 sec - exiting 16:07:15 (7651): No heartbeat from core client for 30 sec - exiting 16:07:29 (7651): No heartbeat from core client for 30 sec - exiting 16:07:30 (7651): No heartbeat from core client for 30 sec - exiting 16:07:31 (7651): No heartbeat from core client for 30 sec - exiting 16:07:36 (7651): No heartbeat from core client for 30 sec - exiting 16:07:37 (7651): No heartbeat from core client for 30 sec - exiting 16:07:38 (7651): No heartbeat from core client for 30 sec - exiting 16:07:44 (7651): No heartbeat from core client for 30 sec - exiting 16:07:45 (7651): No heartbeat from core client for 30 sec - exiting 16:07:46 (7651): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:54 (7651): No heartbeat from core client for 30 sec - exiting 16:07:55 (7651): No heartbeat from core client for 30 sec - exiting 16:07:58 (7651): No heartbeat from core client for 30 sec - exiting 16:07:59 (7651): No heartbeat from core client for 30 sec - exiting 16:08:00 (7651): No heartbeat from core client for 30 sec - exiting 16:08:01 (7651): No heartbeat from core client for 30 sec - exiting 16:08:13 (7651): No heartbeat from core client for 30 sec - exiting 16:08:19 (7651): No heartbeat from core client for 30 sec - exiting 16:08:20 (7651): No heartbeat from core client for 30 sec - exiting 16:08:21 (7651): No heartbeat from core client for 30 sec - exiting 16:08:23 (7651): No heartbeat from core client for 30 sec - exiting 16:08:24 (7651): No heartbeat from core client for 30 sec - exiting 16:08:25 (7651): No heartbeat from core client for 30 sec - exiting 16:08:26 (7651): No heartbeat from core client for 30 sec - exiting 16:08:27 (7651): No heartbeat from core client for 30 sec - exiting 16:08:29 (7651): No heartbeat from core client for 30 sec - exiting 16:08:30 (7651): No heartbeat from core client for 30 sec - exiting 16:08:32 (7651): No heartbeat from core client for 30 sec - exiting 16:08:33 (7651): No heartbeat from core client for 30 sec - exiting 16:08:35 (7651): No heartbeat from core client for 30 sec - exiting 16:08:39 (7651): No heartbeat from core client for 30 sec - exiting 16:08:40 (7651): No heartbeat from core client for 30 sec - exiting 16:08:42 (7651): No heartbeat from core client for 30 sec - exiting 16:08:43 (7651): No heartbeat from core client for 30 sec - exiting 16:08:44 (7651): No heartbeat from core client for 30 sec - exiting 16:08:46 (7651): No heartbeat from core client for 30 sec - exiting 16:08:48 (7651): No heartbeat from core client for 30 sec - exiting 16:08:49 (7651): No heartbeat from core client for 30 sec - exiting 16:08:52 (7651): No heartbeat from core client for 30 sec - exiting 16:08:53 (7651): No heartbeat from core client for 30 sec - exiting 16:08:54 (7651): No heartbeat from core client for 30 sec - exiting 16:08:55 (7651): No heartbeat from core client for 30 sec - exiting 16:08:58 (7651): No heartbeat from core client for 30 sec - exiting 16:08:59 (7651): No heartbeat from core client for 30 sec - exiting 16:09:01 (7651): No heartbeat from core client for 30 sec - exiting 16:09:06 (7651): No heartbeat from core client for 30 sec - exiting 16:09:08 (7651): No heartbeat from core client for 30 sec - exiting 16:09:10 (7651): No heartbeat from core client for 30 sec - exiting 16:09:12 (7651): No heartbeat from core client for 30 sec - exiting 16:09:13 (7651): No heartbeat from core client for 30 sec - exiting 16:09:15 (7651): No heartbeat from core client for 30 sec - exiting 16:09:16 (7651): No heartbeat from core client for 30 sec - exiting 16:09:17 (7651):16:14:06 (7830): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 3 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:01:46 (8688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:05:59 (9528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:33 (9543): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:37 (9543): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:27:33 (22284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:42:51 (25047): No heartbeat from core client for 30 sec - exiting 19:42:52 (25047): No heartbeat from core client for 30 sec - exiting 19:42:53 (25047): No heartbeat from core client for 30 sec - exiting 19:42:54 (25047): No heartbeat from core client for 30 sec - exiting 19:42:55 (25047): No heartbeat from core client for 30 sec - exiting 19:42:56 (25047): No heartbeat from core client for 30 sec - exiting 19:42:57 (25047): No heartbeat from core client for 30 sec - exiting 19:42:58 (25047): No heartbeat from core client for 30 sec - exiting 19:42:59 (25047): No heartbeat from core client for 30 sec - exiting 19:43:00 (25047): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:55:37 (25113): No heartbeat from core client for 30 sec - exiting 19:56:06 (25113): No heartbeat from core client for 30 sec - exiting 19:56:07 (25113): No heartbeat from core client for 30 sec - exiting 19:56:08 (25113): No heartbeat from core client for 30 sec - exiting 19:56:09 (25113): No heartbeat from core client for 30 sec - exiting 19:56:10 (25113): No heartbeat from core client for 30 sec - exiting 19:56:12 (25113): No heartbeat from core client for 30 sec - exiting 19:56:13 (25113): No heartbeat from core client for 30 sec - exiting 19:56:14 (25113): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:17 (25113): No heartbeat from core client for 30 sec - exiting 19:56:18 (25113): No heartbeat from core client for 30 sec - exiting 19:56:19 (25113): No heartbeat from core client for 30 sec - exiting 19:56:20 (25113): No heartbeat from core client for 30 sec - exiting 19:56:22 (25113): No heartbeat from core client for 30 sec - exiting 19:56:23 (25113): No heartbeat from core client for 30 sec - exiting 19:56:24 (25113): No heartbeat from core client for 30 sec - exiting 20:05:57 (25469): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:17:11 (25767): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:17:33 (25767): No heartbeat from core client for 30 sec - exiting 20:18:14 (26108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:27:31 (21334): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:11:29 (30536): No heartbeat from core client for 30 sec - exiting 09:11:45 (30536): No heartbeat from core client for 30 sec - exiting 09:11:46 (30536): No heartbeat from core client for 30 sec - exiting 09:11:47 (30536): No heartbeat from core client for 30 sec - exiting 09:11:51 (30536): No heartbeat from core client for 30 sec - exiting 09:11:52 (30536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:11:53 (30536): No heartbeat from core client for 30 sec - exiting 09:11:56 (30536): No heartbeat from core client for 30 sec - exiting 09:11:57 (30536): No heartbeat from core client for 30 sec - exiting 09:11:58 (30536): No heartbeat from core client for 30 sec - exiting 09:11:59 (30536): No heartbeat from core client for 30 sec - exiting 09:12:00 (30536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=31944, iMonCtr=1 09:24:45 (31944): No heartbeat from core client for 30 sec - exiting Model crash detected, will try to restart... CPDN Monitor - No 'heartbeat' from BOINC... 09:35:25 (32251): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:46:18 (32590): No heartbeat from core client for 30 sec - exiting 09:46:43 (32590): No heartbeat from core client for 30 sec - exiting 09:46:45 (32590): No heartbeat from core client for 30 sec - exiting 09:46:46 (32590): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:46:55 (32590): No heartbeat from core client for 30 sec - exiting 09:46:56 (32590): No heartbeat from core client for 30 sec - exiting 09:46:57 (32590): No heartbeat from core client for 30 sec - exiting 09:46:58 (32590): No heartbeat from core client for 30 sec - exiting 09:46:59 (32590): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:38:21 (32706): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:38:24 (32706): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (15 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf771a400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x806c0d5] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x806e5f2] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8072509] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8077f47] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x80781a3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e1b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7480ca6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Nov 2012 11:15:18	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	777,600	2,217,578	2.8518
31 Oct 2012 18:11:07	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	751,680	2,142,287	2.8500
30 Oct 2012 10:15:10	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	725,760	2,068,711	2.8504
28 Oct 2012 23:55:40	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	699,840	1,993,274	2.8482
27 Oct 2012 14:16:55	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	673,920	1,918,129	2.8462
25 Oct 2012 17:56:17	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	648,000	1,843,833	2.8454
23 Oct 2012 20:23:54	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	622,080	1,770,360	2.8459
22 Oct 2012 01:21:32	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	596,160	1,696,441	2.8456
20 Oct 2012 08:06:25	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	570,240	1,623,322	2.8467
18 Oct 2012 13:42:47	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	544,320	1,550,596	2.8487
16 Oct 2012 21:22:35	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	518,400	1,477,297	2.8497
15 Oct 2012 03:04:46	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	492,480	1,403,470	2.8498
13 Oct 2012 09:42:31	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	466,560	1,329,093	2.8487
11 Oct 2012 16:32:19	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	440,640	1,256,321	2.8511
09 Oct 2012 21:18:41	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	414,720	1,183,148	2.8529
08 Oct 2012 03:09:46	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	388,800	1,109,852	2.8546
06 Oct 2012 10:12:58	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	362,880	1,036,093	2.8552
04 Oct 2012 15:44:41	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	336,960	962,769	2.8572
02 Oct 2012 21:14:02	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	311,040	888,880	2.8578
01 Oct 2012 04:04:27	958227	15280457	hadcm3n_z8rg_1880_40_008200795_1	285,120	815,180	2.8591