Task 15071075

Name	hadcm3n_0027_1980_40_008085819_3
Workunit	8240933
Created	6 Aug 2012, 1:59:32 UTC
Sent	6 Aug 2012, 1:59:38 UTC
Report deadline	5 Nov 2012, 9:26:49 UTC
Received	24 Aug 2012, 15:10:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	1 (0x00000001) Unknown error code
Computer ID	1230144
Run time	17 days 23 hours 27 min 12 sec
CPU time	16 days 14 hours 46 min 8 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	2.21 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> Incorrect function. (0x1) - exit code 1 (0x1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 06:37:49 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:13:48 (4892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:09:35 (3688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:38 (4816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:14:04 (4360): No heartbeat from core client for 30 sec - exiting 18:14:05 (4360): No heartbeat from core client for 30 sec - exiting 18:14:06 (4360): No heartbeat from core client for 30 sec - exiting 18:14:07 (4360): No heartbeat from core client for 30 sec - exiting 18:14:08 (4360): No heartbeat from core client for 30 sec - exiting 18:14:09 (4360): No heartbeat from core client for 30 sec - exiting 18:14:10 (4360): No heartbeat from core client for 30 sec - exiting 18:14:11 (4360): No heartbeat from core client for 30 sec - exiting 18:14:12 (4360): No heartbeat from core client for 30 sec - exiting 18:14:14 (4360): No heartbeat from core client for 30 sec - exiting 18:14:15 (4360): No heartbeat from core client for 30 sec - exiting 18:14:16 (4360): No heartbeat from core client for 30 sec - exiting 18:14:17 (4360): No heartbeat from core client for 30 sec - exiting 18:14:18 (4360): No heartbeat from core client for 30 sec - exiting 18:14:19 (4360): No heartbeat from core client for 30 sec - exiting 18:14:20 (4360): No heartbeat from core client for 30 sec - exiting 18:14:21 (4360): No heartbeat from core client for 30 sec - exiting 18:14:22 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:15:28 (4684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:34 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:36 (4456): No heartbeat from core client for 30 sec - exiting 00:38:38 (4456): No heartbeat from core client for 30 sec - exiting 00:38:39 (4456): No heartbeat from core client for 30 sec - exiting 00:38:40 (4456): No heartbeat from core client for 30 sec - exiting 00:38:41 (4456): No heartbeat from core client for 30 sec - exiting 00:38:42 (4456): No heartbeat from core client for 30 sec - exiting 00:38:43 (4456): No heartbeat from core client for 30 sec - exiting 00:38:44 (4456): No heartbeat from core client for 30 sec - exiting 00:38:45 (4456): No heartbeat from core client for 30 sec - exiting 00:38:46 (4456): No heartbeat from core client for 30 sec - exiting 00:42:18 (1724): No heartbeat from core client for 30 sec - exiting 00:42:28 (1724): No heartbeat from core client for 30 sec - exiting 00:42:29 (1724): No heartbeat from core client for 30 sec - exiting 00:42:30 (1724): No heartbeat from core client for 30 sec - exiting 00:42:31 (1724): No heartbeat from core client for 30 sec - exiting 00:42:32 (1724): No heartbeat from core client for 30 sec - exiting 00:42:34 (1724): No heartbeat from core client for 30 sec - exiting 00:42:35 (1724): No heartbeat from core client for 30 sec - exiting 00:42:36 (1724): No heartbeat from core client for 30 sec - exiting 00:42:37 (1724): No heartbeat from core client for 30 sec - exiting 00:42:38 (1724): No heartbeat from core client for 30 sec - exiting 00:42:39 (1724): No heartbeat from core client for 30 sec - exiting 00:42:40 (1724): No heartbeat from core client for 30 sec - exiting 00:42:41 (1724): No heartbeat from core client for 30 sec - exiting 00:42:42 (1724): No heartbeat from core client for 30 sec - exiting 00:42:43 (1724): No heartbeat from core client for 30 sec - exiting 00:42:44 (1724): No heartbeat from core client for 30 sec - exiting 00:42:46 (1724): No heartbeat from core client for 30 sec - exiting 00:42:47 (1724): No heartbeat from core client for 30 sec - exiting 00:42:48 (1724): No heartbeat from core client for 30 sec - exiting 00:42:49 (1724): No heartbeat from core client for 30 sec - exiting 00:42:50 (1724): No heartbeat from core client for 30 sec - exiting 00:42:51 (1724): No heartbeat from core client for 30 sec - exiting 00:42:52 (1724): No heartbeat from core client for 30 sec - exiting 00:42:53 (1724): No heartbeat from core client for 30 sec - exiting 00:42:54 (1724): No heartbeat from core client for 30 sec - exiting 00:42:55 (1724): No heartbeat from core client for 30 sec - exiting 00:42:56 (1724): No heartbeat from core client for 30 sec - exiting 00:42:58 (1724): No heartbeat from core client for 30 sec - exiting 00:42:59 (1724): No heartbeat from core client for 30 sec - exiting 00:43:00 (1724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:43:01 (1724): No heartbeat from core client for 30 sec - exiting 00:43:02 (1724): No heartbeat from core client for 30 sec - exiting 00:43:03 (1724): No heartbeat from core client for 30 sec - exiting 00:43:04 (1724): No heartbeat from core client for 30 sec - exiting 00:45:01 (4604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:36 (1384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:00:02 (2240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:26:13 (1616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:27:04 (3324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:07:47 (4820): No heartbeat from core client for 30 sec - exiting 06:07:48 (4820): No heartbeat from core client for 30 sec - exiting 06:07:49 (4820): No heartbeat from core client for 30 sec - exiting 06:07:50 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:07:51 (4820): No heartbeat from core client for 30 sec - exiting 06:09:07 (4428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:41 (4776): No heartbeat from core client for 30 sec - exiting 20:03:42 (4776): No heartbeat from core client for 30 sec - exiting 20:03:43 (4776): No heartbeat from core client for 30 sec - exiting 20:03:44 (4776): No heartbeat from core client for 30 sec - exiting 20:03:45 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:53:10 (1644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:54:03 (3292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:57:39 (4512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:57:27 (3508): No heartbeat from core client for 30 sec - exiting 05:57:28 (3508): No heartbeat from core client for 30 sec - exiting 05:57:29 (3508): No heartbeat from core client for 30 sec - exiting 05:57:30 (3508): No heartbeat from core client for 30 sec - exiting 05:57:31 (3508): No heartbeat from core client for 30 sec - exiting 05:57:32 (3508): No heartbeat from core client for 30 sec - exiting 05:57:34 (3508): No heartbeat from core client for 30 sec - exiting 05:57:35 (3508): No heartbeat from core client for 30 sec - exiting 05:57:36 (3508): No heartbeat from core client for 30 sec - exiting 05:57:37 (3508): No heartbeat from core client for 30 sec - exiting 05:57:38 (3508): No heartbeat from core client for 30 sec - exiting 05:57:39 (3508): No heartbeat from core client for 30 sec - exiting 05:57:40 (3508): No heartbeat from core client for 30 sec - exiting 05:57:41 (3508): No heartbeat from core client for 30 sec - exiting 05:57:42 (3508): No heartbeat from core client for 30 sec - exiting 05:57:43 (3508): No heartbeat from core client for 30 sec - exiting 05:57:45 (3508): No heartbeat from core client for 30 sec - exiting 05:57:46 (3508): No heartbeat from core client for 30 sec - exiting 05:57:47 (3508): No heartbeat from core client for 30 sec - exiting 05:57:48 (3508): No heartbeat from core client for 30 sec - exiting 05:57:49 (3508): No heartbeat from core client for 30 sec - exiting 05:57:50 (3508): No heartbeat from core client for 30 sec - exiting 05:57:51 (3508): No heartbeat from core client for 30 sec - exiting 05:57:52 (3508): No heartbeat from core client for 30 sec - exiting 05:57:53 (3508): No heartbeat from core client for 30 sec - exiting 05:57:54 (3508): No heartbeat from core client for 30 sec - exiting 05:57:56 (3508): No heartbeat from core client for 30 sec - exiting 05:57:57 (3508): No heartbeat from core client for 30 sec - exiting 05:57:58 (3508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:58:49 (4376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:48 (4560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 15:27:54 (5604): No heartbeat from core client for 30 sec - exiting 15:27:55 (5604): No heartbeat from core client for 30 sec - exiting 15:27:56 (5604): No heartbeat from core client for 30 sec - exiting 15:27:57 (5604): No heartbeat from core client for 30 sec - exiting 15:27:59 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:28:00 (5604): No heartbeat from core client for 30 sec - exiting 15:28:54 (5592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:30:22 (4236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:11 (4816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:09:22 (1404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5084, iMonCtr=1 Model crash detected, will try to restart... 06:24:21 (4260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:10:49 (4556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:57 (4724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:46:01 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:20:46 (1872): No heartbeat from core client for 30 sec - exiting 17:20:47 (1872): No heartbeat from core client for 30 sec - exiting 17:20:48 (1872): No heartbeat from core client for 30 sec - exiting 17:20:49 (1872): No heartbeat from core client for 30 sec - exiting 17:20:50 (1872): No heartbeat from core client for 30 sec - exiting 17:20:51 (1872): No heartbeat from core client for 30 sec - exiting 17:20:52 (1872): No heartbeat from core client for 30 sec - exiting 17:20:54 (1872): No heartbeat from core client for 30 sec - exiting 17:20:55 (1872): No heartbeat from core client for 30 sec - exiting 17:20:56 (1872): No heartbeat from core client for 30 sec - exiting 17:20:57 (1872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:20:58 (1872): No heartbeat from core client for 30 sec - exiting 05:35:03 (2688): No heartbeat from core client for 30 sec - exiting 05:35:04 (2688): No heartbeat from core client for 30 sec - exiting 05:35:05 (2688): No heartbeat from core client for 30 sec - exiting 05:35:06 (2688): No heartbeat from core client for 30 sec - exiting 05:35:08 (2688): No heartbeat from core client for 30 sec - exiting 05:35:09 (2688): No heartbeat from core client for 30 sec - exiting 05:35:10 (2688): No heartbeat from core client for 30 sec - exiting 05:35:11 (2688): No heartbeat from core client for 30 sec - exiting 05:35:12 (2688): No heartbeat from core client for 30 sec - exiting 05:35:13 (2688): No heartbeat from core client for 30 sec - exiting 05:35:14 (2688): No heartbeat from core client for 30 sec - exiting 05:35:15 (2688): No heartbeat from core client for 30 sec - exiting 05:35:16 (2688): No heartbeat from core client for 30 sec - exiting 05:35:17 (2688): No heartbeat from core client for 30 sec - exiting 05:35:18 (2688): No heartbeat from core client for 30 sec - exiting 05:35:20 (2688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:35:21 (2688): No heartbeat from core client for 30 sec - exiting 06:25:24 (2352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:05:49 (2096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:05:50 (2096): No heartbeat from core client for 30 sec - exiting 05:51:57 (4640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:53:09 (4136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:36:16 (3408): No heartbeat from core client for 30 sec - exiting 06:36:17 (3408): No heartbeat from core client for 30 sec - exiting 06:36:18 (3408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:36:54 (5772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:38:09 (4152): No heartbeat from core client for 30 sec - exiting 18:38:11 (4152): No heartbeat from core client for 30 sec - exiting 18:38:12 (4152): No heartbeat from core client for 30 sec - exiting 18:38:13 (4152): No heartbeat from core client for 30 sec - exiting 18:38:14 (4152): No heartbeat from core client for 30 sec - exiting 18:38:15 (4152): No heartbeat from core client for 30 sec - exiting 18:38:16 (4152): No heartbeat from core client for 30 sec - exiting 18:38:17 (4152): No heartbeat from core client for 30 sec - exiting 18:38:18 (4152): No heartbeat from core client for 30 sec - exiting 18:38:19 (4152): No heartbeat from core client for 30 sec - exiting 18:38:20 (4152): No heartbeat from core client for 30 sec - exiting 18:38:22 (4152): No heartbeat from core client for 30 sec - exiting 18:38:23 (4152): No heartbeat from core client for 30 sec - exiting 18:38:24 (4152): No heartbeat from core client for 30 sec - exiting 18:38:25 (4152): No heartbeat from core client for 30 sec - exiting 18:38:26 (4152): No heartbeat from core client for 30 sec - exiting 18:38:27 (4152): No heartbeat from core client for 30 sec - exiting 18:38:28 (4152): No heartbeat from core client for 30 sec - exiting 18:38:29 (4152): No heartbeat from core client for 30 sec - exiting 18:38:30 (4152): No heartbeat from core client for 30 sec - exiting 18:38:31 (4152): No heartbeat from core client for 30 sec - exiting 18:38:33 (4152): No heartbeat from core client for 30 sec - exiting 18:38:34 (4152): No heartbeat from core client for 30 sec - exiting 18:38:35 (4152): No heartbeat from core client for 30 sec - exiting 18:38:36 (4152): No heartbeat from core client for 30 sec - exiting 18:38:37 (4152): No heartbeat from core client for 30 sec - exiting 18:38:38 (4152): No heartbeat from core client for 30 sec - exiting 18:38:39 (4152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:40:42 (3708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:35:21 (3136): No heartbeat from core client for 30 sec - exiting 05:35:22 (3136): No heartbeat from core client for 30 sec - exiting 05:35:23 (3136): No heartbeat from core client for 30 sec - exiting 05:35:24 (3136): No heartbeat from core client for 30 sec - exiting 05:35:25 (3136): No heartbeat from core client for 30 sec - exiting 05:35:26 (3136): No heartbeat from core client for 30 sec - exiting 05:35:28 (3136): No heartbeat from core client for 30 sec - exiting 05:35:29 (3136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:35:30 (3136): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:29:44 (2596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:30:33 (1108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5732, iMonCt06:52:03 (4372): No heartbeat from core client for 30 sec - exiting 06:52:04 (4372): No heartbeat from core client for 30 sec - exiting 06:52:05 (4372): No heartbeat from core client for 30 sec - exiting 06:52:07 (4372): No heartbeat from core client for 30 sec - exiting 06:52:08 (4372): No heartbeat from core client for 30 sec - exiting 06:52:09 (4372): No heartbeat from core client for 30 sec - exiting 06:52:10 (4372): No heartbeat from core client for 30 sec - exiting 06:52:11 (4372): No heartbeat from core client for 30 sec - exiting 06:52:12 (4372): No heartbeat from core client for 30 sec - exiting 06:52:13 (4372): No heartbeat from core client for 30 sec - exiting 06:52:14 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:52:15 (4372): No heartbeat from core client for 30 sec - exiting 17:21:52 (3920): No heartbeat from core client for 30 sec - exiting 17:21:54 (3920): No heartbeat from core client for 30 sec - exiting 17:21:55 (3920): No heartbeat from core client for 30 sec - exiting 17:21:56 (3920): No heartbeat from core client for 30 sec - exiting 17:21:57 (3920): No heartbeat from core client for 30 sec - exiting 17:21:58 (3920): No heartbeat from core client for 30 sec - exiting 17:21:59 (3920): No heartbeat from core client for 30 sec - exiting 17:22:00 (3920): No heartbeat from core client for 30 sec - exiting 17:22:01 (3920): No heartbeat from core client for 30 sec - exiting 17:22:02 (3920): No heartbeat from core client for 30 sec - exiting 17:22:03 (3920): No heartbeat from core client for 30 sec - exiting 17:22:04 (3920): No heartbeat from core client for 30 sec - exiting 17:22:06 (3920): No heartbeat from core client for 30 sec - exiting 17:22:07 (3920): No heartbeat from core client for 30 sec - exiting 17:22:08 (3920): No heartbeat from core client for 30 sec - exiting 17:22:09 (3920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:22:54 (1444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:29 (1496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:12 (2716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:18:12 (4928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Aug 2012 05:25:29	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	829,440	1,403,708	1.6924
23 Aug 2012 16:18:16	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	803,520	1,360,551	1.6932
23 Aug 2012 04:15:28	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	777,600	1,317,383	1.6942
22 Aug 2012 14:57:49	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	751,680	1,274,076	1.6950
22 Aug 2012 01:54:42	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	725,760	1,231,242	1.6965
21 Aug 2012 13:20:56	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	699,840	1,188,304	1.6980
21 Aug 2012 00:27:59	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	673,920	1,144,750	1.6986
20 Aug 2012 11:04:54	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	648,000	1,100,725	1.6986
19 Aug 2012 19:59:19	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	622,080	1,055,019	1.6960
19 Aug 2012 05:58:39	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	596,160	1,008,955	1.6924
18 Aug 2012 15:10:17	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	570,240	962,696	1.6882
18 Aug 2012 01:12:44	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	544,320	917,315	1.6852
17 Aug 2012 12:40:40	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	518,400	871,918	1.6819
16 Aug 2012 22:37:38	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	492,480	826,280	1.6778
16 Aug 2012 09:56:10	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	466,560	780,906	1.6738
15 Aug 2012 18:34:55	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	440,640	734,225	1.6663
15 Aug 2012 02:58:53	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	414,720	688,592	1.6604
14 Aug 2012 12:32:16	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	388,800	644,983	1.6589
13 Aug 2012 23:09:19	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	362,880	600,456	1.6547
13 Aug 2012 09:54:36	1230144	15071075	hadcm3n_0027_1980_40_008085819_3	336,960	557,321	1.6540