Task 15916384

Name	hadcm3n_3ee1_1980_40_008367598_3
Workunit	8518457
Created	14 Aug 2013, 11:41:49 UTC
Sent	14 Aug 2013, 20:32:23 UTC
Report deadline	14 Nov 2013, 3:59:34 UTC
Received	24 Sep 2013, 14:05:45 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID	1231345
Run time	11 days 8 hours 2 min 29 sec
CPU time	10 days 7 hours 31 min 7 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	2.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 03:18:22 (440): No heartbeat from core client for 30 sec - exiting 03:18:23 (440): No heartbeat from core client for 30 sec - exiting 03:18:24 (440): No heartbeat from core client for 30 sec - exiting 03:18:25 (440): No heartbeat from core client for 30 sec - exiting 03:18:26 (440): No heartbeat from core client for 30 sec - exiting 03:18:27 (440): No heartbeat from core client for 30 sec - exiting 03:18:28 (440): No heartbeat from core client for 30 sec - exiting 03:18:29 (440): No heartbeat from core client for 30 sec - exiting 03:18:30 (440): No heartbeat from core client for 30 sec - exiting 03:18:31 (440): No heartbeat from core client for 30 sec - exiting 03:18:33 (440): No heartbeat from core client for 30 sec - exiting 03:18:34 (440): No heartbeat from core client for 30 sec - exiting 03:18:35 (440): No heartbeat from core client for 30 sec - exiting 03:18:36 (440): No heartbeat from core client for 30 sec - exiting 03:18:37 (440): No heartbeat from core client for 30 sec - exiting 03:18:38 (440): No heartbeat from core client for 30 sec - exiting 03:18:39 (440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:19:31 (4148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:07:33 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:34 (4456): No heartbeat from core client for 30 sec - exiting 00:07:35 (4456): No heartbeat from core client for 30 sec - exiting 00:07:36 (4456): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:20:33 (1196): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 17:20:34 (1196): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:14:44 (3404): No heartbeat from core client for 30 sec - exiting 03:14:45 (3404): No heartbeat from core client for 30 sec - exiting 03:14:47 (3404): No heartbeat from core client for 30 sec - exiting 03:14:48 (3404): No heartbeat from core client for 30 sec - exiting 03:14:49 (3404): No heartbeat from core client for 30 sec - exiting 03:14:50 (3404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:14:51 (3404): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:13:04 (3448): No heartbeat from core client for 30 sec - exiting 03:13:05 (3448): No heartbeat from core client for 30 sec - exiting 03:13:06 (3448): No heartbeat from core client for 30 sec - exiting 03:13:08 (3448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:13:09 (3448): No heartbeat from core client for 30 sec - exiting 03:16:39 (5392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:00:28 (1056): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 02:00:30 (1056): No heartbeat from core client for 30 sec - exiting 02:00:31 (1056): No heartbeat from core client for 30 sec - exiting 02:00:32 (1056): No heartbeat from core client for 30 sec - exiting 03:14:01 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:15:49 (5708): No heartbeat from core client for 30 sec - exiting 03:15:50 (5708): No heartbeat from core client for 30 sec - exiting 03:15:51 (5708): No heartbeat from core client for 30 sec - exiting 03:15:52 (5708): No heartbeat from core client for 30 sec - exiting 03:15:53 (5708): No heartbeat from core client for 30 sec - exiting 03:15:54 (5708): No heartbeat from core client for 30 sec - exiting 03:15:55 (5708): No heartbeat from core client for 30 sec - exiting 03:15:56 (5708): No heartbeat from core client for 30 sec - exiting 03:15:58 (5708): No heartbeat from core client for 30 sec - exiting 03:15:59 (5708): No heartbeat from core client for 30 sec - exiting 03:16:00 (5708): No heartbeat from core client for 30 sec - exiting 03:16:01 (5708): No heartbeat from core client for 30 sec - exiting 03:16:02 (5708): No heartbeat from core client for 30 sec - exiting 03:16:03 (5708): No heartbeat from core client for 30 sec - exiting 03:16:04 (5708): No heartbeat from core client for 30 sec - exiting 03:16:05 (5708): No heartbeat from core client for 30 sec - exiting 03:16:06 (5708): No heartbeat from core client for 30 sec - exiting 03:16:07 (5708): No heartbeat from core client for 30 sec - exiting 03:16:08 (5708): No heartbeat from core client for 30 sec - exiting 03:16:09 (5708): No heartbeat from core client for 30 sec - exiting 03:16:10 (5708): No heartbeat from core client for 30 sec - exiting 03:16:12 (5708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:16:13 (5708): No heartbeat from core client for 30 sec - exiting 03:16:14 (5708): No heartbeat from core client for 30 sec - exiting 03:16:15 (5708): No heartbeat from core client for 30 sec - exiting 03:16:16 (5708): No heartbeat from core client for 30 sec - exiting 03:17:20 (4916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:58:09 (4336): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 21:58:10 (4336): No heartbeat from core client for 30 sec - exiting 21:58:12 (4336): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:53:05 (5068): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:15:21 (5756): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 00:15:22 (5756): No heartbeat from core client for 30 sec - exiting 00:15:23 (5756): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:13:36 (6008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:14:40 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:18:51 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:20:31 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:15:36 (4700): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 11:15:37 (4700): No heartbeat from core client for 30 sec - exiting 11:15:38 (4700): No heartbeat from core client for 30 sec - exiting 11:15:39 (4700): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6208, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 07:51:07 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1220, iMonCtr=1 Model crash detected, will try to restart... 03:12:42 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:14:31 (6068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:58:35 (9132): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 15:58:37 (9132): No heartbeat from core client for 30 sec - exiting 15:58:38 (9132): No heartbeat from core client for 30 sec - exiting 15:58:39 (9132): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:20:33 (1216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:20:34 (1216): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:12:47 (4288): No heartbeat from core client for 30 sec - exiting 03:12:48 (4288): No heartbeat from core client for 30 sec - exiting 03:12:49 (4288): No heartbeat from core client for 30 sec - exiting 03:12:50 (4288): No heartbeat from core client for 30 sec - exiting 03:12:51 (4288): No heartbeat from core client for 30 sec - exiting 03:12:52 (4288): No heartbeat from core client for 30 sec - exiting 03:12:53 (4288): No heartbeat from core client for 30 sec - exiting 03:12:54 (4288): No heartbeat from core client for 30 sec - exiting 03:12:55 (4288): No heartbeat from core client for 30 sec - exiting 03:12:56 (4288): No heartbeat from core client for 30 sec - exiting 03:12:58 (4288): No heartbeat from core client for 30 sec - exiting 03:12:59 (4288): No heartbeat from core client for 30 sec - exiting 03:13:00 (4288): No heartbeat from core client for 30 sec - exiting 03:13:01 (4288): No heartbeat from core client for 30 sec - exiting 03:13:02 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:13:46 (6004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:07:07 (1504): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 01:07:08 (1504): No heartbeat from core client for 30 sec - exiting 01:07:09 (1504): No heartbeat from core client for 30 sec - exiting 03:15:22 (5676): No heartbeat from core client for 30 sec - exiting 03:15:23 (5676): No heartbeat from core client for 30 sec - exiting 03:15:24 (5676): No heartbeat from core client for 30 sec - exiting 03:15:25 (5676): No heartbeat from core client for 30 sec - exiting 03:15:26 (5676): No heartbeat from core client for 30 sec - exiting 03:15:27 (5676): No heartbeat from core client for 30 sec - exiting 03:15:28 (5676): No heartbeat from core client for 30 sec - exiting 03:15:29 (5676): No heartbeat from core client for 30 sec - exiting 03:15:30 (5676): No heartbeat from core client for 30 sec - exiting 03:15:31 (5676): No heartbeat from core client for 30 sec - exiting 03:15:32 (5676): No heartbeat from core client for 30 sec - exiting 03:15:33 (5676): No heartbeat from core client for 30 sec - exiting 03:15:34 (5676): No heartbeat from core client for 30 sec - exiting 03:15:35 (5676): No heartbeat from core client for 30 sec - exiting 03:15:36 (5676): No heartbeat from core client for 30 sec - exiting 03:15:37 (5676): No heartbeat from core client for 30 sec - exiting 03:15:38 (5676): No heartbeat from core client for 30 sec - exiting 03:15:39 (5676): No heartbeat from core client for 30 sec - exiting 03:15:40 (5676): No heartbeat from core client for 30 sec - exiting 03:15:41 (5676): No heartbeat from core client for 30 sec - exiting 03:15:42 (5676): No heartbeat from core client for 30 sec - exiting 03:15:43 (5676): No heartbeat from core client for 30 sec - exiting 03:15:44 (5676): No heartbeat from core client for 30 sec - exiting 03:15:45 (5676): No heartbeat from core client for 30 sec - exiting 03:15:46 (5676): No heartbeat from core client for 30 sec - exiting 03:15:47 (5676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:21:34 (5772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:21:35 (5772): No heartbeat from core client for 30 sec - exiting 17:24:16 (5680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:08 (7812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:59:16 (5044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:29:49 (6052): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 23:29:50 (6052): No heartbeat from core client for 30 sec - exiting 23:29:51 (6052): No heartbeat from core client for 30 sec - exiting 23:29:53 (6052): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 23:12:30 (1952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:13:27 (7196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:10:15 (5692): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:12:00 (4560): No heartbeat from core client for 30 sec - exiting 03:12:01 (4560): No heartbeat from core client for 30 sec - exiting 03:12:02 (4560): No heartbeat from core client for 30 sec - exiting 03:12:03 (4560): No heartbeat from core client for 30 sec - exiting 03:12:04 (4560): No heartbeat from core client for 30 sec - exiting 03:12:05 (4560): No heartbeat from core client for 30 sec - exiting 03:12:06 (4560): No heartbeat from core client for 30 sec - exiting 03:12:07 (4560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:12:08 (4560): No heartbeat from core client for 30 sec - exiting 03:12:10 (4560): No heartbeat from core client for 30 sec - exiting 03:12:11 (4560): No heartbeat from core client for 30 sec - exiting 03:12:55 (5752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:20:31 (3232): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 17:20:32 (3232): No heartbeat from core client for 30 sec - exiting 17:20:33 (3232): No heartbeat from core client for 30 sec - exiting 17:20:34 (3232): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10188, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 13:39:00 (5844): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 13:39:02 (5844): No heartbeat from core client for 30 sec - exiting 13:39:03 (5844): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:06:43 (6576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:07:32 (6700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:07:02 (9356): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 15:07:04 (9356): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:56:21 (5172): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:56:23 (5172): No heartbeat from core client for 30 sec - exiting 10:56:24 (5172): No heartbeat from core client for 30 sec - exiting 10:56:25 (5172): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:05:58 (2480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:05:59 (2480): No heartbeat from core client for 30 sec - exiting 00:06:00 (2480): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:08:27 (4236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Abort request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Sep 2013 09:14:11	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	544,320	889,229	1.6337
23 Sep 2013 11:01:10	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	518,400	846,775	1.6334
20 Sep 2013 07:18:47	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	492,480	804,057	1.6327
18 Sep 2013 07:22:32	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	466,560	761,951	1.6331
17 Sep 2013 01:31:01	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	440,640	719,708	1.6333
15 Sep 2013 11:09:47	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	414,720	677,396	1.6334
13 Sep 2013 10:01:21	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	388,800	635,093	1.6335
12 Sep 2013 02:57:11	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	362,880	593,248	1.6348
10 Sep 2013 07:08:49	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	336,960	551,058	1.6354
05 Sep 2013 09:48:21	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	311,040	508,479	1.6348
04 Sep 2013 10:02:10	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	285,120	465,570	1.6329
03 Sep 2013 08:30:24	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	259,200	423,296	1.6331
02 Sep 2013 00:35:49	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	233,280	380,873	1.6327
31 Aug 2013 20:50:19	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	207,360	338,906	1.6344
29 Aug 2013 10:51:43	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	181,440	296,731	1.6354
27 Aug 2013 02:24:22	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	155,520	253,937	1.6328
25 Aug 2013 06:47:40	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	129,600	211,557	1.6324
24 Aug 2013 00:56:05	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	103,680	169,034	1.6303
22 Aug 2013 04:58:15	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	77,760	126,796	1.6306
20 Aug 2013 08:48:11	1231345	15916384	hadcm3n_3ee1_1980_40_008367598_3	51,840	83,829	1.6171