Task 13346799

Name	hadcm3n_o6sk_1940_40_007442543_1
Workunit	7640046
Created	8 Sep 2011, 21:33:57 UTC
Sent	8 Sep 2011, 21:34:16 UTC
Report deadline	9 Dec 2011, 5:01:27 UTC
Received	25 Nov 2011, 14:56:23 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	990394
Run time	73 days 5 hours 33 min 6 sec
CPU time	60 days 0 hours 14 min 26 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:45:45 (2076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:48:27 (4936): Can't acquire lockfile (32) - waiting 35s 00:48:40 (816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:06 (6028): Can't acquire lockfile (32) - waiting 35s 00:50:06 (4936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:56:15 (4976): Can't acquire lockfile (32) - waiting 35s 00:56:36 (6028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4780, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:08:58 (5012): Can't acquire lockfile (32) - waiting 35s 01:08:58 (4972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:14:35 (5528): Can't acquire lockfile (32) - waiting 35s 01:15:03 (5012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:51:41 (5528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:53:52 (5932): No heartbeat from core client for 30 sec - exiting 13:53:53 (5932): No heartbeat from core client for 30 sec - exiting 13:53:54 (5932): No heartbeat from core client for 30 sec - exiting 13:53:55 (5932): No heartbeat from core client for 30 sec - exiting 13:53:56 (5932): No heartbeat from core client for 30 sec - exiting 13:53:57 (5932): No heartbeat from core client for 30 sec - exiting 13:53:58 (5932): No heartbeat from core client for 30 sec - exiting 13:53:59 (5932): No heartbeat from core client for 30 sec - exiting 13:54:00 (5932): No heartbeat from core client for 30 sec - exiting 13:54:01 (5932): No heartbeat from core client for 30 sec - exiting 13:54:02 (5932): No heartbeat from core client for 30 sec - exiting 13:54:03 (5932): No heartbeat from core client for 30 sec - exiting 13:54:04 (5932): No heartbeat from core client for 30 sec - exiting 13:54:05 (5932): No heartbeat from core client for 30 sec - exiting 13:54:06 (5932): No heartbeat from core client for 30 sec - exiting 13:54:07 (5932): No heartbeat from core client for 30 sec - exiting 13:54:08 (5932): No heartbeat from core client for 30 sec - exiting 13:54:09 (5932): No heartbeat from core client for 30 sec - exiting 13:54:10 (5932): No heartbeat from core client for 30 sec - exiting 13:54:11 (5932): No heartbeat from core client for 30 sec - exiting 13:54:12 (5932): No heartbeat from core client for 30 sec - exiting 13:54:13 (5932): No heartbeat from core client for 30 sec - exiting 13:54:14 (5932): No heartbeat from core client for 30 sec - exiting 13:54:15 (5932): No heartbeat from core client for 30 sec - exiting 13:54:16 (5932): No heartbeat from core client for 30 sec - exiting 13:54:17 (5932): No heartbeat from core client for 30 sec - exiting 13:54:18 (5932): No heartbeat from core client for 30 sec - exiting 13:54:19 (5932): No heartbeat from core client for 30 sec - exiting 13:54:20 (5932): No heartbeat from core client for 30 sec - exiting 13:54:21 (5932): No heartbeat from core client for 30 sec - exiting 13:54:22 (5932): No heartbeat from core client for 30 sec - exiting 13:54:23 (5932): No heartbeat from core client for 30 sec - exiting 13:54:24 (5932): No heartbeat from core client for 30 sec - exiting 13:54:25 (5932): No heartbeat from core client for 30 sec - exiting 13:54:26 (5932): No heartbeat from core client for 30 sec - exiting 13:54:27 (5932): No heartbeat from core client for 30 sec - exiting 13:54:28 (5932): No heartbeat from core client for 30 sec - exiting 13:54:29 (5932): No heartbeat from core client for 30 sec - exiting 13:54:30 (5932): No heartbeat from core client for 30 sec - exiting 13:54:31 (5932): No heartbeat from core client for 30 sec - exiting 13:54:32 (5932): No heartbeat from core client for 30 sec - exiting 13:54:33 (5932): No heartbeat from core client for 30 sec - exiting 13:54:34 (5932): No heartbeat from core client for 30 sec - exiting 13:54:35 (5932): No heartbeat from core client for 30 sec - exiting 13:54:36 (5932): No heartbeat from core client for 30 sec - exiting 13:54:37 (5932): No heartbeat from core client for 30 sec - exiting 13:54:38 (5932): No heartbeat from core client for 30 sec - exiting 13:54:39 (5932): No heartbeat from core client for 30 sec - exiting 13:54:40 (5932): No heartbeat from core client for 30 sec - exiting 13:54:41 (5932): No heartbeat from core client for 30 sec - exiting 13:54:42 (5932): No heartbeat from core client for 30 sec - exiting 13:54:43 (5932): No heartbeat from core client for 30 sec - exiting 13:54:44 (5932): No heartbeat from core client for 30 sec - exiting 13:54:45 (5932): No heartbeat from core client for 30 sec - exiting 13:54:46 (5932): No heartbeat from core client for 30 sec - exiting 13:54:47 (5932): No heartbeat from core client for 30 sec - exiting 13:54:48 (5932): No heartbeat from core client for 30 sec - exiting 13:54:49 (5932): No heartbeat from core client for 30 sec - exiting 13:54:50 (5932): No heartbeat from core client for 30 sec - exiting 13:54:51 (5932): No heartbeat from core client for 30 sec - exiting 13:54:52 (5932): No heartbeat from core client for 30 sec - exiting 13:54:53 (5932): No heartbeat from core client for 30 sec - exiting 13:54:54 (5932): No heartbeat from core client for 30 sec - exiting 13:54:55 (5932): No heartbeat from core client for 30 sec - exiting 13:54:56 (5932): No heartbeat from core client for 30 sec - exiting 13:54:57 (5932): No heartbeat from core client for 30 sec - exiting 13:54:58 (5932): No heartbeat from core client for 30 sec - exiting 13:54:59 (5932): No heartbeat from core client for 30 sec - exiting 13:55:00 (5932): No heartbeat from core client for 30 sec - exiting 13:55:01 (5932): No heartbeat from core client for 30 sec - exiting 13:55:02 (5932): No heartbeat from core client for 30 sec - exiting 13:55:03 (5932): No heartbeat from core client for 30 sec - exiting 13:55:04 (5932): No heartbeat from core client for 30 sec - exiting 13:55:05 (5932): No heartbeat from core client for 30 sec - exiting 13:55:06 (5932): No heartbeat from core client for 30 sec - exiting 13:55:07 (5932): No heartbeat from core client for 30 sec - exiting 13:55:08 (5932): No heartbeat from core client for 30 sec - exiting 13:55:09 (5932): No heartbeat from core client for 30 sec - exiting 13:55:10 (5932): No heartbeat from core client for 30 sec - exiting 13:55:11 (5932): No heartbeat from core client for 30 sec - exiting 13:55:12 (5932): No heartbeat from core client for 30 sec - exiting 13:55:13 (5932): No heartbeat from core client for 30 sec - exiting 13:55:14 (5932): No heartbeat from core client for 30 sec - exiting 13:55:15 (5932): No heartbeat from core client for 30 sec - exiting 13:55:16 (5932): No heartbeat from core client for 30 sec - exiting 13:55:17 (5932): No heartbeat from core client for 30 sec - exiting 13:55:18 (5932): No heartbeat from core client for 30 sec - exiting 13:55:19 (5932): No heartbeat from core client for 30 sec - exiting 13:55:20 (5932): No heartbeat from core client for 30 sec - exiting 13:55:21 (5932): No heartbeat from core client for 30 sec - exiting 13:55:22 (5932): No heartbeat from core client for 30 sec - exiting 13:55:23 (5932): No heartbeat from core client for 30 sec - exiting 13:55:24 (5932): No heartbeat from core client for 30 sec - exiting 13:55:25 (5932): No heartbeat from core client for 30 sec - exiting 13:55:26 (5932): No heartbeat from core client for 30 sec - exiting 13:55:27 (5932): No heartbeat from core client for 30 sec - exiting 13:55:28 (5932): No heartbeat from core client for 30 sec - exiting 13:55:29 (5932): No heartbeat from core client for 30 sec - exiting 13:55:30 (5932): No heartbeat from core client for 30 sec - exiting 13:55:31 (5932): No heartbeat from core client for 30 sec - exiting 13:55:32 (5932): No heartbeat from core client for 30 sec - exiting 13:55:33 (5932): No heartbeat from core client for 30 sec - exiting 13:55:34 (5932): No heartbeat from core client for 30 sec - exiting 13:55:35 (5932): No heartbeat from core client for 30 sec - exiting 13:55:36 (5932): No heartbeat from core client for 30 sec - exiting 13:55:37 (5932): No heartbeat from core client for 30 sec - exiting 13:55:38 (5932): No heartbeat from core client for 30 sec - exiting 13:55:39 (5932): No heartbeat from core client for 30 sec - exiting 13:55:40 (5932): No heartbeat from core client for 30 sec - exiting 13:55:41 (5932): No heartbeat from core client for 30 sec - exiting 13:55:42 (5932): No heartbeat from core client for 30 sec - exiting 13:55:43 (5932): No heartbeat from core client for 30 sec - exiting 13:55:44 (5932): No heartbeat from core client for 30 sec - exiting 13:55:45 (5932): No heartbeat from core client for 30 sec - exiting 13:55:46 (5932): No heartbeat from core client for 30 sec - exiting 13:55:47 (5932): No heartbeat from core client for 30 sec - exiting 13:55:48 (5932): No heartbeat from core client for 30 sec - exiting 13:55:49 (5932): No heartbeat from core client for 30 sec - exiting 13:55:50 (5932): No heartbeat from core client for 30 sec - exiting 13:55:51 (5932): No heartbeat from core client for 30 sec - exiting 13:55:52 (5932): No heartbeat from core client for 30 sec - exiting 13:55:53 (5932): No heartbeat from core client for 30 sec - exiting 13:55:54 (5932): No heartbeat from core client for 30 sec - exiting 13:55:55 (5932): No heartbeat from core client for 30 sec - exiting 13:55:56 (5932): No heartbeat from core client for 30 sec - exiting 13:55:57 (5932): No heartbeat from core client for 30 sec - exiting 13:55:58 (5932): No heartbeat from core client for 30 sec - exiting 13:55:59 (5932): No heartbeat from core client for 30 sec - exiting 13:56:00 (5932): No heartbeat from core client for 30 sec - exiting 13:56:01 (5932): No heartbeat from core client for 30 sec - exiting 13:56:02 (5932): No heartbeat from core client for 30 sec - exiting 13:56:03 (5932): No heartbeat from core client for 30 sec - exiting 13:56:04 (5932): No heartbeat from core client for 30 sec - exiting 13:56:05 (5932): No heartbeat from core client for 30 sec - exiting 13:56:06 (5932): No heartbeat from core client for 30 sec - exiting 13:56:07 (5932): No heartbeat from core client for 30 sec - exiting 13:56:08 (5932): No heartbeat from core client for 30 sec - exiting 13:56:09 (5932): No heartbeat from core client for 30 sec - exiting 13:56:10 (5932): No heartbeat from core client for 30 sec - exiting 13:56:11 (5932): No heartbeat from core client for 30 sec - exiting 13:56:12 (5932): No heartbeat from core client for 30 sec - exiting 13:56:13 (5932): No heartbeat from core client for 30 sec - exiting 13:56:14 (5932): No heartbeat from core client for 30 sec - exiting 13:56:15 (5932): No heartbeat from core client for 30 sec - exiting 13:56:16 (5932): No heartbeat from core client for 30 sec - exiting 13:56:17 (5932): No heartbeat from core client for 30 sec - exiting 13:56:18 (5932): No heartbeat from core client for 30 sec - exiting 13:56:19 (5932): No heartbeat from core client for 30 sec - exiting 13:56:20 (5932): No heartbeat from core client for 30 sec - exiting 13:56:21 (5932): No heartbeat from core client for 30 sec - exiting 13:56:22 (5932): No heartbeat from core client for 30 sec - exiting 13:56:23 (5932): No heartbeat from core client for 30 sec - exiting 13:56:24 (5932): No heartbeat from core client for 30 sec - exiting 13:56:25 (5932): No heartbeat from core client for 30 sec - exiting 13:56:26 (5932): No heartbeat from core client for 30 sec - exiting 13:56:27 (5932): No heartbeat from core client for 30 sec - exiting 13:56:28 (5932): No heartbeat from core client for 30 sec - exiting 13:56:29 (5932): No heartbeat from core client for 30 sec - exiting 13:56:30 (5932): No heartbeat from core client for 30 sec - exiting 13:56:31 (5932): No heartbeat from core client for 30 sec - exiting 13:56:32 (5932): No heartbeat from core client for 30 sec - exiting 13:56:33 (5932): No heartbeat from core client for 30 sec - exiting 13:56:34 (5932): No heartbeat from core client for 30 sec - exiting 13:56:35 (5932): No heartbeat from core client for 30 sec - exiting 13:56:36 (5932): No heartbeat from core client for 30 sec - exiting 13:56:37 (5932): No heartbeat from core client for 30 sec - exiting 13:56:38 (5932): No heartbeat from core client for 30 sec - exiting 13:56:39 (5932): No heartbeat from core client for 30 sec - exiting 13:56:40 (5932): No heartbeat from core client for 30 sec - exiting 13:56:41 (5932): No heartbeat from core client for 30 sec - exiting 13:56:42 (5932): No heartbeat from core client for 30 sec - exiting 13:56:43 (5932): No heartbeat from core client for 30 sec - exiting 13:56:44 (5932): No heartbeat from core client for 30 sec - exiting 13:56:45 (5932): No heartbeat from core client for 30 sec - exiting 13:56:46 (5932): No heartbeat from core client for 30 sec - exiting 13:56:47 (5932): No heartbeat from core client for 30 sec - exiting 13:56:48 (5932): No heartbeat from core client for 30 sec - exiting 13:56:49 (5932): No heartbeat from core client for 30 sec - exiting 13:56:50 (5932): No heartbeat from core client for 30 sec - exiting 13:56:51 (5932): No heartbeat from core client for 30 sec - exiting 13:56:52 (5932): No heartbeat from core client for 30 sec - exiting 13:56:53 (5932): No heartbeat from core client for 30 sec - exiting 13:56:56 (5932): No heartbeat from core client for 30 sec - exiting 13:56:57 (5932): No heartbeat from core client for 30 sec - exiting 13:56:58 (5932): No heartbeat from core client for 30 sec - exiting 13:56:59 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:29:52 (5212): Can't acquire lockfile (32) - waiting 35s 00:30:00 (2296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5212, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:54:36 (4720): Can't acquire lockfile (32) - waiting 35s 12:54:56 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:16:11 (2160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:54:38 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:50:41 (1296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:14:53 (4576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:11:06 (5252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:41:19 (5792): Can't acquire lockfile (32) - waiting 35s 09:41:24 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:30:14 (5596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Nov 2011 13:57:17	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	1,036,800	5,182,812	4.9989
23 Nov 2011 19:06:40	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	1,010,880	5,062,261	5.0078
21 Nov 2011 19:14:20	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	984,960	4,936,574	5.0120
19 Nov 2011 20:14:27	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	959,040	4,808,579	5.0140
18 Nov 2011 04:07:58	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	933,120	4,690,595	5.0268
16 Nov 2011 13:25:46	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	907,200	4,568,728	5.0361
15 Nov 2011 20:07:38	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	881,280	4,445,719	5.0446
15 Nov 2011 20:07:38	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	855,360	4,323,740	5.0549
15 Nov 2011 20:07:38	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	829,440	4,199,264	5.0628
09 Nov 2011 09:07:43	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	803,520	4,075,547	5.0721
07 Nov 2011 14:11:04	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	777,600	3,950,861	5.0808
05 Nov 2011 19:21:23	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	751,680	3,824,977	5.0886
04 Nov 2011 01:35:28	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	725,760	3,696,259	5.0929
02 Nov 2011 06:14:44	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	699,840	3,565,124	5.0942
31 Oct 2011 19:46:08	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	673,920	3,437,874	5.1013
31 Oct 2011 18:35:29	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	648,000	3,295,565	5.0857
31 Oct 2011 16:53:12	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	622,080	3,137,403	5.0434
31 Oct 2011 14:03:10	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	596,160	3,005,791	5.0419
31 Oct 2011 14:03:10	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	570,240	2,873,599	5.0393
31 Oct 2011 14:03:09	990394	13346799	hadcm3n_o6sk_1940_40_007442543_1	544,320	2,740,466	5.0347