Task 15456094

Name	hadcm3n_zbkk_1880_40_008251998_1
Workunit	8407122
Created	23 Nov 2012, 10:46:14 UTC
Sent	23 Nov 2012, 10:46:49 UTC
Report deadline	22 Feb 2013, 18:14:00 UTC
Received	7 Mar 2013, 9:53:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1228571
Run time	18 days 23 hours 25 min 26 sec
CPU time	17 days 7 hours 21 min 24 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.71 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4600, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5844, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:09:05 (5752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:51:25 (3068): No heartbeat from core client for 30 sec - exiting 10:51:26 (3068): No heartbeat from core client for 30 sec - exiting 10:51:27 (3068): No heartbeat from core client for 30 sec - exiting 10:51:28 (3068): No heartbeat from core client for 30 sec - exiting 10:51:29 (3068): No heartbeat from core client for 30 sec - exiting 10:51:30 (3068): No heartbeat from core client for 30 sec - exiting 10:51:31 (3068): No heartbeat from core client for 30 sec - exiting 10:51:33 (3068): No heartbeat from core client for 30 sec - exiting 10:51:34 (3068): No heartbeat from core client for 30 sec - exiting 10:51:35 (3068): No heartbeat from core client for 30 sec - exiting 10:51:36 (3068): No heartbeat from core client for 30 sec - exiting 10:51:37 (3068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:41:53 (5724): No heartbeat from core client for 30 sec - exiting 07:41:54 (5724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:59:07 (5436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:15:02 (6056): No heartbeat from core client for 30 sec - exiting 16:15:03 (6056): No heartbeat from core client for 30 sec - exiting 16:15:04 (6056): No heartbeat from core client for 30 sec - exiting 16:15:05 (6056): No heartbeat from core client for 30 sec - exiting 16:15:06 (6056): No heartbeat from core client for 30 sec - exiting 16:15:07 (6056): No heartbeat from core client for 30 sec - exiting 16:15:08 (6056): No heartbeat from core client for 30 sec - exiting 16:15:09 (6056): No heartbeat from core client for 30 sec - exiting 16:15:10 (6056): No heartbeat from core client for 30 sec - exiting 16:15:11 (6056): No heartbeat from core client for 30 sec - exiting 16:15:12 (6056): No heartbeat from core client for 30 sec - exiting 16:15:13 (6056): No heartbeat from core client for 30 sec - exiting 16:15:14 (6056): No heartbeat from core client for 30 sec - exiting 16:15:15 (6056): No heartbeat from core client for 30 sec - exiting 16:15:16 (6056): No heartbeat from core client for 30 sec - exiting 16:15:17 (6056): No heartbeat from core client for 30 sec - exiting 16:15:18 (6056): No heartbeat from core client for 30 sec - exiting 16:15:19 (6056): No heartbeat from core client for 30 sec - exiting 16:15:20 (6056): No heartbeat from core client for 30 sec - exiting 16:15:21 (6056): No heartbeat from core client for 30 sec - exiting 16:15:22 (6056): No heartbeat from core client for 30 sec - exiting 16:15:23 (6056): No heartbeat from core client for 30 sec - exiting 16:15:24 (6056): No heartbeat from core client for 30 sec - exiting 16:15:25 (6056): No heartbeat from core client for 30 sec - exiting 16:15:26 (6056): No heartbeat from core client for 30 sec - exiting 16:15:27 (6056): No heartbeat from core client for 30 sec - exiting 16:15:28 (6056): No heartbeat from core client for 30 sec - exiting 16:15:29 (6056): No heartbeat from core client for 30 sec - exiting 16:15:30 (6056): No heartbeat from core client for 30 sec - exiting 16:15:31 (6056): No heartbeat from core client for 30 sec - exiting 16:15:32 (6056): No heartbeat from core client for 30 sec - exiting 16:15:33 (6056): No heartbeat from core client for 30 sec - exiting 16:15:34 (6056): No heartbeat from core client for 30 sec - exiting 16:15:35 (6056): No heartbeat from core client for 30 sec - exiting 16:15:36 (6056): No heartbeat from core client for 30 sec - exiting 16:15:37 (6056): No heartbeat from core client for 30 sec - exiting 16:15:38 (6056): No heartbeat from core client for 30 sec - exiting 16:15:39 (6056): No heartbeat from core client for 30 sec - exiting 16:15:40 (6056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:52:37 (5264): No heartbeat from core client for 30 sec - exiting 16:52:38 (5264): No heartbeat from core client for 30 sec - exiting 16:52:39 (5264): No heartbeat from core client for 30 sec - exiting 16:52:40 (5264): No heartbeat from core client for 30 sec - exiting 16:52:41 (5264): No heartbeat from core client for 30 sec - exiting 16:52:42 (5264): No heartbeat from core client for 30 sec - exiting 16:52:43 (5264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3632, iMonCtr=1 Model crash detected, will try to restart... 09:21:25 (6108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4688, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5904, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:28:12 (5244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:08:13 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:46:48 (4832): No heartbeat from core client for 30 sec - exiting 02:46:49 (4832): No heartbeat from core client for 30 sec - exiting 02:46:50 (4832): No heartbeat from core client for 30 sec - exiting 02:46:51 (4832): No heartbeat from core client for 30 sec - exiting 02:46:52 (4832): No heartbeat from core client for 30 sec - exiting 02:46:53 (4832): No heartbeat from core client for 30 sec - exiting 02:46:54 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:46:55 (4832): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4500, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:35:46 (4496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:35:47 (4496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 13:00:01 (5704): No heartbeat from core client for 30 sec - exiting 13:00:02 (5704): No heartbeat from core client for 30 sec - exiting 13:00:03 (5704): No heartbeat from core client for 30 sec - exiting 13:00:05 (5704): No heartbeat from core client for 30 sec - exiting 13:00:06 (5704): No heartbeat from core client for 30 sec - exiting 13:00:07 (5704): No heartbeat from core client for 30 sec - exiting 13:00:08 (5704): No heartbeat from core client for 30 sec - exiting 13:00:09 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 04:25:19 (2460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5548, iMonCtr=1 Model crash detected, will try to restart... 19:15:33 (5468): No heartbeat from core client for 30 sec - exiting 19:15:35 (5468): No heartbeat from core client for 30 sec - exiting 19:15:36 (5468): No heartbeat from core client for 30 sec - exiting 19:15:37 (5468): No heartbeat from core client for 30 sec - exiting 19:15:38 (5468): No heartbeat from core client for 30 sec - exiting 19:15:39 (5468): No heartbeat from core client for 30 sec - exiting 19:15:40 (5468): No heartbeat from core client for 30 sec - exiting 19:15:41 (5468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:15:42 (5468): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5164, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5164, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:44:31 (3688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... 12:04:53 (4380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:05:51 (3008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:12:00 (5876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:53:07 (5984): No heartbeat from core client for 30 sec - exiting 11:53:08 (5984): No heartbeat from core client for 30 sec - exiting 11:53:09 (5984): No heartbeat from core client for 30 sec - exiting 11:53:10 (5984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:44:21 (4436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:19:21 (4764): No heartbeat from core client for 30 sec - exiting 12:19:22 (4764): No heartbeat from core client for 30 sec - exiting 12:19:23 (4764): No heartbeat from core client for 30 sec - exiting 12:19:24 (4764): No heartbeat from core client for 30 sec - exiting 12:19:26 (4764): No heartbeat from core client for 30 sec - exiting 12:19:27 (4764): No heartbeat from core client for 30 sec - exiting 12:19:28 (4764): No heartbeat from core client for 30 sec - exiting 12:19:29 (4764): No heartbeat from core client for 30 sec - exiting 12:19:30 (4764): No heartbeat from core client for 30 sec - exiting 12:19:31 (4764): No heartbeat from core client for 30 sec - exiting 12:19:32 (4764): No heartbeat from core client for 30 sec - exiting 12:19:33 (4764): No heartbeat from core client for 30 sec - exiting 12:19:34 (4764): No heartbeat from core client for 30 sec - exiting 12:19:35 (4764): No heartbeat from core client for 30 sec - exiting 12:19:36 (4764): No heartbeat from core client for 30 sec - exiting 12:19:37 (4764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:21:15 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:21:50 (5088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77121CAF read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Mar 2013 00:02:03	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	1,036,800	1,494,778	1.4417
04 Mar 2013 19:00:46	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	1,010,880	1,457,029	1.4413
03 Mar 2013 21:14:05	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	984,960	1,419,484	1.4412
03 Mar 2013 02:29:21	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	959,040	1,381,789	1.4408
01 Mar 2013 00:25:26	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	933,120	1,344,304	1.4407
26 Feb 2013 21:08:05	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	907,200	1,307,303	1.4410
25 Feb 2013 17:32:36	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	881,280	1,270,797	1.4420
24 Feb 2013 14:35:50	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	855,360	1,234,030	1.4427
23 Feb 2013 18:42:16	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	829,440	1,196,737	1.4428
17 Feb 2013 00:15:16	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	803,520	1,158,192	1.4414
13 Feb 2013 17:16:09	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	777,600	1,120,876	1.4415
12 Feb 2013 19:16:54	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	751,680	1,084,111	1.4423
11 Feb 2013 21:00:34	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	725,760	1,046,903	1.4425
04 Feb 2013 00:30:52	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	699,840	1,008,345	1.4408
03 Feb 2013 13:05:15	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	673,920	970,652	1.4403
02 Feb 2013 17:41:36	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	648,000	933,235	1.4402
01 Feb 2013 14:35:45	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	622,080	896,036	1.4404
30 Jan 2013 14:25:23	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	596,160	858,287	1.4397
30 Jan 2013 03:12:26	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	570,240	821,556	1.4407
29 Jan 2013 16:10:41	1228571	15456094	hadcm3n_zbkk_1880_40_008251998_1	544,320	784,498	1.4412