Task 15441480

Name	hadcm3n_zj7e_1880_40_008246042_1
Workunit	8401166
Created	20 Nov 2012, 22:50:25 UTC
Sent	20 Nov 2012, 22:50:32 UTC
Report deadline	20 Feb 2013, 6:17:43 UTC
Received	27 Jan 2013, 4:20:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1254387
Run time	64 days 18 hours 8 min 28 sec
CPU time	1 days 14 hours 44 min 3 sec
Validate state	Invalid
Credit	7,153.92
Device peak FLOPS	1.63 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 forrtl: The requested operation cannot be performed on a file with a user-mapped section open. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2460, iMonCtr=1 Model crash detected, will try to restart... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 forrtl: The requested operation cannot be performed on a file with a user-mapped section open. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2460, iMonCtr=1 Model crash detected, will try to restart... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 05:45:02 (2460): No heartbeat from core client for 30 sec - exiting 05:45:03 (2460): No heartbeat from core client for 30 sec - exiting 05:45:05 (2460): No heartbeat from core client for 30 sec - exiting 05:45:06 (2460): No heartbeat from core client for 30 sec - exiting 05:45:07 (2460): No heartbeat from core client for 30 sec - exiting 05:45:08 (2460): No heartbeat from core client for 30 sec - exiting 05:45:09 (2460): No heartbeat from core client for 30 sec - exiting 05:45:10 (2460): No heartbeat from core client for 30 sec - exiting 05:45:11 (2460): No heartbeat from core client for 30 sec - exiting 05:45:12 (2460): No heartbeat from core client for 30 sec - exiting 05:45:13 (2460): No heartbeat from core client for 30 sec - exiting 05:45:14 (2460): No heartbeat from core client for 30 sec - exiting 05:45:15 (2460): No heartbeat from core client for 30 sec - exiting 05:45:17 (2460): No heartbeat from core client for 30 sec - exiting 05:45:18 (2460): No heartbeat from core client for 30 sec - exiting 05:45:19 (2460): No heartbeat from core client for 30 sec - exiting 05:45:20 (2460): No heartbeat from core client for 30 sec - exiting 05:45:21 (2460): No heartbeat from core client for 30 sec - exiting 05:45:22 (2460): No heartbeat from core client for 30 sec - exiting 05:45:23 (2460): No heartbeat from core client for 30 sec - exiting 05:45:24 (2460): No heartbeat from core client for 30 sec - exiting 05:45:25 (2460): No heartbeat from core client for 30 sec - exiting 05:45:26 (2460): No heartbeat from core client for 30 sec - exiting 05:45:28 (2460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 20:07:27 (9028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zj7eko.pja1c10 Error converting file to netcdf: dataout/zj7eko.pia1c10 Error converting file to netcdf: dataout/zj7eko.pfa1c10 Error converting file to netcdf: dataout/zj7eka.pha1c10 Error converting file to netcdf: dataout/zj7eka.pga1c10 Error converting file to netcdf: dataout/zj7eka.pea1c10 Error converting file to netcdf: dataout/zj7eka.pda1c10 CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 forrtl: The requested operation cannot be performed on a file with a user-mapped section open. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1828, iMonCtr=1 Model crash detected, will try to restart... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 22:32:50 (1828): No heartbeat from core client for 30 sec - exiting 22:32:51 (1828): No heartbeat from core client for 30 sec - exiting 22:32:52 (1828): No heartbeat from core client for 30 sec - exiting 22:32:54 (1828): No heartbeat from core client for 30 sec - exiting 22:32:55 (1828): No heartbeat from core client for 30 sec - exiting 22:32:56 (1828): No heartbeat from core client for 30 sec - exiting 22:32:57 (1828): No heartbeat from core client for 30 sec - exiting 22:32:59 (1828): No heartbeat from core client for 30 sec - exiting 22:33:00 (1828): No heartbeat from core client for 30 sec - exiting 22:33:01 (1828): No heartbeat from core client for 30 sec - exiting 22:33:02 (1828): No heartbeat from core client for 30 sec - exiting 22:33:04 (1828): No heartbeat from core client for 30 sec - exiting 22:33:05 (1828): No heartbeat from core client for 30 sec - exiting 22:33:06 (1828): No heartbeat from core client for 30 sec - exiting 22:33:07 (1828): No heartbeat from core client for 30 sec - exiting 22:33:09 (1828): No heartbeat from core client for 30 sec - exiting 22:33:10 (1828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:33:11 (1828): No heartbeat from core client for 30 sec - exiting MainError: 03:33:37 AM No files match the supplied pattern. MainError: 03:33:37 AM No files match the supplied pattern. Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2264, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2264, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2264, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2264, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2264, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2264, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Jan 2013 03:24:58	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	596,160	139,666	0.2343
23 Jan 2013 01:36:53	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	570,240	138,863	0.2435
21 Jan 2013 00:59:40	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	544,320	133,098	0.2445
19 Jan 2013 10:35:30	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	518,400	88,293	0.1703
16 Jan 2013 13:20:21	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	492,480	87,823	0.1783
14 Jan 2013 11:29:33	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	466,560	91,062	0.1952
08 Jan 2013 16:56:08	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	440,640	74,579	0.1693
04 Jan 2013 13:54:34	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	414,720	60,972	0.1470
01 Jan 2013 02:14:58	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	388,800	60,023	0.1544
29 Dec 2012 06:39:49	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	362,880	58,288	0.1606
25 Dec 2012 09:07:31	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	336,960	35,720	0.1060
22 Dec 2012 12:30:09	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	311,040	30,012	0.0965
20 Dec 2012 09:16:57	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	285,120	40,009	0.1403
18 Dec 2012 07:56:30	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	259,200	33,803	0.1304
16 Dec 2012 05:59:00	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	233,280	27,560	0.1181
14 Dec 2012 02:54:39	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	207,360	27,905	0.1346
14 Dec 2012 00:43:43	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	181,440	22,570	0.1244
14 Dec 2012 00:43:43	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	155,520	18,629	0.1198
07 Dec 2012 20:12:17	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	129,600	11,174	0.0862
05 Dec 2012 13:05:26	1254387	15441480	hadcm3n_zj7e_1880_40_008246042_1	103,680	12,856	0.1240