Task 15476465

Name	hadcm3n_zidn_1880_40_008250761_3
Workunit	8405885
Created	14 Dec 2012, 3:17:09 UTC
Sent	14 Dec 2012, 3:22:26 UTC
Report deadline	15 Mar 2013, 10:49:37 UTC
Received	13 Feb 2013, 2:15:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-529697949 (0xE06D7363) Unknown error code
Computer ID	1005363
Run time	13 days 15 hours 27 min 34 sec
CPU time	13 days 0 hours 26 min 4 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.67 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -529697949 (0xe06d7363) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76D9D8CB Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 02/12/13 21:45:05 Install Directory : C:\Program Files (x86)\BOINC\ Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\Program Files (x86)\BOINC\\dbghelp.dll ): GetLastError = 193 Loaded Library : dbghelp.dll LoadLibraryA( C:\Program Files (x86)\BOINC\\symsrv.dll ): GetLastError = 193 LoadLibraryA( symsrv.dll ): GetLastError = 193 LoadLibraryA( C:\Program Files (x86)\BOINC\\srcsrv.dll ): GetLastError = 193 LoadLibraryA( srcsrv.dll ): GetLastError = 193 LoadLibraryA( C:\Program Files (x86)\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_zidn_1880_40_008250761;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00111000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77b10000 00160000 C:\Windows\SysWOW64\ntdll.dll (6.0.6002.18541) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 76d80000 00110000 C:\Windows\syswow64\kernel32.dll (6.0.6002.18704) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 75720000 000d0000 C:\Windows\syswow64\USER32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 76f20000 00090000 C:\Windows\syswow64\GDI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 758a0000 000c6000 C:\Windows\syswow64\ADVAPI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 75970000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.0.6002.18024) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 75630000 00060000 C:\Windows\syswow64\Secur32.dll (6.0.6002.18643) (-exported- Symbols Loaded) Linked PDB Filename : wsecur32.pdb File Version : 6.0.6002.18643 (vistasp2_gdr.120601-1144) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18643 ModLoad: 76980000 00060000 C:\Windows\system32\IMM32.DLL (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 771c0000 000c8000 C:\Windows\syswow64\MSCTF.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 757f0000 000aa000 C:\Windows\syswow64\msvcrt.dll (7.0.6002.18551) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.6002.18551 (vistasp2_gdr.111214-0301) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.6002.18551 ModLoad: 76d70000 00009000 C:\Windows\syswow64\LPK.DLL (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : wlpk.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 769e0000 0007d000 C:\Windows\syswow64\USP10.dll (1.626.6002.18244) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.6002.18244 (vistasp2_gdr.100416-0345) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.6002.18244 ModLoad: 75370000 00021000 C:\Windows\system32\NTMARTA.DLL (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : ntmarta.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 76fb0000 00049000 C:\Windows\syswow64\WLDAP32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : wldap32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 76b80000 0002d000 C:\Windows\syswow64\WS2_32.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : ws2_32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 76e90000 00006000 C:\Windows\syswow64\NSI.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : nsi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 75cd0000 00007000 C:\Windows\syswow64\PSAPI.DLL (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : psapi.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 75350000 00011000 C:\Windows\system32\SAMLIB.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : samlib.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 75a60000 00145000 C:\Windows\syswow64\ole32.dll (6.0.6002.18277) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 72b40000 0002c000 C:\Windows\system32\apphelp.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : apphelp.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 72ba0000 000dc000 C:\Windows\system32\dbghelp.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 75560000 00008000 C:\Windows\system32\version.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 376879, Write: 0, Other 112643 - I/O Transfers Counters - Read: 0, Write: 6732, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 107152, QuotaPeakPagedPoolUsage: 113736 QuotaNonPagedPoolUsage: 6544, QuotaPeakNonPagedPoolUsage: 6752 - Virtual Memory Usage - VirtualSize: 62447616, PeakVirtualSize: 146808832 - Pagefile Usage - PagefileUsage: 5103616, PeakPagefileUsage: 85897216 - Working Set Size - WorkingSetSize: 13029376, PeakWorkingSetSize: 93913088, PageFaultCount: 84048 * Dump of thread ID 5076 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 40872264.000000, User Time: 168481088.000000, Wait Time: 179834000.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76D9D8CB - Registers - eax=00112d48 ebx=00000004 ecx=00000003 edx=00000000 esi=004dd3cc edi=00000001 eip=76d9d8cb esp=00112d48 ebp=00112d98 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 00112d98 004748b8 e06d7363 00000001 00000003 00112dc4 kernel32!RaiseException+0x0 00112dd0 0047486f 00112de0 004b2d78 004965d8 004af0e8 hadcm3n_6.07_windows_intelx86!+0x0 00112dec 0040f434 000001b0 00c27898 00d319c8 00d307c8 hadcm3n_6.07_windows_intelx86!+0x0 00112e04 00412703 00000004 00000000 18a04004 00000003 hadcm3n_6.07_windows_intelx86!+0x0 00112e48 00471216 004dd168 00112ee0 004707cd 004dd168 hadcm3n_6.07_windows_intelx86!+0x0 00112e54 004707cd 004dd168 00112f40 00112f5c 00cd7340 hadcm3n_6.07_windows_intelx86!+0x0 00112ee0 0040f327 00112f40 00112fb4 00000a01 00000000 hadcm3n_6.07_windows_intelx86!+0x0 004119b1 41dab854 83e80049 8b000636 758b087d 245d8b20 hadcm3n_6.07_windows_intelx86!+0x0 004119b5 83e80049 8b000636 758b087d 245d8b20 00b06583 hadcm3n_6.07_windows_intelx86!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '41dab854' 004119b9 8b000636 758b087d 245d8b20 00b06583 89ac458d hadcm3n_6.07_windows_intelx86!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '83e80049' 004119bd 758b087d 245d8b20 00b06583 89ac458d 4d8ba44d hadcm3n_6.07_windows_intelx86!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '8b000636' 004119c1 245d8b20 00b06583 89ac458d 4d8ba44d b4e8501c ADVAPI32!CredProfileLoaded+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '758b087d' 758b087d 1ae2c1d6 0b02eec1 ffe281d6 890fffff e1810855 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '245d8b20' 758b0881 0b02eec1 ffe281d6 890fffff e1810855 0fffffff ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1ae2c1d6' 758b0885 ffe281d6 890fffff e1810855 0fffffff eed1f18b ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '0b02eec1' 758b0889 890fffff e1810855 0fffffff eed1f18b e281d68b ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = 'ffe281d6' 758b088d e1810855 0fffffff eed1f18b e281d68b 07000000 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '890fffff' 758b0891 0fffffff eed1f18b e281d68b 07000000 e781f98b ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = 'e1810855' 758b0895 eed1f18b e281d68b 07000000 e781f98b 00c00000 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '0fffffff' 758b0899 e281d68b 07000000 e781f98b 00c00000 ead1d70b ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = 'eed1f18b' 758b089d 07000000 e781f98b 00c00000 ead1d70b e781f98b ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = 'e281d68b' 758b08a1 e781f98b 00c00000 ead1d70b e781f98b 00100000 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '07000000' 758b08a5 00c00000 ead1d70b e781f98b 00100000 fe8bd70b ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = 'e781f98b' 758b08a9 ead1d70b e781f98b 00100000 fe8bd70b 0000e781 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '00c00000' 758b08ad e781f98b 00100000 fe8bd70b 0000e781 eac10006 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = 'ead1d70b' 758b08b1 00100000 fe8bd70b 0000e781 eac10006 95148b14 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = 'e781f98b' 758b08b5 fe8bd70b 0000e781 eac10006 95148b14 758b0ca0 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '00100000' 758b08b9 00000000 eac10006 95148b14 758b0ca0 0f00e681 ADVAPI32!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = 'fe8bd70b' * Dump of thread ID 1756 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 156001.000000, User Time: 0.000000, Wait Time: 179834000.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=0285ff48 edi=00000000 eip=77b5982d esp=0285ff08 ebp=0285ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0285ff6c 76d90c88 00000064 00000000 0285ff88 00413da4 ntdll!NtDelayExecution+0x0 0285ff7c 00413da4 00000064 0285ff94 76e0f271 00000000 kernel32!Sleep+0x0 0285ff88 76e0f271 00000000 0285ffd4 77b8d819 00000000 hadcm3n_6.07_windows_intelx86!+0x0 0285ff94 77b8d819 00000000 d31bacf3 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 0285ffd4 77b8da2b 00413d90 00000000 00000000 00000000 ntdll!RtlCreateUserProcess+0x0 0285ffec 00000000 00413d90 00000000 00000000 2054d64a ntdll!RtlCreateProcessParameters+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4732, selfPID=4732, iMonCtr=1 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Feb 2013 19:47:28	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	777,600	1,124,750	1.4464
12 Feb 2013 05:23:54	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	751,680	1,088,754	1.4484
11 Feb 2013 18:36:50	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	725,760	1,052,965	1.4508
11 Feb 2013 08:06:03	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	699,840	1,017,348	1.4537
10 Feb 2013 17:01:12	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	673,920	981,225	1.4560
10 Feb 2013 05:11:06	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	648,000	943,789	1.4565
09 Feb 2013 18:01:38	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	622,080	907,239	1.4584
08 Feb 2013 16:05:54	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	596,160	871,346	1.4616
08 Feb 2013 04:55:37	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	570,240	834,975	1.4643
07 Feb 2013 12:04:04	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	544,320	799,306	1.4684
20 Jan 2013 12:10:15	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	518,400	763,315	1.4724
19 Jan 2013 22:57:35	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	492,480	725,137	1.4724
19 Jan 2013 11:55:45	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	466,560	686,544	1.4715
19 Jan 2013 01:08:31	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	440,640	648,060	1.4707
18 Jan 2013 14:20:35	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	414,720	609,451	1.4695
18 Jan 2013 03:33:38	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	388,800	570,891	1.4683
17 Jan 2013 16:40:35	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	362,880	532,250	1.4667
17 Jan 2013 05:53:59	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	336,960	493,755	1.4653
16 Jan 2013 19:07:43	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	311,040	455,142	1.4633
16 Jan 2013 08:17:28	1005363	15476465	hadcm3n_zidn_1880_40_008250761_3	285,120	416,565	1.4610