climateprediction.net home page
Task 19954478

Task 19954478

Name wah2_sas50_ksah_198912_13_448_010710381_0
Workunit 10710381
Created 12 Sep 2016, 13:21:21 UTC
Sent 19 Sep 2016, 4:29:21 UTC
Report deadline 1 Sep 2017, 9:49:21 UTC
Received 13 Dec 2016, 2:53:33 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1403926
Run time 5 days 9 hours 17 min 2 sec
CPU time 5 days 9 hours 2 min 24 sec
Validate state Valid
Credit 9,898.90
Device peak FLOPS 3.37 GFLOPS
Application version Weather At Home 2 (wah2) (region independent) v8.12
windows_intelx86
Peak working set size 234.95 MB
Peak swap size 197.75 MB
Peak disk usage 90.68 MB
Stderr 
<core_client_version>7.6.22</core_client_version>
<![CDATA[
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x00463327 read attempt to address 0x0129B3AC

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 10/19/16 15:50:42
Install Directory : C:\Program Files (x86)\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\Program Files (x86)\BOINC\\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\Program Files (x86)\BOINC\\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\Program Files (x86)\BOINC\\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\Program Files (x86)\BOINC\\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\10;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000152000 C:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.12_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000772b0000 000000000017b000 C:\Windows\SYSTEM32\ntdll.dll (6.2.10586.306) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 0000000076db0000 00000000000e0000 C:\Windows\SYSTEM32\KERNEL32.DLL (6.2.10586.589) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 00000000741a0000 000000000017e000 C:\Windows\SYSTEM32\KERNELBASE.dll (6.2.10586.589) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 0000000074490000 0000000000147000 C:\Windows\SYSTEM32\USER32.dll (6.2.10586.306) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 00000000765b0000 000000000014f000 C:\Windows\SYSTEM32\GDI32.dll (6.2.10586.420) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.10586.420 (th2_release_sec.160527-1834)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.420

ModLoad: 0000000075ba0000 000000000007b000 C:\Windows\SYSTEM32\ADVAPI32.dll (6.2.10586.63) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 0000000076700000 00000000000be000 C:\Windows\SYSTEM32\msvcrt.dll (7.0.10586.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.10586.0

ModLoad: 0000000076980000 0000000000044000 C:\Windows\SYSTEM32\sechost.dll (6.2.10586.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 00000000740f0000 00000000000ad000 C:\Windows\SYSTEM32\RPCRT4.dll (6.2.10586.306) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 0000000073fe0000 000000000001e000 C:\Windows\SYSTEM32\SspiCli.dll (6.2.10586.589) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.10586.589 (th2_release.160906-1759)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.589

ModLoad: 0000000073fd0000 000000000000a000 C:\Windows\SYSTEM32\CRYPTBASE.dll (6.2.10586.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 0000000076550000 0000000000058000 C:\Windows\SYSTEM32\bcryptPrimitives.dll (6.2.10586.589) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.10586.589 (th2_release.160906-1759)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.589

ModLoad: 00000000736d0000 0000000000028000 C:\Windows\SYSTEM32\ntmarta.dll (6.2.10586.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntmarta.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 000000006e8b0000 0000000000071000 C:\Windows\SYSTEM32\efswrt.dll (6.2.10586.494) (-exported- Symbols Loaded)
    Linked PDB Filename   : efswrt.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 00000000764c0000 000000000008d000 C:\Windows\SYSTEM32\SHCORE.dll (6.2.10586.589) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 00000000767c0000 00000000001bd000 C:\Windows\SYSTEM32\combase.dll (6.2.10586.589) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 000000006e7e0000 00000000000c8000 C:\Windows\SYSTEM32\wintypes.dll (6.2.10586.103) (-exported- Symbols Loaded)
    Linked PDB Filename   : WinTypes.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 0000000075b90000 000000000000c000 C:\Windows\SYSTEM32\kernel.appcore.dll (6.2.10586.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

Get Product Name Failed.
ModLoad: 0000000075c20000 0000000000095000 C:\Windows\SYSTEM32\OLEAUT32.dll (6.2.10586.589) (-exported- Symbols Loaded)
    Linked PDB Filename   : oleaut32.pdb
    File Version          : 10.0.10586.589
    Company Name          : Microsoft Corporation
    Product Name          : 
    Product Version       : 10.0.10586.589

ModLoad: 000000006e790000 0000000000049000 C:\Windows\SYSTEM32\edputil.dll (6.2.10586.494) (-exported- Symbols Loaded)
    Linked PDB Filename   : edputil.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 0000000074070000 0000000000045000 C:\Windows\SYSTEM32\shlwapi.dll (6.2.10586.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 000000006e650000 000000000013f000 C:\Windows\SYSTEM32\dbghelp.dll (6.2.10586.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0

ModLoad: 0000000072b90000 0000000000008000 C:\Windows\SYSTEM32\version.dll (6.2.10586.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.10586.0 (th2_release.151029-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.0



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 17746, Write: 0, Other 62757

- I/O Transfers Counters -
Read: 0, Write: 3040, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 198248, QuotaPeakPagedPoolUsage: 198824
QuotaNonPagedPoolUsage: 8272, QuotaPeakNonPagedPoolUsage: 9360

- Virtual Memory Usage -
VirtualSize: 107257856, PeakVirtualSize: 109948928

- Pagefile Usage -
PagefileUsage: 2478080, PeakPagefileUsage: 3215360

- Working Set Size -
WorkingSetSize: 2682880, PeakWorkingSetSize: 79863808, PageFaultCount: 21473

*** Dump of thread ID 5932 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 35817692.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x00463327 read attempt to address 0x0129B3AC

- Registers -
eax=00000015 ebx=fffffffe ecx=00000000 edx=00000000 esi=011cb020 edi=00000000
eip=00463327 esp=009cfe7c ebp=009cfe80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010216

- Callstack -
ChildEBP RetAddr  Args to Child
009cfe80 004484ff 011cb020 00000000 011cb020 fffffffe wah2_8.12_windows_intelx86!+0x0 
009cfe98 0044fd08 011cb020 00000000 0051d6a8 009cfee8 wah2_8.12_windows_intelx86!+0x0 
009cfeac 00485b9c 00000015 28db1b9d 00000006 00485b0c wah2_8.12_windows_intelx86!+0x0 
009cfee8 742ba8bf 00000006 5dc9d8ea 742ba700 742ba700 wah2_8.12_windows_intelx86!+0x0 
009cff80 76dc38f4 80000006 76dc38d0 6bf491b4 009cffdc KERNELBASE!CtrlRoutine+0x0 
009cff94 77315de3 80000006 7368d73f 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
009cffdc 77315dae ffffffff 7733b7d5 00000000 00000000 ntdll!RtlUnicodeStringToInteger+0x0 
009cffec 00000000 742ba700 80000006 00000000 00000000 ntdll!RtlUnicodeStringToInteger+0x0 

*** Dump of thread ID 2664 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 15468750.000000, User Time: 46406248.000000, Wait Time: 35817680.000000

- Registers -
eax=00000000 ebx=0019ef6c ecx=00000000 edx=00000000 esi=00000000 edi=00000ef9
eip=77326f1c esp=0019ef2c ebp=0019ef90
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
0019ef90 7425a6ff 000000c8 00000000 0019efb4 004691c1 ntdll!ZwDelayExecution+0x0 
0019efa0 004691c1 000000c8 000000c8 00000000 0019f000 KERNELBASE!Sleep+0x0 
0019efb4 00407b11 a0000000 027f9999 00000000 004d0c50 wah2_8.12_windows_intelx86!+0x0 
0019f000 00407f6e 00000001 004d0c48 00000020 00000000 wah2_8.12_windows_intelx86!+0x0 
0019f034 00409542 00000000 285e1835 0048616e 00000000 wah2_8.12_windows_intelx86!+0x0 
0019fd40 00409731 0019fef0 004097af 004d1368 0019fd60 wah2_8.12_windows_intelx86!+0x0 
0019fd48 004097af 004d1368 0019fd60 007d0f30 0019fd60 wah2_8.12_windows_intelx86!+0x0 
0019fef0 0048611b 00400000 00000000 007d0f6e 0000000a wah2_8.12_windows_intelx86!+0x0 
0019ff80 76dc38f4 0034c000 76dc38d0 6b7191b4 0019ffdc wah2_8.12_windows_intelx86!+0x0 
0019ff94 77315de3 0034c000 73edd73f 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0019ffdc 77315dae ffffffff 7733b7d5 00000000 00000000 ntdll!RtlUnicodeStringToInteger+0x0 
0019ffec 00000000 0048616e 0034c000 00000000 78746341 ntdll!RtlUnicodeStringToInteger+0x0 

*** Dump of thread ID 4352 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 35817688.000000

- Registers -
eax=c0000022 ebx=011bfd1c ecx=00000000 edx=00000000 esi=00000000 edi=00000001
eip=77326f1c esp=011bfcdc ebp=011bfd40
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
011bfd40 7425a6ff 00000064 00000000 011bff80 004790c4 ntdll!ZwDelayExecution+0x0 
011bfd50 004790c4 00000064 004790a0 004790a0 00000000 KERNELBASE!Sleep+0x0 
011bff80 76dc38f4 00000000 76dc38d0 6a7391b4 011bffdc wah2_8.12_windows_intelx86!+0x0 
011bff94 77315de3 00000000 72efd73f 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
011bffdc 77315dae ffffffff 7733b7d5 00000000 00000000 ntdll!RtlUnicodeStringToInteger+0x0 
011bffec 00000000 004790a0 00000000 00000000 00000000 ntdll!RtlUnicodeStringToInteger+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Suspended CPDN Monitor - Suspend request from BOINC...
22:58:22 (4328): start_timer_thread(): CreateThread() failed, errno 0
22:58:24 (3788): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
02:45:31 (4012): start_timer_thread(): CreateThread() failed, errno 0
02:45:32 (4612): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
01:07:09 (7600): start_timer_thread(): CreateThread() failed, errno 0
01:07:10 (8116): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Leaving CPDN_Main::Monitor...
20:52:57 (4028): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
13 Dec 2016 01:49:41 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 150,059 464,231 3.0937
11 Dec 2016 11:08:58 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 138,539 429,899 3.1031
10 Dec 2016 15:07:21 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 127,019 395,757 3.1157
09 Dec 2016 16:40:03 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 115,499 360,072 3.1175
08 Dec 2016 19:56:56 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 103,979 324,758 3.1233
07 Dec 2016 11:58:23 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 92,459 289,667 3.1329
06 Dec 2016 16:51:46 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 80,939 255,047 3.1511
06 Dec 2016 01:12:25 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 69,419 220,884 3.1819
05 Dec 2016 08:31:10 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 57,899 187,387 3.2364
04 Dec 2016 15:18:45 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 46,379 154,275 3.3264
02 Nov 2016 08:07:50 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 23,339 83,184 3.5642
02 Nov 2016 08:07:50 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 23,339 83,184 3.5642
31 Oct 2016 01:33:31 1403926 19954478 wah2_sas50_ksah_198912_13_448_010710381_0 11,819 42,514 3.5971


©2024 climateprediction.net