climateprediction.net home page
Task 21572597

Task 21572597

Name wah2_anz50_n0gy_201612_20_794_011762996_2
Workunit 11762996
Created 19 Mar 2019, 15:41:43 UTC
Sent 19 Mar 2019, 16:11:17 UTC
Report deadline 29 Feb 2020, 21:31:17 UTC
Received 28 Mar 2019, 10:27:17 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID 1443141
Run time 4 days 12 hours 49 min 59 sec
CPU time 4 days 12 hours 4 min 9 sec
Validate state Invalid
Credit 12,938.64
Device peak FLOPS 4.47 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 232.97 MB
Peak swap size 195.75 MB
Peak disk usage 43.47 MB
Stderr 
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
finish file present too long</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:53:50 (11964): start_timer_thread(): CreateThread() failed, errno 0
08:53:51 (14244): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:41:50 (2020): Can't acquire lockfile (32) - waiting 35s
14:42:25 (2020): Can't acquire lockfile (32) - exiting
14:42:25 (2020): Error: Proces nie mo
Leaving CPDN_ain::Monitor...
02:53:58 (6932): called boinc_finish(0)
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Abort request from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x0040A58A write attempt to address 0x00000130

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 03/28/19 09:45:57
Install Directory : 
Data Directory    : D:\BOINC
Project Symstore  : 
LoadLibraryA( D:\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( D:\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: D:\BOINC\projects\climateprediction.net;D:\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000152000 D:\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000077600000 000000000018c000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.16299.936) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : System operacyjny Microsoft&#174; Windows&#174;
    Product Version       : 10.0.16299.15

ModLoad: 00000000746e0000 00000000000d0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.16299.1029) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : System operacyjny Microsoft&#174; Windows&#174;
    Product Version       : 10.0.16299.15

ModLoad: 00000000773a0000 00000000001d8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.16299.1004) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : System operacyjny Microsoft&#174; Windows&#174;
    Product Version       : 10.0.16299.15

ModLoad: 0000000075860000 0000000000175000 C:\WINDOWS\System32\USER32.dll (6.2.16299.125) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : System operacyjny Microsoft&#174; Windows&#174;
    Product Version       : 10.0.16299.15

ModLoad: 0000000077380000 0000000000016000 C:\WINDOWS\System32\win32u.dll (6.2.16299.15) (-exported- Symbols Loaded)
    Linked PDB Filename   : wwin32u.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.15

ModLoad: 0000000077240000 0000000000022000 C:\WINDOWS\System32\GDI32.dll (6.2.16299.665) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.16299.665 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.665

ModLoad: 0000000074220000 000000000015f000 C:\WINDOWS\System32\gdi32full.dll (6.2.16299.967) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32full.pdb
    File Version          : 10.0.16299.967 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.967

ModLoad: 0000000075650000 000000000007c000 C:\WINDOWS\System32\msvcp_win.dll (6.2.16299.1004) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.16299.1004 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.1004

ModLoad: 0000000075110000 0000000000116000 C:\WINDOWS\System32\ucrtbase.dll (6.2.16299.1004) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.16299.1004 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.1004

ModLoad: 0000000074490000 0000000000078000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.16299.696) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : System operacyjny Microsoft&#174; Windows&#174;
    Product Version       : 10.0.16299.15

ModLoad: 00000000743d0000 00000000000bd000 C:\WINDOWS\System32\msvcrt.dll (7.0.16299.125) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.16299.125 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.16299.125

ModLoad: 0000000075040000 0000000000043000 C:\WINDOWS\System32\sechost.dll (6.2.16299.696) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.15

ModLoad: 0000000074610000 00000000000be000 C:\WINDOWS\System32\RPCRT4.dll (6.2.16299.1029) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : System operacyjny Microsoft&#174; Windows&#174;
    Product Version       : 10.0.16299.15

ModLoad: 0000000073f30000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.16299.755) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.16299.755 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.755

ModLoad: 0000000073f20000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.16299.15) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.15

ModLoad: 00000000750b0000 0000000000057000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.16299.1004) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.16299.1004 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.1004

ModLoad: 0000000074840000 0000000000025000 C:\WINDOWS\System32\IMM32.DLL (6.2.16299.15) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.15

ModLoad: 00000000753a0000 0000000000245000 C:\WINDOWS\System32\combase.dll (6.2.16299.1004) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : System operacyjny Microsoft&#174; Windows&#174;
    Product Version       : 10.0.16299.15

ModLoad: 0000000061d70000 0000000000181000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.16299.15) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.15

ModLoad: 00000000719d0000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.16299.15) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.16299.15 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.16299.15



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 560, Write: 0, Other 3267

- I/O Transfers Counters -
Read: 0, Write: 175, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 96712, QuotaPeakPagedPoolUsage: 96808
QuotaNonPagedPoolUsage: 8488, QuotaPeakNonPagedPoolUsage: 8488

- Virtual Memory Usage -
VirtualSize: 91811840, PeakVirtualSize: 91811840

- Pagefile Usage -
PagefileUsage: 3186688, PeakPagefileUsage: 3186688

- Working Set Size -
WorkingSetSize: 6258688, PeakWorkingSetSize: 6258688, PageFaultCount: 1649

*** Dump of thread ID 4240 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 937500.000000, User Time: 1250000.000000, Wait Time: 157487.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x0040A58A write attempt to address 0x00000130

- Registers -
eax=00000000 ebx=00000000 ecx=0000000b edx=0040a560 esi=0000000b edi=00000000
eip=0040a58a esp=0019dca4 ebp=0019dcc8
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010297

- Callstack -
ChildEBP RetAddr  Args to Child
0019dcc8 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
0019dce0 00485ca1 c0000005 0019dd14 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 
0019ff80 746f8674 00319000 746f8650 3938450b 0019ffdc wah2_8.24_windows_intelx86!+0x0 
0019ff94 77665e17 00319000 086e0c72 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0019ffdc 77665de7 ffffffff 7768ad85 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
0019ffec 00000000 00485cce 00319000 00000000 78746341 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 7764 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 156991.000000

- Registers -
eax=00000000 ebx=02916e28 ecx=00000000 edx=00000000 esi=02913b88 edi=02913d48
eip=776712bc esp=0286fd8c ebp=0286ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
0286ff80 746f8674 02916e28 746f8650 3ba7450b 0286ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
0286ff94 77665e17 02916e28 0af10c72 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0286ffdc 77665de7 ffffffff 7768ad85 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
0286ffec 00000000 77632db0 02916e28 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 2136 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 156991.000000

- Registers -
eax=00000000 ebx=02916e28 ecx=00000000 edx=00000000 esi=029145f0 edi=029147b0
eip=776712bc esp=02b0fd8c ebp=02b0ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
02b0ff80 746f8674 02916e28 746f8650 3b91450b 02b0ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
02b0ff94 77665e17 02916e28 0ac70c72 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
02b0ffdc 77665de7 ffffffff 7768ad85 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
02b0ffec 00000000 77632db0 02916e28 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 10508 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 156991.000000

- Registers -
eax=00000000 ebx=02916e28 ecx=00000000 edx=00000000 esi=029149f8 edi=02914bb8
eip=776712bc esp=02c0fd8c ebp=02c0ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
02c0ff80 746f8674 02916e28 746f8650 3be1450b 02c0ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
02c0ff94 77665e17 02916e28 0ab70c72 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
02c0ffdc 77665de7 ffffffff 7768ad85 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
02c0ffec 00000000 77632db0 02916e28 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 8940 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 157485.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001
eip=7766f96c esp=0461fcdc ebp=0461fd40
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
0461fd40 774a65ef 00000064 00000000 0461ff80 00478b44 ntdll!ZwDelayExecution+0x0 
0461fd50 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 
0461ff80 746f8674 00000000 746f8650 3d40450b 0461ffdc wah2_8.24_windows_intelx86!+0x0 
0461ff94 77665e17 00000000 0c160c72 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0461ffdc 77665de7 ffffffff 7768ad85 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
0461ffec 00000000 00478b20 00000000 00000000 be78a1c0 ntdll!RtlGetAppContainerNamedObjectPath+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
24 Mar 2019 13:07:55 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 196,139 387,118 1.9737
24 Mar 2019 08:50:26 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 184,619 368,341 1.9951
24 Mar 2019 03:26:30 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 173,099 349,240 2.0176
23 Mar 2019 20:28:12 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 161,579 328,006 2.0300
23 Mar 2019 14:00:40 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 150,059 304,967 2.0323
23 Mar 2019 07:34:36 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 138,539 281,864 2.0345
23 Mar 2019 01:23:48 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 127,019 256,038 2.0157
22 Mar 2019 17:40:42 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 115,499 231,475 2.0041
22 Mar 2019 11:45:12 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 103,979 207,727 1.9978
22 Mar 2019 04:32:55 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 92,459 184,550 1.9960
21 Mar 2019 21:36:49 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 80,939 159,580 1.9716
21 Mar 2019 15:00:49 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 69,419 136,050 1.9598
21 Mar 2019 08:40:39 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 57,899 113,329 1.9574
21 Mar 2019 02:25:43 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 46,379 91,776 1.9788
20 Mar 2019 20:10:52 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 34,859 69,263 1.9869
20 Mar 2019 05:37:57 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 23,339 46,642 1.9985
19 Mar 2019 23:12:06 1443141 21572597 wah2_anz50_n0gy_201612_20_794_011762996_2 11,819 22,975 1.9439


©2024 climateprediction.net