climateprediction.net home page
Task 20890170

Task 20890170

Name wah2_pnw25_s0gz_200212_24_683_011353342_0
Workunit 11353342
Created 17 Nov 2017, 17:20:00 UTC
Sent 17 Nov 2017, 20:06:40 UTC
Report deadline 31 Oct 2018, 1:26:40 UTC
Received 25 Nov 2017, 2:05:52 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -2147483645 (0x80000003) Unknown error code
Computer ID 1338318
Run time 4 days 2 hours 54 min 49 sec
CPU time 3 days 17 hours 50 min 54 sec
Validate state Invalid
Credit 18,258.20
Device peak FLOPS 4.59 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 186.75 MB
Peak swap size 166.15 MB
Peak disk usage 0.02 MB
Stderr 
<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
One or more arguments are invalid
 (0x80000003) - exit code -2147483645 (0x80000003)</message>
<stderr_txt>
16:06:30 (16900): start_timer_thread(): CreateThread() failed, errno 0
16:06:31 (38308): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
20:25:32 (42120): start_timer_thread(): CreateThread() failed, errno 0
20:25:33 (29956): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
07:13:50 (30836): start_timer_thread(): CreateThread() failed, errno 0
07:13:52 (17308): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
11:29:02 (29972): start_timer_thread(): CreateThread() failed, errno 0
11:29:03 (43092): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:56:40 (41428): start_timer_thread(): CreateThread() failed, errno 0
21:56:41 (32024): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
22:49:35 (43640): start_timer_thread(): CreateThread() failed, errno 0
22:49:37 (43412): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
00:38:32 (42168): start_timer_thread(): CreateThread() failed, errno 0
00:38:33 (35384): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
02:14:08 (11828): start_timer_thread(): CreateThread() failed, errno 0
02:14:10 (44544): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
03:27:24 (36364): start_timer_thread(): CreateThread() failed, errno 0
03:27:26 (43052): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
04:19:32 (15056): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
00:52:43 (8948): start_timer_thread(): CreateThread() failed, errno 0
00:52:44 (6180): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
02:42:12 (31412): start_timer_thread(): CreateThread() failed, errno 0
02:42:13 (42544): start_timer_thread(): CreateThread() failed, errno 0
19:33:44 (15676): start_timer_thread(): CreateThread() failed, errno 0
19:33:46 (16024): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
22:24:27 (21724): start_timer_thread(): CreateThread() failed, errno 0
22:24:28 (26632): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
01:51:48 (16432): start_timer_thread(): CreateThread() failed, errno 0
01:51:50 (32372): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
06:35:57 (27788): start_timer_thread(): CreateThread() failed, errno 0
06:35:59 (25884): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
09:23:09 (27736): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
11:46:27 (34744): start_timer_thread(): CreateThread() failed, errno 0
11:46:29 (27296): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
11:52:23 (11496): start_timer_thread(): CreateThread() failed, errno 0
11:52:25 (28812): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
18:29:47 (34236): start_timer_thread(): CreateThread() failed, errno 0
18:29:49 (6464): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
04:10:52 (39916): start_timer_thread(): CreateThread() failed, errno 0
04:10:53 (29120): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
16:29:43 (33536): start_timer_thread(): CreateThread() failed, errno 0
16:29:44 (33324): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
20:37:32 (4456): start_timer_thread(): CreateThread() failed, errno 0
20:37:34 (14712): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
01:16:38 (6140): start_timer_thread(): CreateThread() failed, errno 0
01:16:40 (30644): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
03:34:32 (27156): start_timer_thread(): CreateThread() failed, errno 0
03:34:33 (40520): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
16:17:39 (8040): start_timer_thread(): CreateThread() failed, errno 0
16:17:40 (8484): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
16:30:22 (33384): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
16:58:24 (30796): start_timer_thread(): CreateThread() failed, errno 0
16:58:25 (6640): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
03:26:23 (32752): start_timer_thread(): CreateThread() failed, errno 0
03:26:25 (33184): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
16:27:21 (38168): start_timer_thread(): CreateThread() failed, errno 0
Leaving CPDN_ain::Monitor...
ERROR: Invalid parameter detected in function (null). File: (null) Line: 0
ERROR: Expression: (null)


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x7683FCE2

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 11/24/17 20:05:37
Install Directory : 
Data Directory    : T:\BOINC
Project Symstore  : 
LoadLibraryA( T:\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( T:\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 193
LoadLibraryA( T:\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( T:\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: T:\BOINC\projects\climateprediction.net\wah2_pnw25_s0gz_200212_24_683_011353342;T:\BOINC\projects\climateprediction.net;cache*t:\symbols;srv*http://msdl.microsoft.com/download/symbols


ModLoad: 0000000000400000 0000000000152000 T:\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000077b90000 000000000018e000 C:\Windows\SYSTEM32\ntdll.dll (6.2.15063.608) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.15063.608 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.608

ModLoad: 0000000077650000 00000000000d0000 C:\Windows\System32\KERNEL32.DLL (6.2.15063.296) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.15063.608 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.608

ModLoad: 00000000766d0000 00000000001c2000 C:\Windows\System32\KERNELBASE.dll (6.2.15063.726) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.15063.608 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.608

ModLoad: 0000000077220000 000000000013c000 C:\Windows\System32\USER32.dll (6.2.15063.726) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000074d40000 0000000000016000 C:\Windows\System32\win32u.dll (6.2.15063.608) (-exported- Symbols Loaded)
    Linked PDB Filename   : wwin32u.pdb
    File Version          : 10.0.15063.608 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.608

ModLoad: 0000000076bf0000 0000000000021000 C:\Windows\System32\GDI32.dll (6.2.15063.483) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.15063.483 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.483

ModLoad: 00000000778a0000 0000000000158000 C:\Windows\System32\gdi32full.dll (6.2.15063.674) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32full.pdb
    File Version          : 10.0.15063.674 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.674

ModLoad: 00000000761e0000 0000000000079000 C:\Windows\System32\msvcp_win.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000077100000 0000000000118000 C:\Windows\System32\ucrtbase.dll (6.2.15063.674) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.15063.674 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.674

ModLoad: 0000000076310000 0000000000077000 C:\Windows\System32\ADVAPI32.dll (6.2.15063.674) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0

ModLoad: 00000000768a0000 00000000000bd000 C:\Windows\System32\msvcrt.dll (7.0.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.15063.0

ModLoad: 0000000077450000 0000000000041000 C:\Windows\System32\sechost.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000077040000 00000000000c0000 C:\Windows\System32\RPCRT4.dll (6.2.15063.674) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000074640000 0000000000020000 C:\Windows\System32\SspiCli.dll (6.2.15063.674) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.15063.674 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.674

ModLoad: 0000000074630000 000000000000a000 C:\Windows\System32\CRYPTBASE.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000076260000 0000000000057000 C:\Windows\System32\bcryptPrimitives.dll (6.2.15063.726) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.15063.726 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.726

ModLoad: 0000000074670000 0000000000025000 C:\Windows\System32\IMM32.DLL (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000072080000 0000000000028000 C:\Windows\SYSTEM32\ntmarta.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntmarta.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0

ModLoad: 000000006d5b0000 000000000015d000 C:\Windows\SYSTEM32\dbghelp.dll (6.2.15063.447) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.15063.447 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.447

ModLoad: 0000000074440000 0000000000008000 C:\Windows\SYSTEM32\version.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.15063.0



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 14497, Write: 0, Other 17233

- I/O Transfers Counters -
Read: 0, Write: 2031, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 240992, QuotaPeakPagedPoolUsage: 242072
QuotaNonPagedPoolUsage: 8824, QuotaPeakNonPagedPoolUsage: 8976

- Virtual Memory Usage -
VirtualSize: 134451200, PeakVirtualSize: 134991872

- Pagefile Usage -
PagefileUsage: 6201344, PeakPagefileUsage: 7602176

- Working Set Size -
WorkingSetSize: 81743872, PeakWorkingSetSize: 85356544, PageFaultCount: 23594

*** Dump of thread ID 15008 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 10625000.000000, User Time: 103281248.000000, Wait Time: 23412870.000000

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x7683FCE2

- Registers -
eax=0000001a ebx=00000000 ecx=00480912 edx=0019ce14 esi=004cefe0 edi=004cefe0
eip=7683fce2 esp=0019eca8 ebp=0019ecc4
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212

- Callstack -
ChildEBP RetAddr  Args to Child
0019ecc4 0048ed78 ffffffff 004cefe0 004a488f 0019ed10 KERNELBASE!DebugBreak+0x0 
0019ecd8 004808e1 004cefe0 9806f287 0019ede0 004a488f wah2_8.24_windows_intelx86!+0x0 
0019ed10 00403875 004cefe0 004a49e8 004d0c50 00000020 wah2_8.24_windows_intelx86!+0x0 
0019ede4 00407333 00000001 00000020 0019fd40 00408220 wah2_8.24_windows_intelx86!+0x0 
0019edf4 00408220 00000000 00000001 9806e2d7 00485cce wah2_8.24_windows_intelx86!+0x0 
0019fd40 004094b7 0019fef0 00409535 004d1370 0019fd60 wah2_8.24_windows_intelx86!+0x0 
0019fd48 00409535 004d1370 0019fd60 00636f98 0019fd60 wah2_8.24_windows_intelx86!+0x0 
0019fef0 00485c7b 00400000 00000000 00636fd6 0000000a wah2_8.24_windows_intelx86!+0x0 
0019ff80 77668744 00231000 77668720 71eb67e2 0019ffdc wah2_8.24_windows_intelx86!+0x0 
0019ff94 77bf582d 00231000 5a908156 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0019ffdc 77bf57fd ffffffff 77c1636e 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
0019ffec 00000000 00485cce 00231000 00000000 78746341 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 25572 (state: Ready): ***

- Information -
Status: Base Priority: Unknown, Priority: Unknown, , Kernel Time: 625000.000000, User Time: 625000.000000, Wait Time: 23412870.000000

- Registers -
eax=c0000034 ebx=025de4ec ecx=00000000 edx=00000000 esi=00000000 edi=004a5cec
eip=77c0205c esp=025de4ac ebp=025de510
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
025de510 767c48cf 00000467 00000000 025de534 00468ec1 ntdll!NtDelayExecution+0x0 
025de520 00468ec1 00000467 00000467 00000000 025de598 KERNELBASE!Sleep+0x0 
025de534 00468a91 11c82390 027f08e4 00000010 00000010 wah2_8.24_windows_intelx86!+0x0 
025de598 00478832 0237a178 004a5cec 9a42e6df 00000001 wah2_8.24_windows_intelx86!+0x0 
025df948 00478b07 6c70753c 5f64616f 656c6966 6174735f wah2_8.24_windows_intelx86!+0x0 
025dfd54 00478c13 00478b20 00478b20 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
025dff80 77668744 00000000 77668720 73af67e2 025dffdc wah2_8.24_windows_intelx86!+0x0 
025dff94 77bf582d 00000000 58d48156 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
025dffdc 77bf57fd ffffffff 77c1636e 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
025dffec 00000000 00478b20 00000000 00000000 32686177 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 39332 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 23408548.000000

- Registers -
eax=00000000 ebx=006368b8 ecx=00000000 edx=00000000 esi=00649d80 edi=00649f40
eip=77c0396c esp=06dcfd7c ebp=06dcff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
06dcff80 77668744 006368b8 77668720 772e67e2 06dcffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
06dcff94 77bf582d 006368b8 5c558156 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
06dcffdc 77bf57fd ffffffff 77c1636e 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
06dcffec 00000000 77bca470 006368b8 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 33432 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 23408548.000000

- Registers -
eax=77bca470 ebx=006368b8 ecx=00000000 edx=00000000 esi=00657d10 edi=00000000
eip=77c0396c esp=06f0fd7c ebp=06f0ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
06f0ff80 77668744 006368b8 77668720 770267e2 06f0ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
06f0ff94 77bf582d 006368b8 5c798156 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
06f0ffdc 77bf57fd ffffffff 77c1636e 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
06f0ffec 00000000 77bca470 006368b8 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Nov 2017 01:07:12 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 276,779 323,395 1.1684
24 Nov 2017 20:45:30 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 265,259 309,826 1.1680
24 Nov 2017 15:53:41 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 253,739 296,492 1.1685
24 Nov 2017 12:32:03 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 242,219 283,206 1.1692
24 Nov 2017 02:16:05 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 230,699 269,920 1.1700
23 Nov 2017 21:42:05 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 219,179 256,722 1.1713
23 Nov 2017 17:40:56 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 207,659 243,489 1.1725
23 Nov 2017 13:46:58 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 196,139 230,069 1.1730
23 Nov 2017 09:59:48 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 184,619 216,790 1.1743
22 Nov 2017 23:30:31 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 173,099 203,641 1.1764
22 Nov 2017 19:32:48 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 161,579 190,739 1.1805
22 Nov 2017 15:41:01 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 150,059 177,576 1.1834
22 Nov 2017 11:38:47 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 138,539 164,289 1.1859
21 Nov 2017 21:57:28 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 127,019 150,966 1.1885
21 Nov 2017 17:50:09 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 115,499 137,694 1.1922
21 Nov 2017 12:36:21 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 103,979 124,325 1.1957
21 Nov 2017 08:12:49 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 92,459 110,994 1.2005
20 Nov 2017 09:22:43 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 80,939 97,825 1.2086
19 Nov 2017 23:21:58 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 69,419 84,214 1.2131
19 Nov 2017 19:00:50 1338318 20890170 wah2_pnw25_s0gz_200212_24_683_011353342_0 57,899 70,175 1.2120


©2024 cpdn.org