climateprediction.net home page
Task 20435235

Task 20435235

Name wah2_global_b75y_199812_25_577_011045872_0
Workunit 11045872
Created 30 May 2017, 11:27:17 UTC
Sent 30 May 2017, 20:23:26 UTC
Report deadline 13 May 2018, 1:43:26 UTC
Received 13 Jun 2017, 3:18:29 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID 1431017
Run time 22 hours 32 min 39 sec
CPU time 21 hours 40 min 38 sec
Validate state Invalid
Credit 1,524.29
Device peak FLOPS 3.43 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 234.19 MB
Peak swap size 170.86 MB
Peak disk usage 0.02 MB
Stderr
<core_client_version>7.6.33</core_client_version>
<![CDATA[
<message>
finish file present too long
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CSuspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Signal 11 received: Segment violation
Signal 11 received: Software termination signal from kill 
Signal 11 received: Abnormal termination triggered by abort call
Signal 11 received, exiting...
22:07:05 (8052): called boinc_finish(193)
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5368, selfPID=5368, iMonCtr=2
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5368, selfPID=9756, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_ain::Monitor...
22:07:16 (9756): called boinc_finish(0)
CPDN Monitor - Abort request from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0254012C

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 06/12/17 22:17:17
Install Directory : 
Data Directory    : D:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( D:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( D:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: D:\ProgramData\BOINC\projects\climateprediction.net;D:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000152000 D:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000077650000 0000000000180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.23796) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000076710000 0000000000110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.23796) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.18015

ModLoad: 0000000076570000 0000000000047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.23796) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.18015

ModLoad: 0000000076aa0000 0000000000100000 C:\Windows\syswow64\USER32.dll (6.1.7601.23594) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17514

ModLoad: 00000000765f0000 0000000000090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.23764) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.23764 (win7sp1_ldr.170407-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.23764

ModLoad: 0000000076370000 000000000000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.23717) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7601.23717 (win7sp1_ldr.170310-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.23717

ModLoad: 0000000076d10000 000000000009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.23688) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.23688 (win7sp1_ldr.170210-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.23688

ModLoad: 0000000076380000 00000000000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 0000000076160000 00000000000a1000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.23796) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7601.23796 (win7sp1_ldr.170427-1518)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.23796

ModLoad: 00000000765c0000 0000000000019000 C:\Windows\SysWOW64\sechost.dll (6.1.7601.18869) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075410000 00000000000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.23796) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000074ef0000 0000000000060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.23796) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.23796 (win7sp1_ldr.170427-1518)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.23796

ModLoad: 0000000074ee0000 000000000000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7601.23796) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7601.23796 (win7sp1_ldr.170427-1518)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.23796

ModLoad: 0000000076430000 0000000000060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 00000000769c0000 00000000000cd000 C:\Windows\syswow64\MSCTF.dll (6.1.7601.23572) (-exported- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000073820000 00000000000b7000 C:\Windows\SysWOW64\guard32.dll (10.0.1.6223) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\jenkins\workspace\CIS_DRV_brunch\Release\x32\Symbols\guard32.pdb
    File Version          : 10, 0, 1, 6223
    Company Name          : COMODO
    Product Name          : COMODO Internet Security
    Product Version       : 10, 0, 1, 6223

ModLoad: 0000000074ce0000 0000000000003000 C:\Windows\system32\api-ms-win-core-synch-l1-2-0.DLL (10.0.10586.788) (-exported- Symbols Loaded)
    Linked PDB Filename   : api-ms-win-core-synch-l1-2-0.pdb
    File Version          : 10.0.10586.788 (th2_release_sec.170113-1732)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10586.788

ModLoad: 0000000073810000 0000000000009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000076db0000 000000000015d000 C:\Windows\system32\ole32.dll (6.1.7601.23775) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 6.1.7601.23775 (win7sp1_ldr.170417-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.23775

ModLoad: 0000000076ba0000 0000000000057000 C:\Windows\system32\shlwapi.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000073800000 0000000000007000 C:\Windows\system32\fltlib.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : fltLib.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 000000006f0b0000 00000000000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 3820, Write: 0, Other 24733

- I/O Transfers Counters -
Read: 0, Write: 3361, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 94008, QuotaPeakPagedPoolUsage: 94008
QuotaNonPagedPoolUsage: 8640, QuotaPeakNonPagedPoolUsage: 9200

- Virtual Memory Usage -
VirtualSize: 54554624, PeakVirtualSize: 54554624

- Pagefile Usage -
PagefileUsage: 3121152, PeakPagefileUsage: 3350528

- Working Set Size -
WorkingSetSize: 6385664, PeakWorkingSetSize: 6389760, PageFaultCount: 1947

*** Dump of thread ID 5784 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 2964019.000000, User Time: 9204059.000000, Wait Time: 11610988.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0254012C

- Registers -
eax=00000065 ebx=004b63b9 ecx=0018dca8 edx=0254012c esi=0018da60 edi=0018dcd8
eip=0048ee46 esp=0018d9f4 ebp=0018dc84
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202

- Callstack -
ChildEBP RetAddr  Args to Child
0018dc84 0048161c 0018dca8 004b63b8 00000000 0018dcd8 wah2_8.24_windows_intelx86!+0x0 
0018dcc8 0044fa52 0254012c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 
0018dce8 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
0018dd00 00485ca1 c0000005 0018dd34 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 
0018ff88 7672336a 7efde000 0018ffd4 77689902 7efde000 wah2_8.24_windows_intelx86!+0x0 
0018ff94 77689902 7efde000 7da640dd 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0018ffd4 776898d5 00485cce 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0018ffec 00000000 00485cce 7efde000 00000000 78746341 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 6840 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 11610985.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=0247fd24 edi=00000000
eip=7766fda1 esp=0247fce0 ebp=0247fd48
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
0247fd48 7658460c 00000064 00000000 0247ff88 00478b44 ntdll!NtDelayExecution+0x0 
0247fd58 00478b44 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 
0247ff88 7672336a 00000000 0247ffd4 77689902 00000000 wah2_8.24_windows_intelx86!+0x0 
0247ff94 77689902 00000000 7ff940dd 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0247ffd4 776898d5 00478b20 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0247ffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
13 Jun 2017 00:03:43 1431017 20435235 wah2_global_b75y_199812_25_577_011045872_0 23,107 74,080 3.2060
08 Jun 2017 07:16:47 1431017 20435235 wah2_global_b75y_199812_25_577_011045872_0 20,227 62,799 3.1047
07 Jun 2017 17:49:26 1431017 20435235 wah2_global_b75y_199812_25_577_011045872_0 17,347 53,762 3.0992
05 Jun 2017 02:31:54 1431017 20435235 wah2_global_b75y_199812_25_577_011045872_0 14,467 44,739 3.0925
05 Jun 2017 00:27:25 1431017 20435235 wah2_global_b75y_199812_25_577_011045872_0 11,587 36,046 3.1109
04 Jun 2017 21:26:38 1431017 20435235 wah2_global_b75y_199812_25_577_011045872_0 8,707 27,344 3.1405
02 Jun 2017 01:35:48 1431017 20435235 wah2_global_b75y_199812_25_577_011045872_0 5,827 18,490 3.1732
01 Jun 2017 23:34:34 1431017 20435235 wah2_global_b75y_199812_25_577_011045872_0 2,947 9,654 3.2759


©2024 cpdn.org