climateprediction.net home page
Task 20331249

Task 20331249

Name wah2_global_a00o_198512_133_553_010979018_0
Workunit 10979018
Created 14 Mar 2017, 14:05:47 UTC
Sent 14 Mar 2017, 14:11:53 UTC
Report deadline 24 Feb 2018, 19:31:53 UTC
Received 7 Apr 2017, 9:37:57 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1412270
Run time 6 days 17 hours 45 min 21 sec
CPU time 6 days 15 hours 57 min 30 sec
Validate state Invalid
Credit 12,353.39
Device peak FLOPS 3.82 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 240.80 MB
Peak swap size 171.27 MB
Peak disk usage 10.95 MB
Stderr 
<core_client_version>7.6.22</core_client_version>
<![CDATA[
<message>
Indirizzo del blocco di gestione della memoria non valido.
 (0x9) - exit code 9 (0x9)
</message>
<stderr_txt>


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7708E064 read attempt to address 0x3060FFFF

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 03/14/17 18:31:25
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\0;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000152000 C:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000077060000 0000000000180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.23572) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075460000 0000000000110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.23572) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.18015

ModLoad: 0000000075770000 0000000000047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.23572) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.18015

ModLoad: 0000000076850000 0000000000100000 C:\Windows\syswow64\USER32.dll (6.1.7601.23594) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17514

ModLoad: 00000000750f0000 0000000000090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.23591) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.23591 (win7sp1_ldr.161106-0500)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.23591

ModLoad: 0000000075660000 000000000000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.23587) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7601.23587 (win7sp1_ldr.161102-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.23587

ModLoad: 0000000075570000 000000000009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.23585) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.23585 (win7sp1_ldr.161027-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.23585

ModLoad: 0000000076b50000 00000000000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 0000000074b60000 00000000000a1000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.23572) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7601.23572 (win7sp1_ldr.161011-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.23572

ModLoad: 0000000074c20000 0000000000019000 C:\Windows\SysWOW64\sechost.dll (6.1.7601.18869) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000076440000 00000000000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.23642) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000074950000 0000000000060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.23642) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.23642 (win7sp1_ldr.170105-0730)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.23642

ModLoad: 0000000074940000 000000000000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7601.23642) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7601.23642 (win7sp1_ldr.170105-0730)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.23642

ModLoad: 00000000749b0000 0000000000060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 0000000076700000 00000000000cd000 C:\Windows\syswow64\MSCTF.dll (6.1.7601.23572) (-exported- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 00000000702b0000 0000000000021000 C:\Windows\system32\ntmarta.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntmarta.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075720000 0000000000045000 C:\Windows\syswow64\WLDAP32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wldap32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 000000006fd90000 000000000004c000 C:\Windows\system32\apphelp.dll (6.1.7601.19050) (-exported- Symbols Loaded)
    Linked PDB Filename   : apphelp.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000074730000 00000000000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 0000000073090000 0000000000009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 24519, Write: 0, Other 106232

- I/O Transfers Counters -
Read: 0, Write: 5223, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 194952, QuotaPeakPagedPoolUsage: 194952
QuotaNonPagedPoolUsage: 7688, QuotaPeakNonPagedPoolUsage: 7768

- Virtual Memory Usage -
VirtualSize: 106020864, PeakVirtualSize: 106020864

- Pagefile Usage -
PagefileUsage: 2445312, PeakPagefileUsage: 2998272

- Working Set Size -
WorkingSetSize: 79060992, PeakWorkingSetSize: 79429632, PageFaultCount: 20276

*** Dump of thread ID 2148 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 156001.000000, User Time: 0.000000, Wait Time: 2256465.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7708E064 read attempt to address 0x3060FFFF

- Registers -
eax=3060fff8 ebx=30610000 ecx=733b1a26 edx=003ee8b8 esi=00280000 edi=00000000
eip=7708e064 esp=0585fe5c ebp=0585fe6c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246

- Callstack -
ChildEBP RetAddr  Args to Child
0585fe6c 754714ad 00280000 00000000 30610000 0585fe94 ntdll!RtlFreeHeap+0x0 
0585fe80 00482c3b 00280000 00000000 30610000 0585feac kernel32!HeapFree+0x0 
0585fe94 00448311 30610000 00000000 01170020 80000000 wah2_8.24_windows_intelx86!+0x0 
0585feac 0044fa68 01170020 00000000 0051d6b0 0585fefc wah2_8.24_windows_intelx86!+0x0 
0585fec0 004856fc 00000015 76bee4da 00000001 00000006 wah2_8.24_windows_intelx86!+0x0 
0585fefc 75517661 00000006 76be621d 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
0585ff88 7547336a 00000006 0585ffd4 77099902 80000006 kernel32!CtrlRoutine+0x0 
0585ff94 77099902 80000006 7301a610 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0585ffd4 770998d5 755174ac 80000006 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0585ffec 00000000 755174ac 80000006 00000000 52b5f865 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 4784 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 12324079.000000, User Time: 68016432.000000, Wait Time: 2256460.000000

- Registers -
eax=00000000 ebx=00000041 ecx=00000000 edx=00000000 esi=0018ed80 edi=00000000
eip=7707fda1 esp=0018ed3c ebp=0018eda4
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
0018eda4 7578460c 000000c8 00000000 0018edc8 00468ec1 ntdll!NtDelayExecution+0x0 
0018edb4 00468ec1 000000c8 000000c8 00000000 0018ee00 KERNELBASE!Sleep+0x0 
0018edc8 00407a89 a0000000 027f9999 00000000 004d0c50 wah2_8.24_windows_intelx86!+0x0 
0018ee00 004092c5 00000000 7323e76e 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
0018fd48 004094b7 0018fef8 00409535 004d1370 0018fd68 wah2_8.24_windows_intelx86!+0x0 
0018fd50 00409535 004d1370 0018fd68 00693130 0018fd68 wah2_8.24_windows_intelx86!+0x0 
0018fef8 00485c7b 00400000 00000000 0069316e 0000000a wah2_8.24_windows_intelx86!+0x0 
0018ff88 7547336a 7efde000 0018ffd4 77099902 7efde000 wah2_8.24_windows_intelx86!+0x0 
0018ff94 77099902 7efde000 769ca610 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0018ffd4 770998d5 00485cce 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0018ffec 00000000 00485cce 7efde000 00000000 78746341 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 4824 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 2256464.000000

- Registers -
eax=c0000022 ebx=00000000 ecx=00000000 edx=00000000 esi=0106fd24 edi=00000000
eip=7707fda1 esp=0106fce0 ebp=0106fd48
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
0106fd48 7578460c 00000064 00000000 0106ff88 00478b44 ntdll!NtDelayExecution+0x0 
0106fd58 00478b44 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 
0106ff88 7547336a 00000000 0106ffd4 77099902 00000000 wah2_8.24_windows_intelx86!+0x0 
0106ff94 77099902 00000000 7782a610 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0106ffd4 770998d5 00478b20 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0106ffec 00000000 00478b20 00000000 00000000 1bd9555e ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3940, selfPID=3940, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3940, selfPID=2748, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3040, selfPID=268, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3040, selfPID=3040, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3056, selfPID=2948, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2512, selfPID=3096, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2512, selfPID=2512, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3036, selfPID=2424, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3036, selfPID=3036, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2056, selfPID=1440, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2056, selfPID=2056, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2152, selfPID=1940, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2152, selfPID=2152, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3020, selfPID=3020, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3020, selfPID=1816, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3044, selfPID=2512, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3044, selfPID=3044, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3028, selfPID=3028, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3028, selfPID=2060, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3988, selfPID=2688, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3988, selfPID=3988, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2052, selfPID=2524, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2052, selfPID=2052, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2524, selfPID=2524, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2524, selfPID=1508, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3260, selfPID=3324, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3260, selfPID=3260, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2408, selfPID=3012, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3044, selfPID=692, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3044, selfPID=3044, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3048, selfPID=1960, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3048, selfPID=3048, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3068, selfPID=2292, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3068, selfPID=3068, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=712, selfPID=2948, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3048, selfPID=2724, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3048, selfPID=3048, iMonCtr=1

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
07 Apr 2017 09:42:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 187,267 567,994 3.0331
07 Apr 2017 09:42:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 184,387 559,505 3.0344
07 Apr 2017 09:42:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 181,507 550,998 3.0357
07 Apr 2017 09:42:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 178,627 542,508 3.0371
07 Apr 2017 09:42:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 175,747 533,951 3.0382
07 Apr 2017 09:42:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 172,867 525,469 3.0397
07 Apr 2017 09:42:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 169,987 516,629 3.0392
07 Apr 2017 09:42:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 167,107 507,553 3.0373
04 Apr 2017 12:38:51 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 164,227 498,812 3.0373
04 Apr 2017 10:14:56 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 161,347 490,177 3.0380
04 Apr 2017 07:48:02 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 158,467 481,513 3.0386
03 Apr 2017 15:12:21 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 155,587 472,869 3.0393
03 Apr 2017 12:47:24 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 152,707 464,283 3.0404
03 Apr 2017 10:22:32 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 149,827 455,643 3.0411
03 Apr 2017 07:48:56 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 146,947 446,552 3.0389
31 Mar 2017 15:02:01 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 144,067 437,950 3.0399
31 Mar 2017 12:41:37 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 141,187 429,527 3.0423
31 Mar 2017 10:21:49 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 138,307 421,016 3.0441
31 Mar 2017 07:46:29 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 135,427 411,927 3.0417
30 Mar 2017 15:18:10 1412270 20331249 wah2_global_a00o_198512_133_553_010979018_0 132,547 403,213 3.0420


©2024 cpdn.org