Name | wah2_global_a01o_208812_145_615_011130682_1 |
Workunit | 11130682 |
Created | 8 Aug 2017, 10:59:24 UTC |
Sent | 8 Aug 2017, 19:19:04 UTC |
Report deadline | 22 Jul 2018, 0:39:04 UTC |
Received | 26 Sep 2017, 9:41:39 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 194 (0x000000C2) EXIT_ABORTED_BY_CLIENT |
Computer ID | 1339864 |
Run time | 33 days 15 hours 38 min 4 sec |
CPU time | 29 days 1 hours 49 min 37 sec |
Validate state | Invalid |
Credit | 25,462.29 |
Device peak FLOPS | 3.65 GFLOPS |
Application version | Weather At Home 2 (wah2) v8.24 windows_intelx86 |
Peak working set size | 248.22 MB |
Peak swap size | 173.68 MB |
Peak disk usage | 47.74 MB |
Stderr | <core_client_version>7.6.33</core_client_version> <![CDATA[ <message> finish file present too long </message> <stderr_txt> diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 15:34:35 (14280): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:30:46 (15196): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 15:27:14 (8116): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:25:20 (11964): start_timer_thread(): CreateThread() failed, errno 0 22:25:21 (13164): start_timer_thread(): CreateThread() failed, errno 0 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11964, selfPID=13164, iMonCtr=1 23:52:25 (8096): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:41:39 (12236): start_timer_thread(): CreateThread() failed, errno 0 19:41:41 (14836): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:21:07 (19572): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:23:07 (13600): start_timer_thread(): CreateThread() failed, errno 0 20:23:09 (8088): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:48:01 (16632): start_timer_thread(): CreateThread() failed, errno 0 15:48:01 (9596): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:17:01 (18336): start_timer_thread(): CreateThread() failed, errno 0 15:17:02 (6844): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:20:27 (16196): start_timer_thread(): CreateThread() failed, errno 0 06:20:29 (10560): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:12:07 (13308): start_timer_thread(): CreateThread() failed, errno 0 21:12:08 (17696): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:25:33 (18080): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:00:05 (3648): start_timer_thread(): CreateThread() failed, errno 0 23:00:06 (17916): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 03:50:07 (2656): start_timer_thread(): CreateThread() failed, errno 0 03:50:09 (19412): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:30:18 (11784): start_timer_thread(): CreateThread() failed, errno 0 17:30:19 (6568): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:56:51 (19220): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 00:35:30 (15112): start_timer_thread(): CreateThread() failed, errno 0 00:35:32 (18180): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:25:31 (17576): start_timer_thread(): CreateThread() failed, errno 0 18:25:33 (14356): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:51:03 (8100): start_timer_thread(): CreateThread() failed, errno 0 20:51:05 (16988): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:31:09 (18920): start_timer_thread(): CreateThread() failed, errno 0 15:31:11 (11684): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:49:59 (1532): start_timer_thread(): CreateThread() failed, errno 0 17:50:01 (7512): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:11:15 (14828): start_timer_thread(): CreateThread() failed, errno 0 21:11:16 (9236): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:41:48 (11428): start_timer_thread(): CreateThread() failed, errno 0 15:41:50 (15640): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:44:57 (10660): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:13:33 (13468): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:57:40 (14364): start_timer_thread(): CreateThread() failed, errno 0 17:57:42 (11728): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:57:04 (2924): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 15:30:29 (14172): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 01:35:41 (11244): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 01:35:43 (15012): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:29:34 (9640): start_timer_thread(): CreateThread() failed, errno 0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9640, selfPID=8772, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_ain::Monitor... 04:25:47 (8772): called boinc_finish(0) CPDN Monitor - Abort request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0268112C Engaging BOINC Windows Runtime Debugger... ******************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 09/26/17 04:40:39 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000152000 C:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077170000 000000000018e000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système d’exploitation Microsoft® Windows® Product Version : 10.0.15063.608 ModLoad: 0000000075030000 00000000000d0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.15063.296) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système d’exploitation Microsoft® Windows® Product Version : 10.0.15063.608 ModLoad: 0000000074e60000 00000000001c2000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système d’exploitation Microsoft® Windows® Product Version : 10.0.15063.608 ModLoad: 00000000749f0000 000000000013c000 C:\WINDOWS\System32\USER32.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système d’exploitation MicrosoftMD WindowsMD Product Version : 10.0.15063.0 ModLoad: 0000000074cd0000 0000000000016000 C:\WINDOWS\System32\win32u.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.608 ModLoad: 0000000074080000 0000000000021000 C:\WINDOWS\System32\GDI32.dll (6.2.15063.483) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.15063.483 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.483 ModLoad: 0000000077000000 0000000000158000 C:\WINDOWS\System32\gdi32full.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.608 ModLoad: 0000000076f80000 0000000000079000 C:\WINDOWS\System32\msvcp_win.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000748d0000 0000000000118000 C:\WINDOWS\System32\ucrtbase.dll (6.2.15063.447) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.15063.447 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.447 ModLoad: 0000000076e10000 0000000000077000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système d’exploitation Microsoft® Windows® Product Version : 10.0.15063.0 ModLoad: 0000000075850000 00000000000bd000 C:\WINDOWS\System32\msvcrt.dll (7.0.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.15063.0 ModLoad: 0000000074200000 0000000000041000 C:\WINDOWS\System32\sechost.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000076d20000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.15063.447) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système d’exploitation Microsoft® Windows® Product Version : 10.0.15063.0 ModLoad: 0000000073c40000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000073c30000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000740b0000 0000000000057000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.15063.138) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.15063.138 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.138 ModLoad: 0000000074310000 0000000000025000 C:\WINDOWS\System32\IMM32.DLL (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 000000006fb70000 000000000015d000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.15063.447) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.15063.447 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.447 ModLoad: 0000000073b60000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 *** Dump of the Process Statistics: *** - I/O Operations Counters - Read: 327, Write: 0, Other 2453 - I/O Transfers Counters - Read: 0, Write: 1944, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 90504, QuotaPeakPagedPoolUsage: 90640 QuotaNonPagedPoolUsage: 8624, QuotaPeakNonPagedPoolUsage: 9048 - Virtual Memory Usage - VirtualSize: 55345152, PeakVirtualSize: 55345152 - Pagefile Usage - PagefileUsage: 3440640, PeakPagefileUsage: 3440640 - Working Set Size - WorkingSetSize: 6320128, PeakWorkingSetSize: 6320128, PageFaultCount: 1842 *** Dump of thread ID 11104 (state: Waiting): *** - Information - Status: Wait Reason: UserRequest, , Kernel Time: 1718750.000000, User Time: 468750.000000, Wait Time: 54441.000000 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0268112C - Registers - eax=00000065 ebx=004b63b9 ecx=0019d8e8 edx=0268112c esi=0019d6a0 edi=0019d918 eip=0048ee46 esp=0019d634 ebp=0019d8c4 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202 - Callstack - ChildEBP RetAddr Args to Child 0019d8c4 0048161c 0019d8e8 004b63b8 00000000 0019d918 wah2_8.24_windows_intelx86!+0x0 0019d908 0044fa52 0268112c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 0019d928 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 0019d940 00485ca1 c0000005 0019d974 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 0019ff80 75048744 00260000 75048720 6bfdcd2e 0019ffdc wah2_8.24_windows_intelx86!+0x0 0019ff94 771d582d 00260000 3ec5a747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0019ffdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 0019ffec 00000000 00485cce 00260000 00000000 78746341 ntdll!RtlGetAppContainerNamedObjectPath+0x0 *** Dump of thread ID 13356 (state: Waiting): *** - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 53136.000000 - Registers - eax=00000000 ebx=007b4bb8 ecx=00000000 edx=00000000 esi=007b51e0 edi=007b53a0 eip=771e396c esp=0079fd7c ebp=0079ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0079ff80 75048744 007b4bb8 75048720 6b9dcd2e 0079ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 0079ff94 771d582d 007b4bb8 3ea5a747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0079ffdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 0079ffec 00000000 771aa470 007b4bb8 00000000 00905a4d ntdll!RtlGetAppContainerNamedObjectPath+0x0 *** Dump of thread ID 14020 (state: Waiting): *** - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 53136.000000 - Registers - eax=00000000 ebx=007b4bb8 ecx=00000000 edx=00000000 esi=007b5bf8 edi=007b5db8 eip=771e396c esp=009efd7c ebp=009eff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 009eff80 75048744 007b4bb8 75048720 6b7acd2e 009effdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 009eff94 771d582d 007b4bb8 3e42a747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 009effdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 009effec 00000000 771aa470 007b4bb8 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 *** Dump of thread ID 11012 (state: Waiting): *** - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 53136.000000 - Registers - eax=00000000 ebx=007b4bb8 ecx=00000000 edx=00000000 esi=007b60b0 edi=007b6270 eip=771e396c esp=00b2fd7c ebp=00b2ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 00b2ff80 75048744 007b4bb8 75048720 6b56cd2e 00b2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 00b2ff94 771d582d 007b4bb8 3e6ea747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 00b2ffdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 00b2ffec 00000000 771aa470 007b4bb8 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 *** Dump of thread ID 12520 (state: Waiting): *** - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 54437.000000 - Registers - eax=00000000 ebx=025bfd1c ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=771e205c esp=025bfcdc ebp=025bfd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 025bfd40 74f5354f 00000064 00000000 025bff80 00478b44 ntdll!NtDelayExecution+0x0 025bfd50 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 025bff80 75048744 00000000 75048720 69bfcd2e 025bffdc wah2_8.24_windows_intelx86!+0x0 025bff94 771d582d 00000000 3c87a747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 025bffdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 025bffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 *** Debug Message Dump **** *** Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
26 Sep 2017 10:13:55 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 385,987 | 2,500,601 | 6.4785 |
26 Sep 2017 10:10:06 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 383,107 | 2,479,453 | 6.4720 |
26 Sep 2017 09:58:43 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 380,227 | 2,458,215 | 6.4651 |
24 Sep 2017 17:17:59 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 377,347 | 2,437,565 | 6.4597 |
24 Sep 2017 11:23:26 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 374,467 | 2,416,612 | 6.4535 |
24 Sep 2017 05:29:03 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 371,587 | 2,395,891 | 6.4477 |
23 Sep 2017 23:14:33 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 368,707 | 2,374,145 | 6.4391 |
23 Sep 2017 18:11:47 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 365,827 | 2,354,731 | 6.4367 |
23 Sep 2017 12:28:12 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 362,947 | 2,335,723 | 6.4354 |
23 Sep 2017 07:06:07 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 360,067 | 2,317,045 | 6.4350 |
22 Sep 2017 06:41:23 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 357,187 | 2,296,710 | 6.4300 |
21 Sep 2017 23:31:22 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 354,307 | 2,273,593 | 6.4170 |
21 Sep 2017 07:52:35 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 351,427 | 2,250,961 | 6.4052 |
21 Sep 2017 00:09:46 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 348,547 | 2,228,573 | 6.3939 |
20 Sep 2017 06:34:17 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 345,667 | 2,205,420 | 6.3802 |
20 Sep 2017 00:11:41 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 342,787 | 2,182,939 | 6.3682 |
19 Sep 2017 17:41:37 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 339,907 | 2,162,166 | 6.3611 |
19 Sep 2017 11:12:59 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 337,027 | 2,140,884 | 6.3523 |
19 Sep 2017 05:41:05 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 334,147 | 2,121,934 | 6.3503 |
19 Sep 2017 00:19:58 | 1339864 | 20613146 | wah2_global_a01o_208812_145_615_011130682_1 | 331,267 | 2,103,377 | 6.3495 |
©2024 cpdn.org