climateprediction.net home page
Task 22484130

Task 22484130

Name wah2_eas25_a36r_201812_24_1024_012326386_0
Workunit 12326386
Created 24 Jul 2024, 18:03:47 UTC
Sent 24 Aug 2024, 3:14:43 UTC
Report deadline 2 Dec 2024, 3:14:43 UTC
Received 3 Oct 2024, 1:04:10 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -529697949 (0xE06D7363) Unknown error code
Computer ID 1439016
Run time 23 days 7 hours 39 min 34 sec
CPU time 21 days 7 hours 9 min 16 sec
Validate state Invalid
Credit 12,446.46
Device peak FLOPS 3.15 GFLOPS
Application version Weather At Home 2 (wah2) (region independent) v8.32
windows_intelx86
Peak working set size 346.40 MB
Peak swap size 310.72 MB
Peak disk usage 200.13 MB
Stderr
<core_client_version>8.0.2</core_client_version>
<![CDATA[
<message>
(unknown error) (317) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
modelGetExecutables: check control files, strTemp0 & 1 : 
D:\BOINC/projects/climateprediction.net/wah2_eas25_a36r_201812_24_1024_012326386/jobs/xadae.namelists
D:\BOINC/projects/climateprediction.net/wah2_eas25_a36r_201812_24_1024_012326386/jobs/xacxf.namelists
modelGetExecutables: unzipping control files : strInput & strTmp 
wah2_eas25_a36r_201812_24_1024_012326386.zip
wah2_eas25_a36r_201812_24_1024_012326386/jobs
gstrDump[0] = generic_phase1_spinup_eas25_global_aabaka_f
gstrDump[1] = generic_phase1_spinup_eas25_regional_aabaka_f
global model: command string: "D:\BOINC/projects/climateprediction.net/wah2am3m2_um_8.32_windows_intelx86.exe" wah2_eas25_a36r_201812_24_1024_012326386 generic_phase1_spinup_eas25_global_aabaka_f ic19611029_11_N96 AERclim_ancil_168months_CMIP6-MRI-ESM2-0_SST_2009-01-01_2022-12-30_v2404b AERclim_ancil_168months_CMIP6-MRI-ESM2-0_SIC_2009-01-01_2022-12-30_v2404b SO2DMS_N96_cmip6hist-ssp245_2009-2020 oxi.addfa ozone_cmip6hist-ssp245_N96_1979_2031
regional model: command string: "D:\BOINC/projects/climateprediction.net/wah2rm3m2t_um_8.32_windows_intelx86.exe" wah2_eas25_a36r_201812_24_1024_012326386
 cpdn_check_running: got RM PID of zero; ignoring this call and waiting for PID via shMem. 
 cpdn_check_running: got RM PID of zero; ignoring this call and waiting for PID via shMem. 
executeModelProcess: MonID=27868, GCM_PID=19008, RCM_PID=30484
Suspended CPDN Monitor - Suspend request from BOINC...
modelGetExecutables: check control files, strTemp0 & 1 : 
D:\BOINC/projects/climateprediction.net/wah2_eas25_a36r_201812_24_1024_012326386/jobs/xadae.namelists
D:\BOINC/projects/climateprediction.net/wah2_eas25_a36r_201812_24_1024_012326386/jobs/xacxf.namelists
gstrDump[0] = generic_phase1_spinup_eas25_global_aabaka_f
gstrDump[1] = generic_phase1_spinup_eas25_regional_aabaka_f
global model: command string: "D:\BOINC/projects/climateprediction.net/wah2am3m2_um_8.32_windows_intelx86.exe" wah2_eas25_a36r_201812_24_1024_012326386 generic_phase1_spinup_eas25_global_aabaka_f ic19611029_11_N96 AERclim_ancil_168months_CMIP6-MRI-ESM2-0_SST_2009-01-01_2022-12-30_v2404b AERclim_ancil_168months_CMIP6-MRI-ESM2-0_SIC_2009-01-01_2022-12-30_v2404b SO2DMS_N96_cmip6hist-ssp245_2009-2020 oxi.addfa ozone_cmip6hist-ssp245_N96_1979_2031
regional model: command string: "D:\BOINC/projects/climateprediction.net/wah2rm3m2t_um_8.32_windows_intelx86.exe" wah2_eas25_a36r_201812_24_1024_012326386
 cpdn_check_running: got RM PID of zero; ignoring this call and waiting for PID via shMem. 
 cpdn_check_running: got RM PID of zero; ignoring this call and waiting for PID via shMem. 
executeModelProcess: MonID=7196, GCM_PID=10472, RCM_PID=4636
Queuing intermediate upload for CPDN/BOINC: cpdnout1.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout2.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout3.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout4.zip
Suspended CPDN Monitor - Suspend request from BOINC...
Queuing intermediate upload for CPDN/BOINC: cpdnout5.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout6.zip
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Queuing intermediate upload for CPDN/BOINC: cpdnout7.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout8.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout9.zip
Suspended CPDN Monitor - Suspend request from BOINC...
Queuing intermediate upload for CPDN/BOINC: cpdnout10.zip
Suspended CPDN Monitor - Suspend request from BOINC...
Queuing intermediate upload for CPDN/BOINC: cpdnout11.zip
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Queuing intermediate upload for CPDN/BOINC: cpdnout_restart.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout12.zip
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Detaching shared memory... Done.
modelGetExecutables: check control files, strTemp0 & 1 : 
D:\BOINC/projects/climateprediction.net/wah2_eas25_a36r_201812_24_1024_012326386/jobs/xadae.namelists
D:\BOINC/projects/climateprediction.net/wah2_eas25_a36r_201812_24_1024_012326386/jobs/xacxf.namelists
gstrDump[0] = generic_phase1_spinup_eas25_global_aabaka_f
gstrDump[1] = generic_phase1_spinup_eas25_regional_aabaka_f
global model: command string: "D:\BOINC/projects/climateprediction.net/wah2am3m2_um_8.32_windows_intelx86.exe" wah2_eas25_a36r_201812_24_1024_012326386 generic_phase1_spinup_eas25_global_aabaka_f ic19611029_11_N96 AERclim_ancil_168months_CMIP6-MRI-ESM2-0_SST_2009-01-01_2022-12-30_v2404b AERclim_ancil_168months_CMIP6-MRI-ESM2-0_SIC_2009-01-01_2022-12-30_v2404b SO2DMS_N96_cmip6hist-ssp245_2009-2020 oxi.addfa ozone_cmip6hist-ssp245_N96_1979_2031
regional model: command string: "D:\BOINC/projects/climateprediction.net/wah2rm3m2t_um_8.32_windows_intelx86.exe" wah2_eas25_a36r_201812_24_1024_012326386
 cpdn_check_running: got RM PID of zero; ignoring this call and waiting for PID via shMem. 
 cpdn_check_running: got RM PID of zero; ignoring this call and waiting for PID via shMem. 
executeModelProcess: MonID=11252, GCM_PID=25784, RCM_PID=18784
Queuing intermediate upload for CPDN/BOINC: cpdnout13.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout14.zip
Queuing intermediate upload for CPDN/BOINC: cpdnout15.zip


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74F61072

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.20.2


Dump Timestamp    : 10/03/24 02:52:06
Install Directory : 
Data Directory    : D:\BOINC
Project Symstore  : 
LoadLibraryA( D:\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( D:\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: D:\BOINC\projects\climateprediction.net\wah2_eas25_a36r_201812_24_1024_012326386;D:\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000189000 D:\BOINC\projects\climateprediction.net\wah2_8.32_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000076f80000 00000000001a4000 C:\Windows\SYSTEM32\ntdll.dll (6.2.19041.4842) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.19041.4842 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.4842

ModLoad: 0000000075370000 00000000000f0000 C:\Windows\System32\KERNEL32.DLL (6.2.19041.4842) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.19041.4842 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.4842

ModLoad: 0000000074e20000 000000000023f000 C:\Windows\System32\KERNELBASE.dll (6.2.19041.4894) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.19041.4842 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.4842

ModLoad: 0000000076060000 000000000019d000 C:\Windows\System32\USER32.dll (6.2.19041.4894) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.19041.4522 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.4522

ModLoad: 00000000757d0000 0000000000018000 C:\Windows\System32\win32u.dll (6.2.19041.4894) (-exported- Symbols Loaded)
    Linked PDB Filename   : wwin32u.pdb
    File Version          : 10.0.19041.4894 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.4894

ModLoad: 00000000756b0000 0000000000023000 C:\Windows\System32\GDI32.dll (6.2.19041.4474) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.19041.4474 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.4474

ModLoad: 0000000076510000 00000000000e6000 C:\Windows\System32\gdi32full.dll (6.2.19041.4894) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32full.pdb
    File Version          : 10.0.19041.4894 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.4894

ModLoad: 0000000075750000 000000000007b000 C:\Windows\System32\msvcp_win.dll (6.2.19041.3636) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.19041.3636 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.3636

ModLoad: 0000000075180000 0000000000120000 C:\Windows\System32\ucrtbase.dll (6.2.19041.3636) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.19041.3636 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.3636

ModLoad: 0000000075100000 000000000007d000 C:\Windows\System32\ADVAPI32.dll (6.2.19041.4597) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000076eb0000 00000000000bf000 C:\Windows\System32\msvcrt.dll (7.0.19041.3636) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.19041.3636 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.19041.3636

ModLoad: 0000000076770000 0000000000078000 C:\Windows\System32\sechost.dll (6.2.19041.4597) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000075460000 00000000000bc000 C:\Windows\System32\RPCRT4.dll (6.2.19041.4597) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.19041.3636 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.3636

ModLoad: 0000000075a60000 0000000000019000 C:\Windows\System32\bcrypt.dll (6.2.19041.3636) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcrypt.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000076e80000 0000000000025000 C:\Windows\System32\IMM32.DLL (6.2.19041.4474) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 10.0.19041.4474 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.4474

ModLoad: 0000000073d80000 000000000000f000 C:\Windows\SYSTEM32\kernel.appcore.dll (6.2.19041.3758) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.19041.3758 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.3758

ModLoad: 00000000746e0000 0000000000029000 C:\Windows\SYSTEM32\ntmarta.dll (6.2.19041.3636) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntmarta.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000055330000 0000000000086000 D:\BOINC\projects\climateprediction.net\wah2_se_8.32_windows_intelx86.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : F:\cpdn\cpdn\cpdnboinc\monitor\win_build\Release\umtonc.pdb

ModLoad: 00000000691f0000 0000000000188000 C:\Windows\SYSTEM32\dbghelp.dll (6.2.19041.3996) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.19041.3996 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.3996

ModLoad: 0000000074850000 0000000000008000 C:\Windows\SYSTEM32\version.dll (6.2.19041.3636) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.19041.3636 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.3636

ModLoad: 00000000752b0000 000000000005f000 C:\Windows\System32\bcryptPrimitives.dll (6.2.19041.3636) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.19041.3636 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.3636



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 225475, Write: 0, Other 162428

- I/O Transfers Counters -
Read: 0, Write: 3904, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 244440, QuotaPeakPagedPoolUsage: 244488
QuotaNonPagedPoolUsage: 10320, QuotaPeakNonPagedPoolUsage: 10584

- Virtual Memory Usage -
VirtualSize: 131100672, PeakVirtualSize: 208105472

- Pagefile Usage -
PagefileUsage: 4980736, PeakPagefileUsage: 81588224

- Working Set Size -
WorkingSetSize: 82595840, PeakWorkingSetSize: 159723520, PageFaultCount: 128755

*** Dump of thread ID 31476 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 192031248.000000, User Time: 733593728.000000, Wait Time: 38370244.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74F61072

- Registers -
eax=0019e600 ebx=00b80048 ecx=00000003 edx=00000000 esi=5539922c edi=553ac450
eip=74f61072 esp=0019e600 ebp=0019e65c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000216

- Callstack -
ChildEBP RetAddr  Args to Child
0019e65c 5537caf9 e06d7363 00000001 00000003 0019e688 KERNELBASE!RaiseException+0x0 
0019e694 55379ebc 0019e6a4 553a59d8 5539922c 5539b27c wah2_se_8.32_windows_intelx86!umtonc+0x0 
0019e6b4 5533e42b 0491f7d4 00b80048 004e7c61 00a00ac0 wah2_se_8.32_windows_intelx86!umtonc+0x0 
0019e6d4 553329b1 0019e70c 0019e7f8 0019e7e0 0019e7d4 wah2_se_8.32_windows_intelx86!+0x0 
0019e91c 5533efac 00a02d58 0019e9b1 00000000 0019eab8 wah2_se_8.32_windows_intelx86!+0x0 
0019eab8 00407759 004e7c60 004e7d68 004e7e70 00000000 wah2_se_8.32_windows_intelx86!umtonc+0x0 
0019eaf4 004028ed 00851570 008515a0 008515a0 00000000 wah2_8.32_windows_intelx86!+0x0 
0019eb3c 004039b8 000007e4 00000003 00000001 004e6928 wah2_8.32_windows_intelx86!+0x0 
0019ecf0 00403e61 000007e4 00000003 03410040 0019ed48 wah2_8.32_windows_intelx86!+0x0 
0019ed04 00409cc6 000007e4 03410040 004e6928 004b963c wah2_8.32_windows_intelx86!+0x0 
0019ed48 0040c9a1 00000000 8478a4c6 00000000 0000000a wah2_8.32_windows_intelx86!+0x0 
0019fd70 0040cc26 0019ff24 0040ca5d 0000000a 0019fd90 wah2_8.32_windows_intelx86!+0x0 
0019fd78 0040ca5d 0000000a 0019fd90 00843af8 0019fd90 wah2_8.32_windows_intelx86!+0x0 
0019ff24 00484e62 00400000 00000000 00843b36 0000000a wah2_8.32_windows_intelx86!+0x0 
0019ff70 7538fcc9 00349000 7538fcb0 0019ffdc 76fe80ce wah2_8.32_windows_intelx86!+0x0 
0019ff80 76fe80ce 00349000 f33ca9d1 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0019ffdc 76fe809e ffffffff 7700919c 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
0019ffec 00000000 00484ee6 00349000 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 7884 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 6875000.000000, User Time: 10937500.000000, Wait Time: 38370240.000000

- Registers -
eax=00000000 ebx=013dff30 ecx=00000000 edx=00000000 esi=00000000 edi=013dff30
eip=76ff334c esp=013dfef0 ebp=013dff54
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000216

- Callstack -
ChildEBP RetAddr  Args to Child
013dff54 74f5b63f 00000064 00000000 013dff80 0047c5eb ntdll!ZwDelayExecution+0x0 
013dff64 0047c5eb 00000064 0047c5e0 7538fcc9 00000000 KERNELBASE!Sleep+0x0 
013dff80 76fe80ce 00000000 f218a9d1 00000000 00000000 wah2_8.32_windows_intelx86!+0x0 
013dffdc 76fe809e ffffffff 7700919c 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
013dffec 00000000 0047c5e0 00000000 00000000 00000010 ntdll!RtlGetAppContainerNamedObjectPath+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Global Worker:: CPDN process is not running, exiting, bRetVal = T, checkPID = 25784, selfPID = 25784, iMonCtr = 1
Regional Worker:: CPDN process is not running, exiting, bRetVal = T, checkPID = 25784, selfPID = 18784, iMonCtr = 1

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
01 Oct 2024 19:37:31 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 173,099 1,740,618 10.0556
30 Sep 2024 13:06:30 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 161,579 1,639,314 10.1456
29 Sep 2024 08:55:18 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 150,059 1,540,883 10.2685
18 Sep 2024 17:08:11 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 138,539 1,426,300 10.2953
18 Sep 2024 17:08:11 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 127,019 1,306,985 10.2897
13 Sep 2024 07:29:34 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 115,499 1,190,222 10.3050
11 Sep 2024 18:00:26 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 103,979 1,068,938 10.2803
10 Sep 2024 05:08:10 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 92,459 947,227 10.2448
08 Sep 2024 16:26:42 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 80,939 825,445 10.1984
07 Sep 2024 02:02:58 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 69,419 703,684 10.1368
05 Sep 2024 13:30:31 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 57,899 583,004 10.0693
04 Sep 2024 00:24:02 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 46,379 463,344 9.9904
02 Sep 2024 11:33:28 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 34,859 343,538 9.8551
01 Sep 2024 00:04:19 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 23,339 226,586 9.7085
30 Aug 2024 11:02:31 1439016 22484130 wah2_eas25_a36r_201812_24_1024_012326386_0 11,819 107,406 9.0876


©2024 cpdn.org