climateprediction.net home page
Task 21344192

Task 21344192

Name wah2_pnw25_pjk4_199809_28_754_011633705_2
Workunit 11633705
Created 25 Oct 2018, 11:38:30 UTC
Sent 25 Oct 2018, 11:38:45 UTC
Report deadline 7 Oct 2019, 16:58:45 UTC
Received 9 Dec 2018, 6:40:50 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1458081
Run time 2 days 13 hours 57 min 34 sec
CPU time 2 days 12 hours 57 min 58 sec
Validate state Invalid
Credit 12,938.64
Device peak FLOPS 4.20 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 182.93 MB
Peak swap size 169.23 MB
Peak disk usage 12.45 MB
Stderr
<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
13:39:37 (20672): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
13:39:40 (1660): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
10:39:59 (21980): start_timer_thread(): CreateThread() failed, errno 0
10:40:01 (14396): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
17:07:57 (13836): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
17:07:59 (23344): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
13:46:59 (15768): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
13:47:01 (19144): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
18:49:24 (21360): start_timer_thread(): CreateThread() failed, errno 0
18:49:26 (20652): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
04:00:09 (22504): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
04:00:11 (21072): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
02:09:04 (7992): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
02:09:05 (5500): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
04:20:43 (11436): start_timer_thread(): CreateThread() failed, errno 0
04:20:45 (20116): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
17:28:09 (20744): start_timer_thread(): CreateThread() failed, errno 0
17:28:11 (20800): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:20:44 (13420): start_timer_thread(): CreateThread() failed, errno 0
11:20:47 (4264): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
14:23:48 (18884): start_timer_thread(): CreateThread() failed, errno 0
14:23:50 (14404): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
15:56:55 (7532): start_timer_thread(): CreateThread() failed, errno 0
15:56:57 (21344): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
13:31:32 (1976): start_timer_thread(): CreateThread() failed, errno 0
13:31:33 (21356): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
16:33:33 (7080): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
16:33:35 (12624): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
19:34:10 (8064): start_timer_thread(): CreateThread() failed, errno 0
19:34:12 (20108): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
23:47:55 (21848): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
23:47:57 (21228): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
10:46:27 (17280): start_timer_thread(): CreateThread() failed, errno 0
10:46:29 (19940): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:31:43 (15052): start_timer_thread(): CreateThread() failed, errno 0
16:31:45 (21620): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
17:08:05 (24980): start_timer_thread(): CreateThread() failed, errno 0
17:08:07 (2096): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
20:54:39 (20924): start_timer_thread(): CreateThread() failed, errno 0
20:54:40 (24924): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
04:07:03 (9316): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
04:07:04 (24984): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
10:23:36 (10936): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
10:23:38 (19644): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
18:46:10 (19512): start_timer_thread(): CreateThread() failed, errno 0
18:46:12 (16828): start_timer_thread(): CreateThread() failed, errno 0
GCM : BUFFIN: C I/O Error feof - Unit 61 - Return code = 16
GCM : BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

GCM : BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
GCM : BUFFIN: C I/O Error feof - Unit 63 - Return code = 16

GCM : BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
GCM : BUFFIN: C I/O Error feof - Unit 64 - Return code = 16

Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16828, selfPID=17352, iMonCtr=1
Model crash detected, will try to restart...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
22:52:15 (19864): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
22:52:17 (21076): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
12:08:49 (15384): start_timer_thread(): CreateThread() failed, errno 0
12:08:51 (11572): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
08:42:39 (15592): start_timer_thread(): CreateThread() failed, errno 0
08:42:41 (1812): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
17:41:10 (11084): start_timer_thread(): CreateThread() failed, errno 0
17:41:12 (8816): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
06:17:04 (9908): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
06:17:06 (14364): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
06:23:47 (19252): start_timer_thread(): CreateThread() failed, errno 0
diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12
WARNING: BOINC Windows Runtime Debugger has been disabled.
06:23:49 (720): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
07:34:03 (11636): start_timer_thread(): CreateThread() failed, errno 0
07:34:05 (14128): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14128, selfPID=18880, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_ain::Monitor...
07:39:48 (18880): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_18.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_19.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_20.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_21.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_22.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_23.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_24.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_25.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_26.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_27.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_28.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_pnw25_pjk4_199809_28_754_011633705_2_r1319282688_restart.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 Dec 2018 05:21:58 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 196,139 215,344 1.0979
07 Dec 2018 10:24:49 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 184,619 203,044 1.0998
06 Dec 2018 13:36:01 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 173,099 191,665 1.1073
27 Nov 2018 21:13:34 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 161,579 180,334 1.1161
27 Nov 2018 13:19:27 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 150,059 168,412 1.1223
27 Nov 2018 09:51:56 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 138,539 156,283 1.1281
25 Nov 2018 19:18:02 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 127,019 143,577 1.1304
24 Nov 2018 12:35:04 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 115,499 130,389 1.1289
24 Nov 2018 00:55:46 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 103,979 116,827 1.1236
23 Nov 2018 19:58:49 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 92,459 103,497 1.1194
23 Nov 2018 14:31:33 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 80,939 89,905 1.1108
21 Nov 2018 14:21:04 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 69,419 76,543 1.1026
20 Nov 2018 16:30:55 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 57,899 63,046 1.0889
14 Nov 2018 02:56:37 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 46,379 50,534 1.0896
01 Nov 2018 11:14:50 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 34,859 38,063 1.0919
01 Nov 2018 11:05:54 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 23,339 25,449 1.0904
25 Oct 2018 15:54:20 1458081 21344192 wah2_pnw25_pjk4_199809_28_754_011633705_2 11,819 12,834 1.0859


©2024 cpdn.org