climateprediction.net home page
Task 10991880

Task 10991880

Name hadsm3dhet2_jn5x_006593015_3
Workunit 6796388
Created 15 Mar 2010, 11:58:12 UTC
Sent 14 Oct 2010, 9:03:06 UTC
Report deadline 26 Sep 2011, 14:23:06 UTC
Received 12 Dec 2010, 8:53:47 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 752666
Run time
CPU time 15 days 10 hours 24 min 43 sec
Validate state Invalid
Credit 2,778.81
Device peak FLOPS 1.00 GFLOPS
Application version UK Met Office HadSM3 Slab Model v6.07
windows_intelx86
Stderr
<core_client_version>6.2.19</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN process is not running, exiting, bRetVal = 1, checkPID=424, selfPID=424, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN process is not running, exiting, bRetVal = 1, checkPID=3964, selfPID=3964, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
forrtl: Access is denied.
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
Can't acquire lockfile - exiting
Can't acquire lockfile - exiting
Can't acquire lockfile - exiting
Can't acquire lockfile - exiting
forrtl: Access is denied.
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2456, iMonCtr=1
Model crash detected, will try to restart...
forrtl: Access is denied.
CPDN process is not running, exiting, bRetVal = 1, checkPID=3436, selfPID=3436, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
MainError:	02:14:46 PM	No files match the supplied pattern.
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
forrtl: Access is denied.
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3172, iMonCtr=1
Model crash detected, will try to restart...
forrtl: Access is denied.
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3172, iMonCtr=1
Model crash detected, will try to restart...
forrtl: Access is denied.
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3172, iMonCtr=1
Model crash detected, will try to restart...
forrtl: Access is denied.
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3172, iMonCtr=1
Model crash detected, will try to restart...
forrtl: Access is denied.
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3172, iMonCtr=1
Model crash detected, will try to restart...
forrtl: Access is denied.
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3172, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
10 Dec 2010 11:51:02 752666 10991880 hadsm3dhet2_jn5x_006593015_3 43,208 1,308,047 4.3248
08 Dec 2010 08:58:25 752666 10991880 hadsm3dhet2_jn5x_006593015_3 32,406 1,261,339 4.3248
07 Dec 2010 09:49:43 752666 10991880 hadsm3dhet2_jn5x_006593015_3 21,604 1,217,569 4.3353
04 Dec 2010 11:18:40 752666 10991880 hadsm3dhet2_jn5x_006593015_3 10,802 1,172,052 4.3401
02 Dec 2010 16:09:47 752666 10991880 hadsm3dhet2_jn5x_006593015_3 259,248 1,128,715 4.3538
30 Nov 2010 23:04:17 752666 10991880 hadsm3dhet2_jn5x_006593015_3 248,446 1,085,297 4.3683
29 Nov 2010 07:41:43 752666 10991880 hadsm3dhet2_jn5x_006593015_3 237,644 1,042,318 4.3860
27 Nov 2010 08:53:55 752666 10991880 hadsm3dhet2_jn5x_006593015_3 226,842 998,890 4.4035
25 Nov 2010 17:35:31 752666 10991880 hadsm3dhet2_jn5x_006593015_3 216,040 956,016 4.4252
23 Nov 2010 18:47:55 752666 10991880 hadsm3dhet2_jn5x_006593015_3 205,238 909,534 4.4316
21 Nov 2010 17:22:53 752666 10991880 hadsm3dhet2_jn5x_006593015_3 194,436 864,867 4.4481
18 Nov 2010 10:14:17 752666 10991880 hadsm3dhet2_jn5x_006593015_3 183,634 820,365 4.4674
16 Nov 2010 07:07:57 752666 10991880 hadsm3dhet2_jn5x_006593015_3 172,832 776,451 4.4925
14 Nov 2010 22:19:41 752666 10991880 hadsm3dhet2_jn5x_006593015_3 162,030 732,624 4.5215
13 Nov 2010 19:16:14 752666 10991880 hadsm3dhet2_jn5x_006593015_3 151,228 689,033 4.5563
13 Nov 2010 09:57:59 752666 10991880 hadsm3dhet2_jn5x_006593015_3 140,426 644,853 4.5921
10 Nov 2010 11:33:44 752666 10991880 hadsm3dhet2_jn5x_006593015_3 129,624 599,694 4.6264
04 Nov 2010 17:03:53 752666 10991880 hadsm3dhet2_jn5x_006593015_3 118,822 537,953 4.5274
02 Nov 2010 21:39:08 752666 10991880 hadsm3dhet2_jn5x_006593015_3 108,020 482,117 4.4632
31 Oct 2010 23:02:11 752666 10991880 hadsm3dhet2_jn5x_006593015_3 97,218 421,028 4.3308


©2024 cpdn.org