climateprediction.net home page
Task 10946731

Task 10946731

Name hadsm3dhet2_jjoi_006588500_6
Workunit 6791873
Created 15 Mar 2010, 11:50:25 UTC
Sent 26 Oct 2010, 16:59:25 UTC
Report deadline 8 Oct 2011, 22:19:25 UTC
Received 1 Dec 2010, 0:37:43 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 952281
Run time 1 days 18 hours 58 min 44 sec
CPU time 1 days 17 hours 29 min 48 sec
Validate state Invalid
Credit 1,190.92
Device peak FLOPS 2.35 GFLOPS
Application version UK Met Office HadSM3 Slab Model v6.09
i686-apple-darwin
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
forrtl: severe (24): end-of-file during read, unit 9, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_jjoi_006588500/tmp/cp.namelists
Image              PC        Routine            Line        Source             
libintlc.dylib     00CCFE5F  Unknown               Unknown  Unknown
libintlc.dylib     00CCF28B  Unknown               Unknown  Unknown
libifcoremt.dylib  008D2290  Unknown               Unknown  Unknown
libifcoremt.dylib  00852D78  Unknown               Unknown  Unknown
libifcoremt.dylib  008524F5  Unknown               Unknown  Unknown
libifcoremt.dylib  0088DF08  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001B742A  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0024C0A2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E362  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0029A2BE  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000023B2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000022D9  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5987, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (24): end-of-file during read, unit 9, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_jjoi_006588500/tmp/cp.namelists
Image              PC        Routine            Line        Source             
libintlc.dylib     00CCFE5F  Unknown               Unknown  Unknown
libintlc.dylib     00CCF28B  Unknown               Unknown  Unknown
libifcoremt.dylib  008D2290  Unknown               Unknown  Unknown
libifcoremt.dylib  00852D78  Unknown               Unknown  Unknown
libifcoremt.dylib  008524F5  Unknown               Unknown  Unknown
libifcoremt.dylib  0088DF08  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001B742A  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0024C0A2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E362  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0029A2BE  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000023B2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000022D9  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5987, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (24): end-of-file during read, unit 9, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_jjoi_006588500/tmp/cp.namelists
Image              PC        Routine            Line        Source             
libintlc.dylib     00CCFE5F  Unknown               Unknown  Unknown
libintlc.dylib     00CCF28B  Unknown               Unknown  Unknown
libifcoremt.dylib  008D2290  Unknown               Unknown  Unknown
libifcoremt.dylib  00852D78  Unknown               Unknown  Unknown
libifcoremt.dylib  008524F5  Unknown               Unknown  Unknown
libifcoremt.dylib  0088DF08  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001B742A  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0024C0A2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E362  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0029A2BE  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000023B2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000022D9  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5987, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (24): end-of-file during read, unit 9, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_jjoi_006588500/tmp/cp.namelists
Image              PC        Routine            Line        Source             
libintlc.dylib     00CCFE5F  Unknown               Unknown  Unknown
libintlc.dylib     00CCF28B  Unknown               Unknown  Unknown
libifcoremt.dylib  008D2290  Unknown               Unknown  Unknown
libifcoremt.dylib  00852D78  Unknown               Unknown  Unknown
libifcoremt.dylib  008524F5  Unknown               Unknown  Unknown
libifcoremt.dylib  0088DF08  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001B742A  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0024C0A2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E362  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0029A2BE  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000023B2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000022D9  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5987, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (24): end-of-file during read, unit 9, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_jjoi_006588500/tmp/cp.namelists
Image              PC        Routine            Line        Source             
libintlc.dylib     00CCFE5F  Unknown               Unknown  Unknown
libintlc.dylib     00CCF28B  Unknown               Unknown  Unknown
libifcoremt.dylib  008D2290  Unknown               Unknown  Unknown
libifcoremt.dylib  00852D78  Unknown               Unknown  Unknown
libifcoremt.dylib  008524F5  Unknown               Unknown  Unknown
libifcoremt.dylib  0088DF08  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001B742A  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0024C0A2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E362  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0029A2BE  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000023B2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000022D9  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5987, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (24): end-of-file during read, unit 9, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadsm3dhet2_jjoi_006588500/tmp/cp.namelists
Image              PC        Routine            Line        Source             
libintlc.dylib     00CCFE5F  Unknown               Unknown  Unknown
libintlc.dylib     00CCF28B  Unknown               Unknown  Unknown
libifcoremt.dylib  008D2290  Unknown               Unknown  Unknown
libifcoremt.dylib  00852D78  Unknown               Unknown  Unknown
libifcoremt.dylib  008524F5  Unknown               Unknown  Unknown
libifcoremt.dylib  0088DF08  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  001B742A  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0024C0A2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0027E362  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  0029A2BE  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000023B2  Unknown               Unknown  Unknown
hadsm3_um_6.09_i6  000022D9  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5987, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
30 Nov 2010 17:55:24 952281 10946731 hadsm3dhet2_jjoi_006588500_6 129,624 142,580 1.1000
30 Nov 2010 06:36:57 952281 10946731 hadsm3dhet2_jjoi_006588500_6 118,822 129,993 1.0940
29 Nov 2010 09:19:22 952281 10946731 hadsm3dhet2_jjoi_006588500_6 108,020 118,130 1.0936
28 Nov 2010 07:28:13 952281 10946731 hadsm3dhet2_jjoi_006588500_6 97,218 106,442 1.0949
27 Nov 2010 11:47:30 952281 10946731 hadsm3dhet2_jjoi_006588500_6 86,416 94,467 1.0932
26 Nov 2010 11:44:08 952281 10946731 hadsm3dhet2_jjoi_006588500_6 75,614 82,398 1.0897
23 Nov 2010 21:34:28 952281 10946731 hadsm3dhet2_jjoi_006588500_6 64,812 70,866 1.0934
23 Nov 2010 10:36:42 952281 10946731 hadsm3dhet2_jjoi_006588500_6 54,010 59,244 1.0969
21 Nov 2010 06:03:54 952281 10946731 hadsm3dhet2_jjoi_006588500_6 43,208 47,339 1.0956
20 Nov 2010 09:25:05 952281 10946731 hadsm3dhet2_jjoi_006588500_6 32,406 35,425 1.0932
19 Nov 2010 20:53:50 952281 10946731 hadsm3dhet2_jjoi_006588500_6 21,604 23,210 1.0743
19 Nov 2010 10:54:26 952281 10946731 hadsm3dhet2_jjoi_006588500_6 10,802 11,713 1.0843


©2024 cpdn.org