climateprediction.net home page
Task 17609769

Task 17609769

Name hadam3pm2_k0wk_1959_10_009271110_1
Workunit 9364026
Created 21 Dec 2014, 4:06:23 UTC
Sent 21 Dec 2014, 11:57:02 UTC
Report deadline 3 Dec 2015, 17:17:02 UTC
Received 26 Dec 2014, 12:41:39 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1327014
Run time 3 days 8 hours 58 min 16 sec
CPU time 4 min 55 sec
Validate state Invalid
Credit 5,074.72
Device peak FLOPS 2.35 GFLOPS
Application version UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03
i686-pc-linux-gnu
Stderr
<core_client_version>6.6.40</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
oa.pc|9dec.nc
oa.pc|0jan.nc
oa.pc|0feb.nc
oa.pc|0mar.nc
oa.pc|0apr.nc
oa.pc|0may.nc
oa.pc|0jun.nc
oa.pc|0jul.nc
oa.pc|0aug.nc
oa.pc|0sep.nc
oa.pc|0oct.nc
oa.pc|0nov.nc
oa.pe|9dec.nc
oa.pe|0jan.nc
oa.pe|0feb.nc
oa.pe|0mar.nc
oa.pe|0apr.nc
oa.pe|0may.nc
oa.pe|0jun.nc
oa.pe|0jul.nc
oa.pe|0aug.nc
oa.pe|0sep.nc
oa.pe|0oct.nc
oa.pe|0nov.nc
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
oa.pc|0dec.nc
oa.pc|1jan.nc
oa.pc|1feb.nc
oa.pc|1mar.nc
oa.pc|1apr.nc
oa.pc|1may.nc
oa.pc|1jun.nc
oa.pc|1jul.nc
oa.pc|1aug.nc
oa.pc|1sep.nc
oa.pc|1oct.nc
oa.pc|1nov.nc
oa.pe|0dec.nc
oa.pe|1jan.nc
oa.pe|1feb.nc
oa.pe|1mar.nc
oa.pe|1apr.nc
oa.pe|1may.nc
oa.pe|1jun.nc
oa.pe|1jul.nc
oa.pe|1aug.nc
oa.pe|1sep.nc
oa.pe|1oct.nc
oa.pe|1nov.nc
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
oa.pc|1dec.nc
oa.pc|2jan.nc
oa.pc|2feb.nc
oa.pc|2mar.nc
oa.pc|2apr.nc
oa.pc|2may.nc
oa.pc|2jun.nc
oa.pc|2jul.nc
oa.pc|2aug.nc
oa.pc|2sep.nc
oa.pc|2oct.nc
oa.pc|2nov.nc
oa.pe|1dec.nc
oa.pe|2jan.nc
oa.pe|2feb.nc
oa.pe|2mar.nc
oa.pe|2apr.nc
oa.pe|2may.nc
oa.pe|2jun.nc
oa.pe|2jul.nc
oa.pe|2aug.nc
oa.pe|2sep.nc
oa.pe|2oct.nc
oa.pe|2nov.nc
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
oa.pc|2dec.nc
oa.pc|3jan.nc
oa.pc|3feb.nc
oa.pc|3mar.nc
oa.pc|3apr.nc
oa.pc|3may.nc
oa.pc|3jun.nc
oa.pc|3jul.nc
oa.pc|3aug.nc
oa.pc|3sep.nc
oa.pc|3oct.nc
oa.pc|3nov.nc
oa.pe|2dec.nc
oa.pe|3jan.nc
oa.pe|3feb.nc
oa.pe|3mar.nc
oa.pe|3apr.nc
oa.pe|3may.nc
oa.pe|3jun.nc
oa.pe|3jul.nc
oa.pe|3aug.nc
oa.pe|3sep.nc
oa.pe|3oct.nc
oa.pe|3nov.nc
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
oa.pc|3dec.nc
oa.pc|4jan.nc
oa.pc|4feb.nc
oa.pc|4mar.nc
oa.pc|4apr.nc
oa.pc|4may.nc
oa.pc|4jun.nc
oa.pc|4jul.nc
oa.pc|4aug.nc
oa.pc|4sep.nc
oa.pc|4oct.nc
oa.pc|4nov.nc
oa.pe|3dec.nc
oa.pe|4jan.nc
oa.pe|4feb.nc
oa.pe|4mar.nc
oa.pe|4apr.nc
oa.pe|4may.nc
oa.pe|4jun.nc
oa.pe|4jul.nc
oa.pe|4aug.nc
oa.pe|4sep.nc
oa.pe|4oct.nc
oa.pe|4nov.nc
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf]
[0xf0f67400]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d]
/lib/libc.so.6(__libc_start_main+0xe6)[0x965d36]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(pthread_cancel+0x69)[0x804c821]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2836, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf]
[0xf0f5c400]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d]
/lib/libc.so.6(__libc_start_main+0xe6)[0x965d36]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(pthread_cancel+0x69)[0x804c821]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2836, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf]
[0xf0fc2400]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d]
/lib/libc.so.6(__libc_start_main+0xe6)[0x965d36]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(pthread_cancel+0x69)[0x804c821]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2836, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf]
[0xf0f9f400]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d]
/lib/libc.so.6(__libc_start_main+0xe6)[0x965d36]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(pthread_cancel+0x69)[0x804c821]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2836, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf]
[0xf0f97400]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d]
/lib/libc.so.6(__libc_start_main+0xe6)[0x965d36]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(pthread_cancel+0x69)[0x804c821]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2836, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf]
[0xf0f1d400]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d]
/lib/libc.so.6(__libc_start_main+0xe6)[0x965d36]
/home/hpc/mimas/dmolina/altres/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(pthread_cancel+0x69)[0x804c821]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2836, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
11:37:33 (2836): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
26 Dec 2014 08:48:05 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 175,748 286,404 1.6296
26 Dec 2014 05:22:24 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 169,988 277,245 1.6310
26 Dec 2014 03:21:47 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 167,108 272,563 1.6311
26 Dec 2014 00:50:36 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 164,228 267,802 1.6307
25 Dec 2014 21:15:04 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 161,348 263,063 1.6304
25 Dec 2014 18:59:44 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 158,468 258,322 1.6301
25 Dec 2014 17:29:30 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 155,588 253,593 1.6299
25 Dec 2014 15:14:10 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 152,708 248,763 1.6290
25 Dec 2014 12:18:41 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 149,828 244,081 1.6291
25 Dec 2014 09:33:15 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 146,948 239,489 1.6298
25 Dec 2014 07:17:47 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 144,068 234,767 1.6296
25 Dec 2014 04:17:15 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 141,188 230,158 1.6302
25 Dec 2014 00:39:21 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 135,428 220,940 1.6314
24 Dec 2014 23:04:07 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 132,548 216,094 1.6303
24 Dec 2014 21:03:49 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 129,668 210,975 1.6270
24 Dec 2014 20:03:40 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 126,788 205,872 1.6237
24 Dec 2014 18:03:20 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 123,908 200,768 1.6203
24 Dec 2014 17:03:08 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 121,028 195,710 1.6171
24 Dec 2014 16:02:59 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 118,148 190,616 1.6134
24 Dec 2014 13:57:36 1327014 17609769 hadam3pm2_k0wk_1959_10_009271110_1 115,268 185,574 1.6099


©2024 cpdn.org