Name | famous_w84t_599_200_006757229_1 |
Workunit | 6960545 |
Created | 13 Nov 2010, 0:57:41 UTC |
Sent | 13 Nov 2010, 1:07:51 UTC |
Report deadline | 12 Feb 2011, 8:35:02 UTC |
Received | 22 Dec 2010, 18:25:48 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 193 (0x000000C1) EXIT_SIGNAL |
Computer ID | 989329 |
Run time | 9 days 7 hours 50 min 48 sec |
CPU time | 8 days 18 hours 5 min 41 sec |
Validate state | Invalid |
Credit | 2,377.97 |
Device peak FLOPS | 1.16 GFLOPS |
Application version | UK Met Office FAMOUS v6.11 i686-pc-linux-gnu |
Stderr | <core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7685, iMonCtr=1 Model crash detected, will try to restart... (7685): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (7802): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting (1356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... (1627): called boinc_finish CPDN Monitor - Quit request from BOINC... (1335): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1275): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0x739400] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f906] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805085a] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x2bbbd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... (1221): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (1360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... (1730): called boinc_finish (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting (1365): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (1309): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... (1713): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0x1fb400] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f906] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805085a] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x2b3bd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Signal 15 received, exiting... Signal 15 received, exiting... (1674): called boinc_finish (1259): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... (1684): called boinc_finish CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (1279): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (1665): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (1732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (1759): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (1783): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (1901): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (1940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (1965): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2149): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2251): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2290): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2613): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2743): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2767): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2807): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2831): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2855): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1321): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xa3c400] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f906] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805085a] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x446bd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Signal 15 received, exiting... (1520): called boinc_finish CPDN Monitor - Quit request from BOINC... (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting (1344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1208): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1212): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (1209): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (1265): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (1280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting (1407): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (2177): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1318): called boinc_finish CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (1318): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (1318): No heartbeat from core client for 30 sec - exiting (1318): No heartbeat from core client for 30 sec - exiting BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (1272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (1321): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... OPEN: File Open Failed: Read-only file system OPEN: Unable to Open File dataout/w84tla#pb000000646c1+ for Read/Write Model crashed: STWORK : Error opening output PP file on unit 61 tmp/pipe_dummy Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1878): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1699, iMonCtr=1 Model crash detected, will try to restart... Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1882): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1699, iMonCtr=1 Model crash detected, will try to restart... Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1898): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1699, iMonCtr=1 Model crash detected, will try to restart... Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1899): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1699, iMonCtr=1 Model crash detected, will try to restart... Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1900): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1699, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( (1699): called boinc_finish Signal 15 received, exiting... (1333): called boinc_finish CPDN Monitor - Quit request from BOINC... (1216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (1280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1335): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... (1927): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2009): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2141): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2173): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 3 received, exiting... SIGSEGV: segmentation violation (1266): called boinc_finish Stack trace (7 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xf30400] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804c7b4] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x80501eb] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x76fbd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
22 Dec 2010 19:41:05 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 720,746 | 752,709 | 1.0443 |
22 Dec 2010 19:41:05 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 711,386 | 742,990 | 1.0444 |
17 Dec 2010 12:07:07 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 702,026 | 733,195 | 1.0444 |
17 Dec 2010 12:07:07 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 692,666 | 723,397 | 1.0444 |
16 Dec 2010 22:38:13 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 683,306 | 713,597 | 1.0443 |
16 Dec 2010 19:50:24 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 673,946 | 703,798 | 1.0443 |
16 Dec 2010 17:05:48 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 664,586 | 693,859 | 1.0440 |
15 Dec 2010 16:45:52 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 655,226 | 684,141 | 1.0441 |
15 Dec 2010 14:18:19 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 645,866 | 674,401 | 1.0442 |
15 Dec 2010 14:18:19 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 636,506 | 664,615 | 1.0442 |
15 Dec 2010 14:18:19 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 627,146 | 654,790 | 1.0441 |
14 Dec 2010 21:57:50 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 617,786 | 644,824 | 1.0438 |
14 Dec 2010 19:29:32 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 608,426 | 635,057 | 1.0438 |
14 Dec 2010 19:29:32 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 599,066 | 625,285 | 1.0438 |
14 Dec 2010 19:29:32 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 589,706 | 615,513 | 1.0438 |
13 Dec 2010 23:49:40 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 580,346 | 605,611 | 1.0435 |
13 Dec 2010 21:01:45 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 570,986 | 595,876 | 1.0436 |
13 Dec 2010 18:29:06 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 561,626 | 586,147 | 1.0437 |
13 Dec 2010 15:35:19 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 552,266 | 576,418 | 1.0437 |
13 Dec 2010 09:22:22 | 989329 | 12020796 | famous_w84t_599_200_006757229_1 | 542,906 | 566,584 | 1.0436 |
©2024 cpdn.org