climateprediction.net home page
Task 12506046

Task 12506046

Name famous_wmrf_1699_200_007107122_1
Workunit 7310294
Created 19 Jan 2011, 16:51:25 UTC
Sent 19 Jan 2011, 16:54:26 UTC
Report deadline 21 Apr 2011, 0:21:37 UTC
Received 31 Jan 2011, 17:03:34 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 25 (0x00000019) Unknown error code
Computer ID 821838
Run time
CPU time 5 days 3 hours 3 min 48 sec
Validate state Invalid
Credit 2,100.04
Device peak FLOPS 1.64 GFLOPS
Application version UK Met Office FAMOUS v6.11
windows_intelx86
Stderr
<core_client_version>5.4.11</core_client_version>
<message>
The drive cannot locate a specific area or track on the disk. (0x19) - exit code 25 (0x19)
</message>
<stderr_txt>
08:24:07 (10532): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:24:08 (10532): No heartbeat from core client for 30 sec - exiting
03:57:31 (4220): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
06:28:14 (5960): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:06:07 (5260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:21:06 (5768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:07:52 (4888): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:13:54 (4760): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:09:17 (660): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:06:52 (5652): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:00:15 (2288): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:00:16 (2288): No heartbeat from core client for 30 sec - exiting
10:52:59 (2996): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
forrtl: The requested operation cannot be performed on a file with a user-mapped section open.
forrtl: severe (38): error during write, unit 6, file C:\Program Files\BOINC\projects\climateprediction.net\famous_wmrf_1699_200_007107122\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown
famous_um_6.11_wi  0088196C  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080BD3E  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080B95B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F0945  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F00F5  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA86B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0FF  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0A0  Unknown               Unknown  Unknown
famous_um_6.11_wi  00776C15  Unknown               Unknown  Unknown
famous_um_6.11_wi  007CA0F6  Unknown               Unknown  Unknown
kernel32.dll       7C817077  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3832, iMonCtr=1
Model crash detected, will try to restart...
10:53:16 (3832): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
forrtl: The requested operation cannot be performed on a file with a user-mapped section open.
forrtl: severe (38): error during write, unit 6, file C:\Program Files\BOINC\projects\climateprediction.net\famous_wmrf_1699_200_007107122\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown
famous_um_6.11_wi  0088196C  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080BD3E  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080B95B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F0945  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F00F5  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA86B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0FF  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0A0  Unknown               Unknown  Unknown
famous_um_6.11_wi  00776C15  Unknown               Unknown  Unknown
famous_um_6.11_wi  007CA0F6  Unknown               Unknown  Unknown
kernel32.dll       7C817077  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4728, iMonCtr=1
Model crash detected, will try to restart...
11:11:22 (4728): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:46:20 (4928): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:12:25 (7192): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:39:02 (7964): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:39:03 (7964): No heartbeat from core client for 30 sec - exiting
13:31:55 (7736): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:14:00 (7804): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:14:02 (7804): No heartbeat from core client for 30 sec - exiting
14:25:23 (6244): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:43:50 (7172): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:18:28 (7020): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:16:01 (7996): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:46:09 (2452): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:30:13 (5320): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:30:15 (5320): No heartbeat from core client for 30 sec - exiting

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
MainError:	09:32:59 PM	No files match the supplied pattern.
MainError:	09:32:59 PM	No files match the supplied pattern.
CPDN Monitor - Quit request from BOINC...
22:05:33 (5900): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:18:31 (4960): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:10:31 (2696): No heartbeat from core client for 30 sec - exiting
15:10:32 (2696): No heartbeat from core client for 30 sec - exiting
15:10:33 (2696): No heartbeat from core client for 30 sec - exiting
15:10:34 (2696): No heartbeat from core client for 30 sec - exiting
15:10:35 (2696): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:32:55 (3896): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:01:40 (4184): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:01:45 (4184): No heartbeat from core client for 30 sec - exiting
18:01:46 (4184): No heartbeat from core client for 30 sec - exiting
18:01:47 (4184): No heartbeat from core client for 30 sec - exiting
18:01:48 (4184): No heartbeat from core client for 30 sec - exiting
18:01:49 (4184): No heartbeat from core client for 30 sec - exiting
18:01:51 (4184): No heartbeat from core client for 30 sec - exiting

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
19:30:35 (5548): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:22:18 (3192): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:22:19 (3192): No heartbeat from core client for 30 sec - exiting
21:22:20 (3192): No heartbeat from core client for 30 sec - exiting
21:22:21 (3192): No heartbeat from core client for 30 sec - exiting
21:22:23 (3192): No heartbeat from core client for 30 sec - exiting
21:22:24 (3192): No heartbeat from core client for 30 sec - exiting
21:22:25 (3192): No heartbeat from core client for 30 sec - exiting
21:22:26 (3192): No heartbeat from core client for 30 sec - exiting
21:22:27 (3192): No heartbeat from core client for 30 sec - exiting
21:22:28 (3192): No heartbeat from core client for 30 sec - exiting
21:22:29 (3192): No heartbeat from core client for 30 sec - exiting
04:48:56 (3720): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:48:57 (3720): No heartbeat from core client for 30 sec - exiting
04:48:58 (3720): No heartbeat from core client for 30 sec - exiting
04:48:59 (3720): No heartbeat from core client for 30 sec - exiting
04:49:01 (3720): No heartbeat from core client for 30 sec - exiting
05:04:25 (5308): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:18:10 (4712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
02:07:53 (6048): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
MainError:	11:10:02 PM	No files match the supplied pattern.
MainError:	11:10:02 PM	No files match the supplied pattern.
04:26:43 (7104): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:26:44 (7104): No heartbeat from core client for 30 sec - exiting

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
05:28:47 (6376): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:39:33 (5672): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:40:06 (764): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:41:03 (7808): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:41:04 (7808): No heartbeat from core client for 30 sec - exiting
07:41:06 (7808): No heartbeat from core client for 30 sec - exiting
07:41:07 (7808): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
07:28:32 (5588): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:14:57 (4156): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:32:15 (6880): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
13:10:02 (860): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:10:26 (5348): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
20:22:52 (5192): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
22:20:33 (7800): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:22:14 (7800): No heartbeat from core client for 30 sec - exiting
22:55:35 (6780): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:29:57 (6012): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:29:58 (6012): No heartbeat from core client for 30 sec - exiting
07:29:59 (6012): No heartbeat from core client for 30 sec - exiting
07:30:01 (6012): No heartbeat from core client for 30 sec - exiting
07:30:02 (6012): No heartbeat from core client for 30 sec - exiting
07:30:03 (6012): No heartbeat from core client for 30 sec - exiting
07:30:04 (6012): No heartbeat from core client for 30 sec - exiting
07:30:05 (6012): No heartbeat from core client for 30 sec - exiting
07:30:06 (6012): No heartbeat from core client for 30 sec - exiting
07:30:07 (6012): No heartbeat from core client for 30 sec - exiting
07:30:08 (6012): No heartbeat from core client for 30 sec - exiting
08:35:44 (6132): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:35:46 (6132): No heartbeat from core client for 30 sec - exiting
08:35:47 (6132): No heartbeat from core client for 30 sec - exiting
08:35:48 (6132): No heartbeat from core client for 30 sec - exiting
08:35:49 (6132): No heartbeat from core client for 30 sec - exiting
08:35:51 (6132): No heartbeat from core client for 30 sec - exiting
08:35:52 (6132): No heartbeat from core client for 30 sec - exiting
15:12:20 (5388): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:12:21 (5388): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:50:13 (5568): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:42:46 (5812): No heartbeat from core client for 30 sec - exiting
04:42:47 (5812): No heartbeat from core client for 30 sec - exiting
04:42:48 (5812): No heartbeat from core client for 30 sec - exiting
04:42:50 (5812): No heartbeat from core client for 30 sec - exiting
04:42:51 (5812): No heartbeat from core client for 30 sec - exiting
04:42:52 (5812): No heartbeat from core client for 30 sec - exiting
04:42:53 (5812): No heartbeat from core client for 30 sec - exiting
04:42:54 (5812): No heartbeat from core client for 30 sec - exiting
04:42:55 (5812): No heartbeat from core client for 30 sec - exiting
04:42:57 (5812): No heartbeat from core client for 30 sec - exiting
04:43:00 (5812): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
17:27:50 (1176): No heartbeat from core client for 30 sec - exiting
17:27:51 (1176): No heartbeat from core client for 30 sec - exiting
17:27:53 (1176): No heartbeat from core client for 30 sec - exiting
17:27:54 (1176): No heartbeat from core client for 30 sec - exiting
17:27:55 (1176): No heartbeat from core client for 30 sec - exiting
17:27:56 (1176): No heartbeat from core client for 30 sec - exiting
17:27:57 (1176): No heartbeat from core client for 30 sec - exiting
17:27:58 (1176): No heartbeat from core client for 30 sec - exiting
17:27:59 (1176): No heartbeat from core client for 30 sec - exiting
17:28:00 (1176): No heartbeat from core client for 30 sec - exiting
17:28:01 (1176): No heartbeat from core client for 30 sec - exiting
17:28:02 (1176): No heartbeat from core client for 30 sec - exiting
17:28:04 (1176): No heartbeat from core client for 30 sec - exiting
17:28:05 (1176): No heartbeat from core client for 30 sec - exiting
17:28:06 (1176): No heartbeat from core client for 30 sec - exiting
17:28:07 (1176): No heartbeat from core client for 30 sec - exiting
17:28:08 (1176): No heartbeat from core client for 30 sec - exiting
17:28:09 (1176): No heartbeat from core client for 30 sec - exiting
17:28:11 (1176): No heartbeat from core client for 30 sec - exiting
17:28:12 (1176): No heartbeat from core client for 30 sec - exiting
17:28:13 (1176): No heartbeat from core client for 30 sec - exiting
17:28:14 (1176): No heartbeat from core client for 30 sec - exiting
17:28:15 (1176): No heartbeat from core client for 30 sec - exiting
17:28:17 (1176): No heartbeat from core client for 30 sec - exiting
17:28:18 (1176): No heartbeat from core client for 30 sec - exiting
17:28:19 (1176): No heartbeat from core client for 30 sec - exiting
17:28:20 (1176): No heartbeat from core client for 30 sec - exiting
17:28:21 (1176): No heartbeat from core client for 30 sec - exiting
17:28:22 (1176): No heartbeat from core client for 30 sec - exiting
17:28:23 (1176): No heartbeat from core client for 30 sec - exiting
17:28:24 (1176): No heartbeat from core client for 30 sec - exiting
17:28:26 (1176): No heartbeat from core client for 30 sec - exiting
17:28:27 (1176): No heartbeat from core client for 30 sec - exiting
17:28:28 (1176): No heartbeat from core client for 30 sec - exiting
17:28:29 (1176): No heartbeat from core client for 30 sec - exiting
17:28:30 (1176): No heartbeat from core client for 30 sec - exiting
17:28:31 (1176): No heartbeat from core client for 30 sec - exiting
17:28:32 (1176): No heartbeat from core client for 30 sec - exiting
17:28:33 (1176): No heartbeat from core client for 30 sec - exiting
17:28:35 (1176): No heartbeat from core client for 30 sec - exiting
17:28:36 (1176): No heartbeat from core client for 30 sec - exiting
17:28:37 (1176): No heartbeat from core client for 30 sec - exiting
17:28:38 (1176): No heartbeat from core client for 30 sec - exiting
17:28:39 (1176): No heartbeat from core client for 30 sec - exiting
17:28:40 (1176): No heartbeat from core client for 30 sec - exiting
17:28:41 (1176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:28:42 (1176): No heartbeat from core client for 30 sec - exiting
17:28:43 (1176): No heartbeat from core client for 30 sec - exiting
17:28:44 (1176): No heartbeat from core client for 30 sec - exiting
17:28:45 (1176): No heartbeat from core client for 30 sec - exiting
17:28:47 (1176): No heartbeat from core client for 30 sec - exiting
17:28:48 (1176): No heartbeat from core client for 30 sec - exiting
17:28:49 (1176): No heartbeat from core client for 30 sec - exiting
19:59:45 (4148): called boinc_finish

</stderr_txt>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
31 Jan 2011 17:07:59 821838 12506046 famous_wmrf_1699_200_007107122_1 636,506 443,089 0.6961
31 Jan 2011 14:46:33 821838 12506046 famous_wmrf_1699_200_007107122_1 627,146 436,628 0.6962
31 Jan 2011 14:46:33 821838 12506046 famous_wmrf_1699_200_007107122_1 617,786 430,206 0.6964
31 Jan 2011 14:46:33 821838 12506046 famous_wmrf_1699_200_007107122_1 608,426 423,833 0.6966
31 Jan 2011 14:46:33 821838 12506046 famous_wmrf_1699_200_007107122_1 599,066 417,474 0.6969
30 Jan 2011 18:40:34 821838 12506046 famous_wmrf_1699_200_007107122_1 589,706 411,057 0.6971
30 Jan 2011 16:10:56 821838 12506046 famous_wmrf_1699_200_007107122_1 580,346 404,537 0.6971
30 Jan 2011 01:22:58 821838 12506046 famous_wmrf_1699_200_007107122_1 570,986 398,231 0.6974
29 Jan 2011 22:38:51 821838 12506046 famous_wmrf_1699_200_007107122_1 561,626 391,811 0.6976
29 Jan 2011 20:01:45 821838 12506046 famous_wmrf_1699_200_007107122_1 552,266 385,403 0.6979
29 Jan 2011 16:06:00 821838 12506046 famous_wmrf_1699_200_007107122_1 542,906 379,078 0.6982
29 Jan 2011 13:45:56 821838 12506046 famous_wmrf_1699_200_007107122_1 533,546 372,538 0.6982
29 Jan 2011 12:19:23 821838 12506046 famous_wmrf_1699_200_007107122_1 524,186 365,963 0.6982
29 Jan 2011 12:19:23 821838 12506046 famous_wmrf_1699_200_007107122_1 514,826 359,471 0.6982
29 Jan 2011 12:19:23 821838 12506046 famous_wmrf_1699_200_007107122_1 505,466 352,892 0.6982
28 Jan 2011 23:23:39 821838 12506046 famous_wmrf_1699_200_007107122_1 496,106 346,332 0.6981
28 Jan 2011 20:31:31 821838 12506046 famous_wmrf_1699_200_007107122_1 486,746 339,766 0.6980
28 Jan 2011 18:48:21 821838 12506046 famous_wmrf_1699_200_007107122_1 477,386 333,451 0.6985
28 Jan 2011 15:10:40 821838 12506046 famous_wmrf_1699_200_007107122_1 468,026 326,996 0.6987
28 Jan 2011 12:50:57 821838 12506046 famous_wmrf_1699_200_007107122_1 458,666 320,715 0.6992


©2024 cpdn.org