climateprediction.net home page
Task 12087763

Task 12087763

Name famous_s3ji_799_200_006672024_1
Workunit 6875277
Created 18 Nov 2010, 10:48:34 UTC
Sent 19 Nov 2010, 19:45:37 UTC
Report deadline 19 Feb 2011, 3:12:48 UTC
Received 13 Dec 2010, 13:51:00 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID 483262
Run time 4 days 13 hours 53 min 47 sec
CPU time 4 days 19 hours 32 min 37 sec
Validate state Invalid
Credit 3,520.59
Device peak FLOPS 2.65 GFLOPS
Application version UK Met Office FAMOUS v6.11
windows_intelx86
Stderr
<core_client_version>6.6.28</core_client_version>
<![CDATA[
<message>
too many exit(0)s
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
12:05:58 (3440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:08:14 (1740): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:20:38 (3612): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:33:30 (5424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:45:53 (5792): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:59:35 (3148): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:01:20 (4600): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:03:22 (5800): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:05:08 (3380): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:06:54 (644): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
forrtl: The requested operation cannot be performed on a file with a user-mapped section open.
forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_s3ji_799_200_006672024\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown
famous_um_6.11_wi  0088196C  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080BD3E  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080B95B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F0945  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F00F5  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA86B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0FF  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0A0  Unknown               Unknown  Unknown
famous_um_6.11_wi  00776C15  Unknown               Unknown  Unknown
famous_um_6.11_wi  007CA0F6  Unknown               Unknown  Unknown
kernel32.dll       7C817077  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4612, iMonCtr=1
Model crash detected, will try to restart...
20:08:55 (4612): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:10:41 (1272): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:12:28 (5464): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:14:29 (3564): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:16:15 (3376): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:18:02 (5900): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:20:03 (2104): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:21:49 (5372): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:23:35 (1924): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:25:37 (4648): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:27:23 (5136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:29:09 (2496): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:31:11 (5752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:32:57 (3552): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:34:43 (5016): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:36:45 (5320): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:38:31 (2032): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:40:17 (4020): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:42:19 (4316): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:44:05 (3528): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:45:51 (2220): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:47:53 (2288): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:49:39 (5036): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:51:25 (5188): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:53:27 (1616): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:55:13 (5256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:56:59 (5836): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:59:01 (4036): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:00:47 (5460): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:02:33 (4184): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:04:35 (4760): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:06:21 (4424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:08:07 (6096): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:10:09 (948): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:11:55 (5344): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:13:41 (4492): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:15:43 (3044): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:17:29 (5296): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:19:15 (5660): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:21:17 (5504): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:23:03 (5176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:24:49 (3652): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:26:51 (4748): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:28:37 (3576): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:30:24 (5888): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:32:25 (3628): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:34:11 (3096): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:35:57 (4784): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
forrtl: The requested operation cannot be performed on a file with a user-mapped section open.
forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_s3ji_799_200_006672024\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown
famous_um_6.11_wi  0088196C  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080BD3E  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080B95B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F0945  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F00F5  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA86B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0FF  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0A0  Unknown               Unknown  Unknown
famous_um_6.11_wi  00776C15  Unknown               Unknown  Unknown
famous_um_6.11_wi  007CA0F6  Unknown               Unknown  Unknown
kernel32.dll       7C817077  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5640, iMonCtr=1
Model crash detected, will try to restart...
21:37:59 (5640): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:39:45 (1360): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:41:32 (1476): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:43:33 (3708): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:45:19 (5236): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:47:06 (3364): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:49:07 (5268): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:50:53 (5952): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:52:39 (5064): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:54:41 (3528): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:56:27 (4512): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:58:13 (3560): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:00:15 (4428): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:02:01 (5484): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:03:47 (4964): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:05:49 (2324): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:07:35 (5660): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:09:21 (5412): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:11:23 (4144): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:13:09 (4272): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:14:55 (4320): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:16:57 (6040): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:18:43 (2328): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:20:29 (3212): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:22:31 (5032): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:24:17 (5576): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:26:03 (4968): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
forrtl: The requested operation cannot be performed on a file with a user-mapped section open.
forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_s3ji_799_200_006672024\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown
famous_um_6.11_wi  0088196C  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080BD3E  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080B95B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F0945  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F00F5  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA86B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0FF  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0A0  Unknown               Unknown  Unknown
famous_um_6.11_wi  00776C15  Unknown               Unknown  Unknown
famous_um_6.11_wi  007CA0F6  Unknown               Unknown  Unknown
kernel32.dll       7C817077  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5536, iMonCtr=1
Model crash detected, will try to restart...
22:28:05 (5536): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:29:51 (6012): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
forrtl: The requested operation cannot be performed on a file with a user-mapped section open.
forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_s3ji_799_200_006672024\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown
famous_um_6.11_wi  0088196C  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080BD3E  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080B95B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F0945  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F00F5  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA86B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0FF  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0A0  Unknown               Unknown  Unknown
famous_um_6.11_wi  00776C15  Unknown               Unknown  Unknown
famous_um_6.11_wi  007CA0F6  Unknown               Unknown  Unknown
kernel32.dll       7C817077  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5388, iMonCtr=1
Model crash detected, will try to restart...
22:31:37 (5388): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:33:39 (3472): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:35:25 (3140): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:37:11 (4244): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:37:57 (5524): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:38:58 (3804): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:40:48 (6048): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:42:34 (5184): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:44:36 (4632): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:46:22 (3220): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:48:08 (4672): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:50:10 (4924): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:51:56 (4088): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:53:42 (4300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
forrtl: The requested operation cannot be performed on a file with a user-mapped section open.
forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_s3ji_799_200_006672024\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown
famous_um_6.11_wi  0088196C  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080BD3E  Unknown               Unknown  Unknown
famous_um_6.11_wi  0080B95B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F0945  Unknown               Unknown  Unknown
famous_um_6.11_wi  007F00F5  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA86B  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0FF  Unknown               Unknown  Unknown
famous_um_6.11_wi  007BA0A0  Unknown               Unknown  Unknown
famous_um_6.11_wi  00776C15  Unknown               Unknown  Unknown
famous_um_6.11_wi  007CA0F6  Unknown               Unknown  Unknown
kernel32.dll       7C817077  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5912, iMonCtr=1
Model crash detected, will try to restart...
22:55:44 (5912): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:57:30 (5480): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:59:16 (4712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:01:18 (5404): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:03:04 (3128): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
13 Dec 2010 13:54:08 483262 12087763 famous_s3ji_799_200_006672024_1 1,067,066 415,176 0.3891
13 Dec 2010 13:54:08 483262 12087763 famous_s3ji_799_200_006672024_1 1,057,706 411,650 0.3892
13 Dec 2010 13:54:08 483262 12087763 famous_s3ji_799_200_006672024_1 1,048,346 408,116 0.3893
13 Dec 2010 13:54:08 483262 12087763 famous_s3ji_799_200_006672024_1 1,038,986 404,581 0.3894
13 Dec 2010 13:54:08 483262 12087763 famous_s3ji_799_200_006672024_1 1,029,626 401,046 0.3895
13 Dec 2010 13:54:08 483262 12087763 famous_s3ji_799_200_006672024_1 1,020,266 397,659 0.3898
13 Dec 2010 13:54:08 483262 12087763 famous_s3ji_799_200_006672024_1 1,010,906 394,118 0.3899
13 Dec 2010 13:54:08 483262 12087763 famous_s3ji_799_200_006672024_1 1,001,546 390,579 0.3900
11 Dec 2010 13:10:28 483262 12087763 famous_s3ji_799_200_006672024_1 992,186 387,017 0.3901
11 Dec 2010 12:11:19 483262 12087763 famous_s3ji_799_200_006672024_1 982,826 383,448 0.3901
11 Dec 2010 11:18:22 483262 12087763 famous_s3ji_799_200_006672024_1 973,466 379,889 0.3902
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 964,106 376,342 0.3904
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 954,746 372,776 0.3904
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 945,386 369,206 0.3905
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 936,026 365,649 0.3906
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 926,666 362,080 0.3907
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 917,306 358,504 0.3908
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 907,946 354,944 0.3909
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 898,586 351,372 0.3910
11 Dec 2010 10:56:58 483262 12087763 famous_s3ji_799_200_006672024_1 889,226 347,815 0.3911


©2024 climateprediction.net