Name | famous_wrq6_1999_200_007122693_0 |
Workunit | 7321053 |
Created | 16 Jan 2011, 16:48:54 UTC |
Sent | 16 Jan 2011, 16:51:18 UTC |
Report deadline | 18 Apr 2011, 0:18:29 UTC |
Received | 15 Mar 2011, 14:08:42 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1121672 |
Run time | 6 days 23 hours 54 min 19 sec |
CPU time | 6 days 19 hours 33 min 40 sec |
Validate state | Invalid |
Credit | 4,014.70 |
Device peak FLOPS | 2.33 GFLOPS |
Application version | UK Met Office FAMOUS v6.11 windows_intelx86 |
Stderr | <core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... forrtl: The requested operation cannot be performed on a file with a user-mapped section open. forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_wrq6_1999_200_007122693\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown famous_um_6.11_wi 0088196C Unknown Unknown Unknown famous_um_6.11_wi 0080BD3E Unknown Unknown Unknown famous_um_6.11_wi 0080B95B Unknown Unknown Unknown famous_um_6.11_wi 007F0945 Unknown Unknown Unknown famous_um_6.11_wi 007F00F5 Unknown Unknown Unknown famous_um_6.11_wi 007BA86B Unknown Unknown Unknown famous_um_6.11_wi 007BA0FF Unknown Unknown Unknown famous_um_6.11_wi 007BA0A0 Unknown Unknown Unknown famous_um_6.11_wi 00776C15 Unknown Unknown Unknown famous_um_6.11_wi 007CA0F6 Unknown Unknown Unknown kernel32.dll 7C817077 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=216, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... forrtl: The requested operation cannot be performed on a file with a user-mapped section open. forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_wrq6_1999_200_007122693\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown famous_um_6.11_wi 0088196C Unknown Unknown Unknown famous_um_6.11_wi 0080BD3E Unknown Unknown Unknown famous_um_6.11_wi 0080B95B Unknown Unknown Unknown famous_um_6.11_wi 007F0945 Unknown Unknown Unknown famous_um_6.11_wi 007F00F5 Unknown Unknown Unknown famous_um_6.11_wi 007BA86B Unknown Unknown Unknown famous_um_6.11_wi 007BA0FF Unknown Unknown Unknown famous_um_6.11_wi 007BA0A0 Unknown Unknown Unknown famous_um_6.11_wi 00776C15 Unknown Unknown Unknown famous_um_6.11_wi 007CA0F6 Unknown Unknown Unknown kernel32.dll 7C817077 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4216, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: The requested operation cannot be performed on a file with a user-mapped section open. forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_wrq6_1999_200_007122693\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown famous_um_6.11_wi 0088196C Unknown Unknown Unknown famous_um_6.11_wi 0080BD3E Unknown Unknown Unknown famous_um_6.11_wi 0080B95B Unknown Unknown Unknown famous_um_6.11_wi 007F0945 Unknown Unknown Unknown famous_um_6.11_wi 007F00F5 Unknown Unknown Unknown famous_um_6.11_wi 007BA86B Unknown Unknown Unknown famous_um_6.11_wi 007BA0FF Unknown Unknown Unknown famous_um_6.11_wi 007BA0A0 Unknown Unknown Unknown famous_um_6.11_wi 00776C15 Unknown Unknown Unknown famous_um_6.11_wi 007CA0F6 Unknown Unknown Unknown kernel32.dll 7C817077 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5276, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... forrtl: The requested operation cannot be performed on a file with a user-mapped section open. forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_wrq6_1999_200_007122693\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown famous_um_6.11_wi 0088196C Unknown Unknown Unknown famous_um_6.11_wi 0080BD3E Unknown Unknown Unknown famous_um_6.11_wi 0080B95B Unknown Unknown Unknown famous_um_6.11_wi 007F0945 Unknown Unknown Unknown famous_um_6.11_wi 007F00F5 Unknown Unknown Unknown famous_um_6.11_wi 007BA86B Unknown Unknown Unknown famous_um_6.11_wi 007BA0FF Unknown Unknown Unknown famous_um_6.11_wi 007BA0A0 Unknown Unknown Unknown famous_um_6.11_wi 00776C15 Unknown Unknown Unknown famous_um_6.11_wi 007CA0F6 Unknown Unknown Unknown kernel32.dll 7C817077 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: The requested operation cannot be performed on a file with a user-mapped section open. forrtl: severe (38): error during write, unit 6, file C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\famous_wrq6_1999_200_007122693\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown famous_um_6.11_wi 0088196C Unknown Unknown Unknown famous_um_6.11_wi 0080BD3E Unknown Unknown Unknown famous_um_6.11_wi 0080B95B Unknown Unknown Unknown famous_um_6.11_wi 007F0945 Unknown Unknown Unknown famous_um_6.11_wi 007F00F5 Unknown Unknown Unknown famous_um_6.11_wi 007BA86B Unknown Unknown Unknown famous_um_6.11_wi 007BA0FF Unknown Unknown Unknown famous_um_6.11_wi 007BA0A0 Unknown Unknown Unknown famous_um_6.11_wi 00776C15 Unknown Unknown Unknown famous_um_6.11_wi 007CA0F6 Unknown Unknown Unknown kernel32.dll 7C817077 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5672, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/famous_wrq6_1999_200_007122693/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/famous_wrq6_1999_200_007122693/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/famous_wrq6_1999_200_007122693/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/famous_wrq6_1999_200_007122693/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/famous_wrq6_1999_200_007122693/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/famous_wrq6_1999_200_007122693/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy Sorry, too many model crashes! :-( 10:54:33 (3740): called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
15 Mar 2011 13:16:50 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,216,826 | 586,626 | 0.4821 |
14 Mar 2011 19:35:48 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,207,466 | 582,201 | 0.4822 |
14 Mar 2011 18:21:26 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,198,106 | 577,923 | 0.4824 |
14 Mar 2011 17:12:30 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,188,746 | 573,642 | 0.4826 |
14 Mar 2011 16:01:47 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,179,386 | 569,370 | 0.4828 |
14 Mar 2011 14:46:31 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,170,026 | 565,095 | 0.4830 |
14 Mar 2011 13:22:14 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,160,666 | 560,680 | 0.4831 |
11 Mar 2011 20:53:45 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,151,306 | 556,182 | 0.4831 |
11 Mar 2011 19:48:26 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,141,946 | 551,817 | 0.4832 |
11 Mar 2011 19:41:57 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,132,586 | 547,139 | 0.4831 |
11 Mar 2011 16:53:52 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,123,226 | 542,854 | 0.4833 |
11 Mar 2011 15:35:32 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,113,866 | 538,492 | 0.4834 |
11 Mar 2011 14:09:16 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,104,506 | 533,547 | 0.4831 |
11 Mar 2011 12:47:53 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,095,146 | 528,821 | 0.4829 |
10 Mar 2011 20:52:33 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,085,786 | 524,203 | 0.4828 |
10 Mar 2011 19:26:07 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,076,426 | 519,674 | 0.4828 |
10 Mar 2011 18:09:37 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,067,066 | 515,178 | 0.4828 |
10 Mar 2011 16:52:40 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,057,706 | 510,685 | 0.4828 |
10 Mar 2011 15:37:05 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,048,346 | 506,204 | 0.4829 |
10 Mar 2011 14:21:09 | 1121672 | 12497352 | famous_wrq6_1999_200_007122693_0 | 1,038,986 | 501,718 | 0.4829 |
©2024 climateprediction.net