Name | famous_wn7q_1199_200_007117197_0 |
Workunit | 7315557 |
Created | 16 Jan 2011, 15:21:28 UTC |
Sent | 19 Jan 2011, 0:44:00 UTC |
Report deadline | 20 Apr 2011, 8:11:11 UTC |
Received | 27 Jan 2011, 11:39:08 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1126407 |
Run time | 3 days 20 hours 3 min 33 sec |
CPU time | 3 days 15 hours 57 min 45 sec |
Validate state | Invalid |
Credit | 1,914.75 |
Device peak FLOPS | 2.68 GFLOPS |
Application version | UK Met Office FAMOUS v6.11 windows_intelx86 |
Stderr | <core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5824, selfPID=5824, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5824, selfPID=5824, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6140, selfPID=6140, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5716, selfPID=5716, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5872, selfPID=5872, iMonCtr=1 19:29:13 (4332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:14 (4332): No heartbeat from core client for 30 sec - exiting 19:29:15 (4332): No heartbeat from core client for 30 sec - exiting 19:29:16 (4332): No heartbeat from core client for 30 sec - exiting 19:29:17 (4332): No heartbeat from core client for 30 sec - exiting 19:29:18 (4332): No heartbeat from core client for 30 sec - exiting 19:29:19 (4332): No heartbeat from core client for 30 sec - exiting 19:29:20 (4332): No heartbeat from core client for 30 sec - exiting 19:29:21 (4332): No heartbeat from core client for 30 sec - exiting 19:29:22 (4332): No heartbeat from core client for 30 sec - exiting 19:29:23 (4332): No heartbeat from core client for 30 sec - exiting 19:29:24 (4332): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4528, selfPID=4528, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6344, selfPID=6344, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7360, selfPID=7360, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6220, selfPID=6220, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2644, selfPID=2644, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7864, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7864, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7864, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7864, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8892, selfPID=8892, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1720, selfPID=1720, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7400, selfPID=7400, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4048, selfPID=4048, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2392, selfPID=2392, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2980, selfPID=2980, iMonCtr=1 06:49:00 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:49:01 (4420): No heartbeat from core client for 30 sec - exiting 06:49:02 (4420): No heartbeat from core client for 30 sec - exiting 06:49:03 (4420): No heartbeat from core client for 30 sec - exiting 06:49:04 (4420): No heartbeat from core client for 30 sec - exiting No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4808, selfPID=4808, iMonCtr=1 06:59:21 (5460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6812, selfPID=6812, iMonCtr=1 07:07:53 (8688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:07:54 (8688): No heartbeat from core client for 30 sec - exiting 07:07:55 (8688): No heartbeat from core client for 30 sec - exiting 07:07:56 (8688): No heartbeat from core client for 30 sec - exiting 07:07:57 (8688): No heartbeat from core client for 30 sec - exiting 07:07:58 (8688): No heartbeat from core client for 30 sec - exiting 07:07:59 (8688): No heartbeat from core client for 30 sec - exiting 07:08:00 (8688): No heartbeat from core client for 30 sec - exiting 07:08:01 (8688): No heartbeat from core client for 30 sec - exiting 07:08:02 (8688): No heartbeat from core client for 30 sec - exiting 07:08:03 (8688): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8804, selfPID=8804, iMonCtr=1 07:15:53 (1696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:54 (1696): No heartbeat from core client for 30 sec - exiting 07:15:55 (1696): No heartbeat from core client for 30 sec - exiting 07:15:56 (1696): No heartbeat from core client for 30 sec - exiting 07:15:57 (1696): No heartbeat from core client for 30 sec - exiting 07:15:58 (1696): No heartbeat from core client for 30 sec - exiting 07:15:59 (1696): No heartbeat from core client for 30 sec - exiting 07:16:00 (1696): No heartbeat from core client for 30 sec - exiting 07:16:01 (1696): No heartbeat from core client for 30 sec - exiting 07:16:02 (1696): No heartbeat from core client for 30 sec - exiting 07:16:03 (1696): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3368, selfPID=3368, iMonCtr=1 07:29:53 (7076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:29:54 (7076): No heartbeat from core client for 30 sec - exiting 07:29:55 (7076): No heartbeat from core client for 30 sec - exiting 07:29:56 (7076): No heartbeat from core client for 30 sec - exiting 07:29:57 (7076): No heartbeat from core client for 30 sec - exiting 07:29:58 (7076): No heartbeat from core client for 30 sec - exiting 07:29:59 (7076): No heartbeat from core client for 30 sec - exiting 07:30:00 (7076): No heartbeat from core client for 30 sec - exiting 07:30:01 (7076): No heartbeat from core client for 30 sec - exiting 07:30:02 (7076): No heartbeat from core client for 30 sec - exiting 07:30:03 (7076): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6560, selfPID=6560, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3852, selfPID=3852, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5572, selfPID=5572, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2472, selfPID=2472, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1672, selfPID=1672, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3176, selfPID=3176, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3600, selfPID=3600, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2836, selfPID=2836, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6468, selfPID=6468, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5152, selfPID=5152, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5344, selfPID=5344, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5320, selfPID=5320, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1160, selfPID=1160, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4456, selfPID=4456, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6412, selfPID=6412, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1184, selfPID=1184, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2136, selfPID=2136, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6632, selfPID=6632, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=704, selfPID=704, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2156, selfPID=2156, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5712, selfPID=5712, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5872, selfPID=5872, iMonCtr=1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6276, selfPID=6276, iMonCtr=1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5744, selfPID=5744, iMonCtr=1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2380, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2380, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2380, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2380, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2380, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_wn7q_1199_200_007117197\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2380, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5556, selfPID=5556, iMonCtr=1 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/jobs/afyel.ihist after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/famous_wn7q_1199_200_007117197/dataout/ocean_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Sorry, too many model crashes! :-( 11:26:07 (5160): called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
27 Jan 2011 11:45:05 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 580,346 | 316,755 | 0.5458 |
27 Jan 2011 11:45:04 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 570,986 | 311,510 | 0.5456 |
27 Jan 2011 00:11:16 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 561,626 | 306,241 | 0.5453 |
26 Jan 2011 22:37:00 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 552,266 | 300,919 | 0.5449 |
26 Jan 2011 21:02:26 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 542,906 | 295,671 | 0.5446 |
26 Jan 2011 19:26:50 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 533,546 | 290,431 | 0.5443 |
26 Jan 2011 18:12:48 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 524,186 | 285,207 | 0.5441 |
26 Jan 2011 16:19:49 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 514,826 | 279,993 | 0.5439 |
26 Jan 2011 14:46:00 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 505,466 | 274,739 | 0.5435 |
26 Jan 2011 13:14:06 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 496,106 | 269,521 | 0.5433 |
26 Jan 2011 11:40:59 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 486,746 | 264,312 | 0.5430 |
26 Jan 2011 10:12:31 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 477,386 | 259,094 | 0.5427 |
26 Jan 2011 08:38:36 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 468,026 | 253,824 | 0.5423 |
26 Jan 2011 07:01:11 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 458,666 | 248,582 | 0.5420 |
26 Jan 2011 05:27:41 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 449,306 | 243,588 | 0.5421 |
26 Jan 2011 03:55:06 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 439,946 | 238,393 | 0.5419 |
26 Jan 2011 02:22:11 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 430,586 | 233,129 | 0.5414 |
26 Jan 2011 00:43:25 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 421,226 | 227,963 | 0.5412 |
25 Jan 2011 23:09:24 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 411,866 | 222,701 | 0.5407 |
25 Jan 2011 21:39:08 | 1126407 | 12491682 | famous_wn7q_1199_200_007117197_0 | 402,506 | 217,429 | 0.5402 |
©2024 climateprediction.net