climateprediction.net home page
Task 21381087

Task 21381087

Name wah2_sam25_a01v_200012_73_762_011651539_2
Workunit 11651539
Created 13 Nov 2018, 17:47:58 UTC
Sent 18 Nov 2018, 15:52:25 UTC
Report deadline 31 Oct 2019, 21:12:25 UTC
Received 18 Dec 2018, 3:27:00 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1400747
Run time 23 days 4 hours 35 min 23 sec
CPU time 22 days 4 hours 11 min 57 sec
Validate state Invalid
Credit 12,178.71
Device peak FLOPS 2.42 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 413.44 MB
Peak swap size 381.26 MB
Peak disk usage 107.11 MB
Stderr 
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk0dec
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1feb
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:50:51 (10244): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1mar
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1apr
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1may
11:40:11 (9668): start_timer_thread(): CreateThread() failed, errno 0
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1jun
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1jul
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1aug
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1sep
CPDN Monitor - Quit request from BOINC...
09:17:09 (2684): start_timer_thread(): CreateThread() failed, errno 0
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1oct
CPDN Monitor - Quit request from BOINC...
11:42:10 (7812): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
06:02:24 (1104): start_timer_thread(): CreateThread() failed, errno 0
06:02:26 (11856): start_timer_thread(): CreateThread() failed, errno 0
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1nov
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk1dec
18:02:07 (3440): start_timer_thread(): CreateThread() failed, errno 0
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk2feb
CPDN Monitor - Quit request from BOINC...
05:42:16 (9304): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
15:03:51 (7884): start_timer_thread(): CreateThread() failed, errno 0
Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_sam25_a01v_200012_73_762_011651539/dataout\a01vga.pdk2mar
CPDN Monitor - Quit request from BOINC...
14:26:24 (10348): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
19:26:04 (5132): start_timer_thread(): CreateThread() failed, errno 0
19:26:06 (9956): start_timer_thread(): CreateThread() failed, errno 0
Signal 11 received: Segment violation
Signal 11 received: Software termination signal from kill 
Signal 11 received: Abnormal termination triggered by abort call
Signal 11 received, exiting...
19:32:29 (9956): called boinc_finish(193)
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5132, iMonCtr=2
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9956, selfPID=9056, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_ain::Monitor...
19:32:34 (9056): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_17.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_18.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_19.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_20.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_21.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_22.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_23.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_24.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_25.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_26.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_27.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_28.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_29.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_30.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_31.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_32.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_33.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_34.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_35.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_36.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_37.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_38.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_39.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_40.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_41.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_42.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_43.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_44.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_45.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_46.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_47.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_48.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_49.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_50.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_51.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_52.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_53.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_54.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_55.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_56.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_57.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_58.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_59.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_60.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_61.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_62.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_63.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_64.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_65.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_66.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_67.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_68.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_69.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_70.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_71.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_72.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_sam25_a01v_200012_73_762_011651539_2_r537040404_73.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Dec 2018 02:45:04 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 184,619 1,903,893 10.3126
13 Dec 2018 06:01:26 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 173,099 1,787,196 10.3247
11 Dec 2018 12:03:47 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 161,579 1,665,292 10.3064
09 Dec 2018 22:46:22 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 150,059 1,546,049 10.3029
08 Dec 2018 13:57:27 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 138,539 1,428,324 10.3099
05 Dec 2018 10:50:10 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 127,019 1,309,346 10.3083
03 Dec 2018 12:52:10 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 115,499 1,189,289 10.2970
02 Dec 2018 02:45:15 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 103,979 1,070,045 10.2910
30 Nov 2018 16:23:30 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 92,459 951,921 10.2956
29 Nov 2018 04:25:45 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 80,939 833,749 10.3010
27 Nov 2018 16:26:50 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 69,419 714,521 10.2929
26 Nov 2018 06:27:02 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 57,899 594,426 10.2666
24 Nov 2018 22:42:20 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 46,379 479,191 10.3321
23 Nov 2018 12:37:32 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 34,859 362,815 10.4081
22 Nov 2018 00:54:02 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 23,339 243,039 10.4134
20 Nov 2018 14:16:06 1400747 21381087 wah2_sam25_a01v_200012_73_762_011651539_2 11,819 122,911 10.3994


©2024 climateprediction.net