Name | hadcm3n_yk2j_1900_40_007358869_2 |
Workunit | 7556299 |
Created | 26 Jul 2011, 11:39:01 UTC |
Sent | 26 Jul 2011, 11:39:08 UTC |
Report deadline | 25 Oct 2011, 19:06:19 UTC |
Received | 11 Aug 2011, 2:08:44 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1244925 |
Run time | 13 days 14 hours 1 min 39 sec |
CPU time | 11 days 6 hours 21 min 6 sec |
Validate state | Invalid |
Credit | 9,331.20 |
Device peak FLOPS | 3.74 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin |
Stderr | <core_client_version>6.12.26</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 62 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/yk2jko.pjb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pjb0c10 Error: Input file: dataout/yk2jko.pib0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pib0c10 Error: Input file: dataout/yk2jko.pfb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pfb0c10 Error: Input file: dataout/yk2jko.pcb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pcb0c10 Error: Input file: dataout/yk2jko.pbb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pbb0c10 Error: Input file: dataout/yk2jko.pab0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pab0c10 Error: Input file: dataout/yk2jka.phb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.phb0c10 Error: Input file: dataout/yk2jka.pgb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.pgb0c10 Error: Input file: dataout/yk2jka.peb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.peb0c10 Error: Input file: dataout/yk2jka.pdb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.pdb0c10 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 62 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/yk2jko.pjb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pjb0c10 Error: Input file: dataout/yk2jko.pib0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pib0c10 Error: Input file: dataout/yk2jko.pfb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pfb0c10 Error: Input file: dataout/yk2jko.pcb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pcb0c10 Error: Input file: dataout/yk2jko.pbb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pbb0c10 Error: Input file: dataout/yk2jko.pab0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pab0c10 Error: Input file: dataout/yk2jka.phb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.phb0c10 Error: Input file: dataout/yk2jka.pgb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.pgb0c10 Error: Input file: dataout/yk2jka.peb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.peb0c10 Error: Input file: dataout/yk2jka.pdb0c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.pdb0c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/yk2jko.pjb8c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pjb8c10 Error: Input file: dataout/yk2jko.pib8c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pib8c10 Error: Input file: dataout/yk2jko.pfb8c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jko.pfb8c10 Error: Input file: dataout/yk2jka.phb8c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.phb8c10 Error: Input file: dataout/yk2jka.pgb8c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.pgb8c10 Error: Input file: dataout/yk2jka.peb8c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.peb8c10 Error: Input file: dataout/yk2jka.pdb8c10 is not a valid UM file. Error converting file to netcdf: dataout/yk2jka.pdb8c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(18017,0xa0564540) malloc: *** error for object 0x4020a04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18017,0xa0564540) malloc: *** error for object 0x301f604: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18017,0xa0564540) malloc: *** error for object 0x303ac04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18017,0xa0564540) malloc: *** error for object 0x303ac00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18017,0xa0564540) malloc: *** error for object 0x303ac04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18017,0xa0564540) malloc: *** error for object 0x303ac00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18017,0xa0564540) malloc: *** error for object 0x301f604: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18026, selfPID=18026, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x843404: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x282c604: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x843404: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x822604: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x822600: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(25322,0xa0564540) malloc: *** error for object 0x2806200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yk2j_1900_40_007358869/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
11 Aug 2011 01:10:26 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 777,600 | 973,287 | 1.2517 |
10 Aug 2011 14:00:28 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 751,680 | 940,596 | 1.2513 |
10 Aug 2011 03:30:58 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 725,760 | 907,789 | 1.2508 |
09 Aug 2011 18:10:58 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 699,840 | 875,084 | 1.2504 |
09 Aug 2011 05:59:38 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 673,920 | 841,653 | 1.2489 |
08 Aug 2011 08:01:56 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 648,000 | 807,214 | 1.2457 |
07 Aug 2011 21:17:01 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 622,080 | 774,551 | 1.2451 |
07 Aug 2011 10:30:14 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 596,160 | 742,079 | 1.2448 |
07 Aug 2011 00:39:08 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 570,240 | 709,445 | 1.2441 |
06 Aug 2011 12:16:20 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 544,320 | 676,759 | 1.2433 |
06 Aug 2011 01:44:24 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 518,400 | 644,217 | 1.2427 |
05 Aug 2011 14:22:35 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 492,480 | 612,556 | 1.2438 |
05 Aug 2011 01:52:07 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 466,560 | 579,955 | 1.2430 |
04 Aug 2011 08:09:01 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 440,640 | 547,503 | 1.2425 |
03 Aug 2011 16:25:01 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 414,720 | 514,988 | 1.2418 |
03 Aug 2011 03:25:50 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 388,800 | 483,964 | 1.2448 |
02 Aug 2011 10:24:30 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 362,880 | 452,102 | 1.2459 |
01 Aug 2011 19:00:48 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 336,960 | 419,880 | 1.2461 |
01 Aug 2011 05:04:03 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 311,040 | 387,466 | 1.2457 |
31 Jul 2011 16:29:21 | 998485 | 13159205 | hadcm3n_yk2j_1900_40_007358869_2 | 285,120 | 354,771 | 1.2443 |
©2024 cpdn.org