Name | hadcm3n_xa09_198012_480_010218839_0 |
Workunit | 10218839 |
Created | 4 Dec 2015, 14:04:36 UTC |
Sent | 4 Dec 2015, 14:11:44 UTC |
Report deadline | 15 Nov 2016, 19:31:44 UTC |
Received | 25 Dec 2015, 6:57:00 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1378735 |
Run time | 20 days 3 hours 54 min 13 sec |
CPU time | 19 days 7 hours 12 min 15 sec |
Validate state | Invalid |
Credit | 11,819.52 |
Device peak FLOPS | 2.19 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
Stderr | <core_client_version>7.0.27</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:08:54 (1154): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:21:34 (5865): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:24:00 (6461): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:25:52 (6481): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:27:49 (6501): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:40 (6521): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:11 (6541): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:12 (6541): No heartbeat from core client for 30 sec - exiting 19:32:14 (6541): No heartbeat from core client for 30 sec - exiting 19:32:15 (6541): No heartbeat from core client for 30 sec - exiting 19:34:28 (6561): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:10 (6581): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:16 (6601): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:41:07 (6621): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:43:05 (6641): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:30 (6661): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:31 (6661): No heartbeat from core client for 30 sec - exiting 19:45:32 (6661): No heartbeat from core client for 30 sec - exiting 19:45:34 (6661): No heartbeat from core client for 30 sec - exiting 19:45:35 (6661): No heartbeat from core client for 30 sec - exiting 19:47:42 (6681): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:49:48 (6701): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:59 (6721): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:00 (6741): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:37 (6761): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:58:43 (6781): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:01:14 (6801): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:01:15 (6801): No heartbeat from core client for 30 sec - exiting 20:01:16 (6801): No heartbeat from core client for 30 sec - exiting 20:01:17 (6801): No heartbeat from core client for 30 sec - exiting 20:03:55 (6821): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:05:51 (6841): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:08:27 (6861): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:10:33 (6881): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:12:45 (6901): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:15:06 (6921): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:16:57 (6941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:18:54 (6964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:20:45 (6984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:23:16 (7004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:23:17 (7004): No heartbeat from core client for 30 sec - exiting 20:23:18 (7004): No heartbeat from core client for 30 sec - exiting 20:23:19 (7004): No heartbeat from core client for 30 sec - exiting 20:25:27 (7024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:03 (7044): No heartbeat from core client for 30 sec - exiting 20:28:04 (7044): No heartbeat from core client for 30 sec - exiting 20:28:05 (7044): No heartbeat from core client for 30 sec - exiting 20:28:06 (7044): No heartbeat from core client for 30 sec - exiting 20:28:07 (7044): No heartbeat from core client for 30 sec - exiting 20:28:08 (7044): No heartbeat from core client for 30 sec - exiting 20:28:09 (7044): No heartbeat from core client for 30 sec - exiting 20:28:10 (7044): No heartbeat from core client for 30 sec - exiting 20:28:11 (7044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/xa09ko.pjl8c10 is not a valid UM file. Error converting file to netcdf: dataout/xa09ko.pjl8c10 Error: Input file: dataout/xa09ko.pil8c10 is not a valid UM file. Error converting file to netcdf: dataout/xa09ko.pil8c10 Error: Input file: dataout/xa09ko.pfl8c10 is not a valid UM file. Error converting file to netcdf: dataout/xa09ko.pfl8c10 Error: Input file: dataout/xa09ka.phl8c10 is not a valid UM file. Error converting file to netcdf: dataout/xa09ka.phl8c10 Error: Input file: dataout/xa09ka.pgl8c10 is not a valid UM file. Error converting file to netcdf: dataout/xa09ka.pgl8c10 Error: Input file: dataout/xa09ka.pel8c10 is not a valid UM file. Error converting file to netcdf: dataout/xa09ka.pel8c10 Error: Input file: dataout/xa09ka.pdl8c10 is not a valid UM file. Error converting file to netcdf: dataout/xa09ka.pdl8c10 20:30:20 (7064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 20:32:10 (7084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 20:33:56 (7104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 20:36:22 (7124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:38:23 (7144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:40:59 (7164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:43:01 (7184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:02 (7204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:47:03 (7224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:49:14 (7244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:51:30 (7264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:54:11 (7284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:56:47 (7304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:59:23 (7324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:01:34 (7344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:32 (7364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:06:03 (7384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:35 (7404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:11:16 (7424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:42 (7444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:15:29 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:18:05 (7484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:19:56 (7507): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:22:07 (7527): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:23 (7547): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:26:24 (7567): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:28:46 (7587): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:28:47 (7587): No heartbeat from core client for 30 sec - exiting 21:31:12 (7607): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:31:13 (7607): No heartbeat from core client for 30 sec - exiting 21:31:14 (7607): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb773e400] [0xb773e424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb756d1df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb7570825] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75584d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7627, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb77a1400] [0xb77a1424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75d01df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb75d3825] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75bb4d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7627, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb76ff400] [0xb76ff424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb752e1df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb7531825] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75194d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7627, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb77b1400] [0xb77b1424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75e01df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb75e3825] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75cb4d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7627, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb77cf400] [0xb77cf424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75fe1df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb7601825] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75e94d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7627, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb7721400] [0xb7721424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75501df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb7553825] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb753b4d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7627, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
25 Dec 2015 06:56:57 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 984,960 | 1,667,552 | 1.6930 |
24 Dec 2015 06:19:49 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 959,040 | 1,623,836 | 1.6932 |
23 Dec 2015 17:17:19 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 933,120 | 1,578,953 | 1.6921 |
23 Dec 2015 04:12:53 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 907,200 | 1,533,983 | 1.6909 |
22 Dec 2015 15:09:51 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 881,280 | 1,489,046 | 1.6896 |
22 Dec 2015 02:09:08 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 855,360 | 1,444,147 | 1.6883 |
21 Dec 2015 13:09:23 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 829,440 | 1,399,345 | 1.6871 |
21 Dec 2015 00:06:05 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 803,520 | 1,354,579 | 1.6858 |
20 Dec 2015 11:18:44 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 777,600 | 1,310,359 | 1.6851 |
19 Dec 2015 22:24:36 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 751,680 | 1,266,208 | 1.6845 |
19 Dec 2015 09:42:04 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 725,760 | 1,222,202 | 1.6840 |
18 Dec 2015 20:55:14 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 699,840 | 1,178,333 | 1.6837 |
18 Dec 2015 08:17:31 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 673,920 | 1,134,469 | 1.6834 |
17 Dec 2015 19:32:37 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 648,000 | 1,090,703 | 1.6832 |
17 Dec 2015 06:49:08 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 622,080 | 1,046,992 | 1.6831 |
16 Dec 2015 17:41:49 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 596,160 | 1,003,290 | 1.6829 |
16 Dec 2015 05:27:31 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 570,240 | 959,571 | 1.6827 |
15 Dec 2015 16:10:50 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 544,320 | 915,762 | 1.6824 |
15 Dec 2015 04:00:32 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 518,400 | 872,060 | 1.6822 |
14 Dec 2015 14:58:01 | 1378735 | 19129798 | hadcm3n_xa09_198012_480_010218839_0 | 492,480 | 829,491 | 1.6843 |
©2024 cpdn.org