climateprediction.net home page
Task 15428762

Task 15428762

Name hadcm3n_zgow_1880_40_008242572_3
Workunit 8397696
Created 6 Nov 2012, 17:37:32 UTC
Sent 6 Nov 2012, 17:37:37 UTC
Report deadline 6 Feb 2013, 1:04:48 UTC
Received 5 Feb 2013, 20:20:40 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1180505
Run time 16 days 12 hours 22 min 17 sec
CPU time 13 days 5 hours 10 min 17 sec
Validate state Invalid
Credit 9,953.28
Device peak FLOPS 2.91 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr
<core_client_version>7.0.31</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
11:40:09 (301): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:41:52 (288): No heartbeat from core client for 30 sec - exiting
16:42:11 (288): No heartbeat from core client for 30 sec - exiting
16:42:12 (288): No heartbeat from core client for 30 sec - exiting
16:42:13 (288): No heartbeat from core client for 30 sec - exiting
16:42:14 (288): No heartbeat from core client for 30 sec - exiting
16:42:15 (288): No heartbeat from core client for 30 sec - exiting
16:42:16 (288): No heartbeat from core client for 30 sec - exiting
16:42:17 (288): No heartbeat from core client for 30 sec - exiting
16:42:18 (288): No heartbeat from core client for 30 sec - exiting
16:42:19 (288): No heartbeat from core client for 30 sec - exiting
16:42:20 (288): No heartbeat from core client for 30 sec - exiting
16:42:21 (288): No heartbeat from core client for 30 sec - exiting
16:42:22 (288): No heartbeat from core client for 30 sec - exiting
16:42:23 (288): No heartbeat from core client for 30 sec - exiting
16:42:24 (288): No heartbeat from core client for 30 sec - exiting
16:42:25 (288): No heartbeat from core client for 30 sec - exiting
16:42:27 (288): No heartbeat from core client for 30 sec - exiting
16:42:28 (288): No heartbeat from core client for 30 sec - exiting
16:42:30 (288): No heartbeat from core client for 30 sec - exiting
16:42:31 (288): No heartbeat from core client for 30 sec - exiting
16:42:32 (288): No heartbeat from core client for 30 sec - exiting
16:42:33 (288): No heartbeat from core client for 30 sec - exiting
16:42:34 (288): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
12:00:17 (326): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:00:18 (326): No heartbeat from core client for 30 sec - exiting
12:00:19 (326): No heartbeat from core client for 30 sec - exiting
12:00:20 (326): No heartbeat from core client for 30 sec - exiting
12:00:21 (326): No heartbeat from core client for 30 sec - exiting
12:00:22 (326): No heartbeat from core client for 30 sec - exiting
12:00:23 (326): No heartbeat from core client for 30 sec - exiting
12:00:24 (326): No heartbeat from core client for 30 sec - exiting
12:00:25 (326): No heartbeat from core client for 30 sec - exiting
12:00:26 (326): No heartbeat from core client for 30 sec - exiting
12:00:27 (326): No heartbeat from core client for 30 sec - exiting
12:00:28 (326): No heartbeat from core client for 30 sec - exiting
12:00:29 (326): No heartbeat from core client for 30 sec - exiting
12:00:31 (326): No heartbeat from core client for 30 sec - exiting
12:00:32 (326): No heartbeat from core client for 30 sec - exiting
12:00:33 (326): No heartbeat from core client for 30 sec - exiting
12:00:34 (326): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
21:40:53 (339): No heartbeat from core client for 30 sec - exiting
21:40:54 (339): No heartbeat from core client for 30 sec - exiting
21:40:55 (339): No heartbeat from core client for 30 sec - exiting
21:40:56 (339): No heartbeat from core client for 30 sec - exiting
21:40:57 (339): No heartbeat from core client for 30 sec - exiting
21:40:58 (339): No heartbeat from core client for 30 sec - exiting
21:40:59 (339): No heartbeat from core client for 30 sec - exiting
21:41:00 (339): No heartbeat from core client for 30 sec - exiting
21:41:01 (339): No heartbeat from core client for 30 sec - exiting
21:41:02 (339): No heartbeat from core client for 30 sec - exiting
21:41:03 (339): No heartbeat from core client for 30 sec - exiting
21:41:04 (339): No heartbeat from core client for 30 sec - exiting
21:41:05 (339): No heartbeat from core client for 30 sec - exiting
21:41:06 (339): No heartbeat from core client for 30 sec - exiting
21:41:07 (339): No heartbeat from core client for 30 sec - exiting
21:41:08 (339): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
11:23:58 (391): No heartbeat from core client for 30 sec - exiting
11:23:59 (391): No heartbeat from core client for 30 sec - exiting
11:24:00 (391): No heartbeat from core client for 30 sec - exiting
11:24:01 (391): No heartbeat from core client for 30 sec - exiting
11:24:02 (391): No heartbeat from core client for 30 sec - exiting
11:24:03 (391): No heartbeat from core client for 30 sec - exiting
11:24:04 (391): No heartbeat from core client for 30 sec - exiting
11:24:05 (391): No heartbeat from core client for 30 sec - exiting
11:24:06 (391): No heartbeat from core client for 30 sec - exiting
11:24:07 (391): No heartbeat from core client for 30 sec - exiting
11:24:08 (391): No heartbeat from core client for 30 sec - exiting
11:24:09 (391): No heartbeat from core client for 30 sec - exiting
11:24:10 (391): No heartbeat from core client for 30 sec - exiting
11:24:11 (391): No heartbeat from core client for 30 sec - exiting
11:24:12 (391): No heartbeat from core client for 30 sec - exiting
11:24:13 (391): No heartbeat from core client for 30 sec - exiting
11:24:14 (391): No heartbeat from core client for 30 sec - exiting
11:24:15 (391): No heartbeat from core client for 30 sec - exiting
11:24:16 (391): No heartbeat from core client for 30 sec - exiting
11:24:17 (391): No heartbeat from core client for 30 sec - exiting
11:24:18 (391): No heartbeat from core client for 30 sec - exiting
11:24:19 (391): No heartbeat from core client for 30 sec - exiting
11:24:20 (391): No heartbeat from core client for 30 sec - exiting
11:24:21 (391): No heartbeat from core client for 30 sec - exiting
11:24:22 (391): No heartbeat from core client for 30 sec - exiting
11:24:23 (391): No heartbeat from core client for 30 sec - exiting
11:24:24 (391): No heartbeat from core client for 30 sec - exiting
11:24:25 (391): No heartbeat from core client for 30 sec - exiting
11:24:26 (391): No heartbeat from core client for 30 sec - exiting
11:24:27 (391): No heartbeat from core client for 30 sec - exiting
11:24:28 (391): No heartbeat from core client for 30 sec - exiting
11:24:29 (391): No heartbeat from core client for 30 sec - exiting
11:24:30 (391): No heartbeat from core client for 30 sec - exiting
11:24:31 (391): No heartbeat from core client for 30 sec - exiting
11:24:32 (391): No heartbeat from core client for 30 sec - exiting
11:24:34 (391): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/zgowko.pj94c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pj94c10
Error: Input file: dataout/zgowko.pi94c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pi94c10
Error: Input file: dataout/zgowko.pf94c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pf94c10
Error: Input file: dataout/zgowka.ph94c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.ph94c10
Error: Input file: dataout/zgowka.pg94c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pg94c10
Error: Input file: dataout/zgowka.pe94c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pe94c10
Error: Input file: dataout/zgowka.pd94c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pd94c10
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
23:24:34 (350): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
13:54:27 (307): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
15:26:27 (301): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:26:35 (301): No heartbeat from core client for 30 sec - exiting
15:26:36 (301): No heartbeat from core client for 30 sec - exiting
15:27:58 (4138): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:21:44 (318): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:22:07 (318): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
13:56:07 (295): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:56:09 (295): No heartbeat from core client for 30 sec - exiting
13:56:10 (295): No heartbeat from core client for 30 sec - exiting
13:56:11 (295): No heartbeat from core client for 30 sec - exiting
13:56:12 (295): No heartbeat from core client for 30 sec - exiting
13:56:13 (295): No heartbeat from core client for 30 sec - exiting
13:56:14 (295): No heartbeat from core client for 30 sec - exiting
13:56:37 (295): No heartbeat from core client for 30 sec - exiting
13:56:38 (295): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/zgowko.pja1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pja1c10
Error: Input file: dataout/zgowko.pia1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pia1c10
Error: Input file: dataout/zgowko.pfa1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pfa1c10
Error: Input file: dataout/zgowka.pha1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pha1c10
Error: Input file: dataout/zgowka.pga1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pga1c10
Error: Input file: dataout/zgowka.pea1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pea1c10
Error: Input file: dataout/zgowka.pda1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pda1c10
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/zgowko.pja1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pja1c10
Error: Input file: dataout/zgowko.pia1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pia1c10
Error: Input file: dataout/zgowko.pfa1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowko.pfa1c10
Error: Input file: dataout/zgowka.pha1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pha1c10
Error: Input file: dataout/zgowka.pga1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pga1c10
Error: Input file: dataout/zgowka.pea1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pea1c10
Error: Input file: dataout/zgowka.pda1c10 is not a valid UM file.
Error converting file to netcdf: dataout/zgowka.pda1c10
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:35:02 (393): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
19:54:19 (3194): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
05 Feb 2013 18:21:52 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 829,440 1,147,839 1.3839
05 Feb 2013 06:16:24 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 803,520 1,112,129 1.3841
31 Jan 2013 15:10:37 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 777,600 1,076,244 1.3841
31 Jan 2013 02:14:33 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 751,680 1,040,458 1.3842
30 Jan 2013 13:00:12 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 725,760 1,004,370 1.3839
30 Jan 2013 00:42:06 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 699,840 969,057 1.3847
28 Jan 2013 13:25:15 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 673,920 933,368 1.3850
27 Jan 2013 19:40:03 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 648,000 897,480 1.3850
27 Jan 2013 02:09:44 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 622,080 861,379 1.3847
26 Jan 2013 10:25:09 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 596,160 825,930 1.3854
25 Jan 2013 16:35:49 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 570,240 790,228 1.3858
22 Jan 2013 20:49:59 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 544,320 754,149 1.3855
19 Jan 2013 08:14:59 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 518,400 718,854 1.3867
18 Jan 2013 19:40:28 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 492,480 683,054 1.3870
17 Jan 2013 15:01:49 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 466,560 646,425 1.3855
05 Dec 2012 17:32:22 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 440,640 610,679 1.3859
04 Dec 2012 16:43:59 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 414,720 574,998 1.3865
03 Dec 2012 15:20:49 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 388,800 538,112 1.3840
03 Dec 2012 00:27:44 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 362,880 501,838 1.3829
29 Nov 2012 07:02:08 1180505 15428762 hadcm3n_zgow_1880_40_008242572_3 336,960 467,183 1.3865


©2024 cpdn.org