climateprediction.net home page
Task 15644282

Task 15644282

Name hadcm3n_48cd_1940_40_008308175_2
Workunit 8459310
Created 1 Mar 2013, 14:59:25 UTC
Sent 1 Mar 2013, 14:59:33 UTC
Report deadline 31 May 2013, 22:26:44 UTC
Received 15 Apr 2013, 5:58:24 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1019749
Run time 18 days 21 hours 3 min 11 sec
CPU time 18 days 12 hours 0 min 5 sec
Validate state Invalid
Credit 9,331.20
Device peak FLOPS 2.53 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.41</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 62 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/48cdko.pjf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.pjf0c10
Error: Input file: dataout/48cdko.pif0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.pif0c10
Error: Input file: dataout/48cdko.pff0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.pff0c10
Error: Input file: dataout/48cdko.pcf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.pcf0c10
Error: Input file: dataout/48cdko.pbf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.pbf0c10
Error: Input file: dataout/48cdko.paf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.paf0c10
Error: Input file: dataout/48cdka.phf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdka.phf0c10
Error: Input file: dataout/48cdka.pgf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdka.pgf0c10
Error: Input file: dataout/48cdka.pef0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdka.pef0c10
Error: Input file: dataout/48cdka.pdf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdka.pdf0c10
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/48cdko.pjg7c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.pjg7c10
Error: Input file: dataout/48cdko.pig7c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.pig7c10
Error: Input file: dataout/48cdko.pfg7c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdko.pfg7c10
Error: Input file: dataout/48cdka.phg7c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdka.phg7c10
Error: Input file: dataout/48cdka.pgg7c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdka.pgg7c10
Error: Input file: dataout/48cdka.peg7c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdka.peg7c10
Error: Input file: dataout/48cdka.pdg7c10 is not a valid UM file.
Error converting file to netcdf: dataout/48cdka.pdg7c10
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08190d00 ***
======= Backtrace: =========
/lib/libc.so.6(+0x71139)[0xf7507139]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xf77048b1]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xf770490d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/libc.so.6(__libc_start_main+0xe6)[0xf74acbb6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 fd:00 3531357                            /home/ryoji/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 fd:00 3531357                            /home/ryoji/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-081c2000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f701a000-f7495000 rw-s 00000000 fd:00 4554819                            /home/ryoji/BOINC/slots/5/133690
f7495000-f7496000 rw-p 00000000 00:00 0 
f7496000-f7604000 r-xp 00000000 fd:00 991330                             /lib/libc-2.11.1.so
f7604000-f7605000 ---p 0016e000 fd:00 991330                             /lib/libc-2.11.1.so
f7605000-f7607000 r--p 0016e000 fd:00 991330                             /lib/libc-2.11.1.so
f7607000-f7608000 rw-p 00170000 fd:00 991330                             /lib/libc-2.11.1.so
f7608000-f760b000 rw-p 00000000 00:00 0 
f760b000-f7627000 r-xp 00000000 fd:00 991323                             /lib/libgcc_s-4.4.5.so.1
f7627000-f7628000 rw-p 0001b000 fd:00 991323                             /lib/libgcc_s-4.4.5.so.1
f7628000-f764e000 r-xp 00000000 fd:00 991362                             /lib/libm-2.11.1.so
f764e000-f764f000 r--p 00025000 fd:00 991362                             /lib/libm-2.11.1.so
f764f000-f7650000 rw-p 00026000 fd:00 991362                             /lib/libm-2.11.1.so
f7650000-f7651000 rw-p 00000000 00:00 0 
f7651000-f7732000 r-xp 00000000 fd:00 3655714                            /usr/lib/libstdc++.so.6.0.13
f7732000-f7736000 r--p 000e0000 fd:00 3655714                            /usr/lib/libstdc++.so.6.0.13
f7736000-f7737000 rw-p 000e4000 fd:00 3655714                            /usr/lib/libstdc++.so.6.0.13
f7737000-f773e000 rw-p 00000000 00:00 0 
f773e000-f7741000 r-xp 00000000 fd:00 991360                             /lib/libdl-2.11.1.so
f7741000-f7742000 r--p 00002000 fd:00 991360                             /lib/libdl-2.11.1.so
f7742000-f7743000 rw-p 00003000 fd:00 991360                             /lib/libdl-2.11.1.so
f7743000-f7759000 r-xp 00000000 fd:00 991391                             /lib/libpthread-2.11.1.so
f7759000-f775a000 r--p 00015000 fd:00 991391                             /lib/libpthread-2.11.1.so
f775a000-f775b000 rw-p 00016000 fd:00 991391                             /lib/libpthread-2.11.1.so
f775b000-f775d000 rw-p 00000000 00:00 0 
f776c000-f776d000 rw-p 00000000 00:00 0 
f776d000-f776e000 ---p 00000000 00:00 0 
f776e000-f7771000 rw-p 00000000 00:00 0 
f7771000-f7773000 rw-s 00000000 fd:00 4554816                            /home/ryoji/BOINC/slots/5/boinc_mmap_file
f7773000-f7774000 rw-p 00000000 00:00 0 
f7774000-f7791000 r-xp 00000000 fd:00 991332                             /lib/ld-2.11.1.so
f7791000-f7792000 r--p 0001d000 fd:00 991332                             /lib/ld-2.11.1.so
f7792000-f7793000 rw-p 0001e000 fd:00 991332                             /lib/ld-2.11.1.so
ff81b000-ff88c000 rw-p 00000000 00:00 0                                  [stack]
ffffe000-fffff000 r-xp 00000000 00:00 0                                  [vdso]
SIGABRT: abort called
Stack trace (17 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xffffe400]
[0xffffe430]
/lib/libc.so.6(gsignal+0x51)[0xf74c0d81]
/lib/libc.so.6(abort+0x182)[0xf74c2632]
/lib/libc.so.6(+0x6abd5)[0xf7500bd5]
/lib/libc.so.6(+0x71139)[0xf7507139]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xf77048b1]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xf770490d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/libc.so.6(__libc_start_main+0xe6)[0xf74acbb6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Apr 2013 05:59:29 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 777,600 1,598,394 2.0555
14 Apr 2013 13:30:01 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 751,680 1,545,292 2.0558
13 Apr 2013 21:52:22 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 725,760 1,493,782 2.0582
13 Apr 2013 03:16:39 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 699,840 1,441,703 2.0600
12 Apr 2013 08:59:25 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 673,920 1,389,406 2.0617
11 Apr 2013 14:20:21 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 648,000 1,336,063 2.0618
10 Apr 2013 19:06:56 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 622,080 1,283,377 2.0630
10 Apr 2013 00:04:58 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 596,160 1,230,162 2.0635
09 Apr 2013 05:53:59 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 570,240 1,177,548 2.0650
08 Apr 2013 10:22:43 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 544,320 1,124,437 2.0658
07 Apr 2013 16:15:50 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 518,400 1,071,979 2.0679
06 Apr 2013 20:48:16 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 492,480 1,018,979 2.0691
06 Apr 2013 02:10:59 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 466,560 966,219 2.0709
25 Mar 2013 10:27:39 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 440,640 913,530 2.0732
24 Mar 2013 12:33:14 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 414,720 861,758 2.0779
23 Mar 2013 14:20:25 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 388,800 809,303 2.0815
22 Mar 2013 23:28:38 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 362,880 756,924 2.0859
22 Mar 2013 08:33:53 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 336,960 704,648 2.0912
21 Mar 2013 17:26:02 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 311,040 651,740 2.0954
21 Mar 2013 02:27:34 1019749 15644282 hadcm3n_48cd_1940_40_008308175_2 285,120 598,928 2.1006


©2024 climateprediction.net