climateprediction.net home page
Task 14034705

Task 14034705

Name hadcm3n_yj4m_1940_40_007743440_1
Workunit 7898548
Created 30 Jan 2012, 22:02:27 UTC
Sent 31 Jan 2012, 1:06:46 UTC
Report deadline 1 May 2012, 8:33:57 UTC
Received 27 Feb 2012, 15:57:11 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1188684
Run time 26 days 1 hours 17 min 44 sec
CPU time 25 days 22 hours 11 min 45 sec
Validate state Invalid
Credit 12,441.60
Device peak FLOPS 2.27 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09e9fce8 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6eb72)[0xf753ab72]
/lib32/libc.so.6(+0x6f812)[0xf753b812]
/lib32/libc.so.6(cfree+0x6d)[0xf753e8cd]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf773bbaf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf773bc0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf74e50f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 fc:00 34344185                           /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 fc:00 34344185                           /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
09e4a000-09eaf000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f704e000-f74c9000 rw-s 00000000 fc:00 34343973                           /var/lib/boinc-client/slots/1/136320
f74c9000-f74cc000 rw-p 00000000 00:00 0 
f74cc000-f763f000 r-xp 00000000 fc:00 10878986                           /lib32/libc-2.13.so
f763f000-f7640000 ---p 00173000 fc:00 10878986                           /lib32/libc-2.13.so
f7640000-f7642000 r--p 00173000 fc:00 10878986                           /lib32/libc-2.13.so
f7642000-f7643000 rw-p 00175000 fc:00 10878986                           /lib32/libc-2.13.so
f7643000-f7646000 rw-p 00000000 00:00 0 
f7646000-f7662000 r-xp 00000000 fc:00 35135338                           /usr/lib32/libgcc_s.so.1
f7662000-f7663000 r--p 0001b000 fc:00 35135338                           /usr/lib32/libgcc_s.so.1
f7663000-f7664000 rw-p 0001c000 fc:00 35135338                           /usr/lib32/libgcc_s.so.1
f7664000-f768c000 r-xp 00000000 fc:00 10878996                           /lib32/libm-2.13.so
f768c000-f768d000 r--p 00028000 fc:00 10878996                           /lib32/libm-2.13.so
f768d000-f768e000 rw-p 00029000 fc:00 10878996                           /lib32/libm-2.13.so
f768e000-f776c000 r-xp 00000000 fc:00 35135339                           /usr/lib32/libstdc++.so.6.0.16
f776c000-f776d000 ---p 000de000 fc:00 35135339                           /usr/lib32/libstdc++.so.6.0.16
f776d000-f7771000 r--p 000de000 fc:00 35135339                           /usr/lib32/libstdc++.so.6.0.16
f7771000-f7772000 rw-p 000e2000 fc:00 35135339                           /usr/lib32/libstdc++.so.6.0.16
f7772000-f777a000 rw-p 00000000 00:00 0 
f777a000-f777d000 r-xp 00000000 fc:00 10878987                           /lib32/libdl-2.13.so
f777d000-f777e000 r--p 00002000 fc:00 10878987                           /lib32/libdl-2.13.so
f777e000-f777f000 rw-p 00003000 fc:00 10878987                           /lib32/libdl-2.13.so
f777f000-f7796000 r-xp 00000000 fc:00 10878994                           /lib32/libpthread-2.13.so
f7796000-f7797000 r--p 00016000 fc:00 10878994                           /lib32/libpthread-2.13.so
f7797000-f7798000 rw-p 00017000 fc:00 10878994                           /lib32/libpthread-2.13.so
f7798000-f779a000 rw-p 00000000 00:00 0 
f77b3000-f77b4000 rw-p 00000000 00:00 0 
f77b4000-f77b5000 ---p 00000000 00:00 0 
f77b5000-f77b8000 rw-p 00000000 00:00 0 
f77b8000-f77ba000 rw-s 00000000 fc:00 34343969                           /var/lib/boinc-client/slots/1/boinc_mmap_file
f77ba000-f77bc000 rw-p 00000000 00:00 0 
f77bc000-f77bd000 r-xp 00000000 00:00 0                                  [vdso]
f77bd000-f77db000 r-xp 00000000 fc:00 32899078                           /lib/i386-linux-gnu/ld-2.13.so
f77db000-f77dc000 r--p 0001d000 fc:00 32899078                           /lib/i386-linux-gnu/ld-2.13.so
f77dc000-f77dd000 rw-p 0001e000 fc:00 32899078                           /lib/i386-linux-gnu/ld-2.13.so
ffceb000-ffd5a000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf77bc400]
[0xf77bc430]
/lib32/libc.so.6(gsignal+0x4f)[0xf74f9c4f]
/lib32/libc.so.6(abort+0x175)[0xf74fd175]
/lib32/libc.so.6(+0x63e8c)[0xf752fe8c]
/lib32/libc.so.6(+0x6eb72)[0xf753ab72]
/lib32/libc.so.6(+0x6f812)[0xf753b812]
/lib32/libc.so.6(cfree+0x6d)[0xf753e8cd]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf773bbaf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf773bc0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf74e50f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
27 Feb 2012 16:00:38 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 1,036,800 2,239,904 2.1604
26 Feb 2012 23:57:01 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 1,010,880 2,182,348 2.1589
26 Feb 2012 07:57:22 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 984,960 2,124,869 2.1573
25 Feb 2012 16:08:41 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 959,040 2,068,090 2.1564
25 Feb 2012 00:31:30 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 933,120 2,012,443 2.1567
24 Feb 2012 09:14:07 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 907,200 1,958,875 2.1593
23 Feb 2012 17:47:08 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 881,280 1,905,182 2.1618
23 Feb 2012 02:17:52 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 855,360 1,851,605 2.1647
22 Feb 2012 11:05:51 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 829,440 1,798,082 2.1678
21 Feb 2012 19:40:52 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 803,520 1,743,911 2.1703
21 Feb 2012 03:30:51 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 777,600 1,685,823 2.1680
20 Feb 2012 11:12:58 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 751,680 1,627,480 2.1651
19 Feb 2012 19:14:54 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 725,760 1,569,199 2.1621
19 Feb 2012 02:45:51 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 699,840 1,510,810 2.1588
18 Feb 2012 11:12:55 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 673,920 1,455,563 2.1598
17 Feb 2012 19:49:17 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 648,000 1,401,797 2.1633
17 Feb 2012 04:25:45 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 622,080 1,348,037 2.1670
16 Feb 2012 13:03:34 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 596,160 1,294,275 2.1710
15 Feb 2012 21:29:29 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 570,240 1,239,842 2.1742
15 Feb 2012 05:47:38 1188684 14034705 hadcm3n_yj4m_1940_40_007743440_1 544,320 1,184,762 2.1766


©2024 cpdn.org