climateprediction.net home page
Task 13142956

Task 13142956

Name hadcm3n_ydgx_1900_40_007350315_2
Workunit 7547745
Created 17 Jul 2011, 2:48:40 UTC
Sent 17 Jul 2011, 2:59:08 UTC
Report deadline 16 Oct 2011, 10:26:19 UTC
Received 26 Jul 2011, 19:47:34 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 803175
Run time 9 days 10 hours 6 min 35 sec
CPU time 8 days 13 hours 14 min 42 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 3.27 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/ydgxko.pja1c10 is not a valid UM file.
Error converting file to netcdf: dataout/ydgxko.pja1c10
Error: Input file: dataout/ydgxko.pia1c10 is not a valid UM file.
Error converting file to netcdf: dataout/ydgxko.pia1c10
Error: Input file: dataout/ydgxko.pfa1c10 is not a valid UM file.
Error converting file to netcdf: dataout/ydgxko.pfa1c10
Error: Input file: dataout/ydgxka.pha1c10 is not a valid UM file.
Error converting file to netcdf: dataout/ydgxka.pha1c10
Error: Input file: dataout/ydgxka.pga1c10 is not a valid UM file.
Error converting file to netcdf: dataout/ydgxka.pga1c10
Error: Input file: dataout/ydgxka.pea1c10 is not a valid UM file.
Error converting file to netcdf: dataout/ydgxka.pea1c10
Error: Input file: dataout/ydgxka.pda1c10 is not a valid UM file.
Error converting file to netcdf: dataout/ydgxka.pda1c10
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x82d604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x82d600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15135,0xa0389540) malloc: *** error for object 0x801400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
hadcm3n_6.07_i686-apple-darwin(93878,0xa0389540) malloc: *** error for object 0x105bc04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(93878,0xa0389540) malloc: *** error for object 0x105bc00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(93878,0xa0389540) malloc: *** error for object 0x835a04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(93878,0xa0389540) malloc: *** error for object 0x851004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(93878,0xa0389540) malloc: *** error for object 0x851000: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(93878,0xa0389540) malloc: *** error for object 0x2024604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Tue Jul 26 12:35:26 2011

hadcm3n_6.07_i686-apple-darwin(93878,0xa0389540) malloc: *** error for object 0x2827004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
Thread 0 Crashed:
0   libSystem.B.dylib                   0x92952b03 _small_free_list_remove_ptr + 234
1   libSystem.B.dylib                   0x9294f5cc _szone_free_definite_size + 3457
2   libSystem.B.dylib                   0x9294e5e8 _free + 244
3   hadcm3n_6.07_i686-apple-darwin      0x0000ba58 __Z12annual_cyclePKSt6vectorISsSaISsEEPKcii + 3482
4   hadcm3n_6.07_i686-apple-darwin      0x0000d36b __Z12decadalMeansiPKc + 957
5   hadcm3n_6.07_i686-apple-darwin      0x000067ff __Z9doCM3Procv + 185
6   hadcm3n_6.07_i686-apple-darwin      0x0000791c __Z8mainLoopv + 410
7   hadcm3n_6.07_i686-apple-darwin      0x000087c7 __Z6workerv + 2989
8   hadcm3n_6.07_i686-apple-darwin      0x00008aa9 _main + 491
9   hadcm3n_6.07_i686-apple-darwin      0x00002676 start + 54

Thread 1:
0   libSystem.B.dylib                   0x92947c0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x929cf429 _nanosleep + 345
2   libSystem.B.dylib                   0x929cf2ca _usleep + 61
3   hadcm3n_6.07_i686-apple-darwin      0x00071a7c __Z11boinc_sleepd + 188
4   hadcm3n_6.07_i686-apple-darwin      0x00067282 __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x92975259 __pthread_start + 345
6   libSystem.B.dylib                   0x929750de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8f6d8 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -    0x93fff /Library/Application Support/BOINC Data/slots/3/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin
0x900b8000 - 0x900c6fff /usr/lib/libz.1.dylib
0x9010a000 - 0x90174fff /usr/lib/libstdc++.6.dylib
0x92947000 - 0x92aeefff /usr/lib/libSystem.B.dylib
0x99689000 - 0x9968cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
26 Jul 2011 19:51:49 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 518,400 738,877 1.4253
26 Jul 2011 07:41:35 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 492,480 696,707 1.4147
25 Jul 2011 23:04:27 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 466,560 659,175 1.4128
25 Jul 2011 22:51:38 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 440,640 623,722 1.4155
25 Jul 2011 22:21:45 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 414,720 588,299 1.4185
25 Jul 2011 21:58:08 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 388,800 553,131 1.4227
25 Jul 2011 21:11:54 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 362,880 517,984 1.4274
25 Jul 2011 20:45:14 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 336,960 482,853 1.4330
25 Jul 2011 20:24:43 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 311,040 447,714 1.4394
25 Jul 2011 19:39:44 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 285,120 412,541 1.4469
25 Jul 2011 19:09:30 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 259,200 378,405 1.4599
25 Jul 2011 19:09:30 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 233,280 341,154 1.4624
25 Jul 2011 19:09:30 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 207,360 303,308 1.4627
25 Jul 2011 19:09:30 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 181,440 265,203 1.4617
25 Jul 2011 18:49:51 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 155,520 227,284 1.4614
25 Jul 2011 18:13:09 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 129,600 189,657 1.4634
25 Jul 2011 17:53:29 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 103,680 153,565 1.4811
25 Jul 2011 17:35:51 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 77,760 116,841 1.5026
25 Jul 2011 16:48:51 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 51,840 78,098 1.5065
25 Jul 2011 16:20:38 803175 13142956 hadcm3n_ydgx_1900_40_007350315_2 25,920 38,929 1.5019


©2024 cpdn.org