climateprediction.net home page
Task 15766576

Task 15766576

Name hadcm3n_4khb_1940_40_008303902_4
Workunit 8455037
Created 9 May 2013, 3:56:30 UTC
Sent 9 May 2013, 3:56:38 UTC
Report deadline 8 Aug 2013, 11:23:49 UTC
Received 1 Jul 2013, 21:24:25 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1251557
Run time 4 days 11 hours 5 min 19 sec
CPU time 4 days 7 hours 27 min 11 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 3.84 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr
<core_client_version>7.0.31</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
CPDN Monitor - Quit request from BOINC...
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
hadcm3n_6.07_i686-apple-darwin(38355,0xac617a28) malloc: *** error for object 0x22f0404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(38355,0xac617a28) malloc: *** error for object 0x22f0400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(38355,0xac617a28) malloc: *** error for object 0x1304004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(38355,0xac617a28) malloc: *** error for object 0x22db004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(38355,0xac617a28) malloc: *** error for object 0x22db000: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
hadcm3n_6.07_i686-apple-darwin(39506,0xac617a28) malloc: *** error for object 0x963404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(39506,0xac617a28) malloc: *** error for object 0x963400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(39506,0xac617a28) malloc: *** error for object 0x3185e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
hadcm3n_6.07_i686-apple-darwin(25789,0xacac0a28) malloc: *** error for object 0x11f9804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(25789,0xacac0a28) malloc: *** error for object 0x11f9800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(25789,0xacac0a28) malloc: *** error for object 0x19d6004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(25789,0xacac0a28) malloc: *** error for object 0x19d6000: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(25789,0xacac0a28) malloc: *** error for object 0x19d6004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(25789,0xacac0a28) malloc: *** error for object 0x19d6000: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(25789,0xacac0a28) malloc: *** error for object 0x19d6004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid
dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/3/../../switcher/switcher) is setuid or setgid
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0xb2c804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0xb2c800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3302e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3303c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3303c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3303c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3303c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3303c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3303c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3303c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(61893,0xacac0a28) malloc: *** error for object 0x3303c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4khb_1940_40_008303902/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
20 May 2013 04:34:29 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 518,400 370,821 0.7153
19 May 2013 20:14:44 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 492,480 352,222 0.7152
19 May 2013 11:48:11 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 466,560 333,463 0.7147
19 May 2013 04:45:47 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 440,640 314,839 0.7145
18 May 2013 19:05:17 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 414,720 296,289 0.7144
18 May 2013 13:05:13 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 388,800 277,614 0.7140
18 May 2013 04:37:00 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 362,880 259,164 0.7142
17 May 2013 17:30:26 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 336,960 240,574 0.7140
17 May 2013 12:06:37 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 311,040 221,969 0.7136
17 May 2013 03:33:26 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 285,120 203,406 0.7134
16 May 2013 20:02:58 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 259,200 184,893 0.7133
16 May 2013 12:03:31 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 233,280 166,435 0.7135
16 May 2013 04:25:40 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 207,360 148,083 0.7141
15 May 2013 21:22:20 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 181,440 129,597 0.7143
15 May 2013 12:46:56 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 155,520 111,145 0.7147
15 May 2013 05:29:22 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 129,600 92,665 0.7150
14 May 2013 20:29:46 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 103,680 74,256 0.7162
14 May 2013 13:45:31 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 77,760 55,777 0.7173
14 May 2013 05:37:34 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 51,840 37,252 0.7186
09 May 2013 15:20:51 1251557 15766576 hadcm3n_4khb_1940_40_008303902_4 25,920 18,698 0.7214


©2024 cpdn.org