Name | hadcm3n_zfxn_1880_40_008198551_1 |
Workunit | 8353675 |
Created | 12 Sep 2012, 19:09:09 UTC |
Sent | 12 Sep 2012, 19:17:36 UTC |
Report deadline | 13 Dec 2012, 2:44:47 UTC |
Received | 7 Oct 2012, 5:36:14 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 193 (0x000000C1) EXIT_SIGNAL |
Computer ID | 383878 |
Run time | 20 days 11 hours 20 min 35 sec |
CPU time | 17 days 20 hours 56 min 11 sec |
Validate state | Invalid |
Credit | 9,331.20 |
Device peak FLOPS | 2.48 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin |
Stderr | <core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x481de04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x481de00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6002604: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6002600: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6001c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6001c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6001c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6001c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6001c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6001c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6001c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18663,0xa0c7a540) malloc: *** error for object 0x6001c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:58:56 (50813): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:58:58 (50813): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:23:50 (73630): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:17:23 (86855): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:32:32 (1423): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:30:36 (9699): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812804: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812800: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812804: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812800: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812804: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812800: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812804: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812800: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16735,0xa0b41540) malloc: *** error for object 0x812804: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Sat Oct 6 23:46:17 2012 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin. 0 libSystem.B.dylib 0x9a157b03 small_free_list_remove_ptr + 234 1 libSystem.B.dylib 0x9a1545cc szone_free_definite_size + 3457 2 libSystem.B.dylib 0x9a1535e8 free + 244 3 hadcm3n_6.07_i686-apple-darwin 0x0000ba58 annual_cycle(std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*, char const*, int, int) + 3482 4 hadcm3n_6.07_i686-apple-darwin 0x0000d36b decadalMeans(int, char const*) + 957 5 hadcm3n_6.07_i686-apple-darwin 0x000067ff doCM3Proc() + 185 6 hadcm3n_6.07_i686-apple-darwin 0x0000791c mainLoop() + 410 7 hadcm3n_6.07_i686-apple-darwin 0x000087c7 worker() + 2989 8 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 main + 491 9 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 1: 0 libSystem.B.dylib 0x9a14cc0e mach_wait_until + 10 1 libSystem.B.dylib 0x9a1d4429 nanosleep + 345 2 libSystem.B.dylib 0x9a1d42ca usleep + 61 3 hadcm3n_6.07_i686-apple-darwin 0x00071a7c boinc_sleep(double) + 188 4 hadcm3n_6.07_i686-apple-darwin 0x00067282 timer_thread(void*) + 78 5 libSystem.B.dylib 0x9a17a259 _pthread_start + 345 6 libSystem.B.dylib 0x9a17a0de thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8ef38 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/6/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x93436000 - 0x934a0fff /usr/lib/libstdc++.6.dylib 0x94873000 - 0x94881fff /usr/lib/libz.1.dylib 0x98242000 - 0x98245fff /usr/lib/system/libmathCommon.A.dylib 0x9a14c000 - 0x9a2f3fff /usr/lib/libSystem.B.dylib Exiting... </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
07 Oct 2012 05:39:02 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 777,600 | 1,544,166 | 1.9858 |
06 Oct 2012 09:27:47 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 751,680 | 1,493,682 | 1.9871 |
05 Oct 2012 07:52:19 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 725,760 | 1,443,777 | 1.9893 |
04 Oct 2012 09:18:11 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 699,840 | 1,392,883 | 1.9903 |
03 Oct 2012 08:46:30 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 673,920 | 1,342,805 | 1.9925 |
02 Oct 2012 14:38:16 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 648,000 | 1,293,169 | 1.9956 |
01 Oct 2012 11:56:03 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 622,080 | 1,240,899 | 1.9948 |
30 Sep 2012 17:31:09 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 596,160 | 1,189,775 | 1.9957 |
30 Sep 2012 00:04:48 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 570,240 | 1,138,563 | 1.9966 |
29 Sep 2012 05:55:15 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 544,320 | 1,087,366 | 1.9977 |
28 Sep 2012 10:26:41 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 518,400 | 1,036,620 | 1.9997 |
27 Sep 2012 12:09:53 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 492,480 | 986,084 | 2.0023 |
26 Sep 2012 15:55:38 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 466,560 | 935,360 | 2.0048 |
25 Sep 2012 15:44:27 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 440,640 | 883,049 | 2.0040 |
24 Sep 2012 22:40:34 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 414,720 | 831,811 | 2.0057 |
24 Sep 2012 04:43:59 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 388,800 | 779,332 | 2.0045 |
23 Sep 2012 11:46:52 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 362,880 | 726,603 | 2.0023 |
22 Sep 2012 20:32:36 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 336,960 | 674,499 | 2.0017 |
22 Sep 2012 04:52:10 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 311,040 | 622,373 | 2.0009 |
21 Sep 2012 13:13:50 | 383878 | 15274158 | hadcm3n_zfxn_1880_40_008198551_1 | 285,120 | 570,272 | 2.0001 |
©2024 cpdn.org