Name | hadcm3n_o13j_1940_40_007267695_1 |
Workunit | 7465935 |
Created | 3 Jun 2011, 5:04:17 UTC |
Sent | 3 Jun 2011, 5:04:21 UTC |
Report deadline | 2 Sep 2011, 12:31:32 UTC |
Received | 8 Jul 2011, 5:49:21 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1130435 |
Run time | 26 days 22 hours 1 min 16 sec |
CPU time | 22 days 21 hours 27 min 5 sec |
Validate state | Invalid |
Credit | 10,575.36 |
Device peak FLOPS | 2.64 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin |
Stderr | <core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Thu Jul 7 12:39:45 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x92c8e156 ___kill + 10 1 libSystem.B.dylib 0x92d20899 _raise + 26 2 libSystem.B.dylib 0x92d369b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x92c2d1ae _mach_wait_until + 10 1 libSystem.B.dylib 0x92cb4809 _nanosleep + 345 2 libSystem.B.dylib 0x92cb46aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x92c5a7fd __pthread_start + 345 6 libSystem.B.dylib 0x92c5a682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca18 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92c2c000 - 0x92dd3fff /usr/lib/libSystem.B.dylib 0x98757000 - 0x98765fff /usr/lib/libz.1.dylib 0x9888b000 - 0x9888efff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2286, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Thu Jul 7 12:39:48 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x92c8e156 ___kill + 10 1 libSystem.B.dylib 0x92d20899 _raise + 26 2 libSystem.B.dylib 0x92d369b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x92c2d1ae _mach_wait_until + 10 1 libSystem.B.dylib 0x92cb4809 _nanosleep + 345 2 libSystem.B.dylib 0x92cb46aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x92c5a7fd __pthread_start + 345 6 libSystem.B.dylib 0x92c5a682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca18 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92c2c000 - 0x92dd3fff /usr/lib/libSystem.B.dylib 0x98757000 - 0x98765fff /usr/lib/libz.1.dylib 0x9888b000 - 0x9888efff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2286, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Thu Jul 7 12:39:51 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x92c8e156 ___kill + 10 1 libSystem.B.dylib 0x92d20899 _raise + 26 2 libSystem.B.dylib 0x92d369b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x92c2d1ae _mach_wait_until + 10 1 libSystem.B.dylib 0x92cb4809 _nanosleep + 345 2 libSystem.B.dylib 0x92cb46aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x92c5a7fd __pthread_start + 345 6 libSystem.B.dylib 0x92c5a682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca18 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92c2c000 - 0x92dd3fff /usr/lib/libSystem.B.dylib 0x98757000 - 0x98765fff /usr/lib/libz.1.dylib 0x9888b000 - 0x9888efff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2286, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Thu Jul 7 12:39:54 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x92c8e156 ___kill + 10 1 libSystem.B.dylib 0x92d20899 _raise + 26 2 libSystem.B.dylib 0x92d369b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x92c2d1ae _mach_wait_until + 10 1 libSystem.B.dylib 0x92cb4809 _nanosleep + 345 2 libSystem.B.dylib 0x92cb46aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x92c5a7fd __pthread_start + 345 6 libSystem.B.dylib 0x92c5a682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca18 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92c2c000 - 0x92dd3fff /usr/lib/libSystem.B.dylib 0x98757000 - 0x98765fff /usr/lib/libz.1.dylib 0x9888b000 - 0x9888efff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2286, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Thu Jul 7 12:39:57 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x92c8e156 ___kill + 10 1 libSystem.B.dylib 0x92d20899 _raise + 26 2 libSystem.B.dylib 0x92d369b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x92c2d1ae _mach_wait_until + 10 1 libSystem.B.dylib 0x92cb4809 _nanosleep + 345 2 libSystem.B.dylib 0x92cb46aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x92c5a7fd __pthread_start + 345 6 libSystem.B.dylib 0x92c5a682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca18 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92c2c000 - 0x92dd3fff /usr/lib/libSystem.B.dylib 0x98757000 - 0x98765fff /usr/lib/libz.1.dylib 0x9888b000 - 0x9888efff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2286, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Thu Jul 7 12:40:00 2011 Thread 0 Crashed: Suspended CPDN Monitor - Suspend request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Thu Jul 7 22:00:54 2011 Thread 0 Crashed: SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Thu Jul 7 22:39:58 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x92c8e156 ___kill + 10 1 libSystem.B.dylib 0x92d20899 _raise + 26 2 libSystem.B.dylib 0x92d369b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x92c2d1ae _mach_wait_until + 10 1 libSystem.B.dylib 0x92cb4809 _nanosleep + 345 2 libSystem.B.dylib 0x92cb46aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x92c5a7fd __pthread_start + 345 6 libSystem.B.dylib 0x92c5a682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca18 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92c2c000 - 0x92dd3fff /usr/lib/libSystem.B.dylib 0x98757000 - 0x98765fff /usr/lib/libz.1.dylib 0x9888b000 - 0x9888efff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=533, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
25 Jun 2011 01:29:29 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 881,280 | 1,168,809 | 1.3263 |
24 Jun 2011 13:38:57 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 855,360 | 1,128,931 | 1.3198 |
24 Jun 2011 01:57:43 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 829,440 | 1,089,059 | 1.3130 |
23 Jun 2011 12:06:50 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 803,520 | 1,047,344 | 1.3034 |
22 Jun 2011 19:06:33 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 777,600 | 1,006,507 | 1.2944 |
22 Jun 2011 05:40:43 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 751,680 | 966,637 | 1.2860 |
21 Jun 2011 17:21:08 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 725,760 | 927,485 | 1.2780 |
21 Jun 2011 05:18:25 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 699,840 | 888,035 | 1.2689 |
20 Jun 2011 14:53:28 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 673,920 | 848,789 | 1.2595 |
20 Jun 2011 02:48:05 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 648,000 | 809,008 | 1.2485 |
19 Jun 2011 22:06:54 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 622,080 | 798,679 | 1.2839 |
19 Jun 2011 22:06:54 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 596,160 | 758,476 | 1.2723 |
19 Jun 2011 22:06:54 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 570,240 | 719,213 | 1.2612 |
19 Jun 2011 21:58:05 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 544,320 | 679,413 | 1.2482 |
17 Jun 2011 11:41:34 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 518,400 | 640,295 | 1.2351 |
16 Jun 2011 21:54:35 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 492,480 | 600,410 | 1.2192 |
16 Jun 2011 09:00:47 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 466,560 | 563,146 | 1.2070 |
15 Jun 2011 20:30:27 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 440,640 | 526,099 | 1.1939 |
15 Jun 2011 08:56:15 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 414,720 | 486,726 | 1.1736 |
14 Jun 2011 21:03:36 | 1130435 | 12929040 | hadcm3n_o13j_1940_40_007267695_1 | 388,800 | 447,801 | 1.1518 |
©2024 cpdn.org