climateprediction.net home page
Task 12745082

Task 12745082

Name hadcm3n_o53j_1900_40_007201938_1
Workunit 7400218
Created 28 Mar 2011, 14:12:31 UTC
Sent 30 Mar 2011, 0:29:34 UTC
Report deadline 29 Jun 2011, 7:56:45 UTC
Received 5 May 2011, 1:16:36 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1116505
Run time 27 days 17 hours 36 min 46 sec
CPU time 18 days 23 hours 24 min 53 sec
Validate state Invalid
Credit 9,953.28
Device peak FLOPS 2.46 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr 
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
22:07:21 (75424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation
22:53:28 (46745): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:11:27 (47951): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:57:23 (384): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:57:24 (384): No heartbeat from core client for 30 sec - exiting
06:55:03 (8285): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:12:59 (17287): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:54:03 (30185): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
Suspended CPDN Monitor - Suspend request from BOINC...
12:57:22 (52791): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:58:42 (37737): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:51:44 (44018): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
19:16:56 (401): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:16:58 (401): No heartbeat from core client for 30 sec - exiting
01:25:29 (8982): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:25:31 (8982): No heartbeat from core client for 30 sec - exiting
00:05:02 (20834): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:07:13 (64647): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:50:01 (72555): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:24:31 (18517): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:26:49 (31118): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:48:18 (25822): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:08:21 (9416): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:29:42 (48296): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:29:43 (48296): No heartbeat from core client for 30 sec - exiting
08:29:44 (48296): No heartbeat from core client for 30 sec - exiting
08:29:45 (48296): No heartbeat from core client for 30 sec - exiting
08:29:46 (48296): No heartbeat from core client for 30 sec - exiting
08:29:47 (48296): No heartbeat from core client for 30 sec - exiting
08:29:48 (48296): No heartbeat from core client for 30 sec - exiting
08:29:49 (48296): No heartbeat from core client for 30 sec - exiting
08:29:50 (48296): No heartbeat from core client for 30 sec - exiting
08:29:51 (48296): No heartbeat from core client for 30 sec - exiting
08:29:52 (48296): No heartbeat from core client for 30 sec - exiting
08:29:53 (48296): No heartbeat from core client for 30 sec - exiting
08:29:54 (48296): No heartbeat from core client for 30 sec - exiting
08:29:55 (48296): No heartbeat from core client for 30 sec - exiting
08:29:56 (48296): No heartbeat from core client for 30 sec - exiting
08:29:57 (48296): No heartbeat from core client for 30 sec - exiting
08:29:58 (48296): No heartbeat from core client for 30 sec - exiting
08:29:59 (48296): No heartbeat from core client for 30 sec - exiting
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.7 build 10J869
Wed Apr 27 08:30:56 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x9194e156 __kill + 10
1   libSystem.B.dylib                   0x919e0899 raise + 26
2   libSystem.B.dylib                   0x919f69b8 abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb _start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x918ed1ae mach_wait_until + 10
1   libSystem.B.dylib                   0x91974809 nanosleep + 345
2   libSystem.B.dylib                   0x919746aa usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 boinc_sleep(double) + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e timer_thread(void*) + 78
5   libSystem.B.dylib                   0x9191a7fd _pthread_start + 345
6   libSystem.B.dylib                   0x9191a682 thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca28 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x918ec000 - 0x91a93fff /usr/lib/libSystem.B.dylib
0x9797f000 - 0x97982fff /usr/lib/system/libmathCommon.A.dylib
0x98338000 - 0x98346fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=86313, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.7 build 10J869
Wed Apr 27 08:30:59 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x9194e156 __kill + 10
1   libSystem.B.dylib                   0x919e0899 raise + 26
2   libSystem.B.dylib                   0x919f69b8 abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb _start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x918ed1ae mach_wait_until + 10
1   libSystem.B.dylib                   0x91974809 nanosleep + 345
2   libSystem.B.dylib                   0x919746aa usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 boinc_sleep(double) + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e timer_thread(void*) + 78
5   libSystem.B.dylib                   0x9191a7fd _pthread_start + 345
6   libSystem.B.dylib                   0x9191a682 thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca28 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x918ec000 - 0x91a93fff /usr/lib/libSystem.B.dylib
0x9797f000 - 0x97982fff /usr/lib/system/libmathCommon.A.dylib
0x98338000 - 0x98346fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=86313, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.7 build 10J869
Wed Apr 27 08:31:02 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x9194e156 __kill + 10
1   libSystem.B.dylib                   0x919e0899 raise + 26
2   libSystem.B.dylib                   0x919f69b8 abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb _start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x918ed1ae mach_wait_until + 10
1   libSystem.B.dylib                   0x91974809 nanosleep + 345
2   libSystem.B.dylib                   0x919746aa usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 boinc_sleep(double) + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e timer_thread(void*) + 78
5   libSystem.B.dylib                   0x9191a7fd _pthread_start + 345
6   libSystem.B.dylib                   0x9191a682 thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca28 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x918ec000 - 0x91a93fff /usr/lib/libSystem.B.dylib
0x9797f000 - 0x97982fff /usr/lib/system/libmathCommon.A.dylib
0x98338000 - 0x98346fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=86313, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.7 build 10J869
Wed Apr 27 08:31:12 2011

Thread 0 Crashed:
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:26:30 (86313): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.7 build 10J869
Wed May  4 08:26:54 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x9194e156 __kill + 10
1   libSystem.B.dylib                   0x919e0899 raise + 26
2   libSystem.B.dylib                   0x919f69b8 abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb _start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x918ed1ae mach_wait_until + 10
1   libSystem.B.dylib                   0x91974809 nanosleep + 345
2   libSystem.B.dylib                   0x919746aa usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 boinc_sleep(double) + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e timer_thread(void*) + 78
5   libSystem.B.dylib                   0x9191a7fd _pthread_start + 345
6   libSystem.B.dylib                   0x9191a682 thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca28 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x918ec000 - 0x91a93fff /usr/lib/libSystem.B.dylib
0x9797f000 - 0x97982fff /usr/lib/system/libmathCommon.A.dylib
0x98338000 - 0x98346fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=55205, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.7 build 10J869
Wed May  4 08:26:57 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x9194e156 __kill + 10
1   libSystem.B.dylib                   0x919e0899 raise + 26
2   libSystem.B.dylib                   0x919f69b8 abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb _start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x918ed1ae mach_wait_until + 10
1   libSystem.B.dylib                   0x91974809 nanosleep + 345
2   libSystem.B.dylib                   0x919746aa usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 boinc_sleep(double) + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e timer_thread(void*) + 78
5   libSystem.B.dylib                   0x9191a7fd _pthread_start + 345
6   libSystem.B.dylib                   0x9191a682 thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca28 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x918ec000 - 0x91a93fff /usr/lib/libSystem.B.dylib
0x9797f000 - 0x97982fff /usr/lib/system/libmathCommon.A.dylib
0x98338000 - 0x98346fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=55205, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.7 build 10J869
Wed May  4 08:27:00 2011

Thread 0 Crashed:
CPDN Monitor - Quit request from BOINC...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.7 build 10J869
Wed May  4 17:56:46 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x9194e156 __kill + 10
1   libSystem.B.dylib                   0x919e0899 raise + 26
2   libSystem.B.dylib                   0x919f69b8 abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb _start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x918ed1ae mach_wait_until + 10
1   libSystem.B.dylib                   0x91974809 nanosleep + 345
2   libSystem.B.dylib                   0x919746aa usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 boinc_sleep(double) + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e timer_thread(void*) + 78
5   libSystem.B.dylib                   0x9191a7fd _pthread_start + 345
6   libSystem.B.dylib                   0x9191a682 thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffca28 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x918ec000 - 0x91a93fff /usr/lib/libSystem.B.dylib
0x9797f000 - 0x97982fff /usr/lib/system/libmathCommon.A.dylib
0x98338000 - 0x98346fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=439, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
26 Apr 2011 21:06:32 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 829,440 1,603,221 1.9329
26 Apr 2011 02:36:03 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 803,520 1,559,123 1.9404
25 Apr 2011 08:14:41 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 777,600 1,515,479 1.9489
24 Apr 2011 13:31:08 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 751,680 1,471,497 1.9576
23 Apr 2011 19:47:33 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 725,760 1,428,054 1.9677
23 Apr 2011 01:55:23 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 699,840 1,384,197 1.9779
22 Apr 2011 06:47:59 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 673,920 1,339,680 1.9879
21 Apr 2011 09:58:10 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 648,000 1,294,352 1.9975
20 Apr 2011 18:21:26 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 622,080 1,249,121 2.0080
20 Apr 2011 18:21:26 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 596,160 1,204,309 2.0201
20 Apr 2011 18:21:25 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 570,240 1,159,557 2.0335
20 Apr 2011 18:21:25 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 544,320 1,114,642 2.0478
20 Apr 2011 18:21:25 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 518,400 1,069,919 2.0639
20 Apr 2011 18:21:25 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 492,480 1,023,836 2.0789
20 Apr 2011 18:21:25 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 466,560 976,662 2.0933
20 Apr 2011 18:21:25 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 440,640 928,473 2.1071
13 Apr 2011 13:28:13 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 414,720 879,249 2.1201
12 Apr 2011 15:43:52 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 388,800 829,051 2.1323
12 Apr 2011 09:07:00 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 362,880 777,689 2.1431
12 Apr 2011 09:05:53 1116505 12745082 hadcm3n_o53j_1900_40_007201938_1 336,960 723,373 2.1468


©2024 climateprediction.net