 |
|
| Name | hadcm3n_zcrm_1920_40_008280298_0 |
| Workunit | 8431433 |
| Created | 29 Dec 2012, 13:54:18 UTC |
| Sent | 29 Dec 2012, 19:00:49 UTC |
| Report deadline | 31 Mar 2013, 2:28:00 UTC |
| Received | 16 Mar 2013, 11:46:10 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1364627 |
| Run time | 21 days 5 hours 48 min 56 sec |
| CPU time | 21 days 0 hours 26 min 49 sec |
| Validate state | Invalid |
| Credit | 12,441.60 |
| Device peak FLOPS | 2.82 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>7.0.27</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:05:31 (14765): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:42:50 (32447): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:40 (17025): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:29:08 (6041): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:30:02 (24364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:31:23 (24395): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:33:42 (24432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:34:33 (24455): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:14 (24480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:36:26 (24502): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:02 (24542): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:58 (24603): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:57:06 (24627): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:57:49 (32588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:59:15 (32618): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:01:48 (32657): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:02:37 (32685): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:03:23 (32701): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:05:55 (32723): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:06 (32767): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:18 (327): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:11:04 (378): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:13:16 (401): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:14:02 (442): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:14:58 (464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:16:00 (489): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:16:46 (514): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:17:33 (537): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:18:19 (562): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:19:03 (584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:19:44 (606): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:21:11 (629): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:21:58 (662): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:54 (674): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:24:20 (700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:07 (731): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:54 (754): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:26:40 (776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:27:27 (799): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:13 (842): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:29:29 (876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:30:16 (904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:02 (927): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:48 (949): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:32:28 (971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:45:51 (1003): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:16:56 (24117): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:58:25 (16464): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:22:39 (13430): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish 19:16:23 (5597): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:23:52 (5516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:27:56 (847): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:35:50 (18859): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x090bf1c8 *** ======= Backtrace: ========= /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70f01)[0xf752cf01] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x72768)[0xf752e768] /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xf753181d] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xf76af4bf] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xf74d2e46] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:08 13336823 /home/mugurel/boinc.data/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:08 13336823 /home/mugurel/boinc.data/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 09069000-090cf000 rw-p 00000000 00:00 0 [heap] f6f00000-f6f21000 rw-p 00000000 00:00 0 f6f21000-f7000000 ---p 00000000 00:00 0 f703e000-f74b9000 rw-s 00000000 08:08 328763 /home/mugurel/boinc.data/slots/4/140210 f74b9000-f74bc000 rw-p 00000000 00:00 0 f74bc000-f7618000 r-xp 00000000 08:06 1864176 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so f7618000-f7619000 ---p 0015c000 08:06 1864176 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so f7619000-f761b000 r--p 0015c000 08:06 1864176 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so f761b000-f761c000 rw-p 0015e000 08:06 1864176 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so f761c000-f761f000 rw-p 00000000 00:00 0 f761f000-f763b000 r-xp 00000000 08:06 1831425 /lib/i386-linux-gnu/libgcc_s.so.1 f763b000-f763c000 rw-p 0001b000 08:06 1831425 /lib/i386-linux-gnu/libgcc_s.so.1 f763c000-f7660000 r-xp 00000000 08:06 1864165 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so f7660000-f7661000 r--p 00023000 08:06 1864165 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so f7661000-f7662000 rw-p 00024000 08:06 1864165 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so f7662000-f7742000 r-xp 00000000 08:06 3109321 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f7742000-f7746000 r--p 000e0000 08:06 3109321 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f7746000-f7747000 rw-p 000e4000 08:06 3109321 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f7747000-f774f000 rw-p 00000000 00:00 0 f774f000-f7751000 r-xp 00000000 08:06 1864154 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so f7751000-f7752000 r--p 00001000 08:06 1864154 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so f7752000-f7753000 rw-p 00002000 08:06 1864154 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so f7753000-f7768000 r-xp 00000000 08:06 1864151 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so f7768000-f7769000 r--p 00014000 08:06 1864151 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so f7769000-f776a000 rw-p 00015000 08:06 1864151 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so f776a000-f776c000 rw-p 00000000 00:00 0 f7786000-f7787000 rw-p 00000000 00:00 0 f7787000-f7788000 ---p 00000000 00:00 0 f7788000-f778b000 rw-p 00000000 00:00 0 f778b000-f778d000 rw-s 00000000 08:08 328760 /home/mugurel/boinc.data/slots/4/boinc_mmap_file f778d000-f778f000 rw-p 00000000 00:00 0 f778f000-f7790000 r-xp 00000000 00:00 0 [vdso] f7790000-f77ac000 r-xp 00000000 08:06 1834489 /lib/i386-linux-gnu/ld-2.13.so f77ac000-f77ad000 r--p 0001b000 08:06 1834489 /lib/i386-linux-gnu/ld-2.13.so f77ad000-f77ae000 rw-p 0001c000 08:06 1834489 /lib/i386-linux-gnu/ld-2.13.so ffeb7000-fff28000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (17 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf778f400] [0xf778f430] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x51)[0xf74e6941] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x182)[0xf74e9d72] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x66e15)[0xf7522e15] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70f01)[0xf752cf01] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x72768)[0xf752e768] /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xf753181d] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xf76af4bf] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xf74d2e46] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 15 Mar 2013 11:49:18 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 1,036,800 | 1,816,000 | 1.7515 |
| 10 Mar 2013 02:30:03 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 1,010,880 | 1,765,973 | 1.7470 |
| 09 Mar 2013 11:32:56 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 984,960 | 1,712,747 | 1.7389 |
| 02 Mar 2013 16:15:52 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 959,040 | 1,656,984 | 1.7278 |
| 26 Feb 2013 14:55:41 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 933,120 | 1,609,892 | 1.7253 |
| 22 Feb 2013 11:57:37 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 907,200 | 1,572,797 | 1.7337 |
| 21 Feb 2013 07:49:27 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 881,280 | 1,525,482 | 1.7310 |
| 19 Feb 2013 19:37:37 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 855,360 | 1,477,861 | 1.7278 |
| 19 Feb 2013 06:17:44 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 829,440 | 1,431,038 | 1.7253 |
| 17 Feb 2013 01:35:34 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 803,520 | 1,386,635 | 1.7257 |
| 16 Feb 2013 16:05:49 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 777,600 | 1,354,701 | 1.7422 |
| 16 Feb 2013 07:13:37 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 751,680 | 1,322,771 | 1.7598 |
| 15 Feb 2013 22:16:31 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 725,760 | 1,290,639 | 1.7783 |
| 15 Feb 2013 13:20:11 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 699,840 | 1,258,548 | 1.7983 |
| 15 Feb 2013 04:26:16 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 673,920 | 1,226,621 | 1.8201 |
| 14 Feb 2013 19:42:12 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 648,000 | 1,194,558 | 1.8435 |
| 14 Feb 2013 10:26:03 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 622,080 | 1,162,010 | 1.8679 |
| 14 Feb 2013 01:29:05 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 596,160 | 1,129,819 | 1.8952 |
| 13 Feb 2013 16:35:51 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 570,240 | 1,097,843 | 1.9252 |
| 12 Feb 2013 16:11:28 | 1184661 | 15516385 | hadcm3n_zcrm_1920_40_008280298_0 | 544,320 | 1,059,225 | 1.9460 |
©2024 climateprediction.net