 |
|
| Name | hadcm3n_ldoy_198012_480_350_010332222_0 |
| Workunit | 10332222 |
| Created | 26 Feb 2016, 20:50:28 UTC |
| Sent | 26 Feb 2016, 21:25:09 UTC |
| Report deadline | 8 Feb 2017, 2:45:09 UTC |
| Received | 8 Mar 2016, 5:27:13 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1376149 |
| Run time | 9 days 21 hours 14 min 16 sec |
| CPU time | 8 days 22 hours 45 min 16 sec |
| Validate state | Invalid |
| Credit | 6,220.80 |
| Device peak FLOPS | 2.57 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>7.2.42</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish 19:07:02 (3689): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... 02:40:45 (3789): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:48:42 (31142): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:50:58 (21867): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:14:31 (27460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:19:33 (9489): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:57:32 (19404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:59:38 (30188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:59:39 (30188): No heartbeat from core client for 30 sec - exiting 03:59:40 (30188): No heartbeat from core client for 30 sec - exiting 03:59:41 (30188): No heartbeat from core client for 30 sec - exiting 03:59:42 (30188): No heartbeat from core client for 30 sec - exiting 03:59:43 (30188): No heartbeat from core client for 30 sec - exiting 03:59:44 (30188): No heartbeat from core client for 30 sec - exiting 03:59:45 (30188): No heartbeat from core client for 30 sec - exiting 03:59:46 (30188): No heartbeat from core client for 30 sec - exiting 04:03:38 (314): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:07:34 (4955): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:08:37 (7813): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:09:41 (7991): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:11:48 (8162): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:14:12 (8491): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:41 (8624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:25 (8873): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:19:27 (9159): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:20:32 (9261): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:22:55 (9421): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:24:57 (18304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:26:22 (18433): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:29:07 (18728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:30:09 (18941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:37 (19074): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:34:25 (19254): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:37:34 (19651): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:40:20 (19989): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:41:21 (20285): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:44:06 (20460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:46:08 (20813): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:48:51 (21109): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:53 (21350): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:51:42 (21445): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:54:30 (21830): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:56:33 (22076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:59:23 (22361): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:01:46 (22689): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:02:47 (22917): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:53 (23056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:06:17 (23302): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:07:18 (23514): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:09:20 (23725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:12:03 (23964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:13:04 (24165): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:14:09 (24342): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:16:50 (24609): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:19:30 (24885): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:11 (25156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:52 (25362): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:25:38 (25634): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:27:19 (25849): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:40 (26047): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:31:22 (26329): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:02 (26596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:36:44 (26865): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:37:46 (27075): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:40:47 (27302): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:42:11 (27596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:43:12 (27725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:13 (27996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:47:13 (28232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:49:34 (28472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:52:38 (28742): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:53:18 (28953): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:55:44 (29153): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:58:06 (29470): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:47 (29778): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:02:08 (29993): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:04:34 (30221): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:15 (30464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:07:21 (30566): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:10:02 (30879): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:12:44 (31158): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:14:05 (31399): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:16:26 (31667): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:18:26 (31905): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:21:08 (32213): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:23:48 (32482): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:26:49 (32749): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:27:50 (493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:29:31 (729): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:33:33 (1040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:34:34 (1368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:16 (1679): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:39:17 (1937): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:22 (2038): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:43:24 (2432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:46:28 (2705): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:48:08 (2955): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:51:10 (3258): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:51:51 (3519): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:54:33 (3798): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:55:55 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:58:16 (4318): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:00:17 (4558): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:03:17 (4834): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:04:59 (5114): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:06:04 (5209): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:26 (5558): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:19 (5867): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:59 (6062): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:21 (6333): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:17:41 (6601): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:19:42 (6837): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:22:43 (7142): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:25:24 (7413): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:26:46 (7622): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:29:26 (7893): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:30:47 (8103): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish *** Error in `../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x08d03118 *** ======= Backtrace: ========= /lib/libc.so.6(+0x69d49)[0xf73f1d49] /lib/libc.so.6(+0x71297)[0xf73f9297] /lib/libc.so.6(cfree+0x55)[0xf73fd2c5] /usr/lib/libstdc++.so.6(_ZdlPv+0x18)[0xf7610f18] /usr/lib/libstdc++.so.6(_ZdaPv+0x18)[0xf7610f58] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xde)[0xf739fefe] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:06 22806700 /home/cibolabwana/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:06 22806700 /home/cibolabwana/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 08c04000-08d97000 rw-p 00000000 00:00 0 [heap] f6d80000-f71fb000 rw-s 00000000 08:06 22937748 /home/cibolabwana/BOINC/slots/5/203365 f7200000-f7221000 rw-p 00000000 00:00 0 f7221000-f7300000 ---p 00000000 00:00 0 f7388000-f754b000 r-xp 00000000 08:01 522259 /lib/libc-2.20.so f754b000-f754e000 r--p 001c2000 08:01 522259 /lib/libc-2.20.so f754e000-f7550000 rw-p 001c5000 08:01 522259 /lib/libc-2.20.so f7550000-f7552000 rw-p 00000000 00:00 0 f7558000-f7572000 r-xp 00000000 08:01 527664 /lib/libgcc_s-4.9.2.so.1 f7572000-f7573000 r--p 00019000 08:01 527664 /lib/libgcc_s-4.9.2.so.1 f7573000-f7574000 rw-p 0001a000 08:01 527664 /lib/libgcc_s-4.9.2.so.1 f7578000-f75c3000 r-xp 00000000 08:01 522267 /lib/libm-2.20.so f75c3000-f75c4000 r--p 0004a000 08:01 522267 /lib/libm-2.20.so f75c4000-f75c5000 rw-p 0004b000 08:01 522267 /lib/libm-2.20.so f75c8000-f76af000 r-xp 00000000 08:01 651902 /usr/lib/libstdc++.so.6.0.20 f76af000-f76b3000 r--p 000e7000 08:01 651902 /usr/lib/libstdc++.so.6.0.20 f76b3000-f76b4000 rw-p 000eb000 08:01 651902 /usr/lib/libstdc++.so.6.0.20 f76b4000-f76bb000 rw-p 00000000 00:00 0 f76c0000-f76c3000 r-xp 00000000 08:01 522265 /lib/libdl-2.20.so f76c3000-f76c4000 r--p 00002000 08:01 522265 /lib/libdl-2.20.so f76c4000-f76c5000 rw-p 00003000 08:01 522265 /lib/libdl-2.20.so f76c8000-f76df000 r-xp 00000000 08:01 525739 /lib/libpthread-2.20.so f76df000-f76e0000 r--p 00017000 08:01 525739 /lib/libpthread-2.20.so f76e0000-f76e1000 rw-p 00018000 08:01 525739 /lib/libpthread-2.20.so f76e1000-f76e3000 rw-p 00000000 00:00 0 f76fe000-f7700000 rw-p 00000000 00:00 0 f7700000-f7702000 rw-s 00000000 08:06 22937739 /home/cibolabwana/BOINC/slots/5/boinc_mmap_file f7702000-f7703000 ---p 00000000 00:00 0 f7703000-f7708000 rw-p 00000000 00:00 0 [stack:3753] f7708000-f7728000 r-xp 00000000 08:01 522251 /lib/ld-2.20.so f7728000-f7729000 r--p 0001f000 08:01 522251 /lib/ld-2.20.so f7729000-f772a000 rw-p 00020000 08:01 522251 /lib/ld-2.20.so f772a000-f772d000 rw-p 00000000 00:00 0 ff84f000-ff8c0000 rw-p 00000000 00:00 0 [stack] ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso] SIGABRT: abort called Stack trace (18 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] linux-gate.so.1(__kernel_sigreturn+0x0)[0xffffe400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xffffe425] /lib/libc.so.6(gsignal+0x47)[0xf73b39f7] /lib/libc.so.6(abort+0x149)[0xf73b5369] /lib/libc.so.6(+0x69d4e)[0xf73f1d4e] /lib/libc.so.6(+0x71297)[0xf73f9297] /lib/libc.so.6(cfree+0x55)[0xf73fd2c5] /usr/lib/libstdc++.so.6(_ZdlPv+0x18)[0xf7610f18] /usr/lib/libstdc++.so.6(_ZdaPv+0x18)[0xf7610f58] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xde)[0xf739fefe] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 08 Mar 2016 04:27:12 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 518,400 | 773,111 | 1.4913 |
| 07 Mar 2016 16:12:07 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 492,480 | 733,628 | 1.4897 |
| 07 Mar 2016 03:57:51 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 466,560 | 694,376 | 1.4883 |
| 06 Mar 2016 15:35:57 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 440,640 | 655,011 | 1.4865 |
| 06 Mar 2016 00:24:32 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 414,720 | 617,148 | 1.4881 |
| 05 Mar 2016 12:18:24 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 388,800 | 578,108 | 1.4869 |
| 05 Mar 2016 00:06:47 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 362,880 | 539,048 | 1.4855 |
| 04 Mar 2016 12:00:56 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 336,960 | 500,181 | 1.4844 |
| 04 Mar 2016 00:01:16 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 311,040 | 461,382 | 1.4834 |
| 03 Mar 2016 11:33:10 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 285,120 | 422,699 | 1.4825 |
| 02 Mar 2016 22:54:09 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 259,200 | 384,172 | 1.4821 |
| 02 Mar 2016 10:20:23 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 233,280 | 345,832 | 1.4825 |
| 01 Mar 2016 21:34:01 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 207,360 | 307,234 | 1.4816 |
| 01 Mar 2016 09:34:30 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 181,440 | 269,115 | 1.4832 |
| 29 Feb 2016 21:34:05 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 155,520 | 230,898 | 1.4847 |
| 29 Feb 2016 09:54:16 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 129,600 | 192,802 | 1.4877 |
| 28 Feb 2016 22:29:07 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 103,680 | 154,493 | 1.4901 |
| 28 Feb 2016 11:11:11 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 77,760 | 116,726 | 1.5011 |
| 28 Feb 2016 00:12:10 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 51,840 | 79,268 | 1.5291 |
| 27 Feb 2016 12:41:20 | 1376149 | 19314915 | hadcm3n_ldoy_198012_480_350_010332222_0 | 25,920 | 40,455 | 1.5608 |
©2024 cpdn.org