Name | hadcm3n_o37l_1940_40_007683701_0 |
Workunit | 7838788 |
Created | 16 Jan 2012, 7:17:56 UTC |
Sent | 16 Jan 2012, 14:41:32 UTC |
Report deadline | 16 Apr 2012, 22:08:43 UTC |
Received | 1 Feb 2012, 10:13:36 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 193 (0x000000C1) EXIT_SIGNAL |
Computer ID | 1066269 |
Run time | 13 days 8 hours 36 min 18 sec |
CPU time | 9 days 8 hours 24 min 54 sec |
Validate state | Invalid |
Credit | 5,909.76 |
Device peak FLOPS | 2.93 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
Stderr | <core_client_version>6.12.42</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> 16:42:49 (13665): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:43:29 (13877): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:43:30 (13877): No heartbeat from core client for 30 sec - exiting 17:43:31 (13877): No heartbeat from core client for 30 sec - exiting 00:22:55 (14106): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:22:56 (14106): No heartbeat from core client for 30 sec - exiting 00:22:58 (14106): No heartbeat from core client for 30 sec - exiting 00:22:59 (14106): No heartbeat from core client for 30 sec - exiting 00:23:00 (14106): No heartbeat from core client for 30 sec - exiting 00:23:01 (14106): No heartbeat from core client for 30 sec - exiting 01:34:13 (22368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:40:51 (15361): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:40:52 (15361): No heartbeat from core client for 30 sec - exiting 01:40:53 (15361): No heartbeat from core client for 30 sec - exiting 01:40:54 (15361): No heartbeat from core client for 30 sec - exiting 01:40:55 (15361): No heartbeat from core client for 30 sec - exiting 01:40:56 (15361): No heartbeat from core client for 30 sec - exiting 01:55:46 (20929): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:55:47 (20929): No heartbeat from core client for 30 sec - exiting 01:55:48 (20929): No heartbeat from core client for 30 sec - exiting 01:55:49 (20929): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 00:36:26 (16592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:49 (17786): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:50 (17786): No heartbeat from core client for 30 sec - exiting 03:52:08 (18337): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:52:09 (18337): No heartbeat from core client for 30 sec - exiting 03:52:16 (18337): No heartbeat from core client for 30 sec - exiting 03:52:17 (18337): No heartbeat from core client for 30 sec - exiting 16:31:19 (18698): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:20 (18698): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:20:17 (16839): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:20:18 (16839): No heartbeat from core client for 30 sec - exiting 20:20:19 (16839): No heartbeat from core client for 30 sec - exiting 20:20:20 (16839): No heartbeat from core client for 30 sec - exiting 20:20:21 (16839): No heartbeat from core client for 30 sec - exiting 20:20:22 (16839): No heartbeat from core client for 30 sec - exiting 20:20:23 (16839): No heartbeat from core client for 30 sec - exiting 20:20:24 (16839): No heartbeat from core client for 30 sec - exiting 20:20:25 (16839): No heartbeat from core client for 30 sec - exiting 20:20:26 (16839): No heartbeat from core client for 30 sec - exiting 20:20:27 (16839): No heartbeat from core client for 30 sec - exiting 01:33:15 (18520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:33:17 (18520): No heartbeat from core client for 30 sec - exiting 01:33:18 (18520): No heartbeat from core client for 30 sec - exiting 01:33:19 (18520): No heartbeat from core client for 30 sec - exiting 01:33:20 (18520): No heartbeat from core client for 30 sec - exiting 01:33:21 (18520): No heartbeat from core client for 30 sec - exiting 01:33:22 (18520): No heartbeat from core client for 30 sec - exiting 01:39:15 (19691): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:39:16 (19691): No heartbeat from core client for 30 sec - exiting 01:39:17 (19691): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:15:49 (23494): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:15:50 (23494): No heartbeat from core client for 30 sec - exiting 18:15:51 (23494): No heartbeat from core client for 30 sec - exiting 18:15:52 (23494): No heartbeat from core client for 30 sec - exiting 18:15:53 (23494): No heartbeat from core client for 30 sec - exiting 18:15:54 (23494): No heartbeat from core client for 30 sec - exiting 18:15:55 (23494): No heartbeat from core client for 30 sec - exiting 18:15:56 (23494): No heartbeat from core client for 30 sec - exiting 18:15:57 (23494): No heartbeat from core client for 30 sec - exiting 18:15:58 (23494): No heartbeat from core client for 30 sec - exiting 18:15:59 (23494): No heartbeat from core client for 30 sec - exiting 18:16:00 (23494): No heartbeat from core client for 30 sec - exiting 18:16:01 (23494): No heartbeat from core client for 30 sec - exiting 18:16:02 (23494): No heartbeat from core client for 30 sec - exiting 18:16:03 (23494): No heartbeat from core client for 30 sec - exiting 18:16:04 (23494): No heartbeat from core client for 30 sec - exiting 18:27:42 (24179): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:42 (3298): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:43 (3298): No heartbeat from core client for 30 sec - exiting 18:30:44 (3298): No heartbeat from core client for 30 sec - exiting 18:30:45 (3298): No heartbeat from core client for 30 sec - exiting 18:30:46 (3298): No heartbeat from core client for 30 sec - exiting 18:30:47 (3298): No heartbeat from core client for 30 sec - exiting 18:30:48 (3298): No heartbeat from core client for 30 sec - exiting 18:30:49 (3298): No heartbeat from core client for 30 sec - exiting 18:30:50 (3298): No heartbeat from core client for 30 sec - exiting 18:30:51 (3298): No heartbeat from core client for 30 sec - exiting 18:30:52 (3298): No heartbeat from core client for 30 sec - exiting 18:30:53 (3298): No heartbeat from core client for 30 sec - exiting 18:30:54 (3298): No heartbeat from core client for 30 sec - exiting 18:30:55 (3298): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 01:19:47 (8583): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:05:33 (12504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:05:35 (12504): No heartbeat from core client for 30 sec - exiting 07:05:36 (12504): No heartbeat from core client for 30 sec - exiting 07:05:37 (12504): No heartbeat from core client for 30 sec - exiting 07:05:38 (12504): No heartbeat from core client for 30 sec - exiting 07:05:39 (12504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:21:48 (30296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:49 (30296): No heartbeat from core client for 30 sec - exiting 22:21:50 (30296): No heartbeat from core client for 30 sec - exiting 22:21:51 (30296): No heartbeat from core client for 30 sec - exiting 22:21:52 (30296): No heartbeat from core client for 30 sec - exiting 22:21:53 (30296): No heartbeat from core client for 30 sec - exiting 22:21:54 (30296): No heartbeat from core client for 30 sec - exiting 22:21:55 (30296): No heartbeat from core client for 30 sec - exiting 00:10:36 (17710): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:21 (18118): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:24 (18118): No heartbeat from core client for 30 sec - exiting 04:29:04 (19405): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:29:05 (19405): No heartbeat from core client for 30 sec - exiting 04:29:06 (19405): No heartbeat from core client for 30 sec - exiting 04:29:07 (19405): No heartbeat from core client for 30 sec - exiting 13:28:10 (19446): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:11 (19446): No heartbeat from core client for 30 sec - exiting 13:28:12 (19446): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 18:53:58 (21524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:53:59 (21524): No heartbeat from core client for 30 sec - exiting 18:54:00 (21524): No heartbeat from core client for 30 sec - exiting 18:54:01 (21524): No heartbeat from core client for 30 sec - exiting 18:54:02 (21524): No heartbeat from core client for 30 sec - exiting 18:54:03 (21524): No heartbeat from core client for 30 sec - exiting 18:54:04 (21524): No heartbeat from core client for 30 sec - exiting 18:54:05 (21524): No heartbeat from core client for 30 sec - exiting 07:52:16 (22756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:52:17 (22756): No heartbeat from core client for 30 sec - exiting 07:52:18 (22756): No heartbeat from core client for 30 sec - exiting 07:52:19 (22756): No heartbeat from core client for 30 sec - exiting 07:52:20 (22756): No heartbeat from core client for 30 sec - exiting 07:52:21 (22756): No heartbeat from core client for 30 sec - exiting 07:52:22 (22756): No heartbeat from core client for 30 sec - exiting 07:52:23 (22756): No heartbeat from core client for 30 sec - exiting 16:51:52 (25796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:53 (25796): No heartbeat from core client for 30 sec - exiting 16:51:54 (25796): No heartbeat from core client for 30 sec - exiting 16:51:55 (25796): No heartbeat from core client for 30 sec - exiting 16:51:56 (25796): No heartbeat from core client for 30 sec - exiting 16:51:57 (25796): No heartbeat from core client for 30 sec - exiting 16:51:58 (25796): No heartbeat from core client for 30 sec - exiting 16:51:59 (25796): No heartbeat from core client for 30 sec - exiting 16:52:00 (25796): No heartbeat from core client for 30 sec - exiting 16:52:01 (25796): No heartbeat from core client for 30 sec - exiting 16:52:02 (25796): No heartbeat from core client for 30 sec - exiting 16:52:03 (25796): No heartbeat from core client for 30 sec - exiting 16:52:04 (25796): No heartbeat from core client for 30 sec - exiting 13:44:52 (2950): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:44:53 (2950): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 14:41:29 (6568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:41:30 (6568): No heartbeat from core client for 30 sec - exiting 14:41:31 (6568): No heartbeat from core client for 30 sec - exiting 14:41:32 (6568): No heartbeat from core client for 30 sec - exiting 14:41:33 (6568): No heartbeat from core client for 30 sec - exiting 14:41:41 (6568): No heartbeat from core client for 30 sec - exiting 14:41:42 (6568): No heartbeat from core client for 30 sec - exiting 14:41:43 (6568): No heartbeat from core client for 30 sec - exiting 14:41:44 (6568): No heartbeat from core client for 30 sec - exiting 14:41:45 (6568): No heartbeat from core client for 30 sec - exiting 14:41:46 (6568): No heartbeat from core client for 30 sec - exiting 16:56:42 (28145): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:56:43 (28145): No heartbeat from core client for 30 sec - exiting 16:56:44 (28145): No heartbeat from core client for 30 sec - exiting 16:56:45 (28145): No heartbeat from core client for 30 sec - exiting 16:56:46 (28145): No heartbeat from core client for 30 sec - exiting 16:56:47 (28145): No heartbeat from core client for 30 sec - exiting 16:56:48 (28145): No heartbeat from core client for 30 sec - exiting 16:56:49 (28145): No heartbeat from core client for 30 sec - exiting 16:56:50 (28145): No heartbeat from core client for 30 sec - exiting 16:56:59 (28145): No heartbeat from core client for 30 sec - exiting 16:57:00 (28145): No heartbeat from core client for 30 sec - exiting 16:57:01 (28145): No heartbeat from core client for 30 sec - exiting 16:57:02 (28145): No heartbeat from core client for 30 sec - exiting 16:57:03 (28145): No heartbeat from core client for 30 sec - exiting 16:57:04 (28145): No heartbeat from core client for 30 sec - exiting 16:57:10 (28145): No heartbeat from core client for 30 sec - exiting 16:57:11 (28145): No heartbeat from core client for 30 sec - exiting 16:57:12 (28145): No heartbeat from core client for 30 sec - exiting 16:57:13 (28145): No heartbeat from core client for 30 sec - exiting 16:57:15 (28145): No heartbeat from core client for 30 sec - exiting 16:57:16 (28145): No heartbeat from core client for 30 sec - exiting 16:57:17 (28145): No heartbeat from core client for 30 sec - exiting 16:57:18 (28145): No heartbeat from core client for 30 sec - exiting 16:57:19 (28145): No heartbeat from core client for 30 sec - exiting 19:43:43 (29861): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:43:44 (29861): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:12:07 (18417): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:12:08 (18417): No heartbeat from core client for 30 sec - exiting 23:12:09 (18417): No heartbeat from core client for 30 sec - exiting 23:12:10 (18417): No heartbeat from core client for 30 sec - exiting 23:12:11 (18417): No heartbeat from core client for 30 sec - exiting 23:12:12 (18417): No heartbeat from core client for 30 sec - exiting 23:12:13 (18417): No heartbeat from core client for 30 sec - exiting 23:12:14 (18417): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:59:55 (5721): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 12:13:22 (28849): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:13:24 (28849): No heartbeat from core client for 30 sec - exiting 12:13:25 (28849): No heartbeat from core client for 30 sec - exiting 12:13:26 (28849): No heartbeat from core client for 30 sec - exiting 12:13:27 (28849): No heartbeat from core client for 30 sec - exiting 12:13:28 (28849): No heartbeat from core client for 30 sec - exiting 12:13:29 (28849): No heartbeat from core client for 30 sec - exiting 12:13:30 (28849): No heartbeat from core client for 30 sec - exiting 12:13:31 (28849): No heartbeat from core client for 30 sec - exiting 12:13:32 (28849): No heartbeat from core client for 30 sec - exiting 12:13:33 (28849): No heartbeat from core client for 30 sec - exiting 12:13:34 (28849): No heartbeat from core client for 30 sec - exiting 12:13:35 (28849): No heartbeat from core client for 30 sec - exiting 12:13:36 (28849): No heartbeat from core client for 30 sec - exiting 12:13:37 (28849): No heartbeat from core client for 30 sec - exiting 12:13:41 (28849): No heartbeat from core client for 30 sec - exiting 12:13:42 (28849): No heartbeat from core client for 30 sec - exiting 12:13:43 (28849): No heartbeat from core client for 30 sec - exiting 12:13:44 (28849): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:24:55 (3601): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:24:56 (3601): No heartbeat from core client for 30 sec - exiting 16:34:06 (4022): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:07 (4022): No heartbeat from core client for 30 sec - exiting 16:34:08 (4022): No heartbeat from core client for 30 sec - exiting 16:34:09 (4022): No heartbeat from core client for 30 sec - exiting 16:34:10 (4022): No heartbeat from core client for 30 sec - exiting 16:34:11 (4022): No heartbeat from core client for 30 sec - exiting 16:34:12 (4022): No heartbeat from core client for 30 sec - exiting 16:34:13 (4022): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 16:43:19 (4066): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:20 (4066): No heartbeat from core client for 30 sec - exiting 16:43:21 (4066): No heartbeat from core client for 30 sec - exiting 16:43:22 (4066): No heartbeat from core client for 30 sec - exiting 17:13:02 (4128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 21:47:27 (4269): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:47:28 (4269): No heartbeat from core client for 30 sec - exiting 21:47:29 (4269): No heartbeat from core client for 30 sec - exiting 21:47:30 (4269): No heartbeat from core client for 30 sec - exiting 21:47:31 (4269): No heartbeat from core client for 30 sec - exiting 21:47:32 (4269): No heartbeat from core client for 30 sec - exiting 21:47:33 (4269): No heartbeat from core client for 30 sec - exiting 21:47:34 (4269): No heartbeat from core client for 30 sec - exiting 21:47:39 (4269): No heartbeat from core client for 30 sec - exiting 21:47:40 (4269): No heartbeat from core client for 30 sec - exiting 00:06:49 (5377): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:10:54 (5953): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:10:55 (5953): No heartbeat from core client for 30 sec - exiting 01:11:00 (5953): No heartbeat from core client for 30 sec - exiting 01:11:01 (5953): No heartbeat from core client for 30 sec - exiting 01:11:02 (5953): No heartbeat from core client for 30 sec - exiting 01:27:16 (6208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:27:17 (6208): No heartbeat from core client for 30 sec - exiting 01:27:18 (6208): No heartbeat from core client for 30 sec - exiting 01:27:19 (6208): No heartbeat from core client for 30 sec - exiting 04:19:57 (6277): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:19:58 (6277): No heartbeat from core client for 30 sec - exiting 04:19:59 (6277): No heartbeat from core client for 30 sec - exiting 04:20:00 (6277): No heartbeat from core client for 30 sec - exiting 04:20:01 (6277): No heartbeat from core client for 30 sec - exiting 04:31:52 (7098): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:53 (7098): No heartbeat from core client for 30 sec - exiting 04:31:54 (7098): No heartbeat from core client for 30 sec - exiting 04:31:55 (7098): No heartbeat from core client for 30 sec - exiting 04:31:56 (7098): No heartbeat from core client for 30 sec - exiting 05:22:37 (7149): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:38 (7149): No heartbeat from core client for 30 sec - exiting 05:22:39 (7149): No heartbeat from core client for 30 sec - exiting 05:22:40 (7149): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Stack trace (14 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf775f400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x806c0d5] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x806e5f2] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8072509] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8077f47] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x80781a3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e1b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xff)[0xf746c37f] Exiting... </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
31 Jan 2012 14:05:20 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 492,480 | 769,433 | 1.5624 |
30 Jan 2012 16:08:27 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 466,560 | 728,455 | 1.5613 |
29 Jan 2012 19:23:17 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 440,640 | 687,714 | 1.5607 |
28 Jan 2012 22:26:22 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 414,720 | 647,062 | 1.5602 |
27 Jan 2012 20:15:06 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 388,800 | 606,396 | 1.5597 |
27 Jan 2012 04:13:12 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 362,880 | 565,673 | 1.5588 |
26 Jan 2012 10:19:45 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 336,960 | 525,962 | 1.5609 |
25 Jan 2012 17:47:04 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 311,040 | 485,684 | 1.5615 |
25 Jan 2012 01:54:15 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 285,120 | 445,648 | 1.5630 |
24 Jan 2012 09:39:21 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 259,200 | 405,057 | 1.5627 |
23 Jan 2012 17:44:03 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 233,280 | 364,842 | 1.5640 |
23 Jan 2012 01:38:40 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 207,360 | 323,942 | 1.5622 |
22 Jan 2012 02:49:56 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 181,440 | 283,191 | 1.5608 |
21 Jan 2012 01:48:52 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 155,520 | 242,973 | 1.5623 |
20 Jan 2012 05:39:07 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 129,600 | 202,498 | 1.5625 |
19 Jan 2012 11:37:22 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 103,680 | 161,932 | 1.5618 |
18 Jan 2012 19:09:00 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 77,760 | 121,364 | 1.5608 |
18 Jan 2012 01:27:25 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 51,840 | 80,596 | 1.5547 |
17 Jan 2012 07:57:59 | 1066269 | 13927659 | hadcm3n_o37l_1940_40_007683701_0 | 25,920 | 40,438 | 1.5601 |
©2024 cpdn.org