Name | hadcm3n_o3fd_1980_40_008384178_3 |
Workunit | 8535037 |
Created | 5 Jul 2013, 10:21:02 UTC |
Sent | 5 Jul 2013, 10:58:14 UTC |
Report deadline | 4 Oct 2013, 18:25:25 UTC |
Received | 28 Dec 2013, 21:56:01 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Aborted by user |
Exit status | 203 (0x000000CB) EXIT_ABORTED_VIA_GUI |
Computer ID | 1369231 |
Run time | 21 days 10 hours 8 min 48 sec |
CPU time | 11 days 5 hours 9 min 54 sec |
Validate state | Invalid |
Credit | 6,220.80 |
Device peak FLOPS | 2.62 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
Stderr | <core_client_version>7.2.33</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> nt for 30 sec - exiting 21:17:05 (6006): No heartbeat from core client for 30 sec - exiting 21:17:06 (6006): No heartbeat from core client for 30 sec - exiting 21:17:07 (6006): No heartbeat from core client for 30 sec - exiting 21:17:08 (6006): No heartbeat from core client for 30 sec - exiting 21:17:09 (6006): No heartbeat from core client for 30 sec - exiting 21:17:10 (6006): No heartbeat from core client for 30 sec - exiting 21:17:11 (6006): No heartbeat from core client for 30 sec - exiting 21:17:12 (6006): No heartbeat from core client for 30 sec - exiting 21:17:13 (6006): No heartbeat from core client for 30 sec - exiting 21:17:14 (6006): No heartbeat from core client for 30 sec - exiting 21:17:15 (6006): No heartbeat from core client for 30 sec - exiting 02:21:08 (6006): No heartbeat from core client for 30 sec - exiting 02:21:09 (6006): No heartbeat from core client for 30 sec - exiting 02:21:10 (6006): No heartbeat from core client for 30 sec - exiting 02:21:11 (6006): No heartbeat from core client for 30 sec - exiting 02:21:12 (6006): No heartbeat from core client for 30 sec - exiting 02:21:13 (6006): No heartbeat from core client for 30 sec - exiting 02:21:14 (6006): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0895b4e8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:42:08 (3377): No heartbeat from core client for 30 sec - exiting 09:42:09 (3377): No heartbeat from core client for 30 sec - exiting 09:42:10 (3377): No heartbeat from core client for 30 sec - exiting 09:42:11 (3377): No heartbeat from core client for 30 sec - exiting 21:33:58 (3377): No heartbeat from core client for 30 sec - exiting 21:33:59 (3377): No heartbeat from core client for 30 sec - exiting 21:34:00 (3377): No heartbeat from core client for 30 sec - exiting 21:34:01 (3377): No heartbeat from core client for 30 sec - exiting 21:34:02 (3377): No heartbeat from core client for 30 sec - exiting 21:34:03 (3377): No heartbeat from core client for 30 sec - exiting 02:29:28 (3377): No heartbeat from core client for 30 sec - exiting 02:29:29 (3377): No heartbeat from core client for 30 sec - exiting 02:29:30 (3377): No heartbeat from core client for 30 sec - exiting 02:29:31 (3377): No heartbeat from core client for 30 sec - exiting 02:29:32 (3377): No heartbeat from core client for 30 sec - exiting 02:29:33 (3377): No heartbeat from core client for 30 sec - exiting 02:29:34 (3377): No heartbeat from core client for 30 sec - exiting 02:29:35 (3377): No heartbeat from core client for 30 sec - exiting 20:53:13 (3377): No heartbeat from core client for 30 sec - exiting 20:53:14 (3377): No heartbeat from core client for 30 sec - exiting 20:53:15 (3377): No heartbeat from core client for 30 sec - exiting 20:53:16 (3377): No heartbeat from core client for 30 sec - exiting 20:53:17 (3377): No heartbeat from core client for 30 sec - exiting 20:53:18 (3377): No heartbeat from core client for 30 sec - exiting 20:53:19 (3377): No heartbeat from core client for 30 sec - exiting 20:53:20 (3377): No heartbeat from core client for 30 sec - exiting 19:37:13 (3377): No heartbeat from core client for 30 sec - exiting 19:37:14 (3377): No heartbeat from core client for 30 sec - exiting 19:37:15 (3377): No heartbeat from core client for 30 sec - exiting 19:37:16 (3377): No heartbeat from core client for 30 sec - exiting 19:37:17 (3377): No heartbeat from core client for 30 sec - exiting 19:37:18 (3377): No heartbeat from core client for 30 sec - exiting 19:37:19 (3377): No heartbeat from core client for 30 sec - exiting 19:37:20 (3377): No heartbeat from core client for 30 sec - exiting 19:37:21 (3377): No heartbeat from core client for 30 sec - exiting 19:37:22 (3377): No heartbeat from core client for 30 sec - exiting 19:37:23 (3377): No heartbeat from core client for 30 sec - exiting 19:37:24 (3377): No heartbeat from core client for 30 sec - exiting 19:37:25 (3377): No heartbeat from core client for 30 sec - exiting 19:37:26 (3377): No heartbeat from core client for 30 sec - exiting 19:37:27 (3377): No heartbeat from core client for 30 sec - exiting 19:37:28 (3377): No heartbeat from core client for 30 sec - exiting 19:37:29 (3377): No heartbeat from core client for 30 sec - exiting 19:37:30 (3377): No heartbeat from core client for 30 sec - exiting 19:37:31 (3377): No heartbeat from core client for 30 sec - exiting 19:37:32 (3377): No heartbeat from core client for 30 sec - exiting 19:37:33 (3377): No heartbeat from core client for 30 sec - exiting 19:37:34 (3377): No heartbeat from core client for 30 sec - exiting 19:37:35 (3377): No heartbeat from core client for 30 sec - exiting 23:56:48 (3377): No heartbeat from core client for 30 sec - exiting 23:56:49 (3377): No heartbeat from core client for 30 sec - exiting 23:56:50 (3377): No heartbeat from core client for 30 sec - exiting 23:56:51 (3377): No heartbeat from core client for 30 sec - exiting 23:56:52 (3377): No heartbeat from core client for 30 sec - exiting 23:56:53 (3377): No heartbeat from core client for 30 sec - exiting 23:56:54 (3377): No heartbeat from core client for 30 sec - exiting 23:56:55 (3377): No heartbeat from core client for 30 sec - exiting 23:56:56 (3377): No heartbeat from core client for 30 sec - exiting 21:18:26 (3377): No heartbeat from core client for 30 sec - exiting 21:18:27 (3377): No heartbeat from core client for 30 sec - exiting 21:18:28 (3377): No heartbeat from core client for 30 sec - exiting 21:18:29 (3377): No heartbeat from core client for 30 sec - exiting 21:18:30 (3377): No heartbeat from core client for 30 sec - exiting 21:18:31 (3377): No heartbeat from core client for 30 sec - exiting 21:18:32 (3377): No heartbeat from core client for 30 sec - exiting 16:56:35 (3377): No heartbeat from core client for 30 sec - exiting 16:56:36 (3377): No heartbeat from core client for 30 sec - exiting 16:56:37 (3377): No heartbeat from core client for 30 sec - exiting 16:56:38 (3377): No heartbeat from core client for 30 sec - exiting 16:56:39 (3377): No heartbeat from core client for 30 sec - exiting 16:56:40 (3377): No heartbeat from core client for 30 sec - exiting 16:56:41 (3377): No heartbeat from core client for 30 sec - exiting 16:56:42 (3377): No heartbeat from core client for 30 sec - exiting 05:19:35 (3377): No heartbeat from core client for 30 sec - exiting 05:19:36 (3377): No heartbeat from core client for 30 sec - exiting 05:19:37 (3377): No heartbeat from core client for 30 sec - exiting 05:19:38 (3377): No heartbeat from core client for 30 sec - exiting 05:19:39 (3377): No heartbeat from core client for 30 sec - exiting 05:19:40 (3377): No heartbeat from core client for 30 sec - exiting 05:19:41 (3377): No heartbeat from core client for 30 sec - exiting 02:21:41 (3377): No heartbeat from core client for 30 sec - exiting 02:21:43 (3377): No heartbeat from core client for 30 sec - exiting 02:21:44 (3377): No heartbeat from core client for 30 sec - exiting 02:21:45 (3377): No heartbeat from core client for 30 sec - exiting 02:21:46 (3377): No heartbeat from core client for 30 sec - exiting 02:21:47 (3377): No heartbeat from core client for 30 sec - exiting 02:21:48 (3377): No heartbeat from core client for 30 sec - exiting 06:13:29 (3377): No heartbeat from core client for 30 sec - exiting 06:13:31 (3377): No heartbeat from core client for 30 sec - exiting 06:13:32 (3377): No heartbeat from core client for 30 sec - exiting 06:13:33 (3377): No heartbeat from core client for 30 sec - exiting 06:13:34 (3377): No heartbeat from core client for 30 sec - exiting 06:13:35 (3377): No heartbeat from core client for 30 sec - exiting 06:13:36 (3377): No heartbeat from core client for 30 sec - exiting 02:49:02 (3377): No heartbeat from core client for 30 sec - exiting 02:49:03 (3377): No heartbeat from core client for 30 sec - exiting 02:49:04 (3377): No heartbeat from core client for 30 sec - exiting 02:49:05 (3377): No heartbeat from core client for 30 sec - exiting 02:49:06 (3377): No heartbeat from core client for 30 sec - exiting 02:49:07 (3377): No heartbeat from core client for 30 sec - exiting 02:49:08 (3377): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x092054e8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 15:50:34 (3683): No heartbeat from core client for 30 sec - exiting 15:50:35 (3683): No heartbeat from core client for 30 sec - exiting 15:50:36 (3683): No heartbeat from core client for 30 sec - exiting 15:50:37 (3683): No heartbeat from core client for 30 sec - exiting 15:50:38 (3683): No heartbeat from core client for 30 sec - exiting 15:50:39 (3683): No heartbeat from core client for 30 sec - exiting 15:50:40 (3683): No heartbeat from core client for 30 sec - exiting 15:50:41 (3683): No heartbeat from core client for 30 sec - exiting 15:50:42 (3683): No heartbeat from core client for 30 sec - exiting 15:50:43 (3683): No heartbeat from core client for 30 sec - exiting 15:50:44 (3683): No heartbeat from core client for 30 sec - exiting 15:50:45 (3683): No heartbeat from core client for 30 sec - exiting 17:16:51 (3683): No heartbeat from core client for 30 sec - exiting 17:16:52 (3683): No heartbeat from core client for 30 sec - exiting 17:16:53 (3683): No heartbeat from core client for 30 sec - exiting 17:16:54 (3683): No heartbeat from core client for 30 sec - exiting 17:16:55 (3683): No heartbeat from core client for 30 sec - exiting 17:16:56 (3683): No heartbeat from core client for 30 sec - exiting 17:16:57 (3683): No heartbeat from core client for 30 sec - exiting 17:16:58 (3683): No heartbeat from core client for 30 sec - exiting 17:16:59 (3683): No heartbeat from core client for 30 sec - exiting 17:17:00 (3683): No heartbeat from core client for 30 sec - exiting 17:17:01 (3683): No heartbeat from core client for 30 sec - exiting 17:17:02 (3683): No heartbeat from core client for 30 sec - exiting 17:17:03 (3683): No heartbeat from core client for 30 sec - exiting 17:17:04 (3683): No heartbeat from core client for 30 sec - exiting 17:17:05 (3683): No heartbeat from core client for 30 sec - exiting 17:17:06 (3683): No heartbeat from core client for 30 sec - exiting 17:17:07 (3683): No heartbeat from core client for 30 sec - exiting 17:17:08 (3683): No heartbeat from core client for 30 sec - exiting 17:17:09 (3683): No heartbeat from core client for 30 sec - exiting 17:17:10 (3683): No heartbeat from core client for 30 sec - exiting 18:11:55 (3683): No heartbeat from core client for 30 sec - exiting 18:11:56 (3683): No heartbeat from core client for 30 sec - exiting 18:11:57 (3683): No heartbeat from core client for 30 sec - exiting 18:11:58 (3683): No heartbeat from core client for 30 sec - exiting 18:11:59 (3683): No heartbeat from core client for 30 sec - exiting 18:12:00 (3683): No heartbeat from core client for 30 sec - exiting 14:18:05 (3683): No heartbeat from core client for 30 sec - exiting 14:18:06 (3683): No heartbeat from core client for 30 sec - exiting 14:18:07 (3683): No heartbeat from core client for 30 sec - exiting 14:18:08 (3683): No heartbeat from core client for 30 sec - exiting 14:18:09 (3683): No heartbeat from core client for 30 sec - exiting 14:18:10 (3683): No heartbeat from core client for 30 sec - exiting 14:18:11 (3683): No heartbeat from core client for 30 sec - exiting 14:18:12 (3683): No heartbeat from core client for 30 sec - exiting 14:18:13 (3683): No heartbeat from core client for 30 sec - exiting 14:18:14 (3683): No heartbeat from core client for 30 sec - exiting 14:18:15 (3683): No heartbeat from core client for 30 sec - exiting 14:18:16 (3683): No heartbeat from core client for 30 sec - exiting 14:18:17 (3683): No heartbeat from core client for 30 sec - exiting 14:18:18 (3683): No heartbeat from core client for 30 sec - exiting 20:18:12 (3683): No heartbeat from core client for 30 sec - exiting 20:18:14 (3683): No heartbeat from core client for 30 sec - exiting 20:18:15 (3683): No heartbeat from core client for 30 sec - exiting 20:18:16 (3683): No heartbeat from core client for 30 sec - exiting 20:18:17 (3683): No heartbeat from core client for 30 sec - exiting 20:18:18 (3683): No heartbeat from core client for 30 sec - exiting 20:18:19 (3683): No heartbeat from core client for 30 sec - exiting 15:12:49 (3683): No heartbeat from core client for 30 sec - exiting 15:12:50 (3683): No heartbeat from core client for 30 sec - exiting 15:12:51 (3683): No heartbeat from core client for 30 sec - exiting 15:12:52 (3683): No heartbeat from core client for 30 sec - exiting 15:12:53 (3683): No heartbeat from core client for 30 sec - exiting 15:12:54 (3683): No heartbeat from core client for 30 sec - exiting 21:06:38 (3683): No heartbeat from core client for 30 sec - exiting 21:06:39 (3683): No heartbeat from core client for 30 sec - exiting 21:06:40 (3683): No heartbeat from core client for 30 sec - exiting 21:06:41 (3683): No heartbeat from core client for 30 sec - exiting 21:06:42 (3683): No heartbeat from core client for 30 sec - exiting 21:06:43 (3683): No heartbeat from core client for 30 sec - exiting 21:06:44 (3683): No heartbeat from core client for 30 sec - exiting 21:06:45 (3683): No heartbeat from core client for 30 sec - exiting 03:20:09 (3683): No heartbeat from core client for 30 sec - exiting 03:20:10 (3683): No heartbeat from core client for 30 sec - exiting 03:20:11 (3683): No heartbeat from core client for 30 sec - exiting 03:20:12 (3683): No heartbeat from core client for 30 sec - exiting 03:20:13 (3683): No heartbeat from core client for 30 sec - exiting 03:20:14 (3683): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x086614e8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 23:09:43 (4053): No heartbeat from core client for 30 sec - exiting 23:09:44 (4053): No heartbeat from core client for 30 sec - exiting 23:09:45 (4053): No heartbeat from core client for 30 sec - exiting 23:09:46 (4053): No heartbeat from core client for 30 sec - exiting 23:09:47 (4053): No heartbeat from core client for 30 sec - exiting 23:09:48 (4053): No heartbeat from core client for 30 sec - exiting 23:09:49 (4053): No heartbeat from core client for 30 sec - exiting 23:09:50 (4053): No heartbeat from core client for 30 sec - exiting 05:24:19 (4053): No heartbeat from core client for 30 sec - exiting 05:24:20 (4053): No heartbeat from core client for 30 sec - exiting 05:24:21 (4053): No heartbeat from core client for 30 sec - exiting 05:24:22 (4053): No heartbeat from core client for 30 sec - exiting 05:24:23 (4053): No heartbeat from core client for 30 sec - exiting 05:24:24 (4053): No heartbeat from core client for 30 sec - exiting 00:22:43 (4053): No heartbeat from core client for 30 sec - exiting 00:22:44 (4053): No heartbeat from core client for 30 sec - exiting 00:22:45 (4053): No heartbeat from core client for 30 sec - exiting 00:22:46 (4053): No heartbeat from core client for 30 sec - exiting 00:22:47 (4053): No heartbeat from core client for 30 sec - exiting 00:22:48 (4053): No heartbeat from core client for 30 sec - exiting 00:22:49 (4053): No heartbeat from core client for 30 sec - exiting 00:22:50 (4053): No heartbeat from core client for 30 sec - exiting 21:39:40 (4053): No heartbeat from core client for 30 sec - exiting 21:39:41 (4053): No heartbeat from core client for 30 sec - exiting 21:39:42 (4053): No heartbeat from core client for 30 sec - exiting 21:39:43 (4053): No heartbeat from core client for 30 sec - exiting 21:39:44 (4053): No heartbeat from core client for 30 sec - exiting 21:39:45 (4053): No heartbeat from core client for 30 sec - exiting 02:08:22 (4053): No heartbeat from core client for 30 sec - exiting 02:08:23 (4053): No heartbeat from core client for 30 sec - exiting 02:08:24 (4053): No heartbeat from core client for 30 sec - exiting 02:08:25 (4053): No heartbeat from core client for 30 sec - exiting 02:08:26 (4053): No heartbeat from core client for 30 sec - exiting 02:08:27 (4053): No heartbeat from core client for 30 sec - exiting 02:08:28 (4053): No heartbeat from core client for 30 sec - exiting 02:08:29 (4053): No heartbeat from core client for 30 sec - exiting 11:02:19 (4053): No heartbeat from core client for 30 sec - exiting 11:02:20 (4053): No heartbeat from core client for 30 sec - exiting 11:02:21 (4053): No heartbeat from core client for 30 sec - exiting 11:02:22 (4053): No heartbeat from core client for 30 sec - exiting 11:02:23 (4053): No heartbeat from core client for 30 sec - exiting 11:02:24 (4053): No heartbeat from core client for 30 sec - exiting 11:02:25 (4053): No heartbeat from core client for 30 sec - exiting 01:10:00 (4053): No heartbeat from core client for 30 sec - exiting 01:10:02 (4053): No heartbeat from core client for 30 sec - exiting 01:10:03 (4053): No heartbeat from core client for 30 sec - exiting 01:10:04 (4053): No heartbeat from core client for 30 sec - exiting 01:10:05 (4053): No heartbeat from core client for 30 sec - exiting 01:10:06 (4053): No heartbeat from core client for 30 sec - exiting 01:10:07 (4053): No heartbeat from core client for 30 sec - exiting 01:10:08 (4053): No heartbeat from core client for 30 sec - exiting 20:25:57 (4053): No heartbeat from core client for 30 sec - exiting 20:25:58 (4053): No heartbeat from core client for 30 sec - exiting 20:25:59 (4053): No heartbeat from core client for 30 sec - exiting 20:26:00 (4053): No heartbeat from core client for 30 sec - exiting 20:26:01 (4053): No heartbeat from core client for 30 sec - exiting 20:26:02 (4053): No heartbeat from core client for 30 sec - exiting 20:26:03 (4053): No heartbeat from core client for 30 sec - exiting 20:26:04 (4053): No heartbeat from core client for 30 sec - exiting 20:26:05 (4053): No heartbeat from core client for 30 sec - exiting 20:26:06 (4053): No heartbeat from core client for 30 sec - exiting 20:26:07 (4053): No heartbeat from core client for 30 sec - exiting 20:26:08 (4053): No heartbeat from core client for 30 sec - exiting 20:26:09 (4053): No heartbeat from core client for 30 sec - exiting 20:26:10 (4053): No heartbeat from core client for 30 sec - exiting 20:26:11 (4053): No heartbeat from core client for 30 sec - exiting 20:26:12 (4053): No heartbeat from core client for 30 sec - exiting 20:26:13 (4053): No heartbeat from core client for 30 sec - exiting 06:09:13 (4053): No heartbeat from core client for 30 sec - exiting 06:09:14 (4053): No heartbeat from core client for 30 sec - exiting 06:09:15 (4053): No heartbeat from core client for 30 sec - exiting 06:09:16 (4053): No heartbeat from core client for 30 sec - exiting 06:09:17 (4053): No heartbeat from core client for 30 sec - exiting 06:09:18 (4053): No heartbeat from core client for 30 sec - exiting 06:09:19 (4053): No heartbeat from core client for 30 sec - exiting 20:29:28 (4053): No heartbeat from core client for 30 sec - exiting 20:29:30 (4053): No heartbeat from core client for 30 sec - exiting 20:29:31 (4053): No heartbeat from core client for 30 sec - exiting 20:29:32 (4053): No heartbeat from core client for 30 sec - exiting 20:29:33 (4053): No heartbeat from core client for 30 sec - exiting 20:29:34 (4053): No heartbeat from core client for 30 sec - exiting 20:29:35 (4053): No heartbeat from core client for 30 sec - exiting 20:29:36 (4053): No heartbeat from core client for 30 sec - exiting 20:29:37 (4053): No heartbeat from core client for 30 sec - exiting 20:29:38 (4053): No heartbeat from core client for 30 sec - exiting 20:29:39 (4053): No heartbeat from core client for 30 sec - exiting 20:29:40 (4053): No heartbeat from core client for 30 sec - exiting 20:29:41 (4053): No heartbeat from core client for 30 sec - exiting 20:29:42 (4053): No heartbeat from core client for 30 sec - exiting 20:29:43 (4053): No heartbeat from core client for 30 sec - exiting 20:29:44 (4053): No heartbeat from core client for 30 sec - exiting 20:29:45 (4053): No heartbeat from core client for 30 sec - exiting 20:29:46 (4053): No heartbeat from core client for 30 sec - exiting 20:29:47 (4053): No heartbeat from core client for 30 sec - exiting 20:29:48 (4053): No heartbeat from core client for 30 sec - exiting 20:29:49 (4053): No heartbeat from core client for 30 sec - exiting 20:29:50 (4053): No heartbeat from core client for 30 sec - exiting 20:29:51 (4053): No heartbeat from core client for 30 sec - exiting 20:29:52 (4053): No heartbeat from core client for 30 sec - exiting 20:29:53 (4053): No heartbeat from core client for 30 sec - exiting 20:29:54 (4053): No heartbeat from core client for 30 sec - exiting 20:29:55 (4053): No heartbeat from core client for 30 sec - exiting 20:29:56 (4053): No heartbeat from core client for 30 sec - exiting 20:29:57 (4053): No heartbeat from core client for 30 sec - exiting 20:29:58 (4053): No heartbeat from core client for 30 sec - exiting 20:29:59 (4053): No heartbeat from core client for 30 sec - exiting 20:30:00 (4053): No heartbeat from core client for 30 sec - exiting 20:30:01 (4053): No heartbeat from core client for 30 sec - exiting 20:30:02 (4053): No heartbeat from core client for 30 sec - exiting 20:30:03 (4053): No heartbeat from core client for 30 sec - exiting 20:30:05 (4053): No heartbeat from core client for 30 sec - exiting 20:30:06 (4053): No heartbeat from core client for 30 sec - exiting 20:30:07 (4053): No heartbeat from core client for 30 sec - exiting 20:30:09 (4053): No heartbeat from core client for 30 sec - exiting 20:30:10 (4053): No heartbeat from core client for 30 sec - exiting 20:30:11 (4053): No heartbeat from core client for 30 sec - exiting 20:30:12 (4053): No heartbeat from core client for 30 sec - exiting 20:30:13 (4053): No heartbeat from core client for 30 sec - exiting 20:30:14 (4053): No heartbeat from core client for 30 sec - exiting 20:30:15 (4053): No heartbeat from core client for 30 sec - exiting 20:30:16 (4053): No heartbeat from core client for 30 sec - exiting 20:30:17 (4053): No heartbeat from core client for 30 sec - exiting 20:30:18 (4053): No heartbeat from core client for 30 sec - exiting 20:30:19 (4053): No heartbeat from core client for 30 sec - exiting 20:30:20 (4053): No heartbeat from core client for 30 sec - exiting 20:30:21 (4053): No heartbeat from core client for 30 sec - exiting 20:30:22 (4053): No heartbeat from core client for 30 sec - exiting 20:30:24 (4053): No heartbeat from core client for 30 sec - exiting 20:30:25 (4053): No heartbeat from core client for 30 sec - exiting 20:30:26 (4053): No heartbeat from core client for 30 sec - exiting 20:30:27 (4053): No heartbeat from core client for 30 sec - exiting 20:30:28 (4053): No heartbeat from core client for 30 sec - exiting 20:30:29 (4053): No heartbeat from core client for 30 sec - exiting 20:30:30 (4053): No heartbeat from core client for 30 sec - exiting 20:30:31 (4053): No heartbeat from core client for 30 sec - exiting 20:30:32 (4053): No heartbeat from core client for 30 sec - exiting 20:30:33 (4053): No heartbeat from core client for 30 sec - exiting 20:30:35 (4053): No heartbeat from core client for 30 sec - exiting 20:30:36 (4053): No heartbeat from core client for 30 sec - exiting 20:30:37 (4053): No heartbeat from core client for 30 sec - exiting 20:30:38 (4053): No heartbeat from core client for 30 sec - exiting 20:30:41 (4053): No heartbeat from core client for 30 sec - exiting 20:30:42 (4053): No heartbeat from core client for 30 sec - exiting 20:30:43 (4053): No heartbeat from core client for 30 sec - exiting 20:30:44 (4053): No heartbeat from core client for 30 sec - exiting 20:30:45 (4053): No heartbeat from core client for 30 sec - exiting 20:30:46 (4053): No heartbeat from core client for 30 sec - exiting 20:30:47 (4053): No heartbeat from core client for 30 sec - exiting 20:30:48 (4053): No heartbeat from core client for 30 sec - exiting 20:30:49 (4053): No heartbeat from core client for 30 sec - exiting 20:30:50 (4053): No heartbeat from core client for 30 sec - exiting 20:30:51 (4053): No heartbeat from core client for 30 sec - exiting 20:30:52 (4053): No heartbeat from core client for 30 sec - exiting 20:30:53 (4053): No heartbeat from core client for 30 sec - exiting 20:30:54 (4053): No heartbeat from core client for 30 sec - exiting 20:30:55 (4053): No heartbeat from core client for 30 sec - exiting 20:30:56 (4053): No heartbeat from core client for 30 sec - exiting 20:30:57 (4053): No heartbeat from core client for 30 sec - exiting 20:30:58 (4053): No heartbeat from core client for 30 sec - exiting 20:30:59 (4053): No heartbeat from core client for 30 sec - exiting 20:31:00 (4053): No heartbeat from core client for 30 sec - exiting 20:31:01 (4053): No heartbeat from core client for 30 sec - exiting 20:31:02 (4053): No heartbeat from core client for 30 sec - exiting 20:31:03 (4053): No heartbeat from core client for 30 sec - exiting 20:31:04 (4053): No heartbeat from core client for 30 sec - exiting 20:31:05 (4053): No heartbeat from core client for 30 sec - exiting 21:35:39 (4053): No heartbeat from core client for 30 sec - exiting 21:35:41 (4053): No heartbeat from core client for 30 sec - exiting 21:35:42 (4053): No heartbeat from core client for 30 sec - exiting 21:35:43 (4053): No heartbeat from core client for 30 sec - exiting 21:35:44 (4053): No heartbeat from core client for 30 sec - exiting 21:35:57 (4053): No heartbeat from core client for 30 sec - exiting 21:35:58 (4053): No heartbeat from core client for 30 sec - exiting 21:35:59 (4053): No heartbeat from core client for 30 sec - exiting 21:36:00 (4053): No heartbeat from core client for 30 sec - exiting 21:36:02 (4053): No heartbeat from core client for 30 sec - exiting 21:36:03 (4053): No heartbeat from core client for 30 sec - exiting 21:36:04 (4053): No heartbeat from core client for 30 sec - exiting 21:36:05 (4053): No heartbeat from core client for 30 sec - exiting 21:36:06 (4053): No heartbeat from core client for 30 sec - exiting 21:36:07 (4053): No heartbeat from core client for 30 sec - exiting 21:36:08 (4053): No heartbeat from core client for 30 sec - exiting 21:36:09 (4053): No heartbeat from core client for 30 sec - exiting 21:36:10 (4053): No heartbeat from core client for 30 sec - exiting 21:36:11 (4053): No heartbeat from core client for 30 sec - exiting 21:36:12 (4053): No heartbeat from core client for 30 sec - exiting 21:36:13 (4053): No heartbeat from core client for 30 sec - exiting 21:36:14 (4053): No heartbeat from core client for 30 sec - exiting 21:36:15 (4053): No heartbeat from core client for 30 sec - exiting 21:36:16 (4053): No heartbeat from core client for 30 sec - exiting 21:36:17 (4053): No heartbeat from core client for 30 sec - exiting 21:36:18 (4053): No heartbeat from core client for 30 sec - exiting 21:36:19 (4053): No heartbeat from core client for 30 sec - exiting 21:36:20 (4053): No heartbeat from core client for 30 sec - exiting 21:36:21 (4053): No heartbeat from core client for 30 sec - exiting 21:36:22 (4053): No heartbeat from core client for 30 sec - exiting 21:36:23 (4053): No heartbeat from core client for 30 sec - exiting 21:36:24 (4053): No heartbeat from core client for 30 sec - exiting 21:36:25 (4053): No heartbeat from core client for 30 sec - exiting 21:36:26 (4053): No heartbeat from core client for 30 sec - exiting 21:36:27 (4053): No heartbeat from core client for 30 sec - exiting 21:36:28 (4053): No heartbeat from core client for 30 sec - exiting 21:36:29 (4053): No heartbeat from core client for 30 sec - exiting 21:36:30 (4053): No heartbeat from core client for 30 sec - exiting 21:36:31 (4053): No heartbeat from core client for 30 sec - exiting 21:36:32 (4053): No heartbeat from core client for 30 sec - exiting 21:36:33 (4053): No heartbeat from core client for 30 sec - exiting 21:36:34 (4053): No heartbeat from core client for 30 sec - exiting 21:36:35 (4053): No heartbeat from core client for 30 sec - exiting 21:36:36 (4053): No heartbeat from core client for 30 sec - exiting 21:36:37 (4053): No heartbeat from core client for 30 sec - exiting 21:36:38 (4053): No heartbeat from core client for 30 sec - exiting 21:36:39 (4053): No heartbeat from core client for 30 sec - exiting 21:36:40 (4053): No heartbeat from core client for 30 sec - exiting 21:36:42 (4053): No heartbeat from core client for 30 sec - exiting 21:36:43 (4053): No heartbeat from core client for 30 sec - exiting 21:36:44 (4053): No heartbeat from core client for 30 sec - exiting 21:36:45 (4053): No heartbeat from core client for 30 sec - exiting 21:36:46 (4053): No heartbeat from core client for 30 sec - exiting 21:36:47 (4053): No heartbeat from core client for 30 sec - exiting 21:36:48 (4053): No heartbeat from core client for 30 sec - exiting 21:36:49 (4053): No heartbeat from core client for 30 sec - exiting 21:36:50 (4053): No heartbeat from core client for 30 sec - exiting 21:36:51 (4053): No heartbeat from core client for 30 sec - exiting 21:36:52 (4053): No heartbeat from core client for 30 sec - exiting 21:36:53 (4053): No heartbeat from core client for 30 sec - exiting 21:36:54 (4053): No heartbeat from core client for 30 sec - exiting 21:36:55 (4053): No heartbeat from core client for 30 sec - exiting 21:36:56 (4053): No heartbeat from core client for 30 sec - exiting 21:36:57 (4053): No heartbeat from core client for 30 sec - exiting 21:36:58 (4053): No heartbeat from core client for 30 sec - exiting 21:36:59 (4053): No heartbeat from core client for 30 sec - exiting 21:37:00 (4053): No heartbeat from core client for 30 sec - exiting 21:37:01 (4053): No heartbeat from core client for 30 sec - exiting 21:37:02 (4053): No heartbeat from core client for 30 sec - exiting 21:37:03 (4053): No heartbeat from core client for 30 sec - exiting 21:37:04 (4053): No heartbeat from core client for 30 sec - exiting 21:37:05 (4053): No heartbeat from core client for 30 sec - exiting 21:37:06 (4053): No heartbeat from core client for 30 sec - exiting 21:37:08 (4053): No heartbeat from core client for 30 sec - exiting 21:37:09 (4053): No heartbeat from core client for 30 sec - exiting 21:37:10 (4053): No heartbeat from core client for 30 sec - exiting 21:37:11 (4053): No heartbeat from core client for 30 sec - exiting 21:37:13 (4053): No heartbeat from core client for 30 sec - exiting 21:37:14 (4053): No heartbeat from core client for 30 sec - exiting 21:37:15 (4053): No heartbeat from core client for 30 sec - exiting 21:37:16 (4053): No heartbeat from core client for 30 sec - exiting 21:37:17 (4053): No heartbeat from core client for 30 sec - exiting 21:37:18 (4053): No heartbeat from core client for 30 sec - exiting 21:37:19 (4053): No heartbeat from core client for 30 sec - exiting 21:37:20 (4053): No heartbeat from core client for 30 sec - exiting 21:37:21 (4053): No heartbeat from core client for 30 sec - exiting 21:37:23 (4053): No heartbeat from core client for 30 sec - exiting 21:37:24 (4053): No heartbeat from core client for 30 sec - exiting 21:37:25 (4053): No heartbeat from core client for 30 sec - exiting 21:37:26 (4053): No heartbeat from core client for 30 sec - exiting 21:37:27 (4053): No heartbeat from core client for 30 sec - exiting 21:37:28 (4053): No heartbeat from core client for 30 sec - exiting 21:37:29 (4053): No heartbeat from core client for 30 sec - exiting 21:37:30 (4053): No heartbeat from core client for 30 sec - exiting 21:37:31 (4053): No heartbeat from core client for 30 sec - exiting 21:37:32 (4053): No heartbeat from core client for 30 sec - exiting 21:37:33 (4053): No heartbeat from core client for 30 sec - exiting 21:37:34 (4053): No heartbeat from core client for 30 sec - exiting 21:37:35 (4053): No heartbeat from core client for 30 sec - exiting 21:37:36 (4053): No heartbeat from core client for 30 sec - exiting 21:37:38 (4053): No heartbeat from core client for 30 sec - exiting 21:37:39 (4053): No heartbeat from core client for 30 sec - exiting 21:37:40 (4053): No heartbeat from core client for 30 sec - exiting 21:37:41 (4053): No heartbeat from core client for 30 sec - exiting 21:37:42 (4053): No heartbeat from core client for 30 sec - exiting 21:37:43 (4053): No heartbeat from core client for 30 sec - exiting 21:37:44 (4053): No heartbeat from core client for 30 sec - exiting 21:37:45 (4053): No heartbeat from core client for 30 sec - exiting 21:37:46 (4053): No heartbeat from core client for 30 sec - exiting 21:37:47 (4053): No heartbeat from core client for 30 sec - exiting 21:37:48 (4053): No heartbeat from core client for 30 sec - exiting 21:37:49 (4053): No heartbeat from core client for 30 sec - exiting 21:37:50 (4053): No heartbeat from core client for 30 sec - exiting 21:37:51 (4053): No heartbeat from core client for 30 sec - exiting 21:37:52 (4053): No heartbeat from core client for 30 sec - exiting 21:37:53 (4053): No heartbeat from core client for 30 sec - exiting 21:37:54 (4053): No heartbeat from core client for 30 sec - exiting 21:37:55 (4053): No heartbeat from core client for 30 sec - exiting 21:37:56 (4053): No heartbeat from core client for 30 sec - exiting 21:37:57 (4053): No heartbeat from core client for 30 sec - exiting 21:37:58 (4053): No heartbeat from core client for 30 sec - exiting 21:37:59 (4053): No heartbeat from core client for 30 sec - exiting 21:38:00 (4053): No heartbeat from core client for 30 sec - exiting 21:38:01 (4053): No heartbeat from core client for 30 sec - exiting 21:38:02 (4053): No heartbeat from core client for 30 sec - exiting 21:38:03 (4053): No heartbeat from core client for 30 sec - exiting 21:38:04 (4053): No heartbeat from core client for 30 sec - exiting 21:38:05 (4053): No heartbeat from core client for 30 sec - exiting 21:38:07 (4053): No heartbeat from core client for 30 sec - exiting 21:38:09 (4053): No heartbeat from core client for 30 sec - exiting 21:38:10 (4053): No heartbeat from core client for 30 sec - exiting 21:38:11 (4053): No heartbeat from core client for 30 sec - exiting 21:38:12 (4053): No heartbeat from core client for 30 sec - exiting 21:38:13 (4053): No heartbeat from core client for 30 sec - exiting 21:38:14 (4053): No heartbeat from core client for 30 sec - exiting 21:38:15 (4053): No heartbeat from core client for 30 sec - exiting 21:38:16 (4053): No heartbeat from core client for 30 sec - exiting 21:38:17 (4053): No heartbeat from core client for 30 sec - exiting 21:38:19 (4053): No heartbeat from core client for 30 sec - exiting 21:38:20 (4053): No heartbeat from core client for 30 sec - exiting 21:38:21 (4053): No heartbeat from core client for 30 sec - exiting 21:38:22 (4053): No heartbeat from core client for 30 sec - exiting 21:38:24 (4053): No heartbeat from core client for 30 sec - exiting 21:38:25 (4053): No heartbeat from core client for 30 sec - exiting 21:38:26 (4053): No heartbeat from core client for 30 sec - exiting 21:38:27 (4053): No heartbeat from core client for 30 sec - exiting 21:38:28 (4053): No heartbeat from core client for 30 sec - exiting 21:38:29 (4053): No heartbeat from core client for 30 sec - exiting 21:38:30 (4053): No heartbeat from core client for 30 sec - exiting 21:38:31 (4053): No heartbeat from core client for 30 sec - exiting 21:38:32 (4053): No heartbeat from core client for 30 sec - exiting 21:38:33 (4053): No heartbeat from core client for 30 sec - exiting 21:38:34 (4053): No heartbeat from core client for 30 sec - exiting 21:38:35 (4053): No heartbeat from core client for 30 sec - exiting 21:38:36 (4053): No heartbeat from core client for 30 sec - exiting 21:38:37 (4053): No heartbeat from core client for 30 sec - exiting 21:38:38 (4053): No heartbeat from core client for 30 sec - exiting 21:38:39 (4053): No heartbeat from core client for 30 sec - exiting 21:38:42 (4053): No heartbeat from core client for 30 sec - exiting 21:38:43 (4053): No heartbeat from core client for 30 sec - exiting 21:38:44 (4053): No heartbeat from core client for 30 sec - exiting 21:38:45 (4053): No heartbeat from core client for 30 sec - exiting 21:38:46 (4053): No heartbeat from core client for 30 sec - exiting 21:38:47 (4053): No heartbeat from core client for 30 sec - exiting 21:38:48 (4053): No heartbeat from core client for 30 sec - exiting 21:38:49 (4053): No heartbeat from core client for 30 sec - exiting 21:38:50 (4053): No heartbeat from core client for 30 sec - exiting 21:38:51 (4053): No heartbeat from core client for 30 sec - exiting 21:38:52 (4053): No heartbeat from core client for 30 sec - exiting 21:38:53 (4053): No heartbeat from core client for 30 sec - exiting 21:38:55 (4053): No heartbeat from core client for 30 sec - exiting 21:38:56 (4053): No heartbeat from core client for 30 sec - exiting 21:38:57 (4053): No heartbeat from core client for 30 sec - exiting 21:38:58 (4053): No heartbeat from core client for 30 sec - exiting 21:38:59 (4053): No heartbeat from core client for 30 sec - exiting 21:39:01 (4053): No heartbeat from core client for 30 sec - exiting 21:39:02 (4053): No heartbeat from core client for 30 sec - exiting 21:39:03 (4053): No heartbeat from core client for 30 sec - exiting 21:39:05 (4053): No heartbeat from core client for 30 sec - exiting 21:39:06 (4053): No heartbeat from core client for 30 sec - exiting 21:39:07 (4053): No heartbeat from core client for 30 sec - exiting 21:39:08 (4053): No heartbeat from core client for 30 sec - exiting 21:39:09 (4053): No heartbeat from core client for 30 sec - exiting 21:39:10 (4053): No heartbeat from core client for 30 sec - exiting 21:39:11 (4053): No heartbeat from core client for 30 sec - exiting 21:39:12 (4053): No heartbeat from core client for 30 sec - exiting 21:39:13 (4053): No heartbeat from core client for 30 sec - exiting 21:39:14 (4053): No heartbeat from core client for 30 sec - exiting 21:39:17 (4053): No heartbeat from core client for 30 sec - exiting 21:39:18 (4053): No heartbeat from core client for 30 sec - exiting 21:39:19 (4053): No heartbeat from core client for 30 sec - exiting 21:39:22 (4053): No heartbeat from core client for 30 sec - exiting 21:39:23 (4053): No heartbeat from core client for 30 sec - exiting 21:39:24 (4053): No heartbeat from core client for 30 sec - exiting 21:39:25 (4053): No heartbeat from core client for 30 sec - exiting 21:39:26 (4053): No heartbeat from core client for 30 sec - exiting 21:39:27 (4053): No heartbeat from core client for 30 sec - exiting 21:39:28 (4053): No heartbeat from core client for 30 sec - exiting 21:39:29 (4053): No heartbeat from core client for 30 sec - exiting 21:39:30 (4053): No heartbeat from core client for 30 sec - exiting 21:39:31 (4053): No heartbeat from core client for 30 sec - exiting 21:39:32 (4053): No heartbeat from core client for 30 sec - exiting 21:39:33 (4053): No heartbeat from core client for 30 sec - exiting 21:39:34 (4053): No heartbeat from core client for 30 sec - exiting 06:58:30 (4053): No heartbeat from core client for 30 sec - exiting 06:58:31 (4053): No heartbeat from core client for 30 sec - exiting 06:58:32 (4053): No heartbeat from core client for 30 sec - exiting 06:58:33 (4053): No heartbeat from core client for 30 sec - exiting 06:58:34 (4053): No heartbeat from core client for 30 sec - exiting 06:58:35 (4053): No heartbeat from core client for 30 sec - exiting 06:58:36 (4053): No heartbeat from core client for 30 sec - exiting 06:58:37 (4053): No heartbeat from core client for 30 sec - exiting 06:58:38 (4053): No heartbeat from core client for 30 sec - exiting 06:58:39 (4053): No heartbeat from core client for 30 sec - exiting 01:18:57 (4053): No heartbeat from core client for 30 sec - exiting 01:18:58 (4053): No heartbeat from core client for 30 sec - exiting 01:18:59 (4053): No heartbeat from core client for 30 sec - exiting 01:19:00 (4053): No heartbeat from core client for 30 sec - exiting 01:19:01 (4053): No heartbeat from core client for 30 sec - exiting 01:19:02 (4053): No heartbeat from core client for 30 sec - exiting 01:19:03 (4053): No heartbeat from core client for 30 sec - exiting 01:19:04 (4053): No heartbeat from core client for 30 sec - exiting 10:53:27 (4053): No heartbeat from core client for 30 sec - exiting 10:53:28 (4053): No heartbeat from core client for 30 sec - exiting 10:53:29 (4053): No heartbeat from core client for 30 sec - exiting 10:53:30 (4053): No heartbeat from core client for 30 sec - exiting 10:53:31 (4053): No heartbeat from core client for 30 sec - exiting 10:53:32 (4053): No heartbeat from core client for 30 sec - exiting 17:43:31 (4053): No heartbeat from core client for 30 sec - exiting 17:43:32 (4053): No heartbeat from core client for 30 sec - exiting 17:43:33 (4053): No heartbeat from core client for 30 sec - exiting 17:43:34 (4053): No heartbeat from core client for 30 sec - exiting 17:43:35 (4053): No heartbeat from core client for 30 sec - exiting 17:43:36 (4053): No heartbeat from core client for 30 sec - exiting 17:43:37 (4053): No heartbeat from core client for 30 sec - exiting 17:43:38 (4053): No heartbeat from core client for 30 sec - exiting 11:11:41 (4053): No heartbeat from core client for 30 sec - exiting 11:11:42 (4053): No heartbeat from core client for 30 sec - exiting 11:11:43 (4053): No heartbeat from core client for 30 sec - exiting 11:11:44 (4053): No heartbeat from core client for 30 sec - exiting 11:11:45 (4053): No heartbeat from core client for 30 sec - exiting 11:11:46 (4053): No heartbeat from core client for 30 sec - exiting 01:02:24 (4053): No heartbeat from core client for 30 sec - exiting 01:02:25 (4053): No heartbeat from core client for 30 sec - exiting 01:02:26 (4053): No heartbeat from core client for 30 sec - exiting 01:02:27 (4053): No heartbeat from core client for 30 sec - exiting 01:02:28 (4053): No heartbeat from core client for 30 sec - exiting 01:02:29 (4053): No heartbeat from core client for 30 sec - exiting 01:02:30 (4053): No heartbeat from core client for 30 sec - exiting 01:02:31 (4053): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08c0c4e8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 18:00:02 (8984): No heartbeat from core client for 30 sec - exiting 18:00:03 (8984): No heartbeat from core client for 30 sec - exiting 18:00:04 (8984): No heartbeat from core client for 30 sec - exiting 18:00:05 (8984): No heartbeat from core client for 30 sec - exiting 18:00:06 (8984): No heartbeat from core client for 30 sec - exiting 18:00:07 (8984): No heartbeat from core client for 30 sec - exiting 22:34:03 (8984): No heartbeat from core client for 30 sec - exiting 22:34:04 (8984): No heartbeat from core client for 30 sec - exiting 22:34:05 (8984): No heartbeat from core client for 30 sec - exiting 22:34:06 (8984): No heartbeat from core client for 30 sec - exiting 22:34:07 (8984): No heartbeat from core client for 30 sec - exiting 22:34:08 (8984): No heartbeat from core client for 30 sec - exiting 22:34:09 (8984): No heartbeat from core client for 30 sec - exiting 22:34:10 (8984): No heartbeat from core client for 30 sec - exiting 14:48:35 (8984): No heartbeat from core client for 30 sec - exiting 14:48:36 (8984): No heartbeat from core client for 30 sec - exiting 14:48:37 (8984): No heartbeat from core client for 30 sec - exiting 14:48:38 (8984): No heartbeat from core client for 30 sec - exiting 14:48:39 (8984): No heartbeat from core client for 30 sec - exiting 14:48:40 (8984): No heartbeat from core client for 30 sec - exiting 14:48:41 (8984): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0970d4e8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 00:59:57 (5575): No heartbeat from core client for 30 sec - exiting 00:59:58 (5575): No heartbeat from core client for 30 sec - exiting 00:59:59 (5575): No heartbeat from core client for 30 sec - exiting 01:00:00 (5575): No heartbeat from core client for 30 sec - exiting 01:00:01 (5575): No heartbeat from core client for 30 sec - exiting 01:00:02 (5575): No heartbeat from core client for 30 sec - exiting 01:00:03 (5575): No heartbeat from core client for 30 sec - exiting 01:00:04 (5575): No heartbeat from core client for 30 sec - exiting 19:39:32 (5575): No heartbeat from core client for 30 sec - exiting 19:39:33 (5575): No heartbeat from core client for 30 sec - exiting 19:39:34 (5575): No heartbeat from core client for 30 sec - exiting 19:39:35 (5575): No heartbeat from core client for 30 sec - exiting 19:39:36 (5575): No heartbeat from core client for 30 sec - exiting 19:39:37 (5575): No heartbeat from core client for 30 sec - exiting 00:35:23 (5575): No heartbeat from core client for 30 sec - exiting 00:35:24 (5575): No heartbeat from core client for 30 sec - exiting 00:35:25 (5575): No heartbeat from core client for 30 sec - exiting 00:35:26 (5575): No heartbeat from core client for 30 sec - exiting 00:35:27 (5575): No heartbeat from core client for 30 sec - exiting 00:35:28 (5575): No heartbeat from core client for 30 sec - exiting 00:35:29 (5575): No heartbeat from core client for 30 sec - exiting 00:35:30 (5575): No heartbeat from core client for 30 sec - exiting 09:44:55 (5575): No heartbeat from core client for 30 sec - exiting 09:44:56 (5575): No heartbeat from core client for 30 sec - exiting 09:44:57 (5575): No heartbeat from core client for 30 sec - exiting 09:44:58 (5575): No heartbeat from core client for 30 sec - exiting 09:44:59 (5575): No heartbeat from core client for 30 sec - exiting 09:45:00 (5575): No heartbeat from core client for 30 sec - exiting 09:45:01 (5575): No heartbeat from core client for 30 sec - exiting 09:45:02 (5575): No heartbeat from core client for 30 sec - exiting 01:15:24 (5575): No heartbeat from core client for 30 sec - exiting 01:15:25 (5575): No heartbeat from core client for 30 sec - exiting 01:15:26 (5575): No heartbeat from core client for 30 sec - exiting 01:15:27 (5575): No heartbeat from core client for 30 sec - exiting 01:15:28 (5575): No heartbeat from core client for 30 sec - exiting 01:15:29 (5575): No heartbeat from core client for 30 sec - exiting 01:15:30 (5575): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x094c04e8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 22:46:09 (3505): No heartbeat from core client for 30 sec - exiting 22:46:10 (3505): No heartbeat from core client for 30 sec - exiting 22:46:11 (3505): No heartbeat from core client for 30 sec - exiting 22:46:12 (3505): No heartbeat from core client for 30 sec - exiting 22:46:13 (3505): No heartbeat from core client for 30 sec - exiting 22:46:14 (3505): No heartbeat from core client for 30 sec - exiting 22:46:15 (3505): No heartbeat from core client for 30 sec - exiting 22:46:16 (3505): No heartbeat from core client for 30 sec - exiting 22:46:17 (3505): No heartbeat from core client for 30 sec - exiting 22:46:18 (3505): No heartbeat from core client for 30 sec - exiting 22:46:19 (3505): No heartbeat from core client for 30 sec - exiting 22:46:20 (3505): No heartbeat from core client for 30 sec - exiting 22:46:21 (3505): No heartbeat from core client for 30 sec - exiting 22:46:22 (3505): No heartbeat from core client for 30 sec - exiting 22:46:23 (3505): No heartbeat from core client for 30 sec - exiting 22:46:24 (3505): No heartbeat from core client for 30 sec - exiting 22:46:25 (3505): No heartbeat from core client for 30 sec - exiting 22:46:26 (3505): No heartbeat from core client for 30 sec - exiting 22:46:27 (3505): No heartbeat from core client for 30 sec - exiting 22:46:28 (3505): No heartbeat from core client for 30 sec - exiting 22:46:29 (3505): No heartbeat from core client for 30 sec - exiting 22:46:30 (3505): No heartbeat from core client for 30 sec - exiting 22:46:31 (3505): No heartbeat from core client for 30 sec - exiting 23:03:44 (3505): No heartbeat from core client for 30 sec - exiting 23:03:45 (3505): No heartbeat from core client for 30 sec - exiting 23:03:46 (3505): No heartbeat from core client for 30 sec - exiting 23:03:47 (3505): No heartbeat from core client for 30 sec - exiting 23:03:48 (3505): No heartbeat from core client for 30 sec - exiting 23:03:49 (3505): No heartbeat from core client for 30 sec - exiting 23:03:50 (3505): No heartbeat from core client for 30 sec - exiting 23:03:51 (3505): No heartbeat from core client for 30 sec - exiting 10:22:33 (3505): No heartbeat from core client for 30 sec - exiting 03:30:18 (3505): No heartbeat from core client for 30 sec - exiting 03:30:19 (3505): No heartbeat from core client for 30 sec - exiting 03:30:20 (3505): No heartbeat from core client for 30 sec - exiting 19:34:29 (3505): No heartbeat from core client for 30 sec - exiting 19:34:30 (3505): No heartbeat from core client for 30 sec - exiting 19:34:31 (3505): No heartbeat from core client for 30 sec - exiting 19:34:32 (3505): No heartbeat from core client for 30 sec - exiting 19:34:33 (3505): No heartbeat from core client for 30 sec - exiting 19:34:34 (3505): No heartbeat from core client for 30 sec - exiting 19:34:35 (3505): No heartbeat from core client for 30 sec - exiting 23:30:56 (3505): No heartbeat from core client for 30 sec - exiting 23:30:57 (3505): No heartbeat from core client for 30 sec - exiting 23:30:58 (3505): No heartbeat from core client for 30 sec - exiting 23:30:59 (3505): No heartbeat from core client for 30 sec - exiting 23:31:00 (3505): No heartbeat from core client for 30 sec - exiting 23:31:01 (3505): No heartbeat from core client for 30 sec - exiting 23:31:02 (3505): No heartbeat from core client for 30 sec - exiting 23:31:03 (3505): No heartbeat from core client for 30 sec - exiting 23:31:04 (3505): No heartbeat from core client for 30 sec - exiting 23:31:05 (3505): No heartbeat from core client for 30 sec - exiting 23:31:06 (3505): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x089224e8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 03:47:30 (13505): No heartbeat from core client for 30 sec - exiting 03:47:31 (13505): No heartbeat from core client for 30 sec - exiting 03:47:32 (13505): No heartbeat from core client for 30 sec - exiting 03:47:33 (13505): No heartbeat from core client for 30 sec - exiting 03:47:34 (13505): No heartbeat from core client for 30 sec - exiting 03:47:35 (13505): No heartbeat from core client for 30 sec - exiting 03:47:36 (13505): No heartbeat from core client for 30 sec - exiting 09:53:04 (13505): No heartbeat from core client for 30 sec - exiting 09:53:05 (13505): No heartbeat from core client for 30 sec - exiting 09:53:06 (13505): No heartbeat from core client for 30 sec - exiting 09:53:07 (13505): No heartbeat from core client for 30 sec - exiting 09:53:08 (13505): No heartbeat from core client for 30 sec - exiting 09:53:09 (13505): No heartbeat from core client for 30 sec - exiting 09:53:10 (13505): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x094a84e8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 23:54:08 (11134): No heartbeat from core client for 30 sec - exiting 23:54:09 (11134): No heartbeat from core client for 30 sec - exiting 23:54:10 (11134): No heartbeat from core client for 30 sec - exiting 23:54:11 (11134): No heartbeat from core client for 30 sec - exiting 23:54:12 (11134): No heartbeat from core client for 30 sec - exiting 23:54:13 (11134): No heartbeat from core client for 30 sec - exiting 23:54:14 (11134): No heartbeat from core client for 30 sec - exiting 18:58:15 (11134): No heartbeat from core client for 30 sec - exiting 18:58:16 (11134): No heartbeat from core client for 30 sec - exiting 18:58:17 (11134): No heartbeat from core client for 30 sec - exiting 18:58:18 (11134): No heartbeat from core client for 30 sec - exiting 18:58:19 (11134): No heartbeat from core client for 30 sec - exiting 00:10:50 (11134): No heartbeat from core client for 30 sec - exiting 00:10:51 (11134): No heartbeat from core client for 30 sec - exiting 00:10:52 (11134): No heartbeat from core client for 30 sec - exiting 00:10:53 (11134): No heartbeat from core client for 30 sec - exiting 00:10:54 (11134): No heartbeat from core client for 30 sec - exiting 00:10:55 (11134): No heartbeat from core client for 30 sec - exiting 00:59:20 (11134): No heartbeat from core client for 30 sec - exiting 00:59:21 (11134): No heartbeat from core client for 30 sec - exiting 00:59:22 (11134): No heartbeat from core client for 30 sec - exiting 00:59:23 (11134): No heartbeat from core client for 30 sec - exiting 00:59:24 (11134): No heartbeat from core client for 30 sec - exiting 00:59:25 (11134): No heartbeat from core client for 30 sec - exiting 00:59:26 (11134): No heartbeat from core client for 30 sec - exiting 00:59:27 (11134): No heartbeat from core client for 30 sec - exiting 23:58:31 (11134): No heartbeat from core client for 30 sec - exiting 23:58:33 (11134): No heartbeat from core client for 30 sec - exiting 23:58:34 (11134): No heartbeat from core client for 30 sec - exiting 23:58:35 (11134): No heartbeat from core client for 30 sec - exiting 23:58:36 (11134): No heartbeat from core client for 30 sec - exiting 23:58:37 (11134): No heartbeat from core client for 30 sec - exiting 23:58:38 (11134): No heartbeat from core client for 30 sec - exiting 23:58:39 (11134): No heartbeat from core client for 30 sec - exiting 01:01:17 (11134): No heartbeat from core client for 30 sec - exiting 01:01:18 (11134): No heartbeat from core client for 30 sec - exiting 01:01:19 (11134): No heartbeat from core client for 30 sec - exiting 01:01:20 (11134): No heartbeat from core client for 30 sec - exiting 01:01:21 (11134): No heartbeat from core client for 30 sec - exiting 01:01:22 (11134): No heartbeat from core client for 30 sec - exiting 01:01:23 (11134): No heartbeat from core client for 30 sec - exiting 01:01:24 (11134): No heartbeat from core client for 30 sec - exiting 19:52:51 (11134): No heartbeat from core client for 30 sec - exiting 19:52:52 (11134): No heartbeat from core client for 30 sec - exiting 19:52:53 (11134): No heartbeat from core client for 30 sec - exiting 19:52:54 (11134): No heartbeat from core client for 30 sec - exiting 19:52:55 (11134): No heartbeat from core client for 30 sec - exiting 19:52:56 (11134): No heartbeat from core client for 30 sec - exiting 19:52:57 (11134): No heartbeat from core client for 30 sec - exiting 19:52:59 (11134): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x091de4d0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 20:37:48 (3474): No heartbeat from core client for 30 sec - exiting 20:37:49 (3474): No heartbeat from core client for 30 sec - exiting 20:37:50 (3474): No heartbeat from core client for 30 sec - exiting 20:37:51 (3474): No heartbeat from core client for 30 sec - exiting 20:37:52 (3474): No heartbeat from core client for 30 sec - exiting 20:37:53 (3474): No heartbeat from core client for 30 sec - exiting 19:55:28 (3474): No heartbeat from core client for 30 sec - exiting 19:55:29 (3474): No heartbeat from core client for 30 sec - exiting 19:55:30 (3474): No heartbeat from core client for 30 sec - exiting 19:55:31 (3474): No heartbeat from core client for 30 sec - exiting 19:55:32 (3474): No heartbeat from core client for 30 sec - exiting 19:55:33 (3474): No heartbeat from core client for 30 sec - exiting 19:55:34 (3474): No heartbeat from core client for 30 sec - exiting 19:55:35 (3474): No heartbeat from core client for 30 sec - exiting 00:38:53 (3474): No heartbeat from core client for 30 sec - exiting 00:38:54 (3474): No heartbeat from core client for 30 sec - exiting 00:38:55 (3474): No heartbeat from core client for 30 sec - exiting 00:38:56 (3474): No heartbeat from core client for 30 sec - exiting 00:38:57 (3474): No heartbeat from core client for 30 sec - exiting 00:38:58 (3474): No heartbeat from core client for 30 sec - exiting 00:38:59 (3474): No heartbeat from core client for 30 sec - exiting 19:14:11 (3474): No heartbeat from core client for 30 sec - exiting 19:14:12 (3474): No heartbeat from core client for 30 sec - exiting 19:14:13 (3474): No heartbeat from core client for 30 sec - exiting 19:14:14 (3474): No heartbeat from core client for 30 sec - exiting 19:14:15 (3474): No heartbeat from core client for 30 sec - exiting 19:14:16 (3474): No heartbeat from core client for 30 sec - exiting 19:14:17 (3474): No heartbeat from core client for 30 sec - exiting 19:14:18 (3474): No heartbeat from core client for 30 sec - exiting 19:14:19 (3474): No heartbeat from core client for 30 sec - exiting 09:30:46 (3474): No heartbeat from core client for 30 sec - exiting 09:30:47 (3474): No heartbeat from core client for 30 sec - exiting 09:30:48 (3474): No heartbeat from core client for 30 sec - exiting 09:30:49 (3474): No heartbeat from core client for 30 sec - exiting 09:30:50 (3474): No heartbeat from core client for 30 sec - exiting 09:30:51 (3474): No heartbeat from core client for 30 sec - exiting 09:30:52 (3474): No heartbeat from core client for 30 sec - exiting 09:30:53 (3474): No heartbeat from core client for 30 sec - exiting 09:30:54 (3474): No heartbeat from core client for 30 sec - exiting 09:30:55 (3474): No heartbeat from core client for 30 sec - exiting 09:30:56 (3474): No heartbeat from core client for 30 sec - exiting 09:30:57 (3474): No heartbeat from core client for 30 sec - exiting 09:30:58 (3474): No heartbeat from core client for 30 sec - exiting 09:30:59 (3474): No heartbeat from core client for 30 sec - exiting 09:31:00 (3474): No heartbeat from core client for 30 sec - exiting 00:09:03 (3474): No heartbeat from core client for 30 sec - exiting 00:09:04 (3474): No heartbeat from core client for 30 sec - exiting 00:09:05 (3474): No heartbeat from core client for 30 sec - exiting 00:09:06 (3474): No heartbeat from core client for 30 sec - exiting 00:09:07 (3474): No heartbeat from core client for 30 sec - exiting 00:09:08 (3474): No heartbeat from core client for 30 sec - exiting 00:09:09 (3474): No heartbeat from core client for 30 sec - exiting 00:09:10 (3474): No heartbeat from core client for 30 sec - exiting 01:56:34 (3474): No heartbeat from core client for 30 sec - exiting 01:56:35 (3474): No heartbeat from core client for 30 sec - exiting 01:56:36 (3474): No heartbeat from core client for 30 sec - exiting 01:56:37 (3474): No heartbeat from core client for 30 sec - exiting 01:56:38 (3474): No heartbeat from core client for 30 sec - exiting 01:56:39 (3474): No heartbeat from core client for 30 sec - exiting 01:56:40 (3474): No heartbeat from core client for 30 sec - exiting 21:44:20 (3474): No heartbeat from core client for 30 sec - exiting 21:44:21 (3474): No heartbeat from core client for 30 sec - exiting 21:44:22 (3474): No heartbeat from core client for 30 sec - exiting 21:44:23 (3474): No heartbeat from core client for 30 sec - exiting 21:44:24 (3474): No heartbeat from core client for 30 sec - exiting 21:44:25 (3474): No heartbeat from core client for 30 sec - exiting 21:44:26 (3474): No heartbeat from core client for 30 sec - exiting 21:44:27 (3474): No heartbeat from core client for 30 sec - exiting 20:38:38 (3474): No heartbeat from core client for 30 sec - exiting 20:38:39 (3474): No heartbeat from core client for 30 sec - exiting 20:38:40 (3474): No heartbeat from core client for 30 sec - exiting 20:38:41 (3474): No heartbeat from core client for 30 sec - exiting 20:38:42 (3474): No heartbeat from core client for 30 sec - exiting 20:38:43 (3474): No heartbeat from core client for 30 sec - exiting 20:38:44 (3474): No heartbeat from core client for 30 sec - exiting 20:38:45 (3474): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
23 Jul 2013 22:03:32 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 518,400 | 968,644 | 1.8685 |
23 Jul 2013 21:29:51 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 492,480 | 922,868 | 1.8739 |
23 Jul 2013 20:47:42 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 466,560 | 876,675 | 1.8790 |
23 Jul 2013 20:10:02 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 440,640 | 830,089 | 1.8838 |
23 Jul 2013 18:55:55 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 414,720 | 783,094 | 1.8882 |
23 Jul 2013 15:36:38 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 388,800 | 736,129 | 1.8933 |
23 Jul 2013 15:36:38 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 362,880 | 688,846 | 1.8983 |
23 Jul 2013 15:36:38 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 336,960 | 641,987 | 1.9052 |
23 Jul 2013 15:36:38 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 311,040 | 595,186 | 1.9135 |
23 Jul 2013 15:36:39 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 285,120 | 548,199 | 1.9227 |
23 Jul 2013 15:36:39 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 259,200 | 501,222 | 1.9337 |
23 Jul 2013 15:36:38 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 233,280 | 453,647 | 1.9446 |
23 Jul 2013 15:36:38 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 207,360 | 405,675 | 1.9564 |
23 Jul 2013 15:36:37 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 181,440 | 357,718 | 1.9715 |
23 Jul 2013 15:36:39 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 155,520 | 309,522 | 1.9902 |
11 Jul 2013 20:29:41 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 129,600 | 261,426 | 2.0172 |
08 Jul 2013 21:45:47 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 103,680 | 211,634 | 2.0412 |
08 Jul 2013 05:46:36 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 77,760 | 158,994 | 2.0447 |
07 Jul 2013 13:41:57 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 51,840 | 106,120 | 2.0471 |
06 Jul 2013 20:24:32 | 1284277 | 15881840 | hadcm3n_o3fd_1980_40_008384178_3 | 25,920 | 53,200 | 2.0525 |
©2024 cpdn.org