Task 13639656

Name	hadcm3n_u0mq_1980_40_007538394_2
Workunit	7735626
Created	16 Nov 2011, 18:50:39 UTC
Sent	16 Nov 2011, 18:57:21 UTC
Report deadline	16 Feb 2012, 2:24:32 UTC
Received	8 Dec 2011, 20:54:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1167875
Run time	3 days 0 hours 14 min 56 sec
CPU time	2 days 19 hours 42 min 39 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	4.15 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 20:28:57 (5336): No heartbeat from core client for 30 sec - exiting 20:28:58 (5336): No heartbeat from core client for 30 sec - exiting 20:28:59 (5336): No heartbeat from core client for 30 sec - exiting 20:29:00 (5336): No heartbeat from core client for 30 sec - exiting 20:29:01 (5336): No heartbeat from core client for 30 sec - exiting 20:29:02 (5336): No heartbeat from core client for 30 sec - exiting 20:29:03 (5336): No heartbeat from core client for 30 sec - exiting 20:29:04 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:43:04 (5056): No heartbeat from core client for 30 sec - exiting 07:43:05 (5056): No heartbeat from core client for 30 sec - exiting 07:43:06 (5056): No heartbeat from core client for 30 sec - exiting 07:43:07 (5056): No heartbeat from core client for 30 sec - exiting 07:43:08 (5056): No heartbeat from core client for 30 sec - exiting 07:43:09 (5056): No heartbeat from core client for 30 sec - exiting 07:43:10 (5056): No heartbeat from core client for 30 sec - exiting 07:43:11 (5056): No heartbeat from core client for 30 sec - exiting 07:43:12 (5056): No heartbeat from core client for 30 sec - exiting 07:43:13 (5056): No heartbeat from core client for 30 sec - exiting 07:43:14 (5056): No heartbeat from core client for 30 sec - exiting 07:43:15 (5056): No heartbeat from core client for 30 sec - exiting 07:43:16 (5056): No heartbeat from core client for 30 sec - exiting 07:43:17 (5056): No heartbeat from core client for 30 sec - exiting 07:43:18 (5056): No heartbeat from core client for 30 sec - exiting 07:43:19 (5056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:56:51 (5152): No heartbeat from core client for 30 sec - exiting 18:56:52 (5152): No heartbeat from core client for 30 sec - exiting 18:56:53 (5152): No heartbeat from core client for 30 sec - exiting 18:56:54 (5152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:56:55 (5152): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 19:19:25 (5492): No heartbeat from core client for 30 sec - exiting 19:19:26 (5492): No heartbeat from core client for 30 sec - exiting 19:19:27 (5492): No heartbeat from core client for 30 sec - exiting 19:19:28 (5492): No heartbeat from core client for 30 sec - exiting 19:19:29 (5492): No heartbeat from core client for 30 sec - exiting 19:19:30 (5492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:48:55 (6140): No heartbeat from core client for 30 sec - exiting 21:48:56 (6140): No heartbeat from core client for 30 sec - exiting 21:48:57 (6140): No heartbeat from core client for 30 sec - exiting 21:48:58 (6140): No heartbeat from core client for 30 sec - exiting 21:48:59 (6140): No heartbeat from core client for 30 sec - exiting 21:49:00 (6140): No heartbeat from core client for 30 sec - exiting 21:49:01 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:49:02 (6140): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 18:22:31 (5996): No heartbeat from core client for 30 sec - exiting 18:22:32 (5996): No heartbeat from core client for 30 sec - exiting 18:22:33 (5996): No heartbeat from core client for 30 sec - exiting 18:22:34 (5996): No heartbeat from core client for 30 sec - exiting 18:22:35 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:29:53 (5456): No heartbeat from core client for 30 sec - exiting 18:29:54 (5456): No heartbeat from core client for 30 sec - exiting 18:29:55 (5456): No heartbeat from core client for 30 sec - exiting 18:29:56 (5456): No heartbeat from core client for 30 sec - exiting 18:29:57 (5456): No heartbeat from core client for 30 sec - exiting 18:29:58 (5456): No heartbeat from core client for 30 sec - exiting 18:29:59 (5456): No heartbeat from core client for 30 sec - exiting 18:30:00 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:38:38 (8516): No heartbeat from core client for 30 sec - exiting 22:38:39 (8516): No heartbeat from core client for 30 sec - exiting 22:38:40 (8516): No heartbeat from core client for 30 sec - exiting 22:38:41 (8516): No heartbeat from core client for 30 sec - exiting 22:38:42 (8516): No heartbeat from core client for 30 sec - exiting 22:38:43 (8516): No heartbeat from core client for 30 sec - exiting 22:38:44 (8516): No heartbeat from core client for 30 sec - exiting 22:38:45 (8516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:33:47 (4872): No heartbeat from core client for 30 sec - exiting 13:33:48 (4872): No heartbeat from core client for 30 sec - exiting 13:33:49 (4872): No heartbeat from core client for 30 sec - exiting 13:33:50 (4872): No heartbeat from core client for 30 sec - exiting 13:33:51 (4872): No heartbeat from core client for 30 sec - exiting 13:33:52 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:33:53 (4872): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 17:05:54 (5060): No heartbeat from core client for 30 sec - exiting 17:05:56 (5060): No heartbeat from core client for 30 sec - exiting 17:05:57 (5060): No heartbeat from core client for 30 sec - exiting 17:05:58 (5060): No heartbeat from core client for 30 sec - exiting 17:05:59 (5060): No heartbeat from core client for 30 sec - exiting 17:06:00 (5060): No heartbeat from core client for 30 sec - exiting 17:06:01 (5060): No heartbeat from core client for 30 sec - exiting 17:06:02 (5060): No heartbeat from core client for 30 sec - exiting 17:06:03 (5060): No heartbeat from core client for 30 sec - exiting 17:06:04 (5060): No heartbeat from core client for 30 sec - exiting 17:06:05 (5060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:57:18 (2768): No heartbeat from core client for 30 sec - exiting 19:57:19 (2768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:29:07 (4360): No heartbeat from core client for 30 sec - exiting 09:29:08 (4360): No heartbeat from core client for 30 sec - exiting 09:29:09 (4360): No heartbeat from core client for 30 sec - exiting 09:29:10 (4360): No heartbeat from core client for 30 sec - exiting 09:29:11 (4360): No heartbeat from core client for 30 sec - exiting 09:29:12 (4360): No heartbeat from core client for 30 sec - exiting 09:29:13 (4360): No heartbeat from core client for 30 sec - exiting 09:29:14 (4360): No heartbeat from core client for 30 sec - exiting 09:29:15 (4360): No heartbeat from core client for 30 sec - exiting 09:29:16 (4360): No heartbeat from core client for 30 sec - exiting 09:29:17 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:31:48 (4892): No heartbeat from core client for 30 sec - exiting 20:31:49 (4892): No heartbeat from core client for 30 sec - exiting 20:31:50 (4892): No heartbeat from core client for 30 sec - exiting 20:31:51 (4892): No heartbeat from core client for 30 sec - exiting 20:31:52 (4892): No heartbeat from core client for 30 sec - exiting 20:31:53 (4892): No heartbeat from core client for 30 sec - exiting 20:31:54 (4892): No heartbeat from core client for 30 sec - exiting 20:31:55 (4892): No heartbeat from core client for 30 sec - exiting 20:31:56 (4892): No heartbeat from core client for 30 sec - exiting 20:31:57 (4892): No heartbeat from core client for 30 sec - exiting 20:31:58 (4892): No heartbeat from core client for 30 sec - exiting 20:31:59 (4892): No heartbeat from core client for 30 sec - exiting 20:32:00 (4892): No heartbeat from core client for 30 sec - exiting 20:32:01 (4892): No heartbeat from core client for 30 sec - exiting 20:32:02 (4892): No heartbeat from core client for 30 sec - exiting 20:32:03 (4892): No heartbeat from core client for 30 sec - exiting 20:32:04 (4892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:36 (6116): No heartbeat from core client for 30 sec - exiting 21:57:37 (6116): No heartbeat from core client for 30 sec - exiting 21:57:38 (6116): No heartbeat from core client for 30 sec - exiting 21:57:39 (6116): No heartbeat from core client for 30 sec - exiting 21:57:40 (6116): No heartbeat from core client for 30 sec - exiting 21:57:41 (6116): No heartbeat from core client for 30 sec - exiting 21:57:42 (6116): No heartbeat from core client for 30 sec - exiting 21:57:43 (6116): No heartbeat from core client for 30 sec - exiting 21:57:44 (6116): No heartbeat from core client for 30 sec - exiting 21:57:45 (6116): No heartbeat from core client for 30 sec - exiting 21:57:46 (6116): No heartbeat from core client for 30 sec - exiting 21:57:47 (6116): No heartbeat from core client for 30 sec - exiting 21:57:48 (6116): No heartbeat from core client for 30 sec - exiting 21:57:49 (6116): No heartbeat from core client for 30 sec - exiting 21:57:50 (6116): No heartbeat from core client for 30 sec - exiting 21:57:51 (6116): No heartbeat from core client for 30 sec - exiting 21:57:52 (6116): No heartbeat from core client for 30 sec - exiting 21:57:53 (6116): No heartbeat from core client for 30 sec - exiting 21:57:54 (6116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:36:34 (5628): No heartbeat from core client for 30 sec - exiting 18:36:35 (5628): No heartbeat from core client for 30 sec - exiting 18:36:36 (5628): No heartbeat from core client for 30 sec - exiting 18:36:37 (5628): No heartbeat from core client for 30 sec - exiting 18:36:38 (5628): No heartbeat from core client for 30 sec - exiting 18:36:39 (5628): No heartbeat from core client for 30 sec - exiting 18:36:40 (5628): No heartbeat from core client for 30 sec - exiting 18:36:41 (5628): No heartbeat from core client for 30 sec - exiting 18:36:42 (5628): No heartbeat from core client for 30 sec - exiting 18:36:43 (5628): No heartbeat from core client for 30 sec - exiting 18:36:44 (5628): No heartbeat from core client for 30 sec - exiting 18:36:45 (5628): No heartbeat from core client for 30 sec - exiting 18:36:46 (5628): No heartbeat from core client for 30 sec - exiting 18:36:47 (5628): No heartbeat from core client for 30 sec - exiting 18:36:48 (5628): No heartbeat from core client for 30 sec - exiting 18:36:49 (5628): No heartbeat from core client for 30 sec - exiting 18:36:50 (5628): No heartbeat from core client for 30 sec - exiting 18:36:51 (5628): No heartbeat from core client for 30 sec - exiting 18:36:52 (5628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:30:59 (5800): No heartbeat from core client for 30 sec - exiting 20:31:00 (5800): No heartbeat from core client for 30 sec - exiting 20:31:01 (5800): No heartbeat from core client for 30 sec - exiting 20:31:02 (5800): No heartbeat from core client for 30 sec - exiting 20:31:03 (5800): No heartbeat from core client for 30 sec - exiting 20:31:04 (5800): No heartbeat from core client for 30 sec - exiting 20:31:05 (5800): No heartbeat from core client for 30 sec - exiting 20:31:06 (5800): No heartbeat from core client for 30 sec - exiting 20:31:07 (5800): No heartbeat from core client for 30 sec - exiting 20:31:08 (5800): No heartbeat from core client for 30 sec - exiting 20:31:09 (5800): No heartbeat from core client for 30 sec - exiting 20:31:10 (5800): No heartbeat from core client for 30 sec - exiting 20:31:11 (5800): No heartbeat from core client for 30 sec - exiting 20:31:12 (5800): No heartbeat from core client for 30 sec - exiting 20:31:13 (5800): No heartbeat from core client for 30 sec - exiting 20:31:14 (5800): No heartbeat from core client for 30 sec - exiting 20:31:15 (5800): No heartbeat from core client for 30 sec - exiting 20:31:16 (5800): No heartbeat from core client for 30 sec - exiting 20:31:17 (5800): No heartbeat from core client for 30 sec - exiting 20:31:18 (5800): No heartbeat from core client for 30 sec - exiting 20:31:19 (5800): No heartbeat from core client for 30 sec - exiting 20:31:20 (5800): No heartbeat from core client for 30 sec - exiting 20:31:21 (5800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:16:10 (5428): No heartbeat from core client for 30 sec - exiting 22:16:11 (5428): No heartbeat from core client for 30 sec - exiting 22:16:12 (5428): No heartbeat from core client for 30 sec - exiting 22:16:13 (5428): No heartbeat from core client for 30 sec - exiting 22:16:14 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:08:24 (5488): No heartbeat from core client for 30 sec - exiting 19:08:25 (5488): No heartbeat from core client for 30 sec - exiting 19:08:26 (5488): No heartbeat from core client for 30 sec - exiting 19:08:27 (5488): No heartbeat from core client for 30 sec - exiting 19:08:28 (5488): No heartbeat from core client for 30 sec - exiting 19:08:29 (5488): No heartbeat from core client for 30 sec - exiting 19:08:30 (5488): No heartbeat from core client for 30 sec - exiting 19:08:31 (5488): No heartbeat from core client for 30 sec - exiting 19:08:32 (5488): No heartbeat from core client for 30 sec - exiting 19:08:33 (5488): No heartbeat from core client for 30 sec - exiting 19:08:34 (5488): No heartbeat from core client for 30 sec - exiting 19:08:35 (5488): No heartbeat from core client for 30 sec - exiting 19:08:36 (5488): No heartbeat from core client for 30 sec - exiting 19:08:37 (5488): No heartbeat from core client for 30 sec - exiting 19:08:38 (5488): No heartbeat from core client for 30 sec - exiting 19:08:39 (5488): No heartbeat from core client for 30 sec - exiting 19:08:40 (5488): No heartbeat from core client for 30 sec - exiting 19:08:41 (5488): No heartbeat from core client for 30 sec - exiting 19:08:42 (5488): No heartbeat from core client for 30 sec - exiting 19:08:43 (5488): No heartbeat from core client for 30 sec - exiting 19:08:44 (5488): No heartbeat from core client for 30 sec - exiting 19:08:45 (5488): No heartbeat from core client for 30 sec - exiting 19:08:46 (5488): No heartbeat from core client for 30 sec - exiting 19:08:47 (5488): No heartbeat from core client for 30 sec - exiting 19:08:48 (5488): No heartbeat from core client for 30 sec - exiting 19:08:49 (5488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:19 (6108): No heartbeat from core client for 30 sec - exiting 21:24:20 (6108): No heartbeat from core client for 30 sec - exiting 21:24:21 (6108): No heartbeat from core client for 30 sec - exiting 21:24:22 (6108): No heartbeat from core client for 30 sec - exiting 21:24:23 (6108): No heartbeat from core client for 30 sec - exiting 21:24:24 (6108): No heartbeat from core client for 30 sec - exiting 21:24:25 (6108): No heartbeat from core client for 30 sec - exiting 21:24:26 (6108): No heartbeat from core client for 30 sec - exiting 21:24:27 (6108): No heartbeat from core client for 30 sec - exiting 21:24:28 (6108): No heartbeat from core client for 30 sec - exiting 21:24:29 (6108): No heartbeat from core client for 30 sec - exiting 21:24:30 (6108): No heartbeat from core client for 30 sec - exiting 21:24:31 (6108): No heartbeat from core client for 30 sec - exiting 21:24:32 (6108): No heartbeat from core client for 30 sec - exiting 21:24:33 (6108): No heartbeat from core client for 30 sec - exiting 21:24:34 (6108): No heartbeat from core client for 30 sec - exiting 21:24:35 (6108): No heartbeat from core client for 30 sec - exiting 21:24:36 (6108): No heartbeat from core client for 30 sec - exiting 21:24:37 (6108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:42:18 (5344): No heartbeat from core client for 30 sec - exiting 07:42:19 (5344): No heartbeat from core client for 30 sec - exiting 07:42:20 (5344): No heartbeat from core client for 30 sec - exiting 07:42:21 (5344): No heartbeat from core client for 30 sec - exiting 07:42:22 (5344): No heartbeat from core client for 30 sec - exiting 07:42:23 (5344): No heartbeat from core client for 30 sec - exiting 07:42:24 (5344): No heartbeat from core client for 30 sec - exiting 07:42:25 (5344): No heartbeat from core client for 30 sec - exiting 07:42:26 (5344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:37:28 (4740): No heartbeat from core client for 30 sec - exiting 18:37:29 (4740): No heartbeat from core client for 30 sec - exiting 18:37:30 (4740): No heartbeat from core client for 30 sec - exiting 18:37:31 (4740): No heartbeat from core client for 30 sec - exiting 18:37:32 (4740): No heartbeat from core client for 30 sec - exiting 18:37:33 (4740): No heartbeat from core client for 30 sec - exiting 18:37:34 (4740): No heartbeat from core client for 30 sec - exiting 18:37:35 (4740): No heartbeat from core client for 30 sec - exiting 18:37:36 (4740): No heartbeat from core client for 30 sec - exiting 18:37:37 (4740): No heartbeat from core client for 30 sec - exiting 18:37:38 (4740): No heartbeat from core client for 30 sec - exiting 18:37:39 (4740): No heartbeat from core client for 30 sec - exiting 18:37:40 (4740): No heartbeat from core client for 30 sec - exiting 18:37:41 (4740): No heartbeat from core client for 30 sec - exiting 18:37:42 (4740): No heartbeat from core client for 30 sec - exiting 18:37:43 (4740): No heartbeat from core client for 30 sec - exiting 18:37:44 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:58:46 (5376): No heartbeat from core client for 30 sec - exiting 18:58:47 (5376): No heartbeat from core client for 30 sec - exiting 18:58:48 (5376): No heartbeat from core client for 30 sec - exiting 18:58:49 (5376): No heartbeat from core client for 30 sec - exiting 18:58:50 (5376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:29:45 (3352): No heartbeat from core client for 30 sec - exiting 16:29:46 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:34:23 (5384): No heartbeat from core client for 30 sec - exiting 21:34:24 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:57:06 (5872): No heartbeat from core client for 30 sec - exiting 18:57:07 (5872): No heartbeat from core client for 30 sec - exiting 18:57:08 (5872): No heartbeat from core client for 30 sec - exiting 18:57:09 (5872): No heartbeat from core client for 30 sec - exiting 18:57:10 (5872): No heartbeat from core client for 30 sec - exiting 18:57:11 (5872): No heartbeat from core client for 30 sec - exiting 18:57:12 (5872): No heartbeat from core client for 30 sec - exiting 18:57:13 (5872): No heartbeat from core client for 30 sec - exiting 18:57:14 (5872): No heartbeat from core client for 30 sec - exiting 18:57:15 (5872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:19:38 (1792): No heartbeat from core client for 30 sec - exiting 21:19:39 (1792): No heartbeat from core client for 30 sec - exiting 21:19:40 (1792): No heartbeat from core client for 30 sec - exiting 21:19:41 (1792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:39:37 (6112): No heartbeat from core client for 30 sec - exiting 22:39:38 (6112): No heartbeat from core client for 30 sec - exiting 22:39:39 (6112): No heartbeat from core client for 30 sec - exiting 22:39:40 (6112): No heartbeat from core client for 30 sec - exiting 22:39:41 (6112): No heartbeat from core client for 30 sec - exiting 22:39:42 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5256, iMonCtr=1 Model crash detected, will try to restart... 22:37:40 (5528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4944, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5564, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5692, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5548, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5996, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5752, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:09:18 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:53:34 (3060): No heartbeat from core client for 30 sec - exiting 15:53:35 (3060): No heartbeat from core client for 30 sec - exiting 15:53:36 (3060): No heartbeat from core client for 30 sec - exiting 15:53:37 (3060): No heartbeat from core client for 30 sec - exiting 15:53:38 (3060): No heartbeat from core client for 30 sec - exiting 15:53:39 (3060): No heartbeat from core client for 30 sec - exiting 15:53:40 (3060): No heartbeat from core client for 30 sec - exiting 15:53:41 (3060): No heartbeat from core client for 30 sec - exiting 15:53:42 (3060): No heartbeat from core client for 30 sec - exiting 15:53:43 (3060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:20:29 (5084): No heartbeat from core client for 30 sec - exiting 19:20:30 (5084): No heartbeat from core client for 30 sec - exiting 19:20:31 (5084): No heartbeat from core client for 30 sec - exiting 19:20:32 (5084): No heartbeat from core client for 30 sec - exiting 19:20:33 (5084): No heartbeat from core client for 30 sec - exiting 19:20:34 (5084): No heartbeat from core client for 30 sec - exiting 19:20:35 (5084): No heartbeat from core client for 30 sec - exiting 19:20:36 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:50:47 (2992): No heartbeat from core client for 30 sec - exiting 00:50:48 (2992): No heartbeat from core client for 30 sec - exiting 00:50:49 (2992): No heartbeat from core client for 30 sec - exiting 00:50:50 (2992): No heartbeat from core client for 30 sec - exiting 00:50:51 (2992): No heartbeat from core client for 30 sec - exiting 00:50:52 (2992): No heartbeat from core client for 30 sec - exiting 00:50:53 (2992): No heartbeat from core client for 30 sec - exiting 00:50:54 (2992): No heartbeat from core client for 30 sec - exiting 00:50:55 (2992): No heartbeat from core client for 30 sec - exiting 00:50:56 (2992): No heartbeat from core client for 30 sec - exiting 00:50:57 (2992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:11:29 (4320): No heartbeat from core client for 30 sec - exiting 16:11:30 (4320): No heartbeat from core client for 30 sec - exiting 16:11:31 (4320): No heartbeat from core client for 30 sec - exiting 16:11:32 (4320): No heartbeat from core client for 30 sec - exiting 16:11:33 (4320): No heartbeat from core client for 30 sec - exiting 16:11:34 (4320): No heartbeat from core client for 30 sec - exiting 16:11:35 (4320): No heartbeat from core client for 30 sec - exiting 16:11:36 (4320): No heartbeat from core client for 30 sec - exiting 16:11:37 (4320): No heartbeat from core client for 30 sec - exiting 16:11:38 (4320): No heartbeat from core client for 30 sec - exiting 16:11:39 (4320): No heartbeat from core client for 30 sec - exiting 16:11:40 (4320): No heartbeat from core client for 30 sec - exiting 16:11:41 (4320): No heartbeat from core client for 30 sec - exiting 16:11:42 (4320): No heartbeat from core client for 30 sec - exiting 16:11:43 (4320): No heartbeat from core client for 30 sec - exiting 16:11:44 (4320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:11:48 (392): No heartbeat from core client for 30 sec - exiting 22:11:50 (392): No heartbeat from core client for 30 sec - exiting 22:11:51 (392): No heartbeat from core client for 30 sec - exiting 22:11:52 (392): No heartbeat from core client for 30 sec - exiting 22:11:53 (392): No heartbeat from core client for 30 sec - exiting 22:11:54 (392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:11:55 (392): No heartbeat from core client for 30 sec - exiting 21:59:51 (5692): No heartbeat from core client for 30 sec - exiting 21:59:52 (5692): No heartbeat from core client for 30 sec - exiting 21:59:53 (5692): No heartbeat from core client for 30 sec - exiting 21:59:54 (5692): No heartbeat from core client for 30 sec - exiting 21:59:55 (5692): No heartbeat from core client for 30 sec - exiting 21:59:56 (5692): No heartbeat from core client for 30 sec - exiting 21:59:57 (5692): No heartbeat from core client for 30 sec - exiting 21:59:58 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:33:28 (6044): No heartbeat from core client for 30 sec - exiting 18:33:29 (6044): No heartbeat from core client for 30 sec - exiting 18:33:30 (6044): No heartbeat from core client for 30 sec - exiting 18:33:31 (6044): No heartbeat from core client for 30 sec - exiting 18:33:32 (6044): No heartbeat from core client for 30 sec - exiting 18:33:33 (6044): No heartbeat from core client for 30 sec - exiting 18:33:34 (6044): No heartbeat from core client for 30 sec - exiting 18:33:35 (6044): No heartbeat from core client for 30 sec - exiting 18:33:36 (6044): No heartbeat from core client for 30 sec - exiting 18:33:37 (6044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:43 (5572): No heartbeat from core client for 30 sec - exiting 19:22:44 (5572): No heartbeat from core client for 30 sec - exiting 19:22:45 (5572): No heartbeat from core client for 30 sec - exiting 19:22:46 (5572): No heartbeat from core client for 30 sec - exiting 19:22:47 (5572): No heartbeat from core client for 30 sec - exiting 19:22:48 (5572): No heartbeat from core client for 30 sec - exiting 19:22:49 (5572): No heartbeat from core client for 30 sec - exiting 19:22:50 (5572): No heartbeat from core client for 30 sec - exiting 19:22:51 (5572): No heartbeat from core client for 30 sec - exiting 19:22:52 (5572): No heartbeat from core client for 30 sec - exiting 19:22:53 (5572): No heartbeat from core client for 30 sec - exiting 19:22:54 (5572): No heartbeat from core client for 30 sec - exiting 19:22:55 (5572): No heartbeat from core client for 30 sec - exiting 19:22:56 (5572): No heartbeat from core client for 30 sec - exiting 19:22:57 (5572): No heartbeat from core client for 30 sec - exiting 19:22:58 (5572): No heartbeat from core client for 30 sec - exiting 19:22:59 (5572): No heartbeat from core client for 30 sec - exiting 19:23:00 (5572): No heartbeat from core client for 30 sec - exiting 19:23:01 (5572): No heartbeat from core client for 30 sec - exiting 19:23:02 (5572): No heartbeat from core client for 30 sec - exiting 19:23:03 (5572): No heartbeat from core client for 30 sec - exiting 19:23:04 (5572): No heartbeat from core client for 30 sec - exiting 19:23:05 (5572): No heartbeat from core client for 30 sec - exiting 19:23:06 (5572): No heartbeat from core client for 30 sec - exiting 19:23:07 (5572): No heartbeat from core client for 30 sec - exiting 19:23:08 (5572): No heartbeat from core client for 30 sec - exiting 19:23:09 (5572): No heartbeat from core client for 30 sec - exiting 19:23:10 (5572): No heartbeat from core client for 30 sec - exiting 19:23:11 (5572): No heartbeat from core client for 30 sec - exiting 19:23:12 (5572): No heartbeat from core client for 30 sec - exiting 19:23:13 (5572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:23:14 (5572): No heartbeat from core client for 30 sec - exiting 19:41:34 (5716): No heartbeat from core client for 30 sec - exiting 19:41:35 (5716): No heartbeat from core client for 30 sec - exiting 19:41:36 (5716): No heartbeat from core client for 30 sec - exiting 19:41:37 (5716): No heartbeat from core client for 30 sec - exiting 19:41:38 (5716): No heartbeat from core client for 30 sec - exiting 19:41:39 (5716): No heartbeat from core client for 30 sec - exiting 19:41:40 (5716): No heartbeat from core client for 30 sec - exiting 19:41:41 (5716): No heartbeat from core client for 30 sec - exiting 19:41:42 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Dec 2011 20:58:47	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	259,200	243,756	0.9404
04 Dec 2011 17:02:00	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	233,280	219,036	0.9389
03 Dec 2011 10:36:41	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	207,360	193,808	0.9346
03 Dec 2011 02:19:45	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	181,440	168,338	0.9278
01 Dec 2011 19:48:02	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	155,520	143,672	0.9238
24 Nov 2011 22:58:10	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	129,600	120,453	0.9294
22 Nov 2011 18:41:42	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	103,680	97,476	0.9402
20 Nov 2011 14:14:03	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	77,760	73,034	0.9392
20 Nov 2011 00:56:05	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	51,840	48,508	0.9357
20 Nov 2011 00:56:05	1167875	13639656	hadcm3n_u0mq_1980_40_007538394_2	25,920	24,122	0.9306