Task 13116678

Name	hadcm3n_yi61_1900_40_007356403_1
Workunit	7553833
Created	6 Jul 2011, 14:46:23 UTC
Sent	9 Jul 2011, 18:32:49 UTC
Report deadline	9 Oct 2011, 2:00:00 UTC
Received	23 Jul 2011, 4:14:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1006135
Run time	7 days 11 hours 15 min 35 sec
CPU time	6 days 5 hours 40 min 48 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.70 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.6.36</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> o heartbeat from core client for 30 sec - exiting 23:05:50 (24224): No heartbeat from core client for 30 sec - exiting 23:05:51 (24224): No heartbeat from core client for 30 sec - exiting 23:05:52 (24224): No heartbeat from core client for 30 sec - exiting 23:05:53 (24224): No heartbeat from core client for 30 sec - exiting 23:05:54 (24224): No heartbeat from core client for 30 sec - exiting 23:05:55 (24224): No heartbeat from core client for 30 sec - exiting 23:05:56 (24224): No heartbeat from core client for 30 sec - exiting 23:05:57 (24224): No heartbeat from core client for 30 sec - exiting 23:05:58 (24224): No heartbeat from core client for 30 sec - exiting 23:05:59 (24224): No heartbeat from core client for 30 sec - exiting 23:06:00 (24224): No heartbeat from core client for 30 sec - exiting 23:06:01 (24224): No heartbeat from core client for 30 sec - exiting 23:08:05 (24396): No heartbeat from core client for 30 sec - exiting 23:08:06 (24396): No heartbeat from core client for 30 sec - exiting 23:08:07 (24396): No heartbeat from core client for 30 sec - exiting 23:08:08 (24396): No heartbeat from core client for 30 sec - exiting 23:08:09 (24396): No heartbeat from core client for 30 sec - exiting 23:08:10 (24396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:09:13 (24412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:11:13 (24441): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:12:25 (24463): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:15 (24491): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:19 (24491): No heartbeat from core client for 30 sec - exiting 23:24:20 (24491): No heartbeat from core client for 30 sec - exiting 23:24:21 (24491): No heartbeat from core client for 30 sec - exiting 23:24:22 (24491): No heartbeat from core client for 30 sec - exiting 23:24:23 (24491): No heartbeat from core client for 30 sec - exiting 23:24:24 (24491): No heartbeat from core client for 30 sec - exiting 23:24:25 (24491): No heartbeat from core client for 30 sec - exiting 23:24:26 (24491): No heartbeat from core client for 30 sec - exiting 23:24:27 (24491): No heartbeat from core client for 30 sec - exiting 23:24:28 (24491): No heartbeat from core client for 30 sec - exiting 23:24:29 (24491): No heartbeat from core client for 30 sec - exiting 23:24:30 (24491): No heartbeat from core client for 30 sec - exiting 23:24:31 (24491): No heartbeat from core client for 30 sec - exiting 23:24:32 (24491): No heartbeat from core client for 30 sec - exiting 23:24:33 (24491): No heartbeat from core client for 30 sec - exiting 23:24:34 (24491): No heartbeat from core client for 30 sec - exiting 23:24:35 (24491): No heartbeat from core client for 30 sec - exiting 23:24:36 (24491): No heartbeat from core client for 30 sec - exiting 23:24:37 (24491): No heartbeat from core client for 30 sec - exiting 23:24:38 (24491): No heartbeat from core client for 30 sec - exiting 23:24:39 (24491): No heartbeat from core client for 30 sec - exiting 23:24:40 (24491): No heartbeat from core client for 30 sec - exiting 23:24:41 (24491): No heartbeat from core client for 30 sec - exiting 23:24:42 (24491): No heartbeat from core client for 30 sec - exiting 23:24:43 (24491): No heartbeat from core client for 30 sec - exiting 23:24:44 (24491): No heartbeat from core client for 30 sec - exiting 23:24:45 (24491): No heartbeat from core client for 30 sec - exiting 23:24:46 (24491): No heartbeat from core client for 30 sec - exiting 23:26:45 (24576): No heartbeat from core client for 30 sec - exiting 23:26:46 (24576): No heartbeat from core client for 30 sec - exiting 23:26:47 (24576): No heartbeat from core client for 30 sec - exiting 23:26:48 (24576): No heartbeat from core client for 30 sec - exiting 23:26:49 (24576): No heartbeat from core client for 30 sec - exiting 23:26:50 (24576): No heartbeat from core client for 30 sec - exiting 23:26:51 (24576): No heartbeat from core client for 30 sec - exiting 23:26:52 (24576): No heartbeat from core client for 30 sec - exiting 23:26:53 (24576): No heartbeat from core client for 30 sec - exiting 23:26:54 (24576): No heartbeat from core client for 30 sec - exiting 23:26:55 (24576): No heartbeat from core client for 30 sec - exiting 23:26:56 (24576): No heartbeat from core client for 30 sec - exiting 23:26:57 (24576): No heartbeat from core client for 30 sec - exiting 23:26:58 (24576): No heartbeat from core client for 30 sec - exiting 23:26:59 (24576): No heartbeat from core client for 30 sec - exiting 23:27:00 (24576): No heartbeat from core client for 30 sec - exiting 23:27:01 (24576): No heartbeat from core client for 30 sec - exiting 23:27:02 (24576): No heartbeat from core client for 30 sec - exiting 23:27:03 (24576): No heartbeat from core client for 30 sec - exiting 23:27:04 (24576): No heartbeat from core client for 30 sec - exiting 23:27:05 (24576): No heartbeat from core client for 30 sec - exiting 23:27:06 (24576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:28:08 (24601): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:30:46 (24629): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:31:54 (24652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:33:05 (24674): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:34:09 (24699): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:04 (24725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:39 (24748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:47:41 (24802): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:52:41 (24861): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:32:53 (24909): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:32:57 (24909): No heartbeat from core client for 30 sec - exiting 00:32:58 (24909): No heartbeat from core client for 30 sec - exiting 00:34:03 (25202): No heartbeat from core client for 30 sec - exiting 00:34:04 (25202): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:52:56 (25218): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:54:07 (25369): No heartbeat from core client for 30 sec - exiting 00:54:08 (25369): No heartbeat from core client for 30 sec - exiting 00:54:09 (25369): No heartbeat from core client for 30 sec - exiting 00:54:10 (25369): No heartbeat from core client for 30 sec - exiting 00:54:11 (25369): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:57:56 (25381): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:59:04 (25435): No heartbeat from core client for 30 sec - exiting 00:59:05 (25435): No heartbeat from core client for 30 sec - exiting 00:59:06 (25435): No heartbeat from core client for 30 sec - exiting 00:59:07 (25435): No heartbeat from core client for 30 sec - exiting 00:59:08 (25435): No heartbeat from core client for 30 sec - exiting 00:59:09 (25435): No heartbeat from core client for 30 sec - exiting 00:59:10 (25435): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:02:58 (25454): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:07:58 (25497): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:12:59 (25547): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:13:01 (25547): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:04:25 (36750): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:17:03 (38355): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:17:12 (38355): No heartbeat from core client for 30 sec - exiting 12:17:13 (38355): No heartbeat from core client for 30 sec - exiting 12:17:14 (38355): No heartbeat from core client for 30 sec - exiting 12:17:15 (38355): No heartbeat from core client for 30 sec - exiting 12:17:16 (38355): No heartbeat from core client for 30 sec - exiting 12:17:17 (38355): No heartbeat from core client for 30 sec - exiting 12:17:18 (38355): No heartbeat from core client for 30 sec - exiting 12:17:19 (38355): No heartbeat from core client for 30 sec - exiting 12:17:20 (38355): No heartbeat from core client for 30 sec - exiting 12:17:21 (38355): No heartbeat from core client for 30 sec - exiting 12:17:22 (38355): No heartbeat from core client for 30 sec - exiting 12:17:23 (38355): No heartbeat from core client for 30 sec - exiting 12:17:24 (38355): No heartbeat from core client for 30 sec - exiting 12:17:25 (38355): No heartbeat from core client for 30 sec - exiting 12:17:26 (38355): No heartbeat from core client for 30 sec - exiting 12:18:04 (38355): No heartbeat from core client for 30 sec - exiting 12:18:05 (38355): No heartbeat from core client for 30 sec - exiting 12:18:06 (38355): No heartbeat from core client for 30 sec - exiting 12:18:07 (38355): No heartbeat from core client for 30 sec - exiting 12:18:08 (38355): No heartbeat from core client for 30 sec - exiting 12:18:09 (38355): No heartbeat from core client for 30 sec - exiting 12:18:10 (38355): No heartbeat from core client for 30 sec - exiting 12:18:11 (38355): No heartbeat from core client for 30 sec - exiting 12:18:12 (38355): No heartbeat from core client for 30 sec - exiting 12:18:13 (38355): No heartbeat from core client for 30 sec - exiting 12:18:14 (38355): No heartbeat from core client for 30 sec - exiting 12:18:15 (38355): No heartbeat from core client for 30 sec - exiting 12:18:16 (38355): No heartbeat from core client for 30 sec - exiting 12:18:17 (38355): No heartbeat from core client for 30 sec - exiting 12:18:18 (38355): No heartbeat from core client for 30 sec - exiting 12:18:19 (38355): No heartbeat from core client for 30 sec - exiting 12:18:20 (38355): No heartbeat from core client for 30 sec - exiting 12:18:21 (38355): No heartbeat from core client for 30 sec - exiting 12:18:22 (38355): No heartbeat from core client for 30 sec - exiting 12:18:23 (38355): No heartbeat from core client for 30 sec - exiting 12:18:24 (38355): No heartbeat from core client for 30 sec - exiting 12:18:25 (38355): No heartbeat from core client for 30 sec - exiting 12:18:26 (38355): No heartbeat from core client for 30 sec - exiting 12:18:27 (38355): No heartbeat from core client for 30 sec - exiting 12:18:28 (38355): No heartbeat from core client for 30 sec - exiting 12:18:29 (38355): No heartbeat from core client for 30 sec - exiting 12:18:30 (38355): No heartbeat from core client for 30 sec - exiting 12:18:31 (38355): No heartbeat from core client for 30 sec - exiting 12:18:32 (38355): No heartbeat from core client for 30 sec - exiting 12:18:33 (38355): No heartbeat from core client for 30 sec - exiting 12:18:34 (38355): No heartbeat from core client for 30 sec - exiting 12:18:35 (38355): No heartbeat from core client for 30 sec - exiting 12:18:36 (38355): No heartbeat from core client for 30 sec - exiting 12:18:37 (38355): No heartbeat from core client for 30 sec - exiting 12:18:38 (38355): No heartbeat from core client for 30 sec - exiting 12:18:39 (38355): No heartbeat from core client for 30 sec - exiting 12:18:40 (38355): No heartbeat from core client for 30 sec - exiting 12:18:41 (38355): No heartbeat from core client for 30 sec - exiting 12:18:42 (38355): No heartbeat from core client for 30 sec - exiting 12:18:43 (38355): No heartbeat from core client for 30 sec - exiting 12:18:44 (38355): No heartbeat from core client for 30 sec - exiting 12:18:45 (38355): No heartbeat from core client for 30 sec - exiting 12:18:46 (38355): No heartbeat from core client for 30 sec - exiting 12:20:04 (38454): No heartbeat from core client for 30 sec - exiting 12:20:05 (38454): No heartbeat from core client for 30 sec - exiting 12:20:06 (38454): No heartbeat from core client for 30 sec - exiting 12:20:07 (38454): No heartbeat from core client for 30 sec - exiting 12:20:08 (38454): No heartbeat from core client for 30 sec - exiting 12:20:09 (38454): No heartbeat from core client for 30 sec - exiting 12:20:10 (38454): No heartbeat from core client for 30 sec - exiting 12:20:11 (38454): No heartbeat from core client for 30 sec - exiting 12:20:12 (38454): No heartbeat from core client for 30 sec - exiting 12:20:13 (38454): No heartbeat from core client for 30 sec - exiting 12:20:14 (38454): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:31:48 (38474): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:31:55 (38474): No heartbeat from core client for 30 sec - exiting 12:31:56 (38474): No heartbeat from core client for 30 sec - exiting 12:31:57 (38474): No heartbeat from core client for 30 sec - exiting 12:31:58 (38474): No heartbeat from core client for 30 sec - exiting 13:30:43 (38589): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:58:03 (39002): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:58:05 (39002): No heartbeat from core client for 30 sec - exiting 13:58:06 (39002): No heartbeat from core client for 30 sec - exiting 13:58:07 (39002): No heartbeat from core client for 30 sec - exiting 14:06:39 (39212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:06:44 (39212): No heartbeat from core client for 30 sec - exiting 14:33:27 (39283): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:52:10 (39485): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:52:14 (39485): No heartbeat from core client for 30 sec - exiting 14:52:15 (39485): No heartbeat from core client for 30 sec - exiting 14:52:16 (39485): No heartbeat from core client for 30 sec - exiting 14:52:17 (39485): No heartbeat from core client for 30 sec - exiting 15:02:08 (39617): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:02:15 (39617): No heartbeat from core client for 30 sec - exiting 15:02:16 (39617): No heartbeat from core client for 30 sec - exiting 15:02:17 (39617): No heartbeat from core client for 30 sec - exiting 15:02:18 (39617): No heartbeat from core client for 30 sec - exiting 15:02:19 (39617): No heartbeat from core client for 30 sec - exiting 15:02:20 (39617): No heartbeat from core client for 30 sec - exiting 15:02:21 (39617): No heartbeat from core client for 30 sec - exiting 15:02:22 (39617): No heartbeat from core client for 30 sec - exiting 15:02:23 (39617): No heartbeat from core client for 30 sec - exiting 15:02:24 (39617): No heartbeat from core client for 30 sec - exiting 15:02:25 (39617): No heartbeat from core client for 30 sec - exiting 15:02:26 (39617): No heartbeat from core client for 30 sec - exiting 15:02:27 (39617): No heartbeat from core client for 30 sec - exiting 15:02:28 (39617): No heartbeat from core client for 30 sec - exiting 15:02:29 (39617): No heartbeat from core client for 30 sec - exiting 15:02:30 (39617): No heartbeat from core client for 30 sec - exiting 15:02:31 (39617): No heartbeat from core client for 30 sec - exiting 15:02:32 (39617): No heartbeat from core client for 30 sec - exiting 15:02:33 (39617): No heartbeat from core client for 30 sec - exiting 15:03:05 (39617): No heartbeat from core client for 30 sec - exiting 15:03:06 (39617): No heartbeat from core client for 30 sec - exiting 15:03:07 (39617): No heartbeat from core client for 30 sec - exiting 15:03:08 (39617): No heartbeat from core client for 30 sec - exiting 15:03:09 (39617): No heartbeat from core client for 30 sec - exiting 15:03:10 (39617): No heartbeat from core client for 30 sec - exiting 15:03:11 (39617): No heartbeat from core client for 30 sec - exiting 15:03:12 (39617): No heartbeat from core client for 30 sec - exiting 15:03:13 (39617): No heartbeat from core client for 30 sec - exiting 15:03:14 (39617): No heartbeat from core client for 30 sec - exiting 15:03:15 (39617): No heartbeat from core client for 30 sec - exiting 15:03:16 (39617): No heartbeat from core client for 30 sec - exiting 15:03:17 (39617): No heartbeat from core client for 30 sec - exiting 15:03:18 (39617): No heartbeat from core client for 30 sec - exiting 15:03:19 (39617): No heartbeat from core client for 30 sec - exiting 15:03:20 (39617): No heartbeat from core client for 30 sec - exiting 15:03:21 (39617): No heartbeat from core client for 30 sec - exiting 15:03:22 (39617): No heartbeat from core client for 30 sec - exiting 15:03:23 (39617): No heartbeat from core client for 30 sec - exiting 15:13:01 (39700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:13:07 (39700): No heartbeat from core client for 30 sec - exiting 15:13:08 (39700): No heartbeat from core client for 30 sec - exiting 15:13:09 (39700): No heartbeat from core client for 30 sec - exiting 15:13:10 (39700): No heartbeat from core client for 30 sec - exiting 15:13:11 (39700): No heartbeat from core client for 30 sec - exiting 15:13:12 (39700): No heartbeat from core client for 30 sec - exiting 15:13:13 (39700): No heartbeat from core client for 30 sec - exiting 15:13:14 (39700): No heartbeat from core client for 30 sec - exiting 15:21:04 (39776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:21:08 (39776): No heartbeat from core client for 30 sec - exiting 15:21:09 (39776): No heartbeat from core client for 30 sec - exiting 15:21:10 (39776): No heartbeat from core client for 30 sec - exiting 15:21:11 (39776): No heartbeat from core client for 30 sec - exiting 15:29:13 (39853): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:29:17 (39853): No heartbeat from core client for 30 sec - exiting 15:29:18 (39853): No heartbeat from core client for 30 sec - exiting 15:29:19 (39853): No heartbeat from core client for 30 sec - exiting 15:29:20 (39853): No heartbeat from core client for 30 sec - exiting 15:37:52 (39927): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:56 (39927): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K540 15:47:43 (40018): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:47:47 (40018): No heartbeat from core client for 30 sec - exiting 15:47:48 (40018): No heartbeat from core client for 30 sec - exiting 15:47:49 (40018): No heartbeat from core client for 30 sec - exiting 15:47:50 (40018): No heartbeat from core client for 30 sec - exiting 15:47:51 (40018): No heartbeat from core client for 30 sec - exiting 15:47:52 (40018): No heartbeat from core client for 30 sec - exiting 15:47:53 (40018): No heartbeat from core client for 30 sec - exiting 15:47:54 (40018): No heartbeat from core client for 30 sec - exiting 15:50:08 (40090): No heartbeat from core client for 30 sec - exiting 15:50:09 (40090): No heartbeat from core client for 30 sec - exiting 15:50:10 (40090): No heartbeat from core client for 30 sec - exiting 15:50:11 (40090): No heartbeat from core client for 30 sec - exiting 15:50:12 (40090): No heartbeat from core client for 30 sec - exiting 15:50:13 (40090): No heartbeat from core client for 30 sec - exiting 15:50:14 (40090): No heartbeat from core client for 30 sec - exiting 15:50:15 (40090): No heartbeat from core client for 30 sec - exiting 15:50:16 (40090): No heartbeat from core client for 30 sec - exiting 15:50:17 (40090): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:27:40 (70266): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:40 (70345): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:42 (70345): No heartbeat from core client for 30 sec - exiting 19:32:43 (70345): No heartbeat from core client for 30 sec - exiting 19:32:44 (70345): No heartbeat from core client for 30 sec - exiting 19:33:19 (70345): No heartbeat from core client for 30 sec - exiting 19:33:20 (70345): No heartbeat from core client for 30 sec - exiting 19:33:21 (70345): No heartbeat from core client for 30 sec - exiting 19:33:22 (70345): No heartbeat from core client for 30 sec - exiting 19:33:23 (70345): No heartbeat from core client for 30 sec - exiting 19:33:24 (70345): No heartbeat from core client for 30 sec - exiting 19:33:25 (70345): No heartbeat from core client for 30 sec - exiting 19:33:26 (70345): No heartbeat from core client for 30 sec - exiting 19:33:27 (70345): No heartbeat from core client for 30 sec - exiting 19:33:28 (70345): No heartbeat from core client for 30 sec - exiting 19:33:29 (70345): No heartbeat from core client for 30 sec - exiting 19:33:30 (70345): No heartbeat from core client for 30 sec - exiting 19:33:31 (70345): No heartbeat from core client for 30 sec - exiting 19:33:32 (70345): No heartbeat from core client for 30 sec - exiting 19:33:33 (70345): No heartbeat from core client for 30 sec - exiting 19:33:34 (70345): No heartbeat from core client for 30 sec - exiting 19:33:35 (70345): No heartbeat from core client for 30 sec - exiting 19:33:36 (70345): No heartbeat from core client for 30 sec - exiting 19:33:37 (70345): No heartbeat from core client for 30 sec - exiting 19:33:38 (70345): No heartbeat from core client for 30 sec - exiting 19:33:39 (70345): No heartbeat from core client for 30 sec - exiting 19:33:40 (70345): No heartbeat from core client for 30 sec - exiting 19:33:41 (70345): No heartbeat from core client for 30 sec - exiting 19:33:42 (70345): No heartbeat from core client for 30 sec - exiting 19:34:38 (70391): No heartbeat from core client for 30 sec - exiting 19:34:39 (70391): No heartbeat from core client for 30 sec - exiting 19:34:40 (70391): No heartbeat from core client for 30 sec - exiting 19:34:41 (70391): No heartbeat from core client for 30 sec - exiting 19:34:42 (70391): No heartbeat from core client for 30 sec - exiting 19:34:43 (70391): No heartbeat from core client for 30 sec - exiting 19:34:44 (70391): No heartbeat from core client for 30 sec - exiting 19:34:45 (70391): No heartbeat from core client for 30 sec - exiting 19:34:46 (70391): No heartbeat from core client for 30 sec - exiting 19:34:47 (70391): No heartbeat from core client for 30 sec - exiting 19:34:48 (70391): No heartbeat from core client for 30 sec - exiting 19:34:49 (70391): No heartbeat from core client for 30 sec - exiting 19:34:50 (70391): No heartbeat from core client for 30 sec - exiting 19:34:51 (70391): No heartbeat from core client for 30 sec - exiting 19:34:52 (70391): No heartbeat from core client for 30 sec - exiting 19:34:53 (70391): No heartbeat from core client for 30 sec - exiting 19:34:54 (70391): No heartbeat from core client for 30 sec - exiting 19:34:55 (70391): No heartbeat from core client for 30 sec - exiting 19:34:56 (70391): No heartbeat from core client for 30 sec - exiting 19:34:57 (70391): No heartbeat from core client for 30 sec - exiting 19:34:58 (70391): No heartbeat from core client for 30 sec - exiting 19:34:59 (70391): No heartbeat from core client for 30 sec - exiting 19:35:00 (70391): No heartbeat from core client for 30 sec - exiting 19:35:01 (70391): No heartbeat from core client for 30 sec - exiting 19:35:02 (70391): No heartbeat from core client for 30 sec - exiting 19:35:03 (70391): No heartbeat from core client for 30 sec - exiting 19:35:04 (70391): No heartbeat from core client for 30 sec - exiting 19:35:05 (70391): No heartbeat from core client for 30 sec - exiting 19:35:06 (70391): No heartbeat from core client for 30 sec - exiting 19:35:07 (70391): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:49:44 (70400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:49:47 (70400): No heartbeat from core client for 30 sec - exiting 19:49:48 (70400): No heartbeat from core client for 30 sec - exiting 19:53:32 (70524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:38 (71804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:05:41 (71899): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:16:42 (71910): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:16:46 (71910): No heartbeat from core client for 30 sec - exiting 20:18:35 (72012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:04 (72023): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:15 (72023): No heartbeat from core client for 30 sec - exiting 20:27:16 (72023): No heartbeat from core client for 30 sec - exiting 20:27:17 (72023): No heartbeat from core client for 30 sec - exiting 20:27:18 (72023): No heartbeat from core client for 30 sec - exiting 20:27:19 (72023): No heartbeat from core client for 30 sec - exiting 20:27:20 (72023): No heartbeat from core client for 30 sec - exiting 20:27:21 (72023): No heartbeat from core client for 30 sec - exiting 20:27:22 (72023): No heartbeat from core client for 30 sec - exiting 20:28:35 (72112): No heartbeat from core client for 30 sec - exiting 20:28:36 (72112): No heartbeat from core client for 30 sec - exiting 20:28:37 (72112): No heartbeat from core client for 30 sec - exiting 20:28:38 (72112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:53 (72124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:55 (72124): No heartbeat from core client for 30 sec - exiting 20:47:55 (72218): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:49:30 (72284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:58:23 (72323): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:58:30 (72323): No heartbeat from core client for 30 sec - exiting 20:58:31 (72323): No heartbeat from core client for 30 sec - exiting 20:58:32 (72323): No heartbeat from core client for 30 sec - exiting 20:58:33 (72323): No heartbeat from core client for 30 sec - exiting 20:58:34 (72323): No heartbeat from core client for 30 sec - exiting 20:58:35 (72323): No heartbeat from core client for 30 sec - exiting 20:58:36 (72323): No heartbeat from core client for 30 sec - exiting 20:58:37 (72323): No heartbeat from core client for 30 sec - exiting 20:58:38 (72323): No heartbeat from core client for 30 sec - exiting 20:58:39 (72323): No heartbeat from core client for 30 sec - exiting 20:59:28 (72323): No heartbeat from core client for 30 sec - exiting 20:59:29 (72323): No heartbeat from core client for 30 sec - exiting 20:59:30 (72323): No heartbeat from core client for 30 sec - exiting 20:59:31 (72323): No heartbeat from core client for 30 sec - exiting 20:59:32 (72323): No heartbeat from core client for 30 sec - exiting 20:59:33 (72323): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 18:57:29	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	259,200	525,731	2.0283
25 Jul 2011 18:15:43	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	233,280	474,053	2.0321
25 Jul 2011 17:50:10	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	207,360	421,649	2.0334
25 Jul 2011 17:15:05	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	181,440	368,865	2.0330
25 Jul 2011 16:09:15	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	155,520	316,403	2.0345
25 Jul 2011 15:06:54	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	129,600	264,108	2.0379
25 Jul 2011 13:33:30	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	103,680	211,860	2.0434
25 Jul 2011 13:33:29	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	77,760	158,579	2.0393
25 Jul 2011 13:33:29	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	51,840	105,726	2.0395
10 Jul 2011 12:58:31	1006135	13116678	hadcm3n_yi61_1900_40_007356403_1	25,920	53,086	2.0481