Task 17302700

Name	hadcm3n_sbnx_1940_40_009111594_1
Workunit	9241930
Created	25 Oct 2014, 16:28:01 UTC
Sent	25 Oct 2014, 16:37:26 UTC
Report deadline	25 Jan 2015, 0:04:37 UTC
Received	29 Nov 2014, 14:20:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1170047
Run time	19 days 22 hours 5 min 36 sec
CPU time	18 days 2 hours 45 min 11 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.69 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.21</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:37:41 (48103): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:37:43 (48103): No heartbeat from core client for 30 sec - exiting 12:37:44 (48103): No heartbeat from core client for 30 sec - exiting 12:37:45 (48103): No heartbeat from core client for 30 sec - exiting 12:37:46 (48103): No heartbeat from core client for 30 sec - exiting 12:37:47 (48103): No heartbeat from core client for 30 sec - exiting 12:37:48 (48103): No heartbeat from core client for 30 sec - exiting 12:37:49 (48103): No heartbeat from core client for 30 sec - exiting 12:37:50 (48103): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation 12:39:24 (32223): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:41:10 (32237): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:41:13 (32237): No heartbeat from core client for 30 sec - exiting 12:41:14 (32237): No heartbeat from core client for 30 sec - exiting 12:41:15 (32237): No heartbeat from core client for 30 sec - exiting 12:41:16 (32237): No heartbeat from core client for 30 sec - exiting 12:41:17 (32237): No heartbeat from core client for 30 sec - exiting 12:41:18 (32237): No heartbeat from core client for 30 sec - exiting 12:41:19 (32237): No heartbeat from core client for 30 sec - exiting 12:41:20 (32237): No heartbeat from core client for 30 sec - exiting 12:41:21 (32237): No heartbeat from core client for 30 sec - exiting 12:41:22 (32237): No heartbeat from core client for 30 sec - exiting 12:41:23 (32237): No heartbeat from core client for 30 sec - exiting 12:41:24 (32237): No heartbeat from core client for 30 sec - exiting 12:41:25 (32237): No heartbeat from core client for 30 sec - exiting 12:41:26 (32237): No heartbeat from core client for 30 sec - exiting 12:42:58 (32280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:44:32 (32291): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:44:33 (32291): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation 12:46:11 (32335): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:52:46 (32351): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:52:49 (32351): No heartbeat from core client for 30 sec - exiting 12:54:24 (32418): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:55:57 (32430): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:58:37 (32470): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:00:13 (32490): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:00:15 (32490): No heartbeat from core client for 30 sec - exiting 13:03:16 (32530): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:04:51 (32548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:06:30 (32590): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:06:31 (32590): No heartbeat from core client for 30 sec - exiting 13:08:05 (32602): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:44 (32642): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:12:19 (32663): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:14:56 (32703): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:16:30 (32722): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:19:09 (32764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:44 (32783): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:21 (32823): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:24 (32823): No heartbeat from core client for 30 sec - exiting 13:23:25 (32823): No heartbeat from core client for 30 sec - exiting 13:25:10 (32844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:25:12 (32844): No heartbeat from core client for 30 sec - exiting 13:26:47 (32886): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:18 (32898): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:19 (32898): No heartbeat from core client for 30 sec - exiting 13:30:55 (32940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:32:30 (32959): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:35:08 (32999): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:38:02 (33019): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:39:39 (33065): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:41:15 (33079): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 13:44:01 (33123): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:44:04 (33123): No heartbeat from core client for 30 sec - exiting 13:45:54 (33144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:45:57 (33144): No heartbeat from core client for 30 sec - exiting 13:45:58 (33144): No heartbeat from core client for 30 sec - exiting 13:45:59 (33144): No heartbeat from core client for 30 sec - exiting 13:47:43 (33188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:49:20 (33206): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:58:00 (33241): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:58:02 (33241): No heartbeat from core client for 30 sec - exiting 13:58:03 (33241): No heartbeat from core client for 30 sec - exiting 13:59:33 (33299): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:59:35 (33299): No heartbeat from core client for 30 sec - exiting 14:01:11 (33334): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:01:12 (33334): No heartbeat from core client for 30 sec - exiting 14:01:13 (33334): No heartbeat from core client for 30 sec - exiting 14:01:14 (33334): No heartbeat from core client for 30 sec - exiting 14:01:15 (33334): No heartbeat from core client for 30 sec - exiting 14:02:49 (33351): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:51 (33351): No heartbeat from core client for 30 sec - exiting 14:06:41 (33386): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:06:43 (33386): No heartbeat from core client for 30 sec - exiting 14:06:44 (33386): No heartbeat from core client for 30 sec - exiting 14:06:45 (33386): No heartbeat from core client for 30 sec - exiting 14:08:11 (33419): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:15:22 (33455): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:16:59 (33500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:18:34 (33535): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:09 (33552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:10 (33552): No heartbeat from core client for 30 sec - exiting 14:22:53 (33587): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:24:27 (33614): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:33 (33649): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:36:33 (33680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:36:37 (33680): No heartbeat from core client for 30 sec - exiting 14:36:38 (33680): No heartbeat from core client for 30 sec - exiting 14:36:39 (33680): No heartbeat from core client for 30 sec - exiting 14:36:40 (33680): No heartbeat from core client for 30 sec - exiting 14:36:41 (33680): No heartbeat from core client for 30 sec - exiting 14:36:42 (33680): No heartbeat from core client for 30 sec - exiting 14:36:43 (33680): No heartbeat from core client for 30 sec - exiting 14:36:44 (33680): No heartbeat from core client for 30 sec - exiting 14:36:45 (33680): No heartbeat from core client for 30 sec - exiting 14:36:46 (33680): No heartbeat from core client for 30 sec - exiting 14:38:16 (33749): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:47:52 (33764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:47:55 (33764): No heartbeat from core client for 30 sec - exiting 14:47:56 (33764): No heartbeat from core client for 30 sec - exiting 14:47:57 (33764): No heartbeat from core client for 30 sec - exiting 14:47:58 (33764): No heartbeat from core client for 30 sec - exiting 14:47:59 (33764): No heartbeat from core client for 30 sec - exiting 14:48:00 (33764): No heartbeat from core client for 30 sec - exiting 14:48:01 (33764): No heartbeat from core client for 30 sec - exiting 14:48:02 (33764): No heartbeat from core client for 30 sec - exiting 14:48:03 (33764): No heartbeat from core client for 30 sec - exiting 14:49:37 (33850): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:51:13 (33866): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:51:16 (33866): No heartbeat from core client for 30 sec - exiting 14:51:17 (33866): No heartbeat from core client for 30 sec - exiting 14:51:18 (33866): No heartbeat from core client for 30 sec - exiting 14:51:19 (33866): No heartbeat from core client for 30 sec - exiting 14:51:20 (33866): No heartbeat from core client for 30 sec - exiting 14:51:21 (33866): No heartbeat from core client for 30 sec - exiting 14:51:22 (33866): No heartbeat from core client for 30 sec - exiting 14:51:23 (33866): No heartbeat from core client for 30 sec - exiting 14:51:24 (33866): No heartbeat from core client for 30 sec - exiting 14:51:25 (33866): No heartbeat from core client for 30 sec - exiting 14:51:26 (33866): No heartbeat from core client for 30 sec - exiting 14:51:27 (33866): No heartbeat from core client for 30 sec - exiting 14:51:28 (33866): No heartbeat from core client for 30 sec - exiting 14:51:29 (33866): No heartbeat from core client for 30 sec - exiting 14:51:30 (33866): No heartbeat from core client for 30 sec - exiting 14:51:31 (33866): No heartbeat from core client for 30 sec - exiting 14:51:32 (33866): No heartbeat from core client for 30 sec - exiting 14:51:33 (33866): No heartbeat from core client for 30 sec - exiting 14:51:34 (33866): No heartbeat from core client for 30 sec - exiting 14:51:35 (33866): No heartbeat from core client for 30 sec - exiting 14:51:36 (33866): No heartbeat from core client for 30 sec - exiting 14:51:37 (33866): No heartbeat from core client for 30 sec - exiting 14:51:38 (33866): No heartbeat from core client for 30 sec - exiting 14:51:39 (33866): No heartbeat from core client for 30 sec - exiting 14:51:40 (33866): No heartbeat from core client for 30 sec - exiting 14:51:41 (33866): No heartbeat from core client for 30 sec - exiting 14:51:42 (33866): No heartbeat from core client for 30 sec - exiting 14:51:43 (33866): No heartbeat from core client for 30 sec - exiting 14:51:44 (33866): No heartbeat from core client for 30 sec - exiting 14:51:45 (33866): No heartbeat from core client for 30 sec - exiting 14:51:46 (33866): No heartbeat from core client for 30 sec - exiting 14:51:47 (33866): No heartbeat from core client for 30 sec - exiting 14:51:48 (33866): No heartbeat from core client for 30 sec - exiting 14:51:49 (33866): No heartbeat from core client for 30 sec - exiting 14:51:50 (33866): No heartbeat from core client for 30 sec - exiting 14:51:51 (33866): No heartbeat from core client for 30 sec - exiting 14:51:52 (33866): No heartbeat from core client for 30 sec - exiting 14:51:53 (33866): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation 14:57:17 (33911): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:58:54 (33946): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:58:55 (33946): No heartbeat from core client for 30 sec - exiting 15:01:43 (33983): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:04:26 (34006): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:04:29 (34006): No heartbeat from core client for 30 sec - exiting 15:04:30 (34006): No heartbeat from core client for 30 sec - exiting 15:04:31 (34006): No heartbeat from core client for 30 sec - exiting 15:06:19 (34049): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:08:19 (34066): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:56 (34106): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:11:32 (34121): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:14:12 (34160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:51 (34180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:54 (34180): No heartbeat from core client for 30 sec - exiting 15:18:30 (34227): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:21:28 (34240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:21:31 (34240): No heartbeat from core client for 30 sec - exiting 15:21:32 (34240): No heartbeat from core client for 30 sec - exiting 15:21:33 (34240): No heartbeat from core client for 30 sec - exiting 15:21:34 (34240): No heartbeat from core client for 30 sec - exiting 15:21:35 (34240): No heartbeat from core client for 30 sec - exiting 15:21:36 (34240): No heartbeat from core client for 30 sec - exiting 15:23:20 (34293): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:26:39 (34305): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:26:42 (34305): No heartbeat from core client for 30 sec - exiting 15:26:43 (34305): No heartbeat from core client for 30 sec - exiting 15:26:44 (34305): No heartbeat from core client for 30 sec - exiting 15:26:45 (34305): No heartbeat from core client for 30 sec - exiting 15:26:46 (34305): No heartbeat from core client for 30 sec - exiting 15:26:47 (34305): No heartbeat from core client for 30 sec - exiting 15:30:41 (34351): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:36:09 (34379): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:36:11 (34379): No heartbeat from core client for 30 sec - exiting 15:36:12 (34379): No heartbeat from core client for 30 sec - exiting 15:36:13 (34379): No heartbeat from core client for 30 sec - exiting 15:36:14 (34379): No heartbeat from core client for 30 sec - exiting 15:39:00 (34439): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:41:55 (34458): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:43:39 (34507): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:45:21 (34520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:46:58 (34564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:48:37 (34579): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:48:39 (34579): No heartbeat from core client for 30 sec - exiting 15:51:15 (34618): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:57:50 (34637): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:57:53 (34637): No heartbeat from core client for 30 sec - exiting 15:57:54 (34637): No heartbeat from core client for 30 sec - exiting 15:57:55 (34637): No heartbeat from core client for 30 sec - exiting 15:57:57 (34637): No heartbeat from core client for 30 sec - exiting 16:00:03 (34706): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:12 (34706): No heartbeat from core client for 30 sec - exiting 16:03:14 (34720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:05:55 (34767): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:08:44 (34786): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:08:46 (34786): No heartbeat from core client for 30 sec - exiting 16:10:19 (34831): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:22 (34831): No heartbeat from core client for 30 sec - exiting 16:10:23 (34831): No heartbeat from core client for 30 sec - exiting 16:10:24 (34831): No heartbeat from core client for 30 sec - exiting 16:12:01 (34846): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:14:41 (34885): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:16:19 (34904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:16:20 (34904): No heartbeat from core client for 30 sec - exiting 16:18:57 (34945): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:20:33 (34964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:20:34 (34964): No heartbeat from core client for 30 sec - exiting 16:20:35 (34964): No heartbeat from core client for 30 sec - exiting 16:20:36 (34964): No heartbeat from core client for 30 sec - exiting 16:20:37 (34964): No heartbeat from core client for 30 sec - exiting 16:23:13 (35005): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:24:50 (35026): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:24:56 (35026): No heartbeat from core client for 30 sec - exiting 16:24:57 (35026): No heartbeat from core client for 30 sec - exiting 16:24:58 (35026): No heartbeat from core client for 30 sec - exiting 16:24:59 (35026): No heartbeat from core client for 30 sec - exiting 16:25:00 (35026): No heartbeat from core client for 30 sec - exiting 16:25:01 (35026): No heartbeat from core client for 30 sec - exiting 16:25:02 (35026): No heartbeat from core client for 30 sec - exiting 16:25:03 (35026): No heartbeat from core client for 30 sec - exiting 16:25:04 (35026): No heartbeat from core client for 30 sec - exiting 16:25:05 (35026): No heartbeat from core client for 30 sec - exiting 16:25:06 (35026): No heartbeat from core client for 30 sec - exiting 16:27:53 (35066): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:29:25 (35087): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:29:26 (35087): No heartbeat from core client for 30 sec - exiting 16:33:17 (35126): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:33:18 (35126): No heartbeat from core client for 30 sec - exiting 16:34:51 (35152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:37:55 (35194): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:39:29 (35212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:39:34 (35212): No heartbeat from core client for 30 sec - exiting 16:39:35 (35212): No heartbeat from core client for 30 sec - exiting 16:39:36 (35212): No heartbeat from core client for 30 sec - exiting 16:39:37 (35212): No heartbeat from core client for 30 sec - exiting 16:39:38 (35212): No heartbeat from core client for 30 sec - exiting 16:39:39 (35212): No heartbeat from core client for 30 sec - exiting 16:39:40 (35212): No heartbeat from core client for 30 sec - exiting 16:39:41 (35212): No heartbeat from core client for 30 sec - exiting 16:39:42 (35212): No heartbeat from core client for 30 sec - exiting 16:41:14 (35253): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:41:17 (35253): No heartbeat from core client for 30 sec - exiting 16:41:18 (35253): No heartbeat from core client for 30 sec - exiting 16:41:19 (35253): No heartbeat from core client for 30 sec - exiting 16:41:20 (35253): No heartbeat from core client for 30 sec - exiting 16:41:21 (35253): No heartbeat from core client for 30 sec - exiting 16:41:22 (35253): No heartbeat from core client for 30 sec - exiting 16:41:23 (35253): No heartbeat from core client for 30 sec - exiting 16:41:24 (35253): No heartbeat from core client for 30 sec - exiting 16:41:25 (35253): No heartbeat from core client for 30 sec - exiting 16:41:26 (35253): No heartbeat from core client for 30 sec - exiting 16:42:56 (35272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:35 (35304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:52:26 (35337): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:53:53 (35398): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:37 (35418): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:40 (35418): No heartbeat from core client for 30 sec - exiting 16:55:41 (35418): No heartbeat from core client for 30 sec - exiting 16:55:42 (35418): No heartbeat from core client for 30 sec - exiting 16:55:43 (35418): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 19:29:04 (35452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:06:17 (36549): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:06:19 (36549): No heartbeat from core client for 30 sec - exiting 07:06:20 (36549): No heartbeat from core client for 30 sec - exiting 07:06:21 (36549): No heartbeat from core client for 30 sec - exiting 07:06:22 (36549): No heartbeat from core client for 30 sec - exiting 07:06:23 (36549): No heartbeat from core client for 30 sec - exiting 07:07:54 (60980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:08:42 (61010): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:10:27 (61034): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:10:29 (61034): No heartbeat from core client for 30 sec - exiting 07:10:30 (61034): No heartbeat from core client for 30 sec - exiting 07:10:31 (61034): No heartbeat from core client for 30 sec - exiting 07:10:32 (61034): No heartbeat from core client for 30 sec - exiting 07:12:05 (61069): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:57:56 (61099): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:59 (61099): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x7004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x801404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62285,0xa05a1540) malloc: * error for object 0x801400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Suspended CPDN Monitor - Suspend request from BOINC... 12:52:58 (62285): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:52:59 (62285): No heartbeat from core client for 30 sec - exiting 12:53:00 (62285): No heartbeat from core client for 30 sec - exiting 12:53:01 (62285): No heartbeat from core client for 30 sec - exiting 13:55:16 (72897): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:55:18 (72897): No heartbeat from core client for 30 sec - exiting 13:55:19 (72897): No heartbeat from core client for 30 sec - exiting 13:55:20 (72897): No heartbeat from core client for 30 sec - exiting 13:55:21 (72897): No heartbeat from core client for 30 sec - exiting 13:55:22 (72897): No heartbeat from core client for 30 sec - exiting 13:55:23 (72897): No heartbeat from core client for 30 sec - exiting 13:56:54 (73328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:47:06 (73346): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:54:10 (74842): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 16:59:41 (23367): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:47 (23367): No heartbeat from core client for 30 sec - exiting 16:59:48 (23367): No heartbeat from core client for 30 sec - exiting 16:59:49 (23367): No heartbeat from core client for 30 sec - exiting 17:01:26 (23382): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:03:00 (23420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:04:35 (23427): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:04:37 (23427): No heartbeat from core client for 30 sec - exiting 17:04:38 (23427): No heartbeat from core client for 30 sec - exiting 17:08:34 (23467): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:08:39 (23467): No heartbeat from core client for 30 sec - exiting 17:08:40 (23467): No heartbeat from core client for 30 sec - exiting 17:08:41 (23467): No heartbeat from core client for 30 sec - exiting 17:08:42 (23467): No heartbeat from core client for 30 sec - exiting 17:10:19 (23487): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:11:55 (23526): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:11:56 (23526): No heartbeat from core client for 30 sec - exiting 17:13:30 (23534): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:32 (23534): No heartbeat from core client for 30 sec - exiting 17:15:06 (23572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:37 (23581): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:41 (23617): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:21:15 (23635): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:21:16 (23635): No heartbeat from core client for 30 sec - exiting 17:24:43 (23672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:26:28 (23692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:26:30 (23692): No heartbeat from core client for 30 sec - exiting 17:26:31 (23692): No heartbeat from core client for 30 sec - exiting 17:28:18 (23728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:29:56 (23742): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:41 (23777): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:45 (23777): No heartbeat from core client for 30 sec - exiting 17:36:46 (23777): No heartbeat from core client for 30 sec - exiting 17:36:47 (23777): No heartbeat from core client for 30 sec - exiting 17:36:48 (23777): No heartbeat from core client for 30 sec - exiting 17:36:49 (23777): No heartbeat from core client for 30 sec - exiting 17:36:50 (23777): No heartbeat from core client for 30 sec - exiting 17:36:51 (23777): No heartbeat from core client for 30 sec - exiting 17:36:52 (23777): No heartbeat from core client for 30 sec - exiting 17:36:53 (23777): No heartbeat from core client for 30 sec - exiting 17:36:54 (23777): No heartbeat from core client for 30 sec - exiting 17:36:55 (23777): No heartbeat from core client for 30 sec - exiting 17:36:56 (23777): No heartbeat from core client for 30 sec - exiting 17:36:57 (23777): No heartbeat from core client for 30 sec - exiting 17:36:58 (23777): No heartbeat from core client for 30 sec - exiting 17:38:30 (23817): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:42:23 (23852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:42:26 (23852): No heartbeat from core client for 30 sec - exiting 17:44:00 (23878): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:44:02 (23878): No heartbeat from core client for 30 sec - exiting 17:46:51 (23914): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:48:28 (23938): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:58:59 (23971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:59:02 (23971): No heartbeat from core client for 30 sec - exiting 17:59:03 (23971): No heartbeat from core client for 30 sec - exiting 17:59:04 (23971): No heartbeat from core client for 30 sec - exiting 18:00:43 (24032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:02:17 (24066): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:29 (24076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:31 (24076): No heartbeat from core client for 30 sec - exiting 18:09:18 (24122): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:09:20 (24122): No heartbeat from core client for 30 sec - exiting 18:10:56 (24144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:10:57 (24144): No heartbeat from core client for 30 sec - exiting 18:13:41 (24175): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:17:22 (24197): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:18:57 (24241): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:21:45 (24256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:21:48 (24256): No heartbeat from core client for 30 sec - exiting 18:23:24 (24296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:29:07 (24310): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:29:09 (24310): No heartbeat from core client for 30 sec - exiting 18:29:10 (24310): No heartbeat from core client for 30 sec - exiting 18:29:11 (24310): No heartbeat from core client for 30 sec - exiting 18:29:12 (24310): No heartbeat from core client for 30 sec - exiting 18:31:59 (24362): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:32 (24383): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:33 (24383): No heartbeat from core client for 30 sec - exiting 18:33:34 (24383): No heartbeat from core client for 30 sec - exiting 18:35:17 (24421): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:54 (24435): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:55 (24435): No heartbeat from core client for 30 sec - exiting 18:36:56 (24435): No heartbeat from core client for 30 sec - exiting 18:36:57 (24435): No heartbeat from core client for 30 sec - exiting 18:40:02 (24467): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:40:05 (24467): No heartbeat from core client for 30 sec - exiting 18:40:06 (24467): No heartbeat from core client for 30 sec - exiting 18:40:07 (24467): No heartbeat from core client for 30 sec - exiting 18:41:37 (24494): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:41:39 (24494): No heartbeat from core client for 30 sec - exiting 18:41:40 (24494): No heartbeat from core client for 30 sec - exiting 18:41:41 (24494): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation 18:43:14 (24523): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 18:44:52 (24543): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:49 (24575): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:22 (24603): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:23 (24603): No heartbeat from core client for 30 sec - exiting 18:57:29 (24630): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:59:16 (24685): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:01:00 (24710): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 19:03:00 (24732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:04 (24732): No heartbeat from core client for 30 sec - exiting 19:04:44 (24756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:04:46 (24756): No heartbeat from core client for 30 sec - exiting 19:04:47 (24756): No heartbeat from core client for 30 sec - exiting 19:06:25 (24773): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:02 (24802): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:04 (24802): No heartbeat from core client for 30 sec - exiting 19:09:36 (24818): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:15:13 (24850): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:26 (24887): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:29 (24887): No heartbeat from core client for 30 sec - exiting 19:19:30 (24887): No heartbeat from core client for 30 sec - exiting 19:24:39 (24929): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:24:43 (24929): No heartbeat from core client for 30 sec - exiting 19:24:44 (24929): No heartbeat from core client for 30 sec - exiting 19:24:45 (24929): No heartbeat from core client for 30 sec - exiting 19:24:46 (24929): No heartbeat from core client for 30 sec - exiting 19:26:18 (24965): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:26:20 (24965): No heartbeat from core client for 30 sec - exiting 19:26:21 (24965): No heartbeat from core client for 30 sec - exiting 19:26:22 (24965): No heartbeat from core client for 30 sec - exiting 19:26:23 (24965): No heartbeat from core client for 30 sec - exiting 19:26:24 (24965): No heartbeat from core client for 30 sec - exiting 19:27:54 (24995): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:29 (25013): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:26 (25043): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:34:06 (25068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:34:08 (25068): No heartbeat from core client for 30 sec - exiting 19:35:42 (25101): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:59 (25116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:41:32 (25161): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:43:12 (25177): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:44:49 (25207): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:24 (25225): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:25 (25225): No heartbeat from core client for 30 sec - exiting 19:48:06 (25260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:58:45 (25276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:58:47 (25276): No heartbeat from core client for 30 sec - exiting 19:58:48 (25276): No heartbeat from core client for 30 sec - exiting 19:58:49 (25276): No heartbeat from core client for 30 sec - exiting 20:00:20 (25353): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:04:41 (25368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:04:44 (25368): No heartbeat from core client for 30 sec - exiting 20:04:45 (25368): No heartbeat from core client for 30 sec - exiting 20:09:42 (25412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:13:38 (25445): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:13:40 (25445): No heartbeat from core client for 30 sec - exiting 20:13:41 (25445): No heartbeat from core client for 30 sec - exiting 20:13:42 (25445): No heartbeat from core client for 30 sec - exiting 20:13:43 (25445): No heartbeat from core client for 30 sec - exiting 20:13:44 (25445): No heartbeat from core client for 30 sec - exiting 20:13:45 (25445): No heartbeat from core client for 30 sec - exiting 20:18:54 (25487): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:45 (25523): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:50 (25523): No heartbeat from core client for 30 sec - exiting 20:32:12 (25594): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:47 (25625): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:48 (25625): No heartbeat from core client for 30 sec - exiting 20:36:20 (25657): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:36:23 (25657): No heartbeat from core client for 30 sec - exiting 20:36:24 (25657): No heartbeat from core client for 30 sec - exiting 20:36:25 (25657): No heartbeat from core client for 30 sec - exiting 20:36:26 (25657): No heartbeat from core client for 30 sec - exiting 20:48:29 (25678): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:48:33 (25678): No heartbeat from core client for 30 sec - exiting 20:48:34 (25678): No heartbeat from core client for 30 sec - exiting 20:48:35 (25678): No heartbeat from core client for 30 sec - exiting 20:50:08 (25769): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:51:45 (25784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:00:34 (25816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:02:15 (25872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:55 (25901): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:05:31 (25919): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:05:33 (25919): No heartbeat from core client for 30 sec - exiting 21:11:57 (25949): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:28 (25992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:16:07 (26021): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:21:21 (26046): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:23:23 (26097): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:25:00 (26114): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:32:30 (26144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:23 (26196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:24 (26196): No heartbeat from core client for 30 sec - exiting 21:37:57 (26233): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:38:01 (26233): No heartbeat from core client for 30 sec - exiting 21:38:02 (26233): No heartbeat from core client for 30 sec - exiting 21:38:03 (26233): No heartbeat from core client for 30 sec - exiting 21:38:04 (26233): No heartbeat from core client for 30 sec - exiting 21:38:05 (26233): No heartbeat from core client for 30 sec - exiting 21:38:06 (26233): No heartbeat from core client for 30 sec - exiting 21:38:07 (26233): No heartbeat from core client for 30 sec - exiting 21:38:08 (26233): No heartbeat from core client for 30 sec - exiting 21:38:09 (26233): No heartbeat from core client for 30 sec - exiting 21:39:43 (26257): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:39:45 (26257): No heartbeat from core client for 30 sec - exiting 21:39:46 (26257): No heartbeat from core client for 30 sec - exiting 21:39:47 (26257): No heartbeat from core client for 30 sec - exiting 21:42:32 (26288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:10 (26312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:12 (26312): No heartbeat from core client for 30 sec - exiting 21:45:13 (26312): No heartbeat from core client for 30 sec - exiting 21:45:14 (26312): No heartbeat from core client for 30 sec - exiting 21:45:15 (26312): No heartbeat from core client for 30 sec - exiting 21:45:16 (26312): No heartbeat from core client for 30 sec - exiting 21:54:14 (26353): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:54:18 (26353): No heartbeat from core client for 30 sec - exiting 21:54:19 (26353): No heartbeat from core client for 30 sec - exiting 21:55:54 (26410): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:00 (26437): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:06:33 (26496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:20:49 (26523): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:20:53 (26523): No heartbeat from core client for 30 sec - exiting 23:20:54 (26523): No heartbeat from core client for 30 sec - exiting 23:20:55 (26523): No heartbeat from core client for 30 sec - exiting 23:20:56 (26523): No heartbeat from core client for 30 sec - exiting 23:20:57 (26523): No heartbeat from core client for 30 sec - exiting 23:20:58 (26523): No heartbeat from core client for 30 sec - exiting 23:20:59 (26523): No heartbeat from core client for 30 sec - exiting 23:21:00 (26523): No heartbeat from core client for 30 sec - exiting 23:21:01 (26523): No heartbeat from core client for 30 sec - exiting 23:21:02 (26523): No heartbeat from core client for 30 sec - exiting 23:21:03 (26523): No heartbeat from core client for 30 sec - exiting 23:21:04 (26523): No heartbeat from core client for 30 sec - exiting 23:21:05 (26523): No heartbeat from core client for 30 sec - exiting 23:21:06 (26523): No heartbeat from core client for 30 sec - exiting 23:21:07 (26523): No heartbeat from core client for 30 sec - exiting 23:21:08 (26523): No heartbeat from core client for 30 sec - exiting 23:21:09 (26523): No heartbeat from core client for 30 sec - exiting 23:21:10 (26523): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 14:40:21 (27026): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:40:24 (27026): No heartbeat from core client for 30 sec - exiting 14:40:26 (27026): No heartbeat from core client for 30 sec - exiting 14:40:27 (27026): No heartbeat from core client for 30 sec - exiting 14:40:28 (27026): No heartbeat from core client for 30 sec - exiting 14:40:29 (27026): No heartbeat from core client for 30 sec - exiting 14:40:30 (27026): No heartbeat from core client for 30 sec - exiting 14:40:31 (27026): No heartbeat from core client for 30 sec - exiting 10:02:24 (32846): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:02:25 (32846): No heartbeat from core client for 30 sec - exiting 10:02:26 (32846): No heartbeat from core client for 30 sec - exiting 10:02:27 (32846): No heartbeat from core client for 30 sec - exiting 10:02:28 (32846): No heartbeat from core client for 30 sec - exiting 10:02:29 (32846): No heartbeat from core client for 30 sec - exiting 10:02:30 (32846): No heartbeat from core client for 30 sec - exiting 10:02:31 (32846): No heartbeat from core client for 30 sec - exiting 10:02:32 (32846): No heartbeat from core client for 30 sec - exiting 10:02:33 (32846): No heartbeat from core client for 30 sec - exiting 10:02:34 (32846): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:32:41 (49857): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:50:51 (41792): No heartbeat from core client for 30 sec - exiting 04:50:52 (41792): No heartbeat from core client for 30 sec - exiting 04:50:54 (41792): No heartbeat from core client for 30 sec - exiting 04:50:55 (41792): No heartbeat from core client for 30 sec - exiting 04:50:56 (41792): No heartbeat from core client for 30 sec - exiting 04:50:57 (41792): No heartbeat from core client for 30 sec - exiting 04:50:58 (41792): No heartbeat from core client for 30 sec - exiting 04:50:59 (41792): No heartbeat from core client for 30 sec - exiting 04:51:00 (41792): No heartbeat from core client for 30 sec - exiting 04:51:01 (41792): No heartbeat from core client for 30 sec - exiting 04:51:02 (41792): No heartbeat from core client for 30 sec - exiting 04:51:03 (41792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_sbnx_1940_40_009111594/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Nov 2014 11:49:29	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	777,600	1,565,157	2.0128
28 Nov 2014 15:41:35	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	751,680	1,508,698	2.0071
27 Nov 2014 21:39:04	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	725,760	1,452,812	2.0018
27 Nov 2014 02:28:52	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	699,840	1,398,287	1.9980
26 Nov 2014 09:37:57	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	673,920	1,344,214	1.9946
25 Nov 2014 16:35:00	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	648,000	1,289,965	1.9907
24 Nov 2014 22:20:54	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	622,080	1,233,780	1.9833
23 Nov 2014 23:45:47	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	596,160	1,179,562	1.9786
22 Nov 2014 23:13:29	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	570,240	1,125,302	1.9734
22 Nov 2014 01:34:47	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	544,320	1,072,125	1.9697
21 Nov 2014 06:53:49	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	518,400	1,019,761	1.9671
20 Nov 2014 07:08:21	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	492,480	967,071	1.9637
19 Nov 2014 10:13:52	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	466,560	911,869	1.9545
18 Nov 2014 16:36:57	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	440,640	855,375	1.9412
17 Nov 2014 20:25:03	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	414,720	802,064	1.9340
17 Nov 2014 02:26:59	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	388,800	746,545	1.9201
16 Nov 2014 10:22:38	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	362,880	689,975	1.9014
15 Nov 2014 17:37:21	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	336,960	634,399	1.8827
15 Nov 2014 01:54:46	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	311,040	578,776	1.8608
14 Nov 2014 08:51:36	1170047	17302700	hadcm3n_sbnx_1940_40_009111594_1	285,120	523,703	1.8368