Task 16004657

Name	hadcm3n_zk0h_1920_40_008324587_3
Workunit	8475722
Created	5 Sep 2013, 15:29:01 UTC
Sent	5 Sep 2013, 15:52:41 UTC
Report deadline	5 Dec 2013, 23:19:52 UTC
Received	19 Oct 2013, 13:55:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1286653
Run time	20 days 8 hours 7 min 11 sec
CPU time	17 days 8 hours 22 min 23 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.71 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10608, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4584, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:47:30 (10164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:47:31 (10164): No heartbeat from core client for 30 sec - exiting 13:47:32 (10164): No heartbeat from core client for 30 sec - exiting 13:47:33 (10164): No heartbeat from core client for 30 sec - exiting 13:47:34 (10164): No heartbeat from core client for 30 sec - exiting 13:47:35 (10164): No heartbeat from core client for 30 sec - exiting 13:47:36 (10164): No heartbeat from core client for 30 sec - exiting 13:47:37 (10164): No heartbeat from core client for 30 sec - exiting 13:47:38 (10164): No heartbeat from core client for 30 sec - exiting 13:47:39 (10164): No heartbeat from core client for 30 sec - exiting 13:47:40 (10164): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6540, iMonCtr=1 Model crash detected, will try to restart... 07:37:54 (7212): No heartbeat from core client for 30 sec - exiting 07:37:55 (7212): No heartbeat from core client for 30 sec - exiting 07:37:56 (7212): No heartbeat from core client for 30 sec - exiting 07:37:57 (7212): No heartbeat from core client for 30 sec - exiting 07:37:58 (7212): No heartbeat from core client for 30 sec - exiting 07:37:59 (7212): No heartbeat from core client for 30 sec - exiting 07:38:00 (7212): No heartbeat from core client for 30 sec - exiting 07:38:01 (7212): No heartbeat from core client for 30 sec - exiting 07:38:03 (7212): No heartbeat from core client for 30 sec - exiting 07:38:04 (7212): No heartbeat from core client for 30 sec - exiting 07:38:05 (7212): No heartbeat from core client for 30 sec - exiting 07:38:06 (7212): No heartbeat from core client for 30 sec - exiting 07:38:07 (7212): No heartbeat from core client for 30 sec - exiting 07:38:08 (7212): No heartbeat from core client for 30 sec - exiting 07:38:09 (7212): No heartbeat from core client for 30 sec - exiting 07:38:10 (7212): No heartbeat from core client for 30 sec - exiting 07:38:11 (7212): No heartbeat from core client for 30 sec - exiting 07:38:12 (7212): No heartbeat from core client for 30 sec - exiting 07:38:13 (7212): No heartbeat from core client for 30 sec - exiting 07:38:14 (7212): No heartbeat from core client for 30 sec - exiting 07:38:15 (7212): No heartbeat from core client for 30 sec - exiting 07:38:17 (7212): No heartbeat from core client for 30 sec - exiting 07:38:18 (7212): No heartbeat from core client for 30 sec - exiting 07:38:19 (7212): No heartbeat from core client for 30 sec - exiting 07:38:20 (7212): No heartbeat from core client for 30 sec - exiting 07:38:21 (7212): No heartbeat from core client for 30 sec - exiting 07:38:22 (7212): No heartbeat from core client for 30 sec - exiting 07:38:23 (7212): No heartbeat from core client for 30 sec - exiting 07:38:24 (7212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:38:25 (7212): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 07:07:52 (8136): No heartbeat from core client for 30 sec - exiting 07:07:54 (8136): No heartbeat from core client for 30 sec - exiting 07:07:55 (8136): No heartbeat from core client for 30 sec - exiting 07:07:56 (8136): No heartbeat from core client for 30 sec - exiting 07:07:57 (8136): No heartbeat from core client for 30 sec - exiting 07:07:58 (8136): No heartbeat from core client for 30 sec - exiting 07:07:59 (8136): No heartbeat from core client for 30 sec - exiting 07:08:00 (8136): No heartbeat from core client for 30 sec - exiting 07:08:01 (8136): No heartbeat from core client for 30 sec - exiting 07:08:02 (8136): No heartbeat from core client for 30 sec - exiting 07:08:03 (8136): No heartbeat from core client for 30 sec - exiting 07:08:05 (8136): No heartbeat from core client for 30 sec - exiting 07:08:06 (8136): No heartbeat from core client for 30 sec - exiting 07:08:07 (8136): No heartbeat from core client for 30 sec - exiting 07:08:08 (8136): No heartbeat from core client for 30 sec - exiting 07:08:09 (8136): No heartbeat from core client for 30 sec - exiting 07:08:10 (8136): No heartbeat from core client for 30 sec - exiting 07:08:11 (8136): No heartbeat from core client for 30 sec - exiting 07:08:12 (8136): No heartbeat from core client for 30 sec - exiting 07:08:13 (8136): No heartbeat from core client for 30 sec - exiting 07:08:14 (8136): No heartbeat from core client for 30 sec - exiting 07:08:15 (8136): No heartbeat from core client for 30 sec - exiting 07:08:17 (8136): No heartbeat from core client for 30 sec - exiting 07:08:18 (8136): No heartbeat from core client for 30 sec - exiting 07:08:19 (8136): No heartbeat from core client for 30 sec - exiting 07:08:20 (8136): No heartbeat from core client for 30 sec - exiting 07:08:21 (8136): No heartbeat from core client for 30 sec - exiting 07:08:22 (8136): No heartbeat from core client for 30 sec - exiting 07:08:23 (8136): No heartbeat from core client for 30 sec - exiting 07:08:24 (8136): No heartbeat from core client for 30 sec - exiting 07:08:25 (8136): No heartbeat from core client for 30 sec - exiting 07:08:26 (8136): No heartbeat from core client for 30 sec - exiting 07:08:27 (8136): No heartbeat from core client for 30 sec - exiting 07:08:28 (8136): No heartbeat from core client for 30 sec - exiting 07:08:29 (8136): No heartbeat from core client for 30 sec - exiting 07:08:30 (8136): No heartbeat from core client for 30 sec - exiting 07:08:32 (8136): No heartbeat from core client for 30 sec - exiting 07:08:33 (8136): No heartbeat from core client for 30 sec - exiting 07:08:34 (8136): No heartbeat from core client for 30 sec - exiting 07:08:35 (8136): No heartbeat from core client for 30 sec - exiting 07:08:36 (8136): No heartbeat from core client for 30 sec - exiting 07:08:37 (8136): No heartbeat from core client for 30 sec - exiting 07:08:38 (8136): No heartbeat from core client for 30 sec - exiting 07:08:39 (8136): No heartbeat from core client for 30 sec - exiting 07:08:40 (8136): No heartbeat from core client for 30 sec - exiting 07:08:41 (8136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:21:47 (7516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: Access is denied. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12400, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6928, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6016, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3532, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:37:29 (7880): No heartbeat from core client for 30 sec - exiting 12:37:31 (7880): No heartbeat from core client for 30 sec - exiting 12:37:32 (7880): No heartbeat from core client for 30 sec - exiting 12:37:33 (7880): No heartbeat from core client for 30 sec - exiting 12:37:34 (7880): No heartbeat from core client for 30 sec - exiting 12:37:35 (7880): No heartbeat from core client for 30 sec - exiting 12:37:36 (7880): No heartbeat from core client for 30 sec - exiting 12:37:37 (7880): No heartbeat from core client for 30 sec - exiting 12:37:38 (7880): No heartbeat from core client for 30 sec - exiting 12:37:39 (7880): No heartbeat from core client for 30 sec - exiting 12:37:40 (7880): No heartbeat from core client for 30 sec - exiting 12:37:41 (7880): No heartbeat from core client for 30 sec - exiting 12:37:43 (7880): No heartbeat from core client for 30 sec - exiting 12:37:44 (7880): No heartbeat from core client for 30 sec - exiting 12:37:45 (7880): No heartbeat from core client for 30 sec - exiting 12:37:46 (7880): No heartbeat from core client for 30 sec - exiting 12:37:47 (7880): No heartbeat from core client for 30 sec - exiting 12:37:48 (7880): No heartbeat from core client for 30 sec - exiting 12:37:49 (7880): No heartbeat from core client for 30 sec - exiting 12:37:50 (7880): No heartbeat from core client for 30 sec - exiting 12:37:51 (7880): No heartbeat from core client for 30 sec - exiting 12:37:52 (7880): No heartbeat from core client for 30 sec - exiting 12:37:53 (7880): No heartbeat from core client for 30 sec - exiting 12:37:55 (7880): No heartbeat from core client for 30 sec - exiting 12:37:56 (7880): No heartbeat from core client for 30 sec - exiting 12:37:57 (7880): No heartbeat from core client for 30 sec - exiting 12:37:58 (7880): No heartbeat from core client for 30 sec - exiting 12:37:59 (7880): No heartbeat from core client for 30 sec - exiting 12:38:00 (7880): No heartbeat from core client for 30 sec - exiting 12:38:01 (7880): No heartbeat from core client for 30 sec - exiting 12:38:02 (7880): No heartbeat from core client for 30 sec - exiting 12:38:03 (7880): No heartbeat from core client for 30 sec - exiting 12:38:04 (7880): No heartbeat from core client for 30 sec - exiting 12:38:05 (7880): No heartbeat from core client for 30 sec - exiting 12:38:07 (7880): No heartbeat from core client for 30 sec - exiting 12:38:08 (7880): No heartbeat from core client for 30 sec - exiting 12:38:09 (7880): No heartbeat from core client for 30 sec - exiting 12:38:10 (7880): No heartbeat from core client for 30 sec - exiting 12:38:11 (7880): No heartbeat from core client for 30 sec - exiting 12:38:12 (7880): No heartbeat from core client for 30 sec - exiting 12:38:13 (7880): No heartbeat from core client for 30 sec - exiting 12:38:14 (7880): No heartbeat from core client for 30 sec - exiting 12:38:15 (7880): No heartbeat from core client for 30 sec - exiting 12:38:16 (7880): No heartbeat from core client for 30 sec - exiting 12:38:18 (7880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:25 (7376): No heartbeat from core client for 30 sec - exiting 07:09:27 (7376): No heartbeat from core client for 30 sec - exiting 07:09:28 (7376): No heartbeat from core client for 30 sec - exiting 07:09:29 (7376): No heartbeat from core client for 30 sec - exiting 07:09:30 (7376): No heartbeat from core client for 30 sec - exiting 07:09:31 (7376): No heartbeat from core client for 30 sec - exiting 07:09:32 (7376): No heartbeat from core client for 30 sec - exiting 07:09:33 (7376): No heartbeat from core client for 30 sec - exiting 07:09:35 (7376): No heartbeat from core client for 30 sec - exiting 07:09:36 (7376): No heartbeat from core client for 30 sec - exiting 07:09:37 (7376): No heartbeat from core client for 30 sec - exiting 07:09:38 (7376): No heartbeat from core client for 30 sec - exiting 07:09:39 (7376): No heartbeat from core client for 30 sec - exiting 07:09:40 (7376): No heartbeat from core client for 30 sec - exiting 07:09:41 (7376): No heartbeat from core client for 30 sec - exiting 07:09:42 (7376): No heartbeat from core client for 30 sec - exiting 07:09:43 (7376): No heartbeat from core client for 30 sec - exiting 07:09:44 (7376): No heartbeat from core client for 30 sec - exiting 07:09:46 (7376): No heartbeat from core client for 30 sec - exiting 07:09:47 (7376): No heartbeat from core client for 30 sec - exiting 07:09:48 (7376): No heartbeat from core client for 30 sec - exiting 07:09:49 (7376): No heartbeat from core client for 30 sec - exiting 07:09:50 (7376): No heartbeat from core client for 30 sec - exiting 07:09:51 (7376): No heartbeat from core client for 30 sec - exiting 07:09:52 (7376): No heartbeat from core client for 30 sec - exiting 07:09:53 (7376): No heartbeat from core client for 30 sec - exiting 07:09:54 (7376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:13:09 (6536): No heartbeat from core client for 30 sec - exiting 07:13:10 (6536): No heartbeat from core client for 30 sec - exiting 07:13:11 (6536): No heartbeat from core client for 30 sec - exiting 07:13:12 (6536): No heartbeat from core client for 30 sec - exiting 07:13:13 (6536): No heartbeat from core client for 30 sec - exiting 07:13:14 (6536): No heartbeat from core client for 30 sec - exiting 07:13:15 (6536): No heartbeat from core client for 30 sec - exiting 07:13:16 (6536): No heartbeat from core client for 30 sec - exiting 07:13:17 (6536): No heartbeat from core client for 30 sec - exiting 07:13:19 (6536): No heartbeat from core client for 30 sec - exiting 07:13:20 (6536): No heartbeat from core client for 30 sec - exiting 07:13:21 (6536): No heartbeat from core client for 30 sec - exiting 07:13:22 (6536): No heartbeat from core client for 30 sec - exiting 07:13:23 (6536): No heartbeat from core client for 30 sec - exiting 07:13:24 (6536): No heartbeat from core client for 30 sec - exiting 07:13:25 (6536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:11:31 (8064): No heartbeat from core client for 30 sec - exiting 07:11:32 (8064): No heartbeat from core client for 30 sec - exiting 07:11:33 (8064): No heartbeat from core client for 30 sec - exiting 07:11:34 (8064): No heartbeat from core client for 30 sec - exiting 07:11:35 (8064): No heartbeat from core client for 30 sec - exiting 07:11:36 (8064): No heartbeat from core client for 30 sec - exiting 07:11:37 (8064): No heartbeat from core client for 30 sec - exiting 07:11:38 (8064): No heartbeat from core client for 30 sec - exiting 07:11:39 (8064): No heartbeat from core client for 30 sec - exiting 07:11:40 (8064): No heartbeat from core client for 30 sec - exiting 07:11:41 (8064): No heartbeat from core client for 30 sec - exiting 07:11:42 (8064): No heartbeat from core client for 30 sec - exiting 07:11:43 (8064): No heartbeat from core client for 30 sec - exiting 07:11:44 (8064): No heartbeat from core client for 30 sec - exiting 07:11:45 (8064): No heartbeat from core client for 30 sec - exiting 07:11:46 (8064): No heartbeat from core client for 30 sec - exiting 07:11:47 (8064): No heartbeat from core client for 30 sec - exiting 07:11:48 (8064): No heartbeat from core client for 30 sec - exiting 07:11:49 (8064): No heartbeat from core client for 30 sec - exiting 07:11:50 (8064): No heartbeat from core client for 30 sec - exiting 07:11:52 (8064): No heartbeat from core client for 30 sec - exiting 07:11:53 (8064): No heartbeat from core client for 30 sec - exiting 07:11:54 (8064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:03:00 (6376): No heartbeat from core client for 30 sec - exiting 07:03:01 (6376): No heartbeat from core client for 30 sec - exiting 07:03:02 (6376): No heartbeat from core client for 30 sec - exiting 07:03:03 (6376): No heartbeat from core client for 30 sec - exiting 07:03:04 (6376): No heartbeat from core client for 30 sec - exiting 07:03:05 (6376): No heartbeat from core client for 30 sec - exiting 07:03:06 (6376): No heartbeat from core client for 30 sec - exiting 07:03:07 (6376): No heartbeat from core client for 30 sec - exiting 07:03:08 (6376): No heartbeat from core client for 30 sec - exiting 07:03:09 (6376): No heartbeat from core client for 30 sec - exiting 07:03:10 (6376): No heartbeat from core client for 30 sec - exiting 07:03:12 (6376): No heartbeat from core client for 30 sec - exiting 07:03:13 (6376): No heartbeat from core client for 30 sec - exiting 07:03:14 (6376): No heartbeat from core client for 30 sec - exiting 07:03:15 (6376): No heartbeat from core client for 30 sec - exiting 07:03:16 (6376): No heartbeat from core client for 30 sec - exiting 07:03:17 (6376): No heartbeat from core client for 30 sec - exiting 07:03:18 (6376): No heartbeat from core client for 30 sec - exiting 07:03:19 (6376): No heartbeat from core client for 30 sec - exiting 07:03:20 (6376): No heartbeat from core client for 30 sec - exiting 07:03:21 (6376): No heartbeat from core client for 30 sec - exiting 07:03:22 (6376): No heartbeat from core client for 30 sec - exiting 07:03:23 (6376): No heartbeat from core client for 30 sec - exiting 07:03:24 (6376): No heartbeat from core client for 30 sec - exiting 07:03:25 (6376): No heartbeat from core client for 30 sec - exiting 07:03:27 (6376): No heartbeat from core client for 30 sec - exiting 07:03:28 (6376): No heartbeat from core client for 30 sec - exiting 07:03:29 (6376): No heartbeat from core client for 30 sec - exiting 07:03:30 (6376): No heartbeat from core client for 30 sec - exiting 07:03:31 (6376): No heartbeat from core client for 30 sec - exiting 07:03:32 (6376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:09:03 (7540): No heartbeat from core client for 30 sec - exiting 07:09:04 (7540): No heartbeat from core client for 30 sec - exiting 07:09:05 (7540): No heartbeat from core client for 30 sec - exiting 07:09:07 (7540): No heartbeat from core client for 30 sec - exiting 07:09:08 (7540): No heartbeat from core client for 30 sec - exiting 07:09:09 (7540): No heartbeat from core client for 30 sec - exiting 07:09:10 (7540): No heartbeat from core client for 30 sec - exiting 07:09:11 (7540): No heartbeat from core client for 30 sec - exiting 07:09:12 (7540): No heartbeat from core client for 30 sec - exiting 07:09:13 (7540): No heartbeat from core client for 30 sec - exiting 07:09:14 (7540): No heartbeat from core client for 30 sec - exiting 07:09:15 (7540): No heartbeat from core client for 30 sec - exiting 07:09:16 (7540): No heartbeat from core client for 30 sec - exiting 07:09:17 (7540): No heartbeat from core client for 30 sec - exiting 07:09:18 (7540): No heartbeat from core client for 30 sec - exiting 07:09:20 (7540): No heartbeat from core client for 30 sec - exiting 07:09:21 (7540): No heartbeat from core client for 30 sec - exiting 07:09:22 (7540): No heartbeat from core client for 30 sec - exiting 07:09:23 (7540): No heartbeat from core client for 30 sec - exiting 07:09:24 (7540): No heartbeat from core client for 30 sec - exiting 07:09:25 (7540): No heartbeat from core client for 30 sec - exiting 07:09:26 (7540): No heartbeat from core client for 30 sec - exiting 07:09:27 (7540): No heartbeat from core client for 30 sec - exiting 07:09:28 (7540): No heartbeat from core client for 30 sec - exiting 07:09:29 (7540): No heartbeat from core client for 30 sec - exiting 07:09:30 (7540): No heartbeat from core client for 30 sec - exiting 07:09:31 (7540): No heartbeat from core client for 30 sec - exiting 07:09:32 (7540): No heartbeat from core client for 30 sec - exiting 07:09:34 (7540): No heartbeat from core client for 30 sec - exiting 07:09:35 (7540): No heartbeat from core client for 30 sec - exiting 07:09:36 (7540): No heartbeat from core client for 30 sec - exiting 07:09:37 (7540): No heartbeat from core client for 30 sec - exiting 07:09:38 (7540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:00:43 (7376): No heartbeat from core client for 30 sec - exiting 07:00:44 (7376): No heartbeat from core client for 30 sec - exiting 07:00:45 (7376): No heartbeat from core client for 30 sec - exiting 07:00:46 (7376): No heartbeat from core client for 30 sec - exiting 07:00:47 (7376): No heartbeat from core client for 30 sec - exiting 07:00:48 (7376): No heartbeat from core client for 30 sec - exiting 07:00:49 (7376): No heartbeat from core client for 30 sec - exiting 07:00:51 (7376): No heartbeat from core client for 30 sec - exiting 07:00:52 (7376): No heartbeat from core client for 30 sec - exiting 07:00:53 (7376): No heartbeat from core client for 30 sec - exiting 07:00:54 (7376): No heartbeat from core client for 30 sec - exiting 07:00:55 (7376): No heartbeat from core client for 30 sec - exiting 07:00:56 (7376): No heartbeat from core client for 30 sec - exiting 07:00:57 (7376): No heartbeat from core client for 30 sec - exiting 07:00:58 (7376): No heartbeat from core client for 30 sec - exiting 07:00:59 (7376): No heartbeat from core client for 30 sec - exiting 07:01:00 (7376): No heartbeat from core client for 30 sec - exiting 07:01:01 (7376): No heartbeat from core client for 30 sec - exiting 07:01:02 (7376): No heartbeat from core client for 30 sec - exiting 07:01:03 (7376): No heartbeat from core client for 30 sec - exiting 07:01:04 (7376): No heartbeat from core client for 30 sec - exiting 07:01:06 (7376): No heartbeat from core client for 30 sec - exiting 07:01:07 (7376): No heartbeat from core client for 30 sec - exiting 07:01:08 (7376): No heartbeat from core client for 30 sec - exiting 07:01:09 (7376): No heartbeat from core client for 30 sec - exiting 07:01:10 (7376): No heartbeat from core client for 30 sec - exiting 07:01:11 (7376): No heartbeat from core client for 30 sec - exiting 07:01:12 (7376): No heartbeat from core client for 30 sec - exiting 07:01:13 (7376): No heartbeat from core client for 30 sec - exiting 07:01:14 (7376): No heartbeat from core client for 30 sec - exiting 07:01:15 (7376): No heartbeat from core client for 30 sec - exiting 07:01:16 (7376): No heartbeat from core client for 30 sec - exiting 07:01:18 (7376): No heartbeat from core client for 30 sec - exiting 07:01:19 (7376): No heartbeat from core client for 30 sec - exiting 07:01:20 (7376): No heartbeat from core client for 30 sec - exiting 07:01:21 (7376): No heartbeat from core client for 30 sec - exiting 07:01:22 (7376): No heartbeat from core client for 30 sec - exiting 07:01:23 (7376): No heartbeat from core client for 30 sec - exiting 07:01:24 (7376): No heartbeat from core client for 30 sec - exiting 07:01:25 (7376): No heartbeat from core client for 30 sec - exiting 07:01:26 (7376): No heartbeat from core client for 30 sec - exiting 07:01:27 (7376): No heartbeat from core client for 30 sec - exiting 07:01:28 (7376): No heartbeat from core client for 30 sec - exiting 07:01:29 (7376): No heartbeat from core client for 30 sec - exiting 07:01:30 (7376): No heartbeat from core client for 30 sec - exiting 07:01:31 (7376): No heartbeat from core client for 30 sec - exiting 07:01:33 (7376): No heartbeat from core client for 30 sec - exiting 07:01:34 (7376): No heartbeat from core client for 30 sec - exiting 07:01:35 (7376): No heartbeat from core client for 30 sec - exiting 07:01:36 (7376): No heartbeat from core client for 30 sec - exiting 07:01:37 (7376): No heartbeat from core client for 30 sec - exiting 07:01:38 (7376): No heartbeat from core client for 30 sec - exiting 07:01:39 (7376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:10:30 (6648): No heartbeat from core client for 30 sec - exiting 07:10:31 (6648): No heartbeat from core client for 30 sec - exiting 07:10:32 (6648): No heartbeat from core client for 30 sec - exiting 07:10:33 (6648): No heartbeat from core client for 30 sec - exiting 07:10:34 (6648): No heartbeat from core client for 30 sec - exiting 07:10:35 (6648): No heartbeat from core client for 30 sec - exiting 07:10:36 (6648): No heartbeat from core client for 30 sec - exiting 07:10:37 (6648): No heartbeat from core client for 30 sec - exiting 07:10:39 (6648): No heartbeat from core client for 30 sec - exiting 07:10:40 (6648): No heartbeat from core client for 30 sec - exiting 07:10:41 (6648): No heartbeat from core client for 30 sec - exiting 07:10:42 (6648): No heartbeat from core client for 30 sec - exiting 07:10:43 (6648): No heartbeat from core client for 30 sec - exiting 07:10:44 (6648): No heartbeat from core client for 30 sec - exiting 07:10:45 (6648): No heartbeat from core client for 30 sec - exiting 07:10:46 (6648): No heartbeat from core client for 30 sec - exiting 07:10:47 (6648): No heartbeat from core client for 30 sec - exiting 07:10:48 (6648): No heartbeat from core client for 30 sec - exiting 07:10:50 (6648): No heartbeat from core client for 30 sec - exiting 07:10:51 (6648): No heartbeat from core client for 30 sec - exiting 07:10:52 (6648): No heartbeat from core client for 30 sec - exiting 07:10:53 (6648): No heartbeat from core client for 30 sec - exiting 07:10:54 (6648): No heartbeat from core client for 30 sec - exiting 07:10:55 (6648): No heartbeat from core client for 30 sec - exiting 07:10:56 (6648): No heartbeat from core client for 30 sec - exiting 07:10:57 (6648): No heartbeat from core client for 30 sec - exiting 07:10:58 (6648): No heartbeat from core client for 30 sec - exiting 07:10:59 (6648): No heartbeat from core client for 30 sec - exiting 07:11:00 (6648): No heartbeat from core client for 30 sec - exiting 07:11:01 (6648): No heartbeat from core client for 30 sec - exiting 07:11:02 (6648): No heartbeat from core client for 30 sec - exiting 07:11:03 (6648): No heartbeat from core client for 30 sec - exiting 07:11:04 (6648): No heartbeat from core client for 30 sec - exiting 07:11:05 (6648): No heartbeat from core client for 30 sec - exiting 07:11:07 (6648): No heartbeat from core client for 30 sec - exiting 07:11:08 (6648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:13:10 (6620): No heartbeat from core client for 30 sec - exiting 14:13:11 (6620): No heartbeat from core client for 30 sec - exiting 14:13:12 (6620): No heartbeat from core client for 30 sec - exiting 14:13:13 (6620): No heartbeat from core client for 30 sec - exiting 14:13:14 (6620): No heartbeat from core client for 30 sec - exiting 14:13:15 (6620): No heartbeat from core client for 30 sec - exiting 14:13:16 (6620): No heartbeat from core client for 30 sec - exiting 14:13:17 (6620): No heartbeat from core client for 30 sec - exiting 14:13:18 (6620): No heartbeat from core client for 30 sec - exiting 14:13:19 (6620): No heartbeat from core client for 30 sec - exiting 14:13:20 (6620): No heartbeat from core client for 30 sec - exiting 14:13:21 (6620): No heartbeat from core client for 30 sec - exiting 14:13:23 (6620): No heartbeat from core client for 30 sec - exiting 14:13:24 (6620): No heartbeat from core client for 30 sec - exiting 14:13:25 (6620): No heartbeat from core client for 30 sec - exiting 14:13:26 (6620): No heartbeat from core client for 30 sec - exiting 14:13:27 (6620): No heartbeat from core client for 30 sec - exiting 14:13:28 (6620): No heartbeat from core client for 30 sec - exiting 14:13:29 (6620): No heartbeat from core client for 30 sec - exiting 14:13:30 (6620): No heartbeat from core client for 30 sec - exiting 14:13:31 (6620): No heartbeat from core client for 30 sec - exiting 14:13:32 (6620): No heartbeat from core client for 30 sec - exiting 14:13:33 (6620): No heartbeat from core client for 30 sec - exiting 14:13:35 (6620): No heartbeat from core client for 30 sec - exiting 14:13:36 (6620): No heartbeat from core client for 30 sec - exiting 14:13:37 (6620): No heartbeat from core client for 30 sec - exiting 14:13:38 (6620): No heartbeat from core client for 30 sec - exiting 14:13:39 (6620): No heartbeat from core client for 30 sec - exiting 14:13:40 (6620): No heartbeat from core client for 30 sec - exiting 14:13:41 (6620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:46 (6572): No heartbeat from core client for 30 sec - exiting 14:25:48 (6572): No heartbeat from core client for 30 sec - exiting 14:25:49 (6572): No heartbeat from core client for 30 sec - exiting 14:25:50 (6572): No heartbeat from core client for 30 sec - exiting 14:25:51 (6572): No heartbeat from core client for 30 sec - exiting 14:25:52 (6572): No heartbeat from core client for 30 sec - exiting 14:25:53 (6572): No heartbeat from core client for 30 sec - exiting 14:25:54 (6572): No heartbeat from core client for 30 sec - exiting 14:25:55 (6572): No heartbeat from core client for 30 sec - exiting 14:25:56 (6572): No heartbeat from core client for 30 sec - exiting 14:25:58 (6572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:59 (6572): No heartbeat from core client for 30 sec - exiting 08:28:27 (7148): No heartbeat from core client for 30 sec - exiting 08:28:28 (7148): No heartbeat from core client for 30 sec - exiting 08:28:29 (7148): No heartbeat from core client for 30 sec - exiting 08:28:30 (7148): No heartbeat from core client for 30 sec - exiting 08:28:31 (7148): No heartbeat from core client for 30 sec - exiting 08:28:32 (7148): No heartbeat from core client for 30 sec - exiting 08:28:33 (7148): No heartbeat from core client for 30 sec - exiting 08:28:34 (7148): No heartbeat from core client for 30 sec - exiting 08:28:35 (7148): No heartbeat from core client for 30 sec - exiting 08:28:36 (7148): No heartbeat from core client for 30 sec - exiting 08:28:37 (7148): No heartbeat from core client for 30 sec - exiting 08:28:38 (7148): No heartbeat from core client for 30 sec - exiting 08:28:40 (7148): No heartbeat from core client for 30 sec - exiting 08:28:41 (7148): No heartbeat from core client for 30 sec - exiting 08:28:42 (7148): No heartbeat from core client for 30 sec - exiting 08:28:43 (7148): No heartbeat from core client for 30 sec - exiting 08:28:44 (7148): No heartbeat from core client for 30 sec - exiting 08:28:45 (7148): No heartbeat from core client for 30 sec - exiting 08:28:46 (7148): No heartbeat from core client for 30 sec - exiting 08:28:47 (7148): No heartbeat from core client for 30 sec - exiting 08:28:48 (7148): No heartbeat from core client for 30 sec - exiting 08:28:49 (7148): No heartbeat from core client for 30 sec - exiting 08:28:50 (7148): No heartbeat from core client for 30 sec - exiting 08:28:51 (7148): No heartbeat from core client for 30 sec - exiting 08:28:53 (7148): No heartbeat from core client for 30 sec - exiting 08:28:54 (7148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:28:55 (7148): No heartbeat from core client for 30 sec - exiting 10:54:29 (1408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:39:22 (5064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:23 (5064): No heartbeat from core client for 30 sec - exiting 19:39:24 (5064): No heartbeat from core client for 30 sec - exiting 19:39:25 (5064): No heartbeat from core client for 30 sec - exiting 19:39:26 (5064): No heartbeat from core client for 30 sec - exiting 19:39:27 (5064): No heartbeat from core client for 30 sec - exiting 19:39:28 (5064): No heartbeat from core client for 30 sec - exiting 19:39:29 (5064): No heartbeat from core client for 30 sec - exiting 19:39:30 (5064): No heartbeat from core client for 30 sec - exiting 19:39:31 (5064): No heartbeat from core client for 30 sec - exiting 19:39:32 (5064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:10:18 (3212): No heartbeat from core client for 30 sec - exiting 07:10:19 (3212): No heartbeat from core client for 30 sec - exiting 07:10:20 (3212): No heartbeat from core client for 30 sec - exiting 07:10:21 (3212): No heartbeat from core client for 30 sec - exiting 07:10:22 (3212): No heartbeat from core client for 30 sec - exiting 07:10:23 (3212): No heartbeat from core client for 30 sec - exiting 07:10:25 (3212): No heartbeat from core client for 30 sec - exiting 07:10:26 (3212): No heartbeat from core client for 30 sec - exiting 07:10:27 (3212): No heartbeat from core client for 30 sec - exiting 07:10:28 (3212): No heartbeat from core client for 30 sec - exiting 07:10:29 (3212): No heartbeat from core client for 30 sec - exiting 07:10:30 (3212): No heartbeat from core client for 30 sec - exiting 07:10:31 (3212): No heartbeat from core client for 30 sec - exiting 07:10:32 (3212): No heartbeat from core client for 30 sec - exiting 07:10:33 (3212): No heartbeat from core client for 30 sec - exiting 07:10:34 (3212): No heartbeat from core client for 30 sec - exiting 07:10:35 (3212): No heartbeat from core client for 30 sec - exiting 07:10:36 (3212): No heartbeat from core client for 30 sec - exiting 07:10:37 (3212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:29:54 (4832): No heartbeat from core client for 30 sec - exiting 07:29:55 (4832): No heartbeat from core client for 30 sec - exiting 07:29:56 (4832): No heartbeat from core client for 30 sec - exiting 07:29:57 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77803AC3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zk0h_1920_40_008324587/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Oct 2013 13:37:49	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	1,036,800	1,498,938	1.4457
19 Oct 2013 02:08:07	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	1,010,880	1,462,814	1.4471
18 Oct 2013 13:02:25	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	984,960	1,425,503	1.4473
16 Oct 2013 23:58:53	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	959,040	1,389,738	1.4491
15 Oct 2013 18:29:29	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	933,120	1,350,841	1.4477
14 Oct 2013 22:09:33	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	907,200	1,311,240	1.4454
13 Oct 2013 15:40:49	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	881,280	1,276,178	1.4481
12 Oct 2013 18:28:10	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	855,360	1,238,292	1.4477
11 Oct 2013 14:29:46	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	829,440	1,201,571	1.4487
10 Oct 2013 04:59:05	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	803,520	1,167,333	1.4528
09 Oct 2013 00:08:40	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	777,600	1,129,356	1.4524
08 Oct 2013 12:03:38	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	751,680	1,092,243	1.4531
08 Oct 2013 01:24:32	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	725,760	1,057,486	1.4571
06 Oct 2013 16:43:10	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	699,840	1,020,214	1.4578
04 Oct 2013 13:14:31	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	673,920	984,162	1.4604
03 Oct 2013 17:37:32	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	648,000	947,087	1.4616
02 Oct 2013 21:45:49	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	622,080	909,867	1.4626
01 Oct 2013 13:18:13	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	596,160	872,530	1.4636
30 Sep 2013 13:58:22	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	570,240	829,900	1.4554
26 Sep 2013 11:13:17	1286653	16004657	hadcm3n_zk0h_1920_40_008324587_3	544,320	795,359	1.4612