Task 16841402

Name	hadcm3n_8d65_1980_40_008726488_2
Workunit	8872466
Created	15 Aug 2014, 18:36:14 UTC
Sent	15 Aug 2014, 18:36:29 UTC
Report deadline	15 Nov 2014, 2:03:40 UTC
Received	29 Nov 2015, 22:53:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1308837
Run time	42 days 20 hours 56 min 16 sec
CPU time	40 days 19 hours 9 min 9 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	1.43 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.4.27</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 01:12:00 (1996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:48:12 (3828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:50 (3192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5616, iMonCtr=1 Model crash detected, will try to restart... 08:19:41 (3544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:46 (1120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3088, iMonCtr=1 Model crash detected, will try to restart... 08:45:59 (2912): No heartbeat from core client for 30 sec - exiting 08:46:00 (2912): No heartbeat from core client for 30 sec - exiting 08:46:01 (2912): No heartbeat from core client for 30 sec - exiting 08:46:03 (2912): No heartbeat from core client for 30 sec - exiting 08:46:04 (2912): No heartbeat from core client for 30 sec - exiting 08:46:05 (2912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:05 (3832): No heartbeat from core client for 30 sec - exiting 09:05:06 (3832): No heartbeat from core client for 30 sec - exiting 09:05:07 (3832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3320, iMonCtr=1 Model crash detected, will try to restart... 00:51:00 (4028): No heartbeat from core client for 30 sec - exiting 00:51:01 (4028): No heartbeat from core client for 30 sec - exiting 00:51:03 (4028): No heartbeat from core client for 30 sec - exiting 00:51:04 (4028): No heartbeat from core client for 30 sec - exiting 00:51:05 (4028): No heartbeat from core client for 30 sec - exiting 00:51:06 (4028): No heartbeat from core client for 30 sec - exiting 00:51:07 (4028): No heartbeat from core client for 30 sec - exiting 00:51:08 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:51:09 (4028): No heartbeat from core client for 30 sec - exiting 08:43:02 (3628): No heartbeat from core client for 30 sec - exiting 08:43:03 (3628): No heartbeat from core client for 30 sec - exiting 08:43:04 (3628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2452, iMonCtr=1 Model crash detected, will try to restart... 03:34:38 (3788): No heartbeat from core client for 30 sec - exiting 03:34:39 (3788): No heartbeat from core client for 30 sec - exiting 03:34:40 (3788): No heartbeat from core client for 30 sec - exiting 03:34:41 (3788): No heartbeat from core client for 30 sec - exiting 03:34:42 (3788): No heartbeat from core client for 30 sec - exiting 03:34:44 (3788): No heartbeat from core client for 30 sec - exiting 03:34:45 (3788): No heartbeat from core client for 30 sec - exiting 03:34:46 (3788): No heartbeat from core client for 30 sec - exiting 03:34:47 (3788): No heartbeat from core client for 30 sec - exiting 03:34:48 (3788): No heartbeat from core client for 30 sec - exiting 03:34:49 (3788): No heartbeat from core client for 30 sec - exiting 03:34:50 (3788): No heartbeat from core client for 30 sec - exiting 03:34:51 (3788): No heartbeat from core client for 30 sec - exiting 03:34:52 (3788): No heartbeat from core client for 30 sec - exiting 03:34:53 (3788): No heartbeat from core client for 30 sec - exiting 03:34:55 (3788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:34:57 (3788): No heartbeat from core client for 30 sec - exiting 04:01:36 (1880): No heartbeat from core client for 30 sec - exiting 04:01:37 (1880): No heartbeat from core client for 30 sec - exiting 04:01:38 (1880): No heartbeat from core client for 30 sec - exiting 04:01:39 (1880): No heartbeat from core client for 30 sec - exiting 04:01:41 (1880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:01:42 (1880): No heartbeat from core client for 30 sec - exiting 09:23:49 (3424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:29:01 (3312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:09:03 (2448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3624, iMonCtr=1 Model crash detected, will try to restart... 22:49:28 (3400): No heartbeat from core client for 30 sec - exiting 22:49:29 (3400): No heartbeat from core client for 30 sec - exiting 22:49:30 (3400): No heartbeat from core client for 30 sec - exiting 22:49:31 (3400): No heartbeat from core client for 30 sec - exiting 22:49:32 (3400): No heartbeat from core client for 30 sec - exiting 22:49:33 (3400): No heartbeat from core client for 30 sec - exiting 22:49:34 (3400): No heartbeat from core client for 30 sec - exiting 22:49:35 (3400): No heartbeat from core client for 30 sec - exiting 22:49:37 (3400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:54:35 (4040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:13 (3900): No heartbeat from core client for 30 sec - exiting 22:03:14 (3900): No heartbeat from core client for 30 sec - exiting 22:03:15 (3900): No heartbeat from core client for 30 sec - exiting 22:03:16 (3900): No heartbeat from core client for 30 sec - exiting 22:03:17 (3900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:18 (3900): No heartbeat from core client for 30 sec - exiting 21:10:30 (1264): No heartbeat from core client for 30 sec - exiting 21:10:32 (1264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:55:29 (1464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:11 (3740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:41:36 (3300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:01:43 (3684): No heartbeat from core client for 30 sec - exiting 14:01:45 (3684): No heartbeat from core client for 30 sec - exiting 14:01:46 (3684): No heartbeat from core client for 30 sec - exiting 14:01:47 (3684): No heartbeat from core client for 30 sec - exiting 14:01:48 (3684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:01:49 (3684): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3924, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3336, iMonCtr=1 Model crash detected, will try to restart... 19:41:07 (3672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:58:59 (2708): No heartbeat from core client for 30 sec - exiting 19:59:00 (2708): No heartbeat from core client for 30 sec - exiting 19:59:01 (2708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:22 (2344): No heartbeat from core client for 30 sec - exiting 08:40:23 (2344): No heartbeat from core client for 30 sec - exiting 08:40:24 (2344): No heartbeat from core client for 30 sec - exiting 08:40:25 (2344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:25:13 (2560): No heartbeat from core client for 30 sec - exiting 21:25:14 (2560): No heartbeat from core client for 30 sec - exiting 21:25:15 (2560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:09 (3576): No heartbeat from core client for 30 sec - exiting 08:48:11 (3576): No heartbeat from core client for 30 sec - exiting 08:48:12 (3576): No heartbeat from core client for 30 sec - exiting 08:48:13 (3576): No heartbeat from core client for 30 sec - exiting 08:48:14 (3576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C08:49:17 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:36 (2956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:38 (3860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:32 (3656): No heartbeat from core client for 30 sec - exiting 23:55:33 (3656): No heartbeat from core client for 30 sec - exiting 23:55:34 (3656): No heartbeat from core client for 30 sec - exiting 23:55:36 (3656): No heartbeat from core client for 30 sec - exiting 23:55:37 (3656): No heartbeat from core client for 30 sec - exiting 23:55:38 (3656): No heartbeat from core client for 30 sec - exiting 23:55:39 (3656): No heartbeat from core client for 30 sec - exiting 23:55:40 (3656): No heartbeat from core client for 30 sec - exiting 23:55:41 (3656): No heartbeat from core client for 30 sec - exiting 23:55:42 (3656): No heartbeat from core client for 30 sec - exiting 23:55:43 (3656): No heartbeat from core client for 30 sec - exiting 23:55:44 (3656): No heartbeat from core client for 30 sec - exiting 23:55:45 (3656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:22 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:07:52 (3136): No heartbeat from core client for 30 sec - exiting 07:07:53 (3136): No heartbeat from core client for 30 sec - exiting 07:07:55 (3136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:04 (2068): No heartbeat from core client for 30 sec - exiting 19:39:06 (2068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:07 (2068): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3928, iMonCtr=1 Model crash detected, will try to restart... 20:10:27 (3104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:51 (3080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:30:23 (1928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:47:11 (3584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:12:56 (3956): No heartbeat from core client for 30 sec - exiting 19:12:57 (3956): No heartbeat from core client for 30 sec - exiting 19:12:58 (3956): No heartbeat from core client for 30 sec - exiting 19:12:59 (3956): No heartbeat from core client for 30 sec - exiting 19:13:01 (3956): No heartbeat from core client for 30 sec - exiting 19:13:02 (3956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:16 (3980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:34:56 (1724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:37:37 (1712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:17 (2252): No heartbeat from core client for 30 sec - exiting 14:38:18 (2252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:59:45 (2588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:53 (1932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:03:46 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:50:51 (2924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3044, iMonCtr=1 Model crash detected, will try to restart... 09:35:00 (2884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:52:23 (3404): No heartbeat from core client for 30 sec - exiting 08:52:24 (3404): No heartbeat from core client for 30 sec - exiting 08:52:26 (3404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:02 (3932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:49:53 (3064): No heartbeat from core client for 30 sec - exiting 13:49:54 (3064): No heartbeat from core client for 30 sec - exiting 13:49:57 (3064): No heartbeat from core client for 30 sec - exiting 13:49:58 (3064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:55:37 (3568): No heartbeat from core client for 30 sec - exiting 08:55:38 (3568): No heartbeat from core client for 30 sec - exiting 08:55:40 (3568): No heartbeat from core client for 30 sec - exiting 08:55:41 (3568): No heartbeat from core client for 30 sec - exiting 08:55:42 (3568): No heartbeat from core client for 30 sec - exiting 08:55:43 (3568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:55:08 (3736): No heartbeat from core client for 30 sec - exiting 08:55:09 (3736): No heartbeat from core client for 30 sec - exiting 08:55:10 (3736): No heartbeat from core client for 30 sec - exiting 08:55:11 (3736): No heartbeat from core client for 30 sec - exiting 08:55:12 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:00:37 (1500): No heartbeat from core client for 30 sec - exiting 09:00:38 (1500): No heartbeat from core client for 30 sec - exiting 09:00:39 (1500): No heartbeat from core client for 30 sec - exiting 09:00:40 (1500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:26 (2176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:02:11 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:59:09 (3576): No heartbeat from core client for 30 sec - exiting 22:59:11 (3576): No heartbeat from core client for 30 sec - exiting 22:59:12 (3576): No heartbeat from core client for 30 sec - exiting 22:59:13 (3576): No heartbeat from core client for 30 sec - exiting 22:59:14 (3576): No heartbeat from core client for 30 sec - exiting 22:59:15 (3576): No heartbeat from core client for 30 sec - exiting 22:59:17 (3576): No heartbeat from core client for 30 sec - exiting 22:59:18 (3576): No heartbeat from core client for 30 sec - exiting 22:59:20 (3576): No heartbeat from core client for 30 sec - exiting 22:59:21 (3576): No heartbeat from core client for 30 sec - exiting 22:59:22 (3576): No heartbeat from core client for 30 sec - exiting 22:59:23 (3576): No heartbeat from core client for 30 sec - exiting 22:59:24 (3576): No heartbeat from core client for 30 sec - exiting 22:59:25 (3576): No heartbeat from core client for 30 sec - exiting 22:59:26 (3576): No heartbeat from core client for 30 sec - exiting 22:59:27 (3576): No heartbeat from core client for 30 sec - exiting 22:59:28 (3576): No heartbeat from core client for 30 sec - exiting 22:59:30 (3576): No heartbeat from core client for 30 sec - exiting 22:59:31 (3576): No heartbeat from core client for 30 sec - exiting 22:59:32 (3576): No heartbeat from core client for 30 sec - exiting 22:59:33 (3576): No heartbeat from core client for 30 sec - exiting 22:59:34 (3576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:59:35 (3576): No heartbeat from core client for 30 sec - exiting 08:34:50 (3792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:53 (4028): No heartbeat from core client for 30 sec - exiting 08:43:54 (4028): No heartbeat from core client for 30 sec - exiting 08:43:55 (4028): No heartbeat from core client for 30 sec - exiting 08:43:56 (4028): No heartbeat from core client for 30 sec - exiting 08:43:57 (4028): No heartbeat from core client for 30 sec - exiting 08:43:58 (4028): No heartbeat from core client for 30 sec - exiting 08:43:59 (4028): No heartbeat from core client for 30 sec - exiting 08:44:01 (4028): No heartbeat from core client for 30 sec - exiting 08:44:02 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:35 (784): No heartbeat from core client for 30 sec - exiting 08:56:36 (784): No heartbeat from core client for 30 sec - exiting 08:56:37 (784): No heartbeat from core client for 30 sec - exiting 08:56:38 (784): No heartbeat from core client for 30 sec - exiting 08:56:39 (784): No heartbeat from core client for 30 sec - exiting 08:56:40 (784): No heartbeat from core client for 30 sec - exiting 08:56:41 (784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:50:33 (4080): No heartbeat from core client for 30 sec - exiting 18:50:34 (4080): No heartbeat from core client for 30 sec - exiting 18:50:35 (4080): No heartbeat from core client for 30 sec - exiting 18:50:36 (4080): No heartbeat from core client for 30 sec - exiting 18:50:37 (4080): No heartbeat from core client for 30 sec - exiting 18:50:38 (4080): No heartbeat from core client for 30 sec - exiting 18:50:39 (4080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:57:03 (784): No heartbeat from core client for 30 sec - exiting 08:57:05 (784): No heartbeat from core client for 30 sec - exiting 08:57:06 (784): No heartbeat from core client for 30 sec - exiting 08:57:07 (784): No heartbeat from core client for 30 sec - exiting 08:57:08 (784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:57:09 (784): No heartbeat from core client for 30 sec - exiting 09:01:17 (3424): No heartbeat from core client for 30 sec - exiting 09:01:18 (3424): No heartbeat from core client for 30 sec - exiting 09:01:19 (3424): No heartbeat from core client for 30 sec - exiting 09:01:20 (3424): No heartbeat from core client for 30 sec - exiting 09:01:21 (3424): No heartbeat from core client for 30 sec - exiting 09:01:22 (3424): No heartbeat from core client for 30 sec - exiting 09:01:23 (3424): No heartbeat from core client for 30 sec - exiting 09:01:24 (3424): No heartbeat from core client for 30 sec - exiting 09:01:26 (3424): No heartbeat from core client for 30 sec - exiting 09:01:27 (3424): No heartbeat from core client for 30 sec - exiting 09:01:28 (3424): No heartbeat from core client for 30 sec - exiting 09:01:29 (3424): No heartbeat from core client for 30 sec - exiting 09:01:30 (3424): No heartbeat from core client for 30 sec - exiting 09:01:31 (3424): No heartbeat from core client for 30 sec - exiting 09:01:32 (3424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:36:43 (2624): No heartbeat from core client for 30 sec - exiting 00:36:44 (2624): No heartbeat from core client for 30 sec - exiting 00:36:45 (2624): No heartbeat from core client for 30 sec - exiting 00:36:46 (2624): No heartbeat from core client for 30 sec - exiting 00:36:47 (2624): No heartbeat from core client for 30 sec - exiting 00:36:48 (2624): No heartbeat from core client for 30 sec - exiting 00:36:49 (2624): No heartbeat from core client for 30 sec - exiting 00:36:51 (2624): No heartbeat from core client for 30 sec - exiting 00:36:52 (2624): No heartbeat from core client for 30 sec - exiting 00:36:53 (2624): No heartbeat from core client for 30 sec - exiting 00:36:54 (2624): No heartbeat from core client for 30 sec - exiting 00:36:55 (2624): No heartbeat from core client for 30 sec - exiting 00:36:56 (2624): No heartbeat from core client for 30 sec - exiting 00:36:57 (2624): No heartbeat from core client for 30 sec - exiting 00:36:58 (2624): No heartbeat from core client for 30 sec - exiting 00:36:59 (2624): No heartbeat from core client for 30 sec - exiting 00:37:01 (2624): No heartbeat from core client for 30 sec - exiting 00:37:02 (2624): No heartbeat from core client for 30 sec - exiting 00:37:03 (2624): No heartbeat from core client for 30 sec - exiting 00:37:04 (2624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:33:57 (5724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:03:46 (3696): No heartbeat from core client for 30 sec - exiting 02:03:47 (3696): No heartbeat from core client for 30 sec - exiting 02:03:48 (3696): No heartbeat from core client for 30 sec - exiting 02:03:49 (3696): No heartbeat from core client for 30 sec - exiting 02:03:50 (3696): No heartbeat from core client for 30 sec - exiting 02:03:51 (3696): No heartbeat from core client for 30 sec - exiting 02:03:52 (3696): No heartbeat from core client for 30 sec - exiting 02:03:54 (3696): No heartbeat from core client for 30 sec - exiting 02:03:55 (3696): No heartbeat from core client for 30 sec - exiting 02:03:56 (3696): No heartbeat from core client for 30 sec - exiting 02:03:57 (3696): No heartbeat from core client for 30 sec - exiting 02:03:58 (3696): No heartbeat from core client for 30 sec - exiting 02:03:59 (3696): No heartbeat from core client for 30 sec - exiting 02:04:00 (3696): No heartbeat from core client for 30 sec - exiting 02:04:01 (3696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:49:56 (3712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3376, iMonCtr=1 Model crash detected, will try to restart... 17:09:17 (3240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:11:18 (884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:22:48 (3772): No heartbeat from core client for 30 sec - exiting 15:22:49 (3772): No heartbeat from core client for 30 sec - exiting 15:22:50 (3772): No heartbeat from core client for 30 sec - exiting 15:22:51 (3772): No heartbeat from core client for 30 sec - exiting 15:22:52 (3772): No heartbeat from core client for 30 sec - exiting 15:22:53 (3772): No heartbeat from core client for 30 sec - exiting 15:22:54 (3772): No heartbeat from core client for 30 sec - exiting 15:22:56 (3772): No heartbeat from core client for 30 sec - exiting 15:22:57 (3772): No heartbeat from core client for 30 sec - exiting 15:22:58 (3772): No heartbeat from core client for 30 sec - exiting 15:22:59 (3772): No heartbeat from core client for 30 sec - exiting 15:23:00 (3772): No heartbeat from core client for 30 sec - exiting 15:23:01 (3772): No heartbeat from core client for 30 sec - exiting 15:23:02 (3772): No heartbeat from core client for 30 sec - exiting 15:23:03 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:23:04 (3772): No heartbeat from core client for 30 sec - exiting 15:25:22 (4044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:25:09 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:33:05 (3260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:16:19 (636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:46 (3552): No heartbeat from core client for 30 sec - exiting 22:04:47 (3552): No heartbeat from core client for 30 sec - exiting 22:04:48 (3552): No heartbeat from core client for 30 sec - exiting 22:04:49 (3552): No heartbeat from core client for 30 sec - exiting 22:04:51 (3552): No heartbeat from core client for 30 sec - exiting 22:04:52 (3552): No heartbeat from core client for 30 sec - exiting 22:04:53 (3552): No heartbeat from core client for 30 sec - exiting 22:04:54 (3552): No heartbeat from core client for 30 sec - exiting 22:04:55 (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:56 (3552): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2872, iMonCtr=1 Model crash detected, will try to restart... 22:17:12 (3492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:07:00 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:02:52 (4052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:51 (196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:59:21 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3992, iMonCtr=1 Model crash detected, will try to restart... 19:18:43 (3708): No heartbeat from core client for 30 sec - exiting 19:18:44 (3708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2848, iMonCtr=1 Model crash detected, will try to restart... 19:26:40 (2848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:02 (2424): No heartbeat from core client for 30 sec - exiting 19:37:03 (2424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:52:54 (4016): No heartbeat from core client for 30 sec - exiting 19:52:55 (4016): No heartbeat from core client for 30 sec - exiting 19:52:56 (4016): No heartbeat from core client for 30 sec - exiting 19:52:57 (4016): No heartbeat from core client for 30 sec - exiting 19:52:58 (4016): No heartbeat from core client for 30 sec - exiting 19:52:59 (4016): No heartbeat from core client for 30 sec - exiting 19:53:00 (4016): No heartbeat from core client for 30 sec - exiting 19:53:01 (4016): No heartbeat from core client for 30 sec - exiting 19:53:02 (4016): No heartbeat from core client for 30 sec - exiting 19:53:03 (4016): No heartbeat from core client for 30 sec - exiting 19:53:04 (4016): No heartbeat from core client for 30 sec - exiting 19:53:06 (4016): No heartbeat from core client for 30 sec - exiting 19:53:07 (4016): No heartbeat from core client for 30 sec - exiting 19:53:08 (4016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:02:51 (4272): No heartbeat from core client for 30 sec - exiting 21:02:52 (4272): No heartbeat from core client for 30 sec - exiting 21:02:53 (4272): No heartbeat from core client for 30 sec - exiting 21:02:54 (4272): No heartbeat from core client for 30 sec - exiting 21:02:55 (4272): No heartbeat from core client for 30 sec - exiting 21:02:56 (4272): No heartbeat from core client for 30 sec - exiting 21:02:57 (4272): No heartbeat from core client for 30 sec - exiting 21:02:58 (4272): No heartbeat from core client for 30 sec - exiting 21:02:59 (4272): No heartbeat from core client for 30 sec - exiting 21:03:00 (4272): No heartbeat from core client for 30 sec - exiting 21:03:01 (4272): No heartbeat from core client for 30 sec - exiting 21:03:02 (4272): No heartbeat from core client for 30 sec - exiting 21:03:03 (4272): No heartbeat from core client for 30 sec - exiting 21:03:04 (4272): No heartbeat from core client for 30 sec - exiting 21:03:05 (4272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77AE6308 read attempt to address 0x00000004 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77AE620B read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Nov 2015 19:40:22	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	777,600	3,515,279	4.5207
19 Apr 2015 17:22:43	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	751,680	3,400,426	4.5238
18 Apr 2015 09:57:15	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	725,760	3,289,237	4.5321
17 Apr 2015 03:05:25	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	699,840	3,179,907	4.5438
01 Feb 2015 01:29:46	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	673,920	3,067,491	4.5517
30 Jan 2015 16:56:37	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	648,000	2,954,491	4.5594
10 Jan 2015 21:46:58	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	622,080	2,843,157	4.5704
09 Jan 2015 12:54:29	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	596,160	2,727,625	4.5753
08 Jan 2015 16:22:41	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	570,240	2,613,653	4.5834
08 Jan 2015 16:19:05	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	544,320	2,503,336	4.5990
05 Jan 2015 12:43:25	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	518,400	2,390,512	4.6113
04 Jan 2015 02:57:18	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	492,480	2,270,413	4.6102
02 Jan 2015 19:02:32	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	466,560	2,156,725	4.6226
01 Jan 2015 10:57:58	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	440,640	2,042,543	4.6354
31 Dec 2014 01:41:48	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	414,720	1,924,314	4.6400
29 Dec 2014 15:40:11	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	388,800	1,803,607	4.6389
28 Dec 2014 06:54:49	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	362,880	1,686,908	4.6487
26 Dec 2014 21:50:20	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	336,960	1,569,532	4.6579
06 Dec 2014 14:49:59	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	311,040	1,452,178	4.6688
11 Oct 2014 23:23:03	1308837	16841402	hadcm3n_8d65_1980_40_008726488_2	285,120	1,337,822	4.6921