Task 15692460

Name	hadcm3n_4k6o_2020_40_008341130_0
Workunit	8491991
Created	29 Mar 2013, 4:13:52 UTC
Sent	29 Mar 2013, 6:10:35 UTC
Report deadline	28 Jun 2013, 13:37:46 UTC
Received	1 May 2013, 11:39:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1217831
Run time	18 days 15 hours 59 min 41 sec
CPU time	11 days 15 hours 22 min 7 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=138524, iMonCtr=1 Model crash detected, will try to restart... 13:05:57 (4728): No heartbeat from core client for 30 sec - exiting 13:05:58 (4728): No heartbeat from core client for 30 sec - exiting 13:05:59 (4728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:09:48 (8404): No heartbeat from core client for 30 sec - exiting 14:09:49 (8404): No heartbeat from core client for 30 sec - exiting 14:09:50 (8404): No heartbeat from core client for 30 sec - exiting 14:09:51 (8404): No heartbeat from core client for 30 sec - exiting 14:09:52 (8404): No heartbeat from core client for 30 sec - exiting 14:09:53 (8404): No heartbeat from core client for 30 sec - exiting 14:09:54 (8404): No heartbeat from core client for 30 sec - exiting 14:09:55 (8404): No heartbeat from core client for 30 sec - exiting 14:09:56 (8404): No heartbeat from core client for 30 sec - exiting 14:09:57 (8404): No heartbeat from core client for 30 sec - exiting 14:09:58 (8404): No heartbeat from core client for 30 sec - exiting 14:09:59 (8404): No heartbeat from core client for 30 sec - exiting 14:10:00 (8404): No heartbeat from core client for 30 sec - exiting 14:10:01 (8404): No heartbeat from core client for 30 sec - exiting 14:10:02 (8404): No heartbeat from core client for 30 sec - exiting 14:10:03 (8404): No heartbeat from core client for 30 sec - exiting 14:10:04 (8404): No heartbeat from core client for 30 sec - exiting 14:10:05 (8404): No heartbeat from core client for 30 sec - exiting 14:10:06 (8404): No heartbeat from core client for 30 sec - exiting 14:10:07 (8404): No heartbeat from core client for 30 sec - exiting 14:10:08 (8404): No heartbeat from core client for 30 sec - exiting 14:10:09 (8404): No heartbeat from core client for 30 sec - exiting 14:10:10 (8404): No heartbeat from core client for 30 sec - exiting 14:10:11 (8404): No heartbeat from core client for 30 sec - exiting 14:10:12 (8404): No heartbeat from core client for 30 sec - exiting 14:10:13 (8404): No heartbeat from core client for 30 sec - exiting 14:10:14 (8404): No heartbeat from core client for 30 sec - exiting 14:10:15 (8404): No heartbeat from core client for 30 sec - exiting 14:10:16 (8404): No heartbeat from core client for 30 sec - exiting 14:10:17 (8404): No heartbeat from core client for 30 sec - exiting 14:10:18 (8404): No heartbeat from core client for 30 sec - exiting 14:10:19 (8404): No heartbeat from core client for 30 sec - exiting 14:10:20 (8404): No heartbeat from core client for 30 sec - exiting 14:10:21 (8404): No heartbeat from core client for 30 sec - exiting 14:10:22 (8404): No heartbeat from core client for 30 sec - exiting 14:10:23 (8404): No heartbeat from core client for 30 sec - exiting 14:10:24 (8404): No heartbeat from core client for 30 sec - exiting 14:10:25 (8404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:10:29 (25812): No heartbeat from core client for 30 sec - exiting 13:10:30 (25812): No heartbeat from core client for 30 sec - exiting 13:10:31 (25812): No heartbeat from core client for 30 sec - exiting 13:10:32 (25812): No heartbeat from core client for 30 sec - exiting 13:10:33 (25812): No heartbeat from core client for 30 sec - exiting 13:10:34 (25812): No heartbeat from core client for 30 sec - exiting 13:10:35 (25812): No heartbeat from core client for 30 sec - exiting 13:10:36 (25812): No heartbeat from core client for 30 sec - exiting 13:10:37 (25812): No heartbeat from core client for 30 sec - exiting 13:10:38 (25812): No heartbeat from core client for 30 sec - exiting 13:10:39 (25812): No heartbeat from core client for 30 sec - exiting 13:10:40 (25812): No heartbeat from core client for 30 sec - exiting 13:10:41 (25812): No heartbeat from core client for 30 sec - exiting 13:10:42 (25812): No heartbeat from core client for 30 sec - exiting 13:10:43 (25812): No heartbeat from core client for 30 sec - exiting 13:10:44 (25812): No heartbeat from core client for 30 sec - exiting 13:10:45 (25812): No heartbeat from core client for 30 sec - exiting 13:10:46 (25812): No heartbeat from core client for 30 sec - exiting 13:10:47 (25812): No heartbeat from core client for 30 sec - exiting 13:10:48 (25812): No heartbeat from core client for 30 sec - exiting 13:10:49 (25812): No heartbeat from core client for 30 sec - exiting 13:10:50 (25812): No heartbeat from core client for 30 sec - exiting 13:10:51 (25812): No heartbeat from core client for 30 sec - exiting 13:10:52 (25812): No heartbeat from core client for 30 sec - exiting 13:10:53 (25812): No heartbeat from core client for 30 sec - exiting 13:10:54 (25812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:28:06 (11060): No heartbeat from core client for 30 sec - exiting 19:28:07 (11060): No heartbeat from core client for 30 sec - exiting 19:28:08 (11060): No heartbeat from core client for 30 sec - exiting 19:28:09 (11060): No heartbeat from core client for 30 sec - exiting 19:28:10 (11060): No heartbeat from core client for 30 sec - exiting 19:28:11 (11060): No heartbeat from core client for 30 sec - exiting 19:28:12 (11060): No heartbeat from core client for 30 sec - exiting 19:28:13 (11060): No heartbeat from core client for 30 sec - exiting 19:28:14 (11060): No heartbeat from core client for 30 sec - exiting 19:28:15 (11060): No heartbeat from core client for 30 sec - exiting 19:28:16 (11060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:58:38 (154272): No heartbeat from core client for 30 sec - exiting 12:58:39 (154272): No heartbeat from core client for 30 sec - exiting 12:58:40 (154272): No heartbeat from core client for 30 sec - exiting 12:58:41 (154272): No heartbeat from core client for 30 sec - exiting 12:58:42 (154272): No heartbeat from core client for 30 sec - exiting 12:58:43 (154272): No heartbeat from core client for 30 sec - exiting 12:58:44 (154272): No heartbeat from core client for 30 sec - exiting 12:58:45 (154272): No heartbeat from core client for 30 sec - exiting 12:58:46 (154272): No heartbeat from core client for 30 sec - exiting 12:58:47 (154272): No heartbeat from core client for 30 sec - exiting 12:58:48 (154272): No heartbeat from core client for 30 sec - exiting 12:58:49 (154272): No heartbeat from core client for 30 sec - exiting 12:58:50 (154272): No heartbeat from core client for 30 sec - exiting 12:58:51 (154272): No heartbeat from core client for 30 sec - exiting 12:58:52 (154272): No heartbeat from core client for 30 sec - exiting 12:58:53 (154272): No heartbeat from core client for 30 sec - exiting 12:58:54 (154272): No heartbeat from core client for 30 sec - exiting 12:58:55 (154272): No heartbeat from core client for 30 sec - exiting 12:58:56 (154272): No heartbeat from core client for 30 sec - exiting 12:58:57 (154272): No heartbeat from core client for 30 sec - exiting 12:58:58 (154272): No heartbeat from core client for 30 sec - exiting 12:58:59 (154272): No heartbeat from core client for 30 sec - exiting 12:59:00 (154272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:27:44 (4420): No heartbeat from core client for 30 sec - exiting 20:27:45 (4420): No heartbeat from core client for 30 sec - exiting 20:27:46 (4420): No heartbeat from core client for 30 sec - exiting 20:27:47 (4420): No heartbeat from core client for 30 sec - exiting 20:27:48 (4420): No heartbeat from core client for 30 sec - exiting 20:27:49 (4420): No heartbeat from core client for 30 sec - exiting 20:27:50 (4420): No heartbeat from core client for 30 sec - exiting 20:27:51 (4420): No heartbeat from core client for 30 sec - exiting 20:27:52 (4420): No heartbeat from core client for 30 sec - exiting 20:27:53 (4420): No heartbeat from core client for 30 sec - exiting 20:27:54 (4420): No heartbeat from core client for 30 sec - exiting 20:27:55 (4420): No heartbeat from core client for 30 sec - exiting 20:27:56 (4420): No heartbeat from core client for 30 sec - exiting 20:27:57 (4420): No heartbeat from core client for 30 sec - exiting 20:27:58 (4420): No heartbeat from core client for 30 sec - exiting 20:27:59 (4420): No heartbeat from core client for 30 sec - exiting 20:28:00 (4420): No heartbeat from core client for 30 sec - exiting 20:28:01 (4420): No heartbeat from core client for 30 sec - exiting 20:28:02 (4420): No heartbeat from core client for 30 sec - exiting 20:28:03 (4420): No heartbeat from core client for 30 sec - exiting 20:28:04 (4420): No heartbeat from core client for 30 sec - exiting 20:28:05 (4420): No heartbeat from core client for 30 sec - exiting 20:28:06 (4420): No heartbeat from core client for 30 sec - exiting 20:28:07 (4420): No heartbeat from core client for 30 sec - exiting 20:28:08 (4420): No heartbeat from core client for 30 sec - exiting 20:28:09 (4420): No heartbeat from core client for 30 sec - exiting 20:28:10 (4420): No heartbeat from core client for 30 sec - exiting 20:28:11 (4420): No heartbeat from core client for 30 sec - exiting 20:28:12 (4420): No heartbeat from core client for 30 sec - exiting 20:28:13 (4420): No heartbeat from core client for 30 sec - exiting 20:28:14 (4420): No heartbeat from core client for 30 sec - exiting 20:28:15 (4420): No heartbeat from core client for 30 sec - exiting 20:28:16 (4420): No heartbeat from core client for 30 sec - exiting 20:28:17 (4420): No heartbeat from core client for 30 sec - exiting 20:28:18 (4420): No heartbeat from core client for 30 sec - exiting 20:28:19 (4420): No heartbeat from core client for 30 sec - exiting 20:28:20 (4420): No heartbeat from core client for 30 sec - exiting 20:28:21 (4420): No heartbeat from core client for 30 sec - exiting 20:28:22 (4420): No heartbeat from core client for 30 sec - exiting 20:28:23 (4420): No heartbeat from core client for 30 sec - exiting 20:28:24 (4420): No heartbeat from core client for 30 sec - exiting 20:28:25 (4420): No heartbeat from core client for 30 sec - exiting 20:28:26 (4420): No heartbeat from core client for 30 sec - exiting 20:28:27 (4420): No heartbeat from core client for 30 sec - exiting 20:28:28 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:29 (4420): No heartbeat from core client for 30 sec - exiting 20:28:30 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=134676, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26536, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:44:08 (26684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=40160, iMonCtr=1 Model crash detected, will try to restart... 10:16:19 (15872): No heartbeat from core client for 30 sec - exiting 10:16:20 (15872): No heartbeat from core client for 30 sec - exiting 10:16:21 (15872): No heartbeat from core client for 30 sec - exiting 10:16:22 (15872): No heartbeat from core client for 30 sec - exiting 10:16:23 (15872): No heartbeat from core client for 30 sec - exiting 10:16:24 (15872): No heartbeat from core client for 30 sec - exiting 10:16:25 (15872): No heartbeat from core client for 30 sec - exiting 10:16:26 (15872): No heartbeat from core client for 30 sec - exiting 10:16:27 (15872): No heartbeat from core client for 30 sec - exiting 10:16:28 (15872): No heartbeat from core client for 30 sec - exiting 10:16:29 (15872): No heartbeat from core client for 30 sec - exiting 10:16:30 (15872): No heartbeat from core client for 30 sec - exiting 10:16:31 (15872): No heartbeat from core client for 30 sec - exiting 10:16:32 (15872): No heartbeat from core client for 30 sec - exiting 10:16:33 (15872): No heartbeat from core client for 30 sec - exiting 10:16:34 (15872): No heartbeat from core client for 30 sec - exiting 10:16:35 (15872): No heartbeat from core client for 30 sec - exiting 10:16:36 (15872): No heartbeat from core client for 30 sec - exiting 10:16:37 (15872): No heartbeat from core client for 30 sec - exiting 10:16:38 (15872): No heartbeat from core client for 30 sec - exiting 10:16:39 (15872): No heartbeat from core client for 30 sec - exiting 10:16:40 (15872): No heartbeat from core client for 30 sec - exiting 10:16:41 (15872): No heartbeat from core client for 30 sec - exiting 10:16:42 (15872): No heartbeat from core client for 30 sec - exiting 10:16:43 (15872): No heartbeat from core client for 30 sec - exiting 10:16:44 (15872): No heartbeat from core client for 30 sec - exiting 10:16:45 (15872): No heartbeat from core client for 30 sec - exiting 10:16:46 (15872): No heartbeat from core client for 30 sec - exiting 10:16:47 (15872): No heartbeat from core client for 30 sec - exiting 10:16:48 (15872): No heartbeat from core client for 30 sec - exiting 10:16:49 (15872): No heartbeat from core client for 30 sec - exiting 10:16:50 (15872): No heartbeat from core client for 30 sec - exiting 10:16:51 (15872): No heartbeat from core client for 30 sec - exiting 10:16:52 (15872): No heartbeat from core client for 30 sec - exiting 10:16:53 (15872): No heartbeat from core client for 30 sec - exiting 10:16:54 (15872): No heartbeat from core client for 30 sec - exiting 10:16:55 (15872): No heartbeat from core client for 30 sec - exiting 10:16:56 (15872): No heartbeat from core client for 30 sec - exiting 10:16:57 (15872): No heartbeat from core client for 30 sec - exiting 10:16:58 (15872): No heartbeat from core client for 30 sec - exiting 10:16:59 (15872): No heartbeat from core client for 30 sec - exiting 10:17:00 (15872): No heartbeat from core client for 30 sec - exiting 10:17:01 (15872): No heartbeat from core client for 30 sec - exiting 10:17:02 (15872): No heartbeat from core client for 30 sec - exiting 10:17:03 (15872): No heartbeat from core client for 30 sec - exiting 10:17:04 (15872): No heartbeat from core client for 30 sec - exiting 10:17:05 (15872): No heartbeat from core client for 30 sec - exiting 10:17:06 (15872): No heartbeat from core client for 30 sec - exiting 10:17:07 (15872): No heartbeat from core client for 30 sec - exiting 10:17:08 (15872): No heartbeat from core client for 30 sec - exiting 10:17:09 (15872): No heartbeat from core client for 30 sec - exiting 10:17:10 (15872): No heartbeat from core client for 30 sec - exiting 10:17:11 (15872): No heartbeat from core client for 30 sec - exiting 10:17:12 (15872): No heartbeat from core client for 30 sec - exiting 10:17:13 (15872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:17:14 (15872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:46:29 (114656): No heartbeat from core client for 30 sec - exiting 15:46:30 (114656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:55:12 (7236): No heartbeat from core client for 30 sec - exiting 13:55:13 (7236): No heartbeat from core client for 30 sec - exiting 13:55:14 (7236): No heartbeat from core client for 30 sec - exiting 13:55:15 (7236): No heartbeat from core client for 30 sec - exiting 13:55:16 (7236): No heartbeat from core client for 30 sec - exiting 13:55:17 (7236): No heartbeat from core client for 30 sec - exiting 13:55:18 (7236): No heartbeat from core client for 30 sec - exiting 13:55:19 (7236): No heartbeat from core client for 30 sec - exiting 13:55:20 (7236): No heartbeat from core client for 30 sec - exiting 13:55:21 (7236): No heartbeat from core client for 30 sec - exiting 13:55:22 (7236): No heartbeat from core client for 30 sec - exiting 13:55:23 (7236): No heartbeat from core client for 30 sec - exiting 13:55:24 (7236): No heartbeat from core client for 30 sec - exiting 13:55:25 (7236): No heartbeat from core client for 30 sec - exiting 13:55:26 (7236): No heartbeat from core client for 30 sec - exiting 13:55:27 (7236): No heartbeat from core client for 30 sec - exiting 13:55:28 (7236): No heartbeat from core client for 30 sec - exiting 13:55:29 (7236): No heartbeat from core client for 30 sec - exiting 13:55:30 (7236): No heartbeat from core client for 30 sec - exiting 13:55:31 (7236): No heartbeat from core client for 30 sec - exiting 13:55:32 (7236): No heartbeat from core client for 30 sec - exiting 13:55:33 (7236): No heartbeat from core client for 30 sec - exiting 13:55:34 (7236): No heartbeat from core client for 30 sec - exiting 13:55:35 (7236): No heartbeat from core client for 30 sec - exiting 13:55:36 (7236): No heartbeat from core client for 30 sec - exiting 13:55:37 (7236): No heartbeat from core client for 30 sec - exiting 13:55:38 (7236): No heartbeat from core client for 30 sec - exiting 13:55:39 (7236): No heartbeat from core client for 30 sec - exiting 13:55:40 (7236): No heartbeat from core client for 30 sec - exiting 13:55:41 (7236): No heartbeat from core client for 30 sec - exiting 13:55:42 (7236): No heartbeat from core client for 30 sec - exiting 13:55:43 (7236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:42:49 (2168): No heartbeat from core client for 30 sec - exiting 15:42:50 (2168): No heartbeat from core client for 30 sec - exiting 15:42:51 (2168): No heartbeat from core client for 30 sec - exiting 15:42:52 (2168): No heartbeat from core client for 30 sec - exiting 15:42:53 (2168): No heartbeat from core client for 30 sec - exiting 15:42:54 (2168): No heartbeat from core client for 30 sec - exiting 15:42:55 (2168): No heartbeat from core client for 30 sec - exiting 15:42:56 (2168): No heartbeat from core client for 30 sec - exiting 15:42:57 (2168): No heartbeat from core client for 30 sec - exiting 15:42:58 (2168): No heartbeat from core client for 30 sec - exiting 15:42:59 (2168): No heartbeat from core client for 30 sec - exiting 15:43:00 (2168): No heartbeat from core client for 30 sec - exiting 15:43:01 (2168): No heartbeat from core client for 30 sec - exiting 15:43:02 (2168): No heartbeat from core client for 30 sec - exiting 15:43:03 (2168): No heartbeat from core client for 30 sec - exiting 15:43:04 (2168): No heartbeat from core client for 30 sec - exiting 15:43:05 (2168): No heartbeat from core client for 30 sec - exiting 15:43:06 (2168): No heartbeat from core client for 30 sec - exiting 15:43:07 (2168): No heartbeat from core client for 30 sec - exiting 15:43:08 (2168): No heartbeat from core client for 30 sec - exiting 15:43:09 (2168): No heartbeat from core client for 30 sec - exiting 15:43:10 (2168): No heartbeat from core client for 30 sec - exiting 15:43:11 (2168): No heartbeat from core client for 30 sec - exiting 15:43:12 (2168): No heartbeat from core client for 30 sec - exiting 15:43:13 (2168): No heartbeat from core client for 30 sec - exiting 15:43:14 (2168): No heartbeat from core client for 30 sec - exiting 15:43:15 (2168): No heartbeat from core client for 30 sec - exiting 15:43:16 (2168): No heartbeat from core client for 30 sec - exiting 15:43:17 (2168): No heartbeat from core client for 30 sec - exiting 15:43:18 (2168): No heartbeat from core client for 30 sec - exiting 15:43:19 (2168): No heartbeat from core client for 30 sec - exiting 15:43:20 (2168): No heartbeat from core client for 30 sec - exiting 15:43:21 (2168): No heartbeat from core client for 30 sec - exiting 15:43:22 (2168): No heartbeat from core client for 30 sec - exiting 15:43:23 (2168): No heartbeat from core client for 30 sec - exiting 15:43:24 (2168): No heartbeat from core client for 30 sec - exiting 15:43:25 (2168): No heartbeat from core client for 30 sec - exiting 15:43:26 (2168): No heartbeat from core client for 30 sec - exiting 15:43:27 (2168): No heartbeat from core client for 30 sec - exiting 15:43:28 (2168): No heartbeat from core client for 30 sec - exiting 15:43:29 (2168): No heartbeat from core client for 30 sec - exiting 15:43:30 (2168): No heartbeat from core client for 30 sec - exiting 15:43:31 (2168): No heartbeat from core client for 30 sec - exiting 15:43:32 (2168): No heartbeat from core client for 30 sec - exiting 15:43:33 (2168): No heartbeat from core client for 30 sec - exiting 15:43:34 (2168): No heartbeat from core client for 30 sec - exiting 15:43:35 (2168): No heartbeat from core client for 30 sec - exiting 15:43:36 (2168): No heartbeat from core client for 30 sec - exiting 15:43:37 (2168): No heartbeat from core client for 30 sec - exiting 15:43:38 (2168): No heartbeat from core client for 30 sec - exiting 15:43:39 (2168): No heartbeat from core client for 30 sec - exiting 15:43:40 (2168): No heartbeat from core client for 30 sec - exiting 15:43:41 (2168): No heartbeat from core client for 30 sec - exiting 15:43:42 (2168): No heartbeat from core client for 30 sec - exiting 15:43:43 (2168): No heartbeat from core client for 30 sec - exiting 15:43:44 (2168): No heartbeat from core client for 30 sec - exiting 15:43:45 (2168): No heartbeat from core client for 30 sec - exiting 15:43:46 (2168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:30:33 (6648): No heartbeat from core client for 30 sec - exiting 04:30:34 (6648): No heartbeat from core client for 30 sec - exiting 04:30:35 (6648): No heartbeat from core client for 30 sec - exiting 04:30:36 (6648): No heartbeat from core client for 30 sec - exiting 04:30:37 (6648): No heartbeat from core client for 30 sec - exiting 04:30:38 (6648): No heartbeat from core client for 30 sec - exiting 04:30:39 (6648): No heartbeat from core client for 30 sec - exiting 04:30:40 (6648): No heartbeat from core client for 30 sec - exiting 04:30:41 (6648): No heartbeat from core client for 30 sec - exiting 04:30:42 (6648): No heartbeat from core client for 30 sec - exiting 04:30:43 (6648): No heartbeat from core client for 30 sec - exiting 04:30:44 (6648): No heartbeat from core client for 30 sec - exiting 04:30:45 (6648): No heartbeat from core client for 30 sec - exiting 04:30:46 (6648): No heartbeat from core client for 30 sec - exiting 04:30:47 (6648): No heartbeat from core client for 30 sec - exiting 04:30:48 (6648): No heartbeat from core client for 30 sec - exiting 04:30:49 (6648): No heartbeat from core client for 30 sec - exiting 04:30:50 (6648): No heartbeat from core client for 30 sec - exiting 04:30:51 (6648): No heartbeat from core client for 30 sec - exiting 04:30:52 (6648): No heartbeat from core client for 30 sec - exiting 04:30:53 (6648): No heartbeat from core client for 30 sec - exiting 04:30:54 (6648): No heartbeat from core client for 30 sec - exiting 04:30:55 (6648): No heartbeat from core client for 30 sec - exiting 04:30:56 (6648): No heartbeat from core client for 30 sec - exiting 04:30:57 (6648): No heartbeat from core client for 30 sec - exiting 04:30:58 (6648): No heartbeat from core client for 30 sec - exiting 04:30:59 (6648): No heartbeat from core client for 30 sec - exiting 04:31:00 (6648): No heartbeat from core client for 30 sec - exiting 04:31:01 (6648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:02 (6648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:18:14 (71856): No heartbeat from core client for 30 sec - exiting 10:18:15 (71856): No heartbeat from core client for 30 sec - exiting 10:18:16 (71856): No heartbeat from core client for 30 sec - exiting 10:18:17 (71856): No heartbeat from core client for 30 sec - exiting 10:18:18 (71856): No heartbeat from core client for 30 sec - exiting 10:18:19 (71856): No heartbeat from core client for 30 sec - exiting 10:18:20 (71856): No heartbeat from core client for 30 sec - exiting 10:18:21 (71856): No heartbeat from core client for 30 sec - exiting 10:18:22 (71856): No heartbeat from core client for 30 sec - exiting 10:18:23 (71856): No heartbeat from core client for 30 sec - exiting 10:18:24 (71856): No heartbeat from core client for 30 sec - exiting 10:18:25 (71856): No heartbeat from core client for 30 sec - exiting 10:18:26 (71856): No heartbeat from core client for 30 sec - exiting 10:18:27 (71856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:16:10 (10992): No heartbeat from core client for 30 sec - exiting 15:16:11 (10992): No heartbeat from core client for 30 sec - exiting 15:16:12 (10992): No heartbeat from core client for 30 sec - exiting 15:16:13 (10992): No heartbeat from core client for 30 sec - exiting 15:16:14 (10992): No heartbeat from core client for 30 sec - exiting 15:16:15 (10992): No heartbeat from core client for 30 sec - exiting 15:16:16 (10992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=57932, iMonCtr=1 Model crash detected, will try to restart... 12:54:33 (24184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:03:26 (84152): No heartbeat from core client for 30 sec - exiting 06:03:27 (84152): No heartbeat from core client for 30 sec - exiting 06:03:28 (84152): No heartbeat from core client for 30 sec - exiting 06:03:29 (84152): No heartbeat from core client for 30 sec - exiting 06:03:30 (84152): No heartbeat from core client for 30 sec - exiting 06:03:31 (84152): No heartbeat from core client for 30 sec - exiting 06:03:32 (84152): No heartbeat from core client for 30 sec - exiting 06:03:33 (84152): No heartbeat from core client for 30 sec - exiting 06:03:34 (84152): No heartbeat from core client for 30 sec - exiting 06:03:35 (84152): No heartbeat from core client for 30 sec - exiting 06:03:36 (84152): No heartbeat from core client for 30 sec - exiting 06:03:37 (84152): No heartbeat from core client for 30 sec - exiting 06:03:38 (84152): No heartbeat from core client for 30 sec - exiting 06:03:39 (84152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:34:23 (6008): No heartbeat from core client for 30 sec - exiting 23:34:24 (6008): No heartbeat from core client for 30 sec - exiting 23:34:25 (6008): No heartbeat from core client for 30 sec - exiting 23:34:26 (6008): No heartbeat from core client for 30 sec - exiting 23:34:27 (6008): No heartbeat from core client for 30 sec - exiting 23:34:28 (6008): No heartbeat from core client for 30 sec - exiting 23:34:29 (6008): No heartbeat from core client for 30 sec - exiting 23:34:30 (6008): No heartbeat from core client for 30 sec - exiting 23:34:31 (6008): No heartbeat from core client for 30 sec - exiting 23:34:32 (6008): No heartbeat from core client for 30 sec - exiting 23:34:33 (6008): No heartbeat from core client for 30 sec - exiting 23:34:34 (6008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:38:33 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=33432, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77873AB3 read attempt to address 0x409E7B2E Engaging BOINC Windows Runtime Debugger... Cannot serialize file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_4k6o_2020_40_008341130/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 May 2013 10:39:26	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	777,600	1,005,721	1.2934
27 Apr 2013 00:00:02	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	751,680	971,381	1.2923
26 Apr 2013 06:02:05	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	725,760	937,974	1.2924
25 Apr 2013 15:20:45	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	699,840	904,899	1.2930
24 Apr 2013 21:22:43	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	673,920	871,652	1.2934
23 Apr 2013 13:10:31	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	648,000	838,416	1.2939
22 Apr 2013 10:04:18	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	622,080	806,072	1.2958
21 Apr 2013 12:48:17	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	596,160	772,406	1.2956
20 Apr 2013 11:29:46	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	570,240	739,463	1.2968
19 Apr 2013 17:01:31	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	544,320	705,570	1.2962
18 Apr 2013 18:10:02	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	518,400	670,745	1.2939
17 Apr 2013 15:35:45	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	492,480	636,406	1.2922
16 Apr 2013 16:56:23	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	466,560	603,106	1.2927
16 Apr 2013 00:26:02	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	440,640	569,509	1.2925
15 Apr 2013 02:58:28	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	414,720	535,605	1.2915
14 Apr 2013 12:02:38	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	388,800	502,446	1.2923
13 Apr 2013 20:41:41	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	362,880	469,156	1.2929
13 Apr 2013 00:00:21	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	336,960	435,313	1.2919
11 Apr 2013 16:10:43	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	311,040	401,806	1.2918
10 Apr 2013 22:10:31	1217831	15692460	hadcm3n_4k6o_2020_40_008341130_0	285,120	367,952	1.2905