Task 13030234

Name	hadcm3n_t048_1940_40_007313524_2
Workunit	7510954
Created	30 Jun 2011, 7:06:37 UTC
Sent	30 Jun 2011, 7:07:16 UTC
Report deadline	29 Sep 2011, 14:34:27 UTC
Received	10 Aug 2011, 21:49:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1098636
Run time	9 days 23 hours 36 min 24 sec
CPU time	9 days 13 hours 4 min 32 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.08 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:11:56 (4916): No heartbeat from core client for 30 sec - exiting 19:11:57 (4916): No heartbeat from core client for 30 sec - exiting 19:11:58 (4916): No heartbeat from core client for 30 sec - exiting 19:11:59 (4916): No heartbeat from core client for 30 sec - exiting 19:12:00 (4916): No heartbeat from core client for 30 sec - exiting 19:12:02 (4916): No heartbeat from core client for 30 sec - exiting 19:12:03 (4916): No heartbeat from core client for 30 sec - exiting 19:12:04 (4916): No heartbeat from core client for 30 sec - exiting 19:12:05 (4916): No heartbeat from core client for 30 sec - exiting 19:12:06 (4916): No heartbeat from core client for 30 sec - exiting 19:12:07 (4916): No heartbeat from core client for 30 sec - exiting 19:12:08 (4916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:26:31 (3000): No heartbeat from core client for 30 sec - exiting 18:26:32 (3000): No heartbeat from core client for 30 sec - exiting 18:26:33 (3000): No heartbeat from core client for 30 sec - exiting 18:26:34 (3000): No heartbeat from core client for 30 sec - exiting 18:26:35 (3000): No heartbeat from core client for 30 sec - exiting 18:26:36 (3000): No heartbeat from core client for 30 sec - exiting 18:26:37 (3000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:17 (2216): No heartbeat from core client for 30 sec - exiting 14:20:18 (2216): No heartbeat from core client for 30 sec - exiting 14:20:19 (2216): No heartbeat from core client for 30 sec - exiting 14:20:21 (2216): No heartbeat from core client for 30 sec - exiting 14:20:22 (2216): No heartbeat from core client for 30 sec - exiting 14:20:23 (2216): No heartbeat from core client for 30 sec - exiting 14:20:24 (2216): No heartbeat from core client for 30 sec - exiting 14:20:25 (2216): No heartbeat from core client for 30 sec - exiting 14:20:26 (2216): No heartbeat from core client for 30 sec - exiting 14:20:27 (2216): No heartbeat from core client for 30 sec - exiting 14:20:28 (2216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:28:36 (4172): No heartbeat from core client for 30 sec - exiting 20:28:37 (4172): No heartbeat from core client for 30 sec - exiting 20:28:38 (4172): No heartbeat from core client for 30 sec - exiting 20:28:39 (4172): No heartbeat from core client for 30 sec - exiting 20:28:40 (4172): No heartbeat from core client for 30 sec - exiting 20:28:41 (4172): No heartbeat from core client for 30 sec - exiting 20:28:42 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C12:07:45 (4208): No heartbeat from core client for 30 sec - exiting 12:07:46 (4208): No heartbeat from core client for 30 sec - exiting 12:07:47 (4208): No heartbeat from core client for 30 sec - exiting 12:07:48 (4208): No heartbeat from core client for 30 sec - exiting 12:07:49 (4208): No heartbeat from core client for 30 sec - exiting 12:07:50 (4208): No heartbeat from core client for 30 sec - exiting 12:07:51 (4208): No heartbeat from core client for 30 sec - exiting 12:07:53 (4208): No heartbeat from core client for 30 sec - exiting 12:07:54 (4208): No heartbeat from core client for 30 sec - exiting 12:07:55 (4208): No heartbeat from core client for 30 sec - exiting 12:07:56 (4208): No heartbeat from core client for 30 sec - exiting 12:07:57 (4208): No heartbeat from core client for 30 sec - exiting 12:07:58 (4208): No heartbeat from core client for 30 sec - exiting 12:07:59 (4208): No heartbeat from core client for 30 sec - exiting 12:08:00 (4208): No heartbeat from core client for 30 sec - exiting 12:08:01 (4208): No heartbeat from core client for 30 sec - exiting 12:08:02 (4208): No heartbeat from core client for 30 sec - exiting 12:08:03 (4208): No heartbeat from core client for 30 sec - exiting 12:08:05 (4208): No heartbeat from core client for 30 sec - exiting 12:08:06 (4208): No heartbeat from core client for 30 sec - exiting 12:08:07 (4208): No heartbeat from core client for 30 sec - exiting 12:08:08 (4208): No heartbeat from core client for 30 sec - exiting 12:08:09 (4208): No heartbeat from core client for 30 sec - exiting 12:08:10 (4208): No heartbeat from core client for 30 sec - exiting 12:08:11 (4208): No heartbeat from core client for 30 sec - exiting 12:08:12 (4208): No heartbeat from core client for 30 sec - exiting 12:08:13 (4208): No heartbeat from core client for 30 sec - exiting 12:08:14 (4208): No heartbeat from core client for 30 sec - exiting 12:08:15 (4208): No heartbeat from core client for 30 sec - exiting 12:08:17 (4208): No heartbeat from core client for 30 sec - exiting 12:08:18 (4208): No heartbeat from core client for 30 sec - exiting 12:08:19 (4208): No heartbeat from core client for 30 sec - exiting 12:08:20 (4208): No heartbeat from core client for 30 sec - exiting 12:08:21 (4208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:09:07 (3488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2448, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:40:46 (3552): No heartbeat from core client for 30 sec - exiting 13:40:47 (3552): No heartbeat from core client for 30 sec - exiting 13:40:48 (3552): No heartbeat from core client for 30 sec - exiting 13:40:49 (3552): No heartbeat from core client for 30 sec - exiting 13:40:50 (3552): No heartbeat from core client for 30 sec - exiting 13:40:51 (3552): No heartbeat from core client for 30 sec - exiting 13:40:52 (3552): No heartbeat from core client for 30 sec - exiting 13:40:54 (3552): No heartbeat from core client for 30 sec - exiting 13:40:55 (3552): No heartbeat from core client for 30 sec - exiting 13:40:56 (3552): No heartbeat from core client for 30 sec - exiting 13:40:57 (3552): No heartbeat from core client for 30 sec - exiting 13:40:58 (3552): No heartbeat from core client for 30 sec - exiting 13:40:59 (3552): No heartbeat from core client for 30 sec - exiting 13:41:00 (3552): No heartbeat from core client for 30 sec - exiting 13:41:01 (3552): No heartbeat from core client for 30 sec - exiting 13:41:03 (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5204, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:42:49 (3968): No heartbeat from core client for 30 sec - exiting 20:42:50 (3968): No heartbeat from core client for 30 sec - exiting 20:42:52 (3968): No heartbeat from core client for 30 sec - exiting 20:42:53 (3968): No heartbeat from core client for 30 sec - exiting 20:42:54 (3968): No heartbeat from core client for 30 sec - exiting 20:42:55 (3968): No heartbeat from core client for 30 sec - exiting 20:42:56 (3968): No heartbeat from core client for 30 sec - exiting 20:42:57 (3968): No heartbeat from core client for 30 sec - exiting 20:42:58 (3968): No heartbeat from core client for 30 sec - exiting 20:42:59 (3968): No heartbeat from core client for 30 sec - exiting 20:43:00 (3968): No heartbeat from core client for 30 sec - exiting 20:43:01 (3968): No heartbeat from core client for 30 sec - exiting 20:43:02 (3968): No heartbeat from core client for 30 sec - exiting 20:43:04 (3968): No heartbeat from core client for 30 sec - exiting 20:43:05 (3968): No heartbeat from core client for 30 sec - exiting 20:43:06 (3968): No heartbeat from core client for 30 sec - exiting 20:43:07 (3968): No heartbeat from core client for 30 sec - exiting 20:43:08 (3968): No heartbeat from core client for 30 sec - exiting 20:43:09 (3968): No heartbeat from core client for 30 sec - exiting 20:43:10 (3968): No heartbeat from core client for 30 sec - exiting 20:43:11 (3968): No heartbeat from core client for 30 sec - exiting 20:43:12 (3968): No heartbeat from core client for 30 sec - exiting 20:43:13 (3968): No heartbeat from core client for 30 sec - exiting 20:43:15 (3968): No heartbeat from core client for 30 sec - exiting 20:43:16 (3968): No heartbeat from core client for 30 sec - exiting 20:43:17 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:43:57 (4704): No heartbeat from core client for 30 sec - exiting 22:43:58 (4704): No heartbeat from core client for 30 sec - exiting 22:43:59 (4704): No heartbeat from core client for 30 sec - exiting 22:44:01 (4704): No heartbeat from core client for 30 sec - exiting 22:44:02 (4704): No heartbeat from core client for 30 sec - exiting 22:44:03 (4704): No heartbeat from core client for 30 sec - exiting 22:44:04 (4704): No heartbeat from core client for 30 sec - exiting 22:44:05 (4704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:51:08 (2572): No heartbeat from core client for 30 sec - exiting 00:51:09 (2572): No heartbeat from core client for 30 sec - exiting 00:51:10 (2572): No heartbeat from core client for 30 sec - exiting 00:51:12 (2572): No heartbeat from core client for 30 sec - exiting 00:51:13 (2572): No heartbeat from core client for 30 sec - exiting 00:51:14 (2572): No heartbeat from core client for 30 sec - exiting 00:51:15 (2572): No heartbeat from core client for 30 sec - exiting 00:51:16 (2572): No heartbeat from core client for 30 sec - exiting 00:51:17 (2572): No heartbeat from core client for 30 sec - exiting 00:51:18 (2572): No heartbeat from core client for 30 sec - exiting 00:51:19 (2572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:56:47 (1928): No heartbeat from core client for 30 sec - exiting 00:56:48 (1928): No heartbeat from core client for 30 sec - exiting 00:56:49 (1928): No heartbeat from core client for 30 sec - exiting 00:56:50 (1928): No heartbeat from core client for 30 sec - exiting 00:56:51 (1928): No heartbeat from core client for 30 sec - exiting 00:56:53 (1928): No heartbeat from core client for 30 sec - exiting 00:56:54 (1928): No heartbeat from core client for 30 sec - exiting 00:56:55 (1928): No heartbeat from core client for 30 sec - exiting 00:56:56 (1928): No heartbeat from core client for 30 sec - exiting 00:56:57 (1928): No heartbeat from core client for 30 sec - exiting 00:56:58 (1928): No heartbeat from core client for 30 sec - exiting 00:56:59 (1928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:58:01 (3356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:59:15 (1068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:39:17 (5264): No heartbeat from core client for 30 sec - exiting 18:39:18 (5264): No heartbeat from core client for 30 sec - exiting 18:39:19 (5264): No heartbeat from core client for 30 sec - exiting 18:39:20 (5264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:56:29 (2692): No heartbeat from core client for 30 sec - exiting 06:56:30 (2692): No heartbeat from core client for 30 sec - exiting 06:56:32 (2692): No heartbeat from core client for 30 sec - exiting 06:56:33 (2692): No heartbeat from core client for 30 sec - exiting 06:56:34 (2692): No heartbeat from core client for 30 sec - exiting 06:56:35 (2692): No heartbeat from core client for 30 sec - exiting 06:56:36 (2692): No heartbeat from core client for 30 sec - exiting 06:56:37 (2692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:20:33 (4324): No heartbeat from core client for 30 sec - exiting 22:20:34 (4324): No heartbeat from core client for 30 sec - exiting 22:20:35 (4324): No heartbeat from core client for 30 sec - exiting 22:20:37 (4324): No heartbeat from core client for 30 sec - exiting 22:20:38 (4324): No heartbeat from core client for 30 sec - exiting 22:20:39 (4324): No heartbeat from core client for 30 sec - exiting 22:20:40 (4324): No heartbeat from core client for 30 sec - exiting 22:20:41 (4324): No heartbeat from core client for 30 sec - exiting 22:20:42 (4324): No heartbeat from core client for 30 sec - exiting 22:20:43 (4324): No heartbeat from core client for 30 sec - exiting 22:20:44 (4324): No heartbeat from core client for 30 sec - exiting 22:20:45 (4324): No heartbeat from core client for 30 sec - exiting 22:20:46 (4324): No heartbeat from core client for 30 sec - exiting 22:20:47 (4324): No heartbeat from core client for 30 sec - exiting 22:20:49 (4324): No heartbeat from core client for 30 sec - exiting 22:20:50 (4324): No heartbeat from core client for 30 sec - exiting 22:20:51 (4324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:52 (4324): No heartbeat from core client for 30 sec - exiting 11:03:45 (1064): No heartbeat from core client for 30 sec - exiting 11:03:46 (1064): No heartbeat from core client for 30 sec - exiting 11:03:47 (1064): No heartbeat from core client for 30 sec - exiting 11:03:48 (1064): No heartbeat from core client for 30 sec - exiting 11:03:49 (1064): No heartbeat from core client for 30 sec - exiting 11:03:51 (1064): No heartbeat from core client for 30 sec - exiting 11:03:52 (1064): No heartbeat from core client for 30 sec - exiting 11:03:53 (1064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:03:54 (1064): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5808, iMonCtr=1 Model crash detected, will try to restart... 11:27:56 (5064): No heartbeat from core client for 30 sec - exiting 11:27:57 (5064): No heartbeat from core client for 30 sec - exiting 11:27:58 (5064): No heartbeat from core client for 30 sec - exiting 11:27:59 (5064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:14:17 (4620): No heartbeat from core client for 30 sec - exiting 13:14:18 (4620): No heartbeat from core client for 30 sec - exiting 13:14:19 (4620): No heartbeat from core client for 30 sec - exiting 13:14:20 (4620): No heartbeat from core client for 30 sec - exiting 13:14:21 (4620): No heartbeat from core client for 30 sec - exiting 13:14:22 (4620): No heartbeat from core client for 30 sec - exiting 13:14:24 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:25:32 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:44:56 (4420): No heartbeat from core client for 30 sec - exiting 17:44:57 (4420): No heartbeat from core client for 30 sec - exiting 17:44:58 (4420): No heartbeat from core client for 30 sec - exiting 17:44:59 (4420): No heartbeat from core client for 30 sec - exiting 17:45:00 (4420): No heartbeat from core client for 30 sec - exiting 17:45:02 (4420): No heartbeat from core client for 30 sec - exiting 17:45:03 (4420): No heartbeat from core client for 30 sec - exiting 17:45:04 (4420): No heartbeat from core client for 30 sec - exiting 17:45:05 (4420): No heartbeat from core client for 30 sec - exiting 17:45:06 (4420): No heartbeat from core client for 30 sec - exiting 17:45:07 (4420): No heartbeat from core client for 30 sec - exiting 17:45:08 (4420): No heartbeat from core client for 30 sec - exiting 17:45:09 (4420): No heartbeat from core client for 30 sec - exiting 17:45:10 (4420): No heartbeat from core client for 30 sec - exiting 17:45:11 (4420): No heartbeat from core client for 30 sec - exiting 17:45:12 (4420): No heartbeat from core client for 30 sec - exiting 17:45:14 (4420): No heartbeat from core client for 30 sec - exiting 17:45:15 (4420): No heartbeat from core client for 30 sec - exiting 17:45:16 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:16:25 (3928): No heartbeat from core client for 30 sec - exiting 22:16:26 (3928): No heartbeat from core client for 30 sec - exiting 22:16:27 (3928): No heartbeat from core client for 30 sec - exiting 22:16:28 (3928): No heartbeat from core client for 30 sec - exiting 22:16:29 (3928): No heartbeat from core client for 30 sec - exiting 22:16:30 (3928): No heartbeat from core client for 30 sec - exiting 22:16:31 (3928): No heartbeat from core client for 30 sec - exiting 22:16:32 (3928): No heartbeat from core client for 30 sec - exiting 22:16:33 (3928): No heartbeat from core client for 30 sec - exiting 22:16:34 (3928): No heartbeat from core client for 30 sec - exiting 22:16:36 (3928): No heartbeat from core client for 30 sec - exiting 22:16:37 (3928): No heartbeat from core client for 30 sec - exiting 22:16:38 (3928): No heartbeat from core client for 30 sec - exiting 22:16:39 (3928): No heartbeat from core client for 30 sec - exiting 22:16:40 (3928): No heartbeat from core client for 30 sec - exiting 22:16:41 (3928): No heartbeat from core client for 30 sec - exiting 22:16:42 (3928): No heartbeat from core client for 30 sec - exiting 22:16:43 (3928): No heartbeat from core client for 30 sec - exiting 22:16:44 (3928): No heartbeat from core client for 30 sec - exiting 22:16:45 (3928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6064, iMonCtr=1 Model crash detected, will try to restart... 12:09:42 (3808): No heartbeat from core client for 30 sec - exiting 12:09:44 (3808): No heartbeat from core client for 30 sec - exiting 12:09:45 (3808): No heartbeat from core client for 30 sec - exiting 12:09:46 (3808): No heartbeat from core client for 30 sec - exiting 12:09:47 (3808): No heartbeat from core client for 30 sec - exiting 12:09:48 (3808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1984, iMonCtr=1 Model crash detected, will try to restart... 05:57:36 (2676): No heartbeat from core client for 30 sec - exiting 05:57:37 (2676): No heartbeat from core client for 30 sec - exiting 05:57:38 (2676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:11:48 (4456): No heartbeat from core client for 30 sec - exiting 21:11:49 (4456): No heartbeat from core client for 30 sec - exiting 21:11:50 (4456): No heartbeat from core client for 30 sec - exiting 21:11:51 (4456): No heartbeat from core client for 30 sec - exiting 21:11:52 (4456): No heartbeat from core client for 30 sec - exiting 21:11:53 (4456): No heartbeat from core client for 30 sec - exiting 21:11:54 (4456): No heartbeat from core client for 30 sec - exiting 21:11:56 (4456): No heartbeat from core client for 30 sec - exiting 21:11:57 (4456): No heartbeat from core client for 30 sec - exiting 21:11:58 (4456): No heartbeat from core client for 30 sec - exiting 21:11:59 (4456): No heartbeat from core client for 30 sec - exiting 21:12:00 (4456): No heartbeat from core client for 30 sec - exiting 21:12:01 (4456): No heartbeat from core client for 30 sec - exiting 21:12:02 (4456): No heartbeat from core client for 30 sec - exiting 21:12:03 (4456): No heartbeat from core client for 30 sec - exiting 21:12:04 (4456): No heartbeat from core client for 30 sec - exiting 21:12:05 (4456): No heartbeat from core client for 30 sec - exiting 21:12:06 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:21:37 (4148): No heartbeat from core client for 30 sec - exiting 19:21:38 (4148): No heartbeat from core client for 30 sec - exiting 19:21:39 (4148): No heartbeat from core client for 30 sec - exiting 19:21:40 (4148): No heartbeat from core client for 30 sec - exiting 19:21:42 (4148): No heartbeat from core client for 30 sec - exiting 19:21:43 (4148): No heartbeat from core client for 30 sec - exiting 19:21:44 (4148): No heartbeat from core client for 30 sec - exiting 19:21:45 (4148): No heartbeat from core client for 30 sec - exiting 19:21:46 (4148): No heartbeat from core client for 30 sec - exiting 19:21:47 (4148): No heartbeat from core client for 30 sec - exiting 19:21:48 (4148): No heartbeat from core client for 30 sec - exiting 19:21:49 (4148): No heartbeat from core client for 30 sec - exiting 19:21:50 (4148): No heartbeat from core client for 30 sec - exiting 19:21:51 (4148): No heartbeat from core client for 30 sec - exiting 19:21:53 (4148): No heartbeat from core client for 30 sec - exiting 19:21:54 (4148): No heartbeat from core client for 30 sec - exiting 19:21:55 (4148): No heartbeat from core client for 30 sec - exiting 19:21:56 (4148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2536, iMonCtr=1 Model crash detected, will try to restart... 16:17:52 (2384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:17:53 (2384): No heartbeat from core client for 30 sec - exiting 16:17:54 (2384): No heartbeat from core client for 30 sec - exiting 16:17:55 (2384): No heartbeat from core client for 30 sec - exiting 16:17:56 (2384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:02:50 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:21:50 (3448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:45:07 (3428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:40 (1156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3480, iMonCtr=1 Model crash detected, will try to restart... 20:17:40 (4244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:11:13 (4996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:39:07 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:00:09 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B73A93 read attempt to address 0x40339CE2 Engaging BOINC Windows Runtime Debugger... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2856, selfPID=2856, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77313A93 read attempt to address 0x40339CE2 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_t048_1940_40_007313524/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Aug 2011 21:56:32	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	518,400	823,741	1.5890
08 Aug 2011 20:48:03	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	492,480	783,319	1.5906
07 Aug 2011 11:47:14	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	466,560	742,926	1.5923
06 Aug 2011 16:13:06	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	440,640	702,349	1.5939
01 Aug 2011 23:27:57	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	414,720	662,157	1.5966
30 Jul 2011 18:16:51	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	388,800	622,153	1.6002
28 Jul 2011 00:35:11	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	362,880	582,063	1.6040
27 Jul 2011 09:42:19	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	336,960	541,840	1.6080
26 Jul 2011 21:44:58	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	311,040	501,373	1.6119
25 Jul 2011 22:16:08	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	285,120	461,066	1.6171
25 Jul 2011 21:03:55	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	259,200	420,041	1.6205
25 Jul 2011 17:42:31	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	233,280	377,466	1.6181
25 Jul 2011 16:15:26	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	207,360	336,301	1.6218
25 Jul 2011 15:31:49	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	181,440	295,208	1.6270
25 Jul 2011 14:46:00	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	155,520	252,844	1.6258
25 Jul 2011 14:46:00	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	129,600	212,464	1.6394
10 Jul 2011 17:41:26	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	103,680	170,233	1.6419
08 Jul 2011 08:48:54	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	77,760	129,876	1.6702
07 Jul 2011 15:39:00	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	51,840	84,970	1.6391
02 Jul 2011 21:43:53	1098636	13030234	hadcm3n_t048_1940_40_007313524_2	25,920	42,968	1.6577