Task 15913765

Name	hadcm3n_4fog_1940_40_008302722_2
Workunit	8453857
Created	14 Aug 2013, 11:37:43 UTC
Sent	15 Aug 2013, 6:16:47 UTC
Report deadline	14 Nov 2013, 13:43:58 UTC
Received	28 Sep 2013, 22:42:27 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1129647
Run time	24 days 2 hours 0 min 5 sec
CPU time	22 days 19 hours 44 min 55 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.90 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <stderr_txt> 13:08:43 (5676): No heartbeat from core client for 30 sec - exiting 13:08:44 (5676): No heartbeat from core client for 30 sec - exiting 13:08:45 (5676): No heartbeat from core client for 30 sec - exiting 13:08:46 (5676): No heartbeat from core client for 30 sec - exiting 13:08:47 (5676): No heartbeat from core client for 30 sec - exiting 13:08:48 (5676): No heartbeat from core client for 30 sec - exiting 13:08:49 (5676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4168, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 05:33:14 (5448): No heartbeat from core client for 30 sec - exiting 05:33:15 (5448): No heartbeat from core client for 30 sec - exiting 05:33:16 (5448): No heartbeat from core client for 30 sec - exiting 05:33:17 (5448): No heartbeat from core client for 30 sec - exiting 05:33:18 (5448): No heartbeat from core client for 30 sec - exiting 05:33:19 (5448): No heartbeat from core client for 30 sec - exiting 05:33:20 (5448): No heartbeat from core client for 30 sec - exiting 05:33:21 (5448): No heartbeat from core client for 30 sec - exiting 05:33:22 (5448): No heartbeat from core client for 30 sec - exiting 05:33:23 (5448): No heartbeat from core client for 30 sec - exiting 05:33:24 (5448): No heartbeat from core client for 30 sec - exiting 05:33:25 (5448): No heartbeat from core client for 30 sec - exiting 05:33:26 (5448): No heartbeat from core client for 30 sec - exiting 05:33:27 (5448): No heartbeat from core client for 30 sec - exiting 05:33:28 (5448): No heartbeat from core client for 30 sec - exiting 05:33:29 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:59 (5512): No heartbeat from core client for 30 sec - exiting 16:44:01 (5512): No heartbeat from core client for 30 sec - exiting 16:44:02 (5512): No heartbeat from core client for 30 sec - exiting 16:44:03 (5512): No heartbeat from core client for 30 sec - exiting 16:44:04 (5512): No heartbeat from core client for 30 sec - exiting 16:44:05 (5512): No heartbeat from core client for 30 sec - exiting 16:44:06 (5512): No heartbeat from core client for 30 sec - exiting 16:44:07 (5512): No heartbeat from core client for 30 sec - exiting 16:44:08 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:31:19 (5428): No heartbeat from core client for 30 sec - exiting 05:31:21 (5428): No heartbeat from core client for 30 sec - exiting 05:31:22 (5428): No heartbeat from core client for 30 sec - exiting 05:31:23 (5428): No heartbeat from core client for 30 sec - exiting 05:31:24 (5428): No heartbeat from core client for 30 sec - exiting 05:31:25 (5428): No heartbeat from core client for 30 sec - exiting 05:31:26 (5428): No heartbeat from core client for 30 sec - exiting 05:31:27 (5428): No heartbeat from core client for 30 sec - exiting 05:31:28 (5428): No heartbeat from core client for 30 sec - exiting 05:31:29 (5428): No heartbeat from core client for 30 sec - exiting 05:31:30 (5428): No heartbeat from core client for 30 sec - exiting 05:31:31 (5428): No heartbeat from core client for 30 sec - exiting 05:31:32 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:47 (4660): No heartbeat from core client for 30 sec - exiting 16:31:48 (4660): No heartbeat from core client for 30 sec - exiting 16:31:49 (4660): No heartbeat from core client for 30 sec - exiting 16:31:50 (4660): No heartbeat from core client for 30 sec - exiting 16:31:52 (4660): No heartbeat from core client for 30 sec - exiting 16:31:53 (4660): No heartbeat from core client for 30 sec - exiting 16:31:54 (4660): No heartbeat from core client for 30 sec - exiting 16:31:55 (4660): No heartbeat from core client for 30 sec - exiting 16:31:56 (4660): No heartbeat from core client for 30 sec - exiting 16:31:57 (4660): No heartbeat from core client for 30 sec - exiting 16:31:58 (4660): No heartbeat from core client for 30 sec - exiting 16:31:59 (4660): No heartbeat from core client for 30 sec - exiting 16:32:00 (4660): No heartbeat from core client for 30 sec - exiting 16:32:01 (4660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:31:10 (5388): No heartbeat from core client for 30 sec - exiting 05:31:11 (5388): No heartbeat from core client for 30 sec - exiting 05:31:12 (5388): No heartbeat from core client for 30 sec - exiting 05:31:13 (5388): No heartbeat from core client for 30 sec - exiting 05:31:14 (5388): No heartbeat from core client for 30 sec - exiting 05:31:15 (5388): No heartbeat from core client for 30 sec - exiting 05:31:16 (5388): No heartbeat from core client for 30 sec - exiting 05:31:17 (5388): No heartbeat from core client for 30 sec - exiting 05:31:18 (5388): No heartbeat from core client for 30 sec - exiting 05:31:19 (5388): No heartbeat from core client for 30 sec - exiting 05:31:20 (5388): No heartbeat from core client for 30 sec - exiting 05:31:21 (5388): No heartbeat from core client for 30 sec - exiting 05:31:22 (5388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5032, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5252, iMonCtr=1 Model crash detected, will try to restart... 07:54:31 (10048): No heartbeat from core client for 30 sec - exiting 07:54:32 (10048): No heartbeat from core client for 30 sec - exiting 07:54:33 (10048): No heartbeat from core client for 30 sec - exiting 07:54:34 (10048): No heartbeat from core client for 30 sec - exiting 07:54:35 (10048): No heartbeat from core client for 30 sec - exiting 07:54:36 (10048): No heartbeat from core client for 30 sec - exiting 07:54:37 (10048): No heartbeat from core client for 30 sec - exiting 07:54:38 (10048): No heartbeat from core client for 30 sec - exiting 07:54:39 (10048): No heartbeat from core client for 30 sec - exiting 07:54:40 (10048): No heartbeat from core client for 30 sec - exiting 07:54:41 (10048): No heartbeat from core client for 30 sec - exiting 07:54:42 (10048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:18:12 (264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:26:20 (8928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:54:28 (8828): No heartbeat from core client for 30 sec - exiting 13:54:29 (8828): No heartbeat from core client for 30 sec - exiting 13:54:30 (8828): No heartbeat from core client for 30 sec - exiting 13:54:31 (8828): No heartbeat from core client for 30 sec - exiting 13:54:32 (8828): No heartbeat from core client for 30 sec - exiting 13:54:33 (8828): No heartbeat from core client for 30 sec - exiting 13:54:34 (8828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:20:26 (5352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:45:10 (5968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:38:27 (2276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:46:05 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:30:40 (856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:48:15 (5432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:56:57 (5752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:00:09 (2840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:44:53 (5480): No heartbeat from core client for 30 sec - exiting 05:44:54 (5480): No heartbeat from core client for 30 sec - exiting 05:44:55 (5480): No heartbeat from core client for 30 sec - exiting 05:44:56 (5480): No heartbeat from core client for 30 sec - exiting 05:44:57 (5480): No heartbeat from core client for 30 sec - exiting 05:44:58 (5480): No heartbeat from core client for 30 sec - exiting 05:44:59 (5480): No heartbeat from core client for 30 sec - exiting 05:45:00 (5480): No heartbeat from core client for 30 sec - exiting 05:45:01 (5480): No heartbeat from core client for 30 sec - exiting 05:45:02 (5480): No heartbeat from core client for 30 sec - exiting 05:45:03 (5480): No heartbeat from core client for 30 sec - exiting 05:45:04 (5480): No heartbeat from core client for 30 sec - exiting 05:45:05 (5480): No heartbeat from core client for 30 sec - exiting 05:45:06 (5480): No heartbeat from core client for 30 sec - exiting 05:45:07 (5480): No heartbeat from core client for 30 sec - exiting 05:45:08 (5480): No heartbeat from core client for 30 sec - exiting 05:45:09 (5480): No heartbeat from core client for 30 sec - exiting 05:45:10 (5480): No heartbeat from core client for 30 sec - exiting 05:45:11 (5480): No heartbeat from core client for 30 sec - exiting 05:45:12 (5480): No heartbeat from core client for 30 sec - exiting 05:45:13 (5480): No heartbeat from core client for 30 sec - exiting 05:45:14 (5480): No heartbeat from core client for 30 sec - exiting 05:45:15 (5480): No heartbeat from core client for 30 sec - exiting 05:45:16 (5480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:13:22 (5660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:11:42 (864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:15:05 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:49:17 (5800): No heartbeat from core client for 30 sec - exiting 16:49:19 (5800): No heartbeat from core client for 30 sec - exiting 16:49:20 (5800): No heartbeat from core client for 30 sec - exiting 16:49:21 (5800): No heartbeat from core client for 30 sec - exiting 16:49:22 (5800): No heartbeat from core client for 30 sec - exiting 16:49:23 (5800): No heartbeat from core client for 30 sec - exiting 16:49:24 (5800): No heartbeat from core client for 30 sec - exiting 16:49:25 (5800): No heartbeat from core client for 30 sec - exiting 16:49:26 (5800): No heartbeat from core client for 30 sec - exiting 16:49:27 (5800): No heartbeat from core client for 30 sec - exiting 16:49:28 (5800): No heartbeat from core client for 30 sec - exiting 16:49:29 (5800): No heartbeat from core client for 30 sec - exiting 16:49:30 (5800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:49:31 (5800): No heartbeat from core client for 30 sec - exiting 16:56:24 (4236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5516, iMonCtr=1 Model crash detected, will try to restart... 14:21:23 (4152): No heartbeat from core client for 30 sec - exiting 14:21:24 (4152): No heartbeat from core client for 30 sec - exiting 14:21:25 (4152): No heartbeat from core client for 30 sec - exiting 14:21:26 (4152): No heartbeat from core client for 30 sec - exiting 14:21:27 (4152): No heartbeat from core client for 30 sec - exiting 14:21:28 (4152): No heartbeat from core client for 30 sec - exiting 14:21:29 (4152): No heartbeat from core client for 30 sec - exiting 14:21:30 (4152): No heartbeat from core client for 30 sec - exiting 14:21:31 (4152): No heartbeat from core client for 30 sec - exiting 14:21:32 (4152): No heartbeat from core client for 30 sec - exiting 14:21:33 (4152): No heartbeat from core client for 30 sec - exiting 14:21:34 (4152): No heartbeat from core client for 30 sec - exiting 14:21:35 (4152): No heartbeat from core client for 30 sec - exiting 14:21:36 (4152): No heartbeat from core client for 30 sec - exiting 14:21:37 (4152): No heartbeat from core client for 30 sec - exiting 14:21:38 (4152): No heartbeat from core client for 30 sec - exiting 14:21:39 (4152): No heartbeat from core client for 30 sec - exiting 14:21:40 (4152): No heartbeat from core client for 30 sec - exiting 14:21:41 (4152): No heartbeat from core client for 30 sec - exiting 14:21:42 (4152): No heartbeat from core client for 30 sec - exiting 14:21:43 (4152): No heartbeat from core client for 30 sec - exiting 14:21:44 (4152): No heartbeat from core client for 30 sec - exiting 14:21:45 (4152): No heartbeat from core client for 30 sec - exiting 14:21:46 (4152): No heartbeat from core client for 30 sec - exiting 14:21:47 (4152): No heartbeat from core client for 30 sec - exiting 14:21:48 (4152): No heartbeat from core client for 30 sec - exiting 14:21:49 (4152): No heartbeat from core client for 30 sec - exiting 14:21:50 (4152): No heartbeat from core client for 30 sec - exiting 14:21:51 (4152): No heartbeat from core client for 30 sec - exiting 14:21:52 (4152): No heartbeat from core client for 30 sec - exiting 14:21:53 (4152): No heartbeat from core client for 30 sec - exiting 14:21:54 (4152): No heartbeat from core client for 30 sec - exiting 14:21:55 (4152): No heartbeat from core client for 30 sec - exiting 14:21:56 (4152): No heartbeat from core client for 30 sec - exiting 14:21:57 (4152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:11:31 (5924): No heartbeat from core client for 30 sec - exiting 12:11:32 (5924): No heartbeat from core client for 30 sec - exiting 12:11:33 (5924): No heartbeat from core client for 30 sec - exiting 12:11:34 (5924): No heartbeat from core client for 30 sec - exiting 12:11:35 (5924): No heartbeat from core client for 30 sec - exiting 12:11:36 (5924): No heartbeat from core client for 30 sec - exiting 12:11:37 (5924): No heartbeat from core client for 30 sec - exiting 12:11:38 (5924): No heartbeat from core client for 30 sec - exiting 12:11:39 (5924): No heartbeat from core client for 30 sec - exiting 12:11:40 (5924): No heartbeat from core client for 30 sec - exiting 12:11:41 (5924): No heartbeat from core client for 30 sec - exiting 12:11:42 (5924): No heartbeat from core client for 30 sec - exiting 12:11:43 (5924): No heartbeat from core client for 30 sec - exiting 12:11:44 (5924): No heartbeat from core client for 30 sec - exiting 12:11:45 (5924): No heartbeat from core client for 30 sec - exiting 12:11:46 (5924): No heartbeat from core client for 30 sec - exiting 12:11:47 (5924): No heartbeat from core client for 30 sec - exiting 12:11:48 (5924): No heartbeat from core client for 30 sec - exiting 12:11:49 (5924): No heartbeat from core client for 30 sec - exiting 12:11:50 (5924): No heartbeat from core client for 30 sec - exiting 12:11:51 (5924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:23:12 (5828): No heartbeat from core client for 30 sec - exiting 05:23:14 (5828): No heartbeat from core client for 30 sec - exiting 05:23:15 (5828): No heartbeat from core client for 30 sec - exiting 05:23:16 (5828): No heartbeat from core client for 30 sec - exiting 05:23:17 (5828): No heartbeat from core client for 30 sec - exiting 05:23:18 (5828): No heartbeat from core client for 30 sec - exiting 05:23:19 (5828): No heartbeat from core client for 30 sec - exiting 05:23:20 (5828): No heartbeat from core client for 30 sec - exiting 05:23:21 (5828): No heartbeat from core client for 30 sec - exiting 05:23:22 (5828): No heartbeat from core client for 30 sec - exiting 05:23:23 (5828): No heartbeat from core client for 30 sec - exiting 05:23:24 (5828): No heartbeat from core client for 30 sec - exiting 05:23:25 (5828): No heartbeat from core client for 30 sec - exiting 05:23:26 (5828): No heartbeat from core client for 30 sec - exiting 05:23:27 (5828): No heartbeat from core client for 30 sec - exiting 05:23:28 (5828): No heartbeat from core client for 30 sec - exiting 05:23:29 (5828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:22 (5720): No heartbeat from core client for 30 sec - exiting 16:45:23 (5720): No heartbeat from core client for 30 sec - exiting 16:45:24 (5720): No heartbeat from core client for 30 sec - exiting 16:45:25 (5720): No heartbeat from core client for 30 sec - exiting 16:45:26 (5720): No heartbeat from core client for 30 sec - exiting 16:45:27 (5720): No heartbeat from core client for 30 sec - exiting 16:45:28 (5720): No heartbeat from core client for 30 sec - exiting 16:45:29 (5720): No heartbeat from core client for 30 sec - exiting 16:45:30 (5720): No heartbeat from core client for 30 sec - exiting 16:45:31 (5720): No heartbeat from core client for 30 sec - exiting 16:45:32 (5720): No heartbeat from core client for 30 sec - exiting 16:45:33 (5720): No heartbeat from core client for 30 sec - exiting 16:45:34 (5720): No heartbeat from core client for 30 sec - exiting 16:45:35 (5720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:36:01 (5808): No heartbeat from core client for 30 sec - exiting 05:36:02 (5808): No heartbeat from core client for 30 sec - exiting 05:36:03 (5808): No heartbeat from core client for 30 sec - exiting 05:36:04 (5808): No heartbeat from core client for 30 sec - exiting 05:36:05 (5808): No heartbeat from core client for 30 sec - exiting 05:36:06 (5808): No heartbeat from core client for 30 sec - exiting 05:36:07 (5808): No heartbeat from core client for 30 sec - exiting 05:36:08 (5808): No heartbeat from core client for 30 sec - exiting 05:36:09 (5808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Ocean Restart file copy failed on 4fogko.dah0c20 16:42:44 (6136): No heartbeat from core client for 30 sec - exiting 16:42:45 (6136): No heartbeat from core client for 30 sec - exiting 16:42:46 (6136): No heartbeat from core client for 30 sec - exiting 16:42:47 (6136): No heartbeat from core client for 30 sec - exiting 16:42:48 (6136): No heartbeat from core client for 30 sec - exiting 16:42:49 (6136): No heartbeat from core client for 30 sec - exiting 16:42:50 (6136): No heartbeat from core client for 30 sec - exiting 16:42:51 (6136): No heartbeat from core client for 30 sec - exiting 16:42:52 (6136): No heartbeat from core client for 30 sec - exiting 16:42:53 (6136): No heartbeat from core client for 30 sec - exiting 16:42:54 (6136): No heartbeat from core client for 30 sec - exiting 16:42:55 (6136): No heartbeat from core client for 30 sec - exiting 16:42:56 (6136): No heartbeat from core client for 30 sec - exiting 16:42:57 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4376, iMonCtr=1 Model crash detected, will try to restart... 13:00:54 (5892): No heartbeat from core client for 30 sec - exiting 13:00:55 (5892): No heartbeat from core client for 30 sec - exiting 13:00:56 (5892): No heartbeat from core client for 30 sec - exiting 13:00:57 (5892): No heartbeat from core client for 30 sec - exiting 13:00:58 (5892): No heartbeat from core client for 30 sec - exiting 13:00:59 (5892): No heartbeat from core client for 30 sec - exiting 13:01:00 (5892): No heartbeat from core client for 30 sec - exiting 13:01:01 (5892): No heartbeat from core client for 30 sec - exiting 13:01:02 (5892): No heartbeat from core client for 30 sec - exiting 13:01:03 (5892): No heartbeat from core client for 30 sec - exiting 13:01:04 (5892): No heartbeat from core client for 30 sec - exiting 13:01:05 (5892): No heartbeat from core client for 30 sec - exiting 13:01:06 (5892): No heartbeat from core client for 30 sec - exiting 13:01:07 (5892): No heartbeat from core client for 30 sec - exiting 13:01:08 (5892): No heartbeat from core client for 30 sec - exiting 13:01:09 (5892): No heartbeat from core client for 30 sec - exiting 13:01:10 (5892): No heartbeat from core client for 30 sec - exiting 13:01:11 (5892): No heartbeat from core client for 30 sec - exiting 13:01:12 (5892): No heartbeat from core client for 30 sec - exiting 13:01:13 (5892): No heartbeat from core client for 30 sec - exiting 13:01:14 (5892): No heartbeat from core client for 30 sec - exiting 13:01:15 (5892): No heartbeat from core client for 30 sec - exiting 13:01:16 (5892): No heartbeat from core client for 30 sec - exiting 13:01:17 (5892): No heartbeat from core client for 30 sec - exiting 13:01:18 (5892): No heartbeat from core client for 30 sec - exiting 13:01:19 (5892): No heartbeat from core client for 30 sec - exiting 13:01:20 (5892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:18:19 (2008): No heartbeat from core client for 30 sec - exiting 12:18:20 (2008): No heartbeat from core client for 30 sec - exiting 12:18:21 (2008): No heartbeat from core client for 30 sec - exiting 12:18:22 (2008): No heartbeat from core client for 30 sec - exiting 12:18:23 (2008): No heartbeat from core client for 30 sec - exiting 12:18:24 (2008): No heartbeat from core client for 30 sec - exiting 12:18:25 (2008): No heartbeat from core client for 30 sec - exiting 12:18:26 (2008): No heartbeat from core client for 30 sec - exiting 12:18:27 (2008): No heartbeat from core client for 30 sec - exiting 12:18:28 (2008): No heartbeat from core client for 30 sec - exiting 12:18:29 (2008): No heartbeat from core client for 30 sec - exiting 12:18:30 (2008): No heartbeat from core client for 30 sec - exiting 12:18:31 (2008): No heartbeat from core client for 30 sec - exiting 12:18:32 (2008): No heartbeat from core client for 30 sec - exiting 12:18:33 (2008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5496, iMonCtr=1 Model crash detected, will try to restart... 07:12:56 (5312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6104, iMonCtr=1 Model crash detected, will try to restart... 05:32:24 (5808): No heartbeat from core client for 30 sec - exiting 05:32:25 (5808): No heartbeat from core client for 30 sec - exiting 05:32:26 (5808): No heartbeat from core client for 30 sec - exiting 05:32:27 (5808): No heartbeat from core client for 30 sec - exiting 05:32:28 (5808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:40:03 (4468): No heartbeat from core client for 30 sec - exiting 16:40:04 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:32:28 (5496): No heartbeat from core client for 30 sec - exiting 05:32:29 (5496): No heartbeat from core client for 30 sec - exiting 05:32:30 (5496): No heartbeat from core client for 30 sec - exiting 05:32:31 (5496): No heartbeat from core client for 30 sec - exiting 05:32:32 (5496): No heartbeat from core client for 30 sec - exiting 05:32:33 (5496): No heartbeat from core client for 30 sec - exiting 05:32:34 (5496): No heartbeat from core client for 30 sec - exiting 05:32:35 (5496): No heartbeat from core client for 30 sec - exiting 05:32:36 (5496): No heartbeat from core client for 30 sec - exiting 05:32:37 (5496): No heartbeat from core client for 30 sec - exiting 05:32:38 (5496): No heartbeat from core client for 30 sec - exiting 05:32:39 (5496): No heartbeat from core client for 30 sec - exiting 05:32:40 (5496): No heartbeat from core client for 30 sec - exiting 05:32:41 (5496): No heartbeat from core client for 30 sec - exiting 05:32:42 (5496): No heartbeat from core client for 30 sec - exiting 05:32:43 (5496): No heartbeat from core client for 30 sec - exiting 05:32:44 (5496): No heartbeat from core client for 30 sec - exiting 05:32:45 (5496): No heartbeat from core client for 30 sec - exiting 05:32:46 (5496): No heartbeat from core client for 30 sec - exiting 05:32:47 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:52 (4420): No heartbeat from core client for 30 sec - exiting 16:43:53 (4420): No heartbeat from core client for 30 sec - exiting 16:43:54 (4420): No heartbeat from core client for 30 sec - exiting 16:43:55 (4420): No heartbeat from core client for 30 sec - exiting 16:43:56 (4420): No heartbeat from core client for 30 sec - exiting 16:43:57 (4420): No heartbeat from core client for 30 sec - exiting 16:43:58 (4420): No heartbeat from core client for 30 sec - exiting 16:43:59 (4420): No heartbeat from core client for 30 sec - exiting 16:44:00 (4420): No heartbeat from core client for 30 sec - exiting 16:44:01 (4420): No heartbeat from core client for 30 sec - exiting 16:44:02 (4420): No heartbeat from core client for 30 sec - exiting 16:44:03 (4420): No heartbeat from core client for 30 sec - exiting 16:44:04 (4420): No heartbeat from core client for 30 sec - exiting 16:44:05 (4420): No heartbeat from core client for 30 sec - exiting 16:44:06 (4420): No heartbeat from core client for 30 sec - exiting 16:44:07 (4420): No heartbeat from core client for 30 sec - exiting 16:44:08 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:38:11 (2040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:38:46 (2776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:42:21 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3052, iMonCtr=1 Model crash detected, will try to restart... 05:23:11 (5704): No heartbeat from core client for 30 sec - exiting 05:23:12 (5704): No heartbeat from core client for 30 sec - exiting 05:23:43 (5704): No heartbeat from core client for 30 sec - exiting 05:23:44 (5704): No heartbeat from core client for 30 sec - exiting 05:23:45 (5704): No heartbeat from core client for 30 sec - exiting 05:23:46 (5704): No heartbeat from core client for 30 sec - exiting 05:23:47 (5704): No heartbeat from core client for 30 sec - exiting 05:23:48 (5704): No heartbeat from core client for 30 sec - exiting 05:23:49 (5704): No heartbeat from core client for 30 sec - exiting 05:23:50 (5704): No heartbeat from core client for 30 sec - exiting 05:23:51 (5704): No heartbeat from core client for 30 sec - exiting 05:23:52 (5704): No heartbeat from core client for 30 sec - exiting 05:23:53 (5704): No heartbeat from core client for 30 sec - exiting 05:23:54 (5704): No heartbeat from core client for 30 sec - exiting 05:23:55 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:31 (7832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:10 (5180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:31:57 (4020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:33:41 (4640): No heartbeat from core client for 30 sec - exiting 12:33:42 (4640): No heartbeat from core client for 30 sec - exiting 12:33:43 (4640): No heartbeat from core client for 30 sec - exiting 12:33:44 (4640): No heartbeat from core client for 30 sec - exiting 12:33:45 (4640): No heartbeat from core client for 30 sec - exiting 12:33:46 (4640): No heartbeat from core client for 30 sec - exiting 12:33:47 (4640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Sep 2013 21:00:06	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	1,036,800	1,971,241	1.9013
28 Sep 2013 06:23:40	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	1,010,880	1,921,246	1.9006
27 Sep 2013 10:38:25	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	984,960	1,870,948	1.8995
26 Sep 2013 10:22:44	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	959,040	1,822,292	1.9001
24 Sep 2013 10:04:48	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	933,120	1,773,501	1.9006
23 Sep 2013 09:36:37	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	907,200	1,725,503	1.9020
21 Sep 2013 18:50:41	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	881,280	1,675,027	1.9007
21 Sep 2013 03:18:52	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	855,360	1,624,750	1.8995
20 Sep 2013 08:44:16	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	829,440	1,573,700	1.8973
19 Sep 2013 17:02:16	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	803,520	1,523,079	1.8955
18 Sep 2013 10:38:57	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	777,600	1,472,223	1.8933
16 Sep 2013 10:27:30	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	751,680	1,421,366	1.8909
15 Sep 2013 11:49:58	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	725,760	1,370,279	1.8881
14 Sep 2013 12:59:07	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	699,840	1,319,236	1.8851
13 Sep 2013 08:05:56	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	673,920	1,268,254	1.8819
12 Sep 2013 09:44:45	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	648,000	1,217,902	1.8795
10 Sep 2013 07:59:08	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	622,080	1,166,900	1.8758
09 Sep 2013 00:26:14	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	596,160	1,115,998	1.8720
08 Sep 2013 00:33:55	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	570,240	1,069,255	1.8751
07 Sep 2013 01:47:16	1129647	15913765	hadcm3n_4fog_1940_40_008302722_2	544,320	1,022,780	1.8790