Task 13029249

Name	hadcm3n_t13z_1940_40_007313979_2
Workunit	7511409
Created	29 Jun 2011, 9:33:24 UTC
Sent	29 Jun 2011, 9:40:51 UTC
Report deadline	28 Sep 2011, 17:08:02 UTC
Received	22 Nov 2011, 6:14:04 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1091933
Run time	24 days 4 hours 36 min 57 sec
CPU time	22 days 18 hours 24 min 53 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.80 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2672, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3292, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3868, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4068, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3964, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=616, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5580, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:28:58 (2268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3196, iMonCtr=1 Model crash detected, will try to restart... 07:08:48 (2212): No heartbeat from core client for 30 sec - exiting 07:08:50 (2212): No heartbeat from core client for 30 sec - exiting 07:08:51 (2212): No heartbeat from core client for 30 sec - exiting 07:08:52 (2212): No heartbeat from core client for 30 sec - exiting 07:08:53 (2212): No heartbeat from core client for 30 sec - exiting 07:08:54 (2212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:08:55 (2212): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:24:08 (2368): No heartbeat from core client for 30 sec - exiting 11:24:09 (2368): No heartbeat from core client for 30 sec - exiting 11:24:11 (2368): No heartbeat from core client for 30 sec - exiting 11:24:12 (2368): No heartbeat from core client for 30 sec - exiting 11:24:13 (2368): No heartbeat from core client for 30 sec - exiting 11:24:14 (2368): No heartbeat from core client for 30 sec - exiting 11:24:15 (2368): No heartbeat from core client for 30 sec - exiting 11:24:16 (2368): No heartbeat from core client for 30 sec - exiting 11:24:17 (2368): No heartbeat from core client for 30 sec - exiting 11:24:18 (2368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:03:58 (1924): No heartbeat from core client for 30 sec - exiting 07:03:59 (1924): No heartbeat from core client for 30 sec - exiting 07:04:00 (1924): No heartbeat from core client for 30 sec - exiting 07:04:01 (1924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1176, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:10:43 (2424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:42:32 (2212): No heartbeat from core client for 30 sec - exiting 07:42:33 (2212): No heartbeat from core client for 30 sec - exiting 07:42:35 (2212): No heartbeat from core client for 30 sec - exiting 07:42:36 (2212): No heartbeat from core client for 30 sec - exiting 07:42:37 (2212): No heartbeat from core client for 30 sec - exiting 07:42:38 (2212): No heartbeat from core client for 30 sec - exiting 07:42:39 (2212): No heartbeat from core client for 30 sec - exiting 07:42:40 (2212): No heartbeat from core client for 30 sec - exiting 07:42:41 (2212): No heartbeat from core client for 30 sec - exiting 07:42:42 (2212): No heartbeat from core client for 30 sec - exiting 07:42:43 (2212): No heartbeat from core client for 30 sec - exiting 07:42:45 (2212): No heartbeat from core client for 30 sec - exiting 07:42:46 (2212): No heartbeat from core client for 30 sec - exiting 07:42:47 (2212): No heartbeat from core client for 30 sec - exiting 07:42:48 (2212): No heartbeat from core client for 30 sec - exiting 07:42:49 (2212): No heartbeat from core client for 30 sec - exiting 07:42:50 (2212): No heartbeat from core client for 30 sec - exiting 07:42:51 (2212): No heartbeat from core client for 30 sec - exiting 07:42:52 (2212): No heartbeat from core client for 30 sec - exiting 07:42:53 (2212): No heartbeat from core client for 30 sec - exiting 07:42:54 (2212): No heartbeat from core client for 30 sec - exiting 07:42:56 (2212): No heartbeat from core client for 30 sec - exiting 07:42:57 (2212): No heartbeat from core client for 30 sec - exiting 07:42:58 (2212): No heartbeat from core client for 30 sec - exiting 07:42:59 (2212): No heartbeat from core client for 30 sec - exiting 07:43:00 (2212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2200, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2200, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3684, iMonCtr=1 Model crash detected, will try to restart... 07:06:43 (3708): No heartbeat from core client for 30 sec - exiting 07:06:44 (3708): No heartbeat from core client for 30 sec - exiting 07:06:45 (3708): No heartbeat from core client for 30 sec - exiting 07:06:46 (3708): No heartbeat from core client for 30 sec - exiting 07:06:47 (3708): No heartbeat from core client for 30 sec - exiting 07:06:48 (3708): No heartbeat from core client for 30 sec - exiting 07:06:49 (3708): No heartbeat from core client for 30 sec - exiting 07:06:50 (3708): No heartbeat from core client for 30 sec - exiting 07:06:52 (3708): No heartbeat from core client for 30 sec - exiting 07:06:53 (3708): No heartbeat from core client for 30 sec - exiting 07:06:54 (3708): No heartbeat from core client for 30 sec - exiting 07:06:55 (3708): No heartbeat from core client for 30 sec - exiting 07:06:56 (3708): No heartbeat from core client for 30 sec - exiting 07:06:57 (3708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3336, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 07:13:27 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:11:20 (2232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:05:05 (2116): No heartbeat from core client for 30 sec - exiting 07:05:06 (2116): No heartbeat from core client for 30 sec - exiting 07:05:07 (2116): No heartbeat from core client for 30 sec - exiting 07:05:08 (2116): No heartbeat from core client for 30 sec - exiting 07:05:09 (2116): No heartbeat from core client for 30 sec - exiting 07:05:10 (2116): No heartbeat from core client for 30 sec - exiting 07:05:11 (2116): No heartbeat from core client for 30 sec - exiting 07:05:12 (2116): No heartbeat from core client for 30 sec - exiting 07:05:13 (2116): No heartbeat from core client for 30 sec - exiting 07:05:15 (2116): No heartbeat from core client for 30 sec - exiting 07:05:16 (2116): No heartbeat from core client for 30 sec - exiting 07:05:17 (2116): No heartbeat from core client for 30 sec - exiting 07:05:18 (2116): No heartbeat from core client for 30 sec - exiting 07:05:19 (2116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:05:20 (2116): No heartbeat from core client for 30 sec - exiting 07:05:21 (2116): No heartbeat from core client for 30 sec - exiting 07:05:22 (2116): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4364, iMonCtr=1 Model crash detected, will try to restart... 07:01:55 (2704): No heartbeat from core client for 30 sec - exiting 07:01:56 (2704): No heartbeat from core client for 30 sec - exiting 07:01:57 (2704): No heartbeat from core client for 30 sec - exiting 07:01:58 (2704): No heartbeat from core client for 30 sec - exiting 07:01:59 (2704): No heartbeat from core client for 30 sec - exiting 07:02:01 (2704): No heartbeat from core client for 30 sec - exiting 07:02:02 (2704): No heartbeat from core client for 30 sec - exiting 07:02:03 (2704): No heartbeat from core client for 30 sec - exiting 07:02:04 (2704): No heartbeat from core client for 30 sec - exiting 07:02:05 (2704): No heartbeat from core client for 30 sec - exiting 07:02:06 (2704): No heartbeat from core client for 30 sec - exiting 07:02:07 (2704): No heartbeat from core client for 30 sec - exiting 07:02:08 (2704): No heartbeat from core client for 30 sec - exiting 07:02:09 (2704): No heartbeat from core client for 30 sec - exiting 07:02:10 (2704): No heartbeat from core client for 30 sec - exiting 07:02:11 (2704): No heartbeat from core client for 30 sec - exiting 07:02:13 (2704): No heartbeat from core client for 30 sec - exiting 07:02:14 (2704): No heartbeat from core client for 30 sec - exiting 07:02:15 (2704): No heartbeat from core client for 30 sec - exiting 07:02:16 (2704): No heartbeat from core client for 30 sec - exiting 07:02:17 (2704): No heartbeat from core client for 30 sec - exiting 07:02:18 (2704): No heartbeat from core client for 30 sec - exiting 07:02:19 (2704): No heartbeat from core client for 30 sec - exiting 07:02:20 (2704): No heartbeat from core client for 30 sec - exiting 07:02:21 (2704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1888, iMonCtr=1 Model crash detected, will try to restart... 07:06:05 (3848): No heartbeat from core client for 30 sec - exiting 07:06:06 (3848): No heartbeat from core client for 30 sec - exiting 07:06:07 (3848): No heartbeat from core client for 30 sec - exiting 07:06:08 (3848): No heartbeat from core client for 30 sec - exiting 07:06:09 (3848): No heartbeat from core client for 30 sec - exiting 07:06:10 (3848): No heartbeat from core client for 30 sec - exiting 07:06:11 (3848): No heartbeat from core client for 30 sec - exiting 07:06:12 (3848): No heartbeat from core client for 30 sec - exiting 07:06:13 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:06:15 (3848): No heartbeat from core client for 30 sec - exiting 07:06:16 (3848): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2852, iMonCtr=1 Model crash detected, will try to restart... 07:06:57 (1136): No heartbeat from core client for 30 sec - exiting 07:06:58 (1136): No heartbeat from core client for 30 sec - exiting 07:06:59 (1136): No heartbeat from core client for 30 sec - exiting 07:07:00 (1136): No heartbeat from core client for 30 sec - exiting 07:07:01 (1136): No heartbeat from core client for 30 sec - exiting 07:07:02 (1136): No heartbeat from core client for 30 sec - exiting 07:07:03 (1136): No heartbeat from core client for 30 sec - exiting 07:07:04 (1136): No heartbeat from core client for 30 sec - exiting 07:07:05 (1136): No heartbeat from core client for 30 sec - exiting 07:07:07 (1136): No heartbeat from core client for 30 sec - exiting 07:07:08 (1136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:07:09 (1136): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3332, iMonCtr=1 Model crash detected, will try to restart... 07:00:29 (3040): No heartbeat from core client for 30 sec - exiting 07:00:31 (3040): No heartbeat from core client for 30 sec - exiting 07:00:32 (3040): No heartbeat from core client for 30 sec - exiting 07:00:33 (3040): No heartbeat from core client for 30 sec - exiting 07:00:34 (3040): No heartbeat from core client for 30 sec - exiting 07:00:35 (3040): No heartbeat from core client for 30 sec - exiting 07:00:37 (3040): No heartbeat from core client for 30 sec - exiting 07:00:38 (3040): No heartbeat from core client for 30 sec - exiting 07:00:39 (3040): No heartbeat from core client for 30 sec - exiting 07:00:40 (3040): No heartbeat from core client for 30 sec - exiting 07:00:41 (3040): No heartbeat from core client for 30 sec - exiting 07:00:42 (3040): No heartbeat from core client for 30 sec - exiting 07:00:43 (3040): No heartbeat from core client for 30 sec - exiting 07:00:44 (3040): No heartbeat from core client for 30 sec - exiting 07:00:45 (3040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:18:42 (4100): No heartbeat from core client for 30 sec - exiting 14:18:44 (4100): No heartbeat from core client for 30 sec - exiting 14:18:45 (4100): No heartbeat from core client for 30 sec - exiting 14:18:46 (4100): No heartbeat from core client for 30 sec - exiting 14:18:47 (4100): No heartbeat from core client for 30 sec - exiting 14:18:48 (4100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:03:36 (1304): No heartbeat from core client for 30 sec - exiting 07:03:37 (1304): No heartbeat from core client for 30 sec - exiting 07:03:39 (1304): No heartbeat from core client for 30 sec - exiting 07:03:40 (1304): No heartbeat from core client for 30 sec - exiting 07:03:41 (1304): No heartbeat from core client for 30 sec - exiting 07:03:42 (1304): No heartbeat from core client for 30 sec - exiting 07:03:43 (1304): No heartbeat from core client for 30 sec - exiting 07:03:44 (1304): No heartbeat from core client for 30 sec - exiting 07:03:45 (1304): No heartbeat from core client for 30 sec - exiting 07:03:46 (1304): No heartbeat from core client for 30 sec - exiting 07:03:47 (1304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1512, iMonCtr=1 Model crash detected, will try to restart... 07:07:18 (3004): No heartbeat from core client for 30 sec - exiting 07:07:19 (3004): No heartbeat from core client for 30 sec - exiting 07:07:20 (3004): No heartbeat from core client for 30 sec - exiting 07:07:21 (3004): No heartbeat from core client for 30 sec - exiting 07:07:23 (3004): No heartbeat from core client for 30 sec - exiting 07:07:24 (3004): No heartbeat from core client for 30 sec - exiting 07:07:25 (3004): No heartbeat from core client for 30 sec - exiting 07:07:26 (3004): No heartbeat from core client for 30 sec - exiting 07:07:27 (3004): No heartbeat from core client for 30 sec - exiting 07:07:28 (3004): No heartbeat from core client for 30 sec - exiting 07:07:29 (3004): No heartbeat from core client for 30 sec - exiting 07:07:30 (3004): No heartbeat from core client for 30 sec - exiting 07:07:31 (3004): No heartbeat from core client for 30 sec - exiting 07:07:32 (3004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3188, iMonCtr=1 Model crash detected, will try to restart... 07:06:52 (2456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:20:20 (3404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:31:41 (1060): No heartbeat from core client for 30 sec - exiting 07:31:42 (1060): No heartbeat from core client for 30 sec - exiting 07:31:43 (1060): No heartbeat from core client for 30 sec - exiting 07:31:44 (1060): No heartbeat from core client for 30 sec - exiting 07:31:45 (1060): No heartbeat from core client for 30 sec - exiting 07:31:46 (1060): No heartbeat from core client for 30 sec - exiting 07:31:47 (1060): No heartbeat from core client for 30 sec - exiting 07:31:48 (1060): No heartbeat from core client for 30 sec - exiting 07:31:50 (1060): No heartbeat from core client for 30 sec - exiting 07:31:51 (1060): No heartbeat from core client for 30 sec - exiting 07:31:52 (1060): No heartbeat from core client for 30 sec - exiting 07:31:53 (1060): No heartbeat from core client for 30 sec - exiting 07:31:54 (1060): No heartbeat from core client for 30 sec - exiting 07:31:55 (1060): No heartbeat from core client for 30 sec - exiting 07:31:56 (1060): No heartbeat from core client for 30 sec - exiting 07:31:57 (1060): No heartbeat from core client for 30 sec - exiting 07:31:58 (1060): No heartbeat from core client for 30 sec - exiting 07:31:59 (1060): No heartbeat from core client for 30 sec - exiting 07:32:00 (1060): No heartbeat from core client for 30 sec - exiting 07:32:02 (1060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:32:03 (1060): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3308, iMonCtr=1 Model crash detected, will try to restart... 07:04:16 (2124): No heartbeat from core client for 30 sec - exiting 07:04:17 (2124): No heartbeat from core client for 30 sec - exiting 07:04:18 (2124): No heartbeat from core client for 30 sec - exiting 07:04:19 (2124): No heartbeat from core client for 30 sec - exiting 07:04:20 (2124): No heartbeat from core client for 30 sec - exiting 07:04:21 (2124): No heartbeat from core client for 30 sec - exiting 07:04:22 (2124): No heartbeat from core client for 30 sec - exiting 07:04:23 (2124): No heartbeat from core client for 30 sec - exiting 07:04:24 (2124): No heartbeat from core client for 30 sec - exiting 07:04:26 (2124): No heartbeat from core client for 30 sec - exiting 07:04:27 (2124): No heartbeat from core client for 30 sec - exiting 07:04:28 (2124): No heartbeat from core client for 30 sec - exiting 07:04:29 (2124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:04:30 (2124): No heartbeat from core client for 30 sec - exiting C07:03:39 (1320): No heartbeat from core client for 30 sec - exiting 07:03:41 (1320): No heartbeat from core client for 30 sec - exiting 07:03:42 (1320): No heartbeat from core client for 30 sec - exiting 07:03:43 (1320): No heartbeat from core client for 30 sec - exiting 07:03:44 (1320): No heartbeat from core client for 30 sec - exiting 07:03:45 (1320): No heartbeat from core client for 30 sec - exiting 07:03:46 (1320): No heartbeat from core client for 30 sec - exiting 07:03:47 (1320): No heartbeat from core client for 30 sec - exiting 07:03:48 (1320): No heartbeat from core client for 30 sec - exiting 07:03:49 (1320): No heartbeat from core client for 30 sec - exiting 07:03:50 (1320): No heartbeat from core client for 30 sec - exiting 07:03:52 (1320): No heartbeat from core client for 30 sec - exiting 07:03:53 (1320): No heartbeat from core client for 30 sec - exiting 07:03:54 (1320): No heartbeat from core client for 30 sec - exiting 07:03:55 (1320): No heartbeat from core client for 30 sec - exiting 07:03:56 (1320): No heartbeat from core client for 30 sec - exiting 07:03:57 (1320): No heartbeat from core client for 30 sec - exiting 07:03:58 (1320): No heartbeat from core client for 30 sec - exiting 07:03:59 (1320): No heartbeat from core client for 30 sec - exiting 07:04:00 (1320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1996, iMonCtr=1 Model crash detected, will try to restart... 07:03:28 (2436): No heartbeat from core client for 30 sec - exiting 07:03:29 (2436): No heartbeat from core client for 30 sec - exiting 07:03:30 (2436): No heartbeat from core client for 30 sec - exiting 07:03:31 (2436): No heartbeat from core client for 30 sec - exiting 07:03:32 (2436): No heartbeat from core client for 30 sec - exiting 07:03:33 (2436): No heartbeat from core client for 30 sec - exiting 07:03:34 (2436): No heartbeat from core client for 30 sec - exiting 07:03:35 (2436): No heartbeat from core client for 30 sec - exiting 07:03:37 (2436): No heartbeat from core client for 30 sec - exiting 07:03:38 (2436): No heartbeat from core client for 30 sec - exiting 07:03:39 (2436): No heartbeat from core client for 30 sec - exiting 07:03:40 (2436): No heartbeat from core client for 30 sec - exiting 07:03:41 (2436): No heartbeat from core client for 30 sec - exiting 07:03:42 (2436): No heartbeat from core client for 30 sec - exiting 07:03:43 (2436): No heartbeat from core client for 30 sec - exiting 07:03:44 (2436): No heartbeat from core client for 30 sec - exiting 07:03:45 (2436): No heartbeat from core client for 30 sec - exiting 07:03:46 (2436): No heartbeat from core client for 30 sec - exiting 07:03:48 (2436): No heartbeat from core client for 30 sec - exiting 07:03:49 (2436): No heartbeat from core client for 30 sec - exiting 07:03:50 (2436): No heartbeat from core client for 30 sec - exiting 07:03:51 (2436): No heartbeat from core client for 30 sec - exiting 07:03:52 (2436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3836, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1996, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1996, iMonCtr=1 Model crash detected, will try to restart... 07:06:55 (3804): No heartbeat from core client for 30 sec - exiting 07:06:56 (3804): No heartbeat from core client for 30 sec - exiting 07:06:57 (3804): No heartbeat from core client for 30 sec - exiting 07:06:58 (3804): No heartbeat from core client for 30 sec - exiting 07:06:59 (3804): No heartbeat from core client for 30 sec - exiting 07:07:00 (3804): No heartbeat from core client for 30 sec - exiting 07:07:01 (3804): No heartbeat from core client for 30 sec - exiting 07:07:03 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C12:38:05 (2164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2580, iMonCtr=1 Model crash detected, will try to restart... 07:01:46 (4408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5456, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2256, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2256, iMonCtr=1 Model crash detected, will try to restart... 08:06:25 (3648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2952, iMonCtr=1 Model crash detected, will try to restart... 07:04:20 (4020): No heartbeat from core client for 30 sec - exiting 07:04:21 (4020): No heartbeat from core client for 30 sec - exiting 07:04:22 (4020): No heartbeat from core client for 30 sec - exiting 07:04:23 (4020): No heartbeat from core client for 30 sec - exiting 07:04:24 (4020): No heartbeat from core client for 30 sec - exiting 07:04:25 (4020): No heartbeat from core client for 30 sec - exiting 07:04:26 (4020): No heartbeat from core client for 30 sec - exiting 07:04:27 (4020): No heartbeat from core client for 30 sec - exiting 07:04:28 (4020): No heartbeat from core client for 30 sec - exiting 07:04:29 (4020): No heartbeat from core client for 30 sec - exiting 07:04:30 (4020): No heartbeat from core client for 30 sec - exiting 07:04:31 (4020): No heartbeat from core client for 30 sec - exiting 07:04:32 (4020): No heartbeat from core client for 30 sec - exiting 07:04:33 (4020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4216, iMonCtr=1 Model crash detected, will try to restart... 07:05:46 (3824): No heartbeat from core client for 30 sec - exiting 07:05:48 (3824): No heartbeat from core client for 30 sec - exiting 07:05:49 (3824): No heartbeat from core client for 30 sec - exiting 07:05:50 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:32 (4044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:06 (312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C07:11:33 (1100): No heartbeat from core client for 30 sec - exiting 07:11:35 (1100): No heartbeat from core client for 30 sec - exiting 07:11:36 (1100): No heartbeat from core client for 30 sec - exiting 07:11:37 (1100): No heartbeat from core client for 30 sec - exiting 07:11:38 (1100): No heartbeat from core client for 30 sec - exiting 07:11:39 (1100): No heartbeat from core client for 30 sec - exiting 07:11:40 (1100): No heartbeat from core client for 30 sec - exiting 07:11:41 (1100): No heartbeat from core client for 30 sec - exiting 07:11:42 (1100): No heartbeat from core client for 30 sec - exiting 07:11:43 (1100): No heartbeat from core client for 30 sec - exiting 07:11:45 (1100): No heartbeat from core client for 30 sec - exiting 07:11:46 (1100): No heartbeat from core client for 30 sec - exiting 07:11:47 (1100): No heartbeat from core client for 30 sec - exiting 07:11:48 (1100): No heartbeat from core client for 30 sec - exiting 07:11:49 (1100): No heartbeat from core client for 30 sec - exiting 07:11:50 (1100): No heartbeat from core client for 30 sec - exiting 07:11:51 (1100): No heartbeat from core client for 30 sec - exiting 07:11:52 (1100): No heartbeat from core client for 30 sec - exiting 07:11:53 (1100): No heartbeat from core client for 30 sec - exiting 07:11:54 (1100): No heartbeat from core client for 30 sec - exiting 07:11:55 (1100): No heartbeat from core client for 30 sec - exiting 07:11:57 (1100): No heartbeat from core client for 30 sec - exiting 07:11:58 (1100): No heartbeat from core client for 30 sec - exiting 07:11:59 (1100): No heartbeat from core client for 30 sec - exiting 07:12:00 (1100): No heartbeat from core client for 30 sec - exiting 07:12:01 (1100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:13:18 (2520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4900, iMonCtr=1 Model crash detected, will try to restart... 07:14:53 (2252): No heartbeat from core client for 30 sec - exiting 07:14:55 (2252): No heartbeat from core client for 30 sec - exiting 07:14:56 (2252): No heartbeat from core client for 30 sec - exiting 07:14:58 (2252): No heartbeat from core client for 30 sec - exiting 07:14:59 (2252): No heartbeat from core client for 30 sec - exiting 07:15:00 (2252): No heartbeat from core client for 30 sec - exiting 07:15:01 (2252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:13:09 (2860): No heartbeat from core client for 30 sec - exiting 07:13:10 (2860): No heartbeat from core client for 30 sec - exiting 07:13:11 (2860): No heartbeat from core client for 30 sec - exiting 07:13:12 (2860): No heartbeat from core client for 30 sec - exiting 07:13:14 (2860): No heartbeat from core client for 30 sec - exiting 07:13:15 (2860): No heartbeat from core client for 30 sec - exiting 07:13:16 (2860): No heartbeat from core client for 30 sec - exiting 07:13:17 (2860): No heartbeat from core client for 30 sec - exiting 07:13:18 (2860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:00:39 (3648): No heartbeat from core client for 30 sec - exiting 13:00:40 (3648): No heartbeat from core client for 30 sec - exiting 13:00:41 (3648): No heartbeat from core client for 30 sec - exiting 13:00:42 (3648): No heartbeat from core client for 30 sec - exiting 13:00:43 (3648): No heartbeat from core client for 30 sec - exiting 13:00:45 (3648): No heartbeat from core client for 30 sec - exiting 13:00:46 (3648): No heartbeat from core client for 30 sec - exiting 13:00:47 (3648): No heartbeat from core client for 30 sec - exiting 13:00:48 (3648): No heartbeat from core client for 30 sec - exiting 13:00:49 (3648): No heartbeat from core client for 30 sec - exiting 13:00:50 (3648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Nov 2011 13:44:46	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	1,036,800	1,966,434	1.8966
16 Nov 2011 07:59:34	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	1,010,880	1,918,906	1.8983
16 Nov 2011 06:18:38	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	984,960	1,871,961	1.9005
09 Nov 2011 11:37:32	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	959,040	1,823,527	1.9014
07 Nov 2011 13:21:35	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	933,120	1,775,551	1.9028
03 Nov 2011 07:41:43	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	907,200	1,727,564	1.9043
01 Nov 2011 09:47:51	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	881,280	1,679,550	1.9058
31 Oct 2011 17:01:58	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	855,360	1,630,159	1.9058
31 Oct 2011 15:38:03	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	829,440	1,582,646	1.9081
31 Oct 2011 15:38:03	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	803,520	1,534,346	1.9095
18 Oct 2011 06:38:19	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	777,600	1,485,413	1.9103
13 Oct 2011 07:22:59	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	751,680	1,436,314	1.9108
11 Oct 2011 09:48:24	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	725,760	1,387,533	1.9118
06 Oct 2011 12:07:24	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	699,840	1,339,287	1.9137
05 Oct 2011 05:26:39	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	673,920	1,290,646	1.9151
03 Oct 2011 08:12:59	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	648,000	1,242,354	1.9172
01 Oct 2011 20:40:15	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	622,080	1,194,191	1.9197
29 Sep 2011 09:38:46	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	596,160	1,146,242	1.9227
26 Sep 2011 09:55:29	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	570,240	1,097,499	1.9246
21 Sep 2011 08:26:14	1091933	13029249	hadcm3n_t13z_1940_40_007313979_2	544,320	1,048,781	1.9268