Task 15500847

Name	hadcm3n_3el1_1940_40_008268538_0
Workunit	8423662
Created	23 Dec 2012, 12:58:25 UTC
Sent	23 Dec 2012, 17:34:20 UTC
Report deadline	25 Mar 2013, 1:01:31 UTC
Received	31 May 2013, 20:35:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1214423
Run time	93 days 9 hours 28 min 30 sec
CPU time	43 days 22 hours 41 min 10 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:49:34 (8176): No heartbeat from core client for 30 sec - exiting 21:49:35 (8176): No heartbeat from core client for 30 sec - exiting 21:49:36 (8176): No heartbeat from core client for 30 sec - exiting 21:49:37 (8176): No heartbeat from core client for 30 sec - exiting 21:49:38 (8176): No heartbeat from core client for 30 sec - exiting 21:49:39 (8176): No heartbeat from core client for 30 sec - exiting 21:49:40 (8176): No heartbeat from core client for 30 sec - exiting 21:49:41 (8176): No heartbeat from core client for 30 sec - exiting 21:49:42 (8176): No heartbeat from core client for 30 sec - exiting 21:49:43 (8176): No heartbeat from core client for 30 sec - exiting 21:49:44 (8176): No heartbeat from core client for 30 sec - exiting 21:49:45 (8176): No heartbeat from core client for 30 sec - exiting 21:49:46 (8176): No heartbeat from core client for 30 sec - exiting 21:49:47 (8176): No heartbeat from core client for 30 sec - exiting 21:49:48 (8176): No heartbeat from core client for 30 sec - exiting 21:49:49 (8176): No heartbeat from core client for 30 sec - exiting 21:49:50 (8176): No heartbeat from core client for 30 sec - exiting 21:49:51 (8176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4148, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:34:24 (1764): No heartbeat from core client for 30 sec - exiting 10:34:25 (1764): No heartbeat from core client for 30 sec - exiting 10:34:26 (1764): No heartbeat from core client for 30 sec - exiting 10:34:27 (1764): No heartbeat from core client for 30 sec - exiting 10:34:28 (1764): No heartbeat from core client for 30 sec - exiting 10:34:29 (1764): No heartbeat from core client for 30 sec - exiting 10:34:30 (1764): No heartbeat from core client for 30 sec - exiting 10:34:31 (1764): No heartbeat from core client for 30 sec - exiting 10:34:32 (1764): No heartbeat from core client for 30 sec - exiting 10:34:33 (1764): No heartbeat from core client for 30 sec - exiting 10:34:34 (1764): No heartbeat from core client for 30 sec - exiting 10:34:35 (1764): No heartbeat from core client for 30 sec - exiting 10:34:36 (1764): No heartbeat from core client for 30 sec - exiting 10:34:37 (1764): No heartbeat from core client for 30 sec - exiting 10:34:38 (1764): No heartbeat from core client for 30 sec - exiting 10:34:39 (1764): No heartbeat from core client for 30 sec - exiting 10:34:40 (1764): No heartbeat from core client for 30 sec - exiting 10:34:41 (1764): No heartbeat from core client for 30 sec - exiting 10:34:42 (1764): No heartbeat from core client for 30 sec - exiting 10:34:43 (1764): No heartbeat from core client for 30 sec - exiting 10:34:44 (1764): No heartbeat from core client for 30 sec - exiting 10:34:45 (1764): No heartbeat from core client for 30 sec - exiting 10:34:46 (1764): No heartbeat from core client for 30 sec - exiting 10:34:47 (1764): No heartbeat from core client for 30 sec - exiting 10:34:48 (1764): No heartbeat from core client for 30 sec - exiting 10:34:49 (1764): No heartbeat from core client for 30 sec - exiting 10:34:50 (1764): No heartbeat from core client for 30 sec - exiting 10:34:51 (1764): No heartbeat from core client for 30 sec - exiting 10:34:52 (1764): No heartbeat from core client for 30 sec - exiting 10:34:53 (1764): No heartbeat from core client for 30 sec - exiting 10:34:54 (1764): No heartbeat from core client for 30 sec - exiting 10:34:55 (1764): No heartbeat from core client for 30 sec - exiting 10:34:56 (1764): No heartbeat from core client for 30 sec - exiting 10:34:57 (1764): No heartbeat from core client for 30 sec - exiting 10:34:58 (1764): No heartbeat from core client for 30 sec - exiting 10:34:59 (1764): No heartbeat from core client for 30 sec - exiting 10:35:00 (1764): No heartbeat from core client for 30 sec - exiting 10:35:01 (1764): No heartbeat from core client for 30 sec - exiting 10:35:02 (1764): No heartbeat from core client for 30 sec - exiting 10:35:03 (1764): No heartbeat from core client for 30 sec - exiting 10:35:04 (1764): No heartbeat from core client for 30 sec - exiting 10:35:05 (1764): No heartbeat from core client for 30 sec - exiting 10:35:06 (1764): No heartbeat from core client for 30 sec - exiting 10:35:07 (1764): No heartbeat from core client for 30 sec - exiting 10:35:08 (1764): No heartbeat from core client for 30 sec - exiting 10:35:09 (1764): No heartbeat from core client for 30 sec - exiting 10:35:10 (1764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:01:32 (6108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3964, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:04:57 (5036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:04:58 (5036): No heartbeat from core client for 30 sec - exiting 18:04:59 (5036): No heartbeat from core client for 30 sec - exiting 18:05:00 (5036): No heartbeat from core client for 30 sec - exiting 18:05:01 (5036): No heartbeat from core client for 30 sec - exiting 18:05:02 (5036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:09:43 (6844): No heartbeat from core client for 30 sec - exiting 11:09:44 (6844): No heartbeat from core client for 30 sec - exiting 11:09:45 (6844): No heartbeat from core client for 30 sec - exiting 11:09:46 (6844): No heartbeat from core client for 30 sec - exiting 11:09:47 (6844): No heartbeat from core client for 30 sec - exiting 11:09:48 (6844): No heartbeat from core client for 30 sec - exiting 11:09:49 (6844): No heartbeat from core client for 30 sec - exiting 11:09:50 (6844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6044, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6044, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:21:44 (7376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:21:45 (7376): No heartbeat from core client for 30 sec - exiting 13:21:46 (7376): No heartbeat from core client for 30 sec - exiting 13:21:47 (7376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:10:29 (1756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:10:30 (1756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4608, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:07:18 (9948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:55:07 (7480): No heartbeat from core client for 30 sec - exiting 12:55:08 (7480): No heartbeat from core client for 30 sec - exiting 12:55:09 (7480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:35 (8456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:36 (8456): No heartbeat from core client for 30 sec - exiting 12:56:37 (8456): No heartbeat from core client for 30 sec - exiting 12:56:38 (8456): No heartbeat from core client for 30 sec - exiting 12:56:39 (8456): No heartbeat from core client for 30 sec - exiting 12:56:40 (8456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:01:20 (4976): No heartbeat from core client for 30 sec - exiting 12:01:21 (4976): No heartbeat from core client for 30 sec - exiting 12:01:22 (4976): No heartbeat from core client for 30 sec - exiting 12:01:23 (4976): No heartbeat from core client for 30 sec - exiting 12:01:24 (4976): No heartbeat from core client for 30 sec - exiting 12:01:25 (4976): No heartbeat from core client for 30 sec - exiting 12:01:26 (4976): No heartbeat from core client for 30 sec - exiting 12:01:27 (4976): No heartbeat from core client for 30 sec - exiting 12:01:28 (4976): No heartbeat from core client for 30 sec - exiting 12:01:29 (4976): No heartbeat from core client for 30 sec - exiting 12:01:30 (4976): No heartbeat from core client for 30 sec - exiting 12:01:31 (4976): No heartbeat from core client for 30 sec - exiting 12:01:32 (4976): No heartbeat from core client for 30 sec - exiting 12:01:33 (4976): No heartbeat from core client for 30 sec - exiting 12:01:34 (4976): No heartbeat from core client for 30 sec - exiting 12:01:35 (4976): No heartbeat from core client for 30 sec - exiting 12:01:36 (4976): No heartbeat from core client for 30 sec - exiting 12:01:37 (4976): No heartbeat from core client for 30 sec - exiting 12:01:38 (4976): No heartbeat from core client for 30 sec - exiting 12:01:39 (4976): No heartbeat from core client for 30 sec - exiting 12:01:40 (4976): No heartbeat from core client for 30 sec - exiting 12:01:41 (4976): No heartbeat from core client for 30 sec - exiting 12:01:42 (4976): No heartbeat from core client for 30 sec - exiting 12:01:43 (4976): No heartbeat from core client for 30 sec - exiting 12:01:44 (4976): No heartbeat from core client for 30 sec - exiting 12:01:45 (4976): No heartbeat from core client for 30 sec - exiting 12:01:46 (4976): No heartbeat from core client for 30 sec - exiting 12:01:47 (4976): No heartbeat from core client for 30 sec - exiting 12:01:48 (4976): No heartbeat from core client for 30 sec - exiting 12:01:49 (4976): No heartbeat from core client for 30 sec - exiting 12:01:50 (4976): No heartbeat from core client for 30 sec - exiting 12:01:51 (4976): No heartbeat from core client for 30 sec - exiting 12:01:52 (4976): No heartbeat from core client for 30 sec - exiting 12:01:53 (4976): No heartbeat from core client for 30 sec - exiting 12:01:54 (4976): No heartbeat from core client for 30 sec - exiting 12:01:55 (4976): No heartbeat from core client for 30 sec - exiting 12:01:56 (4976): No heartbeat from core client for 30 sec - exiting 12:01:57 (4976): No heartbeat from core client for 30 sec - exiting 12:01:58 (4976): No heartbeat from core client for 30 sec - exiting 12:01:59 (4976): No heartbeat from core client for 30 sec - exiting 12:02:00 (4976): No heartbeat from core client for 30 sec - exiting 12:02:01 (4976): No heartbeat from core client for 30 sec - exiting 12:02:02 (4976): No heartbeat from core client for 30 sec - exiting 12:02:03 (4976): No heartbeat from core client for 30 sec - exiting 12:02:04 (4976): No heartbeat from core client for 30 sec - exiting 12:02:05 (4976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:02:06 (4976): No heartbeat from core client for 30 sec - exiting 12:02:07 (4976): No heartbeat from core client for 30 sec - exiting 12:02:08 (4976): No heartbeat from core client for 30 sec - exiting 12:02:09 (4976): No heartbeat from core client for 30 sec - exiting 12:03:03 (8780): No heartbeat from core client for 30 sec - exiting 12:03:04 (8780): No heartbeat from core client for 30 sec - exiting 12:03:05 (8780): No heartbeat from core client for 30 sec - exiting 12:03:06 (8780): No heartbeat from core client for 30 sec - exiting 12:03:07 (8780): No heartbeat from core client for 30 sec - exiting 12:03:08 (8780): No heartbeat from core client for 30 sec - exiting 12:03:09 (8780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:56:27 (6956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:56:28 (6956): No heartbeat from core client for 30 sec - exiting 18:56:29 (6956): No heartbeat from core client for 30 sec - exiting 18:56:30 (6956): No heartbeat from core client for 30 sec - exiting 18:57:34 (3452): No heartbeat from core client for 30 sec - exiting 18:57:35 (3452): No heartbeat from core client for 30 sec - exiting 18:57:36 (3452): No heartbeat from core client for 30 sec - exiting 18:57:37 (3452): No heartbeat from core client for 30 sec - exiting 18:57:38 (3452): No heartbeat from core client for 30 sec - exiting 18:57:39 (3452): No heartbeat from core client for 30 sec - exiting 18:57:40 (3452): No heartbeat from core client for 30 sec - exiting 18:57:41 (3452): No heartbeat from core client for 30 sec - exiting 18:57:42 (3452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3332, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:34:47 (6300): No heartbeat from core client for 30 sec - exiting 12:34:48 (6300): No heartbeat from core client for 30 sec - exiting 12:34:49 (6300): No heartbeat from core client for 30 sec - exiting 12:34:50 (6300): No heartbeat from core client for 30 sec - exiting 12:34:51 (6300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5708, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:14:07 (5040): No heartbeat from core client for 30 sec - exiting 02:14:08 (5040): No heartbeat from core client for 30 sec - exiting 02:14:09 (5040): No heartbeat from core client for 30 sec - exiting 02:14:10 (5040): No heartbeat from core client for 30 sec - exiting 02:14:11 (5040): No heartbeat from core client for 30 sec - exiting 02:14:12 (5040): No heartbeat from core client for 30 sec - exiting 02:14:13 (5040): No heartbeat from core client for 30 sec - exiting 02:14:14 (5040): No heartbeat from core client for 30 sec - exiting 02:14:15 (5040): No heartbeat from core client for 30 sec - exiting 02:14:16 (5040): No heartbeat from core client for 30 sec - exiting 02:14:17 (5040): No heartbeat from core client for 30 sec - exiting 02:14:18 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:14:48 (5504): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:52:08 (5868): No heartbeat from core client for 30 sec - exiting 10:52:09 (5868): No heartbeat from core client for 30 sec - exiting 10:52:10 (5868): No heartbeat from core client for 30 sec - exiting 10:52:11 (5868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:52:38 (7180): Can't acquire lockfile (32) - waiting 35s Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7180, iMonCtr=1 Model crash detected, will try to restart... 03:24:51 (5748): No heartbeat from core client for 30 sec - exiting 03:24:52 (5748): No heartbeat from core client for 30 sec - exiting 03:24:53 (5748): No heartbeat from core client for 30 sec - exiting 03:24:54 (5748): No heartbeat from core client for 30 sec - exiting 03:24:55 (5748): No heartbeat from core client for 30 sec - exiting 03:24:56 (5748): No heartbeat from core client for 30 sec - exiting 03:24:57 (5748): No heartbeat from core client for 30 sec - exiting 03:24:58 (5748): No heartbeat from core client for 30 sec - exiting 03:24:59 (5748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:26:01 (4524): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:25:33 (4580): No heartbeat from core client for 30 sec - exiting 13:25:34 (4580): No heartbeat from core client for 30 sec - exiting 13:25:35 (4580): No heartbeat from core client for 30 sec - exiting 13:25:36 (4580): No heartbeat from core client for 30 sec - exiting 13:25:37 (4580): No heartbeat from core client for 30 sec - exiting 13:25:38 (4580): No heartbeat from core client for 30 sec - exiting 13:25:39 (4580): No heartbeat from core client for 30 sec - exiting 13:25:40 (4580): No heartbeat from core client for 30 sec - exiting 13:25:41 (4580): No heartbeat from core client for 30 sec - exiting 13:25:42 (4580): No heartbeat from core client for 30 sec - exiting 13:25:43 (4580): No heartbeat from core client for 30 sec - exiting 13:25:44 (4580): No heartbeat from core client for 30 sec - exiting 13:25:45 (4580): No heartbeat from core client for 30 sec - exiting 13:25:46 (4580): No heartbeat from core client for 30 sec - exiting 13:25:47 (4580): No heartbeat from core client for 30 sec - exiting 13:25:48 (4580): No heartbeat from core client for 30 sec - exiting 13:25:49 (4580): No heartbeat from core client for 30 sec - exiting 13:25:50 (4580): No heartbeat from core client for 30 sec - exiting 13:25:51 (4580): No heartbeat from core client for 30 sec - exiting 13:25:52 (4580): No heartbeat from core client for 30 sec - exiting 13:25:53 (4580): No heartbeat from core client for 30 sec - exiting 13:25:54 (4580): No heartbeat from core client for 30 sec - exiting 13:25:55 (4580): No heartbeat from core client for 30 sec - exiting 13:25:56 (4580): No heartbeat from core client for 30 sec - exiting 13:25:57 (4580): No heartbeat from core client for 30 sec - exiting 13:25:58 (4580): No heartbeat from core client for 30 sec - exiting 13:25:59 (4580): No heartbeat from core client for 30 sec - exiting 13:26:00 (4580): No heartbeat from core client for 30 sec - exiting 13:26:01 (4580): No heartbeat from core client for 30 sec - exiting 13:26:02 (4580): No heartbeat from core client for 30 sec - exiting 13:26:03 (4580): No heartbeat from core client for 30 sec - exiting 13:26:04 (4580): No heartbeat from core client for 30 sec - exiting 13:26:05 (4580): No heartbeat from core client for 30 sec - exiting 13:26:06 (4580): No heartbeat from core client for 30 sec - exiting 13:26:07 (4580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4344, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4936, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4936, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6008, iMonCtr=1 Model crash detected, will try to restart... 04:31:27 (1648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:28 (1648): No heartbeat from core client for 30 sec - exiting 04:31:29 (1648): No heartbeat from core client for 30 sec - exiting 04:31:30 (1648): No heartbeat from core client for 30 sec - exiting 04:31:31 (1648): No heartbeat from core client for 30 sec - exiting 04:31:32 (1648): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:58:48 (9556): No heartbeat from core client for 30 sec - exiting 18:58:49 (9556): No heartbeat from core client for 30 sec - exiting 18:58:50 (9556): No heartbeat from core client for 30 sec - exiting 19:00:24 (9556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:00:25 (9556): No heartbeat from core client for 30 sec - exiting 19:00:26 (9556): No heartbeat from core client for 30 sec - exiting 19:00:27 (9556): No heartbeat from core client for 30 sec - exiting 19:00:28 (9556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4772, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2572, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 May 2013 19:39:30	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	1,036,800	3,796,852	3.6621
30 May 2013 17:18:45	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	1,010,880	3,713,882	3.6739
29 May 2013 08:17:48	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	984,960	3,626,455	3.6818
27 May 2013 11:44:58	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	959,040	3,539,107	3.6903
24 May 2013 22:08:08	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	933,120	3,453,282	3.7008
23 May 2013 21:02:57	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	907,200	3,370,785	3.7156
22 May 2013 10:08:51	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	881,280	3,286,730	3.7295
19 May 2013 14:44:12	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	855,360	3,199,369	3.7404
16 May 2013 06:41:34	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	829,440	3,106,299	3.7451
14 May 2013 17:17:32	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	803,520	3,022,585	3.7617
13 May 2013 16:58:41	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	777,600	2,939,904	3.7807
12 May 2013 10:41:41	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	751,680	2,856,895	3.8007
28 Apr 2013 02:44:14	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	725,760	2,747,158	3.7852
19 Apr 2013 19:17:15	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	699,840	2,646,947	3.7822
15 Apr 2013 23:40:10	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	673,920	2,553,669	3.7893
13 Apr 2013 03:06:35	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	648,000	2,448,502	3.7786
10 Apr 2013 02:25:50	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	622,080	2,358,076	3.7906
07 Apr 2013 21:02:08	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	596,160	2,264,714	3.7988
05 Apr 2013 15:06:07	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	570,240	2,169,733	3.8049
31 Mar 2013 23:41:04	1214423	15500847	hadcm3n_3el1_1940_40_008268538_0	544,320	2,071,656	3.8060