Task 15542039

Name	hadcm3n_zb6w_1960_40_008282873_0
Workunit	8434008
Created	14 Jan 2013, 4:10:37 UTC
Sent	14 Jan 2013, 4:10:49 UTC
Report deadline	15 Apr 2013, 11:38:00 UTC
Received	17 Jan 2013, 23:43:04 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1260264
Run time	1 days 15 hours 0 min 52 sec
CPU time	16 hours 26 min 1 sec
Validate state	Invalid
Credit	311.04
Device peak FLOPS	2.55 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8560, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3692, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4152, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 07:35:03 (8084): No heartbeat from core client for 30 sec - exiting 07:35:04 (8084): No heartbeat from core client for 30 sec - exiting 07:35:05 (8084): No heartbeat from core client for 30 sec - exiting 07:35:06 (8084): No heartbeat from core client for 30 sec - exiting 07:35:07 (8084): No heartbeat from core client for 30 sec - exiting 07:35:08 (8084): No heartbeat from core client for 30 sec - exiting 07:35:09 (8084): No heartbeat from core client for 30 sec - exiting 07:35:10 (8084): No heartbeat from core client for 30 sec - exiting 07:35:11 (8084): No heartbeat from core client for 30 sec - exiting 07:35:12 (8084): No heartbeat from core client for 30 sec - exiting 07:35:13 (8084): No heartbeat from core client for 30 sec - exiting 07:35:14 (8084): No heartbeat from core client for 30 sec - exiting 07:35:15 (8084): No heartbeat from core client for 30 sec - exiting 07:35:16 (8084): No heartbeat from core client for 30 sec - exiting 07:35:17 (8084): No heartbeat from core client for 30 sec - exiting 07:35:18 (8084): No heartbeat from core client for 30 sec - exiting 07:35:19 (8084): No heartbeat from core client for 30 sec - exiting 07:35:20 (8084): No heartbeat from core client for 30 sec - exiting 07:35:21 (8084): No heartbeat from core client for 30 sec - exiting 07:35:22 (8084): No heartbeat from core client for 30 sec - exiting 07:35:23 (8084): No heartbeat from core client for 30 sec - exiting 07:35:24 (8084): No heartbeat from core client for 30 sec - exiting 07:35:25 (8084): No heartbeat from core client for 30 sec - exiting 07:35:26 (8084): No heartbeat from core client for 30 sec - exiting 07:35:27 (8084): No heartbeat from core client for 30 sec - exiting 07:35:28 (8084): No heartbeat from core client for 30 sec - exiting 07:35:29 (8084): No heartbeat from core client for 30 sec - exiting 07:35:30 (8084): No heartbeat from core client for 30 sec - exiting 07:35:31 (8084): No heartbeat from core client for 30 sec - exiting 07:35:32 (8084): No heartbeat from core client for 30 sec - exiting 07:35:33 (8084): No heartbeat from core client for 30 sec - exiting 07:35:34 (8084): No heartbeat from core client for 30 sec - exiting 07:35:35 (8084): No heartbeat from core client for 30 sec - exiting 07:35:36 (8084): No heartbeat from core client for 30 sec - exiting 07:35:37 (8084): No heartbeat from core client for 30 sec - exiting 07:35:38 (8084): No heartbeat from core client for 30 sec - exiting 07:35:39 (8084): No heartbeat from core client for 30 sec - exiting 07:35:40 (8084): No heartbeat from core client for 30 sec - exiting 07:35:41 (8084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:41:24 (5260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9172, iMonCtr=1 Model crash detected, will try to restart... 17:57:07 (6436): No heartbeat from core client for 30 sec - exiting 17:57:08 (6436): No heartbeat from core client for 30 sec - exiting 17:57:10 (6436): No heartbeat from core client for 30 sec - exiting 17:57:11 (6436): No heartbeat from core client for 30 sec - exiting 17:57:12 (6436): No heartbeat from core client for 30 sec - exiting 17:57:13 (6436): No heartbeat from core client for 30 sec - exiting 17:57:14 (6436): No heartbeat from core client for 30 sec - exiting 17:57:15 (6436): No heartbeat from core client for 30 sec - exiting 17:57:16 (6436): No heartbeat from core client for 30 sec - exiting 17:57:17 (6436): No heartbeat from core client for 30 sec - exiting 17:57:18 (6436): No heartbeat from core client for 30 sec - exiting 17:57:19 (6436): No heartbeat from core client for 30 sec - exiting 17:57:20 (6436): No heartbeat from core client for 30 sec - exiting 17:57:21 (6436): No heartbeat from core client for 30 sec - exiting 17:57:22 (6436): No heartbeat from core client for 30 sec - exiting 17:57:23 (6436): No heartbeat from core client for 30 sec - exiting 17:57:24 (6436): No heartbeat from core client for 30 sec - exiting 17:57:25 (6436): No heartbeat from core client for 30 sec - exiting 17:57:26 (6436): No heartbeat from core client for 30 sec - exiting 17:57:27 (6436): No heartbeat from core client for 30 sec - exiting 17:57:28 (6436): No heartbeat from core client for 30 sec - exiting 17:57:29 (6436): No heartbeat from core client for 30 sec - exiting 17:57:30 (6436): No heartbeat from core client for 30 sec - exiting 17:57:31 (6436): No heartbeat from core client for 30 sec - exiting 17:57:32 (6436): No heartbeat from core client for 30 sec - exiting 17:57:33 (6436): No heartbeat from core client for 30 sec - exiting 17:57:34 (6436): No heartbeat from core client for 30 sec - exiting 17:57:35 (6436): No heartbeat from core client for 30 sec - exiting 17:57:36 (6436): No heartbeat from core client for 30 sec - exiting 17:57:37 (6436): No heartbeat from core client for 30 sec - exiting 17:57:38 (6436): No heartbeat from core client for 30 sec - exiting 17:57:39 (6436): No heartbeat from core client for 30 sec - exiting 17:57:40 (6436): No heartbeat from core client for 30 sec - exiting 17:57:41 (6436): No heartbeat from core client for 30 sec - exiting 17:57:42 (6436): No heartbeat from core client for 30 sec - exiting 17:57:43 (6436): No heartbeat from core client for 30 sec - exiting 17:57:44 (6436): No heartbeat from core client for 30 sec - exiting 17:57:45 (6436): No heartbeat from core client for 30 sec - exiting 17:57:46 (6436): No heartbeat from core client for 30 sec - exiting 17:57:47 (6436): No heartbeat from core client for 30 sec - exiting 17:57:48 (6436): No heartbeat from core client for 30 sec - exiting 17:57:49 (6436): No heartbeat from core client for 30 sec - exiting 17:57:50 (6436): No heartbeat from core client for 30 sec - exiting 17:57:51 (6436): No heartbeat from core client for 30 sec - exiting 17:57:52 (6436): No heartbeat from core client for 30 sec - exiting 17:57:53 (6436): No heartbeat from core client for 30 sec - exiting 17:57:54 (6436): No heartbeat from core client for 30 sec - exiting 17:57:55 (6436): No heartbeat from core client for 30 sec - exiting 17:57:56 (6436): No heartbeat from core client for 30 sec - exiting 17:57:57 (6436): No heartbeat from core client for 30 sec - exiting 17:57:58 (6436): No heartbeat from core client for 30 sec - exiting 17:57:59 (6436): No heartbeat from core client for 30 sec - exiting 17:58:00 (6436): No heartbeat from core client for 30 sec - exiting 17:58:01 (6436): No heartbeat from core client for 30 sec - exiting 17:58:02 (6436): No heartbeat from core client for 30 sec - exiting 17:58:03 (6436): No heartbeat from core client for 30 sec - exiting 17:58:04 (6436): No heartbeat from core client for 30 sec - exiting 17:58:05 (6436): No heartbeat from core client for 30 sec - exiting 17:58:06 (6436): No heartbeat from core client for 30 sec - exiting 17:58:07 (6436): No heartbeat from core client for 30 sec - exiting 17:58:08 (6436): No heartbeat from core client for 30 sec - exiting 17:58:09 (6436): No heartbeat from core client for 30 sec - exiting 17:58:10 (6436): No heartbeat from core client for 30 sec - exiting 17:58:11 (6436): No heartbeat from core client for 30 sec - exiting 17:58:12 (6436): No heartbeat from core client for 30 sec - exiting 17:58:13 (6436): No heartbeat from core client for 30 sec - exiting 17:58:14 (6436): No heartbeat from core client for 30 sec - exiting 17:58:15 (6436): No heartbeat from core client for 30 sec - exiting 17:58:16 (6436): No heartbeat from core client for 30 sec - exiting 17:58:17 (6436): No heartbeat from core client for 30 sec - exiting 17:58:18 (6436): No heartbeat from core client for 30 sec - exiting 17:58:19 (6436): No heartbeat from core client for 30 sec - exiting 17:58:20 (6436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:15:23 (3908): No heartbeat from core client for 30 sec - exiting 08:15:24 (3908): No heartbeat from core client for 30 sec - exiting 08:15:25 (3908): No heartbeat from core client for 30 sec - exiting 08:15:26 (3908): No heartbeat from core client for 30 sec - exiting 08:15:27 (3908): No heartbeat from core client for 30 sec - exiting 08:15:28 (3908): No heartbeat from core client for 30 sec - exiting 08:15:29 (3908): No heartbeat from core client for 30 sec - exiting 08:15:30 (3908): No heartbeat from core client for 30 sec - exiting 08:15:31 (3908): No heartbeat from core client for 30 sec - exiting 08:15:32 (3908): No heartbeat from core client for 30 sec - exiting 08:15:33 (3908): No heartbeat from core client for 30 sec - exiting 08:15:34 (3908): No heartbeat from core client for 30 sec - exiting 08:15:35 (3908): No heartbeat from core client for 30 sec - exiting 08:15:36 (3908): No heartbeat from core client for 30 sec - exiting 08:15:37 (3908): No heartbeat from core client for 30 sec - exiting 08:15:38 (3908): No heartbeat from core client for 30 sec - exiting 08:15:39 (3908): No heartbeat from core client for 30 sec - exiting 08:15:40 (3908): No heartbeat from core client for 30 sec - exiting 08:15:41 (3908): No heartbeat from core client for 30 sec - exiting 08:15:42 (3908): No heartbeat from core client for 30 sec - exiting 08:15:43 (3908): No heartbeat from core client for 30 sec - exiting 08:15:44 (3908): No heartbeat from core client for 30 sec - exiting 08:15:45 (3908): No heartbeat from core client for 30 sec - exiting 08:15:46 (3908): No heartbeat from core client for 30 sec - exiting 08:15:47 (3908): No heartbeat from core client for 30 sec - exiting 08:15:48 (3908): No heartbeat from core client for 30 sec - exiting 08:15:49 (3908): No heartbeat from core client for 30 sec - exiting 08:15:50 (3908): No heartbeat from core client for 30 sec - exiting 08:15:51 (3908): No heartbeat from core client for 30 sec - exiting 08:15:52 (3908): No heartbeat from core client for 30 sec - exiting 08:15:53 (3908): No heartbeat from core client for 30 sec - exiting 08:15:54 (3908): No heartbeat from core client for 30 sec - exiting 08:15:55 (3908): No heartbeat from core client for 30 sec - exiting 08:15:56 (3908): No heartbeat from core client for 30 sec - exiting 08:15:57 (3908): No heartbeat from core client for 30 sec - exiting 08:15:58 (3908): No heartbeat from core client for 30 sec - exiting 08:15:59 (3908): No heartbeat from core client for 30 sec - exiting 08:16:00 (3908): No heartbeat from core client for 30 sec - exiting 08:16:01 (3908): No heartbeat from core client for 30 sec - exiting 08:16:02 (3908): No heartbeat from core client for 30 sec - exiting 08:16:03 (3908): No heartbeat from core client for 30 sec - exiting 08:16:04 (3908): No heartbeat from core client for 30 sec - exiting 08:16:05 (3908): No heartbeat from core client for 30 sec - exiting 08:16:06 (3908): No heartbeat from core client for 30 sec - exiting 08:16:07 (3908): No heartbeat from core client for 30 sec - exiting 08:16:08 (3908): No heartbeat from core client for 30 sec - exiting 08:16:09 (3908): No heartbeat from core client for 30 sec - exiting 08:16:10 (3908): No heartbeat from core client for 30 sec - exiting 08:16:11 (3908): No heartbeat from core client for 30 sec - exiting 08:16:12 (3908): No heartbeat from core client for 30 sec - exiting 08:16:13 (3908): No heartbeat from core client for 30 sec - exiting 08:16:14 (3908): No heartbeat from core client for 30 sec - exiting 08:16:15 (3908): No heartbeat from core client for 30 sec - exiting 08:16:16 (3908): No heartbeat from core client for 30 sec - exiting 08:16:17 (3908): No heartbeat from core client for 30 sec - exiting 08:16:18 (3908): No heartbeat from core client for 30 sec - exiting 08:16:19 (3908): No heartbeat from core client for 30 sec - exiting 08:16:20 (3908): No heartbeat from core client for 30 sec - exiting 08:16:21 (3908): No heartbeat from core client for 30 sec - exiting 08:16:22 (3908): No heartbeat from core client for 30 sec - exiting 08:16:23 (3908): No heartbeat from core client for 30 sec - exiting 08:16:24 (3908): No heartbeat from core client for 30 sec - exiting 08:16:25 (3908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7712, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=936, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Jan 2013 13:25:48	1260264	15542039	hadcm3n_zb6w_1960_40_008282873_0	25,920	50,067	1.9316