Task 13626012

Name	hadcm3n_o6po_1940_40_007543173_1
Workunit	7740405
Created	10 Nov 2011, 0:44:34 UTC
Sent	16 Nov 2011, 14:24:21 UTC
Report deadline	15 Feb 2012, 21:51:32 UTC
Received	21 Nov 2011, 15:00:16 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1176150
Run time	14 hours 15 min 9 sec
CPU time	14 hours 15 min 9 sec
Validate state	Invalid
Credit	311.04
Device peak FLOPS	2.47 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:22:51 (1356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:23:34 (4016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:17 (3412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:28 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:18 (1160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:23 (4844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:42:23 (2068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:43:05 (2584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:22 (564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:47:31 (2732): No heartbeat from core client for 30 sec - exiting 20:47:32 (2732): No heartbeat from core client for 30 sec - exiting 20:47:33 (2732): No heartbeat from core client for 30 sec - exiting 20:47:34 (2732): No heartbeat from core client for 30 sec - exiting 20:47:35 (2732): No heartbeat from core client for 30 sec - exiting 20:47:36 (2732): No heartbeat from core client for 30 sec - exiting 20:47:37 (2732): No heartbeat from core client for 30 sec - exiting 20:47:38 (2732): No heartbeat from core client for 30 sec - exiting 20:47:39 (2732): No heartbeat from core client for 30 sec - exiting 20:47:40 (2732): No heartbeat from core client for 30 sec - exiting 20:47:42 (2732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:50:02 (4052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:52:41 (368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:57:09 (1232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:57:12 (1232): No heartbeat from core client for 30 sec - exiting 20:57:13 (1232): No heartbeat from core client for 30 sec - exiting 20:57:14 (1232): No heartbeat from core client for 30 sec - exiting 20:57:15 (1232): No heartbeat from core client for 30 sec - exiting 20:57:17 (1232): No heartbeat from core client for 30 sec - exiting 20:57:18 (1232): No heartbeat from core client for 30 sec - exiting 20:58:57 (3104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:45 (1200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:08 (440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:11 (440): No heartbeat from core client for 30 sec - exiting 21:08:13 (440): No heartbeat from core client for 30 sec - exiting 21:08:15 (440): No heartbeat from core client for 30 sec - exiting 21:08:16 (440): No heartbeat from core client for 30 sec - exiting 21:08:17 (440): No heartbeat from core client for 30 sec - exiting 21:08:18 (440): No heartbeat from core client for 30 sec - exiting 21:10:32 (4160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:10:34 (4160): No heartbeat from core client for 30 sec - exiting 21:10:35 (4160): No heartbeat from core client for 30 sec - exiting 21:10:36 (4160): No heartbeat from core client for 30 sec - exiting 21:10:37 (4160): No heartbeat from core client for 30 sec - exiting 21:13:55 (2092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:42:41 (4140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:43:17 (2176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:46:03 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:49:26 (4780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:51:10 (3064): No heartbeat from core client for 30 sec - exiting 21:51:11 (3064): No heartbeat from core client for 30 sec - exiting 21:51:12 (3064): No heartbeat from core client for 30 sec - exiting 21:51:13 (3064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:54:51 (4996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:56:24 (228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:32 (872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:39 (872): No heartbeat from core client for 30 sec - exiting 21:57:40 (872): No heartbeat from core client for 30 sec - exiting 21:57:41 (872): No heartbeat from core client for 30 sec - exiting 21:57:42 (872): No heartbeat from core client for 30 sec - exiting 21:57:43 (872): No heartbeat from core client for 30 sec - exiting 21:57:44 (872): No heartbeat from core client for 30 sec - exiting 21:57:46 (872): No heartbeat from core client for 30 sec - exiting 21:57:47 (872): No heartbeat from core client for 30 sec - exiting 21:57:48 (872): No heartbeat from core client for 30 sec - exiting 21:57:49 (872): No heartbeat from core client for 30 sec - exiting 22:01:25 (5036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:01:26 (5036): No heartbeat from core client for 30 sec - exiting 22:01:27 (5036): No heartbeat from core client for 30 sec - exiting 22:02:34 (4884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:37 (4596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:06:24 (552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:54 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:10:25 (2552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:12:10 (56): No heartbeat from core client for 30 sec - exiting 22:12:11 (56): No heartbeat from core client for 30 sec - exiting 22:12:12 (56): No heartbeat from core client for 30 sec - exiting 22:12:13 (56): No heartbeat from core client for 30 sec - exiting 22:12:14 (56): No heartbeat from core client for 30 sec - exiting 22:12:15 (56): No heartbeat from core client for 30 sec - exiting 22:12:16 (56): No heartbeat from core client for 30 sec - exiting 22:12:17 (56): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:17:09 (2628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:17:56 (4768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:12 (4952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:26:29 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:27:08 (2164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:26:56 (1304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:26:58 (1304): No heartbeat from core client for 30 sec - exiting 04:27:38 (4084): No heartbeat from core client for 30 sec - exiting 04:27:39 (4084): No heartbeat from core client for 30 sec - exiting 04:27:40 (4084): No heartbeat from core client for 30 sec - exiting 04:27:41 (4084): No heartbeat from core client for 30 sec - exiting 04:27:42 (4084): No heartbeat from core client for 30 sec - exiting 04:27:44 (4084): No heartbeat from core client for 30 sec - exiting 04:27:45 (4084): No heartbeat from core client for 30 sec - exiting 04:27:46 (4084): No heartbeat from core client for 30 sec - exiting 04:27:47 (4084): No heartbeat from core client for 30 sec - exiting 04:27:48 (4084): No heartbeat from core client for 30 sec - exiting 04:27:49 (4084): No heartbeat from core client for 30 sec - exiting 04:27:50 (4084): No heartbeat from core client for 30 sec - exiting 04:27:51 (4084): No heartbeat from core client for 30 sec - exiting 04:27:52 (4084): No heartbeat from core client for 30 sec - exiting 04:27:53 (4084): No heartbeat from core client for 30 sec - exiting 04:27:55 (4084): No heartbeat from core client for 30 sec - exiting 04:27:56 (4084): No heartbeat from core client for 30 sec - exiting 04:27:57 (4084): No heartbeat from core client for 30 sec - exiting 04:27:58 (4084): No heartbeat from core client for 30 sec - exiting 04:27:59 (4084): No heartbeat from core client for 30 sec - exiting 04:28:00 (4084): No heartbeat from core client for 30 sec - exiting 04:28:01 (4084): No heartbeat from core client for 30 sec - exiting 04:28:02 (4084): No heartbeat from core client for 30 sec - exiting 04:28:03 (4084): No heartbeat from core client for 30 sec - exiting 04:28:04 (4084): No heartbeat from core client for 30 sec - exiting 04:28:05 (4084): No heartbeat from core client for 30 sec - exiting 04:28:07 (4084): No heartbeat from core client for 30 sec - exiting 04:28:08 (4084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:57 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:58 (4064): No heartbeat from core client for 30 sec - exiting 04:32:00 (4064): No heartbeat from core client for 30 sec - exiting 04:32:01 (4064): No heartbeat from core client for 30 sec - exiting 04:33:21 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:36:57 (4772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:37:00 (4772): No heartbeat from core client for 30 sec - exiting 04:37:01 (4772): No heartbeat from core client for 30 sec - exiting 04:38:59 (4144): No heartbeat from core client for 30 sec - exiting 04:39:00 (4144): No heartbeat from core client for 30 sec - exiting 04:39:01 (4144): No heartbeat from core client for 30 sec - exiting 04:39:02 (4144): No heartbeat from core client for 30 sec - exiting 04:39:03 (4144): No heartbeat from core client for 30 sec - exiting 04:39:04 (4144): No heartbeat from core client for 30 sec - exiting 04:39:05 (4144): No heartbeat from core client for 30 sec - exiting 04:39:06 (4144): No heartbeat from core client for 30 sec - exiting 04:39:08 (4144): No heartbeat from core client for 30 sec - exiting 04:39:09 (4144): No heartbeat from core client for 30 sec - exiting 04:39:10 (4144): No heartbeat from core client for 30 sec - exiting 04:39:11 (4144): No heartbeat from core client for 30 sec - exiting 04:39:12 (4144): No heartbeat from core client for 30 sec - exiting 04:39:13 (4144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:45:17 (1664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:47:03 (1048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:03 (3368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:04 (3368): No heartbeat from core client for 30 sec - exiting 04:52:06 (3368): No heartbeat from core client for 30 sec - exiting 04:53:28 (2432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:57:48 (3752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:59:44 (472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:41 (3928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:05:28 (3560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:09:04 (2576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:10:57 (4780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:13:44 (4812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=540, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=540, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=540, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=540, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=540, iMonCtr=1 Model crash detected, will try to restart... 05:16:05 (540): No heartbeat from core client for 30 sec - exiting 05:16:06 (540): No heartbeat from core client for 30 sec - exiting 05:16:07 (540): No heartbeat from core client for 30 sec - exiting 05:16:09 (540): No heartbeat from core client for 30 sec - exiting 05:16:10 (540): No heartbeat from core client for 30 sec - exiting 05:16:11 (540): No heartbeat from core client for 30 sec - exiting 05:16:12 (540): No heartbeat from core client for 30 sec - exiting 05:16:13 (540): No heartbeat from core client for 30 sec - exiting 05:16:14 (540): No heartbeat from core client for 30 sec - exiting 05:16:15 (540): No heartbeat from core client for 30 sec - exiting 05:16:16 (540): No heartbeat from core client for 30 sec - exiting 05:16:17 (540): No heartbeat from core client for 30 sec - exiting 05:16:18 (540): No heartbeat from core client for 30 sec - exiting 05:16:20 (540): No heartbeat from core client for 30 sec - exiting 05:16:21 (540): No heartbeat from core client for 30 sec - exiting 05:16:22 (540): No heartbeat from core client for 30 sec - exiting 05:16:23 (540): No heartbeat from core client for 30 sec - exiting 05:16:24 (540): No heartbeat from core client for 30 sec - exiting 05:16:25 (540): No heartbeat from core client for 30 sec - exiting 05:16:26 (540): No heartbeat from core client for 30 sec - exiting 05:16:27 (540): No heartbeat from core client for 30 sec - exiting 05:16:28 (540): No heartbeat from core client for 30 sec - exiting 05:16:29 (540): No heartbeat from core client for 30 sec - exiting 05:16:30 (540): No heartbeat from core client for 30 sec - exiting 05:16:32 (540): No heartbeat from core client for 30 sec - exiting 05:16:33 (540): No heartbeat from core client for 30 sec - exiting 05:16:34 (540): No heartbeat from core client for 30 sec - exiting 05:16:35 (540): No heartbeat from core client for 30 sec - exiting 05:16:36 (540): No heartbeat from core client for 30 sec - exiting 05:16:37 (540): No heartbeat from core client for 30 sec - exiting 05:16:38 (540): No heartbeat from core client for 30 sec - exiting 05:16:39 (540): No heartbeat from core client for 30 sec - exiting 05:16:40 (540): No heartbeat from core client for 30 sec - exiting 05:16:42 (540): No heartbeat from core client for 30 sec - exiting 05:16:43 (540): No heartbeat from core client for 30 sec - exiting 05:16:44 (540): No heartbeat from core client for 30 sec - exiting 05:16:45 (540): No heartbeat from core client for 30 sec - exiting 05:16:46 (540): No heartbeat from core client for 30 sec - exiting 05:16:47 (540): No heartbeat from core client for 30 sec - exiting 05:16:48 (540): No heartbeat from core client for 30 sec - exiting 05:16:49 (540): No heartbeat from core client for 30 sec - exiting 05:16:50 (540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3572, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 05:19:25 (3572): No heartbeat from core client for 30 sec - exiting 05:19:26 (3572): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Nov 2011 11:14:09	1176150	13626012	hadcm3n_o6po_1940_40_007543173_1	25,920	48,809	1.8831