Task 13779596

Name	hadcm3n_t74i_1940_40_007449153_2
Workunit	7646656
Created	14 Dec 2011, 19:21:14 UTC
Sent	14 Dec 2011, 19:45:29 UTC
Report deadline	15 Mar 2012, 3:12:40 UTC
Received	23 Dec 2011, 18:23:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1129368
Run time	3 days 23 hours 45 min 58 sec
CPU time	3 days 9 hours 12 min 4 sec
Validate state	Invalid
Credit	2,177.28
Device peak FLOPS	2.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:50:52 (6392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:51:03 (6392): No heartbeat from core client for 30 sec - exiting 02:51:04 (6392): No heartbeat from core client for 30 sec - exiting 02:51:05 (6392): No heartbeat from core client for 30 sec - exiting 02:51:06 (6392): No heartbeat from core client for 30 sec - exiting 02:51:07 (6392): No heartbeat from core client for 30 sec - exiting 02:51:08 (6392): No heartbeat from core client for 30 sec - exiting 02:51:11 (6392): No heartbeat from core client for 30 sec - exiting 02:51:12 (6392): No heartbeat from core client for 30 sec - exiting 02:51:17 (6392): No heartbeat from core client for 30 sec - exiting 02:51:18 (6392): No heartbeat from core client for 30 sec - exiting 02:51:21 (6392): No heartbeat from core client for 30 sec - exiting 02:51:22 (6392): No heartbeat from core client for 30 sec - exiting 02:51:24 (6392): No heartbeat from core client for 30 sec - exiting 02:51:27 (6392): No heartbeat from core client for 30 sec - exiting 02:51:28 (6392): No heartbeat from core client for 30 sec - exiting 02:51:35 (6392): No heartbeat from core client for 30 sec - exiting 02:51:36 (6392): No heartbeat from core client for 30 sec - exiting 02:51:41 (6392): No heartbeat from core client for 30 sec - exiting 02:51:42 (6392): No heartbeat from core client for 30 sec - exiting 02:51:53 (6392): No heartbeat from core client for 30 sec - exiting 02:51:54 (6392): No heartbeat from core client for 30 sec - exiting 02:51:55 (6392): No heartbeat from core client for 30 sec - exiting 02:51:56 (6392): No heartbeat from core client for 30 sec - exiting 02:51:57 (6392): No heartbeat from core client for 30 sec - exiting 02:51:58 (6392): No heartbeat from core client for 30 sec - exiting 02:51:59 (6392): No heartbeat from core client for 30 sec - exiting 02:52:02 (6392): No heartbeat from core client for 30 sec - exiting 02:52:05 (6392): No heartbeat from core client for 30 sec - exiting 03:49:23 (5628): No heartbeat from core client for 30 sec - exiting 03:49:43 (5628): No heartbeat from core client for 30 sec - exiting 03:49:44 (5628): No heartbeat from core client for 30 sec - exiting 03:49:45 (5628): No heartbeat from core client for 30 sec - exiting 03:49:46 (5628): No heartbeat from core client for 30 sec - exiting 03:49:47 (5628): No heartbeat from core client for 30 sec - exiting 03:49:48 (5628): No heartbeat from core client for 30 sec - exiting 03:49:49 (5628): No heartbeat from core client for 30 sec - exiting 03:49:50 (5628): No heartbeat from core client for 30 sec - exiting 03:49:51 (5628): No heartbeat from core client for 30 sec - exiting 03:49:52 (5628): No heartbeat from core client for 30 sec - exiting 03:49:53 (5628): No heartbeat from core client for 30 sec - exiting 03:49:54 (5628): No heartbeat from core client for 30 sec - exiting 03:49:55 (5628): No heartbeat from core client for 30 sec - exiting 03:49:56 (5628): No heartbeat from core client for 30 sec - exiting 03:49:57 (5628): No heartbeat from core client for 30 sec - exiting 03:49:58 (5628): No heartbeat from core client for 30 sec - exiting 03:49:59 (5628): No heartbeat from core client for 30 sec - exiting 03:50:00 (5628): No heartbeat from core client for 30 sec - exiting 03:50:01 (5628): No heartbeat from core client for 30 sec - exiting 03:50:02 (5628): No heartbeat from core client for 30 sec - exiting 03:50:03 (5628): No heartbeat from core client for 30 sec - exiting 03:50:04 (5628): No heartbeat from core client for 30 sec - exiting 03:50:05 (5628): No heartbeat from core client for 30 sec - exiting 03:50:06 (5628): No heartbeat from core client for 30 sec - exiting 03:50:07 (5628): No heartbeat from core client for 30 sec - exiting 03:50:08 (5628): No heartbeat from core client for 30 sec - exiting 03:50:09 (5628): No heartbeat from core client for 30 sec - exiting 03:50:10 (5628): No heartbeat from core client for 30 sec - exiting 03:50:11 (5628): No heartbeat from core client for 30 sec - exiting 03:50:12 (5628): No heartbeat from core client for 30 sec - exiting 03:50:13 (5628): No heartbeat from core client for 30 sec - exiting 03:50:14 (5628): No heartbeat from core client for 30 sec - exiting 03:50:15 (5628): No heartbeat from core client for 30 sec - exiting 03:50:16 (5628): No heartbeat from core client for 30 sec - exiting 03:50:17 (5628): No heartbeat from core client for 30 sec - exiting 03:50:18 (5628): No heartbeat from core client for 30 sec - exiting 03:50:19 (5628): No heartbeat from core client for 30 sec - exiting 03:50:20 (5628): No heartbeat from core client for 30 sec - exiting 03:50:21 (5628): No heartbeat from core client for 30 sec - exiting 03:50:22 (5628): No heartbeat from core client for 30 sec - exiting 03:50:23 (5628): No heartbeat from core client for 30 sec - exiting 03:50:24 (5628): No heartbeat from core client for 30 sec - exiting 03:50:25 (5628): No heartbeat from core client for 30 sec - exiting 03:50:26 (5628): No heartbeat from core client for 30 sec - exiting 03:50:27 (5628): No heartbeat from core client for 30 sec - exiting 03:50:28 (5628): No heartbeat from core client for 30 sec - exiting 03:50:29 (5628): No heartbeat from core client for 30 sec - exiting 03:50:30 (5628): No heartbeat from core client for 30 sec - exiting 03:50:31 (5628): No heartbeat from core client for 30 sec - exiting 03:50:32 (5628): No heartbeat from core client for 30 sec - exiting 03:50:33 (5628): No heartbeat from core client for 30 sec - exiting 03:50:37 (5628): No heartbeat from core client for 30 sec - exiting 03:50:38 (5628): No heartbeat from core client for 30 sec - exiting 03:50:40 (5628): No heartbeat from core client for 30 sec - exiting 03:50:46 (5628): No heartbeat from core client for 30 sec - exiting 03:50:47 (5628): No heartbeat from core client for 30 sec - exiting 03:50:48 (5628): No heartbeat from core client for 30 sec - exiting 03:50:49 (5628): No heartbeat from core client for 30 sec - exiting 03:50:50 (5628): No heartbeat from core client for 30 sec - exiting 03:50:51 (5628): No heartbeat from core client for 30 sec - exiting 03:50:52 (5628): No heartbeat from core client for 30 sec - exiting 03:50:53 (5628): No heartbeat from core client for 30 sec - exiting 03:50:54 (5628): No heartbeat from core client for 30 sec - exiting 03:50:55 (5628): No heartbeat from core client for 30 sec - exiting 03:50:56 (5628): No heartbeat from core client for 30 sec - exiting 03:50:57 (5628): No heartbeat from core client for 30 sec - exiting 03:50:58 (5628): No heartbeat from core client for 30 sec - exiting 03:51:18 (5628): No heartbeat from core client for 30 sec - exiting 03:51:19 (5628): No heartbeat from core client for 30 sec - exiting 03:51:21 (5628): No heartbeat from core client for 30 sec - exiting 03:51:24 (5628): No heartbeat from core client for 30 sec - exiting 03:51:25 (5628): No heartbeat from core client for 30 sec - exiting 03:51:29 (5628): No heartbeat from core client for 30 sec - exiting 03:51:30 (5628): No heartbeat from core client for 30 sec - exiting 03:51:31 (5628): No heartbeat from core client for 30 sec - exiting 03:51:32 (5628): No heartbeat from core client for 30 sec - exiting 03:51:33 (5628): No heartbeat from core client for 30 sec - exiting 03:51:34 (5628): No heartbeat from core client for 30 sec - exiting 03:51:35 (5628): No heartbeat from core client for 30 sec - exiting 03:51:36 (5628): No heartbeat from core client for 30 sec - exiting 03:51:37 (5628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:20:22 (2664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:26 (2664): No heartbeat from core client for 30 sec - exiting 13:20:27 (2664): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:52:03 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:52:08 (6132): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Dec 2011 18:34:57	1129368	13779596	hadcm3n_t74i_1940_40_007449153_2	181,440	287,974	1.5872
23 Dec 2011 18:34:57	1129368	13779596	hadcm3n_t74i_1940_40_007449153_2	155,520	249,184	1.6023
23 Dec 2011 18:34:57	1129368	13779596	hadcm3n_t74i_1940_40_007449153_2	129,600	210,423	1.6236
23 Dec 2011 18:34:56	1129368	13779596	hadcm3n_t74i_1940_40_007449153_2	103,680	171,422	1.6534
19 Dec 2011 18:56:46	1129368	13779596	hadcm3n_t74i_1940_40_007449153_2	77,760	128,736	1.6556
19 Dec 2011 18:56:46	1129368	13779596	hadcm3n_t74i_1940_40_007449153_2	51,840	90,393	1.7437
19 Dec 2011 18:56:46	1129368	13779596	hadcm3n_t74i_1940_40_007449153_2	25,920	51,975	2.0052