Task 13285531

Name	hadcm3n_t28t_1940_40_007317387_2
Workunit	7514817
Created	24 Aug 2011, 9:30:27 UTC
Sent	24 Aug 2011, 9:30:33 UTC
Report deadline	23 Nov 2011, 16:57:44 UTC
Received	26 Sep 2011, 8:26:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1120191
Run time	4 days 22 hours 35 min 38 sec
CPU time	4 days 11 hours 37 min 10 sec
Validate state	Invalid
Credit	1,555.20
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 08:47:42 (9700): No heartbeat from core client for 30 sec - exiting 08:47:43 (9700): No heartbeat from core client for 30 sec - exiting 08:47:44 (9700): No heartbeat from core client for 30 sec - exiting 08:47:45 (9700): No heartbeat from core client for 30 sec - exiting 08:47:46 (9700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:15:30 (7752): Can't acquire lockfile (32) - waiting 35s 15:15:42 (6224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:33:14 (7816): No heartbeat from core client for 30 sec - exiting 13:33:16 (7816): No heartbeat from core client for 30 sec - exiting 13:33:17 (7816): No heartbeat from core client for 30 sec - exiting 13:33:18 (7816): No heartbeat from core client for 30 sec - exiting 13:33:19 (7816): No heartbeat from core client for 30 sec - exiting 13:33:20 (7816): No heartbeat from core client for 30 sec - exiting 13:33:21 (7816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:34:53 (3484): No heartbeat from core client for 30 sec - exiting 09:34:54 (3484): No heartbeat from core client for 30 sec - exiting 09:34:55 (3484): No heartbeat from core client for 30 sec - exiting 09:34:56 (3484): No heartbeat from core client for 30 sec - exiting 09:34:57 (3484): No heartbeat from core client for 30 sec - exiting 09:34:58 (3484): No heartbeat from core client for 30 sec - exiting 09:34:59 (3484): No heartbeat from core client for 30 sec - exiting 09:35:00 (3484): No heartbeat from core client for 30 sec - exiting 09:35:01 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:37:46 (3776): No heartbeat from core client for 30 sec - exiting 09:37:47 (3776): No heartbeat from core client for 30 sec - exiting 09:37:48 (3776): No heartbeat from core client for 30 sec - exiting 09:37:49 (3776): No heartbeat from core client for 30 sec - exiting 09:37:50 (3776): No heartbeat from core client for 30 sec - exiting 09:37:51 (3776): No heartbeat from core client for 30 sec - exiting 09:37:52 (3776): No heartbeat from core client for 30 sec - exiting 09:37:53 (3776): No heartbeat from core client for 30 sec - exiting 09:37:54 (3776): No heartbeat from core client for 30 sec - exiting 09:37:55 (3776): No heartbeat from core client for 30 sec - exiting 09:37:56 (3776): No heartbeat from core client for 30 sec - exiting 09:37:57 (3776): No heartbeat from core client for 30 sec - exiting 09:37:58 (3776): No heartbeat from core client for 30 sec - exiting 09:37:59 (3776): No heartbeat from core client for 30 sec - exiting 09:38:00 (3776): No heartbeat from core client for 30 sec - exiting 09:38:01 (3776): No heartbeat from core client for 30 sec - exiting 09:38:02 (3776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:03 (3776): No heartbeat from core client for 30 sec - exiting 09:38:04 (3776): No heartbeat from core client for 30 sec - exiting 09:38:05 (3776): No heartbeat from core client for 30 sec - exiting 09:38:06 (3776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:12:08 (6420): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:03:16 (7036): No heartbeat from core client for 30 sec - exiting 10:03:17 (7036): No heartbeat from core client for 30 sec - exiting 10:03:18 (7036): No heartbeat from core client for 30 sec - exiting 10:03:19 (7036): No heartbeat from core client for 30 sec - exiting 10:03:20 (7036): No heartbeat from core client for 30 sec - exiting 10:03:21 (7036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:03:22 (7036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:02:25 (2536): No heartbeat from core client for 30 sec - exiting 09:02:26 (2536): No heartbeat from core client for 30 sec - exiting 09:02:27 (2536): No heartbeat from core client for 30 sec - exiting 09:02:28 (2536): No heartbeat from core client for 30 sec - exiting 09:02:29 (2536): No heartbeat from core client for 30 sec - exiting 09:02:30 (2536): No heartbeat from core client for 30 sec - exiting 09:02:31 (2536): No heartbeat from core client for 30 sec - exiting 09:02:32 (2536): No heartbeat from core client for 30 sec - exiting 09:02:33 (2536): No heartbeat from core client for 30 sec - exiting 09:02:34 (2536): No heartbeat from core client for 30 sec - exiting 09:02:35 (2536): No heartbeat from core client for 30 sec - exiting 09:02:36 (2536): No heartbeat from core client for 30 sec - exiting 09:02:37 (2536): No heartbeat from core client for 30 sec - exiting 09:02:38 (2536): No heartbeat from core client for 30 sec - exiting 09:02:39 (2536): No heartbeat from core client for 30 sec - exiting 09:02:40 (2536): No heartbeat from core client for 30 sec - exiting 09:02:41 (2536): No heartbeat from core client for 30 sec - exiting 09:02:42 (2536): No heartbeat from core client for 30 sec - exiting 09:02:43 (2536): No heartbeat from core client for 30 sec - exiting 09:02:44 (2536): No heartbeat from core client for 30 sec - exiting 09:02:45 (2536): No heartbeat from core client for 30 sec - exiting 09:02:46 (2536): No heartbeat from core client for 30 sec - exiting 09:02:47 (2536): No heartbeat from core client for 30 sec - exiting 09:02:48 (2536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:31:02 (6204): Can't acquire lockfile (32) - waiting 35s 11:31:29 (1384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Cforrtl: Access is denied. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7564, selfPID=7564, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:48:47 (3372): No heartbeat from core client for 30 sec - exiting 21:48:48 (3372): No heartbeat from core client for 30 sec - exiting 21:48:49 (3372): No heartbeat from core client for 30 sec - exiting 21:48:50 (3372): No heartbeat from core client for 30 sec - exiting 21:48:51 (3372): No heartbeat from core client for 30 sec - exiting 21:48:52 (3372): No heartbeat from core client for 30 sec - exiting 21:48:53 (3372): No heartbeat from core client for 30 sec - exiting 21:48:54 (3372): No heartbeat from core client for 30 sec - exiting 21:48:55 (3372): No heartbeat from core client for 30 sec - exiting 21:48:56 (3372): No heartbeat from core client for 30 sec - exiting 21:48:57 (3372): No heartbeat from core client for 30 sec - exiting 21:48:58 (3372): No heartbeat from core client for 30 sec - exiting 21:48:59 (3372): No heartbeat from core client for 30 sec - exiting 21:49:00 (3372): No heartbeat from core client for 30 sec - exiting 21:49:01 (3372): No heartbeat from core client for 30 sec - exiting 21:49:02 (3372): No heartbeat from core client for 30 sec - exiting 21:49:03 (3372): No heartbeat from core client for 30 sec - exiting 21:49:04 (3372): No heartbeat from core client for 30 sec - exiting 21:49:05 (3372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:49:06 (3372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5916, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5916, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5916, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5916, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5916, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5916, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Sep 2011 15:16:53	1120191	13285531	hadcm3n_t28t_1940_40_007317387_2	129,600	327,839	2.5296
15 Sep 2011 16:02:29	1120191	13285531	hadcm3n_t28t_1940_40_007317387_2	103,680	261,741	2.5245
07 Sep 2011 11:58:11	1120191	13285531	hadcm3n_t28t_1940_40_007317387_2	77,760	196,919	2.5324
02 Sep 2011 15:30:13	1120191	13285531	hadcm3n_t28t_1940_40_007317387_2	51,840	131,181	2.5305
26 Aug 2011 13:59:22	1120191	13285531	hadcm3n_t28t_1940_40_007317387_2	25,920	65,992	2.5460