Task 16042174

Name	hadcm3n_ocmj_1900_40_008471486_0
Workunit	8622325
Created	27 Sep 2013, 10:05:48 UTC
Sent	1 Oct 2013, 3:32:46 UTC
Report deadline	31 Dec 2013, 10:59:57 UTC
Received	3 Nov 2013, 5:36:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1204618
Run time	7 days 7 hours 47 min 1 sec
CPU time	7 days 4 hours 1 min 9 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	2.57 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:10:53 (6728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:54:37 (4200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:25:20 (7120): No heartbeat from core client for 30 sec - exiting 14:25:21 (7120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:22 (7120): No heartbeat from core client for 30 sec - exiting 14:25:23 (7120): No heartbeat from core client for 30 sec - exiting 14:56:53 (2228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:13:09 (4464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:43:10 (8004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4704, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4704, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4704, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:33:47 (4352): No heartbeat from core client for 30 sec - exiting 10:33:49 (4352): No heartbeat from core client for 30 sec - exiting 10:33:50 (4352): No heartbeat from core client for 30 sec - exiting 10:33:51 (4352): No heartbeat from core client for 30 sec - exiting 10:33:52 (4352): No heartbeat from core client for 30 sec - exiting 10:33:53 (4352): No heartbeat from core client for 30 sec - exiting 10:33:54 (4352): No heartbeat from core client for 30 sec - exiting 10:33:55 (4352): No heartbeat from core client for 30 sec - exiting 10:33:56 (4352): No heartbeat from core client for 30 sec - exiting 10:33:57 (4352): No heartbeat from core client for 30 sec - exiting 10:33:58 (4352): No heartbeat from core client for 30 sec - exiting 10:33:59 (4352): No heartbeat from core client for 30 sec - exiting 10:34:00 (4352): No heartbeat from core client for 30 sec - exiting 10:34:01 (4352): No heartbeat from core client for 30 sec - exiting 10:34:02 (4352): No heartbeat from core client for 30 sec - exiting 10:34:03 (4352): No heartbeat from core client for 30 sec - exiting 10:34:04 (4352): No heartbeat from core client for 30 sec - exiting 10:34:05 (4352): No heartbeat from core client for 30 sec - exiting 10:34:06 (4352): No heartbeat from core client for 30 sec - exiting 10:34:07 (4352): No heartbeat from core client for 30 sec - exiting 10:34:08 (4352): No heartbeat from core client for 30 sec - exiting 10:34:09 (4352): No heartbeat from core client for 30 sec - exiting 10:34:10 (4352): No heartbeat from core client for 30 sec - exiting 10:34:11 (4352): No heartbeat from core client for 30 sec - exiting 10:34:12 (4352): No heartbeat from core client for 30 sec - exiting 10:34:13 (4352): No heartbeat from core client for 30 sec - exiting 10:34:14 (4352): No heartbeat from core client for 30 sec - exiting 10:34:15 (4352): No heartbeat from core client for 30 sec - exiting 10:34:16 (4352): No heartbeat from core client for 30 sec - exiting 10:34:17 (4352): No heartbeat from core client for 30 sec - exiting 10:34:18 (4352): No heartbeat from core client for 30 sec - exiting 10:34:19 (4352): No heartbeat from core client for 30 sec - exiting 10:34:20 (4352): No heartbeat from core client for 30 sec - exiting 10:34:21 (4352): No heartbeat from core client for 30 sec - exiting 10:34:22 (4352): No heartbeat from core client for 30 sec - exiting 10:34:23 (4352): No heartbeat from core client for 30 sec - exiting 10:34:24 (4352): No heartbeat from core client for 30 sec - exiting 10:34:25 (4352): No heartbeat from core client for 30 sec - exiting 10:34:26 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1464, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Oct 2013 15:36:16	1204618	16042174	hadcm3n_ocmj_1900_40_008471486_0	155,520	576,540	3.7072
26 Oct 2013 12:08:29	1204618	16042174	hadcm3n_ocmj_1900_40_008471486_0	129,600	517,813	3.9955
18 Oct 2013 02:48:14	1204618	16042174	hadcm3n_ocmj_1900_40_008471486_0	103,680	200,601	1.9348
17 Oct 2013 09:43:56	1204618	16042174	hadcm3n_ocmj_1900_40_008471486_0	77,760	159,862	2.0558
11 Oct 2013 12:12:24	1204618	16042174	hadcm3n_ocmj_1900_40_008471486_0	51,840	106,660	2.0575
07 Oct 2013 11:37:35	1204618	16042174	hadcm3n_ocmj_1900_40_008471486_0	25,920	53,945	2.0812