Task 13678053

Name	hadcm3n_u478_1980_40_007548320_1
Workunit	7745552
Created	30 Nov 2011, 21:17:46 UTC
Sent	30 Nov 2011, 21:21:05 UTC
Report deadline	1 Mar 2012, 4:48:16 UTC
Received	29 Dec 2011, 7:29:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1176018
Run time	15 days 6 hours 9 min 29 sec
CPU time	15 days 1 hours 44 min 23 sec
Validate state	Invalid
Credit	8,709.12
Device peak FLOPS	2.54 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 10:35:08 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:33:31 (1908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:31:23 (2784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3820, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 15:16:42 (2548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:15:09 (3864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:29 (2496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:10:54 (2244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:13:24 (2512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:12:03 (4452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:10:39 (1108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3156, iMonCtr=1 Model crash detected, will try to restart... 22:42:49 (2748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:09 (1544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:28:22 (3324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:26:54 (4264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3924, iMonCtr=1 Model crash detected, will try to restart... 11:36:18 (2840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:34:40 (820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:33:10 (1388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:31:52 (2412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:30:43 (1164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:29:31 (2876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:51:18 (2504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:50:14 (3028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:49:08 (972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:46:53 (1876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:33 (3108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:44:26 (3224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:40:12 (1360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:00 (2480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:24:26 (2180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:16 (1568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:22:05 (3044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:52 (1332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:40:01 (3648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:39:02 (2396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:15 (2696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:25:36 (2748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:24:33 (2196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:23:24 (2828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:04 (1900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2940, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2940, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2940, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2940, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2940, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2940, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Dec 2011 16:31:36	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	725,760	1,274,708	1.7564
27 Dec 2011 20:21:32	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	699,840	1,231,017	1.7590
26 Dec 2011 16:41:38	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	673,920	1,187,377	1.7619
25 Dec 2011 22:07:55	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	648,000	1,143,172	1.7642
25 Dec 2011 10:39:20	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	622,080	1,102,090	1.7716
24 Dec 2011 14:10:57	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	596,160	1,058,700	1.7759
24 Dec 2011 01:58:41	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	570,240	1,015,203	1.7803
23 Dec 2011 13:42:20	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	544,320	971,446	1.7847
22 Dec 2011 17:05:46	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	518,400	928,141	1.7904
21 Dec 2011 19:12:18	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	492,480	885,628	1.7983
21 Dec 2011 06:02:15	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	466,560	841,312	1.8032
20 Dec 2011 17:32:30	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	440,640	797,088	1.8089
20 Dec 2011 06:57:24	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	414,720	755,798	1.8224
19 Dec 2011 10:07:58	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	388,800	712,590	1.8328
18 Dec 2011 14:28:39	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	362,880	666,976	1.8380
17 Dec 2011 16:25:34	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	336,960	623,247	1.8496
16 Dec 2011 17:48:47	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	311,040	579,434	1.8629
16 Dec 2011 05:35:42	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	285,120	536,297	1.8810
15 Dec 2011 18:04:42	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	259,200	494,834	1.9091
15 Dec 2011 06:18:42	1176018	13678053	hadcm3n_u478_1980_40_007548320_1	233,280	451,041	1.9335