Task 16169673

Name	hadcm3n_odnj_1900_40_008472818_1
Workunit	8623657
Created	29 Dec 2013, 22:41:17 UTC
Sent	29 Dec 2013, 22:41:29 UTC
Report deadline	31 Mar 2014, 6:08:40 UTC
Received	15 Jan 2014, 16:10:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1308610
Run time	6 days 20 hours 31 min 40 sec
CPU time	6 days 18 hours 36 min 33 sec
Validate state	Invalid
Credit	11,508.48
Device peak FLOPS	3.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:57:33 (4212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:17:10 (6348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:26:20 (5208): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 20:26:21 (5208): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5792, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:51:17 (6412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:25:12 (5900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6976, iMonCtr=1 Model crash detected, will try to restart... 23:39:26 (7072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:39:27 (7072): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6396, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6396, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6396, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6396, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6396, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6396, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Jan 2014 15:52:08	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	959,040	582,329	0.6072
15 Jan 2014 03:42:39	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	933,120	566,447	0.6070
14 Jan 2014 13:04:22	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	907,200	550,491	0.6068
14 Jan 2014 04:07:13	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	881,280	534,694	0.6067
14 Jan 2014 00:41:32	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	855,360	519,008	0.6068
14 Jan 2014 00:41:13	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	829,440	503,247	0.6067
13 Jan 2014 13:00:19	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	803,520	487,521	0.6067
13 Jan 2014 01:11:18	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	777,600	471,657	0.6066
12 Jan 2014 22:30:33	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	751,680	455,934	0.6066
12 Jan 2014 02:47:01	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	725,760	440,317	0.6067
11 Jan 2014 21:40:49	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	699,840	424,652	0.6068
11 Jan 2014 03:31:10	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	673,920	408,862	0.6067
11 Jan 2014 02:29:56	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	648,000	393,118	0.6067
10 Jan 2014 01:55:23	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	622,080	377,232	0.6064
09 Jan 2014 06:01:23	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	596,160	361,599	0.6065
09 Jan 2014 02:10:21	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	570,240	345,932	0.6066
08 Jan 2014 19:23:04	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	544,320	330,285	0.6068
08 Jan 2014 05:49:54	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	518,400	314,600	0.6069
08 Jan 2014 02:03:49	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	492,480	298,698	0.6065
07 Jan 2014 20:52:09	1308610	16169673	hadcm3n_odnj_1900_40_008472818_1	466,560	282,875	0.6063