Task 16032376

Name	hadcm3n_4gpo_2020_40_008392679_1
Workunit	8543538
Created	23 Sep 2013, 20:30:04 UTC
Sent	23 Sep 2013, 20:31:24 UTC
Report deadline	24 Dec 2013, 3:58:35 UTC
Received	14 Oct 2013, 20:58:17 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1294197
Run time	14 days 9 hours 21 min 50 sec
CPU time	14 days 2 hours 54 min 26 sec
Validate state	Invalid
Credit	7,776.00
Device peak FLOPS	2.69 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 20:06:02 (9744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:38:45 (5164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:38:04 (8356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:38:10 (8356): No heartbeat from core client for 30 sec - exiting 11:38:11 (8356): No heartbeat from core client for 30 sec - exiting 11:39:03 (9988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:35:59 (12008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:36:35 (12140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:45:00 (5420): No heartbeat from core client for 30 sec - exiting 08:45:02 (5420): No heartbeat from core client for 30 sec - exiting 08:45:03 (5420): No heartbeat from core client for 30 sec - exiting 08:45:04 (5420): No heartbeat from core client for 30 sec - exiting 08:45:05 (5420): No heartbeat from core client for 30 sec - exiting 08:45:06 (5420): No heartbeat from core client for 30 sec - exiting 08:45:07 (5420): No heartbeat from core client for 30 sec - exiting 08:45:08 (5420): No heartbeat from core client for 30 sec - exiting 08:45:09 (5420): No heartbeat from core client for 30 sec - exiting 08:45:10 (5420): No heartbeat from core client for 30 sec - exiting 08:45:11 (5420): No heartbeat from core client for 30 sec - exiting 08:45:12 (5420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:14 (5420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6408, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Oct 2013 16:46:46	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	648,000	1,179,963	1.8209
12 Oct 2013 03:02:37	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	622,080	1,133,545	1.8222
11 Oct 2013 14:01:44	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	596,160	1,087,551	1.8243
11 Oct 2013 00:50:45	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	570,240	1,041,508	1.8264
10 Oct 2013 04:23:55	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	544,320	996,936	1.8315
09 Oct 2013 09:17:55	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	518,400	951,190	1.8349
07 Oct 2013 23:08:00	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	492,480	904,899	1.8374
07 Oct 2013 00:24:19	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	466,560	857,929	1.8388
05 Oct 2013 17:46:52	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	440,640	809,609	1.8373
05 Oct 2013 04:42:19	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	414,720	762,815	1.8393
04 Oct 2013 10:22:30	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	388,800	714,720	1.8383
03 Oct 2013 20:04:42	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	362,880	663,946	1.8297
03 Oct 2013 00:01:58	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	336,960	616,530	1.8297
02 Oct 2013 07:43:52	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	311,040	568,723	1.8285
01 Oct 2013 10:41:48	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	285,120	521,733	1.8299
30 Sep 2013 20:50:15	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	259,200	474,353	1.8301
30 Sep 2013 01:52:07	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	233,280	427,172	1.8312
29 Sep 2013 04:32:10	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	207,360	380,022	1.8327
28 Sep 2013 06:53:54	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	181,440	331,740	1.8284
27 Sep 2013 12:08:58	1294197	16032376	hadcm3n_4gpo_2020_40_008392679_1	155,520	283,954	1.8258