Task 15684234

Name	hadcm3n_u0zy_2020_40_008336082_0
Workunit	8486943
Created	26 Mar 2013, 19:38:08 UTC
Sent	26 Mar 2013, 19:38:18 UTC
Report deadline	26 Jun 2013, 3:05:29 UTC
Received	29 Apr 2013, 11:59:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1254204
Run time	9 days 1 hours 40 min 19 sec
CPU time	8 days 7 hours 18 min 30 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	2.45 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5216, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5828, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5312, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5352, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5596, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5368, iMonCtr=1 Model crash detected, will try to restart... 15:08:58 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:08:59 (4420): No heartbeat from core client for 30 sec - exiting 15:09:00 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:53:37 (6028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2392, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4052, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5524, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5960, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5144, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1836, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 12:26:15 (5292): No heartbeat from core client for 30 sec - exiting 12:26:21 (5292): No heartbeat from core client for 30 sec - exiting 12:26:22 (5292): No heartbeat from core client for 30 sec - exiting 12:26:23 (5292): No heartbeat from core client for 30 sec - exiting 12:26:24 (5292): No heartbeat from core client for 30 sec - exiting 12:26:25 (5292): No heartbeat from core client for 30 sec - exiting 12:26:26 (5292): No heartbeat from core client for 30 sec - exiting 12:26:27 (5292): No heartbeat from core client for 30 sec - exiting 12:26:28 (5292): No heartbeat from core client for 30 sec - exiting 12:26:29 (5292): No heartbeat from core client for 30 sec - exiting 12:26:30 (5292): No heartbeat from core client for 30 sec - exiting 12:26:31 (5292): No heartbeat from core client for 30 sec - exiting 12:26:32 (5292): No heartbeat from core client for 30 sec - exiting 12:26:33 (5292): No heartbeat from core client for 30 sec - exiting 12:26:34 (5292): No heartbeat from core client for 30 sec - exiting 12:26:35 (5292): No heartbeat from core client for 30 sec - exiting 12:26:36 (5292): No heartbeat from core client for 30 sec - exiting 12:26:37 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:26:38 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2700, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5612, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5612, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5612, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Apr 2013 15:05:41	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	414,720	693,918	1.6732
24 Apr 2013 16:56:13	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	388,800	650,507	1.6731
19 Apr 2013 07:13:07	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	362,880	607,211	1.6733
16 Apr 2013 16:41:16	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	336,960	560,333	1.6629
15 Apr 2013 14:12:05	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	311,040	518,163	1.6659
13 Apr 2013 18:54:30	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	285,120	474,974	1.6659
09 Apr 2013 15:56:10	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	259,200	431,841	1.6661
08 Apr 2013 12:13:22	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	233,280	388,787	1.6666
07 Apr 2013 10:38:29	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	207,360	346,004	1.6686
06 Apr 2013 01:50:44	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	181,440	302,354	1.6664
05 Apr 2013 10:35:31	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	155,520	259,521	1.6687
04 Apr 2013 09:41:00	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	129,600	217,403	1.6775
02 Apr 2013 20:34:37	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	103,680	174,997	1.6879
31 Mar 2013 16:51:14	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	77,760	130,616	1.6797
30 Mar 2013 15:25:36	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	51,840	88,175	1.7009
28 Mar 2013 13:05:47	1254204	15684234	hadcm3n_u0zy_2020_40_008336082_0	25,920	44,310	1.7095