Task 14689445

Name	hadcm3n_o5uu_2060_40_007959649_3
Workunit	8114763
Created	16 May 2012, 17:43:28 UTC
Sent	16 May 2012, 17:43:40 UTC
Report deadline	16 Aug 2012, 1:10:51 UTC
Received	11 Jun 2012, 19:30:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1167875
Run time	1 days 1 hours 14 min 14 sec
CPU time	23 hours 46 min 9 sec
Validate state	Invalid
Credit	933.12
Device peak FLOPS	4.14 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:36:31 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:26:13 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:39:56 (1092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:45:34 (3012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:50:44 (4396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:14:22 (5488): No heartbeat from core client for 30 sec - exiting 14:14:24 (5488): No heartbeat from core client for 30 sec - exiting 14:14:25 (5488): No heartbeat from core client for 30 sec - exiting 14:14:26 (5488): No heartbeat from core client for 30 sec - exiting 14:14:27 (5488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:34:17 (3980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:34:19 (3980): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:53:05 (6000): No heartbeat from core client for 30 sec - exiting 15:53:06 (6000): No heartbeat from core client for 30 sec - exiting 15:53:07 (6000): No heartbeat from core client for 30 sec - exiting 15:53:08 (6000): No heartbeat from core client for 30 sec - exiting 15:53:09 (6000): No heartbeat from core client for 30 sec - exiting 15:53:10 (6000): No heartbeat from core client for 30 sec - exiting 15:53:11 (6000): No heartbeat from core client for 30 sec - exiting 15:53:12 (6000): No heartbeat from core client for 30 sec - exiting 15:53:13 (6000): No heartbeat from core client for 30 sec - exiting 15:53:14 (6000): No heartbeat from core client for 30 sec - exiting 15:53:15 (6000): No heartbeat from core client for 30 sec - exiting 15:53:16 (6000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:53:17 (6000): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:13:02 (1456): No heartbeat from core client for 30 sec - exiting 13:13:03 (1456): No heartbeat from core client for 30 sec - exiting 13:13:04 (1456): No heartbeat from core client for 30 sec - exiting 13:13:05 (1456): No heartbeat from core client for 30 sec - exiting 13:13:06 (1456): No heartbeat from core client for 30 sec - exiting 13:13:07 (1456): No heartbeat from core client for 30 sec - exiting 13:13:08 (1456): No heartbeat from core client for 30 sec - exiting 13:13:09 (1456): No heartbeat from core client for 30 sec - exiting 13:13:10 (1456): No heartbeat from core client for 30 sec - exiting 13:13:11 (1456): No heartbeat from core client for 30 sec - exiting 13:13:12 (1456): No heartbeat from core client for 30 sec - exiting 13:13:13 (1456): No heartbeat from core client for 30 sec - exiting 13:13:14 (1456): No heartbeat from core client for 30 sec - exiting 13:13:15 (1456): No heartbeat from core client for 30 sec - exiting 13:13:16 (1456): No heartbeat from core client for 30 sec - exiting 13:13:17 (1456): No heartbeat from core client for 30 sec - exiting 13:13:18 (1456): No heartbeat from core client for 30 sec - exiting 13:13:19 (1456): No heartbeat from core client for 30 sec - exiting 13:13:20 (1456): No heartbeat from core client for 30 sec - exiting 13:13:21 (1456): No heartbeat from core client for 30 sec - exiting 13:13:22 (1456): No heartbeat from core client for 30 sec - exiting 13:13:23 (1456): No heartbeat from core client for 30 sec - exiting 13:13:24 (1456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:14 (5956): No heartbeat from core client for 30 sec - exiting 16:22:16 (5956): No heartbeat from core client for 30 sec - exiting 16:22:17 (5956): No heartbeat from core client for 30 sec - exiting 16:22:18 (5956): No heartbeat from core client for 30 sec - exiting 16:22:19 (5956): No heartbeat from core client for 30 sec - exiting 16:22:20 (5956): No heartbeat from core client for 30 sec - exiting 16:22:21 (5956): No heartbeat from core client for 30 sec - exiting 16:22:22 (5956): No heartbeat from core client for 30 sec - exiting 16:22:23 (5956): No heartbeat from core client for 30 sec - exiting 16:22:24 (5956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:27:26 (5840): No heartbeat from core client for 30 sec - exiting 19:27:27 (5840): No heartbeat from core client for 30 sec - exiting 19:27:28 (5840): No heartbeat from core client for 30 sec - exiting 19:27:29 (5840): No heartbeat from core client for 30 sec - exiting 19:27:30 (5840): No heartbeat from core client for 30 sec - exiting 19:27:31 (5840): No heartbeat from core client for 30 sec - exiting 19:27:32 (5840): No heartbeat from core client for 30 sec - exiting 19:27:33 (5840): No heartbeat from core client for 30 sec - exiting 19:27:34 (5840): No heartbeat from core client for 30 sec - exiting 19:27:35 (5840): No heartbeat from core client for 30 sec - exiting 19:27:36 (5840): No heartbeat from core client for 30 sec - exiting 19:27:37 (5840): No heartbeat from core client for 30 sec - exiting 19:27:38 (5840): No heartbeat from core client for 30 sec - exiting 19:27:39 (5840): No heartbeat from core client for 30 sec - exiting 19:27:40 (5840): No heartbeat from core client for 30 sec - exiting 19:27:41 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Jun 2012 16:42:47	1167875	14689445	hadcm3n_o5uu_2060_40_007959649_3	77,760	68,090	0.8756
07 Jun 2012 14:21:17	1167875	14689445	hadcm3n_o5uu_2060_40_007959649_3	51,840	44,013	0.8490
05 Jun 2012 14:13:25	1167875	14689445	hadcm3n_o5uu_2060_40_007959649_3	25,920	20,418	0.7877