Task 15692093

Name	hadcm3n_3k1u_1980_40_008318179_1
Workunit	8469314
Created	29 Mar 2013, 3:26:03 UTC
Sent	29 Mar 2013, 10:37:51 UTC
Report deadline	28 Jun 2013, 18:05:02 UTC
Received	4 Apr 2013, 6:07:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1270025
Run time	5 days 18 hours 2 min 39 sec
CPU time	5 days 1 hours 23 min 45 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.25 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 14:40:08 (15048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:40:09 (15048): No heartbeat from core client for 30 sec - exiting 14:40:10 (15048): No heartbeat from core client for 30 sec - exiting 14:40:11 (15048): No heartbeat from core client for 30 sec - exiting 14:40:12 (15048): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:32:10 (16916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:32:11 (16916): No heartbeat from core client for 30 sec - exiting 17:32:12 (16916): No heartbeat from core client for 30 sec - exiting 17:32:56 (15044): No heartbeat from core client for 30 sec - exiting 17:32:57 (15044): No heartbeat from core client for 30 sec - exiting 17:32:58 (15044): No heartbeat from core client for 30 sec - exiting 17:32:59 (15044): No heartbeat from core client for 30 sec - exiting 17:33:00 (15044): No heartbeat from core client for 30 sec - exiting 17:33:01 (15044): No heartbeat from core client for 30 sec - exiting 17:33:02 (15044): No heartbeat from core client for 30 sec - exiting 17:33:03 (15044): No heartbeat from core client for 30 sec - exiting 17:33:04 (15044): No heartbeat from core client for 30 sec - exiting 17:33:05 (15044): No heartbeat from core client for 30 sec - exiting 17:33:06 (15044): No heartbeat from core client for 30 sec - exiting 17:33:07 (15044): No heartbeat from core client for 30 sec - exiting 17:33:08 (15044): No heartbeat from core client for 30 sec - exiting 17:33:09 (15044): No heartbeat from core client for 30 sec - exiting 17:33:10 (15044): No heartbeat from core client for 30 sec - exiting 17:33:11 (15044): No heartbeat from core client for 30 sec - exiting 17:33:12 (15044): No heartbeat from core client for 30 sec - exiting 17:33:13 (15044): No heartbeat from core client for 30 sec - exiting 17:33:14 (15044): No heartbeat from core client for 30 sec - exiting 17:33:15 (15044): No heartbeat from core client for 30 sec - exiting 17:33:16 (15044): No heartbeat from core client for 30 sec - exiting 17:33:17 (15044): No heartbeat from core client for 30 sec - exiting 17:33:18 (15044): No heartbeat from core client for 30 sec - exiting 17:33:19 (15044): No heartbeat from core client for 30 sec - exiting 17:33:20 (15044): No heartbeat from core client for 30 sec - exiting 17:33:21 (15044): No heartbeat from core client for 30 sec - exiting 17:33:22 (15044): No heartbeat from core client for 30 sec - exiting 17:33:23 (15044): No heartbeat from core client for 30 sec - exiting 17:33:24 (15044): No heartbeat from core client for 30 sec - exiting 17:33:25 (15044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:26 (15044): No heartbeat from core client for 30 sec - exiting 17:33:27 (15044): No heartbeat from core client for 30 sec - exiting 17:33:28 (15044): No heartbeat from core client for 30 sec - exiting 17:34:49 (19120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:03:52 (8228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9160, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9160, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9160, iMonCtr=1 Model crash detected, will try to restart... 23:04:54 (9160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5516, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5516, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5516, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Apr 2013 23:01:44	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	259,200	423,059	1.6322
03 Apr 2013 09:10:43	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	233,280	379,638	1.6274
02 Apr 2013 18:25:21	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	207,360	337,217	1.6262
02 Apr 2013 05:12:15	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	181,440	295,387	1.6280
01 Apr 2013 15:55:55	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	155,520	253,995	1.6332
01 Apr 2013 02:47:22	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	129,600	210,444	1.6238
31 Mar 2013 13:45:08	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	103,680	168,363	1.6239
31 Mar 2013 01:27:10	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	77,760	126,136	1.6221
30 Mar 2013 12:15:30	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	51,840	83,705	1.6147
29 Mar 2013 23:26:12	1270025	15692093	hadcm3n_3k1u_1980_40_008318179_1	25,920	42,529	1.6408