Task 16481355

Name	hadcm3n_oabo_1900_40_008468503_4
Workunit	8619342
Created	5 Apr 2014, 21:24:34 UTC
Sent	5 Apr 2014, 21:30:40 UTC
Report deadline	6 Jul 2014, 4:57:51 UTC
Received	27 Apr 2014, 14:53:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1242215
Run time	10 days 7 hours 2 min 41 sec
CPU time	9 days 8 hours 5 min 53 sec
Validate state	Invalid
Credit	8,709.12
Device peak FLOPS	3.56 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 17:41:43 (184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:42:55 (7648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:55:18 (1396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:57:02 (6452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:59:08 (6284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:49 (5380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:51 (5380): No heartbeat from core client for 30 sec - exiting 07:12:52 (5380): No heartbeat from core client for 30 sec - exiting 07:12:53 (5380): No heartbeat from core client for 30 sec - exiting 07:12:54 (5380): No heartbeat from core client for 30 sec - exiting 09:07:44 (5608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:14:10 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:27:23 (5072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:06 (6796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:14:19 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:14:23 (5140): No heartbeat from core client for 30 sec - exiting 08:14:24 (5140): No heartbeat from core client for 30 sec - exiting 08:14:25 (5140): No heartbeat from core client for 30 sec - exiting 08:14:26 (5140): No heartbeat from core client for 30 sec - exiting 08:14:27 (5140): No heartbeat from core client for 30 sec - exiting 08:14:28 (5140): No heartbeat from core client for 30 sec - exiting 08:14:29 (5140): No heartbeat from core client for 30 sec - exiting 08:14:30 (5140): No heartbeat from core client for 30 sec - exiting 08:14:31 (5140): No heartbeat from core client for 30 sec - exiting 19:12:15 (6508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:14:28 (2624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:04:21 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:49:17 (4928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Apr 2014 14:57:28	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	725,760	793,660	1.0936
27 Apr 2014 01:02:09	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	699,840	764,302	1.0921
26 Apr 2014 17:24:34	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	673,920	735,296	1.0911
26 Apr 2014 04:27:37	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	648,000	706,332	1.0900
24 Apr 2014 00:53:20	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	622,080	677,145	1.0885
20 Apr 2014 22:13:02	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	596,160	647,993	1.0869
20 Apr 2014 14:05:17	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	570,240	618,810	1.0852
20 Apr 2014 12:23:46	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	544,320	590,038	1.0840
19 Apr 2014 23:21:30	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	518,400	561,961	1.0840
19 Apr 2014 23:21:30	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	492,480	533,629	1.0836
19 Apr 2014 03:38:35	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	466,560	504,946	1.0823
18 Apr 2014 20:57:14	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	440,640	479,635	1.0885
18 Apr 2014 20:57:14	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	414,720	452,161	1.0903
18 Apr 2014 20:57:14	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	388,800	423,581	1.0895
16 Apr 2014 00:14:07	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	362,880	395,613	1.0902
14 Apr 2014 02:19:19	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	336,960	367,887	1.0918
14 Apr 2014 02:19:19	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	311,040	340,164	1.0936
14 Apr 2014 02:19:19	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	285,120	310,971	1.0907
14 Apr 2014 02:19:19	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	259,200	282,307	1.0891
14 Apr 2014 02:19:19	1242215	16481355	hadcm3n_oabo_1900_40_008468503_4	233,280	253,542	1.0869