Task 13132509

Name	hadcm3n_ykfi_1900_40_007359336_2
Workunit	7556766
Created	8 Jul 2011, 1:05:32 UTC
Sent	8 Jul 2011, 1:17:15 UTC
Report deadline	7 Oct 2011, 8:44:26 UTC
Received	25 Apr 2012, 3:07:15 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	-197 (0xFFFFFF3B) ERR_ABORTED_VIA_GUI
Computer ID	1095421
Run time	10 days 13 hours 9 min 28 sec
CPU time	9 days 18 hours 17 min 28 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	2.93 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5228, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4352, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5352, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5700, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4132, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4132, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/ykfiko.pja1c10 Error converting file to netcdf: dataout/ykfiko.pia1c10 Error converting file to netcdf: dataout/ykfiko.pfa1c10 Error converting file to netcdf: dataout/ykfika.pha1c10 Error converting file to netcdf: dataout/ykfika.pga1c10 Error converting file to netcdf: dataout/ykfika.pea1c10 Error converting file to netcdf: dataout/ykfika.pda1c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:28:19 (2552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4924, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4440, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1632, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 21:08:39 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:41 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:21:53 (3488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:54 (3488): No heartbeat from core client for 30 sec - exiting 22:21:55 (3488): No heartbeat from core client for 30 sec - exiting 22:21:56 (3488): No heartbeat from core client for 30 sec - exiting 22:21:57 (3488): No heartbeat from core client for 30 sec - exiting 22:21:58 (3488): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4664, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:36:17 (3300): No heartbeat from core client for 30 sec - exiting 11:36:19 (3300): No heartbeat from core client for 30 sec - exiting 11:36:20 (3300): No heartbeat from core client for 30 sec - exiting 11:36:21 (3300): No heartbeat from core client for 30 sec - exiting 11:36:22 (3300): No heartbeat from core client for 30 sec - exiting 11:36:23 (3300): No heartbeat from core client for 30 sec - exiting 11:36:24 (3300): No heartbeat from core client for 30 sec - exiting 11:36:25 (3300): No heartbeat from core client for 30 sec - exiting 11:36:26 (3300): No heartbeat from core client for 30 sec - exiting 11:36:27 (3300): No heartbeat from core client for 30 sec - exiting 11:36:28 (3300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4480, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5072, iMonCtr=1 Model crash detected, will try to restart... 23:29:32 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:04:44 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:54:55 (4436): No heartbeat from core client for 30 sec - exiting 15:54:56 (4436): No heartbeat from core client for 30 sec - exiting 15:54:57 (4436): No heartbeat from core client for 30 sec - exiting 15:54:58 (4436): No heartbeat from core client for 30 sec - exiting 15:54:59 (4436): No heartbeat from core client for 30 sec - exiting 15:55:00 (4436): No heartbeat from core client for 30 sec - exiting 15:55:01 (4436): No heartbeat from core client for 30 sec - exiting 15:55:02 (4436): No heartbeat from core client for 30 sec - exiting 15:55:03 (4436): No heartbeat from core client for 30 sec - exiting 15:55:04 (4436): No heartbeat from core client for 30 sec - exiting 15:55:05 (4436): No heartbeat from core client for 30 sec - exiting 15:55:06 (4436): No heartbeat from core client for 30 sec - exiting 15:55:07 (4436): No heartbeat from core client for 30 sec - exiting 15:55:08 (4436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/ykfiko.pjb1c10 Error converting file to netcdf: dataout/ykfiko.pib1c10 Error converting file to netcdf: dataout/ykfiko.pfb1c10 Error converting file to netcdf: dataout/ykfika.phb1c10 Error converting file to netcdf: dataout/ykfika.pgb1c10 Error converting file to netcdf: dataout/ykfika.peb1c10 CPDN Monitor - Quit request from BOINC... 20:30:09 (4524): No heartbeat from core client for 30 sec - exiting 20:30:10 (4524): No heartbeat from core client for 30 sec - exiting 20:30:11 (4524): No heartbeat from core client for 30 sec - exiting 20:30:12 (4524): No heartbeat from core client for 30 sec - exiting 20:30:13 (4524): No heartbeat from core client for 30 sec - exiting 20:30:14 (4524): No heartbeat from core client for 30 sec - exiting 20:30:15 (4524): No heartbeat from core client for 30 sec - exiting 20:30:16 (4524): No heartbeat from core client for 30 sec - exiting 20:30:17 (4524): No heartbeat from core client for 30 sec - exiting 20:30:18 (4524): No heartbeat from core client for 30 sec - exiting 20:30:19 (4524): No heartbeat from core client for 30 sec - exiting 20:30:20 (4524): No heartbeat from core client for 30 sec - exiting 20:30:22 (4524): No heartbeat from core client for 30 sec - exiting 20:30:23 (4524): No heartbeat from core client for 30 sec - exiting 20:30:24 (4524): No heartbeat from core client for 30 sec - exiting 20:30:25 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4948, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4480, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 20:57:56 (1684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2160, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2160, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2160, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2160, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2160, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 18:24:24 (4820): No heartbeat from core client for 30 sec - exiting 18:24:25 (4820): No heartbeat from core client for 30 sec - exiting 18:24:26 (4820): No heartbeat from core client for 30 sec - exiting 18:24:27 (4820): No heartbeat from core client for 30 sec - exiting 18:24:28 (4820): No heartbeat from core client for 30 sec - exiting 18:24:29 (4820): No heartbeat from core client for 30 sec - exiting 18:24:30 (4820): No heartbeat from core client for 30 sec - exiting 18:24:31 (4820): No heartbeat from core client for 30 sec - exiting 18:24:32 (4820): No heartbeat from core client for 30 sec - exiting 18:24:33 (4820): No heartbeat from core client for 30 sec - exiting 18:24:34 (4820): No heartbeat from core client for 30 sec - exiting 18:24:35 (4820): No heartbeat from core client for 30 sec - exiting 18:24:36 (4820): No heartbeat from core client for 30 sec - exiting 18:24:37 (4820): No heartbeat from core client for 30 sec - exiting 18:24:38 (4820): No heartbeat from core client for 30 sec - exiting 18:24:39 (4820): No heartbeat from core client for 30 sec - exiting 18:24:40 (4820): No heartbeat from core client for 30 sec - exiting 18:24:41 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4248, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5944, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 21:02:10 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5924, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3504, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4448, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5252, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1196, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:08:41 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3944, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Abort request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Oct 2011 00:34:24	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	440,640	803,908	1.8244
30 Sep 2011 01:34:23	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	414,720	759,536	1.8314
25 Sep 2011 16:50:52	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	388,800	716,310	1.8424
20 Sep 2011 01:01:59	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	362,880	674,233	1.8580
18 Sep 2011 05:07:48	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	336,960	627,889	1.8634
29 Aug 2011 01:47:04	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	311,040	584,057	1.8778
28 Aug 2011 00:40:39	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	285,120	532,120	1.8663
21 Aug 2011 21:44:18	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	259,200	485,804	1.8742
21 Aug 2011 06:55:11	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	233,280	438,459	1.8795
19 Aug 2011 05:08:02	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	207,360	389,992	1.8807
17 Aug 2011 11:02:18	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	181,440	342,037	1.8851
13 Aug 2011 01:17:35	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	155,520	288,818	1.8571
11 Aug 2011 04:21:30	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	129,600	238,617	1.8412
06 Aug 2011 12:16:12	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	103,680	190,148	1.8340
01 Aug 2011 21:10:51	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	77,760	141,940	1.8254
30 Jul 2011 18:39:29	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	51,840	94,601	1.8249
25 Jul 2011 17:43:24	1095421	13132509	hadcm3n_ykfi_1900_40_007359336_2	25,920	46,009	1.7750