Task 15073289

Name	hadcm3n_100j_1980_40_007955748_4
Workunit	8110860
Created	8 Aug 2012, 6:12:44 UTC
Sent	8 Aug 2012, 6:12:50 UTC
Report deadline	7 Nov 2012, 13:40:01 UTC
Received	9 Oct 2012, 15:17:04 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1156294
Run time	25 days 3 hours 15 min 20 sec
CPU time	20 days 16 hours 33 min 40 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.41 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3716, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3900, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3924, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4468, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=792, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3140, iMonCtr=1 Model crash detected, will try to restart... 13:35:11 (3104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/100jko.pji4c10 Error converting file to netcdf: dataout/100jko.pii4c10 Error converting file to netcdf: dataout/100jko.pfi4c10 Error converting file to netcdf: dataout/100jka.phi4c10 Error converting file to netcdf: dataout/100jka.pgi4c10 Error converting file to netcdf: dataout/100jka.pei4c10 Error converting file to netcdf: dataout/100jka.pdi4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:20:44 (5496): No heartbeat from core client for 30 sec - exiting 14:20:45 (5496): No heartbeat from core client for 30 sec - exiting 14:20:46 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:53:57 (4164): No heartbeat from core client for 30 sec - exiting 11:53:58 (4164): No heartbeat from core client for 30 sec - exiting 11:53:59 (4164): No heartbeat from core client for 30 sec - exiting 11:54:00 (4164): No heartbeat from core client for 30 sec - exiting 11:54:01 (4164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3436, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4756, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3320, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1512, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4564, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4008, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4604, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4604, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4148, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3816, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:07:41 (5288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4144, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3844, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2744, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3124, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Oct 2012 14:16:36	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	518,400	1,787,618	3.4483
05 Oct 2012 20:23:33	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	492,480	1,684,291	3.4200
01 Oct 2012 11:00:50	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	466,560	1,580,568	3.3877
27 Sep 2012 05:23:25	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	440,640	1,485,258	3.3707
23 Sep 2012 16:28:20	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	414,720	1,382,486	3.3335
19 Sep 2012 20:17:34	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	388,800	1,283,811	3.3020
16 Sep 2012 17:30:18	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	362,880	1,181,538	3.2560
13 Sep 2012 12:58:15	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	336,960	1,083,361	3.2151
09 Sep 2012 16:33:23	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	311,040	990,038	3.1830
06 Sep 2012 18:04:08	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	285,120	896,558	3.1445
03 Sep 2012 18:59:23	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	259,200	803,230	3.0989
31 Aug 2012 12:16:10	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	233,280	714,311	3.0620
29 Aug 2012 07:55:08	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	207,360	635,218	3.0634
26 Aug 2012 11:40:47	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	181,440	554,462	3.0559
23 Aug 2012 13:57:34	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	155,520	479,031	3.0802
20 Aug 2012 18:32:03	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	129,600	381,691	2.9451
17 Aug 2012 09:50:06	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	103,680	292,804	2.8241
14 Aug 2012 10:41:51	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	77,760	210,374	2.7054
12 Aug 2012 14:41:47	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	51,840	143,323	2.7647
10 Aug 2012 07:31:35	1156294	15073289	hadcm3n_100j_1980_40_007955748_4	25,920	78,500	3.0285