Task 15914082

Name	hadcm3n_4n10_1980_40_008361198_1
Workunit	8512057
Created	14 Aug 2013, 11:38:19 UTC
Sent	14 Aug 2013, 23:01:25 UTC
Report deadline	14 Nov 2013, 6:28:36 UTC
Received	26 Sep 2013, 22:39:39 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1006742
Run time	23 days 20 hours 24 min 34 sec
CPU time	22 days 3 hours 56 min 3 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	1.26 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 19:22:21 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2140, selfPID=2140, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4n10ko.pji4c10 Error converting file to netcdf: dataout/4n10ko.pii4c10 Error converting file to netcdf: dataout/4n10ko.pfi4c10 Error converting file to netcdf: dataout/4n10ka.phi4c10 Error converting file to netcdf: dataout/4n10ka.pgi4c10 Error converting file to netcdf: dataout/4n10ka.pei4c10 Error converting file to netcdf: dataout/4n10ka.pdi4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3648, selfPID=3648, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3792, selfPID=3792, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4n10ko.pji8c10 Error converting file to netcdf: dataout/4n10ko.pii8c10 Error converting file to netcdf: dataout/4n10ko.pfi8c10 Error converting file to netcdf: dataout/4n10ka.phi8c10 19:15:09 (880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:15:10 (880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=412, selfPID=412, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4n10ko.pjj3c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:49:37 (1220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:08:21 (3232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:38:48 (2820): No heartbeat from core client for 30 sec - exiting 09:38:49 (2820): No heartbeat from core client for 30 sec - exiting 09:38:50 (2820): No heartbeat from core client for 30 sec - exiting 09:38:52 (2820): No heartbeat from core client for 30 sec - exiting 09:38:53 (2820): No heartbeat from core client for 30 sec - exiting 09:38:54 (2820): No heartbeat from core client for 30 sec - exiting 09:38:55 (2820): No heartbeat from core client for 30 sec - exiting 09:38:56 (2820): No heartbeat from core client for 30 sec - exiting 09:38:57 (2820): No heartbeat from core client for 30 sec - exiting 09:38:58 (2820): No heartbeat from core client for 30 sec - exiting 09:38:59 (2820): No heartbeat from core client for 30 sec - exiting 09:39:00 (2820): No heartbeat from core client for 30 sec - exiting 09:39:01 (2820): No heartbeat from core client for 30 sec - exiting 09:39:02 (2820): No heartbeat from core client for 30 sec - exiting 09:39:04 (2820): No heartbeat from core client for 30 sec - exiting 09:39:05 (2820): No heartbeat from core client for 30 sec - exiting 09:39:06 (2820): No heartbeat from core client for 30 sec - exiting 09:39:07 (2820): No heartbeat from core client for 30 sec - exiting 09:39:08 (2820): No heartbeat from core client for 30 sec - exiting 09:39:09 (2820): No heartbeat from core client for 30 sec - exiting 09:39:10 (2820): No heartbeat from core client for 30 sec - exiting 09:39:11 (2820): No heartbeat from core client for 30 sec - exiting 09:39:12 (2820): No heartbeat from core client for 30 sec - exiting 09:39:13 (2820): No heartbeat from core client for 30 sec - exiting 09:39:15 (2820): No heartbeat from core client for 30 sec - exiting 09:39:16 (2820): No heartbeat from core client for 30 sec - exiting 09:39:17 (2820): No heartbeat from core client for 30 sec - exiting 09:39:18 (2820): No heartbeat from core client for 30 sec - exiting 09:39:19 (2820): No heartbeat from core client for 30 sec - exiting 09:39:20 (2820): No heartbeat from core client for 30 sec - exiting 09:39:21 (2820): No heartbeat from core client for 30 sec - exiting 09:39:22 (2820): No heartbeat from core client for 30 sec - exiting 09:39:23 (2820): No heartbeat from core client for 30 sec - exiting 09:39:24 (2820): No heartbeat from core client for 30 sec - exiting 09:39:25 (2820): No heartbeat from core client for 30 sec - exiting 09:39:27 (2820): No heartbeat from core client for 30 sec - exiting 09:39:28 (2820): No heartbeat from core client for 30 sec - exiting 09:39:29 (2820): No heartbeat from core client for 30 sec - exiting 09:39:30 (2820): No heartbeat from core client for 30 sec - exiting 09:39:31 (2820): No heartbeat from core client for 30 sec - exiting 09:39:32 (2820): No heartbeat from core client for 30 sec - exiting 09:39:33 (2820): No heartbeat from core client for 30 sec - exiting 09:39:34 (2820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4n10ko.pjj7c10 Error converting file to netcdf: dataout/4n10ko.pij7c10 Error converting file to netcdf: dataout/4n10ko.pfj7c10 Error converting file to netcdf: dataout/4n10ka.phj7c10 Error converting file to netcdf: dataout/4n10ka.pgj7c10 Error converting file to netcdf: dataout/4n10ka.pej7c10 Error converting file to netcdf: dataout/4n10ka.pdj7c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:40:10 (3424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Sep 2013 17:46:05	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	518,400	1,914,955	3.6940
23 Sep 2013 09:31:03	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	492,480	1,810,519	3.6763
20 Sep 2013 02:21:59	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	466,560	1,720,059	3.6867
18 Sep 2013 01:33:13	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	440,640	1,619,609	3.6756
15 Sep 2013 04:02:39	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	414,720	1,515,413	3.6541
12 Sep 2013 06:33:43	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	388,800	1,413,425	3.6354
10 Sep 2013 07:13:50	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	362,880	1,325,868	3.6537
09 Sep 2013 01:21:26	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	336,960	1,239,191	3.6776
07 Sep 2013 06:23:25	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	311,040	1,153,430	3.7083
05 Sep 2013 06:07:27	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	285,120	1,063,327	3.7294
03 Sep 2013 13:32:29	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	259,200	973,909	3.7574
02 Sep 2013 02:01:31	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	233,280	885,824	3.7973
30 Aug 2013 08:12:40	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	207,360	787,220	3.7964
28 Aug 2013 06:42:33	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	181,440	678,536	3.7397
26 Aug 2013 10:34:48	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	155,520	583,108	3.7494
24 Aug 2013 20:55:52	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	129,600	492,245	3.7982
23 Aug 2013 10:11:45	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	103,680	387,111	3.7337
21 Aug 2013 05:00:53	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	77,760	299,179	3.8475
19 Aug 2013 17:56:45	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	51,840	210,786	4.0661
17 Aug 2013 08:04:26	1006742	15914082	hadcm3n_4n10_1980_40_008361198_1	25,920	101,208	3.9046