Task 19038889

Name	hadcm3n_xd7e_1980_40_010022639_3
Workunit	10020709
Created	26 Oct 2015, 16:30:47 UTC
Sent	26 Oct 2015, 16:31:02 UTC
Report deadline	25 Jan 2016, 23:58:13 UTC
Received	10 Jan 2016, 13:55:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1287911
Run time	16 days 6 hours 21 min 5 sec
CPU time	12 days 19 hours 25 min 52 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.44 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.6.9</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/xd7eko.pji4c10 Error converting file to netcdf: dataout/xd7eko.pii4c10 Error converting file to netcdf: dataout/xd7eko.pfi4c10 Error converting file to netcdf: dataout/xd7eka.phi4c10 Error converting file to netcdf: dataout/xd7eka.pgi4c10 Error converting file to netcdf: dataout/xd7eka.pei4c10 Error converting file to netcdf: dataout/xd7eka.pdi4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:19:13 (7336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1236, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2004, iMonCtr=1 Model crash detected, will try to restart... 21:09:12 (6952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x778D31A5 read attempt to address 0x40E99051 Engaging BOINC Windows Runtime Debugger... Cannot serialize file Z:\BOINC/projects/climateprediction.net/hadcm3n_xd7e_1980_40_010022639/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Jan 2016 13:57:20	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	777,600	1,106,748	1.4233
06 Jan 2016 16:30:18	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	751,680	1,069,189	1.4224
04 Jan 2016 19:23:23	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	725,760	1,030,655	1.4201
02 Jan 2016 14:03:02	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	699,840	994,556	1.4211
30 Dec 2015 13:07:42	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	673,920	957,551	1.4209
27 Dec 2015 11:16:39	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	648,000	919,902	1.4196
24 Dec 2015 19:37:30	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	622,080	883,446	1.4201
23 Dec 2015 21:07:55	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	596,160	848,363	1.4230
22 Dec 2015 15:44:57	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	570,240	812,312	1.4245
21 Dec 2015 16:30:16	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	544,320	775,777	1.4252
20 Dec 2015 17:29:59	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	518,400	737,878	1.4234
19 Dec 2015 16:53:37	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	492,480	699,841	1.4211
16 Dec 2015 17:21:45	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	466,560	664,117	1.4234
12 Dec 2015 17:47:34	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	440,640	624,785	1.4179
06 Dec 2015 21:34:43	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	414,720	587,372	1.4163
05 Dec 2015 20:10:10	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	388,800	550,950	1.4171
02 Dec 2015 20:45:43	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	362,880	513,141	1.4141
29 Nov 2015 16:04:50	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	336,960	476,936	1.4154
26 Nov 2015 19:23:35	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	311,040	440,364	1.4158
24 Nov 2015 09:32:39	1287911	19038889	hadcm3n_xd7e_1980_40_010022639_3	285,120	404,695	1.4194