Task 15767651

Name	hadcm3n_4kdn_1940_40_008305076_1
Workunit	8456211
Created	9 May 2013, 12:23:38 UTC
Sent	9 May 2013, 12:23:45 UTC
Report deadline	8 Aug 2013, 19:50:56 UTC
Received	24 Sep 2013, 8:18:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1122348
Run time	18 days 23 hours 25 min 27 sec
CPU time	18 days 18 hours 39 min 31 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6588, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6224, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6188, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4kdnko.pjf6c10 Error converting file to netcdf: dataout/4kdnko.pif6c10 Error converting file to netcdf: dataout/4kdnko.pff6c10 Error converting file to netcdf: dataout/4kdnka.phf6c10 Error converting file to netcdf: dataout/4kdnka.pgf6c10 Error converting file to netcdf: dataout/4kdnka.pef6c10 Error converting file to netcdf: dataout/4kdnka.pdf6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:00:15 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3572, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3152, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5240, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5240, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4292, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5344, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:02:09 (6708): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5460, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5636, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 16:28:55 (5500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:05:21 (4252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:07:03 (7368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:11:40 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77653AC3 read attempt to address 0x40DC9EF4 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x774F3AC3 read attempt to address 0x40DC9EF4 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Sep 2013 08:18:58	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	777,600	1,608,911	2.0691
24 Sep 2013 08:18:58	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	751,680	1,558,154	2.0729
21 Sep 2013 20:36:19	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	725,760	1,503,870	2.0721
15 Sep 2013 03:17:30	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	699,840	1,449,685	2.0715
30 Aug 2013 03:24:25	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	673,920	1,408,292	2.0897
15 Aug 2013 04:44:07	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	648,000	1,360,783	2.1000
15 Aug 2013 00:02:25	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	622,080	1,306,621	2.1004
26 Jul 2013 00:14:03	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	596,160	1,251,731	2.0997
18 Jun 2013 04:11:42	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	570,240	1,197,754	2.1004
14 Jun 2013 02:39:47	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	544,320	1,143,537	2.1009
12 Jun 2013 10:02:05	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	518,400	1,089,899	2.1024
11 Jun 2013 03:46:10	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	492,480	1,035,567	2.1028
09 Jun 2013 19:54:53	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	466,560	981,022	2.1027
09 Jun 2013 00:26:24	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	440,640	926,362	2.1023
07 Jun 2013 23:49:21	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	414,720	872,561	2.1040
06 Jun 2013 01:07:52	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	388,800	818,306	2.1047
01 Jun 2013 05:01:07	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	362,880	763,979	2.1053
29 May 2013 02:40:52	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	336,960	710,082	2.1073
27 May 2013 03:32:05	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	311,040	656,090	2.1093
26 May 2013 12:31:15	1122348	15767651	hadcm3n_4kdn_1940_40_008305076_1	285,120	602,149	2.1119