Task 15461021

Name	hadcm3n_z7r6_1880_40_008200193_4
Workunit	8355317
Created	26 Nov 2012, 2:47:32 UTC
Sent	26 Nov 2012, 2:47:38 UTC
Report deadline	25 Feb 2013, 10:14:49 UTC
Received	18 Dec 2012, 14:14:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1065718
Run time	5 days 21 hours 26 min 45 sec
CPU time	5 days 9 hours 23 min 1 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.67 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/z7r6ko.pj83c10 Error converting file to netcdf: dataout/z7r6ko.pi83c10 Error converting file to netcdf: dataout/z7r6ko.pf83c10 Error converting file to netcdf: dataout/z7r6ka.ph83c10 Error converting file to netcdf: dataout/z7r6ka.pg83c10 Error converting file to netcdf: dataout/z7r6ka.pe83c10 Error converting file to netcdf: dataout/z7r6ka.pd83c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:10:09 (1464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:10:13 (1464): No heartbeat from core client for 30 sec - exiting 21:10:14 (1464): No heartbeat from core client for 30 sec - exiting 21:10:15 (1464): No heartbeat from core client for 30 sec - exiting 21:10:16 (1464): No heartbeat from core client for 30 sec - exiting 21:10:17 (1464): No heartbeat from core client for 30 sec - exiting 21:11:13 (10900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:14:30 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:50:47 (9732): No heartbeat from core client for 30 sec - exiting 03:50:49 (9732): No heartbeat from core client for 30 sec - exiting 03:50:50 (9732): No heartbeat from core client for 30 sec - exiting 03:50:51 (9732): No heartbeat from core client for 30 sec - exiting 03:50:52 (9732): No heartbeat from core client for 30 sec - exiting 03:50:53 (9732): No heartbeat from core client for 30 sec - exiting 03:50:54 (9732): No heartbeat from core client for 30 sec - exiting 03:50:55 (9732): No heartbeat from core client for 30 sec - exiting 03:50:57 (9732): No heartbeat from core client for 30 sec - exiting 03:50:58 (9732): No heartbeat from core client for 30 sec - exiting 03:50:59 (9732): No heartbeat from core client for 30 sec - exiting 03:51:00 (9732): No heartbeat from core client for 30 sec - exiting 03:51:01 (9732): No heartbeat from core client for 30 sec - exiting 03:51:02 (9732): No heartbeat from core client for 30 sec - exiting 03:51:03 (9732): No heartbeat from core client for 30 sec - exiting 03:51:04 (9732): No heartbeat from core client for 30 sec - exiting 03:51:05 (9732): No heartbeat from core client for 30 sec - exiting 03:51:06 (9732): No heartbeat from core client for 30 sec - exiting 03:51:07 (9732): No heartbeat from core client for 30 sec - exiting 03:51:09 (9732): No heartbeat from core client for 30 sec - exiting 03:51:10 (9732): No heartbeat from core client for 30 sec - exiting 03:51:11 (9732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:51:12 (9732): No heartbeat from core client for 30 sec - exiting 03:54:48 (8756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:54:51 (8756): No heartbeat from core client for 30 sec - exiting 03:54:53 (8756): No heartbeat from core client for 30 sec - exiting 03:54:54 (8756): No heartbeat from core client for 30 sec - exiting 04:00:50 (7144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:00:51 (7144): No heartbeat from core client for 30 sec - exiting 04:05:19 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:05:23 (6080): No heartbeat from core client for 30 sec - exiting 04:05:24 (6080): No heartbeat from core client for 30 sec - exiting 04:05:25 (6080): No heartbeat from core client for 30 sec - exiting 04:05:26 (6080): No heartbeat from core client for 30 sec - exiting 04:05:27 (6080): No heartbeat from core client for 30 sec - exiting 04:05:28 (6080): No heartbeat from core client for 30 sec - exiting 04:05:30 (6080): No heartbeat from core client for 30 sec - exiting 04:10:06 (872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:10:12 (872): No heartbeat from core client for 30 sec - exiting 04:10:13 (872): No heartbeat from core client for 30 sec - exiting 04:10:15 (872): No heartbeat from core client for 30 sec - exiting 04:10:16 (872): No heartbeat from core client for 30 sec - exiting 04:10:17 (872): No heartbeat from core client for 30 sec - exiting 04:11:22 (10788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:49 (7068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:51 (7068): No heartbeat from core client for 30 sec - exiting 04:21:28 (3860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:21:31 (3860): No heartbeat from core client for 30 sec - exiting 04:26:34 (9292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:26:36 (9292): No heartbeat from core client for 30 sec - exiting 04:26:37 (9292): No heartbeat from core client for 30 sec - exiting 04:26:38 (9292): No heartbeat from core client for 30 sec - exiting 04:26:40 (9292): No heartbeat from core client for 30 sec - exiting 04:32:15 (3904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:32:17 (3904): No heartbeat from core client for 30 sec - exiting 04:32:18 (3904): No heartbeat from core client for 30 sec - exiting 04:32:21 (3904): No heartbeat from core client for 30 sec - exiting 04:32:22 (3904): No heartbeat from core client for 30 sec - exiting 04:32:23 (3904): No heartbeat from core client for 30 sec - exiting 04:32:24 (3904): No heartbeat from core client for 30 sec - exiting 04:33:29 (5256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:37:30 (7236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:37:37 (7236): No heartbeat from core client for 30 sec - exiting 04:37:38 (7236): No heartbeat from core client for 30 sec - exiting 04:43:25 (10052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:43:26 (10052): No heartbeat from core client for 30 sec - exiting 04:43:27 (10052): No heartbeat from core client for 30 sec - exiting 04:43:28 (10052): No heartbeat from core client for 30 sec - exiting 04:43:29 (10052): No heartbeat from core client for 30 sec - exiting 04:48:13 (6892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:48:15 (6892): No heartbeat from core client for 30 sec - exiting 04:48:16 (6892): No heartbeat from core client for 30 sec - exiting 04:48:17 (6892): No heartbeat from core client for 30 sec - exiting 04:48:18 (6892): No heartbeat from core client for 30 sec - exiting 04:52:29 (10516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:30 (10516): No heartbeat from core client for 30 sec - exiting 04:52:31 (10516): No heartbeat from core client for 30 sec - exiting 04:52:32 (10516): No heartbeat from core client for 30 sec - exiting 04:52:34 (10516): No heartbeat from core client for 30 sec - exiting 04:52:35 (10516): No heartbeat from core client for 30 sec - exiting 04:56:47 (8340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:56:53 (8340): No heartbeat from core client for 30 sec - exiting 04:56:54 (8340): No heartbeat from core client for 30 sec - exiting 04:56:55 (8340): No heartbeat from core client for 30 sec - exiting 04:56:56 (8340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 05:46:10 (10896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:49:50 (8888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:49:55 (8888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:36:04 (13000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1120, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A23AB3 read attempt to address 0x40BBEDFB Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A23AB3 read attempt to address 0x40BBEDFB Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Dec 2012 01:35:26	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	259,200	464,005	1.7901
16 Dec 2012 17:16:37	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	233,280	419,383	1.7978
14 Dec 2012 16:47:11	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	207,360	371,827	1.7931
14 Dec 2012 16:47:11	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	181,440	324,497	1.7885
14 Dec 2012 16:47:11	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	155,520	278,101	1.7882
14 Dec 2012 16:47:11	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	129,600	230,467	1.7783
06 Dec 2012 12:59:17	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	103,680	183,473	1.7696
04 Dec 2012 07:21:51	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	77,760	138,094	1.7759
30 Nov 2012 14:08:55	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	51,840	92,793	1.7900
30 Nov 2012 00:04:49	1065718	15461021	hadcm3n_z7r6_1880_40_008200193_4	25,920	47,356	1.8270