Task 15806970

Name	hadcm3n_n3dq_1920_40_008377526_0
Workunit	8528385
Created	30 May 2013, 12:08:34 UTC
Sent	30 May 2013, 12:20:25 UTC
Report deadline	29 Aug 2013, 19:47:36 UTC
Received	18 Jun 2013, 1:06:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1110511
Run time	6 days 15 hours 37 min 26 sec
CPU time	6 days 9 hours 51 min 30 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4512, iMonCtr=1 Model crash detected, will try to restart... 07:22:22 (4184): No heartbeat from core client for 30 sec - exiting 07:22:24 (4184): No heartbeat from core client for 30 sec - exiting 07:22:25 (4184): No heartbeat from core client for 30 sec - exiting 07:22:26 (4184): No heartbeat from core client for 30 sec - exiting 07:22:27 (4184): No heartbeat from core client for 30 sec - exiting 07:22:28 (4184): No heartbeat from core client for 30 sec - exiting 07:22:29 (4184): No heartbeat from core client for 30 sec - exiting 07:22:30 (4184): No heartbeat from core client for 30 sec - exiting 07:22:31 (4184): No heartbeat from core client for 30 sec - exiting 07:22:32 (4184): No heartbeat from core client for 30 sec - exiting 07:22:33 (4184): No heartbeat from core client for 30 sec - exiting 07:22:34 (4184): No heartbeat from core client for 30 sec - exiting 07:22:36 (4184): No heartbeat from core client for 30 sec - exiting 07:22:37 (4184): No heartbeat from core client for 30 sec - exiting 07:22:38 (4184): No heartbeat from core client for 30 sec - exiting 07:22:39 (4184): No heartbeat from core client for 30 sec - exiting 07:22:40 (4184): No heartbeat from core client for 30 sec - exiting 07:22:41 (4184): No heartbeat from core client for 30 sec - exiting 07:22:42 (4184): No heartbeat from core client for 30 sec - exiting 07:22:43 (4184): No heartbeat from core client for 30 sec - exiting 07:22:44 (4184): No heartbeat from core client for 30 sec - exiting 07:22:45 (4184): No heartbeat from core client for 30 sec - exiting 07:22:46 (4184): No heartbeat from core client for 30 sec - exiting 07:22:48 (4184): No heartbeat from core client for 30 sec - exiting 07:22:49 (4184): No heartbeat from core client for 30 sec - exiting 07:22:50 (4184): No heartbeat from core client for 30 sec - exiting 07:22:51 (4184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:46:06 (3172): No heartbeat from core client for 30 sec - exiting 08:46:07 (3172): No heartbeat from core client for 30 sec - exiting 08:46:08 (3172): No heartbeat from core client for 30 sec - exiting 08:46:09 (3172): No heartbeat from core client for 30 sec - exiting 08:46:10 (3172): No heartbeat from core client for 30 sec - exiting 08:46:11 (3172): No heartbeat from core client for 30 sec - exiting 08:46:12 (3172): No heartbeat from core client for 30 sec - exiting 08:46:14 (3172): No heartbeat from core client for 30 sec - exiting 08:46:15 (3172): No heartbeat from core client for 30 sec - exiting 08:46:16 (3172): No heartbeat from core client for 30 sec - exiting 08:46:17 (3172): No heartbeat from core client for 30 sec - exiting 08:46:18 (3172): No heartbeat from core client for 30 sec - exiting 08:46:19 (3172): No heartbeat from core client for 30 sec - exiting 08:46:20 (3172): No heartbeat from core client for 30 sec - exiting 08:46:21 (3172): No heartbeat from core client for 30 sec - exiting 08:46:22 (3172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2892, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:39:40 (5452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n3dqko.pjc8c10 Error converting file to netcdf: dataout/n3dqko.pic8c10 Error converting file to netcdf: dataout/n3dqko.pfc8c10 Error converting file to netcdf: dataout/n3dqka.phc8c10 Error converting file to netcdf: dataout/n3dqka.pgc8c10 Error converting file to netcdf: dataout/n3dqka.pec8c10 Error converting file to netcdf: dataout/n3dqka.pdc8c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776E3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776E7373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776E7373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776E7373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776E7373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776E3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Jun 2013 14:08:05	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	259,200	547,506	2.1123
15 Jun 2013 15:52:39	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	233,280	491,917	2.1087
14 Jun 2013 14:00:46	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	207,360	436,693	2.1060
13 Jun 2013 02:41:30	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	181,440	381,566	2.1030
11 Jun 2013 21:39:54	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	155,520	326,990	2.1026
10 Jun 2013 15:25:40	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	129,600	272,122	2.0997
09 Jun 2013 02:57:21	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	103,680	217,686	2.0996
06 Jun 2013 23:46:27	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	77,760	162,810	2.0938
04 Jun 2013 19:22:14	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	51,840	108,232	2.0878
03 Jun 2013 11:30:48	1110511	15806970	hadcm3n_n3dq_1920_40_008377526_0	25,920	54,064	2.0858