Task 13363200

Name	hadcm3n_t0wg_1940_40_007449251_0
Workunit	7646754
Created	9 Sep 2011, 23:37:46 UTC
Sent	14 Sep 2011, 10:41:31 UTC
Report deadline	14 Dec 2011, 18:08:42 UTC
Received	25 Oct 2011, 15:23:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1024848
Run time	9 days 19 hours 7 min 14 sec
CPU time	7 days 11 hours 58 min 8 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.6.41</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:14:45 (5928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:06:24 (3004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t0wgko.pjf0c10 Error converting file to netcdf: dataout/t0wgko.pif0c10 Error converting file to netcdf: dataout/t0wgko.pff0c10 Error converting file to netcdf: dataout/t0wgko.pcf0c10 Error converting file to netcdf: dataout/t0wgko.pbf0c10 Error converting file to netcdf: dataout/t0wgko.paf0c10 Error converting file to netcdf: dataout/t0wgka.phf0c10 Error converting file to netcdf: dataout/t0wgka.pgf0c10 Error converting file to netcdf: dataout/t0wgka.pef0c10 Error converting file to netcdf: dataout/t0wgka.pdf0c10 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F0B84B write attempt to address 0xC21308D3 Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9604, selfPID=9604, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77717353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77717353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 01:45:32 (6212): No heartbeat from core client for 30 sec - exiting 01:45:33 (6212): No heartbeat from core client for 30 sec - exiting 01:45:34 (6212): No heartbeat from core client for 30 sec - exiting 01:45:35 (6212): No heartbeat from core client for 30 sec - exiting 01:45:36 (6212): No heartbeat from core client for 30 sec - exiting 01:45:37 (6212): No heartbeat from core client for 30 sec - exiting 01:45:38 (6212): No heartbeat from core client for 30 sec - exiting 01:45:39 (6212): No heartbeat from core client for 30 sec - exiting 01:45:40 (6212): No heartbeat from core client for 30 sec - exiting 01:45:41 (6212): No heartbeat from core client for 30 sec - exiting 01:45:42 (6212): No heartbeat from core client for 30 sec - exiting 01:45:43 (6212): No heartbeat from core client for 30 sec - exiting 01:45:44 (6212): No heartbeat from core client for 30 sec - exiting 01:45:45 (6212): No heartbeat from core client for 30 sec - exiting 01:45:46 (6212): No heartbeat from core client for 30 sec - exiting 01:45:47 (6212): No heartbeat from core client for 30 sec - exiting 01:45:48 (6212): No heartbeat from core client for 30 sec - exiting 01:45:49 (6212): No heartbeat from core client for 30 sec - exiting 01:45:50 (6212): No heartbeat from core client for 30 sec - exiting 01:45:51 (6212): No heartbeat from core client for 30 sec - exiting 01:45:52 (6212): No heartbeat from core client for 30 sec - exiting 01:45:53 (6212): No heartbeat from core client for 30 sec - exiting 01:45:54 (6212): No heartbeat from core client for 30 sec - exiting 01:45:55 (6212): No heartbeat from core client for 30 sec - exiting 01:45:56 (6212): No heartbeat from core client for 30 sec - exiting 01:45:57 (6212): No heartbeat from core client for 30 sec - exiting 01:45:58 (6212): No heartbeat from core client for 30 sec - exiting 01:45:59 (6212): No heartbeat from core client for 30 sec - exiting 01:46:01 (6212): No heartbeat from core client for 30 sec - exiting 01:46:02 (6212): No heartbeat from core client for 30 sec - exiting 01:46:03 (6212): No heartbeat from core client for 30 sec - exiting 01:46:04 (6212): No heartbeat from core client for 30 sec - exiting 01:46:05 (6212): No heartbeat from core client for 30 sec - exiting 01:46:06 (6212): No heartbeat from core client for 30 sec - exiting 01:46:07 (6212): No heartbeat from core client for 30 sec - exiting 01:46:08 (6212): No heartbeat from core client for 30 sec - exiting 01:46:09 (6212): No heartbeat from core client for 30 sec - exiting 01:46:10 (6212): No heartbeat from core client for 30 sec - exiting 01:46:11 (6212): No heartbeat from core client for 30 sec - exiting 01:46:12 (6212): No heartbeat from core client for 30 sec - exiting 01:46:13 (6212): No heartbeat from core client for 30 sec - exiting 05:44:15 (6212): No heartbeat from core client for 30 sec - exiting 05:44:16 (6212): No heartbeat from core client for 30 sec - exiting 05:44:17 (6212): No heartbeat from core client for 30 sec - exiting 05:44:18 (6212): No heartbeat from core client for 30 sec - exiting 05:44:19 (6212): No heartbeat from core client for 30 sec - exiting 05:44:20 (6212): No heartbeat from core client for 30 sec - exiting 05:44:21 (6212): No heartbeat from core client for 30 sec - exiting 05:44:22 (6212): No heartbeat from core client for 30 sec - exiting 05:44:23 (6212): No heartbeat from core client for 30 sec - exiting 05:44:24 (6212): No heartbeat from core client for 30 sec - exiting 05:44:25 (6212): No heartbeat from core client for 30 sec - exiting 05:44:26 (6212): No heartbeat from core client for 30 sec - exiting 05:44:27 (6212): No heartbeat from core client for 30 sec - exiting 05:44:28 (6212): No heartbeat from core client for 30 sec - exiting 05:44:29 (6212): No heartbeat from core client for 30 sec - exiting 05:44:30 (6212): No heartbeat from core client for 30 sec - exiting 05:44:31 (6212): No heartbeat from core client for 30 sec - exiting 05:44:32 (6212): No heartbeat from core client for 30 sec - exiting 05:44:33 (6212): No heartbeat from core client for 30 sec - exiting 05:44:34 (6212): No heartbeat from core client for 30 sec - exiting 05:44:35 (6212): No heartbeat from core client for 30 sec - exiting 05:44:36 (6212): No heartbeat from core client for 30 sec - exiting 05:44:37 (6212): No heartbeat from core client for 30 sec - exiting 05:44:38 (6212): No heartbeat from core client for 30 sec - exiting 05:44:39 (6212): No heartbeat from core client for 30 sec - exiting 05:44:40 (6212): No heartbeat from core client for 30 sec - exiting 05:44:41 (6212): No heartbeat from core client for 30 sec - exiting 05:44:42 (6212): No heartbeat from core client for 30 sec - exiting 05:44:43 (6212): No heartbeat from core client for 30 sec - exiting 05:44:44 (6212): No heartbeat from core client for 30 sec - exiting 05:44:45 (6212): No heartbeat from core client for 30 sec - exiting 05:44:46 (6212): No heartbeat from core client for 30 sec - exiting 05:44:47 (6212): No heartbeat from core client for 30 sec - exiting 05:44:48 (6212): No heartbeat from core client for 30 sec - exiting 05:44:49 (6212): No heartbeat from core client for 30 sec - exiting 05:44:50 (6212): No heartbeat from core client for 30 sec - exiting 05:44:51 (6212): No heartbeat from core client for 30 sec - exiting 05:44:52 (6212): No heartbeat from core client for 30 sec - exiting 05:44:53 (6212): No heartbeat from core client for 30 sec - exiting 05:44:54 (6212): No heartbeat from core client for 30 sec - exiting 05:44:55 (6212): No heartbeat from core client for 30 sec - exiting 05:44:56 (6212): No heartbeat from core client for 30 sec - exiting 05:44:57 (6212): No heartbeat from core client for 30 sec - exiting 05:44:58 (6212): No heartbeat from core client for 30 sec - exiting 05:44:59 (6212): No heartbeat from core client for 30 sec - exiting 05:45:00 (6212): No heartbeat from core client for 30 sec - exiting 05:45:01 (6212): No heartbeat from core client for 30 sec - exiting 09:42:52 (6212): No heartbeat from core client for 30 sec - exiting 09:42:54 (6212): No heartbeat from core client for 30 sec - exiting 09:42:55 (6212): No heartbeat from core client for 30 sec - exiting 09:42:56 (6212): No heartbeat from core client for 30 sec - exiting 09:42:57 (6212): No heartbeat from core client for 30 sec - exiting 09:42:58 (6212): No heartbeat from core client for 30 sec - exiting 09:42:59 (6212): No heartbeat from core client for 30 sec - exiting 09:43:00 (6212): No heartbeat from core client for 30 sec - exiting 09:43:01 (6212): No heartbeat from core client for 30 sec - exiting 09:43:02 (6212): No heartbeat from core client for 30 sec - exiting 09:43:03 (6212): No heartbeat from core client for 30 sec - exiting 09:43:04 (6212): No heartbeat from core client for 30 sec - exiting 09:43:05 (6212): No heartbeat from core client for 30 sec - exiting 09:43:06 (6212): No heartbeat from core client for 30 sec - exiting 09:43:07 (6212): No heartbeat from core client for 30 sec - exiting 09:43:08 (6212): No heartbeat from core client for 30 sec - exiting 09:43:09 (6212): No heartbeat from core client for 30 sec - exiting 09:43:10 (6212): No heartbeat from core client for 30 sec - exiting 09:43:11 (6212): No heartbeat from core client for 30 sec - exiting 09:43:12 (6212): No heartbeat from core client for 30 sec - exiting 09:43:13 (6212): No heartbeat from core client for 30 sec - exiting 09:43:14 (6212): No heartbeat from core client for 30 sec - exiting 09:43:15 (6212): No heartbeat from core client for 30 sec - exiting 09:43:16 (6212): No heartbeat from core client for 30 sec - exiting 09:43:17 (6212): No heartbeat from core client for 30 sec - exiting 09:43:18 (6212): No heartbeat from core client for 30 sec - exiting 09:43:19 (6212): No heartbeat from core client for 30 sec - exiting 09:43:20 (6212): No heartbeat from core client for 30 sec - exiting 09:43:21 (6212): No heartbeat from core client for 30 sec - exiting 09:43:22 (6212): No heartbeat from core client for 30 sec - exiting 09:43:23 (6212): No heartbeat from core client for 30 sec - exiting 09:43:24 (6212): No heartbeat from core client for 30 sec - exiting 09:43:25 (6212): No heartbeat from core client for 30 sec - exiting 09:43:26 (6212): No heartbeat from core client for 30 sec - exiting 09:43:27 (6212): No heartbeat from core client for 30 sec - exiting 09:43:28 (6212): No heartbeat from core client for 30 sec - exiting 09:43:29 (6212): No heartbeat from core client for 30 sec - exiting 09:43:30 (6212): No heartbeat from core client for 30 sec - exiting 09:43:31 (6212): No heartbeat from core client for 30 sec - exiting 09:43:32 (6212): No heartbeat from core client for 30 sec - exiting 09:43:33 (6212): No heartbeat from core client for 30 sec - exiting 09:43:34 (6212): No heartbeat from core client for 30 sec - exiting 09:43:35 (6212): No heartbeat from core client for 30 sec - exiting 09:43:36 (6212): No heartbeat from core client for 30 sec - exiting 09:43:37 (6212): No heartbeat from core client for 30 sec - exiting 09:43:38 (6212): No heartbeat from core client for 30 sec - exiting 09:43:39 (6212): No heartbeat from core client for 30 sec - exiting 09:43:40 (6212): No heartbeat from core client for 30 sec - exiting 09:43:41 (6212): No heartbeat from core client for 30 sec - exiting 09:43:42 (6212): No heartbeat from core client for 30 sec - exiting 09:43:43 (6212): No heartbeat from core client for 30 sec - exiting 09:43:44 (6212): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77717353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77717353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77713A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77717353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B57353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 05:05:43 (6476): No heartbeat from core client for 30 sec - exiting 05:05:56 (6476): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B53A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B57353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B57353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B53A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B53A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B53A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B53740 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Oct 2011 09:30:03	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	259,200	623,950	2.4072
07 Oct 2011 14:32:56	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	233,280	564,390	2.4194
05 Oct 2011 11:53:17	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	207,360	502,625	2.4239
03 Oct 2011 12:06:11	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	181,440	443,357	2.4435
30 Sep 2011 15:52:32	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	155,520	381,500	2.4531
28 Sep 2011 15:00:34	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	129,600	315,702	2.4360
24 Sep 2011 16:27:14	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	103,680	253,395	2.4440
22 Sep 2011 11:35:09	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	77,760	188,605	2.4255
19 Sep 2011 17:06:36	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	51,840	126,836	2.4467
16 Sep 2011 15:53:59	1024848	13363200	hadcm3n_t0wg_1940_40_007449251_0	25,920	65,113	2.5121