Task 15557943

Name	hadcm3n_103q_1980_40_008239693_3
Workunit	8394817
Created	26 Jan 2013, 3:33:20 UTC
Sent	26 Jan 2013, 3:33:27 UTC
Report deadline	27 Apr 2013, 11:00:38 UTC
Received	12 Feb 2013, 8:52:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1217848
Run time	4 days 5 hours 35 min 2 sec
CPU time	3 days 20 hours 43 min 44 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.44</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 13:56:20 (6212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:56:21 (6212): No heartbeat from core client for 30 sec - exiting 13:56:22 (6212): No heartbeat from core client for 30 sec - exiting 13:56:23 (6212): No heartbeat from core client for 30 sec - exiting 13:56:24 (6212): No heartbeat from core client for 30 sec - exiting 13:56:25 (6212): No heartbeat from core client for 30 sec - exiting 13:56:26 (6212): No heartbeat from core client for 30 sec - exiting 13:56:27 (6212): No heartbeat from core client for 30 sec - exiting 13:56:28 (6212): No heartbeat from core client for 30 sec - exiting 13:56:29 (6212): No heartbeat from core client for 30 sec - exiting 14:06:35 (8460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:13:16 (4904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 15:20:20 (9164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:20:21 (9164): No heartbeat from core client for 30 sec - exiting 15:20:22 (9164): No heartbeat from core client for 30 sec - exiting 15:20:23 (9164): No heartbeat from core client for 30 sec - exiting 15:20:24 (9164): No heartbeat from core client for 30 sec - exiting 15:20:25 (9164): No heartbeat from core client for 30 sec - exiting 15:20:26 (9164): No heartbeat from core client for 30 sec - exiting 15:20:27 (9164): No heartbeat from core client for 30 sec - exiting 15:20:28 (9164): No heartbeat from core client for 30 sec - exiting 15:20:29 (9164): No heartbeat from core client for 30 sec - exiting 15:20:30 (9164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:47:15 (7276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:08:23 (8676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... 17:08:58 (8296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:42:49 (1664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:28:44 (8604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:59:41 (7824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:45:15 (4664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:08:39 (6656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:17:24 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:17:25 (5856): No heartbeat from core client for 30 sec - exiting 21:17:26 (5856): No heartbeat from core client for 30 sec - exiting 21:17:27 (5856): No heartbeat from core client for 30 sec - exiting BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 20:12:16 (10228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:30:04 (8456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:11:11 (9228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:33:38 (9240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:11:34 (8788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:58 (9096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:59 (9096): No heartbeat from core client for 30 sec - exiting 14:21:00 (9096): No heartbeat from core client for 30 sec - exiting 14:21:01 (9096): No heartbeat from core client for 30 sec - exiting 14:21:02 (9096): No heartbeat from core client for 30 sec - exiting 14:21:03 (9096): No heartbeat from core client for 30 sec - exiting 14:21:04 (9096): No heartbeat from core client for 30 sec - exiting 14:21:05 (9096): No heartbeat from core client for 30 sec - exiting 14:21:06 (9096): No heartbeat from core client for 30 sec - exiting 14:21:07 (9096): No heartbeat from core client for 30 sec - exiting 14:21:08 (9096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 14:47:13 (1672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:47:14 (1672): No heartbeat from core client for 30 sec - exiting 14:47:15 (1672): No heartbeat from core client for 30 sec - exiting 14:47:16 (1672): No heartbeat from core client for 30 sec - exiting 14:47:17 (1672): No heartbeat from core client for 30 sec - exiting 14:47:18 (1672): No heartbeat from core client for 30 sec - exiting 14:47:19 (1672): No heartbeat from core client for 30 sec - exiting 14:47:20 (1672): No heartbeat from core client for 30 sec - exiting 14:47:21 (1672): No heartbeat from core client for 30 sec - exiting 14:47:22 (1672): No heartbeat from core client for 30 sec - exiting 14:47:23 (1672): No heartbeat from core client for 30 sec - exiting 15:31:58 (9980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:40:23 (2000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:11:02 (5784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:13:30 (9568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 16:32:47 (9436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:32:48 (9436): No heartbeat from core client for 30 sec - exiting 16:32:49 (9436): No heartbeat from core client for 30 sec - exiting 16:32:50 (9436): No heartbeat from core client for 30 sec - exiting 16:32:51 (9436): No heartbeat from core client for 30 sec - exiting 16:32:52 (9436): No heartbeat from core client for 30 sec - exiting 16:32:53 (9436): No heartbeat from core client for 30 sec - exiting 16:32:54 (9436): No heartbeat from core client for 30 sec - exiting 16:32:55 (9436): No heartbeat from core client for 30 sec - exiting 16:32:56 (9436): No heartbeat from core client for 30 sec - exiting 16:32:57 (9436): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... 17:37:07 (10996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:03:14 (9904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:57 (10504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:15:03 (4512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:08:28 (8600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:29 (8600): No heartbeat from core client for 30 sec - exiting 09:08:30 (8600): No heartbeat from core client for 30 sec - exiting 09:08:31 (8600): No heartbeat from core client for 30 sec - exiting 09:08:32 (8600): No heartbeat from core client for 30 sec - exiting 09:08:33 (8600): No heartbeat from core client for 30 sec - exiting 09:08:34 (8600): No heartbeat from core client for 30 sec - exiting 09:08:35 (8600): No heartbeat from core client for 30 sec - exiting 09:08:36 (8600): No heartbeat from core client for 30 sec - exiting 09:08:37 (8600): No heartbeat from core client for 30 sec - exiting 09:08:38 (8600): No heartbeat from core client for 30 sec - exiting 11:30:38 (9324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:41:09 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:02:17 (7764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:02:18 (7764): No heartbeat from core client for 30 sec - exiting 18:02:19 (7764): No heartbeat from core client for 30 sec - exiting 18:02:20 (7764): No heartbeat from core client for 30 sec - exiting 18:02:21 (7764): No heartbeat from core client for 30 sec - exiting 18:02:22 (7764): No heartbeat from core client for 30 sec - exiting 18:02:23 (7764): No heartbeat from core client for 30 sec - exiting 18:02:24 (7764): No heartbeat from core client for 30 sec - exiting 18:02:25 (7764): No heartbeat from core client for 30 sec - exiting 18:02:26 (7764): No heartbeat from core client for 30 sec - exiting 18:02:27 (7764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9032, iMonCtr=1 Model crash detected, will try to restart... 19:40:52 (8904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:25:14 (9912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:36:16 (1752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8088, iMonCtr=1 Model crash detected, will try to restart... 08:51:57 (9068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:44:00 (8116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:53:17 (7664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1504, iMonCtr=1 Model crash detected, will try to restart... 21:20:31 (8384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:15:28 (8416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:47:17 (8088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:20:02 (7812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x779E3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A071F3 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Feb 2013 23:29:01	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	259,200	309,417	1.1937
10 Feb 2013 02:10:24	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	233,280	278,166	1.1924
09 Feb 2013 07:01:24	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	207,360	246,881	1.1906
08 Feb 2013 10:02:30	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	181,440	215,720	1.1889
04 Feb 2013 09:12:54	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	155,520	185,130	1.1904
03 Feb 2013 03:36:30	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	129,600	153,907	1.1876
02 Feb 2013 05:49:57	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	103,680	122,984	1.1862
30 Jan 2013 12:25:06	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	77,760	91,736	1.1797
27 Jan 2013 11:36:35	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	51,840	60,415	1.1654
27 Jan 2013 00:19:24	1217848	15557943	hadcm3n_103q_1980_40_008239693_3	25,920	30,436	1.1742