Task 16033219

Name	hadcm3n_4edf_1980_40_008394484_1
Workunit	8545343
Created	24 Sep 2013, 15:10:09 UTC
Sent	24 Sep 2013, 15:11:06 UTC
Report deadline	24 Dec 2013, 22:38:17 UTC
Received	31 Oct 2013, 0:58:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1288774
Run time	12 days 2 hours 57 min 53 sec
CPU time	9 days 1 hours 37 min 57 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:20:05 (5024): No heartbeat from core client for 30 sec - exiting 23:20:06 (5024): No heartbeat from core client for 30 sec - exiting 23:20:07 (5024): No heartbeat from core client for 30 sec - exiting 23:20:08 (5024): No heartbeat from core client for 30 sec - exiting 23:20:09 (5024): No heartbeat from core client for 30 sec - exiting 23:20:10 (5024): No heartbeat from core client for 30 sec - exiting 23:20:11 (5024): No heartbeat from core client for 30 sec - exiting 23:20:12 (5024): No heartbeat from core client for 30 sec - exiting 23:20:13 (5024): No heartbeat from core client for 30 sec - exiting 23:20:14 (5024): No heartbeat from core client for 30 sec - exiting 23:20:15 (5024): No heartbeat from core client for 30 sec - exiting 23:20:16 (5024): No heartbeat from core client for 30 sec - exiting 23:20:17 (5024): No heartbeat from core client for 30 sec - exiting 23:20:18 (5024): No heartbeat from core client for 30 sec - exiting 23:20:19 (5024): No heartbeat from core client for 30 sec - exiting 23:20:20 (5024): No heartbeat from core client for 30 sec - exiting 23:20:21 (5024): No heartbeat from core client for 30 sec - exiting 23:20:22 (5024): No heartbeat from core client for 30 sec - exiting 23:20:23 (5024): No heartbeat from core client for 30 sec - exiting 23:20:24 (5024): No heartbeat from core client for 30 sec - exiting 23:20:25 (5024): No heartbeat from core client for 30 sec - exiting 23:20:26 (5024): No heartbeat from core client for 30 sec - exiting 23:20:27 (5024): No heartbeat from core client for 30 sec - exiting 23:20:28 (5024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:21:57 (3660): No heartbeat from core client for 30 sec - exiting 23:21:58 (3660): No heartbeat from core client for 30 sec - exiting 23:21:59 (3660): No heartbeat from core client for 30 sec - exiting 23:22:00 (3660): No heartbeat from core client for 30 sec - exiting 23:22:01 (3660): No heartbeat from core client for 30 sec - exiting 23:22:02 (3660): No heartbeat from core client for 30 sec - exiting 23:22:03 (3660): No heartbeat from core client for 30 sec - exiting 23:22:04 (3660): No heartbeat from core client for 30 sec - exiting 23:22:05 (3660): No heartbeat from core client for 30 sec - exiting 23:22:06 (3660): No heartbeat from core client for 30 sec - exiting 23:22:07 (3660): No heartbeat from core client for 30 sec - exiting 23:22:08 (3660): No heartbeat from core client for 30 sec - exiting 23:22:09 (3660): No heartbeat from core client for 30 sec - exiting 23:22:10 (3660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:14 (3952): No heartbeat from core client for 30 sec - exiting 23:24:15 (3952): No heartbeat from core client for 30 sec - exiting 23:24:16 (3952): No heartbeat from core client for 30 sec - exiting 23:24:17 (3952): No heartbeat from core client for 30 sec - exiting 23:24:18 (3952): No heartbeat from core client for 30 sec - exiting 23:24:19 (3952): No heartbeat from core client for 30 sec - exiting 23:24:21 (3952): No heartbeat from core client for 30 sec - exiting 23:24:22 (3952): No heartbeat from core client for 30 sec - exiting 23:24:23 (3952): No heartbeat from core client for 30 sec - exiting 23:24:24 (3952): No heartbeat from core client for 30 sec - exiting 23:24:25 (3952): No heartbeat from core client for 30 sec - exiting 23:24:26 (3952): No heartbeat from core client for 30 sec - exiting 23:24:27 (3952): No heartbeat from core client for 30 sec - exiting 23:24:28 (3952): No heartbeat from core client for 30 sec - exiting 23:24:29 (3952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:25:11 (5884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:25:54 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:12:10 (3332): No heartbeat from core client for 30 sec - exiting 21:12:11 (3332): No heartbeat from core client for 30 sec - exiting 21:12:12 (3332): No heartbeat from core client for 30 sec - exiting 21:12:13 (3332): No heartbeat from core client for 30 sec - exiting 21:12:14 (3332): No heartbeat from core client for 30 sec - exiting 21:12:15 (3332): No heartbeat from core client for 30 sec - exiting 21:12:16 (3332): No heartbeat from core client for 30 sec - exiting 21:12:17 (3332): No heartbeat from core client for 30 sec - exiting 21:12:18 (3332): No heartbeat from core client for 30 sec - exiting 21:12:19 (3332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4484, iMonCtr=1 Model crash detected, will try to restart... 22:07:50 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=916, iMonCtr=1 Model crash detected, will try to restart... 11:40:45 (5008): No heartbeat from core client for 30 sec - exiting 11:40:46 (5008): No heartbeat from core client for 30 sec - exiting 11:40:47 (5008): No heartbeat from core client for 30 sec - exiting 11:40:48 (5008): No heartbeat from core client for 30 sec - exiting 11:40:49 (5008): No heartbeat from core client for 30 sec - exiting 11:40:50 (5008): No heartbeat from core client for 30 sec - exiting 11:40:51 (5008): No heartbeat from core client for 30 sec - exiting 11:40:52 (5008): No heartbeat from core client for 30 sec - exiting 11:40:53 (5008): No heartbeat from core client for 30 sec - exiting 11:40:54 (5008): No heartbeat from core client for 30 sec - exiting 11:40:55 (5008): No heartbeat from core client for 30 sec - exiting 11:40:56 (5008): No heartbeat from core client for 30 sec - exiting 11:40:57 (5008): No heartbeat from core client for 30 sec - exiting 11:40:58 (5008): No heartbeat from core client for 30 sec - exiting 11:40:59 (5008): No heartbeat from core client for 30 sec - exiting 11:41:00 (5008): No heartbeat from core client for 30 sec - exiting 11:41:01 (5008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4176, iMonCtr=1 Model crash detected, will try to restart... 12:08:21 (3464): No heartbeat from core client for 30 sec - exiting 12:08:22 (3464): No heartbeat from core client for 30 sec - exiting 12:08:23 (3464): No heartbeat from core client for 30 sec - exiting 12:08:24 (3464): No heartbeat from core client for 30 sec - exiting 12:08:25 (3464): No heartbeat from core client for 30 sec - exiting 12:08:27 (3464): No heartbeat from core client for 30 sec - exiting 12:08:28 (3464): No heartbeat from core client for 30 sec - exiting 12:08:29 (3464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:14:47 (3016): No heartbeat from core client for 30 sec - exiting 11:14:48 (3016): No heartbeat from core client for 30 sec - exiting 11:14:49 (3016): No heartbeat from core client for 30 sec - exiting 11:14:50 (3016): No heartbeat from core client for 30 sec - exiting 11:14:51 (3016): No heartbeat from core client for 30 sec - exiting 11:14:52 (3016): No heartbeat from core client for 30 sec - exiting 11:14:53 (3016): No heartbeat from core client for 30 sec - exiting 11:14:54 (3016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:10:44 (4392): No heartbeat from core client for 30 sec - exiting 19:10:45 (4392): No heartbeat from core client for 30 sec - exiting 19:10:46 (4392): No heartbeat from core client for 30 sec - exiting 19:10:47 (4392): No heartbeat from core client for 30 sec - exiting 19:10:48 (4392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:11:44 (4808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:30:10 (1956): No heartbeat from core client for 30 sec - exiting 18:30:11 (1956): No heartbeat from core client for 30 sec - exiting 18:30:13 (1956): No heartbeat from core client for 30 sec - exiting 18:30:14 (1956): No heartbeat from core client for 30 sec - exiting 18:30:15 (1956): No heartbeat from core client for 30 sec - exiting 18:30:16 (1956): No heartbeat from core client for 30 sec - exiting 18:30:17 (1956): No heartbeat from core client for 30 sec - exiting 18:30:18 (1956): No heartbeat from core client for 30 sec - exiting 18:30:19 (1956): No heartbeat from core client for 30 sec - exiting 18:30:20 (1956): No heartbeat from core client for 30 sec - exiting 18:30:21 (1956): No heartbeat from core client for 30 sec - exiting 18:30:22 (1956): No heartbeat from core client for 30 sec - exiting 18:30:24 (1956): No heartbeat from core client for 30 sec - exiting 18:30:25 (1956): No heartbeat from core client for 30 sec - exiting 18:30:26 (1956): No heartbeat from core client for 30 sec - exiting 18:30:27 (1956): No heartbeat from core client for 30 sec - exiting 18:30:28 (1956): No heartbeat from core client for 30 sec - exiting 18:30:29 (1956): No heartbeat from core client for 30 sec - exiting 18:30:30 (1956): No heartbeat from core client for 30 sec - exiting 18:30:31 (1956): No heartbeat from core client for 30 sec - exiting 18:30:32 (1956): No heartbeat from core client for 30 sec - exiting 18:30:33 (1956): No heartbeat from core client for 30 sec - exiting 18:30:34 (1956): No heartbeat from core client for 30 sec - exiting 18:30:36 (1956): No heartbeat from core client for 30 sec - exiting 18:30:37 (1956): No heartbeat from core client for 30 sec - exiting 18:30:38 (1956): No heartbeat from core client for 30 sec - exiting 18:30:39 (1956): No heartbeat from core client for 30 sec - exiting 18:30:40 (1956): No heartbeat from core client for 30 sec - exiting 18:30:41 (1956): No heartbeat from core client for 30 sec - exiting 18:30:42 (1956): No heartbeat from core client for 30 sec - exiting 18:30:43 (1956): No heartbeat from core client for 30 sec - exiting 18:30:44 (1956): No heartbeat from core client for 30 sec - exiting 18:30:45 (1956): No heartbeat from core client for 30 sec - exiting 18:30:46 (1956): No heartbeat from core client for 30 sec - exiting 18:30:48 (1956): No heartbeat from core client for 30 sec - exiting 18:30:49 (1956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:50 (1956): No heartbeat from core client for 30 sec - exiting 18:30:51 (1956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5332, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77003FA9 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77277383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77467383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76FE7383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 19:31:23 (4676): No heartbeat from core client for 30 sec - exiting 19:31:24 (4676): No heartbeat from core client for 30 sec - exiting 19:31:25 (4676): No heartbeat from core client for 30 sec - exiting 19:31:26 (4676): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77927383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77D93AC3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Oct 2013 13:24:26	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	518,400	757,330	1.4609
26 Oct 2013 01:28:50	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	492,480	727,466	1.4771
23 Oct 2013 07:21:06	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	466,560	697,333	1.4946
22 Oct 2013 13:15:23	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	440,640	665,971	1.5114
22 Oct 2013 03:36:12	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	414,720	635,436	1.5322
20 Oct 2013 05:02:34	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	388,800	604,962	1.5560
19 Oct 2013 18:38:14	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	362,880	574,532	1.5833
19 Oct 2013 02:23:13	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	336,960	543,522	1.6130
04 Oct 2013 08:11:01	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	311,040	502,450	1.6154
03 Oct 2013 15:34:39	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	285,120	460,633	1.6156
02 Oct 2013 22:36:13	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	259,200	418,820	1.6158
02 Oct 2013 06:12:54	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	233,280	377,014	1.6161
01 Oct 2013 13:23:15	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	207,360	335,237	1.6167
30 Sep 2013 20:35:04	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	181,440	292,716	1.6133
30 Sep 2013 04:13:10	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	155,520	249,915	1.6070
28 Sep 2013 00:36:06	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	129,600	207,027	1.5974
27 Sep 2013 08:42:05	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	103,680	165,534	1.5966
26 Sep 2013 15:39:23	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	77,760	124,060	1.5954
25 Sep 2013 23:28:23	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	51,840	82,647	1.5943
25 Sep 2013 09:29:47	1288774	16033219	hadcm3n_4edf_1980_40_008394484_1	25,920	41,982	1.6197