Task 15996616

Name	hadcm3n_83lk_1980_40_008462603_0
Workunit	8613459
Created	30 Aug 2013, 22:50:35 UTC
Sent	31 Aug 2013, 19:59:22 UTC
Report deadline	1 Dec 2013, 3:26:33 UTC
Received	25 Oct 2013, 4:31:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1169946
Run time	8 days 1 hours 7 min 59 sec
CPU time	7 days 9 hours 16 min 27 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.28 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 22:15:47 (5848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5888, iMonCtr=1 Model crash detected, will try to restart... 22:15:37 (5528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:54:45 (6960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4580, iMonCtr=1 Model crash detected, will try to restart... 10:02:53 (4520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:07:26 (3828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:44:05 (3312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2000, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 21:38:46 (5124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:21:42 (3300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3576, iMonCtr=1 Model crash detected, will try to restart... 13:45:59 (1076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:27 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:02:49 (1092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2260, iMonCtr=1 Model crash detected, will try to restart... 21:55:06 (5252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1424, iMonCtr=1 Model crash detected, will try to restart... 09:10:10 (4492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:27:45 (5648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:42:49 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:13:14 (5332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:07:24 (1452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:38:00 (2148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:00 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5096, iMonCtr=1 Model crash detected, will try to restart... 22:07:20 (4212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6976, iMonCtr=1 Model crash detected, will try to restart... 23:42:45 (5796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:54:03 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:30:48 (5776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:57:49 (4544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:39:47 (816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:55:29 (5264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:59:57 (6832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:57:27 (3504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3584, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3584, iMonCtr=1 Model crash detected, will try to restart... 18:30:27 (6104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:23 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6276, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3960, iMonCtr=1 Model crash detected, will try to restart... 13:52:12 (3104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:04 (6524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1196, iMonCtr=1 Model crash detected, will try to restart... 08:06:16 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:50:49 (5164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:51:39 (4284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:02:56 (3796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:24:00 (5484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:24:54 (2704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:27:20 (1760): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 17:32:25 (5128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:15 (7144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:31:58 (3160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:47 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:13:21 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:55:47 (3356): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 13:37:07 (2188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:59:26 (7516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:20:51 (6528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:28:53 (2516): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 08:29:15 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:35:11 (2632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:36:12 (6304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:25:57 (6008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:58:38 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:59:43 (3184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:10:37 (5896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:11:47 (1892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:41:14 (2272): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 19:57:34 (6208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:03:15 (5516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:38:23 (6960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5276, iMonCtr=1 Model crash detected, will try to restart... 08:14:17 (4516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5728, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F53FA9 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 20:55:14 (5676): No heartbeat from core client for 30 sec - exiting 20:55:15 (5676): No heartbeat from core client for 30 sec - exiting 20:55:16 (5676): No heartbeat from core client for 30 sec - exiting 20:55:17 (5676): No heartbeat from core client for 30 sec - exiting 20:55:18 (5676): No heartbeat from core client for 30 sec - exiting 20:55:19 (5676): No heartbeat from core client for 30 sec - exiting 20:55:20 (5676): No heartbeat from core client for 30 sec - exiting 20:55:21 (5676): No heartbeat from core client for 30 sec - exiting 20:55:22 (5676): No heartbeat from core client for 30 sec - exiting 20:55:23 (5676): No heartbeat from core client for 30 sec - exiting 20:55:24 (5676): No heartbeat from core client for 30 sec - exiting 20:55:25 (5676): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x777D3AC3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... 20:55:26 (5676): No heartbeat from core client for 30 sec - exiting 20:55:27 (5676): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Oct 2013 23:53:50	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	518,400	631,220	1.2176
23 Oct 2013 04:55:29	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	492,480	600,856	1.2201
17 Oct 2013 05:11:36	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	466,560	570,301	1.2224
15 Oct 2013 16:22:28	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	440,640	539,352	1.2240
09 Oct 2013 21:34:25	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	414,720	508,123	1.2252
06 Oct 2013 02:46:20	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	388,800	477,038	1.2269
04 Oct 2013 03:58:46	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	362,880	445,653	1.2281
29 Sep 2013 02:01:37	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	336,960	413,621	1.2275
28 Sep 2013 16:06:27	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	311,040	380,821	1.2243
24 Sep 2013 04:37:10	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	285,120	349,369	1.2253
21 Sep 2013 16:02:44	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	259,200	318,156	1.2275
19 Sep 2013 09:52:36	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	233,280	286,406	1.2277
17 Sep 2013 16:27:41	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	207,360	254,252	1.2261
15 Sep 2013 01:42:10	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	181,440	222,400	1.2257
14 Sep 2013 02:50:37	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	155,520	189,933	1.2213
12 Sep 2013 04:47:50	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	129,600	158,590	1.2237
11 Sep 2013 18:08:33	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	103,680	126,628	1.2213
09 Sep 2013 03:41:55	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	77,760	95,665	1.2303
08 Sep 2013 17:24:28	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	51,840	64,154	1.2375
04 Sep 2013 05:30:19	1169946	15996616	hadcm3n_83lk_1980_40_008462603_0	25,920	33,156	1.2792