Task 15549992

Name	hadcm3n_n4p6_1880_40_008286925_0
Workunit	8438060
Created	17 Jan 2013, 22:05:52 UTC
Sent	17 Jan 2013, 22:06:53 UTC
Report deadline	19 Apr 2013, 5:34:04 UTC
Received	13 Mar 2013, 9:43:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1262119
Run time	34 days 16 hours 7 min 37 sec
CPU time	25 days 8 hours 55 min 37 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.94 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:52:47 (2476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:52:48 (2476): No heartbeat from core client for 30 sec - exiting 11:52:49 (2476): No heartbeat from core client for 30 sec - exiting 11:52:50 (2476): No heartbeat from core client for 30 sec - exiting 11:52:51 (2476): No heartbeat from core client for 30 sec - exiting 11:52:52 (2476): No heartbeat from core client for 30 sec - exiting 11:55:47 (2924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:55:48 (2924): No heartbeat from core client for 30 sec - exiting 11:55:49 (2924): No heartbeat from core client for 30 sec - exiting 11:55:50 (2924): No heartbeat from core client for 30 sec - exiting 11:55:51 (2924): No heartbeat from core client for 30 sec - exiting 11:55:52 (2924): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4460, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:22:35 (3852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 13:22:40 (3852): No heartbeat from core client for 30 sec - exiting 13:22:41 (3852): No heartbeat from core client for 30 sec - exiting 13:22:42 (3852): No heartbeat from core client for 30 sec - exiting 13:22:43 (3852): No heartbeat from core client for 30 sec - exiting 13:22:44 (3852): No heartbeat from core client for 30 sec - exiting 13:22:45 (3852): No heartbeat from core client for 30 sec - exiting 13:22:46 (3852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 14:55:34 (1604): No heartbeat from core client for 30 sec - exiting 14:55:35 (1604): No heartbeat from core client for 30 sec - exiting 14:55:36 (1604): No heartbeat from core client for 30 sec - exiting 14:55:37 (1604): No heartbeat from core client for 30 sec - exiting 14:55:38 (1604): No heartbeat from core client for 30 sec - exiting 14:55:39 (1604): No heartbeat from core client for 30 sec - exiting 14:55:40 (1604): No heartbeat from core client for 30 sec - exiting 14:55:42 (1604): No heartbeat from core client for 30 sec - exiting 14:55:43 (1604): No heartbeat from core client for 30 sec - exiting 14:55:44 (1604): No heartbeat from core client for 30 sec - exiting 14:55:45 (1604): No heartbeat from core client for 30 sec - exiting 14:55:46 (1604): No heartbeat from core client for 30 sec - exiting 14:55:47 (1604): No heartbeat from core client for 30 sec - exiting 14:55:48 (1604): No heartbeat from core client for 30 sec - exiting 14:55:49 (1604): No heartbeat from core client for 30 sec - exiting 14:55:50 (1604): No heartbeat from core client for 30 sec - exiting 14:55:51 (1604): No heartbeat from core client for 30 sec - exiting 14:55:52 (1604): No heartbeat from core client for 30 sec - exiting 14:55:54 (1604): No heartbeat from core client for 30 sec - exiting 14:55:55 (1604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2696, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77193AB3 read attempt to address 0x403A7AF1 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77193AB3 read attempt to address 0x403A7AF1 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Mar 2013 13:50:14	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	1,036,800	2,191,267	2.1135
11 Mar 2013 18:11:32	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	1,010,880	2,138,587	2.1156
09 Mar 2013 23:59:27	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	984,960	2,082,126	2.1139
08 Mar 2013 20:48:17	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	959,040	2,024,040	2.1105
07 Mar 2013 22:07:41	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	933,120	1,966,845	2.1078
06 Mar 2013 10:56:16	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	907,200	1,910,656	2.1061
04 Mar 2013 00:20:53	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	881,280	1,852,425	2.1020
02 Mar 2013 23:13:30	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	855,360	1,798,568	2.1027
02 Mar 2013 08:03:07	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	829,440	1,744,703	2.1035
01 Mar 2013 13:30:27	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	803,520	1,687,474	2.1001
27 Feb 2013 20:43:20	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	777,600	1,628,413	2.0942
26 Feb 2013 16:00:57	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	751,680	1,572,845	2.0924
23 Feb 2013 21:29:58	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	725,760	1,518,302	2.0920
22 Feb 2013 17:41:17	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	699,840	1,461,545	2.0884
21 Feb 2013 02:18:14	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	673,920	1,403,938	2.0832
18 Feb 2013 15:50:42	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	648,000	1,349,040	2.0819
17 Feb 2013 11:42:33	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	622,080	1,294,988	2.0817
16 Feb 2013 17:11:08	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	596,160	1,240,961	2.0816
15 Feb 2013 21:15:41	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	570,240	1,182,390	2.0735
14 Feb 2013 16:00:44	1262119	15549992	hadcm3n_n4p6_1880_40_008286925_0	544,320	1,129,707	2.0754