Task 15769410

Name	hadcm3n_4j4f_1940_40_008306802_1
Workunit	8457937
Created	9 May 2013, 18:52:20 UTC
Sent	9 May 2013, 18:55:32 UTC
Report deadline	9 Aug 2013, 2:22:43 UTC
Received	27 Jun 2013, 19:55:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1263648
Run time	3 days 13 hours 26 min 29 sec
CPU time	2 days 23 hours 44 min 37 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.82 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7352, iMonCtr=1 Model crash detected, will try to restart... 19:55:40 (3476): No heartbeat from core client for 30 sec - exiting 19:55:41 (3476): No heartbeat from core client for 30 sec - exiting 19:55:42 (3476): No heartbeat from core client for 30 sec - exiting 19:55:43 (3476): No heartbeat from core client for 30 sec - exiting 19:55:44 (3476): No heartbeat from core client for 30 sec - exiting 19:55:45 (3476): No heartbeat from core client for 30 sec - exiting 19:55:46 (3476): No heartbeat from core client for 30 sec - exiting 19:55:47 (3476): No heartbeat from core client for 30 sec - exiting 19:55:48 (3476): No heartbeat from core client for 30 sec - exiting 19:55:49 (3476): No heartbeat from core client for 30 sec - exiting 19:55:50 (3476): No heartbeat from core client for 30 sec - exiting 19:55:51 (3476): No heartbeat from core client for 30 sec - exiting 19:55:52 (3476): No heartbeat from core client for 30 sec - exiting 19:55:53 (3476): No heartbeat from core client for 30 sec - exiting 19:55:54 (3476): No heartbeat from core client for 30 sec - exiting 19:55:55 (3476): No heartbeat from core client for 30 sec - exiting 19:55:56 (3476): No heartbeat from core client for 30 sec - exiting 19:55:57 (3476): No heartbeat from core client for 30 sec - exiting 19:55:58 (3476): No heartbeat from core client for 30 sec - exiting 19:55:59 (3476): No heartbeat from core client for 30 sec - exiting 19:56:00 (3476): No heartbeat from core client for 30 sec - exiting 19:56:01 (3476): No heartbeat from core client for 30 sec - exiting 19:56:02 (3476): No heartbeat from core client for 30 sec - exiting 19:56:03 (3476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:46:47 (24128): No heartbeat from core client for 30 sec - exiting 18:46:48 (24128): No heartbeat from core client for 30 sec - exiting 18:46:49 (24128): No heartbeat from core client for 30 sec - exiting 18:46:50 (24128): No heartbeat from core client for 30 sec - exiting 18:46:51 (24128): No heartbeat from core client for 30 sec - exiting 18:46:52 (24128): No heartbeat from core client for 30 sec - exiting 18:46:53 (24128): No heartbeat from core client for 30 sec - exiting 18:46:54 (24128): No heartbeat from core client for 30 sec - exiting 18:46:55 (24128): No heartbeat from core client for 30 sec - exiting 18:46:56 (24128): No heartbeat from core client for 30 sec - exiting 18:46:57 (24128): No heartbeat from core client for 30 sec - exiting 18:46:58 (24128): No heartbeat from core client for 30 sec - exiting 18:46:59 (24128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:00 (24128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:52:13 (3496): No heartbeat from core client for 30 sec - exiting 19:52:14 (3496): No heartbeat from core client for 30 sec - exiting 19:52:15 (3496): No heartbeat from core client for 30 sec - exiting 19:52:16 (3496): No heartbeat from core client for 30 sec - exiting 19:52:17 (3496): No heartbeat from core client for 30 sec - exiting 19:52:18 (3496): No heartbeat from core client for 30 sec - exiting 19:52:19 (3496): No heartbeat from core client for 30 sec - exiting 19:52:20 (3496): No heartbeat from core client for 30 sec - exiting 19:52:21 (3496): No heartbeat from core client for 30 sec - exiting 19:52:22 (3496): No heartbeat from core client for 30 sec - exiting 19:52:23 (3496): No heartbeat from core client for 30 sec - exiting 19:52:24 (3496): No heartbeat from core client for 30 sec - exiting 19:52:25 (3496): No heartbeat from core client for 30 sec - exiting 19:52:26 (3496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:15 (5860): No heartbeat from core client for 30 sec - exiting 19:29:16 (5860): No heartbeat from core client for 30 sec - exiting 19:29:17 (5860): No heartbeat from core client for 30 sec - exiting 19:29:18 (5860): No heartbeat from core client for 30 sec - exiting 19:29:19 (5860): No heartbeat from core client for 30 sec - exiting 19:29:20 (5860): No heartbeat from core client for 30 sec - exiting 19:29:21 (5860): No heartbeat from core client for 30 sec - exiting 19:29:22 (5860): No heartbeat from core client for 30 sec - exiting 19:29:23 (5860): No heartbeat from core client for 30 sec - exiting 19:29:24 (5860): No heartbeat from core client for 30 sec - exiting 19:29:25 (5860): No heartbeat from core client for 30 sec - exiting 19:29:26 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:29 (5156): No heartbeat from core client for 30 sec - exiting 19:03:30 (5156): No heartbeat from core client for 30 sec - exiting 19:03:31 (5156): No heartbeat from core client for 30 sec - exiting 19:03:32 (5156): No heartbeat from core client for 30 sec - exiting 19:03:33 (5156): No heartbeat from core client for 30 sec - exiting 19:03:34 (5156): No heartbeat from core client for 30 sec - exiting 19:03:35 (5156): No heartbeat from core client for 30 sec - exiting 19:03:36 (5156): No heartbeat from core client for 30 sec - exiting 19:03:37 (5156): No heartbeat from core client for 30 sec - exiting 19:03:38 (5156): No heartbeat from core client for 30 sec - exiting 19:03:39 (5156): No heartbeat from core client for 30 sec - exiting 19:03:40 (5156): No heartbeat from core client for 30 sec - exiting 19:03:41 (5156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6044, iMonCtr=1 Model crash detected, will try to restart... 15:57:12 (22308): No heartbeat from core client for 30 sec - exiting 15:57:13 (22308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:57:14 (22308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:14:30 (14436): No heartbeat from core client for 30 sec - exiting 12:14:31 (14436): No heartbeat from core client for 30 sec - exiting 12:14:32 (14436): No heartbeat from core client for 30 sec - exiting 12:14:33 (14436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:42:21 (5688): No heartbeat from core client for 30 sec - exiting 13:42:22 (5688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:42:23 (5688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 14:47:35 (5720): No heartbeat from core client for 30 sec - exiting 14:47:36 (5720): No heartbeat from core client for 30 sec - exiting 14:47:37 (5720): No heartbeat from core client for 30 sec - exiting 14:47:38 (5720): No heartbeat from core client for 30 sec - exiting 14:47:39 (5720): No heartbeat from core client for 30 sec - exiting 14:47:40 (5720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:57:35 (5836): No heartbeat from core client for 30 sec - exiting 18:57:36 (5836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:31:04 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:56:21 (5636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:49:13 (5396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4064, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:15:07 (19076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x779A7373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x779C7373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77E73AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jun 2013 20:27:12	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	259,200	255,196	0.9846
23 Jun 2013 12:18:13	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	233,280	230,144	0.9866
20 Jun 2013 18:26:26	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	207,360	205,230	0.9897
09 Jun 2013 17:28:32	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	181,440	180,137	0.9928
08 Jun 2013 18:46:06	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	155,520	155,130	0.9975
01 Jun 2013 16:27:46	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	129,600	129,395	0.9984
27 May 2013 18:40:27	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	103,680	104,120	1.0042
19 May 2013 18:00:55	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	77,760	78,471	1.0091
17 May 2013 20:53:14	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	51,840	52,242	1.0078
11 May 2013 14:00:04	1263648	15769410	hadcm3n_4j4f_1940_40_008306802_1	25,920	26,487	1.0219