Task 15438290

Name	hadcm3n_ycgh_1940_40_008239220_2
Workunit	8394344
Created	18 Nov 2012, 1:00:11 UTC
Sent	18 Nov 2012, 1:00:14 UTC
Report deadline	17 Feb 2013, 8:27:25 UTC
Received	19 Dec 2012, 11:42:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1131656
Run time	9 days 8 hours 49 min 55 sec
CPU time	9 days 6 hours 1 min 29 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.48 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:41:56 (4820): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 11:21:37 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:15:42 (1748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:23:37 (5804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8112, iMonCtr=1 Model crash detected, will try to restart... 20:02:06 (4180): No heartbeat from core client for 30 sec - exiting 20:02:07 (4180): No heartbeat from core client for 30 sec - exiting 20:02:08 (4180): No heartbeat from core client for 30 sec - exiting 20:02:09 (4180): No heartbeat from core client for 30 sec - exiting 20:02:10 (4180): No heartbeat from core client for 30 sec - exiting 20:02:11 (4180): No heartbeat from core client for 30 sec - exiting 20:02:12 (4180): No heartbeat from core client for 30 sec - exiting 20:02:13 (4180): No heartbeat from core client for 30 sec - exiting 20:02:14 (4180): No heartbeat from core client for 30 sec - exiting 20:02:15 (4180): No heartbeat from core client for 30 sec - exiting 20:02:16 (4180): No heartbeat from core client for 30 sec - exiting 20:02:17 (4180): No heartbeat from core client for 30 sec - exiting 20:02:18 (4180): No heartbeat from core client for 30 sec - exiting 20:02:19 (4180): No heartbeat from core client for 30 sec - exiting 20:02:20 (4180): No heartbeat from core client for 30 sec - exiting 20:02:21 (4180): No heartbeat from core client for 30 sec - exiting 20:02:22 (4180): No heartbeat from core client for 30 sec - exiting 20:02:23 (4180): No heartbeat from core client for 30 sec - exiting 20:02:24 (4180): No heartbeat from core client for 30 sec - exiting 20:02:25 (4180): No heartbeat from core client for 30 sec - exiting 20:02:26 (4180): No heartbeat from core client for 30 sec - exiting 20:02:27 (4180): No heartbeat from core client for 30 sec - exiting 20:02:28 (4180): No heartbeat from core client for 30 sec - exiting 20:02:29 (4180): No heartbeat from core client for 30 sec - exiting 20:02:30 (4180): No heartbeat from core client for 30 sec - exiting 20:02:31 (4180): No heartbeat from core client for 30 sec - exiting 20:02:32 (4180): No heartbeat from core client for 30 sec - exiting 20:02:33 (4180): No heartbeat from core client for 30 sec - exiting 20:02:34 (4180): No heartbeat from core client for 30 sec - exiting 20:02:35 (4180): No heartbeat from core client for 30 sec - exiting 20:02:36 (4180): No heartbeat from core client for 30 sec - exiting 20:02:37 (4180): No heartbeat from core client for 30 sec - exiting 20:02:38 (4180): No heartbeat from core client for 30 sec - exiting 20:02:39 (4180): No heartbeat from core client for 30 sec - exiting 20:02:40 (4180): No heartbeat from core client for 30 sec - exiting 20:02:41 (4180): No heartbeat from core client for 30 sec - exiting 20:02:42 (4180): No heartbeat from core client for 30 sec - exiting 20:02:43 (4180): No heartbeat from core client for 30 sec - exiting 20:02:44 (4180): No heartbeat from core client for 30 sec - exiting 20:02:45 (4180): No heartbeat from core client for 30 sec - exiting 20:02:46 (4180): No heartbeat from core client for 30 sec - exiting 20:02:47 (4180): No heartbeat from core client for 30 sec - exiting 20:02:48 (4180): No heartbeat from core client for 30 sec - exiting 20:02:49 (4180): No heartbeat from core client for 30 sec - exiting 20:02:50 (4180): No heartbeat from core client for 30 sec - exiting 20:02:51 (4180): No heartbeat from core client for 30 sec - exiting 20:02:52 (4180): No heartbeat from core client for 30 sec - exiting 20:02:53 (4180): No heartbeat from core client for 30 sec - exiting 20:02:54 (4180): No heartbeat from core client for 30 sec - exiting 20:02:55 (4180): No heartbeat from core client for 30 sec - exiting 20:02:56 (4180): No heartbeat from core client for 30 sec - exiting 20:02:57 (4180): No heartbeat from core client for 30 sec - exiting 20:02:58 (4180): No heartbeat from core client for 30 sec - exiting 20:02:59 (4180): No heartbeat from core client for 30 sec - exiting 20:03:00 (4180): No heartbeat from core client for 30 sec - exiting 20:03:01 (4180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:25:26 (4088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:43:35 (3668): No heartbeat from core client for 30 sec - exiting 22:43:36 (3668): No heartbeat from core client for 30 sec - exiting 22:43:37 (3668): No heartbeat from core client for 30 sec - exiting 22:43:38 (3668): No heartbeat from core client for 30 sec - exiting 22:43:39 (3668): No heartbeat from core client for 30 sec - exiting 22:43:40 (3668): No heartbeat from core client for 30 sec - exiting 22:43:41 (3668): No heartbeat from core client for 30 sec - exiting 22:43:42 (3668): No heartbeat from core client for 30 sec - exiting 22:43:43 (3668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3332, iMonCtr=1 Model crash detected, will try to restart... 11:47:01 (2216): No heartbeat from core client for 30 sec - exiting 11:47:02 (2216): No heartbeat from core client for 30 sec - exiting 11:47:03 (2216): No heartbeat from core client for 30 sec - exiting 11:47:04 (2216): No heartbeat from core client for 30 sec - exiting 11:47:05 (2216): No heartbeat from core client for 30 sec - exiting 11:47:06 (2216): No heartbeat from core client for 30 sec - exiting 11:47:07 (2216): No heartbeat from core client for 30 sec - exiting 11:47:08 (2216): No heartbeat from core client for 30 sec - exiting 11:47:09 (2216): No heartbeat from core client for 30 sec - exiting 11:47:10 (2216): No heartbeat from core client for 30 sec - exiting 11:47:11 (2216): No heartbeat from core client for 30 sec - exiting 11:47:12 (2216): No heartbeat from core client for 30 sec - exiting 11:47:13 (2216): No heartbeat from core client for 30 sec - exiting 11:47:14 (2216): No heartbeat from core client for 30 sec - exiting 11:47:15 (2216): No heartbeat from core client for 30 sec - exiting 11:47:16 (2216): No heartbeat from core client for 30 sec - exiting 11:47:17 (2216): No heartbeat from core client for 30 sec - exiting 11:47:18 (2216): No heartbeat from core client for 30 sec - exiting 11:47:19 (2216): No heartbeat from core client for 30 sec - exiting 11:47:20 (2216): No heartbeat from core client for 30 sec - exiting 11:47:21 (2216): No heartbeat from core client for 30 sec - exiting 11:47:22 (2216): No heartbeat from core client for 30 sec - exiting 11:47:23 (2216): No heartbeat from core client for 30 sec - exiting 11:47:24 (2216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:47:25 (2216): No heartbeat from core client for 30 sec - exiting 14:50:07 (1016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:50:08 (1016): No heartbeat from core client for 30 sec - exiting 14:50:09 (1016): No heartbeat from core client for 30 sec - exiting 14:50:10 (1016): No heartbeat from core client for 30 sec - exiting 14:50:11 (1016): No heartbeat from core client for 30 sec - exiting 14:50:12 (1016): No heartbeat from core client for 30 sec - exiting 14:50:13 (1016): No heartbeat from core client for 30 sec - exiting 14:50:14 (1016): No heartbeat from core client for 30 sec - exiting 14:50:15 (1016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:55:43 (3520): No heartbeat from core client for 30 sec - exiting 05:55:44 (3520): No heartbeat from core client for 30 sec - exiting 05:55:45 (3520): No heartbeat from core client for 30 sec - exiting 05:55:46 (3520): No heartbeat from core client for 30 sec - exiting 05:55:47 (3520): No heartbeat from core client for 30 sec - exiting 05:55:48 (3520): No heartbeat from core client for 30 sec - exiting 05:55:49 (3520): No heartbeat from core client for 30 sec - exiting 05:55:50 (3520): No heartbeat from core client for 30 sec - exiting 05:55:51 (3520): No heartbeat from core client for 30 sec - exiting 05:55:52 (3520): No heartbeat from core client for 30 sec - exiting 05:55:53 (3520): No heartbeat from core client for 30 sec - exiting 05:55:54 (3520): No heartbeat from core client for 30 sec - exiting 05:55:55 (3520): No heartbeat from core client for 30 sec - exiting 05:55:56 (3520): No heartbeat from core client for 30 sec - exiting 05:55:57 (3520): No heartbeat from core client for 30 sec - exiting 05:55:58 (3520): No heartbeat from core client for 30 sec - exiting 05:55:59 (3520): No heartbeat from core client for 30 sec - exiting 05:56:00 (3520): No heartbeat from core client for 30 sec - exiting 05:56:01 (3520): No heartbeat from core client for 30 sec - exiting 05:56:02 (3520): No heartbeat from core client for 30 sec - exiting 05:56:03 (3520): No heartbeat from core client for 30 sec - exiting 05:56:04 (3520): No heartbeat from core client for 30 sec - exiting 05:56:05 (3520): No heartbeat from core client for 30 sec - exiting 05:56:06 (3520): No heartbeat from core client for 30 sec - exiting 05:56:07 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:12:30 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:14:35 (320): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C13AB3 read attempt to address 0x40E05C7B Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F53AB3 read attempt to address 0x40E05C7B Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_ycgh_1940_40_008239220/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Dec 2012 19:32:37	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	1,036,800	798,997	0.7706
16 Dec 2012 15:05:47	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	1,010,880	778,946	0.7706
15 Dec 2012 16:10:56	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	984,960	758,445	0.7700
14 Dec 2012 16:15:53	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	959,040	738,733	0.7703
14 Dec 2012 10:36:39	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	933,120	718,673	0.7702
13 Dec 2012 22:27:36	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	907,200	698,611	0.7701
13 Dec 2012 21:26:59	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	881,280	678,458	0.7699
13 Dec 2012 21:26:59	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	855,360	658,256	0.7696
13 Dec 2012 21:26:59	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	829,440	638,325	0.7696
13 Dec 2012 21:26:59	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	803,520	618,302	0.7695
13 Dec 2012 21:26:59	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	777,600	597,908	0.7689
07 Dec 2012 04:41:18	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	751,680	577,731	0.7686
06 Dec 2012 05:02:41	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	725,760	557,952	0.7688
04 Dec 2012 19:33:44	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	699,840	537,901	0.7686
03 Dec 2012 19:26:57	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	673,920	517,449	0.7678
02 Dec 2012 18:59:40	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	648,000	497,393	0.7676
02 Dec 2012 13:34:30	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	622,080	477,974	0.7683
02 Dec 2012 08:28:34	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	596,160	458,580	0.7692
02 Dec 2012 02:47:30	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	570,240	439,195	0.7702
01 Dec 2012 21:30:54	1131656	15438290	hadcm3n_ycgh_1940_40_008239220_2	544,320	419,550	0.7708