Task 16608137

Name	hadcm3n_8bfq_1980_40_008724241_1
Workunit	8870219
Created	3 May 2014, 3:18:55 UTC
Sent	3 May 2014, 5:30:49 UTC
Report deadline	2 Aug 2014, 12:58:00 UTC
Received	28 Jun 2014, 14:57:21 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1149805
Run time	13 days 16 hours 30 min 17 sec
CPU time	12 days 19 hours 15 min 33 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:41:47 (4592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:01:44 (2468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:01:45 (2468): No heartbeat from core client for 30 sec - exiting 20:02:43 (2460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:06:20 (5768): No heartbeat from core client for 30 sec - exiting 10:06:21 (5768): No heartbeat from core client for 30 sec - exiting 10:06:22 (5768): No heartbeat from core client for 30 sec - exiting 10:06:23 (5768): No heartbeat from core client for 30 sec - exiting 10:06:24 (5768): No heartbeat from core client for 30 sec - exiting 10:06:25 (5768): No heartbeat from core client for 30 sec - exiting 10:06:26 (5768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5548, iMonCtr=1 Model crash detected, will try to restart... 12:15:48 (3612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:51:00 (5952): No heartbeat from core client for 30 sec - exiting 08:51:01 (5952): No heartbeat from core client for 30 sec - exiting 08:51:02 (5952): No heartbeat from core client for 30 sec - exiting 08:51:03 (5952): No heartbeat from core client for 30 sec - exiting 08:51:04 (5952): No heartbeat from core client for 30 sec - exiting 08:51:05 (5952): No heartbeat from core client for 30 sec - exiting 08:51:06 (5952): No heartbeat from core client for 30 sec - exiting 08:51:07 (5952): No heartbeat from core client for 30 sec - exiting 08:51:08 (5952): No heartbeat from core client for 30 sec - exiting 08:51:09 (5952): No heartbeat from core client for 30 sec - exiting 08:51:10 (5952): No heartbeat from core client for 30 sec - exiting 08:51:11 (5952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:52:42 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:31:28 (5404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:50 (5556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:26:46 (4604): No heartbeat from core client for 30 sec - exiting 08:26:47 (4604): No heartbeat from core client for 30 sec - exiting 08:26:48 (4604): No heartbeat from core client for 30 sec - exiting 08:26:49 (4604): No heartbeat from core client for 30 sec - exiting 08:26:50 (4604): No heartbeat from core client for 30 sec - exiting 08:26:51 (4604): No heartbeat from core client for 30 sec - exiting 08:26:52 (4604): No heartbeat from core client for 30 sec - exiting 08:26:53 (4604): No heartbeat from core client for 30 sec - exiting 08:26:54 (4604): No heartbeat from core client for 30 sec - exiting 08:26:55 (4604): No heartbeat from core client for 30 sec - exiting 08:26:56 (4604): No heartbeat from core client for 30 sec - exiting 08:26:57 (4604): No heartbeat from core client for 30 sec - exiting 08:26:58 (4604): No heartbeat from core client for 30 sec - exiting 08:26:59 (4604): No heartbeat from core client for 30 sec - exiting 08:27:00 (4604): No heartbeat from core client for 30 sec - exiting 08:27:01 (4604): No heartbeat from core client for 30 sec - exiting 08:27:02 (4604): No heartbeat from core client for 30 sec - exiting 08:27:03 (4604): No heartbeat from core client for 30 sec - exiting 08:27:04 (4604): No heartbeat from core client for 30 sec - exiting 08:27:05 (4604): No heartbeat from core client for 30 sec - exiting 08:27:06 (4604): No heartbeat from core client for 30 sec - exiting 08:27:07 (4604): No heartbeat from core client for 30 sec - exiting 08:27:08 (4604): No heartbeat from core client for 30 sec - exiting 08:27:09 (4604): No heartbeat from core client for 30 sec - exiting 08:27:10 (4604): No heartbeat from core client for 30 sec - exiting 08:27:11 (4604): No heartbeat from core client for 30 sec - exiting 08:27:12 (4604): No heartbeat from core client for 30 sec - exiting 08:27:13 (4604): No heartbeat from core client for 30 sec - exiting 08:27:14 (4604): No heartbeat from core client for 30 sec - exiting 08:27:15 (4604): No heartbeat from core client for 30 sec - exiting 08:27:16 (4604): No heartbeat from core client for 30 sec - exiting 08:27:17 (4604): No heartbeat from core client for 30 sec - exiting 08:27:18 (4604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:28:44 (3716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:27:12 (3840): No heartbeat from core client for 30 sec - exiting 10:27:13 (3840): No heartbeat from core client for 30 sec - exiting 10:27:14 (3840): No heartbeat from core client for 30 sec - exiting 10:27:15 (3840): No heartbeat from core client for 30 sec - exiting 10:27:16 (3840): No heartbeat from core client for 30 sec - exiting 10:27:17 (3840): No heartbeat from core client for 30 sec - exiting 10:27:19 (3840): No heartbeat from core client for 30 sec - exiting 10:27:20 (3840): No heartbeat from core client for 30 sec - exiting 10:27:21 (3840): No heartbeat from core client for 30 sec - exiting 10:27:22 (3840): No heartbeat from core client for 30 sec - exiting 10:27:23 (3840): No heartbeat from core client for 30 sec - exiting 10:27:24 (3840): No heartbeat from core client for 30 sec - exiting 10:27:25 (3840): No heartbeat from core client for 30 sec - exiting 10:27:26 (3840): No heartbeat from core client for 30 sec - exiting 10:27:27 (3840): No heartbeat from core client for 30 sec - exiting 10:27:28 (3840): No heartbeat from core client for 30 sec - exiting 10:27:29 (3840): No heartbeat from core client for 30 sec - exiting 10:27:30 (3840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:27:31 (3840): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 08:37:15 (3468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:56:14 (4736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:56:15 (4736): No heartbeat from core client for 30 sec - exiting 07:56:16 (4736): No heartbeat from core client for 30 sec - exiting 07:56:17 (4736): No heartbeat from core client for 30 sec - exiting 07:56:18 (4736): No heartbeat from core client for 30 sec - exiting 07:56:19 (4736): No heartbeat from core client for 30 sec - exiting 07:56:20 (4736): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:08:44 (4808): No heartbeat from core client for 30 sec - exiting 11:08:46 (4808): No heartbeat from core client for 30 sec - exiting 11:08:47 (4808): No heartbeat from core client for 30 sec - exiting 11:08:48 (4808): No heartbeat from core client for 30 sec - exiting 11:08:49 (4808): No heartbeat from core client for 30 sec - exiting 11:08:50 (4808): No heartbeat from core client for 30 sec - exiting 11:08:51 (4808): No heartbeat from core client for 30 sec - exiting 11:08:52 (4808): No heartbeat from core client for 30 sec - exiting 11:08:53 (4808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:08:54 (4808): No heartbeat from core client for 30 sec - exiting 11:08:55 (4808): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2436, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2432, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:11:04 (2040): No heartbeat from core client for 30 sec - exiting 08:11:06 (2040): No heartbeat from core client for 30 sec - exiting 08:11:07 (2040): No heartbeat from core client for 30 sec - exiting 08:11:08 (2040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:13:27 (5020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:56:04 (5024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:33:45 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:05:05 (3492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:07:18 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773D3AC3 read attempt to address 0x4104B0C9 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773D3AC3 read attempt to address 0x4104B0C9 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_8bfq_1980_40_008724241/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Jun 2014 08:09:45	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	777,600	1,106,076	1.4224
27 Jun 2014 12:19:02	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	751,680	1,071,772	1.4258
26 Jun 2014 09:34:26	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	725,760	1,036,239	1.4278
25 Jun 2014 10:02:44	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	699,840	1,000,010	1.4289
23 Jun 2014 15:24:16	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	673,920	965,493	1.4327
20 Jun 2014 14:17:29	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	648,000	929,885	1.4350
20 Jun 2014 06:31:01	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	622,080	894,371	1.4377
14 Jun 2014 08:29:02	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	596,160	858,195	1.4395
12 Jun 2014 05:34:05	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	570,240	823,324	1.4438
11 Jun 2014 06:11:43	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	544,320	790,782	1.4528
10 Jun 2014 04:43:50	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	518,400	757,935	1.4621
06 Jun 2014 14:28:36	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	492,480	717,238	1.4564
05 Jun 2014 19:19:44	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	466,560	685,235	1.4687
05 Jun 2014 11:15:10	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	440,640	658,024	1.4933
05 Jun 2014 06:19:26	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	414,720	629,609	1.5182
25 May 2014 09:11:38	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	388,800	587,902	1.5121
23 May 2014 14:21:27	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	362,880	545,461	1.5031
20 May 2014 08:31:50	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	336,960	506,126	1.5020
19 May 2014 06:32:56	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	311,040	468,374	1.5058
18 May 2014 07:19:41	1149805	16608137	hadcm3n_8bfq_1980_40_008724241_1	285,120	431,395	1.5130