Task 14795760

Name	hadcm3n_ym5z_1980_40_007832889_4
Workunit	7988001
Created	18 Jun 2012, 7:57:44 UTC
Sent	18 Jun 2012, 7:57:49 UTC
Report deadline	17 Sep 2012, 15:25:00 UTC
Received	26 Jul 2012, 1:25:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1100141
Run time	19 days 22 hours 53 min 18 sec
CPU time	16 days 7 hours 20 min 42 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.55 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 21:05:05 (8516): No heartbeat from core client for 30 sec - exiting 21:05:06 (8516): No heartbeat from core client for 30 sec - exiting 21:05:07 (8516): No heartbeat from core client for 30 sec - exiting 21:05:09 (8516): No heartbeat from core client for 30 sec - exiting 21:05:10 (8516): No heartbeat from core client for 30 sec - exiting 21:05:11 (8516): No heartbeat from core client for 30 sec - exiting 21:05:12 (8516): No heartbeat from core client for 30 sec - exiting 21:05:13 (8516): No heartbeat from core client for 30 sec - exiting 21:05:14 (8516): No heartbeat from core client for 30 sec - exiting 21:05:15 (8516): No heartbeat from core client for 30 sec - exiting 21:05:16 (8516): No heartbeat from core client for 30 sec - exiting 21:05:17 (8516): No heartbeat from core client for 30 sec - exiting 21:05:18 (8516): No heartbeat from core client for 30 sec - exiting 21:05:19 (8516): No heartbeat from core client for 30 sec - exiting 21:05:21 (8516): No heartbeat from core client for 30 sec - exiting 21:05:22 (8516): No heartbeat from core client for 30 sec - exiting 21:05:23 (8516): No heartbeat from core client for 30 sec - exiting 21:05:24 (8516): No heartbeat from core client for 30 sec - exiting 21:05:25 (8516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:05:27 (8516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7764, selfPID=7764, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:17:05 (5584): No heartbeat from core client for 30 sec - exiting 02:17:06 (5584): No heartbeat from core client for 30 sec - exiting 02:17:07 (5584): No heartbeat from core client for 30 sec - exiting 02:17:08 (5584): No heartbeat from core client for 30 sec - exiting 02:17:09 (5584): No heartbeat from core client for 30 sec - exiting 02:17:10 (5584): No heartbeat from core client for 30 sec - exiting 02:17:11 (5584): No heartbeat from core client for 30 sec - exiting 02:17:12 (5584): No heartbeat from core client for 30 sec - exiting 02:17:13 (5584): No heartbeat from core client for 30 sec - exiting 02:17:14 (5584): No heartbeat from core client for 30 sec - exiting 02:17:15 (5584): No heartbeat from core client for 30 sec - exiting 02:17:16 (5584): No heartbeat from core client for 30 sec - exiting 02:17:17 (5584): No heartbeat from core client for 30 sec - exiting 02:17:18 (5584): No heartbeat from core client for 30 sec - exiting 02:17:19 (5584): No heartbeat from core client for 30 sec - exiting 02:17:20 (5584): No heartbeat from core client for 30 sec - exiting 02:17:21 (5584): No heartbeat from core client for 30 sec - exiting 02:17:22 (5584): No heartbeat from core client for 30 sec - exiting 02:17:23 (5584): No heartbeat from core client for 30 sec - exiting 02:17:24 (5584): No heartbeat from core client for 30 sec - exiting 02:17:25 (5584): No heartbeat from core client for 30 sec - exiting 02:17:26 (5584): No heartbeat from core client for 30 sec - exiting 02:17:27 (5584): No heartbeat from core client for 30 sec - exiting 02:17:28 (5584): No heartbeat from core client for 30 sec - exiting 02:17:29 (5584): No heartbeat from core client for 30 sec - exiting 02:17:30 (5584): No heartbeat from core client for 30 sec - exiting 02:17:31 (5584): No heartbeat from core client for 30 sec - exiting 02:17:32 (5584): No heartbeat from core client for 30 sec - exiting 02:17:33 (5584): No heartbeat from core client for 30 sec - exiting 02:17:34 (5584): No heartbeat from core client for 30 sec - exiting 02:17:35 (5584): No heartbeat from core client for 30 sec - exiting 02:17:36 (5584): No heartbeat from core client for 30 sec - exiting 02:17:37 (5584): No heartbeat from core client for 30 sec - exiting 02:17:38 (5584): No heartbeat from core client for 30 sec - exiting 02:17:39 (5584): No heartbeat from core client for 30 sec - exiting 02:17:40 (5584): No heartbeat from core client for 30 sec - exiting 02:17:41 (5584): No heartbeat from core client for 30 sec - exiting 02:17:42 (5584): No heartbeat from core client for 30 sec - exiting 02:17:43 (5584): No heartbeat from core client for 30 sec - exiting 02:17:44 (5584): No heartbeat from core client for 30 sec - exiting 02:17:45 (5584): No heartbeat from core client for 30 sec - exiting 02:17:46 (5584): No heartbeat from core client for 30 sec - exiting 02:17:47 (5584): No heartbeat from core client for 30 sec - exiting 02:17:48 (5584): No heartbeat from core client for 30 sec - exiting 02:17:49 (5584): No heartbeat from core client for 30 sec - exiting 02:17:50 (5584): No heartbeat from core client for 30 sec - exiting 02:17:51 (5584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:17:52 (5584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:33:06 (10172): No heartbeat from core client for 30 sec - exiting 03:33:07 (10172): No heartbeat from core client for 30 sec - exiting 03:33:08 (10172): No heartbeat from core client for 30 sec - exiting 03:33:09 (10172): No heartbeat from core client for 30 sec - exiting 03:33:10 (10172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:11:09 (4776): No heartbeat from core client for 30 sec - exiting 19:11:10 (4776): No heartbeat from core client for 30 sec - exiting 19:11:11 (4776): No heartbeat from core client for 30 sec - exiting 19:11:12 (4776): No heartbeat from core client for 30 sec - exiting 19:11:13 (4776): No heartbeat from core client for 30 sec - exiting 19:11:15 (4776): No heartbeat from core client for 30 sec - exiting 19:11:16 (4776): No heartbeat from core client for 30 sec - exiting 19:11:17 (4776): No heartbeat from core client for 30 sec - exiting 19:11:18 (4776): No heartbeat from core client for 30 sec - exiting 19:11:19 (4776): No heartbeat from core client for 30 sec - exiting 19:11:20 (4776): No heartbeat from core client for 30 sec - exiting 19:11:21 (4776): No heartbeat from core client for 30 sec - exiting 19:11:22 (4776): No heartbeat from core client for 30 sec - exiting 19:11:23 (4776): No heartbeat from core client for 30 sec - exiting 19:11:24 (4776): No heartbeat from core client for 30 sec - exiting 19:11:25 (4776): No heartbeat from core client for 30 sec - exiting 19:11:27 (4776): No heartbeat from core client for 30 sec - exiting 19:11:28 (4776): No heartbeat from core client for 30 sec - exiting 19:11:29 (4776): No heartbeat from core client for 30 sec - exiting 19:11:30 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6100, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A23AB3 read attempt to address 0x40B9A1CA Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77703AB3 read attempt to address 0x40B9A1CA Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_ym5z_1980_40_007832889/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Jul 2012 14:02:57	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	1,036,800	1,408,361	1.3584
22 Jul 2012 02:40:45	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	1,010,880	1,373,475	1.3587
21 Jul 2012 13:57:14	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	984,960	1,338,330	1.3588
21 Jul 2012 00:59:45	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	959,040	1,303,315	1.3590
20 Jul 2012 08:27:01	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	933,120	1,266,858	1.3577
19 Jul 2012 16:48:31	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	907,200	1,230,936	1.3569
19 Jul 2012 03:10:08	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	881,280	1,195,747	1.3568
18 Jul 2012 12:34:51	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	855,360	1,160,429	1.3567
17 Jul 2012 22:26:39	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	829,440	1,125,847	1.3574
17 Jul 2012 10:40:42	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	803,520	1,090,632	1.3573
16 Jul 2012 23:43:12	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	777,600	1,055,682	1.3576
16 Jul 2012 12:17:08	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	751,680	1,020,744	1.3580
15 Jul 2012 23:39:03	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	725,760	986,024	1.3586
15 Jul 2012 11:07:44	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	699,840	950,885	1.3587
14 Jul 2012 22:55:16	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	673,920	915,976	1.3592
14 Jul 2012 11:05:26	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	648,000	881,054	1.3597
13 Jul 2012 23:33:11	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	622,080	845,905	1.3598
13 Jul 2012 12:00:08	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	596,160	810,806	1.3600
12 Jul 2012 22:07:02	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	570,240	775,529	1.3600
11 Jul 2012 22:13:59	1100141	14795760	hadcm3n_ym5z_1980_40_007832889_4	544,320	740,296	1.3600