Task 13356948

Name	hadcm3n_t4wd_1940_40_007446431_1
Workunit	7643934
Created	9 Sep 2011, 15:36:06 UTC
Sent	17 Sep 2011, 5:47:10 UTC
Report deadline	17 Dec 2011, 13:14:21 UTC
Received	20 Jan 2013, 2:14:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1141779
Run time	18 days 17 hours 49 min 23 sec
CPU time	18 days 9 hours 14 min 17 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.37 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3672, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 14:21:55 (2180): No heartbeat from core client for 30 sec - exiting 14:21:56 (2180): No heartbeat from core client for 30 sec - exiting 14:21:57 (2180): No heartbeat from core client for 30 sec - exiting 14:21:58 (2180): No heartbeat from core client for 30 sec - exiting 14:21:59 (2180): No heartbeat from core client for 30 sec - exiting 14:22:00 (2180): No heartbeat from core client for 30 sec - exiting 14:22:01 (2180): No heartbeat from core client for 30 sec - exiting 14:22:02 (2180): No heartbeat from core client for 30 sec - exiting 14:22:03 (2180): No heartbeat from core client for 30 sec - exiting 14:22:04 (2180): No heartbeat from core client for 30 sec - exiting 14:22:05 (2180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:00:09 (7776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:22:29 (7080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:13:56 (9152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:14:35 (9152): No heartbeat from core client for 30 sec - exiting 14:14:36 (9152): No heartbeat from core client for 30 sec - exiting 14:14:37 (9152): No heartbeat from core client for 30 sec - exiting 14:14:38 (9152): No heartbeat from core client for 30 sec - exiting 14:14:39 (9152): No heartbeat from core client for 30 sec - exiting 14:14:40 (9152): No heartbeat from core client for 30 sec - exiting 14:14:41 (9152): No heartbeat from core client for 30 sec - exiting 14:14:42 (9152): No heartbeat from core client for 30 sec - exiting 14:14:43 (9152): No heartbeat from core client for 30 sec - exiting 14:14:44 (9152): No heartbeat from core client for 30 sec - exiting 14:14:46 (9152): No heartbeat from core client for 30 sec - exiting 14:14:47 (9152): No heartbeat from core client for 30 sec - exiting 14:14:48 (9152): No heartbeat from core client for 30 sec - exiting 14:14:49 (9152): No heartbeat from core client for 30 sec - exiting 14:14:50 (9152): No heartbeat from core client for 30 sec - exiting 14:14:51 (9152): No heartbeat from core client for 30 sec - exiting 14:14:52 (9152): No heartbeat from core client for 30 sec - exiting 14:14:53 (9152): No heartbeat from core client for 30 sec - exiting 14:14:54 (9152): No heartbeat from core client for 30 sec - exiting 14:14:55 (9152): No heartbeat from core client for 30 sec - exiting 14:14:56 (9152): No heartbeat from core client for 30 sec - exiting 14:14:58 (9152): No heartbeat from core client for 30 sec - exiting 14:14:59 (9152): No heartbeat from core client for 30 sec - exiting 14:15:00 (9152): No heartbeat from core client for 30 sec - exiting 14:15:17 (9152): No heartbeat from core client for 30 sec - exiting 14:15:18 (9152): No heartbeat from core client for 30 sec - exiting 14:15:19 (9152): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 21:43:29 (10680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:47:06 (10680): No heartbeat from core client for 30 sec - exiting 21:47:07 (10680): No heartbeat from core client for 30 sec - exiting 21:47:08 (10680): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 01:29:09 (2316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:24:19 (3228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11132, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:46:32 (5672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:25 (5672): No heartbeat from core client for 30 sec - exiting 08:48:27 (5672): No heartbeat from core client for 30 sec - exiting 08:48:28 (5672): No heartbeat from core client for 30 sec - exiting 08:48:29 (5672): No heartbeat from core client for 30 sec - exiting 08:48:30 (5672): No heartbeat from core client for 30 sec - exiting 08:48:31 (5672): No heartbeat from core client for 30 sec - exiting 08:48:32 (5672): No heartbeat from core client for 30 sec - exiting 08:48:33 (5672): No heartbeat from core client for 30 sec - exiting 08:48:34 (5672): No heartbeat from core client for 30 sec - exiting 08:48:35 (5672): No heartbeat from core client for 30 sec - exiting 08:48:36 (5672): No heartbeat from core client for 30 sec - exiting 08:48:37 (5672): No heartbeat from core client for 30 sec - exiting 08:48:39 (5672): No heartbeat from core client for 30 sec - exiting 08:48:40 (5672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:36:03 (6392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:38:57 (5492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:40:52 (5492): No heartbeat from core client for 30 sec - exiting 02:40:53 (5492): No heartbeat from core client for 30 sec - exiting 02:40:54 (5492): No heartbeat from core client for 30 sec - exiting 02:40:55 (5492): No heartbeat from core client for 30 sec - exiting 02:40:56 (5492): No heartbeat from core client for 30 sec - exiting 02:40:57 (5492): No heartbeat from core client for 30 sec - exiting 02:40:58 (5492): No heartbeat from core client for 30 sec - exiting 02:40:59 (5492): No heartbeat from core client for 30 sec - exiting 02:41:00 (5492): No heartbeat from core client for 30 sec - exiting 02:41:01 (5492): No heartbeat from core client for 30 sec - exiting 02:41:02 (5492): No heartbeat from core client for 30 sec - exiting 02:41:03 (5492): No heartbeat from core client for 30 sec - exiting 02:41:04 (5492): No heartbeat from core client for 30 sec - exiting 02:41:05 (5492): No heartbeat from core client for 30 sec - exiting 02:41:06 (5492): No heartbeat from core client for 30 sec - exiting 02:41:07 (5492): No heartbeat from core client for 30 sec - exiting 02:41:08 (5492): No heartbeat from core client for 30 sec - exiting 02:41:09 (5492): No heartbeat from core client for 30 sec - exiting 02:41:10 (5492): No heartbeat from core client for 30 sec - exiting 02:41:11 (5492): No heartbeat from core client for 30 sec - exiting 02:41:12 (5492): No heartbeat from core client for 30 sec - exiting 02:41:13 (5492): No heartbeat from core client for 30 sec - exiting 02:41:14 (5492): No heartbeat from core client for 30 sec - exiting 02:41:15 (5492): No heartbeat from core client for 30 sec - exiting 02:41:16 (5492): No heartbeat from core client for 30 sec - exiting 02:41:17 (5492): No heartbeat from core client for 30 sec - exiting 02:41:18 (5492): No heartbeat from core client for 30 sec - exiting 02:41:19 (5492): No heartbeat from core client for 30 sec - exiting 02:41:20 (5492): No heartbeat from core client for 30 sec - exiting 02:41:21 (5492): No heartbeat from core client for 30 sec - exiting 06:27:18 (10696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 22:12:24 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77713242 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_t4wd_1940_40_007446431/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Jan 2013 02:18:26	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	777,600	1,588,450	2.0428
03 Feb 2012 09:55:05	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	751,680	1,537,837	2.0459
02 Feb 2012 19:36:47	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	725,760	1,488,843	2.0514
13 Jan 2012 13:28:41	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	699,840	1,436,698	2.0529
06 Jan 2012 06:57:39	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	673,920	1,385,222	2.0555
05 Jan 2012 13:30:42	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	648,000	1,337,826	2.0645
02 Jan 2012 14:06:06	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	622,080	1,290,461	2.0744
24 Dec 2011 17:26:52	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	596,160	1,243,757	2.0863
22 Dec 2011 11:29:16	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	570,240	1,196,796	2.0988
21 Dec 2011 06:57:28	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	544,320	1,149,338	2.1115
19 Dec 2011 02:31:16	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	518,400	1,097,257	2.1166
16 Dec 2011 10:31:56	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	492,480	1,040,298	2.1124
09 Dec 2011 06:32:04	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	466,560	987,204	2.1159
06 Dec 2011 07:39:32	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	440,640	927,401	2.1047
03 Dec 2011 12:42:09	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	414,720	871,356	2.1011
02 Dec 2011 08:21:53	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	388,800	816,307	2.0996
22 Nov 2011 10:44:15	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	362,880	762,051	2.1000
19 Nov 2011 05:42:51	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	336,960	706,906	2.0979
19 Nov 2011 05:42:51	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	311,040	652,947	2.0992
08 Nov 2011 09:53:45	1141779	13356948	hadcm3n_t4wd_1940_40_007446431_1	285,120	599,386	2.1022