Task 13332771

Name	hadcm3n_t3f7_1940_40_007315149_2
Workunit	7512579
Created	5 Sep 2011, 3:25:04 UTC
Sent	5 Sep 2011, 3:25:06 UTC
Report deadline	5 Dec 2011, 10:52:17 UTC
Received	22 Nov 2011, 13:07:39 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1080897
Run time	19 days 8 hours 40 min 31 sec
CPU time	19 days 8 hours 40 min 31 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.34 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 12:07:53 (2112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2004, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2964, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4424, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2164, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2740, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4800, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2916, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3008, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2816, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3028, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3328, iMonCtr=1 Model crash detected, will try to restart... 19:49:48 (3236): No heartbeat from core client for 30 sec - exiting 19:49:51 (3236): No heartbeat from core client for 30 sec - exiting 19:49:53 (3236): No heartbeat from core client for 30 sec - exiting 19:49:54 (3236): No heartbeat from core client for 30 sec - exiting 19:49:55 (3236): No heartbeat from core client for 30 sec - exiting 19:49:56 (3236): No heartbeat from core client for 30 sec - exiting 19:49:57 (3236): No heartbeat from core client for 30 sec - exiting 19:49:58 (3236): No heartbeat from core client for 30 sec - exiting 19:49:59 (3236): No heartbeat from core client for 30 sec - exiting 19:50:00 (3236): No heartbeat from core client for 30 sec - exiting 19:50:01 (3236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2884, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2364, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 08:09:27 (3048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:20 (3048): No heartbeat from core client for 30 sec - exiting 15:35:29 (2992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A73A93 read attempt to address 0x403FA800 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x004703A7 write attempt to address 0x77F5334B Engaging BOINC Windows Runtime Debugger... Signal 22 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Nov 2011 02:47:21	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	777,600	1,672,772	2.1512
20 Nov 2011 17:09:44	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	751,680	1,617,061	2.1513
20 Nov 2011 01:36:15	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	725,760	1,561,297	2.1513
18 Nov 2011 12:50:16	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	699,840	1,504,684	2.1500
17 Nov 2011 20:55:04	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	673,920	1,448,183	2.1489
15 Nov 2011 20:38:32	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	648,000	1,390,280	2.1455
15 Nov 2011 20:38:32	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	622,080	1,333,338	2.1434
08 Nov 2011 01:01:15	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	596,160	1,278,154	2.1440
06 Nov 2011 16:07:30	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	570,240	1,223,174	2.1450
05 Nov 2011 03:17:36	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	544,320	1,166,996	2.1440
02 Nov 2011 22:54:46	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	518,400	1,109,836	2.1409
31 Oct 2011 19:25:42	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	492,480	1,053,950	2.1401
31 Oct 2011 18:46:59	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	466,560	999,132	2.1415
31 Oct 2011 17:19:42	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	440,640	944,438	2.1433
31 Oct 2011 15:26:09	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	414,720	889,654	2.1452
31 Oct 2011 15:26:09	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	388,800	833,643	2.1441
31 Oct 2011 15:26:09	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	362,880	777,652	2.1430
12 Oct 2011 23:50:28	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	336,960	721,973	2.1426
10 Oct 2011 00:05:55	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	311,040	666,974	2.1443
04 Oct 2011 21:38:38	1080897	13332771	hadcm3n_t3f7_1940_40_007315149_2	285,120	610,490	2.1412