Task 13290528

Name	hadcm3n_p5ol_1940_40_007421161_1
Workunit	7618796
Created	25 Aug 2011, 1:23:42 UTC
Sent	25 Aug 2011, 1:28:08 UTC
Report deadline	24 Nov 2011, 8:55:19 UTC
Received	13 Sep 2011, 15:52:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	775427
Run time	13 days 1 hours 10 min 13 sec
CPU time	11 days 23 hours 29 min 5 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 17:27:33 (3060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:53:51 (6472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:27:31 (4572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:21:22 (7872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1388, iMonCtr=1 Model crash detected, will try to restart... 09:37:25 (4932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6960, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5384, iMonCtr=1 Model crash detected, will try to restart... 10:52:11 (5224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... 21:56:44 (3832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4200, iMonCtr=1 Model crash detected, will try to restart... 22:23:40 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5848, iMonCtr=1 Model crash detected, will try to restart... 08:46:11 (5160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3072, iMonCtr=1 Model crash detected, will try to restart... 17:02:43 (5960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:03 (244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:03:26 (6032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:04:06 (2832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:22 (4124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:02:00 (2320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:36:21 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:37:53 (5348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:04:41 (3792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:04:42 (3792): No heartbeat from core client for 30 sec - exiting 12:05:33 (2004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:06:27 (1368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:45:00 (3888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:45:01 (3888): No heartbeat from core client for 30 sec - exiting 12:46:42 (5564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:50:12 (5264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:00:01 (4664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:04:49 (5276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:41 (4000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6000, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4912, iMonCtr=1 Model crash detected, will try to restart... 10:23:09 (3224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:05:19 (3300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:13:19 (2732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:15:42 (7044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7120, iMonCtr=1 Model crash detected, will try to restart... 09:41:37 (5320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:58:04 (3844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:43:32 (7816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:38:32 (7048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:35:31 (5120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5412, iMonCtr=1 Model crash detected, will try to restart... 13:21:04 (2080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:21:45 (7460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:25:25 (7256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:26 (7256): No heartbeat from core client for 30 sec - exiting 22:07:15 (7976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:16 (7976): No heartbeat from core client for 30 sec - exiting 23:08:43 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5148, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2828, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Sep 2011 15:51:11	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	518,400	1,034,939	1.9964
12 Sep 2011 07:33:28	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	492,480	983,186	1.9964
11 Sep 2011 16:48:17	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	466,560	930,398	1.9942
11 Sep 2011 00:39:48	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	440,640	877,847	1.9922
09 Sep 2011 21:38:06	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	414,720	825,611	1.9908
08 Sep 2011 20:18:17	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	388,800	772,601	1.9871
07 Sep 2011 15:10:08	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	362,880	721,362	1.9879
06 Sep 2011 17:45:01	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	336,960	671,269	1.9921
06 Sep 2011 03:06:00	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	311,040	620,572	1.9952
05 Sep 2011 01:09:57	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	285,120	569,287	1.9967
03 Sep 2011 19:34:33	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	259,200	518,300	1.9996
02 Sep 2011 19:07:51	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	233,280	467,456	2.0038
01 Sep 2011 19:48:55	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	207,360	417,012	2.0111
01 Sep 2011 00:54:23	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	181,440	364,114	2.0068
30 Aug 2011 22:32:49	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	155,520	312,007	2.0062
30 Aug 2011 00:00:34	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	129,600	259,818	2.0048
29 Aug 2011 00:11:43	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	103,680	207,508	2.0014
28 Aug 2011 00:20:06	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	77,760	154,743	1.9900
27 Aug 2011 09:44:18	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	51,840	103,718	2.0007
26 Aug 2011 17:54:52	775427	13290528	hadcm3n_p5ol_1940_40_007421161_1	25,920	51,854	2.0005