Task 15775161

Name	hadcm3n_zj27_1920_40_008365332_0
Workunit	8516191
Created	11 May 2013, 0:35:07 UTC
Sent	11 May 2013, 0:41:05 UTC
Report deadline	10 Aug 2013, 8:08:16 UTC
Received	23 Jun 2013, 23:48:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1254207
Run time	10 days 6 hours 42 min 45 sec
CPU time	9 days 2 hours 15 min 39 sec
Validate state	Invalid
Credit	5,909.76
Device peak FLOPS	3.03 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:23:59 (780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:24:08 (780): No heartbeat from core client for 30 sec - exiting 16:24:10 (780): No heartbeat from core client for 30 sec - exiting 16:24:11 (780): No heartbeat from core client for 30 sec - exiting 16:24:12 (780): No heartbeat from core client for 30 sec - exiting 16:24:13 (780): No heartbeat from core client for 30 sec - exiting 16:24:14 (780): No heartbeat from core client for 30 sec - exiting 16:24:15 (780): No heartbeat from core client for 30 sec - exiting 16:24:16 (780): No heartbeat from core client for 30 sec - exiting 16:24:17 (780): No heartbeat from core client for 30 sec - exiting 16:24:18 (780): No heartbeat from core client for 30 sec - exiting 16:24:19 (780): No heartbeat from core client for 30 sec - exiting 16:24:20 (780): No heartbeat from core client for 30 sec - exiting 16:26:52 (2528): No heartbeat from core client for 30 sec - exiting 16:26:53 (2528): No heartbeat from core client for 30 sec - exiting 16:26:54 (2528): No heartbeat from core client for 30 sec - exiting 16:26:55 (2528): No heartbeat from core client for 30 sec - exiting 16:26:56 (2528): No heartbeat from core client for 30 sec - exiting 16:26:57 (2528): No heartbeat from core client for 30 sec - exiting 16:26:58 (2528): No heartbeat from core client for 30 sec - exiting 16:26:59 (2528): No heartbeat from core client for 30 sec - exiting 16:27:00 (2528): No heartbeat from core client for 30 sec - exiting 16:27:01 (2528): No heartbeat from core client for 30 sec - exiting 16:27:02 (2528): No heartbeat from core client for 30 sec - exiting 16:27:03 (2528): No heartbeat from core client for 30 sec - exiting 16:27:04 (2528): No heartbeat from core client for 30 sec - exiting 16:27:05 (2528): No heartbeat from core client for 30 sec - exiting 16:27:06 (2528): No heartbeat from core client for 30 sec - exiting 16:27:07 (2528): No heartbeat from core client for 30 sec - exiting 16:27:08 (2528): No heartbeat from core client for 30 sec - exiting 16:27:09 (2528): No heartbeat from core client for 30 sec - exiting 16:27:10 (2528): No heartbeat from core client for 30 sec - exiting 16:27:11 (2528): No heartbeat from core client for 30 sec - exiting 16:27:12 (2528): No heartbeat from core client for 30 sec - exiting 16:27:13 (2528): No heartbeat from core client for 30 sec - exiting 16:27:14 (2528): No heartbeat from core client for 30 sec - exiting 16:27:15 (2528): No heartbeat from core client for 30 sec - exiting 16:27:16 (2528): No heartbeat from core client for 30 sec - exiting 16:27:17 (2528): No heartbeat from core client for 30 sec - exiting 16:27:18 (2528): No heartbeat from core client for 30 sec - exiting 16:27:19 (2528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:19:59 (4804): No heartbeat from core client for 30 sec - exiting 16:20:00 (4804): No heartbeat from core client for 30 sec - exiting 16:20:02 (4804): No heartbeat from core client for 30 sec - exiting 16:20:03 (4804): No heartbeat from core client for 30 sec - exiting 16:20:04 (4804): No heartbeat from core client for 30 sec - exiting 16:20:05 (4804): No heartbeat from core client for 30 sec - exiting 16:20:06 (4804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5664, iMonCtr=1 Model crash detected, will try to restart... 13:10:01 (3476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4844, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7252, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:07:23 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:21:56 (1228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:31:02 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:58:17 (3844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:59:27 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:33:28 (4896): No heartbeat from core client for 30 sec - exiting 10:33:29 (4896): No heartbeat from core client for 30 sec - exiting 10:33:30 (4896): No heartbeat from core client for 30 sec - exiting 10:33:31 (4896): No heartbeat from core client for 30 sec - exiting 10:33:33 (4896): No heartbeat from core client for 30 sec - exiting 10:33:34 (4896): No heartbeat from core client for 30 sec - exiting 10:33:35 (4896): No heartbeat from core client for 30 sec - exiting 10:33:36 (4896): No heartbeat from core client for 30 sec - exiting 10:33:37 (4896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:52:08 (6308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8328, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 14:01:12 (3540): No heartbeat from core client for 30 sec - exiting 14:01:13 (3540): No heartbeat from core client for 30 sec - exiting 14:01:14 (3540): No heartbeat from core client for 30 sec - exiting 14:01:15 (3540): No heartbeat from core client for 30 sec - exiting 14:01:16 (3540): No heartbeat from core client for 30 sec - exiting 14:01:17 (3540): No heartbeat from core client for 30 sec - exiting 14:01:18 (3540): No heartbeat from core client for 30 sec - exiting 14:01:19 (3540): No heartbeat from core client for 30 sec - exiting 14:01:20 (3540): No heartbeat from core client for 30 sec - exiting 14:01:21 (3540): No heartbeat from core client for 30 sec - exiting 14:01:23 (3540): No heartbeat from core client for 30 sec - exiting 14:01:24 (3540): No heartbeat from core client for 30 sec - exiting 14:01:25 (3540): No heartbeat from core client for 30 sec - exiting 14:01:26 (3540): No heartbeat from core client for 30 sec - exiting 14:01:27 (3540): No heartbeat from core client for 30 sec - exiting 14:01:29 (3540): No heartbeat from core client for 30 sec - exiting 14:01:30 (3540): No heartbeat from core client for 30 sec - exiting 14:01:31 (3540): No heartbeat from core client for 30 sec - exiting 14:01:32 (3540): No heartbeat from core client for 30 sec - exiting 14:01:33 (3540): No heartbeat from core client for 30 sec - exiting 14:01:34 (3540): No heartbeat from core client for 30 sec - exiting 14:01:35 (3540): No heartbeat from core client for 30 sec - exiting 14:01:36 (3540): No heartbeat from core client for 30 sec - exiting 14:01:38 (3540): No heartbeat from core client for 30 sec - exiting 14:01:39 (3540): No heartbeat from core client for 30 sec - exiting 14:01:40 (3540): No heartbeat from core client for 30 sec - exiting 14:01:41 (3540): No heartbeat from core client for 30 sec - exiting 14:01:42 (3540): No heartbeat from core client for 30 sec - exiting 14:01:43 (3540): No heartbeat from core client for 30 sec - exiting 14:01:44 (3540): No heartbeat from core client for 30 sec - exiting 14:01:45 (3540): No heartbeat from core client for 30 sec - exiting 14:01:46 (3540): No heartbeat from core client for 30 sec - exiting 14:01:47 (3540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:47:40 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:57:32 (3360): No heartbeat from core client for 30 sec - exiting 07:57:34 (3360): No heartbeat from core client for 30 sec - exiting 07:57:35 (3360): No heartbeat from core client for 30 sec - exiting 07:57:36 (3360): No heartbeat from core client for 30 sec - exiting 07:57:37 (3360): No heartbeat from core client for 30 sec - exiting 07:57:38 (3360): No heartbeat from core client for 30 sec - exiting 07:57:39 (3360): No heartbeat from core client for 30 sec - exiting 07:57:40 (3360): No heartbeat from core client for 30 sec - exiting 07:57:41 (3360): No heartbeat from core client for 30 sec - exiting 07:57:42 (3360): No heartbeat from core client for 30 sec - exiting 07:57:43 (3360): No heartbeat from core client for 30 sec - exiting 07:57:44 (3360): No heartbeat from core client for 30 sec - exiting 07:57:45 (3360): No heartbeat from core client for 30 sec - exiting 07:57:46 (3360): No heartbeat from core client for 30 sec - exiting 07:57:47 (3360): No heartbeat from core client for 30 sec - exiting 07:57:48 (3360): No heartbeat from core client for 30 sec - exiting 07:57:49 (3360): No heartbeat from core client for 30 sec - exiting 07:57:50 (3360): No heartbeat from core client for 30 sec - exiting 07:57:51 (3360): No heartbeat from core client for 30 sec - exiting 07:57:52 (3360): No heartbeat from core client for 30 sec - exiting 07:57:53 (3360): No heartbeat from core client for 30 sec - exiting 07:57:54 (3360): No heartbeat from core client for 30 sec - exiting 07:57:55 (3360): No heartbeat from core client for 30 sec - exiting 07:57:56 (3360): No heartbeat from core client for 30 sec - exiting 07:57:57 (3360): No heartbeat from core client for 30 sec - exiting 07:57:58 (3360): No heartbeat from core client for 30 sec - exiting 07:58:00 (3360): No heartbeat from core client for 30 sec - exiting 07:58:01 (3360): No heartbeat from core client for 30 sec - exiting 07:58:02 (3360): No heartbeat from core client for 30 sec - exiting 07:58:03 (3360): No heartbeat from core client for 30 sec - exiting 07:58:04 (3360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Jun 2013 09:44:28	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	492,480	757,158	1.5374
14 Jun 2013 06:46:45	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	466,560	715,300	1.5331
09 Jun 2013 22:07:19	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	440,640	674,569	1.5309
09 Jun 2013 02:42:17	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	414,720	632,283	1.5246
08 Jun 2013 12:04:57	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	388,800	588,481	1.5136
07 Jun 2013 18:02:33	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	362,880	546,845	1.5070
07 Jun 2013 01:52:31	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	336,960	505,259	1.4995
04 Jun 2013 19:16:49	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	311,040	468,035	1.5047
03 Jun 2013 18:24:02	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	285,120	428,357	1.5024
03 Jun 2013 04:04:46	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	259,200	389,026	1.5009
02 Jun 2013 17:02:02	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	233,280	350,709	1.5034
01 Jun 2013 08:08:33	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	207,360	313,244	1.5106
31 May 2013 21:23:04	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	181,440	275,540	1.5186
30 May 2013 11:20:24	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	155,520	237,353	1.5262
29 May 2013 15:45:21	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	129,600	200,456	1.5467
29 May 2013 04:36:29	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	103,680	162,802	1.5702
27 May 2013 08:28:48	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	77,760	121,566	1.5633
22 May 2013 23:54:56	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	51,840	80,438	1.5517
14 May 2013 05:37:33	1254207	15775161	hadcm3n_zj27_1920_40_008365332_0	25,920	39,950	1.5413