Task 13745184

Name	hadcm3n_t2lb_1940_40_007536096_4
Workunit	7733328
Created	7 Dec 2011, 11:26:24 UTC
Sent	7 Dec 2011, 11:39:14 UTC
Report deadline	7 Mar 2012, 19:06:25 UTC
Received	23 Jan 2012, 15:40:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1424609
Run time	21 days 11 hours 2 min 15 sec
CPU time	19 days 8 hours 56 min 39 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	1.72 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:38:52 (4584): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:55:31 (6280): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:55:32 (6280): No heartbeat from core client for 30 sec - exiting 03:55:33 (6280): No heartbeat from core client for 30 sec - exiting 03:55:34 (6280): No heartbeat from core client for 30 sec - exiting 03:55:35 (6280): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4940, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:02:48 (3840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:43:09 (764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:43:10 (764): No heartbeat from core client for 30 sec - exiting 21:43:11 (764): No heartbeat from core client for 30 sec - exiting 21:43:12 (764): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:46:21 (3872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C19:34:44 (5032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:34:45 (5032): No heartbeat from core client for 30 sec - exiting 19:34:46 (5032): No heartbeat from core client for 30 sec - exiting 19:34:47 (5032): No heartbeat from core client for 30 sec - exiting 19:34:48 (5032): No heartbeat from core client for 30 sec - exiting 19:34:49 (5032): No heartbeat from core client for 30 sec - exiting 19:34:50 (5032): No heartbeat from core client for 30 sec - exiting 19:34:51 (5032): No heartbeat from core client for 30 sec - exiting 19:34:52 (5032): No heartbeat from core client for 30 sec - exiting 19:34:53 (5032): No heartbeat from core client for 30 sec - exiting 19:34:54 (5032): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:14:25 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:14:26 (3736): No heartbeat from core client for 30 sec - exiting 18:14:27 (3736): No heartbeat from core client for 30 sec - exiting 19:27:13 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:55:10 (1960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:04:14 (4908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:13:42 (4164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:13:43 (4164): No heartbeat from core client for 30 sec - exiting 05:09:06 (1108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jan 2012 15:01:52	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	777,600	1,673,795	2.1525
22 Jan 2012 20:06:57	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	751,680	1,614,638	2.1480
21 Jan 2012 15:42:16	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	725,760	1,548,099	2.1331
20 Jan 2012 12:15:38	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	699,840	1,494,494	2.1355
19 Jan 2012 19:05:25	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	673,920	1,442,680	2.1407
18 Jan 2012 15:05:38	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	648,000	1,391,875	2.1480
17 Jan 2012 13:44:22	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	622,080	1,339,122	2.1527
16 Jan 2012 21:45:39	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	596,160	1,286,424	2.1579
15 Jan 2012 13:11:21	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	570,240	1,231,625	2.1598
14 Jan 2012 21:12:49	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	544,320	1,179,329	2.1666
14 Jan 2012 06:06:38	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	518,400	1,128,773	2.1774
13 Jan 2012 17:39:44	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	492,480	1,076,383	2.1856
13 Jan 2012 17:39:44	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	466,560	1,021,252	2.1889
13 Jan 2012 17:39:44	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	440,640	966,512	2.1934
13 Jan 2012 17:39:44	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	414,720	910,963	2.1966
13 Jan 2012 17:39:44	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	388,800	854,414	2.1976
13 Jan 2012 17:39:44	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	362,880	800,021	2.2046
13 Jan 2012 17:39:44	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	336,960	745,361	2.2120
05 Jan 2012 17:41:56	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	311,040	667,017	2.1445
19 Dec 2011 19:32:49	1155189	13745184	hadcm3n_t2lb_1940_40_007536096_4	285,120	593,403	2.0812