Task 12732602

Name	hadcm3n_o0ao_1900_40_007195715_0
Workunit	7393995
Created	28 Mar 2011, 13:56:35 UTC
Sent	3 Apr 2011, 5:52:50 UTC
Report deadline	3 Jul 2011, 13:20:01 UTC
Received	21 May 2011, 23:19:38 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1115097
Run time	35 days 11 hours 50 min 28 sec
CPU time	24 days 6 hours 29 min 12 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.18 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10116, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10116, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10116, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10116, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10116, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10116, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6440, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6440, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6440, iMonCtr=1 Model crash detected, will try to restart... 15:52:45 (6440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1104, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 13:44:48 (9368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:06:13 (11844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:33:44 (12648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:39:55 (10344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:47:31 (12856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:20 (7916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:55 (12392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:38:37 (12632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:11 (11924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:29:22 (11212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:51:03 (7304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:36:46 (12896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:50 (13232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:04:17 (13128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:10:27 (13260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:00:16 (7088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:40:52 (11584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:23:32 (7328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:41:56 (9624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:36:56 (12256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:14:33 (12916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:20:44 (13740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:26:54 (14744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:42:02 (15116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:28 (9488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:06:37 (13960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:21:10 (15356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:00:14 (15240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:04:39 (13668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4908, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6080, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 01:37:54 (3692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:37:55 (3692): No heartbeat from core client for 30 sec - exiting 01:37:56 (3692): No heartbeat from core client for 30 sec - exiting 01:37:57 (3692): No heartbeat from core client for 30 sec - exiting 01:37:58 (3692): No heartbeat from core client for 30 sec - exiting 01:37:59 (3692): No heartbeat from core client for 30 sec - exiting 01:38:01 (3692): No heartbeat from core client for 30 sec - exiting 01:38:02 (3692): No heartbeat from core client for 30 sec - exiting 01:38:03 (3692): No heartbeat from core client for 30 sec - exiting 01:38:04 (3692): No heartbeat from core client for 30 sec - exiting 01:38:05 (3692): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4148, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4148, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4148, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4148, iMonCtr=1 Model crash detected, will try to restart... 07:25:32 (7560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:25:34 (7560): No heartbeat from core client for 30 sec - exiting 07:25:35 (7560): No heartbeat from core client for 30 sec - exiting 07:25:36 (7560): No heartbeat from core client for 30 sec - exiting 07:25:37 (7560): No heartbeat from core client for 30 sec - exiting 07:25:38 (7560): No heartbeat from core client for 30 sec - exiting 07:25:39 (7560): No heartbeat from core client for 30 sec - exiting 07:25:40 (7560): No heartbeat from core client for 30 sec - exiting 07:25:41 (7560): No heartbeat from core client for 30 sec - exiting 07:25:42 (7560): No heartbeat from core client for 30 sec - exiting 07:25:43 (7560): No heartbeat from core client for 30 sec - exiting 07:25:44 (7560): No heartbeat from core client for 30 sec - exiting 07:25:46 (7560): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6300, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 03:18:07 (760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:36:32 (9648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:36:33 (9648): No heartbeat from core client for 30 sec - exiting 03:36:34 (9648): No heartbeat from core client for 30 sec - exiting 03:36:35 (9648): No heartbeat from core client for 30 sec - exiting 03:36:36 (9648): No heartbeat from core client for 30 sec - exiting 03:36:37 (9648): No heartbeat from core client for 30 sec - exiting 03:36:38 (9648): No heartbeat from core client for 30 sec - exiting 03:36:39 (9648): No heartbeat from core client for 30 sec - exiting 04:46:45 (10216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:50:02 (684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:11:27 (7720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:20:36 (5984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:23:44 (9792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:07:23 (6440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:07:24 (6440): No heartbeat from core client for 30 sec - exiting 07:07:25 (6440): No heartbeat from core client for 30 sec - exiting 07:07:26 (6440): No heartbeat from core client for 30 sec - exiting 07:07:27 (6440): No heartbeat from core client for 30 sec - exiting 07:22:37 (8156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:37 (2664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:39 (2664): No heartbeat from core client for 30 sec - exiting 16:43:40 (2664): No heartbeat from core client for 30 sec - exiting 16:43:41 (2664): No heartbeat from core client for 30 sec - exiting 16:43:42 (2664): No heartbeat from core client for 30 sec - exiting 22:56:35 (2232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:56:36 (2232): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 02:28:28 (8128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:58:47 (7512): No heartbeat from core client for 30 sec - exiting 05:58:48 (7512): No heartbeat from core client for 30 sec - exiting 05:58:49 (7512): No heartbeat from core client for 30 sec - exiting 05:58:50 (7512): No heartbeat from core client for 30 sec - exiting 05:58:52 (7512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:14:01 (7884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8340, iMonCtr=1 Model crash detected, will try to restart... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 May 2011 08:49:08	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	1,036,800	2,096,017	2.0216
19 May 2011 00:20:58	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	1,010,880	2,037,676	2.0157
18 May 2011 10:20:23	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	984,960	1,987,978	2.0183
17 May 2011 12:37:09	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	959,040	1,937,856	2.0206
16 May 2011 14:46:17	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	933,120	1,880,047	2.0148
15 May 2011 20:10:19	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	907,200	1,819,819	2.0060
15 May 2011 01:52:42	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	881,280	1,758,644	1.9956
12 May 2011 00:40:51	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	855,360	1,699,901	1.9874
09 May 2011 14:27:08	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	829,440	1,643,109	1.9810
08 May 2011 12:06:24	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	803,520	1,587,596	1.9758
07 May 2011 10:43:21	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	777,600	1,530,593	1.9684
06 May 2011 02:50:04	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	751,680	1,471,488	1.9576
05 May 2011 02:19:09	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	725,760	1,721,697	2.3723
04 May 2011 08:25:27	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	699,840	1,660,181	2.3722
03 May 2011 14:18:18	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	673,920	1,598,185	2.3715
02 May 2011 20:57:16	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	648,000	1,536,934	2.3718
02 May 2011 03:30:27	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	622,080	1,475,751	2.3723
01 May 2011 08:00:04	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	596,160	1,415,755	2.3748
30 Apr 2011 15:08:43	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	570,240	1,356,351	2.3786
26 Apr 2011 22:37:44	1115097	12732602	hadcm3n_o0ao_1900_40_007195715_0	544,320	1,296,335	2.3816