Task 17905358

Name	hadcm3n_sczl_1940_40_009115026_4
Workunit	9245362
Created	6 Feb 2015, 16:38:45 UTC
Sent	6 Feb 2015, 20:59:58 UTC
Report deadline	9 May 2015, 4:27:09 UTC
Received	22 Mar 2015, 19:27:03 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1314566
Run time	23 days 10 hours 36 min 18 sec
CPU time	20 days 11 hours 32 min 35 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	3.36 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:03:14 (648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:23:49 (17697): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:30:00 (26706): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:06:39 (33609): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:55:57 (37532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:42:32 (43357): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:32:13 (47444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:23:23 (53522): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:26:19 (59478): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:29:08 (59519): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:34:16 (62979): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:24:21 (66451): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 15:58:10 (70548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:54:47 (76081): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:29:02 (80379): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:02:01 (85279): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:14:49 (92074): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:17:31 (95668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:01:59 (99135): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:40:06 (3480): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:17:49 (84119): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 20:17:50 (84119): No heartbeat from core client for 30 sec - exiting 00:35:37 (20924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:36:35 (23126): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:46:33 (32280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:46:34 (32280): No heartbeat from core client for 30 sec - exiting 11:46:35 (32280): No heartbeat from core client for 30 sec - exiting 11:46:36 (32280): No heartbeat from core client for 30 sec - exiting 11:46:37 (32280): No heartbeat from core client for 30 sec - exiting 11:46:38 (32280): No heartbeat from core client for 30 sec - exiting 11:46:39 (32280): No heartbeat from core client for 30 sec - exiting 11:46:41 (32280): No heartbeat from core client for 30 sec - exiting 11:46:42 (32280): No heartbeat from core client for 30 sec - exiting 11:46:43 (32280): No heartbeat from core client for 30 sec - exiting 11:46:44 (32280): No heartbeat from core client for 30 sec - exiting 11:46:45 (32280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:19:55 (38422): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 02:25:47 (50971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:57:02 (54541): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:13:18 (62275): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:48:36 (67987): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:40:40 (76342): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 11:36:10 (86864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:37:52 (48545): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:41:19 (52166): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:07:36 (55983): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:08 (71630): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:08:14 (32915): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:54:58 (38007): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:06 (56354): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:47:31 (84798): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:18:30 (88434): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:46 (93307): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:04:55 (99511): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:25:20 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:03:59 (96125): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:07:59 (13147): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 21:08:00 (13147): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:59:23 (7080): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:09:21 (814): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:28 (613): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:50:30 (1237): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:06:24 (14091): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 22:18:55 (41776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:21:39 (45531): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:22 (49041): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:23 (54769): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:02:25 (28361): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:32 (36835): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:28:34 (46198): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:23:27 (88347): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:07:59 (44276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:00 (44276): No heartbeat from core client for 30 sec - exiting 21:08:01 (44276): No heartbeat from core client for 30 sec - exiting 21:08:02 (44276): No heartbeat from core client for 30 sec - exiting 21:08:03 (44276): No heartbeat from core client for 30 sec - exiting 21:08:04 (44276): No heartbeat from core client for 30 sec - exiting 21:08:05 (44276): No heartbeat from core client for 30 sec - exiting 21:08:07 (44276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:34:01 (92459): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:11:52 (62513): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:05:55 (70889): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:30:52 (93002): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: severe (17): syntax error in NAMELIST input, unit 5, file /Volumes/Macintosh HD/Users/Shared/BOINC Data/projects/climateprediction.net/hadcm3n_sczl_1940_40_009115026/jobs/climate.cpdc, line 393, position 19 Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003CACB0 Unknown Unknown Unknown hadcm3n_um_6.07_i 00261D1E Unknown Unknown Unknown hadcm3n_um_6.07_i 0016269A Unknown Unknown Unknown hadcm3n_um_6.07_i 00164453 Unknown Unknown Unknown hadcm3n_um_6.07_i 0038CE7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0038A1ED Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=96937, iMonCtr=1 Model crash detected, will try to restart... 22:09:14 (96937): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:31:52 (14658): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:46:12 (21123): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:38 (46318): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:02:47 (638): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:02:48 (638): No heartbeat from core client for 30 sec - exiting 11:02:50 (638): No heartbeat from core client for 30 sec - exiting 11:02:51 (638): No heartbeat from core client for 30 sec - exiting 11:02:52 (638): No heartbeat from core client for 30 sec - exiting 11:02:53 (638): No heartbeat from core client for 30 sec - exiting 11:02:54 (638): No heartbeat from core client for 30 sec - exiting 11:02:55 (638): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 15:20:31 (97960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Mar 2015 18:28:59	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	1,036,800	1,768,877	1.7061
22 Mar 2015 01:09:44	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	1,010,880	1,726,132	1.7076
21 Mar 2015 11:01:33	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	984,960	1,682,011	1.7077
20 Mar 2015 18:11:42	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	959,040	1,638,442	1.7084
20 Mar 2015 01:25:28	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	933,120	1,594,287	1.7086
18 Mar 2015 20:45:34	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	907,200	1,551,741	1.7105
18 Mar 2015 07:26:42	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	881,280	1,509,258	1.7126
12 Mar 2015 23:51:45	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	855,360	1,464,918	1.7126
12 Mar 2015 10:08:46	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	829,440	1,421,246	1.7135
11 Mar 2015 11:47:34	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	803,520	1,377,272	1.7140
10 Mar 2015 16:33:51	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	777,600	1,332,235	1.7133
10 Mar 2015 16:32:30	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	751,680	1,289,130	1.7150
10 Mar 2015 16:28:06	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	725,760	1,245,013	1.7155
10 Mar 2015 16:26:10	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	699,840	1,200,288	1.7151
10 Mar 2015 16:23:11	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	673,920	1,156,658	1.7163
03 Mar 2015 16:00:47	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	648,000	1,114,030	1.7192
03 Mar 2015 00:55:17	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	622,080	1,067,273	1.7157
02 Mar 2015 08:01:14	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	596,160	1,020,009	1.7110
01 Mar 2015 15:29:43	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	570,240	970,934	1.7027
28 Feb 2015 23:58:44	1314566	17905358	hadcm3n_sczl_1940_40_009115026_4	544,320	924,662	1.6987