Task 16974117

Name	hadcm3s_1dqx_1990_2_008922099_3
Workunit	9066274
Created	4 Sep 2014, 14:39:04 UTC
Sent	4 Sep 2014, 15:15:23 UTC
Report deadline	4 Dec 2014, 22:42:34 UTC
Received	10 Oct 2014, 10:45:15 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1272748
Run time	3 days 3 hours 51 min 31 sec
CPU time	2 days 20 hours 2 min 2 sec
Validate state	Valid
Credit	622.08
Device peak FLOPS	0.75 GFLOPS
Application version	UK Met Office HadCM3 short v7.24 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> 10:03:07 (12080): No heartbeat from client for 30 sec - exiting 10:03:07 (12080): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:03:08 (12080): No heartbeat from client for 30 sec - exiting 10:03:08 (12080): timer handler: client dead, exiting 10:03:09 (12080): No heartbeat from client for 30 sec - exiting 10:03:09 (12080): timer handler: client dead, exiting 10:03:10 (12080): No heartbeat from client for 30 sec - exiting 10:03:10 (12080): timer handler: client dead, exiting 10:03:11 (12080): No heartbeat from client for 30 sec - exiting 10:03:11 (12080): timer handler: client dead, exiting 10:03:12 (12080): No heartbeat from client for 30 sec - exiting 10:03:12 (12080): timer handler: client dead, exiting 10:03:13 (12080): No heartbeat from client for 30 sec - exiting 10:03:13 (12080): timer handler: client dead, exiting 10:03:14 (12080): No heartbeat from client for 30 sec - exiting 10:03:14 (12080): timer handler: client dead, exiting 10:03:15 (12080): No heartbeat from client for 30 sec - exiting 10:03:15 (12080): timer handler: client dead, exiting 10:03:16 (12080): No heartbeat from client for 30 sec - exiting 10:03:16 (12080): timer handler: client dead, exiting 10:03:17 (12080): No heartbeat from client for 30 sec - exiting 10:03:17 (12080): timer handler: client dead, exiting Suspended CPDN Monitor - Suspend request from BOINC... 09:31:31 (5768): No heartbeat from client for 30 sec - exiting 09:32:41 (5768): timer handler: client dead, exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 14:59:47 (8340): No heartbeat from client for 30 sec - exiting 14:59:47 (8340): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:59:48 (8340): No heartbeat from client for 30 sec - exiting 14:59:48 (8340): timer handler: client dead, exiting 14:59:49 (8340): No heartbeat from client for 30 sec - exiting 14:59:49 (8340): timer handler: client dead, exiting 14:59:50 (8340): No heartbeat from client for 30 sec - exiting 14:59:50 (8340): timer handler: client dead, exiting 14:59:51 (8340): No heartbeat from client for 30 sec - exiting 14:59:51 (8340): timer handler: client dead, exiting 14:59:52 (8340): No heartbeat from client for 30 sec - exiting 14:59:52 (8340): timer handler: client dead, exiting 14:59:53 (8340): No heartbeat from client for 30 sec - exiting 14:59:53 (8340): timer handler: client dead, exiting 14:59:54 (8340): No heartbeat from client for 30 sec - exiting 14:59:54 (8340): timer handler: client dead, exiting 14:59:55 (8340): No heartbeat from client for 30 sec - exiting 14:59:55 (8340): timer handler: client dead, exiting 14:59:56 (8340): No heartbeat from client for 30 sec - exiting 14:59:56 (8340): timer handler: client dead, exiting 14:59:57 (8340): No heartbeat from client for 30 sec - exiting 14:59:57 (8340): timer handler: client dead, exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11020, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:33:04 (4188): No heartbeat from client for 30 sec - exiting 09:34:24 (4188): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:29:01 (7224): No heartbeat from client for 30 sec - exiting 09:30:22 (7224): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on 1dqxko.daj1c20 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:06:12 (10012): No heartbeat from client for 30 sec - exiting 11:06:44 (10012): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:54:00 (8256): No heartbeat from client for 30 sec - exiting 11:54:01 (8256): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:44:04 (14240): No heartbeat from client for 30 sec - exiting 12:44:04 (14240): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:59:29 (11580): No heartbeat from client for 30 sec - exiting 15:59:29 (11580): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:59:30 (11580): No heartbeat from client for 30 sec - exiting 15:59:30 (11580): timer handler: client dead, exiting 15:59:31 (11580): No heartbeat from client for 30 sec - exiting 15:59:31 (11580): timer handler: client dead, exiting 15:59:32 (11580): No heartbeat from client for 30 sec - exiting 15:59:32 (11580): timer handler: client dead, exiting 15:59:33 (11580): No heartbeat from client for 30 sec - exiting 15:59:33 (11580): timer handler: client dead, exiting 15:59:34 (11580): No heartbeat from client for 30 sec - exiting 15:59:34 (11580): timer handler: client dead, exiting 15:59:35 (11580): No heartbeat from client for 30 sec - exiting 15:59:35 (11580): timer handler: client dead, exiting 15:59:36 (11580): No heartbeat from client for 30 sec - exiting 15:59:36 (11580): timer handler: client dead, exiting 15:59:37 (11580): No heartbeat from client for 30 sec - exiting 15:59:37 (11580): timer handler: client dead, exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:28:01 (8556): No heartbeat from client for 30 sec - exiting 11:28:01 (8556): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:28:02 (8556): No heartbeat from client for 30 sec - exiting 11:28:02 (8556): timer handler: client dead, exiting 11:28:03 (8556): No heartbeat from client for 30 sec - exiting 11:28:03 (8556): timer handler: client dead, exiting 11:28:04 (8556): No heartbeat from client for 30 sec - exiting 11:28:04 (8556): timer handler: client dead, exiting 11:28:05 (8556): No heartbeat from client for 30 sec - exiting 11:28:05 (8556): timer handler: client dead, exiting 11:28:06 (8556): No heartbeat from client for 30 sec - exiting 11:28:06 (8556): timer handler: client dead, exiting 11:34:10 (11848): No heartbeat from client for 30 sec - exiting 11:34:10 (11848): timer handler: client dead, exiting 11:34:11 (11848): No heartbeat from client for 30 sec - exiting 11:34:11 (11848): timer handler: client dead, exiting 11:34:12 (11848): No heartbeat from client for 30 sec - exiting 11:34:12 (11848): timer handler: client dead, exiting 11:34:13 (11848): No heartbeat from client for 30 sec - exiting 11:34:13 (11848): timer handler: client dead, exiting 11:34:14 (11848): No heartbeat from client for 30 sec - exiting 11:34:14 (11848): timer handler: client dead, exiting 11:34:15 (11848): No heartbeat from client for 30 sec - exiting 11:34:15 (11848): timer handler: client dead, exiting 11:34:16 (11848): No heartbeat from client for 30 sec - exiting 11:34:16 (11848): timer handler: client dead, exiting 11:34:17 (11848): No heartbeat from client for 30 sec - exiting 11:34:17 (11848): timer handler: client dead, exiting 11:34:18 (11848): No heartbeat from client for 30 sec - exiting 11:34:18 (11848): timer handler: client dead, exiting 11:34:19 (11848): No heartbeat from client for 30 sec - exiting 11:34:19 (11848): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:38:45 (10836): No heartbeat from client for 30 sec - exiting 12:38:45 (10836): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:38:46 (10836): No heartbeat from client for 30 sec - exiting 12:38:46 (10836): timer handler: client dead, exiting 12:38:47 (10836): No heartbeat from client for 30 sec - exiting 12:38:47 (10836): timer handler: client dead, exiting 12:38:48 (10836): No heartbeat from client for 30 sec - exiting 12:38:48 (10836): timer handler: client dead, exiting 12:38:49 (10836): No heartbeat from client for 30 sec - exiting 12:38:49 (10836): timer handler: client dead, exiting 12:47:07 (5772): No heartbeat from client for 30 sec - exiting 12:47:07 (5772): timer handler: client dead, exiting 12:47:08 (5772): No heartbeat from client for 30 sec - exiting 12:47:08 (5772): timer handler: client dead, exiting 12:47:09 (5772): No heartbeat from client for 30 sec - exiting 12:47:09 (5772): timer handler: client dead, exiting 12:47:10 (5772): No heartbeat from client for 30 sec - exiting 12:47:10 (5772): timer handler: client dead, exiting 12:47:11 (5772): No heartbeat from client for 30 sec - exiting 12:47:11 (5772): timer handler: client dead, exiting 12:47:12 (5772): No heartbeat from client for 30 sec - exiting 12:47:12 (5772): timer handler: client dead, exiting 12:47:13 (5772): No heartbeat from client for 30 sec - exiting 12:47:13 (5772): timer handler: client dead, exiting 12:47:14 (5772): No heartbeat from client for 30 sec - exiting 12:47:14 (5772): timer handler: client dead, exiting 12:47:15 (5772): No heartbeat from client for 30 sec - exiting 12:47:15 (5772): timer handler: client dead, exiting 12:47:16 (5772): No heartbeat from client for 30 sec - exiting 12:47:16 (5772): timer handler: client dead, exiting 12:47:29 (5772): No heartbeat from client for 30 sec - exiting 12:47:29 (5772): timer handler: client dead, exiting 12:47:30 (5772): No heartbeat from client for 30 sec - exiting 12:47:30 (5772): timer handler: client dead, exiting 12:47:31 (5772): No heartbeat from client for 30 sec - exiting 12:47:31 (5772): timer handler: client dead, exiting 12:47:32 (5772): No heartbeat from client for 30 sec - exiting 12:47:32 (5772): timer handler: client dead, exiting 12:47:33 (5772): No heartbeat from client for 30 sec - exiting 12:47:33 (5772): timer handler: client dead, exiting 12:47:34 (5772): No heartbeat from client for 30 sec - exiting 12:47:34 (5772): timer handler: client dead, exiting 12:47:35 (5772): No heartbeat from client for 30 sec - exiting 12:47:35 (5772): timer handler: client dead, exiting 12:47:36 (5772): No heartbeat from client for 30 sec - exiting 12:47:36 (5772): timer handler: client dead, exiting 12:47:37 (5772): No heartbeat from client for 30 sec - exiting 12:47:37 (5772): timer handler: client dead, exiting 12:47:38 (5772): No heartbeat from client for 30 sec - exiting 12:47:38 (5772): timer handler: client dead, exiting 12:47:39 (5772): No heartbeat from client for 30 sec - exiting 12:47:39 (5772): timer handler: client dead, exiting 12:47:40 (5772): No heartbeat from client for 30 sec - exiting 12:47:40 (5772): timer handler: client dead, exiting 12:47:53 (5772): No heartbeat from client for 30 sec - exiting 12:47:53 (5772): timer handler: client dead, exiting 12:47:54 (5772): No heartbeat from client for 30 sec - exiting 12:47:54 (5772): timer handler: client dead, exiting 12:47:55 (5772): No heartbeat from client for 30 sec - exiting 12:47:55 (5772): timer handler: client dead, exiting 12:47:56 (5772): No heartbeat from client for 30 sec - exiting 12:47:56 (5772): timer handler: client dead, exiting 12:47:57 (5772): No heartbeat from client for 30 sec - exiting 12:47:57 (5772): timer handler: client dead, exiting 12:47:58 (5772): No heartbeat from client for 30 sec - exiting 12:47:58 (5772): timer handler: client dead, exiting 12:47:59 (5772): No heartbeat from client for 30 sec - exiting 12:47:59 (5772): timer handler: client dead, exiting 12:48:00 (5772): No heartbeat from client for 30 sec - exiting 12:48:00 (5772): timer handler: client dead, exiting 12:48:01 (5772): No heartbeat from client for 30 sec - exiting 12:48:01 (5772): timer handler: client dead, exiting 12:48:02 (5772): No heartbeat from client for 30 sec - exiting 12:48:02 (5772): timer handler: client dead, exiting 12:48:03 (5772): No heartbeat from client for 30 sec - exiting 12:48:03 (5772): timer handler: client dead, exiting 12:48:04 (5772): No heartbeat from client for 30 sec - exiting 12:48:04 (5772): timer handler: client dead, exiting 12:48:17 (5772): No heartbeat from client for 30 sec - exiting 12:48:17 (5772): timer handler: client dead, exiting 12:48:18 (5772): No heartbeat from client for 30 sec - exiting 12:48:18 (5772): timer handler: client dead, exiting 12:48:19 (5772): No heartbeat from client for 30 sec - exiting 12:48:19 (5772): timer handler: client dead, exiting 12:48:20 (5772): No heartbeat from client for 30 sec - exiting 12:48:20 (5772): timer handler: client dead, exiting 12:48:21 (5772): No heartbeat from client for 30 sec - exiting 12:48:21 (5772): timer handler: client dead, exiting 12:48:22 (5772): No heartbeat from client for 30 sec - exiting 12:48:22 (5772): timer handler: client dead, exiting 12:48:23 (5772): No heartbeat from client for 30 sec - exiting 12:48:23 (5772): timer handler: client dead, exiting 12:48:24 (5772): No heartbeat from client for 30 sec - exiting 12:48:24 (5772): timer handler: client dead, exiting 12:48:25 (5772): No heartbeat from client for 30 sec - exiting 12:48:25 (5772): timer handler: client dead, exiting 12:48:26 (5772): No heartbeat from client for 30 sec - exiting 12:48:26 (5772): timer handler: client dead, exiting 12:48:27 (5772): No heartbeat from client for 30 sec - exiting 12:48:27 (5772): timer handler: client dead, exiting 12:48:28 (5772): No heartbeat from client for 30 sec - exiting 12:48:28 (5772): timer handler: client dead, exiting 12:48:41 (5772): No heartbeat from client for 30 sec - exiting 12:48:41 (5772): timer handler: client dead, exiting 12:48:42 (5772): No heartbeat from client for 30 sec - exiting 12:48:42 (5772): timer handler: client dead, exiting 12:48:43 (5772): No heartbeat from client for 30 sec - exiting 12:48:43 (5772): timer handler: client dead, exiting 12:48:44 (5772): No heartbeat from client for 30 sec - exiting 12:48:44 (5772): timer handler: client dead, exiting 12:48:45 (5772): No heartbeat from client for 30 sec - exiting 12:48:45 (5772): timer handler: client dead, exiting 12:48:46 (5772): No heartbeat from client for 30 sec - exiting 12:48:46 (5772): timer handler: client dead, exiting 12:48:47 (5772): No heartbeat from client for 30 sec - exiting 12:48:47 (5772): timer handler: client dead, exiting 12:48:48 (5772): No heartbeat from client for 30 sec - exiting 12:48:48 (5772): timer handler: client dead, exiting 12:48:49 (5772): No heartbeat from client for 30 sec - exiting 12:48:49 (5772): timer handler: client dead, exiting 12:48:50 (5772): No heartbeat from client for 30 sec - exiting 12:48:50 (5772): timer handler: client dead, exiting 12:48:51 (5772): No heartbeat from client for 30 sec - exiting 12:48:51 (5772): timer handler: client dead, exiting 12:48:52 (5772): No heartbeat from client for 30 sec - exiting 12:48:52 (5772): timer handler: client dead, exiting 12:49:05 (5772): No heartbeat from client for 30 sec - exiting 12:49:05 (5772): timer handler: client dead, exiting 12:49:06 (5772): No heartbeat from client for 30 sec - exiting 12:49:06 (5772): timer handler: client dead, exiting 12:49:07 (5772): No heartbeat from client for 30 sec - exiting 12:49:07 (5772): timer handler: client dead, exiting 12:49:08 (5772): No heartbeat from client for 30 sec - exiting 12:49:08 (5772): timer handler: client dead, exiting 12:49:09 (5772): No heartbeat from client for 30 sec - exiting 12:49:09 (5772): timer handler: client dead, exiting 12:49:10 (5772): No heartbeat from client for 30 sec - exiting 12:49:10 (5772): timer handler: client dead, exiting 12:49:11 (5772): No heartbeat from client for 30 sec - exiting 12:49:11 (5772): timer handler: client dead, exiting 12:49:12 (5772): No heartbeat from client for 30 sec - exiting 12:49:12 (5772): timer handler: client dead, exiting 12:49:13 (5772): No heartbeat from client for 30 sec - exiting 12:49:13 (5772): timer handler: client dead, exiting 12:49:14 (5772): No heartbeat from client for 30 sec - exiting 12:49:14 (5772): timer handler: client dead, exiting 12:49:15 (5772): No heartbeat from client for 30 sec - exiting 12:49:15 (5772): timer handler: client dead, exiting 12:49:16 (5772): No heartbeat from client for 30 sec - exiting 12:49:16 (5772): timer handler: client dead, exiting 12:49:29 (5772): No heartbeat from client for 30 sec - exiting 12:49:29 (5772): timer handler: client dead, exiting 12:49:30 (5772): No heartbeat from client for 30 sec - exiting 12:49:30 (5772): timer handler: client dead, exiting 12:49:31 (5772): No heartbeat from client for 30 sec - exiting 12:49:31 (5772): timer handler: client dead, exiting 12:49:32 (5772): No heartbeat from client for 30 sec - exiting 12:49:32 (5772): timer handler: client dead, exiting 12:49:33 (5772): No heartbeat from client for 30 sec - exiting 12:49:33 (5772): timer handler: client dead, exiting 12:49:34 (5772): No heartbeat from client for 30 sec - exiting 12:49:34 (5772): timer handler: client dead, exiting 12:49:35 (5772): No heartbeat from client for 30 sec - exiting 12:49:35 (5772): timer handler: client dead, exiting 12:49:36 (5772): No heartbeat from client for 30 sec - exiting 12:49:36 (5772): timer handler: client dead, exiting 12:49:37 (5772): No heartbeat from client for 30 sec - exiting 12:49:37 (5772): timer handler: client dead, exiting 12:49:38 (5772): No heartbeat from client for 30 sec - exiting 12:49:38 (5772): timer handler: client dead, exiting 12:49:39 (5772): No heartbeat from client for 30 sec - exiting 12:49:39 (5772): timer handler: client dead, exiting 12:49:40 (5772): No heartbeat from client for 30 sec - exiting 12:49:40 (5772): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:29 (10072): No heartbeat from client for 30 sec - exiting 12:56:29 (10072): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:30 (10072): No heartbeat from client for 30 sec - exiting 12:56:30 (10072): timer handler: client dead, exiting 12:56:31 (10072): No heartbeat from client for 30 sec - exiting 12:56:31 (10072): timer handler: client dead, exiting 12:56:32 (10072): No heartbeat from client for 30 sec - exiting 12:56:32 (10072): timer handler: client dead, exiting 12:56:33 (10072): No heartbeat from client for 30 sec - exiting 12:56:33 (10072): timer handler: client dead, exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:14:35 (6480): No heartbeat from client for 30 sec - exiting 14:14:35 (6480): timer handler: client dead, exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:30:22 (6972): No heartbeat from client for 30 sec - exiting 13:30:54 (6972): timer handler: client dead, exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:29:25 (1980): No heartbeat from client for 30 sec - exiting 09:29:25 (1980): timer handler: client dead, exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:01:38 (7684): No heartbeat from client for 30 sec - exiting 11:02:12 (7684): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:26:49 (11340): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Oct 2014 21:54:50	1272748	16974117	hadcm3s_1dqx_1990_2_008922099_3	51,840	268,564	5.1806
25 Sep 2014 01:32:37	1272748	16974117	hadcm3s_1dqx_1990_2_008922099_3	25,920	144,775	5.5855