Name | hadcm3n_200p_1940_40_007958935_4 |
Workunit | 8114047 |
Created | 10 Aug 2012, 23:46:33 UTC |
Sent | 10 Aug 2012, 23:46:51 UTC |
Report deadline | 10 Nov 2012, 7:14:02 UTC |
Received | 22 Aug 2012, 22:11:31 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1188461 |
Run time | 10 days 4 hours 45 min 37 sec |
CPU time | 9 days 19 hours 59 min 49 sec |
Validate state | Invalid |
Credit | 9,331.20 |
Device peak FLOPS | 3.30 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
Stderr | <core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> m core client for 30 sec - exiting 22:13:05 (4940): No heartbeat from core client for 30 sec - exiting 22:13:06 (4940): No heartbeat from core client for 30 sec - exiting 22:13:07 (4940): No heartbeat from core client for 30 sec - exiting 22:13:08 (4940): No heartbeat from core client for 30 sec - exiting 22:13:09 (4940): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 22:18:00 (4904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:18:01 (4904): No heartbeat from core client for 30 sec - exiting 22:18:02 (4904): No heartbeat from core client for 30 sec - exiting 22:18:03 (4904): No heartbeat from core client for 30 sec - exiting 22:18:04 (4904): No heartbeat from core client for 30 sec - exiting 22:18:05 (4904): No heartbeat from core client for 30 sec - exiting 22:18:06 (4904): No heartbeat from core client for 30 sec - exiting 22:18:07 (4904): No heartbeat from core client for 30 sec - exiting 22:18:08 (4904): No heartbeat from core client for 30 sec - exiting 22:18:09 (4904): No heartbeat from core client for 30 sec - exiting 22:18:10 (4904): No heartbeat from core client for 30 sec - exiting 22:19:02 (6160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:23:00 (11664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:23:01 (11664): No heartbeat from core client for 30 sec - exiting 22:23:02 (11664): No heartbeat from core client for 30 sec - exiting 22:23:03 (11664): No heartbeat from core client for 30 sec - exiting 22:23:04 (11664): No heartbeat from core client for 30 sec - exiting 22:23:05 (11664): No heartbeat from core client for 30 sec - exiting 22:23:06 (11664): No heartbeat from core client for 30 sec - exiting 22:23:07 (11664): No heartbeat from core client for 30 sec - exiting 22:23:08 (11664): No heartbeat from core client for 30 sec - exiting 22:23:09 (11664): No heartbeat from core client for 30 sec - exiting 22:23:10 (11664): No heartbeat from core client for 30 sec - exiting 22:28:54 (11452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:28:55 (11452): No heartbeat from core client for 30 sec - exiting 22:28:56 (11452): No heartbeat from core client for 30 sec - exiting 22:28:57 (11452): No heartbeat from core client for 30 sec - exiting 22:28:58 (11452): No heartbeat from core client for 30 sec - exiting 22:28:59 (11452): No heartbeat from core client for 30 sec - exiting 22:29:00 (11452): No heartbeat from core client for 30 sec - exiting 22:29:01 (11452): No heartbeat from core client for 30 sec - exiting 22:29:02 (11452): No heartbeat from core client for 30 sec - exiting 22:29:03 (11452): No heartbeat from core client for 30 sec - exiting 22:29:04 (11452): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 22:33:02 (12012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:33:03 (12012): No heartbeat from core client for 30 sec - exiting 22:33:04 (12012): No heartbeat from core client for 30 sec - exiting 22:33:05 (12012): No heartbeat from core client for 30 sec - exiting 22:33:06 (12012): No heartbeat from core client for 30 sec - exiting 22:33:07 (12012): No heartbeat from core client for 30 sec - exiting 22:33:08 (12012): No heartbeat from core client for 30 sec - exiting 22:33:09 (12012): No heartbeat from core client for 30 sec - exiting 22:33:10 (12012): No heartbeat from core client for 30 sec - exiting 22:33:11 (12012): No heartbeat from core client for 30 sec - exiting 22:33:12 (12012): No heartbeat from core client for 30 sec - exiting 22:34:49 (9404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:45:29 (10104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:45:30 (10104): No heartbeat from core client for 30 sec - exiting 22:45:31 (10104): No heartbeat from core client for 30 sec - exiting 22:45:32 (10104): No heartbeat from core client for 30 sec - exiting 22:45:33 (10104): No heartbeat from core client for 30 sec - exiting 22:45:34 (10104): No heartbeat from core client for 30 sec - exiting 22:45:35 (10104): No heartbeat from core client for 30 sec - exiting 22:45:36 (10104): No heartbeat from core client for 30 sec - exiting 22:45:37 (10104): No heartbeat from core client for 30 sec - exiting 22:45:38 (10104): No heartbeat from core client for 30 sec - exiting 22:45:39 (10104): No heartbeat from core client for 30 sec - exiting 22:47:00 (9980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:47:01 (9980): No heartbeat from core client for 30 sec - exiting 22:47:02 (9980): No heartbeat from core client for 30 sec - exiting 22:47:03 (9980): No heartbeat from core client for 30 sec - exiting 22:47:04 (9980): No heartbeat from core client for 30 sec - exiting 22:47:05 (9980): No heartbeat from core client for 30 sec - exiting 22:47:06 (9980): No heartbeat from core client for 30 sec - exiting 22:47:07 (9980): No heartbeat from core client for 30 sec - exiting 22:47:08 (9980): No heartbeat from core client for 30 sec - exiting 22:47:09 (9980): No heartbeat from core client for 30 sec - exiting 22:47:10 (9980): No heartbeat from core client for 30 sec - exiting 23:28:54 (11308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:28:55 (11308): No heartbeat from core client for 30 sec - exiting 23:28:56 (11308): No heartbeat from core client for 30 sec - exiting 23:28:57 (11308): No heartbeat from core client for 30 sec - exiting 23:28:58 (11308): No heartbeat from core client for 30 sec - exiting 23:28:59 (11308): No heartbeat from core client for 30 sec - exiting 23:29:00 (11308): No heartbeat from core client for 30 sec - exiting 23:29:01 (11308): No heartbeat from core client for 30 sec - exiting 23:29:02 (11308): No heartbeat from core client for 30 sec - exiting 23:29:03 (11308): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 23:51:06 (10884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:51:07 (10884): No heartbeat from core client for 30 sec - exiting 23:51:08 (10884): No heartbeat from core client for 30 sec - exiting 23:51:09 (10884): No heartbeat from core client for 30 sec - exiting 23:51:10 (10884): No heartbeat from core client for 30 sec - exiting 23:51:11 (10884): No heartbeat from core client for 30 sec - exiting 23:51:12 (10884): No heartbeat from core client for 30 sec - exiting 23:51:13 (10884): No heartbeat from core client for 30 sec - exiting 23:51:14 (10884): No heartbeat from core client for 30 sec - exiting 23:51:15 (10884): No heartbeat from core client for 30 sec - exiting 23:51:16 (10884): No heartbeat from core client for 30 sec - exiting 23:56:48 (11856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:05:57 (8720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:14:52 (11872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:14:53 (11872): No heartbeat from core client for 30 sec - exiting 00:14:54 (11872): No heartbeat from core client for 30 sec - exiting 00:14:55 (11872): No heartbeat from core client for 30 sec - exiting 00:14:56 (11872): No heartbeat from core client for 30 sec - exiting 00:14:57 (11872): No heartbeat from core client for 30 sec - exiting 00:14:58 (11872): No heartbeat from core client for 30 sec - exiting 00:14:59 (11872): No heartbeat from core client for 30 sec - exiting 00:15:00 (11872): No heartbeat from core client for 30 sec - exiting 00:15:01 (11872): No heartbeat from core client for 30 sec - exiting 00:15:02 (11872): No heartbeat from core client for 30 sec - exiting 00:16:44 (9180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:16:45 (9180): No heartbeat from core client for 30 sec - exiting 00:16:46 (9180): No heartbeat from core client for 30 sec - exiting 00:16:47 (9180): No heartbeat from core client for 30 sec - exiting 00:16:48 (9180): No heartbeat from core client for 30 sec - exiting 00:16:49 (9180): No heartbeat from core client for 30 sec - exiting 00:16:50 (9180): No heartbeat from core client for 30 sec - exiting 00:16:51 (9180): No heartbeat from core client for 30 sec - exiting 00:16:52 (9180): No heartbeat from core client for 30 sec - exiting 00:16:53 (9180): No heartbeat from core client for 30 sec - exiting 00:16:54 (9180): No heartbeat from core client for 30 sec - exiting 00:18:49 (1204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:18:50 (1204): No heartbeat from core client for 30 sec - exiting 00:18:51 (1204): No heartbeat from core client for 30 sec - exiting 00:18:52 (1204): No heartbeat from core client for 30 sec - exiting 00:18:53 (1204): No heartbeat from core client for 30 sec - exiting 00:18:54 (1204): No heartbeat from core client for 30 sec - exiting 00:18:55 (1204): No heartbeat from core client for 30 sec - exiting 00:18:56 (1204): No heartbeat from core client for 30 sec - exiting 00:18:57 (1204): No heartbeat from core client for 30 sec - exiting 00:18:58 (1204): No heartbeat from core client for 30 sec - exiting 00:18:59 (1204): No heartbeat from core client for 30 sec - exiting 00:20:58 (10772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:20:59 (10772): No heartbeat from core client for 30 sec - exiting 00:21:00 (10772): No heartbeat from core client for 30 sec - exiting 00:21:01 (10772): No heartbeat from core client for 30 sec - exiting 00:21:02 (10772): No heartbeat from core client for 30 sec - exiting 00:21:03 (10772): No heartbeat from core client for 30 sec - exiting 00:21:04 (10772): No heartbeat from core client for 30 sec - exiting 00:21:05 (10772): No heartbeat from core client for 30 sec - exiting 00:21:06 (10772): No heartbeat from core client for 30 sec - exiting 00:21:07 (10772): No heartbeat from core client for 30 sec - exiting 00:21:08 (10772): No heartbeat from core client for 30 sec - exiting 00:22:46 (11480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:22:47 (11480): No heartbeat from core client for 30 sec - exiting 00:22:48 (11480): No heartbeat from core client for 30 sec - exiting 00:22:49 (11480): No heartbeat from core client for 30 sec - exiting 00:22:50 (11480): No heartbeat from core client for 30 sec - exiting 00:22:51 (11480): No heartbeat from core client for 30 sec - exiting 00:22:52 (11480): No heartbeat from core client for 30 sec - exiting 00:22:53 (11480): No heartbeat from core client for 30 sec - exiting 00:22:54 (11480): No heartbeat from core client for 30 sec - exiting 00:22:55 (11480): No heartbeat from core client for 30 sec - exiting 00:22:56 (11480): No heartbeat from core client for 30 sec - exiting 00:24:51 (11800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:24:52 (11800): No heartbeat from core client for 30 sec - exiting 00:24:53 (11800): No heartbeat from core client for 30 sec - exiting 00:24:54 (11800): No heartbeat from core client for 30 sec - exiting 00:24:55 (11800): No heartbeat from core client for 30 sec - exiting 00:24:56 (11800): No heartbeat from core client for 30 sec - exiting 00:24:57 (11800): No heartbeat from core client for 30 sec - exiting 00:24:58 (11800): No heartbeat from core client for 30 sec - exiting 00:24:59 (11800): No heartbeat from core client for 30 sec - exiting 00:25:00 (11800): No heartbeat from core client for 30 sec - exiting 00:25:01 (11800): No heartbeat from core client for 30 sec - exiting 00:26:53 (10352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:26:54 (10352): No heartbeat from core client for 30 sec - exiting 00:26:55 (10352): No heartbeat from core client for 30 sec - exiting 00:26:56 (10352): No heartbeat from core client for 30 sec - exiting 00:26:57 (10352): No heartbeat from core client for 30 sec - exiting 00:26:58 (10352): No heartbeat from core client for 30 sec - exiting 00:26:59 (10352): No heartbeat from core client for 30 sec - exiting 00:27:00 (10352): No heartbeat from core client for 30 sec - exiting 00:27:01 (10352): No heartbeat from core client for 30 sec - exiting 00:27:02 (10352): No heartbeat from core client for 30 sec - exiting 00:27:03 (10352): No heartbeat from core client for 30 sec - exiting 00:28:53 (4940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:54 (4940): No heartbeat from core client for 30 sec - exiting 00:28:55 (4940): No heartbeat from core client for 30 sec - exiting 00:28:56 (4940): No heartbeat from core client for 30 sec - exiting 00:28:57 (4940): No heartbeat from core client for 30 sec - exiting 00:28:58 (4940): No heartbeat from core client for 30 sec - exiting 00:28:59 (4940): No heartbeat from core client for 30 sec - exiting 00:29:00 (4940): No heartbeat from core client for 30 sec - exiting 00:29:01 (4940): No heartbeat from core client for 30 sec - exiting 00:29:02 (4940): No heartbeat from core client for 30 sec - exiting 00:29:03 (4940): No heartbeat from core client for 30 sec - exiting 00:31:06 (11324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:31:07 (11324): No heartbeat from core client for 30 sec - exiting 00:31:08 (11324): No heartbeat from core client for 30 sec - exiting 00:31:09 (11324): No heartbeat from core client for 30 sec - exiting 00:31:10 (11324): No heartbeat from core client for 30 sec - exiting 00:31:11 (11324): No heartbeat from core client for 30 sec - exiting 00:31:12 (11324): No heartbeat from core client for 30 sec - exiting 00:31:13 (11324): No heartbeat from core client for 30 sec - exiting 00:31:14 (11324): No heartbeat from core client for 30 sec - exiting 00:31:15 (11324): No heartbeat from core client for 30 sec - exiting 00:31:16 (11324): No heartbeat from core client for 30 sec - exiting 00:33:07 (11948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:33:08 (11948): No heartbeat from core client for 30 sec - exiting 00:33:09 (11948): No heartbeat from core client for 30 sec - exiting 00:33:10 (11948): No heartbeat from core client for 30 sec - exiting 00:33:11 (11948): No heartbeat from core client for 30 sec - exiting 00:33:12 (11948): No heartbeat from core client for 30 sec - exiting 00:33:13 (11948): No heartbeat from core client for 30 sec - exiting 00:33:14 (11948): No heartbeat from core client for 30 sec - exiting 00:33:15 (11948): No heartbeat from core client for 30 sec - exiting 00:33:16 (11948): No heartbeat from core client for 30 sec - exiting 00:33:17 (11948): No heartbeat from core client for 30 sec - exiting 00:34:52 (10324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:34:53 (10324): No heartbeat from core client for 30 sec - exiting 00:34:54 (10324): No heartbeat from core client for 30 sec - exiting 00:34:55 (10324): No heartbeat from core client for 30 sec - exiting 00:34:56 (10324): No heartbeat from core client for 30 sec - exiting 00:34:57 (10324): No heartbeat from core client for 30 sec - exiting 00:34:58 (10324): No heartbeat from core client for 30 sec - exiting 00:34:59 (10324): No heartbeat from core client for 30 sec - exiting 00:35:00 (10324): No heartbeat from core client for 30 sec - exiting 00:35:01 (10324): No heartbeat from core client for 30 sec - exiting 00:35:02 (10324): No heartbeat from core client for 30 sec - exiting 00:36:52 (12096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:36:53 (12096): No heartbeat from core client for 30 sec - exiting 00:36:54 (12096): No heartbeat from core client for 30 sec - exiting 00:36:55 (12096): No heartbeat from core client for 30 sec - exiting 00:36:56 (12096): No heartbeat from core client for 30 sec - exiting 00:36:57 (12096): No heartbeat from core client for 30 sec - exiting 00:36:58 (12096): No heartbeat from core client for 30 sec - exiting 00:36:59 (12096): No heartbeat from core client for 30 sec - exiting 00:37:00 (12096): No heartbeat from core client for 30 sec - exiting 00:37:01 (12096): No heartbeat from core client for 30 sec - exiting 00:37:02 (12096): No heartbeat from core client for 30 sec - exiting 00:38:29 (7212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:41:50 (8676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:41:51 (8676): No heartbeat from core client for 30 sec - exiting 00:41:52 (8676): No heartbeat from core client for 30 sec - exiting 00:43:26 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:43:27 (4172): No heartbeat from core client for 30 sec - exiting 00:43:28 (4172): No heartbeat from core client for 30 sec - exiting 00:43:29 (4172): No heartbeat from core client for 30 sec - exiting 00:43:30 (4172): No heartbeat from core client for 30 sec - exiting 00:43:31 (4172): No heartbeat from core client for 30 sec - exiting 00:43:32 (4172): No heartbeat from core client for 30 sec - exiting 00:43:33 (4172): No heartbeat from core client for 30 sec - exiting 00:43:34 (4172): No heartbeat from core client for 30 sec - exiting 00:45:07 (11236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:45:08 (11236): No heartbeat from core client for 30 sec - exiting 00:45:09 (11236): No heartbeat from core client for 30 sec - exiting 00:45:10 (11236): No heartbeat from core client for 30 sec - exiting 00:45:11 (11236): No heartbeat from core client for 30 sec - exiting 00:45:12 (11236): No heartbeat from core client for 30 sec - exiting 00:45:13 (11236): No heartbeat from core client for 30 sec - exiting 00:45:14 (11236): No heartbeat from core client for 30 sec - exiting 00:45:15 (11236): No heartbeat from core client for 30 sec - exiting 00:45:16 (11236): No heartbeat from core client for 30 sec - exiting 00:46:53 (11752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:46:54 (11752): No heartbeat from core client for 30 sec - exiting 00:46:55 (11752): No heartbeat from core client for 30 sec - exiting 00:46:56 (11752): No heartbeat from core client for 30 sec - exiting 00:46:57 (11752): No heartbeat from core client for 30 sec - exiting 00:46:58 (11752): No heartbeat from core client for 30 sec - exiting 00:46:59 (11752): No heartbeat from core client for 30 sec - exiting 00:47:00 (11752): No heartbeat from core client for 30 sec - exiting 00:47:01 (11752): No heartbeat from core client for 30 sec - exiting 00:47:02 (11752): No heartbeat from core client for 30 sec - exiting 00:47:03 (11752): No heartbeat from core client for 30 sec - exiting 00:50:08 (8836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:09 (8836): No heartbeat from core client for 30 sec - exiting 00:50:10 (8836): No heartbeat from core client for 30 sec - exiting 00:50:11 (8836): No heartbeat from core client for 30 sec - exiting 00:50:12 (8836): No heartbeat from core client for 30 sec - exiting 00:50:13 (8836): No heartbeat from core client for 30 sec - exiting 00:50:14 (8836): No heartbeat from core client for 30 sec - exiting 00:50:15 (8836): No heartbeat from core client for 30 sec - exiting 00:50:16 (8836): No heartbeat from core client for 30 sec - exiting 00:50:17 (8836): No heartbeat from core client for 30 sec - exiting 00:50:18 (8836): No heartbeat from core client for 30 sec - exiting 00:51:14 (10700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:55:09 (9976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:55:10 (9976): No heartbeat from core client for 30 sec - exiting 00:55:11 (9976): No heartbeat from core client for 30 sec - exiting 00:55:12 (9976): No heartbeat from core client for 30 sec - exiting 00:55:13 (9976): No heartbeat from core client for 30 sec - exiting 00:55:14 (9976): No heartbeat from core client for 30 sec - exiting 00:55:15 (9976): No heartbeat from core client for 30 sec - exiting 00:55:16 (9976): No heartbeat from core client for 30 sec - exiting 00:55:17 (9976): No heartbeat from core client for 30 sec - exiting 00:55:18 (9976): No heartbeat from core client for 30 sec - exiting 00:55:19 (9976): No heartbeat from core client for 30 sec - exiting 01:00:55 (11112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:56 (11112): No heartbeat from core client for 30 sec - exiting 01:00:57 (11112): No heartbeat from core client for 30 sec - exiting 01:00:58 (11112): No heartbeat from core client for 30 sec - exiting 01:00:59 (11112): No heartbeat from core client for 30 sec - exiting 01:01:00 (11112): No heartbeat from core client for 30 sec - exiting 01:01:01 (11112): No heartbeat from core client for 30 sec - exiting 01:01:02 (11112): No heartbeat from core client for 30 sec - exiting 01:01:03 (11112): No heartbeat from core client for 30 sec - exiting 01:01:04 (11112): No heartbeat from core client for 30 sec - exiting 01:01:05 (11112): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 01:02:54 (11088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:02:55 (11088): No heartbeat from core client for 30 sec - exiting 01:02:56 (11088): No heartbeat from core client for 30 sec - exiting 01:02:57 (11088): No heartbeat from core client for 30 sec - exiting 01:02:58 (11088): No heartbeat from core client for 30 sec - exiting 01:02:59 (11088): No heartbeat from core client for 30 sec - exiting 01:03:00 (11088): No heartbeat from core client for 30 sec - exiting 01:03:01 (11088): No heartbeat from core client for 30 sec - exiting 01:03:02 (11088): No heartbeat from core client for 30 sec - exiting 01:03:03 (11088): No heartbeat from core client for 30 sec - exiting 01:03:04 (11088): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 01:04:58 (9652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:07:19 (11752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 01:21:19 (2780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:21:20 (2780): No heartbeat from core client for 30 sec - exiting 01:21:21 (2780): No heartbeat from core client for 30 sec - exiting 01:21:22 (2780): No heartbeat from core client for 30 sec - exiting 01:21:23 (2780): No heartbeat from core client for 30 sec - exiting 01:21:24 (2780): No heartbeat from core client for 30 sec - exiting 01:21:25 (2780): No heartbeat from core client for 30 sec - exiting 01:21:26 (2780): No heartbeat from core client for 30 sec - exiting 01:30:56 (11728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:30:57 (11728): No heartbeat from core client for 30 sec - exiting 01:30:58 (11728): No heartbeat from core client for 30 sec - exiting 01:30:59 (11728): No heartbeat from core client for 30 sec - exiting 01:31:00 (11728): No heartbeat from core client for 30 sec - exiting 01:31:01 (11728): No heartbeat from core client for 30 sec - exiting 01:31:02 (11728): No heartbeat from core client for 30 sec - exiting 01:31:03 (11728): No heartbeat from core client for 30 sec - exiting 01:31:04 (11728): No heartbeat from core client for 30 sec - exiting 01:31:05 (11728): No heartbeat from core client for 30 sec - exiting 01:31:06 (11728): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 02:16:49 (11096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:16:50 (11096): No heartbeat from core client for 30 sec - exiting 02:16:51 (11096): No heartbeat from core client for 30 sec - exiting 02:16:52 (11096): No heartbeat from core client for 30 sec - exiting 02:16:53 (11096): No heartbeat from core client for 30 sec - exiting 02:16:54 (11096): No heartbeat from core client for 30 sec - exiting 02:16:55 (11096): No heartbeat from core client for 30 sec - exiting 02:16:56 (11096): No heartbeat from core client for 30 sec - exiting 02:16:57 (11096): No heartbeat from core client for 30 sec - exiting 02:16:58 (11096): No heartbeat from core client for 30 sec - exiting 02:16:59 (11096): No heartbeat from core client for 30 sec - exiting 02:17:38 (8776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:32:02 (5012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:38:48 (11828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:38:49 (11828): No heartbeat from core client for 30 sec - exiting 02:38:50 (11828): No heartbeat from core client for 30 sec - exiting 02:38:51 (11828): No heartbeat from core client for 30 sec - exiting 02:38:52 (11828): No heartbeat from core client for 30 sec - exiting 02:38:53 (11828): No heartbeat from core client for 30 sec - exiting 02:38:54 (11828): No heartbeat from core client for 30 sec - exiting 02:38:55 (11828): No heartbeat from core client for 30 sec - exiting 02:38:56 (11828): No heartbeat from core client for 30 sec - exiting 02:38:57 (11828): No heartbeat from core client for 30 sec - exiting 02:38:58 (11828): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 03:14:50 (10748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:14:51 (10748): No heartbeat from core client for 30 sec - exiting 03:14:52 (10748): No heartbeat from core client for 30 sec - exiting 03:14:53 (10748): No heartbeat from core client for 30 sec - exiting 03:14:54 (10748): No heartbeat from core client for 30 sec - exiting 03:14:55 (10748): No heartbeat from core client for 30 sec - exiting 03:14:56 (10748): No heartbeat from core client for 30 sec - exiting 03:14:57 (10748): No heartbeat from core client for 30 sec - exiting 03:14:58 (10748): No heartbeat from core client for 30 sec - exiting 03:14:59 (10748): No heartbeat from core client for 30 sec - exiting 03:15:00 (10748): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 03:46:43 (9512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:46:44 (9512): No heartbeat from core client for 30 sec - exiting 03:46:45 (9512): No heartbeat from core client for 30 sec - exiting 03:46:46 (9512): No heartbeat from core client for 30 sec - exiting 03:46:47 (9512): No heartbeat from core client for 30 sec - exiting 03:46:48 (9512): No heartbeat from core client for 30 sec - exiting 03:46:49 (9512): No heartbeat from core client for 30 sec - exiting 03:46:50 (9512): No heartbeat from core client for 30 sec - exiting 03:46:51 (9512): No heartbeat from core client for 30 sec - exiting 03:46:52 (9512): No heartbeat from core client for 30 sec - exiting 03:46:53 (9512): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 04:08:55 (8420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:08:56 (8420): No heartbeat from core client for 30 sec - exiting 04:08:57 (8420): No heartbeat from core client for 30 sec - exiting 04:08:58 (8420): No heartbeat from core client for 30 sec - exiting 04:08:59 (8420): No heartbeat from core client for 30 sec - exiting 04:09:00 (8420): No heartbeat from core client for 30 sec - exiting 04:09:01 (8420): No heartbeat from core client for 30 sec - exiting 04:09:02 (8420): No heartbeat from core client for 30 sec - exiting 04:09:03 (8420): No heartbeat from core client for 30 sec - exiting 04:09:04 (8420): No heartbeat from core client for 30 sec - exiting 04:09:05 (8420): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11512, iMonCtr=1 Model crash detected, will try to restart... Ocean Restart file copy failed on 200pko.daf16j0 Ocean Restart file copy failed on 200pko.daf1720 Ocean Restart file copy failed on 200pko.daf17c0 Ocean Restart file copy failed on 200pko.daf17f0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.daf18c0 Ocean Restart file copy failed on 200pko.daf18k0 Ocean Restart file copy failed on 200pko.daf19n0 Ocean Restart file copy failed on 200pko.daf1a50 Ocean Restart file copy failed on 200pko.daf1b60 Ocean Restart file copy failed on 200pko.daf1ba0 Ocean Restart file copy failed on 200pko.daf1bi0 Ocean Restart file copy failed on 200pko.daf1bn0 Ocean Restart file copy failed on 200pko.daf1cf0 Ocean Restart file copy failed on 200pko.daf1cr0 Ocean Restart file copy failed on 200pko.daf21g0 Ocean Restart file copy failed on 200pko.daf21k0 Ocean Restart file copy failed on 200pko.daf21o0 Ocean Restart file copy failed on 200pko.daf21r0 Ocean Restart file copy failed on 200pko.daf22a0 Ocean Restart file copy failed on 200pko.daf22d0 Ocean Restart file copy failed on 200pko.daf23b0 Ocean Restart file copy failed on 200pko.daf24n0 Ocean Restart file copy failed on 200pko.daf24q0 Ocean Restart file copy failed on 200pko.daf24t0 Ocean Restart file copy failed on 200pko.daf2580 Ocean Restart file copy failed on 200pko.daf25b0 Ocean Restart file copy failed on 200pko.daf25p0 Ocean Restart file copy failed on 200pko.daf26m0 Ocean Restart file copy failed on 200pko.daf26q0 Ocean Restart file copy failed on 200pko.daf2790 Ocean Restart file copy failed on 200pko.daf27d0 Ocean Restart file copy failed on 200pko.daf27h0 Ocean Restart file copy failed on 200pko.daf2820 Ocean Restart file copy failed on 200pko.daf2870 Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on 200pko.daf2a60 Ocean Restart file copy failed on 200pko.daf2cl0 Ocean Restart file copy failed on 200pko.daf3290 Ocean Restart file copy failed on 200pko.daf32g0 Ocean Restart file copy failed on 200pko.daf32j0 Ocean Restart file copy failed on 200pko.daf33p0 Ocean Restart file copy failed on 200pko.daf35g0 Ocean Restart file copy failed on 200pko.daf3750 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.daf39h0 Ocean Restart file copy failed on 200pko.daf3as0 Ocean Restart file copy failed on 200pko.daf3c70 Ocean Restart file copy failed on 200pko.daf3cf0 Ocean Restart file copy failed on 200pko.daf3cm0 Ocean Restart file copy failed on 200pko.daf41a0 Ocean Restart file copy failed on 200pko.daf44m0 Ocean Restart file copy failed on 200pko.daf4a50 Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on 200pko.daf4as0 Ocean Restart file copy failed on 200pko.daf4bp0 Ocean Restart file copy failed on 200pko.daf4ca0 Ocean Restart file copy failed on 200pko.daf4cg0 Ocean Restart file copy failed on 200pko.daf4ci0 Ocean Restart file copy failed on 200pko.daf5150 Ocean Restart file copy failed on 200pko.daf51f0 Ocean Restart file copy failed on 200pko.daf51m0 Ocean Restart file copy failed on 200pko.daf51r0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.daf52c0 Ocean Restart file copy failed on 200pko.daf5390 Ocean Restart file copy failed on 200pko.daf53r0 Ocean Restart file copy failed on 200pko.daf55d0 Ocean Restart file copy failed on 200pko.daf55g0 Ocean Restart file copy failed on 200pko.daf5850 Ocean Restart file copy failed on 200pko.daf59m0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.daf5bk0 Ocean Restart file copy failed on 200pko.daf5c40 Ocean Restart file copy failed on 200pko.daf6150 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.daf63t0 Ocean Restart file copy failed on 200pko.daf6490 Ocean Restart file copy failed on 200pko.daf64c0 Ocean Restart file copy failed on 200pko.daf65s0 Ocean Restart file copy failed on 200pko.daf66t0 Ocean Restart file copy failed on 200pko.daf6710 Ocean Restart file copy failed on 200pko.daf6790 Ocean Restart file copy failed on 200pko.daf6820 Ocean Restart file copy failed on 200pko.daf6830 Ocean Restart file copy failed on 200pko.daf6840 Ocean Restart file copy failed on 200pko.daf6850 Ocean Restart file copy failed on 200pko.daf6bb0 Ocean Restart file copy failed on 200pko.daf71h0 Ocean Restart file copy failed on 200pko.daf72b0 Ocean Restart file copy failed on 200pko.daf72h0 Ocean Restart file copy failed on 200pko.daf72s0 Ocean Restart file copy failed on 200pko.daf73h0 Ocean Restart file copy failed on 200pko.daf74k0 06:48:03 (6052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:50:05 (5464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on 200pko.daf75j0 Ocean Restart file copy failed on 200pko.daf75u0 Ocean Restart file copy failed on 200pko.daf7670 Ocean Restart file copy failed on 200pko.daf77i0 Ocean Restart file copy failed on 200pko.daf77m0 Ocean Restart file copy failed on 200pko.daf7870 Ocean Restart file copy failed on 200pko.daf78b0 Ocean Restart file copy failed on 200pko.daf7960 Ocean Restart file copy failed on 200pko.daf79d0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.daf7ah0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.daf7ar0 Ocean Restart file copy failed on 200pko.daf7b20 Ocean Restart file copy failed on 200pko.daf7bd0 Ocean Restart file copy failed on 200pko.daf8280 Ocean Restart file copy failed on 200pko.daf82u0 Ocean Restart file copy failed on 200pko.daf83d0 Ocean Restart file copy failed on 200pko.daf83l0 Ocean Restart file copy failed on 200pko.daf83t0 Ocean Restart file copy failed on 200pko.daf84q0 Ocean Restart file copy failed on 200pko.daf85f0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.daf86n0 Ocean Restart file copy failed on 200pko.daf8720 Ocean Restart file copy failed on 200pko.daf87b0 Ocean Restart file copy failed on 200pko.daf87p0 Ocean Restart file copy failed on 200pko.daf88c0 Ocean Restart file copy failed on 200pko.daf88k0 Ocean Restart file copy failed on 200pko.daf89e0 Ocean Restart file copy failed on 200pko.daf89h0 Ocean Restart file copy failed on 200pko.daf8ag0 Ocean Restart file copy failed on 200pko.daf8c10 Ocean Restart file copy failed on 200pko.daf8ck0 Ocean Restart file copy failed on 200pko.daf8cn0 Ocean Restart file copy failed on 200pko.daf8cp0 Ocean Restart file copy failed on 200pko.daf91d0 Ocean Restart file copy failed on 200pko.daf91l0 Ocean Restart file copy failed on 200pko.daf91n0 Ocean Restart file copy failed on 200pko.daf92l0 Ocean Restart file copy failed on 200pko.daf94p0 Ocean Restart file copy failed on 200pko.daf94s0 Ocean Restart file copy failed on 200pko.daf9580 Ocean Restart file copy failed on 200pko.daf95j0 Ocean Restart file copy failed on 200pko.daf97p0 Ocean Restart file copy failed on 200pko.daf98c0 Ocean Restart file copy failed on 200pko.daf98s0 Ocean Restart file copy failed on 200pko.daf99p0 Ocean Restart file copy failed on 200pko.daf9a70 Ocean Restart file copy failed on 200pko.daf9b90 Ocean Restart file copy failed on 200pko.daf9cf0 Ocean Restart file copy failed on 200pko.dag0130 Ocean Restart file copy failed on 200pko.dag02n0 Ocean Restart file copy failed on 200pko.dag03q0 Ocean Restart file copy failed on 200pko.dag05d0 Ocean Restart file copy failed on 200pko.dag05r0 Ocean Restart file copy failed on 200pko.dag0660 Ocean Restart file copy failed on 200pko.dag07f0 Ocean Restart file copy failed on 200pko.dag07m0 Ocean Restart file copy failed on 200pko.dag0930 Ocean Restart file copy failed on 200pko.dag0a40 Ocean Restart file copy failed on 200pko.dag0a60 Ocean Restart file copy failed on 200pko.dag0b70 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dag11l0 Ocean Restart file copy failed on 200pko.dag13f0 Ocean Restart file copy failed on 200pko.dag14p0 Ocean Restart file copy failed on 200pko.dag15a0 Ocean Restart file copy failed on 200pko.dag15u0 Ocean Restart file copy failed on 200pko.dag16c0 Ocean Restart file copy failed on 200pko.dag16e0 Ocean Restart file copy failed on 200pko.dag16s0 Ocean Restart file copy failed on 200pko.dag17a0 Ocean Restart file copy failed on 200pko.dag17n0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dag23j0 Ocean Restart file copy failed on 200pko.dag27h0 Ocean Restart file copy failed on 200pko.dag27r0 Ocean Restart file copy failed on 200pko.dag29h0 Ocean Restart file copy failed on 200pko.dag33s0 Ocean Restart file copy failed on 200pko.dag3680 Ocean Restart file copy failed on 200pko.dag36k0 Ocean Restart file copy failed on 200pko.dag38i0 Ocean Restart file copy failed on 200pko.dag3920 Ocean Restart file copy failed on 200pko.dag39d0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dag3a70 Ocean Restart file copy failed on 200pko.dag3cm0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dag4320 Ocean Restart file copy failed on 200pko.dag45g0 Ocean Restart file copy failed on 200pko.dag47d0 Ocean Restart file copy failed on 200pko.dag47g0 Ocean Restart file copy failed on 200pko.dag47o0 Ocean Restart file copy failed on 200pko.dag48c0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dag4co0 Ocean Restart file copy failed on 200pko.dag5240 Ocean Restart file copy failed on 200pko.dag52e0 Ocean Restart file copy failed on 200pko.dag5490 Ocean Restart file copy failed on 200pko.dag5690 Ocean Restart file copy failed on 200pko.dag56q0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dag57m0 Ocean Restart file copy failed on 200pko.dag58d0 Ocean Restart file copy failed on 200pko.dag59f0 Ocean Restart file copy failed on 200pko.dag5ar0 Ocean Restart file copy failed on 200pko.dag5b60 Ocean Restart file copy failed on 200pko.dag5cn0 Ocean Restart file copy failed on 200pko.dag62c0 Ocean Restart file copy failed on 200pko.dag6320 Ocean Restart file copy failed on 200pko.dag63m0 Ocean Restart file copy failed on 200pko.dag65u0 Ocean Restart file copy failed on 200pko.dag67a0 Ocean Restart file copy failed on 200pko.dag67g0 Ocean Restart file copy failed on 200pko.dag6990 Ocean Restart file copy failed on 200pko.dag6a70 Ocean Restart file copy failed on 200pko.dag6b40 Ocean Restart file copy failed on 200pko.dag6ct0 Ocean Restart file copy failed on 200pko.dag71q0 Ocean Restart file copy failed on 200pko.dag74a0 Ocean Restart file copy failed on 200pko.dag74p0 Ocean Restart file copy failed on 200pko.dag75l0 Ocean Restart file copy failed on 200pko.dag7620 Ocean Restart file copy failed on 200pko.dag76j0 Ocean Restart file copy failed on 200pko.dag7730 Ocean Restart file copy failed on 200pko.dag78c0 Ocean Restart file copy failed on 200pko.dag78e0 Ocean Restart file copy failed on 200pko.dag7970 Ocean Restart file copy failed on 200pko.dag79o0 Ocean Restart file copy failed on 200pko.dag7ae0 Ocean Restart file copy failed on 200pko.dag7ak0 Ocean Restart file copy failed on 200pko.dag7b20 Ocean Restart file copy failed on 200pko.dag7b60 Ocean Restart file copy failed on 200pko.dag8110 Ocean Restart file copy failed on 200pko.dag8260 Ocean Restart file copy failed on 200pko.dag84e0 Ocean Restart file copy failed on 200pko.dag85l0 Ocean Restart file copy failed on 200pko.dag85r0 Ocean Restart file copy failed on 200pko.dag85t0 Ocean Restart file copy failed on 200pko.dag87k0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dag89o0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dag8af0 Ocean Restart file copy failed on 200pko.dag8c10 Ocean Restart file copy failed on 200pko.dag8cc0 Ocean Restart file copy failed on 200pko.dag8ce0 Ocean Restart file copy failed on 200pko.dag9160 Ocean Restart file copy failed on 200pko.dag92k0 Ocean Restart file copy failed on 200pko.dag93a0 Ocean Restart file copy failed on 200pko.dag9450 Ocean Restart file copy failed on 200pko.dag95f0 Ocean Restart file copy failed on 200pko.dag95o0 Ocean Restart file copy failed on 200pko.dag95q0 Ocean Restart file copy failed on 200pko.dag97g0 Ocean Restart file copy failed on 200pko.dag97t0 Ocean Restart file copy failed on 200pko.dag9850 Ocean Restart file copy failed on 200pko.dag98r0 Ocean Restart file copy failed on 200pko.dag99q0 Ocean Restart file copy failed on 200pko.dag9b40 Ocean Restart file copy failed on 200pko.dag9bn0 Ocean Restart file copy failed on 200pko.dag9bp0 Ocean Restart file copy failed on 200pko.dag9cn0 Ocean Restart file copy failed on 200pko.dah0210 Ocean Restart file copy failed on 200pko.dah02f0 Ocean Restart file copy failed on 200pko.dah02m0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on 200pko.dah04u0 Ocean Restart file copy failed on 200pko.dah05b0 Ocean Restart file copy failed on 200pko.dah05k0 Ocean Restart file copy failed on 200pko.dah05n0 Ocean Restart file copy failed on 200pko.dah05u0 Ocean Restart file copy failed on 200pko.dah06k0 Ocean Restart file copy failed on 200pko.dah0740 Ocean Restart file copy failed on 200pko.dah07c0 Ocean Restart file copy failed on 200pko.dah0850 Ocean Restart file copy failed on 200pko.dah0880 Ocean Restart file copy failed on 200pko.dah08o0 Ocean Restart file copy failed on 200pko.dah08s0 Ocean Restart file copy failed on 200pko.dah0aj0 Ocean Restart file copy failed on 200pko.dah0bg0 22:04:46 (8944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:47 (8944): No heartbeat from core client for 30 sec - exiting 22:04:48 (8944): No heartbeat from core client for 30 sec - exiting 22:04:49 (8944): No heartbeat from core client for 30 sec - exiting 22:04:50 (8944): No heartbeat from core client for 30 sec - exiting 22:04:51 (8944): No heartbeat from core client for 30 sec - exiting 22:04:52 (8944): No heartbeat from core client for 30 sec - exiting 22:04:53 (8944): No heartbeat from core client for 30 sec - exiting 22:04:54 (8944): No heartbeat from core client for 30 sec - exiting 22:04:55 (8944): No heartbeat from core client for 30 sec - exiting 22:04:56 (8944): No heartbeat from core client for 30 sec - exiting 22:11:03 (7492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:11:04 (7492): No heartbeat from core client for 30 sec - exiting 22:11:05 (7492): No heartbeat from core client for 30 sec - exiting 22:11:06 (7492): No heartbeat from core client for 30 sec - exiting 22:11:07 (7492): No heartbeat from core client for 30 sec - exiting 22:11:08 (7492): No heartbeat from core client for 30 sec - exiting 22:11:09 (7492): No heartbeat from core client for 30 sec - exiting 22:11:10 (7492): No heartbeat from core client for 30 sec - exiting 22:11:11 (7492): No heartbeat from core client for 30 sec - exiting 22:11:12 (7492): No heartbeat from core client for 30 sec - exiting 22:11:13 (7492): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 22:16:49 (12124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:16:50 (12124): No heartbeat from core client for 30 sec - exiting 22:16:51 (12124): No heartbeat from core client for 30 sec - exiting 22:16:52 (12124): No heartbeat from core client for 30 sec - exiting 22:16:53 (12124): No heartbeat from core client for 30 sec - exiting 22:16:54 (12124): No heartbeat from core client for 30 sec - exiting 22:16:55 (12124): No heartbeat from core client for 30 sec - exiting 22:16:56 (12124): No heartbeat from core client for 30 sec - exiting 22:16:57 (12124): No heartbeat from core client for 30 sec - exiting 22:16:58 (12124): No heartbeat from core client for 30 sec - exiting 22:16:59 (12124): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 22:21:04 (12664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:05 (12664): No heartbeat from core client for 30 sec - exiting 22:21:06 (12664): No heartbeat from core client for 30 sec - exiting 22:21:07 (12664): No heartbeat from core client for 30 sec - exiting 22:21:08 (12664): No heartbeat from core client for 30 sec - exiting 22:21:09 (12664): No heartbeat from core client for 30 sec - exiting 22:21:10 (12664): No heartbeat from core client for 30 sec - exiting 22:21:11 (12664): No heartbeat from core client for 30 sec - exiting 22:21:12 (12664): No heartbeat from core client for 30 sec - exiting 22:21:13 (12664): No heartbeat from core client for 30 sec - exiting 22:21:14 (12664): No heartbeat from core client for 30 sec - exiting 22:31:04 (9128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:31:05 (9128): No heartbeat from core client for 30 sec - exiting 22:31:06 (9128): No heartbeat from core client for 30 sec - exiting 22:31:07 (9128): No heartbeat from core client for 30 sec - exiting 22:31:08 (9128): No heartbeat from core client for 30 sec - exiting 22:31:09 (9128): No heartbeat from core client for 30 sec - exiting 22:31:10 (9128): No heartbeat from core client for 30 sec - exiting 22:31:11 (9128): No heartbeat from core client for 30 sec - exiting 22:31:12 (9128): No heartbeat from core client for 30 sec - exiting 22:31:13 (9128): No heartbeat from core client for 30 sec - exiting 22:31:14 (9128): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 22:41:06 (9944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:41:07 (9944): No heartbeat from core client for 30 sec - exiting 22:41:08 (9944): No heartbeat from core client for 30 sec - exiting 22:41:09 (9944): No heartbeat from core client for 30 sec - exiting 22:41:10 (9944): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 22:51:06 (13304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:51:07 (13304): No heartbeat from core client for 30 sec - exiting 22:51:08 (13304): No heartbeat from core client for 30 sec - exiting 22:51:09 (13304): No heartbeat from core client for 30 sec - exiting 22:51:10 (13304): No heartbeat from core client for 30 sec - exiting 22:51:11 (13304): No heartbeat from core client for 30 sec - exiting 22:51:12 (13304): No heartbeat from core client for 30 sec - exiting 22:51:13 (13304): No heartbeat from core client for 30 sec - exiting 22:51:14 (13304): No heartbeat from core client for 30 sec - exiting 22:51:15 (13304): No heartbeat from core client for 30 sec - exiting 22:51:16 (13304): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 23:00:55 (12276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:00:56 (12276): No heartbeat from core client for 30 sec - exiting 23:00:57 (12276): No heartbeat from core client for 30 sec - exiting 23:00:58 (12276): No heartbeat from core client for 30 sec - exiting 23:00:59 (12276): No heartbeat from core client for 30 sec - exiting 23:01:00 (12276): No heartbeat from core client for 30 sec - exiting 23:01:01 (12276): No heartbeat from core client for 30 sec - exiting 23:01:02 (12276): No heartbeat from core client for 30 sec - exiting 23:01:03 (12276): No heartbeat from core client for 30 sec - exiting 23:01:04 (12276): No heartbeat from core client for 30 sec - exiting 23:01:05 (12276): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 23:06:43 (1664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:06:44 (1664): No heartbeat from core client for 30 sec - exiting 23:06:45 (1664): No heartbeat from core client for 30 sec - exiting 23:06:46 (1664): No heartbeat from core client for 30 sec - exiting 23:06:47 (1664): No heartbeat from core client for 30 sec - exiting 23:06:48 (1664): No heartbeat from core client for 30 sec - exiting 23:06:49 (1664): No heartbeat from core client for 30 sec - exiting 23:06:50 (1664): No heartbeat from core client for 30 sec - exiting 23:06:51 (1664): No heartbeat from core client for 30 sec - exiting 23:06:52 (1664): No heartbeat from core client for 30 sec - exiting 23:06:53 (1664): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on 200pko.dah11s0 Atmos Hold Restart file rename failed on atmos_restart.hold 23:12:49 (12184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:22:51 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:22:52 (4304): No heartbeat from core client for 30 sec - exiting 23:22:53 (4304): No heartbeat from core client for 30 sec - exiting 23:22:54 (4304): No heartbeat from core client for 30 sec - exiting 23:22:55 (4304): No heartbeat from core client for 30 sec - exiting 23:22:56 (4304): No heartbeat from core client for 30 sec - exiting 23:22:57 (4304): No heartbeat from core client for 30 sec - exiting 23:22:58 (4304): No heartbeat from core client for 30 sec - exiting 23:22:59 (4304): No heartbeat from core client for 30 sec - exiting 23:23:00 (4304): No heartbeat from core client for 30 sec - exiting 23:23:01 (4304): No heartbeat from core client for 30 sec - exiting 23:30:27 (12296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:32:52 (13148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:32:53 (13148): No heartbeat from core client for 30 sec - exiting 23:32:54 (13148): No heartbeat from core client for 30 sec - exiting 23:32:55 (13148): No heartbeat from core client for 30 sec - exiting 23:32:56 (13148): No heartbeat from core client for 30 sec - exiting 23:32:57 (13148): No heartbeat from core client for 30 sec - exiting 23:32:58 (13148): No heartbeat from core client for 30 sec - exiting 23:32:59 (13148): No heartbeat from core client for 30 sec - exiting 23:33:00 (13148): No heartbeat from core client for 30 sec - exiting 23:33:01 (13148): No heartbeat from core client for 30 sec - exiting 23:33:02 (13148): No heartbeat from core client for 30 sec - exiting 23:41:00 (9492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:41:01 (9492): No heartbeat from core client for 30 sec - exiting 23:41:02 (9492): No heartbeat from core client for 30 sec - exiting 23:41:03 (9492): No heartbeat from core client for 30 sec - exiting 23:41:04 (9492): No heartbeat from core client for 30 sec - exiting 23:41:05 (9492): No heartbeat from core client for 30 sec - exiting 23:41:06 (9492): No heartbeat from core client for 30 sec - exiting 23:41:07 (9492): No heartbeat from core client for 30 sec - exiting 23:41:08 (9492): No heartbeat from core client for 30 sec - exiting 23:41:09 (9492): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 23:42:52 (12812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:42:53 (12812): No heartbeat from core client for 30 sec - exiting 23:42:54 (12812): No heartbeat from core client for 30 sec - exiting 23:42:55 (12812): No heartbeat from core client for 30 sec - exiting 23:42:56 (12812): No heartbeat from core client for 30 sec - exiting 23:42:57 (12812): No heartbeat from core client for 30 sec - exiting 23:42:58 (12812): No heartbeat from core client for 30 sec - exiting 23:42:59 (12812): No heartbeat from core client for 30 sec - exiting 23:43:00 (12812): No heartbeat from core client for 30 sec - exiting 23:43:01 (12812): No heartbeat from core client for 30 sec - exiting 23:43:02 (12812): No heartbeat from core client for 30 sec - exiting 23:52:54 (12036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:52:55 (12036): No heartbeat from core client for 30 sec - exiting 23:52:56 (12036): No heartbeat from core client for 30 sec - exiting 23:52:57 (12036): No heartbeat from core client for 30 sec - exiting 23:52:58 (12036): No heartbeat from core client for 30 sec - exiting 23:52:59 (12036): No heartbeat from core client for 30 sec - exiting 23:53:00 (12036): No heartbeat from core client for 30 sec - exiting 23:53:01 (12036): No heartbeat from core client for 30 sec - exiting 23:53:02 (12036): No heartbeat from core client for 30 sec - exiting 23:53:03 (12036): No heartbeat from core client for 30 sec - exiting 23:53:04 (12036): No heartbeat from core client for 30 sec - exiting 00:02:56 (5584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:02:57 (5584): No heartbeat from core client for 30 sec - exiting 00:02:58 (5584): No heartbeat from core client for 30 sec - exiting 00:02:59 (5584): No heartbeat from core client for 30 sec - exiting 00:03:00 (5584): No heartbeat from core client for 30 sec - exiting 00:03:01 (5584): No heartbeat from core client for 30 sec - exiting 00:03:02 (5584): No heartbeat from core client for 30 sec - exiting 00:03:03 (5584): No heartbeat from core client for 30 sec - exiting 00:03:04 (5584): No heartbeat from core client for 30 sec - exiting 00:03:05 (5584): No heartbeat from core client for 30 sec - exiting 00:03:06 (5584): No heartbeat from core client for 30 sec - exiting 00:12:56 (11848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:12:57 (11848): No heartbeat from core client for 30 sec - exiting 00:12:58 (11848): No heartbeat from core client for 30 sec - exiting 00:12:59 (11848): No heartbeat from core client for 30 sec - exiting 00:13:00 (11848): No heartbeat from core client for 30 sec - exiting 00:13:01 (11848): No heartbeat from core client for 30 sec - exiting 00:13:02 (11848): No heartbeat from core client for 30 sec - exiting 00:13:03 (11848): No heartbeat from core client for 30 sec - exiting 00:13:04 (11848): No heartbeat from core client for 30 sec - exiting 00:13:05 (11848): No heartbeat from core client for 30 sec - exiting 00:13:06 (11848): No heartbeat from core client for 30 sec - exiting 00:17:57 (11980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:17:58 (11980): No heartbeat from core client for 30 sec - exiting 00:17:59 (11980): No heartbeat from core client for 30 sec - exiting 00:18:00 (11980): No heartbeat from core client for 30 sec - exiting 00:18:01 (11980): No heartbeat from core client for 30 sec - exiting 00:18:02 (11980): No heartbeat from core client for 30 sec - exiting 00:18:03 (11980): No heartbeat from core client for 30 sec - exiting 00:18:04 (11980): No heartbeat from core client for 30 sec - exiting 00:18:05 (11980): No heartbeat from core client for 30 sec - exiting 00:18:06 (11980): No heartbeat from core client for 30 sec - exiting 00:22:57 (12388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:53 (13192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:54 (13192): No heartbeat from core client for 30 sec - exiting 00:28:55 (13192): No heartbeat from core client for 30 sec - exiting 00:28:56 (13192): No heartbeat from core client for 30 sec - exiting 00:28:57 (13192): No heartbeat from core client for 30 sec - exiting 00:28:58 (13192): No heartbeat from core client for 30 sec - exiting 00:28:59 (13192): No heartbeat from core client for 30 sec - exiting 00:29:00 (13192): No heartbeat from core client for 30 sec - exiting 00:29:01 (13192): No heartbeat from core client for 30 sec - exiting 00:29:02 (13192): No heartbeat from core client for 30 sec - exiting 00:29:03 (13192): No heartbeat from core client for 30 sec - exiting 00:32:59 (12892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:43:00 (12952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:53:01 (11332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:03:01 (8464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:13:00 (10824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:15:01 (11848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:18:48 (9812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:38:50 (10560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:48:50 (1664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:58:51 (3224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:08:52 (7020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:18:53 (11180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:28:53 (12544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:38:54 (13008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:43:55 (12872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:58:56 (13096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:08:56 (8876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:18:57 (13172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:28:57 (9360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:32:35 (12468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:36:29 (7112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12712, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12712, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8480, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
22 Aug 2012 11:16:48 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 777,600 | 843,139 | 1.0843 |
22 Aug 2012 04:05:16 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 751,680 | 814,237 | 1.0832 |
21 Aug 2012 19:00:39 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 725,760 | 785,378 | 1.0821 |
21 Aug 2012 10:55:27 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 699,840 | 756,449 | 1.0809 |
21 Aug 2012 02:43:29 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 673,920 | 727,541 | 1.0796 |
20 Aug 2012 18:37:55 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 648,000 | 698,481 | 1.0779 |
20 Aug 2012 10:29:47 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 622,080 | 669,631 | 1.0764 |
20 Aug 2012 02:23:12 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 596,160 | 640,763 | 1.0748 |
19 Aug 2012 18:30:34 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 570,240 | 611,832 | 1.0729 |
19 Aug 2012 10:09:33 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 544,320 | 583,020 | 1.0711 |
19 Aug 2012 02:02:54 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 518,400 | 554,229 | 1.0691 |
18 Aug 2012 17:56:09 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 492,480 | 525,493 | 1.0670 |
18 Aug 2012 09:49:18 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 466,560 | 496,816 | 1.0648 |
18 Aug 2012 01:42:50 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 440,640 | 468,173 | 1.0625 |
17 Aug 2012 17:29:42 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 414,720 | 439,413 | 1.0595 |
17 Aug 2012 09:04:54 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 388,800 | 410,268 | 1.0552 |
17 Aug 2012 01:03:23 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 362,880 | 381,687 | 1.0518 |
16 Aug 2012 17:05:38 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 336,960 | 353,129 | 1.0480 |
16 Aug 2012 09:01:01 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 311,040 | 324,608 | 1.0436 |
16 Aug 2012 01:24:16 | 1188461 | 15091727 | hadcm3n_200p_1940_40_007958935_4 | 285,120 | 297,921 | 1.0449 |
©2024 cpdn.org