Task 15598148

Name	hadcm3n_4a2e_1940_40_008310745_0
Workunit	8461880
Created	8 Feb 2013, 2:00:21 UTC
Sent	9 Feb 2013, 1:11:04 UTC
Report deadline	11 May 2013, 8:38:15 UTC
Received	1 Apr 2013, 8:47:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1175123
Run time	41 days 17 hours 10 min 55 sec
CPU time	19 days 21 hours 3 min 52 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	0.82 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:48:45 (2168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:33:33 (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:57:41 (2252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C01:28:29 (924): No heartbeat from core client for 30 sec - exiting 01:28:31 (924): No heartbeat from core client for 30 sec - exiting 01:28:32 (924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:09:26 (1276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:03:42 (2756): No heartbeat from core client for 30 sec - exiting 02:03:45 (2756): No heartbeat from core client for 30 sec - exiting 02:03:46 (2756): No heartbeat from core client for 30 sec - exiting 02:03:47 (2756): No heartbeat from core client for 30 sec - exiting 02:03:48 (2756): No heartbeat from core client for 30 sec - exiting 02:03:50 (2756): No heartbeat from core client for 30 sec - exiting 02:03:51 (2756): No heartbeat from core client for 30 sec - exiting 02:03:52 (2756): No heartbeat from core client for 30 sec - exiting 02:03:53 (2756): No heartbeat from core client for 30 sec - exiting 02:03:54 (2756): No heartbeat from core client for 30 sec - exiting 02:03:55 (2756): No heartbeat from core client for 30 sec - exiting 02:03:57 (2756): No heartbeat from core client for 30 sec - exiting 02:03:58 (2756): No heartbeat from core client for 30 sec - exiting 02:03:59 (2756): No heartbeat from core client for 30 sec - exiting 02:04:00 (2756): No heartbeat from core client for 30 sec - exiting 02:04:01 (2756): No heartbeat from core client for 30 sec - exiting 02:04:02 (2756): No heartbeat from core client for 30 sec - exiting 02:04:03 (2756): No heartbeat from core client for 30 sec - exiting 02:04:04 (2756): No heartbeat from core client for 30 sec - exiting 02:04:05 (2756): No heartbeat from core client for 30 sec - exiting 02:04:06 (2756): No heartbeat from core client for 30 sec - exiting 02:04:08 (2756): No heartbeat from core client for 30 sec - exiting 02:04:09 (2756): No heartbeat from core client for 30 sec - exiting 02:04:10 (2756): No heartbeat from core client for 30 sec - exiting 02:04:11 (2756): No heartbeat from core client for 30 sec - exiting 02:04:12 (2756): No heartbeat from core client for 30 sec - exiting 02:04:13 (2756): No heartbeat from core client for 30 sec - exiting 02:04:14 (2756): No heartbeat from core client for 30 sec - exiting 02:04:15 (2756): No heartbeat from core client for 30 sec - exiting 02:04:16 (2756): No heartbeat from core client for 30 sec - exiting 02:04:17 (2756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:04:19 (2756): No heartbeat from core client for 30 sec - exiting 02:04:20 (2756): No heartbeat from core client for 30 sec - exiting 02:04:22 (2756): No heartbeat from core client for 30 sec - exiting 02:04:23 (2756): No heartbeat from core client for 30 sec - exiting 02:04:24 (2756): No heartbeat from core client for 30 sec - exiting 02:04:25 (2756): No heartbeat from core client for 30 sec - exiting 02:04:26 (2756): No heartbeat from core client for 30 sec - exiting 02:04:27 (2756): No heartbeat from core client for 30 sec - exiting 02:04:28 (2756): No heartbeat from core client for 30 sec - exiting 02:04:29 (2756): No heartbeat from core client for 30 sec - exiting 03:12:24 (392): No heartbeat from core client for 30 sec - exiting 03:12:37 (392): No heartbeat from core client for 30 sec - exiting 03:12:38 (392): No heartbeat from core client for 30 sec - exiting 03:12:39 (392): No heartbeat from core client for 30 sec - exiting 03:12:40 (392): No heartbeat from core client for 30 sec - exiting 03:12:42 (392): No heartbeat from core client for 30 sec - exiting 03:12:43 (392): No heartbeat from core client for 30 sec - exiting 03:12:44 (392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:12:46 (392): No heartbeat from core client for 30 sec - exiting 03:12:52 (392): No heartbeat from core client for 30 sec - exiting 03:12:54 (392): No heartbeat from core client for 30 sec - exiting 03:12:56 (392): No heartbeat from core client for 30 sec - exiting 03:12:57 (392): No heartbeat from core client for 30 sec - exiting 03:12:59 (392): No heartbeat from core client for 30 sec - exiting 03:13:00 (392): No heartbeat from core client for 30 sec - exiting 03:13:01 (392): No heartbeat from core client for 30 sec - exiting 03:13:02 (392): No heartbeat from core client for 30 sec - exiting 03:13:03 (392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 03:35:09 (2836): No heartbeat from core client for 30 sec - exiting 03:35:12 (2836): No heartbeat from core client for 30 sec - exiting 03:35:13 (2836): No heartbeat from core client for 30 sec - exiting 03:35:14 (2836): No heartbeat from core client for 30 sec - exiting 03:35:15 (2836): No heartbeat from core client for 30 sec - exiting 03:35:16 (2836): No heartbeat from core client for 30 sec - exiting 03:35:17 (2836): No heartbeat from core client for 30 sec - exiting 03:35:18 (2836): No heartbeat from core client for 30 sec - exiting 03:35:19 (2836): No heartbeat from core client for 30 sec - exiting 03:35:21 (2836): No heartbeat from core client for 30 sec - exiting 03:35:22 (2836): No heartbeat from core client for 30 sec - exiting 03:35:23 (2836): No heartbeat from core client for 30 sec - exiting 03:35:24 (2836): No heartbeat from core client for 30 sec - exiting 03:35:25 (2836): No heartbeat from core client for 30 sec - exiting 03:35:26 (2836): No heartbeat from core client for 30 sec - exiting 03:35:27 (2836): No heartbeat from core client for 30 sec - exiting 03:35:28 (2836): No heartbeat from core client for 30 sec - exiting 03:35:29 (2836): No heartbeat from core client for 30 sec - exiting 03:35:30 (2836): No heartbeat from core client for 30 sec - exiting 03:35:32 (2836): No heartbeat from core client for 30 sec - exiting 03:35:33 (2836): No heartbeat from core client for 30 sec - exiting 03:35:34 (2836): No heartbeat from core client for 30 sec - exiting 03:35:37 (2836): No heartbeat from core client for 30 sec - exiting 03:35:39 (2836): No heartbeat from core client for 30 sec - exiting 03:35:40 (2836): No heartbeat from core client for 30 sec - exiting 03:35:41 (2836): No heartbeat from core client for 30 sec - exiting 03:35:42 (2836): No heartbeat from core client for 30 sec - exiting 03:35:43 (2836): No heartbeat from core client for 30 sec - exiting 03:35:45 (2836): No heartbeat from core client for 30 sec - exiting 03:35:46 (2836): No heartbeat from core client for 30 sec - exiting 03:35:47 (2836): No heartbeat from core client for 30 sec - exiting 03:35:48 (2836): No heartbeat from core client for 30 sec - exiting 03:35:49 (2836): No heartbeat from core client for 30 sec - exiting 03:35:51 (2836): No heartbeat from core client for 30 sec - exiting 03:35:52 (2836): No heartbeat from core client for 30 sec - exiting 03:35:53 (2836): No heartbeat from core client for 30 sec - exiting 03:35:54 (2836): No heartbeat from core client for 30 sec - exiting 03:35:55 (2836): No heartbeat from core client for 30 sec - exiting 03:35:56 (2836): No heartbeat from core client for 30 sec - exiting 03:35:57 (2836): No heartbeat from core client for 30 sec - exiting 03:35:58 (2836): No heartbeat from core client for 30 sec - exiting 03:35:59 (2836): No heartbeat from core client for 30 sec - exiting 03:36:01 (2836): No heartbeat from core client for 30 sec - exiting 03:36:02 (2836): No heartbeat from core client for 30 sec - exiting 03:36:03 (2836): No heartbeat from core client for 30 sec - exiting 03:36:04 (2836): No heartbeat from core client for 30 sec - exiting 03:36:05 (2836): No heartbeat from core client for 30 sec - exiting 03:36:06 (2836): No heartbeat from core client for 30 sec - exiting 03:36:07 (2836): No heartbeat from core client for 30 sec - exiting 03:36:08 (2836): No heartbeat from core client for 30 sec - exiting 03:36:09 (2836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:44:29 (8452): No heartbeat from core client for 30 sec - exiting 22:44:31 (8452): No heartbeat from core client for 30 sec - exiting 22:44:32 (8452): No heartbeat from core client for 30 sec - exiting 22:44:33 (8452): No heartbeat from core client for 30 sec - exiting 22:44:34 (8452): No heartbeat from core client for 30 sec - exiting 22:44:35 (8452): No heartbeat from core client for 30 sec - exiting 22:44:36 (8452): No heartbeat from core client for 30 sec - exiting 22:44:37 (8452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:08:16 (5712): No heartbeat from core client for 30 sec - exiting 01:08:17 (5712): No heartbeat from core client for 30 sec - exiting 01:08:18 (5712): No heartbeat from core client for 30 sec - exiting 01:08:19 (5712): No heartbeat from core client for 30 sec - exiting 01:08:20 (5712): No heartbeat from core client for 30 sec - exiting 01:08:21 (5712): No heartbeat from core client for 30 sec - exiting 01:08:22 (5712): No heartbeat from core client for 30 sec - exiting 01:08:23 (5712): No heartbeat from core client for 30 sec - exiting 01:08:24 (5712): No heartbeat from core client for 30 sec - exiting 01:08:25 (5712): No heartbeat from core client for 30 sec - exiting 01:08:26 (5712): No heartbeat from core client for 30 sec - exiting 01:08:27 (5712): No heartbeat from core client for 30 sec - exiting 01:08:28 (5712): No heartbeat from core client for 30 sec - exiting 01:08:29 (5712): No heartbeat from core client for 30 sec - exiting 01:08:30 (5712): No heartbeat from core client for 30 sec - exiting 01:08:31 (5712): No heartbeat from core client for 30 sec - exiting 01:08:32 (5712): No heartbeat from core client for 30 sec - exiting 01:08:33 (5712): No heartbeat from core client for 30 sec - exiting 01:08:34 (5712): No heartbeat from core client for 30 sec - exiting 01:08:35 (5712): No heartbeat from core client for 30 sec - exiting 01:08:36 (5712): No heartbeat from core client for 30 sec - exiting 01:08:38 (5712): No heartbeat from core client for 30 sec - exiting 01:08:39 (5712): No heartbeat from core client for 30 sec - exiting 01:08:40 (5712): No heartbeat from core client for 30 sec - exiting 01:08:41 (5712): No heartbeat from core client for 30 sec - exiting 01:08:42 (5712): No heartbeat from core client for 30 sec - exiting 01:08:43 (5712): No heartbeat from core client for 30 sec - exiting 01:08:44 (5712): No heartbeat from core client for 30 sec - exiting 01:08:45 (5712): No heartbeat from core client for 30 sec - exiting 01:08:46 (5712): No heartbeat from core client for 30 sec - exiting 01:08:47 (5712): No heartbeat from core client for 30 sec - exiting 01:08:48 (5712): No heartbeat from core client for 30 sec - exiting 01:08:49 (5712): No heartbeat from core client for 30 sec - exiting 01:08:50 (5712): No heartbeat from core client for 30 sec - exiting 01:08:51 (5712): No heartbeat from core client for 30 sec - exiting 01:08:52 (5712): No heartbeat from core client for 30 sec - exiting 01:08:53 (5712): No heartbeat from core client for 30 sec - exiting 01:08:55 (5712): No heartbeat from core client for 30 sec - exiting 01:08:56 (5712): No heartbeat from core client for 30 sec - exiting 01:08:57 (5712): No heartbeat from core client for 30 sec - exiting 01:08:58 (5712): No heartbeat from core client for 30 sec - exiting 01:08:59 (5712): No heartbeat from core client for 30 sec - exiting 01:09:01 (5712): No heartbeat from core client for 30 sec - exiting 01:09:02 (5712): No heartbeat from core client for 30 sec - exiting 01:09:03 (5712): No heartbeat from core client for 30 sec - exiting 01:09:04 (5712): No heartbeat from core client for 30 sec - exiting 01:09:05 (5712): No heartbeat from core client for 30 sec - exiting 01:09:06 (5712): No heartbeat from core client for 30 sec - exiting 01:09:07 (5712): No heartbeat from core client for 30 sec - exiting 01:09:08 (5712): No heartbeat from core client for 30 sec - exiting 01:09:09 (5712): No heartbeat from core client for 30 sec - exiting 01:09:10 (5712): No heartbeat from core client for 30 sec - exiting 01:09:12 (5712): No heartbeat from core client for 30 sec - exiting 01:09:13 (5712): No heartbeat from core client for 30 sec - exiting 01:09:14 (5712): No heartbeat from core client for 30 sec - exiting 01:09:15 (5712): No heartbeat from core client for 30 sec - exiting 01:09:16 (5712): No heartbeat from core client for 30 sec - exiting 01:09:17 (5712): No heartbeat from core client for 30 sec - exiting 01:09:18 (5712): No heartbeat from core client for 30 sec - exiting 01:09:19 (5712): No heartbeat from core client for 30 sec - exiting 01:09:20 (5712): No heartbeat from core client for 30 sec - exiting 01:09:21 (5712): No heartbeat from core client for 30 sec - exiting 01:09:22 (5712): No heartbeat from core client for 30 sec - exiting 01:09:23 (5712): No heartbeat from core client for 30 sec - exiting 01:09:24 (5712): No heartbeat from core client for 30 sec - exiting 01:09:25 (5712): No heartbeat from core client for 30 sec - exiting 01:09:26 (5712): No heartbeat from core client for 30 sec - exiting 01:09:27 (5712): No heartbeat from core client for 30 sec - exiting 01:09:28 (5712): No heartbeat from core client for 30 sec - exiting 01:09:29 (5712): No heartbeat from core client for 30 sec - exiting 01:09:30 (5712): No heartbeat from core client for 30 sec - exiting 01:09:31 (5712): No heartbeat from core client for 30 sec - exiting 01:09:32 (5712): No heartbeat from core client for 30 sec - exiting 01:09:33 (5712): No heartbeat from core client for 30 sec - exiting 01:09:34 (5712): No heartbeat from core client for 30 sec - exiting 01:09:35 (5712): No heartbeat from core client for 30 sec - exiting 01:09:36 (5712): No heartbeat from core client for 30 sec - exiting 01:09:37 (5712): No heartbeat from core client for 30 sec - exiting 01:09:38 (5712): No heartbeat from core client for 30 sec - exiting 01:09:39 (5712): No heartbeat from core client for 30 sec - exiting 01:09:40 (5712): No heartbeat from core client for 30 sec - exiting 01:09:41 (5712): No heartbeat from core client for 30 sec - exiting 01:09:42 (5712): No heartbeat from core client for 30 sec - exiting 01:09:43 (5712): No heartbeat from core client for 30 sec - exiting 01:09:45 (5712): No heartbeat from core client for 30 sec - exiting 01:09:46 (5712): No heartbeat from core client for 30 sec - exiting 01:09:47 (5712): No heartbeat from core client for 30 sec - exiting 01:09:48 (5712): No heartbeat from core client for 30 sec - exiting 01:09:49 (5712): No heartbeat from core client for 30 sec - exiting 01:09:50 (5712): No heartbeat from core client for 30 sec - exiting 01:09:51 (5712): No heartbeat from core client for 30 sec - exiting 01:09:53 (5712): No heartbeat from core client for 30 sec - exiting 01:09:54 (5712): No heartbeat from core client for 30 sec - exiting 01:09:55 (5712): No heartbeat from core client for 30 sec - exiting 01:09:56 (5712): No heartbeat from core client for 30 sec - exiting 01:09:57 (5712): No heartbeat from core client for 30 sec - exiting 01:09:58 (5712): No heartbeat from core client for 30 sec - exiting 01:09:59 (5712): No heartbeat from core client for 30 sec - exiting 01:10:00 (5712): No heartbeat from core client for 30 sec - exiting 01:10:01 (5712): No heartbeat from core client for 30 sec - exiting 01:10:02 (5712): No heartbeat from core client for 30 sec - exiting 01:10:03 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:10:04 (5712): No heartbeat from core client for 30 sec - exiting 01:10:05 (5712): No heartbeat from core client for 30 sec - exiting 01:10:06 (5712): No heartbeat from core client for 30 sec - exiting 01:10:07 (5712): No heartbeat from core client for 30 sec - exiting 16:07:49 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:19:05 (2956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:50:55 (3168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:35:57 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:50 (3952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:04:23 (1288): No heartbeat from core client for 30 sec - exiting 22:04:25 (1288): No heartbeat from core client for 30 sec - exiting 22:04:26 (1288): No heartbeat from core client for 30 sec - exiting 22:04:27 (1288): No heartbeat from core client for 30 sec - exiting 22:04:28 (1288): No heartbeat from core client for 30 sec - exiting 22:04:30 (1288): No heartbeat from core client for 30 sec - exiting 22:04:31 (1288): No heartbeat from core client for 30 sec - exiting 22:04:33 (1288): No heartbeat from core client for 30 sec - exiting 22:04:34 (1288): No heartbeat from core client for 30 sec - exiting 22:04:35 (1288): No heartbeat from core client for 30 sec - exiting 22:04:36 (1288): No heartbeat from core client for 30 sec - exiting 22:04:37 (1288): No heartbeat from core client for 30 sec - exiting 22:04:38 (1288): No heartbeat from core client for 30 sec - exiting 22:04:39 (1288): No heartbeat from core client for 30 sec - exiting 22:04:40 (1288): No heartbeat from core client for 30 sec - exiting 22:04:42 (1288): No heartbeat from core client for 30 sec - exiting 22:04:44 (1288): No heartbeat from core client for 30 sec - exiting 22:04:45 (1288): No heartbeat from core client for 30 sec - exiting 22:04:47 (1288): No heartbeat from core client for 30 sec - exiting 22:04:48 (1288): No heartbeat from core client for 30 sec - exiting 22:04:49 (1288): No heartbeat from core client for 30 sec - exiting 22:04:51 (1288): No heartbeat from core client for 30 sec - exiting 22:04:52 (1288): No heartbeat from core client for 30 sec - exiting 22:04:53 (1288): No heartbeat from core client for 30 sec - exiting 22:04:55 (1288): No heartbeat from core client for 30 sec - exiting 22:04:56 (1288): No heartbeat from core client for 30 sec - exiting 22:04:57 (1288): No heartbeat from core client for 30 sec - exiting 22:04:58 (1288): No heartbeat from core client for 30 sec - exiting 22:05:01 (1288): No heartbeat from core client for 30 sec - exiting 22:05:05 (1288): No heartbeat from core client for 30 sec - exiting 22:05:06 (1288): No heartbeat from core client for 30 sec - exiting 22:05:07 (1288): No heartbeat from core client for 30 sec - exiting 22:05:08 (1288): No heartbeat from core client for 30 sec - exiting 22:05:09 (1288): No heartbeat from core client for 30 sec - exiting 22:05:10 (1288): No heartbeat from core client for 30 sec - exiting 22:05:11 (1288): No heartbeat from core client for 30 sec - exiting 22:05:12 (1288): No heartbeat from core client for 30 sec - exiting 22:05:14 (1288): No heartbeat from core client for 30 sec - exiting 22:05:15 (1288): No heartbeat from core client for 30 sec - exiting 22:05:16 (1288): No heartbeat from core client for 30 sec - exiting 22:05:17 (1288): No heartbeat from core client for 30 sec - exiting 22:05:18 (1288): No heartbeat from core client for 30 sec - exiting 22:05:20 (1288): No heartbeat from core client for 30 sec - exiting 22:05:21 (1288): No heartbeat from core client for 30 sec - exiting 22:05:22 (1288): No heartbeat from core client for 30 sec - exiting 22:05:23 (1288): No heartbeat from core client for 30 sec - exiting 22:05:24 (1288): No heartbeat from core client for 30 sec - exiting 22:05:26 (1288): No heartbeat from core client for 30 sec - exiting 22:05:27 (1288): No heartbeat from core client for 30 sec - exiting 22:05:28 (1288): No heartbeat from core client for 30 sec - exiting 22:05:30 (1288): No heartbeat from core client for 30 sec - exiting 22:05:31 (1288): No heartbeat from core client for 30 sec - exiting 22:05:32 (1288): No heartbeat from core client for 30 sec - exiting 22:05:33 (1288): No heartbeat from core client for 30 sec - exiting 22:05:35 (1288): No heartbeat from core client for 30 sec - exiting 22:05:36 (1288): No heartbeat from core client for 30 sec - exiting 22:05:37 (1288): No heartbeat from core client for 30 sec - exiting 22:05:38 (1288): No heartbeat from core client for 30 sec - exiting 22:05:40 (1288): No heartbeat from core client for 30 sec - exiting 22:05:41 (1288): No heartbeat from core client for 30 sec - exiting 22:05:42 (1288): No heartbeat from core client for 30 sec - exiting 22:05:43 (1288): No heartbeat from core client for 30 sec - exiting 22:05:44 (1288): No heartbeat from core client for 30 sec - exiting 22:05:45 (1288): No heartbeat from core client for 30 sec - exiting 22:05:46 (1288): No heartbeat from core client for 30 sec - exiting 22:05:47 (1288): No heartbeat from core client for 30 sec - exiting 22:05:48 (1288): No heartbeat from core client for 30 sec - exiting 22:05:49 (1288): No heartbeat from core client for 30 sec - exiting 22:05:50 (1288): No heartbeat from core client for 30 sec - exiting 22:05:51 (1288): No heartbeat from core client for 30 sec - exiting 22:05:52 (1288): No heartbeat from core client for 30 sec - exiting 22:05:54 (1288): No heartbeat from core client for 30 sec - exiting 22:05:55 (1288): No heartbeat from core client for 30 sec - exiting 22:05:56 (1288): No heartbeat from core client for 30 sec - exiting 22:05:58 (1288): No heartbeat from core client for 30 sec - exiting 22:05:59 (1288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:03:16 (6096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:32 (8364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:01:50 (764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:16:27 (2784): No heartbeat from core client for 30 sec - exiting 14:16:32 (2784): No heartbeat from core client for 30 sec - exiting 14:16:34 (2784): No heartbeat from core client for 30 sec - exiting 14:16:35 (2784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:29:24 (4504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:05:39 (7372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:39 (4768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:53:29 (5156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:01:32 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:04:17 (7128): No heartbeat from core client for 30 sec - exiting 03:04:18 (7128): No heartbeat from core client for 30 sec - exiting 03:04:19 (7128): No heartbeat from core client for 30 sec - exiting 03:04:20 (7128): No heartbeat from core client for 30 sec - exiting 03:04:21 (7128): No heartbeat from core client for 30 sec - exiting 03:04:22 (7128): No heartbeat from core client for 30 sec - exiting 03:04:23 (7128): No heartbeat from core client for 30 sec - exiting 03:04:24 (7128): No heartbeat from core client for 30 sec - exiting 03:04:25 (7128): No heartbeat from core client for 30 sec - exiting 03:04:26 (7128): No heartbeat from core client for 30 sec - exiting 03:04:27 (7128): No heartbeat from core client for 30 sec - exiting 03:04:28 (7128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:43:36 (5364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 20:43:14 (2308): No heartbeat from core client for 30 sec - exiting 20:43:15 (2308): Can't open init data file - running in standalone mode Could not launch model process. Last Error=1450 Called boinc_finish 20:43:21 (2308): No heartbeat from core client for 30 sec - exiting 00:35:17 (4596): No heartbeat from core client for 30 sec - exiting 00:35:21 (4596): No heartbeat from core client for 30 sec - exiting 00:35:24 (4596): No heartbeat from core client for 30 sec - exiting 00:35:25 (4596): No heartbeat from core client for 30 sec - exiting 00:35:33 (4596): No heartbeat from core client for 30 sec - exiting 00:35:35 (4596): No heartbeat from core client for 30 sec - exiting 00:35:36 (4596): No heartbeat from core client for 30 sec - exiting 00:35:37 (4596): No heartbeat from core client for 30 sec - exiting 00:35:38 (4596): No heartbeat from core client for 30 sec - exiting 00:35:41 (4596): No heartbeat from core client for 30 sec - exiting 00:35:42 (4596): No heartbeat from core client for 30 sec - exiting 00:35:43 (4596): No heartbeat from core client for 30 sec - exiting 00:35:44 (4596): No heartbeat from core client for 30 sec - exiting 00:35:45 (4596): No heartbeat from core client for 30 sec - exiting 00:35:46 (4596): No heartbeat from core client for 30 sec - exiting 00:35:47 (4596): No heartbeat from core client for 30 sec - exiting 00:35:48 (4596): No heartbeat from core client for 30 sec - exiting 00:35:49 (4596): No heartbeat from core client for 30 sec - exiting 00:35:50 (4596): No heartbeat from core client for 30 sec - exiting 00:35:51 (4596): No heartbeat from core client for 30 sec - exiting 00:35:52 (4596): No heartbeat from core client for 30 sec - exiting 00:35:53 (4596): No heartbeat from core client for 30 sec - exiting 00:35:54 (4596): No heartbeat from core client for 30 sec - exiting 00:35:55 (4596): No heartbeat from core client for 30 sec - exiting 00:35:56 (4596): No heartbeat from core client for 30 sec - exiting 00:35:57 (4596): No heartbeat from core client for 30 sec - exiting 00:35:58 (4596): No heartbeat from core client for 30 sec - exiting 00:35:59 (4596): No heartbeat from core client for 30 sec - exiting 00:36:00 (4596): No heartbeat from core client for 30 sec - exiting 00:36:01 (4596): No heartbeat from core client for 30 sec - exiting 00:36:02 (4596): No heartbeat from core client for 30 sec - exiting 00:36:03 (4596): No heartbeat from core client for 30 sec - exiting 00:36:04 (4596): No heartbeat from core client for 30 sec - exiting 00:36:05 (4596): No heartbeat from core client for 30 sec - exiting 00:36:07 (4596): No heartbeat from core client for 30 sec - exiting 00:36:08 (4596): No heartbeat from core client for 30 sec - exiting 00:36:09 (4596): No heartbeat from core client for 30 sec - exiting 00:36:10 (4596): No heartbeat from core client for 30 sec - exiting 00:36:11 (4596): No heartbeat from core client for 30 sec - exiting 00:36:12 (4596): No heartbeat from core client for 30 sec - exiting 00:36:13 (4596): No heartbeat from core client for 30 sec - exiting 00:36:14 (4596): No heartbeat from core client for 30 sec - exiting 00:36:15 (4596): No heartbeat from core client for 30 sec - exiting 00:36:16 (4596): No heartbeat from core client for 30 sec - exiting 00:36:17 (4596): No heartbeat from core client for 30 sec - exiting 00:36:18 (4596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:16:28 (2752): No heartbeat from core client for 30 sec - exiting 01:16:29 (2752): No heartbeat from core client for 30 sec - exiting 01:16:30 (2752): No heartbeat from core client for 30 sec - exiting 01:16:31 (2752): No heartbeat from core client for 30 sec - exiting 01:16:33 (2752): No heartbeat from core client for 30 sec - exiting 01:16:34 (2752): No heartbeat from core client for 30 sec - exiting 01:16:35 (2752): No heartbeat from core client for 30 sec - exiting 01:16:41 (2752): No heartbeat from core client for 30 sec - exiting 01:16:44 (2752): No heartbeat from core client for 30 sec - exiting 01:16:45 (2752): No heartbeat from core client for 30 sec - exiting 01:16:46 (2752): No heartbeat from core client for 30 sec - exiting 01:16:47 (2752): No heartbeat from core client for 30 sec - exiting 01:16:48 (2752): No heartbeat from core client for 30 sec - exiting 01:16:49 (2752): No heartbeat from core client for 30 sec - exiting 01:16:50 (2752): No heartbeat from core client for 30 sec - exiting 01:16:51 (2752): No heartbeat from core client for 30 sec - exiting 01:16:52 (2752): No heartbeat from core client for 30 sec - exiting 01:16:53 (2752): No heartbeat from core client for 30 sec - exiting 01:16:54 (2752): No heartbeat from core client for 30 sec - exiting 01:16:55 (2752): No heartbeat from core client for 30 sec - exiting 01:16:56 (2752): No heartbeat from core client for 30 sec - exiting 01:16:57 (2752): No heartbeat from core client for 30 sec - exiting 01:16:58 (2752): No heartbeat from core client for 30 sec - exiting 01:16:59 (2752): No heartbeat from core client for 30 sec - exiting 01:17:00 (2752): No heartbeat from core client for 30 sec - exiting 01:17:01 (2752): No heartbeat from core client for 30 sec - exiting 01:17:02 (2752): No heartbeat from core client for 30 sec - exiting 01:17:03 (2752): No heartbeat from core client for 30 sec - exiting 01:17:04 (2752): No heartbeat from core client for 30 sec - exiting 01:17:06 (2752): No heartbeat from core client for 30 sec - exiting 01:17:07 (2752): No heartbeat from core client for 30 sec - exiting 01:17:08 (2752): No heartbeat from core client for 30 sec - exiting 01:17:10 (2752): No heartbeat from core client for 30 sec - exiting 01:17:12 (2752): No heartbeat from core client for 30 sec - exiting 01:17:13 (2752): No heartbeat from core client for 30 sec - exiting 01:17:15 (2752): No heartbeat from core client for 30 sec - exiting 01:17:17 (2752): No heartbeat from core client for 30 sec - exiting 01:17:18 (2752): No heartbeat from core client for 30 sec - exiting 01:17:19 (2752): No heartbeat from core client for 30 sec - exiting 01:17:22 (2752): No heartbeat from core client for 30 sec - exiting 01:17:23 (2752): No heartbeat from core client for 30 sec - exiting 01:17:26 (2752): No heartbeat from core client for 30 sec - exiting 01:17:28 (2752): No heartbeat from core client for 30 sec - exiting 01:17:33 (2752): No heartbeat from core client for 30 sec - exiting 01:17:34 (2752): No heartbeat from core client for 30 sec - exiting 01:17:35 (2752): No heartbeat from core client for 30 sec - exiting 01:17:36 (2752): No heartbeat from core client for 30 sec - exiting 01:17:37 (2752): No heartbeat from core client for 30 sec - exiting 01:17:38 (2752): No heartbeat from core client for 30 sec - exiting 01:17:39 (2752): No heartbeat from core client for 30 sec - exiting 01:17:40 (2752): No heartbeat from core client for 30 sec - exiting 01:17:42 (2752): No heartbeat from core client for 30 sec - exiting 01:17:43 (2752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:17:44 (2752): No heartbeat from core client for 30 sec - exiting 01:17:45 (2752): No heartbeat from core client for 30 sec - exiting 01:17:46 (2752): No heartbeat from core client for 30 sec - exiting 01:17:47 (2752): No heartbeat from core client for 30 sec - exiting 01:17:48 (2752): No heartbeat from core client for 30 sec - exiting 01:17:49 (2752): No heartbeat from core client for 30 sec - exiting 01:17:50 (2752): No heartbeat from core client for 30 sec - exiting 23:38:05 (3252): No heartbeat from core client for 30 sec - exiting 23:38:07 (3252): No heartbeat from core client for 30 sec - exiting 23:38:08 (3252): No heartbeat from core client for 30 sec - exiting 23:38:09 (3252): No heartbeat from core client for 30 sec - exiting 23:38:11 (3252): No heartbeat from core client for 30 sec - exiting 23:38:14 (3252): No heartbeat from core client for 30 sec - exiting 23:38:22 (3252): No heartbeat from core client for 30 sec - exiting 23:38:24 (3252): No heartbeat from core client for 30 sec - exiting 23:38:25 (3252): No heartbeat from core client for 30 sec - exiting 23:38:26 (3252): No heartbeat from core client for 30 sec - exiting 23:38:27 (3252): No heartbeat from core client for 30 sec - exiting 23:38:29 (3252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:36:17 (4212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CSuspended CPDN Monitor - Suspend request from BOINC... 02:34:21 (3240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:20:14 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1060, iMonCtr=1 Model crash detected, will try to restart... forrtl: There is not enough space on the disk. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1060, iMonCtr=1 Model crash detected, will try to restart... forrtl: There is not enough space on the disk. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1060, iMonCtr=1 Model crash detected, will try to restart... forrtl: There is not enough space on the disk. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1060, iMonCtr=1 Model crash detected, will try to restart... forrtl: There is not enough space on the diController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1060, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Mar 2013 12:42:24	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	233,280	1,632,573	6.9983
25 Mar 2013 06:51:47	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	207,360	1,466,187	7.0707
20 Mar 2013 00:19:58	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	181,440	1,298,432	7.1563
16 Mar 2013 11:04:01	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	155,520	1,131,804	7.2775
12 Mar 2013 17:41:26	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	129,600	965,167	7.4473
08 Mar 2013 18:09:51	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	103,680	793,951	7.6577
05 Mar 2013 08:25:48	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	77,760	625,208	8.0402
01 Mar 2013 00:40:31	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	51,840	444,283	8.5703
14 Feb 2013 20:23:01	1175123	15598148	hadcm3n_4a2e_1940_40_008310745_0	25,920	230,886	8.9076