Task 13382585

Name	hadcm3n_o3dx_1940_40_007454794_4
Workunit	7652297
Created	13 Sep 2011, 2:05:36 UTC
Sent	13 Sep 2011, 2:12:06 UTC
Report deadline	13 Dec 2011, 9:39:17 UTC
Received	17 Sep 2011, 15:20:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1156833
Run time	1 days 13 hours 5 min 46 sec
CPU time	1 days 6 hours 54 min 44 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	2.34 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 03:38:57 (4636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:39:41 (9308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:42 (9308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:48:34 (6384): No heartbeat from core client for 30 sec - exiting 12:48:35 (6384): No heartbeat from core client for 30 sec - exiting 12:48:36 (6384): No heartbeat from core client for 30 sec - exiting 12:48:37 (6384): No heartbeat from core client for 30 sec - exiting 12:48:38 (6384): No heartbeat from core client for 30 sec - exiting 12:48:39 (6384): No heartbeat from core client for 30 sec - exiting 12:48:40 (6384): No heartbeat from core client for 30 sec - exiting 12:48:41 (6384): No heartbeat from core client for 30 sec - exiting 12:48:42 (6384): No heartbeat from core client for 30 sec - exiting 12:48:43 (6384): No heartbeat from core client for 30 sec - exiting 12:48:44 (6384): No heartbeat from core client for 30 sec - exiting 12:48:45 (6384): No heartbeat from core client for 30 sec - exiting 12:48:46 (6384): No heartbeat from core client for 30 sec - exiting 12:48:47 (6384): No heartbeat from core client for 30 sec - exiting 12:48:48 (6384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:50:23 (7404): No heartbeat from core client for 30 sec - exiting 12:50:24 (7404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 16:23:16 (10028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:19:11 (6832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:19:12 (6832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:15:37 (8512): No heartbeat from core client for 30 sec - exiting 19:15:38 (8512): No heartbeat from core client for 30 sec - exiting 19:15:39 (8512): No heartbeat from core client for 30 sec - exiting 19:15:40 (8512): No heartbeat from core client for 30 sec - exiting 19:15:41 (8512): No heartbeat from core client for 30 sec - exiting 19:15:42 (8512): No heartbeat from core client for 30 sec - exiting 19:15:43 (8512): No heartbeat from core client for 30 sec - exiting 19:15:44 (8512): No heartbeat from core client for 30 sec - exiting 19:15:45 (8512): No heartbeat from core client for 30 sec - exiting 19:15:46 (8512): No heartbeat from core client for 30 sec - exiting 19:15:47 (8512): No heartbeat from core client for 30 sec - exiting 19:15:48 (8512): No heartbeat from core client for 30 sec - exiting 19:15:49 (8512): No heartbeat from core client for 30 sec - exiting 19:15:50 (8512): No heartbeat from core client for 30 sec - exiting 19:15:51 (8512): No heartbeat from core client for 30 sec - exiting 19:15:52 (8512): No heartbeat from core client for 30 sec - exiting 19:15:53 (8512): No heartbeat from core client for 30 sec - exiting 19:15:54 (8512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:58:59 (8652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:59:00 (8652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:43:01 (7876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:03:20 (9524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:03:22 (9524): No heartbeat from core client for 30 sec - exiting 00:04:37 (6372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:45:05 (6952): No heartbeat from core client for 30 sec - exiting 00:45:06 (6952): No heartbeat from core client for 30 sec - exiting 00:45:07 (6952): No heartbeat from core client for 30 sec - exiting 00:45:08 (6952): No heartbeat from core client for 30 sec - exiting 00:45:09 (6952): No heartbeat from core client for 30 sec - exiting 00:45:10 (6952): No heartbeat from core client for 30 sec - exiting 00:45:11 (6952): No heartbeat from core client for 30 sec - exiting 00:45:12 (6952): No heartbeat from core client for 30 sec - exiting 00:45:13 (6952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:20:12 (7476): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:09:59 (4656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:18:30 (6664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:37 (7312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:39 (7312): No heartbeat from core client for 30 sec - exiting 04:15:16 (5728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:17 (5728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 04:30:42 (9588): No heartbeat from core client for 30 sec - exiting 04:30:43 (9588): No heartbeat from core client for 30 sec - exiting 04:30:44 (9588): No heartbeat from core client for 30 sec - exiting 04:30:45 (9588): No heartbeat from core client for 30 sec - exiting 04:30:46 (9588): No heartbeat from core client for 30 sec - exiting 04:30:47 (9588): No heartbeat from core client for 30 sec - exiting 04:30:48 (9588): No heartbeat from core client for 30 sec - exiting 04:30:49 (9588): No heartbeat from core client for 30 sec - exiting 04:30:50 (9588): No heartbeat from core client for 30 sec - exiting 04:30:51 (9588): No heartbeat from core client for 30 sec - exiting 04:30:52 (9588): No heartbeat from core client for 30 sec - exiting 04:30:54 (9588): No heartbeat from core client for 30 sec - exiting 04:30:55 (9588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:43:45 (7824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:52:29 (9288): No heartbeat from core client for 30 sec - exiting 04:52:30 (9288): No heartbeat from core client for 30 sec - exiting 04:52:31 (9288): No heartbeat from core client for 30 sec - exiting 04:52:32 (9288): No heartbeat from core client for 30 sec - exiting 04:52:33 (9288): No heartbeat from core client for 30 sec - exiting 04:52:34 (9288): No heartbeat from core client for 30 sec - exiting 04:52:35 (9288): No heartbeat from core client for 30 sec - exiting 04:52:36 (9288): No heartbeat from core client for 30 sec - exiting 04:52:37 (9288): No heartbeat from core client for 30 sec - exiting 04:52:38 (9288): No heartbeat from core client for 30 sec - exiting 04:52:39 (9288): No heartbeat from core client for 30 sec - exiting 04:52:40 (9288): No heartbeat from core client for 30 sec - exiting 04:52:41 (9288): No heartbeat from core client for 30 sec - exiting 04:52:42 (9288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:02:42 (6660): No heartbeat from core client for 30 sec - exiting 05:02:43 (6660): No heartbeat from core client for 30 sec - exiting 05:02:44 (6660): No heartbeat from core client for 30 sec - exiting 05:02:45 (6660): No heartbeat from core client for 30 sec - exiting 05:02:46 (6660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:39:46 (6620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:39:48 (6620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:34:25 (924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:28:43 (6176): No heartbeat from core client for 30 sec - exiting 08:28:44 (6176): No heartbeat from core client for 30 sec - exiting 08:28:45 (6176): No heartbeat from core client for 30 sec - exiting 08:28:46 (6176): No heartbeat from core client for 30 sec - exiting 08:28:47 (6176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:58:45 (9540): No heartbeat from core client for 30 sec - exiting 08:58:46 (9540): No heartbeat from core client for 30 sec - exiting 08:58:47 (9540): No heartbeat from core client for 30 sec - exiting 08:58:48 (9540): No heartbeat from core client for 30 sec - exiting 08:58:49 (9540): No heartbeat from core client for 30 sec - exiting 08:58:50 (9540): No heartbeat from core client for 30 sec - exiting 08:58:51 (9540): No heartbeat from core client for 30 sec - exiting 08:58:52 (9540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:01:47 (8076): No heartbeat from core client for 30 sec - exiting 09:01:48 (8076): No heartbeat from core client for 30 sec - exiting 09:01:49 (8076): No heartbeat from core client for 30 sec - exiting 09:01:50 (8076): No heartbeat from core client for 30 sec - exiting 09:01:51 (8076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:50 (9616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:55 (6968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:39:56 (3128): No heartbeat from core client for 30 sec - exiting 09:39:57 (3128): No heartbeat from core client for 30 sec - exiting 09:39:58 (3128): No heartbeat from core client for 30 sec - exiting 09:39:59 (3128): No heartbeat from core client for 30 sec - exiting 09:40:00 (3128): No heartbeat from core client for 30 sec - exiting 09:40:01 (3128): No heartbeat from core client for 30 sec - exiting 09:40:02 (3128): No heartbeat from core client for 30 sec - exiting 09:40:03 (3128): No heartbeat from core client for 30 sec - exiting 09:40:04 (3128): No heartbeat from core client for 30 sec - exiting 09:40:05 (3128): No heartbeat from core client for 30 sec - exiting 09:40:06 (3128): No heartbeat from core client for 30 sec - exiting 09:40:07 (3128): No heartbeat from core client for 30 sec - exiting 09:40:08 (3128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:52:25 (6324): No heartbeat from core client for 30 sec - exiting 09:52:26 (6324): No heartbeat from core client for 30 sec - exiting 09:52:27 (6324): No heartbeat from core client for 30 sec - exiting 09:52:28 (6324): No heartbeat from core client for 30 sec - exiting 09:52:29 (6324): No heartbeat from core client for 30 sec - exiting 09:52:30 (6324): No heartbeat from core client for 30 sec - exiting 09:52:31 (6324): No heartbeat from core client for 30 sec - exiting 09:52:32 (6324): No heartbeat from core client for 30 sec - exiting 09:52:33 (6324): No heartbeat from core client for 30 sec - exiting 09:52:34 (6324): No heartbeat from core client for 30 sec - exiting 09:52:35 (6324): No heartbeat from core client for 30 sec - exiting 09:52:36 (6324): No heartbeat from core client for 30 sec - exiting 09:52:37 (6324): No heartbeat from core client for 30 sec - exiting 09:52:38 (6324): No heartbeat from core client for 30 sec - exiting 09:52:39 (6324): No heartbeat from core client for 30 sec - exiting 09:52:40 (6324): No heartbeat from core client for 30 sec - exiting 09:52:41 (6324): No heartbeat from core client for 30 sec - exiting 09:52:42 (6324): No heartbeat from core client for 30 sec - exiting 09:52:43 (6324): No heartbeat from core client for 30 sec - exiting 09:52:44 (6324): No heartbeat from core client for 30 sec - exiting 09:52:45 (6324): No heartbeat from core client for 30 sec - exiting 09:52:46 (6324): No heartbeat from core client for 30 sec - exiting 09:52:47 (6324): No heartbeat from core client for 30 sec - exiting 09:52:48 (6324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:52:49 (6324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 10:17:19 (2796): No heartbeat from core client for 30 sec - exiting 10:17:20 (2796): No heartbeat from core client for 30 sec - exiting 10:17:21 (2796): No heartbeat from core client for 30 sec - exiting 10:17:22 (2796): No heartbeat from core client for 30 sec - exiting 10:17:23 (2796): No heartbeat from core client for 30 sec - exiting 10:17:24 (2796): No heartbeat from core client for 30 sec - exiting 10:17:25 (2796): No heartbeat from core client for 30 sec - exiting 10:17:26 (2796): No heartbeat from core client for 30 sec - exiting 10:17:27 (2796): No heartbeat from core client for 30 sec - exiting 10:17:28 (2796): No heartbeat from core client for 30 sec - exiting 10:17:29 (2796): No heartbeat from core client for 30 sec - exiting 10:17:30 (2796): No heartbeat from core client for 30 sec - exiting 10:17:31 (2796): No heartbeat from core client for 30 sec - exiting 10:17:32 (2796): No heartbeat from core client for 30 sec - exiting 10:17:33 (2796): No heartbeat from core client for 30 sec - exiting 10:17:34 (2796): No heartbeat from core client for 30 sec - exiting 10:17:35 (2796): No heartbeat from core client for 30 sec - exiting 10:17:36 (2796): No heartbeat from core client for 30 sec - exiting 10:17:37 (2796): No heartbeat from core client for 30 sec - exiting 10:17:38 (2796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:35:24 (8344): No heartbeat from core client for 30 sec - exiting 10:35:25 (8344): No heartbeat from core client for 30 sec - exiting 10:35:26 (8344): No heartbeat from core client for 30 sec - exiting 10:35:27 (8344): No heartbeat from core client for 30 sec - exiting 10:35:28 (8344): No heartbeat from core client for 30 sec - exiting 10:35:29 (8344): No heartbeat from core client for 30 sec - exiting 10:35:30 (8344): No heartbeat from core client for 30 sec - exiting 10:35:31 (8344): No heartbeat from core client for 30 sec - exiting 10:35:32 (8344): No heartbeat from core client for 30 sec - exiting 10:35:33 (8344): No heartbeat from core client for 30 sec - exiting 10:35:34 (8344): No heartbeat from core client for 30 sec - exiting 10:35:35 (8344): No heartbeat from core client for 30 sec - exiting 10:35:36 (8344): No heartbeat from core client for 30 sec - exiting 10:35:37 (8344): No heartbeat from core client for 30 sec - exiting 10:35:38 (8344): No heartbeat from core client for 30 sec - exiting 10:35:39 (8344): No heartbeat from core client for 30 sec - exiting 10:35:40 (8344): No heartbeat from core client for 30 sec - exiting 10:35:41 (8344): No heartbeat from core client for 30 sec - exiting 10:35:42 (8344): No heartbeat from core client for 30 sec - exiting 10:35:43 (8344): No heartbeat from core client for 30 sec - exiting 10:35:44 (8344): No heartbeat from core client for 30 sec - exiting 10:35:45 (8344): No heartbeat from core client for 30 sec - exiting 10:35:46 (8344): No heartbeat from core client for 30 sec - exiting 10:35:47 (8344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:05:04 (6636): No heartbeat from core client for 30 sec - exiting 11:05:05 (6636): No heartbeat from core client for 30 sec - exiting 11:05:06 (6636): No heartbeat from core client for 30 sec - exiting 11:05:07 (6636): No heartbeat from core client for 30 sec - exiting 11:05:08 (6636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:22:09 (4248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:45:01 (9580): No heartbeat from core client for 30 sec - exiting 11:45:02 (9580): No heartbeat from core client for 30 sec - exiting 11:45:03 (9580): No heartbeat from core client for 30 sec - exiting 11:45:04 (9580): No heartbeat from core client for 30 sec - exiting 11:45:05 (9580): No heartbeat from core client for 30 sec - exiting 11:45:06 (9580): No heartbeat from core client for 30 sec - exiting 11:45:07 (9580): No heartbeat from core client for 30 sec - exiting 11:45:08 (9580): No heartbeat from core client for 30 sec - exiting 11:45:09 (9580): No heartbeat from core client for 30 sec - exiting 11:45:10 (9580): No heartbeat from core client for 30 sec - exiting 11:45:11 (9580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:09:18 (6304): No heartbeat from core client for 30 sec - exiting 12:09:19 (6304): No heartbeat from core client for 30 sec - exiting 12:09:20 (6304): No heartbeat from core client for 30 sec - exiting 12:09:21 (6304): No heartbeat from core client for 30 sec - exiting 12:09:22 (6304): No heartbeat from core client for 30 sec - exiting 12:09:23 (6304): No heartbeat from core client for 30 sec - exiting 12:09:24 (6304): No heartbeat from core client for 30 sec - exiting 12:09:25 (6304): No heartbeat from core client for 30 sec - exiting 12:09:26 (6304): No heartbeat from core client for 30 sec - exiting 12:09:27 (6304): No heartbeat from core client for 30 sec - exiting 12:09:28 (6304): No heartbeat from core client for 30 sec - exiting 12:09:29 (6304): No heartbeat from core client for 30 sec - exiting 12:09:30 (6304): No heartbeat from core client for 30 sec - exiting 12:09:31 (6304): No heartbeat from core client for 30 sec - exiting 12:09:32 (6304): No heartbeat from core client for 30 sec - exiting 12:09:33 (6304): No heartbeat from core client for 30 sec - exiting 12:09:34 (6304): No heartbeat from core client for 30 sec - exiting 12:09:35 (6304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:34:31 (5132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:22:13 (4992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:47:53 (9172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... forrtl: There is not enough space on the disk. CPDN Monitor - Quit request from BOINC... 10:13:11 (3044): No heartbeat from core client for 30 sec - exiting 10:13:12 (3044): No heartbeat from core client for 30 sec - exiting 10:13:13 (3044): No heartbeat from core client for 30 sec - exiting 10:13:14 (3044): No heartbeat from core client for 30 sec - exiting 10:13:15 (3044): No heartbeat from core client for 30 sec - exiting 10:13:16 (3044): No heartbeat from core client for 30 sec - exiting 10:13:17 (3044): No heartbeat from core client for 30 sec - exiting 10:13:18 (3044): No heartbeat from core client for 30 sec - exiting 10:13:19 (3044): No heartbeat from core client for 30 sec - exiting 10:13:20 (3044): No heartbeat from core client for 30 sec - exiting 10:13:21 (3044): No heartbeat from core client for 30 sec - exiting 10:13:22 (3044): No heartbeat from core client for 30 sec - exiting 10:13:23 (3044): No heartbeat from core client for 30 sec - exiting 10:13:24 (3044): No heartbeat from core client for 30 sec - exiting 10:13:25 (3044): No heartbeat from core client for 30 sec - exiting 10:13:26 (3044): No heartbeat from core client for 30 sec - exiting 10:13:27 (3044): No heartbeat from core client for 30 sec - exiting 10:13:28 (3044): No heartbeat from core client for 30 sec - exiting 10:13:29 (3044): No heartbeat from core client for 30 sec - exiting 10:13:30 (3044): No heartbeat from core client for 30 sec - exiting 10:13:31 (3044): No heartbeat from core client for 30 sec - exiting 10:13:32 (3044): No heartbeat from core client for 30 sec - exiting 10:13:33 (3044): No heartbeat from core client for 30 sec - exiting 10:13:34 (3044): No heartbeat from core client for 30 sec - exiting 10:13:35 (3044): No heartbeat from core client for 30 sec - exiting 10:13:36 (3044): No heartbeat from core client for 30 sec - exiting 10:13:37 (3044): No heartbeat from core client for 30 sec - exiting 10:13:38 (3044): No heartbeat from core client for 30 sec - exiting 10:13:39 (3044): No heartbeat from core client for 30 sec - exiting 10:13:40 (3044): No heartbeat from core client for 30 sec - exiting 10:13:41 (3044): No heartbeat from core client for 30 sec - exiting 10:13:42 (3044): No heartbeat from core client for 30 sec - exiting 10:13:43 (3044): No heartbeat from core client for 30 sec - exiting 10:13:44 (3044): No heartbeat from core client for 30 sec - exiting 10:13:45 (3044): No heartbeat from core client for 30 sec - exiting 10:13:46 (3044): No heartbeat from core client for 30 sec - exiting 10:13:47 (3044): No heartbeat from core client for 30 sec - exiting 10:13:48 (3044): No heartbeat from core client for 30 sec - exiting 10:13:49 (3044): No heartbeat from core client for 30 sec - exiting 10:13:50 (3044): No heartbeat from core client for 30 sec - exiting 10:13:51 (3044): No heartbeat from core client for 30 sec - exiting 10:13:52 (3044): No heartbeat from core client for 30 sec - exiting 10:13:53 (3044): No heartbeat from core client for 30 sec - exiting 10:13:54 (3044): No heartbeat from core client for 30 sec - exiting 10:13:55 (3044): No heartbeat from core client for 30 sec - exiting 10:13:56 (3044): No heartbeat from core client for 30 sec - exiting 10:13:57 (3044): No heartbeat from core client for 30 sec - exiting 10:13:58 (3044): No heartbeat from core client for 30 sec - exiting 10:13:59 (3044): No heartbeat from core client for 30 sec - exiting 10:14:00 (3044): No heartbeat from core client for 30 sec - exiting 10:14:01 (3044): No heartbeat from core client for 30 sec - exiting 10:14:02 (3044): No heartbeat from core client for 30 sec - exiting 10:14:03 (3044): No heartbeat from core client for 30 sec - exiting 10:14:04 (3044): No heartbeat from core client for 30 sec - exiting 10:14:05 (3044): No heartbeat from core client for 30 sec - exiting 10:14:06 (3044): No heartbeat from core client for 30 sec - exiting 10:14:07 (3044): No heartbeat from core client for 30 sec - exiting 10:14:08 (3044): No heartbeat from core client for 30 sec - exiting 10:14:09 (3044): No heartbeat from core client for 30 sec - exiting 10:14:10 (3044): No heartbeat from core client for 30 sec - exiting 10:14:11 (3044): No heartbeat from core client for 30 sec - exiting 10:14:12 (3044): No heartbeat from core client for 30 sec - exiting 10:14:13 (3044): No heartbeat from core client for 30 sec - exiting 10:14:14 (3044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:17:35 (3296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:21:17 (4488): No heartbeat from core client for 30 sec - exiting 10:21:18 (4488): No heartbeat from core client for 30 sec - exiting 10:21:19 (4488): No heartbeat from core client for 30 sec - exiting 10:21:20 (4488): No heartbeat from core client for 30 sec - exiting 10:21:21 (4488): No heartbeat from core client for 30 sec - exiting 10:21:22 (4488): No heartbeat from core client for 30 sec - exiting 10:21:23 (4488): No heartbeat from core client for 30 sec - exiting 10:21:24 (4488): No heartbeat from core client for 30 sec - exiting 10:21:25 (4488): No heartbeat from core client for 30 sec - exiting 10:21:26 (4488): No heartbeat from core client for 30 sec - exiting 10:21:27 (4488): No heartbeat from core client for 30 sec - exiting 10:21:28 (4488): No heartbeat from core client for 30 sec - exiting 10:21:29 (4488): No heartbeat from core client for 30 sec - exiting 10:21:30 (4488): No heartbeat from core client for 30 sec - exiting 10:21:31 (4488): No heartbeat from core client for 30 sec - exiting 10:21:32 (4488): No heartbeat from core client for 30 sec - exiting 10:21:33 (4488): No heartbeat from core client for 30 sec - exiting 10:21:34 (4488): No heartbeat from core client for 30 sec - exiting 10:21:35 (4488): No heartbeat from core client for 30 sec - exiting 10:21:36 (4488): No heartbeat from core client for 30 sec - exiting 10:21:37 (4488): No heartbeat from core client for 30 sec - exiting 10:21:38 (4488): No heartbeat from core client for 30 sec - exiting 10:21:39 (4488): No heartbeat from core client for 30 sec - exiting 10:21:40 (4488): No heartbeat from core client for 30 sec - exiting 10:21:41 (4488): No heartbeat from core client for 30 sec - exiting 10:21:42 (4488): No heartbeat from core client for 30 sec - exiting 10:21:43 (4488): No heartbeat from core client for 30 sec - exiting 10:21:44 (4488): No heartbeat from core client for 30 sec - exiting 10:21:45 (4488): No heartbeat from core client for 30 sec - exiting 10:21:46 (4488): No heartbeat from core client for 30 sec - exiting 10:21:47 (4488): No heartbeat from core client for 30 sec - exiting 10:21:48 (4488): No heartbeat from core client for 30 sec - exiting 10:21:49 (4488): No heartbeat from core client for 30 sec - exiting 10:21:50 (4488): No heartbeat from core client for 30 sec - exiting 10:21:51 (4488): No heartbeat from core client for 30 sec - exiting 10:21:52 (4488): No heartbeat from core client for 30 sec - exiting 10:21:53 (4488): No heartbeat from core client for 30 sec - exiting 10:21:54 (4488): No heartbeat from core client for 30 sec - exiting 10:21:55 (4488): No heartbeat from core client for 30 sec - exiting 10:21:56 (4488): No heartbeat from core client for 30 sec - exiting 10:21:57 (4488): No heartbeat from core client for 30 sec - exiting 10:21:58 (4488): No heartbeat from core client for 30 sec - exiting 10:21:59 (4488): No heartbeat from core client for 30 sec - exiting 10:22:00 (4488): No heartbeat from core client for 30 sec - exiting 10:22:01 (4488): No heartbeat from core client for 30 sec - exiting 10:22:02 (4488): No heartbeat from core client for 30 sec - exiting 10:22:03 (4488): No heartbeat from core client for 30 sec - exiting 10:22:04 (4488): No heartbeat from core client for 30 sec - exiting 10:22:05 (4488): No heartbeat from core client for 30 sec - exiting 10:22:06 (4488): No heartbeat from core client for 30 sec - exiting 10:22:07 (4488): No heartbeat from core client for 30 sec - exiting 10:22:08 (4488): No heartbeat from core client for 30 sec - exiting 10:22:09 (4488): No heartbeat from core client for 30 sec - exiting 10:22:10 (4488): No heartbeat from core client for 30 sec - exiting 10:22:11 (4488): No heartbeat from core client for 30 sec - exiting 10:22:12 (4488): No heartbeat from core client for 30 sec - exiting 10:22:13 (4488): No heartbeat from core client for 30 sec - exiting 10:22:14 (4488): No heartbeat from core client for 30 sec - exiting 10:22:15 (4488): No heartbeat from core client for 30 sec - exiting 10:22:16 (4488): No heartbeat from core client for 30 sec - exiting 10:22:17 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. 10:39:07 (4896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Sep 2011 15:57:26	1156833	13382585	hadcm3n_o3dx_1940_40_007454794_4	51,840	90,906	1.7536
13 Sep 2011 17:42:54	1156833	13382585	hadcm3n_o3dx_1940_40_007454794_4	25,920	46,422	1.7910