Task 12819167

Name	hadcm3n_p1kn_1900_40_007219623_0
Workunit	7417863
Created	26 Apr 2011, 15:18:04 UTC
Sent	3 May 2011, 19:20:02 UTC
Report deadline	3 Aug 2011, 2:47:13 UTC
Received	23 May 2011, 3:31:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1144509
Run time	10 days 16 hours 0 min 52 sec
CPU time	8 days 23 hours 42 min 14 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	2.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:37:57 (3800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:56:42 (1036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:46:28 (1460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:46:29 (1460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=696, selfPID=696, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:30:56 (2556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:04:17 (3936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:57:31 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:09:00 (3204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:48 (2688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:11:14 (2600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:35:34 (2336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:19:46 (2408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:48:59 (708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:49:01 (708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 15:54:27 (2076): No heartbeat from core client for 30 sec - exiting 15:54:28 (2076): No heartbeat from core client for 30 sec - exiting 15:54:29 (2076): No heartbeat from core client for 30 sec - exiting 15:54:30 (2076): No heartbeat from core client for 30 sec - exiting 15:54:31 (2076): No heartbeat from core client for 30 sec - exiting 15:54:32 (2076): No heartbeat from core client for 30 sec - exiting 15:54:34 (2076): No heartbeat from core client for 30 sec - exiting 15:54:35 (2076): No heartbeat from core client for 30 sec - exiting 15:54:36 (2076): No heartbeat from core client for 30 sec - exiting 15:54:37 (2076): No heartbeat from core client for 30 sec - exiting 15:54:38 (2076): No heartbeat from core client for 30 sec - exiting 15:54:39 (2076): No heartbeat from core client for 30 sec - exiting 15:54:40 (2076): No heartbeat from core client for 30 sec - exiting 15:54:41 (2076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:26:46 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:53:14 (2816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:09:23 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:23:12 (2368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:45:29 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:21:14 (1692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:55:29 (3168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:33:23 (2476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:38:17 (2516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:39:09 (2780): No heartbeat from core client for 30 sec - exiting 18:39:10 (2780): No heartbeat from core client for 30 sec - exiting 18:39:11 (2780): No heartbeat from core client for 30 sec - exiting 18:39:12 (2780): No heartbeat from core client for 30 sec - exiting 18:39:13 (2780): No heartbeat from core client for 30 sec - exiting 18:39:14 (2780): No heartbeat from core client for 30 sec - exiting 18:39:15 (2780): No heartbeat from core client for 30 sec - exiting 18:39:16 (2780): No heartbeat from core client for 30 sec - exiting 18:39:17 (2780): No heartbeat from core client for 30 sec - exiting 18:39:18 (2780): No heartbeat from core client for 30 sec - exiting 18:39:19 (2780): No heartbeat from core client for 30 sec - exiting 18:39:20 (2780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:39:21 (2780): No heartbeat from core client for 30 sec - exiting 18:39:22 (2780): No heartbeat from core client for 30 sec - exiting 18:39:23 (2780): No heartbeat from core client for 30 sec - exiting 18:44:17 (2632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:45:14 (3200): No heartbeat from core client for 30 sec - exiting 18:45:15 (3200): No heartbeat from core client for 30 sec - exiting 18:45:16 (3200): No heartbeat from core client for 30 sec - exiting 18:45:17 (3200): No heartbeat from core client for 30 sec - exiting 18:45:18 (3200): No heartbeat from core client for 30 sec - exiting 18:45:19 (3200): No heartbeat from core client for 30 sec - exiting 18:45:20 (3200): No heartbeat from core client for 30 sec - exiting 18:45:21 (3200): No heartbeat from core client for 30 sec - exiting 18:45:22 (3200): No heartbeat from core client for 30 sec - exiting 18:45:23 (3200): No heartbeat from core client for 30 sec - exiting 18:45:24 (3200): No heartbeat from core client for 30 sec - exiting 18:45:25 (3200): No heartbeat from core client for 30 sec - exiting 18:45:26 (3200): No heartbeat from core client for 30 sec - exiting 18:45:27 (3200): No heartbeat from core client for 30 sec - exiting 18:45:28 (3200): No heartbeat from core client for 30 sec - exiting 18:45:29 (3200): No heartbeat from core client for 30 sec - exiting 18:45:30 (3200): No heartbeat from core client for 30 sec - exiting 18:45:31 (3200): No heartbeat from core client for 30 sec - exiting 18:45:32 (3200): No heartbeat from core client for 30 sec - exiting 18:45:33 (3200): No heartbeat from core client for 30 sec - exiting 18:45:34 (3200): No heartbeat from core client for 30 sec - exiting 18:45:35 (3200): No heartbeat from core client for 30 sec - exiting 18:45:36 (3200): No heartbeat from core client for 30 sec - exiting 18:45:37 (3200): No heartbeat from core client for 30 sec - exiting 18:45:38 (3200): No heartbeat from core client for 30 sec - exiting 18:45:39 (3200): No heartbeat from core client for 30 sec - exiting 18:45:40 (3200): No heartbeat from core client for 30 sec - exiting 18:45:41 (3200): No heartbeat from core client for 30 sec - exiting 18:45:42 (3200): No heartbeat from core client for 30 sec - exiting 18:45:43 (3200): No heartbeat from core client for 30 sec - exiting 18:45:44 (3200): No heartbeat from core client for 30 sec - exiting 18:45:45 (3200): No heartbeat from core client for 30 sec - exiting 18:45:46 (3200): No heartbeat from core client for 30 sec - exiting 18:45:47 (3200): No heartbeat from core client for 30 sec - exiting 18:45:48 (3200): No heartbeat from core client for 30 sec - exiting 18:45:50 (3200): No heartbeat from core client for 30 sec - exiting 18:45:51 (3200): No heartbeat from core client for 30 sec - exiting 18:45:52 (3200): No heartbeat from core client for 30 sec - exiting 18:45:53 (3200): No heartbeat from core client for 30 sec - exiting 18:45:54 (3200): No heartbeat from core client for 30 sec - exiting 18:45:55 (3200): No heartbeat from core client for 30 sec - exiting 18:45:56 (3200): No heartbeat from core client for 30 sec - exiting 18:45:57 (3200): No heartbeat from core client for 30 sec - exiting 18:45:58 (3200): No heartbeat from core client for 30 sec - exiting 18:45:59 (3200): No heartbeat from core client for 30 sec - exiting 18:46:00 (3200): No heartbeat from core client for 30 sec - exiting 18:46:01 (3200): No heartbeat from core client for 30 sec - exiting 18:46:02 (3200): No heartbeat from core client for 30 sec - exiting 18:46:03 (3200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:35:36 (2520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:29:34 (2108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:29:36 (2108): No heartbeat from core client for 30 sec - exiting 08:31:00 (2300): No heartbeat from core client for 30 sec - exiting 08:31:01 (2300): No heartbeat from core client for 30 sec - exiting 08:31:02 (2300): No heartbeat from core client for 30 sec - exiting 08:31:03 (2300): No heartbeat from core client for 30 sec - exiting 08:31:04 (2300): No heartbeat from core client for 30 sec - exiting 08:31:05 (2300): No heartbeat from core client for 30 sec - exiting 08:31:06 (2300): No heartbeat from core client for 30 sec - exiting 08:31:07 (2300): No heartbeat from core client for 30 sec - exiting 08:31:08 (2300): No heartbeat from core client for 30 sec - exiting 08:31:09 (2300): No heartbeat from core client for 30 sec - exiting 08:31:10 (2300): No heartbeat from core client for 30 sec - exiting 08:31:11 (2300): No heartbeat from core client for 30 sec - exiting 08:31:12 (2300): No heartbeat from core client for 30 sec - exiting 08:31:13 (2300): No heartbeat from core client for 30 sec - exiting 08:31:14 (2300): No heartbeat from core client for 30 sec - exiting 08:31:15 (2300): No heartbeat from core client for 30 sec - exiting 08:31:16 (2300): No heartbeat from core client for 30 sec - exiting 08:31:17 (2300): No heartbeat from core client for 30 sec - exiting 08:31:18 (2300): No heartbeat from core client for 30 sec - exiting 08:31:19 (2300): No heartbeat from core client for 30 sec - exiting 08:31:20 (2300): No heartbeat from core client for 30 sec - exiting 08:31:21 (2300): No heartbeat from core client for 30 sec - exiting 08:31:22 (2300): No heartbeat from core client for 30 sec - exiting 08:31:23 (2300): No heartbeat from core client for 30 sec - exiting 08:31:24 (2300): No heartbeat from core client for 30 sec - exiting 08:31:25 (2300): No heartbeat from core client for 30 sec - exiting 08:31:26 (2300): No heartbeat from core client for 30 sec - exiting 08:31:27 (2300): No heartbeat from core client for 30 sec - exiting 08:31:28 (2300): No heartbeat from core client for 30 sec - exiting 08:31:29 (2300): No heartbeat from core client for 30 sec - exiting 08:31:30 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:53 (996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:55 (996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:55:20 (2496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:51:06 (2468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:51:07 (2468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:43:53 (2576): No heartbeat from core client for 30 sec - exiting 08:43:54 (2576): No heartbeat from core client for 30 sec - exiting 08:43:56 (2576): No heartbeat from core client for 30 sec - exiting 08:43:57 (2576): No heartbeat from core client for 30 sec - exiting 08:43:58 (2576): No heartbeat from core client for 30 sec - exiting 08:43:59 (2576): No heartbeat from core client for 30 sec - exiting 08:44:00 (2576): No heartbeat from core client for 30 sec - exiting 08:44:01 (2576): No heartbeat from core client for 30 sec - exiting 08:44:02 (2576): No heartbeat from core client for 30 sec - exiting 08:44:03 (2576): No heartbeat from core client for 30 sec - exiting 08:44:04 (2576): No heartbeat from core client for 30 sec - exiting 08:44:05 (2576): No heartbeat from core client for 30 sec - exiting 08:44:07 (2576): No heartbeat from core client for 30 sec - exiting 08:44:08 (2576): No heartbeat from core client for 30 sec - exiting 08:44:09 (2576): No heartbeat from core client for 30 sec - exiting 08:44:10 (2576): No heartbeat from core client for 30 sec - exiting 08:44:11 (2576): No heartbeat from core client for 30 sec - exiting 08:44:12 (2576): No heartbeat from core client for 30 sec - exiting 08:44:13 (2576): No heartbeat from core client for 30 sec - exiting 08:44:14 (2576): No heartbeat from core client for 30 sec - exiting 08:44:15 (2576): No heartbeat from core client for 30 sec - exiting 08:44:16 (2576): No heartbeat from core client for 30 sec - exiting 08:44:17 (2576): No heartbeat from core client for 30 sec - exiting 08:44:19 (2576): No heartbeat from core client for 30 sec - exiting 08:44:20 (2576): No heartbeat from core client for 30 sec - exiting 08:44:21 (2576): No heartbeat from core client for 30 sec - exiting 08:44:22 (2576): No heartbeat from core client for 30 sec - exiting 08:44:23 (2576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:17:20 (3676): No heartbeat from core client for 30 sec - exiting 09:17:21 (3676): No heartbeat from core client for 30 sec - exiting 09:17:23 (3676): No heartbeat from core client for 30 sec - exiting 09:17:24 (3676): No heartbeat from core client for 30 sec - exiting 09:17:25 (3676): No heartbeat from core client for 30 sec - exiting 09:17:26 (3676): No heartbeat from core client for 30 sec - exiting 09:17:27 (3676): No heartbeat from core client for 30 sec - exiting 09:17:28 (3676): No heartbeat from core client for 30 sec - exiting 09:17:29 (3676): No heartbeat from core client for 30 sec - exiting 09:17:30 (3676): No heartbeat from core client for 30 sec - exiting 09:17:31 (3676): No heartbeat from core client for 30 sec - exiting 09:17:32 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:21:39 (396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:46:25 (3504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:46:27 (3504): No heartbeat from core client for 30 sec - exiting 10:46:28 (3504): No heartbeat from core client for 30 sec - exiting 13:13:09 (2632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... 16:43:52 (1440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:19:21 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:21:29 (492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:18 (1628): No heartbeat from core client for 30 sec - exiting 05:22:19 (1628): No heartbeat from core client for 30 sec - exiting 05:22:20 (1628): No heartbeat from core client for 30 sec - exiting 05:22:21 (1628): No heartbeat from core client for 30 sec - exiting 05:22:22 (1628): No heartbeat from core client for 30 sec - exiting 05:22:23 (1628): No heartbeat from core client for 30 sec - exiting 05:22:24 (1628): No heartbeat from core client for 30 sec - exiting 05:22:25 (1628): No heartbeat from core client for 30 sec - exiting 05:22:26 (1628): No heartbeat from core client for 30 sec - exiting 05:22:27 (1628): No heartbeat from core client for 30 sec - exiting 05:22:28 (1628): No heartbeat from core client for 30 sec - exiting 05:22:29 (1628): No heartbeat from core client for 30 sec - exiting 05:22:30 (1628): No heartbeat from core client for 30 sec - exiting 05:22:31 (1628): No heartbeat from core client for 30 sec - exiting 05:22:32 (1628): No heartbeat from core client for 30 sec - exiting 05:22:33 (1628): No heartbeat from core client for 30 sec - exiting 05:22:34 (1628): No heartbeat from core client for 30 sec - exiting 05:22:35 (1628): No heartbeat from core client for 30 sec - exiting 05:22:36 (1628): No heartbeat from core client for 30 sec - exiting 05:22:37 (1628): No heartbeat from core client for 30 sec - exiting 05:22:38 (1628): No heartbeat from core client for 30 sec - exiting 05:22:39 (1628): No heartbeat from core client for 30 sec - exiting 05:22:40 (1628): No heartbeat from core client for 30 sec - exiting 05:22:41 (1628): No heartbeat from core client for 30 sec - exiting 05:22:42 (1628): No heartbeat from core client for 30 sec - exiting 05:22:43 (1628): No heartbeat from core client for 30 sec - exiting 05:22:44 (1628): No heartbeat from core client for 30 sec - exiting 05:22:45 (1628): No heartbeat from core client for 30 sec - exiting 05:22:46 (1628): No heartbeat from core client for 30 sec - exiting 05:22:47 (1628): No heartbeat from core client for 30 sec - exiting 05:22:48 (1628): No heartbeat from core client for 30 sec - exiting 05:22:49 (1628): No heartbeat from core client for 30 sec - exiting 05:22:50 (1628): No heartbeat from core client for 30 sec - exiting 05:22:51 (1628): No heartbeat from core client for 30 sec - exiting 05:22:52 (1628): No heartbeat from core client for 30 sec - exiting 05:22:53 (1628): No heartbeat from core client for 30 sec - exiting 05:22:54 (1628): No heartbeat from core client for 30 sec - exiting 05:22:55 (1628): No heartbeat from core client for 30 sec - exiting 05:22:56 (1628): No heartbeat from core client for 30 sec - exiting 05:22:57 (1628): No heartbeat from core client for 30 sec - exiting 05:22:58 (1628): No heartbeat from core client for 30 sec - exiting 05:22:59 (1628): No heartbeat from core client for 30 sec - exiting 05:23:00 (1628): No heartbeat from core client for 30 sec - exiting 05:23:01 (1628): No heartbeat from core client for 30 sec - exiting 05:23:02 (1628): No heartbeat from core client for 30 sec - exiting 05:23:03 (1628): No heartbeat from core client for 30 sec - exiting 05:23:04 (1628): No heartbeat from core client for 30 sec - exiting 05:23:05 (1628): No heartbeat from core client for 30 sec - exiting 05:23:06 (1628): No heartbeat from core client for 30 sec - exiting 05:23:07 (1628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:23:08 (1628): No heartbeat from core client for 30 sec - exiting 05:23:09 (1628): No heartbeat from core client for 30 sec - exiting 05:23:10 (1628): No heartbeat from core client for 30 sec - exiting 10:22:05 (2200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:23:07 (220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:50:09 (4040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:54:56 (2528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:55:04 (1724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:56:15 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:28:44 (2868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:51:22 (4020): No heartbeat from core client for 30 sec - exiting 10:51:24 (4020): No heartbeat from core client for 30 sec - exiting 10:51:25 (4020): No heartbeat from core client for 30 sec - exiting 10:51:26 (4020): No heartbeat from core client for 30 sec - exiting 10:51:27 (4020): No heartbeat from core client for 30 sec - exiting 10:51:28 (4020): No heartbeat from core client for 30 sec - exiting 10:51:29 (4020): No heartbeat from core client for 30 sec - exiting 10:51:30 (4020): No heartbeat from core client for 30 sec - exiting 10:51:31 (4020): No heartbeat from core client for 30 sec - exiting 10:51:32 (4020): No heartbeat from core client for 30 sec - exiting 10:51:33 (4020): No heartbeat from core client for 30 sec - exiting 10:51:34 (4020): No heartbeat from core client for 30 sec - exiting 10:51:35 (4020): No heartbeat from core client for 30 sec - exiting 10:51:36 (4020): No heartbeat from core client for 30 sec - exiting 10:51:37 (4020): No heartbeat from core client for 30 sec - exiting 10:51:38 (4020): No heartbeat from core client for 30 sec - exiting 10:51:39 (4020): No heartbeat from core client for 30 sec - exiting 10:51:40 (4020): No heartbeat from core client for 30 sec - exiting 10:51:41 (4020): No heartbeat from core client for 30 sec - exiting 10:51:42 (4020): No heartbeat from core client for 30 sec - exiting 10:51:43 (4020): No heartbeat from core client for 30 sec - exiting 10:51:44 (4020): No heartbeat from core client for 30 sec - exiting 10:51:45 (4020): No heartbeat from core client for 30 sec - exiting 10:51:46 (4020): No heartbeat from core client for 30 sec - exiting 10:51:47 (4020): No heartbeat from core client for 30 sec - exiting 10:51:48 (4020): No heartbeat from core client for 30 sec - exiting 10:51:49 (4020): No heartbeat from core client for 30 sec - exiting 10:51:50 (4020): No heartbeat from core client for 30 sec - exiting 10:51:51 (4020): No heartbeat from core client for 30 sec - exiting 10:51:52 (4020): No heartbeat from core client for 30 sec - exiting 10:51:53 (4020): No heartbeat from core client for 30 sec - exiting 10:51:54 (4020): No heartbeat from core client for 30 sec - exiting 10:51:55 (4020): No heartbeat from core client for 30 sec - exiting 10:51:56 (4020): No heartbeat from core client for 30 sec - exiting 10:51:57 (4020): No heartbeat from core client for 30 sec - exiting 10:51:58 (4020): No heartbeat from core client for 30 sec - exiting 10:51:59 (4020): No heartbeat from core client for 30 sec - exiting 10:52:00 (4020): No heartbeat from core client for 30 sec - exiting 10:52:01 (4020): No heartbeat from core client for 30 sec - exiting 10:52:02 (4020): No heartbeat from core client for 30 sec - exiting 10:52:03 (4020): No heartbeat from core client for 30 sec - exiting 10:52:04 (4020): No heartbeat from core client for 30 sec - exiting 10:52:05 (4020): No heartbeat from core client for 30 sec - exiting 10:52:06 (4020): No heartbeat from core client for 30 sec - exiting 10:52:07 (4020): No heartbeat from core client for 30 sec - exiting 10:52:08 (4020): No heartbeat from core client for 30 sec - exiting 10:52:09 (4020): No heartbeat from core client for 30 sec - exiting 10:52:10 (4020): No heartbeat from core client for 30 sec - exiting 10:52:11 (4020): No heartbeat from core client for 30 sec - exiting 10:52:12 (4020): No heartbeat from core client for 30 sec - exiting 10:52:13 (4020): No heartbeat from core client for 30 sec - exiting 10:52:14 (4020): No heartbeat from core client for 30 sec - exiting 10:52:15 (4020): No heartbeat from core client for 30 sec - exiting 10:52:16 (4020): No heartbeat from core client for 30 sec - exiting 10:52:17 (4020): No heartbeat from core client for 30 sec - exiting 10:52:18 (4020): No heartbeat from core client for 30 sec - exiting 10:52:19 (4020): No heartbeat from core client for 30 sec - exiting 10:52:20 (4020): No heartbeat from core client for 30 sec - exiting 10:52:21 (4020): No heartbeat from core client for 30 sec - exiting 10:52:22 (4020): No heartbeat from core client for 30 sec - exiting 10:52:23 (4020): No heartbeat from core client for 30 sec - exiting 10:52:24 (4020): No heartbeat from core client for 30 sec - exiting 10:52:25 (4020): No heartbeat from core client for 30 sec - exiting 10:52:26 (4020): No heartbeat from core client for 30 sec - exiting 10:52:27 (4020): No heartbeat from core client for 30 sec - exiting 10:52:28 (4020): No heartbeat from core client for 30 sec - exiting 10:52:29 (4020): No heartbeat from core client for 30 sec - exiting 10:52:30 (4020): No heartbeat from core client for 30 sec - exiting 10:52:31 (4020): No heartbeat from core client for 30 sec - exiting 10:52:32 (4020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:43:15 (1004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:44:10 (1272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:45:47 (2108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:02:21 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:19:44 (3148): No heartbeat from core client for 30 sec - exiting 19:19:45 (3148): No heartbeat from core client for 30 sec - exiting 19:19:46 (3148): No heartbeat from core client for 30 sec - exiting 19:19:47 (3148): No heartbeat from core client for 30 sec - exiting 19:19:48 (3148): No heartbeat from core client for 30 sec - exiting 19:19:49 (3148): No heartbeat from core client for 30 sec - exiting 19:19:50 (3148): No heartbeat from core client for 30 sec - exiting 19:19:52 (3148): No heartbeat from core client for 30 sec - exiting 19:19:53 (3148): No heartbeat from core client for 30 sec - exiting 19:19:54 (3148): No heartbeat from core client for 30 sec - exiting 19:19:55 (3148): No heartbeat from core client for 30 sec - exiting 19:19:56 (3148): No heartbeat from core client for 30 sec - exiting 19:19:57 (3148): No heartbeat from core client for 30 sec - exiting 19:19:58 (3148): No heartbeat from core client for 30 sec - exiting 19:19:59 (3148): No heartbeat from core client for 30 sec - exiting 19:20:00 (3148): No heartbeat from core client for 30 sec - exiting 19:20:01 (3148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:55 (2000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:13:13 (3600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:39 (3600): No heartbeat from core client for 30 sec - exiting 20:14:40 (3600): No heartbeat from core client for 30 sec - exiting 20:14:41 (3600): No heartbeat from core client for 30 sec - exiting 20:14:42 (3600): No heartbeat from core client for 30 sec - exiting 20:14:43 (3600): No heartbeat from core client for 30 sec - exiting 20:14:44 (3600): No heartbeat from core client for 30 sec - exiting 20:14:45 (3600): No heartbeat from core client for 30 sec - exiting 20:14:46 (3600): No heartbeat from core client for 30 sec - exiting 20:14:47 (3600): No heartbeat from core client for 30 sec - exiting 20:14:48 (3600): No heartbeat from core client for 30 sec - exiting 20:16:21 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:18:38 (3612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:26:37 (3168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:25:40 (2100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 Model crashed: SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 Model crashed: SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 Model crashed: SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 Model crashed: SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 Model crashed: SETPOS: Unit 21 to Word Address -1076373002 Failed with Error Code -1 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 May 2011 20:52:22	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	414,720	769,304	1.8550
21 May 2011 17:13:12	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	388,800	721,705	1.8562
20 May 2011 19:55:32	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	362,880	674,014	1.8574
19 May 2011 06:16:34	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	336,960	626,943	1.8606
18 May 2011 04:54:45	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	311,040	579,233	1.8622
16 May 2011 20:58:51	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	285,120	530,836	1.8618
15 May 2011 21:46:32	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	259,200	483,061	1.8637
15 May 2011 00:57:06	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	233,280	433,655	1.8589
14 May 2011 05:27:13	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	207,360	385,833	1.8607
12 May 2011 12:00:17	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	181,440	338,474	1.8655
11 May 2011 12:12:41	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	155,520	291,001	1.8711
10 May 2011 15:46:48	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	129,600	242,511	1.8712
09 May 2011 19:01:17	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	103,680	194,079	1.8719
06 May 2011 22:24:29	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	77,760	145,511	1.8713
06 May 2011 00:47:51	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	51,840	98,122	1.8928
05 May 2011 03:26:28	1144509	12819167	hadcm3n_p1kn_1900_40_007219623_0	25,920	48,689	1.8784