Task 14798189

Name	hadam3p_pnw_aqll_1977_1_008001450_0
Workunit	8156564
Created	19 Jun 2012, 11:53:14 UTC
Sent	19 Jun 2012, 11:53:18 UTC
Report deadline	1 Jun 2013, 17:13:18 UTC
Received	23 Sep 2012, 6:02:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1129423
Run time	2 days 16 hours 24 min 46 sec
CPU time	2 days 13 hours 54 min 14 sec
Validate state	Invalid
Credit	1,503.98
Device peak FLOPS	2.69 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... ReCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5852, selfPID=5852, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5988, selfPID=5988, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6236, selfPID=6236, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:37:28 (6152): No heartbeat from core client for 30 sec - exiting 23:37:29 (6152): No heartbeat from core client for 30 sec - exiting 23:37:31 (6152): No heartbeat from core client for 30 sec - exiting 23:37:32 (6152): No heartbeat from core client for 30 sec - exiting 23:37:33 (6152): No heartbeat from core client for 30 sec - exiting 23:37:34 (6152): No heartbeat from core client for 30 sec - exiting 23:37:35 (6152): No heartbeat from core client for 30 sec - exiting 23:37:36 (6152): No heartbeat from core client for 30 sec - exiting 23:37:37 (6152): No heartbeat from core client for 30 sec - exiting 23:37:38 (6152): No heartbeat from core client for 30 sec - exiting 23:37:39 (6152): No heartbeat from core client for 30 sec - exiting 23:37:40 (6152): No heartbeat from core client for 30 sec - exiting 23:37:41 (6152): No heartbeat from core client for 30 sec - exiting 23:37:43 (6152): No heartbeat from core client for 30 sec - exiting 23:37:44 (6152): No heartbeat from core client for 30 sec - exiting 23:37:45 (6152): No heartbeat from core client for 30 sec - exiting 23:37:46 (6152): No heartbeat from core client for 30 sec - exiting 23:37:47 (6152): No heartbeat from core client for 30 sec - exiting 23:37:48 (6152): No heartbeat from core client for 30 sec - exiting 23:37:49 (6152): No heartbeat from core client for 30 sec - exiting 23:37:50 (6152): No heartbeat from core client for 30 sec - exiting 23:37:51 (6152): No heartbeat from core client for 30 sec - exiting 23:37:53 (6152): No heartbeat from core client for 30 sec - exiting 23:37:54 (6152): No heartbeat from core client for 30 sec - exiting 23:37:55 (6152): No heartbeat from core client for 30 sec - exiting 23:37:56 (6152): No heartbeat from core client for 30 sec - exiting 23:37:57 (6152): No heartbeat from core client for 30 sec - exiting 23:37:58 (6152): No heartbeat from core client for 30 sec - exiting 23:37:59 (6152): No heartbeat from core client for 30 sec - exiting 23:38:00 (6152): No heartbeat from core client for 30 sec - exiting 23:38:01 (6152): No heartbeat from core client for 30 sec - exiting 23:38:02 (6152): No heartbeat from core client for 30 sec - exiting 23:38:36 (6152): No heartbeat from core client for 30 sec - exiting 23:38:37 (6152): No heartbeat from core client for 30 sec - exiting 23:38:38 (6152): No heartbeat from core client for 30 sec - exiting 23:38:40 (6152): No heartbeat from core client for 30 sec - exiting 23:38:41 (6152): No heartbeat from core client for 30 sec - exiting 23:38:42 (6152): No heartbeat from core client for 30 sec - exiting 23:38:43 (6152): No heartbeat from core client for 30 sec - exiting 23:38:44 (6152): No heartbeat from core client for 30 sec - exiting 23:38:45 (6152): No heartbeat from core client for 30 sec - exiting 23:38:46 (6152): No heartbeat from core client for 30 sec - exiting 23:38:47 (6152): No heartbeat from core client for 30 sec - exiting 23:38:48 (6152): No heartbeat from core client for 30 sec - exiting 23:38:49 (6152): No heartbeat from core client for 30 sec - exiting 23:38:50 (6152): No heartbeat from core client for 30 sec - exiting 23:38:52 (6152): No heartbeat from core client for 30 sec - exiting 23:38:53 (6152): No heartbeat from core client for 30 sec - exiting 23:38:54 (6152): No heartbeat from core client for 30 sec - exiting 23:38:55 (6152): No heartbeat from core client for 30 sec - exiting 23:38:56 (6152): No heartbeat from core client for 30 sec - exiting 23:38:57 (6152): No heartbeat from core client for 30 sec - exiting 23:38:58 (6152): No heartbeat from core client for 30 sec - exiting 23:38:59 (6152): No heartbeat from core client for 30 sec - exiting 23:39:00 (6152): No heartbeat from core client for 30 sec - exiting 23:39:01 (6152): No heartbeat from core client for 30 sec - exiting 23:39:02 (6152): No heartbeat from core client for 30 sec - exiting 23:39:04 (6152): No heartbeat from core client for 30 sec - exiting 23:39:05 (6152): No heartbeat from core client for 30 sec - exiting 23:39:06 (6152): No heartbeat from core client for 30 sec - exiting 23:39:07 (6152): No heartbeat from core client for 30 sec - exiting 23:39:08 (6152): No heartbeat from core client for 30 sec - exiting 23:39:09 (6152): No heartbeat from core client for 30 sec - exiting 23:39:10 (6152): No heartbeat from core client for 30 sec - exiting 23:39:11 (6152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7792, selfPID=7792, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:54:14 (5236): No heartbeat from core client for 30 sec - exiting 12:54:16 (5236): No heartbeat from core client for 30 sec - exiting 12:54:17 (5236): No heartbeat from core client for 30 sec - exiting 12:54:18 (5236): No heartbeat from core client for 30 sec - exiting 12:54:19 (5236): No heartbeat from core client for 30 sec - exiting 12:54:20 (5236): No heartbeat from core client for 30 sec - exiting 12:54:21 (5236): No heartbeat from core client for 30 sec - exiting 12:54:22 (5236): No heartbeat from core client for 30 sec - exiting 12:54:23 (5236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6432, selfPID=7160, iMonCtr=1 Model crash detected, will try to restart... 13:07:48 (6048): No heartbeat from core client for 30 sec - exiting 13:07:49 (6048): No heartbeat from core client for 30 sec - exiting 13:07:50 (6048): No heartbeat from core client for 30 sec - exiting 13:07:51 (6048): No heartbeat from core client for 30 sec - exiting 13:07:52 (6048): No heartbeat from core client for 30 sec - exiting 13:07:53 (6048): No heartbeat from core client for 30 sec - exiting 13:07:54 (6048): No heartbeat from core client for 30 sec - exiting 13:07:55 (6048): No heartbeat from core client for 30 sec - exiting 13:07:56 (6048): No heartbeat from core client for 30 sec - exiting 13:07:57 (6048): No heartbeat from core client for 30 sec - exiting 13:07:58 (6048): No heartbeat from core client for 30 sec - exiting 13:07:59 (6048): No heartbeat from core client for 30 sec - exiting 13:08:00 (6048): No heartbeat from core client for 30 sec - exiting 13:08:01 (6048): No heartbeat from core client for 30 sec - exiting 13:08:02 (6048): No heartbeat from core client for 30 sec - exiting 13:08:03 (6048): No heartbeat from core client for 30 sec - exiting 13:08:04 (6048): No heartbeat from core client for 30 sec - exiting 13:08:05 (6048): No heartbeat from core client for 30 sec - exiting 13:08:06 (6048): No heartbeat from core client for 30 sec - exiting 13:08:07 (6048): No heartbeat from core client for 30 sec - exiting 13:08:08 (6048): No heartbeat from core client for 30 sec - exiting 13:08:09 (6048): No heartbeat from core client for 30 sec - exiting 13:08:10 (6048): No heartbeat from core client for 30 sec - exiting 13:08:11 (6048): No heartbeat from core client for 30 sec - exiting 13:08:12 (6048): No heartbeat from core client for 30 sec - exiting 13:08:13 (6048): No heartbeat from core client for 30 sec - exiting 13:08:14 (6048): No heartbeat from core client for 30 sec - exiting 13:08:15 (6048): No heartbeat from core client for 30 sec - exiting 13:08:16 (6048): No heartbeat from core client for 30 sec - exiting 13:08:17 (6048): No heartbeat from core client for 30 sec - exiting 13:08:18 (6048): No heartbeat from core client for 30 sec - exiting 13:08:19 (6048): No heartbeat from core client for 30 sec - exiting 13:08:20 (6048): No heartbeat from core client for 30 sec - exiting 13:08:21 (6048): No heartbeat from core client for 30 sec - exiting 13:08:22 (6048): No heartbeat from core client for 30 sec - exiting 13:08:23 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5564, iMonCtr=2 14:27:49 (6052): No heartbeat from core client for 30 sec - exiting 14:27:50 (6052): No heartbeat from core client for 30 sec - exiting 14:27:51 (6052): No heartbeat from core client for 30 sec - exiting 14:27:52 (6052): No heartbeat from core client for 30 sec - exiting 14:27:53 (6052): No heartbeat from core client for 30 sec - exiting 14:27:54 (6052): No heartbeat from core client for 30 sec - exiting 14:27:55 (6052): No heartbeat from core client for 30 sec - exiting 14:27:56 (6052): No heartbeat from core client for 30 sec - exiting 14:27:58 (6052): No heartbeat from core client for 30 sec - exiting 14:27:59 (6052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:18:33 (4648): No heartbeat from core client for 30 sec - exiting 10:18:34 (4648): No heartbeat from core client for 30 sec - exiting 10:18:35 (4648): No heartbeat from core client for 30 sec - exiting 10:18:37 (4648): No heartbeat from core client for 30 sec - exiting 10:18:38 (4648): No heartbeat from core client for 30 sec - exiting 10:18:39 (4648): No heartbeat from core client for 30 sec - exiting 10:18:40 (4648): No heartbeat from core client for 30 sec - exiting 10:18:41 (4648): No heartbeat from core client for 30 sec - exiting 10:18:42 (4648): No heartbeat from core client for 30 sec - exiting 10:18:43 (4648): No heartbeat from core client for 30 sec - exiting 10:18:44 (4648): No heartbeat from core client for 30 sec - exiting 10:18:45 (4648): No heartbeat from core client for 30 sec - exiting 10:18:46 (4648): No heartbeat from core client for 30 sec - exiting 10:18:47 (4648): No heartbeat from core client for 30 sec - exiting 10:18:49 (4648): No heartbeat from core client for 30 sec - exiting 10:18:50 (4648): No heartbeat from core client for 30 sec - exiting 10:18:51 (4648): No heartbeat from core client for 30 sec - exiting 10:18:52 (4648): No heartbeat from core client for 30 sec - exiting 10:18:53 (4648): No heartbeat from core client for 30 sec - exiting 10:18:54 (4648): No heartbeat from core client for 30 sec - exiting 10:18:55 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:32:48 (5364): No heartbeat from core client for 30 sec - exiting 11:32:49 (5364): No heartbeat from core client for 30 sec - exiting 11:32:50 (5364): No heartbeat from core client for 30 sec - exiting 11:32:51 (5364): No heartbeat from core client for 30 sec - exiting 11:32:52 (5364): No heartbeat from core client for 30 sec - exiting 11:32:54 (5364): No heartbeat from core client for 30 sec - exiting 11:32:55 (5364): No heartbeat from core client for 30 sec - exiting 11:32:56 (5364): No heartbeat from core client for 30 sec - exiting 11:32:57 (5364): No heartbeat from core client for 30 sec - exiting 11:32:58 (5364): No heartbeat from core client for 30 sec - exiting 11:32:59 (5364): No heartbeat from core client for 30 sec - exiting 11:33:00 (5364): No heartbeat from core client for 30 sec - exiting 11:33:01 (5364): No heartbeat from core client for 30 sec - exiting 11:33:02 (5364): No heartbeat from core client for 30 sec - exiting 11:33:03 (5364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:49:54 (4276): No heartbeat from core client for 30 sec - exiting 09:49:55 (4276): No heartbeat from core client for 30 sec - exiting 09:49:56 (4276): No heartbeat from core client for 30 sec - exiting 09:49:57 (4276): No heartbeat from core client for 30 sec - exiting 09:49:58 (4276): No heartbeat from core client for 30 sec - exiting 09:49:59 (4276): No heartbeat from core client for 30 sec - exiting 09:50:00 (4276): No heartbeat from core client for 30 sec - exiting 09:50:01 (4276): No heartbeat from core client for 30 sec - exiting 09:50:03 (4276): No heartbeat from core client for 30 sec - exiting 09:50:04 (4276): No heartbeat from core client for 30 sec - exiting 09:50:05 (4276): No heartbeat from core client for 30 sec - exiting 09:50:06 (4276): No heartbeat from core client for 30 sec - exiting 09:50:07 (4276): No heartbeat from core client for 30 sec - exiting 09:50:08 (4276): No heartbeat from core client for 30 sec - exiting 09:50:09 (4276): No heartbeat from core client for 30 sec - exiting 09:50:10 (4276): No heartbeat from core client for 30 sec - exiting 09:50:11 (4276): No heartbeat from core client for 30 sec - exiting 09:50:12 (4276): No heartbeat from core client for 30 sec - exiting 09:50:13 (4276): No heartbeat from core client for 30 sec - exiting 09:50:15 (4276): No heartbeat from core client for 30 sec - exiting 09:50:16 (4276): No heartbeat from core client for 30 sec - exiting 09:50:17 (4276): No heartbeat from core client for 30 sec - exiting 09:50:18 (4276): No heartbeat from core client for 30 sec - exiting 09:50:19 (4276): No heartbeat from core client for 30 sec - exiting 09:50:20 (4276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:28:35 (6816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:30:34 (6872): No heartbeat from core client for 30 sec - exiting 23:30:35 (6872): No heartbeat from core client for 30 sec - exiting 23:30:36 (6872): No heartbeat from core client for 30 sec - exiting 23:30:37 (6872): No heartbeat from core client for 30 sec - exiting 23:30:38 (6872): No heartbeat from core client for 30 sec - exiting 23:30:39 (6872): No heartbeat from core client for 30 sec - exiting 23:30:40 (6872): No heartbeat from core client for 30 sec - exiting 23:30:41 (6872): No heartbeat from core client for 30 sec - exiting 23:30:42 (6872): No heartbeat from core client for 30 sec - exiting 23:30:43 (6872): No heartbeat from core client for 30 sec - exiting 23:30:44 (6872): No heartbeat from core client for 30 sec - exiting 23:30:45 (6872): No heartbeat from core client for 30 sec - exiting 23:30:46 (6872): No heartbeat from core client for 30 sec - exiting 23:30:47 (6872): No heartbeat from core client for 30 sec - exiting 23:30:48 (6872): No heartbeat from core client for 30 sec - exiting 23:30:49 (6872): No heartbeat from core client for 30 sec - exiting 23:30:50 (6872): No heartbeat from core client for 30 sec - exiting 23:30:51 (6872): No heartbeat from core client for 30 sec - exiting 23:30:52 (6872): No heartbeat from core client for 30 sec - exiting 23:30:53 (6872): No heartbeat from core client for 30 sec - exiting 23:30:54 (6872): No heartbeat from core client for 30 sec - exiting 23:30:55 (6872): No heartbeat from core client for 30 sec - exiting 23:30:56 (6872): No heartbeat from core client for 30 sec - exiting 23:30:57 (6872): No heartbeat from core client for 30 sec - exiting 23:30:58 (6872): No heartbeat from core client for 30 sec - exiting 23:30:59 (6872): No heartbeat from core client for 30 sec - exiting 23:31:00 (6872): No heartbeat from core client for 30 sec - exiting 23:31:01 (6872): No heartbeat from core client for 30 sec - exiting 23:31:02 (6872): No heartbeat from core client for 30 sec - exiting 23:31:03 (6872): No heartbeat from core client for 30 sec - exiting 23:31:04 (6872): No heartbeat from core client for 30 sec - exiting 23:31:05 (6872): No heartbeat from core client for 30 sec - exiting 23:31:06 (6872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5812, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5160, selfPID=7704, iMonCtr=1 Model crash detected, will try to restart... 11:19:28 (5344): No heartbeat from core client for 30 sec - exiting 11:19:29 (5344): No heartbeat from core client for 30 sec - exiting 11:19:30 (5344): No heartbeat from core client for 30 sec - exiting 11:19:31 (5344): No heartbeat from core client for 30 sec - exiting 11:19:32 (5344): No heartbeat from core client for 30 sec - exiting 11:19:33 (5344): No heartbeat from core client for 30 sec - exiting 11:19:34 (5344): No heartbeat from core client for 30 sec - exiting 11:19:35 (5344): No heartbeat from core client for 30 sec - exiting 11:19:36 (5344): No heartbeat from core client for 30 sec - exiting 11:19:37 (5344): No heartbeat from core client for 30 sec - exiting 11:19:38 (5344): No heartbeat from core client for 30 sec - exiting 11:19:39 (5344): No heartbeat from core client for 30 sec - exiting 11:19:40 (5344): No heartbeat from core client for 30 sec - exiting 11:19:41 (5344): No heartbeat from core client for 30 sec - exiting 11:19:42 (5344): No heartbeat from core client for 30 sec - exiting 11:19:43 (5344): No heartbeat from core client for 30 sec - exiting 11:19:44 (5344): No heartbeat from core client for 30 sec - exiting 11:19:46 (5344): No heartbeat from core client for 30 sec - exiting 11:19:47 (5344): No heartbeat from core client for 30 sec - exiting 11:19:48 (5344): No heartbeat from core client for 30 sec - exiting 11:19:49 (5344): No heartbeat from core client for 30 sec - exiting 11:19:50 (5344): No heartbeat from core client for 30 sec - exiting 11:19:51 (5344): No heartbeat from core client for 30 sec - exiting 11:19:52 (5344): No heartbeat from core client for 30 sec - exiting 11:19:53 (5344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1544, selfPID=1544, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Glob11:18:15 (5368): No heartbeat from core client for 30 sec - exiting 11:18:16 (5368): No heartbeat from core client for 30 sec - exiting 11:18:17 (5368): No heartbeat from core client for 30 sec - exiting 11:18:18 (5368): No heartbeat from core client for 30 sec - exiting 11:18:19 (5368): No heartbeat from core client for 30 sec - exiting 11:18:20 (5368): No heartbeat from core client for 30 sec - exiting 11:18:21 (5368): No heartbeat from core client for 30 sec - exiting 11:18:23 (5368): No heartbeat from core client for 30 sec - exiting 11:18:24 (5368): No heartbeat from core client for 30 sec - exiting 11:18:25 (5368): No heartbeat from core client for 30 sec - exiting 11:18:26 (5368): No heartbeat from core client for 30 sec - exiting 11:18:27 (5368): No heartbeat from core client for 30 sec - exiting 11:18:28 (5368): No heartbeat from core client for 30 sec - exiting 11:18:29 (5368): No heartbeat from core client for 30 sec - exiting 11:18:30 (5368): No heartbeat from core client for 30 sec - exiting 11:18:31 (5368): No heartbeat from core client for 30 sec - exiting 11:18:32 (5368): No heartbeat from core client for 30 sec - exiting 11:18:34 (5368): No heartbeat from core client for 30 sec - exiting 11:18:35 (5368): No heartbeat from core client for 30 sec - exiting 11:18:36 (5368): No heartbeat from core client for 30 sec - exiting 11:18:37 (5368): No heartbeat from core client for 30 sec - exiting 11:18:38 (5368): No heartbeat from core client for 30 sec - exiting 11:18:39 (5368): No heartbeat from core client for 30 sec - exiting 11:18:40 (5368): No heartbeat from core client for 30 sec - exiting 11:18:41 (5368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:43:57 (5072): No heartbeat from core client for 30 sec - exiting 10:43:59 (5072): No heartbeat from core client for 30 sec - exiting 10:44:00 (5072): No heartbeat from core client for 30 sec - exiting 10:44:01 (5072): No heartbeat from core client for 30 sec - exiting 10:44:02 (5072): No heartbeat from core client for 30 sec - exiting 10:44:03 (5072): No heartbeat from core client for 30 sec - exiting 10:44:04 (5072): No heartbeat from core client for 30 sec - exiting 10:44:05 (5072): No heartbeat from core client for 30 sec - exiting 10:44:06 (5072): No heartbeat from core client for 30 sec - exiting 10:44:07 (5072): No heartbeat from core client for 30 sec - exiting 10:44:08 (5072): No heartbeat from core client for 30 sec - exiting 10:44:10 (5072): No heartbeat from core client for 30 sec - exiting 10:44:11 (5072): No heartbeat from core client for 30 sec - exiting 10:44:12 (5072): No heartbeat from core client for 30 sec - exiting 10:44:13 (5072): No heartbeat from core client for 30 sec - exiting 10:44:14 (5072): No heartbeat from core client for 30 sec - exiting 10:44:15 (5072): No heartbeat from core client for 30 sec - exiting 10:44:16 (5072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:08:30 (5360): No heartbeat from core client for 30 sec - exiting 14:08:31 (5360): No heartbeat from core client for 30 sec - exiting 14:08:32 (5360): No heartbeat from core client for 30 sec - exiting 14:08:33 (5360): No heartbeat from core client for 30 sec - exiting 14:08:35 (5360): No heartbeat from core client for 30 sec - exiting 14:08:36 (5360): No heartbeat from core client for 30 sec - exiting 14:08:37 (5360): No heartbeat from core client for 30 sec - exiting 14:08:38 (5360): No heartbeat from core client for 30 sec - exiting 14:08:39 (5360): No heartbeat from core client for 30 sec - exiting 14:08:40 (5360): No heartbeat from core client for 30 sec - exiting 14:08:41 (5360): No heartbeat from core client for 30 sec - exiting 14:08:42 (5360): No heartbeat from core client for 30 sec - exiting 14:08:43 (5360): No heartbeat from core client for 30 sec - exiting 14:08:44 (5360): No heartbeat from core client for 30 sec - exiting 14:08:45 (5360): No heartbeat from core client for 30 sec - exiting 14:08:47 (5360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:08:48 (5360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 13:25:20 (5560): No heartbeat from core client for 30 sec - exiting 13:25:22 (5560): No heartbeat from core client for 30 sec - exiting 13:25:23 (5560): No heartbeat from core client for 30 sec - exiting 13:25:24 (5560): No heartbeat from core client for 30 sec - exiting 13:25:25 (5560): No heartbeat from core client for 30 sec - exiting 13:25:26 (5560): No heartbeat from core client for 30 sec - exiting 13:25:27 (5560): No heartbeat from core client for 30 sec - exiting 13:25:28 (5560): No heartbeat from core client for 30 sec - exiting 13:25:29 (5560): No heartbeat from core client for 30 sec - exiting 13:25:30 (5560): No heartbeat from core client for 30 sec - exiting 13:25:31 (5560): No heartbeat from core client for 30 sec - exiting 13:25:32 (5560): No heartbeat from core client for 30 sec - exiting 13:25:33 (5560): No heartbeat from core client for 30 sec - exiting 13:25:34 (5560): No heartbeat from core client for 30 sec - exiting 13:25:35 (5560): No heartbeat from core client for 30 sec - exiting 13:25:36 (5560): No heartbeat from core client for 30 sec - exiting 13:25:37 (5560): No heartbeat from core client for 30 sec - exiting 13:25:38 (5560): No heartbeat from core client for 30 sec - exiting 13:25:39 (5560): No heartbeat from core client for 30 sec - exiting 13:25:40 (5560): No heartbeat from core client for 30 sec - exiting 13:25:41 (5560): No heartbeat from core client for 30 sec - exiting 13:25:42 (5560): No heartbeat from core client for 30 sec - exiting 13:25:43 (5560): No heartbeat from core client for 30 sec - exiting 13:25:44 (5560): No heartbeat from core client for 30 sec - exiting 13:25:45 (5560): No heartbeat from core client for 30 sec - exiting 13:25:46 (5560): No heartbeat from core client for 30 sec - exiting 13:25:47 (5560): No heartbeat from core client for 30 sec - exiting 13:25:48 (5560): No heartbeat from core client for 30 sec - exiting 13:25:49 (5560): No heartbeat from core client for 30 sec - exiting 13:25:50 (5560): No heartbeat from core client for 30 sec - exiting 13:25:51 (5560): No heartbeat from core client for 30 sec - exiting 13:25:52 (5560): No heartbeat from core client for 30 sec - exiting 13:25:53 (5560): No heartbeat from core client for 30 sec - exiting 13:25:54 (5560): No heartbeat from core client for 30 sec - exiting 13:25:55 (5560): No heartbeat from core client for 30 sec - exiting 13:25:56 (5560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:24:28 (2356): No heartbeat from core client for 30 sec - exiting 17:24:29 (2356): No heartbeat from core client for 30 sec - exiting 17:24:30 (2356): No heartbeat from core client for 30 sec - exiting 17:24:31 (2356): No heartbeat from core client for 30 sec - exiting 17:24:33 (2356): No heartbeat from core client for 30 sec - exiting 17:24:34 (2356): No heartbeat from core client for 30 sec - exiting 17:24:35 (2356): No heartbeat from core client for 30 sec - exiting 17:24:36 (2356): No heartbeat from core client for 30 sec - exiting 17:24:37 (2356): No heartbeat from core client for 30 sec - exiting 17:24:38 (2356): No heartbeat from core client for 30 sec - exiting 17:24:39 (2356): No heartbeat from core client for 30 sec - exiting 17:24:40 (2356): No heartbeat from core client for 30 sec - exiting 17:24:41 (2356): No heartbeat from core client for 30 sec - exiting 17:24:42 (2356): No heartbeat from core client for 30 sec - exiting 17:24:43 (2356): No heartbeat from core client for 30 sec - exiting 17:24:45 (2356): No heartbeat from core client for 30 sec - exiting 17:24:46 (2356): No heartbeat from core client for 30 sec - exiting 17:24:47 (2356): No heartbeat from core client for 30 sec - exiting 17:24:48 (2356): No heartbeat from core client for 30 sec - exiting 17:24:49 (2356): No heartbeat from core client for 30 sec - exiting 17:24:50 (2356): No heartbeat from core client for 30 sec - exiting 17:24:51 (2356): No heartbeat from core client for 30 sec - exiting 17:24:52 (2356): No heartbeat from core client for 30 sec - exiting 17:24:53 (2356): No heartbeat from core client for 30 sec - exiting 17:24:54 (2356): No heartbeat from core client for 30 sec - exiting 17:24:55 (2356): No heartbeat from core client for 30 sec - exiting 17:24:57 (2356): No heartbeat from core client for 30 sec - exiting 17:24:58 (2356): No heartbeat from core client for 30 sec - exiting 17:24:59 (2356): No heartbeat from core client for 30 sec - exiting 17:25:00 (2356): No heartbeat from core client for 30 sec - exiting 17:25:01 (2356): No heartbeat from core client for 30 sec - exiting 17:25:02 (2356): No heartbeat from core client for 30 sec - exiting 17:25:03 (2356): No heartbeat from core client for 30 sec - exiting 17:25:04 (2356): No heartbeat from core client for 30 sec - exiting 17:25:05 (2356): No heartbeat from core client for 30 sec - exiting 17:25:06 (2356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5900, selfPID=5900, iMonCtr=2 11:50:18 (5584): No heartbeat from core client for 30 sec - exiting 11:50:19 (5584): No heartbeat from core client for 30 sec - exiting 11:50:20 (5584): No heartbeat from core client for 30 sec - exiting 11:50:21 (5584): No heartbeat from core client for 30 sec - exiting 11:50:22 (5584): No heartbeat from core client for 30 sec - exiting 11:50:23 (5584): No heartbeat from core client for 30 sec - exiting 11:50:24 (5584): No heartbeat from core client for 30 sec - exiting 11:50:25 (5584): No heartbeat from core client for 30 sec - exiting 11:50:26 (5584): No heartbeat from core client for 30 sec - exiting 11:50:27 (5584): No heartbeat from core client for 30 sec - exiting 11:50:29 (5584): No heartbeat from core client for 30 sec - exiting 11:50:30 (5584): No heartbeat from core client for 30 sec - exiting 11:50:31 (5584): No heartbeat from core client for 30 sec - exiting 11:50:32 (5584): No heartbeat from core client for 30 sec - exiting 11:50:33 (5584): No heartbeat from core client for 30 sec - exiting 11:50:34 (5584): No heartbeat from core client for 30 sec - exiting 11:50:35 (5584): No heartbeat from core client for 30 sec - exiting 11:50:36 (5584): No heartbeat from core client for 30 sec - exiting 11:50:37 (5584): No heartbeat from core client for 30 sec - exiting 11:50:38 (5584): No heartbeat from core client for 30 sec - exiting 11:50:39 (5584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:50:40 (5584): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:30:08 (5568): No heartbeat from core client for 30 sec - exiting 12:30:09 (5568): No heartbeat from core client for 30 sec - exiting 12:30:10 (5568): No heartbeat from core client for 30 sec - exiting 12:30:11 (5568): No heartbeat from core client for 30 sec - exiting 12:30:12 (5568): No heartbeat from core client for 30 sec - exiting 12:30:14 (5568): No heartbeat from core client for 30 sec - exiting 12:30:15 (5568): No heartbeat from core client for 30 sec - exiting 12:30:16 (5568): No heartbeat from core client for 30 sec - exiting 12:30:17 (5568): No heartbeat from core client for 30 sec - exiting 12:30:18 (5568): No heartbeat from core client for 30 sec - exiting 12:30:19 (5568): No heartbeat from core client for 30 sec - exiting 12:30:20 (5568): No heartbeat from core client for 30 sec - exiting 12:30:21 (5568): No heartbeat from core client for 30 sec - exiting 12:30:22 (5568): No heartbeat from core client for 30 sec - exiting 12:30:23 (5568): No heartbeat from core client for 30 sec - exiting 12:30:24 (5568): No heartbeat from core client for 30 sec - exiting 12:30:26 (5568): No heartbeat from core client for 30 sec - exiting 12:30:27 (5568): No heartbeat from core client for 30 sec - exiting 12:30:28 (5568): No heartbeat from core client for 30 sec - exiting 12:30:29 (5568): No heartbeat from core client for 30 sec - exiting 12:30:30 (5568): No heartbeat from core client for 30 sec - exiting 12:30:31 (5568): No heartbeat from core client for 30 sec - exiting 12:30:32 (5568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4936, selfPID=4936, iMonCtr=2 12:38:08 (4924): No heartbeat from core client for 30 sec - exiting 12:38:09 (4924): No heartbeat from core client for 30 sec - exiting 12:38:10 (4924): No heartbeat from core client for 30 sec - exiting 12:38:11 (4924): No heartbeat from core client for 30 sec - exiting 12:38:12 (4924): No heartbeat from core client for 30 sec - exiting 12:38:13 (4924): No heartbeat from core client for 30 sec - exiting 12:38:14 (4924): No heartbeat from core client for 30 sec - exiting 12:38:15 (4924): No heartbeat from core client for 30 sec - exiting 12:38:16 (4924): No heartbeat from core client for 30 sec - exiting 12:38:17 (4924): No heartbeat from core client for 30 sec - exiting 12:38:18 (4924): No heartbeat from core client for 30 sec - exiting 12:38:19 (4924): No heartbeat from core client for 30 sec - exiting 12:38:20 (4924): No heartbeat from core client for 30 sec - exiting 12:38:21 (4924): No heartbeat from core client for 30 sec - exiting 12:38:22 (4924): No heartbeat from core client for 30 sec - exiting 12:38:23 (4924): No heartbeat from core client for 30 sec - exiting 12:38:24 (4924): No heartbeat from core client for 30 sec - exiting 12:38:25 (4924): No heartbeat from core client for 30 sec - exiting 12:38:26 (4924): No heartbeat from core client for 30 sec - exiting 12:38:27 (4924): No heartbeat from core client for 30 sec - exiting 12:38:28 (4924): No heartbeat from core client for 30 sec - exiting 12:38:29 (4924): No heartbeat from core client for 30 sec - exiting 12:38:30 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7116, selfPID=7116, iMonCtr=2 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 12:40:02 (7008): No heartbeat from core client for 30 sec - exiting 12:40:03 (7008): No heartbeat from core client for 30 sec - exiting 12:40:04 (7008): No heartbeat from core client for 30 sec - exiting 12:40:05 (7008): No heartbeat from core client for 30 sec - exiting 12:40:06 (7008): No heartbeat from core client for 30 sec - exiting 12:40:07 (7008): No heartbeat from core client for 30 sec - exiting 12:40:08 (7008): No heartbeat from core client for 30 sec - exiting 12:40:09 (7008): No heartbeat from core client for 30 sec - exiting 12:40:10 (7008): No heartbeat from core client for 30 sec - exiting 12:40:11 (7008): No heartbeat from core client for 30 sec - exiting 12:40:12 (7008): No heartbeat from core client for 30 sec - exiting 12:40:13 (7008): No heartbeat from core client for 30 sec - exiting 12:40:14 (7008): No heartbeat from core client for 30 sec - exiting 12:40:15 (7008): No heartbeat from core client for 30 sec - exiting Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_pnw_aqll_1977_1_008001450_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_aqll_1977_1_008001450_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_aqll_1977_1_008001450_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_aqll_1977_1_008001450_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_aqll_1977_1_008001450_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_aqll_1977_1_008001450_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Sep 2012 18:10:36	1129423	14798189	hadam3p_pnw_aqll_1977_1_008001450_0	69,216	221,994	3.2073
19 Sep 2012 17:36:49	1129423	14798189	hadam3p_pnw_aqll_1977_1_008001450_0	57,696	184,699	3.2012
31 Aug 2012 16:48:00	1129423	14798189	hadam3p_pnw_aqll_1977_1_008001450_0	46,176	148,275	3.2111
26 Aug 2012 19:05:45	1129423	14798189	hadam3p_pnw_aqll_1977_1_008001450_0	34,656	111,127	3.2066
22 Aug 2012 19:00:52	1129423	14798189	hadam3p_pnw_aqll_1977_1_008001450_0	23,136	74,319	3.2123
13 Aug 2012 03:43:19	1129423	14798189	hadam3p_pnw_aqll_1977_1_008001450_0	11,616	37,332	3.2138