Task 14689164

Name	hadam3p_eu_d6lv_2005_1_007974773_0
Workunit	8129887
Created	16 May 2012, 17:40:07 UTC
Sent	31 May 2012, 3:42:19 UTC
Report deadline	13 May 2013, 9:02:19 UTC
Received	25 Jun 2012, 6:42:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1016212
Run time	7 days 20 hours 11 min 45 sec
CPU time	5 days 16 hours 21 min 1 sec
Validate state	Invalid
Credit	1,790.21
Device peak FLOPS	1.63 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>6.6.38</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1584, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 04:55:51 (4272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:58:46 (5004): No heartbeat from core client for 30 sec - exiting 04:58:47 (5004): No heartbeat from core client for 30 sec - exiting 04:58:48 (5004): No heartbeat from core client for 30 sec - exiting 04:58:49 (5004): No heartbeat from core client for 30 sec - exiting 04:58:50 (5004): No heartbeat from core client for 30 sec - exiting 04:58:51 (5004): No heartbeat from core client for 30 sec - exiting 04:58:52 (5004): No heartbeat from core client for 30 sec - exiting 04:58:53 (5004): No heartbeat from core client for 30 sec - exiting 04:58:54 (5004): No heartbeat from core client for 30 sec - exiting 04:58:55 (5004): No heartbeat from core client for 30 sec - exiting 04:58:56 (5004): No heartbeat from core client for 30 sec - exiting 04:58:57 (5004): No heartbeat from core client for 30 sec - exiting 04:58:58 (5004): No heartbeat from core client for 30 sec - exiting 04:58:59 (5004): No heartbeat from core client for 30 sec - exiting 04:59:00 (5004): No heartbeat from core client for 30 sec - exiting 04:59:01 (5004): No heartbeat from core client for 30 sec - exiting 04:59:02 (5004): No heartbeat from core client for 30 sec - exiting 04:59:03 (5004): No heartbeat from core client for 30 sec - exiting 04:59:04 (5004): No heartbeat from core client for 30 sec - exiting 04:59:05 (5004): No heartbeat from core client for 30 sec - exiting 04:59:06 (5004): No heartbeat from core client for 30 sec - exiting 04:59:07 (5004): No heartbeat from core client for 30 sec - exiting 04:59:08 (5004): No heartbeat from core client for 30 sec - exiting 04:59:09 (5004): No heartbeat from core client for 30 sec - exiting 04:59:10 (5004): No heartbeat from core client for 30 sec - exiting 04:59:11 (5004): No heartbeat from core client for 30 sec - exiting 04:59:12 (5004): No heartbeat from core client for 30 sec - exiting 04:59:13 (5004): No heartbeat from core client for 30 sec - exiting 04:59:14 (5004): No heartbeat from core client for 30 sec - exiting 04:59:15 (5004): No heartbeat from core client for 30 sec - exiting 04:59:16 (5004): No heartbeat from core client for 30 sec - exiting 04:59:17 (5004): No heartbeat from core client for 30 sec - exiting 04:59:18 (5004): No heartbeat from core client for 30 sec - exiting 04:59:19 (5004): No heartbeat from core client for 30 sec - exiting 04:59:20 (5004): No heartbeat from core client for 30 sec - exiting 04:59:21 (5004): No heartbeat from core client for 30 sec - exiting 04:59:22 (5004): No heartbeat from core client for 30 sec - exiting 04:59:23 (5004): No heartbeat from core client for 30 sec - exiting 04:59:24 (5004): No heartbeat from core client for 30 sec - exiting 04:59:25 (5004): No heartbeat from core client for 30 sec - exiting 04:59:26 (5004): No heartbeat from core client for 30 sec - exiting 04:59:27 (5004): No heartbeat from core client for 30 sec - exiting 04:59:28 (5004): No heartbeat from core client for 30 sec - exiting 04:59:29 (5004): No heartbeat from core client for 30 sec - exiting 04:59:30 (5004): No heartbeat from core client for 30 sec - exiting 04:59:31 (5004): No heartbeat from core client for 30 sec - exiting 04:59:32 (5004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4100, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... 05:15:41 (4188): No heartbeat from core client for 30 sec - exiting 05:15:42 (4188): No heartbeat from core client for 30 sec - exiting 05:15:43 (4188): No heartbeat from core client for 30 sec - exiting 05:15:44 (4188): No heartbeat from core client for 30 sec - exiting 05:15:45 (4188): No heartbeat from core client for 30 sec - exiting 05:15:46 (4188): No heartbeat from core client for 30 sec - exiting 05:15:48 (4188): No heartbeat from core client for 30 sec - exiting 05:15:49 (4188): No heartbeat from core client for 30 sec - exiting 05:15:50 (4188): No heartbeat from core client for 30 sec - exiting 05:15:51 (4188): No heartbeat from core client for 30 sec - exiting 05:15:52 (4188): No heartbeat from core client for 30 sec - exiting 05:15:53 (4188): No heartbeat from core client for 30 sec - exiting 05:15:54 (4188): No heartbeat from core client for 30 sec - exiting 05:15:55 (4188): No heartbeat from core client for 30 sec - exiting 05:15:56 (4188): No heartbeat from core client for 30 sec - exiting 05:15:57 (4188): No heartbeat from core client for 30 sec - exiting 05:15:58 (4188): No heartbeat from core client for 30 sec - exiting 05:16:00 (4188): No heartbeat from core client for 30 sec - exiting 05:16:01 (4188): No heartbeat from core client for 30 sec - exiting 05:16:02 (4188): No heartbeat from core client for 30 sec - exiting 05:16:03 (4188): No heartbeat from core client for 30 sec - exiting 05:16:04 (4188): No heartbeat from core client for 30 sec - exiting 05:16:05 (4188): No heartbeat from core client for 30 sec - exiting 05:16:06 (4188): No heartbeat from core client for 30 sec - exiting 05:16:07 (4188): No heartbeat from core client for 30 sec - exiting 05:16:08 (4188): No heartbeat from core client for 30 sec - exiting 05:16:09 (4188): No heartbeat from core client for 30 sec - exiting 05:16:11 (4188): No heartbeat from core client for 30 sec - exiting 05:16:12 (4188): No heartbeat from core client for 30 sec - exiting 05:16:13 (4188): No heartbeat from core client for 30 sec - exiting 05:16:14 (4188): No heartbeat from core client for 30 sec - exiting 05:16:15 (4188): No heartbeat from core client for 30 sec - exiting 05:16:16 (4188): No heartbeat from core client for 30 sec - exiting 05:16:17 (4188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:24:20 (5832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:50:32 (4740): No heartbeat from core client for 30 sec - exiting 06:50:34 (4740): No heartbeat from core client for 30 sec - exiting 06:50:35 (4740): No heartbeat from core client for 30 sec - exiting 06:50:36 (4740): No heartbeat from core client for 30 sec - exiting 06:50:37 (4740): No heartbeat from core client for 30 sec - exiting 06:50:38 (4740): No heartbeat from core client for 30 sec - exiting 06:50:39 (4740): No heartbeat from core client for 30 sec - exiting 06:50:40 (4740): No heartbeat from core client for 30 sec - exiting 06:50:41 (4740): No heartbeat from core client for 30 sec - exiting 06:50:42 (4740): No heartbeat from core client for 30 sec - exiting 06:50:44 (4740): No heartbeat from core client for 30 sec - exiting 06:50:45 (4740): No heartbeat from core client for 30 sec - exiting 06:50:46 (4740): No heartbeat from core client for 30 sec - exiting 06:50:47 (4740): No heartbeat from core client for 30 sec - exiting 06:50:48 (4740): No heartbeat from core client for 30 sec - exiting 06:50:49 (4740): No heartbeat from core client for 30 sec - exiting 06:50:50 (4740): No heartbeat from core client for 30 sec - exiting 06:50:51 (4740): No heartbeat from core client for 30 sec - exiting 06:50:52 (4740): No heartbeat from core client for 30 sec - exiting 06:50:53 (4740): No heartbeat from core client for 30 sec - exiting 06:50:54 (4740): No heartbeat from core client for 30 sec - exiting 06:50:56 (4740): No heartbeat from core client for 30 sec - exiting 06:50:57 (4740): No heartbeat from core client for 30 sec - exiting 06:50:58 (4740): No heartbeat from core client for 30 sec - exiting 06:50:59 (4740): No heartbeat from core client for 30 sec - exiting 06:51:00 (4740): No heartbeat from core client for 30 sec - exiting 06:51:01 (4740): No heartbeat from core client for 30 sec - exiting 06:51:02 (4740): No heartbeat from core client for 30 sec - exiting 06:51:03 (4740): No heartbeat from core client for 30 sec - exiting 06:51:04 (4740): No heartbeat from core client for 30 sec - exiting 06:51:05 (4740): No heartbeat from core client for 30 sec - exiting 06:51:06 (4740): No heartbeat from core client for 30 sec - exiting 06:51:08 (4740): No heartbeat from core client for 30 sec - exiting 06:51:09 (4740): No heartbeat from core client for 30 sec - exiting 06:51:10 (4740): No heartbeat from core client for 30 sec - exiting 06:51:11 (4740): No heartbeat from core client for 30 sec - exiting 06:51:12 (4740): No heartbeat from core client for 30 sec - exiting 06:51:13 (4740): No heartbeat from core client for 30 sec - exiting 06:51:14 (4740): No heartbeat from core client for 30 sec - exiting 06:51:15 (4740): No heartbeat from core client for 30 sec - exiting 06:51:16 (4740): No heartbeat from core client for 30 sec - exiting 06:51:17 (4740): No heartbeat from core client for 30 sec - exiting 06:51:19 (4740): No heartbeat from core client for 30 sec - exiting 06:51:20 (4740): No heartbeat from core client for 30 sec - exiting 06:51:21 (4740): No heartbeat from core client for 30 sec - exiting 06:51:22 (4740): No heartbeat from core client for 30 sec - exiting 06:51:23 (4740): No heartbeat from core client for 30 sec - exiting 06:51:24 (4740): No heartbeat from core client for 30 sec - exiting 06:51:25 (4740): No heartbeat from core client for 30 sec - exiting 06:51:26 (4740): No heartbeat from core client for 30 sec - exiting 06:51:27 (4740): No heartbeat from core client for 30 sec - exiting 06:51:28 (4740): No heartbeat from core client for 30 sec - exiting 06:51:29 (4740): No heartbeat from core client for 30 sec - exiting 06:51:31 (4740): No heartbeat from core client for 30 sec - exiting 06:51:32 (4740): No heartbeat from core client for 30 sec - exiting 06:51:33 (4740): No heartbeat from core client for 30 sec - exiting 06:51:34 (4740): No heartbeat from core client for 30 sec - exiting 06:51:35 (4740): No heartbeat from core client for 30 sec - exiting 06:51:36 (4740): No heartbeat from core client for 30 sec - exiting 06:51:37 (4740): No heartbeat from core client for 30 sec - exiting 06:51:38 (4740): No heartbeat from core client for 30 sec - exiting 06:51:39 (4740): No heartbeat from core client for 30 sec - exiting 06:20:12 (5752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:23:24 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4672, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 05:11:35 (5748): No heartbeat from core client for 30 sec - exiting 05:11:36 (5748): No heartbeat from core client for 30 sec - exiting 05:11:37 (5748): No heartbeat from core client for 30 sec - exiting 05:11:38 (5748): No heartbeat from core client for 30 sec - exiting 05:11:39 (5748): No heartbeat from core client for 30 sec - exiting 05:11:40 (5748): No heartbeat from core client for 30 sec - exiting 05:11:41 (5748): No heartbeat from core client for 30 sec - exiting 05:11:43 (5748): No heartbeat from core client for 30 sec - exiting 05:11:44 (5748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5980, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3580, selfPID=3016, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 05:36:54 (6092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3764, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5376, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:21:49 (5268): No heartbeat from core client for 30 sec - exiting 07:21:50 (5268): No heartbeat from core client for 30 sec - exiting 07:21:51 (5268): No heartbeat from core client for 30 sec - exiting 07:21:52 (5268): No heartbeat from core client for 30 sec - exiting 07:21:53 (5268): No heartbeat from core client for 30 sec - exiting 07:21:54 (5268): No heartbeat from core client for 30 sec - exiting 07:21:55 (5268): No heartbeat from core client for 30 sec - exiting 07:21:56 (5268): No heartbeat from core client for 30 sec - exiting 07:21:57 (5268): No heartbeat from core client for 30 sec - exiting 07:21:59 (5268): No heartbeat from core client for 30 sec - exiting 07:22:00 (5268): No heartbeat from core client for 30 sec - exiting 07:22:01 (5268): No heartbeat from core client for 30 sec - exiting 07:22:02 (5268): No heartbeat from core client for 30 sec - exiting 07:22:03 (5268): No heartbeat from core client for 30 sec - exiting 07:22:04 (5268): No heartbeat from core client for 30 sec - exiting 07:22:05 (5268): No heartbeat from core client for 30 sec - exiting 07:22:06 (5268): No heartbeat from core client for 30 sec - exiting 07:22:08 (5268): No heartbeat from core client for 30 sec - exiting 07:22:09 (5268): No heartbeat from core client for 30 sec - exiting 07:22:10 (5268): No heartbeat from core client for 30 sec - exiting 07:22:11 (5268): No heartbeat from core client for 30 sec - exiting 07:22:12 (5268): No heartbeat from core client for 30 sec - exiting 07:22:13 (5268): No heartbeat from core client for 30 sec - exiting 07:22:14 (5268): No heartbeat from core client for 30 sec - exiting 07:22:15 (5268): No heartbeat from core client for 30 sec - exiting 07:22:16 (5268): No heartbeat from core client for 30 sec - exiting 07:22:17 (5268): No heartbeat from core client for 30 sec - exiting 07:22:19 (5268): No heartbeat from core client for 30 sec - exiting 07:22:20 (5268): No heartbeat from core client for 30 sec - exiting 07:22:21 (5268): No heartbeat from core client for 30 sec - exiting 07:22:22 (5268): No heartbeat from core client for 30 sec - exiting 07:22:23 (5268): No heartbeat from core client for 30 sec - exiting 07:22:24 (5268): No heartbeat from core client for 30 sec - exiting 07:22:25 (5268): No heartbeat from core client for 30 sec - exiting 07:22:26 (5268): No heartbeat from core client for 30 sec - exiting 07:22:27 (5268): No heartbeat from core client for 30 sec - exiting 07:22:29 (5268): No heartbeat from core client for 30 sec - exiting 07:22:30 (5268): No heartbeat from core client for 30 sec - exiting 07:22:31 (5268): No heartbeat from core client for 30 sec - exiting 07:22:32 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:27:52 (6024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3328, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2600, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 06:36:58 (5748): No heartbeat from core client for 30 sec - exiting 06:36:59 (5748): No heartbeat from core client for 30 sec - exiting 06:37:00 (5748): No heartbeat from core client for 30 sec - exiting 06:37:01 (5748): No heartbeat from core client for 30 sec - exiting 06:37:02 (5748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5408, selfPID=5408, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5980, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3132, selfPID=4712, iMonCtr=1 Model crash detected, will try to restart... 07:47:41 (5924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7300, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5880, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 08:31:52 (2160): No heartbeat from core client for 30 sec - exiting 08:31:53 (2160): No heartbeat from core client for 30 sec - exiting 08:31:54 (2160): No heartbeat from core client for 30 sec - exiting 08:31:55 (2160): No heartbeat from core client for 30 sec - exiting 08:31:56 (2160): No heartbeat from core client for 30 sec - exiting 08:31:57 (2160): No heartbeat from core client for 30 sec - exiting 08:31:58 (2160): No heartbeat from core client for 30 sec - exiting 08:32:00 (2160): No heartbeat from core client for 30 sec - exiting 08:32:01 (2160): No heartbeat from core client for 30 sec - exiting 08:32:02 (2160): No heartbeat from core client for 30 sec - exiting 08:32:03 (2160): No heartbeat from core client for 30 sec - exiting 08:32:04 (2160): No heartbeat from core client for 30 sec - exiting 08:32:05 (2160): No heartbeat from core client for 30 sec - exiting 08:32:06 (2160): No heartbeat from core client for 30 sec - exiting 08:32:07 (2160): No heartbeat from core client for 30 sec - exiting 08:32:08 (2160): No heartbeat from core client for 30 sec - exiting 08:32:09 (2160): No heartbeat from core client for 30 sec - exiting 08:32:10 (2160): No heartbeat from core client for 30 sec - exiting 08:32:12 (2160): No heartbeat from core client for 30 sec - exiting 08:32:13 (2160): No heartbeat from core client for 30 sec - exiting 08:32:14 (2160): No heartbeat from core client for 30 sec - exiting 08:32:15 (2160): No heartbeat from core client for 30 sec - exiting 08:32:16 (2160): No heartbeat from core client for 30 sec - exiting 08:32:17 (2160): No heartbeat from core client for 30 sec - exiting 08:32:18 (2160): No heartbeat from core client for 30 sec - exiting 08:32:19 (2160): No heartbeat from core client for 30 sec - exiting 08:32:20 (2160): No heartbeat from core client for 30 sec - exiting 08:32:21 (2160): No heartbeat from core client for 30 sec - exiting 08:32:22 (2160): No heartbeat from core client for 30 sec - exiting 08:32:24 (2160): No heartbeat from core client for 30 sec - exiting 08:32:25 (2160): No heartbeat from core client for 30 sec - exiting 08:32:26 (2160): No heartbeat from core client for 30 sec - exiting 08:32:27 (2160): No heartbeat from core client for 30 sec - exiting 08:32:28 (2160): No heartbeat from core client for 30 sec - exiting 08:32:29 (2160): No heartbeat from core client for 30 sec - exiting 08:32:30 (2160): No heartbeat from core client for 30 sec - exiting 08:32:31 (2160): No heartbeat from core client for 30 sec - exiting 08:32:32 (2160): No heartbeat from core client for 30 sec - exiting 08:32:33 (2160): No heartbeat from core client for 30 sec - exiting 08:32:34 (2160): No heartbeat from core client for 30 sec - exiting 08:32:36 (2160): No heartbeat from core client for 30 sec - exiting 08:32:37 (2160): No heartbeat from core client for 30 sec - exiting 08:32:38 (2160): No heartbeat from core client for 30 sec - exiting 08:32:39 (2160): No heartbeat from core client for 30 sec - exiting 08:32:40 (2160): No heartbeat from core client for 30 sec - exiting 08:32:41 (2160): No heartbeat from core client for 30 sec - exiting 08:32:42 (2160): No heartbeat from core client for 30 sec - exiting 08:32:43 (2160): No heartbeat from core client for 30 sec - exiting 08:32:44 (2160): No heartbeat from core client for 30 sec - exiting 08:32:46 (2160): No heartbeat from core client for 30 sec - exiting 08:33:20 (2160): No heartbeat from core client for 30 sec - exiting 08:33:21 (2160): No heartbeat from core client for 30 sec - exiting 08:33:23 (2160): No heartbeat from core client for 30 sec - exiting 08:33:24 (2160): No heartbeat from core client for 30 sec - exiting 08:33:25 (2160): No heartbeat from core client for 30 sec - exiting 08:33:26 (2160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4192, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2784, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1420, selfPID=5904, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 06:08:48 (5592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3112, selfPID=3356, iMonCtr=1 Model crash detected, will try to restart... 07:03:54 (5544): No heartbeat from core client for 30 sec - exiting 07:03:55 (5544): No heartbeat from core client for 30 sec - exiting 07:03:56 (5544): No heartbeat from core client for 30 sec - exiting 07:03:57 (5544): No heartbeat from core client for 30 sec - exiting 07:03:59 (5544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4852, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 22:10:03 (5848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5028, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=520, selfPID=1448, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3736, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 09:19:19 (5568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5632, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1040, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2656, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5172, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 06:29:00 (5888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5972, selfPID=5104, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 05:26:07 (4340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:30:49 (4600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5376, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5480, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2608, selfPID=5696, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:33:09 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5316, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 05:41:07 (3540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6136, selfPID=6136, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6236, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6452, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5548, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5728, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 05:18:09 (5840): No heartbeat from core client for 30 sec - exiting 05:18:10 (5840): No heartbeat from core client for 30 sec - exiting 05:18:11 (5840): No heartbeat from core client for 30 sec - exiting 05:18:12 (5840): No heartbeat from core client for 30 sec - exiting 05:18:13 (5840): No heartbeat from core client for 30 sec - exiting 05:18:14 (5840): No heartbeat from core client for 30 sec - exiting 05:18:15 (5840): No heartbeat from core client for 30 sec - exiting 05:18:16 (5840): No heartbeat from core client for 30 sec - exiting 05:18:17 (5840): No heartbeat from core client for 30 sec - exiting 05:18:18 (5840): No heartbeat from core client for 30 sec - exiting 05:18:20 (5840): No heartbeat from core client for 30 sec - exiting 05:18:21 (5840): No heartbeat from core client for 30 sec - exiting 05:18:22 (5840): No heartbeat from core client for 30 sec - exiting 05:18:23 (5840): No heartbeat from core client for 30 sec - exiting 05:18:24 (5840): No heartbeat from core client for 30 sec - exiting 05:18:25 (5840): No heartbeat from core client for 30 sec - exiting 05:18:26 (5840): No heartbeat from core client for 30 sec - exiting 05:18:27 (5840): No heartbeat from core client for 30 sec - exiting 05:18:28 (5840): No heartbeat from core client for 30 sec - exiting 05:18:29 (5840): No heartbeat from core client for 30 sec - exiting 05:18:31 (5840): No heartbeat from core client for 30 sec - exiting 05:18:32 (5840): No heartbeat from core client for 30 sec - exiting 05:18:33 (5840): No heartbeat from core client for 30 sec - exiting 05:18:34 (5840): No heartbeat from core client for 30 sec - exiting 05:18:35 (5840): No heartbeat from core client for 30 sec - exiting 05:18:36 (5840): No heartbeat from core client for 30 sec - exiting 05:18:37 (5840): No heartbeat from core client for 30 sec - exiting 05:18:38 (5840): No heartbeat from core client for 30 sec - exiting 05:18:39 (5840): No heartbeat from core client for 30 sec - exiting 05:18:40 (5840): No heartbeat from core client for 30 sec - exiting 05:18:41 (5840): No heartbeat from core client for 30 sec - exiting 05:18:43 (5840): No heartbeat from core client for 30 sec - exiting 05:18:44 (5840): No heartbeat from core client for 30 sec - exiting 05:18:45 (5840): No heartbeat from core client for 30 sec - exiting 05:18:46 (5840): No heartbeat from core client for 30 sec - exiting 05:18:47 (5840): No heartbeat from core client for 30 sec - exiting 05:18:48 (5840): No heartbeat from core client for 30 sec - exiting 05:18:49 (5840): No heartbeat from core client for 30 sec - exiting 05:18:50 (5840): No heartbeat from core client for 30 sec - exiting 05:18:51 (5840): No heartbeat from core client for 30 sec - exiting 05:18:52 (5840): No heartbeat from core client for 30 sec - exiting 05:18:53 (5840): No heartbeat from core client for 30 sec - exiting 05:18:55 (5840): No heartbeat from core client for 30 sec - exiting 05:18:56 (5840): No heartbeat from core client for 30 sec - exiting 05:18:57 (5840): No heartbeat from core client for 30 sec - exiting 05:18:58 (5840): No heartbeat from core client for 30 sec - exiting 05:18:59 (5840): No heartbeat from core client for 30 sec - exiting 05:19:00 (5840): No heartbeat from core client for 30 sec - exiting 05:19:01 (5840): No heartbeat from core client for 30 sec - exiting 05:19:02 (5840): No heartbeat from core client for 30 sec - exiting 05:19:03 (5840): No heartbeat from core client for 30 sec - exiting 05:19:04 (5840): No heartbeat from core client for 30 sec - exiting 05:19:06 (5840): No heartbeat from core client for 30 sec - exiting 05:19:07 (5840): No heartbeat from core client for 30 sec - exiting 05:19:08 (5840): No heartbeat from core client for 30 sec - exiting 05:19:09 (5840): No heartbeat from core client for 30 sec - exiting 05:19:10 (5840): No heartbeat from core client for 30 sec - exiting 05:19:11 (5840): No heartbeat from core client for 30 sec - exiting 05:19:12 (5840): No heartbeat from core client for 30 sec - exiting 05:19:13 (5840): No heartbeat from core client for 30 sec - exiting 05:19:14 (5840): No heartbeat from core client for 30 sec - exiting 05:19:15 (5840): No heartbeat from core client for 30 sec - exiting 05:19:16 (5840): No heartbeat from core client for 30 sec - exiting 05:19:18 (5840): No heartbeat from core client for 30 sec - exiting 05:19:19 (5840): No heartbeat from core client for 30 sec - exiting 05:19:20 (5840): No heartbeat from core client for 30 sec - exiting 05:19:21 (5840): No heartbeat from core client for 30 sec - exiting 05:19:22 (5840): No heartbeat from core client for 30 sec - exiting 05:19:23 (5840): No heartbeat from core client for 30 sec - exiting 05:19:24 (5840): No heartbeat from core client for 30 sec - exiting 05:19:25 (5840): No heartbeat from core client for 30 sec - exiting 05:19:26 (5840): No heartbeat from core client for 30 sec - exiting 05:19:27 (5840): No heartbeat from core client for 30 sec - exiting 05:19:28 (5840): No heartbeat from core client for 30 sec - exiting 05:19:30 (5840): No heartbeat from core client for 30 sec - exiting 05:19:31 (5840): No heartbeat from core client for 30 sec - exiting 05:19:32 (5840): No heartbeat from core client for 30 sec - exiting 05:19:33 (5840): No heartbeat from core client for 30 sec - exiting 05:19:34 (5840): No heartbeat from core client for 30 sec - exiting 05:19:35 (5840): No heartbeat from core client for 30 sec - exiting 05:19:36 (5840): No heartbeat from core client for 30 sec - exiting 05:19:37 (5840): No heartbeat from core client for 30 sec - exiting 05:19:38 (5840): No heartbeat from core client for 30 sec - exiting 05:19:39 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4248, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:22:32 (336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2544, selfPID=2544, iMonCtr=2 GCobal Workerr:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5872, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5356, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 15:17:32 (1828): No heartbeat from core client for 30 sec - exiting 15:17:34 (1828): No heartbeat from core client for 30 sec - exiting 15:17:35 (1828): No heartbeat from core client for 30 sec - exiting 15:17:36 (1828): No heartbeat from core client for 30 sec - exiting 15:17:37 (1828): No heartbeat from core client for 30 sec - exiting 15:17:38 (1828): No heartbeat from core client for 30 sec - exiting 15:17:39 (1828): No heartbeat from core client for 30 sec - exiting 15:17:40 (1828): No heartbeat from core client for 30 sec - exiting 15:17:41 (1828): No heartbeat from core client for 30 sec - exiting 15:17:42 (1828): No heartbeat from core client for 30 sec - exiting 15:17:44 (1828): No heartbeat from core client for 30 sec - exiting 15:17:45 (1828): No heartbeat from core client for 30 sec - exiting 15:17:46 (1828): No heartbeat from core client for 30 sec - exiting 15:17:47 (1828): No heartbeat from core client for 30 sec - exiting 15:17:48 (1828): No heartbeat from core client for 30 sec - exiting 15:17:49 (1828): No heartbeat from core client for 30 sec - exiting 15:17:50 (1828): No heartbeat from core client for 30 sec - exiting 15:17:51 (1828): No heartbeat from core client for 30 sec - exiting 15:17:52 (1828): No heartbeat from core client for 30 sec - exiting 15:17:54 (1828): No heartbeat from core client for 30 sec - exiting 15:17:55 (1828): No heartbeat from core client for 30 sec - exiting 15:17:56 (1828): No heartbeat from core client for 30 sec - exiting 15:17:57 (1828): No heartbeat from core client for 30 sec - exiting 15:17:58 (1828): No heartbeat from core client for 30 sec - exiting 15:17:59 (1828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4248, iMonCtr=2 Model crash detected, will try to restart... GCobal Workerr:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3116, iMonCtr=2 Mod el crash detected, will try to restart... Leaving CPDN_Main::Monitor... 16:43:30 (4488): No heartbeat from core client for 30 sec - exiting 16:43:31 (4488): No heartbeat from core client for 30 sec - exiting 16:43:32 (4488): No heartbeat from core client for 30 sec - exiting 16:43:33 (4488): No heartbeat from core client for 30 sec - exiting 16:43:34 (4488): No heartbeat from core client for 30 sec - exiting 16:43:35 (4488): No heartbeat from core client for 30 sec - exiting 16:43:36 (4488): No heartbeat from core client for 30 sec - exiting 16:43:38 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2512, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6124, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5904, iMonCtr=2 Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4572, selfPID=3608, iMonCtr=1 Model crash detected, will try to restart... 09:22:16 (4948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:01:33 (5968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:41:03 (6096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4468, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 11:20:03 (6104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:25:16 (3964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:29:29 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:30:39 (2136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5480, selfPID=5480, iMonCtr=2 12:34:18 (2896): No heartbeat from core client for 30 sec - exiting 12:34:19 (2896): No heartbeat from core client for 30 sec - exiting 12:34:20 (2896): No heartbeat from core client for 30 sec - exiting 12:34:21 (2896): No heartbeat from core client for 30 sec - exiting 12:34:22 (2896): No heartbeat from core client for 30 sec - exiting 12:34:23 (2896): No heartbeat from core client for 30 sec - exiting 12:34:24 (2896): No heartbeat from core client for 30 sec - exiting 12:34:25 (2896): No heartbeat from core client for 30 sec - exiting 12:34:26 (2896): No heartbeat from core client for 30 sec - exiting 12:34:27 (2896): No heartbeat from core client for 30 sec - exiting 12:34:28 (2896): No heartbeat from core client for 30 sec - exiting 12:34:29 (2896): No heartbeat from core client for 30 sec - exiting 12:34:30 (2896): No heartbeat from core client for 30 sec - exiting 12:34:31 (2896): No heartbeat from core client for 30 sec - exiting 12:34:32 (2896): No heartbeat from core client for 30 sec - exiting 12:34:33 (2896): No heartbeat from core client for 30 sec - exiting 12:34:34 (2896): No heartbeat from core client for 30 sec - exiting 12:34:35 (2896): No heartbeat from core client for 30 sec - exiting 12:34:36 (2896): No heartbeat from core client for 30 sec - exiting 12:34:37 (2896): No heartbeat from core client for 30 sec - exiting 12:34:38 (2896): No heartbeat from core client for 30 sec - exiting 12:34:39 (2896): No heartbeat from core client for 30 sec - exiting 12:34:40 (2896): No heartbeat from core client for 30 sec - exiting 12:34:41 (2896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1656, iMonCtr=2 Model crash detected, will21:26:30 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:43:29 (5572): No heartbeat from core client for 30 sec - exiting 06:43:30 (5572): No heartbeat from core client for 30 sec - exiting 06:43:31 (5572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3840, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4632, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_eu_d6lv_2005_1_007974773_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_d6lv_2005_1_007974773_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_d6lv_2005_1_007974773_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Jun 2012 07:24:17	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	103,776	461,169	4.4439
20 Jun 2012 14:35:00	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	92,256	410,511	4.4497
19 Jun 2012 15:48:49	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	80,736	358,376	4.4389
18 Jun 2012 08:59:53	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	69,216	306,713	4.4312
17 Jun 2012 08:39:06	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	57,696	255,114	4.4217
16 Jun 2012 13:50:22	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	46,176	203,895	4.4156
14 Jun 2012 10:08:24	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	34,656	152,162	4.3906
11 Jun 2012 17:05:29	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	23,136	101,793	4.3998
01 Jun 2012 09:54:00	1016212	14689164	hadam3p_eu_d6lv_2005_1_007974773_0	11,616	51,128	4.4015