Task 12857249

Name	hadam3p_saf_27gy_1995_1_007239984_0
Workunit	7438224
Created	4 May 2011, 6:46:07 UTC
Sent	10 May 2011, 9:26:06 UTC
Report deadline	21 Apr 2012, 14:46:06 UTC
Received	5 Sep 2011, 11:16:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	941998
Run time	10 days 9 hours 27 min 16 sec
CPU time	10 days 9 hours 27 min 16 sec
Validate state	Invalid
Credit	2,057.21
Device peak FLOPS	0.92 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <stderr_txt> 19:24:47 (4828): No heartbeat from core client for 30 sec - exiting 19:24:48 (4828): No heartbeat from core client for 30 sec - exiting 19:24:49 (4828): No heartbeat from core client for 30 sec - exiting 19:24:50 (4828): No heartbeat from core client for 30 sec - exiting 19:24:51 (4828): No heartbeat from core client for 30 sec - exiting 19:24:52 (4828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3292, selfPID=4984, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4344, selfPID=4344, iMonCtr=2 11:30:46 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:30:47 (4172): No heartbeat from core client for 30 sec - exiting 11:30:48 (4172): No heartbeat from core client for 30 sec - exiting 11:30:49 (4172): No heartbeat from core client for 30 sec - exiting 11:30:50 (4172): No heartbeat from core client for 30 sec - exiting 11:30:51 (4172): No heartbeat from core client for 30 sec - exiting 11:30:52 (4172): No heartbeat from core client for 30 sec - exiting 11:30:53 (4172): No heartbeat from core client for 30 sec - exiting 11:30:54 (4172): No heartbeat from core client for 30 sec - exiting 11:30:55 (4172): No heartbeat from core client for 30 sec - exiting 11:30:56 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2744, selfPID=2744, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6932, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7492, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2852, selfPID=2852, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4624, selfPID=4624, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4908, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4964, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4628, selfPID=4628, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7708, selfPID=7708, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2908, selfPID=2908, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4620, selfPID=4620, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6396, selfPID=6396, iMonCtr=2 08:48:10 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3632, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3700, selfPID=512, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:07:33 (4696): No heartbeat from core client for 30 sec - exiting 09:07:34 (4696): No heartbeat from core client for 30 sec - exiting 09:07:36 (4696): No heartbeat from core client for 30 sec - exiting 09:07:37 (4696): No heartbeat from core client for 30 sec - exiting 09:07:38 (4696): No heartbeat from core client for 30 sec - exiting 09:07:39 (4696): No heartbeat from core client for 30 sec - exiting 09:07:40 (4696): No heartbeat from core client for 30 sec - exiting 09:07:41 (4696): No heartbeat from core client for 30 sec - exiting 09:07:42 (4696): No heartbeat from core client for 30 sec - exiting 09:07:43 (4696): No heartbeat from core client for 30 sec - exiting 09:07:44 (4696): No heartbeat from core client for 30 sec - exiting 09:07:45 (4696): No heartbeat from core client for 30 sec - exiting 09:07:46 (4696): No heartbeat from core client for 30 sec - exiting 09:07:47 (4696): No heartbeat from core client for 30 sec - exiting 09:07:48 (4696): No heartbeat from core client for 30 sec - exiting 09:07:49 (4696): No heartbeat from core client for 30 sec - exiting 09:07:50 (4696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5252, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5752, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3252, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6084, iMonCtr=2 10:04:23 (1564): No heartbeat from core client for 30 sec - exiting 10:04:24 (1564): No heartbeat from core client for 30 sec - exiting 10:04:25 (1564): No heartbeat from core client for 30 sec - exiting 10:04:26 (1564): No heartbeat from core client for 30 sec - exiting 10:04:27 (1564): No heartbeat from core client for 30 sec - exiting 10:04:28 (1564): No heartbeat from core client for 30 sec - exiting 10:04:29 (1564): No heartbeat from core client for 30 sec - exiting 10:04:30 (1564): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5356, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5056, iMonCtr=2 Model crash detected, will try to restart... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:18:58 (3988): No heartbeat from core client for 30 sec - exiting 19:18:59 (3988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5976, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 09:36:39 (4212): No heartbeat from core client for 30 sec - exiting 09:36:40 (4212): No heartbeat from core client for 30 sec - exiting 09:36:41 (4212): No heartbeat from core client for 30 sec - exiting 09:36:42 (4212): No heartbeat from core client for 30 sec - exiting 09:36:43 (4212): No heartbeat from core client for 30 sec - exiting 09:36:44 (4212): No heartbeat from core client for 30 sec - exiting 09:36:45 (4212): No heartbeat from core client for 30 sec - exiting 09:36:46 (4212): No heartbeat from core client for 30 sec - exiting 09:36:47 (4212): No heartbeat from core client for 30 sec - exiting 09:36:48 (4212): No heartbeat from core client for 30 sec - exiting 09:36:49 (4212): No heartbeat from core client for 30 sec - exiting 09:36:50 (4212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6084, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6124, iMonCtr=2 21:38:39 (4440): No heartbeat from core client for 30 sec - exiting 21:38:40 (4440): No heartbeat from core client for 30 sec - exiting 21:38:41 (4440): No heartbeat from core client for 30 sec - exiting 21:38:42 (4440): No heartbeat from core client for 30 sec - exiting 21:38:43 (4440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:17 (1988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:18 (1988): No heartbeat from core client for 30 sec - exiting 00:23:05 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1136, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5964, iMonCtr=2 Model crash detected, will try to restart... 10:55:12 (6052): No heartbeat from core client for 30 sec - exiting 10:55:13 (6052): No heartbeat from core client for 30 sec - exiting 10:55:14 (6052): No heartbeat from core client for 30 sec - exiting 10:55:15 (6052): No heartbeat from core client for 30 sec - exiting 10:55:16 (6052): No heartbeat from core client for 30 sec - exiting 10:55:17 (6052): No heartbeat from core client for 30 sec - exiting 10:55:18 (6052): No heartbeat from core client for 30 sec - exiting 10:55:19 (6052): No heartbeat from core client for 30 sec - exiting 10:55:20 (6052): No heartbeat from core client for 30 sec - exiting 10:55:21 (6052): No heartbeat from core client for 30 sec - exiting 10:55:22 (6052): No heartbeat from core client for 30 sec - exiting 10:55:23 (6052): No heartbeat from core client for 30 sec - exiting 10:55:24 (6052): No heartbeat from core client for 30 sec - exiting 10:55:25 (6052): No heartbeat from core client for 30 sec - exiting 10:55:26 (6052): No heartbeat from core client for 30 sec - exiting 10:55:27 (6052): No heartbeat from core client for 30 sec - exiting 10:55:28 (6052): No heartbeat from core client for 30 sec - exiting 10:55:29 (6052): No heartbeat from core client for 30 sec - exiting 10:55:30 (6052): No heartbeat from core client for 30 sec - exiting 10:55:31 (6052): No heartbeat from core client for 30 sec - exiting 10:55:32 (6052): No heartbeat from core client for 30 sec - exiting 10:55:33 (6052): No heartbeat from core client for 30 sec - exiting 10:55:35 (6052): No heartbeat from core client for 30 sec - exiting 10:55:36 (6052): No heartbeat from core client for 30 sec - exiting 10:55:37 (6052): No heartbeat from core client for 30 sec - exiting 10:55:38 (6052): No heartbeat from core client for 30 sec - exiting 10:55:39 (6052): No heartbeat from core client for 30 sec - exiting 10:55:40 (6052): No heartbeat from core client for 30 sec - exiting 10:55:41 (6052): No heartbeat from core client for 30 sec - exiting 10:55:42 (6052): No heartbeat from core client for 30 sec - exiting 10:55:43 (6052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3824, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3312, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 09:26:30 (5968): No heartbeat from core client for 30 sec - exiting 09:26:31 (5968): No heartbeat from core client for 30 sec - exiting 09:26:32 (5968): No heartbeat from core client for 30 sec - exiting 09:26:33 (5968): No heartbeat from core client for 30 sec - exiting 09:26:34 (5968): No heartbeat from core client for 30 sec - exiting 09:26:35 (5968): No heartbeat from core client for 30 sec - exiting 09:26:36 (5968): No heartbeat from core client for 30 sec - exiting 09:26:37 (5968): No heartbeat from core client for 30 sec - exiting 09:26:38 (5968): No heartbeat from core client for 30 sec - exiting 09:26:39 (5968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9672, selfPID=9672, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9240, selfPID=9240, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4312, selfPID=4312, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7428, selfPID=7428, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7184, selfPID=7184, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5156, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:56:09 (4148): No heartbeat from core client for 30 sec - exiting 07:56:10 (4148): No heartbeat from core client for 30 sec - exiting 07:56:11 (4148): No heartbeat from core client for 30 sec - exiting 07:56:12 (4148): No heartbeat from core client for 30 sec - exiting 07:56:13 (4148): No heartbeat from core client for 30 sec - exiting 07:56:15 (4148): No heartbeat from core client for 30 sec - exiting 07:56:16 (4148): No heartbeat from core client for 30 sec - exiting 07:56:17 (4148): No heartbeat from core client for 30 sec - exiting 07:56:18 (4148): No heartbeat from core client for 30 sec - exiting 07:56:19 (4148): No heartbeat from core client for 30 sec - exiting 07:56:20 (4148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4744, selfPID=4744, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3036, selfPID=3036, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3344, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4232, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:37:43 (1344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3240, selfPID=3240, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4948, selfPID=4948, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 08:45:04 (4536): No heartbeat from core client for 30 sec - exiting 08:45:05 (4536): No heartbeat from core client for 30 sec - exiting 08:45:06 (4536): No heartbeat from core client for 30 sec - exiting 08:45:07 (4536): No heartbeat from core client for 30 sec - exiting 08:45:08 (4536): No heartbeat from core client for 30 sec - exiting 08:45:09 (4536): No heartbeat from core client for 30 sec - exiting 08:45:10 (4536): No heartbeat from core client for 30 sec - exiting 08:45:11 (4536): No heartbeat from core client for 30 sec - exiting 08:45:12 (4536): No heartbeat from core client for 30 sec - exiting 08:45:13 (4536): No heartbeat from core client for 30 sec - exiting 08:45:14 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1908, iMonCtr=2 06:32:24 (5756): No heartbeat from core client for 30 sec - exiting 06:32:25 (5756): No heartbeat from core client for 30 sec - exiting 06:32:26 (5756): No heartbeat from core client for 30 sec - exiting 06:32:27 (5756): No heartbeat from core client for 30 sec - exiting 06:32:28 (5756): No heartbeat from core client for 30 sec - exiting 06:32:29 (5756): No heartbeat from core client for 30 sec - exiting 06:32:30 (5756): No heartbeat from core client for 30 sec - exiting 06:32:31 (5756): No heartbeat from core client for 30 sec - exiting 06:32:32 (5756): No heartbeat from core client for 30 sec - exiting 06:32:33 (5756): No heartbeat from core client for 30 sec - exiting 06:32:34 (5756): No heartbeat from core client for 30 sec - exiting 06:32:35 (5756): No heartbeat from core client for 30 sec - exiting 06:32:36 (5756): No heartbeat from core client for 30 sec - exiting 06:32:37 (5756): No heartbeat from core client for 30 sec - exiting 06:32:38 (5756): No heartbeat from core client for 30 sec - exiting 06:32:39 (5756): No heartbeat from core client for 30 sec - exiting 06:32:40 (5756): No heartbeat from core client for 30 sec - exiting 06:32:41 (5756): No heartbeat from core client for 30 sec - exiting 06:32:42 (5756): No heartbeat from core client for 30 sec - exiting 06:32:43 (5756): No heartbeat from core client for 30 sec - exiting 06:32:44 (5756): No heartbeat from core client for 30 sec - exiting 06:32:45 (5756): No heartbeat from core client for 30 sec - exiting 06:32:46 (5756): No heartbeat from core client for 30 sec - exiting 06:32:47 (5756): No heartbeat from core client for 30 sec - exiting 06:32:48 (5756): No heartbeat from core client for 30 sec - exiting 06:32:49 (5756): No heartbeat from core client for 30 sec - exiting 06:32:50 (5756): No heartbeat from core client for 30 sec - exiting 06:32:51 (5756): No heartbeat from core client for 30 sec - exiting 06:32:52 (5756): No heartbeat from core client for 30 sec - exiting 06:32:53 (5756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4896, iMonCtr=2 Model crash detected, will try to restart... 17:38:11 (5340): No heartbeat from core client for 30 sec - exiting 17:38:12 (5340): No heartbeat from core client for 30 sec - exiting 17:38:13 (5340): No heartbeat from core client for 30 sec - exiting 17:38:14 (5340): No heartbeat from core client for 30 sec - exiting 17:38:15 (5340): No heartbeat from core client for 30 sec - exiting 17:38:16 (5340): No heartbeat from core client for 30 sec - exiting 17:38:17 (5340): No heartbeat from core client for 30 sec - exiting 17:38:18 (5340): No heartbeat from core client for 30 sec - exiting 17:38:19 (5340): No heartbeat from core client for 30 sec - exiting 17:38:20 (5340): No heartbeat from core client for 30 sec - exiting 17:38:21 (5340): No heartbeat from core client for 30 sec - exiting 17:38:22 (5340): No heartbeat from core client for 30 sec - exiting 17:38:23 (5340): No heartbeat from core client for 30 sec - exiting 17:38:24 (5340): No heartbeat from core client for 30 sec - exiting 17:38:25 (5340): No heartbeat from core client for 30 sec - exiting 17:38:26 (5340): No heartbeat from core client for 30 sec - exiting 17:38:27 (5340): No heartbeat from core client for 30 sec - exiting 17:38:28 (5340): No heartbeat from core client for 30 sec - exiting 17:38:29 (5340): No heartbeat from core client for 30 sec - exiting 17:38:30 (5340): No heartbeat from core client for 30 sec - exiting 17:38:31 (5340): No heartbeat from core client for 30 sec - exiting 17:38:32 (5340): No heartbeat from core client for 30 sec - exiting 17:38:33 (5340): No heartbeat from core client for 30 sec - exiting 17:38:34 (5340): No heartbeat from core client for 30 sec - exiting 17:38:35 (5340): No heartbeat from core client for 30 sec - exiting 17:38:36 (5340): No heartbeat from core client for 30 sec - exiting 17:38:37 (5340): No heartbeat from core client for 30 sec - exiting 17:38:38 (5340): No heartbeat from core client for 30 sec - exiting 17:38:39 (5340): No heartbeat from core client for 30 sec - exiting 17:38:40 (5340): No heartbeat from core client for 30 sec - exiting 17:38:41 (5340): No heartbeat from core client for 30 sec - exiting 17:38:42 (5340): No heartbeat from core client for 30 sec - exiting 17:38:43 (5340): No heartbeat from core client for 30 sec - exiting 17:38:44 (5340): No heartbeat from core client for 30 sec - exiting 17:38:45 (5340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4476, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4148, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2960, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3640, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5968, selfPID=5576, iMonCtr=1 Model crash detected, will try to restart... 01:07:19 (5768): No heartbeat from core client for 30 sec - exiting 01:07:20 (5768): No heartbeat from core client for 30 sec - exiting 01:07:21 (5768): No heartbeat from core client for 30 sec - exiting 01:07:22 (5768): No heartbeat from core client for 30 sec - exiting 01:07:23 (5768): No heartbeat from core client for 30 sec - exiting 01:07:24 (5768): No heartbeat from core client for 30 sec - exiting 01:07:25 (5768): No heartbeat from core client for 30 sec - exiting 01:07:26 (5768): No heartbeat from core client for 30 sec - exiting 01:07:27 (5768): No heartbeat from core client for 30 sec - exiting 01:07:28 (5768): No heartbeat from core client for 30 sec - exiting 01:07:29 (5768): No heartbeat from core client for 30 sec - exiting 01:07:30 (5768): No heartbeat from core client for 30 sec - exiting 01:07:31 (5768): No heartbeat from core client for 30 sec - exiting 01:07:32 (5768): No heartbeat from core client for 30 sec - exiting 01:07:33 (5768): No heartbeat from core client for 30 sec - exiting 01:07:34 (5768): No heartbeat from core client for 30 sec - exiting 01:07:35 (5768): No heartbeat from core client for 30 sec - exiting 01:07:36 (5768): No heartbeat from core client for 30 sec - exiting 01:07:37 (5768): No heartbeat from core client for 30 sec - exiting 01:07:38 (5768): No heartbeat from core client for 30 sec - exiting 01:07:39 (5768): No heartbeat from core client for 30 sec - exiting 01:07:40 (5768): No heartbeat from core client for 30 sec - exiting 01:07:41 (5768): No heartbeat from core client for 30 sec - exiting 01:07:42 (5768): No heartbeat from core client for 30 sec - exiting 01:07:43 (5768): No heartbeat from core client for 30 sec - exiting 01:07:44 (5768): No heartbeat from core client for 30 sec - exiting 01:07:46 (5768): No heartbeat from core client for 30 sec - exiting 01:07:47 (5768): No heartbeat from core client for 30 sec - exiting 01:07:48 (5768): No heartbeat from core client for 30 sec - exiting 01:07:49 (5768): No heartbeat from core client for 30 sec - exiting 01:07:50 (5768): No heartbeat from core client for 30 sec - exiting 01:07:51 (5768): No heartbeat from core client for 30 sec - exiting 01:07:52 (5768): No heartbeat from core client for 30 sec - exiting 01:07:53 (5768): No heartbeat from core client for 30 sec - exiting 01:07:54 (5768): No heartbeat from core client for 30 sec - exiting 01:07:55 (5768): No heartbeat from core client for 30 sec - exiting 01:07:56 (5768): No heartbeat from core client for 30 sec - exiting 01:07:57 (5768): No heartbeat from core client for 30 sec - exiting 01:07:58 (5768): No heartbeat from core client for 30 sec - exiting 01:07:59 (5768): No heartbeat from core client for 30 sec - exiting 01:08:00 (5768): No heartbeat from core client for 30 sec - exiting 01:08:01 (5768): No heartbeat from core client for 30 sec - exiting 01:08:02 (5768): No heartbeat from core client for 30 sec - exiting 01:08:03 (5768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:50:15 (4532): No heartbeat from core client for 30 sec - exiting 01:50:16 (4532): No heartbeat from core client for 30 sec - exiting 01:50:17 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2248, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4668, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global WorkController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6468, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5540, iMonCtr=2 Model crash detected, will try to restart... 09:29:08 (6116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4852, selfPID=2756, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3380, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2584, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3200, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4720, selfPID=5584, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5460, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4548, selfPID=4284, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=248, selfPID=248, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2864, selfPID=2864, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2560, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2236, selfPID=3720, iMonCtr=1 Model crash detected, will try to restart... 18:36:13 (5508): No heartbeat from core client for 30 sec - exiting 18:36:14 (5508): No heartbeat from core client for 30 sec - exiting 18:36:15 (5508): No heartbeat from core client for 30 sec - exiting 18:36:16 (5508): No heartbeat from core client for 30 sec - exiting 18:36:17 (5508): No heartbeat from core client for 30 sec - exiting 18:36:18 (5508): No heartbeat from core client for 30 sec - exiting 18:36:19 (5508): No heartbeat from core client for 30 sec - exiting 18:36:20 (5508): No heartbeat from core client for 30 sec - exiting 18:36:21 (5508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:14:18 (4132): No heartbeat from core client for 30 sec - exiting 08:14:19 (4132): No heartbeat from core client for 30 sec - exiting 08:14:20 (4132): No heartbeat from core client for 30 sec - exiting 08:14:21 (4132): No heartbeat from core client for 30 sec - exiting 08:14:22 (4132): No heartbeat from core client for 30 sec - exiting 08:14:23 (4132): No heartbeat from core client for 30 sec - exiting 08:14:24 (4132): No heartbeat from core client for 30 sec - exiting 08:14:25 (4132): No heartbeat from core client for 30 sec - exiting 08:14:26 (4132): No heartbeat from core client for 30 sec - exiting 08:14:27 (4132): No heartbeat from core client for 30 sec - exiting 08:14:28 (4132): No heartbeat from core client for 30 sec - exiting 08:14:29 (4132): No heartbeat from core client for 30 sec - exiting 08:14:30 (4132): No heartbeat from core client for 30 sec - exiting 08:14:31 (4132): No heartbeat from core client for 30 sec - exiting 08:14:32 (4132): No heartbeat from core client for 30 sec - exiting 08:14:33 (4132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakg.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_saf_27gy_1995_1_007239984_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Sep 2011 11:14:40	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	126,816	894,060	7.0501
16 Aug 2011 13:57:41	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	115,304	815,188	7.0699
15 Aug 2011 00:28:49	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	115,296	813,877	7.0590
02 Aug 2011 14:11:51	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	103,787	733,387	7.0663
30 Jul 2011 02:08:24	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	103,776	732,146	7.0551
25 Jul 2011 14:05:10	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	92,256	650,572	7.0518
09 Jul 2011 05:09:00	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	80,736	570,434	7.0654
05 Jul 2011 23:47:41	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	69,216	491,400	7.0995
14 Jun 2011 22:24:34	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	57,696	408,847	7.0862
06 Jun 2011 13:28:35	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	46,176	327,990	7.1030
02 Jun 2011 22:43:36	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	34,656	248,147	7.1603
16 May 2011 22:19:44	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	23,136	168,109	7.2661
14 May 2011 10:51:21	941998	12857249	hadam3p_saf_27gy_1995_1_007239984_0	11,616	86,316	7.4308