Task 16007962

Name	hadcm3n_o7df_1980_40_008384389_2
Workunit	8535248
Created	8 Sep 2013, 9:04:51 UTC
Sent	8 Sep 2013, 9:44:08 UTC
Report deadline	8 Dec 2013, 17:11:19 UTC
Received	11 Feb 2014, 16:29:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	255 (0x000000FF) Unknown error code
Computer ID	1292123
Run time	18 days 4 hours 6 min 56 sec
CPU time	15 days 5 hours 30 min 32 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.51 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> Die erweiterten Attribute sind inkonsistent. (0xff) - exit code 255 (0xff) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2784, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4168, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:57:54 (2672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3836, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 19:24:14 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:18:50 (5568): No heartbeat from core client for 30 sec - exiting 18:18:51 (5568): No heartbeat from core client for 30 sec - exiting 18:18:52 (5568): No heartbeat from core client for 30 sec - exiting 18:18:53 (5568): No heartbeat from core client for 30 sec - exiting 18:18:54 (5568): No heartbeat from core client for 30 sec - exiting 18:18:56 (5568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:04 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5348, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4252, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4232, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1840, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3108, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... 17:09:29 (5664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1992, iMonCtr=1 Model crash detected, will try to restart... 21:52:40 (4408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:52:41 (4408): No heartbeat from core client for 30 sec - exiting 21:52:42 (4408): No heartbeat from core client for 30 sec - exiting 21:52:43 (4408): No heartbeat from core client for 30 sec - exiting 21:52:44 (4408): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4336, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4888, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2276, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3512, iMonCtr=1 Model crash detected, will try to restart... 12:26:07 (212): No heartbeat from core client for 30 sec - exiting 12:26:08 (212): No heartbeat from core client for 30 sec - exiting 12:26:09 (212): No heartbeat from core client for 30 sec - exiting 12:26:10 (212): No heartbeat from core client for 30 sec - exiting 12:26:11 (212): No heartbeat from core client for 30 sec - exiting 12:26:12 (212): No heartbeat from core client for 30 sec - exiting 12:26:13 (212): No heartbeat from core client for 30 sec - exiting 12:26:14 (212): No heartbeat from core client for 30 sec - exiting 12:26:16 (212): No heartbeat from core client for 30 sec - exiting 12:26:17 (212): No heartbeat from core client for 30 sec - exiting 12:26:18 (212): No heartbeat from core client for 30 sec - exiting 12:26:19 (212): No heartbeat from core client for 30 sec - exiting 12:26:20 (212): No heartbeat from core client for 30 sec - exiting 12:26:21 (212): No heartbeat from core client for 30 sec - exiting 12:26:22 (212): No heartbeat from core client for 30 sec - exiting 12:26:23 (212): No heartbeat from core client for 30 sec - exiting 12:26:24 (212): No heartbeat from core client for 30 sec - exiting 12:26:25 (212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3040, iMonCtr=1 Model crash detected, will try to restart... 16:45:57 (1520): No heartbeat from core client for 30 sec - exiting 16:45:59 (1520): No heartbeat from core client for 30 sec - exiting 16:46:00 (1520): No heartbeat from core client for 30 sec - exiting 16:46:01 (1520): No heartbeat from core client for 30 sec - exiting 16:46:02 (1520): No heartbeat from core client for 30 sec - exiting 16:46:03 (1520): No heartbeat from core client for 30 sec - exiting 16:46:04 (1520): No heartbeat from core client for 30 sec - exiting 16:46:05 (1520): No heartbeat from core client for 30 sec - exiting 16:46:06 (1520): No heartbeat from core client for 30 sec - exiting 16:46:07 (1520): No heartbeat from core client for 30 sec - exiting 16:46:08 (1520): No heartbeat from core client for 30 sec - exiting 16:46:10 (1520): No heartbeat from core client for 30 sec - exiting 16:46:11 (1520): No heartbeat from core client for 30 sec - exiting 16:46:12 (1520): No heartbeat from core client for 30 sec - exiting 16:46:13 (1520): No heartbeat from core client for 30 sec - exiting 16:46:14 (1520): No heartbeat from core client for 30 sec - exiting 16:46:15 (1520): No heartbeat from core client for 30 sec - exiting 16:46:16 (1520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:37:27 (4232): No heartbeat from core client for 30 sec - exiting 17:37:29 (4232): No heartbeat from core client for 30 sec - exiting 17:37:30 (4232): No heartbeat from core client for 30 sec - exiting 17:37:31 (4232): No heartbeat from core client for 30 sec - exiting 17:37:32 (4232): No heartbeat from core client for 30 sec - exiting 17:37:33 (4232): No heartbeat from core client for 30 sec - exiting 17:37:34 (4232): No heartbeat from core client for 30 sec - exiting 17:37:36 (4232): No heartbeat from core client for 30 sec - exiting 17:37:42 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... C21:13:08 (3020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4208, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5928, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4616, iMonCtr=1 Model crash detected, will try to restart... 20:35:25 (4324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:35:26 (4324): No heartbeat from core client for 30 sec - exiting 20:35:27 (4324): No heartbeat from core client for 30 sec - exiting 20:35:28 (4324): No heartbeat from core client for 30 sec - exiting 20:35:29 (4324): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4876, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5268, iMonCtr=1 Model crash detected, will try to restart... 20:40:51 (5940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:40:52 (5940): No heartbeat from core client for 30 sec - exiting 20:40:53 (5940): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1740, iMonCtr=1 Model crash detected, will try to restart... 18:40:25 (5168): No heartbeat from core client for 30 sec - exiting 18:40:26 (5168): No heartbeat from core client for 30 sec - exiting 18:40:27 (5168): No heartbeat from core client for 30 sec - exiting 18:40:29 (5168): No heartbeat from core client for 30 sec - exiting 18:40:30 (5168): No heartbeat from core client for 30 sec - exiting 18:40:31 (5168): No heartbeat from core client for 30 sec - exiting 18:40:32 (5168): No heartbeat from core client for 30 sec - exiting 18:40:33 (5168): No heartbeat from core client for 30 sec - exiting 18:40:34 (5168): No heartbeat from core client for 30 sec - exiting 18:40:35 (5168): No heartbeat from core client for 30 sec - exiting 18:40:36 (5168): No heartbeat from core client for 30 sec - exiting 18:40:37 (5168): No heartbeat from core client for 30 sec - exiting 18:40:38 (5168): No heartbeat from core client for 30 sec - exiting 18:40:39 (5168): No heartbeat from core client for 30 sec - exiting 18:40:40 (5168): No heartbeat from core client for 30 sec - exiting 18:40:42 (5168): No heartbeat from core client for 30 sec - exiting 18:40:43 (5168): No heartbeat from core client for 30 sec - exiting 18:40:44 (5168): No heartbeat from core client for 30 sec - exiting 18:40:45 (5168): No heartbeat from core client for 30 sec - exiting 18:40:46 (5168): No heartbeat from core client for 30 sec - exiting 18:40:47 (5168): No heartbeat from core client for 30 sec - exiting 18:40:48 (5168): No heartbeat from core client for 30 sec - exiting 18:40:49 (5168): No heartbeat from core client for 30 sec - exiting 18:40:50 (5168): No heartbeat from core client for 30 sec - exiting 18:40:51 (5168): No heartbeat from core client for 30 sec - exiting 18:40:52 (5168): No heartbeat from core client for 30 sec - exiting 18:40:54 (5168): No heartbeat from core client for 30 sec - exiting 18:40:55 (5168): No heartbeat from core client for 30 sec - exiting 18:40:56 (5168): No heartbeat from core client for 30 sec - exiting 18:40:57 (5168): No heartbeat from core client for 30 sec - exiting 18:40:58 (5168): No heartbeat from core client for 30 sec - exiting 18:40:59 (5168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5644, iMonCtr=1 Model crash detected, will try to restart... 13:14:12 (5848): No heartbeat from core client for 30 sec - exiting 13:14:13 (5848): No heartbeat from core client for 30 sec - exiting 13:14:14 (5848): No heartbeat from core client for 30 sec - exiting 13:14:16 (5848): No heartbeat from core client for 30 sec - exiting 13:14:17 (5848): No heartbeat from core client for 30 sec - exiting 13:14:18 (5848): No heartbeat from core client for 30 sec - exiting 13:14:19 (5848): No heartbeat from core client for 30 sec - exiting 13:14:20 (5848): No heartbeat from core client for 30 sec - exiting 13:14:21 (5848): No heartbeat from core client for 30 sec - exiting 13:14:22 (5848): No heartbeat from core client for 30 sec - exiting 13:14:23 (5848): No heartbeat from core client for 30 sec - exiting 13:14:24 (5848): No heartbeat from core client for 30 sec - exiting 13:14:25 (5848): No heartbeat from core client for 30 sec - exiting 13:14:26 (5848): No heartbeat from core client for 30 sec - exiting 13:14:28 (5848): No heartbeat from core client for 30 sec - exiting 13:14:29 (5848): No heartbeat from core client for 30 sec - exiting 13:14:30 (5848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:28 (1124): No heartbeat from core client for 30 sec - exiting 21:35:29 (1124): No heartbeat from core client for 30 sec - exiting 21:35:30 (1124): No heartbeat from core client for 30 sec - exiting 21:35:31 (1124): No heartbeat from core client for 30 sec - exiting 21:35:32 (1124): No heartbeat from core client for 30 sec - exiting 21:35:33 (1124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2636, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2636, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5392, iMonCtr=1 Model crash detected, will try to restart... 14:23:32 (3192): No heartbeat from core client for 30 sec - exiting 14:23:33 (3192): No heartbeat from core client for 30 sec - exiting 14:23:34 (3192): No heartbeat from core client for 30 sec - exiting 14:23:35 (3192): No heartbeat from core client for 30 sec - exiting 14:23:36 (3192): No heartbeat from core client for 30 sec - exiting 14:23:37 (3192): No heartbeat from core client for 30 sec - exiting 14:23:38 (3192): No heartbeat from core client for 30 sec - exiting 14:23:40 (3192): No heartbeat from core client for 30 sec - exiting 14:23:41 (3192): No heartbeat from core client for 30 sec - exiting 14:23:42 (3192): No heartbeat from core client for 30 sec - exiting 14:23:43 (3192): No heartbeat from core client for 30 sec - exiting 14:23:44 (3192): No heartbeat from core client for 30 sec - exiting 14:23:45 (3192): No heartbeat from core client for 30 sec - exiting 14:23:46 (3192): No heartbeat from core client for 30 sec - exiting 14:23:48 (3192): No heartbeat from core client for 30 sec - exiting 14:23:49 (3192): No heartbeat from core client for 30 sec - exiting 14:23:50 (3192): No heartbeat from core client for 30 sec - exiting 14:23:51 (3192): No heartbeat from core client for 30 sec - exiting 14:23:52 (3192): No heartbeat from core client for 30 sec - exiting 14:23:53 (3192): No heartbeat from core client for 30 sec - exiting 14:23:54 (3192): No heartbeat from core client for 30 sec - exiting 14:23:55 (3192): No heartbeat from core client for 30 sec - exiting 14:23:56 (3192): No heartbeat from core client for 30 sec - exiting 14:23:57 (3192): No heartbeat from core client for 30 sec - exiting 14:23:58 (3192): No heartbeat from core client for 30 sec - exiting 14:23:59 (3192): No heartbeat from core client for 30 sec - exiting 14:24:00 (3192): No heartbeat from core client for 30 sec - exiting 14:24:01 (3192): No heartbeat from core client for 30 sec - exiting 14:24:02 (3192): No heartbeat from core client for 30 sec - exiting 14:24:03 (3192): No heartbeat from core client for 30 sec - exiting 14:24:04 (3192): No heartbeat from core client for 30 sec - exiting 14:24:05 (3192): No heartbeat from core client for 30 sec - exiting 14:24:06 (3192): No heartbeat from core client for 30 sec - exiting 14:24:07 (3192): No heartbeat from core client for 30 sec - exiting 14:24:08 (3192): No heartbeat from core client for 30 sec - exiting 14:24:09 (3192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:06:27 (4816): No heartbeat from core client for 30 sec - exiting 17:06:28 (4816): No heartbeat from core client for 30 sec - exiting 17:06:29 (4816): No heartbeat from core client for 30 sec - exiting 17:06:30 (4816): No heartbeat from core client for 30 sec - exiting 17:06:31 (4816): No heartbeat from core client for 30 sec - exiting 17:06:32 (4816): No heartbeat from core client for 30 sec - exiting 17:06:33 (4816): No heartbeat from core client for 30 sec - exiting 17:06:34 (4816): No heartbeat from core client for 30 sec - exiting 17:06:35 (4816): No heartbeat from core client for 30 sec - exiting 17:06:36 (4816): No heartbeat from core client for 30 sec - exiting 17:06:37 (4816): No heartbeat from core client for 30 sec - exiting 17:06:38 (4816): No heartbeat from core client for 30 sec - exiting 17:06:39 (4816): No heartbeat from core client for 30 sec - exiting 17:06:40 (4816): No heartbeat from core client for 30 sec - exiting 17:06:41 (4816): No heartbeat from core client for 30 sec - exiting 17:06:42 (4816): No heartbeat from core client for 30 sec - exiting 17:06:43 (4816): No heartbeat from core client for 30 sec - exiting 17:06:44 (4816): No heartbeat from core client for 30 sec - exiting 17:06:45 (4816): No heartbeat from core client for 30 sec - exiting 17:06:46 (4816): No heartbeat from core client for 30 sec - exiting 17:06:47 (4816): No heartbeat from core client for 30 sec - exiting 17:06:48 (4816): No heartbeat from core client for 30 sec - exiting 17:06:49 (4816): No heartbeat from core client for 30 sec - exiting 17:06:50 (4816): No heartbeat from core client for 30 sec - exiting 17:06:51 (4816): No heartbeat from core client for 30 sec - exiting 17:06:52 (4816): No heartbeat from core client for 30 sec - exiting 17:06:53 (4816): No heartbeat from core client for 30 sec - exiting 17:06:54 (4816): No heartbeat from core client for 30 sec - exiting 17:06:56 (4816): No heartbeat from core client for 30 sec - exiting 17:06:57 (4816): No heartbeat from core client for 30 sec - exiting 17:06:58 (4816): No heartbeat from core client for 30 sec - exiting 17:06:59 (4816): No heartbeat from core client for 30 sec - exiting 17:07:00 (4816): No heartbeat from core client for 30 sec - exiting 17:07:01 (4816): No heartbeat from core client for 30 sec - exiting 17:07:02 (4816): No heartbeat from core client for 30 sec - exiting 17:07:03 (4816): No heartbeat from core client for 30 sec - exiting 17:07:04 (4816): No heartbeat from core client for 30 sec - exiting 17:07:05 (4816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:08:20 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4928, iMonCtr=1 Model crash detected, will try to restart... 17:26:02 (4824): No heartbeat from core client for 30 sec - exiting 17:26:03 (4824): No heartbeat from core client for 30 sec - exiting 17:26:04 (4824): No heartbeat from core client for 30 sec - exiting 17:26:05 (4824): No heartbeat from core client for 30 sec - exiting 17:26:06 (4824): No heartbeat from core client for 30 sec - exiting 17:26:07 (4824): No heartbeat from core client for 30 sec - exiting 17:26:09 (4824): No heartbeat from core client for 30 sec - exiting 17:26:10 (4824): No heartbeat from core client for 30 sec - exiting 17:26:12 (4824): No heartbeat from core client for 30 sec - exiting 17:26:13 (4824): No heartbeat from core client for 30 sec - exiting 17:26:15 (4824): No heartbeat from core client for 30 sec - exiting 17:26:23 (4824): No heartbeat from core client for 30 sec - exiting 17:26:25 (4824): No heartbeat from core client for 30 sec - exiting 17:26:27 (4824): No heartbeat from core client for 30 sec - exiting 17:26:30 (4824): No heartbeat from core client for 30 sec - exiting 17:26:33 (4824): No heartbeat from core client for 30 sec - exiting 17:26:35 (4824): No heartbeat from core client for 30 sec - exiting 17:26:36 (4824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6024, iMonCtr=1 Model crash detected, will try to restart... 21:31:43 (4012): No heartbeat from core client for 30 sec - exiting 21:31:49 (4012): No heartbeat from core client for 30 sec - exiting 21:31:50 (4012): No heartbeat from core client for 30 sec - exiting 21:31:51 (4012): No heartbeat from core client for 30 sec - exiting 21:31:52 (4012): No heartbeat from core client for 30 sec - exiting 21:31:53 (4012): No heartbeat from core client for 30 sec - exiting 21:31:54 (4012): No heartbeat from core client for 30 sec - exiting 21:31:55 (4012): No heartbeat from core client for 30 sec - exiting 21:31:56 (4012): No heartbeat from core client for 30 sec - exiting 21:31:57 (4012): No heartbeat from core client for 30 sec - exiting 21:31:58 (4012): No heartbeat from core client for 30 sec - exiting 21:31:59 (4012): No heartbeat from core client for 30 sec - exiting 21:32:00 (4012): No heartbeat from core client for 30 sec - exiting 21:32:01 (4012): No heartbeat from core client for 30 sec - exiting 21:32:02 (4012): No heartbeat from core client for 30 sec - exiting 21:32:03 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4016, iMonCtr=1 Model crash detected, will try to restart... 11:40:32 (5104): No heartbeat from core client for 30 sec - exiting 11:40:33 (5104): No heartbeat from core client for 30 sec - exiting 11:40:34 (5104): No heartbeat from core client for 30 sec - exiting 11:40:35 (5104): No heartbeat from core client for 30 sec - exiting 11:40:36 (5104): No heartbeat from core client for 30 sec - exiting 11:40:37 (5104): No heartbeat from core client for 30 sec - exiting 11:40:38 (5104): No heartbeat from core client for 30 sec - exiting 11:40:39 (5104): No heartbeat from core client for 30 sec - exiting 11:40:41 (5104): No heartbeat from core client for 30 sec - exiting 11:40:42 (5104): No heartbeat from core client for 30 sec - exiting 11:40:43 (5104): No heartbeat from core client for 30 sec - exiting 11:40:44 (5104): No heartbeat from core client for 30 sec - exiting 11:40:45 (5104): No heartbeat from core client for 30 sec - exiting 11:40:46 (5104): No heartbeat from core client for 30 sec - exiting 11:40:47 (5104): No heartbeat from core client for 30 sec - exiting 11:40:49 (5104): No heartbeat from core client for 30 sec - exiting 11:40:50 (5104): No heartbeat from core client for 30 sec - exiting 11:40:51 (5104): No heartbeat from core client for 30 sec - exiting 11:40:52 (5104): No heartbeat from core client for 30 sec - exiting 11:40:53 (5104): No heartbeat from core client for 30 sec - exiting 11:40:55 (5104): No heartbeat from core client for 30 sec - exiting 11:40:56 (5104): No heartbeat from core client for 30 sec - exiting 11:40:57 (5104): No heartbeat from core client for 30 sec - exiting 11:40:58 (5104): No heartbeat from core client for 30 sec - exiting 11:40:59 (5104): No heartbeat from core client for 30 sec - exiting 11:41:00 (5104): No heartbeat from core client for 30 sec - exiting 11:41:01 (5104): No heartbeat from core client for 30 sec - exiting 11:41:02 (5104): No heartbeat from core client for 30 sec - exiting 11:41:03 (5104): No heartbeat from core client for 30 sec - exiting 11:41:04 (5104): No heartbeat from core client for 30 sec - exiting 11:41:06 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4800, iMonCtr=1 Model crash detected, will try to restart... 13:39:30 (3284): No heartbeat from core client for 30 sec - exiting 13:39:31 (3284): No heartbeat from core client for 30 sec - exiting 13:39:32 (3284): No heartbeat from core client for 30 sec - exiting 13:39:33 (3284): No heartbeat from core client for 30 sec - exiting 13:39:34 (3284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:46:06 (4484): No heartbeat from core client for 30 sec - exiting 14:46:07 (4484): No heartbeat from core client for 30 sec - exiting 14:46:08 (4484): No heartbeat from core client for 30 sec - exiting 14:46:09 (4484): No heartbeat from core client for 30 sec - exiting 14:46:10 (4484): No heartbeat from core client for 30 sec - exiting 14:46:11 (4484): No heartbeat from core client for 30 sec - exiting 14:46:12 (4484): No heartbeat from core client for 30 sec - exiting 14:46:13 (4484): No heartbeat from core client for 30 sec - exiting 14:46:14 (4484): No heartbeat from core client for 30 sec - exiting 14:46:15 (4484): No heartbeat from core client for 30 sec - exiting 14:46:16 (4484): No heartbeat from core client for 30 sec - exiting 14:46:17 (4484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4104, iMonCtr=1 Model crash detected, will try to restart... 18:02:48 (4588): No heartbeat from core client for 30 sec - exiting 18:02:49 (4588): No heartbeat from core client for 30 sec - exiting 18:02:51 (4588): No heartbeat from core client for 30 sec - exiting 18:02:52 (4588): No heartbeat from core client for 30 sec - exiting 18:02:53 (4588): No heartbeat from core client for 30 sec - exiting 18:02:54 (4588): No heartbeat from core client for 30 sec - exiting 18:02:55 (4588): No heartbeat from core client for 30 sec - exiting 18:02:56 (4588): No heartbeat from core client for 30 sec - exiting 18:02:57 (4588): No heartbeat from core client for 30 sec - exiting 18:02:58 (4588): No heartbeat from core client for 30 sec - exiting 18:02:59 (4588): No heartbeat from core client for 30 sec - exiting 18:03:00 (4588): No heartbeat from core client for 30 sec - exiting 18:03:02 (4588): No heartbeat from core client for 30 sec - exiting 18:03:04 (4588): No heartbeat from core client for 30 sec - exiting 18:03:05 (4588): No heartbeat from core client for 30 sec - exiting 18:03:06 (4588): No heartbeat from core client for 30 sec - exiting 18:03:07 (4588): No heartbeat from core client for 30 sec - exiting 18:03:08 (4588): No heartbeat from core client for 30 sec - exiting 18:03:09 (4588): No heartbeat from core client for 30 sec - exiting 18:03:10 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5356, iMonCtr=1 Model crash detected, will try to restart... 18:42:12 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:12:17 (5368): No heartbeat from core client for 30 sec - exiting 14:12:18 (5368): No heartbeat from core client for 30 sec - exiting 14:12:19 (5368): No heartbeat from core client for 30 sec - exiting 14:12:20 (5368): No heartbeat from core client for 30 sec - exiting 14:12:22 (5368): No heartbeat from core client for 30 sec - exiting 14:12:23 (5368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:12:24 (5368): No heartbeat from core client for 30 sec - exiting 18:12:38 (5428): No heartbeat from core client for 30 sec - exiting 18:12:40 (5428): No heartbeat from core client for 30 sec - exiting 18:12:41 (5428): No heartbeat from core client for 30 sec - exiting 18:12:42 (5428): No heartbeat from core client for 30 sec - exiting 18:12:44 (5428): No heartbeat from core client for 30 sec - exiting 18:12:45 (5428): No heartbeat from core client for 30 sec - exiting 18:12:46 (5428): No heartbeat from core client for 30 sec - exiting 18:12:47 (5428): No heartbeat from core client for 30 sec - exiting 18:12:48 (5428): No heartbeat from core client for 30 sec - exiting 18:12:53 (5428): No heartbeat from core client for 30 sec - exiting 18:12:59 (5428): No heartbeat from core client for 30 sec - exiting 18:13:00 (5428): No heartbeat from core client for 30 sec - exiting 18:13:01 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5312, iMonCtr=1 Model crash detected, will try to restart... 18:37:39 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:32:14 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:35:22 (4916): No heartbeat from core client for 30 sec - exiting 16:35:23 (4916): No heartbeat from core client for 30 sec - exiting 16:35:24 (4916): No heartbeat from core client for 30 sec - exiting 16:35:25 (4916): No heartbeat from core client for 30 sec - exiting 16:35:26 (4916): No heartbeat from core client for 30 sec - exiting 16:35:27 (4916): No heartbeat from core client for 30 sec - exiting 16:35:28 (4916): No heartbeat from core client for 30 sec - exiting 16:35:29 (4916): No heartbeat from core client for 30 sec - exiting 16:35:30 (4916): No heartbeat from core client for 30 sec - exiting 16:35:32 (4916): No heartbeat from core client for 30 sec - exiting 16:35:33 (4916): No heartbeat from core client for 30 sec - exiting 16:35:34 (4916): No heartbeat from core client for 30 sec - exiting 16:35:35 (4916): No heartbeat from core client for 30 sec - exiting 16:35:36 (4916): No heartbeat from core client for 30 sec - exiting 16:35:37 (4916): No heartbeat from core client for 30 sec - exiting 16:35:39 (4916): No heartbeat from core client for 30 sec - exiting 16:35:40 (4916): No heartbeat from core client for 30 sec - exiting 16:35:41 (4916): No heartbeat from core client for 30 sec - exiting 16:35:42 (4916): No heartbeat from core client for 30 sec - exiting 16:35:43 (4916): No heartbeat from core client for 30 sec - exiting 16:35:44 (4916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:31:42 (4664): No heartbeat from core client for 30 sec - exiting 18:31:43 (4664): No heartbeat from core client for 30 sec - exiting 18:31:44 (4664): No heartbeat from core client for 30 sec - exiting 18:31:45 (4664): No heartbeat from core client for 30 sec - exiting 18:31:46 (4664): No heartbeat from core client for 30 sec - exiting 18:31:47 (4664): No heartbeat from core client for 30 sec - exiting 18:31:48 (4664): No heartbeat from core client for 30 sec - exiting 18:31:49 (4664): No heartbeat from core client for 30 sec - exiting 18:31:50 (4664): No heartbeat from core client for 30 sec - exiting 18:31:51 (4664): No heartbeat from core client for 30 sec - exiting 18:31:52 (4664): No heartbeat from core client for 30 sec - exiting 18:31:53 (4664): No heartbeat from core client for 30 sec - exiting 18:31:54 (4664): No heartbeat from core client for 30 sec - exiting 18:31:55 (4664): No heartbeat from core client for 30 sec - exiting 18:31:56 (4664): No heartbeat from core client for 30 sec - exiting 18:31:57 (4664): No heartbeat from core client for 30 sec - exiting 18:31:58 (4664): No heartbeat from core client for 30 sec - exiting 18:31:59 (4664): No heartbeat from core client for 30 sec - exiting 18:32:00 (4664): No heartbeat from core client for 30 sec - exiting 18:32:01 (4664): No heartbeat from core client for 30 sec - exiting 18:32:02 (4664): No heartbeat from core client for 30 sec - exiting 18:32:03 (4664): No heartbeat from core client for 30 sec - exiting 18:32:04 (4664): No heartbeat from core client for 30 sec - exiting 18:32:05 (4664): No heartbeat from core client for 30 sec - exiting 18:32:06 (4664): No heartbeat from core client for 30 sec - exiting 18:32:07 (4664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:44:06 (5784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:46:58 (3928): No heartbeat from core client for 30 sec - exiting 18:47:00 (3928): No heartbeat from core client for 30 sec - exiting 18:47:01 (3928): No heartbeat from core client for 30 sec - exiting 18:47:02 (3928): No heartbeat from core client for 30 sec - exiting 18:47:03 (3928): No heartbeat from core client for 30 sec - exiting 18:47:04 (3928): No heartbeat from core client for 30 sec - exiting 18:47:05 (3928): No heartbeat from core client for 30 sec - exiting 18:47:06 (3928): No heartbeat from core client for 30 sec - exiting 18:47:07 (3928): No heartbeat from core client for 30 sec - exiting 18:47:08 (3928): No heartbeat from core client for 30 sec - exiting 18:47:09 (3928): No heartbeat from core client for 30 sec - exiting 18:47:10 (3928): No heartbeat from core client for 30 sec - exiting 18:47:12 (3928): No heartbeat from core client for 30 sec - exiting 18:47:13 (3928): No heartbeat from core client for 30 sec - exiting 18:47:14 (3928): No heartbeat from core client for 30 sec - exiting 18:47:15 (3928): No heartbeat from core client for 30 sec - exiting 18:47:16 (3928): No heartbeat from core client for 30 sec - exiting 18:47:17 (3928): No heartbeat from core client for 30 sec - exiting 18:47:18 (3928): No heartbeat from core client for 30 sec - exiting 18:47:19 (3928): No heartbeat from core client for 30 sec - exiting 18:47:20 (3928): No heartbeat from core client for 30 sec - exiting 18:47:21 (3928): No heartbeat from core client for 30 sec - exiting 18:47:23 (3928): No heartbeat from core client for 30 sec - exiting 18:47:24 (3928): No heartbeat from core client for 30 sec - exiting 18:47:25 (3928): No heartbeat from core client for 30 sec - exiting 18:47:26 (3928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:04:34 (4992): No heartbeat from core client for 30 sec - exiting 19:04:35 (4992): No heartbeat from core client for 30 sec - exiting 19:04:36 (4992): No heartbeat from core client for 30 sec - exiting 19:04:37 (4992): No heartbeat from core client for 30 sec - exiting 19:04:39 (4992): No heartbeat from core client for 30 sec - exiting 19:04:40 (4992): No heartbeat from core client for 30 sec - exiting 19:04:41 (4992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:17 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:20 (2812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:09:42 (1316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:18:59 (5100): No heartbeat from core client for 30 sec - exiting 19:19:00 (5100): No heartbeat from core client for 30 sec - exiting 19:19:01 (5100): No heartbeat from core client for 30 sec - exiting 19:19:02 (5100): No heartbeat from core client for 30 sec - exiting 19:19:03 (5100): No heartbeat from core client for 30 sec - exiting 19:19:04 (5100): No heartbeat from core client for 30 sec - exiting 19:19:05 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:08:09 (5948): No heartbeat from core client for 30 sec - exiting 14:08:11 (5948): No heartbeat from core client for 30 sec - exiting 14:08:12 (5948): No heartbeat from core client for 30 sec - exiting 14:08:13 (5948): No heartbeat from core client for 30 sec - exiting 14:08:14 (5948): No heartbeat from core client for 30 sec - exiting 14:08:15 (5948): No heartbeat from core client for 30 sec - exiting 14:08:16 (5948): No heartbeat from core client for 30 sec - exiting 14:08:17 (5948): No heartbeat from core client for 30 sec - exiting 14:08:20 (5948): No heartbeat from core client for 30 sec - exiting 14:08:22 (5948): No heartbeat from core client for 30 sec - exiting 14:08:23 (5948): No heartbeat from core client for 30 sec - exiting 14:08:24 (5948): No heartbeat from core client for 30 sec - exiting 14:08:27 (5948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:58:56 (4868): No heartbeat from core client for 30 sec - exiting 20:58:57 (4868): No heartbeat from core client for 30 sec - exiting 20:58:58 (4868): No heartbeat from core client for 30 sec - exiting 20:58:59 (4868): No heartbeat from core client for 30 sec - exiting 20:59:00 (4868): No heartbeat from core client for 30 sec - exiting 20:59:01 (4868): No heartbeat from core client for 30 sec - exiting 20:59:02 (4868): No heartbeat from core client for 30 sec - exiting 20:59:03 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:53:24 (2500): No heartbeat from core client for 30 sec - exiting 13:53:25 (2500): No heartbeat from core client for 30 sec - exiting 13:53:26 (2500): No heartbeat from core client for 30 sec - exiting 13:53:27 (2500): No heartbeat from core client for 30 sec - exiting 13:53:29 (2500): No heartbeat from core client for 30 sec - exiting 13:53:30 (2500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:11:09 (2888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:11:10 (2888): No heartbeat from core client for 30 sec - exiting 13:21:58 (4708): No heartbeat from core client for 30 sec - exiting 13:21:59 (4708): No heartbeat from core client for 30 sec - exiting 13:22:00 (4708): No heartbeat from core client for 30 sec - exiting 13:22:01 (4708): No heartbeat from core client for 30 sec - exiting 13:22:02 (4708): No heartbeat from core client for 30 sec - exiting 13:22:03 (4708): No heartbeat from core client for 30 sec - exiting 13:22:05 (4708): No heartbeat from core client for 30 sec - exiting 13:22:06 (4708): No heartbeat from core client for 30 sec - exiting 13:22:07 (4708): No heartbeat from core client for 30 sec - exiting 13:22:08 (4708): No heartbeat from core client for 30 sec - exiting 13:22:09 (4708): No heartbeat from core client for 30 sec - exiting 13:22:10 (4708): No heartbeat from core client for 30 sec - exiting 13:22:11 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:25 (2196): No heartbeat from core client for 30 sec - exiting 16:10:27 (2196): No heartbeat from core client for 30 sec - exiting 16:10:28 (2196): No heartbeat from core client for 30 sec - exiting 16:10:29 (2196): No heartbeat from core client for 30 sec - exiting 16:10:30 (2196): No heartbeat from core client for 30 sec - exiting 16:10:31 (2196): No heartbeat from core client for 30 sec - exiting 16:10:32 (2196): No heartbeat from core client for 30 sec - exiting 16:10:33 (2196): No heartbeat from core client for 30 sec - exiting 16:10:34 (2196): No heartbeat from core client for 30 sec - exiting 16:10:35 (2196): No heartbeat from core client for 30 sec - exiting 16:10:37 (2196): No heartbeat from core client for 30 sec - exiting 16:10:38 (2196): No heartbeat from core client for 30 sec - exiting 16:10:39 (2196): No heartbeat from core client for 30 sec - exiting 16:10:40 (2196): No heartbeat from core client for 30 sec - exiting 16:10:41 (2196): No heartbeat from core client for 30 sec - exiting 16:10:42 (2196): No heartbeat from core client for 30 sec - exiting 16:10:43 (2196): No heartbeat from core client for 30 sec - exiting 16:10:44 (2196): No heartbeat from core client for 30 sec - exiting 16:10:45 (2196): No heartbeat from core client for 30 sec - exiting 16:10:46 (2196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3784, iMonCtr=1 Model crash detected, will try to restart... 14:07:09 (4616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:08:38 (1532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:16:07 (5824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4696, iMonCtr=1 Model crash detected, will try to restart... 14:04:54 (3724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:30 (5176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:31 (5176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5324, iMonCtr=1 Model crash detected, will try to restart... 13:16:52 (5264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4284, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o7dfko.pjj8c10 Error converting file to netcdf: dataout/o7dfko.pij8c10 Error converting file to netcdf: dataout/o7dfko.pfj8c10 Error converting file to netcdf: dataout/o7dfka.phj8c10 Error converting file to netcdf: dataout/o7dfka.pgj8c10 Error converting file to netcdf: dataout/o7dfka.pej8c10 Error converting file to netcdf: dataout/o7dfka.pdj8c10 21:19:24 (5344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4964, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5732, iMonCtr=1 Model crash detected, will try to restart... 17:15:26 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4980, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4980, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4980, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4980, iMonCtr=1 Model crash detected, will try to restart... 22:44:38 (5360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=868, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77906E9F read attempt to address 0x409F4C88 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Feb 2014 16:32:12	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	518,400	1,315,813	2.5382
10 Feb 2014 11:32:41	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	492,480	1,253,593	2.5455
04 Feb 2014 22:08:57	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	466,560	1,191,770	2.5544
01 Feb 2014 20:56:18	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	440,640	1,127,317	2.5584
25 Jan 2014 14:58:32	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	414,720	1,062,166	2.5612
24 Jan 2014 10:34:49	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	388,800	1,001,188	2.5751
05 Jan 2014 20:35:09	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	362,880	937,390	2.5832
18 Dec 2013 23:12:30	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	336,960	873,110	2.5911
27 Nov 2013 21:54:50	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	311,040	807,412	2.5958
16 Nov 2013 18:15:28	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	285,120	739,946	2.5952
10 Nov 2013 19:18:02	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	259,200	673,961	2.6002
03 Nov 2013 22:00:13	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	233,280	608,272	2.6075
28 Oct 2013 19:46:03	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	207,360	541,246	2.6102
20 Oct 2013 17:22:22	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	181,440	473,028	2.6071
14 Oct 2013 14:21:05	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	155,520	407,385	2.6195
09 Oct 2013 19:13:06	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	129,600	340,417	2.6267
03 Oct 2013 16:05:38	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	103,680	270,069	2.6048
29 Sep 2013 15:55:54	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	77,760	201,530	2.5917
23 Sep 2013 16:20:15	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	51,840	132,690	2.5596
12 Sep 2013 19:53:02	1292123	16007962	hadcm3n_o7df_1980_40_008384389_2	25,920	65,969	2.5451