Task 16013278

Name	hadcm3n_4918_1980_40_008363565_2
Workunit	8514424
Created	12 Sep 2013, 11:23:02 UTC
Sent	12 Sep 2013, 11:54:55 UTC
Report deadline	12 Dec 2013, 19:22:06 UTC
Received	28 Dec 2013, 19:36:00 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1266922
Run time	40 days 18 hours 54 min 44 sec
CPU time	36 days 2 hours 44 min 39 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	1.39 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1088, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3580, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3532, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:13:58 (5100): No heartbeat from core client for 30 sec - exiting 14:13:59 (5100): No heartbeat from core client for 30 sec - exiting 14:14:00 (5100): No heartbeat from core client for 30 sec - exiting 14:14:01 (5100): No heartbeat from core client for 30 sec - exiting 14:14:02 (5100): No heartbeat from core client for 30 sec - exiting 14:14:04 (5100): No heartbeat from core client for 30 sec - exiting 14:14:05 (5100): No heartbeat from core client for 30 sec - exiting 14:14:06 (5100): No heartbeat from core client for 30 sec - exiting 14:14:07 (5100): No heartbeat from core client for 30 sec - exiting 14:14:08 (5100): No heartbeat from core client for 30 sec - exiting 14:14:09 (5100): No heartbeat from core client for 30 sec - exiting 14:14:10 (5100): No heartbeat from core client for 30 sec - exiting 14:14:11 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1956, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=356, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:27:27 (4592): No heartbeat from core client for 30 sec - exiting 22:27:28 (4592): No heartbeat from core client for 30 sec - exiting 22:27:29 (4592): No heartbeat from core client for 30 sec - exiting 22:27:30 (4592): No heartbeat from core client for 30 sec - exiting 22:27:31 (4592): No heartbeat from core client for 30 sec - exiting 22:27:32 (4592): No heartbeat from core client for 30 sec - exiting 22:27:33 (4592): No heartbeat from core client for 30 sec - exiting 22:27:34 (4592): No heartbeat from core client for 30 sec - exiting 22:27:36 (4592): No heartbeat from core client for 30 sec - exiting 22:27:37 (4592): No heartbeat from core client for 30 sec - exiting 22:27:38 (4592): No heartbeat from core client for 30 sec - exiting 22:27:39 (4592): No heartbeat from core client for 30 sec - exiting 22:27:40 (4592): No heartbeat from core client for 30 sec - exiting 22:27:41 (4592): No heartbeat from core client for 30 sec - exiting 22:27:42 (4592): No heartbeat from core client for 30 sec - exiting 22:27:43 (4592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3524, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4120, iMonCtr=1 Model crash detected, will try to restart... 16:31:32 (936): No heartbeat from core client for 30 sec - exiting 16:31:33 (936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5048, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3580, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5000, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4688, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4924, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4704, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4720, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4956, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4852, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4788, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4732, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4808, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4512, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4848, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4672, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3428, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4688, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4900, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4024, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3424, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3208, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4468, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3828, iMonCtr=1 Model crash detected, will try to restart... 21:06:30 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1376, iMonCtr=1 Model crash detected, will try to restart... 17:28:00 (2316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... 21:29:41 (5208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:38:41 (5024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5584, iMonCtr=1 Model crash detected, will try to restart... 17:49:13 (1244): No heartbeat from core client for 30 sec - exiting 17:49:14 (1244): No heartbeat from core client for 30 sec - exiting 17:49:15 (1244): No heartbeat from core client for 30 sec - exiting 17:49:17 (1244): No heartbeat from core client for 30 sec - exiting 17:49:18 (1244): No heartbeat from core client for 30 sec - exiting 17:49:19 (1244): No heartbeat from core client for 30 sec - exiting 17:49:20 (1244): No heartbeat from core client for 30 sec - exiting 17:49:21 (1244): No heartbeat from core client for 30 sec - exiting 17:49:22 (1244): No heartbeat from core client for 30 sec - exiting 17:49:23 (1244): No heartbeat from core client for 30 sec - exiting 17:49:24 (1244): No heartbeat from core client for 30 sec - exiting 17:49:25 (1244): No heartbeat from core client for 30 sec - exiting 17:49:26 (1244): No heartbeat from core client for 30 sec - exiting 17:49:27 (1244): No heartbeat from core client for 30 sec - exiting 17:49:29 (1244): No heartbeat from core client for 30 sec - exiting 17:49:30 (1244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1208, iMonCtr=1 Model crash detected, will try to restart... 13:36:15 (4888): No heartbeat from core client for 30 sec - exiting 13:36:16 (4888): No heartbeat from core client for 30 sec - exiting 13:36:17 (4888): No heartbeat from core client for 30 sec - exiting 13:36:18 (4888): No heartbeat from core client for 30 sec - exiting 13:36:19 (4888): No heartbeat from core client for 30 sec - exiting 13:36:20 (4888): No heartbeat from core client for 30 sec - exiting 13:36:21 (4888): No heartbeat from core client for 30 sec - exiting 13:36:22 (4888): No heartbeat from core client for 30 sec - exiting 13:36:23 (4888): No heartbeat from core client for 30 sec - exiting 13:36:24 (4888): No heartbeat from core client for 30 sec - exiting 13:36:26 (4888): No heartbeat from core client for 30 sec - exiting 13:36:27 (4888): No heartbeat from core client for 30 sec - exiting 13:36:28 (4888): No heartbeat from core client for 30 sec - exiting 13:36:29 (4888): No heartbeat from core client for 30 sec - exiting 13:36:30 (4888): No heartbeat from core client for 30 sec - exiting 13:36:31 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:40:08 (4728): No heartbeat from core client for 30 sec - exiting 10:40:09 (4728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:32:55 (3700): No heartbeat from core client for 30 sec - exiting 12:32:56 (3700): No heartbeat from core client for 30 sec - exiting 12:32:57 (3700): No heartbeat from core client for 30 sec - exiting 12:32:58 (3700): No heartbeat from core client for 30 sec - exiting 12:33:00 (3700): No heartbeat from core client for 30 sec - exiting 12:33:01 (3700): No heartbeat from core client for 30 sec - exiting 12:33:02 (3700): No heartbeat from core client for 30 sec - exiting 12:33:03 (3700): No heartbeat from core client for 30 sec - exiting 12:33:04 (3700): No heartbeat from core client for 30 sec - exiting 12:33:05 (3700): No heartbeat from core client for 30 sec - exiting 12:33:06 (3700): No heartbeat from core client for 30 sec - exiting 12:33:07 (3700): No heartbeat from core client for 30 sec - exiting 12:33:08 (3700): No heartbeat from core client for 30 sec - exiting 12:33:09 (3700): No heartbeat from core client for 30 sec - exiting 12:33:10 (3700): No heartbeat from core client for 30 sec - exiting 12:33:12 (3700): No heartbeat from core client for 30 sec - exiting 12:33:13 (3700): No heartbeat from core client for 30 sec - exiting 12:33:14 (3700): No heartbeat from core client for 30 sec - exiting 12:33:15 (3700): No heartbeat from core client for 30 sec - exiting 12:33:16 (3700): No heartbeat from core client for 30 sec - exiting 12:33:17 (3700): No heartbeat from core client for 30 sec - exiting 12:33:18 (3700): No heartbeat from core client for 30 sec - exiting 12:33:19 (3700): No heartbeat from core client for 30 sec - exiting 12:33:20 (3700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9644, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4112, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4936, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4456, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 23:57:28 (4956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4896, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3332, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4652, iMonCtr=1 Model crash detected, will try to restart... Ocean Restart file copy failed on 4918ko.dal0c20 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5032, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4996, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4948, iMonCtr=1 Model crash detected, will try to restart... 01:20:50 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4668, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4976, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... 14:24:13 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=620, iMonCtr=1 Model crash detected, will try to restart... 14:44:44 (4924): No heartbeat from core client for 30 sec - exiting 14:44:45 (4924): No heartbeat from core client for 30 sec - exiting 14:44:46 (4924): No heartbeat from core client for 30 sec - exiting 14:44:47 (4924): No heartbeat from core client for 30 sec - exiting 14:44:48 (4924): No heartbeat from core client for 30 sec - exiting 14:44:49 (4924): No heartbeat from core client for 30 sec - exiting 14:44:51 (4924): No heartbeat from core client for 30 sec - exiting 14:44:52 (4924): No heartbeat from core client for 30 sec - exiting 14:44:53 (4924): No heartbeat from core client for 30 sec - exiting 14:44:54 (4924): No heartbeat from core client for 30 sec - exiting 14:44:55 (4924): No heartbeat from core client for 30 sec - exiting 14:44:56 (4924): No heartbeat from core client for 30 sec - exiting 14:44:57 (4924): No heartbeat from core client for 30 sec - exiting 14:44:58 (4924): No heartbeat from core client for 30 sec - exiting 14:44:59 (4924): No heartbeat from core client for 30 sec - exiting 14:45:00 (4924): No heartbeat from core client for 30 sec - exiting 14:45:01 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4884, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5048, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Dec 2013 18:39:08	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	1,036,800	3,119,124	3.0084
26 Dec 2013 06:13:13	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	1,010,880	3,056,650	3.0238
24 Dec 2013 07:43:23	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	984,960	2,987,527	3.0331
22 Dec 2013 07:29:09	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	959,040	2,906,152	3.0303
21 Dec 2013 15:33:55	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	933,120	2,850,388	3.0547
19 Dec 2013 14:04:56	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	907,200	2,781,809	3.0664
17 Dec 2013 07:25:55	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	881,280	2,699,874	3.0636
14 Dec 2013 23:23:48	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	855,360	2,617,332	3.0599
13 Dec 2013 12:34:17	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	829,440	2,533,040	3.0539
09 Dec 2013 12:34:45	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	803,520	2,448,906	3.0477
07 Dec 2013 16:51:13	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	777,600	2,365,084	3.0415
04 Dec 2013 09:10:17	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	751,680	2,283,174	3.0374
01 Dec 2013 05:22:48	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	725,760	2,209,132	3.0439
30 Nov 2013 06:24:18	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	699,840	2,132,832	3.0476
26 Nov 2013 04:59:18	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	673,920	2,056,337	3.0513
24 Nov 2013 11:42:22	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	648,000	1,994,801	3.0784
23 Nov 2013 15:14:42	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	622,080	1,924,836	3.0942
21 Nov 2013 12:50:00	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	596,160	1,838,998	3.0847
17 Nov 2013 10:08:34	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	570,240	1,757,989	3.0829
16 Nov 2013 09:35:34	1266922	16013278	hadcm3n_4918_1980_40_008363565_2	544,320	1,680,478	3.0873