Task 16064360

Name	hadcm3n_n0pz_1920_40_008360931_4
Workunit	8511790
Created	10 Oct 2013, 13:03:54 UTC
Sent	10 Oct 2013, 13:06:24 UTC
Report deadline	9 Jan 2014, 20:33:35 UTC
Received	25 Oct 2013, 12:40:24 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1123998
Run time	9 days 14 hours 27 min 57 sec
CPU time	9 days 5 hours 23 min 22 sec
Validate state	Invalid
Credit	5,598.72
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... 00:24:15 (11364): No heartbeat from core client for 30 sec - exiting 00:24:18 (11364): No heartbeat from core client for 30 sec - exiting 00:24:19 (11364): No heartbeat from core client for 30 sec - exiting 00:24:20 (11364): No heartbeat from core client for 30 sec - exiting 00:24:21 (11364): No heartbeat from core client for 30 sec - exiting 00:24:22 (11364): No heartbeat from core client for 30 sec - exiting 00:24:23 (11364): No heartbeat from core client for 30 sec - exiting 00:24:24 (11364): No heartbeat from core client for 30 sec - exiting 00:24:25 (11364): No heartbeat from core client for 30 sec - exiting 00:24:26 (11364): No heartbeat from core client for 30 sec - exiting 00:24:27 (11364): No heartbeat from core client for 30 sec - exiting 00:24:28 (11364): No heartbeat from core client for 30 sec - exiting 00:24:29 (11364): No heartbeat from core client for 30 sec - exiting 00:24:30 (11364): No heartbeat from core client for 30 sec - exiting 00:24:31 (11364): No heartbeat from core client for 30 sec - exiting 00:24:32 (11364): No heartbeat from core client for 30 sec - exiting 00:24:33 (11364): No heartbeat from core client for 30 sec - exiting 00:24:34 (11364): No heartbeat from core client for 30 sec - exiting 00:24:35 (11364): No heartbeat from core client for 30 sec - exiting 00:24:36 (11364): No heartbeat from core client for 30 sec - exiting 00:24:37 (11364): No heartbeat from core client for 30 sec - exiting 00:24:38 (11364): No heartbeat from core client for 30 sec - exiting 00:24:39 (11364): No heartbeat from core client for 30 sec - exiting 00:24:40 (11364): No heartbeat from core client for 30 sec - exiting 00:24:41 (11364): No heartbeat from core client for 30 sec - exiting 00:24:42 (11364): No heartbeat from core client for 30 sec - exiting 00:24:43 (11364): No heartbeat from core client for 30 sec - exiting 00:24:44 (11364): No heartbeat from core client for 30 sec - exiting 00:24:45 (11364): No heartbeat from core client for 30 sec - exiting 00:24:46 (11364): No heartbeat from core client for 30 sec - exiting 00:24:47 (11364): No heartbeat from core client for 30 sec - exiting 00:24:48 (11364): No heartbeat from core client for 30 sec - exiting 00:24:49 (11364): No heartbeat from core client for 30 sec - exiting 00:24:50 (11364): No heartbeat from core client for 30 sec - exiting 00:24:51 (11364): No heartbeat from core client for 30 sec - exiting 00:24:52 (11364): No heartbeat from core client for 30 sec - exiting 00:24:53 (11364): No heartbeat from core client for 30 sec - exiting 00:24:54 (11364): No heartbeat from core client for 30 sec - exiting 00:25:25 (11364): No heartbeat from core client for 30 sec - exiting 00:25:26 (11364): No heartbeat from core client for 30 sec - exiting 00:25:27 (11364): No heartbeat from core client for 30 sec - exiting 00:25:28 (11364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:26:58 (7376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:27:46 (7940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:54 (12052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:30:16 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:30:17 (6064): No heartbeat from core client for 30 sec - exiting 00:30:18 (6064): No heartbeat from core client for 30 sec - exiting 00:31:28 (14156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:32:20 (8136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:39:49 (10756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:39:55 (10756): No heartbeat from core client for 30 sec - exiting 00:39:56 (10756): No heartbeat from core client for 30 sec - exiting 00:39:57 (10756): No heartbeat from core client for 30 sec - exiting 00:40:35 (10400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:41:59 (4944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:43:19 (5936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:44:58 (11872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:45:51 (2092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:46:34 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:53:44 (10108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:59:54 (13392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:44:42 (1944): No heartbeat from core client for 30 sec - exiting 07:44:45 (1944): No heartbeat from core client for 30 sec - exiting 07:44:46 (1944): No heartbeat from core client for 30 sec - exiting 07:44:48 (1944): No heartbeat from core client for 30 sec - exiting 07:45:24 (1944): No heartbeat from core client for 30 sec - exiting 07:45:25 (1944): No heartbeat from core client for 30 sec - exiting 07:45:26 (1944): No heartbeat from core client for 30 sec - exiting 07:45:27 (1944): No heartbeat from core client for 30 sec - exiting 07:45:28 (1944): No heartbeat from core client for 30 sec - exiting 07:45:29 (1944): No heartbeat from core client for 30 sec - exiting 07:45:31 (1944): No heartbeat from core client for 30 sec - exiting 07:45:32 (1944): No heartbeat from core client for 30 sec - exiting 07:45:33 (1944): No heartbeat from core client for 30 sec - exiting 07:45:34 (1944): No heartbeat from core client for 30 sec - exiting 07:45:35 (1944): No heartbeat from core client for 30 sec - exiting 07:45:36 (1944): No heartbeat from core client for 30 sec - exiting 07:45:37 (1944): No heartbeat from core client for 30 sec - exiting 07:45:38 (1944): No heartbeat from core client for 30 sec - exiting 07:45:39 (1944): No heartbeat from core client for 30 sec - exiting 07:45:40 (1944): No heartbeat from core client for 30 sec - exiting 07:45:41 (1944): No heartbeat from core client for 30 sec - exiting 07:45:43 (1944): No heartbeat from core client for 30 sec - exiting 07:45:44 (1944): No heartbeat from core client for 30 sec - exiting 07:45:45 (1944): No heartbeat from core client for 30 sec - exiting 07:45:46 (1944): No heartbeat from core client for 30 sec - exiting 07:45:47 (1944): No heartbeat from core client for 30 sec - exiting 07:45:48 (1944): No heartbeat from core client for 30 sec - exiting 07:45:49 (1944): No heartbeat from core client for 30 sec - exiting 07:45:50 (1944): No heartbeat from core client for 30 sec - exiting 07:45:52 (1944): No heartbeat from core client for 30 sec - exiting 07:45:54 (1944): No heartbeat from core client for 30 sec - exiting 07:45:55 (1944): No heartbeat from core client for 30 sec - exiting 07:45:56 (1944): No heartbeat from core client for 30 sec - exiting 07:45:57 (1944): No heartbeat from core client for 30 sec - exiting 07:45:59 (1944): No heartbeat from core client for 30 sec - exiting 07:46:00 (1944): No heartbeat from core client for 30 sec - exiting 07:46:01 (1944): No heartbeat from core client for 30 sec - exiting 07:46:02 (1944): No heartbeat from core client for 30 sec - exiting 07:46:03 (1944): No heartbeat from core client for 30 sec - exiting 07:46:04 (1944): No heartbeat from core client for 30 sec - exiting 07:46:05 (1944): No heartbeat from core client for 30 sec - exiting 07:46:06 (1944): No heartbeat from core client for 30 sec - exiting 07:46:07 (1944): No heartbeat from core client for 30 sec - exiting 07:46:08 (1944): No heartbeat from core client for 30 sec - exiting 07:46:10 (1944): No heartbeat from core client for 30 sec - exiting 07:46:11 (1944): No heartbeat from core client for 30 sec - exiting 07:46:12 (1944): No heartbeat from core client for 30 sec - exiting 07:46:13 (1944): No heartbeat from core client for 30 sec - exiting 07:46:14 (1944): No heartbeat from core client for 30 sec - exiting 07:46:15 (1944): No heartbeat from core client for 30 sec - exiting 07:46:16 (1944): No heartbeat from core client for 30 sec - exiting 07:46:17 (1944): No heartbeat from core client for 30 sec - exiting 07:46:18 (1944): No heartbeat from core client for 30 sec - exiting 07:46:19 (1944): No heartbeat from core client for 30 sec - exiting 07:46:20 (1944): No heartbeat from core client for 30 sec - exiting 07:46:22 (1944): No heartbeat from core client for 30 sec - exiting 07:46:23 (1944): No heartbeat from core client for 30 sec - exiting 07:46:24 (1944): No heartbeat from core client for 30 sec - exiting 07:46:25 (1944): No heartbeat from core client for 30 sec - exiting 07:46:26 (1944): No heartbeat from core client for 30 sec - exiting 07:46:27 (1944): No heartbeat from core client for 30 sec - exiting 07:46:28 (1944): No heartbeat from core client for 30 sec - exiting 07:46:29 (1944): No heartbeat from core client for 30 sec - exiting 07:46:30 (1944): No heartbeat from core client for 30 sec - exiting 07:46:31 (1944): No heartbeat from core client for 30 sec - exiting 07:46:32 (1944): No heartbeat from core client for 30 sec - exiting 07:46:34 (1944): No heartbeat from core client for 30 sec - exiting 07:46:35 (1944): No heartbeat from core client for 30 sec - exiting 07:46:36 (1944): No heartbeat from core client for 30 sec - exiting 07:46:37 (1944): No heartbeat from core client for 30 sec - exiting 07:46:38 (1944): No heartbeat from core client for 30 sec - exiting 07:46:39 (1944): No heartbeat from core client for 30 sec - exiting 07:46:40 (1944): No heartbeat from core client for 30 sec - exiting 07:46:41 (1944): No heartbeat from core client for 30 sec - exiting 07:46:42 (1944): No heartbeat from core client for 30 sec - exiting 07:46:44 (1944): No heartbeat from core client for 30 sec - exiting 07:46:45 (1944): No heartbeat from core client for 30 sec - exiting 07:46:47 (1944): No heartbeat from core client for 30 sec - exiting 07:46:48 (1944): No heartbeat from core client for 30 sec - exiting 07:46:49 (1944): No heartbeat from core client for 30 sec - exiting 07:46:50 (1944): No heartbeat from core client for 30 sec - exiting 07:46:51 (1944): No heartbeat from core client for 30 sec - exiting 07:46:52 (1944): No heartbeat from core client for 30 sec - exiting 07:46:53 (1944): No heartbeat from core client for 30 sec - exiting 07:46:54 (1944): No heartbeat from core client for 30 sec - exiting 07:46:55 (1944): No heartbeat from core client for 30 sec - exiting 07:46:57 (1944): No heartbeat from core client for 30 sec - exiting 07:46:58 (1944): No heartbeat from core client for 30 sec - exiting 07:46:59 (1944): No heartbeat from core client for 30 sec - exiting 07:47:00 (1944): No heartbeat from core client for 30 sec - exiting 07:47:01 (1944): No heartbeat from core client for 30 sec - exiting 07:47:02 (1944): No heartbeat from core client for 30 sec - exiting 07:47:03 (1944): No heartbeat from core client for 30 sec - exiting 07:47:04 (1944): No heartbeat from core client for 30 sec - exiting 07:47:05 (1944): No heartbeat from core client for 30 sec - exiting 07:47:06 (1944): No heartbeat from core client for 30 sec - exiting 07:47:07 (1944): No heartbeat from core client for 30 sec - exiting 07:47:09 (1944): No heartbeat from core client for 30 sec - exiting 07:47:10 (1944): No heartbeat from core client for 30 sec - exiting 07:47:11 (1944): No heartbeat from core client for 30 sec - exiting 07:47:12 (1944): No heartbeat from core client for 30 sec - exiting 07:47:13 (1944): No heartbeat from core client for 30 sec - exiting 07:47:14 (1944): No heartbeat from core client for 30 sec - exiting 07:47:15 (1944): No heartbeat from core client for 30 sec - exiting 07:47:16 (1944): No heartbeat from core client for 30 sec - exiting 07:47:17 (1944): No heartbeat from core client for 30 sec - exiting 07:47:18 (1944): No heartbeat from core client for 30 sec - exiting 07:47:19 (1944): No heartbeat from core client for 30 sec - exiting 07:47:21 (1944): No heartbeat from core client for 30 sec - exiting 07:47:22 (1944): No heartbeat from core client for 30 sec - exiting 07:47:23 (1944): No heartbeat from core client for 30 sec - exiting 07:47:24 (1944): No heartbeat from core client for 30 sec - exiting 07:47:25 (1944): No heartbeat from core client for 30 sec - exiting 07:47:26 (1944): No heartbeat from core client for 30 sec - exiting 07:47:27 (1944): No heartbeat from core client for 30 sec - exiting 07:47:28 (1944): No heartbeat from core client for 30 sec - exiting 07:47:29 (1944): No heartbeat from core client for 30 sec - exiting 07:47:30 (1944): No heartbeat from core client for 30 sec - exiting 07:47:31 (1944): No heartbeat from core client for 30 sec - exiting 07:47:33 (1944): No heartbeat from core client for 30 sec - exiting 07:47:34 (1944): No heartbeat from core client for 30 sec - exiting 07:47:35 (1944): No heartbeat from core client for 30 sec - exiting 07:47:36 (1944): No heartbeat from core client for 30 sec - exiting 07:47:37 (1944): No heartbeat from core client for 30 sec - exiting 07:47:39 (1944): No heartbeat from core client for 30 sec - exiting 07:47:40 (1944): No heartbeat from core client for 30 sec - exiting 07:47:41 (1944): No heartbeat from core client for 30 sec - exiting 07:47:42 (1944): No heartbeat from core client for 30 sec - exiting 07:47:43 (1944): No heartbeat from core client for 30 sec - exiting 07:47:52 (1944): No heartbeat from core client for 30 sec - exiting 07:47:53 (1944): No heartbeat from core client for 30 sec - exiting 07:47:54 (1944): No heartbeat from core client for 30 sec - exiting 07:47:55 (1944): No heartbeat from core client for 30 sec - exiting 07:47:56 (1944): No heartbeat from core client for 30 sec - exiting 07:47:57 (1944): No heartbeat from core client for 30 sec - exiting 07:47:59 (1944): No heartbeat from core client for 30 sec - exiting 07:48:00 (1944): No heartbeat from core client for 30 sec - exiting 07:48:01 (1944): No heartbeat from core client for 30 sec - exiting 07:48:02 (1944): No heartbeat from core client for 30 sec - exiting 07:48:03 (1944): No heartbeat from core client for 30 sec - exiting 07:48:04 (1944): No heartbeat from core client for 30 sec - exiting 07:48:05 (1944): No heartbeat from core client for 30 sec - exiting 07:48:06 (1944): No heartbeat from core client for 30 sec - exiting 07:48:07 (1944): No heartbeat from core client for 30 sec - exiting 07:48:08 (1944): No heartbeat from core client for 30 sec - exiting 07:48:09 (1944): No heartbeat from core client for 30 sec - exiting 07:48:11 (1944): No heartbeat from core client for 30 sec - exiting 07:48:12 (1944): No heartbeat from core client for 30 sec - exiting 07:48:13 (1944): No heartbeat from core client for 30 sec - exiting 07:48:14 (1944): No heartbeat from core client for 30 sec - exiting 07:48:15 (1944): No heartbeat from core client for 30 sec - exiting 07:48:16 (1944): No heartbeat from core client for 30 sec - exiting 07:48:17 (1944): No heartbeat from core client for 30 sec - exiting 07:48:18 (1944): No heartbeat from core client for 30 sec - exiting 07:48:19 (1944): No heartbeat from core client for 30 sec - exiting 07:48:20 (1944): No heartbeat from core client for 30 sec - exiting 07:48:21 (1944): No heartbeat from core client for 30 sec - exiting 07:48:23 (1944): No heartbeat from core client for 30 sec - exiting 07:48:24 (1944): No heartbeat from core client for 30 sec - exiting 07:48:25 (1944): No heartbeat from core client for 30 sec - exiting 07:48:26 (1944): No heartbeat from core client for 30 sec - exiting 07:48:27 (1944): No heartbeat from core client for 30 sec - exiting 07:48:30 (1944): No heartbeat from core client for 30 sec - exiting 07:48:31 (1944): No heartbeat from core client for 30 sec - exiting 07:48:32 (1944): No heartbeat from core client for 30 sec - exiting 07:48:34 (1944): No heartbeat from core client for 30 sec - exiting 07:48:35 (1944): No heartbeat from core client for 30 sec - exiting 07:48:36 (1944): No heartbeat from core client for 30 sec - exiting 07:48:37 (1944): No heartbeat from core client for 30 sec - exiting 07:48:38 (1944): No heartbeat from core client for 30 sec - exiting 07:48:39 (1944): No heartbeat from core client for 30 sec - exiting 07:48:40 (1944): No heartbeat from core client for 30 sec - exiting 07:48:41 (1944): No heartbeat from core client for 30 sec - exiting 07:48:42 (1944): No heartbeat from core client for 30 sec - exiting 07:48:43 (1944): No heartbeat from core client for 30 sec - exiting 07:48:44 (1944): No heartbeat from core client for 30 sec - exiting 07:48:46 (1944): No heartbeat from core client for 30 sec - exiting 07:48:47 (1944): No heartbeat from core client for 30 sec - exiting 07:48:48 (1944): No heartbeat from core client for 30 sec - exiting 07:48:49 (1944): No heartbeat from core client for 30 sec - exiting 07:48:50 (1944): No heartbeat from core client for 30 sec - exiting 07:48:51 (1944): No heartbeat from core client for 30 sec - exiting 07:48:52 (1944): No heartbeat from core client for 30 sec - exiting 07:48:53 (1944): No heartbeat from core client for 30 sec - exiting 07:48:54 (1944): No heartbeat from core client for 30 sec - exiting 07:48:55 (1944): No heartbeat from core client for 30 sec - exiting 07:48:56 (1944): No heartbeat from core client for 30 sec - exiting 07:48:58 (1944): No heartbeat from core client for 30 sec - exiting 07:48:59 (1944): No heartbeat from core client for 30 sec - exiting 07:49:00 (1944): No heartbeat from core client for 30 sec - exiting 07:49:01 (1944): No heartbeat from core client for 30 sec - exiting 07:49:02 (1944): No heartbeat from core client for 30 sec - exiting 07:49:03 (1944): No heartbeat from core client for 30 sec - exiting 07:49:04 (1944): No heartbeat from core client for 30 sec - exiting 07:49:05 (1944): No heartbeat from core client for 30 sec - exiting 07:49:06 (1944): No heartbeat from core client for 30 sec - exiting 07:49:07 (1944): No heartbeat from core client for 30 sec - exiting 07:49:08 (1944): No heartbeat from core client for 30 sec - exiting 07:49:10 (1944): No heartbeat from core client for 30 sec - exiting 07:49:11 (1944): No heartbeat from core client for 30 sec - exiting 07:49:12 (1944): No heartbeat from core client for 30 sec - exiting 07:49:13 (1944): No heartbeat from core client for 30 sec - exiting 07:49:14 (1944): No heartbeat from core client for 30 sec - exiting 07:49:15 (1944): No heartbeat from core client for 30 sec - exiting 07:49:16 (1944): No heartbeat from core client for 30 sec - exiting 07:49:17 (1944): No heartbeat from core client for 30 sec - exiting 07:49:18 (1944): No heartbeat from core client for 30 sec - exiting 07:49:19 (1944): No heartbeat from core client for 30 sec - exiting 07:49:21 (1944): No heartbeat from core client for 30 sec - exiting 07:49:22 (1944): No heartbeat from core client for 30 sec - exiting 07:49:23 (1944): No heartbeat from core client for 30 sec - exiting 07:49:24 (1944): No heartbeat from core client for 30 sec - exiting 07:49:25 (1944): No heartbeat from core client for 30 sec - exiting 07:49:26 (1944): No heartbeat from core client for 30 sec - exiting 07:49:27 (1944): No heartbeat from core client for 30 sec - exiting 07:49:28 (1944): No heartbeat from core client for 30 sec - exiting 07:49:29 (1944): No heartbeat from core client for 30 sec - exiting 07:49:30 (1944): No heartbeat from core client for 30 sec - exiting 07:49:31 (1944): No heartbeat from core client for 30 sec - exiting 07:49:33 (1944): No heartbeat from core client for 30 sec - exiting 07:49:34 (1944): No heartbeat from core client for 30 sec - exiting 07:49:35 (1944): No heartbeat from core client for 30 sec - exiting 07:49:36 (1944): No heartbeat from core client for 30 sec - exiting 07:49:37 (1944): No heartbeat from core client for 30 sec - exiting 07:49:38 (1944): No heartbeat from core client for 30 sec - exiting 07:49:39 (1944): No heartbeat from core client for 30 sec - exiting 07:49:40 (1944): No heartbeat from core client for 30 sec - exiting 07:49:41 (1944): No heartbeat from core client for 30 sec - exiting 07:49:42 (1944): No heartbeat from core client for 30 sec - exiting 07:49:43 (1944): No heartbeat from core client for 30 sec - exiting 07:49:45 (1944): No heartbeat from core client for 30 sec - exiting 07:49:46 (1944): No heartbeat from core client for 30 sec - exiting 07:49:47 (1944): No heartbeat from core client for 30 sec - exiting 07:49:48 (1944): No heartbeat from core client for 30 sec - exiting 07:49:49 (1944): No heartbeat from core client for 30 sec - exiting 07:49:50 (1944): No heartbeat from core client for 30 sec - exiting 07:49:51 (1944): No heartbeat from core client for 30 sec - exiting 07:49:52 (1944): No heartbeat from core client for 30 sec - exiting 07:49:53 (1944): No heartbeat from core client for 30 sec - exiting 07:49:54 (1944): No heartbeat from core client for 30 sec - exiting 07:49:55 (1944): No heartbeat from core client for 30 sec - exiting 07:49:57 (1944): No heartbeat from core client for 30 sec - exiting 07:49:58 (1944): No heartbeat from core client for 30 sec - exiting 07:49:59 (1944): No heartbeat from core client for 30 sec - exiting 07:50:00 (1944): No heartbeat from core client for 30 sec - exiting 07:50:01 (1944): No heartbeat from core client for 30 sec - exiting 07:50:02 (1944): No heartbeat from core client for 30 sec - exiting 07:50:03 (1944): No heartbeat from core client for 30 sec - exiting 07:50:06 (1944): No heartbeat from core client for 30 sec - exiting 07:50:07 (1944): No heartbeat from core client for 30 sec - exiting 07:50:09 (1944): No heartbeat from core client for 30 sec - exiting 07:50:10 (1944): No heartbeat from core client for 30 sec - exiting 07:50:11 (1944): No heartbeat from core client for 30 sec - exiting 07:50:13 (1944): No heartbeat from core client for 30 sec - exiting 07:50:14 (1944): No heartbeat from core client for 30 sec - exiting 07:50:15 (1944): No heartbeat from core client for 30 sec - exiting 07:50:16 (1944): No heartbeat from core client for 30 sec - exiting 07:50:17 (1944): No heartbeat from core client for 30 sec - exiting 07:50:18 (1944): No heartbeat from core client for 30 sec - exiting 07:50:19 (1944): No heartbeat from core client for 30 sec - exiting 07:50:20 (1944): No heartbeat from core client for 30 sec - exiting 07:50:21 (1944): No heartbeat from core client for 30 sec - exiting 07:50:22 (1944): No heartbeat from core client for 30 sec - exiting 07:50:23 (1944): No heartbeat from core client for 30 sec - exiting 07:50:25 (1944): No heartbeat from core client for 30 sec - exiting 07:50:26 (1944): No heartbeat from core client for 30 sec - exiting 07:50:27 (1944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 09:03:22 (2848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=764, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=764, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=764, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=764, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=764, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=764, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Oct 2013 22:48:36	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	466,560	796,812	1.7078
23 Oct 2013 06:05:46	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	440,640	753,010	1.7089
22 Oct 2013 15:46:24	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	414,720	708,118	1.7075
21 Oct 2013 21:48:21	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	388,800	664,112	1.7081
21 Oct 2013 06:47:59	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	362,880	619,744	1.7078
20 Oct 2013 10:20:07	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	336,960	575,034	1.7065
19 Oct 2013 14:26:43	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	311,040	530,550	1.7057
18 Oct 2013 10:25:47	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	285,120	486,185	1.7052
17 Oct 2013 04:01:11	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	259,200	442,200	1.7060
16 Oct 2013 11:03:18	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	233,280	397,126	1.7024
15 Oct 2013 21:41:09	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	207,360	354,498	1.7096
15 Oct 2013 07:50:16	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	181,440	311,283	1.7156
14 Oct 2013 18:37:27	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	155,520	266,933	1.7164
14 Oct 2013 05:40:35	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	129,600	224,056	1.7288
13 Oct 2013 16:37:28	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	103,680	180,947	1.7452
13 Oct 2013 03:42:35	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	77,760	138,212	1.7774
12 Oct 2013 14:08:35	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	51,840	95,221	1.8368
12 Oct 2013 00:35:52	1123998	16064360	hadcm3n_n0pz_1920_40_008360931_4	25,920	49,208	1.8985