Task 19420618

Name	hadcm3n_sdkg_198012_480_375_010414947_0
Workunit	10414947
Created	21 Mar 2016, 16:50:05 UTC
Sent	22 Mar 2016, 12:27:23 UTC
Report deadline	4 Mar 2017, 17:47:23 UTC
Received	2 May 2016, 6:10:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1392139
Run time	13 days 7 hours 25 min 47 sec
CPU time	9 days 4 hours 8 min 13 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	3.72 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> ntroller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2802e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x201ce04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x202b404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(396,0xa3b3a000) malloc: * error for object 0x2039a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(478,0xa3b3a000) malloc: * error for object 0x1811c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(1384,0xa3b3a000) malloc: * error for object 0x182de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation 07:49:38 (1384): No heartbeat from core client for 30 sec - exiting 07:49:39 (1384): No heartbeat from core client for 30 sec - exiting 07:49:40 (1384): No heartbeat from core client for 30 sec - exiting 07:49:41 (1384): No heartbeat from core client for 30 sec - exiting 07:49:42 (1384): No heartbeat from core client for 30 sec - exiting 07:49:43 (1384): No heartbeat from core client for 30 sec - exiting 07:49:44 (1384): No heartbeat from core client for 30 sec - exiting 07:49:45 (1384): No heartbeat from core client for 30 sec - exiting 07:49:46 (1384): No heartbeat from core client for 30 sec - exiting 07:49:47 (1384): No heartbeat from core client for 30 sec - exiting 07:49:48 (1384): No heartbeat from core client for 30 sec - exiting 07:49:49 (1384): No heartbeat from core client for 30 sec - exiting 07:49:50 (1384): No heartbeat from core client for 30 sec - exiting 07:49:51 (1384): No heartbeat from core client for 30 sec - exiting 07:49:52 (1384): No heartbeat from core client for 30 sec - exiting 07:49:53 (1384): No heartbeat from core client for 30 sec - exiting 07:49:54 (1384): No heartbeat from core client for 30 sec - exiting 07:49:55 (1384): No heartbeat from core client for 30 sec - exiting 07:49:56 (1384): No heartbeat from core client for 30 sec - exiting 07:49:57 (1384): No heartbeat from core client for 30 sec - exiting 07:49:58 (1384): No heartbeat from core client for 30 sec - exiting 07:49:59 (1384): No heartbeat from core client for 30 sec - exiting 17:46:59 (1384): No heartbeat from core client for 30 sec - exiting 17:47:00 (1384): No heartbeat from core client for 30 sec - exiting 17:47:01 (1384): No heartbeat from core client for 30 sec - exiting 17:47:02 (1384): No heartbeat from core client for 30 sec - exiting 17:47:03 (1384): No heartbeat from core client for 30 sec - exiting 17:47:04 (1384): No heartbeat from core client for 30 sec - exiting 17:47:05 (1384): No heartbeat from core client for 30 sec - exiting 17:47:06 (1384): No heartbeat from core client for 30 sec - exiting 17:47:07 (1384): No heartbeat from core client for 30 sec - exiting 17:47:08 (1384): No heartbeat from core client for 30 sec - exiting 17:47:09 (1384): No heartbeat from core client for 30 sec - exiting 17:47:10 (1384): No heartbeat from core client for 30 sec - exiting 17:47:11 (1384): No heartbeat from core client for 30 sec - exiting 17:47:12 (1384): No heartbeat from core client for 30 sec - exiting 17:47:13 (1384): No heartbeat from core client for 30 sec - exiting 17:47:14 (1384): No heartbeat from core client for 30 sec - exiting 17:47:15 (1384): No heartbeat from core client for 30 sec - exiting 17:47:16 (1384): No heartbeat from core client for 30 sec - exiting 17:47:17 (1384): No heartbeat from core client for 30 sec - exiting 17:47:18 (1384): No heartbeat from core client for 30 sec - exiting 17:47:19 (1384): No heartbeat from core client for 30 sec - exiting 17:47:51 (1384): No heartbeat from core client for 30 sec - exiting 17:47:52 (1384): No heartbeat from core client for 30 sec - exiting 17:47:53 (1384): No heartbeat from core client for 30 sec - exiting 17:47:54 (1384): No heartbeat from core client for 30 sec - exiting 17:47:55 (1384): No heartbeat from core client for 30 sec - exiting 17:47:56 (1384): No heartbeat from core client for 30 sec - exiting 17:47:57 (1384): No heartbeat from core client for 30 sec - exiting 17:47:58 (1384): No heartbeat from core client for 30 sec - exiting 17:47:59 (1384): No heartbeat from core client for 30 sec - exiting 17:48:00 (1384): No heartbeat from core client for 30 sec - exiting 17:48:01 (1384): No heartbeat from core client for 30 sec - exiting 17:48:02 (1384): No heartbeat from core client for 30 sec - exiting 17:48:03 (1384): No heartbeat from core client for 30 sec - exiting 17:48:04 (1384): No heartbeat from core client for 30 sec - exiting 17:48:06 (1384): No heartbeat from core client for 30 sec - exiting 17:48:07 (1384): No heartbeat from core client for 30 sec - exiting 17:48:08 (1384): No heartbeat from core client for 30 sec - exiting 17:48:09 (1384): No heartbeat from core client for 30 sec - exiting 17:48:10 (1384): No heartbeat from core client for 30 sec - exiting 17:48:11 (1384): No heartbeat from core client for 30 sec - exiting 17:48:12 (1384): No heartbeat from core client for 30 sec - exiting 17:48:13 (1384): No heartbeat from core client for 30 sec - exiting 17:48:14 (1384): No heartbeat from core client for 30 sec - exiting 17:48:15 (1384): No heartbeat from core client for 30 sec - exiting 17:48:16 (1384): No heartbeat from core client for 30 sec - exiting 17:48:17 (1384): No heartbeat from core client for 30 sec - exiting 17:48:18 (1384): No heartbeat from core client for 30 sec - exiting 17:48:19 (1384): No heartbeat from core client for 30 sec - exiting 17:48:20 (1384): No heartbeat from core client for 30 sec - exiting 17:48:21 (1384): No heartbeat from core client for 30 sec - exiting 17:48:22 (1384): No heartbeat from core client for 30 sec - exiting 17:48:23 (1384): No heartbeat from core client for 30 sec - exiting 17:48:24 (1384): No heartbeat from core client for 30 sec - exiting 17:48:25 (1384): No heartbeat from core client for 30 sec - exiting 17:48:26 (1384): No heartbeat from core client for 30 sec - exiting 17:48:27 (1384): No heartbeat from core client for 30 sec - exiting 17:48:28 (1384): No heartbeat from core client for 30 sec - exiting 17:48:29 (1384): No heartbeat from core client for 30 sec - exiting 17:48:30 (1384): No heartbeat from core client for 30 sec - exiting 17:48:31 (1384): No heartbeat from core client for 30 sec - exiting 17:48:32 (1384): No heartbeat from core client for 30 sec - exiting 17:48:33 (1384): No heartbeat from core client for 30 sec - exiting 17:48:34 (1384): No heartbeat from core client for 30 sec - exiting 17:48:35 (1384): No heartbeat from core client for 30 sec - exiting 17:48:36 (1384): No heartbeat from core client for 30 sec - exiting 17:48:37 (1384): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x100a804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x100a800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x181dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x181dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x181dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x181dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x181dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x181dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x181dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x181dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(459,0xa4223000) malloc: * error for object 0x1001204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Sun Apr 24 20:01:49 2016 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. 20:16:14 (459): No heartbeat from core client for 30 sec - exiting 20:16:15 (459): No heartbeat from core client for 30 sec - exiting 20:16:16 (459): No heartbeat from core client for 30 sec - exiting 20:16:17 (459): No heartbeat from core client for 30 sec - exiting 20:16:18 (459): No heartbeat from core client for 30 sec - exiting 20:16:19 (459): No heartbeat from core client for 30 sec - exiting 20:16:20 (459): No heartbeat from core client for 30 sec - exiting 20:16:21 (459): No heartbeat from core client for 30 sec - exiting 20:16:22 (459): No heartbeat from core client for 30 sec - exiting 20:16:23 (459): No heartbeat from core client for 30 sec - exiting 20:16:24 (459): No heartbeat from core client for 30 sec - exiting 20:16:25 (459): No heartbeat from core client for 30 sec - exiting 20:16:26 (459): No heartbeat from core client for 30 sec - exiting 20:16:27 (459): No heartbeat from core client for 30 sec - exiting 20:16:28 (459): No heartbeat from core client for 30 sec - exiting 20:16:29 (459): No heartbeat from core client for 30 sec - exiting 20:16:30 (459): No heartbeat from core client for 30 sec - exiting 20:16:31 (459): No heartbeat from core client for 30 sec - exiting 20:16:32 (459): No heartbeat from core client for 30 sec - exiting 20:16:33 (459): No heartbeat from core client for 30 sec - exiting 20:16:34 (459): No heartbeat from core client for 30 sec - exiting 20:16:35 (459): No heartbeat from core client for 30 sec - exiting 20:16:36 (459): No heartbeat from core client for 30 sec - exiting 20:16:37 (459): No heartbeat from core client for 30 sec - exiting 20:16:38 (459): No heartbeat from core client for 30 sec - exiting 20:16:39 (459): No heartbeat from core client for 30 sec - exiting 20:16:41 (459): No heartbeat from core client for 30 sec - exiting 20:16:42 (459): No heartbeat from core client for 30 sec - exiting 20:16:43 (459): No heartbeat from core client for 30 sec - exiting 20:16:44 (459): No heartbeat from core client for 30 sec - exiting 20:16:45 (459): No heartbeat from core client for 30 sec - exiting 20:16:46 (459): No heartbeat from core client for 30 sec - exiting 20:16:47 (459): No heartbeat from core client for 30 sec - exiting 20:16:48 (459): No heartbeat from core client for 30 sec - exiting 20:16:49 (459): No heartbeat from core client for 30 sec - exiting 20:16:50 (459): No heartbeat from core client for 30 sec - exiting 20:16:51 (459): No heartbeat from core client for 30 sec - exiting 20:16:52 (459): No heartbeat from core client for 30 sec - exiting 20:16:53 (459): No heartbeat from core client for 30 sec - exiting 20:16:54 (459): No heartbeat from core client for 30 sec - exiting 20:16:55 (459): No heartbeat from core client for 30 sec - exiting 20:16:56 (459): No heartbeat from core client for 30 sec - exiting 20:16:57 (459): No heartbeat from core client for 30 sec - exiting 20:16:58 (459): No heartbeat from core client for 30 sec - exiting 20:16:59 (459): No heartbeat from core client for 30 sec - exiting 20:17:00 (459): No heartbeat from core client for 30 sec - exiting 20:17:01 (459): No heartbeat from core client for 30 sec - exiting 20:17:02 (459): No heartbeat from core client for 30 sec - exiting 20:17:03 (459): No heartbeat from core client for 30 sec - exiting 20:17:04 (459): No heartbeat from core client for 30 sec - exiting 20:17:05 (459): No heartbeat from core client for 30 sec - exiting 20:17:06 (459): No heartbeat from core client for 30 sec - exiting 20:17:07 (459): No heartbeat from core client for 30 sec - exiting 20:17:08 (459): No heartbeat from core client for 30 sec - exiting 20:17:09 (459): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(402,0xa4223000) malloc: * error for object 0x101e804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Fri Apr 29 11:22:41 2016 11:29:50 (402): No heartbeat from core client for 30 sec - exiting 11:29:51 (402): No heartbeat from core client for 30 sec - exiting 11:29:52 (402): No heartbeat from core client for 30 sec - exiting 11:29:53 (402): No heartbeat from core client for 30 sec - exiting 11:29:54 (402): No heartbeat from core client for 30 sec - exiting 11:29:55 (402): No heartbeat from core client for 30 sec - exiting 11:29:56 (402): No heartbeat from core client for 30 sec - exiting 11:29:57 (402): No heartbeat from core client for 30 sec - exiting 11:29:58 (402): No heartbeat from core client for 30 sec - exiting 11:29:59 (402): No heartbeat from core client for 30 sec - exiting 11:30:00 (402): No heartbeat from core client for 30 sec - exiting 11:30:01 (402): No heartbeat from core client for 30 sec - exiting 11:30:02 (402): No heartbeat from core client for 30 sec - exiting 11:30:03 (402): No heartbeat from core client for 30 sec - exiting 11:30:04 (402): No heartbeat from core client for 30 sec - exiting 11:30:05 (402): No heartbeat from core client for 30 sec - exiting 11:30:06 (402): No heartbeat from core client for 30 sec - exiting 11:30:07 (402): No heartbeat from core client for 30 sec - exiting 11:30:08 (402): No heartbeat from core client for 30 sec - exiting 11:30:09 (402): No heartbeat from core client for 30 sec - exiting 11:30:10 (402): No heartbeat from core client for 30 sec - exiting 11:30:11 (402): No heartbeat from core client for 30 sec - exiting 11:30:12 (402): No heartbeat from core client for 30 sec - exiting 11:30:13 (402): No heartbeat from core client for 30 sec - exiting 11:30:14 (402): No heartbeat from core client for 30 sec - exiting 11:30:15 (402): No heartbeat from core client for 30 sec - exiting 11:30:16 (402): No heartbeat from core client for 30 sec - exiting 11:30:18 (402): No heartbeat from core client for 30 sec - exiting 11:30:19 (402): No heartbeat from core client for 30 sec - exiting 11:30:20 (402): No heartbeat from core client for 30 sec - exiting 11:30:21 (402): No heartbeat from core client for 30 sec - exiting 11:30:22 (402): No heartbeat from core client for 30 sec - exiting 11:30:23 (402): No heartbeat from core client for 30 sec - exiting 11:30:24 (402): No heartbeat from core client for 30 sec - exiting 11:30:25 (402): No heartbeat from core client for 30 sec - exiting 11:30:26 (402): No heartbeat from core client for 30 sec - exiting 11:30:27 (402): No heartbeat from core client for 30 sec - exiting 11:30:28 (402): No heartbeat from core client for 30 sec - exiting 11:30:29 (402): No heartbeat from core client for 30 sec - exiting 11:30:30 (402): No heartbeat from core client for 30 sec - exiting 11:30:31 (402): No heartbeat from core client for 30 sec - exiting 11:30:32 (402): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x2025200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(471,0xa4223000) malloc: * error for object 0x1001204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Sat Apr 30 06:57:50 2016 Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x1002804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x1002800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(410,0xa4223000) malloc: * error for object 0x2800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Mon May 2 07:10:18 2016 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fe48 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/1/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90010000 - 0x90038fff /usr/lib/system/libxpc.dylib 0x90b48000 - 0x90b4bfff /usr/lib/system/libdyld.dylib 0x90cbd000 - 0x90cc3fff /usr/lib/system/libunwind.dylib 0x90ccd000 - 0x90cd8fff /usr/lib/system/libcommonCrypto.dylib 0x90f58000 - 0x90f5cfff /usr/lib/system/libcache.dylib 0x91a2c000 - 0x91a32fff /usr/lib/system/libsystem_platform.dylib 0x938a2000 - 0x938b3fff /usr/lib/system/libsystem_trace.dylib 0x9449b000 - 0x944a3fff /usr/lib/system/libsystem_dnssd.dylib 0x94966000 - 0x94987fff /usr/lib/system/libsystem_kernel.dylib 0x949d8000 - 0x949ddfff /usr/lib/system/libmacho.dylib 0x94f87000 - 0x94fbafff /usr/lib/system/libsystem_m.dylib 0x959b6000 - 0x959bffff /usr/lib/system/libsystem_networkextension.dylib 0x95c18000 - 0x95cadfff /usr/lib/system/libsystem_c.dylib 0x969dc000 - 0x96a06fff /usr/lib/system/libsystem_info.dylib 0x96c9a000 - 0x96c9afff /usr/lib/system/libunc.dylib 0x96ca3000 - 0x96d11fff /usr/lib/system/libsystem_network.dylib 0x96d2c000 - 0x96d7ffff /usr/lib/libstdc++.6.dylib 0x97d4d000 - 0x97d5bfff /usr/lib/libz.1.dylib 0x98223000 - 0x98224fff /usr/lib/libSystem.B.dylib 0x98797000 - 0x98797fff /usr/lib/system/libkeymgr.dylib 0x987a6000 - 0x987bffff /usr/lib/system/libsystem_malloc.dylib 0x98ac4000 - 0x98ac4fff /usr/lib/system/liblaunch.dylib 0x99456000 - 0x99458fff /usr/lib/system/libsystem_configuration.dylib 0x99459000 - 0x9945cfff /usr/lib/system/libsystem_sandbox.dylib 0x99711000 - 0x99712fff /usr/lib/system/libremovefile.dylib 0x99781000 - 0x99783fff /usr/lib/system/libsystem_secinit.dylib 0x99c15000 - 0x99c1ffff /usr/lib/system/libsystem_notify.dylib 0x99e9f000 - 0x99e9ffff /usr/lib/system/libsystem_blocks.dylib 0x9a20b000 - 0x9a272fff /usr/lib/system/libcorecrypto.dylib 0x9af70000 - 0x9af72fff /usr/lib/system/libsystem_coreservices.dylib 0x9b558000 - 0x9b561fff /usr/lib/system/libcopyfile.dylib 0x9be19000 - 0x9be43fff /usr/lib/system/libdispatch.dylib 0x9be44000 - 0x9be49fff /usr/lib/system/libcompiler_rt.dylib 0x9be4a000 - 0x9be6dfff /usr/lib/libc++abi.dylib 0x9c1be000 - 0x9c1c6fff /usr/lib/system/libsystem_pthread.dylib 0x9d380000 - 0x9d382fff /usr/lib/system/libquarantine.dylib 0x9dc63000 - 0x9dc7afff /usr/lib/system/libsystem_coretls.dylib 0x9dc7b000 - 0x9dc94fff /usr/lib/system/libsystem_asl.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Apr 2016 05:43:34	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	881,280	792,493	0.8993
08 Apr 2016 18:19:28	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	777,600	689,490	0.8867
07 Apr 2016 17:46:47	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	751,680	666,470	0.8866
06 Apr 2016 22:32:20	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	725,760	643,324	0.8864
06 Apr 2016 10:06:07	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	699,840	620,387	0.8865
06 Apr 2016 01:04:44	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	673,920	597,462	0.8865
05 Apr 2016 13:37:30	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	648,000	574,627	0.8868
05 Apr 2016 02:38:06	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	622,080	551,772	0.8870
05 Apr 2016 00:17:48	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	596,160	528,696	0.8868
04 Apr 2016 06:02:37	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	570,240	505,596	0.8866
03 Apr 2016 14:04:34	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	544,320	482,819	0.8870
02 Apr 2016 22:04:57	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	518,400	459,557	0.8865
02 Apr 2016 11:38:53	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	492,480	436,539	0.8864
01 Apr 2016 17:22:20	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	466,560	413,447	0.8862
01 Apr 2016 06:24:41	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	440,640	390,304	0.8858
31 Mar 2016 14:18:15	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	414,720	367,309	0.8857
30 Mar 2016 22:50:34	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	388,800	344,455	0.8859
30 Mar 2016 12:05:31	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	362,880	321,705	0.8865
29 Mar 2016 22:12:18	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	336,960	298,538	0.8860
29 Mar 2016 12:34:05	1392139	19420618	hadcm3n_sdkg_198012_480_375_010414947_0	311,040	275,371	0.8853