Task 13787137

Name	hadcm3n_o619_1940_40_007545401_2
Workunit	7742633
Created	16 Dec 2011, 13:28:52 UTC
Sent	16 Dec 2011, 13:37:58 UTC
Report deadline	16 Mar 2012, 21:05:09 UTC
Received	4 Apr 2012, 17:55:18 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1176286
Run time	18 days 20 hours 46 min 54 sec
CPU time	11 days 8 hours 50 min 21 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.79 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.35</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:18:06 (94522): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(28795,0xa0937540) malloc: * error for object 0x1023204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(28795,0xa0937540) malloc: * error for object 0x1023200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(28795,0xa0937540) malloc: * error for object 0x821804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(28795,0xa0937540) malloc: * error for object 0x821800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(28795,0xa0937540) malloc: * error for object 0x821804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=28797, selfPID=28797, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(48036,0xa0937540) malloc: * error for object 0x83e804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(48036,0xa0937540) malloc: * error for object 0x83e800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(48036,0xa0937540) malloc: * error for object 0x83e804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(48036,0xa0937540) malloc: * error for object 0x83e800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(48036,0xa0937540) malloc: * error for object 0x83e804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(48036,0xa0937540) malloc: * error for object 0x83e800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(48036,0xa0937540) malloc: * error for object 0x83e804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation 14:43:43 (48036): No heartbeat from core client for 30 sec - exiting 14:43:44 (48036): No heartbeat from core client for 30 sec - exiting 14:43:45 (48036): No heartbeat from core client for 30 sec - exiting 14:43:46 (48036): No heartbeat from core client for 30 sec - exiting 14:43:47 (48036): No heartbeat from core client for 30 sec - exiting 14:43:48 (48036): No heartbeat from core client for 30 sec - exiting 14:43:49 (48036): No heartbeat from core client for 30 sec - exiting 14:43:50 (48036): No heartbeat from core client for 30 sec - exiting 14:43:51 (48036): No heartbeat from core client for 30 sec - exiting 14:43:52 (48036): No heartbeat from core client for 30 sec - exiting 14:43:53 (48036): No heartbeat from core client for 30 sec - exiting 14:43:54 (48036): No heartbeat from core client for 30 sec - exiting 14:43:55 (48036): No heartbeat from core client for 30 sec - exiting 14:43:56 (48036): No heartbeat from core client for 30 sec - exiting 14:43:57 (48036): No heartbeat from core client for 30 sec - exiting 14:43:58 (48036): No heartbeat from core client for 30 sec - exiting 14:43:59 (48036): No heartbeat from core client for 30 sec - exiting 14:44:00 (48036): No heartbeat from core client for 30 sec - exiting 14:44:01 (48036): No heartbeat from core client for 30 sec - exiting 14:44:03 (48036): No heartbeat from core client for 30 sec - exiting 14:44:04 (48036): No heartbeat from core client for 30 sec - exiting 14:44:05 (48036): No heartbeat from core client for 30 sec - exiting 14:44:06 (48036): No heartbeat from core client for 30 sec - exiting 14:44:07 (48036): No heartbeat from core client for 30 sec - exiting 14:44:08 (48036): No heartbeat from core client for 30 sec - exiting 14:44:09 (48036): No heartbeat from core client for 30 sec - exiting 14:44:10 (48036): No heartbeat from core client for 30 sec - exiting 14:44:12 (48036): No heartbeat from core client for 30 sec - exiting 14:44:13 (48036): No heartbeat from core client for 30 sec - exiting 14:44:14 (48036): No heartbeat from core client for 30 sec - exiting 14:44:16 (48036): No heartbeat from core client for 30 sec - exiting 14:44:17 (48036): No heartbeat from core client for 30 sec - exiting 14:44:18 (48036): No heartbeat from core client for 30 sec - exiting 14:44:19 (48036): No heartbeat from core client for 30 sec - exiting 14:44:20 (48036): No heartbeat from core client for 30 sec - exiting 14:44:21 (48036): No heartbeat from core client for 30 sec - exiting 14:44:22 (48036): No heartbeat from core client for 30 sec - exiting 14:44:23 (48036): No heartbeat from core client for 30 sec - exiting 14:44:24 (48036): No heartbeat from core client for 30 sec - exiting 14:44:25 (48036): No heartbeat from core client for 30 sec - exiting 14:44:27 (48036): No heartbeat from core client for 30 sec - exiting 14:44:28 (48036): No heartbeat from core client for 30 sec - exiting 14:44:29 (48036): No heartbeat from core client for 30 sec - exiting 14:44:30 (48036): No heartbeat from core client for 30 sec - exiting 14:44:31 (48036): No heartbeat from core client for 30 sec - exiting 14:44:32 (48036): No heartbeat from core client for 30 sec - exiting 14:44:33 (48036): No heartbeat from core client for 30 sec - exiting 14:44:34 (48036): No heartbeat from core client for 30 sec - exiting 14:44:35 (48036): No heartbeat from core client for 30 sec - exiting 14:44:36 (48036): No heartbeat from core client for 30 sec - exiting 14:44:37 (48036): No heartbeat from core client for 30 sec - exiting 14:44:38 (48036): No heartbeat from core client for 30 sec - exiting 14:44:39 (48036): No heartbeat from core client for 30 sec - exiting 14:44:41 (48036): No heartbeat from core client for 30 sec - exiting 14:44:42 (48036): No heartbeat from core client for 30 sec - exiting 14:44:43 (48036): No heartbeat from core client for 30 sec - exiting 14:44:44 (48036): No heartbeat from core client for 30 sec - exiting 14:44:45 (48036): No heartbeat from core client for 30 sec - exiting 14:44:46 (48036): No heartbeat from core client for 30 sec - exiting 14:44:47 (48036): No heartbeat from core client for 30 sec - exiting 14:44:48 (48036): No heartbeat from core client for 30 sec - exiting 14:44:49 (48036): No heartbeat from core client for 30 sec - exiting 14:44:50 (48036): No heartbeat from core client for 30 sec - exiting 14:44:51 (48036): No heartbeat from core client for 30 sec - exiting 14:44:52 (48036): No heartbeat from core client for 30 sec - exiting 14:44:53 (48036): No heartbeat from core client for 30 sec - exiting 14:44:54 (48036): No heartbeat from core client for 30 sec - exiting 14:44:55 (48036): No heartbeat from core client for 30 sec - exiting 14:44:56 (48036): No heartbeat from core client for 30 sec - exiting 14:44:57 (48036): No heartbeat from core client for 30 sec - exiting 14:44:58 (48036): No heartbeat from core client for 30 sec - exiting 14:44:59 (48036): No heartbeat from core client for 30 sec - exiting 14:45:00 (48036): No heartbeat from core client for 30 sec - exiting 14:45:01 (48036): No heartbeat from core client for 30 sec - exiting 14:45:02 (48036): No heartbeat from core client for 30 sec - exiting 14:45:03 (48036): No heartbeat from core client for 30 sec - exiting 14:45:04 (48036): No heartbeat from core client for 30 sec - exiting 14:45:05 (48036): No heartbeat from core client for 30 sec - exiting 14:45:06 (48036): No heartbeat from core client for 30 sec - exiting 14:45:07 (48036): No heartbeat from core client for 30 sec - exiting 14:45:08 (48036): No heartbeat from core client for 30 sec - exiting 14:45:09 (48036): No heartbeat from core client for 30 sec - exiting 14:45:10 (48036): No heartbeat from core client for 30 sec - exiting 14:45:11 (48036): No heartbeat from core client for 30 sec - exiting 14:45:12 (48036): No heartbeat from core client for 30 sec - exiting 14:45:13 (48036): No heartbeat from core client for 30 sec - exiting 14:45:14 (48036): No heartbeat from core client for 30 sec - exiting 14:45:15 (48036): No heartbeat from core client for 30 sec - exiting 14:45:16 (48036): No heartbeat from core client for 30 sec - exiting 14:45:17 (48036): No heartbeat from core client for 30 sec - exiting 14:45:18 (48036): No heartbeat from core client for 30 sec - exiting 14:45:19 (48036): No heartbeat from core client for 30 sec - exiting 14:45:20 (48036): No heartbeat from core client for 30 sec - exiting 14:45:21 (48036): No heartbeat from core client for 30 sec - exiting 14:45:22 (48036): No heartbeat from core client for 30 sec - exiting 14:45:23 (48036): No heartbeat from core client for 30 sec - exiting 14:45:24 (48036): No heartbeat from core client for 30 sec - exiting 14:45:25 (48036): No heartbeat from core client for 30 sec - exiting 14:45:26 (48036): No heartbeat from core client for 30 sec - exiting 14:45:27 (48036): No heartbeat from core client for 30 sec - exiting 14:45:28 (48036): No heartbeat from core client for 30 sec - exiting 14:45:29 (48036): No heartbeat from core client for 30 sec - exiting 14:45:30 (48036): No heartbeat from core client for 30 sec - exiting 14:45:31 (48036): No heartbeat from core client for 30 sec - exiting 14:45:32 (48036): No heartbeat from core client for 30 sec - exiting 14:45:33 (48036): No heartbeat from core client for 30 sec - exiting 14:45:34 (48036): No heartbeat from core client for 30 sec - exiting 14:45:35 (48036): No heartbeat from core client for 30 sec - exiting 14:45:36 (48036): No heartbeat from core client for 30 sec - exiting 14:45:37 (48036): No heartbeat from core client for 30 sec - exiting 14:45:38 (48036): No heartbeat from core client for 30 sec - exiting 14:45:39 (48036): No heartbeat from core client for 30 sec - exiting 14:45:40 (48036): No heartbeat from core client for 30 sec - exiting 14:45:41 (48036): No heartbeat from core client for 30 sec - exiting 14:45:42 (48036): No heartbeat from core client for 30 sec - exiting 14:45:43 (48036): No heartbeat from core client for 30 sec - exiting 14:45:44 (48036): No heartbeat from core client for 30 sec - exiting 14:45:45 (48036): No heartbeat from core client for 30 sec - exiting 14:45:46 (48036): No heartbeat from core client for 30 sec - exiting 14:45:47 (48036): No heartbeat from core client for 30 sec - exiting 14:45:48 (48036): No heartbeat from core client for 30 sec - exiting 14:45:49 (48036): No heartbeat from core client for 30 sec - exiting 14:45:50 (48036): No heartbeat from core client for 30 sec - exiting 14:45:51 (48036): No heartbeat from core client for 30 sec - exiting 14:45:52 (48036): No heartbeat from core client for 30 sec - exiting 14:45:53 (48036): No heartbeat from core client for 30 sec - exiting 14:45:54 (48036): No heartbeat from core client for 30 sec - exiting 14:45:55 (48036): No heartbeat from core client for 30 sec - exiting 14:45:56 (48036): No heartbeat from core client for 30 sec - exiting 14:45:57 (48036): No heartbeat from core client for 30 sec - exiting 14:45:58 (48036): No heartbeat from core client for 30 sec - exiting 14:45:59 (48036): No heartbeat from core client for 30 sec - exiting 14:46:00 (48036): No heartbeat from core client for 30 sec - exiting 14:46:01 (48036): No heartbeat from core client for 30 sec - exiting 14:46:02 (48036): No heartbeat from core client for 30 sec - exiting 14:46:03 (48036): No heartbeat from core client for 30 sec - exiting 14:46:04 (48036): No heartbeat from core client for 30 sec - exiting 14:46:05 (48036): No heartbeat from core client for 30 sec - exiting 14:46:06 (48036): No heartbeat from core client for 30 sec - exiting 14:46:07 (48036): No heartbeat from core client for 30 sec - exiting 14:46:08 (48036): No heartbeat from core client for 30 sec - exiting 14:46:09 (48036): No heartbeat from core client for 30 sec - exiting 14:46:10 (48036): No heartbeat from core client for 30 sec - exiting 14:46:11 (48036): No heartbeat from core client for 30 sec - exiting 14:46:12 (48036): No heartbeat from core client for 30 sec - exiting 14:46:13 (48036): No heartbeat from core client for 30 sec - exiting 14:46:14 (48036): No heartbeat from core client for 30 sec - exiting 14:46:15 (48036): No heartbeat from core client for 30 sec - exiting 14:46:16 (48036): No heartbeat from core client for 30 sec - exiting 14:46:17 (48036): No heartbeat from core client for 30 sec - exiting 14:46:18 (48036): No heartbeat from core client for 30 sec - exiting 14:46:19 (48036): No heartbeat from core client for 30 sec - exiting 14:46:20 (48036): No heartbeat from core client for 30 sec - exiting 14:46:21 (48036): No heartbeat from core client for 30 sec - exiting 14:46:22 (48036): No heartbeat from core client for 30 sec - exiting 14:46:23 (48036): No heartbeat from core client for 30 sec - exiting 14:46:24 (48036): No heartbeat from core client for 30 sec - exiting 14:46:25 (48036): No heartbeat from core client for 30 sec - exiting 14:46:26 (48036): No heartbeat from core client for 30 sec - exiting 14:46:27 (48036): No heartbeat from core client for 30 sec - exiting 14:46:28 (48036): No heartbeat from core client for 30 sec - exiting 14:46:29 (48036): No heartbeat from core client for 30 sec - exiting 14:46:30 (48036): No heartbeat from core client for 30 sec - exiting 14:46:31 (48036): No heartbeat from core client for 30 sec - exiting 14:46:32 (48036): No heartbeat from core client for 30 sec - exiting 14:46:33 (48036): No heartbeat from core client for 30 sec - exiting 14:46:34 (48036): No heartbeat from core client for 30 sec - exiting 14:46:35 (48036): No heartbeat from core client for 30 sec - exiting 14:46:36 (48036): No heartbeat from core client for 30 sec - exiting 14:46:37 (48036): No heartbeat from core client for 30 sec - exiting 14:46:38 (48036): No heartbeat from core client for 30 sec - exiting 14:46:39 (48036): No heartbeat from core client for 30 sec - exiting 14:46:40 (48036): No heartbeat from core client for 30 sec - exiting 14:46:41 (48036): No heartbeat from core client for 30 sec - exiting 14:46:42 (48036): No heartbeat from core client for 30 sec - exiting 14:46:43 (48036): No heartbeat from core client for 30 sec - exiting 14:46:44 (48036): No heartbeat from core client for 30 sec - exiting 14:46:45 (48036): No heartbeat from core client for 30 sec - exiting 14:46:46 (48036): No heartbeat from core client for 30 sec - exiting 14:46:47 (48036): No heartbeat from core client for 30 sec - exiting 14:46:48 (48036): No heartbeat from core client for 30 sec - exiting 14:46:49 (48036): No heartbeat from core client for 30 sec - exiting 14:46:50 (48036): No heartbeat from core client for 30 sec - exiting 14:46:51 (48036): No heartbeat from core client for 30 sec - exiting 14:46:52 (48036): No heartbeat from core client for 30 sec - exiting 14:46:53 (48036): No heartbeat from core client for 30 sec - exiting 14:46:54 (48036): No heartbeat from core client for 30 sec - exiting 14:46:55 (48036): No heartbeat from core client for 30 sec - exiting 14:46:57 (48036): No heartbeat from core client for 30 sec - exiting 14:46:58 (48036): No heartbeat from core client for 30 sec - exiting 14:46:59 (48036): No heartbeat from core client for 30 sec - exiting 14:47:00 (48036): No heartbeat from core client for 30 sec - exiting 14:47:01 (48036): No heartbeat from core client for 30 sec - exiting 14:47:02 (48036): No heartbeat from core client for 30 sec - exiting 14:47:03 (48036): No heartbeat from core client for 30 sec - exiting 14:47:04 (48036): No heartbeat from core client for 30 sec - exiting 14:47:05 (48036): No heartbeat from core client for 30 sec - exiting 14:47:06 (48036): No heartbeat from core client for 30 sec - exiting 14:47:07 (48036): No heartbeat from core client for 30 sec - exiting 14:47:08 (48036): No heartbeat from core client for 30 sec - exiting 14:47:09 (48036): No heartbeat from core client for 30 sec - exiting 14:47:10 (48036): No heartbeat from core client for 30 sec - exiting 14:47:11 (48036): No heartbeat from core client for 30 sec - exiting 14:47:12 (48036): No heartbeat from core client for 30 sec - exiting 14:47:13 (48036): No heartbeat from core client for 30 sec - exiting 14:47:14 (48036): No heartbeat from core client for 30 sec - exiting 14:47:15 (48036): No heartbeat from core client for 30 sec - exiting 14:47:16 (48036): No heartbeat from core client for 30 sec - exiting 14:47:17 (48036): No heartbeat from core client for 30 sec - exiting 14:47:18 (48036): No heartbeat from core client for 30 sec - exiting 14:47:19 (48036): No heartbeat from core client for 30 sec - exiting 14:47:20 (48036): No heartbeat from core client for 30 sec - exiting 14:47:21 (48036): No heartbeat from core client for 30 sec - exiting 14:47:22 (48036): No heartbeat from core client for 30 sec - exiting 14:47:23 (48036): No heartbeat from core client for 30 sec - exiting 14:47:24 (48036): No heartbeat from core client for 30 sec - exiting 14:47:25 (48036): No heartbeat from core client for 30 sec - exiting 14:47:26 (48036): No heartbeat from core client for 30 sec - exiting 14:47:27 (48036): No heartbeat from core client for 30 sec - exiting 14:47:28 (48036): No heartbeat from core client for 30 sec - exiting 14:47:29 (48036): No heartbeat from core client for 30 sec - exiting 14:47:30 (48036): No heartbeat from core client for 30 sec - exiting 14:47:31 (48036): No heartbeat from core client for 30 sec - exiting 14:47:32 (48036): No heartbeat from core client for 30 sec - exiting 14:47:33 (48036): No heartbeat from core client for 30 sec - exiting 14:47:34 (48036): No heartbeat from core client for 30 sec - exiting 14:47:35 (48036): No heartbeat from core client for 30 sec - exiting 14:47:36 (48036): No heartbeat from core client for 30 sec - exiting 14:47:37 (48036): No heartbeat from core client for 30 sec - exiting 14:47:38 (48036): No heartbeat from core client for 30 sec - exiting 14:47:39 (48036): No heartbeat from core client for 30 sec - exiting 14:47:40 (48036): No heartbeat from core client for 30 sec - exiting 14:47:41 (48036): No heartbeat from core client for 30 sec - exiting 14:47:42 (48036): No heartbeat from core client for 30 sec - exiting 14:47:43 (48036): No heartbeat from core client for 30 sec - exiting 14:47:44 (48036): No heartbeat from core client for 30 sec - exiting 14:47:45 (48036): No heartbeat from core client for 30 sec - exiting 14:47:46 (48036): No heartbeat from core client for 30 sec - exiting 14:47:47 (48036): No heartbeat from core client for 30 sec - exiting 14:47:48 (48036): No heartbeat from core client for 30 sec - exiting 14:47:49 (48036): No heartbeat from core client for 30 sec - exiting 14:47:50 (48036): No heartbeat from core client for 30 sec - exiting 14:47:51 (48036): No heartbeat from core client for 30 sec - exiting 14:47:52 (48036): No heartbeat from core client for 30 sec - exiting 14:47:53 (48036): No heartbeat from core client for 30 sec - exiting 14:47:54 (48036): No heartbeat from core client for 30 sec - exiting 14:47:55 (48036): No heartbeat from core client for 30 sec - exiting 14:47:56 (48036): No heartbeat from core client for 30 sec - exiting 14:47:57 (48036): No heartbeat from core client for 30 sec - exiting 14:47:58 (48036): No heartbeat from core client for 30 sec - exiting 14:47:59 (48036): No heartbeat from core client for 30 sec - exiting 14:48:00 (48036): No heartbeat from core client for 30 sec - exiting 14:48:01 (48036): No heartbeat from core client for 30 sec - exiting 14:48:02 (48036): No heartbeat from core client for 30 sec - exiting 14:48:03 (48036): No heartbeat from core client for 30 sec - exiting 14:48:04 (48036): No heartbeat from core client for 30 sec - exiting 14:48:05 (48036): No heartbeat from core client for 30 sec - exiting 14:48:06 (48036): No heartbeat from core client for 30 sec - exiting 14:48:07 (48036): No heartbeat from core client for 30 sec - exiting 14:48:08 (48036): No heartbeat from core client for 30 sec - exiting 14:48:09 (48036): No heartbeat from core client for 30 sec - exiting 14:48:10 (48036): No heartbeat from core client for 30 sec - exiting 14:48:11 (48036): No heartbeat from core client for 30 sec - exiting 14:48:12 (48036): No heartbeat from core client for 30 sec - exiting 14:48:13 (48036): No heartbeat from core client for 30 sec - exiting 14:48:14 (48036): No heartbeat from core client for 30 sec - exiting 14:48:15 (48036): No heartbeat from core client for 30 sec - exiting 14:48:16 (48036): No heartbeat from core client for 30 sec - exiting 14:48:17 (48036): No heartbeat from core client for 30 sec - exiting 14:48:18 (48036): No heartbeat from core client for 30 sec - exiting 14:48:19 (48036): No heartbeat from core client for 30 sec - exiting 14:48:20 (48036): No heartbeat from core client for 30 sec - exiting 14:48:21 (48036): No heartbeat from core client for 30 sec - exiting 14:48:22 (48036): No heartbeat from core client for 30 sec - exiting 14:48:23 (48036): No heartbeat from core client for 30 sec - exiting 14:48:24 (48036): No heartbeat from core client for 30 sec - exiting 14:48:25 (48036): No heartbeat from core client for 30 sec - exiting 14:48:26 (48036): No heartbeat from core client for 30 sec - exiting 14:48:27 (48036): No heartbeat from core client for 30 sec - exiting 14:48:28 (48036): No heartbeat from core client for 30 sec - exiting 14:48:29 (48036): No heartbeat from core client for 30 sec - exiting 14:48:30 (48036): No heartbeat from core client for 30 sec - exiting 14:48:31 (48036): No heartbeat from core client for 30 sec - exiting 14:48:32 (48036): No heartbeat from core client for 30 sec - exiting 14:48:33 (48036): No heartbeat from core client for 30 sec - exiting 14:48:34 (48036): No heartbeat from core client for 30 sec - exiting 14:48:35 (48036): No heartbeat from core client for 30 sec - exiting 14:48:37 (48036): No heartbeat from core client for 30 sec - exiting 14:48:38 (48036): No heartbeat from core client for 30 sec - exiting 14:48:39 (48036): No heartbeat from core client for 30 sec - exiting 14:48:40 (48036): No heartbeat from core client for 30 sec - exiting 14:48:41 (48036): No heartbeat from core client for 30 sec - exiting 14:48:42 (48036): No heartbeat from core client for 30 sec - exiting 14:48:43 (48036): No heartbeat from core client for 30 sec - exiting 14:48:44 (48036): No heartbeat from core client for 30 sec - exiting 14:48:45 (48036): No heartbeat from core client for 30 sec - exiting 14:48:46 (48036): No heartbeat from core client for 30 sec - exiting 14:48:47 (48036): No heartbeat from core client for 30 sec - exiting 14:48:48 (48036): No heartbeat from core client for 30 sec - exiting 14:48:49 (48036): No heartbeat from core client for 30 sec - exiting 14:48:50 (48036): No heartbeat from core client for 30 sec - exiting 14:48:51 (48036): No heartbeat from core client for 30 sec - exiting 14:48:52 (48036): No heartbeat from core client for 30 sec - exiting 14:48:53 (48036): No heartbeat from core client for 30 sec - exiting 14:48:54 (48036): No heartbeat from core client for 30 sec - exiting 14:48:55 (48036): No heartbeat from core client for 30 sec - exiting 14:48:56 (48036): No heartbeat from core client for 30 sec - exiting 14:48:57 (48036): No heartbeat from core client for 30 sec - exiting 14:48:58 (48036): No heartbeat from core client for 30 sec - exiting 14:48:59 (48036): No heartbeat from core client for 30 sec - exiting 14:49:00 (48036): No heartbeat from core client for 30 sec - exiting 14:49:01 (48036): No heartbeat from core client for 30 sec - exiting 14:49:02 (48036): No heartbeat from core client for 30 sec - exiting 14:49:03 (48036): No heartbeat from core client for 30 sec - exiting 14:49:04 (48036): No heartbeat from core client for 30 sec - exiting 14:49:05 (48036): No heartbeat from core client for 30 sec - exiting 14:49:06 (48036): No heartbeat from core client for 30 sec - exiting 14:49:07 (48036): No heartbeat from core client for 30 sec - exiting 14:49:08 (48036): No heartbeat from core client for 30 sec - exiting 14:49:09 (48036): No heartbeat from core client for 30 sec - exiting 14:49:10 (48036): No heartbeat from core client for 30 sec - exiting 14:49:11 (48036): No heartbeat from core client for 30 sec - exiting 14:49:12 (48036): No heartbeat from core client for 30 sec - exiting 14:49:13 (48036): No heartbeat from core client for 30 sec - exiting 14:49:14 (48036): No heartbeat from core client for 30 sec - exiting 14:49:15 (48036): No heartbeat from core client for 30 sec - exiting 14:49:16 (48036): No heartbeat from core client for 30 sec - exiting 14:49:17 (48036): No heartbeat from core client for 30 sec - exiting 14:49:18 (48036): No heartbeat from core client for 30 sec - exiting 14:49:19 (48036): No heartbeat from core client for 30 sec - exiting 14:49:20 (48036): No heartbeat from core client for 30 sec - exiting 14:49:21 (48036): No heartbeat from core client for 30 sec - exiting 14:49:22 (48036): No heartbeat from core client for 30 sec - exiting 14:49:23 (48036): No heartbeat from core client for 30 sec - exiting 14:49:24 (48036): No heartbeat from core client for 30 sec - exiting 14:49:25 (48036): No heartbeat from core client for 30 sec - exiting 14:49:26 (48036): No heartbeat from core client for 30 sec - exiting 14:49:27 (48036): No heartbeat from core client for 30 sec - exiting 14:49:28 (48036): No heartbeat from core client for 30 sec - exiting 14:49:29 (48036): No heartbeat from core client for 30 sec - exiting 14:49:30 (48036): No heartbeat from core client for 30 sec - exiting 14:49:31 (48036): No heartbeat from core client for 30 sec - exiting 14:49:32 (48036): No heartbeat from core client for 30 sec - exiting 14:49:33 (48036): No heartbeat from core client for 30 sec - exiting 14:49:34 (48036): No heartbeat from core client for 30 sec - exiting 14:49:35 (48036): No heartbeat from core client for 30 sec - exiting 14:49:36 (48036): No heartbeat from core client for 30 sec - exiting 14:49:37 (48036): No heartbeat from core client for 30 sec - exiting 14:49:38 (48036): No heartbeat from core client for 30 sec - exiting 14:49:39 (48036): No heartbeat from core client for 30 sec - exiting 14:49:40 (48036): No heartbeat from core client for 30 sec - exiting 14:49:41 (48036): No heartbeat from core client for 30 sec - exiting 14:49:42 (48036): No heartbeat from core client for 30 sec - exiting 14:49:43 (48036): No heartbeat from core client for 30 sec - exiting 14:49:44 (48036): No heartbeat from core client for 30 sec - exiting 14:49:45 (48036): No heartbeat from core client for 30 sec - exiting 14:49:46 (48036): No heartbeat from core client for 30 sec - exiting 14:49:47 (48036): No heartbeat from core client for 30 sec - exiting 14:49:48 (48036): No heartbeat from core client for 30 sec - exiting 14:49:49 (48036): No heartbeat from core client for 30 sec - exiting 14:49:50 (48036): No heartbeat from core client for 30 sec - exiting 14:49:51 (48036): No heartbeat from core client for 30 sec - exiting 14:49:52 (48036): No heartbeat from core client for 30 sec - exiting 14:49:53 (48036): No heartbeat from core client for 30 sec - exiting 14:49:54 (48036): No heartbeat from core client for 30 sec - exiting 14:49:55 (48036): No heartbeat from core client for 30 sec - exiting 14:49:56 (48036): No heartbeat from core client for 30 sec - exiting 14:49:57 (48036): No heartbeat from core client for 30 sec - exiting 14:49:58 (48036): No heartbeat from core client for 30 sec - exiting 14:49:59 (48036): No heartbeat from core client for 30 sec - exiting 14:50:00 (48036): No heartbeat from core client for 30 sec - exiting 14:50:01 (48036): No heartbeat from core client for 30 sec - exiting 14:50:02 (48036): No heartbeat from core client for 30 sec - exiting 14:50:03 (48036): No heartbeat from core client for 30 sec - exiting 14:50:04 (48036): No heartbeat from core client for 30 sec - exiting 14:50:05 (48036): No heartbeat from core client for 30 sec - exiting 14:50:06 (48036): No heartbeat from core client for 30 sec - exiting 14:50:07 (48036): No heartbeat from core client for 30 sec - exiting 14:50:09 (48036): No heartbeat from core client for 30 sec - exiting 14:50:10 (48036): No heartbeat from core client for 30 sec - exiting 14:50:11 (48036): No heartbeat from core client for 30 sec - exiting 14:50:12 (48036): No heartbeat from core client for 30 sec - exiting 14:50:13 (48036): No heartbeat from core client for 30 sec - exiting 14:50:14 (48036): No heartbeat from core client for 30 sec - exiting 14:50:15 (48036): No heartbeat from core client for 30 sec - exiting 14:50:16 (48036): No heartbeat from core client for 30 sec - exiting 14:50:17 (48036): No heartbeat from core client for 30 sec - exiting 14:50:18 (48036): No heartbeat from core client for 30 sec - exiting 14:50:19 (48036): No heartbeat from core client for 30 sec - exiting 14:50:20 (48036): No heartbeat from core client for 30 sec - exiting 14:50:21 (48036): No heartbeat from core client for 30 sec - exiting 14:50:22 (48036): No heartbeat from core client for 30 sec - exiting 14:50:23 (48036): No heartbeat from core client for 30 sec - exiting 14:50:24 (48036): No heartbeat from core client for 30 sec - exiting 14:50:25 (48036): No heartbeat from core client for 30 sec - exiting 14:50:26 (48036): No heartbeat from core client for 30 sec - exiting 14:50:27 (48036): No heartbeat from core client for 30 sec - exiting 14:50:28 (48036): No heartbeat from core client for 30 sec - exiting 14:50:29 (48036): No heartbeat from core client for 30 sec - exiting 14:50:30 (48036): No heartbeat from core client for 30 sec - exiting 14:50:31 (48036): No heartbeat from core client for 30 sec - exiting 14:50:32 (48036): No heartbeat from core client for 30 sec - exiting 14:50:33 (48036): No heartbeat from core client for 30 sec - exiting 14:50:34 (48036): No heartbeat from core client for 30 sec - exiting 14:50:35 (48036): No heartbeat from core client for 30 sec - exiting 14:50:36 (48036): No heartbeat from core client for 30 sec - exiting 14:50:37 (48036): No heartbeat from core client for 30 sec - exiting 14:50:38 (48036): No heartbeat from core client for 30 sec - exiting 14:50:39 (48036): No heartbeat from core client for 30 sec - exiting 14:50:40 (48036): No heartbeat from core client for 30 sec - exiting 14:50:41 (48036): No heartbeat from core client for 30 sec - exiting 14:50:42 (48036): No heartbeat from core client for 30 sec - exiting 14:50:43 (48036): No heartbeat from core client for 30 sec - exiting 14:50:44 (48036): No heartbeat from core client for 30 sec - exiting 14:50:45 (48036): No heartbeat from core client for 30 sec - exiting 14:50:46 (48036): No heartbeat from core client for 30 sec - exiting 14:50:47 (48036): No heartbeat from core client for 30 sec - exiting 14:50:48 (48036): No heartbeat from core client for 30 sec - exiting 14:50:49 (48036): No heartbeat from core client for 30 sec - exiting 14:50:50 (48036): No heartbeat from core client for 30 sec - exiting 14:50:51 (48036): No heartbeat from core client for 30 sec - exiting 14:50:52 (48036): No heartbeat from core client for 30 sec - exiting 14:50:53 (48036): No heartbeat from core client for 30 sec - exiting 14:50:54 (48036): No heartbeat from core client for 30 sec - exiting 14:50:55 (48036): No heartbeat from core client for 30 sec - exiting 14:50:56 (48036): No heartbeat from core client for 30 sec - exiting 14:50:57 (48036): No heartbeat from core client for 30 sec - exiting 14:50:58 (48036): No heartbeat from core client for 30 sec - exiting 14:50:59 (48036): No heartbeat from core client for 30 sec - exiting 14:51:00 (48036): No heartbeat from core client for 30 sec - exiting 14:51:01 (48036): No heartbeat from core client for 30 sec - exiting 14:51:02 (48036): No heartbeat from core client for 30 sec - exiting 14:51:03 (48036): No heartbeat from core client for 30 sec - exiting 14:51:04 (48036): No heartbeat from core client for 30 sec - exiting 14:51:05 (48036): No heartbeat from core client for 30 sec - exiting 14:51:06 (48036): No heartbeat from core client for 30 sec - exiting 14:51:07 (48036): No heartbeat from core client for 30 sec - exiting 14:51:08 (48036): No heartbeat from core client for 30 sec - exiting 14:51:09 (48036): No heartbeat from core client for 30 sec - exiting 14:51:10 (48036): No heartbeat from core client for 30 sec - exiting 14:51:11 (48036): No heartbeat from core client for 30 sec - exiting 14:51:12 (48036): No heartbeat from core client for 30 sec - exiting 14:51:13 (48036): No heartbeat from core client for 30 sec - exiting 14:51:14 (48036): No heartbeat from core client for 30 sec - exiting 14:51:15 (48036): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(173,0xa025b540) malloc: * error for object 0x1001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Apr 2012 16:56:47	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	1,036,800	981,542	0.9467
04 Apr 2012 05:59:00	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	1,010,880	943,594	0.9334
03 Apr 2012 18:23:08	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	984,960	905,819	0.9197
03 Apr 2012 07:12:33	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	959,040	867,139	0.9042
02 Apr 2012 19:59:40	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	933,120	828,633	0.8880
02 Apr 2012 08:43:16	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	907,200	1,200,379	1.3232
01 Apr 2012 21:16:43	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	881,280	1,160,982	1.3174
01 Apr 2012 10:05:58	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	855,360	1,122,341	1.3121
31 Mar 2012 23:00:01	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	829,440	1,084,075	1.3070
31 Mar 2012 11:53:04	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	803,520	1,045,542	1.3012
31 Mar 2012 00:40:27	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	777,600	1,006,563	1.2944
30 Mar 2012 13:41:46	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	751,680	968,649	1.2886
30 Mar 2012 02:29:42	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	725,760	929,843	1.2812
29 Mar 2012 15:24:57	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	699,840	891,466	1.2738
29 Mar 2012 04:10:46	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	673,920	852,680	1.2653
28 Mar 2012 17:08:24	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	648,000	814,318	1.2567
28 Mar 2012 05:54:22	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	622,080	820,473	1.3189
27 Mar 2012 18:49:04	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	596,160	874,230	1.4664
27 Mar 2012 07:53:48	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	570,240	836,456	1.4668
26 Mar 2012 20:49:18	1176286	13787137	hadcm3n_o619_1940_40_007545401_2	544,320	798,070	1.4662