Task 15493767

Name	hadcm3n_38q7_1940_40_008263980_1
Workunit	8419104
Created	21 Dec 2012, 10:00:21 UTC
Sent	21 Dec 2012, 10:09:40 UTC
Report deadline	22 Mar 2013, 17:36:51 UTC
Received	26 Jan 2013, 8:50:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	958046
Run time	19 days 18 hours 41 min 53 sec
CPU time	17 days 22 hours 14 min 1 sec
Validate state	Invalid
Credit	10,886.40
Device peak FLOPS	2.98 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> . Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:31:48 (24044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/38q7ko.pjf9c10 is not a valid UM file. Error converting file to netcdf: dataout/38q7ko.pjf9c10 Error: Input file: dataout/38q7ko.pif9c10 is not a valid UM file. Error converting file to netcdf: dataout/38q7ko.pif9c10 Error: Input file: dataout/38q7ko.pff9c10 is not a valid UM file. Error converting file to netcdf: dataout/38q7ko.pff9c10 Error: Input file: dataout/38q7ka.phf9c10 is not a valid UM file. Error converting file to netcdf: dataout/38q7ka.phf9c10 Error: Input file: dataout/38q7ka.pgf9c10 is not a valid UM file. Error converting file to netcdf: dataout/38q7ka.pgf9c10 Error: Input file: dataout/38q7ka.pef9c10 is not a valid UM file. Error converting file to netcdf: dataout/38q7ka.pef9c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:03:41 (70213): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:12:08 (74591): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:07:44 (93345): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16303,0xa013d540) malloc: * error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:51:11 (99924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:56:50 (85018): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:02:55 (92193): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:23:24 (93395): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:21:36 (96723): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:22:48 (15680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:45 (15983): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:28:31 (16020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:48:41 (16234): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:49:45 (26245): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:51:36 (26328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:51:37 (26328): No heartbeat from core client for 30 sec - exiting 22:51:38 (26328): No heartbeat from core client for 30 sec - exiting 22:51:39 (26328): No heartbeat from core client for 30 sec - exiting 22:51:40 (26328): No heartbeat from core client for 30 sec - exiting 22:51:41 (26328): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:19:19 (9828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:19 (9983): No heartbeat from core client for 30 sec - exiting 16:22:20 (9983): No heartbeat from core client for 30 sec - exiting 16:22:21 (9983): No heartbeat from core client for 30 sec - exiting 16:22:22 (9983): No heartbeat from core client for 30 sec - exiting 16:22:23 (9983): No heartbeat from core client for 30 sec - exiting 16:22:24 (9983): No heartbeat from core client for 30 sec - exiting 16:22:25 (9983): No heartbeat from core client for 30 sec - exiting 16:22:26 (9983): No heartbeat from core client for 30 sec - exiting 16:22:27 (9983): No heartbeat from core client for 30 sec - exiting 16:22:28 (9983): No heartbeat from core client for 30 sec - exiting 16:22:29 (9983): No heartbeat from core client for 30 sec - exiting 16:22:30 (9983): No heartbeat from core client for 30 sec - exiting 16:22:31 (9983): No heartbeat from core client for 30 sec - exiting 16:22:32 (9983): No heartbeat from core client for 30 sec - exiting 16:22:33 (9983): No heartbeat from core client for 30 sec - exiting 16:22:34 (9983): No heartbeat from core client for 30 sec - exiting 16:22:35 (9983): No heartbeat from core client for 30 sec - exiting 16:22:36 (9983): No heartbeat from core client for 30 sec - exiting 16:22:37 (9983): No heartbeat from core client for 30 sec - exiting 16:22:38 (9983): No heartbeat from core client for 30 sec - exiting 16:22:39 (9983): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:54 (10100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:55 (10100): No heartbeat from core client for 30 sec - exiting 16:28:56 (10100): No heartbeat from core client for 30 sec - exiting 16:28:57 (10100): No heartbeat from core client for 30 sec - exiting 16:28:58 (10100): No heartbeat from core client for 30 sec - exiting 16:28:59 (10100): No heartbeat from core client for 30 sec - exiting 16:29:00 (10100): No heartbeat from core client for 30 sec - exiting 16:29:01 (10100): No heartbeat from core client for 30 sec - exiting 16:29:02 (10100): No heartbeat from core client for 30 sec - exiting 16:29:03 (10100): No heartbeat from core client for 30 sec - exiting 16:29:04 (10100): No heartbeat from core client for 30 sec - exiting 16:29:05 (10100): No heartbeat from core client for 30 sec - exiting 16:29:06 (10100): No heartbeat from core client for 30 sec - exiting 16:29:07 (10100): No heartbeat from core client for 30 sec - exiting 16:29:08 (10100): No heartbeat from core client for 30 sec - exiting 16:29:09 (10100): No heartbeat from core client for 30 sec - exiting 16:29:10 (10100): No heartbeat from core client for 30 sec - exiting 16:29:11 (10100): No heartbeat from core client for 30 sec - exiting 16:29:12 (10100): No heartbeat from core client for 30 sec - exiting 16:29:13 (10100): No heartbeat from core client for 30 sec - exiting 16:29:14 (10100): No heartbeat from core client for 30 sec - exiting 16:29:15 (10100): No heartbeat from core client for 30 sec - exiting 16:29:16 (10100): No heartbeat from core client for 30 sec - exiting 16:29:17 (10100): No heartbeat from core client for 30 sec - exiting 16:29:18 (10100): No heartbeat from core client for 30 sec - exiting 16:29:19 (10100): No heartbeat from core client for 30 sec - exiting 16:29:20 (10100): No heartbeat from core client for 30 sec - exiting 16:29:21 (10100): No heartbeat from core client for 30 sec - exiting 16:29:22 (10100): No heartbeat from core client for 30 sec - exiting 16:29:53 (10100): No heartbeat from core client for 30 sec - exiting 16:29:54 (10100): No heartbeat from core client for 30 sec - exiting 16:29:55 (10100): No heartbeat from core client for 30 sec - exiting 16:29:56 (10100): No heartbeat from core client for 30 sec - exiting 16:29:57 (10100): No heartbeat from core client for 30 sec - exiting 16:29:58 (10100): No heartbeat from core client for 30 sec - exiting 16:29:59 (10100): No heartbeat from core client for 30 sec - exiting 16:30:00 (10100): No heartbeat from core client for 30 sec - exiting 16:30:01 (10100): No heartbeat from core client for 30 sec - exiting 16:30:02 (10100): No heartbeat from core client for 30 sec - exiting 16:30:03 (10100): No heartbeat from core client for 30 sec - exiting 16:30:04 (10100): No heartbeat from core client for 30 sec - exiting 16:30:05 (10100): No heartbeat from core client for 30 sec - exiting 16:30:06 (10100): No heartbeat from core client for 30 sec - exiting 16:30:07 (10100): No heartbeat from core client for 30 sec - exiting 16:30:08 (10100): No heartbeat from core client for 30 sec - exiting 16:30:09 (10100): No heartbeat from core client for 30 sec - exiting 16:30:10 (10100): No heartbeat from core client for 30 sec - exiting 16:30:11 (10100): No heartbeat from core client for 30 sec - exiting 16:30:12 (10100): No heartbeat from core client for 30 sec - exiting 16:30:13 (10100): No heartbeat from core client for 30 sec - exiting 16:30:14 (10100): No heartbeat from core client for 30 sec - exiting 16:30:15 (10100): No heartbeat from core client for 30 sec - exiting 16:30:16 (10100): No heartbeat from core client for 30 sec - exiting 16:30:17 (10100): No heartbeat from core client for 30 sec - exiting 16:30:18 (10100): No heartbeat from core client for 30 sec - exiting 16:30:19 (10100): No heartbeat from core client for 30 sec - exiting 16:30:20 (10100): No heartbeat from core client for 30 sec - exiting 16:30:21 (10100): No heartbeat from core client for 30 sec - exiting 16:30:22 (10100): No heartbeat from core client for 30 sec - exiting 16:30:23 (10100): No heartbeat from core client for 30 sec - exiting 16:30:24 (10100): No heartbeat from core client for 30 sec - exiting 16:30:25 (10100): No heartbeat from core client for 30 sec - exiting 16:30:26 (10100): No heartbeat from core client for 30 sec - exiting 16:30:27 (10100): No heartbeat from core client for 30 sec - exiting 16:30:28 (10100): No heartbeat from core client for 30 sec - exiting 16:30:29 (10100): No heartbeat from core client for 30 sec - exiting 16:30:30 (10100): No heartbeat from core client for 30 sec - exiting 16:30:31 (10100): No heartbeat from core client for 30 sec - exiting 16:30:32 (10100): No heartbeat from core client for 30 sec - exiting 16:30:33 (10100): No heartbeat from core client for 30 sec - exiting 16:30:34 (10100): No heartbeat from core client for 30 sec - exiting 16:30:35 (10100): No heartbeat from core client for 30 sec - exiting 16:30:36 (10100): No heartbeat from core client for 30 sec - exiting 16:30:37 (10100): No heartbeat from core client for 30 sec - exiting 16:30:38 (10100): No heartbeat from core client for 30 sec - exiting 16:30:39 (10100): No heartbeat from core client for 30 sec - exiting 16:30:40 (10100): No heartbeat from core client for 30 sec - exiting 16:30:41 (10100): No heartbeat from core client for 30 sec - exiting 16:30:42 (10100): No heartbeat from core client for 30 sec - exiting 16:30:43 (10100): No heartbeat from core client for 30 sec - exiting 16:30:44 (10100): No heartbeat from core client for 30 sec - exiting 16:30:45 (10100): No heartbeat from core client for 30 sec - exiting 16:30:46 (10100): No heartbeat from core client for 30 sec - exiting 16:30:47 (10100): No heartbeat from core client for 30 sec - exiting 16:30:48 (10100): No heartbeat from core client for 30 sec - exiting 16:30:49 (10100): No heartbeat from core client for 30 sec - exiting 16:30:50 (10100): No heartbeat from core client for 30 sec - exiting 16:30:51 (10100): No heartbeat from core client for 30 sec - exiting 16:30:52 (10100): No heartbeat from core client for 30 sec - exiting 16:30:53 (10100): No heartbeat from core client for 30 sec - exiting 16:30:54 (10100): No heartbeat from core client for 30 sec - exiting 16:30:55 (10100): No heartbeat from core client for 30 sec - exiting 16:30:56 (10100): No heartbeat from core client for 30 sec - exiting 16:30:57 (10100): No heartbeat from core client for 30 sec - exiting 16:30:58 (10100): No heartbeat from core client for 30 sec - exiting 16:30:59 (10100): No heartbeat from core client for 30 sec - exiting 16:31:00 (10100): No heartbeat from core client for 30 sec - exiting 16:31:01 (10100): No heartbeat from core client for 30 sec - exiting 16:31:02 (10100): No heartbeat from core client for 30 sec - exiting 16:31:03 (10100): No heartbeat from core client for 30 sec - exiting 16:31:04 (10100): No heartbeat from core client for 30 sec - exiting 16:31:05 (10100): No heartbeat from core client for 30 sec - exiting 16:31:06 (10100): No heartbeat from core client for 30 sec - exiting 16:31:07 (10100): No heartbeat from core client for 30 sec - exiting 16:31:08 (10100): No heartbeat from core client for 30 sec - exiting 16:31:09 (10100): No heartbeat from core client for 30 sec - exiting 16:31:10 (10100): No heartbeat from core client for 30 sec - exiting 16:32:40 (10493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:31:10 (13873): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:09:43 (3103): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:18:03 (6982): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:54:23 (9004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:29:45 (14404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:35:31 (21492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:39:28 (21604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 21:40:49 (21733): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:41 (21765): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:42 (21765): No heartbeat from core client for 30 sec - exiting 21:45:43 (21765): No heartbeat from core client for 30 sec - exiting 21:45:44 (21765): No heartbeat from core client for 30 sec - exiting 21:45:45 (21765): No heartbeat from core client for 30 sec - exiting 21:45:46 (21765): No heartbeat from core client for 30 sec - exiting 21:45:47 (21765): No heartbeat from core client for 30 sec - exiting 21:45:48 (21765): No heartbeat from core client for 30 sec - exiting 21:45:49 (21765): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Suspended CPDN Monitor - Suspend request from BOINC... 21:51:46 (22168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:51:47 (22168): No heartbeat from core client for 30 sec - exiting 21:51:48 (22168): No heartbeat from core client for 30 sec - exiting 21:51:49 (22168): No heartbeat from core client for 30 sec - exiting 21:51:50 (22168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:58:57 (69902): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:58:58 (69902): No heartbeat from core client for 30 sec - exiting 12:23:58 (69969): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:24:02 (69969): No heartbeat from core client for 30 sec - exiting 12:24:03 (69969): No heartbeat from core client for 30 sec - exiting 12:24:04 (69969): No heartbeat from core client for 30 sec - exiting 12:24:05 (69969): No heartbeat from core client for 30 sec - exiting 12:24:06 (69969): No heartbeat from core client for 30 sec - exiting 12:24:07 (69969): No heartbeat from core client for 30 sec - exiting 12:24:08 (69969): No heartbeat from core client for 30 sec - exiting 12:24:09 (69969): No heartbeat from core client for 30 sec - exiting 12:24:10 (69969): No heartbeat from core client for 30 sec - exiting 12:24:11 (69969): No heartbeat from core client for 30 sec - exiting 12:24:12 (69969): No heartbeat from core client for 30 sec - exiting 12:24:13 (69969): No heartbeat from core client for 30 sec - exiting 12:24:14 (69969): No heartbeat from core client for 30 sec - exiting 12:24:15 (69969): No heartbeat from core client for 30 sec - exiting 12:24:16 (69969): No heartbeat from core client for 30 sec - exiting 12:24:17 (69969): No heartbeat from core client for 30 sec - exiting 12:24:18 (69969): No heartbeat from core client for 30 sec - exiting 12:24:19 (69969): No heartbeat from core client for 30 sec - exiting 12:24:20 (69969): No heartbeat from core client for 30 sec - exiting 12:24:21 (69969): No heartbeat from core client for 30 sec - exiting 12:24:22 (69969): No heartbeat from core client for 30 sec - exiting 12:24:23 (69969): No heartbeat from core client for 30 sec - exiting 12:24:24 (69969): No heartbeat from core client for 30 sec - exiting 12:24:25 (69969): No heartbeat from core client for 30 sec - exiting 12:24:26 (69969): No heartbeat from core client for 30 sec - exiting 12:24:27 (69969): No heartbeat from core client for 30 sec - exiting 12:24:28 (69969): No heartbeat from core client for 30 sec - exiting 12:24:29 (69969): No heartbeat from core client for 30 sec - exiting 12:25:00 (69969): No heartbeat from core client for 30 sec - exiting 12:25:01 (69969): No heartbeat from core client for 30 sec - exiting 12:25:02 (69969): No heartbeat from core client for 30 sec - exiting 12:25:03 (69969): No heartbeat from core client for 30 sec - exiting 12:25:04 (69969): No heartbeat from core client for 30 sec - exiting 12:25:05 (69969): No heartbeat from core client for 30 sec - exiting 12:25:06 (69969): No heartbeat from core client for 30 sec - exiting 12:25:07 (69969): No heartbeat from core client for 30 sec - exiting 12:25:08 (69969): No heartbeat from core client for 30 sec - exiting 12:25:09 (69969): No heartbeat from core client for 30 sec - exiting 12:25:10 (69969): No heartbeat from core client for 30 sec - exiting 12:25:11 (69969): No heartbeat from core client for 30 sec - exiting 12:25:12 (69969): No heartbeat from core client for 30 sec - exiting 12:25:13 (69969): No heartbeat from core client for 30 sec - exiting 12:25:14 (69969): No heartbeat from core client for 30 sec - exiting 12:25:15 (69969): No heartbeat from core client for 30 sec - exiting 12:25:16 (69969): No heartbeat from core client for 30 sec - exiting 12:25:17 (69969): No heartbeat from core client for 30 sec - exiting 12:25:18 (69969): No heartbeat from core client for 30 sec - exiting 12:25:19 (69969): No heartbeat from core client for 30 sec - exiting 12:25:20 (69969): No heartbeat from core client for 30 sec - exiting 12:25:21 (69969): No heartbeat from core client for 30 sec - exiting 12:25:22 (69969): No heartbeat from core client for 30 sec - exiting 12:25:23 (69969): No heartbeat from core client for 30 sec - exiting 12:25:24 (69969): No heartbeat from core client for 30 sec - exiting 12:25:25 (69969): No heartbeat from core client for 30 sec - exiting 12:25:26 (69969): No heartbeat from core client for 30 sec - exiting 12:25:27 (69969): No heartbeat from core client for 30 sec - exiting 12:25:28 (69969): No heartbeat from core client for 30 sec - exiting 12:25:29 (69969): No heartbeat from core client for 30 sec - exiting 12:25:30 (69969): No heartbeat from core client for 30 sec - exiting 12:25:31 (69969): No heartbeat from core client for 30 sec - exiting 12:25:32 (69969): No heartbeat from core client for 30 sec - exiting 12:25:33 (69969): No heartbeat from core client for 30 sec - exiting 12:25:34 (69969): No heartbeat from core client for 30 sec - exiting 12:25:35 (69969): No heartbeat from core client for 30 sec - exiting 12:25:36 (69969): No heartbeat from core client for 30 sec - exiting 12:25:37 (69969): No heartbeat from core client for 30 sec - exiting 12:25:38 (69969): No heartbeat from core client for 30 sec - exiting 12:25:39 (69969): No heartbeat from core client for 30 sec - exiting 12:25:40 (69969): No heartbeat from core client for 30 sec - exiting 12:25:41 (69969): No heartbeat from core client for 30 sec - exiting 12:25:42 (69969): No heartbeat from core client for 30 sec - exiting 12:28:46 (71440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:47 (71440): No heartbeat from core client for 30 sec - exiting 12:28:48 (71440): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:39:42 (75478): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:44:41 (75697): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:44:43 (75697): No heartbeat from core client for 30 sec - exiting 13:44:44 (75697): No heartbeat from core client for 30 sec - exiting 13:44:45 (75697): No heartbeat from core client for 30 sec - exiting 13:44:46 (75697): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:08:30 (64008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:44:40 (69599): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:52:11 (73311): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:37:13 (82776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:18:20 (45643): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:18:55 (45678): No heartbeat from core client for 30 sec - exiting 11:18:56 (45678): No heartbeat from core client for 30 sec - exiting 11:18:57 (45678): No heartbeat from core client for 30 sec - exiting 11:18:58 (45678): No heartbeat from core client for 30 sec - exiting 11:18:59 (45678): No heartbeat from core client for 30 sec - exiting 11:19:00 (45678): No heartbeat from core client for 30 sec - exiting 11:19:01 (45678): No heartbeat from core client for 30 sec - exiting 11:19:02 (45678): No heartbeat from core client for 30 sec - exiting 11:19:03 (45678): No heartbeat from core client for 30 sec - exiting 11:19:04 (45678): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:44:13 (69416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x283b004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x283b000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x281fa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x283b004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x283b000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x4820e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x4820e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x82f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x82f600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x1801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x200e204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x200e200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x200e204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x200e200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(96598,0xa013d540) malloc: * error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:26:25 (46601): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:08 (46671): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:32:48 (46753): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:59 (47029): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:08:16 (60924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:49:46 (60965): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:53:12 (79274): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:33:17 (55064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:18 (55064): No heartbeat from core client for 30 sec - exiting 17:33:19 (55064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:38:16 (99647): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:38:20 (99647): No heartbeat from core client for 30 sec - exiting 04:38:21 (99647): No heartbeat from core client for 30 sec - exiting 04:38:22 (99647): No heartbeat from core client for 30 sec - exiting 04:38:23 (99647): No heartbeat from core client for 30 sec - exiting 04:38:24 (99647): No heartbeat from core client for 30 sec - exiting 04:38:25 (99647): No heartbeat from core client for 30 sec - exiting 04:38:26 (99647): No heartbeat from core client for 30 sec - exiting 04:38:27 (99647): No heartbeat from core client for 30 sec - exiting 04:38:28 (99647): No heartbeat from core client for 30 sec - exiting 04:38:29 (99647): No heartbeat from core client for 30 sec - exiting 04:38:30 (99647): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 04:41:03 (123): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:04:01 (66165): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:58 (66213): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:33 (66329): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:15:26 (32246): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:15:28 (32246): No heartbeat from core client for 30 sec - exiting 13:15:29 (32246): No heartbeat from core client for 30 sec - exiting 13:15:30 (32246): No heartbeat from core client for 30 sec - exiting 13:15:31 (32246): No heartbeat from core client for 30 sec - exiting 13:15:32 (32246): No heartbeat from core client for 30 sec - exiting 13:15:33 (32246): No heartbeat from core client for 30 sec - exiting 13:15:34 (32246): No heartbeat from core client for 30 sec - exiting 13:15:35 (32246): No heartbeat from core client for 30 sec - exiting 13:15:36 (32246): No heartbeat from core client for 30 sec - exiting 13:15:37 (32246): No heartbeat from core client for 30 sec - exiting 13:15:38 (32246): No heartbeat from core client for 30 sec - exiting 13:15:39 (32246): No heartbeat from core client for 30 sec - exiting 13:15:40 (32246): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 16:37:28 (43999): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:14 (44450): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:13:05 (49563): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:16:28 (49648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:18:03 (58295): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:18:04 (58295): No heartbeat from core client for 30 sec - exiting 20:18:05 (58295): No heartbeat from core client for 30 sec - exiting 20:18:06 (58295): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 20:20:53 (58378): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:22:52 (58554): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:24:24 (58622): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:26:20 (58732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:46:39 (9352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:49:41 (9410): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:49:42 (9410): No heartbeat from core client for 30 sec - exiting 11:50:27 (9556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:51:45 (9611): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:51:47 (9611): No heartbeat from core client for 30 sec - exiting 11:51:48 (9611): No heartbeat from core client for 30 sec - exiting 11:51:49 (9611): No heartbeat from core client for 30 sec - exiting 11:51:50 (9611): No heartbeat from core client for 30 sec - exiting 11:51:51 (9611): No heartbeat from core client for 30 sec - exiting 11:51:52 (9611): No heartbeat from core client for 30 sec - exiting 11:51:53 (9611): No heartbeat from core client for 30 sec - exiting 11:51:54 (9611): No heartbeat from core client for 30 sec - exiting 11:51:55 (9611): No heartbeat from core client for 30 sec - exiting 11:51:56 (9611): No heartbeat from core client for 30 sec - exiting 11:51:57 (9611): No heartbeat from core client for 30 sec - exiting 11:51:58 (9611): No heartbeat from core client for 30 sec - exiting 11:51:59 (9611): No heartbeat from core client for 30 sec - exiting 11:52:00 (9611): No heartbeat from core client for 30 sec - exiting 11:52:01 (9611): No heartbeat from core client for 30 sec - exiting 11:52:02 (9611): No heartbeat from core client for 30 sec - exiting 11:52:03 (9611): No heartbeat from core client for 30 sec - exiting 11:52:04 (9611): No heartbeat from core client for 30 sec - exiting 11:52:05 (9611): No heartbeat from core client for 30 sec - exiting 11:52:06 (9611): No heartbeat from core client for 30 sec - exiting 11:52:07 (9611): No heartbeat from core client for 30 sec - exiting 11:52:08 (9611): No heartbeat from core client for 30 sec - exiting 11:52:09 (9611): No heartbeat from core client for 30 sec - exiting 11:53:21 (9710): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:53:22 (9710): No heartbeat from core client for 30 sec - exiting 12:53:35 (12418): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jan 2013 09:10:22	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	907,200	1,529,257	1.6857
24 Jan 2013 11:46:43	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	881,280	1,488,576	1.6891
23 Jan 2013 12:55:10	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	855,360	1,447,871	1.6927
22 Jan 2013 20:49:59	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	829,440	1,407,114	1.6965
22 Jan 2013 06:04:58	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	803,520	1,366,519	1.7007
21 Jan 2013 12:01:56	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	777,600	1,325,968	1.7052
20 Jan 2013 22:48:58	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	751,680	1,285,586	1.7103
19 Jan 2013 09:05:15	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	725,760	1,244,874	1.7153
18 Jan 2013 13:05:16	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	699,840	1,203,771	1.7201
17 Jan 2013 17:15:57	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	673,920	1,162,617	1.7252
15 Jan 2013 20:06:40	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	648,000	1,121,104	1.7301
15 Jan 2013 06:17:57	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	622,080	1,081,032	1.7378
14 Jan 2013 13:05:18	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	596,160	1,039,959	1.7444
14 Jan 2013 00:01:36	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	570,240	1,002,203	1.7575
13 Jan 2013 07:09:12	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	544,320	962,742	1.7687
12 Jan 2013 06:48:57	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	518,400	921,600	1.7778
11 Jan 2013 12:55:11	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	492,480	880,609	1.7881
10 Jan 2013 12:15:06	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	466,560	840,980	1.8025
09 Jan 2013 11:50:51	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	440,640	801,601	1.8192
08 Jan 2013 17:21:19	958046	15493767	hadcm3n_38q7_1940_40_008263980_1	414,720	761,984	1.8373