Task 13338539

Name	hadcm3n_o6t1_1900_40_007440653_3
Workunit	7638156
Created	5 Sep 2011, 19:54:08 UTC
Sent	5 Sep 2011, 20:01:24 UTC
Report deadline	6 Dec 2011, 3:28:35 UTC
Received	6 Nov 2011, 22:35:39 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	876324
Run time	26 days 3 hours 54 min 49 sec
CPU time	18 days 18 hours 0 min 4 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	3.45 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.21</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/o6t1ko.daa5ar0 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/o6t1ka.daa5ar0 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/o6t1ka.daa5ar0 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/o6t1ka.daa5ar0 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:15:48 (44996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(88676,0xa08c8540) malloc: * error for object 0x80de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=88677, selfPID=88677, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(92111,0xa08c8540) malloc: * error for object 0x282de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3420,0xa08c8540) malloc: * error for object 0x805e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x200a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x200a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x200a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x200a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x200a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x200a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x200a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x200a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8913,0xa08c8540) malloc: * error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(11808,0xa0c2b540) malloc: * error for object 0x2802204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(11808,0xa0c2b540) malloc: * error for object 0x2802200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(11808,0xa0c2b540) malloc: * error for object 0x1024804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x282de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x282de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x282de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x282de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x2001204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x2002004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x2002000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x2001204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x2002004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x2002000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x2800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46929,0xa0c2b540) malloc: * error for object 0x2800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2011 01:10:26	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	1,036,800	1,619,360	1.5619
03 Nov 2011 23:09:10	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	1,010,880	1,578,100	1.5611
02 Nov 2011 10:31:48	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	984,960	1,537,960	1.5614
01 Nov 2011 14:17:55	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	959,040	1,497,649	1.5616
31 Oct 2011 20:07:13	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	933,120	1,456,684	1.5611
31 Oct 2011 19:31:18	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	907,200	1,415,454	1.5602
31 Oct 2011 19:07:27	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	881,280	1,373,434	1.5585
31 Oct 2011 18:43:46	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	855,360	1,332,689	1.5580
31 Oct 2011 18:24:54	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	829,440	1,292,369	1.5581
31 Oct 2011 17:25:56	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	803,520	1,252,630	1.5589
31 Oct 2011 17:17:40	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	777,600	1,213,274	1.5603
31 Oct 2011 17:17:40	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	751,680	1,174,095	1.5620
31 Oct 2011 17:17:40	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	725,760	1,133,710	1.5621
31 Oct 2011 17:17:40	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	699,840	1,093,341	1.5623
13 Oct 2011 03:03:29	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	673,920	1,052,520	1.5618
11 Oct 2011 01:30:26	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	648,000	1,012,782	1.5629
10 Oct 2011 10:31:34	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	622,080	974,123	1.5659
09 Oct 2011 18:29:23	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	596,160	933,287	1.5655
08 Oct 2011 20:09:53	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	570,240	892,135	1.5645
07 Oct 2011 22:32:58	876324	13338539	hadcm3n_o6t1_1900_40_007440653_3	544,320	851,871	1.5650