Task 12929694

Name	hadcm3n_o08z_1900_40_007195654_2
Workunit	7393934
Created	3 Jun 2011, 14:04:46 UTC
Sent	3 Jun 2011, 14:04:51 UTC
Report deadline	2 Sep 2011, 21:32:02 UTC
Received	3 Aug 2011, 14:31:46 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1112860
Run time	14 days 18 hours 38 min 12 sec
CPU time	13 days 22 hours 47 min 29 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	3.13 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21259,0xa0638540) malloc: * error for object 0x6801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x3812004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x3812000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x9011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x9011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x9011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x9011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x9011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x9011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(79245,0xa0638540) malloc: * error for object 0x9011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=79254, selfPID=79254, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7497,0xa0638540) malloc: * error for object 0x701a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x6033604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x6033600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19730,0xa0638540) malloc: * error for object 0x5835200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:01:32 (91495): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:01:33 (35504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Aug 2011 20:28:48	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	1,036,800	1,204,792	1.1620
01 Aug 2011 18:34:52	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	1,010,880	1,175,183	1.1625
31 Jul 2011 17:38:43	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	984,960	1,146,073	1.1636
30 Jul 2011 15:30:04	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	959,040	1,116,746	1.1644
28 Jul 2011 21:10:46	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	933,120	1,087,638	1.1656
27 Jul 2011 18:41:42	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	907,200	1,065,120	1.1741
26 Jul 2011 16:31:08	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	881,280	1,035,469	1.1750
25 Jul 2011 23:00:09	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	855,360	1,005,816	1.1759
25 Jul 2011 22:45:14	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	829,440	975,050	1.1756
25 Jul 2011 18:09:14	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	803,520	944,433	1.1754
25 Jul 2011 16:30:05	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	777,600	913,099	1.1743
25 Jul 2011 15:43:45	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	751,680	881,902	1.1732
25 Jul 2011 14:14:43	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	725,760	851,922	1.1738
25 Jul 2011 12:45:49	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	699,840	821,545	1.1739
25 Jul 2011 12:45:49	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	673,920	790,947	1.1737
07 Jul 2011 18:05:22	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	648,000	760,765	1.1740
07 Jul 2011 15:42:37	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	622,080	730,709	1.1746
04 Jul 2011 20:57:24	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	596,160	700,229	1.1746
01 Jul 2011 18:56:21	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	570,240	669,730	1.1745
30 Jun 2011 17:37:34	1112860	12929694	hadcm3n_o08z_1900_40_007195654_2	544,320	639,234	1.1744