Task 16588447

Name	hadcm3n_8aye_1980_40_008723617_0
Workunit	8869595
Created	23 Apr 2014, 13:02:43 UTC
Sent	3 May 2014, 17:00:50 UTC
Report deadline	3 Aug 2014, 0:28:01 UTC
Received	10 Jun 2014, 6:27:21 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	4 (0x00000004) Unknown error code
Computer ID	1110299
Run time	21 days 15 hours 25 min 30 sec
CPU time	20 days 11 hours 44 min 41 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	2.91 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process got signal 4 </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/8ayeko.pji1c10 is not a valid UM file. Error converting file to netcdf: dataout/8ayeko.pji1c10 Error: Input file: dataout/8ayeko.pii1c10 is not a valid UM file. Error converting file to netcdf: dataout/8ayeko.pii1c10 Error: Input file: dataout/8ayeko.pfi1c10 is not a valid UM file. Error converting file to netcdf: dataout/8ayeko.pfi1c10 Error: Input file: dataout/8ayeka.phi1c10 is not a valid UM file. Error converting file to netcdf: dataout/8ayeka.phi1c10 Error: Input file: dataout/8ayeka.pgi1c10 is not a valid UM file. Error converting file to netcdf: dataout/8ayeka.pgi1c10 Error: Input file: dataout/8ayeka.pei1c10 is not a valid UM file. Error converting file to netcdf: dataout/8ayeka.pei1c10 Error: Input file: dataout/8ayeka.pdi1c10 is not a valid UM file. Error converting file to netcdf: dataout/8ayeka.pdi1c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x2168004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x2168000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x95ca04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x978004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x978000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x4967404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x4967400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x5170404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x5170400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x894ca04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x894ca00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x95ca04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x978004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x978000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x314c204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x314c200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x294dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x294dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(23767,0xacd4b2c0) malloc: * error for object 0x292e600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:18:31 (26732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:27:33 (34847): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:55:56 (419): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:56:00 (419): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 05:54:48 (771): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:54:50 (771): No heartbeat from core client for 30 sec - exiting 05:54:51 (771): No heartbeat from core client for 30 sec - exiting 05:54:52 (771): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Tue Jun 10 05:54:53 2014 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Jun 2014 09:03:35	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	881,280	1,748,292	1.9838
10 Jun 2014 09:00:47	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	855,360	1,704,131	1.9923
06 Jun 2014 15:45:23	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	829,440	1,659,747	2.0010
05 Jun 2014 07:54:39	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	803,520	1,610,605	2.0044
04 Jun 2014 11:41:32	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	777,600	1,559,631	2.0057
03 Jun 2014 11:12:08	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	751,680	1,511,851	2.0113
02 Jun 2014 11:57:51	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	725,760	1,463,067	2.0159
01 Jun 2014 13:09:20	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	699,840	1,413,452	2.0197
31 May 2014 19:25:28	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	673,920	1,368,251	2.0303
31 May 2014 03:25:22	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	648,000	1,322,766	2.0413
29 May 2014 21:14:34	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	622,080	1,274,720	2.0491
28 May 2014 19:14:56	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	596,160	1,224,973	2.0548
27 May 2014 23:14:32	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	570,240	1,176,584	2.0633
27 May 2014 06:10:38	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	544,320	1,129,780	2.0756
26 May 2014 01:31:43	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	518,400	1,080,108	2.0835
25 May 2014 03:50:36	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	492,480	1,029,415	2.0903
24 May 2014 01:04:55	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	466,560	979,767	2.1000
23 May 2014 07:30:15	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	440,640	924,086	2.0971
22 May 2014 15:00:47	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	414,720	868,730	2.0947
21 May 2014 11:41:43	1110299	16588447	hadcm3n_8aye_1980_40_008723617_0	388,800	808,665	2.0799