Task 18699499

Name	hadcm3n_sdhp_1940_40_010020410_0
Workunit	10018481
Created	14 Jul 2015, 13:25:08 UTC
Sent	14 Jul 2015, 14:44:08 UTC
Report deadline	13 Oct 2015, 22:11:19 UTC
Received	12 Dec 2016, 1:23:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1368392
Run time	6 days 7 hours 9 min 19 sec
CPU time	5 days 0 hours 2 min 10 sec
Validate state	Invalid
Credit	7,776.00
Device peak FLOPS	4.18 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Peak working set size	200.29 MB
Peak swap size	981.89 MB
Peak disk usage	24.29 MB
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(2013,0xa3ee2000) malloc: * error for object 0x2012600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(2013,0xa3ee2000) malloc: * error for object 0x1807e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2013,0xa3ee2000) malloc: * error for object 0x1807e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) hadcm3n_6.07_i686-apple-darwin(8817,0xa3ee2000) malloc: * error for object 0x183a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8817,0xa3ee2000) malloc: * error for object 0x183a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8817,0xa3ee2000) malloc: * error for object 0x183a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(498,0xa3ee2000) malloc: * error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.11.1 build 15B42 Sun Nov 29 11:29:31 2015 hadcm3n_6.07_i686-apple-darwin(13856,0xa3ee2000) malloc: * error for object 0x182a004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13856,0xa3ee2000) malloc: * error for object 0x182a000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(13856,0xa3ee2000) malloc: * error for object 0x2003000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.11.1 build 15B42 Mon Nov 30 18:21:10 2015 hadcm3n_6.07_i686-apple-darwin(511,0xa3ee2000) malloc: * error for object 0x1800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(511,0xa3ee2000) malloc: * error for object 0x1801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(2495,0xa3ee2000) malloc: * error for object 0x282de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2495,0xa3ee2000) malloc: * error for object 0x282de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2495,0xa3ee2000) malloc: * error for object 0x282de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.11.1 build 15B42 Thu Dec 3 05:14:47 2015 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. CPDN Monitor - Quit request from BOINC... 20:52:33 (491): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 3 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Dec 2016 08:00:26	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	648,000	432,096	0.6668
06 Dec 2015 17:03:19	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	622,080	412,921	0.6638
05 Dec 2015 12:56:17	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	596,160	395,976	0.6642
05 Dec 2015 02:29:56	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	570,240	376,713	0.6606
04 Dec 2015 19:42:34	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	544,320	356,988	0.6558
28 Nov 2015 15:31:01	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	518,400	337,106	0.6503
28 Nov 2015 07:49:20	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	492,480	317,647	0.6450
23 Jul 2015 13:01:58	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	466,560	299,640	0.6422
23 Jul 2015 06:49:55	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	440,640	282,868	0.6419
21 Jul 2015 06:45:38	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	414,720	265,995	0.6414
20 Jul 2015 20:29:57	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	388,800	249,211	0.6410
20 Jul 2015 06:45:23	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	362,880	232,456	0.6406
19 Jul 2015 19:24:06	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	336,960	216,447	0.6424
19 Jul 2015 13:52:44	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	311,040	199,738	0.6422
19 Jul 2015 08:05:57	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	285,120	183,113	0.6422
19 Jul 2015 02:52:46	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	259,200	166,479	0.6423
17 Jul 2015 17:29:26	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	233,280	149,694	0.6417
17 Jul 2015 11:42:43	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	207,360	133,134	0.6420
17 Jul 2015 06:42:32	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	181,440	116,569	0.6425
16 Jul 2015 18:17:07	1368392	18699499	hadcm3n_sdhp_1940_40_010020410_0	155,520	99,800	0.6417