Task 19421388

Name	hadcm3n_xbs7_198012_480_374_010413629_1
Workunit	10413629
Created	21 Mar 2016, 18:06:50 UTC
Sent	24 Mar 2016, 0:39:36 UTC
Report deadline	6 Mar 2017, 5:59:36 UTC
Received	17 Apr 2016, 10:43:17 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1377941
Run time	6 days 0 hours 6 min 35 sec
CPU time	5 days 3 hours 0 min 51 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.62 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.6.12</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:51:26 (38037): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:51:27 (38037): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:49:38 (39201): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:32 (39216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:08:48 (39452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:11:33 (39465): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:11:34 (39465): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:26:42 (43590): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:30:39 (51853): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:40:51 (52000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:42:20 (52015): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:42:21 (52015): No heartbeat from core client for 30 sec - exiting 19:42:22 (52015): No heartbeat from core client for 30 sec - exiting 19:42:23 (52015): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:10:03 (28168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(22435,0xa3414000) malloc: * error for object 0x2800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22435,0xa3414000) malloc: * error for object 0x2800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22435,0xa3414000) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(24487,0xa3414000) malloc: * error for object 0x82e604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24487,0xa3414000) malloc: * error for object 0x82e600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24487,0xa3414000) malloc: * error for object 0x2802600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(26499,0xa3414000) malloc: * error for object 0x103e604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26499,0xa3414000) malloc: * error for object 0x103e600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26499,0xa3414000) malloc: * error for object 0x103e600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(28544,0xa3414000) malloc: * error for object 0x4035204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(28544,0xa3414000) malloc: * error for object 0x4035200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(28544,0xa3414000) malloc: * error for object 0x80a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Sun Apr 17 05:43:13 2016 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fd48 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/10/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x127000 - 0x17afff /usr/lib/libstdc++.6.dylib 0x908fa000 - 0x908fafff /usr/lib/system/liblaunch.dylib 0x911c1000 - 0x911d8fff /usr/lib/system/libsystem_coretls.dylib 0x911f7000 - 0x91205fff /usr/lib/libz.1.dylib 0x91207000 - 0x9120ffff /usr/lib/system/libsystem_dnssd.dylib 0x912b1000 - 0x912b3fff /usr/lib/system/libquarantine.dylib 0x912b4000 - 0x912dcfff /usr/lib/system/libxpc.dylib 0x91365000 - 0x9138ffff /usr/lib/system/libdispatch.dylib 0x918b6000 - 0x9191dfff /usr/lib/system/libcorecrypto.dylib 0x9204f000 - 0x92050fff /usr/lib/libSystem.B.dylib 0x92054000 - 0x9207efff /usr/lib/system/libsystem_info.dylib 0x923d8000 - 0x923d8fff /usr/lib/system/libunc.dylib 0x923da000 - 0x923dffff /usr/lib/system/libcompiler_rt.dylib 0x93e11000 - 0x93e11fff /usr/lib/system/libkeymgr.dylib 0x9436b000 - 0x943d9fff /usr/lib/system/libsystem_network.dylib 0x944f2000 - 0x944f5fff /usr/lib/system/libdyld.dylib 0x94a61000 - 0x94a63fff /usr/lib/system/libsystem_secinit.dylib 0x94c1f000 - 0x94cb4fff /usr/lib/system/libsystem_c.dylib 0x94e2e000 - 0x94e2ffff /usr/lib/system/libremovefile.dylib 0x94ec5000 - 0x94edefff /usr/lib/system/libsystem_asl.dylib 0x94edf000 - 0x94f12fff /usr/lib/system/libsystem_m.dylib 0x96893000 - 0x9689bfff /usr/lib/system/libsystem_pthread.dylib 0x969b5000 - 0x969bbfff /usr/lib/system/libunwind.dylib 0x98045000 - 0x9804bfff /usr/lib/system/libsystem_platform.dylib 0x9842e000 - 0x9843ffff /usr/lib/system/libsystem_trace.dylib 0x98985000 - 0x98987fff /usr/lib/system/libsystem_configuration.dylib 0x989ff000 - 0x98a02fff /usr/lib/system/libsystem_sandbox.dylib 0x99037000 - 0x9905afff /usr/lib/libc++abi.dylib 0x9934b000 - 0x99356fff /usr/lib/system/libcommonCrypto.dylib 0x993e3000 - 0x993ecfff /usr/lib/system/libcopyfile.dylib 0x9940e000 - 0x99418fff /usr/lib/system/libsystem_notify.dylib 0x9a9d6000 - 0x9a9effff /usr/lib/system/libsystem_malloc.dylib 0x9b073000 - 0x9b073fff /usr/lib/system/libsystem_blocks.dylib 0x9b074000 - 0x9b076fff /usr/lib/system/libsystem_coreservices.dylib 0x9b433000 - 0x9b438fff /usr/lib/system/libmacho.dylib 0x9c5c2000 - 0x9c5cbfff /usr/lib/system/libsystem_networkextension.dylib 0x9d53c000 - 0x9d55dfff /usr/lib/system/libsystem_kernel.dylib 0x9d6da000 - 0x9d6defff /usr/lib/system/libcache.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Apr 2016 23:37:38	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	259,200	438,298	1.6910
15 Apr 2016 16:58:12	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	233,280	395,261	1.6944
15 Apr 2016 12:16:38	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	207,360	351,248	1.6939
15 Apr 2016 12:14:49	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	181,440	304,834	1.6801
15 Apr 2016 12:09:00	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	155,520	263,825	1.6964
09 Apr 2016 12:14:31	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	129,600	220,753	1.7033
28 Mar 2016 17:43:12	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	103,680	174,402	1.6821
28 Mar 2016 03:06:44	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	77,760	130,368	1.6765
25 Mar 2016 07:22:26	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	51,840	86,593	1.6704
24 Mar 2016 16:09:57	1377941	19421388	hadcm3n_xbs7_198012_480_374_010413629_1	25,920	43,240	1.6682