Task 13420978

Name	hadcm3n_o0ac_1940_40_007445442_3
Workunit	7642945
Created	25 Sep 2011, 18:42:48 UTC
Sent	25 Sep 2011, 18:46:43 UTC
Report deadline	26 Dec 2011, 2:13:54 UTC
Received	15 Nov 2011, 17:21:23 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	784648
Run time	18 days 1 hours 4 min 25 sec
CPU time	16 days 15 hours 38 min 44 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.08 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x5842c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x83f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x85ac04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x85ac00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x1064e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x383f004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x385a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x385a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x385a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x385a600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x505ac04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x5842c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x5842c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x5058204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x5058200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x820e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x820e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x820e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x820e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x820e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x820e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17584,0xac6e12c0) malloc: * error for object 0x820e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=17586, selfPID=17586, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(34841,0xac6e12c0) malloc: * error for object 0x388a004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34841,0xac6e12c0) malloc: * error for object 0x209c804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34841,0xac6e12c0) malloc: * error for object 0x209c800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34841,0xac6e12c0) malloc: * error for object 0x209c804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34841,0xac6e12c0) malloc: * error for object 0x209c800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34841,0xac6e12c0) malloc: * error for object 0x209c800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 Sat Nov 12 05:02:22 2011 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fde8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/0/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90031000 - 0x90039fff /usr/lib/system/liblaunch.dylib 0x90fcb000 - 0x90fd2fff /usr/lib/system/libsystem_dnssd.dylib 0x91064000 - 0x91065fff /usr/lib/system/libsystem_blocks.dylib 0x92678000 - 0x9267bfff /usr/lib/system/libcompiler_rt.dylib 0x92890000 - 0x92897fff /usr/lib/system/libsystem_notify.dylib 0x92a1b000 - 0x92a29fff /usr/lib/system/libdispatch.dylib 0x93f59000 - 0x93f62fff /usr/lib/libc++abi.dylib 0x940f6000 - 0x940f7fff /usr/lib/system/libremovefile.dylib 0x94cd2000 - 0x94cd2fff /usr/lib/system/libkeymgr.dylib 0x94dec000 - 0x94df4fff /usr/lib/system/libunwind.dylib 0x959a7000 - 0x959a8fff /usr/lib/system/libquarantine.dylib 0x96972000 - 0x96a3dfff /usr/lib/system/libsystem_c.dylib 0x970f8000 - 0x9715afff /usr/lib/libstdc++.6.dylib 0x9717d000 - 0x971acfff /usr/lib/system/libsystem_info.dylib 0x973cd000 - 0x973d1fff /usr/lib/system/libsystem_network.dylib 0x98937000 - 0x9893cfff /usr/lib/system/libmacho.dylib 0x98b70000 - 0x98b73fff /usr/lib/system/libmathCommon.A.dylib 0x99666000 - 0x9967cfff /usr/lib/system/libxpc.dylib 0x99c44000 - 0x99c48fff /usr/lib/system/libcache.dylib 0x99c4a000 - 0x99c58fff /usr/lib/libz.1.dylib 0x9a697000 - 0x9a698fff /usr/lib/system/libunc.dylib 0x9c0f7000 - 0x9c125fff /usr/lib/libSystem.B.dylib 0x9c6f4000 - 0x9c712fff /usr/lib/system/libsystem_kernel.dylib 0x9c8ce000 - 0x9c8d0fff /usr/lib/system/libdyld.dylib 0x9ca3a000 - 0x9ca3afff /usr/lib/system/libdnsinfo.dylib 0x9ca3b000 - 0x9ca7efff /usr/lib/system/libcommonCrypto.dylib 0x9cabc000 - 0x9cabdfff /usr/lib/system/libsystem_sandbox.dylib 0x9cabe000 - 0x9cac6fff /usr/lib/system/libcopyfile.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Nov 2011 17:46:39	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	518,400	1,414,251	2.7281
15 Nov 2011 17:46:39	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	492,480	1,351,105	2.7435
10 Nov 2011 00:50:24	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	466,560	1,288,592	2.7619
09 Nov 2011 05:10:04	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	440,640	1,221,747	2.7727
08 Nov 2011 05:45:50	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	414,720	1,155,361	2.7859
07 Nov 2011 06:44:04	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	388,800	1,090,946	2.8059
06 Nov 2011 08:34:44	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	362,880	1,018,983	2.8080
03 Nov 2011 17:13:26	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	336,960	945,802	2.8069
01 Nov 2011 20:57:10	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	311,040	868,682	2.7928
31 Oct 2011 19:36:17	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	285,120	793,791	2.7841
31 Oct 2011 18:30:49	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	259,200	725,656	2.7996
31 Oct 2011 17:36:51	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	233,280	656,131	2.8126
31 Oct 2011 17:09:19	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	207,360	585,757	2.8248
31 Oct 2011 16:32:11	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	181,440	511,992	2.8218
31 Oct 2011 15:06:02	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	155,520	440,987	2.8356
31 Oct 2011 15:06:02	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	129,600	375,159	2.8947
31 Oct 2011 15:06:02	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	103,680	304,013	2.9322
31 Oct 2011 15:06:02	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	77,760	229,050	2.9456
31 Oct 2011 15:06:02	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	51,840	154,109	2.9728
03 Oct 2011 06:04:45	784648	13420978	hadcm3n_o0ac_1940_40_007445442_3	25,920	82,859	3.1967