Task 13634789

Name	hadcm3n_u37b_1980_40_007459737_3
Workunit	7657240
Created	15 Nov 2011, 18:08:43 UTC
Sent	15 Nov 2011, 18:08:51 UTC
Report deadline	15 Feb 2012, 1:36:02 UTC
Received	6 Dec 2011, 2:52:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1171022
Run time	3 days 21 hours 33 min 32 sec
CPU time	3 days 10 hours 21 min 41 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.28 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/u37bko.pji2c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pji2c10 Error: Input file: dataout/u37bko.pii2c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pii2c10 Error: Input file: dataout/u37bko.pfi2c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pfi2c10 Error: Input file: dataout/u37bka.phi2c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.phi2c10 Error: Input file: dataout/u37bka.pgi2c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pgi2c10 Error: Input file: dataout/u37bka.pei2c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pei2c10 Error: Input file: dataout/u37bka.pdi2c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pdi2c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/u37bko.pji3c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pji3c10 Error: Input file: dataout/u37bko.pii3c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pii3c10 Error: Input file: dataout/u37bko.pfi3c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pfi3c10 Error: Input file: dataout/u37bka.phi3c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.phi3c10 Error: Input file: dataout/u37bka.pgi3c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pgi3c10 Error: Input file: dataout/u37bka.pei3c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pei3c10 Error: Input file: dataout/u37bka.pdi3c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pdi3c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:27:54 (36079): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/u37bko.pji8c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pji8c10 Error: Input file: dataout/u37bko.pii8c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pii8c10 Error: Input file: dataout/u37bko.pfi8c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pfi8c10 Error: Input file: dataout/u37bka.phi8c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.phi8c10 Error: Input file: dataout/u37bka.pgi8c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pgi8c10 Error: Input file: dataout/u37bka.pei8c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pei8c10 Error: Input file: dataout/u37bka.pdi8c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pdi8c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3501, selfPID=3501, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/u37bko.pji9c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pji9c10 Error: Input file: dataout/u37bko.pii9c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pii9c10 Error: Input file: dataout/u37bko.pfi9c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bko.pfi9c10 Error: Input file: dataout/u37bka.phi9c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.phi9c10 Error: Input file: dataout/u37bka.pgi9c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pgi9c10 Error: Input file: dataout/u37bka.pei9c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pei9c10 Error: Input file: dataout/u37bka.pdi9c10 is not a valid UM file. Error converting file to netcdf: dataout/u37bka.pdi9c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(1873,0xac5762c0) malloc: * error for object 0x3b96404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1873,0xac5762c0) malloc: * error for object 0x3b96400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 Mon Dec 5 20:08:49 2011 hadcm3n_6.07_i686-apple-darwin(1873,0xac5762c0) malloc: * error for object 0xb79e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fad8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/7/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90cdc000 - 0x90ce4fff /usr/lib/system/liblaunch.dylib 0x90ce5000 - 0x90cedfff /usr/lib/system/libcopyfile.dylib 0x917b9000 - 0x917bafff /usr/lib/system/libsystem_sandbox.dylib 0x92515000 - 0x92523fff /usr/lib/libz.1.dylib 0x926f7000 - 0x926f8fff /usr/lib/system/libunc.dylib 0x92fb5000 - 0x92fcbfff /usr/lib/system/libxpc.dylib 0x935f9000 - 0x93600fff /usr/lib/system/libsystem_dnssd.dylib 0x93b84000 - 0x93b86fff /usr/lib/system/libdyld.dylib 0x946ff000 - 0x94700fff /usr/lib/system/libremovefile.dylib 0x948b6000 - 0x94981fff /usr/lib/system/libsystem_c.dylib 0x9581a000 - 0x95822fff /usr/lib/system/libunwind.dylib 0x9588b000 - 0x9588cfff /usr/lib/system/libsystem_blocks.dylib 0x95d81000 - 0x95d82fff /usr/lib/system/libquarantine.dylib 0x96151000 - 0x961b3fff /usr/lib/libstdc++.6.dylib 0x96e69000 - 0x96e6cfff /usr/lib/system/libmathCommon.A.dylib 0x974a1000 - 0x974a1fff /usr/lib/system/libdnsinfo.dylib 0x97d0a000 - 0x97d11fff /usr/lib/system/libsystem_notify.dylib 0x9884f000 - 0x98858fff /usr/lib/libc++abi.dylib 0x98859000 - 0x9885dfff /usr/lib/system/libsystem_network.dylib 0x9914d000 - 0x9916bfff /usr/lib/system/libsystem_kernel.dylib 0x99964000 - 0x99993fff /usr/lib/system/libsystem_info.dylib 0x99caf000 - 0x99cb4fff /usr/lib/system/libmacho.dylib 0x9b30e000 - 0x9b33cfff /usr/lib/libSystem.B.dylib 0x9b4c6000 - 0x9b4cafff /usr/lib/system/libcache.dylib 0x9b5f1000 - 0x9b5fffff /usr/lib/system/libdispatch.dylib 0x9b63f000 - 0x9b63ffff /usr/lib/system/libkeymgr.dylib 0x9c9b4000 - 0x9c9f7fff /usr/lib/system/libcommonCrypto.dylib 0x9caff000 - 0x9cb02fff /usr/lib/system/libcompiler_rt.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Dec 2011 02:52:55	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	259,200	296,494	1.1439
05 Dec 2011 05:20:09	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	233,280	266,469	1.1423
01 Dec 2011 15:38:10	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	207,360	236,400	1.1400
29 Nov 2011 04:39:49	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	181,440	206,678	1.1391
25 Nov 2011 12:52:02	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	155,520	176,675	1.1360
24 Nov 2011 02:04:48	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	129,600	146,835	1.1330
23 Nov 2011 01:15:47	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	103,680	116,975	1.1282
21 Nov 2011 13:39:44	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	77,760	87,192	1.1213
19 Nov 2011 02:01:57	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	51,840	57,342	1.1061
17 Nov 2011 08:03:07	1171022	13634789	hadcm3n_u37b_1980_40_007459737_3	25,920	30,486	1.1762