Task 15680129

Name	hadcm3n_4mp4_1940_40_008306275_3
Workunit	8457410
Created	23 Mar 2013, 14:00:07 UTC
Sent	23 Mar 2013, 14:00:12 UTC
Report deadline	22 Jun 2013, 21:27:23 UTC
Received	6 Apr 2013, 17:05:49 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1152409
Run time	8 days 1 hours 29 min 10 sec
CPU time	6 days 2 hours 59 min 49 sec
Validate state	Invalid
Credit	4,354.56
Device peak FLOPS	2.84 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:30:01 (10998): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x81b604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x836c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x836c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x836c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x836c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x1058c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x818e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22942,0xa0cfc540) malloc: * error for object 0x819c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:06:38 (73325): No heartbeat from core client for 30 sec - exiting 14:06:39 (73325): No heartbeat from core client for 30 sec - exiting 14:06:40 (73325): No heartbeat from core client for 30 sec - exiting 14:06:41 (73325): No heartbeat from core client for 30 sec - exiting 14:06:42 (73325): No heartbeat from core client for 30 sec - exiting 14:06:43 (73325): No heartbeat from core client for 30 sec - exiting 14:06:44 (73325): No heartbeat from core client for 30 sec - exiting 14:06:46 (73325): No heartbeat from core client for 30 sec - exiting 14:06:47 (73325): No heartbeat from core client for 30 sec - exiting 14:06:48 (73325): No heartbeat from core client for 30 sec - exiting 14:06:49 (73325): No heartbeat from core client for 30 sec - exiting 14:06:50 (73325): No heartbeat from core client for 30 sec - exiting 14:06:51 (73325): No heartbeat from core client for 30 sec - exiting 14:06:52 (73325): No heartbeat from core client for 30 sec - exiting 14:06:53 (73325): No heartbeat from core client for 30 sec - exiting 14:06:54 (73325): No heartbeat from core client for 30 sec - exiting 14:06:55 (73325): No heartbeat from core client for 30 sec - exiting 14:06:56 (73325): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/4mp4ko.daf5bg0 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_4mp4_1940_40_008306275/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0042E62Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Apr 2013 07:49:06	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	362,880	512,767	1.4130
03 Apr 2013 14:20:28	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	336,960	476,015	1.4127
02 Apr 2013 23:56:59	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	311,040	439,060	1.4116
02 Apr 2013 09:18:47	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	285,120	402,013	1.4100
01 Apr 2013 12:55:58	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	259,200	365,678	1.4108
31 Mar 2013 07:28:47	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	233,280	328,229	1.4070
29 Mar 2013 09:26:09	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	207,360	291,351	1.4050
28 Mar 2013 18:25:51	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	181,440	255,271	1.4069
28 Mar 2013 05:42:35	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	155,520	219,018	1.4083
27 Mar 2013 06:46:37	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	129,600	182,724	1.4099
26 Mar 2013 14:35:15	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	103,680	145,795	1.4062
26 Mar 2013 01:13:13	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	77,760	109,436	1.4074
25 Mar 2013 08:42:14	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	51,840	72,967	1.4075
24 Mar 2013 08:01:54	1152409	15680129	hadcm3n_4mp4_1940_40_008306275_3	25,920	36,707	1.4162