Task 15801775

Name	hadcm3n_o47u_2020_40_008373346_0
Workunit	8524205
Created	29 May 2013, 18:53:17 UTC
Sent	31 May 2013, 20:11:12 UTC
Report deadline	31 Aug 2013, 3:38:23 UTC
Received	31 Aug 2013, 23:36:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1099925
Run time	13 days 18 hours 9 min 54 sec
CPU time	11 days 10 hours 3 min 49 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.51 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x20eca04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x20eca00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x10dd404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x10f8a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x10f8a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x40f4e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x40f4e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48e3004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x20eca04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x20eca00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x30e9c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x40d9804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x40d9800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x10dd404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x10dd400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x38d2604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x38d2600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(29315,0xac81da28) malloc: * error for object 0x48b6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/2/../../switcher/switcher) is setuid or setgid cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o47u_2020_40_008373346/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Aug 2013 22:36:00	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	777,600	986,665	1.2689
30 Aug 2013 20:58:39	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	751,680	953,701	1.2688
29 Aug 2013 18:19:31	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	725,760	920,324	1.2681
27 Aug 2013 18:23:51	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	699,840	886,559	1.2668
26 Aug 2013 13:10:19	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	673,920	853,531	1.2665
26 Aug 2013 00:11:41	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	648,000	820,849	1.2667
26 Aug 2013 00:11:41	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	622,080	788,071	1.2668
26 Aug 2013 00:11:41	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	596,160	755,504	1.2673
26 Aug 2013 00:11:41	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	570,240	722,212	1.2665
10 Jul 2013 21:46:59	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	544,320	689,088	1.2660
09 Jul 2013 23:43:24	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	518,400	656,242	1.2659
08 Jul 2013 16:54:38	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	492,480	622,761	1.2645
06 Jul 2013 05:15:35	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	466,560	589,323	1.2631
06 Jul 2013 04:38:45	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	440,640	557,461	1.2651
04 Jul 2013 01:30:12	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	414,720	524,596	1.2649
02 Jul 2013 18:44:00	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	388,800	491,378	1.2638
02 Jul 2013 11:13:17	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	362,880	458,310	1.2630
29 Jun 2013 04:20:19	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	336,960	425,558	1.2629
26 Jun 2013 20:32:02	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	311,040	392,694	1.2625
24 Jun 2013 23:26:25	1099925	15801775	hadcm3n_o47u_2020_40_008373346_0	285,120	359,655	1.2614