Task 13341802

Name	hadcm3n_o0mf_1900_40_007438699_2
Workunit	7636202
Created	6 Sep 2011, 23:12:04 UTC
Sent	6 Sep 2011, 23:16:15 UTC
Report deadline	7 Dec 2011, 6:43:26 UTC
Received	26 Sep 2011, 6:20:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	944551
Run time	5 days 15 hours 9 min 54 sec
CPU time	5 days 15 hours 9 min 54 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09de8ed0 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6c231)[0xf747d231] /lib32/libc.so.6(+0x6dab8)[0xf747eab8] /lib32/libc.so.6(cfree+0x6d)[0xf7481b9d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf766ba91] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf766baed] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7427bd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:01 1288 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:01 1288 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 09d93000-09df7000 rw-p 00000000 00:00 0 [heap] f6e00000-f6e21000 rw-p 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6f94000-f740f000 rw-s 00000000 08:01 102002784 /var/lib/boinc-client/slots/2/137915 f740f000-f7411000 rw-p 00000000 00:00 0 f7411000-f7564000 r-xp 00000000 08:01 34543760 /lib32/libc-2.11.1.so f7564000-f7565000 ---p 00153000 08:01 34543760 /lib32/libc-2.11.1.so f7565000-f7567000 r--p 00153000 08:01 34543760 /lib32/libc-2.11.1.so f7567000-f7568000 rw-p 00155000 08:01 34543760 /lib32/libc-2.11.1.so f7568000-f756b000 rw-p 00000000 00:00 0 f756b000-f7588000 r-xp 00000000 08:01 160205 /usr/lib32/libgcc_s.so.1 f7588000-f7589000 r--p 0001c000 08:01 160205 /usr/lib32/libgcc_s.so.1 f7589000-f758a000 rw-p 0001d000 08:01 160205 /usr/lib32/libgcc_s.so.1 f758a000-f75ae000 r-xp 00000000 08:01 34543765 /lib32/libm-2.11.1.so f75ae000-f75af000 r--p 00023000 08:01 34543765 /lib32/libm-2.11.1.so f75af000-f75b0000 rw-p 00024000 08:01 34543765 /lib32/libm-2.11.1.so f75b0000-f7699000 r-xp 00000000 08:01 160206 /usr/lib32/libstdc++.so.6.0.13 f7699000-f769a000 ---p 000e9000 08:01 160206 /usr/lib32/libstdc++.so.6.0.13 f769a000-f769e000 r--p 000e9000 08:01 160206 /usr/lib32/libstdc++.so.6.0.13 f769e000-f769f000 rw-p 000ed000 08:01 160206 /usr/lib32/libstdc++.so.6.0.13 f769f000-f76a6000 rw-p 00000000 00:00 0 f76a6000-f76a8000 r-xp 00000000 08:01 34543764 /lib32/libdl-2.11.1.so f76a8000-f76a9000 r--p 00001000 08:01 34543764 /lib32/libdl-2.11.1.so f76a9000-f76aa000 rw-p 00002000 08:01 34543764 /lib32/libdl-2.11.1.so f76aa000-f76ab000 rw-p 00000000 00:00 0 f76ab000-f76c0000 r-xp 00000000 08:01 34660253 /lib32/libpthread-2.11.1.so f76c0000-f76c1000 r--p 00014000 08:01 34660253 /lib32/libpthread-2.11.1.so f76c1000-f76c2000 rw-p 00015000 08:01 34660253 /lib32/libpthread-2.11.1.so f76c2000-f76c4000 rw-p 00000000 00:00 0 f76d9000-f76da000 rw-p 00000000 00:00 0 f76da000-f76db000 ---p 00000000 00:00 0 f76db000-f76de000 rw-p 00000000 00:00 0 f76de000-f76e0000 rw-s 00000000 08:01 101393979 /var/lib/boinc-client/slots/2/boinc_mmap_file f76e0000-f76e2000 rw-p 00000000 00:00 0 f76e2000-f76e3000 r-xp 00000000 00:00 0 [vdso] f76e3000-f76ff000 r-xp 00000000 08:01 34479465 /lib32/ld-2.11.1.so f76ff000-f7700000 r--p 0001b000 08:01 34479465 /lib32/ld-2.11.1.so f7700000-f7701000 rw-p 0001c000 08:01 34479465 /lib32/ld-2.11.1.so ffb5c000-ffbcb000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf76e2400] [0xf76e2430] /lib32/libc.so.6(gsignal+0x51)[0xf743b921] /lib32/libc.so.6(abort+0x182)[0xf743ed52] /lib32/libc.so.6(+0x6213d)[0xf747313d] /lib32/libc.so.6(+0x6c231)[0xf747d231] /lib32/libc.so.6(+0x6dab8)[0xf747eab8] /lib32/libc.so.6(cfree+0x6d)[0xf7481b9d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf766ba91] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf766baed] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7427bd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Sep 2011 06:18:15	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	259,200	486,586	1.8773
25 Sep 2011 00:24:29	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	233,280	437,601	1.8759
24 Sep 2011 05:29:39	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	207,360	388,953	1.8757
23 Sep 2011 03:01:28	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	181,440	340,361	1.8759
22 Sep 2011 04:35:10	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	155,520	291,724	1.8758
21 Sep 2011 04:13:46	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	129,600	243,120	1.8759
19 Sep 2011 21:39:50	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	103,680	194,456	1.8755
18 Sep 2011 18:36:12	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	77,760	145,852	1.8757
17 Sep 2011 16:07:27	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	51,840	97,315	1.8772
16 Sep 2011 17:52:48	944551	13341802	hadcm3n_o0mf_1900_40_007438699_2	25,920	48,800	1.8827