Task 13385614

Name	hadcm3n_t1bm_1940_40_007449203_2
Workunit	7646706
Created	14 Sep 2011, 12:10:28 UTC
Sent	14 Sep 2011, 12:35:31 UTC
Report deadline	14 Dec 2011, 20:02:42 UTC
Received	24 Sep 2011, 9:10:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1162459
Run time	6 days 8 hours 7 min 50 sec
CPU time	5 days 16 hours 35 min 13 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.76 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09b22ea8 *** ======= Backtrace: ========= /lib/libc.so.6[0x719fe1] /usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0xc63632] /usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0xc6368e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe6)[0x6c1cc6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 00689000-006a7000 r-xp 00000000 fd:00 1050260 /lib/ld-2.12.so 006a7000-006a8000 r--p 0001d000 fd:00 1050260 /lib/ld-2.12.so 006a8000-006a9000 rw-p 0001e000 fd:00 1050260 /lib/ld-2.12.so 006ab000-00830000 r-xp 00000000 fd:00 1060460 /lib/libc-2.12.so 00830000-00831000 ---p 00185000 fd:00 1060460 /lib/libc-2.12.so 00831000-00833000 r--p 00185000 fd:00 1060460 /lib/libc-2.12.so 00833000-00834000 rw-p 00187000 fd:00 1060460 /lib/libc-2.12.so 00834000-00837000 rw-p 00000000 00:00 0 0087f000-00882000 r-xp 00000000 fd:00 1062647 /lib/libdl-2.12.so 00882000-00883000 r--p 00002000 fd:00 1062647 /lib/libdl-2.12.so 00883000-00884000 rw-p 00003000 fd:00 1062647 /lib/libdl-2.12.so 00977000-0099f000 r-xp 00000000 fd:00 1062653 /lib/libm-2.12.so 0099f000-009a0000 r--p 00027000 fd:00 1062653 /lib/libm-2.12.so 009a0000-009a1000 rw-p 00028000 fd:00 1062653 /lib/libm-2.12.so 00b77000-00b8e000 r-xp 00000000 fd:00 1063075 /lib/libpthread-2.12.so 00b8e000-00b8f000 r--p 00016000 fd:00 1063075 /lib/libpthread-2.12.so 00b8f000-00b90000 rw-p 00017000 fd:00 1063075 /lib/libpthread-2.12.so 00b90000-00b92000 rw-p 00000000 00:00 0 00b94000-00bb1000 r-xp 00000000 fd:00 1063703 /lib/libgcc_s-4.4.4-20100726.so.1 00bb1000-00bb2000 rw-p 0001d000 fd:00 1063703 /lib/libgcc_s-4.4.4-20100726.so.1 00bb4000-00c95000 r-xp 00000000 fd:00 923221 /usr/lib/libstdc++.so.6.0.13 00c95000-00c99000 r--p 000e0000 fd:00 923221 /usr/lib/libstdc++.so.6.0.13 00c99000-00c9b000 rw-p 000e4000 fd:00 923221 /usr/lib/libstdc++.so.6.0.13 00c9b000-00ca1000 rw-p 00000000 00:00 0 00e93000-00e94000 r-xp 00000000 00:00 0 [vdso] 08048000-080e3000 r-xp 00000000 fd:02 6031010 /home/suntao/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 fd:02 6031010 /home/suntao/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 09acd000-09b54000 rw-p 00000000 00:00 0 [heap] f72e0000-f775b000 rw-s 00000000 fd:02 10485990 /home/suntao/BOINC/slots/6/135880 f775b000-f775e000 rw-p 00000000 00:00 0 f7776000-f7777000 rw-p 00000000 00:00 0 f7777000-f7778000 ---p 00000000 00:00 0 f7778000-f777b000 rw-p 00000000 00:00 0 f777b000-f777d000 rw-s 00000000 fd:02 10485987 /home/suntao/BOINC/slots/6/boinc_mmap_file f777d000-f777e000 rw-p 00000000 00:00 0 ff9d8000-ffa48000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (17 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xe93400] [0xe93430] /lib/libc.so.6(gsignal+0x51)[0x6d5d11] /lib/libc.so.6(abort+0x17a)[0x6d75ea] /lib/libc.so.6[0x713b9d] /lib/libc.so.6[0x719fe1] /usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0xc63632] /usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0xc6368e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe6)[0x6c1cc6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Sep 2011 09:11:40	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	259,200	491,707	1.8970
23 Sep 2011 17:39:01	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	233,280	443,750	1.9022
20 Sep 2011 02:43:23	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	207,360	396,595	1.9126
18 Sep 2011 23:41:21	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	181,440	347,222	1.9137
18 Sep 2011 08:31:34	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	155,520	297,883	1.9154
17 Sep 2011 17:41:48	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	129,600	249,107	1.9221
17 Sep 2011 01:21:29	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	103,680	200,056	1.9296
16 Sep 2011 08:41:44	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	77,760	151,871	1.9531
15 Sep 2011 18:33:00	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	51,840	101,945	1.9665
15 Sep 2011 03:37:42	1162459	13385614	hadcm3n_t1bm_1940_40_007449203_2	25,920	51,959	2.0046