climateprediction.net home page
Task 13620367

Task 13620367

Name hadcm3n_o3pi_1940_40_007539294_3
Workunit 7736526
Created 9 Nov 2011, 8:30:18 UTC
Sent 9 Nov 2011, 9:26:20 UTC
Report deadline 8 Feb 2012, 16:53:31 UTC
Received 15 Nov 2011, 17:23:04 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 673670
Run time 5 days 13 hours 2 min 49 sec
CPU time 17 hours 48 min 56 sec
Validate state Invalid
Credit 3,110.40
Device peak FLOPS 2.23 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.12.42</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
SIGSEGV: segmentation violation
Stack trace (3 frames):
/u/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77dc400]
/u/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8064fb6]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=24242, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
SIGSEGV: segmentation violation
Stack trace (3 frames):
/u/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf76fb400]
/u/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8064fb6]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=24242, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0879c728 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6b36f)[0xf756a36f]
/lib32/libc.so.6(+0x6cc18)[0xf756bc18]
/lib32/libc.so.6(cfree+0x6d)[0xf756eccd]
/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/32/libstdc++.so.6(_ZdlPv+0x21)[0xf7746841]
/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/32/libstdc++.so.6(_ZdaPv+0x1d)[0xf774689d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7515cc6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:06 956722                             /u/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:06 956722                             /u/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
08746000-087ad000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f7081000-f74fc000 rw-s 00000000 08:06 958267                             /u/boinc/slots/0/137650
f74fc000-f74ff000 rw-p 00000000 00:00 0 
f74ff000-f7651000 r-xp 00000000 08:03 778733                             /lib32/libc-2.12.2.so
f7651000-f7653000 r--p 00152000 08:03 778733                             /lib32/libc-2.12.2.so
f7653000-f7654000 rw-p 00154000 08:03 778733                             /lib32/libc-2.12.2.so
f7654000-f7657000 rw-p 00000000 00:00 0 
f7657000-f7670000 r-xp 00000000 08:03 638262                             /lib32/libgcc_s.so.1
f7670000-f7671000 r--p 00018000 08:03 638262                             /lib32/libgcc_s.so.1
f7671000-f7672000 rw-p 00019000 08:03 638262                             /lib32/libgcc_s.so.1
f7672000-f7696000 r-xp 00000000 08:03 781237                             /lib32/libm-2.12.2.so
f7696000-f7697000 r--p 00023000 08:03 781237                             /lib32/libm-2.12.2.so
f7697000-f7698000 rw-p 00024000 08:03 781237                             /lib32/libm-2.12.2.so
f7698000-f7699000 rw-p 00000000 00:00 0 
f7699000-f777a000 r-xp 00000000 08:03 1136674                            /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.3/32/libstdc++.so.6.0.14
f777a000-f777e000 r--p 000e0000 08:03 1136674                            /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.3/32/libstdc++.so.6.0.14
f777e000-f777f000 rw-p 000e4000 08:03 1136674                            /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.3/32/libstdc++.so.6.0.14
f777f000-f7786000 rw-p 00000000 00:00 0 
f7786000-f7788000 r-xp 00000000 08:03 781260                             /lib32/libdl-2.12.2.so
f7788000-f7789000 r--p 00001000 08:03 781260                             /lib32/libdl-2.12.2.so
f7789000-f778a000 rw-p 00002000 08:03 781260                             /lib32/libdl-2.12.2.so
f778a000-f779f000 r-xp 00000000 08:03 781269                             /lib32/libpthread-2.12.2.so
f779f000-f77a0000 r--p 00014000 08:03 781269                             /lib32/libpthread-2.12.2.so
f77a0000-f77a1000 rw-p 00015000 08:03 781269                             /lib32/libpthread-2.12.2.so
f77a1000-f77a3000 rw-p 00000000 00:00 0 
f77d2000-f77d3000 rw-p 00000000 00:00 0 
f77d3000-f77d4000 ---p 00000000 00:00 0 
f77d4000-f77d7000 rw-p 00000000 00:00 0 
f77d7000-f77d9000 rw-s 00000000 08:06 957988                             /u/boinc/slots/0/boinc_mmap_file
f77d9000-f77da000 rw-p 00000000 00:00 0 
f77da000-f77db000 r-xp 00000000 00:00 0                                  [vdso]
f77db000-f77f7000 r-xp 00000000 08:03 781236                             /lib32/ld-2.12.2.so
f77f7000-f77f8000 r--p 0001b000 08:03 781236                             /lib32/ld-2.12.2.so
f77f8000-f77f9000 rw-p 0001c000 08:03 781236                             /lib32/ld-2.12.2.so
ffcd8000-ffd48000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf77da400]
[0xf77da425]
/lib32/libc.so.6(gsignal+0x51)[0xf7529761]
/lib32/libc.so.6(abort+0x183)[0xf752aec3]
/lib32/libc.so.6(+0x65535)[0xf7564535]
/lib32/libc.so.6(+0x6b36f)[0xf756a36f]
/lib32/libc.so.6(+0x6cc18)[0xf756bc18]
/lib32/libc.so.6(cfree+0x6d)[0xf756eccd]
/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/32/libstdc++.so.6(_ZdlPv+0x21)[0xf7746841]
/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/32/libstdc++.so.6(_ZdaPv+0x1d)[0xf774689d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7515cc6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Nov 2011 17:48:49 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 259,200 64,134 0.2474
15 Nov 2011 17:48:49 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 233,280 16,118 0.0691
15 Nov 2011 17:48:49 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 207,360 329,126 1.5872
15 Nov 2011 17:48:48 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 181,440 281,141 1.5495
15 Nov 2011 17:48:49 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 155,520 233,264 1.4999
15 Nov 2011 17:48:49 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 129,600 185,487 1.4312
15 Nov 2011 17:48:49 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 103,680 137,771 1.3288
15 Nov 2011 17:48:49 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 77,760 89,919 1.1564
15 Nov 2011 17:48:49 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 51,840 42,167 0.8134
09 Nov 2011 22:49:39 673670 13620367 hadcm3n_o3pi_1940_40_007539294_3 25,920 47,649 1.8383


©2024 cpdn.org