climateprediction.net home page
Task 13351269

Task 13351269

Name hadcm3n_t7g7_1940_40_007444358_0
Workunit 7641861
Created 9 Sep 2011, 2:10:26 UTC
Sent 9 Sep 2011, 4:00:55 UTC
Report deadline 9 Dec 2011, 11:28:06 UTC
Received 30 Sep 2011, 11:35:37 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1164072
Run time 6 days 8 hours 58 min 50 sec
CPU time 6 days 7 hours 37 min 44 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 3.67 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.59</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:49:09 (16917): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09fee0c8 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6b511)[0xf7484511]
/lib32/libc.so.6(+0x6ce1b)[0xf7485e1b]
/lib32/libc.so.6(cfree+0x6d)[0xf7488f8d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf7662841]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf766289d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf742fe37]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:12 57409557                           /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:12 57409557                           /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
09f98000-09ffc000 rw-p 00000000 00:00 0                                  [heap]
f6e00000-f6e21000 rw-p 00000000 00:00 0 
f6e21000-f6f00000 ---p 00000000 00:00 0 
f6f9b000-f7416000 rw-s 00000000 08:12 57540751                           /var/lib/boinc-client/slots/0/134340
f7416000-f7419000 rw-p 00000000 00:00 0 
f7419000-f756f000 r-xp 00000000 08:12 119537669                          /lib32/libc-2.13.so
f756f000-f7570000 ---p 00156000 08:12 119537669                          /lib32/libc-2.13.so
f7570000-f7572000 r--p 00156000 08:12 119537669                          /lib32/libc-2.13.so
f7572000-f7573000 rw-p 00158000 08:12 119537669                          /lib32/libc-2.13.so
f7573000-f7576000 rw-p 00000000 00:00 0 
f7576000-f7590000 r-xp 00000000 08:12 43779439                           /usr/lib32/libgcc_s.so.1
f7590000-f7591000 r--p 00019000 08:12 43779439                           /usr/lib32/libgcc_s.so.1
f7591000-f7592000 rw-p 0001a000 08:12 43779439                           /usr/lib32/libgcc_s.so.1
f7592000-f75b6000 r-xp 00000000 08:12 119537673                          /lib32/libm-2.13.so
f75b6000-f75b7000 r--p 00023000 08:12 119537673                          /lib32/libm-2.13.so
f75b7000-f75b8000 rw-p 00024000 08:12 119537673                          /lib32/libm-2.13.so
f75b8000-f7697000 r-xp 00000000 08:12 43779442                           /usr/lib32/libstdc++.so.6.0.14
f7697000-f769b000 r--p 000de000 08:12 43779442                           /usr/lib32/libstdc++.so.6.0.14
f769b000-f769c000 rw-p 000e2000 08:12 43779442                           /usr/lib32/libstdc++.so.6.0.14
f769c000-f76a3000 rw-p 00000000 00:00 0 
f76a3000-f76a5000 r-xp 00000000 08:12 119537672                          /lib32/libdl-2.13.so
f76a5000-f76a6000 r--p 00001000 08:12 119537672                          /lib32/libdl-2.13.so
f76a6000-f76a7000 rw-p 00002000 08:12 119537672                          /lib32/libdl-2.13.so
f76a7000-f76bc000 r-xp 00000000 08:12 119537683                          /lib32/libpthread-2.13.so
f76bc000-f76bd000 r--p 00014000 08:12 119537683                          /lib32/libpthread-2.13.so
f76bd000-f76be000 rw-p 00015000 08:12 119537683                          /lib32/libpthread-2.13.so
f76be000-f76c1000 rw-p 00000000 00:00 0 
f76dd000-f76de000 rw-p 00000000 00:00 0 
f76de000-f76df000 ---p 00000000 00:00 0 
f76df000-f76e2000 rw-p 00000000 00:00 0 
f76e2000-f76e4000 rw-s 00000000 08:12 57540748                           /var/lib/boinc-client/slots/0/boinc_mmap_file
f76e4000-f76e5000 rw-p 00000000 00:00 0 
f76e5000-f76e6000 r-xp 00000000 00:00 0                                  [vdso]
f76e6000-f7702000 r-xp 00000000 08:12 119537666                          /lib32/ld-2.13.so
f7702000-f7703000 r--p 0001b000 08:12 119537666                          /lib32/ld-2.13.so
f7703000-f7704000 rw-p 0001c000 08:12 119537666                          /lib32/ld-2.13.so
ff810000-ff880000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf76e5400]
[0xf76e5430]
/lib32/libc.so.6(gsignal+0x51)[0xf7443ea1]
/lib32/libc.so.6(abort+0x17e)[0xf74472ce]
/lib32/libc.so.6(+0x61277)[0xf747a277]
/lib32/libc.so.6(+0x6b511)[0xf7484511]
/lib32/libc.so.6(+0x6ce1b)[0xf7485e1b]
/lib32/libc.so.6(cfree+0x6d)[0xf7488f8d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf7662841]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf766289d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf742fe37]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
30 Sep 2011 11:33:43 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 518,400 545,861 1.0530
30 Sep 2011 02:40:16 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 492,480 518,479 1.0528
29 Sep 2011 17:10:27 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 466,560 491,049 1.0525
29 Sep 2011 08:48:02 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 440,640 463,485 1.0518
28 Sep 2011 19:05:52 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 414,720 436,072 1.0515
28 Sep 2011 08:20:13 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 388,800 408,273 1.0501
27 Sep 2011 20:50:15 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 362,880 381,117 1.0503
27 Sep 2011 10:39:23 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 336,960 353,494 1.0491
26 Sep 2011 11:01:08 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 311,040 326,291 1.0490
26 Sep 2011 02:10:30 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 285,120 298,758 1.0478
25 Sep 2011 17:48:24 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 259,200 271,349 1.0469
25 Sep 2011 02:11:13 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 233,280 244,061 1.0462
24 Sep 2011 09:42:05 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 207,360 216,983 1.0464
23 Sep 2011 19:36:11 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 181,440 189,915 1.0467
23 Sep 2011 09:54:45 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 155,520 162,845 1.0471
22 Sep 2011 17:58:40 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 129,600 135,749 1.0474
21 Sep 2011 12:44:18 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 103,680 108,611 1.0476
21 Sep 2011 01:57:21 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 77,760 81,491 1.0480
15 Sep 2011 20:56:00 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 51,840 54,384 1.0491
12 Sep 2011 12:42:55 1164072 13351269 hadcm3n_t7g7_1940_40_007444358_0 25,920 27,197 1.0493


©2024 climateprediction.net