climateprediction.net home page
Task 13770634

Task 13770634

Name hadcm3n_005m_1900_40_007486399_4
Workunit 7683874
Created 12 Dec 2011, 21:07:31 UTC
Sent 14 Dec 2011, 0:51:25 UTC
Report deadline 14 Mar 2012, 8:18:36 UTC
Received 25 Dec 2011, 17:34:26 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1124964
Run time 9 days 1 hours 2 min 37 sec
CPU time 8 days 21 hours 14 min 30 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 3.22 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:38:47 (17459): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:55:19 (2214): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:55:28 (22633): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
16:56:57 (24380): No heartbeat from core client for 30 sec - exiting
16:56:58 (24380): No heartbeat from core client for 30 sec - exiting
16:56:59 (24380): No heartbeat from core client for 30 sec - exiting
16:57:00 (24380): No heartbeat from core client for 30 sec - exiting
16:57:01 (24380): No heartbeat from core client for 30 sec - exiting
16:57:02 (24380): No heartbeat from core client for 30 sec - exiting
16:57:03 (24380): No heartbeat from core client for 30 sec - exiting
16:57:04 (24380): No heartbeat from core client for 30 sec - exiting
16:57:05 (24380): No heartbeat from core client for 30 sec - exiting
16:57:06 (24380): No heartbeat from core client for 30 sec - exiting
16:57:07 (24380): No heartbeat from core client for 30 sec - exiting
16:57:08 (24380): No heartbeat from core client for 30 sec - exiting
16:57:09 (24380): No heartbeat from core client for 30 sec - exiting
16:57:10 (24380): No heartbeat from core client for 30 sec - exiting
16:57:11 (24380): No heartbeat from core client for 30 sec - exiting
16:57:12 (24380): No heartbeat from core client for 30 sec - exiting
16:57:13 (24380): No heartbeat from core client for 30 sec - exiting
16:57:14 (24380): No heartbeat from core client for 30 sec - exiting
16:57:15 (24380): No heartbeat from core client for 30 sec - exiting
16:57:16 (24380): No heartbeat from core client for 30 sec - exiting
16:57:17 (24380): No heartbeat from core client for 30 sec - exiting
16:57:18 (24380): No heartbeat from core client for 30 sec - exiting
16:57:19 (24380): No heartbeat from core client for 30 sec - exiting
16:57:20 (24380): No heartbeat from core client for 30 sec - exiting
16:57:21 (24380): No heartbeat from core client for 30 sec - exiting
16:57:22 (24380): No heartbeat from core client for 30 sec - exiting
16:57:23 (24380): No heartbeat from core client for 30 sec - exiting
16:57:24 (24380): No heartbeat from core client for 30 sec - exiting
16:57:25 (24380): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:17:51 (26549): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09711a88 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6eb72)[0xf74d7b72]
/lib32/libc.so.6(+0x6f812)[0xf74d8812]
/lib32/libc.so.6(cfree+0x6d)[0xf74db8cd]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf76d8baf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf76d8c0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf74820f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:01 7733689                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:01 7733689                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
096bc000-09721000 rw-p 00000000 00:00 0                                  [heap]
f6e00000-f6e21000 rw-p 00000000 00:00 0 
f6e21000-f6f00000 ---p 00000000 00:00 0 
f6feb000-f7466000 rw-s 00000000 08:01 8652713                            /var/lib/boinc-client/slots/1/132940
f7466000-f7469000 rw-p 00000000 00:00 0 
f7469000-f75dc000 r-xp 00000000 08:01 20447286                           /lib32/libc-2.13.so
f75dc000-f75dd000 ---p 00173000 08:01 20447286                           /lib32/libc-2.13.so
f75dd000-f75df000 r--p 00173000 08:01 20447286                           /lib32/libc-2.13.so
f75df000-f75e0000 rw-p 00175000 08:01 20447286                           /lib32/libc-2.13.so
f75e0000-f75e3000 rw-p 00000000 00:00 0 
f75e3000-f75ff000 r-xp 00000000 08:01 28051663                           /usr/lib32/libgcc_s.so.1
f75ff000-f7600000 r--p 0001b000 08:01 28051663                           /usr/lib32/libgcc_s.so.1
f7600000-f7601000 rw-p 0001c000 08:01 28051663                           /usr/lib32/libgcc_s.so.1
f7601000-f7629000 r-xp 00000000 08:01 20447298                           /lib32/libm-2.13.so
f7629000-f762a000 r--p 00028000 08:01 20447298                           /lib32/libm-2.13.so
f762a000-f762b000 rw-p 00029000 08:01 20447298                           /lib32/libm-2.13.so
f762b000-f7709000 r-xp 00000000 08:01 28051292                           /usr/lib32/libstdc++.so.6.0.16
f7709000-f770a000 ---p 000de000 08:01 28051292                           /usr/lib32/libstdc++.so.6.0.16
f770a000-f770e000 r--p 000de000 08:01 28051292                           /usr/lib32/libstdc++.so.6.0.16
f770e000-f770f000 rw-p 000e2000 08:01 28051292                           /usr/lib32/libstdc++.so.6.0.16
f770f000-f7717000 rw-p 00000000 00:00 0 
f7717000-f771a000 r-xp 00000000 08:01 20447287                           /lib32/libdl-2.13.so
f771a000-f771b000 r--p 00002000 08:01 20447287                           /lib32/libdl-2.13.so
f771b000-f771c000 rw-p 00003000 08:01 20447287                           /lib32/libdl-2.13.so
f771c000-f7733000 r-xp 00000000 08:01 20447294                           /lib32/libpthread-2.13.so
f7733000-f7734000 r--p 00016000 08:01 20447294                           /lib32/libpthread-2.13.so
f7734000-f7735000 rw-p 00017000 08:01 20447294                           /lib32/libpthread-2.13.so
f7735000-f7737000 rw-p 00000000 00:00 0 
f775e000-f775f000 rw-p 00000000 00:00 0 
f775f000-f7760000 ---p 00000000 00:00 0 
f7760000-f7763000 rw-p 00000000 00:00 0 
f7763000-f7765000 rw-s 00000000 08:01 8652519                            /var/lib/boinc-client/slots/1/boinc_mmap_file
f7765000-f7767000 rw-p 00000000 00:00 0 
f7767000-f7768000 r-xp 00000000 00:00 0                                  [vdso]
f7768000-f7786000 r-xp 00000000 08:01 17563841                           /lib/i386-linux-gnu/ld-2.13.so
f7786000-f7787000 r--p 0001d000 08:01 17563841                           /lib/i386-linux-gnu/ld-2.13.so
f7787000-f7788000 rw-p 0001e000 08:01 17563841                           /lib/i386-linux-gnu/ld-2.13.so
ffd5d000-ffdce000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf7767400]
[0xf7767425]
/lib32/libc.so.6(gsignal+0x4f)[0xf7496c4f]
/lib32/libc.so.6(abort+0x175)[0xf749a175]
/lib32/libc.so.6(+0x63e8c)[0xf74cce8c]
/lib32/libc.so.6(+0x6eb72)[0xf74d7b72]
/lib32/libc.so.6(+0x6f812)[0xf74d8812]
/lib32/libc.so.6(cfree+0x6d)[0xf74db8cd]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf76d8baf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf76d8c0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf74820f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Dec 2011 17:40:37 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 518,400 767,664 1.4808
25 Dec 2011 02:52:28 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 492,480 729,160 1.4806
24 Dec 2011 14:41:03 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 466,560 690,289 1.4795
24 Dec 2011 01:28:34 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 440,640 651,827 1.4793
23 Dec 2011 13:27:17 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 414,720 613,483 1.4793
22 Dec 2011 23:38:33 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 388,800 574,985 1.4789
22 Dec 2011 07:38:10 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 362,880 536,093 1.4773
21 Dec 2011 19:12:15 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 336,960 497,266 1.4757
21 Dec 2011 07:37:37 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 311,040 459,173 1.4763
20 Dec 2011 17:58:50 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 285,120 420,962 1.4764
20 Dec 2011 07:07:27 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 259,200 382,942 1.4774
19 Dec 2011 19:12:22 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 233,280 344,822 1.4781
19 Dec 2011 06:12:04 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 207,360 306,779 1.4795
18 Dec 2011 18:43:41 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 181,440 268,895 1.4820
18 Dec 2011 05:16:54 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 155,520 230,667 1.4832
17 Dec 2011 10:14:13 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 129,600 192,598 1.4861
16 Dec 2011 23:21:22 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 103,680 153,963 1.4850
16 Dec 2011 12:37:53 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 77,760 115,345 1.4833
16 Dec 2011 01:34:42 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 51,840 76,859 1.4826
15 Dec 2011 13:41:49 1124964 13770634 hadcm3n_005m_1900_40_007486399_4 25,920 38,277 1.4767


©2024 climateprediction.net