climateprediction.net home page
Task 15773486

Task 15773486

Name hadcm3n_3lb4_1980_40_008364122_0
Workunit 8514981
Created 10 May 2013, 19:15:27 UTC
Sent 10 May 2013, 19:21:52 UTC
Report deadline 10 Aug 2013, 2:49:03 UTC
Received 29 May 2013, 2:36:55 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1179592
Run time 18 days 3 hours 17 min 34 sec
CPU time 16 days 17 hours 51 min 52 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 1.77 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
20:08:12 (27443): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:16:26 (29885): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:39:57 (30799): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:13:32 (33265): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:14:01 (33265): No heartbeat from core client for 30 sec - exiting
22:14:02 (33265): No heartbeat from core client for 30 sec - exiting
22:14:03 (33265): No heartbeat from core client for 30 sec - exiting
22:14:04 (33265): No heartbeat from core client for 30 sec - exiting
22:14:05 (33265): No heartbeat from core client for 30 sec - exiting
22:14:06 (33265): No heartbeat from core client for 30 sec - exiting
22:14:07 (33265): No heartbeat from core client for 30 sec - exiting
23:00:23 (38173): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:57:37 (38425): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:02:27 (38725): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:03:15 (38981): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:03:27 (39232): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:06:18 (39591): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:10:56 (39847): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:21:30 (40334): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:11:44 (40544): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:48 (40782): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:51:46 (41179): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:16:39 (41650): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:18:35 (41880): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:53 (42238): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:18:20 (42513): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:25:47 (43116): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:13:25 (43317): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:26:29 (43553): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:52:26 (43765): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:28:05 (43982): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:39:26 (44216): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:47:46 (44647): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:30:21 (44851): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:31:01 (45088): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:31:42 (45339): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:06:45 (45610): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:33:05 (45920): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:33:46 (46153): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:39:16 (46423): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:43:47 (46628): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:16:30 (47676): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:26:21 (2625): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:16:55 (3353): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:42:38 (6277): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:52:43 (6605): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:47:29 (6873): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:52:38 (7235): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:47:51 (7440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:01:35 (7670): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:49:15 (8207): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:55:27 (8440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:02:27 (8643): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:08:22 (8843): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:03:05 (9045): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:04:02 (9276): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:38:53 (9508): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:45:29 (9741): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:40:27 (9939): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08344a10 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6bff1)[0xf753aff1]
/lib32/libc.so.6(+0x6d880)[0xf753c880]
/lib32/libc.so.6(cfree+0x6d)[0xf753f92d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf77167b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf771680d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf74e5ce7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:02 3514599                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:02 3514599                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
082ef000-08354000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f7051000-f74cc000 rw-s 00000000 08:02 5210216                            /var/lib/boinc-client/slots/22/132695
f74cc000-f74cf000 rw-p 00000000 00:00 0 
f74cf000-f7623000 r-xp 00000000 08:02 9879703                            /lib32/libc-2.12.1.so
f7623000-f7624000 ---p 00154000 08:02 9879703                            /lib32/libc-2.12.1.so
f7624000-f7626000 r--p 00154000 08:02 9879703                            /lib32/libc-2.12.1.so
f7626000-f7627000 rw-p 00156000 08:02 9879703                            /lib32/libc-2.12.1.so
f7627000-f762a000 rw-p 00000000 00:00 0 
f762a000-f7644000 r-xp 00000000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f7644000-f7645000 r--p 00019000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f7645000-f7646000 rw-p 0001a000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f7646000-f766a000 r-xp 00000000 08:02 9879707                            /lib32/libm-2.12.1.so
f766a000-f766b000 r--p 00023000 08:02 9879707                            /lib32/libm-2.12.1.so
f766b000-f766c000 rw-p 00024000 08:02 9879707                            /lib32/libm-2.12.1.so
f766c000-f774b000 r-xp 00000000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f774b000-f774f000 r--p 000de000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f774f000-f7750000 rw-p 000e2000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f7750000-f7757000 rw-p 00000000 00:00 0 
f7757000-f7759000 r-xp 00000000 08:02 9879706                            /lib32/libdl-2.12.1.so
f7759000-f775a000 r--p 00001000 08:02 9879706                            /lib32/libdl-2.12.1.so
f775a000-f775b000 rw-p 00002000 08:02 9879706                            /lib32/libdl-2.12.1.so
f775b000-f775c000 rw-p 00000000 00:00 0 
f775c000-f7771000 r-xp 00000000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7771000-f7772000 r--p 00014000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7772000-f7773000 rw-p 00015000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7773000-f7775000 rw-p 00000000 00:00 0 
f7786000-f7787000 rw-p 00000000 00:00 0 
f7787000-f7788000 ---p 00000000 00:00 0 
f7788000-f778b000 rw-p 00000000 00:00 0 
f778b000-f778d000 rw-s 00000000 08:02 5210134                            /var/lib/boinc-client/slots/22/boinc_mmap_file
f778d000-f778f000 rw-p 00000000 00:00 0 
f778f000-f7790000 r-xp 00000000 00:00 0                                  [vdso]
f7790000-f77ac000 r-xp 00000000 08:02 9879700                            /lib32/ld-2.12.1.so
f77ac000-f77ad000 r--p 0001b000 08:02 9879700                            /lib32/ld-2.12.1.so
f77ad000-f77ae000 rw-p 0001c000 08:02 9879700                            /lib32/ld-2.12.1.so
fff1f000-fff90000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf778f400]
[0xf778f425]
/lib32/libc.so.6(gsignal+0x51)[0xf74f9a01]
/lib32/libc.so.6(abort+0x182)[0xf74fce42]
/lib32/libc.so.6(+0x61f15)[0xf7530f15]
/lib32/libc.so.6(+0x6bff1)[0xf753aff1]
/lib32/libc.so.6(+0x6d880)[0xf753c880]
/lib32/libc.so.6(cfree+0x6d)[0xf753f92d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf77167b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf771680d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf74e5ce7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
29 May 2013 02:40:51 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 518,400 1,446,700 2.7907
28 May 2013 06:42:55 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 492,480 1,378,218 2.7985
27 May 2013 11:09:45 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 466,560 1,307,343 2.8021
26 May 2013 14:51:56 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 440,640 1,235,862 2.8047
25 May 2013 19:13:52 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 414,720 1,164,583 2.8081
24 May 2013 22:38:24 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 388,800 1,093,205 2.8117
24 May 2013 03:21:04 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 362,880 1,022,705 2.8183
23 May 2013 08:02:48 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 336,960 954,072 2.8314
22 May 2013 13:50:14 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 311,040 888,693 2.8572
21 May 2013 19:36:21 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 285,120 823,096 2.8868
21 May 2013 00:19:53 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 259,200 756,080 2.9170
20 May 2013 05:04:39 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 233,280 689,319 2.9549
19 May 2013 07:11:40 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 207,360 614,259 2.9623
18 May 2013 10:39:21 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 181,440 539,222 2.9719
17 May 2013 13:52:20 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 155,520 464,464 2.9865
16 May 2013 16:36:50 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 129,600 389,122 3.0025
15 May 2013 17:33:54 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 103,680 305,147 2.9432
14 May 2013 17:17:29 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 77,760 220,671 2.8378
13 May 2013 22:09:14 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 51,840 150,183 2.8970
13 May 2013 00:44:31 1179592 15773486 hadcm3n_3lb4_1980_40_008364122_0 25,920 75,351 2.9071


©2024 climateprediction.net