climateprediction.net home page
Task 14654652

Task 14654652

Name hadcm3n_202o_1980_40_007958906_0
Workunit 8114018
Created 10 May 2012, 10:19:25 UTC
Sent 10 May 2012, 10:53:37 UTC
Report deadline 9 Aug 2012, 18:20:48 UTC
Received 24 Jun 2012, 1:30:25 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1179592
Run time 37 days 12 hours 20 min 11 sec
CPU time 37 days 9 hours 23 min 16 sec
Validate state Invalid
Credit 12,441.60
Device peak FLOPS 1.37 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:34:52 (19558): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:04:10 (23712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:24:50 (24207): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:24:57 (24207): No heartbeat from core client for 30 sec - exiting
17:27:19 (26434): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:20:18 (26664): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:17:04 (26902): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:22:34 (27193): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:27:53 (27402): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:33:58 (27608): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:36:32 (27820): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:39:39 (28225): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:47:17 (28449): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:06:00 (28657): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:47:29 (29040): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:53:18 (29407): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:57:56 (29616): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:45:18 (29820): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:32:12 (30119): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:48:23 (30337): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:49:03 (31387): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:03:07 (33346): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:35:17 (33598): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:47:10 (34063): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:39:54 (34315): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:43:47 (34533): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:50:01 (34773): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:55:39 (35124): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:57:47 (35399): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:44:04 (35617): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:56:07 (35826): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:57:23 (36075): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:17:07 (36297): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:17:08 (36297): No heartbeat from core client for 30 sec - exiting
14:17:09 (36297): No heartbeat from core client for 30 sec - exiting
14:17:10 (36297): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
13:05:57 (36933): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:05:58 (36933): No heartbeat from core client for 30 sec - exiting
13:05:59 (36933): No heartbeat from core client for 30 sec - exiting
13:06:00 (36933): No heartbeat from core client for 30 sec - exiting
13:06:01 (36933): No heartbeat from core client for 30 sec - exiting
13:06:02 (36933): No heartbeat from core client for 30 sec - exiting
13:06:03 (36933): No heartbeat from core client for 30 sec - exiting
14:07:59 (37727): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:16:26 (37971): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:21:00 (38189): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:23:15 (38404): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:50:02 (41562): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:55:34 (41847): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:26:46 (42079): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x084f5070 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6bff1)[0xf7543ff1]
/lib32/libc.so.6(+0x6d880)[0xf7545880]
/lib32/libc.so.6(cfree+0x6d)[0xf754892d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf771f7b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf771f80d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf74eece7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:02 3514599                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:02 3514599                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
0849f000-08504000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f705a000-f74d5000 rw-s 00000000 08:02 3637273                            /var/lib/boinc-client/slots/41/133180
f74d5000-f74d8000 rw-p 00000000 00:00 0 
f74d8000-f762c000 r-xp 00000000 08:02 9879703                            /lib32/libc-2.12.1.so
f762c000-f762d000 ---p 00154000 08:02 9879703                            /lib32/libc-2.12.1.so
f762d000-f762f000 r--p 00154000 08:02 9879703                            /lib32/libc-2.12.1.so
f762f000-f7630000 rw-p 00156000 08:02 9879703                            /lib32/libc-2.12.1.so
f7630000-f7633000 rw-p 00000000 00:00 0 
f7633000-f764d000 r-xp 00000000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f764d000-f764e000 r--p 00019000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f764e000-f764f000 rw-p 0001a000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f764f000-f7673000 r-xp 00000000 08:02 9879707                            /lib32/libm-2.12.1.so
f7673000-f7674000 r--p 00023000 08:02 9879707                            /lib32/libm-2.12.1.so
f7674000-f7675000 rw-p 00024000 08:02 9879707                            /lib32/libm-2.12.1.so
f7675000-f7754000 r-xp 00000000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f7754000-f7758000 r--p 000de000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f7758000-f7759000 rw-p 000e2000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f7759000-f7760000 rw-p 00000000 00:00 0 
f7760000-f7762000 r-xp 00000000 08:02 9879706                            /lib32/libdl-2.12.1.so
f7762000-f7763000 r--p 00001000 08:02 9879706                            /lib32/libdl-2.12.1.so
f7763000-f7764000 rw-p 00002000 08:02 9879706                            /lib32/libdl-2.12.1.so
f7764000-f7765000 rw-p 00000000 00:00 0 
f7765000-f777a000 r-xp 00000000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f777a000-f777b000 r--p 00014000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f777b000-f777c000 rw-p 00015000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f777c000-f777e000 rw-p 00000000 00:00 0 
f778f000-f7790000 rw-p 00000000 00:00 0 
f7790000-f7791000 ---p 00000000 00:00 0 
f7791000-f7794000 rw-p 00000000 00:00 0 
f7794000-f7796000 rw-s 00000000 08:02 3637270                            /var/lib/boinc-client/slots/41/boinc_mmap_file
f7796000-f7798000 rw-p 00000000 00:00 0 
f7798000-f7799000 r-xp 00000000 00:00 0                                  [vdso]
f7799000-f77b5000 r-xp 00000000 08:02 9879700                            /lib32/ld-2.12.1.so
f77b5000-f77b6000 r--p 0001b000 08:02 9879700                            /lib32/ld-2.12.1.so
f77b6000-f77b7000 rw-p 0001c000 08:02 9879700                            /lib32/ld-2.12.1.so
ffc72000-ffce3000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf7798400]
[0xf7798425]
/lib32/libc.so.6(gsignal+0x51)[0xf7502a01]
/lib32/libc.so.6(abort+0x182)[0xf7505e42]
/lib32/libc.so.6(+0x61f15)[0xf7539f15]
/lib32/libc.so.6(+0x6bff1)[0xf7543ff1]
/lib32/libc.so.6(+0x6d880)[0xf7545880]
/lib32/libc.so.6(cfree+0x6d)[0xf754892d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf771f7b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf771f80d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf74eece7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
24 Jun 2012 01:32:15 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 1,036,800 3,230,586 3.1159
23 Jun 2012 04:13:20 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 1,010,880 3,153,368 3.1194
22 Jun 2012 05:57:50 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 984,960 3,075,072 3.1220
21 Jun 2012 08:15:50 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 959,040 2,996,989 3.1250
20 Jun 2012 09:54:41 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 933,120 2,919,094 3.1283
19 Jun 2012 12:40:08 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 907,200 2,841,401 3.1321
18 Jun 2012 15:12:19 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 881,280 2,763,882 3.1362
17 Jun 2012 16:20:24 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 855,360 2,684,729 3.1387
16 Jun 2012 17:05:27 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 829,440 2,601,990 3.1370
15 Jun 2012 16:44:44 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 803,520 2,517,838 3.1335
14 Jun 2012 18:10:14 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 777,600 2,437,362 3.1345
13 Jun 2012 20:33:05 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 751,680 2,360,376 3.1401
12 Jun 2012 23:16:39 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 725,760 2,282,722 3.1453
12 Jun 2012 01:13:36 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 699,840 2,204,952 3.1507
11 Jun 2012 03:55:42 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 673,920 2,127,364 3.1567
10 Jun 2012 06:15:49 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 648,000 2,050,220 3.1639
09 Jun 2012 07:35:37 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 622,080 1,971,073 3.1685
08 Jun 2012 07:42:52 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 596,160 1,885,180 3.1622
07 Jun 2012 06:57:23 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 570,240 1,798,689 3.1543
06 Jun 2012 07:44:27 1179592 14654652 hadcm3n_202o_1980_40_007958906_0 544,320 1,714,854 3.1505


©2024 climateprediction.net