climateprediction.net home page
Task 13771268

Task 13771268

Name hadcm3n_o0y2_1980_40_007615428_1
Workunit 7793558
Created 12 Dec 2011, 23:42:01 UTC
Sent 13 Dec 2011, 0:02:39 UTC
Report deadline 13 Mar 2012, 7:29:50 UTC
Received 12 Jan 2012, 10:47:47 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1366213
Run time 21 days 9 hours 20 min 13 sec
CPU time 21 days 9 hours 20 min 13 sec
Validate state Invalid
Credit 12,441.60
Device peak FLOPS 2.33 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.2.15</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
00:00:48 (31704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:00:49 (31704): No heartbeat from core client for 30 sec - exiting
00:00:50 (31704): No heartbeat from core client for 30 sec - exiting
00:00:51 (31704): No heartbeat from core client for 30 sec - exiting
00:00:52 (31704): No heartbeat from core client for 30 sec - exiting
00:00:53 (31704): No heartbeat from core client for 30 sec - exiting
00:00:54 (31704): No heartbeat from core client for 30 sec - exiting
00:00:55 (31704): No heartbeat from core client for 30 sec - exiting
00:00:56 (31704): No heartbeat from core client for 30 sec - exiting
00:00:57 (31704): No heartbeat from core client for 30 sec - exiting
00:00:58 (31704): No heartbeat from core client for 30 sec - exiting
00:00:59 (31704): No heartbeat from core client for 30 sec - exiting
00:01:00 (31704): No heartbeat from core client for 30 sec - exiting
00:02:40 (26908): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:02:43 (26908): No heartbeat from core client for 30 sec - exiting
00:02:44 (26908): No heartbeat from core client for 30 sec - exiting
00:02:45 (26908): No heartbeat from core client for 30 sec - exiting
00:02:46 (26908): No heartbeat from core client for 30 sec - exiting
00:16:41 (20089): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:20:04 (20687): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:02:30 (20851): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:02:36 (20851): No heartbeat from core client for 30 sec - exiting
00:02:37 (20851): No heartbeat from core client for 30 sec - exiting
00:02:38 (20851): No heartbeat from core client for 30 sec - exiting
00:02:39 (20851): No heartbeat from core client for 30 sec - exiting
00:02:40 (20851): No heartbeat from core client for 30 sec - exiting
00:02:41 (20851): No heartbeat from core client for 30 sec - exiting
00:02:42 (20851): No heartbeat from core client for 30 sec - exiting
00:02:43 (20851): No heartbeat from core client for 30 sec - exiting
00:02:44 (20851): No heartbeat from core client for 30 sec - exiting
00:02:45 (20851): No heartbeat from core client for 30 sec - exiting
00:02:46 (20851): No heartbeat from core client for 30 sec - exiting
00:02:47 (20851): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
00:02:53 (2409): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:02:55 (2409): No heartbeat from core client for 30 sec - exiting
00:02:56 (2409): No heartbeat from core client for 30 sec - exiting
00:00:49 (16175): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:00:51 (16175): No heartbeat from core client for 30 sec - exiting
00:00:52 (16175): No heartbeat from core client for 30 sec - exiting
00:00:53 (16175): No heartbeat from core client for 30 sec - exiting
00:00:54 (16175): No heartbeat from core client for 30 sec - exiting
00:00:55 (16175): No heartbeat from core client for 30 sec - exiting
00:00:56 (16175): No heartbeat from core client for 30 sec - exiting
00:00:57 (16175): No heartbeat from core client for 30 sec - exiting
00:04:41 (10479): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:04:42 (10479): No heartbeat from core client for 30 sec - exiting
00:02:27 (10569): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:00:59 (4874): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:01:00 (4874): No heartbeat from core client for 30 sec - exiting
00:01:01 (4874): No heartbeat from core client for 30 sec - exiting
00:01:02 (4874): No heartbeat from core client for 30 sec - exiting
00:01:03 (4874): No heartbeat from core client for 30 sec - exiting
00:04:11 (31361): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:04:12 (31361): No heartbeat from core client for 30 sec - exiting
00:04:13 (31361): No heartbeat from core client for 30 sec - exiting
00:04:14 (31361): No heartbeat from core client for 30 sec - exiting
00:04:15 (31361): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
00:01:56 (31438): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:05:09 (25837): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:05:11 (25837): No heartbeat from core client for 30 sec - exiting
00:05:12 (25837): No heartbeat from core client for 30 sec - exiting
00:05:13 (25837): No heartbeat from core client for 30 sec - exiting
00:04:20 (20166): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:04:21 (20166): No heartbeat from core client for 30 sec - exiting
00:04:22 (20166): No heartbeat from core client for 30 sec - exiting
00:04:23 (20166): No heartbeat from core client for 30 sec - exiting
00:04:24 (20166): No heartbeat from core client for 30 sec - exiting
00:04:25 (20166): No heartbeat from core client for 30 sec - exiting
00:04:26 (20166): No heartbeat from core client for 30 sec - exiting
00:04:27 (20166): No heartbeat from core client for 30 sec - exiting
00:04:28 (20166): No heartbeat from core client for 30 sec - exiting
00:04:29 (20166): No heartbeat from core client for 30 sec - exiting
00:02:31 (14472): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:02:32 (14472): No heartbeat from core client for 30 sec - exiting
00:02:33 (14472): No heartbeat from core client for 30 sec - exiting
00:02:34 (14472): No heartbeat from core client for 30 sec - exiting
00:02:35 (14472): No heartbeat from core client for 30 sec - exiting
00:02:36 (14472): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (!prev): 0x0a1765a0 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xb759a591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xb759bde8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xb759eecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xb7789741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xb778979d]
hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7545bd6]
hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:01 1057101    /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rwxp 0009b000 08:01 1057101    /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rwxp 00000000 00:00 0 
0a0b2000-0a1b2000 rwxp 00000000 00:00 0          [heap]
b6f00000-b6f21000 rwxp 00000000 00:00 0 
b6f21000-b7000000 ---p 00000000 00:00 0 
b70b2000-b752d000 rwxs 00000000 08:01 1056880    /home/boinc/BOINC/slots/0/134460
b752d000-b752f000 rwxp 00000000 00:00 0 
b752f000-b7682000 r-xp 00000000 08:05 1153095    /lib/tls/i686/cmov/libc-2.11.1.so
b7682000-b7683000 ---p 00153000 08:05 1153095    /lib/tls/i686/cmov/libc-2.11.1.so
b7683000-b7685000 r-xp 00153000 08:05 1153095    /lib/tls/i686/cmov/libc-2.11.1.so
b7685000-b7686000 rwxp 00155000 08:05 1153095    /lib/tls/i686/cmov/libc-2.11.1.so
b7686000-b7689000 rwxp 00000000 00:00 0 
b7689000-b76a6000 r-xp 00000000 08:05 1153143    /lib/libgcc_s.so.1
b76a6000-b76a7000 r-xp 0001c000 08:05 1153143    /lib/libgcc_s.so.1
b76a7000-b76a8000 rwxp 0001d000 08:05 1153143    /lib/libgcc_s.so.1
b76a8000-b76cc000 r-xp 00000000 08:05 1153106    /lib/tls/i686/cmov/libm-2.11.1.so
b76cc000-b76cd000 r-xp 00023000 08:05 1153106    /lib/tls/i686/cmov/libm-2.11.1.so
b76cd000-b76ce000 rwxp 00024000 08:05 1153106    /lib/tls/i686/cmov/libm-2.11.1.so
b76ce000-b77b7000 r-xp 00000000 08:05 344160     /usr/lib/libstdc++.so.6.0.13
b77b7000-b77b8000 ---p 000e9000 08:05 344160     /usr/lib/libstdc++.so.6.0.13
b77b8000-b77bc000 r-xp 000e9000 08:05 344160     /usr/lib/libstdc++.so.6.0.13
b77bc000-b77bd000 rwxp 000ed000 08:05 344160     /usr/lib/libstdc++.so.6.0.13
b77bd000-b77c5000 rwxp 00000000 00:00 0 
b77c5000-b77c7000 r-xp 00000000 08:05 1153105    /lib/tls/i686/cmov/libdl-2.11.1.so
b77c7000-b77c8000 r-xp 00001000 08:05 1153105    /lib/tls/i686/cmov/libdl-2.11.1.so
b77c8000-b77c9000 rwxp 00002000 08:05 1153105    /lib/tls/i686/cmov/libdl-2.11.1.so
b77c9000-b77de000 r-xp 00000000 08:05 1154175    /lib/tls/i686/cmov/libpthread-2.11.1.so
b77de000-b77df000 r-xp 00014000 08:05 1154175    /lib/tls/i686/cmov/libpthread-2.11.1.so
b77df000-b77e0000 rwxp 00015000 08:05 1154175    /lib/tls/i686/cmov/libpthread-2.11.1.so
b77e0000-b77e2000 rwxp 00000000 00:00 0 
b77ef000-b77f0000 rwxp 00000000 00:00 0 
b77f0000-b77f1000 ---p 00000000 00:00 0 
b77f1000-b77f4000 rwxp 00000000 00:00 0 
b77f4000-b77f6000 rwxs 00000000 08:01 1056844    /home/boinc/BOINC/slots/0/boinc_mmap_file
b77f6000-b77f8000 rwxp 00000000 00:00 0 
b77f8000-b77f9000 r-xp 00000000 00:00 0          [vdso]
b77f9000-b7814000 r-xp 00000000 08:05 1152819    /lib/ld-2.11.1.so
b7814000-b7815000 r-xp 0001a000 08:05 1152819    /lib/ld-2.11.1.so
b7815000-b7816000 rwxp 0001b000 08:05 1152819    /lib/ld-2.11.1.so
bf9a5000-bfa15000 rw-p 00000000 00:00 0          [stack]
SIGABRT: abort called
Stack trace (19 frames):
hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xb77f8400]
[0xb77f8422]
/lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0xb7559651]
/lib/tls/i686/cmov/libc.so.6(abort+0x182)[0xb755ca82]
/lib/tls/i686/cmov/libc.so.6(+0x6149d)[0xb759049d]
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xb759a591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xb759bde8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xb759eecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xb7789741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xb778979d]
hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7545bd6]
hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
12 Jan 2012 10:48:48 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 1,036,800 1,848,011 1.7824
11 Jan 2012 22:30:09 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 1,010,880 1,801,776 1.7824
11 Jan 2012 08:13:49 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 984,960 1,755,525 1.7823
10 Jan 2012 19:09:27 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 959,040 1,709,192 1.7822
10 Jan 2012 06:52:54 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 933,120 1,662,927 1.7821
09 Jan 2012 17:45:47 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 907,200 1,616,752 1.7821
09 Jan 2012 04:27:17 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 881,280 1,570,530 1.7821
08 Jan 2012 15:26:25 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 855,360 1,524,321 1.7821
08 Jan 2012 02:18:36 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 829,440 1,478,099 1.7820
07 Jan 2012 13:18:10 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 803,520 1,431,984 1.7821
07 Jan 2012 00:05:23 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 777,600 1,385,953 1.7823
06 Jan 2012 12:03:52 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 751,680 1,339,927 1.7826
05 Jan 2012 23:00:24 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 725,760 1,293,849 1.7828
05 Jan 2012 09:44:45 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 699,840 1,247,781 1.7830
04 Jan 2012 20:05:21 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 673,920 1,201,678 1.7831
04 Jan 2012 06:50:40 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 648,000 1,155,618 1.7834
03 Jan 2012 17:53:50 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 622,080 1,109,544 1.7836
02 Jan 2012 23:59:02 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 596,160 1,063,430 1.7838
02 Jan 2012 11:00:20 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 570,240 1,017,244 1.7839
01 Jan 2012 22:17:15 805269 13771268 hadcm3n_o0y2_1980_40_007615428_1 544,320 970,922 1.7837


©2024 climateprediction.net