climateprediction.net home page
Task 15768940

Task 15768940

Name hadcm3n_498o_1980_40_008361729_0
Workunit 8512588
Created 9 May 2013, 16:02:06 UTC
Sent 9 May 2013, 16:02:27 UTC
Report deadline 8 Aug 2013, 23:29:38 UTC
Received 28 May 2013, 17:55:31 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1179592
Run time 18 days 22 hours 16 min 58 sec
CPU time 16 days 12 hours 58 min
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 1.77 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
20:08:12 (26932): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:39:57 (29801): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:13:31 (33136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:14:01 (33136): No heartbeat from core client for 30 sec - exiting
22:14:02 (33136): No heartbeat from core client for 30 sec - exiting
22:14:03 (33136): No heartbeat from core client for 30 sec - exiting
22:14:04 (33136): No heartbeat from core client for 30 sec - exiting
22:14:05 (33136): No heartbeat from core client for 30 sec - exiting
22:14:06 (33136): No heartbeat from core client for 30 sec - exiting
22:14:07 (33136): No heartbeat from core client for 30 sec - exiting
23:00:23 (38048): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:57:37 (38301): No heartbeat from core client for 30 sec - exiting
00:57:38 (38301): No heartbeat from core client for 30 sec - exiting
00:57:39 (38301): No heartbeat from core client for 30 sec - exiting
00:57:40 (38301): No heartbeat from core client for 30 sec - exiting
00:57:41 (38301): No heartbeat from core client for 30 sec - exiting
00:57:42 (38301): No heartbeat from core client for 30 sec - exiting
00:57:43 (38301): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:02:27 (38655): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:03:14 (38910): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:03:27 (39160): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:06:18 (39519): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:27:05 (39775): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:10:56 (40187): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:21:29 (40416): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:11:44 (40628): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:47 (40866): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:51:46 (41260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:16:39 (41730): No heartbeat from core client for 30 sec - exiting
18:16:40 (41730): No heartbeat from core client for 30 sec - exiting
18:16:41 (41730): No heartbeat from core client for 30 sec - exiting
18:16:42 (41730): No heartbeat from core client for 30 sec - exiting
18:16:43 (41730): No heartbeat from core client for 30 sec - exiting
18:16:44 (41730): No heartbeat from core client for 30 sec - exiting
18:16:45 (41730): No heartbeat from core client for 30 sec - exiting
18:16:46 (41730): No heartbeat from core client for 30 sec - exiting
18:16:47 (41730): No heartbeat from core client for 30 sec - exiting
18:16:48 (41730): No heartbeat from core client for 30 sec - exiting
18:16:49 (41730): No heartbeat from core client for 30 sec - exiting
18:16:50 (41730): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/498oko.pjj1c10 is not a valid UM file.
Error converting file to netcdf: dataout/498oko.pjj1c10
Error: Input file: dataout/498oko.pij1c10 is not a valid UM file.
Error converting file to netcdf: dataout/498oko.pij1c10
Error: Input file: dataout/498oko.pfj1c10 is not a valid UM file.
Error converting file to netcdf: dataout/498oko.pfj1c10
Error: Input file: dataout/498oka.phj1c10 is not a valid UM file.
Error converting file to netcdf: dataout/498oka.phj1c10
Error: Input file: dataout/498oka.pgj1c10 is not a valid UM file.
Error converting file to netcdf: dataout/498oka.pgj1c10
Error: Input file: dataout/498oka.pej1c10 is not a valid UM file.
Error converting file to netcdf: dataout/498oka.pej1c10
Error: Input file: dataout/498oka.pdj1c10 is not a valid UM file.
Error converting file to netcdf: dataout/498oka.pdj1c10
20:18:34 (41960): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:54 (42318): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:18:20 (42589): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:25:47 (43192): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:13:26 (43395): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:26:29 (43629): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:52:26 (43841): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:28:05 (44058): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:39:26 (44290): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:47:46 (44723): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:30:21 (44927): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:31:00 (45164): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:31:41 (45416): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:06:44 (45682): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:33:05 (45996): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:33:46 (46229): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:39:16 (46497): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:43:48 (46704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:16:29 (47744): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:26:21 (2694): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:16:55 (3418): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:42:38 (6332): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:52:44 (6657): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:47:30 (6925): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:52:38 (7290): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:47:51 (7496): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0926f9e0 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6bff1)[0xf74daff1]
/lib32/libc.so.6(+0x6d880)[0xf74dc880]
/lib32/libc.so.6(cfree+0x6d)[0xf74df92d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76b67b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76b680d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf7485ce7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:02 3514599                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:02 3514599                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
0921a000-0927f000 rw-p 00000000 00:00 0                                  [heap]
f6e00000-f6e21000 rw-p 00000000 00:00 0 
f6e21000-f6f00000 ---p 00000000 00:00 0 
f6ff1000-f746c000 rw-s 00000000 08:02 3563612                            /var/lib/boinc-client/slots/1/133020
f746c000-f746f000 rw-p 00000000 00:00 0 
f746f000-f75c3000 r-xp 00000000 08:02 9879703                            /lib32/libc-2.12.1.so
f75c3000-f75c4000 ---p 00154000 08:02 9879703                            /lib32/libc-2.12.1.so
f75c4000-f75c6000 r--p 00154000 08:02 9879703                            /lib32/libc-2.12.1.so
f75c6000-f75c7000 rw-p 00156000 08:02 9879703                            /lib32/libc-2.12.1.so
f75c7000-f75ca000 rw-p 00000000 00:00 0 
f75ca000-f75e4000 r-xp 00000000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f75e4000-f75e5000 r--p 00019000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f75e5000-f75e6000 rw-p 0001a000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f75e6000-f760a000 r-xp 00000000 08:02 9879707                            /lib32/libm-2.12.1.so
f760a000-f760b000 r--p 00023000 08:02 9879707                            /lib32/libm-2.12.1.so
f760b000-f760c000 rw-p 00024000 08:02 9879707                            /lib32/libm-2.12.1.so
f760c000-f76eb000 r-xp 00000000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f76eb000-f76ef000 r--p 000de000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f76ef000-f76f0000 rw-p 000e2000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f76f0000-f76f7000 rw-p 00000000 00:00 0 
f76f7000-f76f9000 r-xp 00000000 08:02 9879706                            /lib32/libdl-2.12.1.so
f76f9000-f76fa000 r--p 00001000 08:02 9879706                            /lib32/libdl-2.12.1.so
f76fa000-f76fb000 rw-p 00002000 08:02 9879706                            /lib32/libdl-2.12.1.so
f76fb000-f76fc000 rw-p 00000000 00:00 0 
f76fc000-f7711000 r-xp 00000000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7711000-f7712000 r--p 00014000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7712000-f7713000 rw-p 00015000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7713000-f7715000 rw-p 00000000 00:00 0 
f7726000-f7727000 rw-p 00000000 00:00 0 
f7727000-f7728000 ---p 00000000 00:00 0 
f7728000-f772b000 rw-p 00000000 00:00 0 
f772b000-f772d000 rw-s 00000000 08:02 3563542                            /var/lib/boinc-client/slots/1/boinc_mmap_file
f772d000-f772f000 rw-p 00000000 00:00 0 
f772f000-f7730000 r-xp 00000000 00:00 0                                  [vdso]
f7730000-f774c000 r-xp 00000000 08:02 9879700                            /lib32/ld-2.12.1.so
f774c000-f774d000 r--p 0001b000 08:02 9879700                            /lib32/ld-2.12.1.so
f774d000-f774e000 rw-p 0001c000 08:02 9879700                            /lib32/ld-2.12.1.so
ffcb5000-ffd26000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf772f400]
[0xf772f425]
/lib32/libc.so.6(gsignal+0x51)[0xf7499a01]
/lib32/libc.so.6(abort+0x182)[0xf749ce42]
/lib32/libc.so.6(+0x61f15)[0xf74d0f15]
/lib32/libc.so.6(+0x6bff1)[0xf74daff1]
/lib32/libc.so.6(+0x6d880)[0xf74dc880]
/lib32/libc.so.6(cfree+0x6d)[0xf74df92d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76b67b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76b680d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf7485ce7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
28 May 2013 17:57:35 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 518,400 1,428,986 2.7565
27 May 2013 22:34:37 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 492,480 1,358,901 2.7593
27 May 2013 03:01:56 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 466,560 1,290,735 2.7665
26 May 2013 07:44:46 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 440,640 1,222,204 2.7737
25 May 2013 12:48:28 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 414,720 1,152,449 2.7789
24 May 2013 17:30:10 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 388,800 1,082,897 2.7852
23 May 2013 15:05:19 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 362,880 991,462 2.7322
22 May 2013 14:50:36 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 336,960 902,036 2.6770
21 May 2013 16:31:54 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 311,040 822,279 2.6436
20 May 2013 18:16:09 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 285,120 745,312 2.6140
19 May 2013 23:57:49 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 259,200 684,827 2.6421
19 May 2013 03:05:10 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 233,280 608,746 2.6095
18 May 2013 06:37:50 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 207,360 537,261 2.5910
17 May 2013 10:51:10 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 181,440 466,079 2.5688
16 May 2013 15:34:59 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 155,520 397,317 2.5548
15 May 2013 21:57:45 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 129,600 333,836 2.5759
15 May 2013 03:58:52 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 103,680 270,996 2.6138
14 May 2013 11:14:36 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 77,760 207,366 2.6667
13 May 2013 16:03:10 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 51,840 141,283 2.7254
12 May 2013 20:01:36 1179592 15768940 hadcm3n_498o_1980_40_008361729_0 25,920 68,123 2.6282


©2024 climateprediction.net