Task 22134099

Name	hadam4_a15d_201310_6_915_012101949_4
Workunit	12101949
Created	6 Aug 2021, 18:47:44 UTC
Sent	6 Aug 2021, 20:00:12 UTC
Report deadline	20 Jul 2022, 1:20:12 UTC
Received	19 Aug 2021, 16:53:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1514328
Run time	2 days 2 hours 39 min 54 sec
CPU time	1 days 22 hours 6 min 9 sec
Validate state	Invalid
Credit	4,827.12
Device peak FLOPS	5.46 GFLOPS
Application version	UK Met Office HadAM4 at N144 resolution v8.09 i686-pc-linux-gnu
Peak working set size	650.78 MB
Peak swap size	925.42 MB
Peak disk usage	5.79 MB
Stderr	<core_client_version>7.16.6</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234)</message> <stderr_txt> Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 15:18:23 (7124): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f22b60] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a0e552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7a92d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7a930ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7a9103b] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f22b60] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f22b49] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7a9d693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7a9d892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7aa4314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7ad8475] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf79f5ee5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 00:00:20 (3918): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f3bb60] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a27552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7aabd2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7aac0ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7aaa03b] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f3bb60] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f3bb49] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7ab6693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7ab6892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7abd314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7af1475] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a0eee5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:18:16 (3861): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 18:09:44 (3755): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f3ab60] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a26552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7aaad2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7aab0ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7aa903b] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f3ab60] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f3ab49] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7ab5693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7ab5892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7abc314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7af0475] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a0dee5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:19:59 (3783): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fbfb60] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7aab552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7b2fd2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7b300ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7b2e03b] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fbfb60] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fbfb49] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b3a693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b3a892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b41314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b75475] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a92ee5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : NEGATIVE THETA DETECTED. tmp/xnnuj.pipe_dummy Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:56:21 (90733): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:56:21 (90733): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/xnnuj.pipe_dummy Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/xnnuj.pipe_dummy Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/xnnuj.pipe_dummy Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/xnnuj.pipe_dummy Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/xnnuj.pipe_dummy Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/xnnuj.pipe_dummy Sorry, too many model crashes! :-( 19:41:57 (149895): called boinc_finish(22) </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Aug 2021 16:56:23	1514328	22134099	hadam4_a15d_201310_6_915_012101949_4	13,061	158,552	12.1393
19 Aug 2021 16:56:23	1514328	22134099	hadam4_a15d_201310_6_915_012101949_4	8,741	110,705	12.6650
12 Aug 2021 15:01:04	1514328	22134099	hadam4_a15d_201310_6_915_012101949_4	4,421	55,675	12.5933