Task 22127903

Name	hadam4_a1n2_201310_6_914_012099691_1
Workunit	12099691
Created	25 Jul 2021, 15:47:17 UTC
Sent	29 Aug 2021, 11:33:07 UTC
Report deadline	11 Aug 2022, 16:53:07 UTC
Received	21 Sep 2021, 8:08:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1521373
Run time	5 days 3 hours 29 min 3 sec
CPU time	5 days 2 hours 37 min 30 sec
Validate state	Invalid
Credit	3,230.53
Device peak FLOPS	4.54 GFLOPS
Application version	UK Met Office HadAM4 at N144 resolution v8.09 i686-pc-linux-gnu
Peak working set size	650.64 MB
Peak swap size	674.43 MB
Peak disk usage	5.79 MB
Stderr	<core_client_version>7.16.6</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234)</message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:02:38 (111106): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:02:39 (111106): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:03:13 (2410): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb6560] /lib32/libc.so.6(getenv+0xa2)[0xf7abb372] /lib32/libc.so.6(+0xbae9d)[0xf7b3ee9d] /lib32/libc.so.6(+0xbb25e)[0xf7b3f25e] /lib32/libc.so.6(localtime_r+0x1b)[0xf7b3d1ab] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb6560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fb6549] /lib32/libc.so.6(+0xc57c3)[0xf7b497c3] /lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b499b2] /lib32/libc.so.6(nanosleep+0x24)[0xf7b500c4] /lib32/libc.so.6(usleep+0x45)[0xf7b839e5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051d8b] /lib32/libc.so.6(__libc_start_main+0xf5)[0xf7aa2ee5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7eec560] /lib32/libc.so.6(getenv+0xa2)[0xf79f1372] /lib32/libc.so.6(+0xbae9d)[0xf7a74e9d] /lib32/libc.so.6(+0xbb25e)[0xf7a7525e] /lib32/libc.so.6(localtime_r+0x1b)[0xf7a731ab] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7eec560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7eec549] /lib32/libc.so.6(+0xc57c3)[0xf7a7f7c3] /lib32/libc.so.6(clock_nanosleep+0x52)[0xf7a7f9b2] /lib32/libc.so.6(nanosleep+0x24)[0xf7a860c4] /lib32/libc.so.6(usleep+0x45)[0xf7ab99e5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051d8b] /lib32/libc.so.6(__libc_start_main+0xf5)[0xf79d8ee5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:47:23 (2005): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:47:24 (2005): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 12:58:54 (2437): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f02560] /lib32/libc.so.6(getenv+0xa2)[0xf7a07372] /lib32/libc.so.6(+0xbae9d)[0xf7a8ae9d] /lib32/libc.so.6(+0xbb25e)[0xf7a8b25e] /lib32/libc.so.6(localtime_r+0x1b)[0xf7a891ab] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f02560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f02549] /lib32/libc.so.6(+0xc57c3)[0xf7a957c3] /lib32/libc.so.6(clock_nanosleep+0x52)[0xf7a959b2] /lib32/libc.so.6(nanosleep+0x24)[0xf7a9c0c4] /lib32/libc.so.6(usleep+0x45)[0xf7acf9e5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x8051d8b] /lib32/libc.so.6(__libc_start_main+0xf5)[0xf79eeee5] ../../projects/climateprediction.net/hadam4_8.09_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:29:44 (2018): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:29:45 (2018): called boinc_finish(193) Model crashed: ATM_DYN : NEGATIVE THETA DETECTED. tmp/xnnuj.pipe_dummy Model crashed: ATM_DYN : NEGATIVE THETA DETECTED. tmp/xnnuj.pipe_dummy Model crashed: ATM_DYN : NEGATIVE THETA DETECTED. tmp/xnnuj.pipe_dummy Model crashed: ATM_DYN : NEGATIVE THETA DETECTED. tmp/xnnuj.pipe_dummy Model crashed: ATM_DYN : NEGATIVE THETA DETECTED. tmp/xnnuj.pipe_dummy Model crashed: ATM_DYN : NEGATIVE THETA DETECTED. tmp/xnnuj.pipe_dummy Sorry, too many model crashes! :-( 01:06:57 (2286): called boinc_finish(22) </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Sep 2021 03:07:35	1521373	22127903	hadam4_a1n2_201310_6_914_012099691_1	8,741	287,981	32.9460
16 Sep 2021 05:57:30	1521373	22127903	hadam4_a1n2_201310_6_914_012099691_1	4,421	147,728	33.4151