Task 21287965

Name	hadcm3s_n113_190012_120_749_011622224_2
Workunit	11622224
Created	7 Sep 2018, 20:35:57 UTC
Sent	7 Sep 2018, 20:36:31 UTC
Report deadline	21 Aug 2019, 1:56:31 UTC
Received	21 Sep 2018, 21:14:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1461809
Run time	2 days 12 hours 26 min
CPU time	2 days 6 hours 48 min 23 sec
Validate state	Invalid
Credit	3,111.26
Device peak FLOPS	3.79 GFLOPS
Application version	UK Met Office HadCM3 short v8.34 i686-pc-linux-gnu
Peak working set size	166.11 MB
Peak swap size	214.17 MB
Peak disk usage	113.21 MB
Stderr	<core_client_version>7.6.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 17:27:54 (2233): called boinc_finish(193) In boinc_exit called with status 193 Calloing set_signal_exit_code with status 193 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bcc77] [0xf7744bc0] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf738418a] /lib/i386-linux-gnu/libc.so.6(+0xa232b)[0xf73f832b] /lib/i386-linux-gnu/libc.so.6(+0xa269a)[0xf73f869a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1e)[0xf73f6d2e] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b8132] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b88c0] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b89b1] [0xf7744bc0] [0xf7744be9] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4a)[0xf74063ca] /lib/i386-linux-gnu/libc.so.6(usleep+0x35)[0xf74362c5] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80cf745] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804f748] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804fe7b] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80515db] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80517db] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xf736e637] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804c771] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bcc77] [0xf7779bc0] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf73b918a] /lib/i386-linux-gnu/libc.so.6(+0xa232b)[0xf742d32b] /lib/i386-linux-gnu/libc.so.6(+0xa269a)[0xf742d69a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1e)[0xf742bd2e] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b8132] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b88c0] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b89b1] [0xf7779bc0] [0xf7779be9] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4a)[0xf743b3ca] /lib/i386-linux-gnu/libc.so.6(usleep+0x35)[0xf746b2c5] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80cf745] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804f748] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804fe7b] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80515db] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80517db] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xf73a3637] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804c771] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 08:59:45 (6414): called boinc_finish(193) In boinc_exit called with status 193 Calloing set_signal_exit_code with status 193 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bcc77] [0xf7765bc0] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf73a518a] /lib/i386-linux-gnu/libc.so.6(+0xa232b)[0xf741932b] /lib/i386-linux-gnu/libc.so.6(+0xa269a)[0xf741969a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1e)[0xf7417d2e] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b8132] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b88c0] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b89b1] [0xf7765bc0] [0xf7765be9] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4a)[0xf74273ca] /lib/i386-linux-gnu/libc.so.6(usleep+0x35)[0xf74572c5] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80cf745] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804f748] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804fe7b] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80515db] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80517db] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xf738f637] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804c771] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 17:34:23 (12781): called boinc_finish(193) In boinc_exit called with status 193 Calloing set_signal_exit_code with status 193 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bcc77] [0xf77c0bc0] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf740018a] /lib/i386-linux-gnu/libc.so.6(+0xa232b)[0xf747432b] /lib/i386-linux-gnu/libc.so.6(+0xa269a)[0xf747469a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1e)[0xf7472d2e] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b8132] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b88c0] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80b89b1] [0xf77c0bc0] [0xf77c0be9] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4a)[0xf74823ca] /lib/i386-linux-gnu/libc.so.6(usleep+0x35)[0xf74b22c5] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80cf745] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804f748] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804fe7b] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80515db] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x80517db] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xf73ea637] ../../projects/climateprediction.net/hadcm3s_8.34_i686-pc-linux-gnu[0x804c771] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 08:51:31 (9556): called boinc_finish(193) In boinc_exit called with status 193 Calloing set_signal_exit_code with status 193 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 08:57:46 (2253): called boinc_finish(193) In boinc_exit called with status 193 Calloing set_signal_exit_code with status 193 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( Calling boinc_finish...15:13:50 (11134): called boinc_finish(22) In boinc_exit called with status 22 Calloing set_signal_exit_code with status 22 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Sep 2018 17:20:04	1461809	21287965	hadcm3s_n113_190012_120_749_011622224_2	259,272	40,623	0.1567