Task 21744730

Name	hadcm3s_c555_191512_240_835_011893022_0
Workunit	11893022
Created	25 Sep 2019, 10:45:28 UTC
Sent	26 Sep 2019, 6:41:06 UTC
Report deadline	7 Sep 2020, 12:01:06 UTC
Received	8 Oct 2019, 3:38:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	14 (0x0000000E) Unknown error code
Computer ID	1455853
Run time	10 days 6 hours 21 min 48 sec
CPU time	8 days 19 hours 45 min 8 sec
Validate state	Invalid
Credit	4,665.60
Device peak FLOPS	4.83 GFLOPS
Application version	UK Met Office HadCM3 short v8.36 i686-pc-linux-gnu
Peak working set size	166.49 MB
Peak swap size	214.92 MB
Peak disk usage	99.14 MB
Stderr	<core_client_version>7.14.2</core_client_version> <![CDATA[ <message> process exited with code 14 (0xe, -242)</message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... MainError: 12:23:50 AM No files match the supplied pattern. MainError: 12:23:50 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... MainError: 03:44:03 PM No files match the supplied pattern. MainError: 03:44:03 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 10:32:14 AM No files match the supplied pattern. MainError: 10:32:14 AM No files match the supplied pattern. MainError: 12:00:51 AM No files match the supplied pattern. MainError: 12:00:51 AM No files match the supplied pattern. MainError: 01:11:27 PM No files match the supplied pattern. MainError: 01:11:27 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 04:28:29 AM No files match the supplied pattern. MainError: 04:28:29 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 02:14:17 AM No files match the supplied pattern. MainError: 02:14:17 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 08:12:33 PM No files match the supplied pattern. MainError: 08:12:33 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 04:04:26 PM No files match the supplied pattern. MainError: 04:04:26 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bc177] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fca950] /lib/i386-linux-gnu/libc.so.6(getenv+0x99)[0xf7abb0c9] /lib/i386-linux-gnu/libc.so.6(+0xb538d)[0xf7b3b38d] /lib/i386-linux-gnu/libc.so.6(+0xb571b)[0xf7b3b71b] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7b39857] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7632] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7d80] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7e71] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fca950] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fca939] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4b)[0xf7b4b61b] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7b7ecd1] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80cead5] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804fe8b] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80515bd] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80517bd] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0xf7aa4751] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804c7a1] Exiting... MainError: 12:41:00 AM No files match the supplied pattern. MainError: 12:41:00 AM No files match the supplied pattern. MainError: 05:49:43 AM No files match the supplied pattern. MainError: 05:49:43 AM No files match the supplied pattern. MainError: 11:23:33 PM No files match the supplied pattern. MainError: 11:23:33 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 08:14:12 PM No files match the supplied pattern. MainError: 08:14:12 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... MainError: 04:17:39 PM No files match the supplied pattern. MainError: 04:17:39 PM No files match the supplied pattern. MainError: 09:16:23 AM No files match the supplied pattern. MainError: 09:16:23 AM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 forrtl: No space left on device forrtl: severe (38): error during write, unit -1, file /proc/27342/fd/1 Image PC Routine Line Source hadcm3s_um_8.36_i 0851D9E5 Unknown Unknown Unknown hadcm3s_um_8.36_i 0856077D Unknown Unknown Unknown hadcm3s_um_8.36_i 0855C53F Unknown Unknown Unknown hadcm3s_um_8.36_i 083E3EA1 Unknown Unknown Unknown hadcm3s_um_8.36_i 0848B335 Unknown Unknown Unknown hadcm3s_um_8.36_i 08496BAD Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27329, iMonCtr=1 Model crash detected, will try to restart... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 forrtl: No space left on device forrtl: severe (38): error during write, unit -1, file /proc/9381/fd/1 Image PC Routine Line Source hadcm3s_um_8.36_i 0851D9E5 Unknown Unknown Unknown hadcm3s_um_8.36_i 0856077D Unknown Unknown Unknown hadcm3s_um_8.36_i 0855C53F Unknown Unknown Unknown hadcm3s_um_8.36_i 083E3EA1 Unknown Unknown Unknown hadcm3s_um_8.36_i 0848B335 Unknown Unknown Unknown hadcm3s_um_8.36_i 08496BAD Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27329, iMonCtr=1 Model crash detected, will try to restart... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 forrtl: No space left on device forrtl: severe (38): error during write, unit -1, file /proc/11796/fd/1 Image PC Routine Line Source hadcm3s_um_8.36_i 0851D9E5 Unknown Unknown Unknown hadcm3s_um_8.36_i 0856077D Unknown Unknown Unknown hadcm3s_um_8.36_i 0855C53F Unknown Unknown Unknown hadcm3s_um_8.36_i 083E3EA1 Unknown Unknown Unknown hadcm3s_um_8.36_i 0848B335 Unknown Unknown Unknown hadcm3s_um_8.36_i 08496BAD Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27329, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/b </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Oct 2019 10:15:31	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	388,800	722,128	1.8573
06 Oct 2019 17:18:51	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	362,880	666,910	1.8378
05 Oct 2019 21:13:06	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	336,960	610,361	1.8114
05 Oct 2019 00:26:21	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	311,040	555,334	1.7854
04 Oct 2019 06:49:29	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	285,120	501,521	1.7590
03 Oct 2019 13:41:04	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	259,200	448,408	1.7300
02 Oct 2019 17:03:21	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	233,280	392,046	1.6806
01 Oct 2019 21:15:58	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	207,360	341,005	1.6445
01 Oct 2019 10:44:50	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	181,440	290,526	1.6012
01 Oct 2019 10:44:05	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	155,520	245,078	1.5759
01 Oct 2019 10:43:20	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	129,600	199,190	1.5370
01 Oct 2019 10:42:44	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	103,680	159,514	1.5385
28 Sep 2019 10:32:38	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	77,760	118,954	1.5298
27 Sep 2019 15:48:34	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	51,840	79,811	1.5396
27 Sep 2019 00:29:15	1455853	21744730	hadcm3s_c555_191512_240_835_011893022_0	25,920	39,936	1.5407