Task 21744123

Name	hadcm3s_hf53_190012_240_835_011892415_0
Workunit	11892415
Created	25 Sep 2019, 10:40:00 UTC
Sent	26 Sep 2019, 6:41:06 UTC
Report deadline	7 Sep 2020, 12:01:06 UTC
Received	8 Oct 2019, 3:38:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	14 (0x0000000E) Unknown error code
Computer ID	1455853
Run time	10 days 6 hours 18 min 55 sec
CPU time	8 days 19 hours 44 min 47 sec
Validate state	Invalid
Credit	4,665.60
Device peak FLOPS	4.83 GFLOPS
Application version	UK Met Office HadCM3 short v8.36 i686-pc-linux-gnu
Peak working set size	166.39 MB
Peak swap size	214.91 MB
Peak disk usage	99.17 MB
Stderr	<core_client_version>7.14.2</core_client_version> <![CDATA[ <message> process exited with code 14 (0xe, -242)</message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... MainError: 12:24:01 AM No files match the supplied pattern. MainError: 12:24:01 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... MainError: 03:46:36 PM No files match the supplied pattern. MainError: 03:46:36 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 10:35:35 AM No files match the supplied pattern. MainError: 10:35:35 AM No files match the supplied pattern. MainError: 12:16:27 AM No files match the supplied pattern. MainError: 12:16:27 AM No files match the supplied pattern. MainError: 01:39:23 PM No files match the supplied pattern. MainError: 01:39:23 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 05:09:43 AM No files match the supplied pattern. MainError: 05:09:43 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 02:57:42 AM No files match the supplied pattern. MainError: 02:57:42 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 09:01:21 PM No files match the supplied pattern. MainError: 09:01:21 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 04:58:36 PM No files match the supplied pattern. MainError: 04:58:36 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bc177] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f92950] /lib/i386-linux-gnu/libc.so.6(getenv+0x99)[0xf7a830c9] /lib/i386-linux-gnu/libc.so.6(+0xb538d)[0xf7b0338d] /lib/i386-linux-gnu/libc.so.6(+0xb571b)[0xf7b0371b] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7b01857] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7632] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7d80] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7e71] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f92950] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f92939] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4b)[0xf7b1361b] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7b46cd1] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80cead5] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804fe8b] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80515bd] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80517bd] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0xf7a6c751] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804c7a1] Exiting... MainError: 01:43:32 PM No files match the supplied pattern. MainError: 01:43:32 PM No files match the supplied pattern. MainError: 06:58:40 AM No files match the supplied pattern. MainError: 06:58:40 AM No files match the supplied pattern. MainError: 12:36:48 AM No files match the supplied pattern. MainError: 12:36:48 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 09:27:16 PM No files match the supplied pattern. MainError: 09:27:16 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... MainError: 05:48:17 PM No files match the supplied pattern. MainError: 05:48:17 PM No files match the supplied pattern. MainError: 10:44:33 AM No files match the supplied pattern. MainError: 10:44:33 AM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 forrtl: No space left on device forrtl: severe (38): error during write, unit -1, file /proc/27346/fd/1 Image PC Routine Line Source hadcm3s_um_8.36_i 0851D9E5 Unknown Unknown Unknown hadcm3s_um_8.36_i 0856077D Unknown Unknown Unknown hadcm3s_um_8.36_i 0855C53F Unknown Unknown Unknown hadcm3s_um_8.36_i 083E3EA1 Unknown Unknown Unknown hadcm3s_um_8.36_i 0848B335 Unknown Unknown Unknown hadcm3s_um_8.36_i 08496BAD Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27330, iMonCtr=1 Model crash detected, will try to restart... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 forrtl: No space left on device forrtl: severe (38): error during write, unit -1, file /proc/9057/fd/1 Image PC Routine Line Source hadcm3s_um_8.36_i 0851D9E5 Unknown Unknown Unknown hadcm3s_um_8.36_i 0856077D Unknown Unknown Unknown hadcm3s_um_8.36_i 0855C53F Unknown Unknown Unknown hadcm3s_um_8.36_i 083E3EA1 Unknown Unknown Unknown hadcm3s_um_8.36_i 0848B335 Unknown Unknown Unknown hadcm3s_um_8.36_i 08496BAD Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27330, iMonCtr=1 Model crash detected, will try to restart... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 forrtl: No space left on device forrtl: severe (38): error during write, unit -1, file /proc/11450/fd/1 Image PC Routine Line Source hadcm3s_um_8.36_i 0851D9E5 Unknown Unknown Unknown hadcm3s_um_8.36_i 0856077D Unknown Unknown Unknown hadcm3s_um_8.36_i 0855C53F Unknown Unknown Unknown hadcm3s_um_8.36_i 083E3EA1 Unknown Unknown Unknown hadcm3s_um_8.36_i 0848B335 Unknown Unknown Unknown hadcm3s_um_8.36_i 08496BAD Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27330, iMonCtr=1 Model crash detected, will try to restart... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 forrtl: N </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Oct 2019 11:18:31	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	388,800	726,847	1.8695
06 Oct 2019 18:21:01	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	362,880	671,639	1.8509
05 Oct 2019 22:15:17	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	336,960	614,459	1.8235
05 Oct 2019 01:23:27	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	311,040	559,136	1.7976
04 Oct 2019 07:51:30	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	285,120	505,089	1.7715
03 Oct 2019 14:42:57	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	259,200	451,653	1.7425
02 Oct 2019 17:03:21	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	233,280	394,858	1.6926
01 Oct 2019 21:15:58	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	207,360	343,846	1.6582
01 Oct 2019 10:44:52	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	181,440	292,951	1.6146
01 Oct 2019 10:44:08	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	155,520	247,450	1.5911
01 Oct 2019 10:43:29	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	129,600	200,935	1.5504
01 Oct 2019 10:42:48	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	103,680	160,604	1.5490
01 Oct 2019 10:41:38	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	77,760	119,438	1.5360
27 Sep 2019 16:47:19	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	51,840	80,146	1.5460
27 Sep 2019 15:48:34	1455853	21744123	hadcm3s_hf53_190012_240_835_011892415_0	25,920	40,040	1.5448