Task 17229283

Name	hadcm3n_s0dx_1940_40_009093662_0
Workunit	9223998
Created	20 Oct 2014, 16:43:37 UTC
Sent	20 Oct 2014, 18:58:27 UTC
Report deadline	20 Jan 2015, 2:25:38 UTC
Received	22 Dec 2014, 2:22:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1286624
Run time	2 days 13 hours 8 min 21 sec
CPU time	2 days 9 hours 53 min 18 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	4.70 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.4.23</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 20:45:26 (14057): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:43:06 (10623): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:41:05 (6028): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf776ed30] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf776ed60] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x47)[0xf7554307] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x143)[0xf75559c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf3)[0xf753fa63] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf773cd30] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf773cd60] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x47)[0xf7522307] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x143)[0xf75239c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf3)[0xf750da63] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7701d30] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7701d60] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x47)[0xf74e7307] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x143)[0xf74e89c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf3)[0xf74d2a63] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7771d30] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7771d60] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x47)[0xf7557307] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x143)[0xf75589c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf3)[0xf7542a63] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf775bd30] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf775bd60] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x47)[0xf7541307] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x143)[0xf75429c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf3)[0xf752ca63] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7706d30] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7706d60] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x47)[0xf74ec307] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x143)[0xf74ed9c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf3)[0xf74d7a63] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Dec 2014 20:35:27	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	233,280	195,640	0.8386
19 Dec 2014 09:25:51	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	207,360	173,811	0.8382
17 Dec 2014 11:52:19	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	181,440	151,552	0.8353
15 Dec 2014 22:06:33	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	155,520	129,117	0.8302
14 Dec 2014 11:35:11	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	129,600	107,146	0.8267
09 Dec 2014 19:01:29	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	103,680	85,974	0.8292
09 Dec 2014 19:01:29	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	77,760	65,933	0.8479
09 Dec 2014 05:36:57	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	51,840	44,276	0.8541
08 Dec 2014 23:49:22	1286624	17229283	hadcm3n_s0dx_1940_40_009093662_0	25,920	22,465	0.8667