Task 16075419

Name	hadcm3n_4kgi_1980_40_008326037_4
Workunit	8477172
Created	29 Oct 2013, 8:59:55 UTC
Sent	29 Oct 2013, 9:00:01 UTC
Report deadline	28 Jan 2014, 16:27:12 UTC
Received	5 Nov 2013, 18:24:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	910708
Run time	7 days 4 hours 25 min 31 sec
CPU time	7 days 1 hours 41 min 41 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	3.26 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:21:06 (6943): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:21:09 (6943): No heartbeat from core client for 30 sec - exiting 10:21:10 (6943): No heartbeat from core client for 30 sec - exiting 10:21:11 (6943): No heartbeat from core client for 30 sec - exiting 10:21:12 (6943): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (10 frames): /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xe64400] [0xe64430] /lib/libc.so.6(gsignal+0x51)[0x9e4ad1] /lib/libc.so.6(abort+0x17a)[0x9e63aa] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0x9d0ce6] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6991, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf84400] [0xf84430] /lib/libc.so.6(gsignal+0x51)[0x9e4ad1] /lib/libc.so.6(abort+0x17a)[0x9e63aa] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0x9d0ce6] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6991, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0x178400] [0x178430] /lib/libc.so.6(gsignal+0x51)[0x9e4ad1] /lib/libc.so.6(abort+0x17a)[0x9e63aa] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0x9d0ce6] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6991, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0x6bf400] [0x6bf430] /lib/libc.so.6(gsignal+0x51)[0x9e4ad1] /lib/libc.so.6(abort+0x17a)[0x9e63aa] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0x9d0ce6] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6991, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xc13400] [0xc13430] /lib/libc.so.6(gsignal+0x51)[0x9e4ad1] /lib/libc.so.6(abort+0x17a)[0x9e63aa] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0x9d0ce6] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6991, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0x646400] [0x646430] /lib/libc.so.6(gsignal+0x51)[0x9e4ad1] /lib/libc.so.6(abort+0x17a)[0x9e63aa] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0x9d0ce6] /root/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6991, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2013 09:58:22	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	544,320	582,959	1.0710
05 Nov 2013 02:00:51	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	518,400	554,793	1.0702
04 Nov 2013 15:47:53	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	492,480	526,964	1.0700
04 Nov 2013 07:43:57	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	466,560	499,197	1.0700
03 Nov 2013 23:51:06	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	440,640	471,188	1.0693
03 Nov 2013 15:59:14	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	414,720	443,362	1.0691
03 Nov 2013 08:02:51	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	388,800	415,615	1.0690
02 Nov 2013 23:39:53	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	362,880	387,812	1.0687
02 Nov 2013 15:54:28	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	336,960	360,052	1.0685
02 Nov 2013 08:02:58	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	311,040	332,329	1.0684
01 Nov 2013 23:49:53	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	285,120	304,363	1.0675
01 Nov 2013 15:57:53	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	259,200	276,560	1.0670
01 Nov 2013 08:07:08	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	233,280	248,805	1.0666
01 Nov 2013 00:19:10	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	207,360	221,163	1.0666
31 Oct 2013 15:54:56	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	181,440	192,736	1.0623
31 Oct 2013 07:48:58	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	155,520	164,267	1.0562
30 Oct 2013 23:39:58	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	129,600	135,345	1.0443
30 Oct 2013 15:36:17	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	103,680	107,038	1.0324
30 Oct 2013 07:57:27	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	77,760	80,295	1.0326
30 Oct 2013 00:24:09	910708	16075419	hadcm3n_4kgi_1980_40_008326037_4	51,840	53,524	1.0325