Task 12834269

Name	hadcm3n_p7e9_1900_40_007227169_1
Workunit	7425409
Created	26 Apr 2011, 15:39:25 UTC
Sent	26 Apr 2011, 20:40:46 UTC
Report deadline	27 Jul 2011, 4:07:57 UTC
Received	6 May 2011, 9:13:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1069729
Run time	6 days 5 hours 25 min 10 sec
CPU time	6 days 2 hours 16 min 13 sec
Validate state	Invalid
Credit	3,421.44
Device peak FLOPS	2.71 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> 18:44:47 (31414): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... Called boinc_finish 09:32:42 (415): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 1 received, exiting... Called boinc_finish 21:47:08 (8959): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:31:02 (21644): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:20:48 (12454): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:21:42 (27636): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 20:50:02 (311): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:53:07 (24745): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Stack trace (10 frames): /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] [0xffffe430] /lib/libc.so.6(gsignal+0x51)[0xf7626ce1] /lib/libc.so.6(abort+0x182)[0xf7628632] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0xf7613b96] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] [0xffffe430] /lib/libc.so.6(gsignal+0x51)[0xf75eace1] /lib/libc.so.6(abort+0x182)[0xf75ec632] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0xf75d7b96] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] [0xffffe430] /lib/libc.so.6(gsignal+0x51)[0xf7586ce1] /lib/libc.so.6(abort+0x182)[0xf7588632] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0xf7573b96] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] [0xffffe430] /lib/libc.so.6(gsignal+0x51)[0xf75a5ce1] /lib/libc.so.6(abort+0x182)[0xf75a7632] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0xf7592b96] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] [0xffffe430] /lib/libc.so.6(gsignal+0x51)[0xf75b3ce1] /lib/libc.so.6(abort+0x182)[0xf75b5632] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0xf75a0b96] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] [0xffffe430] /lib/libc.so.6(gsignal+0x51)[0xf758ece1] /lib/libc.so.6(abort+0x182)[0xf7590632] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xe6)[0xf757bb96] /home/john/Downloads/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 May 2011 13:18:34	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	285,120	488,401	1.7130
05 May 2011 00:06:14	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	259,200	442,497	1.7072
04 May 2011 13:10:57	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	233,280	403,599	1.7301
04 May 2011 02:03:56	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	207,360	363,608	1.7535
03 May 2011 03:55:11	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	181,440	322,954	1.7799
30 Apr 2011 22:02:59	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	155,520	276,073	1.7752
30 Apr 2011 08:18:57	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	129,600	228,253	1.7612
29 Apr 2011 13:51:31	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	103,680	181,983	1.7552
29 Apr 2011 01:03:37	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	77,760	136,666	1.7575
27 Apr 2011 23:11:50	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	51,840	89,844	1.7331
27 Apr 2011 10:30:05	1069729	12834269	hadcm3n_p7e9_1900_40_007227169_1	25,920	45,010	1.7365