Task 12571489

Name	famous_v4o8_1199_200_006732636_3
Workunit	6935977
Created	10 Feb 2011, 14:40:51 UTC
Sent	10 Feb 2011, 14:41:01 UTC
Report deadline	12 May 2011, 22:08:12 UTC
Received	8 Mar 2011, 13:00:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1122015
Run time	5 days 10 hours 53 min 33 sec
CPU time	4 days 2 hours 42 min 56 sec
Validate state	Invalid
Credit	2,439.74
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (2070): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xf7750400] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f906] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805085a] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7492bd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (3056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 SIGSEGV: segmentation violation Stack trace (12 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf77d9400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x80db371] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x812c1e9] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x80de1cf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x807b44a] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7630bd6] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13372, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf77ad400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x816c5b5] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81756ab] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8325c74] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7604bd6] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13416, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: severe (24): end-of-file during read, unit 22, file /var/lib/boinc-client/projects/climateprediction.net/famous_v4o8_1199_200_006732636/datain/ancil/ctldata/STASHmaster/STASHmaster_A Image PC Routine Line Source famous_um_6.11_i6 0840BA88 Unknown Unknown Unknown famous_um_6.11_i6 0840A221 Unknown Unknown Unknown famous_um_6.11_i6 083ED787 Unknown Unknown Unknown famous_um_6.11_i6 083BA000 Unknown Unknown Unknown famous_um_6.11_i6 083CC8A5 Unknown Unknown Unknown famous_um_6.11_i6 0813BBA2 Unknown Unknown Unknown famous_um_6.11_i6 08325740 Unknown Unknown Unknown famous_um_6.11_i6 08330BDF Unknown Unknown Unknown famous_um_6.11_i6 083415D2 Unknown Unknown Unknown libc.so.6 F757DBD6 Unknown Unknown Unknown famous_um_6.11_i6 0804C041 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13072, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (31942): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (17 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xf7733400] /lib32/libpthread.so.0(pthread_once+0x4)[0xf7701784] /usr/lib32/libstdc++.so.6(+0x5a104)[0xf7655104] /usr/lib32/libstdc++.so.6(_ZNSt6localeC1Ev+0x28)[0xf76551a8] /usr/lib32/libstdc++.so.6(_ZNSt8ios_baseC2Ev+0x8c)[0xf764ff2c] /var/lib/boinc-client/projects/climateprediction.net/famous_se_6.11_i686-pc-linux-gnu.so(_Z11ConvertFileSsSsSsSsi+0x25)[0xf7353ef9] /var/lib/boinc-client/projects/climateprediction.net/famous_se_6.11_i686-pc-linux-gnu.so(umtonc+0x105)[0xf7359525] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805b0a8] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8053f4d] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8057641] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f878] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f9e9] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805085a] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7472bd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Mar 2011 13:01:46	1122015	12571489	famous_v4o8_1199_200_006732636_3	739,466	351,044	0.4747
08 Mar 2011 13:01:46	1122015	12571489	famous_v4o8_1199_200_006732636_3	730,106	346,692	0.4749
28 Feb 2011 06:53:27	1122015	12571489	famous_v4o8_1199_200_006732636_3	720,746	342,325	0.4750
28 Feb 2011 03:42:20	1122015	12571489	famous_v4o8_1199_200_006732636_3	711,386	337,978	0.4751
27 Feb 2011 21:39:36	1122015	12571489	famous_v4o8_1199_200_006732636_3	702,026	333,633	0.4752
27 Feb 2011 15:08:28	1122015	12571489	famous_v4o8_1199_200_006732636_3	692,666	329,286	0.4754
27 Feb 2011 15:08:28	1122015	12571489	famous_v4o8_1199_200_006732636_3	683,306	324,943	0.4755
26 Feb 2011 23:08:13	1122015	12571489	famous_v4o8_1199_200_006732636_3	673,946	320,596	0.4757
26 Feb 2011 19:06:44	1122015	12571489	famous_v4o8_1199_200_006732636_3	664,586	316,182	0.4758
26 Feb 2011 15:01:30	1122015	12571489	famous_v4o8_1199_200_006732636_3	655,226	311,716	0.4757
26 Feb 2011 12:13:50	1122015	12571489	famous_v4o8_1199_200_006732636_3	645,866	307,307	0.4758
26 Feb 2011 07:23:25	1122015	12571489	famous_v4o8_1199_200_006732636_3	636,506	302,899	0.4759
26 Feb 2011 04:22:36	1122015	12571489	famous_v4o8_1199_200_006732636_3	627,146	298,495	0.4760
24 Feb 2011 15:17:27	1122015	12571489	famous_v4o8_1199_200_006732636_3	617,786	294,103	0.4761
24 Feb 2011 08:30:21	1122015	12571489	famous_v4o8_1199_200_006732636_3	608,426	289,738	0.4762
22 Feb 2011 21:47:25	1122015	12571489	famous_v4o8_1199_200_006732636_3	599,066	285,376	0.4764
22 Feb 2011 16:25:30	1122015	12571489	famous_v4o8_1199_200_006732636_3	589,706	281,034	0.4766
22 Feb 2011 13:36:29	1122015	12571489	famous_v4o8_1199_200_006732636_3	580,346	276,695	0.4768
22 Feb 2011 07:51:52	1122015	12571489	famous_v4o8_1199_200_006732636_3	570,986	272,335	0.4770
22 Feb 2011 01:55:03	1122015	12571489	famous_v4o8_1199_200_006732636_3	561,626	267,950	0.4771