Task 11454774

Name	famous_u64t_1999_200_006640792_1
Workunit	6844164
Created	10 Jun 2010, 12:10:21 UTC
Sent	28 Jul 2010, 9:14:06 UTC
Report deadline	27 Oct 2010, 16:41:17 UTC
Received	18 Aug 2010, 14:46:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1043969
Run time	13 days 1 hours 59 min 36 sec
CPU time	13 days 16 hours 15 min 58 sec
Validate state	Invalid
Credit	2,254.45
Device peak FLOPS	0.40 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (6187): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (21034): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17931): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17949): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (18314): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (18679): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (19393): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb77de400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb7667b55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... CPDN Monitor - No 'heartbeat' from BOINC... (20801): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7828400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb76b1b55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... CPDN Monitor - No 'heartbeat' from BOINC... (21861): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25702): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (3005): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (5109): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (12076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16610): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (20796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (21511): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (22570): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23981): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25405): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (1303): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (4791): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (9330): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16622): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17683): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (22991): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23695): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (26847): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2393): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2755): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (3809): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (5210): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14251): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14963): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17746): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (18106): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb782e400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb76b7b55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... CPDN Monitor - No 'heartbeat' from BOINC... (21934): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27145): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (29932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (3759): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (7931): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (9329): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13502): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16289): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (21155): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (21861): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (26725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (519): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (882): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7751400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb75dab55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... CPDN Monitor - No 'heartbeat' from BOINC... (1950): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (3001): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (5789): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16909): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17270): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17978): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (19383): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (21577): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb772b400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb75b4b55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... CPDN Monitor - No 'heartbeat' from BOINC... (31666): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7761400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb75eab55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... CPDN Monitor - No 'heartbeat' from BOINC... MainError: 07:25:09 AM No files match the supplied pattern. MainError: 07:25:09 AM No files match the supplied pattern. SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb77cb400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb7654b55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1319, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7843400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb76ccb55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1319, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb771f400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb75a8b55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1319, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7747400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb75d0b55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1319, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb77d1400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb765ab55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1319, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7778400] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x834189c] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x830b5a9] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb7601b55] /var/lib/boinc-client/projects/climateapps2.oucs.ox.ac.uk_cpdnboinc/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1319, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( (1319): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	683,306	1,181,758	1.7295
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	673,946	1,162,330	1.7247
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	664,586	1,151,726	1.7330
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	655,226	1,141,111	1.7416
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	645,866	1,130,880	1.7510
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	636,506	1,120,296	1.7601
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	627,146	1,107,201	1.7655
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	617,786	1,094,266	1.7713
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	608,426	1,077,574	1.7711
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	599,066	1,060,590	1.7704
18 Aug 2010 14:50:27	1043969	11454774	famous_u64t_1999_200_006640792_1	589,706	1,042,480	1.7678
16 Aug 2010 05:32:14	1043969	11454774	famous_u64t_1999_200_006640792_1	580,346	1,024,194	1.7648
16 Aug 2010 00:30:17	1043969	11454774	famous_u64t_1999_200_006640792_1	570,986	1,006,110	1.7621
15 Aug 2010 19:33:41	1043969	11454774	famous_u64t_1999_200_006640792_1	561,626	988,360	1.7598
15 Aug 2010 14:26:00	1043969	11454774	famous_u64t_1999_200_006640792_1	552,266	970,125	1.7566
15 Aug 2010 09:29:05	1043969	11454774	famous_u64t_1999_200_006640792_1	542,906	952,079	1.7537
15 Aug 2010 04:13:55	1043969	11454774	famous_u64t_1999_200_006640792_1	533,546	934,579	1.7516
14 Aug 2010 19:56:40	1043969	11454774	famous_u64t_1999_200_006640792_1	524,186	917,590	1.7505
14 Aug 2010 03:20:07	1043969	11454774	famous_u64t_1999_200_006640792_1	505,466	882,474	1.7459
13 Aug 2010 20:32:46	1043969	11454774	famous_u64t_1999_200_006640792_1	496,106	865,342	1.7443