Task 15774578

Name	hadcm3n_39mr_1980_40_008364900_0
Workunit	8515759
Created	10 May 2013, 22:35:23 UTC
Sent	11 May 2013, 0:24:00 UTC
Report deadline	10 Aug 2013, 7:51:11 UTC
Received	23 Jul 2013, 4:56:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	910934
Run time	33 days 5 hours 6 min 35 sec
CPU time	32 days 15 hours 32 min 9 sec
Validate state	Invalid
Credit	11,508.48
Device peak FLOPS	1.63 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.29</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:44:06 (29709): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:44:07 (29709): No heartbeat from core client for 30 sec - exiting 19:44:08 (29709): No heartbeat from core client for 30 sec - exiting 22:45:00 (30298): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:45:01 (30298): No heartbeat from core client for 30 sec - exiting 02:22:27 (31495): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:22:29 (31495): No heartbeat from core client for 30 sec - exiting 05:03:21 (32469): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:22 (32469): No heartbeat from core client for 30 sec - exiting 08:46:37 (1238): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:46:38 (1238): No heartbeat from core client for 30 sec - exiting 12:20:30 (2349): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:20:31 (2349): No heartbeat from core client for 30 sec - exiting 14:49:12 (3666): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:13 (3666): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Signal 3 received, exiting... Called boinc_finish Stack trace (10 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xb7736400] /lib/libc.so.6[0x425d26c8] /lib/libstdc++.so.6(_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsgetnEPci+0x75)[0x428d8605] /lib/libstdc++.so.6(_ZNSi4readEPci+0x69)[0x428b3f39] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804e44f] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050f96] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xf5)[0x424aa635] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jul 2013 21:21:30	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	959,040	2,807,735	2.9277
23 Jul 2013 20:41:18	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	933,120	2,732,849	2.9287
23 Jul 2013 20:08:46	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	907,200	2,658,050	2.9299
23 Jul 2013 19:15:28	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	881,280	2,583,284	2.9313
23 Jul 2013 16:23:20	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	855,360	2,508,477	2.9327
23 Jul 2013 16:23:21	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	829,440	2,433,514	2.9339
23 Jul 2013 16:23:21	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	803,520	2,359,733	2.9367
12 Jul 2013 00:31:47	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	777,600	2,285,332	2.9390
09 Jul 2013 08:26:55	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	751,680	2,208,881	2.9386
08 Jul 2013 04:16:20	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	725,760	2,132,695	2.9386
07 Jul 2013 06:45:40	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	699,840	2,056,579	2.9386
06 Jul 2013 09:38:34	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	673,920	1,981,333	2.9400
06 Jul 2013 05:03:08	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	648,000	1,905,665	2.9408
04 Jul 2013 16:10:57	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	622,080	1,830,208	2.9421
03 Jul 2013 17:43:52	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	596,160	1,753,431	2.9412
02 Jul 2013 20:47:15	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	570,240	1,679,341	2.9450
02 Jul 2013 11:59:16	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	544,320	1,604,636	2.9480
02 Jul 2013 11:11:22	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	518,400	1,529,469	2.9504
02 Jul 2013 10:30:28	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	492,480	1,453,401	2.9512
02 Jul 2013 10:01:21	910934	15774578	hadcm3n_39mr_1980_40_008364900_0	466,560	1,377,917	2.9534