Task 15814328

Name	hadcm3n_zfut_1960_40_008371203_1
Workunit	8522062
Created	1 Jun 2013, 5:59:53 UTC
Sent	1 Jun 2013, 6:18:44 UTC
Report deadline	31 Aug 2013, 13:45:55 UTC
Received	12 Jun 2013, 6:19:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1179592
Run time	7 days 16 hours 11 min 59 sec
CPU time	7 days 12 hours 29 min 37 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	2.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> 09:29:34 (19945): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:01:28 (20143): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:18:22 (20348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:41 (20582): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:11:30 (20853): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:12:43 (21069): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:41:16 (21285): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:30:00 (21828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:54:29 (22089): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:55:09 (22304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:34:12 (22503): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:34:59 (22725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:43:30 (22959): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:44:15 (23233): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:49:31 (23465): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:00 (23668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:35 (23895): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:52:31 (24177): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:47:22 (24382): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:48:06 (24612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:49:49 (24848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:31 (25082): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:51:59 (25296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:54:26 (25524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:36:17 (25982): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:43:57 (26205): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:59:14 (26477): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:59:15 (26477): No heartbeat from core client for 30 sec - exiting 13:59:16 (26477): No heartbeat from core client for 30 sec - exiting 13:59:17 (26477): No heartbeat from core client for 30 sec - exiting 13:59:18 (26477): No heartbeat from core client for 30 sec - exiting 13:59:19 (26477): No heartbeat from core client for 30 sec - exiting 13:59:20 (26477): No heartbeat from core client for 30 sec - exiting 13:59:21 (26477): No heartbeat from core client for 30 sec - exiting 13:59:22 (26477): No heartbeat from core client for 30 sec - exiting 13:59:23 (26477): No heartbeat from core client for 30 sec - exiting 13:59:24 (26477): No heartbeat from core client for 30 sec - exiting 13:59:25 (26477): No heartbeat from core client for 30 sec - exiting 13:59:26 (26477): No heartbeat from core client for 30 sec - exiting 13:59:27 (26477): No heartbeat from core client for 30 sec - exiting 13:59:28 (26477): No heartbeat from core client for 30 sec - exiting 13:59:29 (26477): No heartbeat from core client for 30 sec - exiting 13:59:30 (26477): No heartbeat from core client for 30 sec - exiting 13:59:31 (26477): No heartbeat from core client for 30 sec - exiting 13:59:32 (26477): No heartbeat from core client for 30 sec - exiting 14:44:41 (26683): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:45:30 (26908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:29:48 (27190): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:05 (27453): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:04:23 (27657): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:40 (27869): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:33 (28081): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:25 (28287): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:10 (28514): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:32:57 (28750): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:38:41 (28985): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:33:39 (29187): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:34:29 (29423): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:35:52 (29645): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:56:35 (29875): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:01:02 (30076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:36:40 (30277): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:37:42 (30482): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:38:28 (30696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:39:35 (30924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:19 (31134): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:46:18 (31347): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:40:54 (31553): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:27 (31801): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:14 (32100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:43:12 (32314): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:52:03 (32547): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:07:26 (32751): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:36:19 (32954): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:42:16 (33203): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:37:06 (33409): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:50 (33637): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:38:32 (33871): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:39:16 (34098): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:44:52 (34326): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:40:10 (34528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:32 (34756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:34:18 (35023): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:44 (35247): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:00 (35489): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:38:43 (35683): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:54:09 (35877): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:05:38 (36086): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:41:06 (36262): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:42:38 (36447): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:45:04 (36654): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:17:22 (37006): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:18:10 (37211): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:02:48 (37387): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:02:49 (37387): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... SIGSEGV: segmentation violation Stack trace (15 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf77dd400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x806c0d5] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x806e5f2] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8072509] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8077f47] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x80781a3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e1b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf7533ce7] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Jun 2013 13:50:25	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	233,280	590,599	2.5317
10 Jun 2013 21:12:14	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	207,360	530,779	2.5597
10 Jun 2013 03:45:43	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	181,440	470,118	2.5910
09 Jun 2013 10:20:19	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	155,520	408,555	2.6270
08 Jun 2013 16:57:31	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	129,600	346,116	2.6706
07 Jun 2013 22:33:16	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	103,680	283,305	2.7325
07 Jun 2013 00:32:00	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	77,760	213,429	2.7447
06 Jun 2013 03:49:05	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	51,840	143,971	2.7772
05 Jun 2013 06:24:21	1179592	15814328	hadcm3n_zfut_1960_40_008371203_1	25,920	74,848	2.8877