Task 14018199

Name	hadam3p_pnw_8xv9_2007_1_007716849_1
Workunit	7871957
Created	27 Jan 2012, 13:04:14 UTC
Sent	27 Jan 2012, 14:00:55 UTC
Report deadline	8 Jan 2013, 19:20:55 UTC
Received	3 Mar 2012, 11:16:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1123964
Run time	1 days 9 hours 46 min 32 sec
CPU time	1 days 9 hours 0 min 13 sec
Validate state	Invalid
Credit	502.72
Device peak FLOPS	1.75 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:24:03 (2191): No heartbeat from core client for 30 sec - exiting 07:24:04 (2191): No heartbeat from core client for 30 sec - exiting 07:24:05 (2191): No heartbeat from core client for 30 sec - exiting 07:24:06 (2191): No heartbeat from core client for 30 sec - exiting 07:24:07 (2191): No heartbeat from core client for 30 sec - exiting 07:24:08 (2191): No heartbeat from core client for 30 sec - exiting 07:24:09 (2191): No heartbeat from core client for 30 sec - exiting 07:24:10 (2191): No heartbeat from core client for 30 sec - exiting 07:24:11 (2191): No heartbeat from core client for 30 sec - exiting 07:24:12 (2191): No heartbeat from core client for 30 sec - exiting 07:24:13 (2191): No heartbeat from core client for 30 sec - exiting 07:24:14 (2191): No heartbeat from core client for 30 sec - exiting 07:24:15 (2191): No heartbeat from core client for 30 sec - exiting 07:24:17 (2191): No heartbeat from core client for 30 sec - exiting 07:24:18 (2191): No heartbeat from core client for 30 sec - exiting 07:24:19 (2191): No heartbeat from core client for 30 sec - exiting 07:24:20 (2191): No heartbeat from core client for 30 sec - exiting 07:24:21 (2191): No heartbeat from core client for 30 sec - exiting 07:24:22 (2191): No heartbeat from core client for 30 sec - exiting 07:24:23 (2191): No heartbeat from core client for 30 sec - exiting 07:24:24 (2191): No heartbeat from core client for 30 sec - exiting 07:24:25 (2191): No heartbeat from core client for 30 sec - exiting 07:24:26 (2191): No heartbeat from core client for 30 sec - exiting 07:24:27 (2191): No heartbeat from core client for 30 sec - exiting 07:24:28 (2191): No heartbeat from core client for 30 sec - exiting 07:24:29 (2191): No heartbeat from core client for 30 sec - exiting 07:24:30 (2191): No heartbeat from core client for 30 sec - exiting 07:24:31 (2191): No heartbeat from core client for 30 sec - exiting 07:24:34 (2191): No heartbeat from core client for 30 sec - exiting 07:24:35 (2191): No heartbeat from core client for 30 sec - exiting 07:24:36 (2191): No heartbeat from core client for 30 sec - exiting 07:24:37 (2191): No heartbeat from core client for 30 sec - exiting 07:24:38 (2191): No heartbeat from core client for 30 sec - exiting 07:24:39 (2191): No heartbeat from core client for 30 sec - exiting 07:24:40 (2191): No heartbeat from core client for 30 sec - exiting 07:24:41 (2191): No heartbeat from core client for 30 sec - exiting 07:24:42 (2191): No heartbeat from core client for 30 sec - exiting 07:24:43 (2191): No heartbeat from core client for 30 sec - exiting 07:24:44 (2191): No heartbeat from core client for 30 sec - exiting 07:24:45 (2191): No heartbeat from core client for 30 sec - exiting 07:24:46 (2191): No heartbeat from core client for 30 sec - exiting 07:24:47 (2191): No heartbeat from core client for 30 sec - exiting 07:24:48 (2191): No heartbeat from core client for 30 sec - exiting 07:24:49 (2191): No heartbeat from core client for 30 sec - exiting 07:24:50 (2191): No heartbeat from core client for 30 sec - exiting 07:24:51 (2191): No heartbeat from core client for 30 sec - exiting 07:24:53 (2191): No heartbeat from core client for 30 sec - exiting 07:24:54 (2191): No heartbeat from core client for 30 sec - exiting 07:24:55 (2191): No heartbeat from core client for 30 sec - exiting 07:24:56 (2191): No heartbeat from core client for 30 sec - exiting 07:24:57 (2191): No heartbeat from core client for 30 sec - exiting 07:24:58 (2191): No heartbeat from core client for 30 sec - exiting 07:24:59 (2191): No heartbeat from core client for 30 sec - exiting 07:25:00 (2191): No heartbeat from core client for 30 sec - exiting 07:25:01 (2191): No heartbeat from core client for 30 sec - exiting 07:25:02 (2191): No heartbeat from core client for 30 sec - exiting 07:25:03 (2191): No heartbeat from core client for 30 sec - exiting 07:25:04 (2191): No heartbeat from core client for 30 sec - exiting 07:25:05 (2191): No heartbeat from core client for 30 sec - exiting 07:25:06 (2191): No heartbeat from core client for 30 sec - exiting 07:25:07 (2191): No heartbeat from core client for 30 sec - exiting 07:25:08 (2191): No heartbeat from core client for 30 sec - exiting 07:25:09 (2191): No heartbeat from core client for 30 sec - exiting 07:25:10 (2191): No heartbeat from core client for 30 sec - exiting 07:25:11 (2191): No heartbeat from core client for 30 sec - exiting 07:25:12 (2191): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:40:04 (2189): No heartbeat from core client for 30 sec - exiting 07:40:05 (2189): No heartbeat from core client for 30 sec - exiting 07:40:06 (2189): No heartbeat from core client for 30 sec - exiting 07:40:07 (2189): No heartbeat from core client for 30 sec - exiting 07:40:08 (2189): No heartbeat from core client for 30 sec - exiting 07:40:09 (2189): No heartbeat from core client for 30 sec - exiting 07:40:10 (2189): No heartbeat from core client for 30 sec - exiting 07:40:11 (2189): No heartbeat from core client for 30 sec - exiting 07:40:12 (2189): No heartbeat from core client for 30 sec - exiting 07:40:13 (2189): No heartbeat from core client for 30 sec - exiting 07:40:14 (2189): No heartbeat from core client for 30 sec - exiting 07:40:15 (2189): No heartbeat from core client for 30 sec - exiting 07:40:16 (2189): No heartbeat from core client for 30 sec - exiting 07:40:17 (2189): No heartbeat from core client for 30 sec - exiting 07:40:18 (2189): No heartbeat from core client for 30 sec - exiting 07:40:19 (2189): No heartbeat from core client for 30 sec - exiting 07:40:20 (2189): No heartbeat from core client for 30 sec - exiting 07:40:21 (2189): No heartbeat from core client for 30 sec - exiting 07:40:22 (2189): No heartbeat from core client for 30 sec - exiting 07:40:24 (2189): No heartbeat from core client for 30 sec - exiting 07:40:25 (2189): No heartbeat from core client for 30 sec - exiting 07:40:26 (2189): No heartbeat from core client for 30 sec - exiting 07:40:27 (2189): No heartbeat from core client for 30 sec - exiting 07:40:28 (2189): No heartbeat from core client for 30 sec - exiting 07:40:29 (2189): No heartbeat from core client for 30 sec - exiting 07:40:30 (2189): No heartbeat from core client for 30 sec - exiting 07:40:31 (2189): No heartbeat from core client for 30 sec - exiting 07:40:32 (2189): No heartbeat from core client for 30 sec - exiting 07:40:33 (2189): No heartbeat from core client for 30 sec - exiting 07:40:34 (2189): No heartbeat from core client for 30 sec - exiting 07:40:35 (2189): No heartbeat from core client for 30 sec - exiting 07:40:36 (2189): No heartbeat from core client for 30 sec - exiting 07:40:37 (2189): No heartbeat from core client for 30 sec - exiting 07:40:38 (2189): No heartbeat from core client for 30 sec - exiting 07:40:39 (2189): No heartbeat from core client for 30 sec - exiting 07:40:40 (2189): No heartbeat from core client for 30 sec - exiting 07:40:41 (2189): No heartbeat from core client for 30 sec - exiting 07:40:42 (2189): No heartbeat from core client for 30 sec - exiting 07:40:43 (2189): No heartbeat from core client for 30 sec - exiting 07:40:44 (2189): No heartbeat from core client for 30 sec - exiting 07:40:45 (2189): No heartbeat from core client for 30 sec - exiting 07:40:46 (2189): No heartbeat from core client for 30 sec - exiting 07:40:47 (2189): No heartbeat from core client for 30 sec - exiting 07:40:48 (2189): No heartbeat from core client for 30 sec - exiting 07:40:49 (2189): No heartbeat from core client for 30 sec - exiting 07:40:50 (2189): No heartbeat from core client for 30 sec - exiting 07:40:51 (2189): No heartbeat from core client for 30 sec - exiting 07:40:52 (2189): No heartbeat from core client for 30 sec - exiting 07:40:53 (2189): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:48:04 (1780): No heartbeat from core client for 30 sec - exiting 16:48:05 (1780): No heartbeat from core client for 30 sec - exiting 16:48:06 (1780): No heartbeat from core client for 30 sec - exiting 16:48:07 (1780): No heartbeat from core client for 30 sec - exiting 16:48:08 (1780): No heartbeat from core client for 30 sec - exiting 16:48:09 (1780): No heartbeat from core client for 30 sec - exiting 16:48:10 (1780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 3 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadam3p_pnw_6.09_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c35ff] [0xf777e400] ../../projects/climateprediction.net/hadam3p_pnw_6.09_i686-pc-linux-gnu[0x804d58f] ../../projects/climateprediction.net/hadam3p_pnw_6.09_i686-pc-linux-gnu[0x8051223] ../../projects/climateprediction.net/hadam3p_pnw_6.09_i686-pc-linux-gnu[0x8051c59] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf74cbce7] ../../projects/climateprediction.net/hadam3p_pnw_6.09_i686-pc-linux-gnu(pthread_cancel+0x71)[0x804d201] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Mar 2012 21:56:13	1123964	14018199	hadam3p_pnw_8xv9_2007_1_007716849_1	23,136	115,634	4.9980
28 Feb 2012 08:56:43	1123964	14018199	hadam3p_pnw_8xv9_2007_1_007716849_1	11,616	58,185	5.0090