Task 14336434

Name	hadam3p_pnw_8pgo_2006_1_007851385_0
Workunit	8006497
Created	1 Apr 2012, 5:53:59 UTC
Sent	1 Apr 2012, 5:54:07 UTC
Report deadline	14 Mar 2013, 11:14:07 UTC
Received	4 Apr 2012, 17:26:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	895482
Run time	2 days 1 hours 35 min 25 sec
CPU time	1 days 17 hours 47 min 58 sec
Validate state	Invalid
Credit	1,003.35
Device peak FLOPS	3.26 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 i686-apple-darwin
Stderr	<core_client_version>7.0.8</core_client_version> <![CDATA[ <stderr_txt> 00:28:03 (88430): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation 00:34:03 (25675): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:21:48 (27843): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:09:16 (13533): No heartbeat from core client for 30 sec - exiting 09:09:17 (13533): No heartbeat from core client for 30 sec - exiting 09:09:18 (13533): No heartbeat from core client for 30 sec - exiting 09:09:19 (13533): No heartbeat from core client for 30 sec - exiting 09:09:20 (13533): No heartbeat from core client for 30 sec - exiting 09:09:21 (13533): No heartbeat from core client for 30 sec - exiting 09:09:22 (13533): No heartbeat from core client for 30 sec - exiting 09:09:23 (13533): No heartbeat from core client for 30 sec - exiting 09:09:24 (13533): No heartbeat from core client for 30 sec - exiting 09:09:25 (13533): No heartbeat from core client for 30 sec - exiting 09:09:26 (13533): No heartbeat from core client for 30 sec - exiting 09:09:27 (13533): No heartbeat from core client for 30 sec - exiting 09:09:28 (13533): No heartbeat from core client for 30 sec - exiting 09:09:29 (13533): No heartbeat from core client for 30 sec - exiting 09:09:30 (13533): No heartbeat from core client for 30 sec - exiting 09:09:31 (13533): No heartbeat from core client for 30 sec - exiting 09:09:32 (13533): No heartbeat from core client for 30 sec - exiting 09:09:33 (13533): No heartbeat from core client for 30 sec - exiting 09:09:34 (13533): No heartbeat from core client for 30 sec - exiting 09:09:35 (13533): No heartbeat from core client for 30 sec - exiting 09:09:36 (13533): No heartbeat from core client for 30 sec - exiting 09:09:37 (13533): No heartbeat from core client for 30 sec - exiting 09:09:38 (13533): No heartbeat from core client for 30 sec - exiting 09:09:39 (13533): No heartbeat from core client for 30 sec - exiting 09:09:40 (13533): No heartbeat from core client for 30 sec - exiting 09:09:41 (13533): No heartbeat from core client for 30 sec - exiting 09:09:42 (13533): No heartbeat from core client for 30 sec - exiting 09:09:44 (13533): No heartbeat from core client for 30 sec - exiting 09:09:45 (13533): No heartbeat from core client for 30 sec - exiting 09:09:46 (13533): No heartbeat from core client for 30 sec - exiting 09:09:47 (13533): No heartbeat from core client for 30 sec - exiting 09:09:48 (13533): No heartbeat from core client for 30 sec - exiting 09:09:49 (13533): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:52 (13672): No heartbeat from core client for 30 sec - exiting 09:10:53 (13672): No heartbeat from core client for 30 sec - exiting 09:10:54 (13672): No heartbeat from core client for 30 sec - exiting 09:10:55 (13672): No heartbeat from core client for 30 sec - exiting 09:10:56 (13672): No heartbeat from core client for 30 sec - exiting 09:10:57 (13672): No heartbeat from core client for 30 sec - exiting 09:10:58 (13672): No heartbeat from core client for 30 sec - exiting 09:10:59 (13672): No heartbeat from core client for 30 sec - exiting 09:11:00 (13672): No heartbeat from core client for 30 sec - exiting 09:11:01 (13672): No heartbeat from core client for 30 sec - exiting 09:11:02 (13672): No heartbeat from core client for 30 sec - exiting 09:11:03 (13672): No heartbeat from core client for 30 sec - exiting 09:11:04 (13672): No heartbeat from core client for 30 sec - exiting 09:11:05 (13672): No heartbeat from core client for 30 sec - exiting 09:11:06 (13672): No heartbeat from core client for 30 sec - exiting 09:11:07 (13672): No heartbeat from core client for 30 sec - exiting 09:11:08 (13672): No heartbeat from core client for 30 sec - exiting 09:11:09 (13672): No heartbeat from core client for 30 sec - exiting 09:11:10 (13672): No heartbeat from core client for 30 sec - exiting 09:11:11 (13672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:12:59 (21525): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:15:23 (30323): No heartbeat from core client for 30 sec - exiting 11:15:24 (30323): No heartbeat from core client for 30 sec - exiting 11:15:25 (30323): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:45:01 (72794): No heartbeat from core client for 30 sec - exiting 16:45:02 (72794): No heartbeat from core client for 30 sec - exiting 16:45:03 (72794): No heartbeat from core client for 30 sec - exiting 16:45:04 (72794): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:56:27 (90386): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:00:03 (98740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:03:54 (23653): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:30:12 (25621): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:15 (36274): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:16:28 (39519): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:20:54 (5487): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:45:21 (81138): No heartbeat from core client for 30 sec - exiting 18:45:23 (81138): No heartbeat from core client for 30 sec - exiting 18:45:24 (81138): No heartbeat from core client for 30 sec - exiting 18:45:25 (81138): No heartbeat from core client for 30 sec - exiting 18:45:26 (81138): No heartbeat from core client for 30 sec - exiting 18:45:27 (81138): No heartbeat from core client for 30 sec - exiting 18:45:28 (81138): No heartbeat from core client for 30 sec - exiting 18:45:29 (81138): No heartbeat from core client for 30 sec - exiting 18:45:30 (81138): No heartbeat from core client for 30 sec - exiting 18:45:31 (81138): No heartbeat from core client for 30 sec - exiting 18:45:32 (81138): No heartbeat from core client for 30 sec - exiting 18:45:33 (81138): No heartbeat from core client for 30 sec - exiting 18:45:34 (81138): No heartbeat from core client for 30 sec - exiting 18:45:35 (81138): No heartbeat from core client for 30 sec - exiting 18:45:36 (81138): No heartbeat from core client for 30 sec - exiting 18:45:37 (81138): No heartbeat from core client for 30 sec - exiting 18:45:38 (81138): No heartbeat from core client for 30 sec - exiting 18:45:39 (81138): No heartbeat from core client for 30 sec - exiting 18:45:40 (81138): No heartbeat from core client for 30 sec - exiting 18:45:42 (81138): No heartbeat from core client for 30 sec - exiting 18:45:43 (81138): No heartbeat from core client for 30 sec - exiting 18:45:44 (81138): No heartbeat from core client for 30 sec - exiting 18:45:45 (81138): No heartbeat from core client for 30 sec - exiting 18:45:46 (81138): No heartbeat from core client for 30 sec - exiting 18:45:47 (81138): No heartbeat from core client for 30 sec - exiting 18:46:18 (81138): No heartbeat from core client for 30 sec - exiting 18:46:19 (81138): No heartbeat from core client for 30 sec - exiting 18:46:21 (81138): No heartbeat from core client for 30 sec - exiting 18:46:22 (81138): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:04:14 (90432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:14:16 (13390): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:19:16 (14424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:39:17 (14939): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:19:24 (17241): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:22:19 (21713): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:18:36 (94858): No heartbeat from core client for 30 sec - exiting 10:18:37 (94858): No heartbeat from core client for 30 sec - exiting 10:18:38 (94858): No heartbeat from core client for 30 sec - exiting 10:18:39 (94858): No heartbeat from core client for 30 sec - exiting 10:18:40 (94858): No heartbeat from core client for 30 sec - exiting 10:18:41 (94858): No heartbeat from core client for 30 sec - exiting 10:18:42 (94858): No heartbeat from core client for 30 sec - exiting 10:18:43 (94858): No heartbeat from core client for 30 sec - exiting 10:18:44 (94858): No heartbeat from core client for 30 sec - exiting 10:18:45 (94858): No heartbeat from core client for 30 sec - exiting 10:18:46 (94858): No heartbeat from core client for 30 sec - exiting 10:18:47 (94858): No heartbeat from core client for 30 sec - exiting 10:18:48 (94858): No heartbeat from core client for 30 sec - exiting 10:18:49 (94858): No heartbeat from core client for 30 sec - exiting 10:18:50 (94858): No heartbeat from core client for 30 sec - exiting 10:18:51 (94858): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:19:57 (1959): No heartbeat from core client for 30 sec - exiting 11:19:58 (1959): No heartbeat from core client for 30 sec - exiting 11:19:59 (1959): No heartbeat from core client for 30 sec - exiting 11:20:00 (1959): No heartbeat from core client for 30 sec - exiting 11:20:01 (1959): No heartbeat from core client for 30 sec - exiting 11:20:02 (1959): No heartbeat from core client for 30 sec - exiting 11:20:03 (1959): No heartbeat from core client for 30 sec - exiting 11:20:04 (1959): No heartbeat from core client for 30 sec - exiting 11:20:05 (1959): No heartbeat from core client for 30 sec - exiting 11:20:06 (1959): No heartbeat from core client for 30 sec - exiting 11:20:07 (1959): No heartbeat from core client for 30 sec - exiting 11:20:08 (1959): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:20:57 (1964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:23 (1971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:25 (1971): No heartbeat from core client for 30 sec - exiting 11:33:26 (1971): No heartbeat from core client for 30 sec - exiting 11:33:27 (1971): No heartbeat from core client for 30 sec - exiting 11:33:28 (1971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 13:48:39 (7454): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:43:33 (31734): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=51796, selfPID=51732, iMonCtr=1 Model crash detected, will try to restart... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=51796, selfPID=51796, iMonCtr=2 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 4 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_8pgo_2006_1_007851385_0_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8pgo_2006_1_007851385_0_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8pgo_2006_1_007851385_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8pgo_2006_1_007851385_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8pgo_2006_1_007851385_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8pgo_2006_1_007851385_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8pgo_2006_1_007851385_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8pgo_2006_1_007851385_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Apr 2012 01:34:34	895482	14336434	hadam3p_pnw_8pgo_2006_1_007851385_0	46,176	129,365	2.8016
03 Apr 2012 02:27:39	895482	14336434	hadam3p_pnw_8pgo_2006_1_007851385_0	34,656	97,339	2.8087
02 Apr 2012 03:58:35	895482	14336434	hadam3p_pnw_8pgo_2006_1_007851385_0	23,136	65,334	2.8239
01 Apr 2012 16:38:39	895482	14336434	hadam3p_pnw_8pgo_2006_1_007851385_0	11,616	32,581	2.8048