Task 18739095

Name	hadam3p_pnw_q6pk_2013_1_010030533_1
Workunit	10028595
Created	25 Jul 2015, 17:39:09 UTC
Sent	26 Jul 2015, 0:04:32 UTC
Report deadline	7 Jul 2016, 5:24:32 UTC
Received	16 Aug 2015, 21:43:17 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1369999
Run time	2 days 4 hours 24 min 52 sec
CPU time	2 days 2 hours 19 min 8 sec
Validate state	Invalid
Credit	2,509.66
Device peak FLOPS	4.63 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v7.27 windows_intelx86
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10396, selfPID=5960, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:59:51 (7708): start_timer_thread(): CreateThread() failed, errno 0 22:59:53 (7172): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:42:59 (10880): start_timer_thread(): CreateThread() failed, errno 0 23:43:00 (11972): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:05:02 (9608): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 00:11:32 (9592): start_timer_thread(): CreateThread() failed, errno 0 00:11:33 (9912): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:45:41 (14660): start_timer_thread(): CreateThread() failed, errno 0 18:45:42 (7372): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:56:12 (10672): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:09:39 (14200): start_timer_thread(): CreateThread() failed, errno 0 21:09:41 (12860): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:44:24 (17524): start_timer_thread(): CreateThread() failed, errno 0 09:44:25 (10824): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:25:39 (22304): start_timer_thread(): CreateThread() failed, errno 0 11:25:41 (1628): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:00:25 (28112): start_timer_thread(): CreateThread() failed, errno 0 12:00:26 (6468): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:20:05 (6100): start_timer_thread(): CreateThread() failed, errno 0 18:20:06 (10292): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:16:03 (9872): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:47:48 (496): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:22:37 (20976): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:26:52 (15712): start_timer_thread(): CreateThread() failed, errno 0 22:26:54 (19408): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:58:13 (26308): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:35:17 (27976): start_timer_thread(): CreateThread() failed, errno 0 15:35:19 (24144): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:54:06 (23020): start_timer_thread(): CreateThread() failed, errno 0 15:54:08 (26076): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 11:39:46 (30092): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:03:41 (15860): start_timer_thread(): CreateThread() failed, errno 0 17:03:42 (31160): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:49:55 (37360): start_timer_thread(): CreateThread() failed, errno 0 20:49:57 (37340): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:14:14 (19848): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 00:03:04 (43836): start_timer_thread(): CreateThread() failed, errno 0 00:03:05 (44972): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:34:54 (46992): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:57:28 (53696): start_timer_thread(): CreateThread() failed, errno 0 18:57:29 (51696): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:43:26 (55380): start_timer_thread(): CreateThread() failed, errno 0 20:43:28 (55520): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:18:03 (52512): start_timer_thread(): CreateThread() failed, errno 0 21:18:04 (54300): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:49:55 (45812): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:55:11 (54508): start_timer_thread(): CreateThread() failed, errno 0 21:55:12 (53544): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:20:33 (25676): start_timer_thread(): CreateThread() failed, errno 0 19:20:34 (25844): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:26:33 (17872): start_timer_thread(): CreateThread() failed, errno 0 19:26:34 (14460): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:58:11 (8560): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:27:59 (5652): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:22:12 (25312): start_timer_thread(): CreateThread() failed, errno 0 21:22:14 (13664): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:38:10 (14388): start_timer_thread(): CreateThread() failed, errno 0 21:38:12 (26808): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:28:45 (26952): start_timer_thread(): CreateThread() failed, errno 0 22:28:46 (15104): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:51:31 (16916): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:28:34 (8796): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=8796, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6796, selfPID=11824, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 13:28:50 (11824): called boinc_finish(0) </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_q6pk_2013_1_010030533_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_q6pk_2013_1_010030533_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_q6pk_2013_1_010030533_1_13.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_q6pk_2013_1_010030533_1_14.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_q6pk_2013_1_010030533_1_15.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_q6pk_2013_1_010030533_1_16.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_q6pk_2013_1_010030533_1_17.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_q6pk_2013_1_010030533_1_18.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2015 06:37:06	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	115,499	180,456	1.5624
12 Aug 2015 03:00:02	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	103,979	162,752	1.5652
10 Aug 2015 02:51:01	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	92,459	144,827	1.5664
08 Aug 2015 06:36:02	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	80,939	127,218	1.5718
05 Aug 2015 04:32:44	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	69,419	109,760	1.5811
03 Aug 2015 01:55:28	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	57,899	91,497	1.5803
01 Aug 2015 06:53:33	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	46,379	73,427	1.5832
31 Jul 2015 06:47:46	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	34,859	55,160	1.5824
30 Jul 2015 06:52:42	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	23,339	37,604	1.6112
29 Jul 2015 22:20:24	1369999	18739095	hadam3p_pnw_q6pk_2013_1_010030533_1	11,819	19,222	1.6264