Task 19399209

Name	wah2_sas50_fmnn_201412_13_370_010399921_0
Workunit	10399921
Created	18 Mar 2016, 16:28:02 UTC
Sent	1 Aug 2016, 12:40:35 UTC
Report deadline	14 Jul 2017, 18:00:35 UTC
Received	23 Aug 2016, 6:16:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1292474
Run time	4 days 2 hours 36 min
CPU time	3 days 18 hours 43 min 14 sec
Validate state	Invalid
Credit	6,099.22
Device peak FLOPS	4.05 GFLOPS
Application version	Weather At Home 2 (wah2) (region independent) v8.12 windows_intelx86
Peak working set size	233.78 MB
Peak swap size	197.82 MB
Peak disk usage	28.65 MB
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <stderr_txt> 21:46:05 (6524): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:39:59 (11416): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:26:06 (1876): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 13:00:02 (6076): start_timer_thread(): CreateThread() failed, errno 0 13:00:04 (3820): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 15:22:38 (3560): start_timer_thread(): CreateThread() failed, errno 0 15:22:39 (2268): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3560, selfPID=2268, iMonCtr=1 15:51:06 (11072): start_timer_thread(): CreateThread() failed, errno 0 15:51:08 (11236): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11072, selfPID=11072, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11072, selfPID=11236, iMonCtr=1 20:05:54 (10728): start_timer_thread(): CreateThread() failed, errno 0 20:05:56 (10444): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:43:36 (6668): start_timer_thread(): CreateThread() failed, errno 0 07:43:37 (11256): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:53:52 (4800): start_timer_thread(): CreateThread() failed, errno 0 09:53:53 (6864): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:31:55 (5572): start_timer_thread(): CreateThread() failed, errno 0 13:31:56 (10308): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 13:40:09 (8052): start_timer_thread(): CreateThread() failed, errno 0 13:40:10 (1872): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8052, selfPID=8052, iMonCtr=1 15:12:01 (6792): start_timer_thread(): CreateThread() failed, errno 0 15:12:02 (5132): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6792, selfPID=5132, iMonCtr=1 21:03:40 (10412): start_timer_thread(): CreateThread() failed, errno 0 21:03:41 (9740): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:12:14 (6556): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5340, selfPID=5340, iMonCtr=1 18:05:25 (12616): start_timer_thread(): CreateThread() failed, errno 0 18:05:27 (10916): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12616, selfPID=10916, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9500, selfPID=10572, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10132, selfPID=9720, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9116, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2916, selfPID=9340, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8968, selfPID=11540, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7100, selfPID=8940, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1380, selfPID=1380, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2860, selfPID=2860, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8136, selfPID=5296, iMonCtr=1 01:31:40 (8772): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:22:57 (7584): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 02:53:55 (3720): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 03:24:33 (2876): start_timer_thread(): CreateThread() failed, errno 0 03:24:35 (6284): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 03:55:07 (5940): start_timer_thread(): CreateThread() failed, errno 0 03:55:08 (2236): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 04:25:41 (7912): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 04:56:16 (3084): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 05:26:49 (10004): start_timer_thread(): CreateThread() failed, errno 0 05:26:51 (812): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 05:57:21 (756): start_timer_thread(): CreateThread() failed, errno 0 05:57:23 (10856): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:58:34 (9224): start_timer_thread(): CreateThread() failed, errno 0 06:58:36 (6632): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 11:25:55 (9700): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:04:36 (6332): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10836, selfPID=6332, iMonCtr=1 20:07:34 (2064): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:13:14 (1620): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:06:56 (10500): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10500, selfPID=3916, iMonCtr=1 09:50:06 (9732): start_timer_thread(): CreateThread() failed, errno 0 09:50:07 (12076): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4080, selfPID=11692, iMonCtr=1 16:16:31 (7020): start_timer_thread(): CreateThread() failed, errno 0 16:16:33 (6176): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7020, selfPID=6176, iMonCtr=1 00:51:42 (4636): start_timer_thread(): CreateThread() failed, errno 0 00:51:44 (5952): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4636, selfPID=5952, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8616, selfPID=9204, iMonCtr=1 07:20:26 (10256): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10156, selfPID=9580, iMonCtr=1 07:03:11 (696): start_timer_thread(): CreateThread() failed, errno 0 07:03:12 (6468): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=696, selfPID=6468, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=696, selfPID=696, iMonCtr=1 07:08:58 (9176): start_timer_thread(): CreateThread() failed, errno 0 07:09:02 (8092): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 12:15:11 (9224): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2668, selfPID=2668, iMonCtr=1 15:56:46 (8912): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8912, selfPID=8204, iMonCtr=1 16:17:33 (7388): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7388, selfPID=9556, iMonCtr=1 18:57:46 (4316): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8356, selfPID=4316, iMonCtr=1 23:11:21 (11984): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9636, selfPID=9636, iMonCtr=1 23:27:17 (7888): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3828, selfPID=8404, iMonCtr=1 16:13:05 (4064): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4064, selfPID=4064, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4064, selfPID=4256, iMonCtr=1 17:55:12 (10008): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:16:19 (8720): start_timer_thread(): CreateThread() failed, errno 0 00:16:22 (7704): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8720, selfPID=7704, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=772, selfPID=9372, iMonCtr=1 07:59:05 (10984): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10984, selfPID=10984, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10984, selfPID=4976, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8524, selfPID=9380, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8524, selfPID=8524, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7136, selfPID=10588, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7796, selfPID=8748, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2692, selfPID=4692, iMonCtr=1 20:22:19 (12108): start_timer_thread(): CreateThread() failed, errno 0 20:22:23 (5436): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12108, selfPID=5436, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4048, selfPID=3268, iMonCtr=1 07:42:43 (9176): start_timer_thread(): CreateThread() failed, errno 0 07:42:45 (8932): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:57:01 (6132): start_timer_thread(): CreateThread() failed, errno 0 11:57:04 (1496): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6132, selfPID=1496, iMonCtr=1 18:17:29 (3592): start_timer_thread(): CreateThread() failed, errno 0 18:17:30 (6968): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 20:18:36 (4300): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4300, selfPID=10660, iMonCtr=1 20:25:35 (5976): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:22:06 (5372): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:49:45 (8572): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8572, selfPID=9708, iMonCtr=1 10:25:41 (10336): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:34:44 (8436): start_timer_thread(): CreateThread() failed, errno 0 20:34:47 (4376): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:43:37 (8844): start_timer_thread(): CreateThread() failed, errno 0 20:43:39 (8956): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8844, selfPID=8956, iMonCtr=1 10:26:27 (4380): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5688, selfPID=8928, iMonCtr=1 19:01:17 (3032): start_timer_thread(): CreateThread() failed, errno 0 19:01:18 (10424): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3032, selfPID=3032, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3032, selfPID=10424, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... 20:48:55 (8448): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:50:54 (940): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10748, selfPID=940, iMonCtr=1 12:51:19 (8124): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8124, selfPID=5496, iMonCtr=1 15:13:09 (10340): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 18:59:58 (9252): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9252, selfPID=9252, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9252, selfPID=9500, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6872, selfPID=6872, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7360, selfPID=11468, iMonCtr=1 11:43:29 (9200): start_timer_thread(): CreateThread() failed, errno 0 11:43:31 (7280): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:07:38 (9032): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:38:01 (9392): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=9392, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2516, selfPID=2516, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2516, selfPID=524, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 14:38:13 (524): called boinc_finish(0) </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>wah2_sas50_fmnn_201412_13_370_010399921_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>wah2_sas50_fmnn_201412_13_370_010399921_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>wah2_sas50_fmnn_201412_13_370_010399921_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>wah2_sas50_fmnn_201412_13_370_010399921_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>wah2_sas50_fmnn_201412_13_370_010399921_0_13.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>wah2_sas50_fmnn_201412_13_370_010399921_0_14.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Aug 2016 04:18:10	1292474	19399209	wah2_sas50_fmnn_201412_13_370_010399921_0	92,459	321,019	3.4720
20 Aug 2016 14:43:07	1292474	19399209	wah2_sas50_fmnn_201412_13_370_010399921_0	80,939	279,268	3.4504
18 Aug 2016 01:20:47	1292474	19399209	wah2_sas50_fmnn_201412_13_370_010399921_0	69,419	238,454	3.4350
16 Aug 2016 16:33:37	1292474	19399209	wah2_sas50_fmnn_201412_13_370_010399921_0	57,899	198,452	3.4276
12 Aug 2016 13:40:31	1292474	19399209	wah2_sas50_fmnn_201412_13_370_010399921_0	46,379	158,465	3.4167
10 Aug 2016 20:09:24	1292474	19399209	wah2_sas50_fmnn_201412_13_370_010399921_0	34,859	120,557	3.4584
08 Aug 2016 14:35:51	1292474	19399209	wah2_sas50_fmnn_201412_13_370_010399921_0	23,339	80,228	3.4375
07 Aug 2016 01:55:32	1292474	19399209	wah2_sas50_fmnn_201412_13_370_010399921_0	11,819	40,009	3.3851