Task 21708946

Name	wah2_pnw25_cakt_190312_49_695_011383755_0
Workunit	11383755
Created	13 Jun 2019, 9:55:03 UTC
Sent	13 Jun 2019, 10:07:09 UTC
Report deadline	25 May 2020, 15:27:09 UTC
Received	16 Jul 2019, 8:57:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID	1458099
Run time	7 days 13 hours 29 min 46 sec
CPU time	6 days 7 hours 57 min 22 sec
Validate state	Invalid
Credit	15,978.39
Device peak FLOPS	3.51 GFLOPS
Application version	Weather At Home 2 (wah2) v8.24 windows_intelx86
Peak working set size	180.60 MB
Peak swap size	165.79 MB
Peak disk usage	1.79 MB
Stderr	<core_client_version>7.14.2</core_client_version> <![CDATA[ <message> finish file present too long</message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:52:14 (12780): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda3dec 18:51:50 (18788): BOINC client no longer exists - exiting 18:51:50 (18788): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4feb Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4mar CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4apr CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4may Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4jun CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4jul CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4aug Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4sep CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4oct CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4nov Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda4dec CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda5feb CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda5mar CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda5apr CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12940, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14904, selfPID=9548, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda5may CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda5jun CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda5jul CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/wah2_pnw25_cakt_190312_49_695_011383755/dataout\caktga.pda5aug CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=21892, selfPID=21252, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_ain::Monitor... 10:53:37 (21252): called boinc_finish(0) CPDN Monitor - Abort request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0235A12C Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 07/16/19 10:56:29 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000152000 C:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077d40000 000000000019c000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17763.475) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.17763.641 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Sistema operativo Microsoft® Windows® Product Version : 10.0.17763.641 ModLoad: 0000000076660000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17763.475) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Sistema operativo Microsoft® Windows® Product Version : 10.0.17763.1 ModLoad: 000000006d9c0000 0000000000010000 C:\Program Files (x86)\AVG\Antivirus\x86\aswhook.dll (19.5.4444.0) (-nosymbols- Symbols Loaded) Linked PDB Filename : D:\BUILD\work\01\c0cb082698b95679\idp\build\avgast\RU_NTDLL-lH-Y2A!Win32_vs141\bin\Release_Unicode_NTDLL_vs141\Win32\aswhook.pdb File Version : 19.5.4444.0 Company Name : AVG Technologies CZ, s.r.o. Product Name : AVG Product Version : 19.5.4444.0 ModLoad: 0000000075970000 00000000001fa000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17763.475) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Sistema operativo Microsoft® Windows® Product Version : 10.0.17763.1 ModLoad: 0000000076e10000 0000000000199000 C:\WINDOWS\System32\USER32.dll (6.2.17763.168) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.17763.641 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Sistema operativo Microsoft® Windows® Product Version : 10.0.17763.641 ModLoad: 0000000075b70000 0000000000017000 C:\WINDOWS\System32\win32u.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 ModLoad: 0000000076fb0000 0000000000023000 C:\WINDOWS\System32\GDI32.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 ModLoad: 0000000075de0000 0000000000166000 C:\WINDOWS\System32\gdi32full.dll (6.2.17763.557) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.17763.557 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.557 ModLoad: 0000000077510000 0000000000080000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 ModLoad: 00000000753d0000 0000000000122000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17763.404) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.17763.404 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.404 ModLoad: 0000000076000000 000000000007e000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.17763.641 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Sistema operativo Microsoft® Windows® Product Version : 10.0.17763.641 ModLoad: 0000000076740000 00000000000c0000 C:\WINDOWS\System32\msvcrt.dll (7.0.17763.475) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.17763.475 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.17763.475 ModLoad: 0000000075c20000 0000000000079000 C:\WINDOWS\System32\sechost.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 ModLoad: 0000000077290000 00000000000bf000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17763.379) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.17763.641 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Sistema operativo Microsoft® Windows® Product Version : 10.0.17763.641 ModLoad: 00000000753b0000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 ModLoad: 00000000753a0000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 ModLoad: 00000000773a0000 0000000000062000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17763.557) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.17763.557 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.557 ModLoad: 0000000075ca0000 0000000000025000 C:\WINDOWS\System32\IMM32.DLL (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 ModLoad: 0000000059680000 000000000018f000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 ModLoad: 0000000075190000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17763.1) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.17763.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17763.1 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 4281, Write: 0, Other 54683 - I/O Transfers Counters - Read: 0, Write: 1686, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 93232, QuotaPeakPagedPoolUsage: 93392 QuotaNonPagedPoolUsage: 8696, QuotaPeakNonPagedPoolUsage: 8848 - Virtual Memory Usage - VirtualSize: 53403648, PeakVirtualSize: 53403648 - Pagefile Usage - PagefileUsage: 3989504, PeakPagefileUsage: 3989504 - Working Set Size - WorkingSetSize: 6414336, PeakWorkingSetSize: 6418432, PageFaultCount: 1871 * Dump of thread ID 20056 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 16562500.000000, User Time: 23125000.000000, Wait Time: 8191.000000 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0235A12C - Registers - eax=00000065 ebx=004b63b9 ecx=0019dc00 edx=0235a12c esi=0019d9b8 edi=0019dc30 eip=0048ee46 esp=0019d94c ebp=0019dbdc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202 - Callstack - ChildEBP RetAddr Args to Child 0019dbdc 0048161c 0019dc00 004b63b8 00000000 0019dc30 wah2_8.24_windows_intelx86!+0x0 0019dc20 0044fa52 0235a12c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 0019dc40 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 0019dc58 00485ca1 c0000005 0019dc8c 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 0019ff70 76680419 002d0000 76680400 0019ffdc 77da662d wah2_8.24_windows_intelx86!+0x0 0019ff80 77da662d 002d0000 d337df4d 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0019ffdc 77da65fd ffffffff 77dc51e5 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 0019ffec 00000000 00485cce 002d0000 00000000 78746341 ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Dump of thread ID 20308 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 8185.000000 - Registers - eax=00000000 ebx=0228fd0c ecx=00000000 edx=00000000 esi=00000000 edi=0228fd0c eip=77db07ec esp=0228fccc ebp=0228fd30 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 0228fd30 75a7dbdf 00000064 00000000 0228ff70 00478b44 ntdll!ZwDelayExecution+0x0 0228fd40 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 0228ff70 76680419 00000000 76680400 0228ffdc 77da662d wah2_8.24_windows_intelx86!+0x0 0228ff80 77da662d 00000000 d106df4d 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0228ffdc 77da65fd ffffffff 77dc51e5 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 0228ffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Jul 2019 10:39:34	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	242,219	525,768	2.1706
10 Jul 2019 14:12:09	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	230,699	503,116	2.1808
09 Jul 2019 14:57:08	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	219,179	480,159	2.1907
04 Jul 2019 16:26:25	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	207,659	456,511	2.1984
01 Jul 2019 09:48:02	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	196,139	427,308	2.1786
30 Jun 2019 02:41:05	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	184,619	404,117	2.1889
29 Jun 2019 20:35:03	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	173,099	381,378	2.2032
29 Jun 2019 13:38:35	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	161,579	358,498	2.2187
29 Jun 2019 06:01:05	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	150,059	336,388	2.2417
28 Jun 2019 23:40:27	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	138,539	314,543	2.2704
28 Jun 2019 15:06:34	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	127,019	292,285	2.3011
28 Jun 2019 08:51:21	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	115,499	270,146	2.3389
28 Jun 2019 01:28:43	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	103,979	248,067	2.3857
27 Jun 2019 14:40:49	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	92,459	226,971	2.4548
27 Jun 2019 07:13:44	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	80,939	205,174	2.5349
27 Jun 2019 00:07:13	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	69,419	183,199	2.6390
26 Jun 2019 14:51:18	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	57,899	161,284	2.7856
26 Jun 2019 07:59:58	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	46,379	139,424	3.0062
26 Jun 2019 01:39:05	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	34,859	117,259	3.3638
25 Jun 2019 02:15:28	1458099	21708946	wah2_pnw25_cakt_190312_49_695_011383755_0	23,339	71,177	3.0497