Task 21416126

Name	wah2_sas50_rafq_201803_10_775_011684498_2
Workunit	11684498
Created	11 Dec 2018, 22:13:04 UTC
Sent	11 Dec 2018, 22:13:22 UTC
Report deadline	24 Nov 2019, 3:33:22 UTC
Received	12 Dec 2018, 19:12:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID	1387653
Run time	14 hours 43 min 54 sec
CPU time	14 hours 35 min 34 sec
Validate state	Invalid
Credit	1,539.60
Device peak FLOPS	3.69 GFLOPS
Application version	Weather At Home 2 (wah2) v8.24 windows_intelx86
Peak working set size	234.38 MB
Peak swap size	198.14 MB
Peak disk usage	47.69 MB
Stderr	<core_client_version>7.14.2</core_client_version> <![CDATA[ <message> finish file present too long</message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12724, selfPID=8212, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_ain::Monitor... 19:10:03 (8212): called boinc_finish(0) CPDN Monitor - Abort request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x026AF12C Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 12/12/18 19:15:16 Install Directory : Data Directory : F:\Data Boinc Project Symstore : LoadLibraryA( F:\Data Boinc\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( F:\Data Boinc\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( F:\Data Boinc\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( F:\Data Boinc\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: F:\Data Boinc\projects\climateprediction.net;F:\Data Boinc\projects\climateprediction.net ModLoad: 0000000000400000 0000000000152000 F:\Data Boinc\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077110000 0000000000190000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.471) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.17134.464 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.17134.464 ModLoad: 0000000075740000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.17134.376 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.17134.376 ModLoad: 0000000075fe0000 00000000001e4000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.441) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.17134.376 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.17134.376 ModLoad: 0000000076230000 000000000018d000 C:\WINDOWS\System32\USER32.dll (6.2.17134.376) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.17134.464 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.17134.464 ModLoad: 0000000073c50000 0000000000017000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000075410000 0000000000022000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.285) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.17134.285 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.285 ModLoad: 00000000766c0000 0000000000164000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.471) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.17134.471 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.471 ModLoad: 0000000076f90000 000000000007d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 00000000765a0000 000000000011e000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.17134.319 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.319 ModLoad: 0000000075390000 0000000000078000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.471) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.17134.464 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.17134.464 ModLoad: 0000000075210000 00000000000bf000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.17134.1 ModLoad: 00000000761d0000 0000000000044000 C:\WINDOWS\System32\sechost.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000075970000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.471) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.17134.464 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.17134.464 ModLoad: 00000000739d0000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.17134.376) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.17134.376 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.376 ModLoad: 00000000739c0000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000075510000 0000000000058000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.471) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.17134.471 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.471 ModLoad: 00000000755d0000 0000000000026000 C:\WINDOWS\System32\IMM32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 000000006e2a0000 0000000000182000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000072d50000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 1023, Write: 0, Other 10446 - I/O Transfers Counters - Read: 0, Write: 1681, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 91944, QuotaPeakPagedPoolUsage: 92104 QuotaNonPagedPoolUsage: 8760, QuotaPeakNonPagedPoolUsage: 9048 - Virtual Memory Usage - VirtualSize: 56377344, PeakVirtualSize: 56377344 - Pagefile Usage - PagefileUsage: 3588096, PeakPagefileUsage: 3588096 - Working Set Size - WorkingSetSize: 6123520, PeakWorkingSetSize: 6123520, PageFaultCount: 1782 * Dump of thread ID 14724 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 937500.000000, User Time: 1875000.000000, Wait Time: 16916.000000 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x026AF12C - Registers - eax=00000065 ebx=004b63b9 ecx=0019dc08 edx=026af12c esi=0019d9c0 edi=0019dc38 eip=0048ee46 esp=0019d954 ebp=0019dbe4 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202 - Callstack - ChildEBP RetAddr Args to Child 0019dbe4 0048161c 0019dc08 004b63b8 00000000 0019dc38 wah2_8.24_windows_intelx86!+0x0 0019dc28 0044fa52 026af12c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 0019dc48 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 0019dc60 00485ca1 c0000005 0019dc94 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 0019ff80 75758484 00230000 75758460 220941f8 0019ffdc wah2_8.24_windows_intelx86!+0x0 0019ff94 77173ab8 00230000 6c8c9d0b 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0019ffdc 77173a88 ffffffff 7718f317 00000000 00000000 ntdll!RtlAreBitsSet+0x0 0019ffec 00000000 00485cce 00230000 00000000 78746341 ntdll!RtlAreBitsSet+0x0 * Dump of thread ID 15468 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 15664.000000 - Registers - eax=00000000 ebx=00814a18 ecx=00000000 edx=00000000 esi=00814858 edi=00813768 eip=7717c58c esp=0080fdc8 ebp=0080ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0080ff80 75758484 00813768 75758460 229041f8 0080ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 0080ff94 77173ab8 00813768 6c159d0b 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0080ffdc 77173a88 ffffffff 7718f317 00000000 00000000 ntdll!RtlAreBitsSet+0x0 0080ffec 00000000 77161440 00813768 00000000 0c90a6db ntdll!RtlAreBitsSet+0x0 * Dump of thread ID 14472 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 15664.000000 - Registers - eax=00000000 ebx=00815808 ecx=00000000 edx=00000000 esi=00815648 edi=00813768 eip=7717c58c esp=00a0fdc8 ebp=00a0ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 00a0ff80 75758484 00813768 75758460 22b041f8 00a0ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 00a0ff94 77173ab8 00813768 6c359d0b 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 00a0ffdc 77173a88 ffffffff 7718f317 00000000 00000000 ntdll!RtlAreBitsSet+0x0 00a0ffec 00000000 77161440 00813768 00000000 00000000 ntdll!RtlAreBitsSet+0x0 * Dump of thread ID 14516 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 15664.000000 - Registers - eax=00000000 ebx=00815cb0 ecx=00000000 edx=00000000 esi=00815af0 edi=00813768 eip=7717c58c esp=00b0fdc8 ebp=00b0ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 00b0ff80 75758484 00813768 75758460 22a041f8 00b0ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 00b0ff94 77173ab8 00813768 6c259d0b 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 00b0ffdc 77173a88 ffffffff 7718f317 00000000 00000000 ntdll!RtlAreBitsSet+0x0 00b0ffec 00000000 77161440 00813768 00000000 00000000 ntdll!RtlAreBitsSet+0x0 * Dump of thread ID 7176 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 16915.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=7717ac1c esp=025dfcdc ebp=025dfd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 025dfd40 760e70ef 00000064 00000000 025dff80 00478b44 ntdll!NtDelayExecution+0x0 025dfd50 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 025dff80 75758484 00000000 75758460 204d41f8 025dffdc wah2_8.24_windows_intelx86!+0x0 025dff94 77173ab8 00000000 6ec89d0b 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 025dffdc 77173a88 ffffffff 7718f317 00000000 00000000 ntdll!RtlAreBitsSet+0x0 025dffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlAreBitsSet+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Dec 2018 14:29:39	1387653	21416126	wah2_sas50_rafq_201803_10_775_011684498_2	23,339	46,723	2.0019
12 Dec 2018 06:30:00	1387653	21416126	wah2_sas50_rafq_201803_10_775_011684498_2	11,819	23,546	1.9922