Software Diagnostics Library

A page to reference all different kinds of collection-related analysis patterns is necessary, so I created this post:

• Stack Trace Collection (unmanaged space)
• Stack Trace Collection (managed space)
• Stack Trace Collection (predicate)
• Stack Trace Collection (I/O requests)
• Stack Trace Collection (CPUs)
• Last Error Collection
• Module Collection
• Module Collection (predicate)
• Structure Field Collection
• Rough Stack Trace Collection (unmanaged space)
• Exception Collection
• Interrupt Stack Collection
• DPC Stack Collection

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When analyzing Execution Residue we are interested in valid symbolic references, for example, valid return addresses for Rough Stack Trace and Past Stack Trace reconstruction. Some of them may be Coincidental Symbolic Information that we check with backward disassembly. However, some symbolic references may be Function Pointers if forward disassembly starts with valid function prologue. Some references may also point to the start of exception processing in the middle of a normal function body. These two variants are illustrated with the following two addresses taken from the raw stack data when debugging an x64 Linux process in WSL using the latest version of WinDbg:

0:000> u 00007f73`e45820b0
libgcc_s_so!Unwind_Backtrace+0x7c0:
00007f73`e45820b0 push r15
00007f73`e45820b2 push r14
00007f73`e45820b4 push r13
00007f73`e45820b6 push r12
00007f73`e45820b8 push rbp
00007f73`e45820b9 push rbx
00007f73`e45820ba sub rsp,58h
00007f73`e45820be mov ecx,dword ptr [rdx+28h]

0:000> u 00005564`99d582c8
ud5bv3!start_modeling+0xd7:
00005564`99d582c8 mov rdi,rax
00005564`99d582cb call ud5bv3!_cxa_begin_catch$plt (00005564`99d58040)
00005564`99d582d0 call ud5bv3!_cxa_end_catch$plt (00005564`99d580b0)
00005564`99d582d5 jmp ud5bv3!start_modeling+0x64 (00005564`99d58255)
00005564`99d582da leave
00005564`99d582db ret
ud5bv3!new_feature:
00005564`99d582dc push rbp
00005564`99d582dd mov rbp,rsp

0:000> ub 00005564`99d582c8
ud5bv3!start_modeling+0xb5:
00005564`99d582a6 mov rdi,rax
00005564`99d582a9 call ud5bv3!sem_close$plt (00005564`99d580a0)
00005564`99d582ae mov rax,qword ptr [rbp-8]
00005564`99d582b2 mov rdi,rax
00005564`99d582b5 call ud5bv3!sem_close$plt (00005564`99d580a0)
00005564`99d582ba lea rdi,[ud5bv3!IO_stdin_used+0x4 (00005564`99d59004)]
00005564`99d582c1 call ud5bv3!sem_unlink$plt (00005564`99d580c0)
00005564`99d582c6 jmp ud5bv3!start_modeling+0xe9 (00005564`99d582da)

As we see, some Function Pointers may have symbolic name plus some offset if they don’t have associated symbols.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This analysis pattern is a Linux variant of the previous Not My Version Windows crash dump analysis pattern. In case of segmentation faults/core dumps with Stack Traces involving OS and 3rd-party shared libraries it is worth noting directories they come from:

(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x00007fad170d5950 0x00007fad170e2dc8 Yes (*) /lib64/lib3rdparty.so.0
0x00007fad170c3130 0x00007fad170c3eb5 Yes /lib/x86_64-linux-gnu/libdl.so.2
0x00007fad16f24320 0x00007fad1706a14b Yes /lib/x86_64-linux-gnu/libc.so.6
0x00007fad17103090 0x00007fad17120b50 Yes /lib64/ld-linux-x86-64.so.2
0x00007fad16bfd300 0x00007fad16c03578 Yes /lib/x86_64-linux-gnu/libnss_files.so.2

If you use WinDbg to analyze Linux core dumps, you can use lmv command or its variant. It is worth noting that referenced libraries can be symbolic links to different versions than you expect. Unfortunately, it is not so easy to extract library timestamps, so it is recommended to additionally use Paratext analysis pattern to supply such information.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

A page to reference all different kinds of region-related analysis patterns is necessary, so I created this post:

• Hidden Exception (user space)
• Hidden Exception (kernel space)
• Hidden Exception (managed space)
• Rough Stack Trace (unmanaged space)
• Rough Stack Trace (managed space)
• Region Profile
• Region Clusters
• Region Summary
• Region Spectrum
• Normalized Region

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Modern x64 Windows targets may support hardware shadow stacks. In such a case, WinDbg shows this message even if you open a memory dump on computers that do not support it:

This target supports Hardware-enforced Stack Protection. A HW based
"Shadow Stack" may be available to assist in debugging and analysis.
See aka.ms/userhsp for more info.
dps @ssp

The data from shadow stacks may be useful in case of Local Buffer Overflow. In such a case, we can compare the problem Stack Trace with the Shadow Stack Trace that was supposed to be without the stack region corruption.

For example, if see this exception and Incorrect Stack Trace, we can see that the stack trace should have been if the the return address were not modified:

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(5a34.4bb8): Security check failure or stack buffer overrun - code c0000409 (first/second chance not available)
Subcode: 0×39 FAST_FAIL_CONTROL_INVALID_RETURN_ADDRESS Shadow stack violation

0:000> k
# Child-SP RetAddr Call Site
00 000000fa`b94ffdf8 000002aa`5d420588 user32!GetMessageW+0×5c
01 000000fa`b94ffe00 000002aa`5d420588 0×000002aa`5d420588
02 000000fa`b94ffe08 00000000`00000000 0×000002aa`5d420588

0:000> r
rax=0000000000000001 rbx=000002aa5d420588 rcx=00007ff9c3d31534
rdx=0000000000000000 rsi=0000000000000000 rdi=000002aa5d420530
rip=00007ff9c3ea538c rsp=000000fab94ffdf8 rbp=000002aa5d420588
r8=000000fab94ffd98 r9=0000000000000000 r10=0000000000000000
r11=0000000000000244 r12=00007ff66b204070 r13=0000000000000000
r14=0000000000000001 r15=00000000ffffffff
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
user32!GetMessageW+0x5c:
00007ff9`c3ea538c ret

0:000> dps @rsp L1
000000fa`b94ffdf8 000002aa`5d420588

0:000> dps @ssp
000000fa`b95fefd0 00007ff9`4ae2f877 mfc140u!AfxInternalPumpMessage+0x27
000000fa`b95fefd8 00007ff9`4ae301b1 mfc140u!CWinThread::Run+0x81
000000fa`b95fefe0 00007ff9`4ae63230 mfc140u!AfxWinMain+0xc0
000000fa`b95fefe8 00007ff6`6b135742 mspaint+0xc5742
000000fa`b95feff0 00007ff9`c500257d kernel32!BaseThreadInitThunk+0x1d
000000fa`b95feff8 00007ff9`c618aa58 ntdll!RtlUserThreadStart+0x28
000000fa`b95ff000 ????????`????????
000000fa`b95ff008 ????????`????????
000000fa`b95ff010 ????????`????????
000000fa`b95ff018 ????????`????????
000000fa`b95ff020 ????????`????????
000000fa`b95ff028 ????????`????????
000000fa`b95ff030 ????????`????????
000000fa`b95ff038 ????????`????????
000000fa`b95ff040 ????????`????????
000000fa`b95ff048 ????????`????????

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This is an unmanaged code analysis pattern variant of the previously published Annotated Disassembly. In modern WinDbg (which was previously called WinDbg Preview), the Disassembly window may annotate local variables in the presence of debugging symbols (this is absent from the output of the uf WinDbg command):

; uf command output
511 00007ff6`6ab22a44 mov dword ptr [rbp+2078h],1
511 00007ff6`6ab22a4e mov dword ptr [rbp+207Ch],2
513 00007ff6`6ab22a58 mov eax,dword ptr [rbp+2078h]
513 00007ff6`6ab22a5e mov dword ptr [rbp+0Ch],eax
514 00007ff6`6ab22a61 mov dword ptr [rbp+0Ch],64h
515 00007ff6`6ab22a68 mov dword ptr [rbp+48h],3
515 00007ff6`6ab22a6f mov dword ptr [rbp+4Ch],4
516 00007ff6`6ab22a76 mov eax,dword ptr [rbp+0Ch]


; Disassembly window
00007ff6`6ab22a4e c7857c20000002000000 mov dword ptr [myDerived.field2 (rbp+207Ch)], 2
00007ff6`6ab22a58 8b8578200000 mov eax, dword ptr [myDerived{.field} (rbp+2078h)]
00007ff6`6ab22a5e 89450c mov dword ptr [myBase{.field} (rbp+Ch)], eax
00007ff6`6ab22a61 c7450c64000000 mov dword ptr [myBase{.field} (rbp+Ch)], 64h
00007ff6`6ab22a68 c7454803000000 mov dword ptr [myDerived2{.field} (rbp+48h)], 3
00007ff6`6ab22a6f c7454c04000000 mov dword ptr [myDerived2.field2 (rbp+4Ch)], 4
00007ff6`6ab22a76 8b450c mov eax, dword ptr [myBase{.field} (rbp+Ch)]


- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes, when we have debugging symbols, information about local variables may be helpful in making sense of function disassembly. For example, we have this code fragment from WinDbg uf command:

511 00007ff6`6ab22a44 mov dword ptr [rbp+2078h],1
511 00007ff6`6ab22a4e mov dword ptr [rbp+207Ch],2
513 00007ff6`6ab22a58 mov eax,dword ptr [rbp+2078h]
513 00007ff6`6ab22a5e mov dword ptr [rbp+0Ch],eax
514 00007ff6`6ab22a61 mov dword ptr [rbp+0Ch],64h
515 00007ff6`6ab22a68 mov dword ptr [rbp+48h],3
515 00007ff6`6ab22a6f mov dword ptr [rbp+4Ch],4
516 00007ff6`6ab22a76 mov eax,dword ptr [rbp+0Ch]

Although source code lines are shown, suppose we don’t have source code to match. However, we can match Address Representations, such as [rbp+xxx], from the output of dv /V WinDbg command:

0:000> dv /V
...
000000ab`740fd00c @rbp+0x000c myBase = struct wmain::__l2::Base
...
000000ab`740ff078 @rbp+0x2078 myDerived = struct wmain::__l2::Derived
...
000000ab`740fd048 @rbp+0x0048 myDerived2 = struct wmain::__l2::Derived
...

Another usage is matching values in raw stack data with local variable addresses. Values as addresses and their symbolic representations here have some connection to ADDR Symbolic and Interpreted Pointers.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Stack Overflow caused by managed code is manifested as Stack Overflow (User Mode) with JIT Code recursive entries. !CLRStack WinDbg SOS extension command may work for very long if stack frame are small so we may need to increase the number of frames to show (.kframes command) and then manually check the originating frames using !IP2MD SOS extension command.

0:000> !CLRStack
OS Thread Id: 0x1da0 (0)
Child SP IP Call Site
000000F83D205FE0 00007ffc82570539 UserQuery.g__foo|4_1()
000000F83D206010 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206040 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206070 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D2060A0 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D2060D0 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206100 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206130 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206160 00007ffc8257053e UserQuery.g__foo|4_1()
[...]

0:000> .kframes 0xFFFF
Default stack trace depth is 0n65535 frames

0:000> kL
# Child-SP RetAddr Call Site
00 000000f8`3d205fe0 00007ffc`8257053e 0x00007ffc`82570539
01 000000f8`3d206010 00007ffc`8257053e 0x00007ffc`8257053e
02 000000f8`3d206040 00007ffc`8257053e 0x00007ffc`8257053e
03 000000f8`3d206070 00007ffc`8257053e 0x00007ffc`8257053e
04 000000f8`3d2060a0 00007ffc`8257053e 0x00007ffc`8257053e
05 000000f8`3d2060d0 00007ffc`8257053e 0x00007ffc`8257053e
06 000000f8`3d206100 00007ffc`8257053e 0x00007ffc`8257053e
07 000000f8`3d206130 00007ffc`8257053e 0x00007ffc`8257053e
08 000000f8`3d206160 00007ffc`8257053e 0x00007ffc`8257053e
09 000000f8`3d206190 00007ffc`8257053e 0x00007ffc`8257053e
[...]
7cfa 000000f8`3d37cec0 00007ffc`8257053e 0x00007ffc`8257053e
7cfb 000000f8`3d37cef0 00007ffc`8257053e 0x00007ffc`8257053e
7cfc 000000f8`3d37cf20 00007ffc`8257053e 0x00007ffc`8257053e
7cfd 000000f8`3d37cf50 00007ffc`8257053e 0x00007ffc`8257053e
7cfe 000000f8`3d37cf80 00007ffc`8257053e 0x00007ffc`8257053e
7cff 000000f8`3d37cfb0 00007ffc`8257053e 0x00007ffc`8257053e
7d00 000000f8`3d37cfe0 00007ffc`8257053e 0x00007ffc`8257053e
7d01 000000f8`3d37d010 00007ffc`825704fe 0×00007ffc`8257053e
7d02 000000f8`3d37d040 00007ffc`825704c4 0×00007ffc`825704fe
7d03 000000f8`3d37d070 00007ffc`82582bdd 0×00007ffc`825704c4
7d04 000000f8`3d37d0a0 00007ffc`8236b45e 0×00007ffc`82582bdd
7d05 000000f8`3d37d940 00007ffc`82366850 0×00007ffc`8236b45e
7d06 000000f8`3d37dc10 00007ffc`82365faf 0×00007ffc`82366850
7d07 000000f8`3d37dd50 00007ffc`82365edc 0×00007ffc`82365faf
7d08 000000f8`3d37dd90 00007ffc`823316f5 0×00007ffc`82365edc
7d09 000000f8`3d37dde0 00007ffc`8233144b 0×00007ffc`823316f5
7d0a 000000f8`3d37de70 00007ffc`81de8db1 0×00007ffc`8233144b
7d0b 000000f8`3d37df60 00007ffc`81de59fa 0×00007ffc`81de8db1
7d0c 000000f8`3d37e0c0 00007ffc`81de5985 0×00007ffc`81de59fa
7d0d 000000f8`3d37e110 00007ffc`81de4d59 0×00007ffc`81de5985
7d0e 000000f8`3d37e160 00007ffc`81de45f5 0×00007ffc`81de4d59
7d0f 000000f8`3d37e1e0 00007ffc`e196a573 0×00007ffc`81de45f5
7d10 000000f8`3d37e220 00007ffc`e18902d0 coreclr!CallDescrWorkerInternal+0×83
7d11 (Inline Function) ——–`——– coreclr!CallDescrWorkerWithHandler+0×30
7d12 000000f8`3d37e260 00007ffc`e189202c coreclr!CallDescrWorkerReflectionWrapper+0×48
7d13 000000f8`3d37e2b0 00007ffc`d5ddc9d7 coreclr!RuntimeMethodHandle::InvokeMethod+0×91c
[…]
7d1b 000000f8`3d37ed60 00007ffc`e18e0d95 coreclr!RunMain+0xd2
7d1c 000000f8`3d37ee10 00007ffc`e18e0b56 coreclr!Assembly::ExecuteMainMethod+0×1c9
7d1d 000000f8`3d37f1a0 00007ffc`e19152b2 coreclr!CorHost2::ExecuteAssembly+0×1c6
7d1e 000000f8`3d37f310 00007ffd`053896bb coreclr!coreclr_execute_assembly+0xe2
7d1f (Inline Function) ——–`——– hostpolicy!coreclr_t::execute_assembly+0×2a
7d20 000000f8`3d37f3b0 00007ffd`053899ec hostpolicy!run_app_for_context+0×56b
7d21 000000f8`3d37f550 00007ffd`0538a387 hostpolicy!run_app+0×3c
7d22 000000f8`3d37f590 00007ffd`07fab539 hostpolicy!corehost_main+0×107
7d23 000000f8`3d37f740 00007ffd`07fae506 hostfxr!execute_app+0×2e9
7d24 000000f8`3d37f840 00007ffd`07fb0821 hostfxr!`anonymous namespace’::read_config_and_execute+0xa6
7d25 000000f8`3d37f940 00007ffd`07faeb62 hostfxr!fx_muxer_t::handle_exec_host_command+0×161
7d26 000000f8`3d37f9f0 00007ffd`07fa82ab hostfxr!fx_muxer_t::execute+0×482
7d27 000000f8`3d37fb30 00007ff6`64fe2351 hostfxr!hostfxr_main_startupinfo+0xab
7d28 000000f8`3d37fc30 00007ff6`64fe2748 LINQPad7_Query_exe!exe_start+0×651
7d29 000000f8`3d37fe60 00007ff6`64fe45f8 LINQPad7_Query_exe!wmain+0×88
7d2a (Inline Function) ——–`——– LINQPad7_Query_exe!invoke_main+0×22
7d2b 000000f8`3d37fe90 00007ffd`164b54e0 LINQPad7_Query_exe!__scrt_common_main_seh+0×10c
7d2c 000000f8`3d37fed0 00007ffd`185e485b kernel32!BaseThreadInitThunk+0×10
7d2d 000000f8`3d37ff00 00000000`00000000 ntdll!RtlUserThreadStart+0×2b

0:000> !IP2MD 0×00007ffc`8257053e
MethodDesc: 00007ffc8257ce18
Method Name: UserQuery.<Main>g__foo|4_1()
Class: 00007ffc8257cd08
MethodTable: 00007ffc8257ce48
mdToken: 0000000006000007
Module: 00007ffc8257c060
IsJitted: yes
Current CodeAddr: 00007ffc82570520
Version History:
ILCodeVersion: 0000000000000000
ReJIT ID: 0
IL Addr: 0000027575cc20f2
CodeAddr: 00007ffc82570520 (MinOptJitted)
NativeCodeVersion: 0000000000000000

0:000> !DumpIL 00007ffc8257ce18
ilAddr is 0000027575CC20F2 pImport is 000001C7B44109C0
ilAddr = 0000027575CC20F2
IL_0000: nop
IL_0001: call void UserQuery::<Main>g__foo|4_1()
IL_0006: nop
IL_0007: ret

0:000> !IP2MD 0×00007ffc`825704fe
MethodDesc: 00007ffc8257ce00
Method Name: UserQuery.<Main>g__bar|4_0()
Class: 00007ffc8257cd08
MethodTable: 00007ffc8257ce48
mdToken: 0000000006000006
Module: 00007ffc8257c060
IsJitted: yes
Current CodeAddr: 00007ffc825704e0
Version History:
ILCodeVersion: 0000000000000000
ReJIT ID: 0
IL Addr: 0000027575cc20f2
CodeAddr: 00007ffc825704e0 (MinOptJitted)
NativeCodeVersion: 0000000000000000

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

A page to reference all different kinds of CPU consumption analysis patterns is necessary, so I created this post:

• Spiking Thread
• Distributed Spike
• Spike Interval
• Spiking Interrupts

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

I resume online training sessions. You can now register: https://www.patterndiagnostics.com/accelerated-windows-memory-dump-analysis

Usually, when we find an interesting message in a log (maybe also a frame from Exception Stack Trace), especially from an unfamiliar component, we also want to search past problem cases either on the Internet or in some internal database. However, we just put the message as is or some small fragment of it we may get a lot of noise results. The problem is to find the optimal Message Essence. Often, this is done by omitting variable data (including Adjoint Thread of Activity fields) but leaving Message Invariants and Trace Constants usually refine a diagnostic error:

This analysis pattern is different from Message Invariant. where the latter is useful when finding its emitter’s source code lines (PLOT).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

COM Exceptions are Software Exceptions and their information can be extracted from C++ Exception record as shown in this post. Here we show the case of Nested and Hidden Exceptions.

We see a COM exception raising function on Exception Stack Trace:

0:008> .exr -1
ExceptionAddress: 00007ff97800cadf (ntdll!LdrpICallHandler+0x000000000000000f)
ExceptionCode: c0000409 (Security check failure or stack buffer overrun)
ExceptionFlags: 00000001
NumberParameters: 1
Parameter[0]: 000000000000000a
Subcode: 0xa FAST_FAIL_GUARD_ICALL_CHECK_FAILURE

0:008> kL
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP          RetAddr           Call Site
00 0000009e`393f9e78 00007ff9`7802184f ntdll!LdrpICallHandler+0xf
01 0000009e`393f9e80 00007ff9`77fea889 ntdll!RtlpExecuteHandlerForException+0xf
02 0000009e`393f9eb0 00007ff9`780204be ntdll!RtlDispatchException+0x219
03 0000009e`393fa5c0 00007ff9`7800cb9e ntdll!KiUserExceptionDispatch+0x2e
04 0000009e`393fad78 00007ff9`72591030 ntdll!LdrpDispatchUserCallTarget+0xe
05 0000009e`393fad80 00007ff9`72594a52 VCRUNTIME140_APP!_CallSettingFrame+0x20
06 0000009e`393fadb0 00007ff9`7259e514 VCRUNTIME140_APP!__FrameHandler3::FrameUnwindToState+0x112
07 0000009e`393fae20 00007ff9`72593cc8 VCRUNTIME140_APP!__FrameHandler3::FrameUnwindToEmptyState+0x54
08 0000009e`393fae50 00007ff9`7259ee51 VCRUNTIME140_APP!__InternalCxxFrameHandler<__FrameHandler3>+0x10c
09 0000009e`393faeb0 00007ff8`f83ea850 VCRUNTIME140_APP!__CxxFrameHandler3+0x71
0a 0000009e`393faf00 00007ff9`780218cf PaintStudio_ViewModel!DllGetActivationFactory+0x100
0b 0000009e`393faf30 00007ff9`77f9d9b2 ntdll!RtlpExecuteHandlerForUnwind+0xf
0c 0000009e`393faf60 00007ff9`7259e9de ntdll!RtlUnwindEx+0x522
0d 0000009e`393fb670 00007ff9`72592955 VCRUNTIME140_APP!__FrameHandler3::UnwindNestedFrames+0xee
0e 0000009e`393fb760 00007ff9`72592d81 VCRUNTIME140_APP!CatchIt<__FrameHandler3>+0xb9
0f 0000009e`393fb800 00007ff9`72593dc4 VCRUNTIME140_APP!FindHandler<__FrameHandler3>+0x33d
10 0000009e`393fb970 00007ff9`7259ee51 VCRUNTIME140_APP!__InternalCxxFrameHandler<__FrameHandler3>+0x208
11 0000009e`393fb9d0 00007ff9`7802184f VCRUNTIME140_APP!__CxxFrameHandler3+0x71
12 0000009e`393fba20 00007ff9`77fea889 ntdll!RtlpExecuteHandlerForException+0xf
13 0000009e`393fba50 00007ff9`77fea643 ntdll!RtlDispatchException+0x219
14 0000009e`393fc160 00007ff9`759d3b29 ntdll!RtlRaiseException+0×153
15 0000009e`393fc9d0 00007ff9`72596220 KERNELBASE!RaiseException+0×69
16 0000009e`393fcab0 00007ff9`4919a58c VCRUNTIME140_APP!_CxxThrowException+0×90
17 0000009e`393fcb10 00007ff8`f8057628 vccorlib140_app!__abi_WinRTraiseCOMException+0×2c
18 0000009e`393fcb40 00007ff8`f8093e81 PaintStudio_ViewModel+0×7628
19 0000009e`393fcb70 00007ff8`f818f27f PaintStudio_ViewModel+0×43e81
1a 0000009e`393fcbc0 00007ff8`f818c26f PaintStudio_ViewModel+0×13f27f
1b 0000009e`393fcc90 00007ff8`f811935a PaintStudio_ViewModel+0×13c26f
1c 0000009e`393fcd40 00007ff8`f827ce8e PaintStudio_ViewModel+0xc935a
1d 0000009e`393fd110 00007ff8`f82723ab PaintStudio_ViewModel+0×22ce8e
1e 0000009e`393fd5c0 00007ff8`f83bf09d PaintStudio_ViewModel+0×2223ab
1f 0000009e`393fd7b0 00007ff8`f83c16bd PaintStudio_ViewModel+0×36f09d
20 0000009e`393fdc60 00007ff8`f80e1331 PaintStudio_ViewModel+0×3716bd
21 0000009e`393fdd10 00007ff7`2030d3b9 PaintStudio_ViewModel+0×91331
22 0000009e`393fdd50 00007ff7`202f772f PaintStudio_View+0×2d3b9
23 0000009e`393fddb0 00007ff7`202f702b PaintStudio_View+0×1772f
24 0000009e`393fdee0 00007ff7`202f520e PaintStudio_View+0×1702b
25 0000009e`393fe010 00007ff7`203266d6 PaintStudio_View+0×1520e
26 0000009e`393fe100 00007ff9`4af9d25b PaintStudio_View+0×466d6
27 0000009e`393fe140 00007ff9`4af9d1ce Windows_UI_Xaml!DirectUI::FrameworkApplicationGenerated:: OnActivatedProtected+0×4b
28 0000009e`393fe170 00007ff9`4af9ebe6 Windows_UI_Xaml!DirectUI::FrameworkApplication::DispatchGenericActivation+0×4a
29 0000009e`393fe1a0 00007ff9`4aeb39eb Windows_UI_Xaml!DirectUI::FrameworkView::OnActivated+0×186
2a (Inline Function) ——–`——– Windows_UI_Xaml!Microsoft::WRL::Callback::__l2::<lambda_772c64e6f5ddba6f719dbbabda2a0901>::operator()+0×15
2b 0000009e`393fe220 00007ff9`72cd55cf Windows_UI_Xaml!Microsoft::WRL::Details::DelegateArgTraits<long (__cdecl Windows::Foundation:: ITypedEventHandler_impl<Windows::Foundation::Internal:: AggregateType<Windows::UI::Core::CoreWindow *,Windows::UI::Core::ICoreWindow *>,IInspectable *>::*)(Windows::UI::Core::ICoreWindow *,IInspectable *)>::DelegateInvokeHelper<Windows::Foundation:: ITypedEventHandler<Windows::UI::Core::CoreWindow *,IInspectable *>,<lambda_772c64e6f5ddba6f719dbbabda2a0901>,-1,Windows::UI::Core::ICoreWindow *,IInspectable *>::Invoke+0×1b
2c 0000009e`393fe250 00007ff9`72cd8a22 twinapi_appcore!Microsoft::WRL::InvokeTraits<-2>:: InvokeDelegates<<lambda_3ad0adb09957fd62cbc86618ebbeb8fa>,Windows::Foundation:: ITypedEventHandler<Windows::ApplicationModel::Core::CoreApplicationView *,Windows::ApplicationModel::Activation::IActivatedEventArgs *> >+0×67
2d 0000009e`393fe2c0 00007ff9`76cb6a63 twinapi_appcore!Windows::ApplicationModel::Core:: CoreApplicationView::Activate+0×3d2
2e 0000009e`393fe430 00007ff9`76d1a036 rpcrt4!Invoke+0×73
2f 0000009e`393fe490 00007ff9`76c783b9 rpcrt4!Ndr64StubWorker+0xb56
30 0000009e`393feb30 00007ff9`76fd5d13 rpcrt4!NdrStubCall3+0xc9
31 0000009e`393feb90 00007ff9`76c99bab combase!CStdStubBuffer_Invoke+0×73
32 0000009e`393febd0 00007ff9`76fbd0e3 rpcrt4!CStdStubBuffer_Invoke+0×3b
33 (Inline Function) ——–`——– combase!InvokeStubWithExceptionPolicyAndTracing::__l6:: <lambda_c9f3956a20c9da92a64affc24fdd69ec>::operator()+0×18
34 0000009e`393fec00 00007ff9`76fbced3 combase!ObjectMethodExceptionHandlingAction< <lambda_c9f3956a20c9da92a64affc24fdd69ec> >+0×43
35 (Inline Function) ——–`——– combase!InvokeStubWithExceptionPolicyAndTracing+0xa8
36 0000009e`393fec60 00007ff9`76fd9556 combase!DefaultStubInvoke+0×1c3
37 (Inline Function) ——–`——– combase!SyncStubCall::Invoke+0×22
38 0000009e`393fedb0 00007ff9`76fba4fa combase!SyncServerCall::StubInvoke+0×26
39 (Inline Function) ——–`——– combase!StubInvoke+0×259
3a 0000009e`393fedf0 00007ff9`76fda81b combase!ServerCall::ContextInvoke+0×42a
3b (Inline Function) ——–`——– combase!CServerChannel::ContextInvoke+0xc0
3c (Inline Function) ——–`——– combase!DefaultInvokeInApartment+0xc0
3d 0000009e`393ff1f0 00007ff9`76f701ac combase!ASTAInvokeInApartment+0×15b
3e 0000009e`393ff400 00007ff9`76f70a11 combase!AppInvoke+0×1ec
3f 0000009e`393ff490 00007ff9`76f918c2 combase!ComInvokeWithLockAndIPID+0×681
40 (Inline Function) ——–`——– combase!ComInvoke+0×1c1
41 0000009e`393ff7c0 00007ff9`76f90a99 combase!ThreadDispatch+0×272
42 0000009e`393ff890 00007ff9`76f947ba combase!ModernSTAState::HandleMessage+0×51
43 0000009e`393ff8e0 00007ff9`4eac92f5 combase!ModernSTAWaitContext::HandlePriorityEventsFromMessagePump+0×66
44 0000009e`393ff910 00007ff9`4eac8fee Windows_UI!Windows::UI::Core::CDispatcher::ProcessMessage+0×1b5
45 0000009e`393ff9c0 00007ff9`4eac8f21 Windows_UI!Windows::UI::Core::CDispatcher::WaitAndProcessMessagesInternal+0xae
46 0000009e`393ffad0 00007ff9`72cea89f Windows_UI!Windows::UI::Core::CDispatcher::WaitAndProcessMessages+0×31
47 0000009e`393ffb00 00007ff9`76eac235 twinapi_appcore!<lambda_643db08282a766b00cec20194396f531>::operator()+0xff
48 0000009e`393ffbf0 00007ff9`77aa7c24 SHCore!_WrapperThreadProc+0xf5
49 0000009e`393ffcd0 00007ff9`77fed4d1 kernel32!BaseThreadInitThunk+0×14
4a 0000009e`393ffd00 00000000`00000000 ntdll!RtlUserThreadStart+0×21

We dump doubly dereferenced raw stack region around such exception processing calls:

0:008> dpp 0000009e`393fc160 0000009e`393fcb70
[…]
0000009e`393fcb38 00007ff8`f8057628 cc003f4c`6115ffcc
0000009e`393fcb40 0000009e`393fcb88 0000009e`393fcb98
0000009e`393fcb48 000001e8`69af9450 00007ff9`491c6170 vccorlib140_app!Platform::COMException::`vftable’
0000009e`393fcb50 000001e8`69af9450 00007ff9`491c6170 vccorlib140_app!Platform::COMException::`vftable’
[…]

We see C++ Object references and apply object structure to them:

0:008> dt vccorlib140_app!Platform::COMException 000001e8`69af9450
+0×000 __VFN_table : 0×00007ff9`491c6170
+0×008 __VFN_table : 0×00007ff9`491c5bf8
+0×010 __VFN_table : 0×00007ff9`491c5e20
+0×018 __VFN_table : 0×00007ff9`491c5ec0
+0×020 __description    : 0×000001e8`5e1e30a8 Void
+0×028 __restrictedErrorString : 0×000001e8`5ba83728 Void
+0×030 __restrictedErrorReference : (null)
+0×038 __capabilitySid  : (null)
+0×040 __hresult        : 0n-2147024894
+0×048 __restrictedInfo : 0×000001e8`699f4308 Void
+0×050 __throwInfo      : 0×00007ff9`491baf60 Void
+0×058 __size           : 0×40
+0×060 __prepare        : Platform::IntPtr
+0×068 __abi_reference_count : __abi_FTMWeakRefData
+0×078 __abi_disposed   : 0
+0×080 __abi_disposed   : 0

0:008> du 0x000001e8`5e1e30a8
000001e8`5e1e30a8  "The system cannot find the file "
000001e8`5e1e30e8  "specified..."

0:008> du 0x000001e8`5ba83728
000001e8`5ba83728  "Error trying to initialize appli"
000001e8`5ba83768  "cation data storage folder"

0:008> !error 0n-2147024894
Error code: (HRESULT) 0x80070002 (2147942402) - The system cannot find the file specified.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

C++ Objects may leave virtual function table pointer traces in Execution Residue and, therefore, their adjacent data can be inspected:

0:000> !teb
TEB at 0000000000306000
ExceptionList: 0000000000000000
StackBase: 0000000000150000
StackLimit: 000000000014d000
SubSystemTib: 0000000000000000
FiberData: 0000000000001e00
ArbitraryUserPointer: 0000000000000000
Self: 0000000000306000
EnvironmentPointer: 0000000000000000
ClientId: 0000000000000214 . 00000000000011b0
RpcHandle: 0000000000000000
Tls Storage: 0000000000306058
PEB Address: 0000000000305000
LastErrorValue: 0
LastStatusValue: c0000034
Count Owned Locks: 0
HardErrorMode: 0

0:000> dps 000000000014d000 0000000000150000
00000000`0014d000 00000000`00000000
00000000`0014d008 00000000`00000000
00000000`0014d010 00000000`00000000
00000000`0014d018 00000000`00000000
00000000`0014d020 00000000`00000000
[…]
00000000`0014fe08 00000000`00000000
00000000`0014fe10 00000000`005d4550
00000000`0014fe18 00000000`00000000
00000000`0014fe20 00000000`005cd7e0
00000000`0014fe28 00000000`005cd7e0
00000000`0014fe30 00000000`005cd7e0
00000000`0014fe38 00000001`40017778 ExecutionResidueC__Objects!CObject::`vftable’
00000000`0014fe40 624f206f`6c6c6548
00000000`0014fe48 00000021`7463656a
00000000`0014fe50 00000000`00000000
00000000`0014fe58 00000000`00000000
00000000`0014fe60 00000000`00000000
00000000`0014fe68 00000000`00000000
00000000`0014fe70 00000000`00000000
00000000`0014fe78 00000000`00000000
00000000`0014fe80 00000000`00000000
00000000`0014fe88 00000000`00000000
00000000`0014fe90 00000000`00000000
00000000`0014fe98 00000000`00000000
00000000`0014fea0 00000000`00000000
00000000`0014fea8 00000000`00000000
00000000`0014feb0 00000001`40017778 ExecutionResidueC__Objects!CObject::`vftable’
00000000`0014feb8 624f206f`6c6c6548
00000000`0014fec0 00000021`7463656a
00000000`0014fec8 00000000`00000000
00000000`0014fed0 00000000`00000000
00000000`0014fed8 0000e111`9d4d4b61
[…]

0:000> dps 00000001`40017778
00000001`40017778 00000001`40001040 ExecutionResidueC__Objects!CObject::`scalar deleting destructor’
00000001`40017780 00000001`40001020 ExecutionResidueC__Objects!CObject::foo
00000001`40017788 00000001`40001030 ExecutionResidueC__Objects!CObject::bar
00000001`40017790 600e149f`00000000
00000001`40017798 00000002`00000000
00000001`400177a0 00017c6c`00000069
00000001`400177a8 00000000`00016e6c
00000001`400177b0 00000000`600e149f
00000001`400177b8 00000014`0000000c
00000001`400177c0 00016ed8`00017cd8
00000001`400177c8 600e149f`00000000
00000001`400177d0 0000000d`00000000
00000001`400177d8 00017cec`000002f0
00000001`400177e0 00000000`00016eec
00000001`400177e8 00000000`600e149f
00000001`400177f0 00000000`0000000e

0:000> da 00000000`0014feb8
00000000`0014feb8 “Hello Object!”

0:000> dt ExecutionResidueC__Objects!CObject 00000000`0014feb0
+0×000 __VFN_table : 0×00000001`40017778
+0×008 data : [32] “Hello Object!”

We see that two objects were allocated on the stack. However, finding dynamically allocated objects may require another level of pointer redirection when pointers to such objects are stored on the stack, for example with dpp WinDbg command:

0:000> dpp 000000000014d000 0000000000150000
00000000`0014d000 00000000`00000000
00000000`0014d008 00000000`00000000
00000000`0014d010 00000000`00000000
00000000`0014d018 00000000`00000000
00000000`0014d020 00000000`00000000
[…]
00000000`0014fe08 00000000`00000000
00000000`0014fe10 00000000`005d4550 00000000`005d4560
00000000`0014fe18 00000000`00000000
00000000`0014fe20 00000000`005cd7e0 00000001`40017778 ExecutionResidueC__Objects!CObject::`vftable’
00000000`0014fe28 00000000`005cd7e0 00000001`40017778 ExecutionResidueC__Objects!CObject::`vftable’
00000000`0014fe30 00000000`005cd7e0 00000001`40017778 ExecutionResidueC__Objects!CObject::`vftable’
00000000`0014fe38 00000001`40017778 00000001`40001040 ExecutionResidueC__Objects!CObject::`scalar deleting destructor’
00000000`0014fe40 624f206f`6c6c6548
00000000`0014fe48 00000021`7463656a
00000000`0014fe50 00000000`00000000
00000000`0014fe58 00000000`00000000
00000000`0014fe60 00000000`00000000
00000000`0014fe68 00000000`00000000
00000000`0014fe70 00000000`00000000
00000000`0014fe78 00000000`00000000
00000000`0014fe80 00000000`00000000
00000000`0014fe88 00000000`00000000
00000000`0014fe90 00000000`00000000
00000000`0014fe98 00000000`00000000
00000000`0014fea0 00000000`00000000
00000000`0014fea8 00000000`00000000
00000000`0014feb0 00000001`40017778 00000001`40001040 ExecutionResidueC__Objects!CObject::`scalar deleting destructor’
00000000`0014feb8 624f206f`6c6c6548
00000000`0014fec0 00000021`7463656a
00000000`0014fec8 00000000`00000000
00000000`0014fed0 00000000`00000000
00000000`0014fed8 0000e111`9d4d4b61
[…]

0:000> !address 00000000`005cd7e0

Usage: Heap
Base Address: 00000000`005c0000
End Address: 00000000`005d8000
Region Size: 00000000`00018000 ( 96.000 kB)
State: 00001000 MEM_COMMIT
Protect: 00000004 PAGE_READWRITE
Type: 00020000 MEM_PRIVATE
Allocation Base: 00000000`005c0000
Allocation Protect: 00000004 PAGE_READWRITE
More info: heap owning the address: !heap 0×5c0000
More info: heap segment
More info: heap entry containing the address: !heap -x 0×5cd7e0

0:000> dps 00000000`005cd7e0
00000000`005cd7e0 00000001`40017778 ExecutionResidueC__Objects!CObject::`vftable’
00000000`005cd7e8 624f206f`6c6c6548
00000000`005cd7f0 00000021`7463656a
00000000`005cd7f8 00000000`00000000
00000000`005cd800 00000000`00000000
00000000`005cd808 93002500`6c5ec8a3
00000000`005cd810 4f535345`434f5250
00000000`005cd818 54494843`52415f52
00000000`005cd820 413d4552`55544345
00000000`005cd828 00000000`3436444d
00000000`005cd830 00000000`00000000
00000000`005cd838 92002600`6c5bc8a0
00000000`005cd840 576d6172`676f7250
00000000`005cd848 5c3a433d`32333436
00000000`005cd850 206d6172`676f7250
00000000`005cd858 00000073`656c6946

0:000> da 00000000`005cd7e8
00000000`005cd7e8 “Hello Object!”

0:000> dps 00000001`40017778
00000001`40017778 00000001`40001040 ExecutionResidueC__Objects!CObject::`scalar deleting destructor’
00000001`40017780 00000001`40001020 ExecutionResidueC__Objects!CObject::foo
00000001`40017788 00000001`40001030 ExecutionResidueC__Objects!CObject::bar
00000001`40017790 600e149f`00000000
00000001`40017798 00000002`00000000
00000001`400177a0 00017c6c`00000069
00000001`400177a8 00000000`00016e6c
00000001`400177b0 00000000`600e149f
00000001`400177b8 00000014`0000000c
00000001`400177c0 00016ed8`00017cd8
00000001`400177c8 600e149f`00000000
00000001`400177d0 0000000d`00000000
00000001`400177d8 00017cec`000002f0
00000001`400177e0 00000000`00016eec
00000001`400177e8 00000000`600e149f
00000001`400177f0 00000000`0000000e

0:000> dt ExecutionResidueC__Objects!CObject 00000000`005cd7e0
+0×000 __VFN_table : 0×00000001`40017778
+0×008 data : [32] “Hello Object!”

We created a modeling C++ program for better illustration:

struct CObject
{
    virtual ~CObject() {};
    virtual int foo() { return 1; };
    virtual int bar() { return 2; };

    char data[32] = "Hello Object!";
};

int main()
{
    CObject  localObj;
    int      _[20]{};	// padding the stack
    CObject* dynamicObj{new CObject};

    throw CObject();
}

The example memory dump, PDB file, and source code can be downloaded from here.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

A page to reference all different kinds of stack trace frames is necessary, so I created this post:

• Coincidental Frames
• Least Common Frame
• Foreign Module Frame
• Aggregated Frames
• Frame Regularity
• Frame Trace
• False Frame

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Often a debugger is not able to reconstruct a stack trace correctly, for example, when symbols to guide the process are not available due to Reduced Symbol Information or complete absence due to Unloaded Module:

0:008> k
# ChildEBP RetAddr
00 0250f4b8 76d21775 ntdll!NtWaitForMultipleObjects+0x15
01 0250f554 75c419fc KERNELBASE!WaitForMultipleObjectsEx+0x100
02 0250f59c 75c4268c kernel32!WaitForMultipleObjectsExImplementation+0xe0
03 0250f5b8 75c681fc kernel32!WaitForMultipleObjects+0x18
04 0250f624 75c680bb kernel32!WerpReportFaultInternal+0x186
05 0250f638 75c679b0 kernel32!WerpReportFault+0x70
06 0250f648 75c6792f kernel32!BasepReportFault+0x20
07 0250f6d4 00e21e86 kernel32!UnhandledExceptionFilter+0x1af
08 0250f6f0 75c803cf ModuleA!UnhandledExceptionFilter+0x3d
09 0250f778 77e250d7 kernel32!UnhandledExceptionFilter+0x127
0a 0250f780 77e24fb4 ntdll!__RtlUserThreadStart+0x62
0b 0250f794 77e24e59 ntdll!_EH4_CallFilterFunc+0x12
0c 0250f7bc 77e134a1 ntdll!_except_handler4+0x8e
0d 0250f7e0 77e13473 ntdll!ExecuteHandler2+0x26
0e 0250f804 77e13414 ntdll!ExecuteHandler+0x24
0f 0250f890 77dc0133 ntdll!RtlDispatchException+0x127
10 0250f890 68a8e0ca ntdll!KiUserExceptionDispatcher+0xf
WARNING: Frame IP not in any known module. Following frames may be wrong.
11 0250fd58 02c45f58 <Unloaded_ModuleB.dll>+0x1e0ca
12 0250fd84 75c4343d 0×2c45f58
13 0250fd90 77de9812 kernel32!BaseThreadInitThunk+0xe
14 0250fdd0 77de97e5 ntdll!__RtlUserThreadStart+0×70
15 0250fde8 00000000 ntdll!_RtlUserThreadStart+0×1b

The address may be the valid return address from Execution Residue, but may also be completely random, non-executable:

0:008> ub 0×2c45f58
^ Unable to find valid previous instruction for ‘ub 0×2c45f58′

0:008> !address 0×2c45f58

Usage: Free
Base Address: 02bb0000
End Address: 02cb0000
Region Size: 00100000 ( 1.000 MB)
State: 00010000 MEM_FREE
Protect: 00000001 PAGE_NOACCESS
Type: <info not present at the target>

In our case, we have symbol files for ModuleB.dll but they don’t help.

0:008> .sympath+ C:\MemoryDumps\Modules\PDBs

If we have normal Manual Dumps we can compare Stack Trace Collections and take the advantage of existing Thread Posets to get the correct stack trace.

Alternatively, we can either use manual stack trace reconstruction techniques or use Injected Symbols:

0:008> lm
[...]
Unloaded modules:
[...]
68a70000 68ac0000 ModuleB.dll
[…]

0:008> .reload /f /i ModuleB.dll=68a70000
*** WARNING: Unable to verify timestamp for ModuleB.dll

0:008> kL
# ChildEBP RetAddr
00 0250f4b8 76d21775 ntdll!NtWaitForMultipleObjects+0x15
01 0250f554 75c419fc KERNELBASE!WaitForMultipleObjectsEx+0x100
02 0250f59c 75c4268c kernel32!WaitForMultipleObjectsExImplementation+0xe0
03 0250f5b8 75c681fc kernel32!WaitForMultipleObjects+0x18
04 0250f624 75c680bb kernel32!WerpReportFaultInternal+0x186
05 0250f638 75c679b0 kernel32!WerpReportFault+0x70
06 0250f648 75c6792f kernel32!BasepReportFault+0x20
07 0250f6d4 00e21e86 kernel32!UnhandledExceptionFilter+0x1af
08 0250f6f0 75c803cf ModuleA!UnhandledExceptionFilter+0x3d
09 0250f778 77e250d7 kernel32!UnhandledExceptionFilter+0x127
0a 0250f780 77e24fb4 ntdll!__RtlUserThreadStart+0x62
0b 0250f794 77e24e59 ntdll!_EH4_CallFilterFunc+0x12
0c 0250f7bc 77e134a1 ntdll!_except_handler4+0x8e
0d 0250f7e0 77e13473 ntdll!ExecuteHandler2+0x26
0e 0250f804 77e13414 ntdll!ExecuteHandler+0x24
0f 0250f890 77dc0133 ntdll!RtlDispatchException+0x127
10 0250f890 68a8e0ca ntdll!KiUserExceptionDispatcher+0xf
11 0250fd64 68a8f284 ModuleB!foo+0x5a
12 0250fd84 75c4343d ModuleB!bar+0xf4
13 0250fd90 77de9812 kernel32!BaseThreadInitThunk+0xe
14 0250fdd0 77de97e5 ntdll!__RtlUserThreadStart+0×70
15 0250fde8 00000000 ntdll!_RtlUserThreadStart+0×1b

We call this analysis pattern False Frame. Although we have Incorrect Stack Trace, just one stack trace frame is wrong. Sometimes, if there is Coincidental Symbolic Information available we get Coincidental Frames.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

A page to reference all different kinds of analysis patterns related to pointers is necessary, so I created this post:

• Invalid Pointer (general)
• Invalid Pointer (objects)
• NULL Pointer (code)
• NULL Pointer (data)
• Wild Pointer
• Corrupt Structure
• Pointer Cone
• Context Pointer
• Pointer Class
• Function Pointer
• Encoded Pointer

I update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Inspecting memory region type to which a pointer points to may help diagnosing some coding mistakes. We call this analysis pattern Pointer Class. Below is a case study modeled on a real-life case.

The application was crashing sporadically and the memory dump was pointing to invalid objects reused after free. We recreated similar source code pattern and got the similar crash (PointerClass.exe.8752.dmp):

0:000> kL
# Child-SP          RetAddr           Call Site
00 000000e9`a10fe448 00007ff9`64e08037 ntdll!NtWaitForMultipleObjects+0x14
01 000000e9`a10fe450 00007ff9`64e07f1e KERNELBASE!WaitForMultipleObjectsEx+0x107
02 000000e9`a10fe750 00007ff9`653c71fb KERNELBASE!WaitForMultipleObjects+0xe
03 000000e9`a10fe790 00007ff9`653c6ca8 kernel32!WerpReportFaultInternal+0x51b
04 000000e9`a10fe8b0 00007ff9`64eb00b8 kernel32!WerpReportFault+0xac
05 000000e9`a10fe8f0 00007ff9`672a4ab2 KERNELBASE!UnhandledExceptionFilter+0x3b8
06 000000e9`a10fea10 00007ff9`6728c656 ntdll!RtlUserThreadStart$filt$0+0xa2
07 000000e9`a10fea50 00007ff9`672a11cf ntdll!_C_specific_handler+0x96
08 000000e9`a10feac0 00007ff9`6726a209 ntdll!RtlpExecuteHandlerForException+0xf
09 000000e9`a10feaf0 00007ff9`6729fe3e ntdll!RtlDispatchException+0x219
0a 000000e9`a10ff200 00007ff7`df32103a ntdll!KiUserExceptionDispatch+0×2e
0b 000000e9`a10ff918 00007ff7`df321081 PointerClass!Data::GetData+0xa
0c 000000e9`a10ff920 00007ff7`df32121a PointerClass!Work::DoWork+0×21
0d 000000e9`a10ff960 00007ff7`df321494 PointerClass!main+0×4a
0e (Inline Function) ——–`——– PointerClass!invoke_main+0×22
0f 000000e9`a10ff9b0 00007ff9`65377bd4 PointerClass!__scrt_common_main_seh+0×10c
10 000000e9`a10ff9f0 00007ff9`6726ce51 kernel32!BaseThreadInitThunk+0×14
11 000000e9`a10ffa20 00000000`00000000 ntdll!RtlUserThreadStart+0×21

0:000> .frame b
0b 000000e9`a10ff918 00007ff7`df321081 PointerClass!Data::GetData+0xa [C:\NewWork\PointerClass\PointerClass.cpp @ 7]

0:000> dv
this = 0×00000227`eb030000

0:000> dp poi(this)
00000227`eb030000  ????????`???????? ????????`????????
00000227`eb030010  ????????`???????? ????????`????????
00000227`eb030020  ????????`???????? ????????`????????
00000227`eb030030  ????????`???????? ????????`????????
00000227`eb030040  ????????`???????? ????????`????????
00000227`eb030050  ????????`???????? ????????`????????
00000227`eb030060  ????????`???????? ????????`????????
00000227`eb030070  ????????`???????? ????????`????????

struct Data
{
void SetData(int newData) { data = newData; }
int  GetData() { return data; }
private:
int data{};
};

An engineer found out that a pointer to an outside object was used and it was not updated when the object was freed:

struct Work
{
void SetData(Data* newData)
{
data = newData;
}

void DoWork()
{
if (data)
{
auto value = data->GetData();
++value;
data->SetData(value);
}
}

private:
Data* data{};
};

void Init(Work& work, Model& model)
{
unsigned long long dummy{};
if (Data* pData = model.GetData(); pData)
{
work.SetData(pData);
}
}

The solution was to use a double pointer but it also crashed (PointerClassFixNotWorking.exe.7452.dmp):

struct Work
{
void SetData(Data** newData)
{
data = newData;
}

void DoWork()
{
if (data && *data)
{
auto value = (*data)->GetData();
++value;
(*data)->SetData(value);
}
}

private:
Data** data{};
};

void Init(Work& work, Model& model)
{
unsigned long long dummy{};
if (Data* pData = model.GetData(); pData)
{
work.SetData(&pData);
}
}

0:000> .ecxr
*** WARNING: Unable to verify checksum for PointerClassFixNotWorking.exe
rax=0000019a4be10000 rbx=0000019a4bff29c0 rcx=0000019a4be10000
rdx=0000000000000000 rsi=0000000000000000 rdi=0000019a4bff6cf0
rip=00007ff766d5103a rsp=0000007ede6ff958 rbp=0000000000000000
r8=0000007ede6ff938  r9=0000000000000000 r10=0000000000000000
r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010206
PointerClassFixNotWorking!Data::GetData+0xa:
00007ff7`66d5103a 8b00            mov     eax,dword ptr [rax] ds:0000019a`4be10000=????????

0:000> kL
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP          RetAddr           Call Site
00 0000007e`de6ff958 00007ff7`66d51092 PointerClassFixNotWorking!Data::GetData+0xa
01 0000007e`de6ff960 00007ff7`66d5124a PointerClassFixNotWorking!Work::DoWork+0x32
02 0000007e`de6ff9a0 00007ff7`66d514d4 PointerClassFixNotWorking!main+0x4a
03 (Inline Function) --------`-------- PointerClassFixNotWorking!invoke_main+0x22
04 0000007e`de6ff9f0 00007ff9`65377bd4 PointerClassFixNotWorking!__scrt_common_main_seh+0x10c
05 0000007e`de6ffa30 00007ff9`6726ce51 kernel32!BaseThreadInitThunk+0x14
06 0000007e`de6ffa60 00000000`00000000 ntdll!RtlUserThreadStart+0x21

struct Data
{
void SetData(int newData) { data = newData; }
int  GetData() { return data; }
private:
int data{};
};

It was hypothesized that the object was also freed somewhere else and the debugging continued. However, the simple inspection of this->data Pointer Class would have revealed that it is pointing to a stack location (that was reused by subsequent calls to other functions):

0:000> .frame 1
01 0000007e`de6ff960 00007ff7`66d5124a PointerClassFixNotWorking!Work::DoWork+0x32 [C:\NewWork\PointerClassFixNotWorking\PointerClassFixNotWorking.cpp @ 24]

0:000> dv /i /v
prv local  0000007e`de6ff980           value = 0n1275013568
prv local  0000007e`de6ff9a0            this = 0x0000007e`de6ff9c0

0:000> dt this
Local var @ 0x7ede6ff9a0 Type Work*
0x0000007e`de6ff9c0
+0×000 data             : 0×0000007e`de6ff978  -> 0×0000019a`4be10000 Data

0:000> !address 0×0000007e`de6ff978
Usage:                  Stack
Base Address:           0000007e`de6fd000
End Address:            0000007e`de700000
Region Size:            00000000`00003000 (  12.000 kB)
State:                  00001000          MEM_COMMIT
Protect:                00000004          PAGE_READWRITE
Type:                   00020000          MEM_PRIVATE
Allocation Base:        0000007e`de600000
Allocation Protect:     00000004          PAGE_READWRITE
More info:              ~0k

So the correct fix should have been be to use an address (heap region in the original case) of a pointer stored inside an owner object (allocated on heap in the original case):

void Init(Work& work, Model& model)
{
unsigned long long dummy{};
if (Data** ppData = model.GetData(); ppData && *ppData)
{
work.SetData(ppData);
}
}

The example memory dumps, PDB files, and source code of applications can be downloaded from here.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When developers look at crash dumps they are more interested at parameters and local variables in particular stack frames of interest. However, sometimes it is useful to look at all such frames especially to gather information that may be useful for technical support or to correlate to additional traces and logs (for example, Historical Information to establish additional Basic Facts and build Vocabulary Index).

Listing the parameters can be done, for example, by using Stack Trace command variant (kP WinDbg commend, but we use kPL to exclude source code references to reduce visual clutter):

0:000> kPL
# Child-SP          RetAddr           Call Site
00 000000e4`c0afe488 00007ffe`cc888037 ntdll!NtWaitForMultipleObjects+0x14
01 000000e4`c0afe490 00007ffe`cc887f1e KERNELBASE!WaitForMultipleObjectsEx+0x107
02 000000e4`c0afe790 00007ffe`cd8271fb KERNELBASE!WaitForMultipleObjects+0xe
03 000000e4`c0afe7d0 00007ffe`cd826ca8 kernel32!WerpReportFaultInternal+0x51b
04 000000e4`c0afe8f0 00007ffe`cc9300b8 kernel32!WerpReportFault+0xac
05 000000e4`c0afe930 00007ffe`cf6c4ab2 KERNELBASE!UnhandledExceptionFilter+0x3b8
06 000000e4`c0afea50 00007ffe`cf6ac656 ntdll!RtlUserThreadStart$filt$0+0xa2
07 000000e4`c0afea90 00007ffe`cf6c11cf ntdll!_C_specific_handler+0x96
08 000000e4`c0afeb00 00007ffe`cf68a209 ntdll!RtlpExecuteHandlerForException+0xf
09 000000e4`c0afeb30 00007ffe`cf6bfe3e ntdll!RtlDispatchException+0x219
0a 000000e4`c0aff240 00007ffe`cc8f0aa2 ntdll!KiUserExceptionDispatch+0x2e
0b 000000e4`c0aff948 00007ff6`c8ab1568 KERNELBASE!wil::details::DebugBreak+0x2
0c 000000e4`c0aff950 00007ff6`c8ab1560 FrameTrace!foo(
unsigned int64 num = 0,
class std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > * str = 0×000000e4`c0aff9d0 “Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! “)+0×68
0d 000000e4`c0aff9b0 00007ff6`c8ab1560 FrameTrace!foo(
unsigned int64 num = 0,
class std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > * str = 0×000000e4`c0affa30 “Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! “)+0×60
0e 000000e4`c0affa10 00007ff6`c8ab1560 FrameTrace!foo(
unsigned int64 num = 1,
class std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > * str = 0×000000e4`c0affa90 “Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! “)+0×60
0f 000000e4`c0affa70 00007ff6`c8ab1560 FrameTrace!foo(
unsigned int64 num = 2,
class std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > * str = 0×000000e4`c0affaf0 “Hello World! Hello World! Hello World! Hello World! “)+0×60
10 000000e4`c0affad0 00007ff6`c8ab1560 FrameTrace!foo(
unsigned int64 num = 3,
class std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > * str = 0×000000e4`c0affb50 “Hello World! Hello World! “)+0×60
11 000000e4`c0affb30 00007ff6`c8ab15b5 FrameTrace!foo(
unsigned int64 num = 4,
class std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > * str = 0×000000e4`c0affbb0 “Hello World! “)+0×60
12 000000e4`c0affb90 00007ff6`c8ab2b14 FrameTrace!main(void)+0×25
13 (Inline Function) ——–`——– FrameTrace!invoke_main+0×22
14 000000e4`c0affbe0 00007ffe`cd7d7bd4 FrameTrace!__scrt_common_main_seh(void)+0×10c
15 000000e4`c0affc20 00007ffe`cf68ce51 kernel32!BaseThreadInitThunk+0×14
16 000000e4`c0affc50 00000000`00000000 ntdll!RtlUserThreadStart+0×21

The stack trace comes from the following modeling application:

void foo(std::size_t num, const std::wstring& str)
{
if (std::wstring concatStr{ str }; num)
{
concatStr += str;

foo(–num, concatStr);
}
else
{
::DebugBreak();
}
}

int main()
{
foo(5, L”Hello World! “);
}

To list local variable we need to use !for_each_frame WinDbg command:

0:000> !for_each_frame "dv"
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
00 000000e4`c0afe488 00007ffe`cc888037 ntdll!NtWaitForMultipleObjects+0x14
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
01 000000e4`c0afe490 00007ffe`cc887f1e KERNELBASE!WaitForMultipleObjectsEx+0x107
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
02 000000e4`c0afe790 00007ffe`cd8271fb KERNELBASE!WaitForMultipleObjects+0xe
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
03 000000e4`c0afe7d0 00007ffe`cd826ca8 kernel32!WerpReportFaultInternal+0x51b
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
04 000000e4`c0afe8f0 00007ffe`cc9300b8 kernel32!WerpReportFault+0xac
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
05 000000e4`c0afe930 00007ffe`cf6c4ab2 KERNELBASE!UnhandledExceptionFilter+0x3b8
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
06 000000e4`c0afea50 00007ffe`cf6ac656 ntdll!RtlUserThreadStart$filt$0+0xa2
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
07 000000e4`c0afea90 00007ffe`cf6c11cf ntdll!_C_specific_handler+0x96
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
08 000000e4`c0afeb00 00007ffe`cf68a209 ntdll!RtlpExecuteHandlerForException+0xf
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
09 000000e4`c0afeb30 00007ffe`cf6bfe3e ntdll!RtlDispatchException+0x219
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0a 000000e4`c0aff240 00007ffe`cc8f0aa2 ntdll!KiUserExceptionDispatch+0x2e
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0b 000000e4`c0aff948 00007ff6`c8ab1568 KERNELBASE!wil::details::DebugBreak+0x2
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0c 000000e4`c0aff950 00007ff6`c8ab1560 FrameTrace!foo+0x68 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 14]
concatStr = "Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! "
num = 0
str = 0x000000e4`c0aff9d0 "Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! "
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0d 000000e4`c0aff9b0 00007ff6`c8ab1560 FrameTrace!foo+0x60 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 11]
concatStr = "Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! "
num = 0
str = 0x000000e4`c0affa30 "Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! "
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0e 000000e4`c0affa10 00007ff6`c8ab1560 FrameTrace!foo+0x60 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 11]
concatStr = "Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! "
num = 1
str = 0x000000e4`c0affa90 "Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! "
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0f 000000e4`c0affa70 00007ff6`c8ab1560 FrameTrace!foo+0x60 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 11]
concatStr = "Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! Hello World! "
num = 2
str = 0x000000e4`c0affaf0 "Hello World! Hello World! Hello World! Hello World! "
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
10 000000e4`c0affad0 00007ff6`c8ab1560 FrameTrace!foo+0x60 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 11]
concatStr = "Hello World! Hello World! Hello World! Hello World! "
num = 3
str = 0x000000e4`c0affb50 "Hello World! Hello World! "
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
11 000000e4`c0affb30 00007ff6`c8ab15b5 FrameTrace!foo+0x60 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 11]
concatStr = "Hello World! Hello World! "
num = 4
str = 0x000000e4`c0affbb0 "Hello World! "
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
12 000000e4`c0affb90 00007ff6`c8ab2b14 FrameTrace!main+0x25 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 20]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
13 (Inline Function) --------`-------- FrameTrace!invoke_main+0x22 [d:\A01\_work\6\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
14 000000e4`c0affbe0 00007ffe`cd7d7bd4 FrameTrace!__scrt_common_main_seh+0x10c [d:\A01\_work\6\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288]
has_cctor = false
main_result = <value unavailable>
tls_init_callback = <value unavailable>
is_nested = <value unavailable>
tls_dtor_callback = <value unavailable>
main_result = <value unavailable>
__scrt_current_native_startup_state = <value unavailable>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
15 000000e4`c0affc20 00007ffe`cf68ce51 kernel32!BaseThreadInitThunk+0x14
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
16 000000e4`c0affc50 00000000`00000000 ntdll!RtlUserThreadStart+0x21
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
15 000000e4`c0affc20 00007ffe`cf68ce51 kernel32!BaseThreadInitThunk+0x14

We can also apply “dv /i /V” command to each frame to get additional low-level frame details:

[...]
11 000000e4`c0affb30 00007ff6`c8ab15b5 FrameTrace!foo+0x60 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 11]
prv local  000000e4`c0affb50 @rsp+0x0020             concatStr = "Hello World! Hello World! "
prv param  000000e4`c0affb90 @rsp+0x0060                   num = 4
prv param  000000e4`c0affb98 @rsp+0x0068                   str = 0x000000e4`c0affbb0 "Hello World! "
[...]

We see this as a form of back tracing Execution Residue, for example:

0:000> !for_each_frame ".frame /c @$frame; dps rsp"
[...]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
04 000000e4`c0afe8f0 00007ffe`cc9300b8 kernel32!WerpReportFault+0xac
04 000000e4`c0afe8f0 00007ffe`cc9300b8 kernel32!WerpReportFault+0xac
rax=000000000000005b rbx=0000000000000000 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=0000000000000000 rdi=0000000000000003
rip=00007ffecd826ca8 rsp=000000e4c0afe8f0 rbp=0000000000000000
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=000000e4c0afeac0 r13=ffffffffffffffff
r14=000000e4c0afeac0 r15=0000000000001a38
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
kernel32!WerpReportFault+0xac:
00007ffe`cd826ca8 8bf8            mov     edi,eax
000000e4`c0afe8f0  00000000`00000000
000000e4`c0afe8f8  00000000`00000000
000000e4`c0afe900  00000000`00000003
000000e4`c0afe908  000000e4`c0afeac0
000000e4`c0afe910  00000000`00000004
000000e4`c0afe918  00000000`00000001
000000e4`c0afe920  00000000`00000000
000000e4`c0afe928  00007ffe`cc9300b8 KERNELBASE!UnhandledExceptionFilter+0x3b8
000000e4`c0afe930  00000000`00000000
000000e4`c0afe938  000000e4`c0affc50
000000e4`c0afe940  00007ffe`cd7c0000 kernel32!RtlVirtualUnwindStub <PERF> (kernel32+0x0)
000000e4`c0afe948  00000207`5d660000
000000e4`c0afe950  00000000`00000000
000000e4`c0afe958  00007ffe`cf6660b9 ntdll!RtlpFindEntry+0x4d
000000e4`c0afe960  00000004`00000006
000000e4`c0afe968  00000001`00000000
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
05 000000e4`c0afe930 00007ffe`cf6c4ab2 KERNELBASE!UnhandledExceptionFilter+0x3b8
05 000000e4`c0afe930 00007ffe`cf6c4ab2 KERNELBASE!UnhandledExceptionFilter+0x3b8
rax=000000000000005b rbx=0000000000000000 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=00007ffecd7c0000 rdi=0000000000000000
rip=00007ffecc9300b8 rsp=000000e4c0afe930 rbp=000000e4c0affc50
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=000000e4c0afeac0 r13=ffffffffffffffff
r14=0000000000000001 r15=0000000000000004
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
KERNELBASE!UnhandledExceptionFilter+0x3b8:
00007ffe`cc9300b8 0f1f440000      nop     dword ptr [rax+rax]
000000e4`c0afe930  00000000`00000000
000000e4`c0afe938  000000e4`c0affc50
000000e4`c0afe940  00007ffe`cd7c0000 kernel32!RtlVirtualUnwindStub <PERF> (kernel32+0x0)
000000e4`c0afe948  00000207`5d660000
000000e4`c0afe950  00000000`00000000
000000e4`c0afe958  00007ffe`cf6660b9 ntdll!RtlpFindEntry+0x4d
000000e4`c0afe960  00000004`00000006
000000e4`c0afe968  00000001`00000000
000000e4`c0afe970  00000000`00000001
000000e4`c0afe978  00007ffe`cd7c0000 kernel32!RtlVirtualUnwindStub <PERF> (kernel32+0x0)
000000e4`c0afe980  00000207`5d662ff0
000000e4`c0afe988  00000000`00000000
000000e4`c0afe990  000000e4`c0afeac0
000000e4`c0afe998  00007ffe`cd7c0000 kernel32!RtlVirtualUnwindStub <PERF> (kernel32+0x0)
000000e4`c0afe9a0  00000000`005a0058
000000e4`c0afe9a8  00007ffe`cca6ff70 KERNELBASE!`string'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
06 000000e4`c0afea50 00007ffe`cf6ac656 ntdll!RtlUserThreadStart$filt$0+0xa2
06 000000e4`c0afea50 00007ffe`cf6ac656 ntdll!RtlUserThreadStart$filt$0+0xa2
rax=000000000000005b rbx=00007ffecf764420 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=0000000000000000 rdi=0000000000000000
rip=00007ffecf6c4ab2 rsp=000000e4c0afea50 rbp=000000e4c0affc50
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=000000e4c0aff730 r13=000000e4c0affc50
r14=000000e4c0aff0c0 r15=00007ffecf620000
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!RtlUserThreadStart$filt$0+0xa2:
00007ffe`cf6c4ab2 eb16            jmp     ntdll!RtlUserThreadStart$filt$0+0xba (00007ffe`cf6c4aca)
000000e4`c0afea50  00000000`00000000
000000e4`c0afea58  00007ffe`cf764420 ntdll!`string'+0x9aa8
000000e4`c0afea60  00000000`00000000
000000e4`c0afea68  000000e4`c0affbe0
000000e4`c0afea70  00000000`00000000
000000e4`c0afea78  00007ffe`cf6457d8 ntdll!LdrpAppendUnicodeStringToFilenameBuffer+0x50
000000e4`c0afea80  00000000`0006ce51
000000e4`c0afea88  00007ffe`cf6ac656 ntdll!_C_specific_handler+0x96
000000e4`c0afea90  000000e4`c0afeb40
000000e4`c0afea98  00007ffe`cf642930 ntdll!LdrpFindLoadedDllByNameLockHeld+0xe4
000000e4`c0afeaa0  000000e4`c0aff088
000000e4`c0afeaa8  000000e4`c0aff110
000000e4`c0afeab0  000000e4`c0aff240
000000e4`c0afeab8  00000000`00000000
000000e4`c0afeac0  000000e4`c0aff730
000000e4`c0afeac8  000000e4`c0aff240
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
07 000000e4`c0afea90 00007ffe`cf6c11cf ntdll!_C_specific_handler+0x96
07 000000e4`c0afea90 00007ffe`cf6c11cf ntdll!_C_specific_handler+0x96
rax=000000000000005b rbx=00007ffecf764420 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=0000000000000000 rdi=0000000000000000
rip=00007ffecf6ac656 rsp=000000e4c0afea90 rbp=000000000006ce51
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=000000e4c0aff730 r13=000000e4c0affc50
r14=000000e4c0aff0c0 r15=00007ffecf620000
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!_C_specific_handler+0x96:
00007ffe`cf6ac656 85c0            test    eax,eax
000000e4`c0afea90  000000e4`c0afeb40
000000e4`c0afea98  00007ffe`cf642930 ntdll!LdrpFindLoadedDllByNameLockHeld+0xe4
000000e4`c0afeaa0  000000e4`c0aff088
000000e4`c0afeaa8  000000e4`c0aff110
000000e4`c0afeab0  000000e4`c0aff240
000000e4`c0afeab8  00000000`00000000
000000e4`c0afeac0  000000e4`c0aff730
000000e4`c0afeac8  000000e4`c0aff240
000000e4`c0afead0  00000000`00000000
000000e4`c0afead8  000000e4`c0afeb70
000000e4`c0afeae0  000000e4`c0aff240
000000e4`c0afeae8  00007ffe`cf6ac5c0 ntdll!_C_specific_handler
000000e4`c0afeaf0  00000000`00000000
000000e4`c0afeaf8  00007ffe`cf6c11cf ntdll!RtlpExecuteHandlerForException+0xf
000000e4`c0afeb00  00000000`00000000
000000e4`c0afeb08  000000e4`c0aff070
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
08 000000e4`c0afeb00 00007ffe`cf68a209 ntdll!RtlpExecuteHandlerForException+0xf
08 000000e4`c0afeb00 00007ffe`cf68a209 ntdll!RtlpExecuteHandlerForException+0xf
rax=000000000000005b rbx=0000000000000000 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=000000e4c0aff730 rdi=0000000000000000
rip=00007ffecf6c11cf rsp=000000e4c0afeb00 rbp=000000e4c0aff070
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=00007ffecf6ac5c0 r13=000000e4c0aff240
r14=000000e4c0afeb70 r15=0000000000000000
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!RtlpExecuteHandlerForException+0xf:
00007ffe`cf6c11cf 90              nop
000000e4`c0afeb00  00000000`00000000
000000e4`c0afeb08  000000e4`c0aff070
000000e4`c0afeb10  000000e4`c0aff730
000000e4`c0afeb18  000000e4`c0aff730
000000e4`c0afeb20  000000e4`c0aff0c0
000000e4`c0afeb28  00007ffe`cf68a209 ntdll!RtlDispatchException+0x219
000000e4`c0afeb30  000000e4`00000001
000000e4`c0afeb38  00007ffe`cf620000 ntdll!RtlStringCchCopyW <PERF> (ntdll+0x0)
000000e4`c0afeb40  00000000`00000000
000000e4`c0afeb48  00007ffe`cf78e9f0 ntdll!__PchSym_ <PERF> (ntdll+0x16e9f0)
000000e4`c0afeb50  000000e4`c0afeb70
000000e4`c0afeb58  000000e4`c0aff090
000000e4`c0afeb60  000000e4`c0aff080
000000e4`c0afeb68  00000000`00000000
000000e4`c0afeb70  000000e4`00000000
000000e4`c0afeb78  00007ffe`cc8300f0 KERNELBASE!UrlHashW <PERF> (KERNELBASE+0xf0)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
09 000000e4`c0afeb30 00007ffe`cf6bfe3e ntdll!RtlDispatchException+0x219
09 000000e4`c0afeb30 00007ffe`cf6bfe3e ntdll!RtlDispatchException+0x219
rax=000000000000005b rbx=0000000000000000 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=000000e4c0aff730 rdi=0000000000000000
rip=00007ffecf68a209 rsp=000000e4c0afeb30 rbp=000000e4c0aff070
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=00007ffecf6ac5c0 r13=000000e4c0aff240
r14=000000e4c0afeb70 r15=0000000000000000
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!RtlDispatchException+0x219:
00007ffe`cf68a209 8bd0            mov     edx,eax
000000e4`c0afeb30  000000e4`00000001
000000e4`c0afeb38  00007ffe`cf620000 ntdll!RtlStringCchCopyW <PERF> (ntdll+0x0)
000000e4`c0afeb40  00000000`00000000
000000e4`c0afeb48  00007ffe`cf78e9f0 ntdll!__PchSym_ <PERF> (ntdll+0x16e9f0)
000000e4`c0afeb50  000000e4`c0afeb70
000000e4`c0afeb58  000000e4`c0aff090
000000e4`c0afeb60  000000e4`c0aff080
000000e4`c0afeb68  00000000`00000000
000000e4`c0afeb70  000000e4`00000000
000000e4`c0afeb78  00007ffe`cc8300f0 KERNELBASE!UrlHashW <PERF> (KERNELBASE+0xf0)
000000e4`c0afeb80  00000001`00000000
000000e4`c0afeb88  00000012`00000018
000000e4`c0afeb90  00000000`00000000
000000e4`c0afeb98  00360030`00300030
000000e4`c0afeba0  00001f80`0010000f
000000e4`c0afeba8  00000000`00000033
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0a 000000e4`c0aff240 00007ffe`cc8f0aa2 ntdll!KiUserExceptionDispatch+0x2e
0a 000000e4`c0aff240 00007ffe`cc8f0aa2 ntdll!KiUserExceptionDispatch+0x2e
rax=000000000000005b rbx=000002075d662a10 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=0000000000000000 rdi=000002075d666e40
rip=00007ffecf6bfe3e rsp=000000e4c0aff240 rbp=0000000000000000
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!KiUserExceptionDispatch+0x2e:
00007ffe`cf6bfe3e 84c0            test    al,al
000000e4`c0aff240  00007ff6`c8ac32f0 FrameTrace!`string'
000000e4`c0aff248  00000000`000a0008
000000e4`c0aff250  00000207`5d662a10
000000e4`c0aff258  00007ff6`00200000
000000e4`c0aff260  000000e4`c0aff2f0
000000e4`c0aff268  000000e4`c0aff2f0
000000e4`c0aff270  00001f80`0010005f
000000e4`c0aff278  0053002b`002b0033
000000e4`c0aff280  00000246`002b002b
000000e4`c0aff288  00000000`00000000
000000e4`c0aff290  00000000`00000000
000000e4`c0aff298  00000000`00000000
000000e4`c0aff2a0  00000000`00000000
000000e4`c0aff2a8  00000000`00000000
000000e4`c0aff2b0  00000000`00000000
000000e4`c0aff2b8  000000e4`c0aff970
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0b 000000e4`c0aff948 00007ff6`c8ab1568 KERNELBASE!wil::details::DebugBreak+0x2
0b 000000e4`c0aff948 00007ff6`c8ab1568 KERNELBASE!wil::details::DebugBreak+0x2
rax=000000000000005b rbx=000002075d662a10 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=0000000000000000 rdi=000002075d666e40
rip=00007ffecc8f0aa2 rsp=000000e4c0aff948 rbp=0000000000000000
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
KERNELBASE!wil::details::DebugBreak+0x2:
00007ffe`cc8f0aa2 cc              int     3
000000e4`c0aff948  00007ff6`c8ab1568 FrameTrace!foo+0x68 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 14]
000000e4`c0aff950  000000e4`c0aff970
000000e4`c0aff958  000000e4`c0aff9d0
000000e4`c0aff960  00000000`000000d0
000000e4`c0aff968  00000207`5d66a990
000000e4`c0aff970  00000207`5d66b070
000000e4`c0aff978  00007ff6`c8ab15ed FrameTrace!std::basic_string<wchar_t,std::char_traits<wchar_t>, std::allocator<wchar_t> >::operator+=+0x1d [C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.26.28801\include\xstring @ 2821]
000000e4`c0aff980  00000000`000001a0
000000e4`c0aff988  00000000`000001a7
000000e4`c0aff990  0000e8d4`e5494150
000000e4`c0aff998  0000e8d4`e5494150
000000e4`c0aff9a0  000000e4`c0affa30
000000e4`c0aff9a8  00007ff6`c8ab1560 FrameTrace!foo+0x60 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 11]
000000e4`c0aff9b0  00000000`00000000
000000e4`c0aff9b8  000000e4`c0aff9d0
000000e4`c0aff9c0  00000000`00000068
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0c 000000e4`c0aff950 00007ff6`c8ab1560 FrameTrace!foo+0x68 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 14]
0c 000000e4`c0aff950 00007ff6`c8ab1560 FrameTrace!foo+0x68 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 14]
rax=000000000000005b rbx=000002075d662a10 rcx=0000000000000003
rdx=000000e4c0afe888 rsi=0000000000000000 rdi=000002075d666e40
rip=00007ff6c8ab1568 rsp=000000e4c0aff950 rbp=0000000000000000
r8=0000000000000000  r9=00000000ffffffff r10=0000000000000000
r11=000000e4c0afdc30 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
FrameTrace!foo+0x68:
00007ff6`c8ab1568 90              nop
000000e4`c0aff950  000000e4`c0aff970
000000e4`c0aff958  000000e4`c0aff9d0
000000e4`c0aff960  00000000`000000d0
000000e4`c0aff968  00000207`5d66a990
000000e4`c0aff970  00000207`5d66b070
000000e4`c0aff978  00007ff6`c8ab15ed FrameTrace!std::basic_string<wchar_t,std::char_traits<wchar_t>, std::allocator<wchar_t> >::operator+=+0x1d [C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.26.28801\include\xstring @ 2821]
000000e4`c0aff980  00000000`000001a0
000000e4`c0aff988  00000000`000001a7
000000e4`c0aff990  0000e8d4`e5494150
000000e4`c0aff998  0000e8d4`e5494150
000000e4`c0aff9a0  000000e4`c0affa30
000000e4`c0aff9a8  00007ff6`c8ab1560 FrameTrace!foo+0x60 [C:\NewWork\FrameTrace\FrameTrace.cpp @ 11]
000000e4`c0aff9b0  00000000`00000000
000000e4`c0aff9b8  000000e4`c0aff9d0
000000e4`c0aff9c0  00000000`00000068
000000e4`c0aff9c8  00000207`5d66a8a0
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[...]

We need to reset the current context after the command above since the last frame becomes the current:

0:000> kc
*** Stack trace for last set context - .thread/.cxr resets it
# Call Site
15 ntdll!RtlUserThreadStart

0:000> .cxr
Resetting default scope

0:000> kc
# Call Site
00 ntdll!NtWaitForMultipleObjects
01 KERNELBASE!WaitForMultipleObjectsEx
02 KERNELBASE!WaitForMultipleObjects
03 kernel32!WerpReportFaultInternal
04 kernel32!WerpReportFault
05 KERNELBASE!UnhandledExceptionFilter
06 ntdll!RtlUserThreadStart$filt$0
07 ntdll!_C_specific_handler
08 ntdll!RtlpExecuteHandlerForException
09 ntdll!RtlDispatchException
0a ntdll!KiUserExceptionDispatch
0b KERNELBASE!wil::details::DebugBreak
0c FrameTrace!foo
0d FrameTrace!foo
0e FrameTrace!foo
0f FrameTrace!foo
10 FrameTrace!foo
11 FrameTrace!foo
12 FrameTrace!main
13 FrameTrace!invoke_main
14 FrameTrace!__scrt_common_main_seh
15 kernel32!BaseThreadInitThunk
16 ntdll!RtlUserThreadStart

We call this analysis pattern Frame Trace.

The example memory dump, the application PDB file, and source code can be downloaded from here.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When modeling Invalid Pointer (Objects) analysis pattern, we noticed that if we use MEM_RELEASE instead of MEM_DECOMMIT in VirtualFree API call, we see page memory contents despite an access violation Stored Exception pointing to that page. Moreover, the page contents were not corresponding to what should have been expected from source code. We had to do live kernel debugging in order to verify what was going on.

We launched InvalidPointerObject.exe that displayed the committed address allocated via VirtualAlloc API call:

The we broke into the system, found our process and inspected that address:

Microsoft (R) Windows Debugger Version 10.0.18362.1 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Opened \\.\pipe\com2
Waiting to reconnect...
Connected to Windows 10 18362 x64 target at (Fri May 1 22:46:00.982 2020 (UTC + 1:00)), ptr64 TRUE
Kernel Debugger connection established.
Symbol search path is: srv*
Executable search path is:
Windows 10 Kernel Version 18362 MP (1 procs) Free x64
Built by: 18362.1.amd64fre.19h1_release.190318-1202
Machine Name:
Kernel base = 0xfffff800`74800000 PsLoadedModuleList = 0xfffff800`74c48190
System Uptime: 0 days 0:00:00.000
KDTARGET: Refreshing KD connection
Break instruction exception - code 80000003 (first chance)
*******************************************************************************
* *
* You are seeing this message because you pressed either *
* CTRL+C (if you run console kernel debugger) or, *
* CTRL+BREAK (if you run GUI kernel debugger), *
* on your debugger machine's keyboard. *
* *
* THIS IS NOT A BUG OR A SYSTEM CRASH *
* *
* If you did not intend to break into the debugger, press the "g" key, then *
* press the "Enter" key now. This message might immediately reappear. If it *
* does, press "g" and "Enter" again. *
* *
*******************************************************************************
nt!DbgBreakPointWithStatus:
fffff800`749c93a0 cc int 3

1: kd> !process 0 0
**** NT ACTIVE PROCESS DUMP ****
PROCESS ffffe00314e89300
SessionId: none Cid: 0004 Peb: 00000000 ParentCid: 0000
DirBase: 001ad002 ObjectTable: ffffc90314806d40 HandleCount: 3136.
Image: System

[...]

PROCESS ffffe00318d60080
SessionId: 1 Cid: 1a90 Peb: 161ab73000 ParentCid: 1474
DirBase: af7ee002 ObjectTable: ffffc9031c02a0c0 HandleCount: 33.
Image: InvalidPointerObject.exe

[...]

1: kd> !process ffffe00318d60080 3f
PROCESS ffffe00318d60080
SessionId: 1 Cid: 1a90 Peb: 161ab73000 ParentCid: 1474
DirBase: af7ee002 ObjectTable: ffffc9031c02a0c0 HandleCount: 33.
Image: InvalidPointerObject.exe
VadRoot ffffe0031a78d1c0 Vads 22 Clone 0 Private 94. Modified 0. Locked 2.
DeviceMap ffffc903193e9bf0
Token ffffc9031c692060
ElapsedTime 00:01:13.571
UserTime 00:00:00.000
KernelTime 00:00:00.000
QuotaPoolUsage[PagedPool] 20344
QuotaPoolUsage[NonPagedPool] 3256
Working Set Sizes (now,min,max) (497, 50, 345) (1988KB, 200KB, 1380KB)
PeakWorkingSetSize 465
VirtualSize 4139 Mb
PeakVirtualSize 4139 Mb
PageFaultCount 499
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 107
Job ffffe00317be8060

[...]

THREAD ffffe003198ba0c0 Cid 1a90.18bc Teb: 000000161ab74000 Win32Thread: 0000000000000000 WAIT: (Executive) KernelMode Alertable
ffffe0031a7b0238 NotificationEvent
IRP List:
ffffe00318a49510: (0006,0238) Flags: 00060900 Mdl: ffffe00319319470
Not impersonating
DeviceMap ffffc903193e9bf0
Owning Process ffffe00318d60080 Image: InvalidPointerObject.exe
Attached Process N/A Image: N/A
Wait Start TickCount 6673 Ticks: 4692 (0:00:01:13.312)
Context Switch Count 118 IdealProcessor: 1
UserTime 00:00:00.000
KernelTime 00:00:00.015
*** WARNING: Unable to verify checksum for InvalidPointerObject.exe
Win32 Start Address InvalidPointerObject!wmainCRTStartup (0x00007ff66357e044)
Stack Init ffff848c00a22c90 Current ffff848c00a22560
Base ffff848c00a23000 Limit ffff848c00a1d000 Call 0000000000000000
Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP RetAddr Call Site
ffff848c`00a225a0 fffff800`7483c7bd nt!KiSwapContext+0x76
ffff848c`00a226e0 fffff800`7483b644 nt!KiSwapThread+0xbfd
ffff848c`00a22780 fffff800`7483ade5 nt!KiCommitThreadWait+0x144
ffff848c`00a22820 fffff800`74de982a nt!KeWaitForSingleObject+0x255
ffff848c`00a22900 fffff800`74de595f nt!IopSynchronousServiceTail+0x24a
ffff848c`00a229a0 fffff800`749d2e15 nt!NtReadFile+0x59f
ffff848c`00a22a90 00007ffb`0ed3c184 nt!KiSystemServiceCopyEnd+0x25 (TrapFrame @ ffff848c`00a22b00)
00000016`1a96f338 00007ffb`0c405227 ntdll!NtReadFile+0x14
00000016`1a96f340 00007ff6`6359b3b9 KERNELBASE!ReadFile+0x77
00000016`1a96f3c0 00000000`00000001 InvalidPointerObject!_read_nolock+0x2f5 [minkernel\crts\ucrt\src\appcrt\lowio\read.cpp @ 566]
00000016`1a96f3c8 00000000`00000000 0x1

1: kd> .thread /r /p ffffe003198ba0c0
Implicit thread is now ffffe003`198ba0c0
Implicit process is now ffffe003`18d60080
.cache forcedecodeuser done
Loading User Symbols
....

1: kd> kL
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 ffff848c`00a225a0 fffff800`7483c7bd nt!KiSwapContext+0x76
01 ffff848c`00a226e0 fffff800`7483b644 nt!KiSwapThread+0xbfd
02 ffff848c`00a22780 fffff800`7483ade5 nt!KiCommitThreadWait+0x144
03 ffff848c`00a22820 fffff800`74de982a nt!KeWaitForSingleObject+0x255
04 ffff848c`00a22900 fffff800`74de595f nt!IopSynchronousServiceTail+0x24a
05 ffff848c`00a229a0 fffff800`749d2e15 nt!NtReadFile+0x59f
06 ffff848c`00a22a90 00007ffb`0ed3c184 nt!KiSystemServiceCopyEnd+0x25
07 00000016`1a96f338 00007ffb`0c405227 ntdll!NtReadFile+0x14
*** WARNING: Unable to verify checksum for InvalidPointerObject.exe
08 00000016`1a96f340 00007ff6`6359b3b9 KERNELBASE!ReadFile+0x77
09 00000016`1a96f3c0 00000000`00000001 InvalidPointerObject!_read_nolock+0x2f5
0a 00000016`1a96f3c8 00000000`00000000 0x1

1: kd> !vad 146e3a70000 1

VAD @ ffffe0031a78eb10
Start VPN 146e3a70 End VPN 146e3a70 Control Area 0000000000000000
FirstProtoPte 0000000000000000 LastPte 0000000000000000 Commit Charge 1 (0n1)
Secured.Flink 0 Blink 0 Banked/Extend 0
File Offset 0
ViewUnmap MemCommit PrivateMemory READWRITE

1: kd> dc 146e3a70000
00000146`e3a70000 00000001 00000000 00000000 00000000 …………….
00000146`e3a70010 00000000 00000000 00000000 00000000 …………….
00000146`e3a70020 00000000 00000000 00000000 00000000 …………….
00000146`e3a70030 00000000 00000000 00000000 00000000 …………….
00000146`e3a70040 00000000 00000000 00000000 00000000 …………….
00000146`e3a70050 00000000 00000000 00000000 00000000 …………….
00000146`e3a70060 00000000 00000000 00000000 00000000 …………….
00000146`e3a70070 00000000 00000000 00000000 00000000 …………….

We see the page memory contents show the correct counter value (1):

struct Resource
{
    void DoSomething()
    {
        ++m_usageCounter;
    }

    std::size_t m_usageCounter{};
}; 

We resume system execution and hit a key. The program crashes in the second DoSomething call after releasing memory that contained Resource object:

	::VirtualFree(pMem, 0, MEM_RELEASE);

	pResource->DoSomething();

We wait until WER dialog appears (we had to add DWORD DontShowUI (0) to \HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting):

We then break in the system again and inspect the same address:

1: kd> g
Break instruction exception - code 80000003 (first chance)
*******************************************************************************
* *
* You are seeing this message because you pressed either *
* CTRL+C (if you run console kernel debugger) or, *
* CTRL+BREAK (if you run GUI kernel debugger), *
* on your debugger machine's keyboard. *
* *
* THIS IS NOT A BUG OR A SYSTEM CRASH *
* *
* If you did not intend to break into the debugger, press the "g" key, then *
* press the "Enter" key now. This message might immediately reappear. If it *
* does, press "g" and "Enter" again. *
* *
*******************************************************************************
nt!DbgBreakPointWithStatus:
fffff800`749c93a0 cc int 3

0: kd> .thread /r /p ffffe003198ba0c0
Implicit thread is now ffffe003`198ba0c0
Implicit process is now ffffe003`18d60080
.cache forcedecodeuser done
Loading User Symbols
....

0: kd> kL
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 ffff848c`00a21f70 fffff800`7483c7bd nt!KiSwapContext+0x76
01 ffff848c`00a220b0 fffff800`7483b644 nt!KiSwapThread+0xbfd
02 ffff848c`00a22150 fffff800`748884e7 nt!KiCommitThreadWait+0x144
03 ffff848c`00a221f0 fffff800`74e1ffe9 nt!KeWaitForMultipleObjects+0x287
04 ffff848c`00a22300 fffff800`74e1fd05 nt!ObWaitForMultipleObjects+0x2a9
05 ffff848c`00a22800 fffff800`749d2e15 nt!NtWaitForMultipleObjects+0x105
06 ffff848c`00a22a90 00007ffb`0ed3cc14 nt!KiSystemServiceCopyEnd+0x25
07 00000016`1a96e208 00007ffb`0c438027 ntdll!NtWaitForMultipleObjects+0x14
08 00000016`1a96e210 00007ffb`0c437f0e KERNELBASE!WaitForMultipleObjectsEx+0x107
09 00000016`1a96e510 00007ffb`0e0071fb KERNELBASE!WaitForMultipleObjects+0xe
0a 00000016`1a96e550 00007ffb`0e006ca8 KERNEL32!WerpReportFaultInternal+0x51b
0b 00000016`1a96e670 00007ffb`0c4df868 KERNEL32!WerpReportFault+0xac
0c 00000016`1a96e6b0 00007ffb`0ed44b32 KERNELBASE!UnhandledExceptionFilter+0x3b8
0d 00000016`1a96e7d0 00007ffb`0ed2c6d6 ntdll!RtlUserThreadStart$filt$0+0xa2
0e 00000016`1a96e810 00007ffb`0ed4121f ntdll!_C_specific_handler+0x96
0f 00000016`1a96e880 00007ffb`0ed0a289 ntdll!RtlpExecuteHandlerForException+0xf
10 00000016`1a96e8b0 00007ffb`0ed3fe8e ntdll!RtlDispatchException+0x219
11 00000016`1a96efc0 00007ff6`6357378a ntdll!KiUserExceptionDispatch+0×2e
*** WARNING: Unable to verify checksum for InvalidPointerObject.exe
12 00000016`1a96f6d8 00007ff6`63573875 InvalidPointerObject!Resource::DoSomething+0xa
13 00000016`1a96f6e0 00007ff6`6357dfd4 InvalidPointerObject!wmain+0xd5
14 (Inline Function) ——–`——– InvalidPointerObject!invoke_main+0×22
15 00000016`1a96f730 00007ffb`0dfb7bd4 InvalidPointerObject!__scrt_common_main_seh+0×10c
16 00000016`1a96f770 00007ffb`0ed0ced1 KERNEL32!BaseThreadInitThunk+0×14
17 00000016`1a96f7a0 00000000`00000000 ntdll!RtlUserThreadStart+0×21

0: kd> .frame 0n18;dv /t /v
12 00000016`1a96f6d8 00007ff6`63573875 InvalidPointerObject!Resource::DoSomething+0xa [C:\NewWork\InvalidPointerObject\InvalidPointerObject.cpp @ 10]
00000016`1a96f6e0 struct Resource * this = 0×00000146`e3a70000

0: kd> !vad 146e3a70000 1

VAD @ ffffe0031ab91080
Start VPN 146e3a70 End VPN 146e3a70 Control Area ffffe0031a66f780
FirstProtoPte ffffc9031b8fdf50 LastPte ffffc9031b8fdf50 Commit Charge 0 (0n0)
Secured.Flink 0 Blink 0 Banked/Extend 0
File Offset 0
ViewShare READWRITE

ControlArea @ ffffe0031a66f780
Segment ffffc9031c7d55d0 Flink ffffe0031ab91f40 Blink ffffe0031ab910e0
Section Ref 1 Pfn Ref 0 Mapped Views 3
User Ref 4 WaitForDel 0 Flush Count 1
File Object 0000000000000000 ModWriteCount 0 System Views 0
WritableRefs 0 PartitionId 0
Flags (2000) Commit

Pagefile-backed section

Segment @ ffffc9031c7d55d0
ControlArea ffffe0031a66f780 ExtendInfo 0000000000000000
Total Ptes 1
Segment Size 1000 Committed 1
CreatingProcessId 1a90 FirstMappedVa 146e3a70000
ProtoPtes ffffc9031b8fdf50
Flags (80000) ProtectionMask

0: kd> !ca ffffe0031a66f780 4

ControlArea @ ffffe0031a66f780
Segment ffffc9031c7d55d0 Flink ffffe0031ab91f40 Blink ffffe0031ab910e0
Section Ref 1 Pfn Ref 0 Mapped Views 3
User Ref 4 WaitForDel 0 Flush Count 1
File Object 0000000000000000 ModWriteCount 0 System Views 0
WritableRefs 0 PartitionId 0
Flags (2000) Commit

Pagefile-backed section

3 mapped view(s):

ffffe0031ab91f40 - VAD ffffe0031ab91ee0, process ffffe0031a8d3080 WerFault.exe
ffffe0031ab93ca0 - VAD ffffe0031ab93c40, process ffffe0031acd3080 InvalidPointer
ffffe0031ab910e0 - VAD ffffe0031ab91080, process ffffe00318d60080 InvalidPointer

0: kd> dc 146e3a70000
00000146`e3a70000 000000f0 00001a90 000018bc 00000000 …………….
00000146`e3a70010 00000000 00000000 00000000 00000000 …………….
00000146`e3a70020 00000000 00000000 00000000 00000000 …………….
00000146`e3a70030 00000000 00000000 00000000 00000000 …………….
00000146`e3a70040 00000000 00000000 00000000 00000000 …………….
00000146`e3a70050 00000000 00000000 00000000 00000000 …………….
00000146`e3a70060 00000000 00000000 00000000 00000000 …………….
00000146`e3a70070 00000000 00000000 00000000 00000000 …………….

We see the page contents changed (it now contains PID and TID) and also its pagefile-backed section lists 3 mapped views including 2 new processes, one is Zombie Process duplicate of the original InvalidPointerObject.exe process and another is WerFault.exe:

0: kd> !process ffffe0031acd3080
PROCESS ffffe0031acd3080
SessionId: 1 Cid: 06cc Peb: 161ab73000 ParentCid: 1a90
DirBase: 9d002002 ObjectTable: ffffc9031c03e5c0 HandleCount: 0.
Image: InvalidPointerObject.exe
VadRoot ffffe0031a789d90 Vads 21 Clone ffffe00318770810 Private 41. Modified 0. Locked 0.
DeviceMap 0000000000000000
Token ffffc9031d32a770
ElapsedTime 00:00:26.741
UserTime 00:00:00.000
KernelTime 00:00:00.000
QuotaPoolUsage[PagedPool] 17776
QuotaPoolUsage[NonPagedPool] 6024
Working Set Sizes (now,min,max) (28, 50, 345) (112KB, 200KB, 1380KB)
PeakWorkingSetSize 10
VirtualSize 4138 Mb
PeakVirtualSize 4138 Mb
PageFaultCount 28
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 51

No active threads

0: kd> !process 1a90
Searching for Process with Cid == 1a90
PROCESS ffffe00318d60080
SessionId: 1 Cid: 1a90 Peb: 161ab73000 ParentCid: 1474
DirBase: af7ee002 ObjectTable: ffffc9031c02a0c0 HandleCount: 39.
Image: InvalidPointerObject.exe
VadRoot ffffe0031a78d1c0 Vads 22 Clone ffffe00318770590 Private 43. Modified 18. Locked 0.
DeviceMap ffffc903193e9bf0
Token ffffc9031c692060
ElapsedTime 00:38:20.130
UserTime 00:00:00.000
KernelTime 00:00:00.000
QuotaPoolUsage[PagedPool] 20352
QuotaPoolUsage[NonPagedPool] 6328
Working Set Sizes (now,min,max) (547, 50, 345) (2188KB, 200KB, 1380KB)
PeakWorkingSetSize 515
VirtualSize 4139 Mb
PeakVirtualSize 4139 Mb
PageFaultCount 552
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 106
Job ffffe00317be8060

[...]

We resume system execution and collect the process crash dump. When we look at the crash address we see the same unexpected False Memory contents:

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(1a90.18bc): Access violation - code c0000005 (first/second chance not available)
For analysis of this file, run !analyze -v
ntdll!NtWaitForMultipleObjects+0x14:
00007ffb`0ed3cc14 c3 ret

0:000> kL
# Child-SP RetAddr Call Site
00 00000016`1a96e208 00007ffb`0c438027 ntdll!NtWaitForMultipleObjects+0x14
01 00000016`1a96e210 00007ffb`0c437f0e KERNELBASE!WaitForMultipleObjectsEx+0x107
02 00000016`1a96e510 00007ffb`0e0071fb KERNELBASE!WaitForMultipleObjects+0xe
03 00000016`1a96e550 00007ffb`0e006ca8 kernel32!WerpReportFaultInternal+0x51b
04 00000016`1a96e670 00007ffb`0c4df868 kernel32!WerpReportFault+0xac
05 00000016`1a96e6b0 00007ffb`0ed44b32 KERNELBASE!UnhandledExceptionFilter+0x3b8
06 00000016`1a96e7d0 00007ffb`0ed2c6d6 ntdll!RtlUserThreadStart$filt$0+0xa2
07 00000016`1a96e810 00007ffb`0ed4121f ntdll!_C_specific_handler+0x96
08 00000016`1a96e880 00007ffb`0ed0a289 ntdll!RtlpExecuteHandlerForException+0xf
09 00000016`1a96e8b0 00007ffb`0ed3fe8e ntdll!RtlDispatchException+0x219
0a 00000016`1a96efc0 00007ff6`6357378a ntdll!KiUserExceptionDispatch+0×2e
*** WARNING: Unable to verify checksum for InvalidPointerObject.exe
0b 00000016`1a96f6d8 00007ff6`63573875 InvalidPointerObject!Resource::DoSomething+0xa
0c 00000016`1a96f6e0 00007ff6`6357dfd4 InvalidPointerObject!wmain+0xd5
0d (Inline Function) ——–`——– InvalidPointerObject!invoke_main+0×22
0e 00000016`1a96f730 00007ffb`0dfb7bd4 InvalidPointerObject!__scrt_common_main_seh+0×10c
0f 00000016`1a96f770 00007ffb`0ed0ced1 kernel32!BaseThreadInitThunk+0×14
10 00000016`1a96f7a0 00000000`00000000 ntdll!RtlUserThreadStart+0×21

0:000> dx Debugger.Sessions[0].Processes[6800].Threads[6332].Stack.Frames[11].SwitchTo();dv /t /v
Debugger.Sessions[0].Processes[6800].Threads[6332].Stack.Frames[11].SwitchTo()
00000016`1a96f6e0 struct Resource * this = 0×00000146`e3a70000

0:000> !address 0×00000146`e3a70000

Usage: <unknown>
Base Address: 00000146`e3a70000
End Address: 00000146`e3a71000
Region Size: 00000000`00001000 ( 4.000 kB)
State: 00001000 MEM_COMMIT
Protect: 00000004 PAGE_READWRITE
Type: 00040000 MEM_MAPPED
Allocation Base: 00000146`e3a70000
Allocation Protect: 00000004 PAGE_READWRITE

Content source: 1 (target), length: 1000

0:000> dc 0×00000146`e3a70000
00000146`e3a70000 000000f0 00001a90 000018bc 00000000 …………….
00000146`e3a70010 00000000 00000000 00000000 00000000 …………….
00000146`e3a70020 00000000 00000000 00000000 00000000 …………….
00000146`e3a70030 00000000 00000000 00000000 00000000 …………….
00000146`e3a70040 00000000 00000000 00000000 00000000 …………….
00000146`e3a70050 00000000 00000000 00000000 00000000 …………….
00000146`e3a70060 00000000 00000000 00000000 00000000 …………….
00000146`e3a70070 00000000 00000000 00000000 00000000 …………….

0:000> ~
. 0 Id: 1a90.18bc Suspend: 0 Teb: 00000016`1ab74000 Unfrozen

0:000> dx -r1 ((InvalidPointerObject!Resource *)0x146e3a70000)
((InvalidPointerObject!Resource *)0x146e3a70000) : 0×146e3a70000 [Type: Resource *]
[+0×000] m_usageCounter : 0×1a90000000f0 [Type: unsigned __int64]

0:000> .ecxr
rax=00000146e3a70000 rbx=00000146e3aa5bf0 rcx=00000146e3a70000
rdx=0000000000000000 rsi=0000000000000000 rdi=00000146e3aa5c70
rip=00007ff66357378a rsp=000000161a96f6d8 rbp=0000000000000000
r8=000000161a96f6a8 r9=0000000000000000 r10=0000000000000000
r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202
InvalidPointerObject!Resource::DoSomething+0xa:
00007ff6`6357378a 488b00 mov rax,qword ptr [rax] ds:00000146`e3a70000=00001a90000000f0

Such False Memory may complicate the analysis of process crash dumps when we want to examine memory contents prior to exception.

The example memory dump, the application PDB file, and source code can be downloaded from here.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In addition to generic Invalid Pointer pattern that maps to visible pointer dereference in C and C++ code, plain NULL Code Pointers and NULL Data Pointers that are visible Small Values, and Wild Pointers showing ASCII or Regular Data (such as UNICODE fragments), we have implicit dereference (from C++ source code perspective) crash dump analysis patterns that we call Invalid Pointer (Objects). When seeing them in a high-level debugger (could be just an exception during debugging) developers are confused since they do not see the usual pointer dereference:

struct Resource
{
    void DoSomething()
    {
        ++m_usageCounter;
    }

    std::size_t m_usageCounter{};
}; 

However, the function call was ordinary (not virtual, otherwise we would have NULL Code Pointer), and the object address to access its members was passed via RCX register, but the memory of the object was invalid, hence we have an exception inside the method call when trying to access object members:

0:000> .ecxr
*** WARNING: Unable to verify checksum for InvalidPointerObject.exe
rax=0000022c837e0000 rbx=0000022c83905ca0 rcx=0000022c837e0000
rdx=0000000000000000 rsi=0000000000000000 rdi=0000022c83907540
rip=00007ff6d65630ba rsp=00000098812ffc18 rbp=0000000000000000
r8=00000098812ffbe8 r9=0000000000000000 r10=0000000000000000
r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202
InvalidPointerObject!Resource::DoSomething+0xa:
00007ff6`d65630ba 488b00 mov rax,qword ptr [rax] ds:0000022c`837e0000=????????????????

0:000> !address @rax

Usage:
Base Address: 0000022c`837e0000
End Address: 0000022c`837e1000
Region Size: 00000000`00001000 ( 4.000 kB)
State: 00002000 MEM_RESERVE
Protect:
Type: 00020000 MEM_PRIVATE
Allocation Base: 0000022c`837e0000
Allocation Protect: 00000004 PAGE_READWRITE

0:000> kL
# Child-SP RetAddr Call Site
00 00000098`812fe748 00007ffd`62278027 ntdll!NtWaitForMultipleObjects+0×14
01 00000098`812fe750 00007ffd`62277f0e KERNELBASE!WaitForMultipleObjectsEx+0×107
02 00000098`812fea50 00007ffd`63d871fb KERNELBASE!WaitForMultipleObjects+0xe
03 00000098`812fea90 00007ffd`63d86ca8 kernel32!WerpReportFaultInternal+0×51b
04 00000098`812febb0 00007ffd`6231f868 kernel32!WerpReportFault+0xac
05 00000098`812febf0 00007ffd`64ee4b32 KERNELBASE!UnhandledExceptionFilter+0×3b8
06 00000098`812fed10 00007ffd`64ecc6d6 ntdll!RtlUserThreadStart$filt$0+0xa2
07 00000098`812fed50 00007ffd`64ee121f ntdll!_C_specific_handler+0×96
08 00000098`812fedc0 00007ffd`64eaa289 ntdll!RtlpExecuteHandlerForException+0xf
09 00000098`812fedf0 00007ffd`64edfe8e ntdll!RtlDispatchException+0×219
0a 00000098`812ff500 00007ff6`d65630ba ntdll!KiUserExceptionDispatch+0×2e
0b 00000098`812ffc18 00007ff6`d656313c InvalidPointerObject!Resource::DoSomething+0xa
0c 00000098`812ffc20 00007ff6`d6568454 InvalidPointerObject!wmain+0×6c
0d (Inline Function) ——–`——– InvalidPointerObject!invoke_main+0×22
0e 00000098`812ffc70 00007ffd`63d37bd4 InvalidPointerObject!__scrt_common_main_seh+0×10c
0f 00000098`812ffcb0 00007ffd`64eaced1 kernel32!BaseThreadInitThunk+0×14
10 00000098`812ffce0 00000000`00000000 ntdll!RtlUserThreadStart+0×21

0:000> ub 00007ff6`d656313c
InvalidPointerObject!wmain+0×45:
00007ff6`d6563115 488b4c2428 mov rcx,qword ptr [rsp+28h]
00007ff6`d656311a e891ffffff call InvalidPointerObject!Resource::DoSomething (00007ff6`d65630b0)
00007ff6`d656311f 41b800400000 mov r8d,4000h
00007ff6`d6563125 33d2 xor edx,edx
00007ff6`d6563127 488b4c2420 mov rcx,qword ptr [rsp+20h]
00007ff6`d656312c ff15ce0e0200 call qword ptr [InvalidPointerObject!_imp_VirtualFree (00007ff6`d6584000)]
00007ff6`d6563132 488b4c2428 mov rcx,qword ptr [rsp+28h]
00007ff6`d6563137 e874ffffff call InvalidPointerObject!Resource::DoSomething (00007ff6`d65630b0)

0:000> u InvalidPointerObject!Resource::DoSomething
InvalidPointerObject!Resource::DoSomething:
00007ff6`d65630b0 48894c2408 mov qword ptr [rsp+8],rcx
00007ff6`d65630b5 488b442408 mov rax,qword ptr [rsp+8]
00007ff6`d65630ba 488b00 mov rax,qword ptr [rax]
00007ff6`d65630bd 48ffc0 inc rax
00007ff6`d65630c0 488b4c2408 mov rcx,qword ptr [rsp+8]
00007ff6`d65630c5 488901 mov qword ptr [rcx],rax
00007ff6`d65630c8 c3 ret
00007ff6`d65630c9 cc int 3

The example memory dump, the application PDB file, and source code can be downloaded from here.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -