Archive for the ‘Crash Dump Analysis’ Category

Crash Dump Analysis Patterns (Part 288)

Friday, February 23rd, 2024

Modern x64 Windows targets may support hardware shadow stacks. In such a case, WinDbg shows this message even if you open a memory dump on computers that do not support it:

This target supports Hardware-enforced Stack Protection. A HW based
"Shadow Stack" may be available to assist in debugging and analysis.
See aka.ms/userhsp for more info.
dps @ssp

The data from shadow stacks may be useful in case of Local Buffer Overflow. In such a case, we can compare the problem Stack Trace with the Shadow Stack Trace that was supposed to be without the stack region corruption.

For example, if see this exception and Incorrect Stack Trace, we can see that the stack trace should have been if the the return address were not modified:

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(5a34.4bb8): Security check failure or stack buffer overrun - code c0000409 (first/second chance not available)
Subcode: 0×39 FAST_FAIL_CONTROL_INVALID_RETURN_ADDRESS Shadow stack violation

0:000> k
# Child-SP RetAddr Call Site
00 000000fa`b94ffdf8 000002aa`5d420588 user32!GetMessageW+0×5c
01 000000fa`b94ffe00 000002aa`5d420588 0×000002aa`5d420588
02 000000fa`b94ffe08 00000000`00000000 0×000002aa`5d420588

0:000> r
rax=0000000000000001 rbx=000002aa5d420588 rcx=00007ff9c3d31534
rdx=0000000000000000 rsi=0000000000000000 rdi=000002aa5d420530
rip=00007ff9c3ea538c rsp=000000fab94ffdf8 rbp=000002aa5d420588
r8=000000fab94ffd98 r9=0000000000000000 r10=0000000000000000
r11=0000000000000244 r12=00007ff66b204070 r13=0000000000000000
r14=0000000000000001 r15=00000000ffffffff
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
user32!GetMessageW+0x5c:
00007ff9`c3ea538c ret

0:000> dps @rsp L1
000000fa`b94ffdf8 000002aa`5d420588

0:000> dps @ssp
000000fa`b95fefd0 00007ff9`4ae2f877 mfc140u!AfxInternalPumpMessage+0x27
000000fa`b95fefd8 00007ff9`4ae301b1 mfc140u!CWinThread::Run+0x81
000000fa`b95fefe0 00007ff9`4ae63230 mfc140u!AfxWinMain+0xc0
000000fa`b95fefe8 00007ff6`6b135742 mspaint+0xc5742
000000fa`b95feff0 00007ff9`c500257d kernel32!BaseThreadInitThunk+0x1d
000000fa`b95feff8 00007ff9`c618aa58 ntdll!RtlUserThreadStart+0x28
000000fa`b95ff000 ????????`????????
000000fa`b95ff008 ????????`????????
000000fa`b95ff010 ????????`????????
000000fa`b95ff018 ????????`????????
000000fa`b95ff020 ????????`????????
000000fa`b95ff028 ????????`????????
000000fa`b95ff030 ????????`????????
000000fa`b95ff038 ????????`????????
000000fa`b95ff040 ????????`????????
000000fa`b95ff048 ????????`????????

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 287)

Sunday, February 11th, 2024

Sometimes, when looking at Stack Traces or disassembly of the currently executing code we may see that continued execution would possibly (and sometimes definitely) generate an exception later on, like these sequences of recursive calls from sequential memory snapshots having the same Constant Subtrace may result in Stack Overflow:

0:000> kL
# ChildEBP RetAddr
00 00efeb60 771706c9 ntdll!NtDelayExecution+0xc
01 00efeb84 75fcd18f ntdll!RtlDelayExecution+0xe9
02 00efebec 75fcd12f KERNELBASE!SleepEx+0x4f
03 00efebfc 0014138e KERNELBASE!Sleep+0xf
04 00efec08 00141338 AppD9!ConnectDB+0xe
05 00efec10 0014121a AppD9!StartModeling+0x8
06 00efec70 75d32e53 AppD9!WndProc+0x7a
07 00efec9c 75d23c26 USER32!_InternalCallWinProc+0x2b
08 00efed94 75d224e5 USER32!UserCallWinProcCheckWow+0x4c6
09 00efee10 75d598f8 USER32!DispatchMessageWorker+0x4a5
0a 00efee58 75d59db3 USER32!DialogBox2+0x143
0b 00efee88 75d7ac60 USER32!InternalDialogBox+0xf3
0c 00efef54 75d799f6 USER32!SoftModalMessageBox+0x6f0
0d 00eff0b0 75d7a4f7 USER32!MessageBoxWorker+0x2fd
0e 00eff138 75d7a565 USER32!MessageBoxTimeoutW+0x187
0f 00eff158 00141371 USER32!MessageBoxW+0x45
10 00eff170 0014121a AppD9!StartModeling+0×41
11 00eff1d0 75d32e53 AppD9!WndProc+0×7a
12 00eff1fc 75d23c26 USER32!_InternalCallWinProc+0×2b
13 00eff2f4 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
14 00eff370 75d598f8 USER32!DispatchMessageWorker+0×4a5
15 00eff3b8 75d59db3 USER32!DialogBox2+0×143
16 00eff3e8 75d7ac60 USER32!InternalDialogBox+0xf3
17 00eff4b4 75d799f6 USER32!SoftModalMessageBox+0×6f0
18 00eff610 75d7a4f7 USER32!MessageBoxWorker+0×2fd
19 00eff698 75d7a565 USER32!MessageBoxTimeoutW+0×187
1a 00eff6b8 00141371 USER32!MessageBoxW+0×45
1b 00eff6d0 0014121a AppD9!StartModeling+0×41
1c 00eff730 75d32e53 AppD9!WndProc+0×7a
1d 00eff75c 75d23c26 USER32!_InternalCallWinProc+0×2b
1e 00eff854 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
1f 00eff8d0 75d598f8 USER32!DispatchMessageWorker+0×4a5
20 00eff918 75d59db3 USER32!DialogBox2+0×143
21 00eff948 75d7ac60 USER32!InternalDialogBox+0xf3
22 00effa14 75d799f6 USER32!SoftModalMessageBox+0×6f0
23 00effb70 75d7a4f7 USER32!MessageBoxWorker+0×2fd
24 00effbf8 75d7a565 USER32!MessageBoxTimeoutW+0×187
25 00effc18 00141371 USER32!MessageBoxW+0×45
26 00effc30 0014121a AppD9!StartModeling+0×41
27 00effc90 75d32e53 AppD9!WndProc+0×7a
28 00effcbc 75d23c26 USER32!_InternalCallWinProc+0×2b
29 00effdb4 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
2a 00effe30 75d22030 USER32!DispatchMessageWorker+0×4a5
2b 00effe3c 0014109d USER32!DispatchMessageW+0×10
2c 00effe68 0014155d AppD9!wWinMain+0×9d
2d (Inline) ——– AppD9!invoke_main+0×1a
2e 00effeb4 769a7ba9 AppD9!__scrt_common_main_seh+0xf8
2f 00effec4 7714bd2b KERNEL32!BaseThreadInitThunk+0×19
30 00efff1c 7714bcaf ntdll!__RtlUserThreadStart+0×2b
31 00efff2c 00000000 ntdll!_RtlUserThreadStart+0×1b

0:000> kL
# ChildEBP RetAddr
00 00efd5e0 771706c9 ntdll!NtDelayExecution+0xc
01 00efd604 75fcd18f ntdll!RtlDelayExecution+0xe9
02 00efd66c 75fcd12f KERNELBASE!SleepEx+0x4f
03 00efd67c 0014138e KERNELBASE!Sleep+0xf
04 00efd688 00141338 AppD9!ConnectDB+0xe
05 00efd690 0014121a AppD9!StartModeling+0x8
06 00efd6f0 75d32e53 AppD9!WndProc+0x7a
07 00efd71c 75d23c26 USER32!_InternalCallWinProc+0x2b
08 00efd814 75d224e5 USER32!UserCallWinProcCheckWow+0x4c6
09 00efd890 75d598f8 USER32!DispatchMessageWorker+0x4a5
0a 00efd8d8 75d59db3 USER32!DialogBox2+0x143
0b 00efd908 75d7ac60 USER32!InternalDialogBox+0xf3
0c 00efd9d4 75d799f6 USER32!SoftModalMessageBox+0x6f0
0d 00efdb30 75d7a4f7 USER32!MessageBoxWorker+0x2fd
0e 00efdbb8 75d7a565 USER32!MessageBoxTimeoutW+0x187
0f 00efdbd8 00141371 USER32!MessageBoxW+0x45
10 00efdbf0 0014121a AppD9!StartModeling+0×41
11 00efdc50 75d32e53 AppD9!WndProc+0×7a
12 00efdc7c 75d23c26 USER32!_InternalCallWinProc+0×2b
13 00efdd74 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
14 00efddf0 75d598f8 USER32!DispatchMessageWorker+0×4a5
15 00efde38 75d59db3 USER32!DialogBox2+0×143
16 00efde68 75d7ac60 USER32!InternalDialogBox+0xf3
17 00efdf34 75d799f6 USER32!SoftModalMessageBox+0×6f0
18 00efe090 75d7a4f7 USER32!MessageBoxWorker+0×2fd
19 00efe118 75d7a565 USER32!MessageBoxTimeoutW+0×187
1a 00efe138 00141371 USER32!MessageBoxW+0×45
1b 00efe150 0014121a AppD9!StartModeling+0×41
1c 00efe1b0 75d32e53 AppD9!WndProc+0×7a
1d 00efe1dc 75d23c26 USER32!_InternalCallWinProc+0×2b
1e 00efe2d4 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
1f 00efe350 75d598f8 USER32!DispatchMessageWorker+0×4a5
20 00efe398 75d59db3 USER32!DialogBox2+0×143
21 00efe3c8 75d7ac60 USER32!InternalDialogBox+0xf3
22 00efe494 75d799f6 USER32!SoftModalMessageBox+0×6f0
23 00efe5f0 75d7a4f7 USER32!MessageBoxWorker+0×2fd
24 00efe678 75d7a565 USER32!MessageBoxTimeoutW+0×187
25 00efe698 00141371 USER32!MessageBoxW+0×45
26 00efe6b0 0014121a AppD9!StartModeling+0×41
27 00efe710 75d32e53 AppD9!WndProc+0×7a
28 00efe73c 75d23c26 USER32!_InternalCallWinProc+0×2b
29 00efe834 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
2a 00efe8b0 75d598f8 USER32!DispatchMessageWorker+0×4a5
2b 00efe8f8 75d59db3 USER32!DialogBox2+0×143
2c 00efe928 75d7ac60 USER32!InternalDialogBox+0xf3
2d 00efe9f4 75d799f6 USER32!SoftModalMessageBox+0×6f0
2e 00efeb50 75d7a4f7 USER32!MessageBoxWorker+0×2fd
2f 00efebd8 75d7a565 USER32!MessageBoxTimeoutW+0×187
30 00efebf8 00141371 USER32!MessageBoxW+0×45
31 00efec10 0014121a AppD9!StartModeling+0×41
32 00efec70 75d32e53 AppD9!WndProc+0×7a
33 00efec9c 75d23c26 USER32!_InternalCallWinProc+0×2b
34 00efed94 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
35 00efee10 75d598f8 USER32!DispatchMessageWorker+0×4a5
36 00efee58 75d59db3 USER32!DialogBox2+0×143
37 00efee88 75d7ac60 USER32!InternalDialogBox+0xf3
38 00efef54 75d799f6 USER32!SoftModalMessageBox+0×6f0
39 00eff0b0 75d7a4f7 USER32!MessageBoxWorker+0×2fd
3a 00eff138 75d7a565 USER32!MessageBoxTimeoutW+0×187
3b 00eff158 00141371 USER32!MessageBoxW+0×45
3c 00eff170 0014121a AppD9!StartModeling+0×41
3d 00eff1d0 75d32e53 AppD9!WndProc+0×7a
3e 00eff1fc 75d23c26 USER32!_InternalCallWinProc+0×2b
3f 00eff2f4 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
40 00eff370 75d598f8 USER32!DispatchMessageWorker+0×4a5
41 00eff3b8 75d59db3 USER32!DialogBox2+0×143
42 00eff3e8 75d7ac60 USER32!InternalDialogBox+0xf3
43 00eff4b4 75d799f6 USER32!SoftModalMessageBox+0×6f0
44 00eff610 75d7a4f7 USER32!MessageBoxWorker+0×2fd
45 00eff698 75d7a565 USER32!MessageBoxTimeoutW+0×187
46 00eff6b8 00141371 USER32!MessageBoxW+0×45
47 00eff6d0 0014121a AppD9!StartModeling+0×41
48 00eff730 75d32e53 AppD9!WndProc+0×7a
49 00eff75c 75d23c26 USER32!_InternalCallWinProc+0×2b
4a 00eff854 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
4b 00eff8d0 75d598f8 USER32!DispatchMessageWorker+0×4a5
4c 00eff918 75d59db3 USER32!DialogBox2+0×143
4d 00eff948 75d7ac60 USER32!InternalDialogBox+0xf3
4e 00effa14 75d799f6 USER32!SoftModalMessageBox+0×6f0
4f 00effb70 75d7a4f7 USER32!MessageBoxWorker+0×2fd
50 00effbf8 75d7a565 USER32!MessageBoxTimeoutW+0×187
51 00effc18 00141371 USER32!MessageBoxW+0×45
52 00effc30 0014121a AppD9!StartModeling+0×41
53 00effc90 75d32e53 AppD9!WndProc+0×7a
54 00effcbc 75d23c26 USER32!_InternalCallWinProc+0×2b
55 00effdb4 75d224e5 USER32!UserCallWinProcCheckWow+0×4c6
56 00effe30 75d22030 USER32!DispatchMessageWorker+0×4a5
57 00effe3c 0014109d USER32!DispatchMessageW+0×10
58 00effe68 0014155d AppD9!wWinMain+0×9d
59 (Inline) ——– AppD9!invoke_main+0×1a
5a 00effeb4 769a7ba9 AppD9!__scrt_common_main_seh+0xf8
5b 00effec4 7714bd2b KERNEL32!BaseThreadInitThunk+0×19
5c 00efff1c 7714bcaf ntdll!__RtlUserThreadStart+0×2b
5d 00efff2c 00000000 ntdll!_RtlUserThreadStart+0×1b

We call such analysis pattern Near Exception.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 151b)

Sunday, January 28th, 2024

This is an unmanaged code analysis pattern variant of the previously published Annotated Disassembly. In modern WinDbg (which was previously called WinDbg Preview), the Disassembly window may annotate local variables in the presence of debugging symbols (this is absent from the output of the uf WinDbg command):

; uf command output
511 00007ff6`6ab22a44 mov dword ptr [rbp+2078h],1
511 00007ff6`6ab22a4e mov dword ptr [rbp+207Ch],2
513 00007ff6`6ab22a58 mov eax,dword ptr [rbp+2078h]
513 00007ff6`6ab22a5e mov dword ptr [rbp+0Ch],eax
514 00007ff6`6ab22a61 mov dword ptr [rbp+0Ch],64h
515 00007ff6`6ab22a68 mov dword ptr [rbp+48h],3
515 00007ff6`6ab22a6f mov dword ptr [rbp+4Ch],4
516 00007ff6`6ab22a76 mov eax,dword ptr [rbp+0Ch]


; Disassembly window
00007ff6`6ab22a4e c7857c20000002000000 mov dword ptr [myDerived.field2 (rbp+207Ch)], 2
00007ff6`6ab22a58 8b8578200000 mov eax, dword ptr [myDerived{.field} (rbp+2078h)]
00007ff6`6ab22a5e 89450c mov dword ptr [myBase{.field} (rbp+Ch)], eax
00007ff6`6ab22a61 c7450c64000000 mov dword ptr [myBase{.field} (rbp+Ch)], 64h
00007ff6`6ab22a68 c7454803000000 mov dword ptr [myDerived2{.field} (rbp+48h)], 3
00007ff6`6ab22a6f c7454c04000000 mov dword ptr [myDerived2.field2 (rbp+4Ch)], 4
00007ff6`6ab22a76 8b450c mov eax, dword ptr [myBase{.field} (rbp+Ch)]

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 286)

Sunday, January 28th, 2024

Sometimes, when we have debugging symbols, information about local variables may be helpful in making sense of function disassembly. For example, we have this code fragment from WinDbg uf command:

511 00007ff6`6ab22a44 mov dword ptr [rbp+2078h],1
511 00007ff6`6ab22a4e mov dword ptr [rbp+207Ch],2
513 00007ff6`6ab22a58 mov eax,dword ptr [rbp+2078h]
513 00007ff6`6ab22a5e mov dword ptr [rbp+0Ch],eax
514 00007ff6`6ab22a61 mov dword ptr [rbp+0Ch],64h
515 00007ff6`6ab22a68 mov dword ptr [rbp+48h],3
515 00007ff6`6ab22a6f mov dword ptr [rbp+4Ch],4
516 00007ff6`6ab22a76 mov eax,dword ptr [rbp+0Ch]

Although source code lines are shown, suppose we don’t have source code to match. However, we can match Address Representations, such as [rbp+xxx], from the output of dv /V WinDbg command:

0:000> dv /V
...
000000ab`740fd00c @rbp+0x000c myBase = struct wmain::__l2::Base
...
000000ab`740ff078 @rbp+0x2078 myDerived = struct wmain::__l2::Derived
...
000000ab`740fd048 @rbp+0x0048 myDerived2 = struct wmain::__l2::Derived
...

Another usage is matching values in raw stack data with local variable addresses. Values as addresses and their symbolic representations here have some connection to ADDR Symbolic and Interpreted Pointers.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 285)

Sunday, January 21st, 2024

Almost 15 years ago we introduced Dereference Fixpoints when the address value is equal to the value at the address. In doing raw stack data classification and pattern matching we may be interested in more general Dereference Nearpoints (especially in position independent ones) illustrated in the following diagram:

Such Dereference Nearpoints may appear due to exception processing when a stack exception address or exception stack pointer address is propagated during exception processing, and multiple structure references, for example, when a local structure address is propagated during function calls.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 284)

Saturday, July 1st, 2023

Sometimes, we are interested in Exception Collection from all different memory parts and space types and using different analysis patterns, for example, for the user and managed spaces:

- Stored Exception;

- Exception Stack Traces from Stack Trace Collection from unmanaged space;

- Managed Code Exceptions from CLR Runtime Threads (~*e !pe -nested and !Threads WinDbg commands) including Nested and Mixed Exceptions;

- Recorded heap failures (!heap -s -v) and other Historical Information;

- Hidden Exceptions (unmanaged space) in Execution Residue (unmanaged user space) for all threads;

- Hidden Exceptions (managed space) in Execution Residue (managed space) for all threads.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 283)

Saturday, April 8th, 2023

Shared Buffer Overwrite may happen via different mechanisms. A virtual address and its underlying physical page may be used by different threads from one process, or if threads from different processes are involved, its underlying physical memory page may be shared between different processes. In the former case, we can check threads’ Execution Residue for the page virtual address range. In the latter case, for example, when we have random crashes in different processes at different virtual addresses, we can compare page frame numbers for problem virtual addresses:

0: kd> !process ffffc38c3010b0c0 0
PROCESS ffffc38c3010b0c0
SessionId: 1 Cid: 1224 Peb: 24fc30b000 ParentCid: 1284
DirBase: 0a953002 ObjectTable: ffffac8a0b2aab40 HandleCount: 184.
Image: conhost.exe

0: kd> !process ffffc38c305e8080 0
PROCESS ffffc38c305e8080
SessionId: 0 Cid: 01c8 Peb: 4acc277000 ParentCid: 0290
DirBase: 10b62b002 ObjectTable: ffffac8a081b33c0 HandleCount: 276.
Image: svchost.exe

0: kd> !pte 00007ffc`884a0000
VA 00007ffc884a0000
PXE at FFFFFB7DBEDF67F8 PPE at FFFFFB7DBECFFF90 PDE at FFFFFB7D9FFF2210 PTE at FFFFFB3FFE442500
contains 8A0000000485F867 contains 0A00000115063867 contains 0A00000009D64867 contains 86000001358EF025
pfn 485f ---DA--UW-V pfn 115063 ---DA--UWEV pfn 9d64 ---DA--UWEV pfn 1358ef —-A–UR-V

0: kd> .process /r /p ffffc38c3010b0c0
Implicit process is now ffffc38c`3010b0c0
Loading User Symbols
.................................

0: kd> .process /r /p ffffc38c305e8080
Implicit process is now ffffc38c`305e8080
Loading User Symbols
..................................

0: kd> !pte 00007ffc`884a0000
VA 00007ffc884a0000
PXE at FFFFFB7DBEDF67F8 PPE at FFFFFB7DBECFFF90 PDE at FFFFFB7D9FFF2210 PTE at FFFFFB3FFE442500
contains 0A00000107137867 contains 0A0000010703A867 contains 0A0000010713B867 contains 81000001358EF005
pfn 107137 ---DA--UWEV pfn 10703a ---DA--UWEV pfn 10713b ---DA--UWEV pfn 1358ef ——-UR-V

We call such an analysis pattern Shared Page.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 282)

Sunday, November 27th, 2022

COM Object analysis pattern is similar to C++ Object because of the same binary compatibility (the first object member is a pointer (vptr) to a table of function pointers (vtbl):

0:003> !teb
TEB at 000000c0033d8000
ExceptionList: 0000000000000000
StackBase: 000000c003480000
StackLimit: 000000c00347a000
SubSystemTib: 0000000000000000
FiberData: 0000000000001e00
ArbitraryUserPointer: 0000000000000000
Self: 000000c0033d8000
EnvironmentPointer: 0000000000000000
ClientId: 00000000000012e8 . 00000000000023c8
RpcHandle: 0000000000000000
Tls Storage: 000002a0b8be97f0
PEB Address: 000000c0033d1000
LastErrorValue: 14007
LastStatusValue: c0150008
Count Owned Locks: 0
HardErrorMode: 0

0:003> dpp 000000c00347a000 000000c003480000
[...]
000000c0`0347d698 000000c0`0347d630 00000001`574f454d
000000c0`0347d6a0 000002a0`b3dc36e0 00007ffc`f5f98430 combase!CObjectContext::`vftable’
000000c0`0347d6a8 000002a0`00000000
000000c0`0347d6b0 000000c0`0347d9e8 00000000`00000000
000000c0`0347d6b8 000000c0`0347d7a0 00000000`00260001
000000c0`0347d6c0 000002a0`b8c07050 4b62055c`8a40a45d
000000c0`0347d6c8 000000c0`0347d9b0 000000c0`0347e2c0
000000c0`0347d6d0 000000c0`0347d9b0 000000c0`0347e2c0
000000c0`0347d6d8 00000000`00000010
000000c0`0347d6e0 0000eda0`2ba8550b
000000c0`0347d6e8 000002a0`b8c0cbe0 00007ffc`f5f9bae8 combase!CClientChannel::`vftable’
000000c0`0347d6f0 00000000`00000002
[…]

We have the following chain of memory addresses: 000000c0`0347d6a0 (the address of the object pointer) -> 000002a0`b3dc36e0 (the address of the object allocated from heap) -> 00007ffc`f5f98430 (vptr, the address of the first vtbl entry).

0:003> dps 00007ffc`f5f98430
00007ffc`f5f98430 00007ffc`f5dacd30 combase!CObjectContext::QueryInterface
00007ffc`f5f98438 00007ffc`f5e25120 combase!CObjectContext::AddRef
00007ffc`f5f98440 00007ffc`f5d8e990 combase!CObjectContext::Release
00007ffc`f5f98448 00007ffc`f5e769e0 combase!CObjectContext::SetProperty
00007ffc`f5f98450 00007ffc`f5ef7000 combase!CObjectContext::RemoveProperty
00007ffc`f5f98458 00007ffc`f5dffb00 combase!CObjectContext::GetProperty
00007ffc`f5f98460 00007ffc`f5ef57e0 combase!CObjectContext::EnumContextProps
00007ffc`f5f98468 00007ffc`f5e20e90 combase!CObjectContext::Freeze
00007ffc`f5f98470 00007ffc`f5ef57a0 combase!CObjectContext::DoCallback
00007ffc`f5f98478 00007ffc`f5ef7140 combase!CObjectContext::SetContextMarshaler
00007ffc`f5f98480 00007ffc`f5df7d50 combase!CObjectContext::GetContextMarshaler
00007ffc`f5f98488 00007ffc`f5ef7120 combase!CObjectContext::SetContextFlags
00007ffc`f5f98490 00007ffc`f5ef5340 combase!CObjectContext::ClearContextFlags
00007ffc`f5f98498 00007ffc`f5ef5960 combase!CObjectContext::GetContextFlags
00007ffc`f5f984a0 00007ffc`f5d8e750 combase!CObjectContext::FreezeWithApartmentSet
00007ffc`f5f984a8 00007ffc`f5d9b4c0 combase!CObjectContext::InternalContextCallback

The difference from a traditional C++ object (with virtual functions) layout is that the first 3 functions in vtbl (vftable) are QueryInterface, AddRef, and Release. In a C++ object, there can be an arbitrary number of function pointers with any corresponding symbolic names.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 29f)

Sunday, November 27th, 2022

This is High Contention pattern variant for network communication via sockets. Stack Trace Collection or Stack Trace Set may show frames with Winsock API (ws2_32 module) or SPI (WSP prefix, mswsock module) based on these template stack trace frames:

06 000000fa`f96eaa90 00007ffb`998d3e9f mswsock!WSPSend+0x1ce
07 000000fa`f96eab90 00007ffb`8bba1062 ws2_32!send+0x197

0:000> !findstack mswsock!WSP
Thread 008, 1 frame(s) match
* 06 000000faf4a0d628 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 009, 1 frame(s) match
* 06 000000faf4b0ca78 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 015, 1 frame(s) match
* 06 000000faf976bf98 00007ffb998d3e9f mswsock!WSPSend+0x1ce

Thread 021, 1 frame(s) match
* 06 000000faf96eaa88 00007ffb998d3e9f mswsock!WSPSend+0x1ce

Thread 026, 1 frame(s) match
* 10 000000fafa1eb168 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 043, 1 frame(s) match
* 06 000000faf8eebe68 00007ffb998d3e9f mswsock!WSPSend+0x1ce

Thread 051, 1 frame(s) match
* 10 000000fafa66bdf8 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 052, 1 frame(s) match
* 06 000000fafa6ebdb8 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 058, 1 frame(s) match
* 06 000000fafa9ea908 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 059, 1 frame(s) match
* 06 000000fafaa6b0e8 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 060, 1 frame(s) match
* 06 000000fafaaeb3b8 00007ffb998d3e9f mswsock!WSPSend+0x1ce

Thread 064, 1 frame(s) match
* 10 000000fafaceb7d8 00007ffb998d3e9f mswsock!WSPSend+0x1ce

Thread 069, 1 frame(s) match
* 06 000000fafaf6bfd8 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 073, 1 frame(s) match
* 06 000000fafb16c798 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 074, 1 frame(s) match
* 06 000000fafb1ec2b8 00007ffb998cffd3 mswsock!WSPSelect+0x4fa

Thread 080, 1 frame(s) match
* 10 000000fafb4eaf38 00007ffb998cf857 mswsock!WSPRecv+0x2ef

Thread 081, 1 frame(s) match
* 10 000000fafb56bd98 00007ffb998d3e9f mswsock!WSPSend+0x1ce

[...]

It is always good to compare the number of such suspicious threads with a normal memory dump.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 281)

Wednesday, June 8th, 2022

We have parallels between various Stack Trace analysis patterns and corresponding Stack Trace Collection analysis patterns, for example, for unmanaged space. The same can be done between Rough Stack Trace and the new analysis pattern that we call Rough Stack Trace Collection, for example, for unmanaged space. In WinDbg, such a collection can be done using a similar script but with dpS command instead. In essence, it is a collection of symbolic Execution Residue from all thread stack regions. This analysis pattern may help in identification of Ubiquitous Components not visible on stack traces, and Past Stack Traces, for example, corresponding to various leaks.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Exception Stack Trace, Stored Exception, Translated Exception, Execution Residue, Hidden Exception, NULL Pointer, Exception Module, Stack Trace Motif, No Component Symbols, and Coincidental Symbolic Information: pattern cooperation

Saturday, March 12th, 2022

We found the number of backgroundTaskHost.exe crash dumps in our honeypot MemoryDumps folder specified in the LocalDumps WER registry setup. All of them have the same Exception Stack Trace:

0:006> kc 10
# Call Site
00 ucrtbase!invoke_watson
01 vccorlib140_app!__abi_FailFast
02 vccorlib140_app!__abi_translateCurrentException
03 Microsoft_Applications_Telemetry_Windows!DllGetActivationFactory
04 VCRUNTIME140_1_APP!_CallSettingFrame_LookupContinuationIndex
05 VCRUNTIME140_1_APP!__FrameHandler4::CxxCallCatchBlock
06 ntdll!RcConsolidateFrames
07 Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: JsonFormatter::getJsonFormattedEvent
08 SurfaceApp!RHBinder__ShimExeMain
09 SurfaceApp!RHBinder__ShimExeMain
0a SurfaceApp!DllGetActivationFactory
0b SurfaceApp!DllGetActivationFactory
0c SurfaceApp!DllGetActivationFactory
0d SurfaceApp!DllGetActivationFactory
0e SurfaceApp!DllGetActivationFactory
0f SurfaceApp!DllGetActivationFactory
[...]

and the same Stored Exception:

0:006> .exr -1
ExceptionAddress: 00007ff96a66c648 (ucrtbase!invoke_watson+0x0000000000000018)
ExceptionCode: c0000409 (Security check failure or stack buffer overrun)
ExceptionFlags: 00000001
NumberParameters: 1
Parameter[0]: 0000000000000005
Subcode: 0×5 FAST_FAIL_INVALID_ARG

0:006> !error c0000409
Error code: (NTSTATUS) 0xc0000409 (3221226505) - The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application.

The !analyze -v command however reports a different exception address and its context that looks like invalid memory access via NULL Pointer (Data):

STACK_TEXT:
00000060`3d8fdaa0 00007ff9`25f36ba2 Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: GetDefaultConfiguration+0×5932
00000060`3d8fdad0 00007ff9`25f3904e Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: GetDefaultConfiguration+0×7dde
00000060`3d8fdda0 00007ff9`25fbc385 Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: JsonFormatter::getJsonFormattedEvent+0×46a25
00000060`3d8fde50 00007ff9`25fb722d Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: JsonFormatter::getJsonFormattedEvent+0×418cd
00000060`3d8fde80 00007ff8`c58bb5e5 SurfaceApp!RHBinder__ShimExeMain+0×4d0c55
00000060`3d8fdf50 00007ff8`c58e921b SurfaceApp!RHBinder__ShimExeMain+0×4fe88b
00000060`3d8fdfb0 00007ff8`c663977f SurfaceApp!DllGetActivationFactory+0×996d5f
00000060`3d8fdfe0 00007ff8`c6debbac SurfaceApp!DllGetActivationFactory+0×114918c
[…]

STACK_COMMAND: .cxr 603d8fd300 ; kb ; ** Pseudo Context ** Pseudo ** Value: 192e03234f0 ** ; kb
[…]

0:006> .cxr 603d8fd300
rax=0000000000000000 rbx=000000603d8fdb30 rcx=0000024030cb3300
rdx=0000024033346ea0 rsi=0000024030c7e910 rdi=0000024033346ea0
rip=00007ff925f36ba2 rsp=000000603d8fdaa0 rbp=000000603d8fdbd0
r8=0000000000000001 r9=0000000000000001 r10=00000fff24be7202
r11=4000000000000004 r12=0000000000000000 r13=0000024033bb2b28
r14=00000240333b9120 r15=00000240339835c8
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: GetDefaultConfiguration+0×5932:
00007ff9`25f36ba2 488b8090010000 mov rax,qword ptr [rax+190h] ds:00000000`00000190=????????????????

So we have a case of Translated Exception here. We can also find the Hidden Exception in Execution Residue:

0:006> !teb
TEB at 000000603d510000
ExceptionList: 0000000000000000
StackBase: 000000603d900000
StackLimit: 000000603d8f6000
SubSystemTib: 0000000000000000
FiberData: 0000000000001e00
ArbitraryUserPointer: 0000000000000000
Self: 000000603d510000
EnvironmentPointer: 0000000000000000
ClientId: 000000000000723c . 0000000000002288
RpcHandle: 0000000000000000
Tls Storage: 0000024030cfcd10
PEB Address: 000000603d503000
LastErrorValue: 0
LastStatusValue: c000007e
Count Owned Locks: 0
HardErrorMode: 0

0:006> dps 000000603d8f6000 000000603d900000
00000060`3d8f6000 00000000`00000000
00000060`3d8f6008 00000000`00000000
00000060`3d8f6010 00000000`00000000
00000060`3d8f6018 00000000`00000000
00000060`3d8f6020 00000000`00000000
00000060`3d8f6028 00000000`00000000
00000060`3d8f6030 00000000`00000000
[…]
00000060`3d8fd2d0 00000240`33bb2b28
00000060`3d8fd2d8 00000000`00000000
00000060`3d8fd2e0 00000240`33346ea0
00000060`3d8fd2e8 00000240`30c7e910
00000060`3d8fd2f0 00000060`3d8fdbd0
00000060`3d8fd2f8 00007ff9`6ce276fe ntdll!KiUserExceptionDispatch+0×2e
00000060`3d8fd300 00000000`00000000
00000060`3d8fd308 00000000`00000002
00000060`3d8fd310 00000060`3d8fdb30
00000060`3d8fd318 00000000`00000158
00000060`3d8fd320 00000000`00000002
00000060`3d8fd328 00000060`3d8fd3d9
00000060`3d8fd330 00001fa0`0010005f
00000060`3d8fd338 0053002b`002b0033
00000060`3d8fd340 00010206`002b002b
00000060`3d8fd348 00000000`00000000
00000060`3d8fd350 00000000`00000000
00000060`3d8fd358 00000000`00000000
00000060`3d8fd360 00000000`00000000
00000060`3d8fd368 00000000`00000000
00000060`3d8fd370 00000000`00000000
00000060`3d8fd378 00000000`00000000
00000060`3d8fd380 00000240`30cb3300
00000060`3d8fd388 00000240`33346ea0
00000060`3d8fd390 00000060`3d8fdb30
00000060`3d8fd398 00000060`3d8fdaa0
00000060`3d8fd3a0 00000060`3d8fdbd0
00000060`3d8fd3a8 00000240`30c7e910
00000060`3d8fd3b0 00000240`33346ea0
00000060`3d8fd3b8 00000000`00000001
00000060`3d8fd3c0 00000000`00000001
00000060`3d8fd3c8 00000fff`24be7202
00000060`3d8fd3d0 40000000`00000004
00000060`3d8fd3d8 00000000`00000000
00000060`3d8fd3e0 00000240`33bb2b28
00000060`3d8fd3e8 00000240`333b9120
00000060`3d8fd3f0 00000240`339835c8
00000060`3d8fd3f8 00007ff9`25f36ba2 Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: GetDefaultConfiguration+0×5932
00000060`3d8fd400 00000000`0000027f
00000060`3d8fd408 00000000`00000000
00000060`3d8fd410 00000000`00000000
00000060`3d8fd418 0000ffff`00001fa0
00000060`3d8fd420 00000000`00000000
[…]
00000060`3d8fd7e0 000001e0`000000f0
00000060`3d8fd7e8 00000000`00000000
00000060`3d8fd7f0 00000000`c0000005
00000060`3d8fd7f8 00000000`00000000
00000060`3d8fd800 00007ff9`25f36ba2 Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: GetDefaultConfiguration+0×5932
00000060`3d8fd808 00000000`00000002
00000060`3d8fd810 00000000`00000000
00000060`3d8fd818 00000000`00000190
00000060`3d8fd820 00000000`00000000
00000060`3d8fd828 00000000`00000000
00000060`3d8fd830 00000000`00000000
00000060`3d8fd838 00000000`00000000
00000060`3d8fd840 00000000`00000000
00000060`3d8fd848 00000000`00000000
[…]

0:006> .cxr 00000060`3d8fd300
[...]

0:006> k 3
# Child-SP RetAddr Call Site
00 00000060`3d8fdaa0 00007ff9`25f3904e Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: GetDefaultConfiguration+0×5932
01 00000060`3d8fdad0 00007ff9`25fbc385 Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: GetDefaultConfiguration+0×7dde
02 00000060`3d8fdda0 00007ff9`25fb722d Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: JsonFormatter::getJsonFormattedEvent+0×46a25

We see that Microsoft_Applications_Telemetry_Windows is Exception Module. We may think that it is related to JSON telemetry data based on Stack Trace Motif but getJsonFormattedEvent function offset is too large for a real function. So we have here Coincidental Symbolic Information of exported function due to No Component Symbols.

0:006> lm m Microsoft_Applications_Telemetry_Windows
Browse full module list
start end module name
00007ff9`25f10000 00007ff9`260f8000 Microsoft_Applications_Telemetry_Windows C (export symbols) Microsoft.Applications.Telemetry.Windows.dll

0:006> uf Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: JsonFormatter::getJsonFormattedEvent
Microsoft_Applications_Telemetry_Windows!Microsoft::Applications::Events:: JsonFormatter::getJsonFormattedEvent:
00007ff9`25f75960 48895c2408 mov qword ptr [rsp+8],rbx
00007ff9`25f75965 55 push rbp
00007ff9`25f75966 56 push rsi
00007ff9`25f75967 57 push rdi
00007ff9`25f75968 4154 push r12
00007ff9`25f7596a 4155 push r13
00007ff9`25f7596c 4156 push r14
00007ff9`25f7596e 4157 push r15
[…]
00007ff9`25f767df 4881c420010000 add rsp,120h
00007ff9`25f767e6 415f pop r15
00007ff9`25f767e8 415e pop r14
00007ff9`25f767ea 415d pop r13
00007ff9`25f767ec 415c pop r12
00007ff9`25f767ee 5f pop rdi
00007ff9`25f767ef 5e pop rsi
00007ff9`25f767f0 5d pop rbp
00007ff9`25f767f1 c3 ret

0:006> ? 00007ff9`25f767f1 - 00007ff9`25f75960
Evaluate expression: 3729 = 00000000`00000e91

We see that the function size is rather small compared to the offset value. This also “explains” that we don’t see any pointers to possible JSON strings in raw stack region data (dpa and dpu WinDbg commands) and if we do memory search there (s-sa and s-su commands).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 280)

Wednesday, March 9th, 2022

Black Box analysis pattern generalizes from the undocumented WinDbg commands !blackbox* to external system information included in process memory dump files which is accessible via .dumpdebug command.

2: kd> !blackboxpnp
PnpActivityId : {00000000-0000-0000-0000-000000000000}
PnpActivityTime : 132804247587428354
PnpEventInformation: 3
PnpEventInProgress : 0
PnpProblemCode : 24
PnpVetoType : 0
DeviceId : SW\{96E080C7-143C-11D1-B40F-00A0C9223196}\{3C0D501A-140B-11D1-B40F-00A0C9223196}
VetoString

Searching the registry we can find that it corresponds to “@ksfilter.inf,%mskssrv.devicedesc%;Microsoft Streaming Service Proxy”. Such commands may be used in conjunction with Historical Information (such as unloaded modules) and Execution Residue analysis patterns to check the last activities.

Other commands include !blackboxbsd and !blackboxntfs.

In process memory dump we may see information from a system the dump came from:

0:000> .dumpdebug
[...]
Stream 10: type SystemMemoryInfoStream (21), size 000001EC, RVA 00002288
Revision : 1
Flags : 0xf
BasicInfo
TimerResolution : 156,250
PageSize : 0x1000
NumberOfPhysicalPages : 4,173,065
LowestPhysicalPageNumber : 0x1
HighestPhysicalPageNumber : 0x46f7ff
AllocationGranularity : 0x10000
MinimumUserModeAddress : 0x10000
MaximumUserModeAddress : 0x7ffffffeffff
ActiveProcessorsAffinityMask : 0xff
NumberOfProcessors : 8
FileCacheInfo
CurrentSize : 514,248,704
PeakSize : 661,852,160
PageFaultCount : 19,464,228
MinimumWorkingSet : 0x100
MaximumWorkingSet : 0x100000000
CurrentSizeIncludingTransitionInPages : 1,327,191
PeakSizeIncludingTransitionInPages : 2,152,355
TransitionRePurposeCount : 8,923,412
Flags : 0
BasicPerfInfo
AvailablePages : 1,536,323
CommittedPages : 4,085,165
CommitLimit : 6,396,880
PeakCommitment : 4,850,269
PerfInfo
IdleProcessTime : 8,086,699,531,250
IoReadTransferCount : 97,860,850,993
IoWriteTransferCount : 55,567,419,561
IoOtherTransferCount : 9,725,039,400
IoReadOperationCount : 55,137,206
IoWriteOperationCount : 39,605,057
IoOtherOperationCount : 82,693,846
AvailablePages : 1,536,323
CommittedPages : 4,085,165
CommitLimit : 6,396,880
PeakCommitment : 4,850,269
CommitLimit : 6,396,880
PageFaultCount : 485,407,430
CopyOnWriteCount : 4,789,295
TransitionCount : 203,364,433
CacheTransitionCount : 0
DemandZeroCount : 275,205,178
PageReadCount : 9,363,018
PageReadIoCount : 1,641,521
CacheReadCount : 0
CacheIoCount : 0
DirtyPagesWriteCount : 295,086
DirtyWriteIoCount : 1,186
MappedPagesWriteCount : 425,398
MappedWriteIoCount : 5,656
PagedPoolPages : 231,590
NonPagedPoolPages : 155,982
PagedPoolAllocs : 0
PagedPoolFrees : 0
NonPagedPoolAllocs : 0
NonPagedPoolFrees : 0
FreeSystemPtes : 16,697,739
ResidentSystemCodePage : 4,175
TotalSystemDriverPages : 15,235
TotalSystemCodePages : 2
NonPagedPoolLookasideHits : 0
PagedPoolLookasideHits : 0
AvailablePagedPoolPages : 12,670,812
ResidentSystemCachePage : 125,549
ResidentPagedPoolPage : 220,095
ResidentSystemDriverPage : 13,012
CcFastReadNoWait : 0
CcFastReadWait : 13,492,886
CcFastReadResourceMiss : 0
CcFastReadNotPossible : 326,025
CcFastMdlReadNoWait : 0
CcFastMdlReadWait : 0
CcFastMdlReadResourceMiss : 0
CcFastMdlReadNotPossible : 0
CcMapDataNoWait : 0
CcMapDataWait : 77,200,777
CcMapDataNoWaitMiss : 0
CcMapDataWaitMiss : 391,734
CcPinMappedDataCount : 13,827,443
CcPinReadNoWait : 2,442
CcPinReadWait : 7,295,776
CcPinReadNoWaitMiss : 1,842,225
CcPinReadWaitMiss : 104,160
CcCopyReadNoWait : 720,327
CcCopyReadWait : 14,332,510
CcCopyReadNoWaitMiss : 73,632
CcCopyReadWaitMiss : 828,820
CcMdlReadNoWait : 0
CcMdlReadWait : 7,430
CcMdlReadNoWaitMiss : 0
CcMdlReadWaitMiss : 0
CcReadAheadIos : 1,577,774
CcLazyWriteIos : 737,095
CcLazyWritePages : 4,455,123
CcDataFlushes : 1,687,345
CcDataPages : 9,178,586
ContextSwitches : 690,599,392
FirstLevelTbFills : 0
SecondLevelTbFills : 0
SystemCalls : 2,382,592,584
CcTotalDirtyPages : 25,337
CcDirtyPageThreshold : 187,360
ResidentAvailablePages : 3,502,801
SharedCommittedPages : 693,491
Stream 11: type ProcessVmCountersStream (22), size 00000098, RVA 00002474
Revision : 2
Process Counters
PageFaultCount : 216,205
PeakWorkingSetSize : 0xdaa6000
WorkingSetSize : 0x160f000
QuotaPeakPagedPoolUsage : 0xfa0f8
QuotaPagedPoolUsage : 0xe8e88
QuotaPeakNonPagedPoolUsage : 0x22258
QuotaNonPagedPoolUsage : 0x180d8
PagefileUsage : 0xe6c000
PeakPagefileUsage : 0xcd67000
PeakVirtualSize : 0x201162a5000
VirtualSize : 0x20111ade000
PrivateUsage : 0xe6c000
PrivateWorkingSetSize : 0xb000
SharedCommitUsage : 0x1f2000
Job Counters
JobSharedCommitUsage : 0x72c000
JobPrivateCommitUsage : 0x71bc9000
JobPeakPrivateCommitUsage : 0x861ac000
JobPrivateCommitLimit : 0
JobTotalCommitLimit : 0
[...]

Other memory acquisition tools may write additional information in memory dump files. The difference between this analysis pattern and Paratext is that the latter involves additional files.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 16e)

Friday, February 4th, 2022

Stack Overflow caused by managed code is manifested as Stack Overflow (User Mode) with JIT Code recursive entries. !CLRStack WinDbg SOS extension command may work for very long if stack frame are small so we may need to increase the number of frames to show (.kframes command) and then manually check the originating frames using !IP2MD SOS extension command.

0:000> !CLRStack
OS Thread Id: 0x1da0 (0)
Child SP IP Call Site
000000F83D205FE0 00007ffc82570539 UserQuery.g__foo|4_1()
000000F83D206010 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206040 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206070 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D2060A0 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D2060D0 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206100 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206130 00007ffc8257053e UserQuery.g__foo|4_1()
000000F83D206160 00007ffc8257053e UserQuery.g__foo|4_1()
[...]

0:000> .kframes 0xFFFF
Default stack trace depth is 0n65535 frames

0:000> kL
# Child-SP RetAddr Call Site
00 000000f8`3d205fe0 00007ffc`8257053e 0x00007ffc`82570539
01 000000f8`3d206010 00007ffc`8257053e 0x00007ffc`8257053e
02 000000f8`3d206040 00007ffc`8257053e 0x00007ffc`8257053e
03 000000f8`3d206070 00007ffc`8257053e 0x00007ffc`8257053e
04 000000f8`3d2060a0 00007ffc`8257053e 0x00007ffc`8257053e
05 000000f8`3d2060d0 00007ffc`8257053e 0x00007ffc`8257053e
06 000000f8`3d206100 00007ffc`8257053e 0x00007ffc`8257053e
07 000000f8`3d206130 00007ffc`8257053e 0x00007ffc`8257053e
08 000000f8`3d206160 00007ffc`8257053e 0x00007ffc`8257053e
09 000000f8`3d206190 00007ffc`8257053e 0x00007ffc`8257053e
[...]
7cfa 000000f8`3d37cec0 00007ffc`8257053e 0x00007ffc`8257053e
7cfb 000000f8`3d37cef0 00007ffc`8257053e 0x00007ffc`8257053e
7cfc 000000f8`3d37cf20 00007ffc`8257053e 0x00007ffc`8257053e
7cfd 000000f8`3d37cf50 00007ffc`8257053e 0x00007ffc`8257053e
7cfe 000000f8`3d37cf80 00007ffc`8257053e 0x00007ffc`8257053e
7cff 000000f8`3d37cfb0 00007ffc`8257053e 0x00007ffc`8257053e
7d00 000000f8`3d37cfe0 00007ffc`8257053e 0x00007ffc`8257053e
7d01 000000f8`3d37d010 00007ffc`825704fe 0×00007ffc`8257053e
7d02 000000f8`3d37d040 00007ffc`825704c4 0×00007ffc`825704fe
7d03 000000f8`3d37d070 00007ffc`82582bdd 0×00007ffc`825704c4
7d04 000000f8`3d37d0a0 00007ffc`8236b45e 0×00007ffc`82582bdd
7d05 000000f8`3d37d940 00007ffc`82366850 0×00007ffc`8236b45e
7d06 000000f8`3d37dc10 00007ffc`82365faf 0×00007ffc`82366850
7d07 000000f8`3d37dd50 00007ffc`82365edc 0×00007ffc`82365faf
7d08 000000f8`3d37dd90 00007ffc`823316f5 0×00007ffc`82365edc
7d09 000000f8`3d37dde0 00007ffc`8233144b 0×00007ffc`823316f5
7d0a 000000f8`3d37de70 00007ffc`81de8db1 0×00007ffc`8233144b
7d0b 000000f8`3d37df60 00007ffc`81de59fa 0×00007ffc`81de8db1
7d0c 000000f8`3d37e0c0 00007ffc`81de5985 0×00007ffc`81de59fa
7d0d 000000f8`3d37e110 00007ffc`81de4d59 0×00007ffc`81de5985
7d0e 000000f8`3d37e160 00007ffc`81de45f5 0×00007ffc`81de4d59
7d0f 000000f8`3d37e1e0 00007ffc`e196a573 0×00007ffc`81de45f5
7d10 000000f8`3d37e220 00007ffc`e18902d0 coreclr!CallDescrWorkerInternal+0×83
7d11 (Inline Function) ——–`——– coreclr!CallDescrWorkerWithHandler+0×30
7d12 000000f8`3d37e260 00007ffc`e189202c coreclr!CallDescrWorkerReflectionWrapper+0×48
7d13 000000f8`3d37e2b0 00007ffc`d5ddc9d7 coreclr!RuntimeMethodHandle::InvokeMethod+0×91c
[…]
7d1b 000000f8`3d37ed60 00007ffc`e18e0d95 coreclr!RunMain+0xd2
7d1c 000000f8`3d37ee10 00007ffc`e18e0b56 coreclr!Assembly::ExecuteMainMethod+0×1c9
7d1d 000000f8`3d37f1a0 00007ffc`e19152b2 coreclr!CorHost2::ExecuteAssembly+0×1c6
7d1e 000000f8`3d37f310 00007ffd`053896bb coreclr!coreclr_execute_assembly+0xe2
7d1f (Inline Function) ——–`——– hostpolicy!coreclr_t::execute_assembly+0×2a
7d20 000000f8`3d37f3b0 00007ffd`053899ec hostpolicy!run_app_for_context+0×56b
7d21 000000f8`3d37f550 00007ffd`0538a387 hostpolicy!run_app+0×3c
7d22 000000f8`3d37f590 00007ffd`07fab539 hostpolicy!corehost_main+0×107
7d23 000000f8`3d37f740 00007ffd`07fae506 hostfxr!execute_app+0×2e9
7d24 000000f8`3d37f840 00007ffd`07fb0821 hostfxr!`anonymous namespace’::read_config_and_execute+0xa6
7d25 000000f8`3d37f940 00007ffd`07faeb62 hostfxr!fx_muxer_t::handle_exec_host_command+0×161
7d26 000000f8`3d37f9f0 00007ffd`07fa82ab hostfxr!fx_muxer_t::execute+0×482
7d27 000000f8`3d37fb30 00007ff6`64fe2351 hostfxr!hostfxr_main_startupinfo+0xab
7d28 000000f8`3d37fc30 00007ff6`64fe2748 LINQPad7_Query_exe!exe_start+0×651
7d29 000000f8`3d37fe60 00007ff6`64fe45f8 LINQPad7_Query_exe!wmain+0×88
7d2a (Inline Function) ——–`——– LINQPad7_Query_exe!invoke_main+0×22
7d2b 000000f8`3d37fe90 00007ffd`164b54e0 LINQPad7_Query_exe!__scrt_common_main_seh+0×10c
7d2c 000000f8`3d37fed0 00007ffd`185e485b kernel32!BaseThreadInitThunk+0×10
7d2d 000000f8`3d37ff00 00000000`00000000 ntdll!RtlUserThreadStart+0×2b

0:000> !IP2MD 0×00007ffc`8257053e
MethodDesc: 00007ffc8257ce18
Method Name: UserQuery.<Main>g__foo|4_1()
Class: 00007ffc8257cd08
MethodTable: 00007ffc8257ce48
mdToken: 0000000006000007
Module: 00007ffc8257c060
IsJitted: yes
Current CodeAddr: 00007ffc82570520
Version History:
ILCodeVersion: 0000000000000000
ReJIT ID: 0
IL Addr: 0000027575cc20f2
CodeAddr: 00007ffc82570520 (MinOptJitted)
NativeCodeVersion: 0000000000000000

0:000> !DumpIL 00007ffc8257ce18
ilAddr is 0000027575CC20F2 pImport is 000001C7B44109C0
ilAddr = 0000027575CC20F2
IL_0000: nop
IL_0001: call void UserQuery::<Main>g__foo|4_1()
IL_0006: nop
IL_0007: ret

0:000> !IP2MD 0×00007ffc`825704fe
MethodDesc: 00007ffc8257ce00
Method Name: UserQuery.<Main>g__bar|4_0()
Class: 00007ffc8257cd08
MethodTable: 00007ffc8257ce48
mdToken: 0000000006000006
Module: 00007ffc8257c060
IsJitted: yes
Current CodeAddr: 00007ffc825704e0
Version History:
ILCodeVersion: 0000000000000000
ReJIT ID: 0
IL Addr: 0000027575cc20f2
CodeAddr: 00007ffc825704e0 (MinOptJitted)
NativeCodeVersion: 0000000000000000

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 279)

Tuesday, November 23rd, 2021

Sometimes, we are interested in field values across many objects of the same type, for example, processes or threads. We call this analysis pattern Structure Field Collection. For example, we may be interested in all thread names or their number of context switches. Here’s an example script that outputs all non-null thread names and their _ETHREAD structure address for further exploration:

0: kd> !for_each_thread "r $t0 = @@C++(((nt!_ETHREAD *) @#Thread )->ThreadName); .if (@$t0 != 0) { .echo _ETHREAD: @#Thread; !ustr @$t0 }"
_ETHREAD: 0xffffad03ba43b080
String(46,46) at ffffad03b9a77790: Win32k Raw Input Thread
_ETHREAD: 0xffffad03ba468080
String(58,58) at ffffad03b6943e80: Win32k Desktop Thread (IO_DT)
_ETHREAD: 0xffffad03ba57d580
String(62,62) at ffffad03ba5aebc0: Win32k Desktop Thread (NOIO_DT)
_ETHREAD: 0xffffad03ba49b080
String(46,46) at ffffad03b9a792c0: Win32k Raw Input Thread
_ETHREAD: 0xffffad03ba49c080
String(58,58) at ffffad03b6945080: Win32k Desktop Thread (IO_DT)
_ETHREAD: 0xffffad03bcb44080
String(62,62) at ffffad03bcb89740: Win32k Desktop Thread (NOIO_DT)
_ETHREAD: 0xffffad03bad74080
String(38,38) at ffffad03bacf5a90: DWM LPC Port Thread
_ETHREAD: 0xffffad03bad70080
String(42,42) at ffffad03bacf6490: DWM Compositor Thread
_ETHREAD: 0xffffad03badbf080
String(32,32) at ffffad03ba5c7910: DWM Token Thread
_ETHREAD: 0xffffad03badbe080
String(46,46) at ffffad03bacf7340: DWM Master Input Thread
_ETHREAD: 0xffffad03badbd080
String(46,46) at ffffad03bacf7660: DWM Manipulation Thread
_ETHREAD: 0xffffad03bae71080
String(34,34) at ffffad03bacf82e0: uDWM Event Thread
_ETHREAD: 0xffffad03baf49080
String(32,32) at ffffad03ba5c8e10: OS Events thread
_ETHREAD: 0xffffad03baf98080
String(30,30) at ffffad03bafb4ed0: EventLog-System
_ETHREAD: 0xffffad03baf33080
String(40,40) at ffffad03baef7490: EventLog-Application
_ETHREAD: 0xffffad03bb00b080
String(34,34) at ffffad03baef74e0: EventLog-Security
_ETHREAD: 0xffffad03bbeee080
String(100,100) at ffffad03bc1ccaa0: MicrosoftWindows.Client.CBS_cw5n1h2txyewy!InputApp
_ETHREAD: 0xffffad03bc590080
String(30,30) at ffffad03bc75cd10: UnknownAppFrame
_ETHREAD: 0xffffad03bc539080
String(30,30) at ffffad03bc75f850: UnknownAppFrame
_ETHREAD: 0xffffad03bc20e300
String(44,44) at ffffad03bc0e44f0: DManip Delegate Thread
_ETHREAD: 0xffffad03bc208080
String(44,44) at ffffad03bc0e4770: DManip Delegate Thread
_ETHREAD: 0xffffad03bc457080
String(30,30) at ffffad03bc365cd0: WebView UI ASTA
_ETHREAD: 0xffffad03bc44a080
String(52,52) at ffffad03bc25be60: Chakra Background Recycler
_ETHREAD: 0xffffad03bc448080
String(52,52) at ffffad03bc25e620: Chakra Background Recycler
_ETHREAD: 0xffffad03bc4d1080
String(58,58) at ffffad03bc25de40: EdgeHtml Independent Hit Test
_ETHREAD: 0xffffad03bc4ce080
String(28,28) at ffffad03bc3671d0: EdgeHtml Timer
_ETHREAD: 0xffffad03bc4c1080
String(42,42) at ffffad03bc0e9950: EdgeHtml Download STA
_ETHREAD: 0xffffad03bc4c0080
String(58,58) at ffffad03bc25f8e0: Chakra Parallel Worker Thread
_ETHREAD: 0xffffad03bc4be080
String(40,40) at ffffad03bc0eae90: EdgeHtml Storage STA
_ETHREAD: 0xffffad03bc4bc080
String(34,34) at ffffad03bc93fbc0: Fetch Idle Worker
_ETHREAD: 0xffffad03bc46d080
String(30,30) at ffffad03bc36a1d0: EdgeHtml Render
_ETHREAD: 0xffffad03bc544080
String(26,26) at ffffad03bc0676d0: MTA Implicit
_ETHREAD: 0xffffad03bc68c040
String(26,26) at ffffad03bc363510: MTA Implicit
_ETHREAD: 0xffffad03bca08040
String(26,26) at ffffad03bc363550: MTA Implicit
_ETHREAD: 0xffffad03bca07080
String(50,50) at ffffad03bac853c0: EdgeHtml IDB MTA Implicit
_ETHREAD: 0xffffad03bca06080
String(26,26) at ffffad03bc363250: MTA Implicit
_ETHREAD: 0xffffad03bca04080
String(50,50) at ffffad03bb0fc170: EdgeHtml IDB MTA Implicit
_ETHREAD: 0xffffad03bc58e080
String(84,84) at ffffad03bbe4af10: WebPlatStorage Events Channel MTA Implicit
_ETHREAD: 0xffffad03bbcd5080
String(36,36) at ffffad03bc9445d0: EdgeHtml Image STA
_ETHREAD: 0xffffad03bc5df080
String(52,52) at ffffad03bc25a600: Chakra Background Recycler
_ETHREAD: 0xffffad03bc5de080
String(52,52) at ffffad03bc260060: Chakra Background Recycler
_ETHREAD: 0xffffad03bc68a080
String(44,44) at ffffad03bc0f3f40: DManip Delegate Thread
_ETHREAD: 0xffffad03bc5d8080
String(58,58) at ffffad03bc264f80: EdgeHtml Independent Hit Test
_ETHREAD: 0xffffad03bc5d7080
String(28,28) at ffffad03bc754510: EdgeHtml Timer
_ETHREAD: 0xffffad03bc0880c0
String(58,58) at ffffad03bc269a20: Chakra Parallel Worker Thread
_ETHREAD: 0xffffad03bc9e6080
String(12,12) at ffffad03bc51bd40: main()
_ETHREAD: 0xffffad03bc591080
String(20,20) at ffffad03bc75d010: InputPanel
_ETHREAD: 0xffffad03bc58c080
String(44,44) at ffffad03bc93e4a0: DManip Delegate Thread
_ETHREAD: 0xffffad03bc50e080
String(44,44) at ffffad03bc93f4e0: DManip Delegate Thread
_ETHREAD: 0xffffad03bc495040
String(26,26) at ffffad03bc3611d0: MTA Implicit
_ETHREAD: 0xffffad03bc494040
String(26,26) at ffffad03bc361a50: MTA Implicit
_ETHREAD: 0xffffad03bc490080
String(26,26) at ffffad03bc760bd0: MTA Implicit
_ETHREAD: 0xffffad03bc48f080
String(88,88) at ffffad03bbbe6890: RPC StorageEvents_WaitForEvents MTA Implicit
_ETHREAD: 0xffffad03bc2a5040
String(26,26) at ffffad03bc3631d0: MTA Implicit
_ETHREAD: 0xffffad03bc1c3040
String(26,26) at ffffad03bc361810: MTA Implicit
_ETHREAD: 0xffffad03bbced0c0
String(26,26) at ffffad03bc361350: MTA Implicit
_ETHREAD: 0xffffad03bca72080
String(52,52) at ffffad03bbdf06b0: Chakra Background Recycler
_ETHREAD: 0xffffad03bca71080
String(58,58) at ffffad03bbdf34d0: Chakra Parallel Worker Thread
_ETHREAD: 0xffffad03bc509080
String(26,26) at ffffad03bc369890: CrBrowserMain
_ETHREAD: 0xffffad03bcb45080
String(34,34) at ffffad03bc94e940: LoaderLockSampler
_ETHREAD: 0xffffad03bcb21080
String(22,22) at ffffad03bc368990: BrokerEvent
_ETHREAD: 0xffffad03bc682080
String(22,22) at ffffad03bc369950: HangWatcher
_ETHREAD: 0xffffad03bc681080
String(46,46) at ffffad03bc94f020: ThreadPoolServiceThread
_ETHREAD: 0xffffad03bc680080
String(106,106) at ffffad03bcb06800: ThreadPoolSingleThreadCOMSTASharedBackgroundBlocking0
_ETHREAD: 0xffffad03bc020080
String(106,106) at ffffad03bc1c8960: ThreadPoolSingleThreadCOMSTASharedForegroundBlocking1
_ETHREAD: 0xffffad03bc01e080
String(52,52) at ffffad03bc2663c0: ThreadPoolBackgroundWorker
_ETHREAD: 0xffffad03bc285080
String(30,30) at ffffad03bc3695d0: Chrome_IOThread
_ETHREAD: 0xffffad03bc284080
String(22,22) at ffffad03bc369910: MemoryInfra
_ETHREAD: 0xffffad03bc283080
String(90,90) at ffffad03bca64110: ThreadPoolSingleThreadCOMSTASharedForeground2
_ETHREAD: 0xffffad03bc1ed080
String(94,94) at ffffad03bbe41f10: ThreadPoolSingleThreadSharedBackgroundBlocking3
_ETHREAD: 0xffffad03bc1f2080
String(52,52) at ffffad03bcb8a040: ThreadPoolForegroundWorker
_ETHREAD: 0xffffad03bc1b7080
String(42,42) at ffffad03bc94fb10: CompositorTileWorker1
_ETHREAD: 0xffffad03bc1b6080
String(36,36) at ffffad03bc94fc00: VideoCaptureThread
_ETHREAD: 0xffffad03bcc0e080
String(30,30) at ffffad03bc75f950: BrowserWatchdog
_ETHREAD: 0xffffad03bcc0d080
String(94,94) at ffffad03bbe42010: ThreadPoolSingleThreadSharedBackgroundBlocking4
_ETHREAD: 0xffffad03bc29b080
String(82,82) at ffffad03bce44110: ThreadPoolSingleThreadForegroundBlocking5
_ETHREAD: 0xffffad03bcdda080
String(42,42) at ffffad03bb09a120: CacheThread_BlockFile
_ETHREAD: 0xffffad03bcdc50c0
String(94,94) at ffffad03bce44590: ThreadPoolSingleThreadSharedForegroundBlocking6
_ETHREAD: 0xffffad03bc67f4c0
String(106,106) at ffffad03bcb05690: ThreadPoolSingleThreadCOMSTASharedBackgroundBlocking7
_ETHREAD: 0xffffad03bca85080
String(52,52) at ffffad03bcb8cf80: ThreadPoolForegroundWorker
_ETHREAD: 0xffffad03bc6af080
String(36,36) at ffffad03bc94e580: CrashpadMainThread
_ETHREAD: 0xffffad03bca97080
String(42,42) at ffffad03bc94e620: ExitCodeWatcherThread
_ETHREAD: 0xffffad03bcc0c080
String(18,18) at ffffad03bc36dd10: CrGpuMain
_ETHREAD: 0xffffad03bcecd080
String(34,34) at ffffad03bc9518c0: LoaderLockSampler
_ETHREAD: 0xffffad03bcecb080
String(22,22) at ffffad03bc36e950: GpuWatchdog
_ETHREAD: 0xffffad03bce7e080
String(46,46) at ffffad03bc9535d0: ThreadPoolServiceThread
_ETHREAD: 0xffffad03bcdcc080
String(52,52) at ffffad03bc26c960: ThreadPoolBackgroundWorker
_ETHREAD: 0xffffad03bcdcb080
String(40,40) at ffffad03bc952c20: Chrome_ChildIOThread
_ETHREAD: 0xffffad03bcea8080
String(52,52) at ffffad03bcb8c8c0: ThreadPoolForegroundWorker
_ETHREAD: 0xffffad03bcea7080
String(38,38) at ffffad03bc952c70: VizCompositorThread
_ETHREAD: 0xffffad03bc7ee080
String(26,26) at ffffad03bc36d390: CrUtilityMain
_ETHREAD: 0xffffad03bc1f1080
String(34,34) at ffffad03bc951a00: LoaderLockSampler
_ETHREAD: 0xffffad03bca3d080
String(46,46) at ffffad03bc951c30: ThreadPoolServiceThread
_ETHREAD: 0xffffad03bca3a080
String(40,40) at ffffad03bc951c80: Chrome_ChildIOThread
_ETHREAD: 0xffffad03bce81080
String(52,52) at ffffad03bc26ca80: ThreadPoolBackgroundWorker
_ETHREAD: 0xffffad03bcbd0080
String(52,52) at ffffad03bcf03da0: ThreadPoolForegroundWorker
_ETHREAD: 0xffffad03b67af080
String(52,52) at ffffad03bdcca800: ThreadPoolForegroundWorker
_ETHREAD: 0xffffad03bd0db080
String(52,52) at ffffad03bcf0fd40: ThreadPoolForegroundWorker
_ETHREAD: 0xffffad03bbdcc080
String(52,52) at ffffad03bcf10280: ThreadPoolForegroundWorker
_ETHREAD: 0xffffad03bcdd94c0
String(26,26) at ffffad03bc36e7d0: CrUtilityMain
_ETHREAD: 0xffffad03baf95080
String(34,34) at ffffad03bc951aa0: LoaderLockSampler
_ETHREAD: 0xffffad03bcec9080
String(46,46) at ffffad03bc952270: ThreadPoolServiceThread
_ETHREAD: 0xffffad03bceb6080
String(52,52) at ffffad03bc26ca20: ThreadPoolBackgroundWorker
_ETHREAD: 0xffffad03bceb5080
String(40,40) at ffffad03bc952d60: Chrome_ChildIOThread
_ETHREAD: 0xffffad03bceb4080
String(52,52) at ffffad03bcb8ca40: ThreadPoolForegroundWorker
_ETHREAD: 0xffffad03bcf3e040
String(166,166) at ffffad03bcc38590: windows.immersivecontrolpanel_cw5n1h2txyewy!microsoft.windows.immersivecontrolpanel
_ETHREAD: 0xffffad03baf0a040
String(80,80) at ffffad03bcedb390: Microsoft.WindowsStore_8wekyb3d8bbwe!App
_ETHREAD: 0xffffad03b9850080
String(30,30) at ffffad03bc75e1d0: UnknownAppFrame
_ETHREAD: 0xffffad03bbf54080
String(30,30) at ffffad03ba344710: UnknownAppFrame
_ETHREAD: 0xffffad03ba48e080
String(44,44) at ffffad03ba32bf10: DManip Delegate Thread
_ETHREAD: 0xffffad03bada5080
String(44,44) at ffffad03bc950a10: DManip Delegate Thread

One of the early analysis patterns, Last Error Collection, is another instance of this general analysis pattern.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

CPU Consumption Patterns

Monday, November 22nd, 2021

A page to reference all different kinds of CPU consumption analysis patterns is necessary, so I created this post:

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Online Training: Accelerated Windows Memory Dump Analysis

Monday, November 22nd, 2021

I resume online training sessions. You can now register: https://www.patterndiagnostics.com/accelerated-windows-memory-dump-analysis

Crash Dump Analysis Patterns (Part 278)

Monday, November 22nd, 2021

In addition to the previous Spiking Thread and Distributed Spike CPU consumption analysis patterns we add Spiking Interrupts since they may account for perceived performance degradation such as response lags and system freezes. This pattern also includes DPC activity. We can see the times spent and the number of interrupts using this command and specify CPU number:

0: kd> !prcb 2
PRCB for Processor 2 at ffffe480b3600180:
Current IRQL — 2
Threads– Current ffffe480b360c240 Next 0000000000000000 Idle ffffe480b360c240
Processor Index 2 Number (0, 2) GroupSetMember 4
Interrupt Count — 0cadbd58
Times — Dpc 0000219c Interrupt 00002ae0

Kernel 00e7808e User 0041303b

0: kd> !whattime 0000219c + 00002ae0
19580 Ticks in Standard Time: 05:05.937s

We can also see the number of DPC requests from the structure itself:

0: kd> dt _KPRCB DPCData
nt!_KPRCB
+0×3340 DpcData : [2] _KDPC_DATA

0: kd> dt _KDPC_DATA
nt!_KDPC_DATA
+0x000 DpcList : _KDPC_LIST
+0x010 DpcLock : Uint8B
+0x018 DpcQueueDepth : Int4B
+0x01c DpcCount : Uint4B
+0x020 ActiveDpc : Ptr64 _KDPC
+0x028 LongDpcPresent : Uint4B
+0×02c Padding : Uint4B

0: kd> dt _KDPC_DATA ffffe480b3600180+0×3340
nt!_KDPC_DATA
+0×000 DpcList : _KDPC_LIST
+0×010 DpcLock : 0
+0×018 DpcQueueDepth : 0n1
+0×01c DpcCount : 0×74d9e0
+0×020 ActiveDpc : 0xffffa30f`e8f1f230 _KDPC
+0×028 LongDpcPresent : 0
+0×02c Padding : 0

0: kd> dt _KDPC_DATA ffffe480b3600180+0×3340+30
nt!_KDPC_DATA
+0×000 DpcList : _KDPC_LIST
+0×010 DpcLock : 0
+0×018 DpcQueueDepth : 0n0
+0×01c DpcCount : 0xd39
+0×020 ActiveDpc : (null)
+0×028 LongDpcPresent : 0
+0×02c Padding : 0

Since these numbers are high and depend on the system age, it is important to compare them with the normal system.

We should be aware that Windows 11 has DPC delegate threads (in addition to Idle threads) that are always shown as running even if they swapped (we can also check their number of context switches and kernel time):

0: kd> !process fffff80443332b00
PROCESS fffff80443332b00
SessionId: none Cid: 0000 Peb: 00000000 ParentCid: 0000
DirBase: 001ae002 ObjectTable: ffff82869fa52800 HandleCount: 3321.
Image: Idle
VadRoot ffffce8384257f70 Vads 2 Clone 0 Private 9. Modified 2094. Locked 0.
DeviceMap 0000000000000000
Token ffff82869fa1f120
ElapsedTime 3 Days 23:10:01.662
UserTime 00:00:00.000
KernelTime 00:00:00.000
QuotaPoolUsage[PagedPool] 0
QuotaPoolUsage[NonPagedPool] 272
Working Set Sizes (now,min,max) (9, 50, 450) (36KB, 200KB, 1800KB)
PeakWorkingSetSize 2
VirtualSize 0 Mb
PeakVirtualSize 0 Mb
PageFaultCount 9
MemoryPriority BACKGROUND
BasePriority 0
CommitCharge 15

THREAD fffff80443335bc0 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 0
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21906707 Ticks: 20013 (0:00:05:12.703)
Context Switch Count 72626555 IdealProcessor: 0
UserTime 00:00:00.000
KernelTime 3 Days 06:22:34.281
Win32 Start Address nt!KiIdleLoop (0xfffff80442a16a20)
Stack Init fffff8043f4beb70 Current fffff8043f4beb00
Base fffff8043f4bf000 Limit fffff8043f4b8000 Call 0000000000000000
Priority 0 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 5
Child-SP RetAddr Call Site
fffff804`3f4be490 fffff804`428eaa41 nt!PpmIdleGuestExecute+0x1d
fffff804`3f4be4d0 fffff804`428ea256 nt!PpmIdleExecuteTransition+0x661
fffff804`3f4be970 fffff804`42a16a74 nt!PoIdle+0x3a6
fffff804`3f4beb40 00000000`00000000 nt!KiIdleLoop+0x54

THREAD ffffe480b3519240 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 1
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21905854 Ticks: 20866 (0:00:05:26.031)
Context Switch Count 83248123 IdealProcessor: 1
UserTime 00:00:00.000
KernelTime 3 Days 08:20:45.812
Win32 Start Address nt!KiIdleLoop (0xfffff80442a16a20)
Stack Init ffffa30fe822fb70 Current ffffa30fe822fb00
Base ffffa30fe8230000 Limit ffffa30fe8229000 Call 0000000000000000
Priority 0 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e822f490 fffff804`428eaa41 nt!PpmIdleGuestExecute+0x1d
ffffa30f`e822f4d0 fffff804`428ea256 nt!PpmIdleExecuteTransition+0x661
ffffa30f`e822f970 fffff804`42a16a74 nt!PoIdle+0x3a6
ffffa30f`e822fb40 00000000`00000000 nt!KiIdleLoop+0x54

THREAD ffffe480b360c240 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 2
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21926718 Ticks: 2 (0:00:00:00.031)
Context Switch Count 90942117 IdealProcessor: 2
UserTime 00:00:00.000
KernelTime 2 Days 15:59:04.671
Win32 Start Address nt!KiIdleLoop (0xfffff80442a16a20)
Stack Init ffffa30fe823fb70 Current ffffa30fe823fb00
Base ffffa30fe8240000 Limit ffffa30fe8239000 Call 0000000000000000
Priority 0 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e823f6c8 fffff804`42b5d0f6 nt!KeBugCheckEx
ffffa30f`e823f6d0 fffff804`43068f46 nt!PnpBugcheckPowerTimeout+0x76
ffffa30f`e823f730 fffff804`428dcc74 nt!PopBuildDeviceNotifyListWatchdog+0x16
ffffa30f`e823f760 fffff804`428db264 nt!KiProcessExpiredTimerList+0x204
ffffa30f`e823f890 fffff804`42a16abe nt!KiRetireDpcList+0x714
ffffa30f`e823fb40 00000000`00000000 nt!KiIdleLoop+0x9e

THREAD ffffe480b370c240 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 3
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21905684 Ticks: 21036 (0:00:05:28.687)
Context Switch Count 66067949 IdealProcessor: 3
UserTime 00:00:00.000
KernelTime 3 Days 08:02:26.906
Win32 Start Address nt!KiIdleLoop (0xfffff80442a16a20)
Stack Init ffffa30fe824fb70 Current ffffa30fe824fb00
Base ffffa30fe8250000 Limit ffffa30fe8249000 Call 0000000000000000
Priority 0 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e824f490 fffff804`428eaa41 nt!PpmIdleGuestExecute+0x1d
ffffa30f`e824f4d0 fffff804`428ea256 nt!PpmIdleExecuteTransition+0x661
ffffa30f`e824f970 fffff804`42a16a74 nt!PoIdle+0x3a6
ffffa30f`e824fb40 00000000`00000000 nt!KiIdleLoop+0x54

THREAD ffffe480b380c240 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 4
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21905843 Ticks: 20877 (0:00:05:26.203)
Context Switch Count 91986345 IdealProcessor: 4
UserTime 00:00:00.000
KernelTime 3 Days 05:20:02.156
Win32 Start Address nt!KiIdleLoop (0xfffff80442a16a20)
Stack Init ffffa30fe825fb70 Current ffffa30fe825fb00
Base ffffa30fe8260000 Limit ffffa30fe8259000 Call 0000000000000000
Priority 0 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e825f490 fffff804`428eaa41 nt!PpmIdleGuestExecute+0x1d
ffffa30f`e825f4d0 fffff804`428ea256 nt!PpmIdleExecuteTransition+0x661
ffffa30f`e825f970 fffff804`42a16a74 nt!PoIdle+0x3a6
ffffa30f`e825fb40 00000000`00000000 nt!KiIdleLoop+0x54

THREAD ffffe480b389d240 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 5
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21905822 Ticks: 20898 (0:00:05:26.531)
Context Switch Count 78668897 IdealProcessor: 5
UserTime 00:00:00.000
KernelTime 3 Days 08:24:03.187
Win32 Start Address nt!KiIdleLoop (0xfffff80442a16a20)
Stack Init ffffa30fe826fb70 Current ffffa30fe826fb00
Base ffffa30fe8270000 Limit ffffa30fe8269000 Call 0000000000000000
Priority 0 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e826f490 fffff804`428eaa41 nt!PpmIdleGuestExecute+0x1d
ffffa30f`e826f4d0 fffff804`428ea256 nt!PpmIdleExecuteTransition+0x661
ffffa30f`e826f970 fffff804`42a16a74 nt!PoIdle+0x3a6
ffffa30f`e826fb40 00000000`00000000 nt!KiIdleLoop+0x54

THREAD ffffe480b39b3240 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 6
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21905853 Ticks: 20867 (0:00:05:26.046)
Context Switch Count 96137826 IdealProcessor: 6
UserTime 00:00:00.000
KernelTime 3 Days 06:36:10.375
Win32 Start Address nt!KiIdleLoop (0xfffff80442a16a20)
Stack Init ffffa30fe827fb70 Current ffffa30fe827fb00
Base ffffa30fe8280000 Limit ffffa30fe8279000 Call 0000000000000000
Priority 0 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e827f490 fffff804`428eaa41 nt!PpmIdleGuestExecute+0x1d
ffffa30f`e827f4d0 fffff804`428ea256 nt!PpmIdleExecuteTransition+0x661
ffffa30f`e827f970 fffff804`42a16a74 nt!PoIdle+0x3a6
ffffa30f`e827fb40 00000000`00000000 nt!KiIdleLoop+0x54

THREAD ffffe480b3b0c240 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 7
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21905670 Ticks: 21050 (0:00:05:28.906)
Context Switch Count 39349487 IdealProcessor: 7
UserTime 00:00:00.000
KernelTime 3 Days 06:49:50.156
Win32 Start Address nt!KiIdleLoop (0xfffff80442a16a20)
Stack Init ffffa30fe828fb70 Current ffffa30fe828fb00
Base ffffa30fe8290000 Limit ffffa30fe8289000 Call 0000000000000000
Priority 0 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e828f490 fffff804`428eaa41 nt!PpmIdleGuestExecute+0x1d
ffffa30f`e828f4d0 fffff804`428ea256 nt!PpmIdleExecuteTransition+0x661
ffffa30f`e828f970 fffff804`42a16a74 nt!PoIdle+0x3a6
ffffa30f`e828fb40 00000000`00000000 nt!KiIdleLoop+0x54

THREAD ffffce8384321140 Cid 0000.002c Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 0
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21906745 Ticks: 19975 (0:00:05:12.109)
Context Switch Count 55086 IdealProcessor: 0
UserTime 00:00:00.000
KernelTime 00:00:00.234
Win32 Start Address nt!KiExecuteDpcDelegate (0xfffff804429bb820)
Stack Init ffffa30fe82bfb70 Current ffffa30fe82bf8b0
Base ffffa30fe82c0000 Limit ffffa30fe82b9000 Call 0000000000000000
Priority 63 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e82bf8f0 fffff804`4285dc17 nt!KiSwapContext+0×76
ffffa30f`e82bfa30 fffff804`429bb87c nt!KiSwapThread+0×3a7
ffffa30f`e82bfb10 fffff804`42a16c24 nt!KiExecuteDpcDelegate+0×5c
ffffa30f`e82bfb40 00000000`00000000 nt!KiStartSystemThread+0×34

THREAD ffffce8384362080 Cid 0000.0034 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 1
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 16926767 Ticks: 4999953 (0:21:42:04.265)
Context Switch Count 4968 IdealProcessor: 1
UserTime 00:00:00.000
KernelTime 00:00:00.015
Win32 Start Address nt!KiExecuteDpcDelegate (0xfffff804429bb820)
Stack Init ffffa30fe82cfb70 Current ffffa30fe82cf8b0
Base ffffa30fe82d0000 Limit ffffa30fe82c9000 Call 0000000000000000
Priority 63 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e82cf8f0 fffff804`4285dc17 nt!KiSwapContext+0×76
ffffa30f`e82cfa30 fffff804`429bb87c nt!KiSwapThread+0×3a7
ffffa30f`e82cfb10 fffff804`42a16c24 nt!KiExecuteDpcDelegate+0×5c
ffffa30f`e82cfb40 00000000`00000000 nt!KiStartSystemThread+0×34

THREAD ffffce83842f7040 Cid 0000.003c Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 2
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21631408 Ticks: 295312 (0:01:16:54.250)
Context Switch Count 522 IdealProcessor: 2
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address nt!KiExecuteDpcDelegate (0xfffff804429bb820)
Stack Init ffffa30fe82dfb70 Current ffffa30fe82df8b0
Base ffffa30fe82e0000 Limit ffffa30fe82d9000 Call 0000000000000000
Priority 63 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e82df8f0 fffff804`4285dc17 nt!KiSwapContext+0×76
ffffa30f`e82dfa30 fffff804`429bb87c nt!KiSwapThread+0×3a7
ffffa30f`e82dfb10 fffff804`42a16c24 nt!KiExecuteDpcDelegate+0×5c
ffffa30f`e82dfb40 00000000`00000000 nt!KiStartSystemThread+0×34

THREAD ffffce8384367040 Cid 0000.0044 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 3
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 21667748 Ticks: 258972 (0:01:07:26.437)
Context Switch Count 301 IdealProcessor: 3
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address nt!KiExecuteDpcDelegate (0xfffff804429bb820)
Stack Init ffffa30fe82efb70 Current ffffa30fe82ef8b0
Base ffffa30fe82f0000 Limit ffffa30fe82e9000 Call 0000000000000000
Priority 63 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e82ef8f0 fffff804`4285dc17 nt!KiSwapContext+0×76
ffffa30f`e82efa30 fffff804`429bb87c nt!KiSwapThread+0×3a7
ffffa30f`e82efb10 fffff804`42a16c24 nt!KiExecuteDpcDelegate+0×5c
ffffa30f`e82efb40 00000000`00000000 nt!KiStartSystemThread+0×34

THREAD ffffce8384369040 Cid 0000.004c Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 4
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 20333183 Ticks: 1593537 (0:06:54:59.015)
Context Switch Count 405 IdealProcessor: 4
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address nt!KiExecuteDpcDelegate (0xfffff804429bb820)
Stack Init ffffa30fe82ffb70 Current ffffa30fe82ff8b0
Base ffffa30fe8300000 Limit ffffa30fe82f9000 Call 0000000000000000
Priority 63 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e82ff8f0 fffff804`4285dc17 nt!KiSwapContext+0×76
ffffa30f`e82ffa30 fffff804`429bb87c nt!KiSwapThread+0×3a7
ffffa30f`e82ffb10 fffff804`42a16c24 nt!KiExecuteDpcDelegate+0×5c
ffffa30f`e82ffb40 00000000`00000000 nt!KiStartSystemThread+0×34

THREAD ffffce838436b040 Cid 0000.0054 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 5
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 4760713 Ticks: 17166007 (3:02:30:18.859)
Context Switch Count 118 IdealProcessor: 5
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address nt!KiExecuteDpcDelegate (0xfffff804429bb820)
Stack Init ffffa30fe830fb70 Current ffffa30fe830f8b0
Base ffffa30fe8310000 Limit ffffa30fe8309000 Call 0000000000000000
Priority 63 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e830f8f0 fffff804`4285dc17 nt!KiSwapContext+0×76
ffffa30f`e830fa30 fffff804`429bb87c nt!KiSwapThread+0×3a7
ffffa30f`e830fb10 fffff804`42a16c24 nt!KiExecuteDpcDelegate+0×5c
ffffa30f`e830fb40 00000000`00000000 nt!KiStartSystemThread+0×34

THREAD ffffce838436d040 Cid 0000.005c Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 6
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 20662898 Ticks: 1263822 (0:05:29:07.218)
Context Switch Count 249 IdealProcessor: 6
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address nt!KiExecuteDpcDelegate (0xfffff804429bb820)
Stack Init ffffa30fe831fb70 Current ffffa30fe831f8b0
Base ffffa30fe8320000 Limit ffffa30fe8319000 Call 0000000000000000
Priority 63 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e831f8f0 fffff804`4285dc17 nt!KiSwapContext+0×76
ffffa30f`e831fa30 fffff804`429bb87c nt!KiSwapThread+0×3a7
ffffa30f`e831fb10 fffff804`42a16c24 nt!KiExecuteDpcDelegate+0×5c
ffffa30f`e831fb40 00000000`00000000 nt!KiStartSystemThread+0×34

THREAD ffffce838436f040 Cid 0000.0064 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 7
Not impersonating
DeviceMap ffff82869fa63c00
Owning Process fffff80443332b00 Image: Idle
Attached Process ffffce83842e6040 Image: System
Wait Start TickCount 20547550 Ticks: 1379170 (0:05:59:09.531)
Context Switch Count 196 IdealProcessor: 7
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address nt!KiExecuteDpcDelegate (0xfffff804429bb820)
Stack Init ffffa30fe832fb70 Current ffffa30fe832f8b0
Base ffffa30fe8330000 Limit ffffa30fe8329000 Call 0000000000000000
Priority 63 BasePriority 0 PriorityDecrement 0 IoPriority 0 PagePriority 0
Child-SP RetAddr Call Site
ffffa30f`e832f8f0 fffff804`4285dc17 nt!KiSwapContext+0×76
ffffa30f`e832fa30 fffff804`429bb87c nt!KiSwapThread+0×3a7
ffffa30f`e832fb10 fffff804`42a16c24 nt!KiExecuteDpcDelegate+0×5c
ffffa30f`e832fb40 00000000`00000000 nt!KiStartSystemThread+0×34

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 36b)

Tuesday, October 5th, 2021

When we added Local Buffer Overflow in 2007, we only added a short WinDbg output snippet of a user space example and didn’t elaborate much on stack reconstruction (although we wrote a separate modeling example, albeit 32-bit). Instead, we referenced a book on that topic that was available at that time. When working on the new exercise for the 5th edition of Accelerated Windows Memory Dump Analysis we realized the missing kernel space example. Many other patterns have both space analysis variants separately.

In addition to Incorrect Stack Traces we may also have Truncated Stack Traces:

1: kd> kc
# Call Site
00 nt!KeBugCheckEx
01 nt!KiDispatchException
02 nt!KiExceptionDispatch
03 nt!KiPageFault

For our try to reconstruct stack trace we need the boundaries of the stack region: its base (upper address, the stack grows towards lower addresses) and the stack pointer address for the current fault. We get both from the output of !thread and .trap WinDbg commands:

1: kd> !thread
THREAD ffff9a8e065f7080 Cid 1e7c.1e80 Teb: 000000ce1b0a7000 Win32Thread: ffff9a8e064c9a60 RUNNING on processor 1
[...]
Base ffffce833784d000 Limit ffffce8337847000 Call 0000000000000000
[…]
ffffce83`3784c950 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 fffff801`7e7f5e51 : nt!KiPageFault+0×443 (TrapFrame @ ffffce83`3784c950)

1: kd> .trap ffffce83`3784c950
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000000 rbx=0000000000000000 rcx=fffff8017f831b7f
rdx=fffff8017f830000 rsi=0000000000000000 rdi=0000000000000000
rip=0000000000000000 rsp=ffffce833784cae0 rbp=0000000000000002
r8=0000000000000000 r9=0000000000000000 r10=ffff9a8e060b62c0
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
00000000`00000000 ?? ???

We see that we have NULL Pointer (Code) here. We now try stack addresses from the top of Execution Residue unless we get a good stack trace:

1: kd> dps ffffce833784cae0 ffffce833784d000
ffffce83`3784cae0 00000000`00000000
ffffce83`3784cae8 00000000`00000000
ffffce83`3784caf0 00000000`00000000
ffffce83`3784caf8 fffff801`7e7f5e51 nt!ObpReferenceObjectByHandleWithTag+0×231
ffffce83`3784cb00 00000000`00000000
ffffce83`3784cb08 ffff868e`00000000
ffffce83`3784cb10 ffff86a6`83360010
ffffce83`3784cb18 ffff9a8e`05e8f990
ffffce83`3784cb20 ffff9a8e`060b62c0
ffffce83`3784cb28 00000000`00000000
ffffce83`3784cb30 ffff9a8e`06794a70
ffffce83`3784cb38 fffff801`7e48f865 nt!IofCallDriver+0×55
ffffce83`3784cb40 ffff9a8e`05e8f960
ffffce83`3784cb48 00000000`00000001
ffffce83`3784cb50 ffffce83`3784cec0
ffffce83`3784cb58 00000000`00000001
ffffce83`3784cb60 ffff9a8e`060b62c0
ffffce83`3784cb68 ffff9a8e`05e8fa78
ffffce83`3784cb70 ffff9a8e`06794a70
ffffce83`3784cb78 fffff801`7e875328 nt!IopSynchronousServiceTail+0×1a8
ffffce83`3784cb80 ffffce83`3784cec0
ffffce83`3784cb88 ffff9a8e`05e8f960
ffffce83`3784cb90 00000000`00000001
[…]

1: kd> k L=ffffce83`3784caf8
# Child-SP RetAddr Call Site
00 ffffce83`3784caf8 fffff801`7e7f5e51 0×0
01 ffffce83`3784cb00 ffff9a8e`05e8f960 nt!ObpReferenceObjectByHandleWithTag+0×231
02 ffffce83`3784cb90 00000000`00000001 0xffff9a8e`05e8f960
03 ffffce83`3784cb98 fffff801`00000000 0×1
04 ffffce83`3784cba0 00000000`00000000 0xfffff801`00000000

1: kd> k L=ffffce83`3784cb38
# Child-SP RetAddr Call Site
00 ffffce83`3784cb38 fffff801`7e48f865 0×0
01 ffffce83`3784cb40 fffff801`7e875328 nt!IofCallDriver+0×55
02 ffffce83`3784cb80 fffff801`7e874bf5 nt!IopSynchronousServiceTail+0×1a8
03 ffffce83`3784cc20 fffff801`7e8745f6 nt!IopXxxControlFile+0×5e5
04 ffffce83`3784cd60 fffff801`7e608bb5 nt!NtDeviceIoControlFile+0×56
05 ffffce83`3784cdd0 00007ffb`8dc6ce54 nt!KiSystemServiceCopyEnd+0×25
06 000000ce`1b2fea68 00000000`00000000 0×00007ffb`8dc6ce54

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 60c)

Sunday, October 3rd, 2021

This part is a kernel space counterpart to unmanaged user space Execution Residue. We get the boundaries of the stack region from the output of !thread command:

THREAD ffff9a8e065f7080 Cid 1e7c.1e80 Teb: 000000ce1b0a7000 Win32Thread: ffff9a8e064c9a60 RUNNING on processor 1
IRP List:
ffff9a8e05e8f960: (0006,0118) Flags: 00060000 Mdl: 00000000
Not impersonating
DeviceMap ffffaa81e3622e30
Owning Process ffff9a8e06992080 Image: process.exe
Attached Process N/A Image: N/A
Wait Start TickCount 7953 Ticks: 1 (0:00:00:00.015)
Context Switch Count 1386 IdealProcessor: 1
UserTime 00:00:00.046
KernelTime 00:00:00.078
Win32 Start Address 0x00007ff79e985384
Stack Init ffffce833784cfd0 Current ffffce833784c690
Base ffffce833784d000 Limit ffffce8337847000 Call 0000000000000000
Priority 12 BasePriority 8 PriorityDecrement 2 IoPriority 2 PagePriority 5

0: kd> dps ffffce8337847000 ffffce833784d000
[…]
ffffce83`3784b720 ffffffff`c0000000
ffffce83`3784b728 00000000`00040000
ffffce83`3784b730 fffff801`7e6f3b90 nt!HvlGetEncryptedData
ffffce83`3784b738 fffff801`7e712bd0 nt!KiBugCheckProgress
ffffce83`3784b740 00000000`00000000
ffffce83`3784b748 00000000`00000001
ffffce83`3784b750 fffff801`860d0b70 crashdmp!Context+0×50
ffffce83`3784b758 fffff801`860c695c crashdmp!DumpWrite+0×474
ffffce83`3784b760 fffff801`860d0b70 crashdmp!Context+0×50
ffffce83`3784b768 fffff801`7e712bd0 nt!KiBugCheckProgress
ffffce83`3784b770 00000000`50404286
ffffce83`3784b778 00000000`00002000
ffffce83`3784b780 00000000`0001f900
ffffce83`3784b788 fffff801`860cc123 crashdmp!CrashdmpTelemetrySaveEnvironmentVariable+0×5f
ffffce83`3784b790 ffff785d`5e18d8e1
ffffce83`3784b798 fffff801`860c290d crashdmp!CheckContextIntegrity+0×6d
ffffce83`3784b7a0 ffffffff`c0000005
ffffce83`3784b7a8 ffff9a8e`065f7080
ffffce83`3784b7b0 fffff801`7e712bd0 nt!KiBugCheckProgress
ffffce83`3784b7b8 00000000`0000001e
ffffce83`3784b7c0 00000000`00000000
ffffce83`3784b7c8 fffff801`860c50d6 crashdmp!CrashdmpWrite+0×1f6
ffffce83`3784b7d0 00000000`00000000
ffffce83`3784b7d8 ffffce83`3784b900
ffffce83`3784b7e0 fffff801`7e712bd0 nt!KiBugCheckProgress
ffffce83`3784b7e8 00000000`00000000
ffffce83`3784b7f0 00000000`00000000
ffffce83`3784b7f8 fffff801`7e6fdf0e nt!IoWriteCrashDump+0×53e
ffffce83`3784b800 ffffce83`3784bae0
ffffce83`3784b808 ffffce83`3784b900
ffffce83`3784b810 ffffce83`3784bae0
ffffce83`3784b818 00000000`00000000
ffffce83`3784b820 0067006f`00720050
ffffce83`3784b828 00730073`00650072
ffffce83`3784b830 00540050`00450000
ffffce83`3784b838 005f004e`004f0000
ffffce83`3784b840 00000000`00000000
ffffce83`3784b848 00000000`00000000
ffffce83`3784b850 00000000`00000000
ffffce83`3784b858 00000000`00000000
ffffce83`3784b860 ffff3902`484e7864
ffffce83`3784b868 fffff801`7e5c6b1a nt!IopIsAddressRangeValid+0×3e
ffffce83`3784b870 00000000`00c33a01
ffffce83`3784b878 00000000`00000008
ffffce83`3784b880 00000000`00000000
ffffce83`3784b888 00000000`00140000
ffffce83`3784b890 ffff9a8e`00f04038
ffffce83`3784b898 00000dff`00000000
ffffce83`3784b8a0 00000000`00000000
ffffce83`3784b8a8 ffff9a8e`065f7080
ffffce83`3784b8b0 ffffffff`c0000005
ffffce83`3784b8b8 fffff801`7e6fd6d0 nt!IoSetDumpRange
ffffce83`3784b8c0 fffff801`7e6fd060 nt!IoFreeDumpRange
ffffce83`3784b8c8 ffffce83`3784b888
ffffce83`3784b8d0 ffff9a8e`00f04000
ffffce83`3784b8d8 00000000`00000000
ffffce83`3784b8e0 00000000`00000000
ffffce83`3784b8e8 ffffffff`c0000005
ffffce83`3784b8f0 00000000`00000000
ffffce83`3784b8f8 00000000`00000008
ffffce83`3784b900 00000000`00000000
ffffce83`3784b908 ffff3902`484e7824
ffffce83`3784b910 00000000`0000001e
ffffce83`3784b918 ffff9a8e`065f7080
ffffce83`3784b920 00000000`00000001
ffffce83`3784b928 00000000`00000000
ffffce83`3784b930 00000000`00000003
ffffce83`3784b938 ffffd581`211c3180
ffffce83`3784b940 00000000`00000001
ffffce83`3784b948 00000000`00000000
ffffce83`3784b950 ffffce83`3784ba60
ffffce83`3784b958 fffff801`7e712456 nt!KeBugCheck2+0xca6
ffffce83`3784b960 00000000`00000001
ffffce83`3784b968 ffff9a8e`032bc000
ffffce83`3784b970 fffff801`7ee31a00 nt!KeBugCheckReasonCallbackListHead
ffffce83`3784b978 fffff801`7ee31a00 nt!KeBugCheckReasonCallbackListHead
ffffce83`3784b980 00000000`00000000
ffffce83`3784b988 ffffce83`3784bae0
ffffce83`3784b990 ffff9a8e`065f7080
ffffce83`3784b998 fffff801`7e712bd0 nt!KiBugCheckProgress
ffffce83`3784b9a0 ffffce83`3784c000
ffffce83`3784b9a8 00000000`00000000
ffffce83`3784b9b0 00000101`01000001
ffffce83`3784b9b8 ffff9a8e`065f7080
ffffce83`3784b9c0 00000000`0000001e
ffffce83`3784b9c8 00000000`00000000
ffffce83`3784b9d0 00000000`0000000f
ffffce83`3784b9d8 fffff801`7caf2100
ffffce83`3784b9e0 00000000`00000000
ffffce83`3784b9e8 00000000`00000000
ffffce83`3784b9f0 ffffd581`211c3180
ffffce83`3784b9f8 ffff86a6`00000004
ffffce83`3784ba00 00000000`00000000
ffffce83`3784ba08 ffff86a6`00000001
ffffce83`3784ba10 ffffce83`3784d000
ffffce83`3784ba18 ffffce83`37847000
ffffce83`3784ba20 fffff801`7e712bd0 nt!KiBugCheckProgress
ffffce83`3784ba28 fffff801`7e489594 nt!ExFreeHeapPool+0×4d4
ffffce83`3784ba30 00000000`00140001
ffffce83`3784ba38 00000000`00000001
ffffce83`3784ba40 00000000`00000000
ffffce83`3784ba48 00000000`00000000
ffffce83`3784ba50 00000000`00000000
ffffce83`3784ba58 00000000`00000000
ffffce83`3784ba60 00000000`00000000
ffffce83`3784ba68 00000000`00000000
ffffce83`3784ba70 00000000`00000000
ffffce83`3784ba78 00000000`00000000
ffffce83`3784ba80 00000000`00000000
ffffce83`3784ba88 00000000`00000000
ffffce83`3784ba90 00000000`00000000
ffffce83`3784ba98 00000000`00000000
ffffce83`3784baa0 00000000`00000000
ffffce83`3784baa8 00000000`00000000
ffffce83`3784bab0 00000000`00000000
ffffce83`3784bab8 00000000`00000000
ffffce83`3784bac0 00000000`00000000
ffffce83`3784bac8 00000000`00000000
ffffce83`3784bad0 00000000`00000000
ffffce83`3784bad8 fffff801`7e40ac67 nt!ExReleasePushLockSharedEx+0×37
ffffce83`3784bae0 00000000`00000000
ffffce83`3784bae8 00000000`00000000
ffffce83`3784baf0 00000000`00000000
ffffce83`3784baf8 00000000`00000000
ffffce83`3784bb00 00000000`00000000
ffffce83`3784bb08 00000000`00000000
ffffce83`3784bb10 00001f80`0010000f
ffffce83`3784bb18 0053002b`002b0010
ffffce83`3784bb20 00040246`0018002b
ffffce83`3784bb28 00000000`00000000
ffffce83`3784bb30 00000000`00000000
ffffce83`3784bb38 00000000`00000000
ffffce83`3784bb40 00000000`00000000
ffffce83`3784bb48 00000000`00000000
ffffce83`3784bb50 00000000`00000000
ffffce83`3784bb58 00000000`00000000
ffffce83`3784bb60 00000000`0000001e
ffffce83`3784bb68 ffffffff`c0000005
ffffce83`3784bb70 ffffce83`3784c8a8
ffffce83`3784bb78 ffffce83`3784c0a8
ffffce83`3784bb80 ffffce83`3784c5e0
ffffce83`3784bb88 ffffce83`3784c0e0
ffffce83`3784bb90 00000000`00000000
ffffce83`3784bb98 00000000`00000000
ffffce83`3784bba0 00000000`00000008
ffffce83`3784bba8 ffffce83`3784c8a8
ffffce83`3784bbb0 fffff801`7f000028 nt!PsInvertedFunctionTable+0×18
ffffce83`3784bbb8 00000000`00000000
ffffce83`3784bbc0 00000000`00000000
ffffce83`3784bbc8 ffffce83`3784c950
ffffce83`3784bbd0 00000000`0010001f
ffffce83`3784bbd8 fffff801`7e5f71c0 nt!KeBugCheckEx
ffffce83`3784bbe0 00000000`0000027f
ffffce83`3784bbe8 00000000`00000000
ffffce83`3784bbf0 00000000`00000000
ffffce83`3784bbf8 00000000`00001f80
ffffce83`3784bc00 00000000`00000000
ffffce83`3784bc08 00000000`00000000
ffffce83`3784bc10 00000000`00000000
ffffce83`3784bc18 00000000`00000000
ffffce83`3784bc20 00000000`00000000
ffffce83`3784bc28 00000000`00000000
ffffce83`3784bc30 00000000`00000000
ffffce83`3784bc38 00000000`00000000
ffffce83`3784bc40 00000000`00000000
ffffce83`3784bc48 00000000`00000000
ffffce83`3784bc50 00000000`00000000
ffffce83`3784bc58 00000000`00000000
ffffce83`3784bc60 00000000`00000000
ffffce83`3784bc68 00000000`00000000
ffffce83`3784bc70 00000000`00000000
ffffce83`3784bc78 00000000`00000000
ffffce83`3784bc80 00000000`00000000
ffffce83`3784bc88 00000000`00000000
ffffce83`3784bc90 00000000`00000000
ffffce83`3784bc98 ffff86a6`006136a0
ffffce83`3784bca0 00000000`00000000
ffffce83`3784bca8 00000000`00000000
ffffce83`3784bcb0 00000000`00000000
ffffce83`3784bcb8 00000000`00000000
ffffce83`3784bcc0 00000000`00000000
ffffce83`3784bcc8 00000000`00000000
ffffce83`3784bcd0 00000000`00000000
ffffce83`3784bcd8 00000000`00000000
ffffce83`3784bce0 00000000`00000000
ffffce83`3784bce8 00000000`00000000
ffffce83`3784bcf0 00000000`00000000
ffffce83`3784bcf8 00000000`00000000
ffffce83`3784bd00 00000000`00000000
ffffce83`3784bd08 00000000`00000000
ffffce83`3784bd10 00000000`00000000
ffffce83`3784bd18 00000000`00000000
ffffce83`3784bd20 00000000`00000000
ffffce83`3784bd28 00000000`00000000
ffffce83`3784bd30 00000000`00000000
ffffce83`3784bd38 00000000`00000000
ffffce83`3784bd40 00000000`00000000
ffffce83`3784bd48 00000000`00000000
ffffce83`3784bd50 00000000`00000000
ffffce83`3784bd58 00000000`00000000
ffffce83`3784bd60 00000000`00000000
ffffce83`3784bd68 00000000`00000000
ffffce83`3784bd70 00000000`00000000
ffffce83`3784bd78 00000000`00000000
ffffce83`3784bd80 00000000`00000000
ffffce83`3784bd88 00000000`00000000
ffffce83`3784bd90 00000000`00000000
ffffce83`3784bd98 00000000`00000000
ffffce83`3784bda0 00000000`00000000
ffffce83`3784bda8 00000000`00000000
ffffce83`3784bdb0 00000000`00000000
ffffce83`3784bdb8 00000000`00000000
ffffce83`3784bdc0 00000000`00000000
ffffce83`3784bdc8 00000000`00000000
ffffce83`3784bdd0 00000000`00000000
ffffce83`3784bdd8 00000000`00000000
ffffce83`3784bde0 00000000`00000000
ffffce83`3784bde8 00000000`00000000
ffffce83`3784bdf0 00000000`00000000
ffffce83`3784bdf8 00000000`00000000
ffffce83`3784be00 00000000`00000000
ffffce83`3784be08 00000000`00000000
ffffce83`3784be10 00000000`00000000
ffffce83`3784be18 00000000`00000000
ffffce83`3784be20 00000000`00000000
ffffce83`3784be28 00000000`00000000
ffffce83`3784be30 00000000`00000000
ffffce83`3784be38 00000000`00000000
ffffce83`3784be40 00000000`00000000
ffffce83`3784be48 00000000`00000000
ffffce83`3784be50 00000000`00000000
ffffce83`3784be58 00000000`00000000
ffffce83`3784be60 00000000`00000000
ffffce83`3784be68 00000000`00000000
ffffce83`3784be70 00000000`00000000
ffffce83`3784be78 00000000`00000000
ffffce83`3784be80 00000000`00000000
ffffce83`3784be88 00000000`00000000
ffffce83`3784be90 00000000`00000000
ffffce83`3784be98 00000000`00000000
ffffce83`3784bea0 00000000`00000000
ffffce83`3784bea8 00000000`00000000
ffffce83`3784beb0 00000000`00000000
ffffce83`3784beb8 00000000`00000000
ffffce83`3784bec0 00000000`00000000
ffffce83`3784bec8 00000000`00000000
ffffce83`3784bed0 00000000`00000000
ffffce83`3784bed8 00000000`00000000
ffffce83`3784bee0 00000000`00000000
ffffce83`3784bee8 00000000`00000000
ffffce83`3784bef0 00000000`00000000
ffffce83`3784bef8 00000000`00000000
ffffce83`3784bf00 00000000`00000000
ffffce83`3784bf08 00000000`00000000
ffffce83`3784bf10 00000000`00000000
ffffce83`3784bf18 00000000`00000000
ffffce83`3784bf20 00000000`00000000
ffffce83`3784bf28 00000000`00000000
ffffce83`3784bf30 00000000`00000000
ffffce83`3784bf38 00000000`00000000
ffffce83`3784bf40 00000000`00000000
ffffce83`3784bf48 00000000`00000000
ffffce83`3784bf50 00000000`00000000
ffffce83`3784bf58 00000000`00000000
ffffce83`3784bf60 00000000`00000000
ffffce83`3784bf68 00000000`00000000
ffffce83`3784bf70 00000000`00000000
ffffce83`3784bf78 00000000`00000000
ffffce83`3784bf80 00000000`00000000
ffffce83`3784bf88 00000000`00000000
ffffce83`3784bf90 00000000`00000000
ffffce83`3784bf98 00000000`00000000
ffffce83`3784bfa0 00000000`00000000
ffffce83`3784bfa8 00000000`00000000
ffffce83`3784bfb0 00000000`00000000
ffffce83`3784bfb8 00000000`00000000
ffffce83`3784bfc0 00000000`00000000
ffffce83`3784bfc8 00000000`00000000
ffffce83`3784bfd0 00000000`00000000
ffffce83`3784bfd8 00000000`00000000
ffffce83`3784bfe0 00000000`00000000
ffffce83`3784bfe8 00000000`00000000
ffffce83`3784bff0 00000000`00000000
ffffce83`3784bff8 00000000`00000000
ffffce83`3784c000 00000000`00000000
ffffce83`3784c008 00000000`00000000
ffffce83`3784c010 00000000`00000000
ffffce83`3784c018 00000000`00000000
ffffce83`3784c020 00000000`00000000
ffffce83`3784c028 00000000`00000000
ffffce83`3784c030 00000000`0010001f
ffffce83`3784c038 ffffce83`3784c950
ffffce83`3784c040 00000000`00000000
ffffce83`3784c048 00000000`00000000
ffffce83`3784c050 00000000`00000000
ffffce83`3784c058 ffffce83`3784c0e0
ffffce83`3784c060 ffffce83`3784c5e0
ffffce83`3784c068 fffff801`7e5f72c7 nt!KeBugCheckEx+0×107
ffffce83`3784c070 ffffce83`3784c8a8
ffffce83`3784c078 ffffce83`3784c5e0
ffffce83`3784c080 ffffce83`3784c8a8
ffffce83`3784c088 00000000`00000000
ffffce83`3784c090 00000000`00000000
ffffce83`3784c098 00000000`00000000
ffffce83`3784c0a0 00000000`00040246
ffffce83`3784c0a8 fffff801`7e659ecb nt!KiDispatchException+0×17467b
ffffce83`3784c0b0 00000000`0000001e
ffffce83`3784c0b8 ffffffff`c0000005
ffffce83`3784c0c0 00000000`00000000
ffffce83`3784c0c8 00000000`00000008
ffffce83`3784c0d0 00000000`00000000
ffffce83`3784c0d8 00000000`00000001
ffffce83`3784c0e0 ffff86a6`0312a600
ffffce83`3784c0e8 ffff86a6`0312a688
ffffce83`3784c0f0 ffff86a6`0312a4e8
ffffce83`3784c0f8 00000000`00000d0d
ffffce83`3784c100 00000001`00000000
ffffce83`3784c108 00000000`00000000
ffffce83`3784c110 00001f80`0010001f
ffffce83`3784c118 0053002b`002b0010
ffffce83`3784c120 00050282`0018002b
ffffce83`3784c128 00000000`00000000
ffffce83`3784c130 00000000`00000000
ffffce83`3784c138 00000000`00000000
ffffce83`3784c140 00000000`00000000
ffffce83`3784c148 00000000`00000000
ffffce83`3784c150 00000000`00000000
[…]
ffffce83`3784cda0 00000000`00000000
ffffce83`3784cda8 ffffce83`00000000
ffffce83`3784cdb0 ffff86a6`00000001
ffffce83`3784cdb8 00000000`00000000
ffffce83`3784cdc0 ffffaa81`e634c9c0
ffffce83`3784cdc8 fffff801`7e608bb5 nt!KiSystemServiceCopyEnd+0×25
ffffce83`3784cdd0 00000000`00000000
ffffce83`3784cdd8 ffff1496`0a767479
ffffce83`3784cde0 00000000`0002034c
ffffce83`3784cde8 000002aa`2d0d0180
ffffce83`3784cdf0 000000ce`1b2feac0
ffffce83`3784cdf8 00000023`83360010
ffffce83`3784ce00 00000000`00000000
ffffce83`3784ce08 00000000`00000000
ffffce83`3784ce10 00000000`00000000
ffffce83`3784ce18 00000000`00000000
ffffce83`3784ce20 ffff9a8e`065f7080
ffffce83`3784ce28 00000000`00000000
ffffce83`3784ce30 ffff9a8e`065f7080
ffffce83`3784ce38 fffff801`7e608bb5 nt!KiSystemServiceCopyEnd+0×25
ffffce83`3784ce40 00000000`00000001
ffffce83`3784ce48 ffffce83`38b5db80
ffffce83`3784ce50 000002aa`00000000
ffffce83`3784ce58 ffff868e`e8876c88 win32k!NtUserKillTimer
ffffce83`3784ce60 000000ce`00000000
ffffce83`3784ce68 00001f80`02080000
ffffce83`3784ce70 00000000`00000007
ffffce83`3784ce78 00000000`000001e4
ffffce83`3784ce80 00000000`00000000
ffffce83`3784ce88 000000ce`1b2ff5b8
ffffce83`3784ce90 000000ce`1b2ff689
ffffce83`3784ce98 00000000`00000000
ffffce83`3784cea0 00000000`00000246
ffffce83`3784cea8 000000ce`1b0a7000
ffffce83`3784ceb0 00000000`00000000
ffffce83`3784ceb8 00000000`00000000
ffffce83`3784cec0 00000000`00000000
ffffce83`3784cec8 00000000`00000000
ffffce83`3784ced0 00000000`00000000
ffffce83`3784ced8 00000000`00000000
ffffce83`3784cee0 00000000`00000000
ffffce83`3784cee8 00000000`00000000
ffffce83`3784cef0 00000000`00000000
ffffce83`3784cef8 00000000`00000000
ffffce83`3784cf00 00000000`00000000
ffffce83`3784cf08 00000000`00000000
ffffce83`3784cf10 00007ffb`8a73a5c2
ffffce83`3784cf18 00000000`00000000
ffffce83`3784cf20 00000000`00000000
ffffce83`3784cf28 00000000`00000000
ffffce83`3784cf30 00000000`00000000
ffffce83`3784cf38 00000000`00000000
ffffce83`3784cf40 00000000`00000000
ffffce83`3784cf48 00000000`00000000
ffffce83`3784cf50 00000000`00000000
ffffce83`3784cf58 00000000`00000000
ffffce83`3784cf60 00000000`00000000
ffffce83`3784cf68 00000000`00000000
ffffce83`3784cf70 00000000`00000000
ffffce83`3784cf78 00000000`00000000
ffffce83`3784cf80 00000000`00000000
ffffce83`3784cf88 00000000`000001e4
ffffce83`3784cf90 00000000`00000000
ffffce83`3784cf98 00000000`000001e4
ffffce83`3784cfa0 00000000`00000100
ffffce83`3784cfa8 00007ffb`8dc6ce54
ffffce83`3784cfb0 00000000`00000033
ffffce83`3784cfb8 00000000`00000246
ffffce83`3784cfc0 000000ce`1b2fea68
ffffce83`3784cfc8 00000000`0000002b
ffffce83`3784cfd0 ffffce83`3784d000
ffffce83`3784cfd8 ffffce83`37847000
ffffce83`3784cfe0 ffffce83`38b5e000
ffffce83`3784cfe8 ffffce83`38b58000
ffffce83`3784cff0 ffffce83`38b5d420
ffffce83`3784cff8 ffffce83`38b5dc90
ffffce83`3784d000 ????????`????????

In the case of Self-Diagnosis bugchecks Effect Components‘ execution residue (such as crashdmp and dump_diskdump) overwrite previous pre-bugcheck execution residue that makes reconstruction of Past Stack Trace impossible.

However, before Effect Components are executed, content of the stack region is saved in a special area:

0: kd> ? ffffce833784d000 - ffffce8337847000
Evaluate expression: 24576 = 00000000`00006000

0: kd> dps KiPreBugcheckStackSaveArea KiPreBugcheckStackSaveArea+6000
[…]
fffff801`7ee2f9f0 00000000`00000034
fffff801`7ee2f9f8 00000000`00000015
fffff801`7ee2fa00 ffff868e`e836edb0 win32kfull!vSrcTranCopyS8D32
fffff801`7ee2fa08 00000000`00000005
fffff801`7ee2fa10 00000000`0000000d
fffff801`7ee2fa18 00000000`00000014
fffff801`7ee2fa20 ffffce83`3784c020
fffff801`7ee2fa28 ffff868e`e8379289 win32kfull!vExpandAndCopyText+0×499
fffff801`7ee2fa30 ffff86a6`03358c65
fffff801`7ee2fa38 00000000`00000005
fffff801`7ee2fa40 ffff86a6`00000024
fffff801`7ee2fa48 ffff86a6`03361644
fffff801`7ee2fa50 00000000`ffcce4f7
fffff801`7ee2fa58 00000000`0000001f
fffff801`7ee2fa60 00000000`00000138
fffff801`7ee2fa68 00000000`00000000
fffff801`7ee2fa70 00000000`00000000
fffff801`7ee2fa78 00000000`ffcce4f7
fffff801`7ee2fa80 ffff86a6`03b27840
fffff801`7ee2fa88 00000000`0000002f
fffff801`7ee2fa90 00000000`00000024
fffff801`7ee2fa98 ffffce83`3784c68c
fffff801`7ee2faa0 00000000`0000002a
fffff801`7ee2faa8 fffff801`00000014
fffff801`7ee2fab0 00000000`0000002a
fffff801`7ee2fab8 ffff86a6`03358a90
fffff801`7ee2fac0 ffff868e`e836edb0 win32kfull!vSrcTranCopyS8D32
fffff801`7ee2fac8 ffffce83`3784c020
fffff801`7ee2fad0 ffff86a6`00000000
fffff801`7ee2fad8 ffff86a6`03b27840
fffff801`7ee2fae0 00000001`00000020
fffff801`7ee2fae8 00000000`00000138
fffff801`7ee2faf0 00000000`00000000
fffff801`7ee2faf8 ffff86a6`03356000
fffff801`7ee2fb00 ffff86a6`00000138
fffff801`7ee2fb08 ffff868e`e83c35a0 win32kfull!draw_clrt_nf_ntb_o_to_temp_start
fffff801`7ee2fb10 ffffce83`3784c010
fffff801`7ee2fb18 ffff86a6`03b27840
fffff801`7ee2fb20 ffff86a6`00911000
fffff801`7ee2fb28 ffff86a6`0312a4e8
fffff801`7ee2fb30 ffff86a6`0312a600
fffff801`7ee2fb38 ffff86a6`03b27840
fffff801`7ee2fb40 ffff868e`e83c35a0 win32kfull!draw_clrt_nf_ntb_o_to_temp_start
fffff801`7ee2fb48 ffff868e`e8559d80 win32kfull!draw_clrt_f_ntb_o_to_temp_start
fffff801`7ee2fb50 ffff86a6`03360000
fffff801`7ee2fb58 fffff801`7e407bae nt!ExAcquirePushLockExclusiveEx+0xee
fffff801`7ee2fb60 ffff9a8e`065f7080
fffff801`7ee2fb68 ffff86a6`00200280
fffff801`7ee2fb70 00000000`00000000
fffff801`7ee2fb78 00000000`00000000
fffff801`7ee2fb80 00000000`00000000
fffff801`7ee2fb88 ffff86a6`00200290
fffff801`7ee2fb90 00000000`00000022
fffff801`7ee2fb98 00000000`00000210
fffff801`7ee2fba0 00000000`00000000
fffff801`7ee2fba8 ffffce83`3784b85c
fffff801`7ee2fbb0 ffff86a6`00911000
fffff801`7ee2fbb8 00000000`00000000
fffff801`7ee2fbc0 00000000`00000000
fffff801`7ee2fbc8 fffff801`7e4dc26a nt!RtlpHpReleaseQueuedLockExclusive+0×20a
fffff801`7ee2fbd0 ffffce83`3784b9e0
fffff801`7ee2fbd8 ffff86a6`00200280
fffff801`7ee2fbe0 00000000`00040246
fffff801`7ee2fbe8 fffff801`7e49af8b nt!KeQueryCurrentStackInformationEx+0×8b
fffff801`7ee2fbf0 00000000`00000000
fffff801`7ee2fbf8 ffffce83`3784b9e0
fffff801`7ee2fc00 00000000`00000210
fffff801`7ee2fc08 ffff86a6`03358a60
fffff801`7ee2fc10 ffffce83`3784d000
fffff801`7ee2fc18 ffffce83`37847000
fffff801`7ee2fc20 00000000`00000000
fffff801`7ee2fc28 ffffce83`3784bf00
fffff801`7ee2fc30 00000000`00000000
fffff801`7ee2fc38 00000000`00000000
fffff801`7ee2fc40 ffffce83`3784b9f8
fffff801`7ee2fc48 fffff801`7e4e6aae nt!KeQueryCurrentStackInformation+0×2e
fffff801`7ee2fc50 ffffce83`3784ba10
fffff801`7ee2fc58 ffffce83`3784ba18
fffff801`7ee2fc60 ffffce83`3784ba60
fffff801`7ee2fc68 00000000`00000000
fffff801`7ee2fc70 00000000`00000008
fffff801`7ee2fc78 fffff801`7e7119e1 nt!KeBugCheck2+0×231
fffff801`7ee2fc80 00000000`00000000
fffff801`7ee2fc88 00000000`00000000
fffff801`7ee2fc90 00000000`00000000
fffff801`7ee2fc98 ffffce83`3784c8a8
fffff801`7ee2fca0 ffffce83`3784b9d0
fffff801`7ee2fca8 fffff801`7e65a4dc nt!RtlDispatchException+0×17399c
fffff801`7ee2fcb0 ffffce83`3784bed0
fffff801`7ee2fcb8 00000000`00000000
fffff801`7ee2fcc0 ffffce83`3784c0e0
fffff801`7ee2fcc8 00000000`00000000
fffff801`7ee2fcd0 00000101`01000000
fffff801`7ee2fcd8 ffff9a8e`065f7080
fffff801`7ee2fce0 00000000`0000001e
fffff801`7ee2fce8 00000000`00000000
fffff801`7ee2fcf0 00000000`0000000f
fffff801`7ee2fcf8 fffff801`7caf2100
fffff801`7ee2fd00 00000000`00000000
fffff801`7ee2fd08 00000000`00000000
fffff801`7ee2fd10 00000000`00000000
fffff801`7ee2fd18 ffff86a6`00000004
fffff801`7ee2fd20 00000000`00000000
fffff801`7ee2fd28 ffff86a6`03350010
fffff801`7ee2fd30 ffffce83`3784d000
fffff801`7ee2fd38 ffffce83`37847000
fffff801`7ee2fd40 fffff801`7e712bd0 nt!KiBugCheckProgress
fffff801`7ee2fd48 fffff801`7e489594 nt!ExFreeHeapPool+0×4d4
fffff801`7ee2fd50 00000000`00000000
fffff801`7ee2fd58 00000000`00000000
fffff801`7ee2fd60 00000000`00000000
fffff801`7ee2fd68 00000000`00000000
fffff801`7ee2fd70 00000000`00000000
fffff801`7ee2fd78 00000000`00000000
fffff801`7ee2fd80 00000000`00000000
fffff801`7ee2fd88 00000000`00000000
fffff801`7ee2fd90 00000000`00000000
fffff801`7ee2fd98 00000000`00000000
fffff801`7ee2fda0 00000000`00000000
fffff801`7ee2fda8 00000000`00000000
fffff801`7ee2fdb0 00000000`00000000
fffff801`7ee2fdb8 00000000`00000000
fffff801`7ee2fdc0 00000000`00000000
fffff801`7ee2fdc8 00000000`00000000
fffff801`7ee2fdd0 00000000`00000000
fffff801`7ee2fdd8 00000000`00000000
fffff801`7ee2fde0 00000000`00000000
fffff801`7ee2fde8 00000000`00000000
fffff801`7ee2fdf0 00000000`00000000
fffff801`7ee2fdf8 fffff801`7e40ac67 nt!ExReleasePushLockSharedEx+0×37
fffff801`7ee2fe00 ffff9a8e`00000002
fffff801`7ee2fe08 ffff86a6`00001f80
fffff801`7ee2fe10 ffff86a6`006136a0
fffff801`7ee2fe18 ffff86a6`0329ccd0
fffff801`7ee2fe20 00000000`000000bd
fffff801`7ee2fe28 ffff86a6`0329ccd0
fffff801`7ee2fe30 ffff86a6`03ac53f0
fffff801`7ee2fe38 ffff868e`e807e1d9 win32kbase!NSInstrumentation::CPlatformReaderWriterLock::ReleaseShared+0×19
fffff801`7ee2fe40 ffff86a6`006163d0
fffff801`7ee2fe48 ffff868e`00000003
fffff801`7ee2fe50 00000000`00000000
fffff801`7ee2fe58 ffff9a8e`00831120
fffff801`7ee2fe60 00000000`00000000
fffff801`7ee2fe68 ffff868e`e8123442 win32kbase!NSInstrumentation::CTypeIsolation<28672,112>::Free+0×8e
fffff801`7ee2fe70 00000000`00000000
fffff801`7ee2fe78 ffff86a6`006136a0
fffff801`7ee2fe80 ffff86a6`000000df
fffff801`7ee2fe88 00000000`00000000
fffff801`7ee2fe90 00000000`00000000
fffff801`7ee2fe98 ffff86a6`03358a70
fffff801`7ee2fea0 00000000`00000000
fffff801`7ee2fea8 ffff86a6`03358a90
fffff801`7ee2feb0 ffffce83`3784bcc0
fffff801`7ee2feb8 ffff868e`e837897f win32kfull!EngTextOut+0×68f
fffff801`7ee2fec0 ffff86a6`03358a90
fffff801`7ee2fec8 ffff86a6`03b27840
fffff801`7ee2fed0 ffffce83`3784bcc0
fffff801`7ee2fed8 00000000`00000005
fffff801`7ee2fee0 ffff86a6`03358a90
fffff801`7ee2fee8 ffff86a6`00000024
fffff801`7ee2fef0 00000000`00000000
fffff801`7ee2fef8 00000000`00000000
fffff801`7ee2ff00 00000000`00000000
fffff801`7ee2ff08 00000000`00000000
fffff801`7ee2ff10 00000000`00000000
fffff801`7ee2ff18 00000000`00000000
fffff801`7ee2ff20 00000000`00000000
fffff801`7ee2ff28 00000000`00000000
fffff801`7ee2ff30 00000000`00000000
fffff801`7ee2ff38 00000000`00000000
fffff801`7ee2ff40 00000000`00000000
fffff801`7ee2ff48 00000000`00000000
fffff801`7ee2ff50 00000000`00000000
fffff801`7ee2ff58 00000000`00000000
fffff801`7ee2ff60 00000000`00000000
fffff801`7ee2ff68 00000000`00000000
fffff801`7ee2ff70 00000000`00000000
fffff801`7ee2ff78 00000000`00000000
fffff801`7ee2ff80 00000000`00000000
fffff801`7ee2ff88 00000000`00000000
fffff801`7ee2ff90 ffffce83`3784c5d0
fffff801`7ee2ff98 00000000`00000000
fffff801`7ee2ffa0 ffff86a6`00000000
fffff801`7ee2ffa8 ffff86a6`03b27840
fffff801`7ee2ffb0 ffff86a6`03116220
fffff801`7ee2ffb8 00000000`000001d4
fffff801`7ee2ffc0 00000000`00000000
fffff801`7ee2ffc8 ffff86a6`03b27840
fffff801`7ee2ffd0 ffff86a6`03b27858
fffff801`7ee2ffd8 ffffce83`3784c68c
fffff801`7ee2ffe0 ffff86a6`0312a4e8
fffff801`7ee2ffe8 ffff86a6`0312a600
fffff801`7ee2fff0 ffff86a6`03b27840
fffff801`7ee2fff8 00000000`00000000
fffff801`7ee30000 00000000`00000000
fffff801`7ee30008 ffff86a6`03358a90
fffff801`7ee30010 00000000`00000000
fffff801`7ee30018 00000000`000000d0
fffff801`7ee30020 ffff86a6`03116220
fffff801`7ee30028 00000000`00000000
fffff801`7ee30030 00000000`00000000
fffff801`7ee30038 00000000`00000000
fffff801`7ee30040 00000000`00000000
fffff801`7ee30048 00000000`00000000
fffff801`7ee30050 00000000`00000000
fffff801`7ee30058 00000000`00000000
fffff801`7ee30060 00000000`00000000
fffff801`7ee30068 00000000`00000000
fffff801`7ee30070 00000000`00000000
fffff801`7ee30078 00000000`00000000
fffff801`7ee30080 00000000`00000000
fffff801`7ee30088 00000000`00000000
fffff801`7ee30090 00000000`00000000
fffff801`7ee30098 00000000`00000000
fffff801`7ee300a0 00000000`00000000
fffff801`7ee300a8 00000000`00000000
fffff801`7ee300b0 00000000`00000000
fffff801`7ee300b8 00000000`00000000
fffff801`7ee300c0 00000000`00000000
fffff801`7ee300c8 00000000`00000000
fffff801`7ee300d0 00000000`00000000
fffff801`7ee300d8 00000000`00000000
fffff801`7ee300e0 00000000`00000000
fffff801`7ee300e8 00000000`00000000
fffff801`7ee300f0 00000000`00000000
fffff801`7ee300f8 00000000`00000000
fffff801`7ee30100 00000000`00000000
fffff801`7ee30108 00000000`00000000
fffff801`7ee30110 00000000`00040293
fffff801`7ee30118 fffff801`7e49af8b nt!KeQueryCurrentStackInformationEx+0×8b
fffff801`7ee30120 00000000`00000000
fffff801`7ee30128 00000000`00000000
fffff801`7ee30130 00000000`00000000
fffff801`7ee30138 00000000`00000000
fffff801`7ee30140 ffffce83`3784d000
fffff801`7ee30148 ffffce83`37847000
fffff801`7ee30150 00000000`00000000
fffff801`7ee30158 00000000`00000000
fffff801`7ee30160 00000000`00000000
fffff801`7ee30168 fffff801`7e4e9cc6 nt!RtlGetExtendedContextLength2+0×46
fffff801`7ee30170 00000000`00000000
fffff801`7ee30178 fffff801`7e4e6a64 nt!RtlpGetStackLimitsEx+0×14
fffff801`7ee30180 ffffce83`3784c0e0
fffff801`7ee30188 ffffce83`3784c8a8
fffff801`7ee30190 00000001`00000010
fffff801`7ee30198 ffffce83`3784c0e0
fffff801`7ee301a0 ffffce83`3784c0a0
fffff801`7ee301a8 fffff801`7e4e6c59 nt!RtlDispatchException+0×119
fffff801`7ee301b0 ffffce83`3784c0e0
fffff801`7ee301b8 00000000`00000000
fffff801`7ee301c0 000004e8`fffffb30
fffff801`7ee301c8 000004d0`fffffb30
fffff801`7ee301d0 00000000`00000019
fffff801`7ee301d8 ffff86a6`03360000
fffff801`7ee301e0 ffffce83`3784c5d0
fffff801`7ee301e8 ffff86a6`0312a688
fffff801`7ee301f0 00000000`00000000
fffff801`7ee301f8 000004f7`00000000
fffff801`7ee30200 00000000`00000000
fffff801`7ee30208 ffffce83`3784d000
fffff801`7ee30210 ffffce83`37847000
fffff801`7ee30218 ffffce83`3784bea0
fffff801`7ee30220 00000000`00000000
fffff801`7ee30228 00000000`00000000
fffff801`7ee30230 00000000`00000000
fffff801`7ee30238 ffffce83`3784c8a8
fffff801`7ee30240 00000000`00000000
fffff801`7ee30248 00000000`00000000
fffff801`7ee30250 00000000`00000000
fffff801`7ee30258 00000000`00000000
fffff801`7ee30260 00000000`00000000
fffff801`7ee30268 00000000`00000000
fffff801`7ee30270 00000000`00000000
fffff801`7ee30278 00000000`00000000
fffff801`7ee30280 00000000`00000000
fffff801`7ee30288 00000000`00000000
fffff801`7ee30290 ffffce83`3784c0e0
fffff801`7ee30298 ffff86a6`00615054
fffff801`7ee302a0 00000000`00000000
fffff801`7ee302a8 ffffffff`ffffffff
fffff801`7ee302b0 00000000`00000000
fffff801`7ee302b8 00000000`00000000
fffff801`7ee302c0 00000000`00000000
fffff801`7ee302c8 00000000`00000000
fffff801`7ee302d0 00000000`00000000
fffff801`7ee302d8 00000000`00000000
fffff801`7ee302e0 00000000`00000000
fffff801`7ee302e8 00000000`00000000
fffff801`7ee302f0 00000000`00000000
fffff801`7ee302f8 00000000`00000000
fffff801`7ee30300 00000000`00000000
fffff801`7ee30308 00000000`00000000
fffff801`7ee30310 00000000`00000000
fffff801`7ee30318 00000000`00000000
fffff801`7ee30320 00000000`00000000
fffff801`7ee30328 00000000`00000000
fffff801`7ee30330 00000000`00000000
fffff801`7ee30338 00000000`00000000
fffff801`7ee30340 00000000`00000000
fffff801`7ee30348 00000000`00000000
fffff801`7ee30350 00000000`0010001f
fffff801`7ee30358 ffffce83`3784c950
fffff801`7ee30360 00000000`00000000
fffff801`7ee30368 00000000`00000000
fffff801`7ee30370 00000000`00000000
fffff801`7ee30378 ffffce83`3784c0e0
fffff801`7ee30380 ffffce83`3784c5e0
fffff801`7ee30388 fffff801`7e5f72c7 nt!KeBugCheckEx+0×107
fffff801`7ee30390 ffffce83`3784c8a8
fffff801`7ee30398 ffffce83`3784c5e0
fffff801`7ee303a0 ffffce83`3784c8a8
fffff801`7ee303a8 00000000`00000000
fffff801`7ee303b0 00000000`00000000
fffff801`7ee303b8 00000000`00000000
fffff801`7ee303c0 00000000`00040246
fffff801`7ee303c8 fffff801`7e659ecb nt!KiDispatchException+0×17467b
fffff801`7ee303d0 00000000`0000001e
fffff801`7ee303d8 ffffffff`c0000005
fffff801`7ee303e0 00000000`00000000
fffff801`7ee303e8 00000000`00000008
fffff801`7ee303f0 00000000`00000000
fffff801`7ee303f8 00000000`00000001
fffff801`7ee30400 ffff86a6`0312a600
fffff801`7ee30408 ffff86a6`0312a688
fffff801`7ee30410 ffff86a6`0312a4e8
fffff801`7ee30418 00000000`00000d0d
fffff801`7ee30420 00000001`00000000
fffff801`7ee30428 00000000`00000000
fffff801`7ee30430 00001f80`0010001f
fffff801`7ee30438 0053002b`002b0010
fffff801`7ee30440 00050282`0018002b
fffff801`7ee30448 00000000`00000000
fffff801`7ee30450 00000000`00000000
fffff801`7ee30458 00000000`00000000
fffff801`7ee30460 00000000`00000000
fffff801`7ee30468 00000000`00000000
fffff801`7ee30470 00000000`00000000
[…]
fffff801`7ee310c0 00000000`00000000
fffff801`7ee310c8 ffffce83`00000000
fffff801`7ee310d0 ffff86a6`00000001
fffff801`7ee310d8 00000000`00000000
fffff801`7ee310e0 ffffaa81`e634c9c0
fffff801`7ee310e8 fffff801`7e608bb5 nt!KiSystemServiceCopyEnd+0×25
fffff801`7ee310f0 00000000`00000000
fffff801`7ee310f8 ffff1496`0a767479
fffff801`7ee31100 00000000`0002034c
fffff801`7ee31108 000002aa`2d0d0180
fffff801`7ee31110 000000ce`1b2feac0
fffff801`7ee31118 00000023`83360010
fffff801`7ee31120 00000000`00000000
fffff801`7ee31128 00000000`00000000
fffff801`7ee31130 00000000`00000000
fffff801`7ee31138 00000000`00000000
fffff801`7ee31140 ffff9a8e`065f7080
fffff801`7ee31148 00000000`00000000
fffff801`7ee31150 ffff9a8e`065f7080
fffff801`7ee31158 fffff801`7e608bb5 nt!KiSystemServiceCopyEnd+0×25
fffff801`7ee31160 00000000`00000001
fffff801`7ee31168 ffffce83`38b5db80
fffff801`7ee31170 000002aa`00000000
fffff801`7ee31178 ffff868e`e8876c88 win32k!NtUserKillTimer
fffff801`7ee31180 000000ce`00000000
fffff801`7ee31188 00001f80`02080000
fffff801`7ee31190 00000000`00000007
fffff801`7ee31198 00000000`000001e4
fffff801`7ee311a0 00000000`00000000
fffff801`7ee311a8 000000ce`1b2ff5b8
fffff801`7ee311b0 000000ce`1b2ff689
fffff801`7ee311b8 00000000`00000000
fffff801`7ee311c0 00000000`00000246
fffff801`7ee311c8 000000ce`1b0a7000
fffff801`7ee311d0 00000000`00000000
fffff801`7ee311d8 00000000`00000000
fffff801`7ee311e0 00000000`00000000
fffff801`7ee311e8 00000000`00000000
fffff801`7ee311f0 00000000`00000000
fffff801`7ee311f8 00000000`00000000
fffff801`7ee31200 00000000`00000000
fffff801`7ee31208 00000000`00000000
fffff801`7ee31210 00000000`00000000
fffff801`7ee31218 00000000`00000000
fffff801`7ee31220 00000000`00000000
fffff801`7ee31228 00000000`00000000
fffff801`7ee31230 00007ffb`8a73a5c2
fffff801`7ee31238 00000000`00000000
fffff801`7ee31240 00000000`00000000
fffff801`7ee31248 00000000`00000000
fffff801`7ee31250 00000000`00000000
fffff801`7ee31258 00000000`00000000
fffff801`7ee31260 00000000`00000000
fffff801`7ee31268 00000000`00000000
fffff801`7ee31270 00000000`00000000
fffff801`7ee31278 00000000`00000000
fffff801`7ee31280 00000000`00000000
fffff801`7ee31288 00000000`00000000
fffff801`7ee31290 00000000`00000000
fffff801`7ee31298 00000000`00000000
fffff801`7ee312a0 00000000`00000000
fffff801`7ee312a8 00000000`000001e4
fffff801`7ee312b0 00000000`00000000
fffff801`7ee312b8 00000000`000001e4
fffff801`7ee312c0 00000000`00000100
fffff801`7ee312c8 00007ffb`8dc6ce54
fffff801`7ee312d0 00000000`00000033
fffff801`7ee312d8 00000000`00000246
fffff801`7ee312e0 000000ce`1b2fea68
fffff801`7ee312e8 00000000`0000002b
fffff801`7ee312f0 ffffce83`3784d000
fffff801`7ee312f8 ffffce83`37847000
fffff801`7ee31300 ffffce83`38b5e000
fffff801`7ee31308 ffffce83`38b58000
fffff801`7ee31310 ffffce83`38b5d420
fffff801`7ee31318 ffffce83`38b5dc90
fffff801`7ee31320 ffff9a8e`002f9448

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Crash Dump Analysis Patterns (Part 277)

Monday, September 27th, 2021

When looking at the kernel and complete memory dumps, the current thread running on the current processor (!thread) may not belong to the current process (not listed in the output of !process WinDbg command). This observation happens when a thread that is owned by one process gets attached to the second process:

0: kd> !thread
THREAD ffffa902d2ff8080 Cid 1f00.02c0 Teb: 000000836c677000 Win32Thread: 0000000000000000 RUNNING on processor 0
IRP List:
ffffa902d0afabb0: (0006,0118) Flags: 00060000 Mdl: 00000000
Not impersonating
DeviceMap ffffba81b0037600
Owning Process ffffa902d1581080 Image: OriginalProcess.exe
Attached Process ffffa902cf41a080 Image: NewProcess.exe
Wait Start TickCount 136814 Ticks: 3 (0:00:00:00.046)
Context Switch Count 4 IdealProcessor: 0
UserTime 00:00:00.000
KernelTime 00:00:00.046
Win32 Start Address 0×00007ff62aabf010
Stack Init ffffe10adff87c90 Current ffffe10adff876a0
Base ffffe10adff88000 Limit ffffe10adff82000 Call 0000000000000000
Priority 14 BasePriority 8 PriorityDecrement 80 IoPriority 2 PagePriority 5
[…]

In this way, a thread can access another process space. We call such analysis pattern Shared Thread. Another example is process creation resulting in Hidden Process. Such Shared Threads can also be found in Stack Trace Collection.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -