Software Diagnostics Library

Finally Citrix has published a tool (written by my colleague Colm Naish, lead escalation engineer) that allows controlled injection of events into CDF (ETW) trace message stream. This is useful in many troubleshooting scenarios where we need to rely on Significant Event and Anchor Message analysis patterns to partition traces into artificial Activity Regions to start our analysis with. This is also analogous for the imposition of the external time on the stream of tracing events from software narratology perspective:

CDFMarker On Demand - For XenApp and XenDesktop

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Announcements, CDF Analysis Tips and Tricks, Citrix, Debugging, Escalation Engineering, Software Engineering, Software Narratology, Software Technical Support, Software Trace Analysis, Software Trace Reading, Tools, Trace Analysis Patterns | 1 Comment »

Reading Notebook: 24-May-10

May 24th, 2010

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

Process integrity levels as SIDs (pp. 464 - 465)

Protected mode IE startup sequence (pp. 467 - 470) - ieuser.exe might block several iexplore.exe instances: http://www.dumpanalysis.org/blog/index.php/2009/02/11/stack-trace-collection-blocked-thread-and-coupled-processes-pattern-cooperation/

Integrity levels and mandatory policies for objects (pp. 471- 473)

Many faces of an Administrator, filtered admin tokens (p. 474)

CreateProcessWithLogonW (p. 474)

The token source field (p. 476)

Token authentication and modified IDs (pp. 476 - 477) - token structure from x64 Windows Server R2:

0: kd> dt _TOKEN nt!_TOKEN +0x000 TokenSource : _TOKEN_SOURCE +0x010 TokenId : _LUID +0x018 AuthenticationId : _LUID +0x020 ParentTokenId : _LUID +0x028 ExpirationTime : _LARGE_INTEGER +0x030 TokenLock : Ptr64 _ERESOURCE +0x038 ModifiedId : _LUID +0x040 Privileges : _SEP_TOKEN_PRIVILEGES +0x058 AuditPolicy : _SEP_AUDIT_POLICY +0x074 SessionId : Uint4B +0x078 UserAndGroupCount : Uint4B +0x07c RestrictedSidCount : Uint4B +0x080 VariableLength : Uint4B +0x084 DynamicCharged : Uint4B +0x088 DynamicAvailable : Uint4B +0x08c DefaultOwnerIndex : Uint4B +0x090 UserAndGroups : Ptr64 _SID_AND_ATTRIBUTES +0x098 RestrictedSids : Ptr64 _SID_AND_ATTRIBUTES +0x0a0 PrimaryGroup : Ptr64 Void +0x0a8 DynamicPart : Ptr64 Uint4B +0x0b0 DefaultDacl : Ptr64 _ACL +0x0b8 TokenType : _TOKEN_TYPE +0x0bc ImpersonationLevel : _SECURITY_IMPERSONATION_LEVEL +0x0c0 TokenFlags : Uint4B +0x0c4 TokenInUse : UChar +0x0c8 IntegrityLevelIndex : Uint4B +0x0cc MandatoryPolicy : Uint4B +0x0d0 LogonSession : Ptr64 _SEP_LOGON_SESSION_REFERENCES +0x0d8 OriginatingLogonSession : _LUID +0x0e0 SidHash : _SID_AND_ATTRIBUTES_HASH +0x1f0 RestrictedSidHash : _SID_AND_ATTRIBUTES_HASH +0x300 pSecurityAttributes : Ptr64 _AUTHZBASEP_SECURITY_ATTRIBUTES_INFORMATION +0x308 VariablePart : Uint8B

Posted in Notes on Windows Internals, Reading Notebook | No Comments »

Models for Memory and Trace Analysis Patterns (Part 2)

May 23rd, 2010

We continue our modeling of software behaviour with the ubiquitous Memory Leak (process heap) pattern. Instead of leaking small heap allocations that are easy to debug with user mode stack trace database our model program leaks large heap allocations:

// MemoryLeak-ProcessHeap // Copyright (c) 2010 Dmitry Vostokov // GNU GENERAL PUBLIC LICENSE // http://www.gnu.org/licenses/gpl-3.0.txt

#include <windows.h>

int _tmain(int argc, _TCHAR* argv[]) { // create extra 25 heaps initially for (int i = 0; i < 25; ++i) HeapCreate(0, 0, 0);

// create a heap to leak within HANDLE hHeap = HeapCreate(0, 0, 0);

while (true) { HeapAlloc(hHeap, 0, 1024*1024); Sleep(1000); }

return 0; }

The program creates extra process heaps to simulate real life heap leaks that usually happen not in a default process heap. Then it slowly leaks 0×100000 bytes every second. The application can be downloaded from here (zip file contains source code, x86 and x64 binaries together with corresponding PDB files):

Download MemoryLeak-ProcessHeap.zip

Here I present the results from x64 Windows Server 2008 R2 but x86 variants (I tested on x86 Vista) should be the same.

First we run the application and save a dump of it after a few seconds (I used Task Manager). Heap statistics shows 9 virtual blocks for the last 0000000001e00000 heap:

0:000> !heap -s LFH Key : 0x000000d529c37801 Termination on corruption : ENABLED Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast (k) (k) (k) (k) length blocks cont. heap ————————————————————————————- 00000000002b0000 00000002 1024 164 1024 3 1 1 0 0 LFH 0000000000010000 00008000 64 4 64 1 1 1 0 0 0000000000020000 00008000 64 64 64 61 1 1 0 0 0000000000220000 00001002 1088 152 1088 3 2 2 0 0 LFH 0000000000630000 00001002 512 8 512 3 1 1 0 0 0000000000870000 00001002 512 8 512 3 1 1 0 0 0000000000ad0000 00001002 512 8 512 3 1 1 0 0 00000000007e0000 00001002 512 8 512 3 1 1 0 0 0000000000cc0000 00001002 512 8 512 3 1 1 0 0 0000000000ed0000 00001002 512 8 512 3 1 1 0 0 00000000010c0000 00001002 512 8 512 3 1 1 0 0 00000000005b0000 00001002 512 8 512 3 1 1 0 0 00000000009f0000 00001002 512 8 512 3 1 1 0 0 00000000004d0000 00001002 512 8 512 3 1 1 0 0 0000000000230000 00001002 512 8 512 3 1 1 0 0 0000000000700000 00001002 512 8 512 3 1 1 0 0 00000000012d0000 00001002 512 8 512 3 1 1 0 0 0000000000950000 00001002 512 8 512 3 1 1 0 0 0000000000b90000 00001002 512 8 512 3 1 1 0 0 00000000014c0000 00001002 512 8 512 3 1 1 0 0 0000000000e50000 00001002 512 8 512 3 1 1 0 0 0000000001020000 00001002 512 8 512 3 1 1 0 0 00000000016e0000 00001002 512 8 512 3 1 1 0 0 0000000001940000 00001002 512 8 512 3 1 1 0 0 0000000001b90000 00001002 512 8 512 3 1 1 0 0 0000000001200000 00001002 512 8 512 3 1 1 0 0 0000000000c20000 00001002 512 8 512 3 1 1 0 0 0000000000db0000 00001002 512 8 512 3 1 1 0 0 0000000000f50000 00001002 512 8 512 3 1 1 0 0 Virtual block: 0000000001350000 - 0000000001350000 (size 0000000000000000) Virtual block: 0000000001540000 - 0000000001540000 (size 0000000000000000) Virtual block: 0000000001760000 - 0000000001760000 (size 0000000000000000) Virtual block: 00000000019c0000 - 00000000019c0000 (size 0000000000000000) Virtual block: 0000000001c10000 - 0000000001c10000 (size 0000000000000000) Virtual block: 0000000001e80000 - 0000000001e80000 (size 0000000000000000) Virtual block: 0000000001f90000 - 0000000001f90000 (size 0000000000000000) Virtual block: 00000000020a0000 - 00000000020a0000 (size 0000000000000000) Virtual block: 00000000021b0000 - 00000000021b0000 (size 0000000000000000) 0000000001e00000 00001002 512 8 512 3 1 1 9 0 ————————————————————————————-

We then wait for a few minutes and save a memory dump again. Heap statistics clearly shows virtual block leaks because now we 276 of them instead of previous 9 (I skipped most of them in the output below):

0:000> !heap -s LFH Key : 0x000000d529c37801 Termination on corruption : ENABLED Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast (k) (k) (k) (k) length blocks cont. heap ————————————————————————————- 00000000002b0000 00000002 1024 164 1024 3 1 1 0 0 LFH 0000000000010000 00008000 64 4 64 1 1 1 0 0 0000000000020000 00008000 64 64 64 61 1 1 0 0 0000000000220000 00001002 1088 152 1088 3 2 2 0 0 LFH 0000000000630000 00001002 512 8 512 3 1 1 0 0 0000000000870000 00001002 512 8 512 3 1 1 0 0 0000000000ad0000 00001002 512 8 512 3 1 1 0 0 00000000007e0000 00001002 512 8 512 3 1 1 0 0 0000000000cc0000 00001002 512 8 512 3 1 1 0 0 0000000000ed0000 00001002 512 8 512 3 1 1 0 0 00000000010c0000 00001002 512 8 512 3 1 1 0 0 00000000005b0000 00001002 512 8 512 3 1 1 0 0 00000000009f0000 00001002 512 8 512 3 1 1 0 0 00000000004d0000 00001002 512 8 512 3 1 1 0 0 0000000000230000 00001002 512 8 512 3 1 1 0 0 0000000000700000 00001002 512 8 512 3 1 1 0 0 00000000012d0000 00001002 512 8 512 3 1 1 0 0 0000000000950000 00001002 512 8 512 3 1 1 0 0 0000000000b90000 00001002 512 8 512 3 1 1 0 0 00000000014c0000 00001002 512 8 512 3 1 1 0 0 0000000000e50000 00001002 512 8 512 3 1 1 0 0 0000000001020000 00001002 512 8 512 3 1 1 0 0 00000000016e0000 00001002 512 8 512 3 1 1 0 0 0000000001940000 00001002 512 8 512 3 1 1 0 0 0000000001b90000 00001002 512 8 512 3 1 1 0 0 0000000001200000 00001002 512 8 512 3 1 1 0 0 0000000000c20000 00001002 512 8 512 3 1 1 0 0 0000000000db0000 00001002 512 8 512 3 1 1 0 0 0000000000f50000 00001002 512 8 512 3 1 1 0 0 Virtual block: 0000000001350000 - 0000000001350000 (size 0000000000000000) Virtual block: 0000000001540000 - 0000000001540000 (size 0000000000000000) Virtual block: 0000000001760000 - 0000000001760000 (size 0000000000000000) Virtual block: 00000000019c0000 - 00000000019c0000 (size 0000000000000000) [… skipped …] Virtual block: 00000000131b0000 - 00000000131b0000 (size 0000000000000000) Virtual block: 00000000132c0000 - 00000000132c0000 (size 0000000000000000) Virtual block: 00000000133d0000 - 00000000133d0000 (size 0000000000000000) Virtual block: 00000000134e0000 - 00000000134e0000 (size 0000000000000000) Virtual block: 00000000135f0000 - 00000000135f0000 (size 0000000000000000) Virtual block: 0000000013700000 - 0000000013700000 (size 0000000000000000) Virtual block: 0000000013810000 - 0000000013810000 (size 0000000000000000) Virtual block: 0000000013920000 - 0000000013920000 (size 0000000000000000) Virtual block: 0000000013a30000 - 0000000013a30000 (size 0000000000000000) Virtual block: 0000000013b40000 - 0000000013b40000 (size 0000000000000000) Virtual block: 0000000013c50000 - 0000000013c50000 (size 0000000000000000) Virtual block: 0000000013d60000 - 0000000013d60000 (size 0000000000000000) 0000000001e00000 00001002 512 8 512 3 1 1 276 0 ————————————————————————————-

We see that size of these blocks is 0×101000 bytes (with hindsight, extra 1000 is probably bookkeeping info):

0:000> !address 0000000013d60000 ProcessParametrs 00000000002b1f20 in range 00000000002b0000 00000000002d9000 Environment 00000000002b1320 in range 00000000002b0000 00000000002d9000 0000000013d60000 : 0000000013d60000 - 0000000000101000 Type 00020000 MEM_PRIVATE Protect 00000004 PAGE_READWRITE State 00001000 MEM_COMMIT Usage RegionUsageHeap Handle 0000000001e00000

We want to know which thread allocates them and we search for the heap address 0000000001e00000 through virtual memory to find any execution residue on thread raw stacks:

0:000> s -q 0 LFFFFFF 0000000001e00000 00000000`001cf608 00000000`01e00000 00000000`01e00000 00000000`001cf610 00000000`01e00000 00000000`00000858 00000000`001cf630 00000000`01e00000 00000000`0000000a 00000000`001cf6c8 00000000`01e00000 00000000`00000000 00000000`001cf6e0 00000000`01e00000 00000000`01e00a80 00000000`001cf720 00000000`01e00000 00000000`00000020 00000000`001cf778 00000000`01e00000 00000000`01e00000 00000000`001cf780 00000000`01e00000 00000000`00000000 00000000`001cf798 00000000`01e00000 00000000`01e02000 00000000`001cf7b0 00000000`01e00000 02100301`00000000 00000000`001cf7c8 00000000`01e00000 00000000`01c10000 00000000`001cf808 00000000`01e00000 00000000`00000001 00000000`001cf830 00000000`01e00000 00000000`00000002 00000000`001cf940 00000000`01e00000 00000000`00000000 00000000`002d8378 00000000`01e00000 004c0044`005c0064 00000000`01e00028 00000000`01e00000 00000000`01e00000 00000000`01e00030 00000000`01e00000 00000000`00000080

Address range 00000000`001cfxxxx belongs to the main thread:

0:000> kL Child-SP RetAddr Call Site 00000000`001cf898 000007fe`fdd91203 ntdll!NtDelayExecution+0xa 00000000`001cf8a0 00000001`3f39104f KERNELBASE!SleepEx+0xab 00000000`001cf940 00000001`3f3911ea MemoryLeak_ProcessHeap!wmain+0×4f 00000000`001cf970 00000000`778cf56d MemoryLeak_ProcessHeap!__tmainCRTStartup+0×15a 00000000`001cf9b0 00000000`77b03281 kernel32!BaseThreadInitThunk+0xd 00000000`001cf9e0 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

We dump the raw stack fragment now:

0:000> dps 00000000`001cf608 00000000`001cf940 00000000`001cf608 00000000`01e00000 00000000`001cf610 00000000`01e00000 00000000`001cf618 00000000`00000858 00000000`001cf620 00000000`00000000 00000000`001cf628 00000000`77b229ac ntdll!RtlAllocateHeap+0×16c 00000000`001cf630 00000000`01e00000 00000000`001cf638 00000000`0000000a 00000000`001cf640 00000000`00000858 00000000`001cf648 00000000`00000860 00000000`001cf650 00000000`00000000 00000000`001cf658 00000000`001cf740 00000000`001cf660 00000020`00001000 00000000`001cf668 fffff680`01000000 00000000`001cf670 00000001`3f390000 MemoryLeak_ProcessHeap!wmain <PERF> (MemoryLeak_ProcessHeap+0×0) 00000000`001cf678 01000000`00000080 00000000`001cf680 00000000`0000f000 00000000`001cf688 02100210`02100210 00000000`001cf690 00000001`3f390000 MemoryLeak_ProcessHeap!wmain <PERF> (MemoryLeak_ProcessHeap+0×0) 00000000`001cf698 00000000`0000f000 00000000`001cf6a0 00000000`01e01fd0 00000000`001cf6a8 00000000`77b07ff3 ntdll!RtlpCreateUCREntry+0xb3 00000000`001cf6b0 00000000`001cf6b8 00000000`001cf6b8 00000000`01e01fc0 00000000`001cf6c0 00000000`00000080 00000000`001cf6c8 00000000`01e00000 00000000`001cf6d0 00000000`00000000 00000000`001cf6d8 00000000`00000a00 00000000`001cf6e0 00000000`01e00000 00000000`001cf6e8 00000000`01e00a80 00000000`001cf6f0 00000000`0007e000 00000000`001cf6f8 00000000`77b0f2bb ntdll!RtlpInitializeHeapSegment+0×19b 00000000`001cf700 00000000`01e00208 00000000`001cf708 00000000`00000000 00000000`001cf710 00000000`01e00230 00000000`001cf718 00000000`00000000 00000000`001cf720 00000000`01e00000 00000000`001cf728 00000000`00000020 00000000`001cf730 00000000`00000000 00000000`001cf738 00000000`77b0f676 ntdll!RtlpInitializeUCRIndex+0×36 00000000`001cf740 000007ff`00000003 00000000`001cf748 00000000`00000100 00000000`001cf750 00000000`00000000 00000000`001cf758 00000000`00001002 00000000`001cf760 00000000`00001002 00000000`001cf768 00000000`77b0fec9 ntdll!RtlCreateHeap+0×8f7 00000000`001cf770 00000000`01e02000 00000000`001cf778 00000000`01e00000 00000000`001cf780 00000000`01e00000 00000000`001cf788 00000000`00000000 00000000`001cf790 03010301`00000000 00000000`001cf798 00000000`01e00000 00000000`001cf7a0 00000000`01e02000 00000000`001cf7a8 00000000`01e80000 00000000`001cf7b0 00000000`01e00000 00000000`001cf7b8 02100301`00000000 00000000`001cf7c0 00000000`001f0000 00000000`001cf7c8 00000000`01e00000 00000000`001cf7d0 00000000`01c10000 00000000`001cf7d8 00000000`01e02000 00000000`001cf7e0 00000000`00270000 00000000`001cf7e8 03020302`00000230 00000000`001cf7f0 00000000`77be7288 ntdll!RtlpInterceptorRoutines 00000000`001cf7f8 00000000`00000000 00000000`001cf800 00000000`00100010 00000000`001cf808 00000000`01e00000 00000000`001cf810 00000000`00000001 00000000`001cf818 00000000`00100000 00000000`001cf820 00000000`00000000 00000000`001cf828 00000000`77b229ac ntdll!RtlAllocateHeap+0×16c 00000000`001cf830 00000000`01e00000 00000000`001cf838 00000000`00000002 00000000`001cf840 00000000`00100000 00000000`001cf848 00000000`00101000 00000000`001cf850 00000000`00000000 00000000`001cf858 00000000`001cf940 00000000`001cf860 00000000`00000000 00000000`001cf868 0000f577`2bd1e0ff 00000000`001cf870 00000000`ffffffff 00000000`001cf878 00000000`10010011 00000000`001cf880 00000000`c00000bb 00000000`001cf888 00000000`00000000 00000000`001cf890 00000000`00000100 00000000`001cf898 000007fe`fdd91203 KERNELBASE!SleepEx+0xab 00000000`001cf8a0 00000000`001cf958 00000000`001cf8a8 00000000`00000000 00000000`001cf8b0 00000000`00000000 00000000`001cf8b8 00000000`00000012 00000000`001cf8c0 ffffffff`ff676980 00000000`001cf8c8 00000000`001cf8c0 00000000`001cf8d0 00000000`00000048 00000000`001cf8d8 00000000`00000001 00000000`001cf8e0 00000000`00000000 00000000`001cf8e8 00000000`00000000 00000000`001cf8f0 00000000`00000000 00000000`001cf8f8 00000000`00000000 00000000`001cf900 00000000`00000000 00000000`001cf908 00000000`00000000 00000000`001cf910 00000000`00000000 00000000`001cf918 00000000`00000000 00000000`001cf920 00000000`00000000 00000000`001cf928 00000000`00000001 00000000`001cf930 00000000`00000000 00000000`001cf938 00000001`3f39104f MemoryLeak_ProcessHeap!wmain+0×4f 00000000`001cf940 00000000`01e00000

We see traces of RtlAllocateHeap but it could be a coincidence (imagine that we don’t have access to source code and can’t do live debugging to put breakpoints). We advise to enable use mode stack trace database as explained for another example process: CTX106970. Then we launch our application again and save a new user dump. We repeat the same procedure to examine the raw stack:

0:000> !heap -s NtGlobalFlag enables following debugging aids for new heaps: stack back traces LFH Key : 0x000000c21e1b31e6 Termination on corruption : ENABLED Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast (k) (k) (k) (k) length blocks cont. heap ------------------------------------------------------------------------------------- 0000000001bc0000 08000002 1024 168 1024 5 1 1 0 0 LFH 0000000000010000 08008000 64 4 64 1 1 1 0 0 0000000000020000 08008000 64 64 64 61 1 1 0 0 0000000000100000 08001002 1088 152 1088 2 2 2 0 0 LFH 0000000001d90000 08001002 512 8 512 3 1 1 0 0 0000000001f90000 08001002 512 8 512 3 1 1 0 0 00000000021c0000 08001002 512 8 512 3 1 1 0 0 0000000002130000 08001002 512 8 512 3 1 1 0 0 0000000002370000 08001002 512 8 512 3 1 1 0 0 0000000001e80000 08001002 512 8 512 3 1 1 0 0 0000000000110000 08001002 512 8 512 3 1 1 0 0 0000000002510000 08001002 512 8 512 3 1 1 0 0 0000000002760000 08001002 512 8 512 3 1 1 0 0 0000000001cc0000 08001002 512 8 512 3 1 1 0 0 0000000002030000 08001002 512 8 512 3 1 1 0 0 0000000002960000 08001002 512 8 512 3 1 1 0 0 0000000002670000 08001002 512 8 512 3 1 1 0 0 0000000002b90000 08001002 512 8 512 3 1 1 0 0 00000000022f0000 08001002 512 8 512 3 1 1 0 0 00000000028b0000 08001002 512 8 512 3 1 1 0 0 0000000001f10000 08001002 512 8 512 3 1 1 0 0 0000000002450000 08001002 512 8 512 3 1 1 0 0 00000000025f0000 08001002 512 8 512 3 1 1 0 0 0000000002a40000 08001002 512 8 512 3 1 1 0 0 0000000002c90000 08001002 512 8 512 3 1 1 0 0 0000000002d90000 08001002 512 8 512 3 1 1 0 0 0000000002e80000 08001002 512 8 512 3 1 1 0 0 0000000002fc0000 08001002 512 8 512 3 1 1 0 0 00000000030b0000 08001002 512 8 512 3 1 1 0 0 Virtual block: 0000000003130000 - 0000000003130000 (size 0000000000000000) Virtual block: 0000000003240000 - 0000000003240000 (size 0000000000000000) Virtual block: 0000000003350000 - 0000000003350000 (size 0000000000000000) Virtual block: 0000000003460000 - 0000000003460000 (size 0000000000000000) Virtual block: 0000000003570000 - 0000000003570000 (size 0000000000000000) Virtual block: 0000000003680000 - 0000000003680000 (size 0000000000000000) Virtual block: 0000000003790000 - 0000000003790000 (size 0000000000000000) Virtual block: 00000000038a0000 - 00000000038a0000 (size 0000000000000000) Virtual block: 00000000039b0000 - 00000000039b0000 (size 0000000000000000) Virtual block: 0000000003ac0000 - 0000000003ac0000 (size 0000000000000000) Virtual block: 0000000003bd0000 - 0000000003bd0000 (size 0000000000000000) Virtual block: 0000000003ce0000 - 0000000003ce0000 (size 0000000000000000) 0000000002270000 08001002 512 8 512 3 1 1 12 0

0:000> s -q 0 LFFFFFF 0000000002270000 00000000`0029f648 00000000`02270000 00000000`000000a8 00000000`0029f660 00000000`02270000 00000000`77b99bc7 00000000`0029f6e8 00000000`02270000 00000000`02270000 00000000`0029f6f0 00000000`02270000 00000000`00000858 00000000`0029f710 00000000`02270000 00000000`0000000a 00000000`0029f758 00000000`02270000 00000000`0029f918 00000000`0029f7c0 00000000`02270000 00000000`03ce0040 00000000`0029f858 00000000`02270000 00000000`02270000 00000000`0029f860 00000000`02270000 00000000`00000000 00000000`0029f878 00000000`02270000 00000000`02272000 00000000`0029f890 00000000`02270000 02100301`00000000 00000000`0029f8f8 00000000`02270000 00000000`03ce0040 00000000`0029fa20 00000000`02270000 00000000`00000000 00000000`01be8a08 00000000`02270000 0064006e`00690057 00000000`02270028 00000000`02270000 00000000`02270000 00000000`02270030 00000000`02270000 00000000`00000080

0:000> dqs 00000000`0029f648 00000000`0029fa20 00000000`0029f648 00000000`02270000 00000000`0029f650 00000000`000000a8 00000000`0029f658 00000000`00000a80 00000000`0029f660 00000000`02270000 00000000`0029f668 00000000`77b99bc7 ntdll!RtlStdLogStackTrace+0x47 00000000`0029f670 00000000`002a0000 00000000`0029f678 00000000`0000007e 00000000`0029f680 00000000`02270a80 00000000`0029f688 00000000`00000000 00000000`0029f690 00000000`02270208 00000000`0029f698 00070000`77b99bc7 00000000`0029f6a0 00000000`77b6cd8a ntdll! ?? ::FNODOBFM::`string'+0xf35a 00000000`0029f6a8 00000000`77b0fd07 ntdll!RtlCreateHeap+0x56e 00000000`0029f6b0 000007fe`fdd9c6a4 KERNELBASE!HeapCreate+0x54 00000000`0029f6b8 00000001`3faa1030 MemoryLeak_ProcessHeap!wmain+0x30 00000000`0029f6c0 00000001`3faa11ea MemoryLeak_ProcessHeap!__tmainCRTStartup+0x15a 00000000`0029f6c8 00000000`778cf56d kernel32!BaseThreadInitThunk+0xd 00000000`0029f6d0 00000000`77be7288 ntdll!RtlpInterceptorRoutines 00000000`0029f6d8 00000000`00000000 00000000`0029f6e0 00000000`00000860 00000000`0029f6e8 00000000`02270000 00000000`0029f6f0 00000000`02270000 00000000`0029f6f8 00000000`00000858 00000000`0029f700 00000000`00000000 00000000`0029f708 00000000`77b229ac ntdll!RtlAllocateHeap+0x16c 00000000`0029f710 00000000`02270000 00000000`0029f718 00000000`0000000a 00000000`0029f720 00000000`00000858 00000000`0029f728 00000000`00000860 00000000`0029f730 00000000`00000000 00000000`0029f738 00000000`0029f820 00000000`0029f740 00000000`77be7288 ntdll!RtlpInterceptorRoutines 00000000`0029f748 00000000`00000002 00000000`0029f750 00000000`00100030 00000000`0029f758 00000000`02270000 00000000`0029f760 00000000`0029f918 00000000`0029f768 00000000`00000020 00000000`0029f770 00000000`00000002 00000000`0029f778 00000000`00000005 00000000`0029f780 00000000`000750f0 00000000`0029f788 00000000`77ba25b2 ntdll!RtlpRegisterStackTrace+0x92 00000000`0029f790 00000000`000750b8 00000000`0029f798 00000000`00000003 00000000`0029f7a0 00000000`00000000 00000000`0029f7a8 00000000`77ad7a0a ntdll!RtlCaptureStackBackTrace+0x4a 00000000`0029f7b0 00000000`00000002 00000000`0029f7b8 00000000`00100030 00000000`0029f7c0 00000000`02270000 00000000`0029f7c8 00000000`03ce0040 00000000`0029f7d0 00000000`00100020 00000000`0029f7d8 00000000`77ba2eb7 ntdll!RtlpStackTraceDatabaseLogPrefix+0x57 00000000`0029f7e0 00000000`03ce0040 00000000`0029f7e8 00000000`00000000 00000000`0029f7f0 00000000`00100020 00000000`0029f7f8 00000000`000750f0 00000000`0029f800 00000000`77b6ed2d ntdll! ?? ::FNODOBFM::`string’+0×1a81b 00000000`0029f808 00000001`3faa1044 MemoryLeak_ProcessHeap!wmain+0×44 00000000`0029f810 00000001`3faa11ea MemoryLeak_ProcessHeap!__tmainCRTStartup+0×15a 00000000`0029f818 00000000`778cf56d kernel32!BaseThreadInitThunk+0xd 00000000`0029f820 00000000`77b03281 ntdll!RtlUserThreadStart+0×1d 00000000`0029f828 00000000`00000100 00000000`0029f830 00000000`00000000 00000000`0029f838 00000000`08001002 00000000`0029f840 00000000`08001002 00000000`0029f848 00000000`77b0fec9 ntdll!RtlCreateHeap+0×8f7 00000000`0029f850 00000000`02272000 00000000`0029f858 00000000`02270000 00000000`0029f860 00000000`02270000 00000000`0029f868 00000000`00000000 00000000`0029f870 03010301`00000000 00000000`0029f878 00000000`02270000 00000000`0029f880 00000000`02272000 00000000`0029f888 00000000`022f0000 00000000`0029f890 00000000`02270000 00000000`0029f898 02100301`00000000 00000000`0029f8a0 00000000`00001000 00000000`0029f8a8 00000000`77b9a886 ntdll!RtlpSetupExtendedBlock+0xc6 00000000`0029f8b0 00000000`00000000 00000000`0029f8b8 00000000`02272000 00000000`0029f8c0 00000000`000b0000 00000000`0029f8c8 03020302`00000230 00000000`0029f8d0 00000000`77be7288 ntdll!RtlpInterceptorRoutines 00000000`0029f8d8 00000000`00000002 00000000`0029f8e0 00000000`77be7288 ntdll!RtlpInterceptorRoutines 00000000`0029f8e8 00000000`00000002 00000000`0029f8f0 00000000`00100030 00000000`0029f8f8 00000000`02270000 00000000`0029f900 00000000`03ce0040 00000000`0029f908 00000000`77b6ed6a ntdll! ?? ::FNODOBFM::`string’+0×1a858 00000000`0029f910 00000000`00000000 00000000`0029f918 00000000`00000000 00000000`0029f920 00000000`00100000 00000000`0029f928 00000000`00101000 00000000`0029f930 00000000`00000020 00000000`0029f938 00000000`00000002 00000000`0029f940 00000000`00000000 00000000`0029f948 0000f569`df709780 00000000`0029f950 00000000`ffffffff 00000000`0029f958 00000000`12010013 00000000`0029f960 00000000`c00000bb 00000000`0029f968 00000000`00000000 00000000`0029f970 00000000`00000100 00000000`0029f978 000007fe`fdd91203 KERNELBASE!SleepEx+0xab 00000000`0029f980 00000000`0029fa38 00000000`0029f988 00000000`00000000 00000000`0029f990 00000000`00000000 00000000`0029f998 00000000`00000012 00000000`0029f9a0 ffffffff`ff676980 00000000`0029f9a8 00000000`0029f9a0 00000000`0029f9b0 00000000`00000048 00000000`0029f9b8 00000000`00000001 00000000`0029f9c0 00000000`00000000 00000000`0029f9c8 00000000`00000000 00000000`0029f9d0 00000000`00000000 00000000`0029f9d8 00000000`00000000 00000000`0029f9e0 00000000`00000000 00000000`0029f9e8 00000000`00000000 00000000`0029f9f0 00000000`00000000 00000000`0029f9f8 00000000`00000000 00000000`0029fa00 00000000`00000000 00000000`0029fa08 00000000`00000001 00000000`0029fa10 00000000`00000000 00000000`0029fa18 00000001`3faa104f MemoryLeak_ProcessHeap!wmain+0×4f 00000000`0029fa20 00000000`02270000

Now we see this stack trace fragment from user mode stack trace database on the raw stack shown above:

00000000`0029f800 00000000`77b6ed2d ntdll! ?? ::FNODOBFM::`string'+0x1a81b 00000000`0029f808 00000001`3faa1044MemoryLeak_ProcessHeap!wmain+0×44 00000000`0029f810 00000001`3faa11ea MemoryLeak_ProcessHeap!__tmainCRTStartup+0×15a 00000000`0029f818 00000000`778cf56d kernel32!BaseThreadInitThunk+0xd 00000000`0029f820 00000000`77b03281 ntdll!RtlUserThreadStart+0×1d

And it looks like HeapAlloc was called from wmain indeed with 0×100000 parameter:

0:000> ub 00000001`3faa1044 MemoryLeak_ProcessHeap!wmain+0x26: 00000001`3faa1026 xor edx,edx 00000001`3faa1028 xor ecx,ecx 00000001`3faa102a call qword ptr [MemoryLeak_ProcessHeap!_imp_HeapCreate (00000001`3faa7000)] 00000001`3faa1030 mov rbx,rax 00000001`3faa1033 xor edx,edx 00000001`3faa1035 mov r8d,100000h 00000001`3faa103b mov rcx,rbx 00000001`3faa103e call qword ptr [MemoryLeak_ProcessHeap!_imp_HeapAlloc (00000001`3faa7008)]

0:000> dps 00000001`3faa7008 L1 00000001`3faa7008 00000000`77b21b70 ntdll!RtlAllocateHeap

The stack trace fragment from x86 Vista user dump even more straightforward:

0040fa00 77946e0c ntdll!RtlAllocateHeap+0×1e3 0040fa04 0022103a MemoryLeak_ProcessHeap!wmain+0×3a 0040fa08 7677d0e9 kernel32!BaseThreadInitThunk+0xe 0040fa0c 779219bb ntdll!__RtlUserThreadStart+0×23 0040fa10 7792198e ntdll!_RtlUserThreadStart+0×1b 0040fa14 7798924f ntdll!RtlpLogCapturedStackTrace+0×103

Of course, we could simply disassemble wmain after identifying our thread but in real life functions are longer and leaking allocations could have happened from frames not present on current stack traces.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging, Pattern Models, Software Behavior Patterns, Software Defect Construction | No Comments »

Ontological and Epistemological Memoidealism

May 22nd, 2010

Just want to clarify the following branches of memoidealism (memory idealism) in addition to its common sense pragmatic panmemoric version:

Ontological memoidealism

The ultimate nature of reality is based on the memory

Epistemological memoidealism

The only things that can be directly known are memories

Another question often asked is why memory idealism and not memory realism. I have chosen the former because memory is often closely associated with the mind. In many cases you can just replace mind with memory, for example:

… science is in reality a classification and analysis of the contents of the memory;¹⁾

We choose the most important property of the mind and computers: memory and try to ground and explain reality and mind in terms of that ontologically elevated property.

1) Karl Pearson, The Grammar of Science

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Memoidealism, Philosophy | No Comments »

Bugtation No.120

May 22nd, 2010

… science is in reality a classification and analysis of the contents of the memory;

Karl Pearson, The Grammar of Science

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Bugtations, Memoidealism, Philosophy, Science of Memory Dump Analysis, Science of Software Tracing | No Comments »

Psychoanalysis of Software Troubleshooting and Debugging (Part 1)

May 21st, 2010

In this part I would like to introduce the notion of Forgotten Facts in opposition to Basic Facts or supporting information. These are facts that engineers often feel uncomfortable to mention because they are troubleshooting information they couldn’t obtain (if they tried) due to some time or customer pressures, failures, incorrectly understood troubleshooting procedures or some other obstacles. Therefore it is important to have a set of counter questions or checklists mapped from common software behaviour patterns to software troubleshooting patterns. Problem descriptions should also be subjected to close reading to reveal unconsciously concealed information. Next part will explore this in more detail with some case studies commissioned by Software Maintenance Institute.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Crash Dump Analysis, Debugging, Escalation Engineering, Psychoanalysis of Software Maintenance and Support, Psychology, Software Behavior Patterns, Software Maintenance Institute, Software Technical Support, Software Trace Analysis, Software Trace Reading, Software Troubleshooting Patterns | 1 Comment »

Two Readings of a Software Trace

May 21st, 2010

When we have a software trace we read it in two directions. The first one is to deconstruct it into a linear ordered source code based on PLOT fragments. The second direction is to construct an interpretation that serve as an explanation for reported software behaviour. During the interpretive reading we remove irrelevant information, compress relevant activity regions and construct the new fictional software trace based on discovered patterns and our problem description.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Code Reading, Debugging, Science of Software Tracing, Software Narratology, Software Technical Support, Software Trace Analysis, Software Trace Reading, Trace Analysis Patterns | No Comments »

Icons for Memory Dump Analysis Patterns (Part 42)

May 19th, 2010

Today we introduce an icon for Double Free (kernel pool) pattern:

B/W

Color

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging, Pattern Icons, Visual Dump Analysis | No Comments »

Icons for Memory Dump Analysis Patterns (Part 41)

May 18th, 2010

Today we introduce an icon for Double Free (process heap) pattern:

B/W

Color

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging, Pattern Icons, Visual Dump Analysis | No Comments »

Icons for Memory Dump Analysis Patterns (Part 40)

May 17th, 2010

Today we introduce an icon for Unknown Component pattern:

B/W

Color

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging, Pattern Icons, Visual Dump Analysis | No Comments »

Dr. DebugLove and Nature

May 16th, 2010

Pictures taken today during weekend nature walks (debugging walkthroughs) in Dublin suburbs:

Before A Debugging Session

Looking For Bugs

Found A Bug

Looking For More Bugs

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Debugging, Debugging and Nature, Fun with Crash Dumps, History | 2 Comments »

MDAA Volumes are Asm Bestsellers

May 16th, 2010

Just noticed a rare event when all three volumes of Memory Dump Analysis Anthology occupy the first 3 positions on Bestsellers in Assembly Language Programming Amazon list (observed at the time of this writing):

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Announcements, Assembly Language, Books, Crash Dump Analysis, Debugging, Publishing | No Comments »

Models for Memory and Trace Analysis Patterns (Part 1)

May 16th, 2010

Due to many requests for memory dumps corresponding to crash dump analysis patterns I’ve started modeling software behaviour and defects. Every pattern will have an example application(s), service(s) or driver(s) or combination of them. Their execution results in memory layout that corresponds to memory or trace analysis patterns. Today we introduce an example model for Multiple Exceptions (user mode) pattern. The following source code models 3 threads each having an exception during their execution on Windows XP, Windows 7 and Windows Server 2008 R2:

// MultipleExceptions-UserMode // Copyright (c) 2010 Dmitry Vostokov // GNU GENERAL PUBLIC LICENSE // http://www.gnu.org/licenses/gpl-3.0.txt

#include <windows.h> #include <process.h>

void thread_one(void *) { *(int *)NULL = 0; }

void thread_two(void *) { *(int *)NULL = 0; }

int main(int argc, WCHAR* argv[]) { _beginthread(thread_two, 0, NULL); _beginthread(thread_one, 0, NULL);

DebugBreak();

return 0; }

In fact, thread_one and thread_two can be replaced with just one function because they are identical. Visual C++ compiler does that during code optimization. On Windows 7 and W2K8 R2 I created LocalDumps registry key to save full crash dumps. On Windows XP I set Dr. Watson as a postmortem debugger (via drwtsn32 -i command and configured it to save full user dumps via drwtsn32 command that brings Dr. Watson GUI). Vista had some peculiar behaviour so I postponed its discussion for another post. The application can be downloaded from here (zip file contains source code, x86 and x64 binaries together with corresponding PDB files):

Download MultipleExceptions-UserMode.zip

Now I provide modeling results for x64 W2K8 R2 running on 2 processor machine. Windows 7 and Windows XP results are very similar. If we run x64 executable it crashes and a dump file is saved (x86 crash dump is similar). Default analysis command gives these results:

0:000> !analyze -v

[...]

FAULTING_IP: MultipleExceptions_UserMode!thread_two+0 00000001`3f8b1000 c704250000000000000000 mov dword ptr [0],0

EXCEPTION_RECORD: ffffffffffffffff -- (.exr 0xffffffffffffffff) ExceptionAddress: 000007fefddc2442 (KERNELBASE!DebugBreak+0x0000000000000002) ExceptionCode: 80000003 (Break instruction exception) ExceptionFlags: 00000000 NumberParameters: 1 Parameter[0]: 0000000000000000

[...]

ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION} Breakpoint A breakpoint has been reached.

[...]

PRIMARY_PROBLEM_CLASS: STATUS_BREAKPOINT

[...]

STACK_TEXT: 00000001`3f8b1000 MultipleExceptions_UserMode!thread_two+0x0 00000001`3f8b10eb MultipleExceptions_UserMode!_callthreadstart+0x17 00000001`3f8b1195 MultipleExceptions_UserMode!_threadstart+0x95 00000000`778cf56d kernel32!BaseThreadInitThunk+0xd 00000000`77b03281 ntdll!RtlUserThreadStart+0x1d

[...]

We see debug break on the first thread:

0:000> kL Child-SP RetAddr Call Site 00000000`002eec78 000007fe`fdd913a6 ntdll!NtWaitForMultipleObjects+0xa 00000000`002eec80 00000000`778d3143 KERNELBASE!WaitForMultipleObjectsEx+0xe8 00000000`002eed80 00000000`77949025 kernel32!WaitForMultipleObjectsExImplementation+0xb3 00000000`002eee10 00000000`779491a7 kernel32!WerpReportFaultInternal+0x215 00000000`002eeeb0 00000000`779491ff kernel32!WerpReportFault+0x77 00000000`002eeee0 00000000`7794941c kernel32!BasepReportFault+0x1f 00000000`002eef10 00000000`77b6573c kernel32!UnhandledExceptionFilter+0x1fc 00000000`002eeff0 00000000`77ae5148 ntdll! ?? ::FNODOBFM::`string'+0x2365 00000000`002ef020 00000000`77b0554d ntdll!_C_specific_handler+0x8c 00000000`002ef090 00000000`77ae5d1c ntdll!RtlpExecuteHandlerForException+0xd 00000000`002ef0c0 00000000`77b1fe48 ntdll!RtlDispatchException+0x3cb 00000000`002ef7a0 000007fe`fddc2442 ntdll!KiUserExceptionDispatcher+0x2e 00000000`002efd58 00000001`3f8b103c KERNELBASE!DebugBreak+0×2 00000000`002efd60 00000001`3f8b13fb MultipleExceptions_UserMode!main+0×2c 00000000`002efd90 00000000`778cf56d MultipleExceptions_UserMode!__tmainCRTStartup+0×15b 00000000`002efdd0 00000000`77b03281 kernel32!BaseThreadInitThunk+0xd 00000000`002efe00 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

2 other threads show exception processing too:

0:000> ~1s; kL ntdll!NtDelayExecution+0xa: 00000000`77b201fa c3 ret Child-SP RetAddr Call Site 00000000`0076ef78 000007fe`fdd91203 ntdll!NtDelayExecution+0xa 00000000`0076ef80 00000000`77949175 KERNELBASE!SleepEx+0xab 00000000`0076f020 00000000`779491ff kernel32!WerpReportFault+0×45 00000000`0076f050 00000000`7794941c kernel32!BasepReportFault+0×1f 00000000`0076f080 00000000`77b6573c kernel32!UnhandledExceptionFilter+0×1fc 00000000`0076f160 00000000`77ae5148 ntdll! ?? ::FNODOBFM::`string’+0×2365 00000000`0076f190 00000000`77b0554d ntdll!_C_specific_handler+0×8c 00000000`0076f200 00000000`77ae5d1c ntdll!RtlpExecuteHandlerForException+0xd 00000000`0076f230 00000000`77b1fe48 ntdll!RtlDispatchException+0×3cb 00000000`0076f910 00000001`3f8b1000 ntdll!KiUserExceptionDispatcher+0×2e 00000000`0076fec8 00000001`3f8b10eb MultipleExceptions_UserMode!thread_two 00000000`0076fed0 00000001`3f8b1195 MultipleExceptions_UserMode!_callthreadstart+0×17 00000000`0076ff00 00000000`778cf56d MultipleExceptions_UserMode!_threadstart+0×95 00000000`0076ff30 00000000`77b03281 kernel32!BaseThreadInitThunk+0xd 00000000`0076ff60 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

0:001> ~2s; kL ntdll!NtDelayExecution+0xa: 00000000`77b201fa c3 ret Child-SP RetAddr Call Site 00000000`0086e968 000007fe`fdd91203 ntdll!NtDelayExecution+0xa 00000000`0086e970 00000000`77949175 KERNELBASE!SleepEx+0xab 00000000`0086ea10 00000000`779491ff kernel32!WerpReportFault+0×45 00000000`0086ea40 00000000`7794941c kernel32!BasepReportFault+0×1f 00000000`0086ea70 00000000`77b6573c kernel32!UnhandledExceptionFilter+0×1fc 00000000`0086eb50 00000000`77ae5148 ntdll! ?? ::FNODOBFM::`string’+0×2365 00000000`0086eb80 00000000`77b0554d ntdll!_C_specific_handler+0×8c 00000000`0086ebf0 00000000`77ae5d1c ntdll!RtlpExecuteHandlerForException+0xd 00000000`0086ec20 00000000`77b1fe48 ntdll!RtlDispatchException+0×3cb 00000000`0086f300 00000001`3f8b1000 ntdll!KiUserExceptionDispatcher+0×2e 00000000`0086f8b8 00000001`3f8b10eb MultipleExceptions_UserMode!thread_two 00000000`0086f8c0 00000001`3f8b1195 MultipleExceptions_UserMode!_callthreadstart+0×17 00000000`0086f8f0 00000000`778cf56d MultipleExceptions_UserMode!_threadstart+0×95 00000000`0086f920 00000000`77b03281 kernel32!BaseThreadInitThunk+0xd 00000000`0086f950 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

We look at unhandled exception filter parameter to get exception pointers information:

0:002> kv Child-SP RetAddr : Args to Child : Call Site [...] 00000000`0086ea70 00000000`77b6573c : 00000000`0086ebb0 00000000`00000006 00000001`00000000 00000000`00000001 : kernel32!UnhandledExceptionFilter+0×1fc

0:002> .exptr 00000000`0086ebb0

----- Exception record at 00000000`0086f7f0: ExceptionAddress: 000000013f8b1000 (MultipleExceptions_UserMode!thread_two) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000000 NumberParameters: 2 Parameter[0]: 0000000000000001 Parameter[1]: 0000000000000000 Attempt to write to address 0000000000000000

----- Context record at 00000000`0086f300: rax=00000000000ef0b0 rbx=00000000000ef0b0 rcx=0000000000000000 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000 rip=000000013f8b1000 rsp=000000000086f8b8 rbp=0000000000000000 r8=000007fffffda000 r9=0000000000000000 r10=0000000000000045 r11=000007fffffd9328 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000 iopl=0 nv up ei pl nz na pe nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202 MultipleExceptions_UserMode!thread_two: 00000001`3f8b1000 c704250000000000000000 mov dword ptr [0],0 ds:00000000`00000000=????????

What we now see that default analysis command showed the break instruction exception record and error code from the first thread but IP and stack trace from other threads having NULL pointer access violation exception.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging, Pattern Models, Software Behavior Patterns, Software Defect Construction | No Comments »

Icons for Memory Dump Analysis Patterns (Part 39)

May 14th, 2010

Today we introduce an icon for Missing Thread pattern:

B/W

Color

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging, Pattern Icons, Visual Dump Analysis | No Comments »

Reading Notebook: 12-May-10

May 13th, 2010

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

SAS -> winlogon.exe starts LogonUI.exe (p. 455) - Here are winlogon.exe threads on x64 W2K8 R2 before SAS:

THREAD fffffa8003cf7060 Cid 01d0.01d4 Teb: 000007fffffdd000 Win32Thread: fffff900c00df900 WAIT: (UserRequest) UserMode Non-Alertable fffffa8004991c90 SynchronizationEvent Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8003cf65a0 Image: winlogon.exe Attached Process N/A Image: N/A Wait Start TickCount 8831 Ticks: 21731 (0:00:05:39.005) Context Switch Count 424 LargeStack UserTime 00:00:00.015 KernelTime 00:00:00.015 Win32 Start Address winlogon!WinMainCRTStartup (0x00000000ff36ec08) Stack Init fffff88003595db0 Current fffff88003595900 Base fffff88003596000 Limit fffff8800358c000 Call 0 Priority 15 BasePriority 15 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Kernel stack not resident. Child-SP RetAddr Call Site fffff880`03595940 fffff800`01ac3752 nt!KiSwapContext+0x7a fffff880`03595a80 fffff800`01ac58af nt!KiCommitThreadWait+0x1d2 fffff880`03595b10 fffff800`01db7db2 nt!KeWaitForSingleObject+0x19f fffff880`03595bb0 fffff800`01abb853 nt!NtWaitForSingleObject+0xb2 fffff880`03595c20 00000000`77bafefa nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`03595c20) 00000000`0018f778 000007fe`fdc910ac ntdll!NtWaitForSingleObject+0xa 00000000`0018f780 00000000`ff3619ad KERNELBASE!WaitForSingleObjectEx+0x79 00000000`0018f820 00000000`ff3616e8 winlogon!SignalManagerWaitForSignal+0x135 00000000`0018f860 00000000`ff36b8b0 winlogon!StateMachineRun+0x404 00000000`0018fb80 00000000`ff36ed85 winlogon!WinMain+0x13a3 00000000`0018fcf0 00000000`77a5f56d winlogon!I_WMsgkSendMessage+0x252 00000000`0018fdb0 00000000`77b93281 kernel32!BaseThreadInitThunk+0xd 00000000`0018fde0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa800498a060 Cid 01d0.0320 Teb: 000007fffffd7000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Alertable fffffa800497bef0 SynchronizationTimer fffffa8004988060 SynchronizationTimer fffffa8004bfe2a0 NotificationEvent fffffa8003c783b0 SynchronizationEvent fffffa8003c78310 SynchronizationEvent fffffa8003c78450 SynchronizationEvent fffffa80049894c0 SynchronizationTimer Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8003cf65a0 Image: winlogon.exe Attached Process N/A Image: N/A Wait Start TickCount 19271 Ticks: 11291 (0:00:02:56.140) Context Switch Count 16 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWaiterpThread (0x0000000077b79a90) Stack Init fffff88004006db0 Current fffff88004005fd0 Base fffff88004007000 Limit fffff88004001000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`04006010 fffff800`01ac3752 nt!KiSwapContext+0x7a fffff880`04006150 fffff800`01abfc4b nt!KiCommitThreadWait+0x1d2 fffff880`040061e0 fffff800`01db8ecf nt!KeWaitForMultipleObjects+0x271 fffff880`04006490 fffff800`01db97d6 nt!ObpWaitForMultipleObjects+0x294 fffff880`04006960 fffff800`01abb853 nt!NtWaitForMultipleObjects+0xe5 fffff880`04006bb0 00000000`77bb046a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`04006c20) 00000000`0139f848 00000000`77b79bd7 ntdll!NtWaitForMultipleObjects+0xa 00000000`0139f850 00000000`77a5f56d ntdll!TppWaiterpThread+0x14d 00000000`0139faf0 00000000`77b93281 kernel32!BaseThreadInitThunk+0xd 00000000`0139fb20 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa8004ed7060 Cid 01d0.0a58 Teb: 000007fffffdb000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable fffffa800489ac20 QueueObject Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8003cf65a0 Image: winlogon.exe Attached Process N/A Image: N/A Wait Start TickCount 27861 Ticks: 2701 (0:00:00:42.135) Context Switch Count 4 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWorkerThread (0x0000000077b78f00) Stack Init fffff88003555db0 Current fffff880035557d0 Base fffff88003556000 Limit fffff88003550000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`03555810 fffff800`01ac3752 nt!KiSwapContext+0x7a fffff880`03555950 fffff800`01ac71c1 nt!KiCommitThreadWait+0x1d2 fffff880`035559e0 fffff800`01db89d7 nt!KeRemoveQueueEx+0x301 fffff880`03555a90 fffff800`01acc996 nt!IoRemoveIoCompletion+0x47 fffff880`03555b20 fffff800`01abb853 nt!NtWaitForWorkViaWorkerFactory+0x285 fffff880`03555c20 00000000`77bb17ba nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`03555c20) 00000000`00dcfa18 00000000`77b7914b ntdll!NtWaitForWorkViaWorkerFactory+0xa 00000000`00dcfa20 00000000`77a5f56d ntdll!TppWorkerThread+0x2c9 00000000`00dcfd20 00000000`77b93281 kernel32!BaseThreadInitThunk+0xd 00000000`00dcfd50 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

Here are main threads from both processes on x64 W2K8 R2 after SAS (I brought change password dialog):

THREAD fffffa8004888770 Cid 01c0.01c4 Teb: 000007fffffde000 Win32Thread: fffff900c00d9c30 WAIT: (UserRequest) UserMode Non-Alertable fffffa80049c25c0 SynchronizationEvent Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa80048879d0 Image: winlogon.exe Attached Process N/A Image: N/A Wait Start TickCount 34664851 Ticks: 1875 (0:00:00:29.250) Context Switch Count 3202 LargeStack UserTime 00:00:00.000 KernelTime 00:00:00.218 Win32 Start Address winlogon!WinMainCRTStartup (0x00000000ffc2ec08) Stack Init fffff880031acdb0 Current fffff880031ac900 Base fffff880031ad000 Limit fffff880031a7000 Call 0 Priority 15 BasePriority 15 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`031ac940 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`031aca80 fffff800`01ad88af nt!KiCommitThreadWait+0x1d2 fffff880`031acb10 fffff800`01dcadb2 nt!KeWaitForSingleObject+0x19f fffff880`031acbb0 fffff800`01ace853 nt!NtWaitForSingleObject+0xb2 fffff880`031acc20 00000000`76e2fefa nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`031acc20) 00000000`0023f398 000007fe`fd0810ac ntdll!NtWaitForSingleObject+0xa 00000000`0023f3a0 00000000`ffc219ad KERNELBASE!WaitForSingleObjectEx+0x79 00000000`0023f440 00000000`ffc216e8 winlogon!SignalManagerWaitForSignal+0x135 00000000`0023f480 00000000`ffc2b8b0 winlogon!StateMachineRun+0x404 00000000`0023f7a0 00000000`ffc2ed85 winlogon!WinMain+0x13a3 00000000`0023f910 00000000`76bdf56d winlogon!I_WMsgkSendMessage+0x252 00000000`0023f9d0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`0023fa00 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa80049ba060 Cid 01c0.0304 Teb: 000007fffffd7000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Alertable fffffa80049b87e0 SynchronizationTimer fffffa80049b4650 SynchronizationTimer fffffa8004e81e20 NotificationEvent fffffa8004edcbf0 SynchronizationEvent fffffa8004edcb50 SynchronizationEvent fffffa8004edcc90 SynchronizationEvent fffffa80049b8670 SynchronizationTimer Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa80048879d0 Image: winlogon.exe Attached Process N/A Image: N/A Wait Start TickCount 34428081 Ticks: 238645 (0:01:02:02.885) Context Switch Count 175 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWaiterpThread (0x0000000076df9a90) Stack Init fffff88004193db0 Current fffff88004192fd0 Base fffff88004194000 Limit fffff8800418e000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Kernel stack not resident. Child-SP RetAddr Call Site fffff880`04193010 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`04193150 fffff800`01ad2c4b nt!KiCommitThreadWait+0x1d2 fffff880`041931e0 fffff800`01dcbecf nt!KeWaitForMultipleObjects+0x271 fffff880`04193490 fffff800`01dcc7d6 nt!ObpWaitForMultipleObjects+0x294 fffff880`04193960 fffff800`01ace853 nt!NtWaitForMultipleObjects+0xe5 fffff880`04193bb0 00000000`76e3046a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`04193c20) 00000000`00d2fb38 00000000`76df9bd7 ntdll!NtWaitForMultipleObjects+0xa 00000000`00d2fb40 00000000`76bdf56d ntdll!TppWaiterpThread+0x14d 00000000`00d2fde0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`00d2fe10 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa8005b8e810 Cid 01c0.12d4 Teb: 000007fffffdc000 Win32Thread: fffff900c37a6250 WAIT: (WrLpcReply) UserMode Non-Alertable fffffa8005b8ebd0 Semaphore Limit 0x1 Waiting for reply to ALPC Message fffff8a00c87e750 : queued at port fffffa800661ec60 : owned by process fffffa8005f442b0 Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa80048879d0 Image: winlogon.exe Attached Process N/A Image: N/A Wait Start TickCount 34664851 Ticks: 1875 (0:00:00:29.250) Context Switch Count 150 LargeStack UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWorkerThread (0×0000000076df8f00) Stack Init fffff88006c8edb0 Current fffff88006c8e620 Base fffff88006c8f000 Limit fffff88006c87000 Call 0 Priority 14 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`06c8e660 fffff800`01ad6752 nt!KiSwapContext+0×7a fffff880`06c8e7a0 fffff800`01ad88af nt!KiCommitThreadWait+0×1d2 fffff880`06c8e830 fffff800`01aedbef nt!KeWaitForSingleObject+0×19f fffff880`06c8e8d0 fffff800`01dd6a36 nt!AlpcpSignalAndWait+0×8f fffff880`06c8e980 fffff800`01dd49c0 nt!AlpcpReceiveSynchronousReply+0×46 fffff880`06c8e9e0 fffff800`01dd1f3b nt!AlpcpProcessSynchronousRequest+0×33d fffff880`06c8eb00 fffff800`01ace853 nt!NtAlpcSendWaitReceivePort+0×1ab fffff880`06c8ebb0 00000000`76e3070a nt!KiSystemServiceCopyEnd+0×13 (TrapFrame @ fffff880`06c8ec20) 00000000`0103f298 000007fe`fea8aa76 ntdll!ZwAlpcSendWaitReceivePort+0xa 00000000`0103f2a0 000007fe`feb2cb64 RPCRT4!LRPC_CCALL::SendReceive+0×156 00000000`0103f360 000007fe`feb2cd55 RPCRT4!NdrpClientCall3+0×244 00000000`0103f620 00000000`ffc24979 RPCRT4!NdrClientCall3+0xf2 00000000`0103f9b0 00000000`ffc4e781 winlogon!WluiRequestCredentials+0×71 00000000`0103fa20 00000000`ffc21d04 winlogon!WLGeneric_Request_Change_Credz_Execute+0xa5 00000000`0103fa90 00000000`76df0fb4 winlogon!StateMachineWorkerCallback+0×7f 00000000`0103fac0 00000000`76df4b1f ntdll!TppWorkpExecuteCallback+0xa4 00000000`0103fb20 00000000`76bdf56d ntdll!TppWorkerThread+0×6c9 00000000`0103fe20 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`0103fe50 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

THREAD fffffa8006480640 Cid 01c0.131c Teb: 000007fffffd9000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable fffffa80042479a0 QueueObject Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa80048879d0 Image: winlogon.exe Attached Process N/A Image: N/A Wait Start TickCount 34664380 Ticks: 2346 (0:00:00:36.597) Context Switch Count 2 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWorkerThread (0x0000000076df8f00) Stack Init fffff8800715ddb0 Current fffff8800715d7d0 Base fffff8800715e000 Limit fffff88007158000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`0715d810 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`0715d950 fffff800`01ada1c1 nt!KiCommitThreadWait+0x1d2 fffff880`0715d9e0 fffff800`01dcb9d7 nt!KeRemoveQueueEx+0x301 fffff880`0715da90 fffff800`01adf996 nt!IoRemoveIoCompletion+0x47 fffff880`0715db20 fffff800`01ace853 nt!NtWaitForWorkViaWorkerFactory+0x285 fffff880`0715dc20 00000000`76e317ba nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`0715dc20) 00000000`010bf908 00000000`76df914b ntdll!NtWaitForWorkViaWorkerFactory+0xa 00000000`010bf910 00000000`76bdf56d ntdll!TppWorkerThread+0x2c9 00000000`010bfc10 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`010bfc40 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa8005916290 Cid 01c0.0c04 Teb: 000007fffffd5000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable fffffa80042479a0 QueueObject Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa80048879d0 Image: winlogon.exe Attached Process N/A Image: N/A Wait Start TickCount 34664851 Ticks: 1875 (0:00:00:29.250) Context Switch Count 3 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWorkerThread (0x0000000076df8f00) Stack Init fffff88007126db0 Current fffff880071267d0 Base fffff88007127000 Limit fffff88007121000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`07126810 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`07126950 fffff800`01ada1c1 nt!KiCommitThreadWait+0x1d2 fffff880`071269e0 fffff800`01dcb9d7 nt!KeRemoveQueueEx+0x301 fffff880`07126a90 fffff800`01adf996 nt!IoRemoveIoCompletion+0x47 fffff880`07126b20 fffff800`01ace853 nt!NtWaitForWorkViaWorkerFactory+0x285 fffff880`07126c20 00000000`76e317ba nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`07126c20) 00000000`009cfaa8 00000000`76df914b ntdll!NtWaitForWorkViaWorkerFactory+0xa 00000000`009cfab0 00000000`76bdf56d ntdll!TppWorkerThread+0x2c9 00000000`009cfdb0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`009cfde0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

We now see the new thread fffffa8005b8e810 waiting for an ALPC message fffff8a00c87e750:

0: kd> !alpc /m fffff8a00c87e750

Message @ fffff8a00c87e750 MessageID : 0x0534 (1332) CallbackID : 0x14152C5 (21058245) SequenceNumber : 0x00000006 (6) Type : LPC_REQUEST DataLength : 0x0060 (96) TotalLength : 0x0088 (136) Canceled : No Release : No ReplyWaitReply : No Continuation : Yes OwnerPort : fffffa80065696c0 [ALPC_CLIENT_COMMUNICATION_PORT] WaitingThread : fffffa8005b8e810 QueueType : ALPC_MSGQUEUE_PENDING QueuePort : fffffa800661ec60 [ALPC_CONNECTION_PORT] QueuePortOwnerProcess : fffffa8005f442b0 (LogonUI.exe) ServerThread : fffffa8005a9b2a0 QuotaCharged : No CancelQueuePort : 0000000000000000 CancelSequencePort : 0000000000000000 CancelSequenceNumber : 0×00000000 (0) ClientContext : 00000000003f5b30 ServerContext : 0000000000000000 PortContext : 00000000015e2640 CancelPortContext : 0000000000000000 SecurityData : 0000000000000000 View : 0000000000000000

The server thread is fffffa8005a9b2a0 and is owned by LogonUI.exe. Here are all threads in that process where I highlighted credential providers:

THREAD fffffa8005f47b60 Cid 06d0.13e0 Teb: 000007fffffde000 Win32Thread: fffff900c1d6ec30 WAIT: (UserRequest) UserMode Non-Alertable fffffa80065be260 SynchronizationEvent fffffa8005bf6240 SynchronizationEvent fffffa8005bcbc70 SynchronizationEvent fffffa80052a9dc0 SynchronizationEvent Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34666693 Ticks: 33 (0:00:00:00.514) Context Switch Count 722 LargeStack UserTime 00:00:00.171 KernelTime 00:00:00.140 Win32 Start Address LogonUI!wWinMainCRTStartup (0x00000000ffb45c58) Stack Init fffff88004911db0 Current fffff88004910fd0 Base fffff88004912000 Limit fffff88004908000 Call 0 Priority 15 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`04911010 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`04911150 fffff800`01ad2c4b nt!KiCommitThreadWait+0x1d2 fffff880`049111e0 fffff800`01dcbecf nt!KeWaitForMultipleObjects+0x271 fffff880`04911490 fffff800`01dcc7d6 nt!ObpWaitForMultipleObjects+0x294 fffff880`04911960 fffff800`01ace853 nt!NtWaitForMultipleObjects+0xe5 fffff880`04911bb0 00000000`76e3046a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`04911c20) 00000000`001bf708 000007fe`fd0813a6 ntdll!NtWaitForMultipleObjects+0xa 00000000`001bf710 00000000`76be3143 KERNELBASE!WaitForMultipleObjectsEx+0xe8 00000000`001bf810 00000000`76cfbc3d kernel32!WaitForMultipleObjectsExImplementation+0xb3 00000000`001bf8a0 000007fe`fae19ecd USER32!RealMsgWaitForMultipleObjectsEx+0x12a 00000000`001bf940 000007fe`fae19d8e DUser!CoreSC::DUIMsgWaitForMultipleObjectsEx+0x17c 00000000`001bf9f0 00000000`76cf9079 DUser!MphMsgWaitForMultipleObjectsEx+0x7a 00000000`001bfa30 000007fe`fb8e407b USER32!MsgWaitForMultipleObjectsEx+0x37 00000000`001bfa70 000007fe`fb8e4f6c authui!CLogonFrame::DoModal+0×67 00000000`001bfaf0 000007fe`fb8e50cf authui!CLogonUI_CreateThenDoModalThenDestroy+0×299 00000000`001bfb50 00000000`ffb454df authui!CLogonUI::DoModal+0×73 00000000`001bfb80 00000000`ffb45ae6 LogonUI!wWinMain+0xfb 00000000`001bfbe0 00000000`76bdf56d LogonUI!ParseCommandLineToStringArrayLocalAlloc+0×33a 00000000`001bfca0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`001bfcd0 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

THREAD fffffa8006595720 Cid 06d0.1158 Teb: 000007fffffdc000 Win32Thread: fffff900c35105f0 WAIT: (UserRequest) UserMode Non-Alertable fffffa8005cad160 SynchronizationEvent fffffa8005618d30 SynchronizationEvent Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664381 Ticks: 2345 (0:00:00:36.582) Context Switch Count 2 LargeStack UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address msvcrt!endthreadex (0x000007feff0573fc) Stack Init fffff88005638db0 Current fffff88005637fd0 Base fffff88005639000 Limit fffff88005632000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`05638010 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`05638150 fffff800`01ad2c4b nt!KiCommitThreadWait+0x1d2 fffff880`056381e0 fffff800`01dcbecf nt!KeWaitForMultipleObjects+0x271 fffff880`05638490 fffff800`01dcc7d6 nt!ObpWaitForMultipleObjects+0x294 fffff880`05638960 fffff800`01ace853 nt!NtWaitForMultipleObjects+0xe5 fffff880`05638bb0 00000000`76e3046a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`05638c20) 00000000`00eaf4d8 000007fe`fd0813a6 ntdll!NtWaitForMultipleObjects+0xa 00000000`00eaf4e0 00000000`76be3143 KERNELBASE!WaitForMultipleObjectsEx+0xe8 00000000`00eaf5e0 00000000`76cfbc3d kernel32!WaitForMultipleObjectsExImplementation+0xb3 00000000`00eaf670 000007fe`fae114e6 USER32!RealMsgWaitForMultipleObjectsEx+0x12a 00000000`00eaf710 000007fe`fae116b2 DUser!CoreSC::Wait+0x62 00000000`00eaf760 000007fe`fae205dd DUser!CoreSC::xwProcessNL+0xed 00000000`00eaf7d0 000007fe`fae20500 DUser!GetMessageExA+0x7b 00000000`00eaf820 000007fe`ff0542bf DUser!ResourceManager::SharedThreadProc+0xe8 00000000`00eaf8b0 000007fe`ff057459 msvcrt!endthreadex+0x47 00000000`00eaf8e0 00000000`76bdf56d msvcrt!endthreadex+0xe0 00000000`00eaf910 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`00eaf940 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa8006646060 Cid 06d0.1174 Teb: 000007fffffda000 Win32Thread: fffff900c397bc30 WAIT: (UserRequest) UserMode Non-Alertable fffffa80059522e0 SynchronizationEvent fffffa80061cf2d0 SynchronizationEvent Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664855 Ticks: 1871 (0:00:00:29.187) Context Switch Count 101 LargeStack UserTime 00:00:00.015 KernelTime 00:00:00.015 Win32 Start Address authui!CCredentialProviderThread::_sThreadProc (0x000007fefb8e51c0) Stack Init fffff880057addb0 Current fffff880057acfd0 Base fffff880057ae000 Limit fffff880057a6000 Call 0 Priority 15 BasePriority 13 UnusualBoost 0 ForegroundBoost 1 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`057ad010 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`057ad150 fffff800`01ad2c4b nt!KiCommitThreadWait+0x1d2 fffff880`057ad1e0 fffff800`01dcbecf nt!KeWaitForMultipleObjects+0x271 fffff880`057ad490 fffff800`01dcc7d6 nt!ObpWaitForMultipleObjects+0x294 fffff880`057ad960 fffff800`01ace853 nt!NtWaitForMultipleObjects+0xe5 fffff880`057adbb0 00000000`76e3046a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`057adc20) 00000000`02c5f9b8 000007fe`fd0813a6 ntdll!NtWaitForMultipleObjects+0xa 00000000`02c5f9c0 00000000`76be3143 KERNELBASE!WaitForMultipleObjectsEx+0xe8 00000000`02c5fac0 00000000`76cfbc3d kernel32!WaitForMultipleObjectsExImplementation+0xb3 00000000`02c5fb50 00000000`76cf905a USER32!RealMsgWaitForMultipleObjectsEx+0x12a 00000000`02c5fbf0 000007fe`febdb46a USER32!MsgWaitForMultipleObjectsEx+0x46 00000000`02c5fc30 000007fe`fecfa542 ole32!CCliModalLoop::BlockFn+0xc2 00000000`02c5fc80 000007fe`fb8e4bc1 ole32!CoWaitForMultipleHandles+0x102 00000000`02c5fd90 000007fe`fb8e4a4a authui!InternalCoWaitForSingleHandle+0×31 00000000`02c5fdd0 000007fe`fb8e51c9 authui!CCredentialProviderThread::_vThreadProc+0xbf 00000000`02c5fe10 00000000`76bdf56d authui!CCredentialProviderThread::_sThreadProc+0×9 00000000`02c5fe40 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`02c5fe70 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

THREAD fffffa8005a9b2a0 Cid 06d0.1248 Teb: 000007fffffd4000 Win32Thread: fffff900c397b850 WAIT: (UserRequest) UserMode Non-Alertable fffffa800559c800 NotificationEvent Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664851 Ticks: 1875 (0:00:00:29.250) Context Switch Count 12 LargeStack UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWorkerThread (0×0000000076df8f00) Stack Init fffff88005871db0 Current fffff88005871900 Base fffff88005872000 Limit fffff8800586b000 Call 0 Priority 15 BasePriority 13 UnusualBoost 0 ForegroundBoost 2 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`05871940 fffff800`01ad6752 nt!KiSwapContext+0×7a fffff880`05871a80 fffff800`01ad88af nt!KiCommitThreadWait+0×1d2 fffff880`05871b10 fffff800`01dcadb2 nt!KeWaitForSingleObject+0×19f fffff880`05871bb0 fffff800`01ace853 nt!NtWaitForSingleObject+0xb2 fffff880`05871c20 00000000`76e2fefa nt!KiSystemServiceCopyEnd+0×13 (TrapFrame @ fffff880`05871c20) 00000000`02aee898 000007fe`fd0810ac ntdll!NtWaitForSingleObject+0xa 00000000`02aee8a0 000007fe`fb8e4586 KERNELBASE!WaitForSingleObjectEx+0×79 00000000`02aee940 000007fe`fb8e891c authui!InternalWaitForSingleObject+0×26 00000000`02aee980 000007fe`fb8e8ac4 authui!WPP_SF_qqddd+0×157d 00000000`02aee9e0 000007fe`fea7c7f5 authui!WluirRequestCredentials+0×44 00000000`02aeea20 000007fe`feb2b62e RPCRT4!Invoke+0×65 00000000`02aeeaa0 000007fe`fea74070 RPCRT4!Ndr64StubWorker+0×61b 00000000`02aef060 000007fe`fea79c24 RPCRT4!NdrServerCallAll+0×40 00000000`02aef0b0 000007fe`fea79d86 RPCRT4!DispatchToStubInCNoAvrf+0×14 00000000`02aef0e0 000007fe`fea7c44b RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0×146 00000000`02aef200 000007fe`fea7c38b RPCRT4!RPC_INTERFACE::DispatchToStub+0×9b 00000000`02aef240 000007fe`fea7c322 RPCRT4!RPC_INTERFACE::DispatchToStubWithObject+0×5b 00000000`02aef2c0 000007fe`fea7a11d RPCRT4!LRPC_SCALL::DispatchRequest+0×422 00000000`02aef3a0 000007fe`fea87ddf RPCRT4!LRPC_SCALL::HandleRequest+0×20d 00000000`02aef4d0 000007fe`fea87995 RPCRT4!LRPC_ADDRESS::ProcessIO+0×3bf 00000000`02aef610 00000000`76dfb43b RPCRT4!LrpcIoComplete+0xa5 00000000`02aef6a0 00000000`76df923f ntdll!TppAlpcpExecuteCallback+0×26b 00000000`02aef730 00000000`76bdf56d ntdll!TppWorkerThread+0×3f8 00000000`02aefa30 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`02aefa60 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

THREAD fffffa8005941a10 Cid 06d0.0f10 Teb: 000007fffffae000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Alertable fffffa800663a9a0 SynchronizationTimer fffffa8005881650 SynchronizationTimer fffffa8006577ef0 SynchronizationTimer fffffa8005a93bd0 NotificationEvent fffffa80063f6450 SynchronizationEvent fffffa80058fe4c0 SynchronizationEvent fffffa80064c0290 SynchronizationEvent fffffa8004e49e90 NotificationEvent Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664421 Ticks: 2305 (0:00:00:35.958) Context Switch Count 11 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWaiterpThread (0x0000000076df9a90) Stack Init fffff88006946db0 Current fffff88006945fd0 Base fffff88006947000 Limit fffff88006941000 Call 0 Priority 15 BasePriority 13 UnusualBoost 0 ForegroundBoost 2 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`06946010 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`06946150 fffff800`01ad2c4b nt!KiCommitThreadWait+0x1d2 fffff880`069461e0 fffff800`01dcbecf nt!KeWaitForMultipleObjects+0x271 fffff880`06946490 fffff800`01dcc7d6 nt!ObpWaitForMultipleObjects+0x294 fffff880`06946960 fffff800`01ace853 nt!NtWaitForMultipleObjects+0xe5 fffff880`06946bb0 00000000`76e3046a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`06946c20) 00000000`02dbf718 00000000`76df9bd7 ntdll!NtWaitForMultipleObjects+0xa 00000000`02dbf720 00000000`76bdf56d ntdll!TppWaiterpThread+0x14d 00000000`02dbf9c0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`02dbf9f0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa80056de060 Cid 06d0.0ba8 Teb: 000007fffffac000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable fffffa8005f7d3e0 QueueObject Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664389 Ticks: 2337 (0:00:00:36.457) Context Switch Count 5 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWorkerThread (0x0000000076df8f00) Stack Init fffff8800569ddb0 Current fffff8800569d7d0 Base fffff8800569e000 Limit fffff88005698000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`0569d810 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`0569d950 fffff800`01ada1c1 nt!KiCommitThreadWait+0x1d2 fffff880`0569d9e0 fffff800`01dcb9d7 nt!KeRemoveQueueEx+0x301 fffff880`0569da90 fffff800`01adf996 nt!IoRemoveIoCompletion+0x47 fffff880`0569db20 fffff800`01ace853 nt!NtWaitForWorkViaWorkerFactory+0x285 fffff880`0569dc20 00000000`76e317ba nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`0569dc20) 00000000`035cfbb8 00000000`76df914b ntdll!NtWaitForWorkViaWorkerFactory+0xa 00000000`035cfbc0 00000000`76bdf56d ntdll!TppWorkerThread+0x2c9 00000000`035cfec0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`035cfef0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa8005ccfa10 Cid 06d0.03a0 Teb: 000007fffffd8000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable fffffa8005f7d3e0 QueueObject Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664420 Ticks: 2306 (0:00:00:35.973) Context Switch Count 7 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWorkerThread (0x0000000076df8f00) Stack Init fffff8800459bdb0 Current fffff8800459b7d0 Base fffff8800459c000 Limit fffff88004596000 Call 0 Priority 15 BasePriority 13 UnusualBoost 0 ForegroundBoost 2 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`0459b810 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`0459b950 fffff800`01ada1c1 nt!KiCommitThreadWait+0x1d2 fffff880`0459b9e0 fffff800`01dcb9d7 nt!KeRemoveQueueEx+0x301 fffff880`0459ba90 fffff800`01adf996 nt!IoRemoveIoCompletion+0x47 fffff880`0459bb20 fffff800`01ace853 nt!NtWaitForWorkViaWorkerFactory+0x285 fffff880`0459bc20 00000000`76e317ba nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`0459bc20) 00000000`02e5f8c8 00000000`76df914b ntdll!NtWaitForWorkViaWorkerFactory+0xa 00000000`02e5f8d0 00000000`76bdf56d ntdll!TppWorkerThread+0x2c9 00000000`02e5fbd0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`02e5fc00 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa800662a800 Cid 06d0.0a54 Teb: 000007fffffaa000 Win32Thread: 0000000000000000 WAIT: (DelayExecution) UserMode Non-Alertable fffffa800662aad8 Semaphore Limit 0x2 Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664389 Ticks: 2337 (0:00:00:36.457) Context Switch Count 1 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ole32!CRpcThreadCache::RpcWorkerThreadEntry (0x000007fefebf3570) Stack Init fffff8800568fdb0 Current fffff8800568f970 Base fffff88005690000 Limit fffff8800568a000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`0568f9b0 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`0568faf0 fffff800`01ad8e56 nt!KiCommitThreadWait+0x1d2 fffff880`0568fb80 fffff800`01dcacee nt!KeDelayExecutionThread+0x186 fffff880`0568fbf0 fffff800`01ace853 nt!NtDelayExecution+0x59 fffff880`0568fc20 00000000`76e301fa nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`0568fc20) 00000000`0371fa68 000007fe`fd081203 ntdll!NtDelayExecution+0xa 00000000`0371fa70 000007fe`febeea00 KERNELBASE!SleepEx+0xab 00000000`0371fb10 000007fe`febf2046 ole32!CROIDTable::WorkerThreadLoop+0x10 00000000`0371fb40 000007fe`febf358a ole32!CRpcThread::WorkerLoop+0x1e 00000000`0371fb80 00000000`76bdf56d ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x1a 00000000`0371fbb0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`0371fbe0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa80063a4490 Cid 06d0.0ca0 Teb: 000007fffffa8000 Win32Thread: fffff900c1fffc30 WAIT: (WrLpcReceive) UserMode Non-Alertable fffffa80063a4850 Semaphore Limit 0x1 Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664404 Ticks: 2322 (0:00:00:36.223) Context Switch Count 11 LargeStack UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address MSCTF!CCtfServerPort::StaticServerThread (0x000007fefe959274) Stack Init fffff88005b30db0 Current fffff88005b30750 Base fffff88005b31000 Limit fffff88005b2a000 Call 0 Priority 15 BasePriority 13 UnusualBoost 0 ForegroundBoost 2 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`05b30790 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`05b308d0 fffff800`01ad88af nt!KiCommitThreadWait+0x1d2 fffff880`05b30960 fffff800`01dcf329 nt!KeWaitForSingleObject+0x19f fffff880`05b30a00 fffff800`01dd0a37 nt!AlpcpReceiveMessagePort+0x189 fffff880`05b30a60 fffff800`01dd1f76 nt!AlpcpReceiveMessage+0x2d4 fffff880`05b30b00 fffff800`01ace853 nt!NtAlpcSendWaitReceivePort+0x1e6 fffff880`05b30bb0 00000000`76e3070a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`05b30c20) 00000000`0390e7b8 000007fe`fe9426a9 ntdll!ZwAlpcSendWaitReceivePort+0xa 00000000`0390e7c0 000007fe`fe959417 MSCTF!CCtfServerPort::ServerLoop+0x16c 00000000`0390f8e0 000007fe`fe959296 MSCTF!CCtfServerPort::ServerThread+0x15b 00000000`0390fc20 00000000`76bdf56d MSCTF!CCtfServerPort::StaticServerThread+0x28 00000000`0390fc50 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`0390fc80 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

THREAD fffffa800489eb60 Cid 06d0.13b8 Teb: 000007fffffa6000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable fffffa8005833be0 NotificationEvent fffffa8005a03ad0 SynchronizationEvent Not impersonating DeviceMap fffff8a000008c10 Owning Process fffffa8005f442b0 Image: LogonUI.exe Attached Process N/A Image: N/A Wait Start TickCount 34664421 Ticks: 2305 (0:00:00:35.958) Context Switch Count 19 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address SmartcardCredentialProvider!I_ReaderMonitorThreadProc (0x000007feed747028) Stack Init fffff88005894db0 Current fffff88005893fd0 Base fffff88005895000 Limit fffff8800588f000 Call 0 Priority 15 BasePriority 13 UnusualBoost 0 ForegroundBoost 1 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`05894010 fffff800`01ad6752 nt!KiSwapContext+0x7a fffff880`05894150 fffff800`01ad2c4b nt!KiCommitThreadWait+0x1d2 fffff880`058941e0 fffff800`01dcbecf nt!KeWaitForMultipleObjects+0x271 fffff880`05894490 fffff800`01dcc7d6 nt!ObpWaitForMultipleObjects+0x294 fffff880`05894960 fffff800`01ace853 nt!NtWaitForMultipleObjects+0xe5 fffff880`05894bb0 00000000`76e3046a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`05894c20) 00000000`02d1f948 000007fe`fd0813a6 ntdll!NtWaitForMultipleObjects+0xa 00000000`02d1f950 00000000`76bcf190 KERNELBASE!WaitForMultipleObjectsEx+0xe8 00000000`02d1fa50 000007fe`ed746b84 kernel32!WaitForMultipleObjects+0xb0 00000000`02d1fae0 000007fe`ed747059 SmartcardCredentialProvider!I_ReaderMonitorWorker+0×9c 00000000`02d1fb80 00000000`76bdf56d SmartcardCredentialProvider!I_ReaderMonitorThreadProc+0×31 00000000`02d1fbc0 00000000`76e13281 kernel32!BaseThreadInitThunk+0xd 00000000`02d1fbf0 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

So according to memory dump analysis pattern terminology these 2 processes are strongly coupled and this fact can be used for analysis logon problems in terminal services environments: http://www.dumpanalysis.org/blog/index.php/2007/09/26/crash-dump-analysis-patterns-part-28/

intrauser isolation (p. 459)

file object security (p. 460) - here is an example from x64 W2K8 R2:

0: kd> !handle [...] 0008: Object: fffffa800658e070 GrantedAccess: 00100020 Entry: fffff8a00445d020 Object: fffffa800658e070 Type: (fffffa8003c0dde0) File ObjectHeader: fffffa800658e040 (new version) HandleCount: 1 PointerCount: 1 Directory Object: 00000000 Name: \DL\Notmyfault\exe\x64\Release {HarddiskVolume2} […] 001c: Object: fffffa8005f44ee0 GrantedAccess: 001f0003 (Protected) Entry: fffff8a00445d070 Object: fffffa8005f44ee0 Type: (fffffa8003c00570) Event ObjectHeader: fffffa8005f44eb0 (new version) HandleCount: 1 PointerCount: 2 […]

0: kd> dt _OBJECT_TYPE fffffa8003c0dde0 ntdll!_OBJECT_TYPE +0x000 TypeList : _LIST_ENTRY [ 0xfffffa80`03c0dde0 - 0xfffffa80`03c0dde0 ] +0x010 Name : _UNICODE_STRING "File" +0x020 DefaultObject : 0x00000000`00000098 +0x028 Index : 0x1c '' +0x02c TotalNumberOfObjects : 0x5645 +0x030 TotalNumberOfHandles : 0x89e +0x034 HighWaterNumberOfObjects : 0x5baf +0x038 HighWaterNumberOfHandles : 0x8b5 +0×040 TypeInfo : _OBJECT_TYPE_INITIALIZER +0×0b0 TypeLock : _EX_PUSH_LOCK +0×0b8 Key : 0×656c6946 +0×0c0 CallbackList : _LIST_ENTRY [ 0xfffffa80`03c0dea0 - 0xfffffa80`03c0dea0 ]

0: kd> dt _OBJECT_TYPE_INITIALIZER fffffa8003c0dde0+40 ntdll!_OBJECT_TYPE_INITIALIZER +0x000 Length : 0x70 +0x002 ObjectTypeFlags : 0x11 '' +0x002 CaseInsensitive : 0y1 +0x002 UnnamedObjectsOnly : 0y0 +0x002 UseDefaultObject : 0y0 +0x002 SecurityRequired : 0y0 +0x002 MaintainHandleCount : 0y1 +0x002 MaintainTypeList : 0y0 +0x002 SupportsObjectCallbacks : 0y0 +0x004 ObjectTypeCode : 1 +0x008 InvalidAttributes : 0x130 +0x00c GenericMapping : _GENERIC_MAPPING +0x01c ValidAccessMask : 0x1f01ff +0x020 RetainAccess : 0 +0x024 PoolType : 0 ( NonPagedPool ) +0x028 DefaultPagedPoolCharge : 0x400 +0x02c DefaultNonPagedPoolCharge : 0x180 +0x030 DumpProcedure : (null) +0x038 OpenProcedure : (null) +0x040 CloseProcedure : 0xfffff800`01de6890 void nt!IopCloseFile+0 +0x048 DeleteProcedure : 0xfffff800`01de6610 void nt!IopDeleteFile+0 +0x050 ParseProcedure : 0xfffff800`01df7370 long nt!IopParseFile+0 +0×058 SecurityProcedure : 0xfffff800`01db7130 long nt!IopGetSetSecurityObject+0 +0×060 QueryNameProcedure : 0xfffff800`01db7470 long nt!IopQueryName+0<> +0×068 OkayToCloseProcedure : (null)

0: kd> dt _OBJECT_TYPE_INITIALIZER fffffa8003c00570+40 ntdll!_OBJECT_TYPE_INITIALIZER +0x000 Length : 0x70 +0x002 ObjectTypeFlags : 0 '' +0x002 CaseInsensitive : 0y0 +0x002 UnnamedObjectsOnly : 0y0 +0x002 UseDefaultObject : 0y0 +0x002 SecurityRequired : 0y0 +0x002 MaintainHandleCount : 0y0 +0x002 MaintainTypeList : 0y0 +0x002 SupportsObjectCallbacks : 0y0 +0x004 ObjectTypeCode : 2 +0x008 InvalidAttributes : 0x100 +0x00c GenericMapping : _GENERIC_MAPPING +0x01c ValidAccessMask : 0x1f0003 +0x020 RetainAccess : 0 +0x024 PoolType : 0 ( NonPagedPool ) +0x028 DefaultPagedPoolCharge : 0 +0x02c DefaultNonPagedPoolCharge : 0x70 +0x030 DumpProcedure : (null) +0x038 OpenProcedure : (null) +0x040 CloseProcedure : (null) +0x048 DeleteProcedure : (null) +0x050 ParseProcedure : (null) +0×058 SecurityProcedure : 0xfffff800`01d97070 long nt!SeDefaultObjectMethod+0 +0×060 QueryNameProcedure : (null) +0×068 OkayToCloseProcedure : (null)