Software Diagnostics Library

In order to start the analysis of a structured memory snapshot a debugger engine needs Anchor Region that describes memory layout and where to start unfolding of analysis. For example, it can be a list of modules (another forthcoming structural pattern). We can observe the importance of such regions when we try to open corrupt or severely truncated memory dumps:

[...]
KdDebuggerDataBlock is not present or unreadable.
[...]
Unable to read PsLoadedModuleList
[...]

For certain types of memory snapshots (like software traces) an anchor region coinsides with its structure description (message trace format for structured snapshots) and a trace file header (if any) for aggregate snapshots.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Shared Buffer Overwrite differs from Local Buffer Overflow and heap / pool memory corruption patterns in not writing over control structures situated at dynamically allocated memory or procedure frame (local call stack) boundaries. Its effect is visible when the buffer data contains pointers that become wild after the overwrite and are later dereferenced resulting in a crash. For example, when the overwriting data contains UNICODE and /or ASCII characters we see them in a pointer data:

1: kd> !analyze -v

[...]

SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)
This is a very common bugcheck.  Usually the exception address pinpoints
the driver/function that caused the problem.  Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: c0000005, The exception code that was not handled
Arg2: 8086c949, The address that the exception occurred at
Arg3: f78eec54, Exception Record Address
Arg4: f78ee950, Context Record Address

[...]

EXCEPTION_RECORD:  f78eec54 -- (.exr 0xfffffffff78eec54)
ExceptionAddress: 8086c949 (nt!ObfDereferenceObject+0x00000023)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 00000001
   Parameter[1]: 006f0058
Attempt to write to address 006f0058

CONTEXT:  f78ee950 -- (.cxr 0xfffffffff78ee950)
eax=f78e0001 ebx=ffffffff ecx=006f0070 edx=00000000 esi=006f0058 edi=8087cdae
eip=8086c949 esp=f78eed1c ebp=f78eed2c iopl=0  nv up ei ng nz na pe nc
cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000     efl=00010286
nt!ObfDereferenceObject+0×23:
8086c949 f00fc11e        lock xadd dword ptr [esi],ebx ds:0023:006f0058=????????

[...]

STACK_TEXT:
f78eed2c f707212e 886e6530 f78eed80 f706e04e nt!ObfDereferenceObject+0x23
f78eed38 f706e04e e47b1258 8b2fcb40 808ae5c0 DriverA!CloseConnection+0x16
f78eed80 80880475 8835f248 00000000 8b2fcb40 DriverA!Resume+0x9f
f78eedac 80949c5a 8835f248 00000000 00000000 nt!ExpWorkerThread+0xeb
f78eeddc 8088e0c2 8088038a 00000000 00000000 nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

1: kd> ub f707212e
DriverA!CloseConnection+0x2:
f707211a push    ebp
f707211b mov     ebp,esp
f707211d push    esi
f707211e mov     esi,dword ptr [ebp+8]
f7072121 mov     ecx,dword ptr [esi+14h]
f7072124 test    ecx,ecx
f7072126 je      DriverA!CloseConnection+0x1a (f7072132)
f7072128 call    dword ptr [DriverA!_imp_ObfDereferenceObject (f70610f4)]

1: kd> db e47b1258 L20
e47b1258  61 67 65 20 57 72 69 74-65 72 00 05 77 00 69 00  age Writer..w.i.
e47b1268  6e 00 73 00 70 00 6f 00-6f 00 6c 00 2c 00 4e 00  n.s.p.o.o.l.,.N.

1: kd> !pool e47b1258
Pool page e47b1258 region is Paged pool
e47b1000 size:  108 previous size:    0  (Allocated)  CM39
e47b1108 size:   38 previous size:  108  (Free)       CMVa
e47b1140 size:   28 previous size:   38  (Allocated)  NtFs
e47b1168 size:    8 previous size:   28  (Free)       CMDa
e47b1170 size:   80 previous size:    8  (Allocated)  FSim
e47b11f0 size:   28 previous size:   80  (Allocated)  CMNb (Protected)
*e47b1218 size:   70 previous size:   28  (Allocated) *CMDa
Pooltag CMDa : value data cache pool tag, Binary : nt!cm
e47b1288 size:   58 previous size:   70  (Allocated)  Sect (Protected)
e47b12e0 size:   18 previous size:   58  (Allocated)  Ntf0
e47b12f8 size:   28 previous size:   18  (Allocated)  NtFs
e47b1320 size:   20 previous size:   28  (Allocated)  CMNb (Protected)
e47b1340 size:   48 previous size:   20  (Allocated)  Ntfc
e47b1388 size:   68 previous size:   48  (Allocated)  Sect (Protected)
e47b13f0 size:   30 previous size:   68  (Allocated)  CMVa
e47b1420 size:   38 previous size:   30  (Allocated)  CMVa
e47b1458 size:    8 previous size:   38  (Free)       CMVa
e47b1460 size:   48 previous size:    8  (Allocated)  CMVa
e47b14a8 size:   d0 previous size:   48  (Allocated)  Ntfo
e47b1578 size:  330 previous size:   d0  (Allocated)  Ntff
e47b18a8 size:   10 previous size:  330  (Free)       NtfE
e47b18b8 size:   e0 previous size:   10  (Allocated)  Ntfo
e47b1998 size:   40 previous size:   e0  (Allocated)  MmSm
e47b19d8 size:    8 previous size:   40  (Free)       Ica
e47b19e0 size:   18 previous size:    8  (Allocated)  Ntf0
e47b19f8 size:   68 previous size:   18  (Allocated)  CMDa
e47b1a60 size:   28 previous size:   68  (Allocated)  ObNm
e47b1a88 size:   b8 previous size:   28  (Allocated)  Port (Protected)
e47b1b40 size:   58 previous size:   b8  (Allocated)  Sect (Protected)
e47b1b98 size:   30 previous size:   58  (Allocated)  CMVa
e47b1bc8 size:    8 previous size:   30  (Free)       NtFA
e47b1bd0 size:  100 previous size:    8  (Allocated)  IoNm
e47b1cd0 size:   18 previous size:  100  (Allocated)  ObDi
e47b1ce8 size:   38 previous size:   18  (Allocated)  CMnb Process: 88469928
e47b1d20 size:   78 previous size:   38  (Free )  NtFI
e47b1d98 size:   68 previous size:   78  (Allocated)  CMDa
e47b1e00 size:   18 previous size:   68  (Allocated)  PsIm (Protected)
e47b1e18 size:   e8 previous size:   18  (Free )  TunP
e47b1f00 size:  100 previous size:   e8  (Allocated)  IoNm

Another example:

0: kd> !analyze -v

[...]

SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)
This is a very common bugcheck.  Usually the exception address pinpoints
the driver/function that caused the problem.  Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: c0000005, The exception code that was not handled
Arg2: 8083e4d6, The address that the exception occurred at
Arg3: f78cec54, Exception Record Address
Arg4: f78ce950, Context Record Address

[...]

EXCEPTION_RECORD:  f78cec54 -- (.exr 0xfffffffff78cec54)
ExceptionAddress: 8083e4d6 (nt!ObfDereferenceObject+0x00000023)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 00000001
   Parameter[1]: 65696c2b
Attempt to write to address 65696c2b

CONTEXT:  f78ce950 -- (.cxr 0xfffffffff78ce950)
eax=f78c0001 ebx=ffffffff ecx=65696c43 edx=00000000 esi=65696c2b edi=8083e407
eip=8083e4d6 esp=f78ced1c ebp=f78ced2c iopl=0  nv up ei ng nz na pe nc
cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000     efl=00010286
nt!ObfDereferenceObject+0×23:
8083e4d6 f00fc11e        lock xadd dword ptr [esi],ebx ds:0023:65696c2b=????????
Resetting default scope

[...]

STACK_TEXT:
f78ced2c f71bd12e 87216470 f78ced80 f71b904e nt!ObfDereferenceObject+0x23
f78ced38 f71b904e e49afb90 8a38eb40 808b70e0 DriverA!CloseConnection+0x16
f78ced80 8082db10 868989e0 00000000 8a38eb40 DriverA!Resume+0x9f
f78cedac 809208bb 868989e0 00000000 00000000 nt!ExpWorkerThread+0xeb
f78ceddc 8083fe9f 8082da53 00000000 00000000 nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

[...]

0: kd> .formats 65696c2b
Evaluate expression:
Hex:     65696c2b
Decimal: 1701407787
Octal:   14532266053
Binary:  01100101 01101001 01101100 00101011
  Chars:   eil+
Time:    Fri Dec 01 05:16:27 2023
Float:   low 6.88942e+022 high 0
Double:  8.40607e-315

0: kd> db e49afb90 L20
e49afb90  41 41 22 00 1e 00 00 00-00 5f 07 00 01 00 00 00  AA”……_……
e49afba0  01 00 00 00 43 6c 69 65-6e 74 41 2f 41 41 41 41  ….ClientA/AAAA

0: kd> !pool e49afb90
Pool page e49afb90 region is Paged pool
e49af000 size:  330 previous size:    0  (Allocated)  Ntff
e49af330 size:  2c0 previous size:  330  (Allocated)  Toke (Protected)
e49af5f0 size:   78 previous size:  2c0  (Allocated)  NtFU
e49af668 size:   10 previous size:   78  (Free)       CMVI
e49af678 size:   a8 previous size:   10  (Allocated)  Ntfo
e49af720 size:   80 previous size:   a8  (Allocated)  NtFU
e49af7a0 size:   78 previous size:   80  (Allocated)  NtFU
e49af818 size:   18 previous size:   78  (Allocated)  Ntf0
e49af830 size:   20 previous size:   18  (Allocated)  ObHd
e49af850 size:   38 previous size:   20  (Allocated)  MmSm
e49af888 size:   78 previous size:   38  (Allocated)  NtFU
e49af900 size:   28 previous size:   78  (Allocated)  NtFs
e49af928 size:   48 previous size:   28  (Allocated)  Ntfc
e49af970 size:   40 previous size:   48  (Allocated)  CMNb (Protected)
e49af9b0 size:   28 previous size:   40  (Allocated)  NtFs
e49af9d8 size:   30 previous size:   28  (Allocated)  AtmA
e49afa08 size:  108 previous size:   30  (Allocated)  CM39
e49afb10 size:   18 previous size:  108  (Allocated)  Ntf0
e49afb28 size:   30 previous size:   18  (Allocated)  CMVw (Protected)
e49afb58 size:   28 previous size:   30  (Allocated)  MPXC
*e49afb80 size:   70 previous size:   28  (Free) *CMDa
Pooltag CMDa : value data cache pool tag, Binary : nt!cm
e49afbf0 size:   b8 previous size:   70  (Allocated)  Port (Protected)
e49afca8 size:   28 previous size:   b8  (Allocated)  CMNb (Protected)
e49afcd0 size:  330 previous size:   28  (Allocated)  Ntff

Notice that in the latter example the pointer references a freed pool element. If a pointer points to an overwritten buffer the result is similar to a dangling pointer pointing to a reallocated freed buffer. If an object was located in a shared buffer and its data becomes overwritten we can also observe Random Object pattern.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Today we introduce an icon for Frame Pointer Omission pattern:

B/W

Color

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

One of the common mistakes that especially happens during a rush to provide an analysis results is overlooking UNICODE or ASCII fragments on thread stacks and mistakenly assuming that found symbolic references have some significance: 

0:001> dds 0bc9e5a8 0bc9e5d4
0bc9e5a8  00b900b8
0bc9e5ac  00bb00ba
0bc9e5b0  00bd00bc
0bc9e5b4  00bf00be ApplicationA!stdext::unchecked_uninitialized_fill_n<std::map<std:: basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> >,std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> >,std::less<std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > >,std::allocator<std::pair<std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > const ,std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > > > > * *,unsigned int,std::map<std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> >,std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> >,std::less<std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > >,std::allocator<std::pair<std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > const ,std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > > > > *,std::allocator<std::map<std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> >,std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> >,std::less<std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > >,std::allocator<std::pair<std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > const ,std::basic_string<unsigned short,std::char_traits<unsigned short>,std::allocator<unsigned short> > > > > *> >+0×1e
0bc9e5b8  00c100c0 ApplicationA!EnumData+0×670
0bc9e5bc  00c300c2 ApplicationA!CloneData+0xe2
0bc9e5c0  00c500c4 ApplicationA!LoadData+0×134
0bc9e5c4  00c700c6 ApplicationA!decompress+0×1ca6
0bc9e5c8  00c900c8 ApplicationA!_TT??_R2?AVout_of_rangestd+0×10c
0bc9e5cc  00cb00ca ApplicationA!AppHandle <PERF> (ApplicationA+0xd00ca)
0bc9e5d0  00cd00cc
0bc9e5d4  00cf00ce

0:001> du 0bc9e5a8
0bc9e5a8  "¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×"
0bc9e5e8  "ØÙÚÛÜÝÞßÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ÷"
0bc9e628  "ØÙÚÛÜÝÞŸĀ"

We can see and also double check from disassembly by using u/ub WinDbg command that function names are coincidental. It just happened that ApplicationA module spans the address range including 00bf00be and 00cb00ca UNICODE fragment values (having the pattern 00xx00xx):

0:001> lm m ApplicationA
start    end      module name
00be0000 00cb8000 ApplicationA (export symbols) ApplicationA.exe

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org - 

In addition to software narratology we would like to introduce another 2 branches of memoretics (borrowed from “geographical” chorography and chorology), the science of memory snapshots:

Software Chorography

The study and visualization of small memory regions compared to the full memory dumps.

Software Chorology

The spacial equivalent of software narratology where the latter is considered in chronological terms. Studies spatial distribution and causal relations between software behavioral and structural phenomena inside particular memory regions.

To give a perspective where usual software traces and memory dumps reside in terms narrativity and non-narrativity (spatiality) we created this diagram:

Memory dumps have some extension in the dimension of narrativity because it is possible to get stack traces and other execution residue from them that provide partial fragments of a software narrative.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Another important pattern category is Memory Hierarchy. Typical examples include a complete memory dump with a physical to virtual mapping and paged out memory. Please note that page file is optional and paging can be implemented without a page file. There can be several layers of hierarchy, for example:

1. physical memory
2. virtualized physical memory
3. virtual memory

Another example is from the protected mode x86 architecture:

1. physical memory
2. linear memory (paging, virtual)
3. logical memory (segments)

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes, when we find an Anchor Message related to our problem description (for example, a COM port error) we are interested in its evolution throughout a software narrative:

#       PID     TID     Message
[...]
126303  5768    1272    OpenComPort returns Status = 0x0
[...]
231610  3464    1576    OpenComPort returns Status = 0x0
[...]
336535  5768    4292    OpenComPort returns Status = 0×0
[…]
423508  5252    2544    OpenComPort returns Status = 0xc000000f
[…]
531247  5768    5492    OpenComPort returns Status = 0xc000000f
[…]
639039  772     3404    OpenComPort returns Status = 0xc000000f
[…]

Then we can check activity between changes. We call this pattern Message Change.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In seeking spiritual faith a software generalist views various religious worldviews as packages providing interfaces (IReligion). The methods of such interface will be discussed in the next part but for now, I show a UML diagram.

Today we introduce an icon for Special Process pattern:

B/W

Color

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

I now transition to 8 similar to 7-to-8 transition from user to kernel space (new dialectical quality):

switch(years_at_citrix)
{
  case 5:
  write_blog_post(”I’ve just passed 5 year mark … “);
  wait_for_certificate();
  write_blog_post(”Shortly after celebrating 5 years … “);
  break;
  case 6:
  write_blog_post(”Threads in my process run very fast. Not long ago … “);
  break;
  case 7:
  write_blog_post(”Transition to kernel mode and space … “);
  break;
  case 8:
  // … TBD
}

I also abandoned monthly counting started earlier because in kernel space time is counted in clock cycles per quantum unit where the latter is 1/3 of a clock tick (Windows Internals, 5th edition, pp. 406 - 408) :-)

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Tool Objects name was motivated by the title of the book Tool and Object: A History and Philosophy of Category Theory that I’m reading:

I realized that troubleshooting can be represented as a category of memory states (or collections of proximate states) as objects and troubleshooting tools as arrows.

We can also consider tools as categories with arrows as troubleshooting actions. In the latter case we can model a collection of tools and transformations (morphisms) between them as a functor.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

It’s time to update my old Resume and CV: As a Book that was on sale for almost 2 years to reflect exponential changes in objectives, skills, achievements and deliverables since summer, 2003. Actually the number of pages has doubled. The 2nd edition is planned to be 111th book scheduled for publication in December:

• Title: Resume and CV: As a Book (2nd Edition)
• Author: Dmitry Vostokov  
• Paperback: 32 pages
• Publisher: OpenTask (01 Dec 2010)
• ISBN-13: 978-1-908043-00-9

The front cover color follows the proverbs about “green fields” (I also fixed a bug that was on the first edition front cover)

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Today we introduce an icon for Suspended Thread pattern:

B/W

Color

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The previously published script to dump raw stack of all threads dumps only 64-bit raw stack from 64-bit WOW64 process memory dumps (a 32-bit process saved in a 64-bit dump). In order to dump WOW64 32-bit raw stack from such 64-bit dumps we need another script. After I found a location of 32-bit TEB pointers (WOW64 TEB32) inside a 64-bit TEB structure I was able to create such a script:

0:000> .load wow64exts

0:000> !teb
Wow64 TEB32 at 000000007efdd000

Wow64 TEB at 000000007efdb000
    ExceptionList:        000000007efdd000
    StackBase:            000000000008fd20
    StackLimit:           0000000000082000
    SubSystemTib:         0000000000000000
    FiberData:            0000000000001e00
    ArbitraryUserPointer: 0000000000000000
    Self:                 000000007efdb000
    EnvironmentPointer:   0000000000000000
    ClientId:             0000000000000f34 . 0000000000000290
    RpcHandle:            0000000000000000
    Tls Storage:          0000000000000000
    PEB Address:          000000007efdf000
    LastErrorValue:       0
    LastStatusValue:      0
    Count Owned Locks:    0
    HardErrorMode:        0

0:000:x86> !wow64exts.info

PEB32: 0x7efde000
PEB64: 0x7efdf000

Wow64 information for current thread:

TEB32: 0x7efdd000
TEB64: 0x7efdb000

32 bit, StackBase   : 0×1a0000
        StackLimit  : 0×190000
        Deallocation: 0xa0000

64 bit, StackBase   : 0x8fd20
        StackLimit  : 0x82000
        Deallocation: 0x50000

[...]

0:000:x86> dd 000000007efdd000 L4
7efdd000  0019fa84 001a0000 00190000 00000000

So the script obviously should be this:

~*e r? $t1 = ((ntdll!_NT_TIB *)@$teb)->ExceptionList; !wow64exts.info; dds poi(@$t1+8) poi(@$t1+4)

Before running it against a freshly opened user dump we need to execute the following commands first after setting symbols right:

.load wow64exts;
.effmach x86

I’ve also created a page to put all such scripts together:

Raw Stack Analysis Scripts 

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

I aim to release a beta version of that unified troubleshooting system together with EasyDbg (as WinDbg add-on) in a few months after finishing porting it to x64 platform. The system is very customizable and similar buttons can be created to launch troubleshooting tools and execute troubleshooting units of work for any client and server applications. It will also be covered in the forthcoming Software Troubleshooting and Debugging Tools presentation.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This pattern is called Invalid Exception Information and shows how to recognize it and get a stack trace right when a debugger is not able to locate a crash point. For example, WinDbg default analysis command is not able to locate the exception context for a crash and provides a heuristic stack trace:

0:000> !analyze -v

[...]

EXCEPTION_RECORD:  001150fc -- (.exr 0x1150fc)
ExceptionAddress: 7c7e2afb (kernel32!RaiseException+0x00000053)
   ExceptionCode: 0eedfade
  ExceptionFlags: 00000001
NumberParameters: 7
   Parameter[0]: 0098fa49
   Parameter[1]: 0374c200
   Parameter[2]: 00000000
   Parameter[3]: 005919b4
   Parameter[4]: 01d80010
   Parameter[5]: 00115704
   Parameter[6]: 001154a4

[...]

CONTEXT:  0012ffb4 — (.cxr 0×12ffb4)
eax=00000000 ebx=00000000 ecx=0000019c edx=00000214 esi=00000000 edi=00000000
eip=000003b0 esp=000002d8 ebp=2d59495b iopl=0 nv up ei ng zr na pe nc
cs=0032 ss=0010 ds=0002 es=0000 fs=0000 gs=0000 efl=000003e4
0032:000003b0 ??              ???

[...]

STACK_TEXT: 
7c910328 ntdll!`string'+0x4
7c7db7d0 kernel32!ConsoleApp+0xe
7c7db7a4 kernel32!ConDllInitialize+0x20f
7c7db7b9 kernel32!ConDllInitialize+0x224
7c915239 ntdll!bsearch+0x42
7c91542b ntdll!RtlpLocateActivationContextSection+0x15a
7c915474 ntdll!RtlpCompareActivationContextDataTOCEntryById+0x0
7c916104 ntdll!RtlpFindUnicodeStringInSection+0x23d
7c91534a ntdll!RtlpFindNextActivationContextSection+0x61
7c915742 ntdll!RtlFindNextActivationContextSection+0x46
7c9155ed ntdll!RtlFindActivationContextSectionString+0xde
7c915ce9 ntdll!RtlDecodeSystemPointer+0x9e7
7c915d47 ntdll!RtlDecodeSystemPointer+0xb0b
7c9158ff ntdll!RtlDecodeSystemPointer+0x45b
7c915bf8 ntdll!RtlDosApplyFileIsolationRedirection_Ustr+0x346
7c915c5d ntdll!RtlDosApplyFileIsolationRedirection_Ustr+0x3de
7c97e214 ntdll!DllExtension+0xc
00800000 ApplicationA+0x400000
7c910000 ntdll!RtlFreeHeap+0x1a4
7c914a53 ntdll!LdrLockLoaderLock+0x146
7c912d04 ntdll!LdrLockLoaderLock+0x1d2
7c912d71 ntdll!LdrUnlockLoaderLock+0x88
7c916768 ntdll!LdrGetDllHandleEx+0xc9
7c912d80 ntdll!`string'+0x84
7c91690e ntdll!LdrGetDllHandleEx+0x2f1
7c912d78 ntdll!LdrUnlockLoaderLock+0xb1
7c97ecc0 ntdll!LdrpHotpatchCount+0x8
7c9167e8 ntdll!`string'+0xc4
7c9168d6 ntdll!LdrGetDllHandleEx+0x2de
7c9166b8 ntdll!LdrGetDllHandle+0x18
7c7de534 kernel32!GetModuleHandleForUnicodeString+0x1d
7c7de544 kernel32!GetModuleHandleForUnicodeString+0xa0
7c7de64b kernel32!BasepGetModuleHandleExW+0x18e
7c7de6cb kernel32!BasepGetModuleHandleExW+0x250
79000000 mscoree!_imp__EnterCriticalSection <PERF> +0x0
7c809ad8 kernel32!_except_handler3+0x0
7c7de548 kernel32!`string'+0x28
79002280 mscoree!`string'+0x0
02080000 xpsp2res+0xc0000
7c7db6d4 kernel32!_BaseDllInitialize+0x7a
7c7db6e9 kernel32!_BaseDllInitialize+0x488
7c917ef3 ntdll!LdrpSnapThunk+0xbd
7c9048b8 ntdll!$$VProc_ImageExportDirectory+0x14b8
7c9000d0 ntdll!RtlDosPathSeperatorsString <PERF> +0x0
7c905d48 ntdll!$$VProc_ImageExportDirectory+0x2948
7c910228 ntdll!RtlpRunTable+0x448
7c910222 ntdll!RtlpAllocateFromHeapLookaside+0x42
7c911086 ntdll!RtlAllocateHeap+0x43d
7c903400 ntdll!$$VProc_ImageExportDirectory+0x0
7c7d9036 kernel32!$$VProc_ImageExportDirectory+0x6a0a
791c6f2d mscorwks!DllMain+0x117
7c917e10 ntdll!`string'+0xc
7c918047 ntdll!LdrpSnapThunk+0x317
7c7d00f0 kernel32!_imp___wcsnicmp <PERF> +0x0
7c7d903c kernel32!$$VProc_ImageExportDirectory+0x6a10
7c917dba ntdll!LdrpGetProcedureAddress+0x186
7c900000 ntdll!RtlDosPathSeperatorsString <PERF> +0x0
7c917e5f ntdll!LdrpGetProcedureAddress+0x29b
7c7d262c kernel32!$$VProc_ImageExportDirectory+0x0
7c7d0000 kernel32!_imp___wcsnicmp <PERF> +0x0
79513870 mscorsn!DllMain+0x119
7c913425 ntdll!RtlDecodePointer+0x0
00726574 ApplicationA+0x326574
7c917e09 ntdll!LdrpGetProcedureAddress+0xa6
7c917ec0 ntdll!LdrGetProcedureAddress+0x18
7c9101e0 ntdll!CheckHeapFillPattern+0x54
7c9101db ntdll!RtlAllocateHeap+0xeac
40ae17ea msxml6!calloc+0xa9
40ae181f msxml6!calloc+0xde
40a30000 msxml6!_imp__OpenThreadToken <PERF> +0x0
7c910323 ntdll!RtlpImageNtHeader+0x56
7c910385 ntdll!RtlImageDirectoryEntryToData+0x57
00400100 ApplicationA+0x100
7c928595 ntdll!LdrpCallTlsInitializers+0x1d
00400000 ApplicationA+0x0
7c9285c7 ntdll!LdrpCallTlsInitializers+0xd8
7c90118a ntdll!LdrpCallInitRoutine+0x14
00a23010 ApplicationA+0x623010
7c9285d0 ntdll!`string'+0x18
7c935e24 ntdll!LdrpInitializeThread+0x147
7c91b1b7 ntdll!LdrpInitializeThread+0x13b
778e159a SETUPAPI!_DllMainCRTStartup+0x0
7c91b100 ntdll!`string'+0x88
7c91b0a4 ntdll!_LdrpInitialize+0x25b
7c90de9a ntdll!NtTestAlert+0xc
7c91b030 ntdll!`string'+0xc8
7c91b02a ntdll!_LdrpInitialize+0x246
7c90d06a ntdll!NtContinue+0xc
7c90e45f ntdll!KiUserApcDispatcher+0xf
00780010 ApplicationA+0x380010
7c951e13 ntdll!DbgUiRemoteBreakin+0x0
7c97e178 ntdll!LdrpLoaderLock+0x0
00d10000 ApplicationA+0x910000
7c951e40 ntdll!DbgUiRemoteBreakin+0x2d
7c90e920 ntdll!_except_handler3+0x0
7c951e60 ntdll!`string'+0x7c

Compare our invalid context data with the normal one having good efl and segment register values:

cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206

We look at our stack trace after resetting context and using kv command. We see that KiUserExceptionDispatcher has the valid exception context but exception pointers for UnhandledExceptionFilter are not valid:

0:000> .ecxr

0:000> kv
ChildEBP RetAddr  Args to Child             
001132d0 7c90df4a 7c7d9590 00000002 001132fc ntdll!KiFastSystemCallRet
001132d4 7c7d9590 00000002 001132fc 00000001 ntdll!ZwWaitForMultipleObjects+0xc
00113370 7c7da115 00000002 001134a0 00000000 kernel32!WaitForMultipleObjectsEx+0x12c
0011338c 6993763c 00000002 001134a0 00000000 kernel32!WaitForMultipleObjects+0x18
00113d20 699382b1 00115018 00000001 00198312 faultrep!StartDWException+0x5df
00114d94 7c834526 00115018 00000001 00000000 faultrep!ReportFault+0x533
00115008 0040550c 00115018 7c9032a8 001150fc kernel32!UnhandledExceptionFilter+0×55b
WARNING: Stack unwind information not available. Following frames may be wrong.
00115034 7c90327a 001150fc 0012ffb4 0011512c ApplicationA+0×550c
001150e4 7c90e48a 00000000 0011512c 001150fc ntdll!ExecuteHandler+0×24
001150e4 7c7e2afb 00000000 0011512c 001150fc ntdll!KiUserExceptionDispatcher+0xe (CONTEXT @ 0011512c)
0011544c 0057ac37 0eedfade 00000001 00000007 kernel32!RaiseException+0×53
00115470 0098fa49 0eedfade 00000001 00000007 ApplicationA+0×17ac37
[…]
0012268c 7e398816 017d0f87 000607e8 0000001a USER32!InternalCallWinProc+0×28
001226f4 7e3a8ea0 00000000 017d0f87 000607e8 USER32!UserCallWinProcCheckWow+0×150

0:000> dd 00115018 L4
00115018  001150fc 0012ffb4 0011512c 001150d0

So we use the valid context pointer now:

0:000> .cxr 0011512c
eax=001153fc ebx=0eedfade ecx=00000000 edx=001537a8 esi=001154a4 edi=00000007
eip=7c7e2afb esp=001153f8 ebp=0011544c iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00200202
kernel32!RaiseException+0×53:
7c7e2afb 5e              pop     esi

0:000> kv
  *** Stack trace for last set context - .thread/.cxr resets it
ChildEBP RetAddr  Args to Child             
0011544c 0057ac37 0eedfade 00000001 00000007 kernel32!RaiseException+0x53
WARNING: Stack unwind information not available. Following frames may be wrong.
00115470 0098fa49 0eedfade 00000001 00000007 ApplicationA+0x17ac37
[...]
0012268c 7e398816 017d0f87 000607e8 0000001a USER32!InternalCallWinProc+0x28
001226f4 7e3a8ea0 00000000 017d0f87 000607e8 USER32!UserCallWinProcCheckWow+0x150
00122748 7e3aacd1 00fd2ad0 0000001a 00000000 USER32!DispatchClientMessage+0xa3
00122778 7c90e473 00122788 00000030 00000030 USER32!__fnINSTRING+0x37
001227b4 7e3993e9 7e3993a8 00122840 00000000 ntdll!KiUserCallbackDispatcher+0x13
001227e0 7e3aa43b 00122840 00000000 00000000 USER32!NtUserPeekMessage+0xc
0012280c 004794d9 00122840 00000000 00000000 USER32!PeekMessageA+0xeb
001228bc 00461667 0012ff7c 00461680 001228e0 ApplicationA+0x794d9
[...]

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

It is time now to introduce a syntactical notation for memory (dump) and software trace analysis pattern languages (in addition to graphical notation proposed earlier). It should be simple and concise: allow easy grammar with plain syntax and obvious reading semantics. We propose to use capitalized letters for major pattern categories, for example, W for wait chains and D for deadlocks. Then use subscripts (or small letters) for pattern subcategories, for example, Wcs and Dlpc. Several categories and subcategories can be combined by using slash (/), for example, Wcs/Dcs/lpc. Slash notation is better viewed using subscripts:

W_cs/D_cs/lpc

Next part will introduce more categories and propose notational adornments for pattern succession, space differentiation and the inclusion of details in notational sentences.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Today we introduce an icon for No System Dumps pattern:

B/W

Color

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The choice of webinars below mostly reflects my personal preferences and long time desire to speak on topics like systems thinking, troubleshooting tool design and development. The other topic about BSOD minidump analysis was requested by participants in an ongoing survey. There will be more topics in 2011. All forthcoming webinars will be hosted by Memory Dump Analysis Services. The planning list includes:

Systems Thinking in Memory Dump and Software Trace Analysis

Software Troubleshooting and Debugging Tools: Objects, Components, Patterns and Frameworks with UML

• UML basics
• DebugWare patterns
• Unified Troubleshooting Framework
• RADII software development process
• Hands-on exercise: designing and building a tool

Blue Screen Of Death Analysis Done Right: Minidump Investigation for System Administrators

• Making sense of !analyze –v output
• Get extra troubleshooting information with additional WinDbg commands
• Guessing culprits with raw stack analysis
• Who’s responsible: hardware or software?
• Checklist and patterns
• Including hands-on exercises: send your own minidumps

More detailed information will be available soon. 

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Don’t name your driver a “Missile” blog post dealt with funny names seen in crash dumps. However, even innocuous driver names may occasionally provoke a laughter from people in the know. For example, SGUB32.SYS can be read 23BUGS in reverse. My recent encounter is a print driver SGNUD64.dll where we read 46DUNGS in reverse. Don’t rush to Google the name to find ISV, it was modified to avoid an engineering embarrassment, although a dung was really there

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -