Software Diagnostics Library

A page to reference all different kinds of stack traces is necessary, so I created this post:

• Stack Trace
• Stack Trace Collection (unmanaged space)
• Special Stack Trace
• Exception Stack Trace
• Dual Stack Trace
• Truncated Stack Trace
• Managed Stack Trace
• Incorrect Stack Trace
• Stack Trace Set
• Stack Trace Collection (managed space)
• Stack Trace Collection (predicate)
• Empty Stack Trace
• Stack Trace Collection (I/O requests)
• Stack Trace Change
• First Fault Stack Trace
• Critical Stack Trace
• RIP Stack Trace
• Glued Stack Trace
• Rough Stack Trace (unmanaged space)
• Past Stack Trace
• Stack Trace (I/O request)
• Stack Trace (file system filters)
• Stack Trace (database)
• Variable Subtrace
• Technology-Specific Subtrace (COM interface invocation)
• Technology-Specific Subtrace (dynamic memory)
• Technology-Specific Subtrace (JIT .NET code)
• Technology-Specific Subtrace (COM client call)
• Internal Stack Trace
• Stack Trace Collection (CPUs)
• Stack Trace Surface
• Hidden Stack Trace
• Constant Subtrace
• Stack Trace Signature
• Quotient Stack Trace
• Module Stack Trace
• Coincidental Frames
• Least Common Frame
• Foreign Module Frame
• Unified Stack Trace
• Aggregated Frames
• Stack Trace (I/O devices)
• Stack Trace Motif
• Stack Trace Race
• Source Stack Trace
• Hidden Stack
• Interrupt Stack
• Frame Trace
• False Frame
• Procedure Call Chain
• Rough Stack Trace (managed space)

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

A page to reference all different kinds of .NET related patterns is necessary, so I created this post:

• CLR Thread
• Managed Code Exception
• Nested Exceptions (managed code)
• Mixed Exception
• Memory Leak (.NET heap)
• JIT Code (.NET)
• Managed Stack Trace
• Multiple Exceptions (managed space)
• Version-Specific Extension
• Caller-n-Callee
• Technology-Specific Subtrace (JIT .NET code)
• Inline Function Optimization (managed code)
• Annotated Disassembly (JIT .NET code)
• Handled Exception (.NET CLR)
• Execution Residue (managed space)
• Deadlock (managed space)
• Duplicate Extension
• Stack Trace Collection (managed space)
• Dynamic Memory Corruption (managed heap)
• Special Thread (.NET CLR)
• Wait Chain (CLR monitors)
• Distributed Exception (managed code)
• Object Distribution Anomaly (.NET heap)
• High Contention (.NET heap)
• Deadlock (.NET Finalizer)
• Hidden Exception (managed space)
• Rough Stack Trace (managed space)
• Saved Exception Context (.NET)

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The first Webinar to start an in-depth discussion of pattern-driven software troubleshooting, debugging and maintenance:

Date: 25th of March 2011
Time: 18:30 (GMT) 14:30 (EST) 11:30 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/448268158

Topics include:

• A Short History of DumpAnalysis.org
• Memory Dump Analysis Patterns
• Troubleshooting and Debugging Tools (Debugware) Patterns
• Software Trace Analysis Patterns
• From Software Defects to Software Behavior
• Workaround Patterns
• Structural Memory Patterns
• Memory Analysis Domain Pattern Hierarchy
• New Directions

Prerequisites: experience in software troubleshooting and/or debugging.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

A page to reference all different kinds of stack overflow is necessary, so I created this post:

• Stack Overflow (kernel mode)
• Stack Overflow (user mode)
• Stack Overflow (software implementation)
• Stack Overflow (insufficient memory)

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

A page to reference all different kinds of contention is necessary, so I created this post:

• High Contention (executive resources)
• High Contention (critical sections)
• High Contention (processors)
• High Contention (.NET CLR monitors)
• Critical Region
• High Contention (.NET heap)

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

I’ve decided to adapt archaeological classificatory framework (using my favourite method of inquiry: metaphorical bijectionism) to lay out foundations for yet another attempt to classify DA+TA patterns):

Attribute  ↔ Pattern
Artefact   ↔ Component Artefact1
Assemblage ↔ Component Assemblage
Culture    ↔ Memory System Culture²

1 Can be either a component-generated artefact or a component like a module or symbol file
2 Typical examples of memory system cultures are Windows, UNIX or even “Multiplatform”

I propose a word Memoarchaeological for such a framework and Memoarchaeology for a branch of Memoretics that studies saved computer memory artifacts from past computations (as opposed to live memory).

Note: In one of the forthcoming issues of Debugged! MZ/PE magazine there will be presented yet another classificatory scheme.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

OpenTask to offer first 3 volumes of Memory Dump Analysis Anthology in one set:

The set is available exclusively from OpenTask e-Commerce web site starting from June. Individual volumes are also available from Amazon, Barnes & Noble and other bookstores worldwide.

Product information:

• Title: Modern Memory Dump and Software Trace Analysis: Volumes 1-3
• Author: Dmitry Vostokov
• Language: English
• Product Dimensions: 22.86 x 15.24
• Paperback: 1600 pages
• Publisher: Opentask (31 May 2010)
• ISBN-13: 978-1-906717-99-5

Information about individual volumes:

• Volume 1
• Volume 2
• Volume 3

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Plan to start providing training and seminars in my free time. If you are interested please answer these questions (you can either respond here in comments or use this form for private communication http://www.dumpanalysis.org/contact):

• Are you interested in on-site training, prefer traveling or attending webinars?
• Are you interested in software trace analysis as well?
• What specific topics are you interested in?
• What training level (beginner, intermediate, advanced) are you interested in? (please provide an example, if possible)

Additional topics of expertise that can be integrated into training include Source Code Reading and Analysis, Debugging, Windows Architecture, Device Drivers, Troubleshooting Tools Design and Implementation, Multithreading, Deep Down C and C++, x86 and x64 Assembly Language Reading.

Looking forward to your responses. Any suggestions are welcome.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This is a revised, edited, cross-referenced and thematically organized volume of selected DumpAnalysis.org blog posts about crash dump analysis and debugging written in July 2009 - January 2010 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software on Windows platforms and technical support and escalation engineers dealing with complex software issues. The fourth volume features:

- 13 new crash dump analysis patterns
- 13 new pattern interaction case studies
- 10 new trace analysis patterns
- 6 new Debugware patterns and case study
- Workaround patterns
- Updated checklist
- Fully cross-referenced with Volume 1, Volume 2 and Volume 3
- New appendixes

Product information:

• Title: Memory Dump Analysis Anthology, Volume 4
• Author: Dmitry Vostokov
• Language: English
• Product Dimensions: 22.86 x 15.24

• Paperback: 410 pages
• Publisher: Opentask (30 March 2010)
• ISBN-13: 978-1-906717-86-5

• Hardcover: 410 pages
• Publisher: Opentask (30 April 2010)
• ISBN-13: 978-1-906717-87-2

Back cover features memory space art image: Internal Process Combustion.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

“Memory dumps are facts.”

I’m very excited to announce that Volume 3 is available in paperback, hardcover and digital editions:

Memory Dump Analysis Anthology, Volume 3

Table of Contents

In two weeks paperback edition should also appear on Amazon and other bookstores. Amazon hardcover edition is planned to be available in January 2010.

The amount of information was so voluminous that I had to split the originally planned volume into two. Volume 4 should appear by the middle of February together with Color Supplement for Volumes 1-4. 

- Dmitry Vostokov @ DumpAnalysis.org -

To commemorate 200 years since Darwin’s birth and 150 years since the publication of The Origin of Species, OpenTask plans to dedicate December, 2009 issue of Debugged! MZ/PE magazine to taxonomy and evolution of software defects and March, 2010 issue to multithreading (as natural selection).

Debugged! MZ/PE: Systematics and Evolution of Software Defects, December, 2009 (Paperback, ISBN: 978-1906717827)

Debugged! MZ/PE: Multithreading, March, 2010 (Paperback, ISBN: 978-1906717834)

Here are draft front covers designed during last weekend:

- Dmitry Vostokov @ DumpAnalysis.org -

While preparing a presentation for an engineering conference I found that a page to reference various wait chain patterns is necessary, so I created this post:

• Wait Chain (general)
• Wait Chain (critical sections)
• Wait Chain (executive resources)
• Wait Chain (thread objects)
• Wait Chain (LPC/ALPC)
• Wait Chain (process objects)
• Wait Chain (RPC)
• Wait Chain (window messaging)
• Wait Chain (named pipes)
• Wait Chain (mutex objects)
• Wait Chain (pushlocks)
• Distributed Wait Chain
• Wait Chain (CLR monitors)
• Screwbolt Wait Chain
• Wait Chain (RTL_RESOURCE)
• Wait Chain (modules)
• Wait Chain (nonstandard synchronization)
• Wait Chain (C++11, condition variable)
• Wait Chain (SRW lock)

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org -

While preparing a presentation for an engineering conference I found that another different page to reference link time problems is necessary, so I created this post:

• Missing Component (general)
• Missing Component (static linking, user mode)
• Duplicated Module
• Unloaded Module

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org -

While preparing a presentation for an engineering conference I found that yet another page to reference different kinds of insufficient memory patterns is necessary, so I created this post:

• Insufficient Memory (committed memory)
• Insufficient Memory (handle leak)
• Insufficient Memory (kernel pool)
• Insufficient Memory (PTE)
• Insufficient Memory (module fragmentation)
• Insufficient Memory (physical memory)
• Insufficient Memory (control blocks)
• Insufficient Memory (reserved virtual memory)
• Memory Leak (process heap)
• Memory Leak (.NET heap)
• Memory Leak (page tables)
• Memory Leak (I/O completion packets)
• Punctuated Memory Leak
• Insufficient Memory (session pool)
• Memory Fluctuation (process heap)
• Insufficient Memory (stack trace database)
• Insufficient Memory (region)
• Memory Leak (regions)
• Handle Leak
• Reference Leak
• Object Distribution Anomaly (.NET heap)
• Handle Limit (GDI, kernel space)
• Handle Limit (GDI, user space)
• Relative Memory Leak
• Insufficient Memory (stack)

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org -

While preparing a presentation for an engineering conference I found that another page to reference different kinds of dynamic memory corruption patterns is necessary, so I created this post too:

• Dynamic Memory Corruption (process heap)
• Dynamic Memory Corruption (kernel pool)
• Double Free (process heap)
• Double Free (kernel pool)
• Dynamic Memory Corruption (managed heap)

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org -

While preparing a presentation for an engineering conference I found that a page to reference all different kinds of deadlocks is necessary, so I created this post:

• Deadlock (critical sections)
• Deadlock (executive resources)
• Deadlock (mixed objects, user space)
• Deadlock (LPC)
• Deadlock (mixed objects, kernel space)
• Deadlock (self)
• Livelock
• Deadlock (managed space)
• Deadlock (.NET Finalizer)

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

As one of the new initiatives for the Year of Debugging  DumpAnalysis Portal will publish bimonthly full color 16 page publication called:

Debugged! MZ/PE: MagaZine for/from Practicing Engineers
The only serial publication dedicated entirely to Windows® debugging

The first issue is planned for March, 2009 and will have ISBN-13: 978-1-906717-38-4. If it goes well I’m planning to have ISSN number assigned to it too. More details will be announced soon.

- Dmitry Vostokov @ DumpAnalysis.org - 

This is a new word I’ve just coined to describe applications heavily dependent on various hooks that are either injected by normal Windows hooking mechanism, registry or via more elaborate tricks like remote threads or patching code. Originally I thought of hookware but found that this term is already in use for completely different purpose.

Now I list various patterns in memory dumps that help in detection, troubleshooting and debugging of hooksware:

- Hooked Functions (user space)

- Hooked Functions (kernel space)

- Hooking Level

This is the primary detection mechanism for hooks that patch code.

See also Raw Pointer and Out-of-Module Pointer patterns.

- Hooked Modules

The WinDbg script to run when you don’t know which module was patched.

- Changed Environment

Loaded hooks shift other DLLs by changing their load address and therefore might expose dormant bugs.

- Insufficient Memory (module fragmentation)

Hooks loaded in the middle of address space limit the maximum amount of memory that can be allocated at once. For example, various virtual machines, like Java, reserve the big chunk of memory at the start up.

- No Component Symbols

We can get an approximate picture of what a 3rd-party hook module does by looking at its import table or in the case of patching by looking at the list of deviations returned by .chkimg command.

- Unknown Component

Might give an idea about the author of the hook.

- Coincidental Symbolic Information

Sometimes hooks are loaded at round addresses like 0×10000000 and these values are very frequently used as flags or constants too.

- Wild Code

When hooking goes wrong the execution path goes into the wild territory.

- Execution Residue

Here we can find various hooks that use normal Windows hooking mechanism. Sometimes the search for “hook” word in symbolic raw stack output of dds command reveals them but beware of Coincidental Symbolic Information. See also Raw Stack Analysis Scripts page.

- Message Hooks - Modeling Example

Windows message hooking pattern example.

- Hidden Module

Some hooks may hide themselves.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The following table provides correspondence between memory dump types (user, kernel and complete) and pattern names. For ALL, where appropriate, I also list the most common dump types where a pattern can be encountered.

Table: CDA Pattern Classification (Memory dump type)

- Dmitry Vostokov @ DumpAnalysis.org -

The following table provides correspondence between kernel / user mode and space and pattern names. ALL means that a pattern is observable in both modes / spaces.

Table: CDA Pattern Classification (Space/Mode)

- Dmitry Vostokov @ DumpAnalysis.org -