Crash Dump Analysis

Crash Dump Analysis Patterns (Part 114)

Forthcoming Webinar on Pattern-Driven Software Diagnostics

2012 - The Year of Software Trace Analysis

Accelerated Memory Dump Analysis Training

Sponsored link: Memory Dump Analysis Services

One of the most common patterns is Crash Signature. It consists of a set of attributes derivable from saved execution context for exceptions, faults and traps. For example, on x64 Windows it is usually RIP and RSP addresses. For x86 it is usually EIP, ESP and EBP. It can also include the application module name.

0:009> !analyze -v

[...]

FAILURE_BUCKET_ID: SOFTWARE_NX_FAULT_c0000005_ApplicationA.exe!Unknown

BUCKET_ID: APPLICATION_FAULT_SOFTWARE_NX_FAULT_STACK_CORRUPTION_BAD_IP_ApplicationA+2d560

[...]

0:009> kL ChildEBP RetAddr 0354f270 75bc0962 ntdll!NtWaitForMultipleObjects+0x15 0354f30c 7651162d KERNELBASE!WaitForMultipleObjectsEx+0x100 0354f354 76511921 kernel32!WaitForMultipleObjectsExImplementation+0xe0 0354f370 76539b0d kernel32!WaitForMultipleObjects+0x18 0354f3dc 76539baa kernel32!WerpReportFaultInternal+0x186 0354f3f0 765398d8 kernel32!WerpReportFault+0x70 0354f400 76539855 kernel32!BasepReportFault+0x20 0354f48c 77750727 kernel32!UnhandledExceptionFilter+0x1af 0354f494 77750604 ntdll!__RtlUserThreadStart+0x62 0354f4a8 777504a9 ntdll!_EH4_CallFilterFunc+0x12 0354f4d0 777387b9 ntdll!_except_handler4+0x8e 0354f4f4 7773878b ntdll!ExecuteHandler2+0x26 0354f5a4 776f010f ntdll!ExecuteHandler+0x24 0354f5a4 0354f958 ntdll!KiUserExceptionDispatcher+0xf WARNING: Frame IP not in any known module. Following frames may be wrong. 0354f908 02ff0340 0×354f958 00000000 00000000 0×2ff0340

0:009> kv ChildEBP RetAddr Args to Child [...] 0354f5a4 0354f958 0154f5bc 0354f60c 0354f5bc ntdll!KiUserExceptionDispatcher+0xf (CONTEXT @ 0354f60c) WARNING: Frame IP not in any known module. Following frames may be wrong. 0354f908 02ff0340 00000000 00000000 00000000 0×354f958 00000000 00000000 00000000 00000000 00000000 0×2ff0340

0:009> .cxr 0354f60c eax=80010105 ebx=0354f924 ecx=00000003 edx=0000ffff esi=00d7dce0 edi=00d7e0c8 eip=0354f958 esp=0354f8f4 ebp=0354f908 iopl=0 nv up ei pl zr na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246 0354f958 64f9 stc

0:009> !address 0354f958 TEB 7efdd000 in range 7efdb000 7efde000 TEB 7efda000 in range 7efd8000 7efdb000 TEB 7efd7000 in range 7efd5000 7efd8000 TEB 7efaf000 in range 7efad000 7efb0000 TEB 7efac000 in range 7efaa000 7efad000 TEB 7efa9000 in range 7efa7000 7efaa000 TEB 7efa6000 in range 7efa4000 7efa7000 TEB 7efa3000 in range 7efa1000 7efa4000 TEB 7ef9f000 in range 7ef9d000 7efa0000 TEB 7ef9c000 in range 7ef9a000 7ef9d000 TEB 7ef99000 in range 7ef97000 7ef9a000 ProcessParametrs 007714b0 in range 00770000 00870000 Environment 007707f0 in range 00770000 00870000 03450000 : 0354d000 - 00003000 Type 00020000 MEM_PRIVATE Protect 00000004 PAGE_READWRITE State 00001000 MEM_COMMIT Usage RegionUsageStack Pid.Tid 1ea0.12dc

0:009> !address 02ff0340 TEB 7efdd000 in range 7efdb000 7efde000 TEB 7efda000 in range 7efd8000 7efdb000 TEB 7efd7000 in range 7efd5000 7efd8000 TEB 7efaf000 in range 7efad000 7efb0000 TEB 7efac000 in range 7efaa000 7efad000 TEB 7efa9000 in range 7efa7000 7efaa000 TEB 7efa6000 in range 7efa4000 7efa7000 TEB 7efa3000 in range 7efa1000 7efa4000 TEB 7ef9f000 in range 7ef9d000 7efa0000 TEB 7ef9c000 in range 7ef9a000 7ef9d000 TEB 7ef99000 in range 7ef97000 7ef9a000 ProcessParametrs 007714b0 in range 00770000 00870000 Environment 007707f0 in range 00770000 00870000 02fc0000 : 02fc0000 - 00043000 Type 00020000 MEM_PRIVATE Protect 00000004 PAGE_READWRITE State 00001000 MEM_COMMIT Usage RegionUsageHeap Handle 00d70000

Stack trace may or may not be included here and it might be incorrect, heuristic and not fully discernible automatically (requires raw stack semantic analysis) like in the example above. In some cases exception information might not be valid though, for example, in the case of laterally damaged or truncated memory dump files.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Citrix and Microsoft Customer Forum

Museum of Debugging and Memory Dumps

7/7/2011 - 8/8/2011 Annual Competition: Tell Your Windows Debugging Story

Crash and Hang Analysis Audit Service

CARE: Crash Analysis Report Environment

Crash Dump and Software Trace Analysis Training and Seminars

Access OpenTask Titles on Safari Books Online

DATA (Dump Analysis + Trace Analysis) Facebook group
Please join the community of memory (dump) and trace analysis engineers. This group promotes scientific methods and memory dump-based worldview.

Twitter @ DumpAnalysis
You can now follow portal and blog news at DumpAnalysis on Twitter

LinkedIn Group Dr. Watson Enthusiasts
All about Dr. Watson errors and more. Get news, excerpts and progress reports about the forthcoming book The Science of Dr. Watson: An Illustrated History of Debugging (ISBN 978-1906717070)

2010 (0x7DA) - The Year of Dump Analysis
2011 (0x7DB) - 2020 (0x7E4) The Debugging Decade

International Memory Analysts and Debuggers Day:
07.07 and/or 08.08 starting from The Year of Dump Analysis, 2010, 7DA

Announcements

Coming Soon:

Resume and CV: As a Book

Fundamentals of Complete Crash and Hang Memory Dump Analysis

Management Bits: An Anthology from Reductionist Manager

Crash Dump Analysis: Practical Foundations (Windows Edition, Systematic Software Fault Analysis Series)

Crash Dump Analysis for System Administrators and Support Engineers

New Magazines:

Debugged! MZ/PE: MagaZine for/from Practicing Engineers

New Books:

Advanced Windows Memory Dump Analysis with Data Structures: Training Course Transcript and WinDbg Practice Exercises with Notes

Accelerated .NET Memory Dump Analysis: Training Course Transcript and WinDbg Practice Exercises with Notes

Accelerated Windows Memory Dump Analysis: Training Course Transcript and WinDbg Practice Exercises with Notes

Introduction to Pattern-Driven Software Problem Solving

Memory Dump Analysis Anthology: Color Supplement for Volumes 4-5

Windows Debugging Notebook: Essential User Space WinDbg Commands

Memory Dump Analysis Anthology, Volume 5

Memory Dump Analysis Anthology, Volume 4

Memory Dump Analysis Anthology: Color Supplement for Volumes 1-3

Memory Dump Analysis Anthology, Volume 3

First Fault Software Problem Solving: A Guide for Engineers, Managers and Users

x64 Windows Debugging: Practical Foundations

Also available:

Windows Debugging: Practical Foundations

DLL List Landscape: The Art from Computer Memory Space

Dumps, Bugs and Debugging Forensics: The Adventures of Dr. Debugalov

WinDbg: A Reference Poster and Learning Cards

Memory Dump Analysis Anthology, Volume 2

Memory Dump Analysis Anthology, Volume 1

New Children's Book:

Baby Turing

Memory Dump It

This entry was posted on Tuesday, November 9th, 2010 at 11:51 am and is filed under Crash Dump Analysis, Crash Dump Patterns, Debugging. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Crash Dump Analysis Patterns (Part 114)

Leave a Reply