Crash Dump Analysis

Crash Dump Analysis Patterns (Part 127b)

Forthcoming Webinar on Pattern-Driven Software Diagnostics

2012 - The Year of Software Trace Analysis

Accelerated Memory Dump Analysis Training

Sponsored link: Memory Dump Analysis Services

Here we continue with Technology-Specific Subtrace pattern series started earlier with COM interface invocation example. In this part we consider dynamic memory allocation example in kernel space (kernel pool). Usually pool corruption is detected during pool memory allocation or release with a special bugcheck code, for example:

BAD_POOL_HEADER (19) The pool is already corrupt at the time of the current request. This may or may not be due to the caller. The internal pool links must be walked to figure out a possible cause of the problem, and then special pool applied to the suspect tags or the driver verifier to a suspect driver. Arguments: Arg1: 00000020, a pool block header size is corrupt. Arg2: 8b79d078, The pool entry we were looking for within the page. Arg3: 8b79d158, The next pool entry. Arg4: 8a1c0004, (reserved)

However, pool corruption might be deeper enough to trigger an access violation even before self-diagnosis. In such cases stack subtraces with functions like ExFreePoolWithTag might point to troubleshooting and debugging directions:

ATTEMPTED_WRITE_TO_READONLY_MEMORY (be) An attempt was made to write to readonly memory. The guilty driver is on the stack trace (and is typically the current instruction pointer). When possible, the guilty driver’s name (Unicode string) is printed on the bugcheck screen and saved in KiBugCheckDriver. Arguments: Arg1: 00470044, Virtual address for the attempted write. Arg2: 06d39025, PTE contents. Arg3: aec0fb30, (reserved) Arg4: 0000000a, (reserved)

TRAP_FRAME: aec0fb30 -- (.trap 0xffffffffaec0fb30) ErrCode = 00000003 eax=8ac12d38 ebx=8b700040 ecx=000001ff edx=00470040 esi=8ac12db8 edi=808b0b40 eip=808949e7 esp=aec0fba4 ebp=aec0fbf0 iopl=0 nv up ei pl nz na po nc cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010202 nt!ExFreePoolWithTag+0x6a3: 808949e7 895a04 mov dword ptr [edx+4],ebx ds:0023:00470044=????????

STACK_TEXT: aec0faa0 80860121 000000be 00470044 06d39025 nt!KeBugCheckEx+0x1b aec0fb18 8088e490 00000001 00470044 00000000 nt!MmAccessFault+0xb25 aec0fb18 808949e7 00000001 00470044 00000000 nt!KiTrap0E+0xdc aec0fbf0 808d93b5 8ac12dc0 00000000 00000000 nt!ExFreePoolWithTag+0×6a3 aec0fc08 808cd304 e5ae5770 8ac12dc0 8aa77db0 nt!CmpFreePostBlock+0×4d aec0fc3c 8082ea53 8ac12dc0 aec0fc88 aec0fc7c nt!CmpPostApc+0xde aec0fc8c 80833eec 00000000 00000000 00000000 nt!KiDeliverApc+0xf9 aec0fcc4 808290bd aec0fd64 8099781c 0160fd44 nt!KiSwapThread+0×300 aec0fd0c 809978a0 00000001 00000000 f77275e0 nt!KeDelayExecutionThread+0×2ab aec0fd54 8088b45c 00000000 0160fd74 0160fd9c nt!NtDelayExecution+0×84 aec0fd54 7c82847c 00000000 0160fd74 0160fd9c nt!KiFastCallEntry+0xfc WARNING: Frame IP not in any known module. Following frames may be wrong. 0160fd9c 00000000 00000000 00000000 00000000 0×7c82847c

1: kd> !pool 8ac12dc0 Pool page 8ac12dc0 region is Nonpaged pool 8ac12000 size: 858 previous size: 0 (Allocated) TWPG 8ac12858 size: 8 previous size: 858 (Free) …. 8ac12860 size: 20 previous size: 8 (Allocated) VadS 8ac12880 size: 8 previous size: 20 (Free) NtFs 8ac12888 size: 20 previous size: 8 (Allocated) VadS 8ac128a8 size: 28 previous size: 20 (Allocated) Ntfn 8ac128d0 size: 30 previous size: 28 (Allocated) Vad 8ac12900 size: 40 previous size: 30 (Allocated) Muta (Protected) 8ac12940 size: 38 previous size: 40 (Allocated) Sema (Protected) 8ac12978 size: 40 previous size: 38 (Allocated) Muta (Protected) 8ac129b8 size: 270 previous size: 40 (Allocated) Thre (Protected) 8ac12c28 size: 40 previous size: 270 (Allocated) Ntfr 8ac12c68 size: d0 previous size: 40 (Allocated) DRIV 8ac12d38 is not a valid large pool allocation, checking large session pool… 8ac12d38 is freed (or corrupt) pool Bad previous allocation size @8ac12d38, last size was 1a

*** *** An error (or corruption) in the pool was detected; *** Attempting to diagnose the problem. *** *** Use !poolval 8ac12000 for more details. ***

Pool page [ 8ac12000 ] is __inVALID.

Analyzing linked list... [ 8ac12c68 --> 8ac12db8 (size = 0x150 bytes)]: Corrupt region Scanning for single bit errors...

None found

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Citrix and Microsoft Customer Forum

Museum of Debugging and Memory Dumps

7/7/2011 - 8/8/2011 Annual Competition: Tell Your Windows Debugging Story

Crash and Hang Analysis Audit Service

CARE: Crash Analysis Report Environment

Crash Dump and Software Trace Analysis Training and Seminars

Access OpenTask Titles on Safari Books Online

DATA (Dump Analysis + Trace Analysis) Facebook group
Please join the community of memory (dump) and trace analysis engineers. This group promotes scientific methods and memory dump-based worldview.

Twitter @ DumpAnalysis
You can now follow portal and blog news at DumpAnalysis on Twitter

LinkedIn Group Dr. Watson Enthusiasts
All about Dr. Watson errors and more. Get news, excerpts and progress reports about the forthcoming book The Science of Dr. Watson: An Illustrated History of Debugging (ISBN 978-1906717070)

2010 (0x7DA) - The Year of Dump Analysis
2011 (0x7DB) - 2020 (0x7E4) The Debugging Decade

International Memory Analysts and Debuggers Day:
07.07 and/or 08.08 starting from The Year of Dump Analysis, 2010, 7DA

Announcements

Coming Soon:

Resume and CV: As a Book