Crash Dump Analysis Patterns (Part 104)
Sometimes we get memory dumps that are difficult to analyze in full because some if not most of information was omitted during saving. These are usually small memory dumps (contrasted with kernel and complete) and user process minidumps. We can easily recognize that when we open a dump:
User Mini Dump File: Only registers, stack and portions of memory are available
or
Mini Kernel Dump File: Only registers and stack trace are available
The same also applies to user dumps where thread times information is omitted so it is not possible to use !runaway WinDbg command or to a dump saved with various options of .dump command (including privacy-aware) instead of /ma or deprecated /f option. On the contrary, manually erased data in crash dumps looks more like an example of another pattern called Lateral Damage.
The similar cases of abridged dumps are discussed in Wrong Dump and Missing Space antipatterns.
Anyway, we shouldn’t dismiss such dumps and should try to analyze them. For example, some approaches (including using image binaries) are listed in kernel minidump analysis series. We can even see portions of raw stack data in search of execution residue:
0: kd> !thread
GetPointerFromAddress: unable to read from 81d315b0
THREAD 82f49020 Cid 0004.0034 Teb: 00000000 Win32Thread: 00000000 RUNNING on processor 0
IRP List:
Unable to read nt!_IRP @ 8391e008
Not impersonating
GetUlongFromAddress: unable to read from 81d0ad90
Owning Process 82f00ab0 Image: System
Attached Process N/A Image: N/A
ffdf0000: Unable to get shared data
Wait Start TickCount 4000214
Context Switch Count 21886
ReadMemory error: Cannot get nt!KeMaximumIncrement value.
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address nt!ExpWorkerThread (0x81c78ea3)
Stack Init 85be0000 Current 85bdf7c0 Base 85be0000 Limit 85bdd000 Call 0
Priority 14 BasePriority 12 PriorityDecrement 0 IoPriority 2 PagePriority 5
[…]
0: kd> dps 85bdd000 85be0000
85bdd000 ????????
85bdd004 ????????
85bdd008 ????????
85bdd00c ????????
85bdd010 ????????
85bdd014 ????????
85bdd018 ????????
85bdd01c ????????
85bdd020 ????????
85bdd024 ????????
85bdd028 ????????
[...]
85bdf8c4 ????????
85bdf8c8 ????????
85bdf8cc ????????
85bdf8d0 0000000a
85bdf8d4 a112883e
85bdf8d8 0000001b
85bdf8dc 00000000
85bdf8e0 81c28750 nt!KeSetEvent+0x4d
85bdf8e4 85bdf8e8
85bdf8e8 85bdf970
85bdf8ec 81c28750 nt!KeSetEvent+0x4d
85bdf8f0 badb0d00
85bdf8f4 00000000
85bdf8f8 00000000
85bdf8fc 81cf4820 nt!KiInitialPCR+0x120
85bdf900 00000000
85bdf904 85bdf938
85bdf908 81cf4820 nt!KiInitialPCR+0x120
85bdf90c 00000000
85bdf910 81d32300 nt!IopTimerLock
85bdf914 00000000
85bdf918 81fa0000 nt!_NULL_IMPORT_DESCRIPTOR <PERF> (nt+0x3a0000)
85bdf91c 85bd0023
85bdf920 00000023
85bdf924 00000000
85bdf928 81d323c0 nt!KiDispatcherLock
85bdf92c a1128828
85bdf930 85bdf9b4
85bdf934 85bdfdb0
85bdf938 00000030
85bdf93c 84ca6f40
85bdf940 84ca6f38
85bdf944 00000001
85bdf948 85bdf970
85bdf94c 00000000
85bdf950 81c28750 nt!KeSetEvent+0x4d
85bdf954 00000008
85bdf958 00010246
85bdf95c 00000000
85bdf960 84ca68a0
[...]
85bdfd2c 82f49020
85bdfd30 835ca4d0
85bdfd34 a6684538
85bdfd38 81cfde7c nt!ExWorkerQueue+0x3c
85bdfd3c 00000001
85bdfd40 00000000
85bdfd44 85bdfd7c
85bdfd48 81c78fa0 nt!ExpWorkerThread+0xfd
85bdfd4c 835ca4d0
85bdfd50 00000000
85bdfd54 82f49020
85bdfd58 00000000
85bdfd5c 00000000
85bdfd60 0069000b
85bdfd64 00000000
85bdfd68 00000001
85bdfd6c 00000000
85bdfd70 835ca4d0
85bdfd74 81da9542 nt!PnpDeviceEventWorker
85bdfd78 00000000
85bdfd7c 85bdfdc0
85bdfd80 81e254e0 nt!PspSystemThreadStartup+0x9d
85bdfd84 835ca4d0
85bdfd88 85bd4680
85bdfd8c 00000000
85bdfd90 00000000
85bdfd94 00000000
85bdfd98 00000002
85bdfd9c 00000000
85bdfda0 00000000
85bdfda4 00000001
85bdfda8 85bdfd88
85bdfdac 85bdfdbc
85bdfdb0 ffffffff
85bdfdb4 81c8aad5 nt!_except_handler4
85bdfdb8 81c9ddb8 nt!`string'+0x4
85bdfdbc 00000000
85bdfdc0 00000000
85bdfdc4 81c9159e nt!KiThreadStartup+0x16
85bdfdc8 81c78ea3 nt!ExpWorkerThread
85bdfdcc 00000001
85bdfdd0 00000000
85bdfdd4 00000000
85bdfdd8 002e0069
85bdfddc 006c0064
85bdfde0 004c006c
85bdfde4 00000000
85bdfde8 000007f0
85bdfdec 00010000
85bdfdf0 0000027f
85bdfdf4 00000000
85bdfdf8 00000000
85bdfdfc 00000000
85bdfe00 00000000
85bdfe04 00000000
85bdfe08 00001f80
85bdfe0c 0000ffff
85bdfe10 00000000
85bdfe14 00000000
85bdfe18 00000000
[...]
85bdffe4 00000000
85bdffe8 00000000
85bdffec 00000000
85bdfff0 00000000
85bdfff4 00000000
85bdfff8 00000000
85bdfffc 00000000
85be0000 ????????
User minidumps are similar here:
0:001> k
ChildEBP RetAddr
099bfe147c90daaa ntdll!KiFastSystemCallRet
099bfe18 77e765e3 ntdll!NtReplyWaitReceivePortEx+0xc
099bff80 77e76caf rpcrt4!LRPC_ADDRESS::ReceiveLotsaCalls+0×12a
099bff88 77e76ad1 rpcrt4!RecvLotsaCallsWrapper+0xd
099bffa8 77e76c97 rpcrt4!BaseCachedThreadRoutine+0×79
099bffb4 7c80b729 rpcrt4!ThreadStartRoutine+0×1a
099bffec 00000000 kernel32!BaseThreadStart+0×37
0:001> dd 099bfe14
099bfe14 099bfe24 7c90daaa 77e765e3 00000224
099bfe24 099bff74 00000000 2db87ae8 099bff48
099bfe34 fbf58e18 00000040 fd629338 b279dbbc
099bfe44 fd5928b8 fbf58ebc b279dbbc e0c1e002
099bfe54 00000000 00000006 00000001 00000000
099bfe64 e637d218 00000000 00000006 00000006
099bfe74 00000006 e1f79698 e39b8b60 00000000
099bfe84 fbe33c40 00000001 e5ce12f8 b279db9c
0:001> dd 099bfe14-20
099bfdf4 ???????? ???????? ???????? ????????
099bfe04 ???????? ???????? ???????? ????????
099bfe14 099bfe24 7c90daaa 77e765e3 00000224
099bfe24 099bff74 00000000 2db87ae8 099bff48
099bfe34 fbf58e18 00000040 fd629338 b279dbbc
099bfe44 fd5928b8 fbf58ebc b279dbbc e0c1e002
099bfe54 00000000 00000006 00000001 00000000
099bfe64 e637d218 00000000 00000006 00000006
As a warning here it is possible to conclude that minidumps can also reveal private information especially when ASCII or Unicode buffers are seen on raw stack data.
I was thinking how to name this pattern and Oxford Thesaurus of English suggested the following name: Abridged Dump by analogy with an abridged book.
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
September 24th, 2010 at 10:59 am
[…] Abridged Dump […]