Crash Dump Analysis Patterns (Part 23b)
Saturday, August 25th, 2007In contrast to Double Free pattern in a user mode process heap double free in a kernel mode pool results in immediate bugcheck in order to identify the driver causing the problem (BAD_POOL_CALLER bugcheck with Arg1 == 7):
2: kd> !analyze -v
...
...
...
BAD_POOL_CALLER (c2)
The current thread is making a bad pool request. Typically this is at a bad IRQL level or double freeing the same allocation, etc.
Arguments:
Arg1: 00000007, Attempt to free pool which was already freed
Arg2: 0000121a, (reserved)
Arg3: 02140001, Memory contents of the pool block
Arg4: 89ba74f0, Address of the block of pool being deallocated
If we look at the block being deallocated we would see that it was marked as “Free” block:
2: kd> !pool 89ba74f0
Pool page 89ba74f0 region is Nonpaged pool
89ba7000 size: 270 previous size: 0 (Allocated) Thre (Protected)
89ba7270 size: 8 previous size: 270 (Free) ....
89ba7278 size: 18 previous size: 8 (Allocated) ReEv
89ba7290 size: 80 previous size: 18 (Allocated) Mdl
89ba7310 size: 80 previous size: 80 (Allocated) Mdl
89ba7390 size: 30 previous size: 80 (Allocated) Vad
89ba73c0 size: 98 previous size: 30 (Allocated) File (Protected)
89ba7458 size: 8 previous size: 98 (Free) Wait
89ba7460 size: 28 previous size: 8 (Allocated) FSfm
89ba74a0 size: 40 previous size: 18 (Allocated) Ntfr
89ba74e0 size: 8 previous size: 40 (Free) File
*89ba74e8 size: a0 previous size: 8 (Free ) *ABCD
Owning component : Unknown (update pooltag.txt)
89ba7588 size: 38 previous size: a0 (Allocated) Sema (Protected)
89ba75c0 size: 38 previous size: 38 (Allocated) Sema (Protected)
89ba75f8 size: 10 previous size: 38 (Free) Nbtl
89ba7608 size: 98 previous size: 10 (Allocated) File (Protected)
89ba76a0 size: 28 previous size: 98 (Allocated) Ntfn
89ba76c8 size: 40 previous size: 28 (Allocated) Ntfr
89ba7708 size: 28 previous size: 40 (Allocated) NtFs
89ba7730 size: 40 previous size: 28 (Allocated) Ntfr
89ba7770 size: 40 previous size: 40 (Allocated) Ntfr
89ba7a10 size: 270 previous size: 260 (Allocated) Thre (Protected)
89ba7c80 size: 20 previous size: 270 (Allocated) VadS
89ba7ca0 size: 40 previous size: 20 (Allocated) Ntfr
89ba7ce0 size: b0 previous size: 40 (Allocated) Ussy
89ba7d90 size: 270 previous size: b0 (Free) Thre
The pool tag is a 4 byte character sequence used to associate drivers with pool blocks and is useful to identify a driver allocated or freed a block. In our case the pool tag is ABCD and it is associated with the driver which previously freed the block. All known pool tags corresponding to kernel components can be found in pooltag.txt located in triage subfolder where you installed WinDbg. However our ABCD tag is not listed there. We can try to find the driver corresponding to ABCD tag using findstr CMD command:
C:\Windows\System32\drivers>findstr /m /l hABCD *.sys
The results of the search will help us to identify the driver which freed the block first. The driver which double freed the same block can be found from the call stack and it might be the same driver or a different driver:
2: kd> k
ChildEBP RetAddr
f78be910 8089c8f4 nt!KeBugCheckEx+0x1b
f78be978 8089c622 nt!ExFreePoolWithTag+0x477
f78be988 f503968b nt!ExFreePool+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
f78be990 f5024a6e driver+0×1768b
f78be9a0 f50249e7 driver+0×2a6e
f78be9a4 84b430e0 driver+0×29e7
Because we don’t have symbol files for driver.sys WinDbg warns us that it was unable to identify the correct stack trace and driver.sys might not have called ExFreePool or ExFreePoolWithTag. To verify that driver.sys called ExFreePool indeed we disassemble backwards the return address of ExFreePool:
2: kd> ub f503968b
driver+0×1767b:
f503967b 90 nop
f503967c 90 nop
f503967d 90 nop
f503967e 90 nop
f503967f 90 nop
f5039680 8b442404 mov eax,dword ptr [esp+4]
f5039684 50 push eax
f5039685 ff15202302f5 call dword ptr [driver+0×320 (f5022320)]
Finally we can get some info from the driver:
2: kd> lmv m driver
start end module name
f5022000 f503e400 driver (no symbols)
Loaded symbol image file: driver.sys
Image path: \SystemRoot\System32\drivers\driver.sys
Image name: driver.sys
Timestamp: Tue Aug 12 11:32:16 2007
If the company name developed the driver is absent we can try techniques outlined in Unknown Component pattern.
If we have symbols it is very easy to identify the code as can be seen from this 64-bit dump:
BAD_POOL_CALLER (c2)
The current thread is making a bad pool request. Typically this is at a bad IRQL level or double freeing the same allocation, etc.
Arguments:
Arg1: 0000000000000007, Attempt to free pool which was already freed
Arg2: 000000000000121a, (reserved)
Arg3: 0000000000000080, Memory contents of the pool block
Arg4: fffffade6d54e270, Address of the block of pool being deallocated
0: kd> kL
fffffade`45517b08 fffff800`011ad905 nt!KeBugCheckEx
fffffade`45517b10 fffffade`5f5991ac nt!ExFreePoolWithTag+0x401
fffffade`45517bd0 fffffade`5f59a0b0 driver64!ProcessDataItem+0×198
fffffade`45517c70 fffffade`5f5885a6 driver64!OnDataArrival+0×2b4
fffffade`45517cd0 fffff800`01299cae driver64!ReaderThread+0×15a
fffffade`45517d70 fffff800`0102bbe6 nt!PspSystemThreadStartup+0×3e
fffffade`45517dd0 00000000`00000000 nt!KiStartSystemThread+0×16
- Dmitry Vostokov @ DumpAnalysis.org -