Crash Dump Analysis Patterns (Part 83)

When constantly looking at Stack Trace Collections from complete or kernel memory dumps we notice that certain processes are always present and remember them. They are no longer suspicious. The same about thread stacks. Some are always present and some are not suspicious because of their function or status, like Passive Threads or Passive System Threads. Going more fine-grained we can talk about components and their specific functions. For example, certain kernel space components have special filter functions, they get an IRP and pass it down the device stack. It doesn’t take much code to check an IRP and forward it. This is usually reflected in small function offsets, for example:

ChildEBP RetAddr
aeced780 80833ec5 nt!KiSwapContext+0x26
aeced7ac 80829bc0 nt!KiSwapThread+0x2e5
aeced7f4 badffece nt!KeWaitForSingleObject+0x346
WARNING: Stack unwind information not available. Following frames may be wrong.
aeced824 bae00208 AVFilterB+0×1ece
aeced868 bae0e45a AVFilterB+0×2208
aeced8a0 8081e095 AVFilterB+0×1045a

aeced8b4 b946673b nt!IofCallDriver+0×45
aeced8c4 b94626ee driverB!FS_Dispatch+0xfb
aeced8d4 8081e095 driverB!dispatch+0×6e

aeced8e8 b96e04e1 nt!IofCallDriver+0×45
aeced90c b96e0755 driverA!PassThrough+0xd1
aeced92c 8081e095 driverA!Create+0×155
aeced940 b882df08 nt!IofCallDriver+0×45
aeceda5c 8081e095 AVFilterA!DispatchPassThrough+0×48
aeceda70 808fb13b nt!IofCallDriver+0×45
aecedb58 80939c6a nt!IopParseDevice+0xa35
aecedbd8 80935d9e nt!ObpLookupObjectName+0×5b0
aecedc2c 808ece57 nt!ObOpenObjectByName+0xea
aecedca8 808ee0f1 nt!IopCreateFile+0×447
aecedd04 808f1e31 nt!IoCreateFile+0xa3
aecedd44 8088ad3c nt!NtOpenFile+0×27

Here, if the thread is blocked, AVFilterB is more suspicious than AVFilterA because it is on top of the stack, waiting and AVFilterA just passed an IRP to driverA. DriverA seems also relayed the IRP to driverB and the latter relayed it to AVFilterB.

Another x64 example shows how these filter functions can be identified. They have “Dispatch” or “PassThrough” in their function names:

Child-SP          RetAddr           Call Site
fffffa60`12610880 fffff800`01875f8a nt!KiSwapContext+0x7f
fffffa60`126109c0 fffff800`0187776a nt!KiSwapThread+0x2fa
fffffa60`12610a30 fffff800`01ab16d6 nt!KeWaitForSingleObject+0x2da
fffffa60`12610fe0 fffffa60`06c5191a rdbss!RxFsdCommonDispatch+0×786
fffffa60`126110d0 fffffa60`07e4f21f rdbss!RxFsdDispatch+0×21a

fffffa60`12611140 fffffa60`011e05f5 mrxsmb!MRxSmbFsdDispatch+0xbf
fffffa60`12611180 fffffa60`011e0130 mup!MupiCallUncProvider+0×159
fffffa60`126111f0 fffffa60`011e17af mup!MupStateMachine+0×120
fffffa60`12611240 fffffa60`00d200b4 mup!MupCreate+0×2c3
fffffa60`126112c0 fffffa60`06d332d6 fltmgr!FltpCreate+0xa4
fffffa60`12611370 fffffa60`06d786c7 driverB!FS_Dispatch+0×156
fffffa60`126113a0 fffffa60`06d7894d driverA!PassThrough+0×177

fffffa60`12611400 fffffa60`090b3f30 driverA!Create+0×14d
fffffa60`12611430 fffff800`01aef360 AVDriverA!LowerDevicePassThrough+0×5c
fffffa60`12611700 fffff800`01aefa59 nt!IopParseDevice+0×5e3
fffffa60`126118a0 fffff800`01af3944 nt!ObpLookupObjectName+0×5eb
fffffa60`126119b0 fffff800`01affee0 nt!ObOpenObjectByName+0×2f4
fffffa60`12611a80 fffff800`01b00a0c nt!IopCreateFile+0×290
fffffa60`12611b20 fffff800`0186fdf3 nt!NtCreateFile+0×78

- Dmitry Vostokov @ -

5 Responses to “Crash Dump Analysis Patterns (Part 83)”

  1. Crash Dump Analysis » Blog Archive » Null data pointer, pass through functions and platformorphic fault: pattern cooperation Says:

    […] DriverA function on the stack trace looks like a passthrough, DriverA was removed from the system. However, the same pattern continued […]

  2. Crash Dump Analysis » Blog Archive » Hunting for a Driver Says:

    […] see that DriverA and DriverB are possibly pass-through and have little influence. For DriverC and DriverD we don’t even have symbol files but they […]

  3. Crash Dump Analysis » Blog Archive » Crash Dump Analysis Patterns (Part 89) Says:

    […] WinDbg to ignore our own functions and modules as well if we are sure they were well-tested or pass-through. For details please see the old minidump analysis case […]

  4. Crash Dump Analysis » Blog Archive » Stack trace collection, blocked threads, pass through functions and main thread: pattern cooperation Says:

    […] functions shown in blue are known from past issues to be pass through forwarding IRP to the lower drivers in a device driver […]

  5. Dmitry Vostokov Says:

    Sometimes we need to check return addresses to see whether a function introduced an alternative execution path such as with hooking:

    retA ntdll!NtCreateFile
    retB DLL!HookCreateFile
    retC kernel32!OtherFunction

    Here we can check what function was called from kernel32!OtherFunction:

    ub retB
    call [kernel32!_imp_NtCreateFile]

    If it is the same function semantically we can assume the hook is pass-through but it is not then we have a divergence of execution flow and we need to pay attention to that.

Leave a Reply