Dmp2Txt: Solving Security Problem

This is a follow up to my previous Q&A about crash dumps and security issues like exposing confidential information stored in memory: Crash Dumps and Security. It seems a solution exists which allows to do some sort of crash dump analysis or at least identify problem components without sending complete or kernel memory dumps.

This solution takes advantage of WinDbg ability to execute scripts of arbitrary complexity. Couple of months ago I wrote about scripts and they really help me in pulling out various information from complete memory dumps:

WinDbg scripts
Yet another Windbg script
Critical sections

Now I created the bigger script that combines together all frequent commands used for identification of potential problems in memory dumps:

  • !analyze -v
  • !vm 4
  • lmv
  • !locks
  • !poolused 3
  • !poolused 4
  • !exqueue f
  • !irpfind
  • !stacks
  • List of all processes’ thread stacks, loaded modues and critical sections (for complete memory dump)

Other commands can be added if necessary.

How does all this work? A customer has to install Debugging Tools for Windows from Microsoft. This can be done on any workstation and not necessarily in a production environment. Then the customer has to run WinDbg.exe with some parameters including path(s) to symbols (-y), a path to memory dump (-z) and a path to script (-c):

C:\Program Files\Debugging Tools for Windows>WinDbg.exe -y "srv*c:\mss*http://msdl.microsoft.com/download/symbols" -z MEMORY.DMP -c "$$><c:\WinDbgScripts\Dmp2Txt.txt;q" -Q -QS -QY –QSY

Once WinDbg.exe finishes (it can run for couple of hours if you have many processes in your complete memory dump) you can copy the .log file created in “C:\Program Files\Debugging Tools for Windows” folder, archive it and send it to support for analysis. Kernel and process data and cached files are not exposed in the log! And because this is a text file the customer can inspect it before sending.

Here are the contents of Dmp2Txt.txt file:

$$
$$ Dmp2Txt: Dump all necessary information from complete full memory dump into log
$$
.logopen /d
!analyze -v
!vm 4
lmv
!locks
!poolused 3
!poolused 4
!exqueue f
!irpfind
!stacks
r $t0 = nt!PsActiveProcessHead
.for (r $t1 = poi(@$t0); (@$t1 != 0) & (@$t1 != @$t0); r $t1 = poi(@$t1))
{
    r? $t2 = #CONTAINING_RECORD(@$t1, nt!_EPROCESS, ActiveProcessLinks);
    .process @$t2
    .reload
    !process @$t2
    !ntsdexts.locks
    lmv
}
.logclose
$$
$$ Dmp2Txt: End of File
$$

For kernel dumps the script is simpler: 

$$
$$ KeDmp2Txt: Dump all necessary information from kernel dump into log
$$
.logopen /d
!analyze -v
!vm 4
lmv
!locks
!poolused 3
!poolused 4
!exqueue f
!irpfind
!stacks
!process 0 7
.logclose
$$
$$ KeDmp2Txt: End of File
$$

Note: if the dump is LiveKd.exe generated then due to inconsistency scripts may run forever 

- Dmitry Vostokov -

7 Responses to “Dmp2Txt: Solving Security Problem”

  1. Dmitry Vostokov Says:

    For XP/W2K3 and higher you can simplify the script at the cost of excluding process critical section locks:

    $$
    $$ Dmp2Txt: Dump all necessary information from complete full memory dump into log
    $$
    .logopen /d
    !analyze -v
    !vm 4
    lmv
    !locks
    !poolused 3
    !poolused 4
    !exqueue f
    !irpfind
    !stacks
    !process 0 ff
    .logclose
    $$
    $$ Dmp2Txt: End of File
    $$

  2. Dmitry Vostokov Says:

    Today I have found that !process 0 ff is less accurate in depicting user space stack traces in some complete memory dumps than the old combination of .reload/!process. To speed up reloading symbols I would recommend .reload /user

  3. Dreamyguy Says:

    I ran the following command:

    C:\Program Files\Debugging Tools for Windows>windbg.exe -y “srv*C:\mysymbols*http://msdl.microsoft.com/download/symbols” -z c:\kk\memory_dipesh.dmp -c “$$> “$$;q”?
    ^ Syntax error in ‘”$$;q”?’

    When I go back and look at the script file i.e. kedmp2.txt, it’s empty.

    Any suggestions?

  4. Dmitry Vostokov Says:

    This is because of quotes changed by Wordpress. In the original post they were “”? but should be "". I enclosed the command under code tag and quotes are shown correct now:

    C:\Program Files\Debugging Tools for Windows>WinDbg.exe -y "srv*c:\mss*http://msdl.microsoft.com/download/symbols" -z MEMORY.DMP -c "$$><c:\WinDbgScripts\Dmp2Txt.txt;q" -Q -QS -QY -QSY

  5. Dreamyguy Says:

    Yes, you were right! It worked this time. Thanks :-)

  6. Dmitry Vostokov Says:

    Instead of .for loop we can use the following command and simplify our script:

    !for_each_process ".process /r /p @#Process; !process @#Process; !ntsdexts.locks; lmv"

  7. Crash Dump Analysis » Blog Archive » Sparse complete x64 memory dumps Says:

    […] For really huge dumps WinDbg scripts collecting data on-site might be a solution too (see Dmp2Txt: Solving Security Problem for WinDbg script […]

Leave a Reply

You must be logged in to post a comment.