Resolving security issues with crash dumps

It is a well known fact that crash dumps may contain sensitive and private information. Crash reports that contain binary process extracts may contain it too. There is a conflict here between the desire to get full memory contents for debugging purposes and possible security implications. The solution would be to have postmortem debuggers and user mode process dumpers to implement an option to save only the activity data like stack traces in a text form. Some problems on a system level can be corrected just by looking at thread stack traces, critical section list, full module information, thread times and so on. This can help to identify components that cause process crashes, hangs or CPU spikes.

Users or system administrators can review text data before sending it outside their environment. This was already implemented as Dr. Watson logs. However these logs don’t usually have sufficient information required for crash dump analysis compared to information we can extract from a dump using WinDbg, for example. If you need to analyze kernel and all process activities you can use scripts to convert your kernel and complete memory dumps to text files:

Dmp2Txt: Solving Security Problem

The similar scripts can be applied to user dumps:

Using scripts to process hundreds of user dumps

Generating good scripts in a production environment has one problem: the conversion tool or debugger needs to know about symbols. This can be easily done with Microsoft modules because of Microsoft public symbol server.  Other companies like Citrix have the option to download public symbols:

Debug Symbols for Citrix Presentation Server

Alternatively one can write a WinDbg extension that loads a text file with stack traces, appropriate module images, finds the right PDB files and presents stack traces with full symbolic information. This can also be a separate program that uses Visual Studio DIA (Debug Interface Access) SDK to access PDB files later after receiving a text file from a customer.

I’m currently experimenting with some approaches and will write about them later. Text files will also be used in Internet Based Crash Dump Analysis Service because it is much easier to process text files than crash dumps. Although it is feasible to submit small mini dumps for this purpose they don’t contain much information and require writing specialized OS specific code to parse them. Also text files having the same file size can contain much more useful information without exposing private and sensitive information.

I would appreciate any comments and suggestions regarding this problem. 

- Dmitry Vostokov @ DumpAnalysis.org -

5 Responses to “Resolving security issues with crash dumps”

  1. Dmitry Vostokov Says:

    One of solutions is to use privacy command options in WinDbg or CDB:

    http://www.dumpanalysis.org/blog/index.php/2007/07/08/windbg-is-privacy-aware/

  2. Dmitry Vostokov Says:

    Examples of data extracted from complete memory dumps:

    http://www.dumpanalysis.org/blog/index.php/reference-stack-traces/

  3. Crash Dump Analysis » Blog Archive » Debugger Log Analyzer: Inception Says:

    […] Resolving security issues with crash dumps […]

  4. Crash Dump Analysis » Blog Archive » All at once: postmortem logs and dump files Says:

    […] the previous post Resolving security issues with crash dumps I mentioned the solution to use logs generated from memory dump files. In the case of process […]

  5. Dmitry Vostokov Says:

    Citrix now has its own symbol server:

    http://www.dumpanalysis.org/blog/index.php/2008/09/30/citrix-joins-symbol-server-club/

Leave a Reply

You must be logged in to post a comment.