Using scripts to process hundreds of user dumps
Suppose you have 100 - 200 user dumps from various user processes in the system and you want to quickly check their thread stacks, locks, etc. to see something suspicious related to your product or its environment your customers complaining about. It is much easier to collect such information into text files and browse them quickly than open every dump in WinDbg. I used shell script (VBScript) to automate loading dumps into WinDbg and used WinDbg scripts to run complex commands against loaded user dumps. For example, I used the following shell script:
'
' UDumps2Txt.vbs
'
Set fso = CreateObject("Scripting.FileSystemObject")
Set Folder = fso.GetFolder(".")
Set Files = Folder.Files
Set WshShell = CreateObject("WScript.Shell")
For Each File In Files
Set oExec = WshShell.Exec("C:\Program Files\Debugging Tools for Windows\WinDbg.exe -y ""srv*c:\mss*http://msdl.microsoft.com/download/symbols"" -z " + File.Name + " -c ""$$><c:\scripts\UDmp2Txt.txt;q"" -Q -QS -QY –QSY")
Do While oExec.Status = 0
WScript.Sleep 1000
Loop
Next
'
' UDumps2Txt.vbs: End of File
'
and the following WinDbg script:
$$
$$ UDmp2Txt: Dump information from user dump into log
$$
.logopen /d
!analyze -v
!locks
~*kv
lmv
.logclose
$$
$$ UDmp2Txt: End of File
$$
The following command launches multiple Dmp2Txt conversions:
C:\UserDumps>cscript /nologo c:\scripts\UDumps2Txt.vbs
You can also use CDB from Debugging Tools for Windows (console debugger) instead of WinDbg. I just use WinDbg uniformly instead of using separately CDB for user process dumps and KD for kernel and complete memory dumps.
Now when you have text files you can search for patterns using regular expressions. I will write more about applying them later. There is a very good book about them from practical point of view I read 6 years ago when I needed to understand them beyond wildcards and question marks. Since that time the book has undergone another two editions:
Mastering Regular Expressions, 3rd edition
Or you can process text files further and feed them into your database - part of automated crash dump analysis system.
- Dmitry Vostokov -
July 11th, 2009 at 8:29 am
this script is not working for me
C:\>UDumps2Txt.vbs
C:\UDumps2Txt.vbs(10, 3) WshShell.Exec: The system cannot find the file specifie
d.
July 11th, 2009 at 10:36 am
Be sure that WshShell.Exec(…) is one line in the script file if you copied it from here. Check if it ws not split into several line after copy-pase and if WinDbg is in the same directory as specified.
November 20th, 2010 at 12:15 am
[…] user process memory dumps taken from several production servers, say 20 files, we can either employ scripts to process all of them or compare their file size and look for a bigger ones for a starter, for example, 85 or 110 Mb. […]