Software Diagnostics Library

OpenTask to offer first 3 volumes of Memory Dump Analysis Anthology in one set:

The set is available exclusively from OpenTask e-Commerce web site starting from June. Individual volumes are also available from Amazon, Barnes & Noble and other bookstores worldwide.

Product information:

• Title: Modern Memory Dump and Software Trace Analysis: Volumes 1-3
• Author: Dmitry Vostokov
• Language: English
• Product Dimensions: 22.86 x 15.24
• Paperback: 1600 pages
• Publisher: Opentask (31 May 2010)
• ISBN-13: 978-1-906717-99-5

Information about individual volumes:

• Volume 1
• Volume 2
• Volume 3

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Plan to start providing training and seminars in my free time. If you are interested please answer these questions (you can either respond here in comments or use this form for private communication http://www.dumpanalysis.org/contact):

• Are you interested in on-site training, prefer traveling or attending webinars?
• Are you interested in software trace analysis as well?
• What specific topics are you interested in?
• What training level (beginner, intermediate, advanced) are you interested in? (please provide an example, if possible)

Additional topics of expertise that can be integrated into training include Source Code Reading and Analysis, Debugging, Windows Architecture, Device Drivers, Troubleshooting Tools Design and Implementation, Multithreading, Deep Down C and C++, x86 and x64 Assembly Language Reading.

Looking forward to your responses. Any suggestions are welcome.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

A friend of mine couldn’t start Windows XP on his notebook. As soon as he entered his credentials in a logon window the system experienced a BSOD event. He booted from another media and collected mini-dumps. All of them were consistent in resisting to my attempts to load symbols and modules. Even explicit downloading the symbol package from Microsoft didn’t help. All bugcheck info and stack traces were like this pointing to pool corruption:

0: kd> !analyze -v
[...]
BAD_POOL_CALLER (c2)
The current thread is making a bad pool request. Typically this is at a bad IRQL level or double freeing the same allocation, etc.
Arguments:
Arg1: 00000043, Attempt to free a virtual address which was never in any pool
Arg2: c9c00000, Address being freed.
Arg3: 00000000, 0
Arg4: 00000000, 0
[...]

1: kd> kv 100
ChildEBP RetAddr  Args to Child             
WARNING: Stack unwind information not available. Following frames may be wrong.
f6cc09e4 80548c2d 000000c2 00000043 c9c00000 nt+0x22f43
f6cc0a24 8054b49a c9c00000 e2039410 e23fd000 nt+0x71c2d
f6cc0a64 8063bf19 c9c00000 00000000 f6cc0ad0 nt+0x7449a
f6cc0a74 8063eb20 c9c00000 00002000 00000000 nt+0x164f19
f6cc0ad0 8063ef05 e1f6e008 00000000 00000000 nt+0x167b20
f6cc0b1c 8063087e e1f6e008 00000000 00000001 nt+0x167f05
f6cc0b34 806383a9 e1f6e101 00000005 00000000 nt+0x15987e
f6cc0ba0 80625bf9 f6cc0bdc 00000005 00000000 nt+0x1613a9
f6cc0bf8 8062ad8b f6cc0d04 00000000 f6cc0c64 nt+0x14ebf9
f6cc0c20 80631f24 f6cc0ccc f6cc0c6c f6cc0c5c nt+0x153d8b
f6cc0cac 806257b4 f6cc0ce4 f6cc0ccc 00000000 nt+0x15af24
f6cc0d40 806259be 0006dcc4 0006dcac 00000000 nt+0x14e7b4
f6cc0d54 8054162c 0006dcc4 0006dcac 0006dcf0 nt+0x14e9be
f6cc0d64 7c91e514 badb0d00 0006dc98 00000000 nt+0x6a62c
f6cc0d68 badb0d00 0006dc98 00000000 00000000 0x7c91e514
f6cc0d6c 0006dc98 00000000 00000000 00000090 0xbadb0d00
f6cc0d70 00000000 00000000 00000090 000000a4 0x6dc98

Portions of raw stack data available in minidump didn’t have any traces of other modules and drivers except nt:

1: kd> !thread
GetPointerFromAddress: unable to read from 80562134
[...]
86485da8: Unable to get thread contents

1: kd> dps f6cc09cc-3000 f6cc09cc+3000
[...]
f6cc095c  ????????
f6cc0960  ????????
f6cc0964  ????????
f6cc0968  00000000
f6cc096c  00000000
f6cc0970  003d0058
f6cc0974  f6cc09a8
f6cc0978  00000000
f6cc097c  0000c000
f6cc0980  00000000
f6cc0984  00000000
f6cc0988  8648b4d8
f6cc098c  863eb240
f6cc0990  00000000
f6cc0994  01ffffff
f6cc0998  f6cc093c
f6cc099c  00000000
f6cc09a0  f6cc0a14
f6cc09a4  80539ac0 nt+0x62ac0
f6cc09a8  804d8228 nt+0x1228
f6cc09ac  ffffffff
f6cc09b0  00000002
f6cc09b4  80506653 nt+0x2f653
f6cc09b8  f78a9548
f6cc09bc  c9c00000
f6cc09c0  0000bb40
[...]
f6cc0fcc  00000000
f6cc0fd0  00000000
f6cc0fd4  00000000
f6cc0fd8  00000000
f6cc0fdc  00000000
f6cc0fe0  7c91d5aa
f6cc0fe4  7c940574
f6cc0fe8  0015fd80
f6cc0fec  00100020
f6cc0ff0  00000000
f6cc0ff4  00000000
f6cc0ff8  00000000
f6cc0ffc  00000000
f6cc1000  ????????
f6cc1004  ????????
f6cc1008  ????????
[...]

So I advised to give me a kernel dump and fortunately there was one available too. It was more amenable for analysis and showed the involvement of registry:

0: kd> kv 100
ChildEBP RetAddr  Args to Child             
f690a9e4 80548c2d 000000c2 00000043 dcf40000 nt!KeBugCheckEx+0x1b
f690aa24 8054b49a dcf40000 e1294410 e17c6000 nt!MiFreePoolPages+0x8b
f690aa64 8063bf19 dcf40000 00000000 f690aad0 nt!ExFreePoolWithTag+0x1ba
f690aa74 8063eb20 dcf40000 00002000 00000000 nt!CmpFree+0×17
f690aad0 8063ef05 e11c4b60 00000000 00000000 nt!HvpRecoverData+0×3ec
f690ab1c 8063087e e11c4b60 00000000 00000001 nt!HvMapHive+0×133
f690ab34 806383a9 e11c4c01 00000005 00000000 nt!HvInitializeHive+0×416
f690aba0 80625bf9 f690abdc 00000005 00000000 nt!CmpInitializeHive+0×26d
f690abf8 8062ad8b f690ad04 00000000 f690ac64 nt!CmpInitHiveFromFile+0xa3
f690ac20 80631f24 f690accc f690ac6c f690ac5c nt!CmpCmdHiveOpen+0×21
f690acac 806257b4 f690ace4 f690accc 00000000 nt!CmLoadKey+0×90
f690ad40 806259be 0006dcc4 0006dcac 00000000 nt!NtLoadKey2+0×1fc
f690ad54 8054162c 0006dcc4 0006dcac 0006dcf0 nt!NtLoadKey+0×12
f690ad54 7c91e514 0006dcc4 0006dcac 0006dcf0 nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
0006dcf0 00000000 00000000 00000000 00000000 0×7c91e514

Examination of parameters on raw stack pointed to a user hive for MyFriend user:

0: kd> dpu f690ace4
f690ace4  00000018
f690ace8  80000ce0
f690acec  f690ad0c “Z\..(”
f690acf0  00000240
f690acf4  00000000
f690acf8  00000000
f690acfc  00660066
f690ad00  00eddea0
f690ad04  00660066
f690ad08  e10b1e60 “\??\C:\Documents and Settings\MyFriend\ntuser.dat”
f690ad14  00000028
[…]

So the solution was to log as Administrator and recreate the user.

- Dmitry Vostokov @ DumpAnalysis.org -

Microsoft to add 5th memory dump type to the final version of Windows 7. In addition to kernel, complete, mini and user dump file types new memory dumps will include all open files to allow full data recovery and postmortem process resurrection on another computer. The new coming soon version of WinDbg includes specialized extensions for process instantiation and recursive data recovery near the point of failure:

blogs.technet.com/5thcolumn

- Dmitry Vostokov @ DumpAnalysis.org -

As one of the new initiatives for the Year of Debugging  DumpAnalysis Portal will publish bimonthly full color 16 page publication called:

Debugged! MZ/PE: MagaZine for/from Practicing Engineers
The only serial publication dedicated entirely to Windows® debugging

The first issue is planned for March, 2009 and will have ISBN-13: 978-1-906717-38-4. If it goes well I’m planning to have ISSN number assigned to it too. More details will be announced soon.

- Dmitry Vostokov @ DumpAnalysis.org - 

Matthieu Suiche has released the new version of win32dd tool with the ability to save physical memory in a WinDbg-compliant memory dump file including pages that normally are not saved in a complete memory dump.

- Dmitry Vostokov @ DumpAnalysis.org -

Paperback edition of Memory Dump Analysis Anthology, Volume 2 is finally available on Amazon and Barnes & Noble. Search Inside is also available on Amazon. In addition, I updated the list of recommended books:

Listmania! Crash Dump Analysis and Debugging

Hardcover edition will be available on Amazon and B&N in 2-3 weeks.

- Dmitry Vostokov @ DumpAnalysis.org -

“Everything is memory dump.”

I’m very excited to announce that Volume 2 is available in paperback, hardcover and digital editions:

Memory Dump Analysis Anthology, Volume 2

In one or two weeks paperback edition should also appear on Amazon and other bookstores. Amazon hardcover edition is planned to be available by the end of October.

I’m often asked when Volume 3 is available and I currently plan to release it in October - November, 2009. In the mean time I’m planning to concentrate on other publishing projects. 

- Dmitry Vostokov @ DumpAnalysis.org -

The book is nearly finished and here is the final TOC:

Memory Dump Analysis Anthology, Volume 2: Table of Contents

- Dmitry Vostokov @ DumpAnalysis.org -

My friend was typing a long message in IE to one of his old schoolmates that he had just found on Internet. He spent about an hour writing and rewriting and when finally hit the Send button he got a page saying that connection was probably lost. Going back in URL history brought the empty edit box and all data was lost. Or was it? He called me and I immediately advised him to save a crash dump of iexplore.exe using Task Manager (Vista). I also asked him for a word he used to start his message. It was “Hello” in Russian. I got his dump file and opened it in WinDbg. Because the language of his message was Russian I assumed that it was still there in local buffers or heap entries in UNICODE format so I typed “ello” in Notepad and saved this in a Unicode text file. Loading it in a binary editor (I used Visual C++) showed the following sequence of bytes:

40 04 38 04 32 04 35 04 42 04

Then I did a search in WinDbg for this sequence from the first loaded module address till the end of user space:

0:000> lm
start    end        module name
003c0000 0045b000   iexplore
[...]

0:000> s 003c0000 L?7FFFFFFF 40 04 38 04 32 04 35 04 42 04
[...]
048971e4 40 04 38 04 32 04 35 04-42 04 2c 00 20 00 1c 04  @.8.2.5.B.,. ...
[...]
08530fe4 40 04 38 04 32 04 35 04-42 04 2c 00 20 00 1c 04 @.8.2.5.B.,. ...
[...]
201ea65c 40 04 38 04 32 04 35 04-42 04 2c 00 20 00 1c 04 @.8.2.5.B.,. ...
[...] 

The number of found entries was big and I decided to output every entry into a file using the following script: 

.foreach ( address { s-[1]b 003c0000 L?7FFFFFFF 40 04 38 04 32 04 35 04 42 04 }) {.writemem c:\dmitry\ieout${address}.txt ${address}-10 ${address}+1000}

I got numerous files:

C:\dmitry>dir ieout*.txt
[...]
09/06/2008  08:53               4112 ieout0x048971e4.txt
09/06/2008  08:53               4112 ieout0x0489784c.txt
09/06/2008  08:53               4112 ieout0x0489b854.txt
09/06/2008  08:53               4112 ieout0x0489bc5c.txt
[...]

I combined all of them into one big file and sent it to my friend:

C:\dmitry>type ieout0x*.txt >ieoutall.txt

The file contained not only the final message but all intermediate typing histories too. He was very happy.

- Dmitry Vostokov @ DumpAnalysis.org -