Software Diagnostics Library

New word - new nickname…

Mr. Heapocrat is a member of a powerful group called heap class and a pseudonym for a historian and journalist that Debugged! MZ/PE magazine editorial board has invited to write a history and current affairs column called “Heap Inquiries”.

- Dmitry Vostokov @ DumpAnalysis.org -

DumpAnalysis.org accepts hardware such as laptops for reviewing in relation to their suitability for extreme debugging, computer forensics, crash dump analysis and memory visualization. If you work for a H/W company like HP, Apple, Dell, Acer, Sony or any other respectable manufacturer please don’t hesitate to forward this post to your management: it could be your company brand or laptop model that debugging and software technical support community chooses next time of upgrade or for T&D / R&D! H/W reviews will be posted on the main portal page which currently has an audience of more than a hundred thousand unique visitors per year from more than 20,000 network locations (*).

If your company is interested please don’t hesitate to use this contact form:

http://www.dumpanalysis.org/contact

(*) From Google Analytics report.

- Dmitry Vostokov @ DumpAnalysis.org -

I’ve updated timeline widget with references to relevant blog posts and also added events that I forgot to add previously and ones that happened since my celebration of 5 years of memory dump analysis in October:

Memory Dump Analysis Portal Timeline

- Dmitry Vostokov @ DumpAnalysis.org -

Last weekend I spent a few hours devising a cover for the forthcoming computer memory visualization book and finally created this one piece cover featuring a journey to the center of pagefile theme and the discovery of cosmic rays in memory:

Coincidentally the whole 100 x 18400 centered slice of pagefile.sys image fit on the cover and nothing was left!

- Dmitry Vostokov @ DumpAnalysis.org -

Finally designed a conceptual cover for DebugWare book using command-line theme:

- Dmitry Vostokov @ DumpAnalysis.org -

Debugged! MZ/PE magazine editorial board has secured a columnist Don T. Quit to write a column “Tips, Bits and Fields”. Don is very eager to offer a (socio- | psycho- | ε) logical advice to debugging community.

Just to remind that a deadline to submit articles for the first issue is set to 15th of February.

- Dmitry Vostokov @ DumpAnalysis.org - 

… taking into account one second because of slowing Earth rotation. I wish everyone a Happy New Year and successful debugging and memory analysis sessions in virtuality and reality!

Dmitry Vostokov @ 45474150 504d5544  PAGEDUMP....(...
0bc6c000 81000000 8054d030 8054f098  ........0.T...T.
0000014c 00000001 000000e2 00000000  L...............
00000000 00000000 00000000 45474100  .............AGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474100  PAGEPAGEPAGE.AGE
8053eee0 00000003 0000f67d 00000001  ..S.....}.......
0000009e 00000100 00000eff 00001000  ................
0000e6e0 45474150 45474150 45474150  ....PAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
45474150 45474150 45474150 45474150  PAGEPAGEPAGEPAGE
00000000 00000000 00000000 00000000  ................
00000000 00000000 00000000 00000000  ................
00000000 00000000 00000000 00000000  ................
00000000 00000000 00000000 00000000  ................
00000000 00000000 00000000 00000000  ................
00000000 00000000 00000000 00000000  ................
00000000 00000000 00000000 00000000  ................
00000000 00000000 00000000 00000000  ................
00000000 00000000 00000000 00000000  ................
[...]


To test various postmortem debuggers and WER to their fullest potential and especially Crash2Hang I wrote another small program that models multiple exceptions in several threads. It is free and can be downloaded with full PDB and source code from here:

Download MTCrash

The source code is simple as possible:

// MTCrash (Multithreaded crash)
// Copyright (c) 2009 Dmitry Vostokov
// GNU GENERAL PUBLIC LICENSE
// http://www.gnu.org/licenses/gpl-3.0.txt

#include <windows.h>
#include <process.h>
#include <iostream>

bool twice = false;

void thread_one(void *)
{
 Sleep(1000);
 std::cout << "Thread 1 is about to experience an AV exception..." << std::endl;
 *(int *)NULL = 0;
}

void thread_two(void *)
{
 Sleep(2000);
 if (twice)
 {
  std::cout << "Thread 2 is about to experience an AV exception..." << std::endl;
  *(int *)NULL = 0;
 }

 while (true)
 {
  std::cout << "Thread 2 is still running..." << std::endl;
  Sleep(1000);
 }
}

int main(int argc, WCHAR* argv[])
{
 if (argc > 1) twice = true;

 _beginthread(thread_two, 0, NULL);
 _beginthread(thread_one, 0, NULL);

 while(true)
 {
  std::cout << "Main Thread is still running..." << std::endl;
  Sleep(1000);
 }

 return 0;
}

It creates 2 additional threads and the first of them tries to access a NULL pointer:

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(d3c.e94): Access violation - code c0000005 (first/second chance not available)
eax=00000000 ebx=005a4660 ecx=0041948c edx=00419ef0 esi=00419488 edi=00000000
eip=004013bd esp=007eff7c ebp=007effb4 iopl=0 nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b  efl=00010246
MTCrash!thread_one+0×6d:
004013bd c7050000000000000000 mov dword ptr ds:[0],0  ds:002b:00000000=????????

0:002> ~*kL

   0  Id: d3c.eb4 Suspend: 1 Teb: 7efdd000 Unfrozen
ChildEBP RetAddr 
002dfee4 7d4d0ec5 ntdll!ZwDelayExecution+0x15
002dff4c 7d4d14ef kernel32!SleepEx+0x68
002dff5c 0040157a kernel32!Sleep+0xf
002dff70 004046ac MTCrash!main+0xaa
002dffc0 7d4e7d2a MTCrash!__tmainCRTStartup+0x15f
002dfff0 00000000 kernel32!BaseProcessStart+0x28

   1  Id: d3c.ebc Suspend: 1 Teb: 7efda000 Unfrozen
ChildEBP RetAddr  
006afee4 7d4d0ec5 ntdll!ZwDelayExecution+0x15
006aff4c 7d4d14ef kernel32!SleepEx+0x68
006aff5c 004014c5 kernel32!Sleep+0xf
006aff7c 00404352 MTCrash!thread_two+0xf5
006affb4 004043eb MTCrash!_callthreadstart+0x1b
006affb8 7d4dfe21 MTCrash!_threadstart+0x73
006affec 00000000 kernel32!BaseThreadStart+0x34

#  2  Id: d3c.e94 Suspend: 1 Teb: 7efd7000 Unfrozen
ChildEBP RetAddr 
007effb8 7d4dfe21 MTCrash!thread_one+0×6d
007effe4 00000000 kernel32!BaseThreadStart+0×34

   3  Id: d3c.f0c Suspend: 1 Teb: 7efaf000 Unfrozen
ChildEBP RetAddr 
0083ffc8 7d665081 ntdll!DbgBreakPoint+0x1
0083fff4 00000000 ntdll!DbgUiRemoteBreakin+0x2d

The second thread and main thread continue to run:

C:\Crash2Hang>MTCrash.exe
Main Thread is still running...
Thread 1 is about to experience an AV exception...
Main Thread is still running...
Thread 2 is still running...
Main Thread is still running...
Thread 2 is still running...
Main Thread is still running...
Thread 2 is still running...
[...]

If launched with any parameter the second thread also experiences an unhandled exception (in red) while the first one is suspended by an unhandled exception filter (in blue):

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(ca4.cb0): Access violation - code c0000005 (first/second chance not available)
eax=00000000 ebx=005a4668 ecx=0041948c edx=00419ef0 esi=00419488 edi=00000000
eip=004013bd esp=007eff7c ebp=007effb4 iopl=0 nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b  efl=00010246
MTCrash!thread_one+0×6d:
004013bd c7050000000000000000 mov dword ptr ds:[0],0  ds:002b:00000000=????????

0:002> ~*kL

   0  Id: ca4.ca0 Suspend: 1 Teb: 7efdd000 Unfrozen
ChildEBP RetAddr 
002dfee4 7d4d0ec5 ntdll!ZwDelayExecution+0x15
002dff4c 7d4d14ef kernel32!SleepEx+0x68
002dff5c 0040157a kernel32!Sleep+0xf
002dff70 004046ac MTCrash!main+0xaa
002dffc0 7d4e7d2a MTCrash!__tmainCRTStartup+0x15f
002dfff0 00000000 kernel32!BaseProcessStart+0x28

   1  Id: ca4.ca8 Suspend: 1 Teb: 7efda000 Unfrozen
ChildEBP RetAddr  
006af8cc 7d5357f3 ntdll!ZwRaiseHardError+0×12
006afb38 7d508f4e kernel32!UnhandledExceptionFilter+0×519
006afb40 7d4d8a25 kernel32!BaseThreadStart+0×4a (FPO: [SEH])
006afb68 7d61ec2a kernel32!_except_handler3+0×61
006afb8c 7d61ebfb ntdll!ExecuteHandler2+0×26
006afc34 7d61ea36 ntdll!ExecuteHandler+0×24
006afc34 0040144f ntdll!KiUserExceptionDispatcher+0xe (CONTEXT @ 006afc9c)
006aff7c 00404352 MTCrash!thread_two+0×7f
006affb4 004043eb MTCrash!_callthreadstart+0×1b
006affb8 7d4dfe21 MTCrash!_threadstart+0×73
006affec 00000000 kernel32!BaseThreadStart+0×34

#  2  Id: ca4.cb0 Suspend: 1 Teb: 7efd7000 Unfrozen
ChildEBP RetAddr 
007effb8 7d4dfe21 MTCrash!thread_one+0×6d
007effe4 00000000 kernel32!BaseThreadStart+0×34

0:002> .cxr 006afc9c
eax=00000000 ebx=005a4448 ecx=0041948c edx=00419ef0 esi=00419488 edi=00000000
eip=0040144f esp=006aff68 ebp=7d4d14e0 iopl=0  nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b  efl=00010246
MTCrash!thread_two+0×7f:
0040144f c7050000000000000000 mov dword ptr ds:[0],0  ds:002b:00000000=????????

However as soon as we dismiss the first error message box or if Auto is set to 1 in AeDebug registry key MTCrash terminates. If Crash2Hang is set as a default postmortem debugger then we get two instances of it running and MTCrash hangs even if we dismiss the first message. The main thread continues to run:

C:\Crash2Hang>MTCrash.exe 1
Main Thread is still running...
Thread 1 is about to experience an AV exception...
Main Thread is still running...
Main Thread is still running...
Thread 2 is about to experience an AV exception...
Main Thread is still running...
Main Thread is still running...
Main Thread is still running...
Main Thread is still running...
[...]

- Dmitry Vostokov @ DumpAnalysis.org -

Here is the front cover image for the first issue of previously announced Debugged! MZ/PE magazine:

- Dmitry Vostokov @ DumpAnalysis.org -

Last quarter was very busy for me and to keep up with schedule I now employ pipeline book writing techniques borrowed from CPU of my laptop to work simultaneously on 10 books. I also feel more relaxed with take it easy attitude towards writing and publishing: TIEP - TIE Publishing. In summary: the following book is planned to be released in Q1, 2009 where I’m an author:

• - Windows® Debugging: Practical Foundations (ISBN: 978-1906717100)

A magazine issue is planned for Q1 where I’m an editor:

• - March issue of Debugged! MZ/PE: MagaZine for/from Practicing Engineers (ISBN: 978-1906717384)

- Dmitry Vostokov @ DumpAnalysis.org -

Sometimes there is a need to preserve a crashing application or a service from termination and keep it in memory without showing any GUI dialogs or message boxes. Here Crash2Hang tool comes handy. It is free and can be downloaded from here:

Download Crash2Hang

The source code is simple as possible:

// Crash2Hang
// Copyright (c) 2009 Dmitry Vostokov
// GNU GENERAL PUBLIC LICENSE
// http://www.gnu.org/licenses/gpl-3.0.txt

#include <windows.h>

int main(int argc, WCHAR* argv[])
{
 if (argc > 1)
  MessageBox(NULL, L"One of processes has called a postmortem debugger!", L"Crash2Hang", MB_OK | MB_ICONSTOP | MB_SETFOREGROUND);
 else
  Sleep(INFINITE);
 return 0;
}

The tool can be used as a postmortem debugger specified in AeDebug registry key, for example, instead of CDB. Any argument specified to Crash2Hang.exe causes it to display a message box when launched

and exit process upon its dismissal. If several threads in a problem process experience an unhandled exception then Crash2Hang process is launched several times which may result in several such message boxes. Without arguments Crash2Hang process hangs infinitely causing the problem thread with an unhandled exception to hang indefinitely too (see my old post Who calls the postmortem debugger? for explanation).  

- Dmitry Vostokov @ DumpAnalysis.org -

It had always been my dream since I left Moscow State University to be associated with a research institute. Until yesterday it became a reality with the announcement of

Memory Analysis & Debugging Institute (MA_&D^I).

From: http://www.dumpanalysis.org/madinstitute-announcement

- Dmitry Vostokov @ DumpAnalysis.org -

OpenTask publisher plans to expand its offering of computer memory visualization titles:

http://www.opentask.com/memory-visualization-titles

More details will be announced soon.

- Dmitry Vostokov @ DumpAnalysis.org -

Seems every respectable publisher now comes with its own debugging book title. Now it is Pragmatic Bookshelf:

Debug It!: Find, Repair, and Prevent Bugs in Your Code

Curiously enough the title sounds similar to Debugged! magazine from Dump Analysis Portal… 

Debugged! MZ/PE: Magazine for/from Practicing Engineers

Despite this similarity both titles also have a pragmatic difference: Debug it! is an imperative but Debugged! is a statement of success

- Dmitry Vostokov @ DumpAnalysis.org -

The analogy between learning a complex tool with its own language and a foreign natural language has been developed further after the release of WinDbg Learning Cards and finally culminated in “WinDbg In Use” book series with the first book to be published during the 1st quarter of 2009:

• Title: WinDbg In Use: Debugging Exercises (Elementary and Intermediate Level)
• Author: Dmitry Vostokov
• Publisher: Opentask (15 March 2009)
• Language: English
• Product Dimensions: 23.5 x 19.1
• ISBN-13: 978-1-906717-50-6
• Paperback: 200 pages
• Book Annotation: Includes 60 programmed exercises from real life debugging and crash dump analysis scenarios and multiple-choice questions with full answers, comments and suggestions for further reading.

Some example exercises will be published on this blog from time to time. I also plan a corresponding column in the forthcoming Debugged! magazine. 

- Dmitry Vostokov @ DumpAnalysis.org -

DumpAnalysis.org wishes a Merry Christmas via a virtual postcard from Narasimha Vedala, the author of Dumps, Bugs and Debugging Forensics book!

http://www.dumpanalysis.org/Merry+Christmas+2008

- Dmitry Vostokov @ DumpAnalysis.org -

The following book is planned for publication during the 1st quarter of 2009:

Title: Reference Stack Traces: Windows Server® 2008 and Windows Vista™
ISBN-13: 978-1-906717-23-0

It features visual separation between kernel and user space in thread stack traces and useful footnotes for IRP and modules. Its publishing was delayed by a few months but fortunately my editing just got new breath by introducing thread stackprint images for kernel stacks (12Kb bitmaps):

Sample pages 13 and 96

Thread stackprints were generated from a complete memory dump using WinDbg scripts and Dump2Picture.

- Dmitry Vostokov @ DumpAnalysis.org -

Thanks to the wonderful real-time memory visualization package from Jamie Fenton developed initially as a FreeFrame plugin for FrameLab (a general FreeFrame host adaptor for DirectShow) and now with its own real-time memory viewer GUI front-end I was able to find the evidence for cosmic rays in computer memory! You can see them on this screenshot where the left panel is a condensed virtual memory map of IE process and the right panel is specific page(s) view (I found rays on pages starting from 0×3B4000 address):

- Dmitry Vostokov @ DumpAnalysis.org -

Previously announced Salary Figures book is on sale:

Salary Figures: A Codebook of Expectations

Front cover:

Back cover:

- Dmitry Vostokov @ DumpAnalysis.org -

- Dmitry Vostokov @ Memory Religion Portal -