Software Diagnostics Library

After writing about large heap allocations I updated the sample chapter for Windows Debugging Notebook:

Draft Sample Chapter

- Dmitry Vostokov @ DumpAnalysis.org -

I discovered today that great IDA book was published last month:

The IDA Pro Book: The Unofficial Guide to the World’s Most Popular Disassembler

Curious about this published book I did some digging and finally ordered it because in the past I was always interested in knowing more about IDA after reading about it in some reverse engineering and disassembly books (more about them later) but was somewhat dissatisfied with IDA freeware version 4.3. The last argument in buying the book was my discovery that the new freeware version is available 4.9 which looks much better:

http://www.hex-rays.com/idapro/idadownfreeware.htm

I’ll keep investigating it for the purposes of memory dump analysis.

- Dmitry Vostokov @ DumpAnalysis.org -

Previously announced  Windows Debugging: Practical Foundations  book has got its front cover done in classic B/W style. A bit frightening, but shouldn’t stop if someone is determined to learn field debugging

Please let me know what do you think. Table of contents to be published next week. 

- Dmitry Vostokov @ DumpAnalysis.org -

A few months ago I wrote about my discovery of the first memory dump book. It actually arrived but only today I got a chance to take pictures of its front and back covers. The latter explans the title of the book (MEMORY DUMP) albeit in Spanish. 

Since many pages are in Basque I decided to learn a bit about this unique language and recommend this guide:

The Basque Language: A Practical Introduction (The Basque Series)

- Dmitry Vostokov @ DumpAnalysis.org -

I’m back from my holidays and here is my 10th anniversary bugtation:

“Coincidences, in general, are great stumbling-blocks in the way of” debugging.

Edgar Allan Poe, The Murders in the Rue Morgue

- Dmitry Vostokov @ DumpAnalysis.org -

There were no updates last two months primarily because I was still reading huge books 1,000 pages each. One of them was The Road to Reality that I started in March and finished only a few weeks ago. When I was about two-thirds through it and I thought that I could never finish it a sign happened to shed the light on the possible end. While commuting from work a shadow from the train window sign projected ”Exit” on the book and I asked  to take a picture of this event:

LiterateScientist Blog:

Under Pressure

Breaking the Spell

Darwin’s Dangerous Idea

The Skeptical Environmentalist

Ideas and Modern Mind 

- Dmitry Vostokov @ DumpAnalysis.org -

First, a definition for a new word that I coined today:  

Bugtation

noun
Date: 21st century
1. a modified quotation showing relation to debugging and troubleshooting

This is the first bugtation I would like to introduce and it is related to heisenbugs:

“There’s no such thing as” heisenbug;
“And what to us seems merest accident
Springs from the deepest source of” computation.

Friedrich Schiller, Early Dramas 

Deviations from original quotations are highlighted in blue. Welcome to the new literary art!

Note: if you notice any bugs in bugtations please let me know…

- Dmitry Vostokov @ DumpAnalysis.org -

Yesterday I sent to print the first draft version with finalized covers for editing in situ. I usually do editing on the real book. Then error corrections and layout improvements can be done in real WYSIWYG hardcopy book mode. What’s new in Volume 2:

- 45 new crash dump analysis patterns
- Pattern interaction and case studies
- Updated checklist
- Fully cross-referenced with Volume 1
- New appendixes

I aim to publish paperback and digital versions on the 3st of October and hardcover version on the 1st of November. Table of Contents will be announced in soon.

Here’s the book cover:

Back cover features visualized virtual process memory generated from a memory dump of colorimetric computer memory dating sample using Dump2Picture.

- Dmitry Vostokov @ DumpAnalysis.org -

New cartoon from Narasimha Vedala provides insight into the new ideology (click on it to enlarge):

Dawn of Debugism

Here I repeat 10 debugging commandments in scripture:

1. Thou shalt not underestimate bugs
2. Thou shalt walk the stack with thy colleagues
3. Thou shalt strive not to corrupt thine memory heap
4. Thou shalt share thine debugging knowledge
5. Thou shalt not overflow the buffer
6. Thou shalt not covet thy neighbour’s dump
7. Thou shalt not reverse engineer for profit
8. Thou shalt not attach debugger to thy neighbor’s wife
9. Thou shalt not commit adultery with bugs
10. Thou shalt not shalt thou to me

- Dmitry Vostokov @ DumpAnalysis.org -

This is already written application (10 years ago by me) that I’m adapting as a high-level interface to WinDbg (can be any GUI debugger actually). The basic idea revolves around floating buttons (listbox and task bar icons, optionally) that dynamically change with every new window or application. The number of buttons can be unlimited, they have tooltips and can be repositioned to any corner of the screen, they can play sounds, show video and pictures. On click they execute elaborated macro commands, including keystrokes and mouse movements, written in a special scripting language. For example, we can create buttons for CDA checklist.

I’ve created 2 buttons for WinDbg window:

When we switch from WinDbg to another application they disappear:

We switch back to WinDbg and they reappear. We can move them around the screen:

We can edit them by right click:

and change their shape:

The set of buttons can be saved as an executable file. When we run it on another PC it recreates all buttons when WinDbg window appears.

Written in C and using only Win32 API EasyDbg process consumes minimum resources. It sits on task bar for easy access:

- Dmitry Vostokov @ DumpAnalysis.org -

New cartoon from Narasimha Vedala based on Dr. Page idea (click on it to enlarge):

Unconventional Debugging: introduce a hot lady bug

- Dmitry Vostokov @ DumpAnalysis.org -

This is a new word I’ve just coined to describe applications heavily dependent on various hooks that are either injected by normal Windows hooking mechanism, registry or via more elaborate tricks like remote threads or patching code. Originally I thought of hookware but found that this term is already in use for completely different purpose.

Now I list various patterns in memory dumps that help in detection, troubleshooting and debugging of hooksware:

- Hooked Functions (user space)

- Hooked Functions (kernel space)

- Hooking Level

This is the primary detection mechanism for hooks that patch code.

See also Raw Pointer and Out-of-Module Pointer patterns.

- Hooked Modules

The WinDbg script to run when you don’t know which module was patched.

- Changed Environment

Loaded hooks shift other DLLs by changing their load address and therefore might expose dormant bugs.

- Insufficient Memory (module fragmentation)

Hooks loaded in the middle of address space limit the maximum amount of memory that can be allocated at once. For example, various virtual machines, like Java, reserve the big chunk of memory at the start up.

- No Component Symbols

We can get an approximate picture of what a 3rd-party hook module does by looking at its import table or in the case of patching by looking at the list of deviations returned by .chkimg command.

- Unknown Component

Might give an idea about the author of the hook.

- Coincidental Symbolic Information

Sometimes hooks are loaded at round addresses like 0×10000000 and these values are very frequently used as flags or constants too.

- Wild Code

When hooking goes wrong the execution path goes into the wild territory.

- Execution Residue

Here we can find various hooks that use normal Windows hooking mechanism. Sometimes the search for “hook” word in symbolic raw stack output of dds command reveals them but beware of Coincidental Symbolic Information. See also Raw Stack Analysis Scripts page.

- Message Hooks - Modeling Example

Windows message hooking pattern example.

- Hidden Module

Some hooks may hide themselves.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Hmm, I was looking at Google Analytics stats for dumpanalysis.org and here is the list of 154 visitor countries sorted by the decreasing number of visits (data for March - August, 2008):

United States
United Kingdom
India
Canada
Germany
China
Russia
France
Japan
South Korea
Ireland
Australia
Taiwan
Netherlands
Israel
Sweden
Italy
Brazil
Spain
Singapore
Romania
Norway
Ukraine
Belgium
Czech Republic
Switzerland
Poland
Denmark
Malaysia
Finland
Turkey
Austria
New Zealand
Hong Kong
Portugal
Argentina
South Africa
Belarus
Greece
(not set)
Philippines
Hungary
Bulgaria
Mexico
Slovakia
Malta
Serbia
Thailand
Croatia
Estonia
Vietnam
Lithuania
Slovenia
Bolivia
United Arab Emirates
Iran
Latvia
Indonesia
Pakistan
Iceland
Saudi Arabia
Egypt
Serbia and Montenegro
Chile
Colombia
Uruguay
Luxembourg
Peru
Morocco
Kazakhstan
Costa Rica
Jordan
Venezuela
Moldova
Cyprus
Jamaica
Algeria
Ecuador
Panama
Bangladesh
Puerto Rico
Sri Lanka
Bosnia and Herzegovina
Lebanon
Guatemala
Qatar
Kuwait
Tunisia
Mongolia
Syria
Guinea
Dominican Republic
Macedonia
Uzbekistan
Nepal
Bahrain
El Salvador
Palestinian Territory
Mauritius
Armenia
Barbados
Trinidad and Tobago
Georgia
Oman
Brunei
Nigeria
Kenya
Bermuda
Yemen
Cuba
Uganda
Bahamas
Netherlands Antilles
Iraq
Reunion
Maldives
Ghana
Ivory Coast
U.S. Virgin Islands
Guyana
Ethiopia
Andorra
Liechtenstein
Sudan
Namibia
Dominica
Saint Lucia
Seychelles
Angola
Guadeloupe
Libya
Paraguay
Cayman Islands
Gibraltar
Aruba
Laos
Somalia
New Caledonia
Zambia
Saint Vincent and the Grenadines
Montenegro
Congo - Kinshasa
Tanzania
Fiji
Azerbaijan
Faroe Islands
Botswana
Antigua and Barbuda
French Guiana
Myanmar
Grenada
Cambodia
Kyrgyzstan
Greenland

Here is the relative graph:

Another possible reason why North Korea is not on the list could be the total absence of Internet even in government and military institutions. Also note the presence of (not set) territory on the list. I suspect these are spies and other security and forensics professionals hiding their true location.

Other countries where people don’t know about memory dumps are:

Nicaragua
Honduras
Senegal
Western Sahara
Guinea-Bissau
Mauritania
Sierra Leone
Liberia
Mali
Burkina Faso
Benin
Niger
Chad
Cameroon
Gabon
Congo - Brazzaville
Central African Republic
Zimbabwe
Mozambique
Malawi
Madagascar
Afghanistan
Turkmenistan
Tajikistan
Papua New Guinea

They are depicted in red:

I’m thinking now about Memory Dump Awareness Index (MDAI) to assign to each country :-) 

- Dmitry Vostokov @ DumpAnalysis.org -

If you need the latest updates of Sysinternals tools you can always check this page:

http://live.sysinternals.com/

and you can also map a drive to this location (it is done automatically via WebDAV redirector):

\\live.sysinternals.com

- Dmitry Vostokov @ DumpAnalysis.org -

New cartoon from Narasimha Vedala, Science series, illustrates bugluon-debugluon interactions:

Dr. Debugalov works out Standard Model of Debugging to save the digital world

For complete explanation, see:

The Standard Model of Debugging

- Dmitry Vostokov @ DumpAnalysis.org -

This model was inspired by Large Hadron Collider (LHC) and NV’s Debugon. It is a simply-symmetrical model consisting of Bugluon - Debugluon pair of particles where one is a particle and the other is the corresponding antiparticle. The interaction between them is completely of non-gravitational nature. When they annihilate we get the light at the end of a long debugging tunnel, called Large Hard-debugging Collider (LHC). A bugluon particle moving in memory space usually leaves traces and various defects. A photographic picture of tracks left by bugluons is called a memory space dump. The analysis of various track patterns is called memory dump analysis

- Dmitry Vostokov @ DumpAnalysis.org -
 

This is a proposal for Debug Awards from Narasimha Vedala:

OSCAR parallel in debugging world - Pointer Award 

- Dmitry Vostokov @ DumpAnalysis.org -

New cartoon from Narasimha Vedala, Science series, provides great insight into incompleteness of debugging:

Debugalov’s Conjecture… “In every sufficiently complex system there is a bug you cannot debug…”

- Dmitry Vostokov @ DumpAnalysis.org -

New version of StressPrinters tool is available that has a fix for the following bug:

When you run the tool it enumerates all installed printer drivers. When Citrix Universal Printer driver is found the enumeration procedure skips the rest of the list. This results in not showing all the drivers installed in Citrix terminal services environment.

You can download the new version from Citrix support website: CTX109374.

- Dmitry Vostokov @ DumpAnalysis.org - 

I’ve just started writing the new series for the Russian version of my blog. It is called English for Software Engineers (in Russian) where I found new uses for UML (Unified Modeling Language) to teach important grammar patterns and concepts. Just to show you an example, here are two sequence diagrams depicting the difference between Present Perfect Tense and Past Tense:

 - Dmitry Vostokov @ DumpAnalysis.org -