Values in memory regions, when interpreted as addresses, such as stack can be classified according to their categories such as zero, module (code and data), heap, stack, Foreign Stack, Small Value, error, and out-of-range (for example, kernel space address in user space). Such classification can be encoded and optionally visualized as Region Spectrum:
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
The advent of virtual machines, the possibility of saving complete memory snapshots without interruption, and the ability to quickly convert such snapshots into a debugger readable memory dump format such as in the case of VMware allows to study how Stack Trace Collections and Wait Chains change over time in complex problem scenarios. Such Stack Trace Surface may also show service restarts if PID changes for processes of interest. We call this pattern by analogy with a memory dump surface where each line corresponds to an individual memory snapshot with coordinates from 0 to the highest address:
In case of orbifold memory space we have a case of a 3D volume (we may call 3D orbifold).
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
Whereas, when naturally visualized with Dump2Picture, a CDF trace looks like a sea with embedded library, a process monitor log looks more like a fragment from a memory dump with some large scale internal structure:
Its 450×1280 slice:
Top and bottom of x4 magnified image:
…
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
Was browsing Amazon and found this book with a cover like an artificially colored memory image and text in black gaps but rotated by 90 degrees counterclockwise:
Noise Channels: Glitch and Error in Digital Culture (Electronic Mediations)
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
After 4 years in print this bestselling title needs an update to address minor changes, include extra examples and reference additional research published in Volumes 2, 3, 4, 5 and 6.
The cover for both paperback and hardcover titles will also have a matte finish. We used A Memory Window artwork for the back cover.
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
The new 6th volume contains revised, edited, cross-referenced, and thematically organized selected DumpAnalysis.org blog posts about memory dump and software trace analysis, software troubleshooting and debugging written in November 2010 - October 2011 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software on Windows platforms, technical support and escalation engineers dealing with complex software issues, and security researchers, malware analysts and reverse engineers. The sixth volume features:
Product information:
Back cover features 3d memory space visualization image created with ParaView.
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
Memoretics as a science of memory snapshots borrows many ideas from the following disciplines (the list is not exhaustive):
It also contributes many ideas back. The following diagram depicts such an interaction:
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
Whereas DNA of a ruptured computation is a coarse grained, software behavioral genome in general is fine grained consisting of multiple behavioral patterns such as seen in memory dumps and software traces. I will elaborate more on it later especially in relation to software behavior profiles. Here’s a nice “memory DNA” metaphorical picture from 3D memory visualization post:
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
If you like the completeness, grand complete memory dumps, multi-volume oeuvres (the more volumes the better) and natural memory visualization you would then like to open this box and listen to this complete performance achievement to get energy and inspiration for long debugging sessions:
Liszt: The Complete Piano Music
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
Finally you can even learn a WinDbg command from a certificate. Memory Dump Analysis Services has created a certificate with dv WinDbg command on it:
Source: http://www.dumpanalysis.com/sample-certificate
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
I’m pleased to announce that MDAA, Volume 5 is available in PDF format:
www.dumpanalysis.org/Memory+Dump+Analysis+Anthology+Volume+5
It features:
- 25 new crash dump analysis patterns
- 11 new pattern interaction case studies (including software tracing)
- 16 new trace analysis patterns
- 7 structural memory patterns
- 4 modeling case studies for memory dump analysis patterns
- Discussion of 3 common analysis mistakes
- Malware analysis case study
- Computer independent architecture of crash analysis report service
- Expanded coverage of software narratology
- Metaphysical and theological implications of memory dump worldview
- More pictures of memory space and physicalist art
- Classification of memory visualization tools
- Memory visualization case studies
- Close reading of the stories of Sherlock Holmes: Dr. Watson’s observational patterns
- Fully cross-referenced with Volume 1, Volume 2, Volume 3, and Volume 4
Its table of contents is available here:
www.dumpanalysis.org/MDAA/MDA-Anthology-V5-TOC.pdf
Paperback and hardcover versions should be available in a week or two. I also started working on Volume 6 that should be available in November-December.
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
Sometimes added instrumentation via gflags, application and driver verifier options affect system, service or application performance and resources. I call this pattern Instrumentation Side Effect. For example, after enabling full page heap, one process on an x64 machine was growing up to 24GB and its user memory dump shows that every heap allocation was recorded in a stack trace database:
0:055> !gflag
Current NtGlobalFlag contents: 0x02000000
hpa - Place heap allocations at ends of pages
0:055> ~*kc
[...]
48 Id: 117fc.c164 Suspend: 1 Teb: 000007ff`fff52000 Unfrozen
Call Site
ntdll!ZwWaitForSingleObject
ntdll!RtlpWaitOnCriticalSection
ntdll!RtlEnterCriticalSection
verifier!AVrfpDphEnterCriticalSection
verifier!AVrfpDphPreProcessing
verifier!AVrfDebugPageHeapAllocate
ntdll!RtlDebugAllocateHeap
ntdll! ?? ::FNODOBFM::`string’
ntdll!RtlAllocateHeap
msvcrt!malloc
ModuleA!foo1
[…]
49 Id: 117fc.de80 Suspend: 1 Teb: 000007ff`fff54000 Unfrozen
Call Site
ntdll!RtlCompareMemory
ntdll!RtlpLogCapturedStackTrace
ntdll!RtlLogStackTrace
verifier!AVrfpDphPlaceOnFreeList
verifier!AVrfDebugPageHeapFree
ntdll!RtlDebugFreeHeap
ntdll! ?? ::FNODOBFM::`string’
ntdll!RtlFreeHeap
kernel32!HeapFree
msvcrt!free
ModuleB!foo2
[…]
50 Id: 117fc.3700 Suspend: 1 Teb: 000007ff`fff4e000 Unfrozen
Call Site
ntdll!ZwWaitForSingleObject
ntdll!RtlpWaitOnCriticalSection
ntdll!RtlEnterCriticalSection
verifier!AVrfpDphEnterCriticalSection
verifier!AVrfpDphPreProcessing
verifier!AVrfDebugPageHeapFree
ntdll!RtlDebugFreeHeap
ntdll! ?? ::FNODOBFM::`string’
ntdll!RtlFreeHeap
kernel32!HeapFree
msvcrt!free
ModuleC!foo3
[…]
0:055> !runaway
User Mode Time
Thread Time
38:d090 0 days 0:02:28.793
44:ca48 0 days 0:01:04.459
48:c164 0 days 0:00:56.909
43:4458 0 days 0:00:54.475
50:3700 0 days 0:00:43.992
45:6f98 0 days 0:00:38.953
49:de80 0 days 0:00:24.211
1:391c 0 days 0:00:00.639
0:7e90 0 days 0:00:00.109
55:a300 0 days 0:00:00.046
34:10c9c 0 days 0:00:00.015
21:d054 0 days 0:00:00.015
56:b0a0 0 days 0:00:00.000
54:8b78 0 days 0:00:00.000
53:155b8 0 days 0:00:00.000
52:b444 0 days 0:00:00.000
Top modules ModuleA(B, C) from the spiking and heap intensive threads are from the same vendor.
I was able to get a 200×27349 slice from that dump using ImageMagick and it shows almost all virtual memory space filled with traces of this pictorial form (magnified by x8):
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
Finally had time today to test python WinDbg extension. I installed python 2.6.6 and used Debugging Tools for Windows x64. The script I used was from the extension web site blog slightly modified and called processes.py:
from pykd import *
nt = loadModule( "nt" )
processList = typedVarList( nt.PsActiveProcessHead, "nt", "_EPROCESS", "ActiveProcessLinks" )
for process in processList:
dprintln( "".join( [ chr(i) for i in process.ImageFileName.values() ] ) )
dprintln( "\r" )
I loaded x64 version of pykd.pyd extension and executed the script:
0: kd> !py c:\PyScripts\processes.py
System
smss.exe
csrss.exe
csrss.exe
psxss.exe
winlogon.exe
wininit.exe
services.exe
lsass.exe
lsm.exe
svchost.exe
svchost.exe
svchost.exe
svchost.exe
svchost.exe
svchost.exe
svchost.exe
svchost.exe
spoolsv.exe
mdm.exe
svchost.exe
svchost.exe
svchost.exe
taskhost.exe
dwm.exe
explorer.exe
DTLite.exe
mmc.exe
WZQKPICK.EXE
concentr.exe
pnamain.exe
jusched.exe
wfcrun32.exe
msdtc.exe
iexplore.exe
iexplore.exe
iexplore.exe
splwow64.exe
iexplore.exe
jucheck.exe
iexplore.exe
notepad.exe
notepad.exe
iexplore.exe
notepad.exe
iexplore.exe
notepad.exe
notepad.exe
iexplore.exe
audiodg.exe
CDViewer.exe
wfica32.exe
iexplore.exe
notepad.exe
cmd.exe
conhost.exe
wuauclt.exe
wfica32.exe
wlrmdr.exe
TrustedInstall
wfica32.exe
notepad.exe
iexplore.exe
wmplayer.exe
VISIO.EXE
mspaint.exe
svchost.exe
sppsvc.exe
windbg.exe
mmc.exe
LogonUI.exe
taskeng.exe
NotMyfault.exe
Very good so far. I’m learning now Python language to write more interesting scripts for crash dump analysis pattern diagnostics, mining statistics, formatting and visualization. After browsing Amazon for some time I chose this book for study:
A Primer on Scientific Programming with Python
About 5 or 6 years ago I was thinking about using a Prolog interpreter to write an expert system for crash dump analysis. I even wrote a simple Prolog program to match product hotfixes and was contemplating writing a WinDbg extension especially after I learnt that SWI-Prolog system I used had an interpreter DLL that could be dynamically loaded into a Win32 API application.
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
This is the 3rd revision of memory map visualization tools list where I include IDA Pro and a visualizer from Debug Analyzer.NET (every category is presented in chronological order of my encounter with links):
1. Synthetic
2. Natural
a. Static
b. Semi-dynamic
c. Dynamic
Please let me know any other approaches or links you know.
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
With the new year starts the new initiative to integrate traditional multidisciplinary debugging approaches and methodologies with multiplatform pattern-driven software problem solving, unified debugging patterns, best practices in memory dump analysis and software tracing, computer security, economics, and the new emerging trends I’m going to write about during this year.
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
This is an artwork commissioned for the New Year of DeBugging 0×7DB. How many bugs can you count there? Click on the picture to expand instead of using a magnifying glass (as seen on debugging books covers):
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
