Software Diagnostics Library

Learn from this Webinar about phenomenological, hermeneutical and analytical approaches to software diagnostics and its knowledge, foundations, norms, theories, logic, methodology, language, ontology, nature and truth. This seminar is hosted by Software Diagnostics Services.

Title: Introduction to Philosophy of Software Diagnostics
Date: 17th of December, 2012
Time: 19:00 GMT
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/872846486

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This is a second Webinar from Memory Dump Analysis Services on software diagnostics. The first one is about pattern recognition. During this Webinar you will learn how to apply systems theory and systems thinking for effective and efficient abnormal software behavior diagnostics: the foundation of software troubleshooting and debugging. The seminar summarizes 6 years of research done by Software Diagnostics Institute started with a short blog post Dumps and Systems Theory.

Title: Introduction to Systemic Software Diagnostics: Systems Thinking in Memory Dump and Software Trace Analysis
Date: 3rd of September, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/377382766

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services organizes a free webinar on a unified malware and victimware analysis by using behavioral and structural patterns including a live memory dump analysis example.

Date: 2nd of July, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/332458406

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In a week this short full color book should appear in online bookstores:

• Title: Software Narratology: An Introduction to the Applied Science of Software Stories
• Authors: Dmitry Vostokov, Memory Dump Analysis Services
• Description: This is a transcript of Memory Dump Analysis Services Webinar about Software Narratology: an exciting new discipline and a field of research founded by DumpAnalysis.org. When software executes it gives us its stories in the form of UI events, software traces and logs. Such stories can be analyzed for their structure and patterns for troubleshooting, debugging and problem resolution purposes. Topics also include software narremes and their types, anticipatory software construction and software diagnostics.
• Publisher: OpenTask (April 2012)
• Language: English
• Product Dimensions: 28.0 x 21.6
• Paperback: 26 pages
• ISBN-13: 978-1908043078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services organizes a free Webinar on Unified Software Diagnostics (USD) and the new scalable cost-effective software support model called Pattern-Driven Software Support devised to address various shortcomings in existing tiered software support organizations. Examples cover Windows, Mac OS  and Linux.

Date: 22nd of June, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/172771078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services (DumpAnalysis.com) organizes a free Webinar about software narratology: an application of ideas from narratology to software narrative stories.

Learn about this exciting new discipline and a field of research. When software executes it gives us its stories in the form of UI events, software traces and logs. Such stories can be analyzed for their structure and patterns for troubleshooting, debugging and problem resolution purposes.

Prerequisites: basic software troubleshooting and ability to read software generated logs.

Audience: anyone can benefit from this new perspective: from software users to software engineers and managers.

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/603136294

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

After the launch of the first episode about symbols I decided to make it recurrent where registration will be needed only once. So I apologize to all who already registered for episode 0×01 that another registration well be required for episode 0×02. However, no registration will be necessary for episode 0×03 and so on. If anyone misses episode 0×02 they can still register for episode 0×03 and all subsequent episodes only once, and so on by induction.

The second episode is about symbol file troubleshooting. All about this topic in 8 slides in 8 minutes including live WinDbg demonstration plus extra 8 minutes for you to ask questions.

Register for Debugging TV Frame 0×02 and further weekly episodes
Date: Friday, October 14, 2011
Time: 5:45 PM - 6:01 PM BST

Space is limited.
Reserve your seat now at:
https://www3.gotomeeting.com/register/318613774

After registering you will receive a confirmation email containing information about joining the show.

Debugging TV Frame 0×01
Recording: https://www3.gotomeeting.com/register/640694470
Slides: DebuggingTV_Frame_0×01.pdf
WinDbg log: DebuggingTV_Frame_0×01.txt

More frames are coming and www.debugging.tv will host TV programme and recordings of past episodes.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

To generate crash dumps for my recent presentation The Old New Crash: Cloud Memory Dump Analysis I created the tool similar to TestWER. It is a .NET worker service role for Windows Azure that does a NULL pointer assignment. Here is a source code from WorkerRole.cs (I left EnableCollection twice just in case :-)):

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Net;
using System.Threading;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.Diagnostics;
using Microsoft.WindowsAzure.ServiceRuntime;
using Microsoft.WindowsAzure.StorageClient;
using Microsoft.WindowsAzure.Diagnostics.Management;

namespace TestWAERWorker
{
    public class WorkerRole : RoleEntryPoint
    {
        public override void Run()
        {
            Trace.WriteLine("TestWAERWorker entry point called", "Information");

            Thread.Sleep(60000);

            unsafe
            {
                int* p = null;
                *p = 1;
            };

            while (true)
            {
                Thread.Sleep(10000);
                Trace.WriteLine("Working", "Information");
            }
        }

        public override bool OnStart()
        {
            // Set the maximum number of concurrent connections
            ServicePointManager.DefaultConnectionLimit = 12;

            CrashDumps.EnableCollection(true);

            DiagnosticMonitorConfiguration config = DiagnosticMonitor.GetDefaultInitialConfiguration();
            string conn_str = RoleEnvironment.GetConfigurationSettingValue("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString");
            CloudStorageAccount account = CloudStorageAccount.Parse(conn_str);
            config.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
            DiagnosticMonitor diagnosticMonitor = DiagnosticMonitor.Start(account, config);

            CrashDumps.EnableCollection(true);

            return base.OnStart();
        }
    }
}

The following forum discussion really helped me to get things right: http://stackoverflow.com/questions/5727692/azure-crash-dumps-not-appearing. So I’d like to thank Oliver Bock here.

If you want to deploy it to Windows Azure then don’t forget to add your storage account name and its key in the service configuration file. For your development environment you will need to replace the whole value with “UseDevelopmentStorage=true”. The full case study with detailed steps is forthcoming.

You can download the package from here: TestWAER.zip

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

I’m pleased to announce that MDAA, Volume 5 is available in PDF format:

www.dumpanalysis.org/Memory+Dump+Analysis+Anthology+Volume+5

It features:

- 25 new crash dump analysis patterns
- 11 new pattern interaction case studies (including software tracing)
- 16 new trace analysis patterns
- 7 structural memory patterns
- 4 modeling case studies for memory dump analysis patterns
- Discussion of 3 common analysis mistakes
- Malware analysis case study
- Computer independent architecture of crash analysis report service
- Expanded coverage of software narratology
- Metaphysical and theological implications of memory dump worldview
- More pictures of memory space and physicalist art
- Classification of memory visualization tools
- Memory visualization case studies
- Close reading of the stories of Sherlock Holmes: Dr. Watson’s observational patterns
- Fully cross-referenced with Volume 1, Volume 2, Volume 3, and Volume 4

Its table of contents is available here:

www.dumpanalysis.org/MDAA/MDA-Anthology-V5-TOC.pdf

Paperback and hardcover versions should be available in a week or two. I also started working on Volume 6 that should be available in November-December.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services plans a free Webinar in July to introduce a uniform methodology and tools for analysis of memory in cyber intelligence and counterintelligence activities.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The first Webinar to start an in-depth discussion of a uniform methodology and tools for analysis of crashes, hangs, and other types of abnormal software behaviour in cloud environments.

Memory Dump Analysis Services (DumpAnalysis.com) organizes a free Webinar

Date: 13th of June 2011
Time: 18:45 (BST) 13:45 (EST) 10:45 (PST)
Duration: 60 minutes

Topics include:

• What’s old
• What’s new
• Orbifold memory spaces
• Methodology and best practices
• Cloud-specific patterns
• Hands-on exercises

Prerequisites: working knowledge of basic memory dump analysis or live debugging.

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/597101206

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The presentation materials from the webinar (25th of March, 2011) are available for download:

http://www.dumpanalysis.com/PDSPSI-materials

Thanks to everyone who registered and attended!

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The first Webinar to start an in-depth discussion of pattern-driven software troubleshooting, debugging and maintenance:

Date: 25th of March 2011
Time: 18:30 (GMT) 14:30 (EST) 11:30 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/448268158

Topics include:

• A Short History of DumpAnalysis.org
• Memory Dump Analysis Patterns
• Troubleshooting and Debugging Tools (Debugware) Patterns
• Software Trace Analysis Patterns
• From Software Defects to Software Behavior
• Workaround Patterns
• Structural Memory Patterns
• Memory Analysis Domain Pattern Hierarchy
• New Directions

Prerequisites: experience in software troubleshooting and/or debugging.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

With the new year starts the new initiative to integrate traditional multidisciplinary debugging approaches and methodologies with multiplatform pattern-driven software problem solving, unified debugging patterns, best practices in memory dump analysis and software tracing, computer security, economics, and the new emerging trends I’m going to write about during this year.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Five volumes of cross-disciplinary Anthology (dubbed by the author “The Summa Memorianica”) lay the foundation of the scientific discipline of Memoretics (study of computer memory snapshots and their evolution in time) that is also called Memory Dump and Software Trace Analysis.ca

The 5th volume contains revised, edited, cross-referenced, and thematically organized selected DumpAnalysis.org blog posts about crash dump, software trace analysis and debugging written in February 2010 - October 2010 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software on Windows platforms, technical support and escalation engineers dealing with complex software issues, and security researchers, malware analysts and reverse engineers. The fifth volume features:

- 25 new crash dump analysis patterns
- 11 new pattern interaction case studies (including software tracing)
- 16 new trace analysis patterns
- 7 structural memory patterns
- 4 modeling case studies for memory dump analysis patterns
- Discussion of 3 common analysis mistakes
- Malware analysis case study
- Computer independent architecture of crash analysis report service
- Expanded coverage of software narratology
- Metaphysical and theological implications of memory dump worldview
- More pictures of memory space and physicalist art
- Classification of memory visualization tools
- Memory visualization case studies
- Close reading of the stories of Sherlock Holmes: Dr. Watson’s observational patterns
- Fully cross-referenced with Volume 1, Volume 2, Volume 3, and Volume 4

Product information:

• Title: Memory Dump Analysis Anthology, Volume 5
• Author: Dmitry Vostokov
• Language: English
• Product Dimensions: 22.86 x 15.24

• Paperback: 400 pages
• Publisher: Opentask (10 December 2010)
• ISBN-13: 978-1-906717-96-4

• Hardcover: 400 pages
• Publisher: Opentask (10 December 2010)
• ISBN-13: 978-1-906717-97-1

Back cover features memory space art image Hot Computation: Memory on Fire.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The choice of webinars below mostly reflects my personal preferences and long time desire to speak on topics like systems thinking, troubleshooting tool design and development. The other topic about BSOD minidump analysis was requested by participants in an ongoing survey. There will be more topics in 2011. All forthcoming webinars will be hosted by Memory Dump Analysis Services. The planning list includes:

Systems Thinking in Memory Dump and Software Trace Analysis

Software Troubleshooting and Debugging Tools: Objects, Components, Patterns and Frameworks with UML

• UML basics
• DebugWare patterns
• Unified Troubleshooting Framework
• RADII software development process
• Hands-on exercise: designing and building a tool

Blue Screen Of Death Analysis Done Right: Minidump Investigation for System Administrators

• Making sense of !analyze –v output
• Get extra troubleshooting information with additional WinDbg commands
• Guessing culprits with raw stack analysis
• Who’s responsible: hardware or software?
• Checklist and patterns
• Including hands-on exercises: send your own minidumps

More detailed information will be available soon. 

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This forthcoming full color book is the complete transcript of a Webinar organized by Memory Dump Analysis Services (www.DumpAnalysis.com).

It discusses user vs. kernel vs. physical (complete) memory space, challenges of complete memory dump analysis, common WinDbg commands, patterns and pattern-driven analysis methodology, common mistakes, fiber bundles, DumpAnalysis.org case studies and illustrates step by step a hands-on exercise in a complete memory dump analysis.

• Title: Fundamentals of Complete Crash and Hang Memory Dump Analysis
• Author: Dmitry Vostokov
• Publisher: OpenTask (October 2010)
• Language: English
• Product Dimensions: 28.0 x 21.6
• Paperback: 48 pages
• ISBN-13: 978-1906717155

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

If you attended Fundamentals of Complete Crash and Hang Memory Dump Analysis you probably remember the memory dump visualization question that I repeat here on this slide fragment:

I got a few responses:

“Unfortunately they are not identical - visual inspection shows that. I tried differencing the relevant sub-images in Photoshop and I can’t get zero.  Of course this can be due to compression artifacts and, more likely, the fact that the duplication is not required to be aligned to the borders. A stronger confirmation/refutation would require unrolling the bitmap to one dimension and sliding it back and forth until maximum correlation is found. Since I have not done the examples step by step, I am left guessing about just what the dump you show illustrates. An aliased memory mapped area is my first guess, and a flip/flop garbage collector is my second.”

“perhaps some module such as a .NET assembly is getting loaded twice in a .NET app, pre .NET 4, such as is dicsussed in this thread:
http://forum.sysinternals.com/why-some-net-assemblies-are-duplicated-in-memory_topic15279_post121591.html“

Initially I also thought that there was the same module loaded twice from different location like in Duplicated Module pattern. Unfortunately lm command didn’t show any duplicated loaded and unloaded modules as well as any hidden modules. So I looked at address information and found two identical relatively large regions at the beginning:

0:000> !address
[...]
BaseAddress EndAddress+1 RegionSize Type       State      Protect       Usage
[...]
0`00470000  0`007f0000   0`00380000 MEM_MAPPED MEM_COMMIT PAGE_READONLY <unclassified>
[…]
0`01f10000  0`02290000   0`00380000 MEM_MAPPED MEM_COMMIT PAGE_READONLY <unclassified>
[…]

The image above was scaled by ImageMagic from a bitmap generated by Dump2Picture:

The original image from Dump2Picture had different colors:

I quickly checked the colorimetric structure of those regions: 0`00470000  0`007f0000 and 0`01f10000  0`02290000 using MemPicture WinDbg script and they seem to conform with the magnified picture above:

0:000> $$>a< d:\Dump2Picture\mempicture.txt 0`00470000 L?0`007f0000-0`00470000
Writing 380000 bytes

C:\Program Files\Debugging Tools for Windows (x64)>dump2picture d2p-range.bin d2p-range.bmp

Dump2Picture version 1.1
Written by Dmitry Vostokov, 2007

d2p-range.bmp
d2p-range.bin
        1 file(s) copied.

C:\Program Files\Debugging Tools for Windows (x64)>d2p-range.bmp
.shell: Process exited

Here is the magnified slice from the original picture:

We see the same partitioning if we juxtapose the original picture and the picture of the address region:

Also these regions are completely identical if we compare their data:

0:000> c 0`00470000 L?(0`007f0000-0`00470000)/8 0`01f10000

So it looks like some file was mapped twice. Inspected via dc command it shows remarkable regularity not seen in executable modules. This regularity also manifests itself in color:

In order to verify I modeled this by writing a simple program that maps a file twice passed as a command line parameter:

int _tmain(int argc, _TCHAR* argv[])
{
 if (argc < 2)
 {
  puts("Usage: MappedFiles.exe <File_Name_To_Map>\n");
  return -1;
 }

 HANDLE hf = CreateFile(argv[1], GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
 HANDLE hm = CreateFileMapping(hf, NULL, PAGE_READONLY, 0, 0, NULL);
 MapViewOfFile(hm, FILE_MAP_READ, 0, 0, 0);

 hf = CreateFile(argv[1], GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
 hm = CreateFileMapping(hf, NULL, PAGE_READONLY, 0, 0, NULL);
 MapViewOfFile(hm, FILE_MAP_READ, 0, 0, 0);

 DebugBreak();
 return 0;
}

I ran it and chose to map explorer.exe because it was a sufficiently large image file:

C:\MappedFiles\Release>MappedFiles.exe c:\windows\explorer.exe

The dump file was saved and its processing shows this picture:

We clearly see identical regions and double check them from the dump file:

0:000> !address
BaseAddr EndAddr+1 RgnSize Type       State         Protect       Usage
[...]
a60000   d1d000    2bd000  MEM_MAPPED MEM_COMMIT    PAGE_READONLY <unclassified>
d1d000   d20000    3000    MEM_FREE   PAGE_NOACCESS Free
d20000   fdd000    2bd000  MEM_MAPPED MEM_COMMIT    PAGE_READONLY <unclassified>
[…] 

0:000> $$>a< d:\Dump2Picture\mempicture.txt 0`00470000 L?0`007f0000-0`00470000
Writing 380000 bytes

C:\Program Files\Debugging Tools for Windows (x64)>dump2picture d2p-range.bin d2p-range.bmp 

Dump2Picture version 1.1
Written by Dmitry Vostokov, 2007

d2p-range.bmp
d2p-range.bin
        1 file(s) copied.

C:\Program Files\Debugging Tools for Windows (x64)>d2p-range.bmp
.shell: Process exited

We see the same partitioning if we juxtapose results:

The application can be downloaded from here: MappedFiles.zip

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Finally I compiled a Questions and Answers page with all necessary links and examples during the weekend:

http://www.dumpanalysis.com/FCMDA-Q-A

I also added text versions of logs (in addition to zip files) to a Webinar materials page:

http://www.dumpanalysis.com/FCMDA-materials

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The presentation materials from the webinar (18th and 23rd of August, 2010) are available for download:

http://www.dumpanalysis.com/FCMDA-materials

Thanks to everyone who registered and attended!

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -