Software Diagnostics Library

After almost a 6-year break we resume extending the catalog of structural memory patterns as a foundation of pattern-oriented memory forensics, root cause analysis, and software internals. The next pattern we add is borrowed from archaeology and is called Region Strata. When we have several memory snapshots we can analyze Memory Regions for their changes at different times:

For example, we got two memory dumps of the same process saved by WER with approx. one minute difference. We saved the raw stack region of the current thread using .writemem WinDbg command in text files (Strata1.txt and Strata2.txt) and then combined them into one Excel table (Strata.xlsx). In that table we can see memory region changes as shown in this picture (click to expand):

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services organizes a free Webinar on Unified Software Diagnostics (USD) and the new scalable cost-effective software support model called Pattern-Driven Software Support devised to address various shortcomings in existing tiered software support organizations. Examples cover Windows, Mac OS  and Linux.

Date: 22nd of June, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/172771078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

More than 4 years passed since I provided a longer structuralist definition. Recently I came to recognize a pattern-driven iterative and incremental nature of memory and software trace analysis and post-construction software problem solving in general and therefore a one sentence definition became necessary:

“Recognition and interpretation of patterns of software behavior”

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memoretics promotes pattern-driven memory dump and software trace analysis which has many uses but not limited to:

• Software and site reliability
• Software Debugging
• QA and Software Testing
• Computer Security
• Software Troubleshooting
• Malware Research and Analysis
• Tools as a Service (TaaS)
• Supportability
• Software Diagnostics

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memoretics as a science of memory snapshots borrows many ideas from the following disciplines (the list is not exhaustive):

• Troubleshooting and Debugging
• Intelligence Analysis
• Critical Thinking
• Forensics
• Linguistics
• Archaeology
• Psychoanalysis
• History
• Mathematics: Sets and Categories
• Literary Criticism and Narratology

It also contributes many ideas back. The following diagram depicts such an interaction:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Due to the need to extend existing basic and intermediate Accelerated Windows Memory Dump Analysis training Memory Dump Analysis Services organises advanced training course. Here is the description and registration information:

Learn how to navigate through memory dump space and Windows data structures to troubleshoot and debug complex software incidents. We use a unique and innovative pattern-driven analysis approach to speed up the learning curve. The training consists of practical step-by-step exercises using WinDbg to diagnose structural and behavioral patterns in 32-bit and 64-bit process, kernel and complete memory dumps.

If you are registered you are allowed to optionally submit your memory dumps before the training. This will allow us in addition to the carefully constructed problems tailor extra examples to the needs of the attendees.

The training consists of one four-hour session and additional homework exercises. When you finish the training you additionally get:

1. A full transcript in PDF format (retail price $200)
2. 5 volumes of Memory Dump Analysis Anthology in PDF format (retail price $100)
3. A personalized attendance certificate with unique CID (PDF format)

Prerequisites: Basic and intermediate level Windows memory dump analysis: ability to list processors, processes, threads, modules, apply symbols, walk through stack traces and raw stack data, diagnose patterns such as heap corruption, CPU spike, memory and handle leaks, access violation, stack overflow, critical section and resource wait chains and deadlocks. If you don’t feel comfortable with prerequisites then Accelerated Windows Memory Dump Analysis training is recommended to take (or purchase a corresponding book) before attending this course.

Audience: Software developers, software technical support and escalation engineers.

Session: December 9, 2011 4:00 PM - 8:00 PM GMT

Price: 210 USD

Space is limited.
Reserve your remote training seat now at:
https://student.gototraining.com/24s4l/register/3788047691824598784

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

To model software behavior at application and system levels, test generative debugging scenarious and construct software defects using metaprogramming I started working on MdTL (Metadefect Template Library). Its consists of C++ templates for structural and behavioral patterns. The simplest examples include Threads<Spike> and Spike<Thread>. The template classes can be more complex, of course, utilizing the full power of C++, STL, and existing libraries like Boost. The unique and novel feature of this library is the inclusion of dual classes of behaviour such as Leak<>, Deadlock<>, Overflow<>, Residue<>, Exception<>, Contention<>, etc. parameterized by various structural memory classes like Process<>, Thread<>, Heap<>, Stack<>, Region<>, Buffer<>, etc. MdTL also includes classes to model software tracing and this will be used for simultaneous software trace and memory dump analysis case study in the forthcoming presentation. I also plan to extend this metaprogramming approach in the future to model malware analysis patterns by introducing Metamalware Template Library (MmTL).

Forthcoming parts will introduce template classes and examples.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

I’m pleased to announce that MDAA, Volume 5 is available in PDF format:

www.dumpanalysis.org/Memory+Dump+Analysis+Anthology+Volume+5

It features:

- 25 new crash dump analysis patterns
- 11 new pattern interaction case studies (including software tracing)
- 16 new trace analysis patterns
- 7 structural memory patterns
- 4 modeling case studies for memory dump analysis patterns
- Discussion of 3 common analysis mistakes
- Malware analysis case study
- Computer independent architecture of crash analysis report service
- Expanded coverage of software narratology
- Metaphysical and theological implications of memory dump worldview
- More pictures of memory space and physicalist art
- Classification of memory visualization tools
- Memory visualization case studies
- Close reading of the stories of Sherlock Holmes: Dr. Watson’s observational patterns
- Fully cross-referenced with Volume 1, Volume 2, Volume 3, and Volume 4

Its table of contents is available here:

www.dumpanalysis.org/MDAA/MDA-Anthology-V5-TOC.pdf

Paperback and hardcover versions should be available in a week or two. I also started working on Volume 6 that should be available in November-December.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Followed by my search for a periodic table of software defects I came up with a novel approach called Generative Debugging partially borrowed from principles and parameters generative linguistics framework. For debugging and post-construction software problem solving purposes we have Behavioral Principles and Structural Parameters. The parameters allow us to describe existing and even generate new software defects (I’m also considering Structural Principles and Behavioural Parameters but whether this is fruitful needs to be explored and seen. We may also consider the name “generative debugging” as a “naming” counterpart to software construction problem solving called generative programming. So the former could also be used to generate problem solutions faster in automatic fashion. Both, in the future, may be combined into a unified generative software problem solving.

As a first example, consider an application functional hang with a thread blocked by a pending I/O request packet (IRP). It is composed from general behavioral principle of blocked (or unresponsive) activity and structural parameters including Thread and IRP.

In the following parts many software incidents will be explored from this generative perspective.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The presentation materials from the webinar (25th of March, 2011) are available for download:

http://www.dumpanalysis.com/PDSPSI-materials

Thanks to everyone who registered and attended!

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The first Webinar to start an in-depth discussion of pattern-driven software troubleshooting, debugging and maintenance:

Date: 25th of March 2011
Time: 18:30 (GMT) 14:30 (EST) 11:30 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/448268158

Topics include:

• A Short History of DumpAnalysis.org
• Memory Dump Analysis Patterns
• Troubleshooting and Debugging Tools (Debugware) Patterns
• Software Trace Analysis Patterns
• From Software Defects to Software Behavior
• Workaround Patterns
• Structural Memory Patterns
• Memory Analysis Domain Pattern Hierarchy
• New Directions

Prerequisites: experience in software troubleshooting and/or debugging.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

With the new year starts the new initiative to integrate traditional multidisciplinary debugging approaches and methodologies with multiplatform pattern-driven software problem solving, unified debugging patterns, best practices in memory dump analysis and software tracing, computer security, economics, and the new emerging trends I’m going to write about during this year.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Borrowing routine activity theory (RAT) from criminology I would like to introduce the similar approach to abnormal software behavior with patterning activities that adds additional unmotivated offenders to combine malware (software rats) with unintentional ordinary common bugware:

The application of RAT to software can be metaphorically named as Function Activity Theory (FAT).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

As the new decade is approaching (2011-2020) we would like to make a few previews and predictions:

- Increased complexity of software will bring more methods from biological, social sciences and humanities in addition to existing methods of automated debugging and computer science techniques

- Focus on first fault software problem solving (when aspect)

- Focus on pattern-driven software problem solving (how aspect)

- Fusion of debugging and malware analysis into a unified structural and behavioral pattern framework

- Visual debugging, memory and software trace visualization techniques

- Software maintenance certification

- Focus on domain-driven troubleshooting and debugging tools as a service (debugware TaaS)

- Focus on security issues related to memory dumps and software traces

- New scripting languages and programming language extensions for debugging

- The maturation of the science of memory snapshots and software traces (memoretics)

Imagining is not not limited to the above and more to come and explain in the forthcoming parts.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

MAaaS includes 2 complementary DA+TA services:

1. Dump Analysis as a Service (DAaaS)
2. Trace Analysis as a Service (TAaaS)

Memory Dump Analysis Services is the first organization to provide such a service at an audit and certification levels.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Five volumes of cross-disciplinary Anthology (dubbed by the author “The Summa Memorianica”) lay the foundation of the scientific discipline of Memoretics (study of computer memory snapshots and their evolution in time) that is also called Memory Dump and Software Trace Analysis.ca

The 5th volume contains revised, edited, cross-referenced, and thematically organized selected DumpAnalysis.org blog posts about crash dump, software trace analysis and debugging written in February 2010 - October 2010 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software on Windows platforms, technical support and escalation engineers dealing with complex software issues, and security researchers, malware analysts and reverse engineers. The fifth volume features:

- 25 new crash dump analysis patterns
- 11 new pattern interaction case studies (including software tracing)
- 16 new trace analysis patterns
- 7 structural memory patterns
- 4 modeling case studies for memory dump analysis patterns
- Discussion of 3 common analysis mistakes
- Malware analysis case study
- Computer independent architecture of crash analysis report service
- Expanded coverage of software narratology
- Metaphysical and theological implications of memory dump worldview
- More pictures of memory space and physicalist art
- Classification of memory visualization tools
- Memory visualization case studies
- Close reading of the stories of Sherlock Holmes: Dr. Watson’s observational patterns
- Fully cross-referenced with Volume 1, Volume 2, Volume 3, and Volume 4

Product information:

• Title: Memory Dump Analysis Anthology, Volume 5
• Author: Dmitry Vostokov
• Language: English
• Product Dimensions: 22.86 x 15.24

• Paperback: 400 pages
• Publisher: Opentask (10 December 2010)
• ISBN-13: 978-1-906717-96-4

• Hardcover: 400 pages
• Publisher: Opentask (10 December 2010)
• ISBN-13: 978-1-906717-97-1

Back cover features memory space art image Hot Computation: Memory on Fire.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

I have discovered rules that make it possible to devise a memory dump and software trace analysis equivalent of the Periodic Table of Elements in Chemistry. It allows prediction of abnormal software behaviour and structural defects and what patterns to look for after deploying software and collecting its artifacts. More on this is in the next part of these series.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Here is the description of a high-level architecture of the previously announced project CARE (Crash Analysis Report Environment). To remind, the main idea of the project is to process memory dumps on a client to save debugger logs. They can be sent to a server for pattern-driven analysis of software behaviour. Textual logs can also be inspected by a client security team before sending. Certain sensitive information can be excluded or modified to have generic meaning according to the built-in processing rules like renaming (for example, server names and folders). Before processing, verified secured logs are converted to abstract debugger logs. Abstracting platform-specific debugger log format allows reuse of the same architecture for different computer platforms. We call it CIA (Computer Independent Architecture). Do not confuse it with ICA (Independent Computer Architecture) and CIA acronym is more appropriate for memory analysis (like similar MAFIA acronym, Memory Analysis Forensics and Intelligence Architecture). These abstract logs are checked for various patterns (in abstracted form) using abstract debugger commands and an abstract report is generated according to various checklists. Abstract reports are then converted to structured reports for the required audience level. Abstract memory analysis pattern descriptions are prepared from platform-specific pattern descriptions. In certain architectural component deployment configurations both client and server parts can reside on the same machine. Here’s the simple diagram depicting the flow of processing:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

I detour for MARS expedition. You may also call it Memory Analysis Report System as malware analysis is always exploration of memory (in general). Why is this sudden change of course? After reading Gilles Deleuze I want to broaden the concept of “malware” and give it new orientation and direction of thinking. Beside that I also want new challenges after many years of research in pattern-driven memory dump and software trace analysis of abnormal software behaviour.

You may have also noticed small restructuring (rebranding) of this blog and DumpAnalysis.org headers.

See you there

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

As a practical example of applying behavioral and structural pattern analysis of computer memory and traces OpenTask plans to publish the following title next year:

• Title: Malware Patterns: Structure and Behavior of Computer Adware, Crimeware, Rootkits, Scareware, Spyware, Trojans, Viruses, Victimware and Worms
• Author: Dmitry Vostokov
• Paperback: 1200 pages
• Publisher: OpenTask (October 2011)
• ISBN-13: 978-1-908043-01-6

The inclusion of victimware is necessary because of the effects of defective malware.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -