Software Diagnostics Library

For some time I was struggling with finding a good name for memory dump and software trace analysis activities. The name Memoretics I use for the science of memory dump analysis (that also incorporates software traces) seems not so good to describe the whole practical activity that should be transparent to everyone in IT. Fortunately, I timely understood that all these activities constitute the essence of software diagnostics that previously lacked any solid foundation. Thus, Software Diagnostics Institute was reborn from the previous Crash Dump Analysis Portal. This institute does pure and applied research and scientific activities and in recent years was funded mainly from OpenTask publisher and recently from Memory Dump Analysis Services. The latter company also recognized that the broadening of its commercial activities requires a new name. So, Software Diagnostics Services was reborn:

The First Comprehensive Software Diagnostics Service

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes I get requests to review application memory dump capture design. Of course, such requests usually come only when such designs don’t work or there are problems with loading saved crash dumps. The common blueprint of such architectures is a top level exception handler that use some API do capture and save process memory state. However, such designs forget why separate processed were introduced in the first place: to guard process memory space of different unrelated tasks (for related tasks there are threads). The data of the module (and its thread state) that does process memory capture may also be corrupt. The right design would be to show a message box with an information on how to use external process memory dumper such as Task Manager. If we need an automation then the right thing is to rely on WER features. Let separate processes do their work in separate spaces.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This is a second Webinar from Memory Dump Analysis Services on software diagnostics. The first one is about pattern recognition. During this Webinar you will learn how to apply systems theory and systems thinking for effective and efficient abnormal software behavior diagnostics: the foundation of software troubleshooting and debugging. The seminar summarizes 6 years of research done by Software Diagnostics Institute started with a short blog post Dumps and Systems Theory.

Title: Introduction to Systemic Software Diagnostics: Systems Thinking in Memory Dump and Software Trace Analysis
Date: 3rd of September, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/377382766

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Counter Value pattern covers performance monitoring and its logs. A counter value is some variable in memory, for example, a module variable, that is updated periodically to reflect some aspect of state or it can be calculated from different such variables and presented in trace messages. Such messages can also be organized in a similar format as ETW based traces we usually consider as examples for our trace patterns:

Source  PID TID   Function         Value
=================================================
[…]
System    0   0   Committed Memory 12,002,234,654
Process 844   0   Private Bytes    345,206,456
System    0   0   Committed Memory 12,002,236,654
Process 844   0   Working Set      122,160,068
[…]

Therefore, all other trace patterns such as adjoint thread (can be visualized via different colors on a graph), focus of tracing, characteristic message block (for graphs), activity region, significant event, and others can be applicable here. There are also some specific patterns such as global monotonicity and constant value that we discuss with examples in subsequent parts.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services will be administering certifications developed by Software Diagnostics Institute for memory dump and software trace analysis:

Software Diagnostics Maturity Enterprise Certification
Memory Dump Analysis Certification is available this September

Debugging TV Frames episode 0×10 contains some background information.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

DumpAnalysis.org portal has been reorganized to Software Diagnostics Institute to reflect the nature of its research activities. More updates later on.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

While preparing a seminar on Software Diagnostics I made a lot of notes and realized that a system of patterns, corresponding vocabulary and pattern language are needed for this discipline. Here patterns are supposed to be broad in nature and be different from patterns for specific artifacts such as memory dumps and software traces. So the first pattern addresses a diagnostic encounter with a First Fault in comparison to subsequent faults where the problem becomes noticeable and diagnostic resources are allocated. Such faults should not be dismissed. Dan Skwire is a passionate advocate of first fault software problem solving and wrote a book:

First Fault Software Problem Solving: A Guide for Engineers, Managers and Users

The following paper proposes distributed control flow reconstruction for first fault diagnosis:

TraceBack: First Fault Diagnosis by Reconstruction of Distributed Control Flow

Memory Dump Analysis Services uses patterns of abnormal software behavior for its first fault diagnostics that doesn’t require any special instrumentation:

Join Debugging Diagnostics Revolution!

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Motivated by 7 Habits of Highly Effective Debuggers I would like to reflect on a distinction between diagnostics and problem solving as separate processes (although highly related). First, we reverse the precept from that article because stories such as software logs and traces are of primary importance to software diagnostics (and not only). And without diagnostics there is no effective debugging (treatment, problem solving, etc.)

The Principle Precept of Diagnostics

Stories NOT Statistics secure certainty.

Next parts will be about actual habits so please stay tuned. I would try to finish this list before the forthcoming Webinar on software diagnostics.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes I hear voices saying that Linux, FreeBSD, and Mac OS X core dumps are uninteresting. This is not true. If you haven’t seen anything interesting there it just simply means you have only encountered a limited amount of abnormal software behaviour. The widespread usage of Windows OS means that most patterns have been diagnosed and described first and other OS are waiting their turn.

My goal is to have a pattern catalog with examples from different OS. For example, currently, all Mac OS X patterns I provide are just examples to existing Windows pattern names. All OS share the same structure and behavior, for example, structural memory analysis patterns and the same computational model. Although structural patterns are different from behavioral patterns I also plan to expand the structural list significantly especially in relation to forthcoming Windows malware analysis training. Regarding behavioral patterns it is possible to model and predict specific pattern examples for another OS by using already existing catalog.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

After 4 years in print this bestselling title needs an update to address minor changes, include extra examples and reference additional research published in Volumes 2, 3, 4, 5 and 6.

• Title: Memory Dump Analysis Anthology, Volume 1
• Author: Dmitry Vostokov
• Publisher: OpenTask (Summer 2012)
• Language: English
• Product Dimensions: 22.86 x 15.24

• Paperback: 800 pages
• ISBN-13: 978-1-908043-35-1

• Hardcover: 800 pages
• ISBN-13: 978-1-908043-36-8

The cover for both paperback and hardcover titles will also have a matte finish. We used A Memory Window artwork for the back cover.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In a week this short full color book should appear in online bookstores:

• Title: Software Narratology: An Introduction to the Applied Science of Software Stories
• Authors: Dmitry Vostokov, Memory Dump Analysis Services
• Description: This is a transcript of Memory Dump Analysis Services Webinar about Software Narratology: an exciting new discipline and a field of research founded by DumpAnalysis.org. When software executes it gives us its stories in the form of UI events, software traces and logs. Such stories can be analyzed for their structure and patterns for troubleshooting, debugging and problem resolution purposes. Topics also include software narremes and their types, anticipatory software construction and software diagnostics.
• Publisher: OpenTask (April 2012)
• Language: English
• Product Dimensions: 28.0 x 21.6
• Paperback: 26 pages
• ISBN-13: 978-1908043078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services organizes a free Webinar on Unified Software Diagnostics (USD) and the new scalable cost-effective software support model called Pattern-Driven Software Support devised to address various shortcomings in existing tiered software support organizations. Examples cover Windows, Mac OS  and Linux.

Date: 22nd of June, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/172771078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -