Software Diagnostics Library

Periodic Message Block is similar to Periodic Error but not limited to errors or failure reports. One such example we recently encountered is when some adjoint activity (such as messages from specific PID) stop to appear after the middle of the trace and after that there are repeated blocks of similar messages from different PIDs with their threads checking for some condition (waiting for event and reporting timeouts):

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Forthcoming Accelerated Windows Software Trace Analysis Training.

The modern software trace recording, visualization and analysis tools such as Process Monitor, Xperf, WPR and WPA provide stack traces associated with trace messages. Consider stack traces as software traces we have, in a more general case, traces (fibers) bundled together on (attached to) a base software trace. For example, a trace message, that mentions an IRP can have its I/O stack attached together with thread stack trace with function calls leading to a function that emitted the trace message. Another example is association of different types of traces with trace messages such as managed and unmanaged ones. This general trace analysis pattern needs a name so we opted for Fiber Bundle as analogy with a fiber bundle from mathematics. Here’s a graphical representation of stack traces recorded for each trace message where one message also has an associated I/O stack trace:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

After some thinking I’ve decided to apply software trace analysis pattern approach to network trace analysis which lacks a unified pattern language. Here I consider a network trace as essentially a software trace where packet headers represent software trace messages coupled with associated transmitted data:

Since we have a trace message stream formatted by a network trace visualization tool we can apply most if not all trace analysis patterns for diagnostics including software narratology for interpretation, discourse and different representations. We provide a few trivial examples here and more in subsequent parts. The first example is Discontinuity pattern:

Other similar patterns are No Activity, Truncated Trace and Time Delta. The second example is Anchor Messages:

Additional example there include Significant Event and Bifurcation Point patterns. Layered protocols are represented through Embedded Message pattern (to be described and added to the pattern list soon). Such traces can be filtered for their embedded protocol headers and therefore naturally represent Adjoint Thread pattern (for the more detailed description of adjoint threads as extension of multithreading please see the article What is an Adjoint Thread):

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When depicting trace analysis patterns I used two-dimensional diagrams based on CDF (ETW) traces such as this one for a bifurcation point:

While working today on a new pattern I needed a new expressive way to graphically illustrate the same idea of trace bifurcation points but without too much drawing. Traces from particle chambers and scattering diagrams came to my imagination after draw the first few diagrams illustrating bifurcation points:

Time directional arrow end can be omitted:

Trace variation after a bifurcation point can be indicated by angle partition:

The case when a variation also happens before is illustrated on this diagram:

and the case with several bifurcations:

Are N-bifurcations like on the diagram below possible?

Yes, they are, if the course of execution depends on some non-binary trace message parameter such as a loaded module that implements a required interface differently.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Motivated by 7 Habits of Highly Effective Debuggers I would like to reflect on a distinction between diagnostics and problem solving as separate processes (although highly related). First, we reverse the precept from that article because stories such as software logs and traces are of primary importance to software diagnostics (and not only). And without diagnostics there is no effective debugging (treatment, problem solving, etc.)

The Principle Precept of Diagnostics

Stories NOT Statistics secure certainty.

Next parts will be about actual habits so please stay tuned. I would try to finish this list before the forthcoming Webinar on software diagnostics.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In some cases it is useful to consider Message Context: a set of surrounding messages having some relation to the chosen message:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes we need to pay attention to Error Distribution, for example, the distribution of the same error across a software log space or different error messages in different parts of the same software log or trace (providing effective partition):

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

If Break-in Activity is usually unrelated to a thread or an adjoint thread which has a discontinuity then Resume Activity pattern highlights messages from that thread:

The difference can be seen on the following graphical representation of two traces we analyzed where in a working trace a break-in preceded resume activity whereas in a non-working trace both patterns were absent:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

We resume our software trace analysis pattern catalog. The next pattern is called Break-in Activity. This is a message or a set of messages that surface just before the end of discontinuity of a adjoint thread and possibly triggered it:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In a week this short full color book should appear in online bookstores:

• Title: Software Narratology: An Introduction to the Applied Science of Software Stories
• Authors: Dmitry Vostokov, Memory Dump Analysis Services
• Description: This is a transcript of Memory Dump Analysis Services Webinar about Software Narratology: an exciting new discipline and a field of research founded by DumpAnalysis.org. When software executes it gives us its stories in the form of UI events, software traces and logs. Such stories can be analyzed for their structure and patterns for troubleshooting, debugging and problem resolution purposes. Topics also include software narremes and their types, anticipatory software construction and software diagnostics.
• Publisher: OpenTask (April 2012)
• Language: English
• Product Dimensions: 28.0 x 21.6
• Paperback: 26 pages
• ISBN-13: 978-1908043078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services organizes a free Webinar on Unified Software Diagnostics (USD) and the new scalable cost-effective software support model called Pattern-Driven Software Support devised to address various shortcomings in existing tiered software support organizations. Examples cover Windows, Mac OS  and Linux.

Date: 22nd of June, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/172771078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Narrative theory distinguishes between frame types such as (Fludernik, McHale, Nelles, Wolf):

- introductory framing (missing end frame) [—————————-

- terminal framing (missing opening frame) —————————-]

- [—————————-]

- interpolated framing [—-[  ]—-[     ]——–]

At the level of the software trace or an adjoint thread as a whole the first 3 types correspond to various types of this pattern Partition: Head, Prologue, Core, Epilogue, Tail where certain parts are missing. The first 2 types can also be instances of Truncated Trace pattern. Interpolated framing can be an instance of multiple discontinuities. All 4 types also correspond to foreground component messages and in general we have multiple Trace Frames as depicted:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The previous definition of software narratology was restricted to software traces and logs (the top left quadrant on a software narrative square, also the part of Memoretics which studies memory snapshots). Now, with the broadening of the domain of software narratology to the whole world of software narrative stories including actor interactions with software in construction requirements use cases and post-construction incidents we give another definition:

Software narrative is a representation of software events and changes of state. Software Narratology is a discipline that studies such software narratives (software narrative science).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Software narratological approach can be useful for writing, analyzing and validating historical narratives. Trace and event log messages play the role of historical events where process ids are assigned to particular historical institutions and their representatives. Threads serve the role of historical entities like persons. Modules play the role of shared ideologies. We already use this approach for writing history books.

This can also work on a different level such as analyzing a history of debugging as a sequence of troubleshooting and debugging stories. More on this later as I plan to provide concrete examples from history. For the time being please read how software narratology and memoretics (the study of memory snapshots) help fiction writers as well.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

After introducing software narrative planes it is logical to expand the field of software narratology to cover the whole domain of software construction and post-construction. We therefore combine both pairs of planes to create a narratological square:

Please also register for the forthcoming free Webinar: Introduction to Software Narratology where I plan to provide a coherent overview of the recent developments in this new field.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Based on an idea of expression and content planes from glossematics with a basic unit of glosseme we can organize software traces with corresponding patterns and software trace narremes (basic units of software narrative such as traces and event logs) into 2 planes: software trace narrative plane (expression) with narremes and the corresponding program lines of traces (PLOTs) source code and design plane (content) with their own set of construction narremes such as collaboration of software constructs. All this corresponds to the following diagram:

The same can be said about actor interaction level of software construction (what ought to be) and post-construction (what is) phases having their own construction and post-construction narratives, patterns and narremes such as in requirements (use cases) and in problem and software incident descriptions:

In the forthcoming articles I provide more examples and explanations including specific software narremes useful from the practical post-construction software problem solving perspective. Please also register for the forthcoming free Webinar: Introduction to Software Narratology where I plan to provide a coherent overview of this new field.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services (DumpAnalysis.com) organizes a free Webinar about software narratology: an application of ideas from narratology to software narrative stories.

Learn about this exciting new discipline and a field of research. When software executes it gives us its stories in the form of UI events, software traces and logs. Such stories can be analyzed for their structure and patterns for troubleshooting, debugging and problem resolution purposes.

Prerequisites: basic software troubleshooting and ability to read software generated logs.

Audience: anyone can benefit from this new perspective: from software users to software engineers and managers.

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/603136294

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In narratology anti-narrative denotes a narrative having sequences of events impossible in reality. In software traces such sequences usually depict abnormal software behaviour. Here are some parallels with corresponding trace analysis patterns:

Fiction                     | Software Trace
================================================
Repeated unrepeatable       | Periodic Error (?)
Denarration (erasure)       | No Activity / Incomplete History
Chronological contradiction | Impossible Trace

Question mark means that possibly another pattern is needed there.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Any DA+TA source (dump artifact + trace artifact) can be used to generate and validate narrative fiction where memory dump components provide fiction structure and software trace components provide behaviour (plot and story):

I already use such help in writing fiction books http://www.opentask.com/fiction-titles (”computational” novels) and use trace analysis patterns in validating historical narratives http://www.opentask.com/history-titles (with Excel and software trace viewers as tools). Here fiction is in no way different from software fictions outlined in the article about software stories. Please also note 2 other fields along with software narratology (SN was defined in mid 2009 where the first pattern appeared earlier in 2009) that are under development: software chorology and chorography. Adjoint threads and trace analysis patterns are the major contributions from SN.

Final note: Because we consider memory dumps in a general sense (e.g. philosophy of memoidealism where any data is a memory dump) and software trace narremes as small memory dumps (mid 2009) any data stream potentially can be represented as some fictional or real story. And vice versa, any story can be transformed into corresponding computer memory dumps and software traces (for example, the narralog language is being designed to model software stories).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Forthcoming Accelerated Software Trace Analysis training requires extensive real life like software logs with multiple software behaviour patterns. The similar accelerated memory dump analysis trainings (unmanaged / native Windows and .NET) also required good memory dumps but this problem was solved by modeling patterns of abnormal software behaviour in an appropriate implementation language such as C++ and C#. Modeling software traces with hundreds of software components, processes and threads would require enormous programming efforts. Therefore, the natural approach is to describe a desired software trace in some declarative language (or minimally imperative) and get a million line software log that models a specific combination of trace analysis patterns. So, welcome to such a language called Narralog: Software Narrative Log  or Narrative Logic. Please note that this language is different from Riemann programming language which is a language to describe software problems and generate software problem solving tools. Language details and whether a kind of a Metadefect Template Library will be used to implement Narralog or simple macroprogramming is suffice will be decided and announced when we further develop our training.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -