Archive for the ‘Software Narratology’ Category

Software Narratology (Literary Theory Terms, Part 2): abstract, accent, act, action, adaptation, address

Sunday, November 8th, 2020

Abstract is usually the summary of an artifact (see Trace Summary analysis pattern) or not concrete description (see Analysis Pattern Square diagram).

Accent as stress in a line of verse has its correspondence to data in Message Pattern, which can be seen as a sequence of variables and Message Invariants.

Act as a play division corresponds to Activity Regions (see also trace partitioning and Activity Theatre analysis patterns).

Action as the main story of a narrative artifact may involve a sequence of selected Significant Events, Macrofunctions, Activity Regions with Motives. In a software narratological framework for presenting software storiesaction is a sequence of selected messages that constitutes a software plot (an acquired software artifact that may not be complete/full due to abridgment like restricting tracing/logging to selected components).

Adaptation as interpreting an artifact as a different one (from one media to another, or a different structure) is similar to treating memory dumps as traces/logs or vice versa as Projective Debugging.

Address as a story written for a specific group of people could be a software execution artifact explicitly acquired and adapted to some external users or Declarative Trace messages crafted for a specific team in mind (see also Embedded Comment analysis pattern).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Software Narratology (Literary Theory Terms, Part 1): ab ovo, in medias res, flashback, abridged edition

Thursday, November 5th, 2020

Ab ovo is a software story (for example, a trace or log, a problem description, see software narratology square) that starts from the beginning of the use case events it narrates (see also Use Case Trail analysis patterns) or the start of software execution (see also Visibility Limit analysis pattern). Logging may start from some middle event of a use case, source code (see also Declarative Trace analysis pattern), or a log may be a part of a larger full trace (see also a software narratological framework for presenting software stories): in medias res. Such software stories may also have flashbacks, for example, stack traces, especially in software problem descriptions. Often, flashbacks are the only available software stories. Some tracing and logging sessions may be deliberately shortened to save space, communication throughput, or other reasons like security, similar to abridged editions of literary works (see also Abridged Dump and Missing Component analysis patterns). Such editions of software execution artifacts often hinder analysis (see Lateral Damage analysis pattern).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 200)

Sunday, September 13th, 2020

Trace and log analysis patterns may be additionally applied not only to a database like tables but also to texts (as an example of general trace and log analysis). Sentences may form trace messages with paragraphs and chapters corresponding to traditional ATIDs (IDs for Adjoint Threads of Activity) such as TID and PID in the most simple syntax mapping case, and certain sentences may be interpreted as Silent Messages.

Different attribute generation schemas may be used, for example, selected vocabulary may be used to assign TID numbers. More complex cases may require paratexts, supplementary texts providing additional structure and semantic information like in the case of Paratext memory analysis pattern, the case of extended traces.

The opposite process of converting traces and logs to text is also possible with additional paratext generation if necessary. We call this two-way analysis pattern Text Trace. After converting texts to logs it is possible to apply the majority of trace and log analysis patterns from the catalog.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 183)

Thursday, July 9th, 2020

Defect Group analysis pattern addresses messages related to source code defects (PLOTs), problem descriptions, and Inter-Correlation with wrong configuration files (Small DA+TA). It differs from Message Set analysis pattern as a predicate to group them may not be easily available.

Such Defect Groups can be results of previous analyses activities. The name of the analysis pattern came from representation theory defect group of a block but at present, it is only name analogy.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 182)

Tuesday, April 14th, 2020

Source code can be considered as a type of a general trace from a corresponding generative narrative plane. We call it Generative Trace since it can generate different traces of execution. If such a trace contains logging code statements, then they form Declarative Trace as a subset of messages. Generative Trace also overlaps with the corresponding Moduli Trace. We can apply many trace and log analysis patterns and even consider line number axis as pseudo-time. The following diagram illustrates Linked Messages analysis pattern in the context of Generative Trace and generated traces:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 181)

Wednesday, November 6th, 2019

Message Flow trace and log analysis pattern generalizes NetFlow to software narratives. We count messages based on the set of Adjoint Threads of Activity, for example PID.TID. This also subsumes network traces aggregated by Src.Dst. Individual single attributes can also be used, for example, aggregation by Thread of Activity, and also by Message Sets.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 175)

Sunday, July 21st, 2019

When we do trace and log analysis (and software data in general) we look at specific messages found from search (Message Patterns), Error Messages, Significant Events, visit Activity Regions, filter Message Sets, walk through (Adjoint) Threads of Activity, and do other actions necessitated by trace and log analysis patterns. All these can be done in random order (starting from some analysis point), not necessarily representing the flow of Time or some other metric:

Analyzed messages form their own analysis trace that we call CoTrace (CoLog, CoData) where the prefix Co- denotes a space dual to trace (log, data) space:

Instead of messages (or in addition to) we can also form CoTraces consisting of visited Activity Regions or some other areas:

We can apply trace analysis patterns to CoTraces as well. The latter can also be used in creation of higher-order pattern narratives.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 157)

Sunday, May 13th, 2018

According to the definition in “Topological Signal Processing” by Michael Robinson (ISBN: 978-3662522844) “a signal consists of a collection of related measurements” (p. 5). For traces and logs we can apply the similar definition and consider Signal as a collection of local messages having the same Message Invariant and related variable data values. Signals are examples of Message Sets. The typical example are sets of related Counter Value messages. Signals can be obtained by obtaining Adjoint Thread of Activity of a specific message (to filter out Background Components “noise”) as illustrated in the following diagram:

Generally, the variable “measurement” part can form Braid of Activity.

We introduce Signal analysis pattern to bridge the gap between Software Narratology and Hardware Narratology.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 105)

Thursday, April 23rd, 2015

Reading Boris Uspensky’s book “A Poetics of Composition: The Structure of the Artistic Text and Typology of a Compositional Form” (in its original Russian version) led me to borrow the concept of viewpoints. The resulting analysis pattern is called Trace Viewpoints. These viewpoints are, “subjective” (semantically laden from the perspective of a trace and log reader), and can be (not limited to):

- Error viewpoints (see also False Positive Error, Periodic Error, and Error Distribution)

- Use case (functional) viewpoints (see also Use Case Trail)

- Architectural (design) viewpoints (see also Milestones)

- Implementation viewpoints (see also Implementation Discourse, Macrofunctions, and Focus of Tracing)

- Non-functional viewpoints (see also Counter Value and Diegetic Messages)

- Signal / noise viewpoints (see also Background and Foreground Components)

In comparison, Activity Regions, Data Flow, Thread of Activity, and Adjoint Thread of Activity are “objective” (structural, syntactical) viewpoints.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 104)

Tuesday, March 17th, 2015

Trace Mask is a superposition of two (or many) different traces. This is different from Inter-Correlation pattern where we may only search for certain messages without the synthesis of a new log. The most useful Trace Mask is when we have different time scales (or significantly different Trace Currents). Then we impose an additional structure on the one of the traces:

We got the idea from Narrative Masks discussed in Miroslav Drozda’s book “Narativní masky ruské prózy” (”Narrative Masks in Russian Prose”).

The very simple example of Trace Mask is shown in Debugging TV Episode 0×15.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 102)

Tuesday, March 3rd, 2015

When we do tracing and logging much of computational activity is not visible. For live tracing and debugging this can be alleviated by adding Watch Threads. These are selected memory locations that may or may not be formatted according to specific data structures and are inspected at each main trace message occurrence or after specific intervals or events:

This analysis pattern is different from State Dump which is about intrinsic tracing where the developer of logging statements already incorporated variable watch in source code. Watch Threads are completely independent from original tracing and may be added independently. Counter Value is the simplest example of Watch Thread if done externally because the former usually doesn’t require source code and often means some OS or module variable independent of product internals. Watch Thread is also similar to Data Flow pattern where specific data we are interested in is a part of every trace message.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 94)

Tuesday, November 11th, 2014

Trace messages may correspond to specific implementation code such as recording the status of an operation, dumping data values, printing errors, or they may correspond to higher levels of software design and architecture, and even to use case stories. We call such messages Milestones by analogy with project management. Alternative names can be Chapter Messages, Summary Messages, Checkpoints, or Use Case Messages. These are different from Macrofunctions which are collections messages grouped by some higher function. Milestone messages are specifically designed distinct trace statements:

They can also be a part of Significant Events, serve the role of Anchor Messages, and be a part of Basic Facts and Vocabulary Index.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 82)

Monday, May 5th, 2014

So far we have been discussing trace analysis patterns related to execution of a particular software version. However, software code changes and also its tracing and logging output: from large scale changes where components are replaced to small scale code refactoring affecting message structure and format. On a software narratological level this corresponds to a narrative about a software trace or log, it evolution. Such Meta Trace analysis pattern is different from Master Trace pattern where the latter is similar to what Metanarrative is usually meant in narratology: a master or grand idea - an expected trace if all functional requirements were correctly identified and implemented during software construction and non-functional ones are met during software execution.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 74)

Friday, July 19th, 2013

Most of the time when we look at software trace fragments we recognize certain Motifs* such as client-server interaction, publisher-subscriber notifications, database queries, plugin sequence initialization, etc. This pattern is different from Master Trace which corresponds to a normal use-case or working software scenario and may actually contain several Motifs as it is usually happens in complex software environments. On the other side of the spectrum there are software narremes (basic narrative units) and Macrofunctions (single semantic units). Motifs help to further bridge the great divide between software construction and software diagnostics with software narremes corresponding to implementation patterns, macrofunctions to design patterns, and motifs to architectural patterns although an overlap between these categories is possible.

* The idea of a pattern name comes from motives in mathematics.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Generalized Software Narrative and Trace

Monday, March 25th, 2013

In the past we viewed software traces and logs as temporarily ordered event sequences. Since events are just memory data we have a map

T  -> M

as can be seen in the definition of a software trace. Here we generalize the domain to any arbitrary set, for example, it can be a list of indexes or pointers or even memory itself. The latter map can give us narrative chains such as

M -> M -> M -> M

and even give us a grand unification of memory and log analysis and the possibility to apply software narratology to memory dump analysis as well. We talk about it soon and provide some generalized software narrative examples.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Software Trace Analysis Patterns Domain Hierarchy

Sunday, November 18th, 2012

I get many questions on whether software log analysis patterns from Software Diagnostics Institute are OS or platform or product specific. My answer is that they are independent from all of them because they are based on viewing software logs as stories of computation and were discovered by application of narratological analysis (software narratology). In addition to these patterns there exist domain specific problem patterns such as wrong hotfix level or specific product error code during software installation or execution. Typical examples of support for such platform and product specific type of patterns include Microsoft Windows Problem Reporting and Citrix Auto Support.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Software Diagnostics Training: Two Approaches

Saturday, November 3rd, 2012

Doing memory dump analysis training for more than 2 years I found that students are divided into 2 types: those who prefer to see source code first and those who want to see a memory dump first. We actually prefer to show a memory dump first and then explore it to find certain patterns of abnormal structure and behavior. Software Diagnostics Services used this approach to design its Accelerated Windows Memory Dump Analysis and Accelerated .NET Memory Dump Analysis courses. Students explore memory dumps and debugger logs to find memory dump analysis patterns which are introduced when necessary. After that they can check source code of modeling applications if they have development experience. Accelerated Windows Software Trace Analysis course uses a different approach. It introduces all software trace analysis patterns at once because they are patterns from software narratology independent from programming languages and software platforms. After that they explore and analyze software traces and logs. We can summarize these 2 approaches on this diagram:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Analysis Patterns (Part 53)

Monday, October 8th, 2012

Periodic Message Block is similar to Periodic Error but not limited to errors or failure reports. One such example we recently encountered is when some adjoint activity (such as messages from specific PID) stop to appear after the middle of the trace and after that there are repeated blocks of similar messages from different PIDs with their threads checking for some condition (waiting for event and reporting timeouts):

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Forthcoming Accelerated Windows Software Trace Analysis Training.

Trace Analysis Patterns (Part 52)

Wednesday, September 26th, 2012

The modern software trace recording, visualization and analysis tools such as Process Monitor, Xperf, WPR and WPA provide stack traces associated with trace messages. Consider stack traces as software traces we have, in a more general case, traces (fibers) bundled together on (attached to) a base software trace. For example, a trace message, that mentions an IRP can have its I/O stack attached together with thread stack trace with function calls leading to a function that emitted the trace message. Another example is association of different types of traces with trace messages such as managed and unmanaged ones. This general trace analysis pattern needs a name so we opted for Fiber Bundle as analogy with a fiber bundle from mathematics. Here’s a graphical representation of stack traces recorded for each trace message where one message also has an associated I/O stack trace:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Network Trace Analysis Patterns (Part 1)

Thursday, July 19th, 2012

After some thinking I’ve decided to apply software trace analysis pattern approach to network trace analysis which lacks a unified pattern language. Here I consider a network trace as essentially a software trace where packet headers represent software trace messages coupled with associated transmitted data:

Since we have a trace message stream formatted by a network trace visualization tool we can apply most if not all trace analysis patterns for diagnostics including software narratology for interpretation, discourse and different representations. We provide a few trivial examples here and more in subsequent parts. The first example is Discontinuity pattern:

Other similar patterns are No Activity, Truncated Trace and Time Delta. The second example is Anchor Messages:

Additional example there include Significant Event and Bifurcation Point patterns. Layered protocols are represented through Embedded Message pattern (to be described and added to the pattern list soon). Such traces can be filtered for their embedded protocol headers and therefore naturally represent Adjoint Thread pattern (for the more detailed description of adjoint threads as extension of multithreading please see the article What is an Adjoint Thread):

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -