Software Diagnostics Library

Sometimes, when we have debugging symbols, information about local variables may be helpful in making sense of function disassembly. For example, we have this code fragment from WinDbg uf command:

511 00007ff6`6ab22a44 mov dword ptr [rbp+2078h],1
511 00007ff6`6ab22a4e mov dword ptr [rbp+207Ch],2
513 00007ff6`6ab22a58 mov eax,dword ptr [rbp+2078h]
513 00007ff6`6ab22a5e mov dword ptr [rbp+0Ch],eax
514 00007ff6`6ab22a61 mov dword ptr [rbp+0Ch],64h
515 00007ff6`6ab22a68 mov dword ptr [rbp+48h],3
515 00007ff6`6ab22a6f mov dword ptr [rbp+4Ch],4
516 00007ff6`6ab22a76 mov eax,dword ptr [rbp+0Ch]

Although source code lines are shown, suppose we don’t have source code to match. However, we can match Address Representations, such as [rbp+xxx], from the output of dv /V WinDbg command:

0:000> dv /V
...
000000ab`740fd00c @rbp+0x000c myBase = struct wmain::__l2::Base
...
000000ab`740ff078 @rbp+0x2078 myDerived = struct wmain::__l2::Derived
...
000000ab`740fd048 @rbp+0x0048 myDerived2 = struct wmain::__l2::Derived
...

Another usage is matching values in raw stack data with local variable addresses. Values as addresses and their symbolic representations here have some connection to ADDR Symbolic and Interpreted Pointers.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Almost 15 years ago we introduced Dereference Fixpoints when the address value is equal to the value at the address. In doing raw stack data classification and pattern matching we may be interested in more general Dereference Nearpoints (especially in position independent ones) illustrated in the following diagram:

Such Dereference Nearpoints may appear due to exception processing when a stack exception address or exception stack pointer address is propagated during exception processing, and multiple structure references, for example, when a local structure address is propagated during function calls.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Lattice is a selection of messages based on a fixed order distance between them (similar to lattices in geometry, one-dimensional in this case) or some other metric:

This analysis pattern is different from Time Scale where fixed time distance is used with additional analysis transformations.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Pressure is a “thermodynamic” variable along with Trace Temperature and Trace Volume. The fourth variable, the number of non-Silent Messages, N, is obvious. In the following diagram, two parts of the trace have approximately the same volume but different “temperature” that result in approx. same “pressure” according to a metaphor of an ideal gas law:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Temperature is an external quantitative (a number) or qualitative (for example, “hot”, “warm”, or “cold”) trace attribute of a trace or log or its fragment such as Activity Region that measures importance of the incident:

It is one example of Trace Field. Being an external attribute, independent from content, the trace with very few or no messages (Sparse Trace) can still have very high “temperature.” However, this analysis pattern is not the same as News Value which is an internal attribute based on trace content that measures its importance.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Volume counts not only messages but also Silent Messages:

The volume metaphor can also be applied to parts of the trace such as Activity Regions, Threads (and Adjoint Threads) of Activity, and various braids.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

How can we view traces and logs to facilitate there classification? One of the possible avenues is to view traces and logs consisting of the very large conceptual units, bricks, or bases to avoid combinatorial explosion. The natural candidates are Activity Region, Discontinuity, and Error Message or Exception Stack Trace Characteristic Message Block (Crash metaphor). So we have A, C, D, and we add B for Bad Activity Region whatever that means. For example, a typical trace that starts with some activity, then we have an exception stack trace belongs to AC Trace Class. Another trace that starts with some activity, then doesn’t show any messaged till the end can be classified as AD. A more elaborate class is ABCABCABCBC with periodicity of normal and bad activities culminating with error messages and finally with the last bad activity and exception. These three Trace Classes are depicted in the following diagram:

We combine sequences of the same bases into one, for example, AAAAA as just A, thus abstracting from semantic and syntactical differences between normal Activity Regions. The same is with other bases and can be considered as an extreme version of Quotient Trace. This classification can also be applied to individual Threads of Activity and Adjoint Threads of Activity.

This genetic-like description parallels the earlier proposal for DNA and RNA of Ruptured Computation.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

If Trace String analysis patterns caters for trace length variations when the sequence of Activity Regions is constant then Trace Amplitude is about different paths between constant regions at both trace ends that may include varying regions in between, as depicted in the following diagram:

Therefore, several different Trace Strings may be embedded in one Trace Amplitude. The idea of this analysis pattern was borrowed from the path integral formulation of QM.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Some Dia|gram language illustrations of trace and log analysis patterns can be easily converted to more usual graphs. For example, Trace Field can be converted to Combed Trace or Adjoint Threads of Activity:

The resulted series can be plotted as a graph:

If time is values are not uniform Time Scale analysis pattern may be involved with the constructed Motivic Trace. We call this analysis pattern Trace Graph.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When comparing traces we may be interested in their Trace Benchmarks, for example (not limited to, just a proposal):

• Performance impact (impact this trace collection has on system performance)
• Problem relevance score (was it really relevant or missed the problem repro?)
• Easy to understand (is there any room for improvement for trace messages?)
• Sensitivity (does it have any sensitive information?)
• Canonicity score (does it contain all required ATIDs?)
• Novelty score (any surprises?)
• Sufficiency (does it require supplemental log from other tools?)
• Pattern metrics (number of distinct error messages, etc., based on trace and log analysis patterns)
• Problem resolution impact (was it the trace that provided missing insights?)

This post-analysis pattern may also be useful for problem postmortems and case studies. This pattern is different from Trace Summary which is a pre-analysis information.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When comparing traces, saving them for further processing, or making their Trace Mask, we may transform them into Canonical Trace form with the same Trace Schema:

This analysis pattern is different from Master Trace and Meta Trace, where the former is about an “ideal” archetype trace of a use case to compare to, and the latter is about trace evolution when underlying code changes.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Most trace and log diagrams we use for analysis pattern illustration are 1-dimensional if we ignore the structure of individual trace messages despite being looking as 2-dimensional. It is, therefore, natural to consider them as periodic and aperiodic tessellations of a line using a set of segments of different lengths and colors. A Trace Tessellation analysis pattern is such an arrangement based on some criteria, for example, a combination of trace and log analysis patterns or simply some A(TID) mapping.

GPT-4 suggests the following outline for 1-dimensional tessellation:

Tessellating a one-dimensional space like a line is fundamentally simpler than tessellating a two-dimensional space. Given that a line segment is the only basic shape you can use to tessellate a line, at first glance, it may seem there isn’t much room for diverse or “interesting” patterns. However, we can introduce variability by considering the following:

Varying Lengths: Instead of using uniform line segments, use segments of varying lengths. For instance, alternating between short and long segments can create a rhythmic pattern.

Color or Texture Variations: Even though it’s a one-dimensional space, you can vary colors or textures. For example, alternate between black and white line segments.

Spacing: While traditional tessellation has no gaps, you could introduce regular or irregular spacing between line segments for a more abstract approach.

Pattern Interruption: Repeating a set pattern and then deliberately interrupting it can be visually interesting. For instance, having three short segments followed by one long segment, and then repeating.

Associative Representation: You can use tessellated patterns on a line to represent data or other concepts. For example, Morse code uses a combination of short (”dots”) and long (”dashes”) signals, essentially tessellating a line with two types of segments to convey information.

Fractals and Recursive Patterns: While not “tessellation” in the traditional sense, you can create patterns on a line using recursive rules. The Cantor set is an example of this, where you take a line segment, remove the middle third, and then repeat the process for the remaining segments ad infinitum.

These are just a few ways to introduce interest and complexity into the “tessellation” of a one-dimensional space. While it’s fundamentally simpler than two-dimensional tessellation, with some creativity, you can still achieve intriguing and meaningful patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes, we are interested in Exception Collection from all different memory parts and space types and using different analysis patterns, for example, for the user and managed spaces:

- Stored Exception;

- Exception Stack Traces from Stack Trace Collection from unmanaged space;

- Managed Code Exceptions from CLR Runtime Threads (~*e !pe -nested and !Threads WinDbg commands) including Nested and Mixed Exceptions;

- Recorded heap failures (!heap -s -v) and other Historical Information;

- Hidden Exceptions (unmanaged space) in Execution Residue (unmanaged user space) for all threads;

- Hidden Exceptions (managed space) in Execution Residue (managed space) for all threads.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Shared Buffer Overwrite may happen via different mechanisms. A virtual address and its underlying physical page may be used by different threads from one process, or if threads from different processes are involved, its underlying physical memory page may be shared between different processes. In the former case, we can check threads’ Execution Residue for the page virtual address range. In the latter case, for example, when we have random crashes in different processes at different virtual addresses, we can compare page frame numbers for problem virtual addresses:

0: kd> !process ffffc38c3010b0c0 0
PROCESS ffffc38c3010b0c0
SessionId: 1 Cid: 1224 Peb: 24fc30b000 ParentCid: 1284
DirBase: 0a953002 ObjectTable: ffffac8a0b2aab40 HandleCount: 184.
Image: conhost.exe

0: kd> !process ffffc38c305e8080 0
PROCESS ffffc38c305e8080
SessionId: 0 Cid: 01c8 Peb: 4acc277000 ParentCid: 0290
DirBase: 10b62b002 ObjectTable: ffffac8a081b33c0 HandleCount: 276.
Image: svchost.exe

0: kd> !pte 00007ffc`884a0000
VA 00007ffc884a0000
PXE at FFFFFB7DBEDF67F8 PPE at FFFFFB7DBECFFF90 PDE at FFFFFB7D9FFF2210 PTE at FFFFFB3FFE442500
contains 8A0000000485F867 contains 0A00000115063867 contains 0A00000009D64867 contains 86000001358EF025
pfn 485f ---DA--UW-V pfn 115063 ---DA--UWEV pfn 9d64 ---DA--UWEV pfn 1358ef —-A–UR-V

0: kd> .process /r /p ffffc38c3010b0c0
Implicit process is now ffffc38c`3010b0c0
Loading User Symbols
.................................

0: kd> .process /r /p ffffc38c305e8080
Implicit process is now ffffc38c`305e8080
Loading User Symbols
..................................

0: kd> !pte 00007ffc`884a0000
VA 00007ffc884a0000
PXE at FFFFFB7DBEDF67F8 PPE at FFFFFB7DBECFFF90 PDE at FFFFFB7D9FFF2210 PTE at FFFFFB3FFE442500
contains 0A00000107137867 contains 0A0000010703A867 contains 0A0000010713B867 contains 81000001358EF005
pfn 107137 ---DA--UWEV pfn 10703a ---DA--UWEV pfn 10713b ---DA--UWEV pfn 1358ef ——-UR-V

We call such an analysis pattern Shared Page.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Suppose we are interested in particular message types, which are Message Patterns. We can form a Pattern Vector with values of corresponding Message Pattern occurrences during each fixed time interval corresponding to selected Time Scale. A typical example is illustrated in the following diagram with counted Silent Messages as well:

Each Message Vector component value is the local Statement Current. We can also select Statement Densities instead, especially if we select only Message Patterns from Foreground Components. Other values are possible, for example, from Trace Field.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Inspired by Laws of Form and works inspired by it including Story and Structure we introduce Iconic Trace symbolic mapping for individual messages, Activity Regions, Motives, and Activity Theatre. A typical example is illustrated in the following diagram:

It is also possible to construct Iconic CoTraces. We will provide different symbolic profiles (with the meaning of various symbols) and examples later in the forthcoming revision of Software Narratology.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Often, we have Basic Facts but are unsure 100% if particular trace messages are relevant or not. We take the idea of Case Messages analysis pattern from fuzzy sets where we have a degree of membership function. It is illustrated in the following diagram:

In comparison, Message Set analysis pattern is about crisp sets where the degree of membership is either 0 or 1.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

If necessary, for example, for the conversion to Text Trace, individual trace messages may need to be converted to message text form blending various message constituents like ATIDs, Message Invariants, data, and Trace Constants into some textual narrative form. We illustrate it in the following diagram:

We call the resulting message text Denormalized Message. It is the reverse process than finding Features of Activity and constructing structured messages conforming to some Trace Schema.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In addition to Trace Skeleton, we can insert Silent Messages, treat non-silent messages as instances of some template message, and even take Quotient Trace of them, leaving the position of final non-silent messages intact. This is similar to retraction in topology, so we name this analysis pattern Trace Retract and illustrate it in the following diagram:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When we construct Trace Molecule, we may find that some Message Complex links from different ATIDs point to the same message and Tracemes. In such a case we have Message Bond by analogy with covalent bonds. One such example is illustrated in the following diagram:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -