Software Diagnostics Library

Trace Network is an analysis pattern in which traces and logs are treated as evidence for constructing an attributed interaction network N=(V, E), where vertices V are Motives, (Adjoint) Threads or Features of Activity, and their combinations, and directed edges E are created by an explicit correspondence rule between them, for example, request/response, causality, correlation propagation, spawn/join relation, or shared resource usage. A scope such as Time Delta or some filtering for Message Patterns may also be applied before the network construction.

Edge aggregation, weighting, and labels are part of the construction specification, so the result is not merely a drawing but a diagnostic network on which structural properties such as fan-in, fan-out, hubs, components, and derived measures such as Trace Divergence can be computed. This differs from Trace Graph, whose primary purpose is plotting or graphing trace data, and from Message Complex, whose primary elements are messages connected geometrically rather than identities connected relationally.

Trace Network analysis pattern differs from Causal History, Causal Messages, and Causal Chains in both primitive elements and construction intent. Causal History is a message-level structure whose arrows represent possible causation; Causal Messages are those messages selected as causally relevant within that history; and Causal Chains are abstractions of causal relations into linked 1-chains, 2-chains, and higher n-chains. By contrast, Trace Network is a general constructed network whose vertices are typically diagnostic identities rather than messages, and whose edges are induced by an explicitly declared relation derived from trace evidence, such as causal linkage, adjoint correspondence, request/response coupling, shared-resource mediation, or correlation transfer. Accordingly, a Trace Network may encode causal structure as one special case, but it is not restricted to causality and does not by itself imply chain-complex abstraction.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes, we want to count the number of (Adjoint) Threads of Activity corresponding to a specified ATID:

We can view this as (adjoint) threads coming into or out of the specified ATID, similar to divergence, which gives the name Trace Divergence log analysis pattern. This analysis pattern differs from Cord of Activity, which is not a number, and the latter may not have a single, unvarying source or target ATID to which other A(TID)s correspond. It is also different from Trace Flux, where the number of threads is an external variable not related to traces and logs, and from Message Flow, which operates on the individual message level, temporal in nature, and counters are set in advance.

Typical examples include SYN floods in network traces (src and dst ATIDs), the number of threads corresponding to the specific PID, or the number of threads contending for the specified API.

Activity Divergence may look similar, but its surface is temporal, whereas Trace Divergence’s, surface is structural. There can be several Trace Divergencies in the same trace or log since they are per ATID.

Formally, Trace Divergence is a property of a constructed graph, for example, D_in​(a)=∣{x∈V∣x→a}∣; Activity Divergence is a property of a constructed signal, interpreted as dynamics, for example, D_in​(a,t).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

We can view traces and logs as abstract polynomials that consists of abstract monomials. For example, if we have trace messages A,B,C, and D, the trace ABCACACACCD represents a single monomial. The multiplication operation in monomials represents message concatenation. But we can also split the trace as an abstract sum of several monomials, for example, ABC + AC + AC + AC + CD, or ABC + 3*AC + CD. The addition operation is a concatenation of traces even if concatenated traces consist of just one message. Note the distinction here between concatenation of messages and traces. By Trace Polynomial we mean a canonical abstract polynomial representation where we divide the trace by monomial when the next message in the message stream is already contained in the previous monomial, for example, ABC + 2AC + AC^2D.

Both addition and multiplication are non-commutative, and no distributivity between them. Mathematically speaking, we have the so-called a non‑distributive bi‑semigroup, or, in a category-theoretic sense, such abstract polynomials are objects in a free 2‑semigroupal category without interchange.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Lattice is a selection of messages based on a fixed order distance between them (similar to lattices in geometry, one-dimensional in this case) or some other metric:

This analysis pattern is different from Time Scale where fixed time distance is used with additional analysis transformations.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Most trace and log diagrams we use for analysis pattern illustration are 1-dimensional if we ignore the structure of individual trace messages despite being looking as 2-dimensional. It is, therefore, natural to consider them as periodic and aperiodic tessellations of a line using a set of segments of different lengths and colors. A Trace Tessellation analysis pattern is such an arrangement based on some criteria, for example, a combination of trace and log analysis patterns or simply some A(TID) mapping.

GPT-4 suggests the following outline for 1-dimensional tessellation:

Tessellating a one-dimensional space like a line is fundamentally simpler than tessellating a two-dimensional space. Given that a line segment is the only basic shape you can use to tessellate a line, at first glance, it may seem there isn’t much room for diverse or “interesting” patterns. However, we can introduce variability by considering the following:

Varying Lengths: Instead of using uniform line segments, use segments of varying lengths. For instance, alternating between short and long segments can create a rhythmic pattern.

Color or Texture Variations: Even though it’s a one-dimensional space, you can vary colors or textures. For example, alternate between black and white line segments.

Spacing: While traditional tessellation has no gaps, you could introduce regular or irregular spacing between line segments for a more abstract approach.

Pattern Interruption: Repeating a set pattern and then deliberately interrupting it can be visually interesting. For instance, having three short segments followed by one long segment, and then repeating.

Associative Representation: You can use tessellated patterns on a line to represent data or other concepts. For example, Morse code uses a combination of short (”dots”) and long (”dashes”) signals, essentially tessellating a line with two types of segments to convey information.

Fractals and Recursive Patterns: While not “tessellation” in the traditional sense, you can create patterns on a line using recursive rules. The Cantor set is an example of this, where you take a line segment, remove the middle third, and then repeat the process for the remaining segments ad infinitum.

These are just a few ways to introduce interest and complexity into the “tessellation” of a one-dimensional space. While it’s fundamentally simpler than two-dimensional tessellation, with some creativity, you can still achieve intriguing and meaningful patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Often, we have Basic Facts but are unsure 100% if particular trace messages are relevant or not. We take the idea of Case Messages analysis pattern from fuzzy sets where we have a degree of membership function. It is illustrated in the following diagram:

In comparison, Message Set analysis pattern is about crisp sets where the degree of membership is either 0 or 1.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Looking at software traces and logs as 2-categories allows us to consider Whisker Traces (horizontal composition) in addition to vertical composition such as Serial Traces, Trace Mask, and Container Traces. The same ATIDs can be combined, and if there is a time mismatch, additional message copies need to be added (whiskering). The process is illustrated in the following diagram:

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Message Complex takes inspiration from simplicial complexes. We select a message, choose TID or ATID, and connect to the nearest messages having the same TID or ATID. This procedure can be repeated for newly connected messages. Then we select another ATID and repeat the procedure. Three connected messages with the same ATID may form a triangle and may also intersect another triangle with a different ATID if they share the same message. A very simple example is illustrated in the following diagram:

Message Complex is more structural and geometric compared to Message Context, which is just a set of surrounding messages regardless of their TID or ATID based on some relationship criteria.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

If we have Step Dumps or Evental Dumps or simply some different memory dumps, for example, from Fiber Bundle and Orbifold memory spaces, we may run debugger commands across them. Then we can track changes in their output like we did in Stack Trace Change analysis pattern. We call the generalization of the latter pattern Structure Sheaf by analogy with structure sheaves of ringed spaces in mathematics. Here we metaphorically treat sequences of debugger commands applied to memory areas (memory structures) as rings of functions on open subsets. We originally wanted to call this analysis pattern Stack Trace (command) for one command and Stack Trace Collection (commands) for a set of commands but realized that the stack trace analogy here makes sense only for sequential memory dumps ordered in time and not for memory dumps taken from different sources.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Several Strands of Activity from different types of ATIDs (Adjoint Threads of Activity) combine into Cord of Activity:

Between cord and rope analogies we chose cord as having “ord” (ordinal) in it (and c as cardinal). It is also possible to combine several Cords of Activity from different traces (Trace Dimension) to form a “cable-laid rope”. We don’t introduce a separate pattern here since in the resulting Trace Mask we have new Cord of Activity due to the additionally created ATID type referencing former separate traces and logs. Data references in messages may provide additional braiding via Braids of Activity.

We started with strands (we got the idea from the discussion of ethnomathematics where strand analysis was mentioned) but then we found the following useful discussion on rope terminology: “Art and Science of Rope“.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes we may want to Flag a message or Activity Region, for example, using Message Annotations. In other cases we may have Activity Regions are sorted by their coordinate-wise inclusion. Or we have inclusion of Message Sets. The analysis pattern name is borrowed from flag filtration in mathematics, where we consider subsets of messages and Activity Regions as subspaces. Dia|gram pictures of Flags may even resemble flags of some countries.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

It is possible to foliate traces into separate traces having the same structure and scale (we also show corresponding Trace Fabric for the original trace):

In the diagram above Trace Foliation was done for message type, for example, error and normal messages. The reverse operation of Trace Mask would produce the same original trace.

Correspondingly Trace Fabric can be foliated too giving rise to “orchestra” representation and vice versa via Trace Mask:

Bars can be added with the help of Silent Messages.

The name of this analysis pattern was also inspired by foliations in mathematics.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Traces and logs from diverse software systems doing different things may have similar Trace Shape despite completely different message content, especially for specific Threads of Activity or Adjoint Threads of Activity:

This may be apparent when we compare Trace Shape of Quotient Trace.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Causal History messages (black circles) pass through Activity Regions which can be marked as hollow circles:

We call this analysis pattern Trace D’Enfant by analogy with dessin d’enfant in mathematics, a bipartite graph embedded in an oriented surface, so in theory Traces D’Enfants can be studied algebraically.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The relations between Causal History messages (0-chains) can be abstracted as Causal Chains (1-chains). Two relations can be linked if an endpoint of one is also a beginning point of another:

The relations of 1-chains can be abstracted as 2-chains and so on (n-chains):

We took the idea of relation spaces and chains from already quoted “Discreet Causal Theory” book. Causal chain terminology is also used in philosophy.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When looking at Causal History we can choose Causal Messages (not necesseraly the top ones):

Causal Messages may not overlap with the trace Defect Group which may not have any causal relevance being only correlation messages.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace Paths and Back Traces form Causal History of the log where arrows point in the direction of possible causation:

Here we borrow the notion of causal sets from physics and corresponding mathematics. The left diagramming idea was taken from Discrete Causal Theory book and Hasse diagrams (which is inverted in our picture). Also, such graphs are internal to software narratives compared to the more general external space we proposed earlier.

We omit Time arrow as it is possible to consider general traces and logs with their causality markers.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Trace analysis gestures that result in CoTrace also produce Trace Path between messages of interest:

Such Trace Paths can also be useful for Trace Homotopy analysis. They also provide the basis for Explanation Traces.

Note that Trace Path is also a reverse for Back Trace analysis pattern. Both are usually selected from Working Set.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Defect Group analysis pattern addresses messages related to source code defects (PLOTs), problem descriptions, and Inter-Correlation with wrong configuration files (Small DA+TA). It differs from Message Set analysis pattern as a predicate to group them may not be easily available.

Such Defect Groups can be results of previous analyses activities. The name of the analysis pattern came from representation theory defect group of a block but at present, it is only name analogy.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

When interpreting a value as a pointer to a memory address we are usually interested in adjacent values pointed to:

0:000> ? rdx
Evaluate expression: 982485297288 = 000000e4`c0afe888

0:000> ? poi(000000e4`c0afe888)
Evaluate expression: 180 = 00000000`000000b4

0:000> ? poi(rdx)
Evaluate expression: 180 = 00000000`000000b4

0:000> dps rdx-10 rdx+10
000000e4`c0afe878 000000e4`c0afeac0
000000e4`c0afe880 000000e4`c0971000
000000e4`c0afe888 00000000`000000b4
000000e4`c0afe890 00000000`0000008c
000000e4`c0afe898 00000000`00000088

0:000> r $t0 = 0

0:000> dps rdx-@$t0 rdx+@$t0
000000e4`c0afe888 00000000`000000b4

0:000> r $t0 = 20

0:000> dps rdx-@$t0 rdx+@$t0
000000e4`c0afe868 00000000`00000001
000000e4`c0afe870 00001a38`00000001
000000e4`c0afe878 000000e4`c0afeac0
000000e4`c0afe880 000000e4`c0971000
000000e4`c0afe888 00000000`000000b4
000000e4`c0afe890 00000000`0000008c
000000e4`c0afe898 00000000`00000088
000000e4`c0afe8a0 00000000`00000088
000000e4`c0afe8a8 00000000`0000008c

We call this analysis pattern Pointer Cone by analogy with cones in category theory and our earlier attempts to use it. The reason for this pattern appearance is that it has been used in many times in other analysis pattern descriptions. We also intend to use this pattern language building block in our next analysis pattern.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -