Trace, Log, Text, Narrative, Data: An Analysis Pattern Reference for Information Mining, Diagnostics, Anomaly Detection

The Fifth Edition is available in PDF format from Software Diagnostics Services

Other Fifth Edition links:

Buy PDF from Leanpub

The first edition is also available for SkillSoft Books24x7 subscribers

General trace and log analysis patterns allow the application of uniform diagnostics and anomaly detection across diverse software environments. This pattern language covers any execution artifact from a small debugging trace to a distributed log with billions of messages from hundreds of computers, thousands of software components, threads, and processes. Pattern-oriented trace and log analysis is applicable to troubleshooting and debugging Windows, macOS, Linux, FreeBSD, Android, iOS, z/OS, and any other possible computer platform, including networking and IoT. Its pattern catalog is a part of pattern-oriented software data analysis, diagnostics, anomaly detection, forensics, prognostics, root cause analysis, and debugging developed by Software Diagnostics Institute. Also, the scope of applicability of such analysis patterns is much wider than just software execution artifacts or temporal data and now includes general data, narratives, text, and image analysis (space-like narratology). This reference reprints with corrections almost 230 patterns originally published in Memory Dump Analysis Anthology volumes 3 - 14 and Software Diagnostics Library. It also includes additional 19 analysis patterns from the forthcoming volume 15, bringing the total analysis pattern count to 229. Full-color diagrams accompany almost all pattern descriptions. The fifth edition includes 28 more patterns, updated diagram pictures, classification, and the list of narratological and mathematical influences, and now includes the long-awaited introduction and two new appendixes.

Product information:

  • Title: Trace, Log, Text, Narrative, Data: An Analysis Pattern Reference for Information Mining, Diagnostics, Anomaly Detection, Fifth Edition
  • Authors: Dmitry Vostokov, Software Diagnostics Institute
  • Language: English
  • Product Dimensions: 21.6 x 14.0
  • Paperback: 400 pages
  • Publisher: OpenTask (March 2023)
  • ISBN-13: 978-1912636587

Table of Contents
Bird's-eye View of Pages

Diagnostics of Artificial Intelligence

If wrong labels are ML bugs, then wrong answers are ML crashes.

We propose to define Diagnostics of AI as a combination of ideas and approaches from different disciplines such as software engineering, computer science, mathematics, physics, chemistry, biology, literary theory and criticism, narratology, semiotics, humanities, sociology, medicine, forensic science, psychology, psychiatry, and psychopathology with the following main directions of our research activities:

It is also important to differentiate diagnostics of artificial intelligence from diagnostics of artificial intelligence systems which is just software and hardware diagnostics, where diagnostics of AI may be just a part. Diagnostics of AI systems is metaphorically like medicine, whereas diagnostics of AI is like psychiatry and psychopathology.

We also organized a few groups for information sharing, such as relevant books and articles:

LinkedIn Diagnostics of Artificial Intelligence group
Facebook Diagnostics of Artificial Intelligence group

Traces and Logs as 2-categories

In the past, we looked at software traces and logs as semigroups or monoids as a single object category. In the latter case, the monoid object is a trace or log, and arrows (morphisms) are trace and log messages.

Online Training: Accelerated Linux API for Software Diagnostics

Software Diagnostics Services organizes this online training course.

New dates/times TBD

For the approximate agenda please check slides from the similar Windows training

Knowledge of Windows API is necessary for:

  • Development
  • Malware analysis
  • Vulnerability analysis and exploitation
  • Reversing
  • Diagnostics
  • Debugging
  • Memory forensics
  • Core dump analysis
  • Secure coding
  • Static code analysis
  • Trace and log analysis

The training uses a unique and innovative pattern-oriented analysis approach and provides:

  • Overview
  • Classification
  • Patterns
  • Internals
  • Development examples
  • Analysis examples
  • Comparison with Windows API

Before the training you get:

  • Access to Software Diagnostics Library

After the training, you also get:

  • The PDF book version of the training
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • Recording

Book: Accelerated Linux Core Dump Analysis, Third Edition

Available in PDF format from Software Diagnostics Technology and Services.

The full transcript of Software Diagnostics Services training. Learn how to analyze Linux process and kernel crashes and hangs, navigate through core memory dump space and diagnose corruption, memory leaks, CPU spikes, blocked threads, deadlocks, wait chains, and much more. This training uses a unique and innovative pattern-oriented diagnostic analysis approach to speed up the learning curve. The training consists of 47 practical step-by-step exercises using GDB and WinDbg debuggers, highlighting almost 40 memory analysis patterns diagnosed in 64-bit core memory dumps from x64 and ARM64 platforms. The training also includes source code of modeling applications, a catalog of relevant patterns from the Software Diagnostics Institute, and an overview of relevant similarities and differences between Windows and Linux memory dump analysis useful for engineers with a Wintel background. In addition to various improvements, the third edition includes a review of relevant x64 and ARM64 disassembly and a new set of ARM64 GDB exercises.

  • Title: Accelerated Linux Core Dump Analysis: Training Course Transcript with GDB and WinDbg Practice Exercises, Third Edition
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (January 2023)
  • Language: English
  • PDF: 637 pages
  • ISBN-13: 978-1912636594

Table of Contents and Sample Exercise

Online Training: Accelerated Linux Disassembly, Reconstruction, and Reversing

Accelerated Disassembly, Reconstruction and Reversing Logo

Software Diagnostics Services (PatternDiagnostics.com) organizes a training course.

Learn disassembly, execution history reconstruction, and binary reversing techniques for better software diagnostics, troubleshooting, debugging, memory forensics, vulnerability and malware analysis on x64 and ARM64 Linux platforms. The course uses a unique and innovative pattern language approach to speed up the learning curve. The training consists of practical step-by-step, hands-on exercises using GDB and Linux core memory dumps. Covered more than 25 ADDR patterns originally introduced for the x64 Windows platform, and many concepts are illustrated with Memory Cell Diagrams. This new training version includes a review of necessary x64 and ARM64 assembly language fundamentals.

Slides from the previous training

Level: Intermediate/Advanced.

Prerequisites: Working knowledge of C and C++. Operating system internals and assembly language concepts are explained when necessary.

Audience: Software technical support and escalation engineers who analyze core dumps from complex software environments and need to go deeper in their analysis of abnormal and malicious software structure and behavior. The course is also useful for software engineers, quality assurance and software maintenance engineers who debug software running on diverse cloud and endpoint computer environments, SRE and DevSecOps, security and vulnerability researchers, malware and memory forensics analysts who have never used GDB for analysis of computer memory.

The training consists of 4 two-hour sessions.

Before the training, you get the following:

  • The current PDF book version
  • The previous training recording
  • Access to Software Diagnostics Library

After the training, you also get the following:

  • The updated PDF book version of the training
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • The new recording

Online Training: Accelerated Linux Core Dump Analysis

Software Diagnostics Services organizes this online training course.

Learn how to analyze Linux process and kernel crashes and hangs, navigate through core memory dump space and diagnose corruption, memory leaks, CPU spikes, blocked threads, deadlocks, wait chains, and much more. This training uses a unique and innovative pattern-oriented diagnostic analysis approach to speed up the learning curve. The training consists of more than 20 practical step-by-step exercises using GDB and WinDbg debuggers, highlighting more than 50 memory analysis patterns diagnosed in 64-bit core memory dumps from x64 and ARM64 platforms. The training also includes source code of modeling applications, a catalog of relevant patterns from Software Diagnostics Institute, and an overview of relevant similarities and differences between Windows and Linux memory dump analysis useful for engineers with Wintel background. The training is based on the 2nd revised and extended edition of the bestselling Accelerated Linux Core Dump Analysis book. There will be additional material added related to x64 and ARM64 disassembly.

Prerequisites: Basic Linux user skills.

Audience: Software technical support and escalation engineers, system administrators, security researchers, reverse engineers, malware and memory forensics analysts, software developers, DevSecOps and SRE, and quality assurance engineers.

Slides from the training

Training outline:

  • Day 1: Overview. Process core dump analysis.
  • Day 2: Process core dump analysis.
  • Day 3: Kernel core dump analysis.
  • Before the training you get:

    • The current PDF book version of the training.
    • Access to Software Diagnostics Library.
    • Past recording.

    After the training, you also get:

    • The new 3rd edition PDF book version of the training.
    • Personalized Certificate of Attendance with unique CID.
    • Optional Personalized Certificate of Completion with unique CID (after the tests).
    • Answers to questions during training sessions.
    • New recording.

Dia|gram Language and Memory Dump Analysis Patterns

We illustrate trace and log analysis patterns using a graphical language named Dia|gram. When we analyze memory dumps, we get textual log output to which we can apply the same analysis patterns. Therefore, we can reuse the same Dia|gram language for memory dump analysis, for example, for analysis pattern illustrations. We start with the first memory dump analysis pattern, Multiple Exceptions. However, we now use some later analysis pattern names in the latter's description, for example, Stack Trace Collection from multiple threads, which consists of Stack Traces from individual threads.

Each individual thread stack trace can be illustrated using this trace diagram with an implicit time arrow but without time values:



If we knew individual stack trace frame timings, we could have assembled the normal trace diagram:

However, we can assemble the combined trace by gluing individual thread traces and sorting them by thread creation time (available in Windows memory dumps):

Additionally, we can impose sparse discrete time labels and structure via Trace Schema:

Having ATIDs, we can use the Adjoint Thread of Activity analysis pattern.

Since we know that top frames existed at the time of the memory snapshot, we can also group them at the end of the trace:

Exception indicators (like exception processing frames) can be highlighted as Error Message and Periodic Error. We can also use Exception Stack Trace:

We can also add other debugger log output to such diagrams for Intra-Correlation.

We now start using Dia|gram in our memory dump analysis training courses and reference materials.

Book: Accelerated macOS Core Dump Analysis, Third Edition

Available for sale in PDF format from Software Diagnostics Services.

The full transcript of Software Diagnostics Services training with 12 step-by-step exercises. Learn how to analyze app crashes and freezes, navigate through process core memory dump space and diagnose corruption, memory leaks, CPU spikes, blocked threads, deadlocks, wait chains, and much more. We use a unique and innovative pattern-driven analysis approach to speed up the learning curve. The training consists of practical step-by-step exercises using Xcode and LLDB environments, highlighting more than 30 analysis patterns from Software Diagnostics Institute diagnosed in ARM64 process core memory dumps. The training also includes an overview of relevant similarities and differences between Windows and macOS user space memory dump analysis useful for engineers with a Wintel background and the relevant ARM64 disassembly tutorial. The course is thoroughly updated for the latest macOS version and M2 platform. The primary audience for this training is software technical support and escalation engineers who analyze crash reports and memory dumps, quality assurance and software engineers who test and debug macOS software, security and vulnerability researchers, and malware and memory forensics analysts who have never used LLDB for the analysis of computer memory.

  • Title: Accelerated macOS Core Dump Analysis, Third Edition: Training Course Transcript with LLDB Practice Exercises
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (December 2022)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • PDF: 250 pages
  • ISBN-13: 978-1912636754

Table of Contents and Sample Exercise
Slides from the training

Book: Accelerated Windows API for Software Diagnostics

Available in PDF format from Software Diagnostics Services.

The book contains the full transcript of Software Diagnostics Services training with 10 hands-on exercises.

Knowledge of Windows API is necessary for:

  • Development
  • Malware analysis
  • Vulnerability analysis and exploitation
  • Reversing
  • Diagnostics
  • Debugging
  • Memory forensics
  • Crash and hang analysis
  • Secure coding
  • Static code analysis
  • Trace and log analysis

The training uses a unique and innovative pattern-oriented analysis approach and provides:

  • Overview
  • Classification
  • Patterns
  • Internals
  • Development examples
  • Analysis examples

  • Title: Accelerated Windows API for Software Diagnostics: With Category Theory in View
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (December 2022)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • PDF: 305 pages
  • ISBN-13: 978-1912636631

Table of Contents and sample exercise
Slides from the training

Online Training: Accelerated macOS (M2) Disassembly, Reconstruction, and Reversing

Accelerated macOS (M2) Disassembly, Reconstruction and Reversing Logo

Software Diagnostics Services (PatternDiagnostics.com) organizes a training course.

New dates/times TBD

Learn disassembly, execution history reconstruction, and binary reversing techniques for better software diagnostics, troubleshooting, debugging, memory forensics, vulnerability and malware analysis on the ARM64 macOS platform. The course uses a unique and innovative pattern language approach to speed up the learning curve. The training consists of practical step-by-step, hands-on exercises using LLDB and macOS core memory dumps. Covered more than 25 ADDR patterns originally introduced for the x64 Windows platform and later expanded to x64 and ARM64 Linux, and many concepts are illustrated with Memory Cell Diagrams. The course builds upon and extends the basic patterns introduced in Practical Foundations of macOS Debugging, Disassembling, Reversing book.

Slides from the simialr Linux-based training

Level: Intermediate/Advanced.

Prerequisites: Working knowledge of C and C++. Operating system internals and assembly language concepts are explained when necessary.

Audience: Software technical support and escalation engineers who analyze core dumps from complex software environments and need to go deeper in their analysis of abnormal and malicious software structure and behavior. The course is also useful for software engineers, quality assurance and software maintenance engineers who debug software running on diverse endpoint computer environments, security and vulnerability researchers, malware and memory forensics analysts who have never used LLDB for analysis of computer memory.

The training consists of 3 two-hour sessions. Before the training, you get:

Before the training, you get:

  • Access to Software Diagnostics Library
  • Practical Foundations of macOS Debugging, Disassembling, Reversing PDF book (January 2023)

After the training, you also get:

  • The PDF book version of the training
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • Recording

Training: Accelerated macOS Core Dump Analysis (M2 Version)

Accelerated Mac OS X Core Dump Analysis Logo

Software Diagnostics Services (PatternDiagnostics.com) organizes a training course:

New dates/times TBD

Learn how to analyze app crashes and freezes, navigate through process core memory dump space and diagnose corruption, memory leaks, CPU spikes, blocked threads, deadlocks, wait chains, and much more. We use a unique and innovative pattern-driven analysis approach to speed up the learning curve. The training consists of practical step-by-step exercises using Xcode and LLDB environments, highlighting more than 30 patterns diagnosed in 64-bit process core memory dumps. The training also includes an overview of relevant similarities and differences between Windows and Mac OS X user space memory dump analysis useful for engineers with a Wintel background. The course is thoroughly updated for the latest macOS version and M2 platform.

Slides from the previous x64-based training

Level: Beginner/Intermediate.

Prerequisites: Prerequisites: Basic macOS troubleshooting and debugging.

Audience: Audience: Software technical support and escalation engineers, system administrators, software developers, security professionals, and quality assurance engineers.

The training consists of 2 two-hour sessions. Before the training, you get:

  • The previous PDF edition of this course (Intel x64 LLDB and GDB)
  • Access to Software Diagnostics Library

After the training, you also get:

  • The new PDF book version of the training
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • Recording
  • Practical Foundations of macOS Debugging, Disassembling, Reversing PDF book (January 2023)

Online Training: Accelerated Windows Trace and Log Analysis

Software Diagnostics Services organizes this online training course.

TBD

Accelerated Software Trace Analysis Logo

Feel frustrated when opening a software trace with millions of messages from hundreds of software components, threads, and processes? Go beyond simple CPU and disk hog monitoring or searching for errors in a text and learn how to efficiently and effectively analyze software traces and logs from complex software environments. In addition to a theoretical part, practical illustrations, examples, and exercises include Microsoft Event Tracing for Windows (ETW), Procmon, Windows Performance Analyzer, and PerfView. This course teaches trace and log analysis using pioneering and innovative pattern-oriented analysis of abnormal software behavior incidents developed by Software Diagnostics Institute.

Sample slides from a theoretical part

The training consists of 5 one-hour sessions. Before the training, you get:

  1. The current version of Malware Narratives (PDF).
  2. Trace, Log, Text, Narrative: An Analysis Pattern Reference for Data Mining, Diagnostics, Anomaly Detection, Fourth Edition (PDF).
  3. Access to Software Diagnostics Library.

After the training, you also get:

  1. The revised edition of Malware Narratives (PDF).
  2. The new edition of Trace, Log, Text, Narrative (PDF).
  3. Personalized Certificate of Attendance with unique CID.
  4. Answers to questions during training sessions.
  5. Recording.

Prerequisites: Basic Windows troubleshooting.

Audience: Software technical support and escalation engineers, system administrators, security researchers, incident response professionals, software developers, platform engineers, DevSecOps and SRE, and quality assurance engineers.

Online Training: Accelerated Windows API for Software Diagnostics

Software Diagnostics Services organizes this online training course.

New dates/times TBD

Slides from the training sessions

Knowledge of Windows API is necessary for:

  • Development
  • Malware analysis
  • Vulnerability analysis and exploitation
  • Reversing
  • Diagnostics
  • Debugging
  • Memory forensics
  • Crash and hang analysis
  • Secure coding
  • Static code analysis
  • Trace and log analysis

The training uses a unique and innovative pattern-oriented analysis approach and provides:

  • Overview
  • Classification
  • Patterns
  • Internals
  • Development examples
  • Analysis examples

Before the training you get:

  • Practical Foundations of Windows Debugging, Disassembling, Reversing, Second Edition PDF book
  • Access to Software Diagnostics Library

After the training, you also get:

  • The PDF book version of the training
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • Recording

Pattern-Oriented Memory Forensics

The following direct links can be used to order the book now:

Buy PDF from Leanpub

Also available in PDF format from Software Diagnostics Services as a part of Accelerated Windows Memory Forensics and Malware Analysis with Memory Dumps training course and Pattern-Oriented Windows Memory Forensics Training Pack

The original edition is available for SkillSoft Books24x7 subscribers

This short book is a fully revised transcript of a lecture introducing a pattern language for memory forensics - an investigation of past software behavior in memory snapshots. It provides a unified language for discussing and communicating detection and analysis results despite the proliferation of operating systems and tools, a base language for checklists, and aid in accelerated learning. The lecture has a short theoretical part and then illustrates various patterns seen in crash dumps by using WinDbg debugger from Microsoft Debugging Tools for Windows.

  • Title: Pattern-Oriented Memory Forensics: A Pattern Language Approach, Revised Edition
  • Author: Dmitry Vostokov, Software Diagnostics Institute, Software Diagnostics Services
  • Publisher: OpenTask (October 2022)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • Paperback: 97 pages
  • ISBN-13: 978-1912636761

Presentation Slides

Book: Advanced Windows Memory Dump Analysis with Data Structures, Fourth Edition, Revised

Available in PDF and paperback format from Software Diagnostics Technology and Services.

The full transcript of Software Diagnostics Services training course with 15 step-by-step exercises, notes, and selected questions and answers. Learn how to navigate through memory dump space and Windows data structures to diagnose, troubleshoot and debug complex software incidents. The training uses a unique and innovative pattern-oriented analysis approach to speed up the learning curve. It consists of practical step-by-step exercises using WinDbg to diagnose structural and behavioral patterns in the 64-bit kernel and complete (physical) memory dumps. Additional topics include memory search, kernel linked list navigation, practical WinDbg scripting, registry, system variables and objects, device drivers, and I/O. Prerequisites are basic and intermediate level Windows memory dump analysis: the ability to list processors, processes, threads, modules, apply symbols, walk through stack traces and raw stack data, diagnose patterns such as heap corruption, CPU spike, memory leaks, access violation, wait chains and deadlocks. If you don't feel comfortable with prerequisites then Accelerated Windows Memory Dump Analysis training book is recommended before purchasing and reading this book course. Audience: Software technical support and escalation engineers, system administrators, security researchers, reverse engineers, malware and memory forensics analysts, software developers, and quality assurance engineers. The 4th edition was fully reworked to use the latest WinDbg and now covers memory dumps from Windows 11. Three new exercises were added and the previous ones now have improved command syntax and color highlighting. This edition also includes a possibility to use a Docker WinDbg image with required symbol files instead of a local Debugging Tools for Windows installation. The current revision 4.1 uses WinDbg Preview for all exercise transcripts.

  • Title: Advanced Windows Memory Dump Analysis with Data Structures: Training Course Transcript and WinDbg Practice Exercises with Notes, Fourth Edition, Revised
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (October 2022)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • Paperback: 293 pages
  • ISBN-13: 978-1912636778

Table of Contents and Sample Exercise

Book: Accelerated .NET Core Memory Dump Analysis, Revised Edition

The following direct links can be used to order the book now:

Available in PDF format from Software Diagnostics Technology and Services.

The full transcript of Software Diagnostics Services training with 9 step-by-step exercises, notes, and source code of specially created modeling applications. The course covers 19 .NET memory dump analysis patterns plus additional 19 unmanaged patterns. Learn how to analyze .NET Core 5/6 application and service crashes and freezes, navigate through memory dump space (managed and unmanaged code) and diagnose corruption, leaks, CPU spikes, blocked threads, deadlocks, wait chains, resource contention, and much more. The training consists of practical step-by-step exercises using Microsoft WinDbg debugger to diagnose patterns in 64-bit process memory dumps. The training uses a unique and innovative pattern-oriented analysis approach to speed up the learning curve. The book is based on the previous fourth edition of Accelerated .NET Memory Dump Analysis that covered .NET Core 5 and Windows 10. It is updated for the latest WinDbg from Windows 11 SDK and has a new .NET Core 6 exercise with a memory dump from Windows 11. This edition also includes a possibility to use a Docker WinDbg image with required symbol files instead of a local Debugging Tools for Windows installation. Prerequisites: Basic .NET programming and debugging. Audience: Software technical support and escalation engineers, system administrators, DevOps, performance and reliability engineers, software developers, and quality assurance engineers. The book may also interest security researchers, reverse engineers, malware and memory forensics analysts. The revised edition uses the latest WinDbg Preview for all exercise transcripts.

  • Title: Accelerated .NET Core Memory Dump Analysis, Revised Edition: Training Course Transcript and WinDbg Practice Exercises
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (October 2022)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • PDF: 203 pages
  • ISBN-13: 978-1912636648

Table of Contents and Sample Exercise

Online Training: Accelerated Linux Disassembly, Reconstruction, and Reversing (WinDbg Version)

Accelerated Disassembly, Reconstruction and Reversing Logo

Software Diagnostics Services (PatternDiagnostics.com) organizes a training course.

New dates/times TBD

Learn disassembly, execution history reconstruction, and binary reversing techniques for better software diagnostics, troubleshooting, debugging, memory forensics, vulnerability and malware analysis on x64 and ARM64 Linux platforms. The course uses a unique and innovative pattern language approach to speed up the learning curve. The training consists of practical step-by-step, hands-on exercises using WinDbg and Linux core memory dumps. Covered more than 25 ADDR patterns originally introduced for the x64 Windows platform, and many concepts are illustrated with Memory Cell Diagrams. The training also features additional diagrams adapted from Linux Practical Foundations books to WinDbg context.

Selected slides of the previous Windows-based training

Level: Intermediate/Advanced.

Prerequisites: Working knowledge of C and C++. Operating system internals and assembly language concepts are explained when necessary.

Audience: Software technical support and escalation engineers who analyze core dumps from complex software environments and need to go deeper in their analysis of abnormal and malicious software structure and behavior. The course is also useful for software engineers, quality assurance and software maintenance engineers who debug software running on diverse cloud and endpoint computer environments, SRE and DevSecOps, security and vulnerability researchers, malware and memory forensics analysts who have never used WinDbg for analysis of computer memory.

The training consists of 3 two-hour sessions.

Before the training, you get:

  • Access to Software Diagnostics Library

After the training, you also get:

  • The PDF book version of the training
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • Recording

Book: Extended Windows Memory Dump Analysis

Available in PDF format from Software Diagnostics Services.

The book contains the full transcript of Software Diagnostics Services training with 16 hands-on exercises. This training course extends pattern-oriented analysis introduced in Accelerated Windows Memory Dump Analysis, Accelerated .NET Core Memory Dump Analysis, and Advanced Windows Memory Dump Analysis with Data Structures courses with:

  • Surveying the current landscape of WinDbg extensions with analysis pattern mappings
  • Writing WinDbg extensions in C and C++
  • Connecting WinDbg to NoSQL databases
  • Connecting WinDbg to streaming and log processing platforms
  • Querying and visualizing WinDbg output data

Prerequisites: Working knowledge of WinDbg. Working knowledge of C or C++ is optional (required only for some exercises). Other concepts are explained when necessary.

Audience: Software developers, software maintenance engineers, escalation engineers, quality assurance engineers, security and vulnerability researchers, malware and memory forensics analysts who want to build memory analysis pipelines.

  • Title: Extended Windows Memory Dump Analysis: Using and Writing WinDbg Extensions, Database and Event Stream Processing, Visualization
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (September 2022)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • PDF: 274 pages
  • ISBN-13: 978-1912636686

Table of Contents and sample exercise
Slides from the training

Online Training: Accelerated Linux Debugging 4D

Software Diagnostics Services organizes this online training course.

May 21 - 23, 2024, 7:00 pm - 8:30 pm (GMT+1) Price 99 USD Registration

Learn live local and remote debugging techniques and tricks in the kernel and user process spaces using GDB and LLDB debuggers for C, C++, and Rust code. The unique and innovative Debugging 4D course teaches unified debugging patterns applied to real problems from complex software environments. The training consists of practical, step-by-step, hands-on exercises.

Before the training, you get:

  • Access to Software Diagnostics Library

After the training, you also get:

  • The PDF book version of the training: Accelerated Linux Debugging 4D
  • Personalized Certificate of Attendance with unique CID
  • Answers to questions during training sessions
  • Recording

Prerequisites:

Working knowledge of one of these languages: C, C++, Rust. Operating system internals and assembly language concepts are explained when necessary.

Audience:

Software engineers, software maintenance engineers, escalation engineers, security and vulnerability researchers, malware and memory forensics analysts who want to learn live memory inspection techniques.

If you are interested in live Windows debugging, there is another training course available.

Online Training: Accelerated Windows Memory Forensics and Malware Analysis with Memory Dumps

Software Diagnostics Services organizes this online training course.

New dates/times TBD

Accelerated Windows Memory Forensics Logo

Learn how to navigate the process, kernel, physical memory spaces, and corresponding Windows data structures, discover forensic artifacts and diagnose structural and behavioral patterns in Windows memory dump files. The course uses a unique and innovative pattern-oriented analysis approach to speed up the learning curve. The training consists of more than 20 practical step-by-step, hands-on exercises using WinDbg, process, kernel, and complete memory dumps. In addition to malware patterns, topics include process and thread navigation, past execution, memory search, kernel linked list navigation, practical WinDbg scripting including built-in language and JavaScript, registry, system variables and objects, device drivers, I/O, file system filters, and security. The training is based on the Pattern-Oriented Memory Forensics: A Pattern Language Approach, the 3rd edition of Accelerated Windows Malware Analysis with Memory Dumps, and the 4th edition of Advanced Windows Memory Dump Analysis with Data Structures books. This course also covers patterns of memory acquisition. It uses the latest WinDbg Preview and is optionally containerized.

Example slides for days 1-3
Example slides for days 4-5

Before the training, you get:

  • Practical Foundations of Windows Debugging, Disassembling, Reversing, Second Edition PDF book
  • Pattern-Oriented Memory Forensics: A Pattern Language Approach PDF book
  • Advanced Windows Memory Dump Analysis with Data Structures, Fourth Edition PDF book
  • Accelerated Windows Malware Analysis with Memory Dumps, Third Edition PDF book
  • The previous training recording
  • Access to Software Diagnostics Library with more than 370 cross-referenced patterns of memory dump analysis, their classification, and more than 70 case studies

After the training, you also get:

  • The updated PDF books
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • Current training sessions recording

Prerequisites: Working knowledge of Windows troubleshooting. Operating system internals concepts are explained when necessary.

Audience: Software technical support and escalation engineers, system administrators, security researchers, reverse engineers, malware and memory forensics analysts, software developers, and quality assurance engineers.

Book: Accelerated Windows Malware Analysis with Memory Dumps, Third Edition

Available in PDF format from Software Diagnostics Technology and Services

The first edition is also available for SkillSoft Books24x7 subscribers

The Korean second edition is available from Acorn publisher.

The full transcript of Software Diagnostics Services training. Learn how to navigate process, kernel, and physical spaces and diagnose various malware patterns in Windows memory dump files. The course uses a unique and innovative pattern-oriented analysis approach to speed up the learning curve. The training consists of practical step-by-step hands-on exercises using WinDbg, process, kernel and complete memory dumps. Covered more than 20 malware analysis patterns. The main audience is software technical support and escalation engineers who analyze memory dumps from complex software environments and need to check for possible malware presence in cases of abnormal software behavior. The course will also be useful for software engineers, quality assurance and software maintenance engineers, security researchers, malware and memory forensics analysts who have never used WinDbg for analysis of computer memory. The third edition uses the latest WinDbg Preview version with some exercises updated to Windows 11 and is optionally containerized.

  • Title: Accelerated Windows Malware Analysis with Memory Dumps: Training Course Transcript and WinDbg Practice Exercises, Third Edition
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (July 2022)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • PDF: 324 pages
  • ISBN-13: 978-1912636969

Table of Contents

Online Training: Accelerated Windows Postmortem Diagnostics and Debugging

Software Diagnostics Services organizes this online training course.

Accelerated Windows Postmortem Diagnostics and Debugging Logo

This comprehensive training includes more than 40 step-by-step exercises and covers more than 85 crash dump analysis patterns from x86 and x64 process, kernel, and complete (physical) memory dumps. Learn how to analyze application (native and .NET Core), service, and system crashes and freezes, navigate through memory dump space (managed and unmanaged code) and diagnose corruption, memory and handle leaks, CPU spikes, blocked threads, deadlocks, wait chains, resource contention, and much more with WinDbg debugger. The training uses a unique and innovative pattern-oriented analysis approach developed by Software Diagnostics Institute< to speed up the learning curve, and it is based on the latest edition of Accelerated Windows Memory Dump Analysis and Accelerated .NET Core Memory Dump Analysis books. It uses the latest WinDbg Preview and is optionally containerized.

Outline slides
Slides from Days 1-3
Slides from Days 4-6
Slides from Days 7-8

The difference between this training and the current book version:

  • You can ask questions
  • .NET Core exercises use the latest WinDbg Preview
  • Certificates and tests

Training outline:

  • Day 1 (2 hours): Overview. Native process memory dump analysis.
  • Day 2 (2 hours): Native process memory dump analysis.
  • Day 3 (2 hours): Native process memory dump analysis.
  • Day 4 (2 hours): .NET Core process memory dump analysis.
  • Day 5 (2 hours): .NET Core process memory dump analysis.
  • Day 6 (2 hours). Kernel memory dump analysis.
  • Day 7 (2 hours). Complete (physical) memory dump analysis.
  • Day 8 (Optional 2 hours): Additional Q&A and memory dump analysis if necessary. Tests.

Before the training, you get:

  • Practical Foundations of Windows Debugging, Disassembling, Reversing, Second Edition PDF book (+300 pages)
  • The current PDF books (+900 pages)
  • The previous training recording
  • Access to Software Diagnostics Library with more than 370 cross-referenced patterns of memory dump analysis, their classification, and more than 70 case studies

After the training, you also get:

  • The updated PDF books (including the new edition of .NET Core book)
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • Current training sessions recording

Prerequisites: Basic Windows troubleshooting

Audience: Software technical support and escalation engineers, system administrators, security researchers, reverse engineers, malware and memory forensics analysts, software developers, and quality assurance engineers.

If you are interested in Linux memory dump analysis there is another forthcoming training: Accelerated Linux Core Dump Analysis

Training: Accelerated Windows Malware Analysis with Memory Dumps

Software Diagnostics Services organizes this online training course.

Accelerated Windows Malware Analysis Logo

New dates/times TBD

Learn how to navigate process, kernel, and physical spaces and diagnose various malware patterns in Windows memory dump files. The course uses a unique and innovative pattern-oriented analysis approach to speed up the learning curve. The training consists of practical step-by-step, hands-on exercises using WinDbg, process, kernel, and complete memory dumps. The training covers more than 20 malware analysis patterns. The main audience is software technical support and escalation engineers who analyze memory dumps from complex software environments and need to check for possible malware presence in cases of abnormal software behavior. The course will also be useful for software engineers, quality assurance and software maintenance engineers, security researchers, malware and memory forensics analysts who have never used WinDbg for analysis of computer memory. The new version uses the latest WinDbg Preview and is optionally containerized.

Slides from the previous training

Before the training, you get:

  • Practical Foundations of Windows Debugging, Disassembling, Reversing, Second Edition PDF book
  • The previous PDF book version of the training
  • Access to Software Diagnostics Library

After the training, you also get:

  • The new 3rd edition PDF book of the training
  • Personalized Certificate of Attendance with unique CID
  • Optional Personalized Certificate of Completion with unique CID (after the tests)
  • Answers to questions during training sessions
  • Recording

Book: Practical Foundations of Windows Debugging, Disassembling, Reversing, Second Edition

Available in PDF format from Software Diagnostics Services

This training course is a combined, reformatted, improved, and modernized version of the two previous books Windows Debugging: Practical Foundations and x64 Windows Debugging: Practical Foundations, that drew inspiration from the original lectures we developed almost 18 years ago to train support and escalation engineers in debugging and crash dump analysis of memory dumps from Windows applications, services, and systems. At that time, when thinking about what material to deliver, we realized that a solid understanding of fundamentals like pointers is needed to analyze stack traces beyond a few WinDbg commands. Therefore, this book is not about bugs or debugging techniques but about the background knowledge everyone needs to start experimenting with WinDbg and learn from practical experience and read other advanced debugging books. This body of knowledge is what the author of this book possessed before starting memory dump analysis using WinDbg 18 years ago, which resulted in the number one debugging bestseller: multi-volume Memory Dump Analysis Anthology. Now, in retrospection, we see these practical foundations as relevant and necessary to acquire for beginners as they were 18 years ago because operating systems internals, assembly language, and compiler architecture haven't changed much in those years.

The book contains two separate sets of chapters and corresponding illustrations. They are named Chapter x86.NN and Chapter x64.NN respectively. The new format makes switching between and comparing x86 and x64 versions easy. There is some repetition of content due to the shared nature of x64 and x86 platforms. Both sets of chapters can be read independently. We included x86 chapters because many 3rd-party Windows applications are still 32-bit and executed in 32-bit compatibility mode on x64 Windows systems. The course consistently uses WinDbg (X86) for 32-bit examples and WinDbg (X64) for 64-bit examples. The book also has a larger format similar to other training courses from Software Diagnostics Services.

Almost 5 years have passed since the first edition of the combined training course that used the earlier version of Windows 10. Since then, we have also published "Practical Foundations of Linux Debugging, Disassembling, Reversing" and "Practical Foundations of ARM64 Linux Debugging, Disassembling, Reversing" books. At that time, we thought about revising our Windows course. Since then, Windows 11 appeared, and we also added Docker support for most of our Windows memory dump analysis courses. While working on the "Accelerated Windows Debugging 4D "course, we decided to make the second edition of Practical Foundations of Windows Debugging based on WinDbg from Windows 11 SDK and Visual Studio 2022 build tools and an optional Docker support for the exercise environment. We also changed the ":=" operator to "<-" in our pseudo-code for Intel disassembly syntax flavor to align with our recent Linux Practical Foundations books, which use "->" in pseudo-code for x64 AT&T disassembly syntax flavor and "<-" in pseudo-code for ARM64 disassembly syntax. All sample projects were recompiled, and many diagrams were redone for the new edition to reflect changes in code generation. WinDbg syntax and code highlighting were also improved. There are also minor additions for C++11 and C++20.

The book is useful for:

  • Software technical support and escalation engineers
  • Software engineers coming from managed code or JVM background
  • Software testers
  • Engineers coming from non-Wintel environments
  • Windows C/C++ software engineers without assembly language background
  • Security researchers without x86/x64 assembly language background
  • Beginners learning Windows software reverse engineering techniques

This introductory training course can complement the more advanced course Accelerated Disassembly, Reconstruction and Reversing, Revised Edition. It may also help with advanced exercises in Accelerated Windows Memory Dump Analysis, Fifth Edition, Part 1, Revised, Process User Space and Accelerated Windows Memory Dump Analysis, Fifth Edition, Part 2, Revised, Kernel and Complete Spaces. This book can also be used as an Intel assembly language and Windows debugging supplement for relevant undergraduate-level courses.

Product information:

  • Title: Practical Foundations of Windows Debugging, Disassembling, Reversing: Training Course, Second Edition
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • Paperback: 338 pages
  • Publisher: OpenTask (April 2022)
  • ISBN-13: 978-1-912636-35-8

Table of Contents

REPL Streaming (REPLS)

When we interact with a debugger, we use the so-called REPL, Read-Execute-Print-Loop (where we replace Eval in traditional REPL). We type some command, it is executed by a debugger engine against a memory dump, the results are printed, and we repeat. However, the results can be streamed for further processing, for example, to automate certain analysis patterns, for example, Structure Sheaves (as distributed logs), Region Profiles, and Region Clusters (we list recently added analysis patterns). In the background, the stream processing results are provided back to the debugger Print phase (or in parallel as a further diagnostic aid).

In the case of software traces and logs, streaming is a natural part of processing. But CoTraces fit into REPLS (processed results can also form Message Annotations), and generally, Diags.

REPLS is an architectural pattern, and it is now added to our catalog of software diagnostics architecture patterns.

Systematic Software Diagnostics


Systematic Software Diagnostics attempts to unify various disorganized and fragmentary individual software diagnostic approaches for software construction and post-construction phases.

Initially, when working on software diagnostics foundations, we recognized the need for systematicity by including some of our books and training courses in the Systematic Software Fault Analysis Series. Over the following years, our many practical books became supplemented by theory and a series of seminars.

Now, after more than 15 years, coherent and complete theoretical, practical, and factual knowledge is systematically unified and ordered into pattern catalogs according to first principles after being integrated across individual observations as fully as possible at this time of the discipline development. In addition to systematicity, software diagnostics is also highly interdisciplinary and systemic, crossing boundaries of other disciplines. Its fundamental methodology is a pattern-oriented analysis of artifacts. Every artifact is considered a trace, log, text, and narrative. Diagnostic analysis patterns, common recurrent analysis techniques and methods in specific contexts, organized into catalogs, are used to identify structural and behavioral patterns, common recurrent problems (sets of indicators, symptoms, signs) together with recommendations and possible solutions to apply in specific contexts.

Consider, for example, a typical diagnostic procedure called measurement. It is an analysis pattern itself applied to artifacts or used to generate other artifacts to which analysis patterns are applied. Systematic Software Diagnostics is equally applicable to software development processes where the same analysis patterns are applied to development artifacts, repositories, documentation, management operations, monitoring, team structure, and dynamics. Even a diagnostic analysis is considered an artifact to apply analysis patterns. Where Systematic Software Diagnostic overlaps with other development and engineering activities, it offers additional pattern languages for software data analysis, troubleshooting, debugging, root cause analysis, performance analysis, writing tools, software and memory forensics, memory dump analysis, network trace analysis, static and dynamic malware analysis, reversing, vulnerability analysis, software internals, and cloud computing.

Book: Practical Foundations of ARM64 Linux Debugging, Disassembling, Reversing

This training course is a Linux ARM64 (A64) version of the previous Practical Foundations of Linux Debugging, Disassembly, Reversing book. It also complements Accelerated Linux Core Dump Analysis training course.

The book skeleton is the same as its x64 Linux predecessor, but the content was revised entirely because of a different Linux distribution and CPU architecture.

The course is useful for:

  • Software support and escalation engineers, cloud security engineers, SRE, and DevSecOps
  • Software engineers coming from JVM background
  • Software testers
  • Engineers coming from non-Linux environments, for example, Windows or Mac OS X
  • Engineers coming from non-ARM environments, for example, x86/x64
  • Linux C/C++ software engineers without assembly language background
  • Security researchers without assembly language background
  • Beginners learning Linux software reverse engineering techniques

This book can also be used as an ARM64 assembly language and Linux debugging supplement for relevant undergraduate-level courses.

Product information:

  • Title: Practical Foundations of ARM64 Linux Debugging, Disassembling, Reversing: Training Course
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • PDF: 176 pages
  • Publisher: OpenTask (January 2022)
  • ISBN-13: 978-1-912636-37-2

Table of Contents

Book: Accelerated Windows Memory Dump Analysis, Fifth Edition, Revised

The following direct links can be used to order the book now:

Buy PDF from Leanpub

Also available for sale in PDF format from Software Diagnostics Services.

The second edition is available for SkillSoft Books24x7 subscribers

The full-color transcript of Software Diagnostics Services training sessions with 32 step-by-step exercises, notes, source code of specially created modeling applications, and more than 120 questions and answers. Covers more than 65 crash dump analysis patterns from x86 and x64 process, kernel, and complete (physical) memory dumps. Learn how to analyze application, service and system crashes and freezes, navigate through memory dump space and diagnose heap corruption, memory leaks, CPU spikes, blocked threads, deadlocks, wait chains, and much more. The training uses a unique and innovative pattern-oriented analysis approach developed by Software Diagnostics Institute to speed up the learning curve. Prerequisites: Basic Windows troubleshooting. Audience: Software technical support and escalation engineers, system administrators, security researchers, reverse engineers, malware and memory forensics analysts, software developers, and quality assurance engineers. The 5th edition was fully reworked with new memory dumps, additional slides, exercises, and analysis patterns. It was further revised with some exercises updated to Windows 11, expanded Q&A, and optional Docker image.

The course consists of two parts:

  • Title: Accelerated Windows Memory Dump Analysis, Fifth Edition, Part 1, Revised, Process User Space: Training Course Transcript and WinDbg Practice Exercises with Notes
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (December 2021)
  • Language: English
  • PDF: 410 pages
  • ISBN-13: 978-1912636051
  • Table of Contents
  • Title: Accelerated Windows Memory Dump Analysis, Fifth Edition, Part 2, Revised, Kernel and Complete Spaces: Training Course Transcript and WinDbg Practice Exercises with Notes
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (December 2021)
  • Language: English
  • PDF: 370 pages
  • ISBN-13: 978-1912636082
  • Table of Contents

Accelerated Software Trace Analysis, Revised Edition, Part 1: Fundamentals and Basic Patterns

The following direct links can be used to order the book:

Buy Kindle print replica from Amazon

Buy PDF from Leanpub

Also available in PDF format from Software Diagnostics Technology and Services

This book is a revised edition of the original Accelerated Windows Software Trace Analysis training course. General trace and log analysis pattern language covers any execution artifact from a small debugging trace to a distributed log with billions of messages from hundreds of computers, thousands of software components, threads, and processes. It also allows the application of uniform diagnostics and anomaly detection across diverse software environments, troubleshooting and debugging Windows, Mac OS X, Linux, Android, iOS, and any other possible computer platform including networking and IoT. Part 1 covers fundamentals and explains more than 60 basic trace and log analysis patterns, which are now cross-referenced in this improved and less Windows-centric edition. It can also serve as a reference.

  • Title: Accelerated Software Trace Analysis, Revised Edition, Part 1: Fundamentals and Basic Patterns
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (December 2021)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • Paperback: 110 pages
  • ISBN-13: 978-1912636310

Revised slides from the book

Syndicate content