In its simplest form the first law of troubleshooting and debugging states that:
The more frequent a problem is, the simpler tool is needed to resolve and fix it.
- Dmitry Vostokov @ DumpAnalysis.org -
I’m very pleased to announce that Dan Skwire’s unique book has been published by OpenTask:
First Fault Software Problem Solving: A Guide for Engineers, Managers and Users
- Dmitry Vostokov @ DumpAnalysis.org -
Suppose we want to write a support tool to troubleshoot a frequent software problem. Would we start with GUI? Most tools are command line tools and we therefore choose Tool Façade pattern to decouple GUI from CLI:
By making our GUI optional we postpone its development until later implementation phase when we want to beautify or simplify the tool user interface. The troubleshooting of software problem we want to automate is quite complex, scattered across several knowledge base articles, with many troubleshooting steps and various checks. We therefore treat different steps as TUWs, Troubleshooting Units of Work (aggregating related ones if necessary) and choose Checklist pattern to coordinate them:
Initially we think that just one checklist is suffice but while studying the domain of troubleshooting we find that the problem usually spans client and server computers, where servers sometimes have client roles as well and different troubleshooting steps and checks are needed. In order to simplify the tool we apply Span Differentiator pattern and our architectural UML component diagram becomes more refined:
We also see that different TUW use similar operations like querying registry values or checking whether certain processes are running and factor them out into separate SMs, Supporting Modules:
Deployment methods are important in troubleshooting where it is not always desirable to change configuration on production servers and we plan for optional Self Extractor pattern:
In diagrams we liberally mixed component and static UML diagramming elements. We also need to design a uniform interface that each TUW module exposes to the main tool coordinator module but we leave it for the part 2.
- Dmitry Vostokov @ DumpAnalysis.org -
Support tools can be packaged for delivery as (not counting optional separate readme and EULA files):
The latter common packaging choice is a pattern itself that I call Self Extractor. Typical examples include sysinternals tools like Process Monitor. Here is a simple unorthodox UML diagram for a single platform tool combining elements from static and component diagrams to illustrate component embedding (not sharing):
SM is a supporting module, for example, a driver, to carry out some task or receive notifications.
Some tools even extract different components depending on the processor architecture, for example, x86 and x64. This is illustrated on a more elaborate UML diagram:
The typical example would be a backward compatible x86 Extractor that extracts an x64 driver (Extractee) for an x64 OS kernel that doesn’t load x86 drivers.
- Dmitry Vostokov @ DumpAnalysis.org -
One ETW trace pointed to a set of intermittent symptoms (messages were simplified for this post):
# PID TID Message
[...]
31278 2300 7060 RequestXMLData entry
31281 2300 7060 RequestXMLData: XML error
[...]
Searching for issues having this error only pointed to a case with a mixed software product environment where some servers had the product version X and other servers the product version X+1. However, in the new case the customer claimed that he had only the product version X+1 on all production servers. We insisted and, after the closer inspection, servers with the product X were found…
- Dmitry Vostokov @ TraceAnalysis.org -
The domain of some troubleshooting and debugging tools sometimes spans several computers, for example, a client, a server, and a middleware server (or a frontware server like a web server). Because such tools are small, there is no need to devise yet another distributed architecture but force the tool to differentiate its functionality based on a command line parameters. For example, the following tool implements Checklist pattern and chooses the right checklist from a checklist repository based on a command line option:
c:\> GreatTool -client
c:\> GreatTool -server
This nicely fits with a case when a server is also a client to another server and we need to troubleshoot its client and server functionality on the same physical computer.
- Dmitry Vostokov @ DumpAnalysis.org -
Many troubleshooting tools and their components (like TUW modules from checklists) reuse the same code over and over, for example, querying registry keys or environment variables. It is beneficial to have a package of supporting modules (or a library) for code reuse purposes, hence the name of this pattern, Supporting Module (SM). In the context of Checklist pattern we can refine its UML component diagram to highlight that TUWs are using some shared SM components:
- Dmitry Vostokov @ DumpAnalysis.org -
Memory Analysis and Debugging Institute (MA&DI), DA+TA Portal (DumpAnalysis.org + TraceAnalysis.org) and OpenTask establish
Software Maintenance Institute (SMI)
More details about R&D Institute of Software Maintenance will be announced soon.
- Dmitry Vostokov @ DumpAnalysis.org -
Threads in my process run very fast. Not long ago the watch variable had the value 5 and now it is 6:
switch (years_at_citrix)
{
case 5:
write_blog_post(”I’ve just passed 5 year mark … “);
wait_for_certificate();
write_blog_post(”Shortly after celebrating 5 years … “);
break;
case 6:
write_blog_post(”Threads in my process run very fast. Not long ago … “);
break;
case 7:
// … TBD
}
- Dmitry Vostokov @ DumpAnalysis.org -
To commemorate 200 years since Darwin’s birth and 150 years since the publication of The Origin of Species, OpenTask plans to dedicate December, 2009 issue of Debugged! MZ/PE magazine to taxonomy and evolution of software defects and March, 2010 issue to multithreading (as natural selection).
Debugged! MZ/PE: Systematics and Evolution of Software Defects, December, 2009 (Paperback, ISBN: 978-1906717827)
Debugged! MZ/PE: Multithreading, March, 2010 (Paperback, ISBN: 978-1906717834)
Here are draft front covers designed during last weekend:
- Dmitry Vostokov @ DumpAnalysis.org -
Software installation may fail: this is a fact (like Evolution). Therefore this is the domain of troubleshooting and debugging proper. Usually such problems are typically analyzed by reading Windows Installer MSI logs that are examples of software traces. The following book is on my desk now:
The Definitive Guide to Windows Installer
- Dmitry Vostokov @ DumpAnalysis.org -
Every PID has its twitter account. Processes emit short trace messages (STM) and others subscribe to them. This is the technical support of the future, the concept of SoftWeet (*):
(*) to weet
to know; to wit (archaic)
- Dmitry Vostokov @ DumpAnalysis.org -
This is a revised, edited, cross-referenced and thematically organized volume of selected DumpAnalysis.org blog posts about crash dump analysis and debugging written in October 2008 - June 2009 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software on Windows platforms and technical support and escalation engineers dealing with complex software issues. The third volume features:
- 15 new crash dump analysis patterns
- 29 new pattern interaction case studies
- Trace analysis patterns
- Updated checklist
- Fully cross-referenced with Volume 1 and Volume 2
- New appendixes
Product information:
Back cover features 3D computer memory visualization image.
- Dmitry Vostokov @ DumpAnalysis.org -
DumpAnalysis.org accepts hardware such as laptops for reviewing in relation to their suitability for extreme debugging, virtualization, trace analysis, computer forensics, memory dump analysis, visualization and auralization. If you work for a H/W company like HP, Apple, Dell, Acer, Sony or any other respectable manufacturer please don’t hesitate to forward this post to your management: it could be your company brand or laptop model that debugging and software technical support community chooses next time of upgrade or for T&D / R&D! H/W reviews will be posted on the main portal page which currently has an audience of more than 200,000 unique visitors per year from more than 30,000 network locations (*).
If your company is interested please don’t hesitate to use this contact form:
http://www.dumpanalysis.org/contact
(*) From Google Analytics report.
- Dmitry Vostokov @ DumpAnalysis.org -
The portal jobs page has been updated:
http://www.dumpanalysis.org/jobs
- Dmitry Vostokov @ DumpAnalysis.org -
Real troubleshooting is usually done by combining several units of work chosen from a manual. Checklist pattern summarizes this recurrent practice. Checklist Coordinator orchestrates troubleshooting units of work (TUWs) components from TUW Repository according to checklists from Checklist Repository (in the simple case it can be just one checklist). This is illustrated on the following UML component diagram:
- Dmitry Vostokov @ DumpAnalysis.org -
Citrix released the powerful GUI CDF trace analyzer tool that can work with hundred Mb trace files with millions of trace messages:
http://support.citrix.com/article/CTX122741
I use it on day-to-day basis while analyzing CDF traces from terminal services environments and highly recommend. Its ability to quick filter out modules is indispensable when dealing with messages from hundreds of processes and thousands of threads. I haven’t tried it yet with MS TMF files though.
- Dmitry Vostokov @ DumpAnalysis.org -
Troubleshooting Unit of Work is another pattern frequently used in manual troubleshooting and debugging. This is usually some independent and self-sufficient list of steps to perform to check something from a troubleshooting checklist or a manual and can be implemented as a separate loadable module, a class to reuse or a function to call. Output from such units of work can be stored in a blackboard system or processed by tools implementing Checklist DebugWare pattern. Typical example is an implementation of the following document:
Required Permissions and Rights for the Ctx_CpsvcUser Account
as a tool:
CTX_CpsvcUser Re-creation Tool for 32-Bit and 64-Bit Versions of Presentation Server 4.5
- Dmitry Vostokov @ DumpAnalysis.org -
Trace Expert pattern came to my mind when I was writing about software trace patterns. It is a very lightweight expert system relying on trace collector and trace formatter (patterns to be written about soon). It is a module that takes a preformatted software trace message file or a buffer and a set of built in rules and uses simple search (peharps involving regular expressions) to dig out diagnostic information and provide troubleshooting and debugging directions.
This module is schematically depicted on the following UML component diagram:
- Dmitry Vostokov @ DumpAnalysis.org -
