Software Diagnostics Library

Some traces and logs may have Periodic Message Blocks with very similar message structure and content (mostly Message Invariants). The only significant difference between them is some unique data. We call such pattern Visitor Trace by analogy with Visitor design pattern where tracing code “visits” each object data or data part to log its content or status.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

There are events that by design or system configuration should be seen in a log only once or not seen at all if code responsible for them was executed before tracing session. For example, the launch of certain services during system initialization shouldn’t be seen again when we trace system activity long after that. It can also be just messages from singleton objects in the application log. The appearance of extra Singleton Events may point to design violations or some abnormal system events such as process restart. The latter may Intra-Correlate with the start of the fault handling process such as WerFault.exe in Windows Process Monitor logs (Guest Component).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The pattern called Small Value deals with easily recognizable values such as handles, timeouts, mouse pointer coordinates, enumeration values, window messages, etc. There is another kind of values we call Design Values, for example, 256 (+/- 1) or some other round value. Here we can also add some regular patterns in hex representation such as window handles or flags, for example, such as 0×10008000. Such designed values may fall into some module range too, the so called Coincidental Symbolic Information pattern. If we see a design value in the output of WinDbg commands especially related to abnormal behaviour patterns, not necessarily as a stack trace parameter, which can be False, then it might point to some design limitations that were reached. For example, Blocked ALPC Queue may have a limitation on I/O Completion Port when we have ALPC Wait Chains in an unresponsive system:

0: kd> !alpc /p <port_address>
[...]
512 thread(s) are registered with port IO completion object:
[…]

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

So far we have been discussing trace analysis patterns related to execution of a particular software version. However, software code changes and also its tracing and logging output: from large scale changes where components are replaced to small scale code refactoring affecting message structure and format. On a software narratological level this corresponds to a narrative about a software trace or log, it evolution. Such Meta Trace analysis pattern is different from Master Trace pattern where the latter is similar to what Metanarrative is usually meant in narratology: a master or grand idea - an expected trace if all functional requirements were correctly identified and implemented during software construction and non-functional ones are met during software execution.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Before deciding on whether to retrospect on 2012 we found that since March 14, 2008 this site has had more than 1 million visitors with more than 33% returning. So instead of just 2012 we decided to retrospect on that interval up to December 31, 2012. Google Analytics has improved since last January, 2012 and now made our task easier. So we start with the first one hundred sites referring to us:

Next table is distribution of visits among countries:

Then the first 100 network locations:

Then the first 100 search keywords and phrases that led to us:

Then browser stats (we have never thought that there are so many of them):

Browser	Visits
Internet Explorer	446051
Firefox	356686
Chrome	184535
Opera	45787
Safari	24123
Mozilla	3780
Mozilla Compatible Agent	2401
Android Browser	1337
Konqueror	1057
IE with Chrome Frame	982
Opera Mini	705
SeaMonkey	503
Safari (in-app)	197
Lunascape	144
BlackBerry8900	128
Camino	126
RockMelt	124
(not set)	96
Netscape	72
Playstation 3	36
IUC	34
Googlebot	29
Lynx	24
Unsupported Browser Version	22
BlackBerry9630	21
NetFront	17
BlackBerry9700	15
Microsoft-Symbol-Server	14
BlackBerry9000	12
Galeon	11
Midori	9
NokiaE63	9
Yahoo! Slurp	9
BlackBerry8530	8
BlackBerry8520	7
PagePeeker.com	7
SAMSUNG-SGH-I617	7
BlackBerry9530	6
JUC	6
MSR-ISRCCrawler	6
OpenWave	6
anonimo	5
BlackBerry9300	5
HTC_HD2_T8585 Opera	5
Nokia5233	5
Space Bison	5
-Vasya	4
Blazer	4
Uzbl	4
-^_^- Hello	3
<?echo ‘<pre>’; system	3
12345	3
BlackBerry9330	3
BlackBerry9650	3
HTC_P3700 Opera	3
HTC_TyTN_II Mozilla	3
NOKIAN78	3
Playstation Portable	3
PPC; 240×320; HTC_P3450	3
undefined GoogleToolbarBB	3
anonymous	2
Empty	2
GreatBrowse	2
Helyi user agent	2
HTC_Touch_Pro2_T7373 Opera	2
HTC_Touch2_T3333 Opera	2
J2ME	2
Mozilla 5.0	2
NokiaC1-01	2
NokiaC3-00	2
NokiaC7-00	2
NokiaX2-01	2
nwzfq	2
test	2
—	1
?M5	1
“PagePeeker.com”	1
<?include	1
<script>alert	1
<SCRIPT>window.location=’http:	1
2.0.0.11	1
31337′	1
8900b	1
AltaVista Intranet V2.0 evreka.com crawler@evreka.com	1
annoying	1
AppEngine-Google;	1
BlackBerry9500	1
BlackBerry9550	1
bwh3_user_agent	1
Citrix	1
EBABrowser	1
EY	1
fake_user_agent Mozilla	1
FAST Enterprise Crawler 6 used by Reed Exhibitions	1
foo	1
General Browser	1
GOOGLEBOT	1
HD_mini_T5555 Opera	1
Hellbrowser 6.66	1
holy_teacher FirePHP	1
HTC_P3490 Opera	1
HTC_P4550 Mozilla	1
HTC_Polaris Mozilla	1
HTC_Touch_3G_T3232 Opera	1
HTC_Touch_HD_T8282 Opera	1
HTC_Touch_Pro_T7272 Opera	1
HTC_Touch2_T3320 Opera	1
HTC-8900	1
IE 8	1
IE6	1
iTunes	1
Keep Out	1
KraftwayBrowser2	1
Links	1
Maemo Browser	1
Medusa	1
MERONG	1
Motorola_ES405B	1
mozilla	1
Mozilla Firefox	1
MS-OC 4.0	1
msie	1
NCSA Mosaic	1
NightDynamo AdminPanel v0.2.1	1
Nokia2700c-2	1
Nokia2730c-1	1
Nokia305	1
Nokia5230	1
Nokia5310XpressMusic	1
Nokia5800 XpressMusic	1
Nokia6300	1
Nokia6700c-1	1
NokiaC2-01	1
NokiaC2-02	1
NokiaC2-03	1
NokiaC5-03	1
nokiac6-00	1
NokiaC6-00	1
NOKIAE65	1
NokiaE66	1
NokiaE71	1
NokiaE71-2;Mozilla	1
NokiaE72-1	1
NokiaN-GageQD	1
NokiaN70-1	1
NokiaNokia 6210s	1
NoneOfYourBusiness	1
nothisname_wangxiaoyang3	1
OmniWeb	1
Palm750	1
Peeplo Screenshot Bot	1
PerTrUsTsQuiD	1
pippos.7	1
PPC; 480×800; HTC_Touch_HD_T8282; OpVer 34.159.1.612	1
PriceGoblin User Agent	1
Private	1
Privoxy	1
Read Later	1
SAMSUNG-GT-E2222	1
samsung-gt-s3653	1
samsung-gt-s3653 UNTRUSTED	1
SAMSUNG-S8000	1
SAMSUNG-SGH-I637	1
Samsung-SPHM540 Polaris	1
SmallProxy 3.5.4	1
SonyEricssonK750	1
Surf	1
tdhbrowser	1
TiFiC Client Z	1
union update table sd_users set userid=9 where username=’coco	1
unknown	1
Unknown	1
UNTRUSTED	1
Updownerbot	1
WIN	1
WinXP SP2	1
Wlwap	1
WM5 PIE	1
Xda_orbit_2; 240×320	1
Xyi znat kakoi browser MRA 5.7	1
ZooShot 0.1a	1
ZooShot 0.42	1

and finally mobile devices stats (you may find your own device there):

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

For some time I was struggling with finding a good name for memory dump and software trace analysis activities. The name Memoretics I use for the science of memory dump analysis (that also incorporates software traces) seems not so good to describe the whole practical activity that should be transparent to everyone in IT. Fortunately, I timely understood that all these activities constitute the essence of software diagnostics that previously lacked any solid foundation. Thus, Software Diagnostics Institute was reborn from the previous Crash Dump Analysis Portal. This institute does pure and applied research and scientific activities and in recent years was funded mainly from OpenTask publisher and recently from Memory Dump Analysis Services. The latter company also recognized that the broadening of its commercial activities requires a new name. So, Software Diagnostics Services was reborn:

The First Comprehensive Software Diagnostics Service

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sometimes I get requests to review application memory dump capture design. Of course, such requests usually come only when such designs don’t work or there are problems with loading saved crash dumps. The common blueprint of such architectures is a top level exception handler that use some API do capture and save process memory state. However, such designs forget why separate processed were introduced in the first place: to guard process memory space of different unrelated tasks (for related tasks there are threads). The data of the module (and its thread state) that does process memory capture may also be corrupt. The right design would be to show a message box with an information on how to use external process memory dumper such as Task Manager. If we need an automation then the right thing is to rely on WER features. Let separate processes do their work in separate spaces.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This is a second Webinar from Memory Dump Analysis Services on software diagnostics. The first one is about pattern recognition. During this Webinar you will learn how to apply systems theory and systems thinking for effective and efficient abnormal software behavior diagnostics: the foundation of software troubleshooting and debugging. The seminar summarizes 6 years of research done by Software Diagnostics Institute started with a short blog post Dumps and Systems Theory.

Title: Introduction to Systemic Software Diagnostics: Systems Thinking in Memory Dump and Software Trace Analysis
Date: 3rd of September, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/377382766

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services will be administering certifications developed by Software Diagnostics Institute for memory dump and software trace analysis:

Software Diagnostics Maturity Enterprise Certification
Memory Dump Analysis Certification is available this September

Debugging TV Frames episode 0×10 contains some background information.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

DumpAnalysis.org portal has been reorganized to Software Diagnostics Institute to reflect the nature of its research activities. More updates later on.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Motivated by 7 Habits of Highly Effective Debuggers I would like to reflect on a distinction between diagnostics and problem solving as separate processes (although highly related). First, we reverse the precept from that article because stories such as software logs and traces are of primary importance to software diagnostics (and not only). And without diagnostics there is no effective debugging (treatment, problem solving, etc.)

The Principle Precept of Diagnostics

Stories NOT Statistics secure certainty.

Next parts will be about actual habits so please stay tuned. I would try to finish this list before the forthcoming Webinar on software diagnostics.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

After 4 years in print this bestselling title needs an update to address minor changes, include extra examples and reference additional research published in Volumes 2, 3, 4, 5 and 6.

• Title: Memory Dump Analysis Anthology, Volume 1
• Author: Dmitry Vostokov
• Publisher: OpenTask (Summer 2012)
• Language: English
• Product Dimensions: 22.86 x 15.24

• Paperback: 800 pages
• ISBN-13: 978-1-908043-35-1

• Hardcover: 800 pages
• ISBN-13: 978-1-908043-36-8

The cover for both paperback and hardcover titles will also have a matte finish. We used A Memory Window artwork for the back cover.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

In a week this short full color book should appear in online bookstores:

• Title: Software Narratology: An Introduction to the Applied Science of Software Stories
• Authors: Dmitry Vostokov, Memory Dump Analysis Services
• Description: This is a transcript of Memory Dump Analysis Services Webinar about Software Narratology: an exciting new discipline and a field of research founded by DumpAnalysis.org. When software executes it gives us its stories in the form of UI events, software traces and logs. Such stories can be analyzed for their structure and patterns for troubleshooting, debugging and problem resolution purposes. Topics also include software narremes and their types, anticipatory software construction and software diagnostics.
• Publisher: OpenTask (April 2012)
• Language: English
• Product Dimensions: 28.0 x 21.6
• Paperback: 26 pages
• ISBN-13: 978-1908043078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

There are many lists for great books but I’d like to share the one that have a more personal touch. These books I wanted to read on the first sight and devoured them when I had an occasion.

First book that made a great impression on me was C Programming Language which I saw and read in 1987 (in Russian translation with very nice hardcover binding). There was a C implementation available on a PDP-11 clone I worked with at that time.

Another book that reinforced my better function prototype reading and pointer declarations was Expert C Programming.

I confess I had a long trouble understanding C++ long back ago despite reading The C++ Programming Language by Bjarne Stroustrup (reading several editions) and Effective C++ series by Scott Meyers partly because of my long practice of Win32 programming using C and partly because I didn’t understand why I needed C++ to do the same. Popular books such as C++ for Real Programmers (in Russian translation) terrified me with their complexity (although I made honest efforts to read them from cover to cover). However, when I saw and started reading the book Advanced CORBA Programming with C++ I really understood why we needed C++ and how it all worked.

After that rereading The C++ Programming Language and (More) Effective C++ books was a great pleasure.

The other part of C++, STL was greatly clarified by reading the following book in 2003 from cover to cover (at that time the first edition): The C++ Standard Library: A Tutorial and Reference.

Inside the C++ Object Model book clarified C++ implementation issues and helped later with crash dump analysis of C++ programs.

I plan to continue this “Great Impression” series with other topics and also to publish a recommended reading list for C++ based on books from my personal library that started growing when I was an employee at Programming Research in 2001 - 2003:

The row of books is all about C++. Except the big thick red book about English language grammar (as I became interested in other grammars than C++) The Cambridge Grammar of the English Language:

Now !ad debugger command: I’m developing a traning course Deep Down C++ with Memory Dump Analysis.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Memory Dump Analysis Services organizes a free Webinar on Unified Software Diagnostics (USD) and the new scalable cost-effective software support model called Pattern-Driven Software Support devised to address various shortcomings in existing tiered software support organizations. Examples cover Windows, Mac OS  and Linux.

Date: 22nd of June, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/172771078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Discovered this forthcoming book and immediately preordered:

Inside Windows Debugging: A Practical Guide to Debugging and Tracing Strategies in Windows

From Safari Books Online table of contents I see it also includes Event Tracing for Windows:

http://my.safaribooksonline.com/book/-/9780735671348

Looking forward to reading it and writing a review.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Software trace analysis is notoriously difficult so a bit of folk music is needed to make debugging sessions less boring. The following album came to my attention in a local library and after listening to it I recommend it for software log analysis sessions:

Make It Through This World

Here’s my own track title interpretation:

1. Dream Away The Defects
2. This I’d Analyze
3. A Magnifier
4. Make It Through This Trace
5. Don’t Let It Go Unanalyzed
6. Request-Reply Pair
7. I Love It When It’s Short
8. No Higher Specs
9. Said, Said.
10. When I Think Of You, My Customers
11. Close to The Defect
12. A Debugger For You Now

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Always imitate the behavior of the working program when it is crashing.

George Meredith

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The previous definition of software narratology was restricted to software traces and logs (the top left quadrant on a software narrative square, also the part of Memoretics which studies memory snapshots). Now, with the broadening of the domain of software narratology to the whole world of software narrative stories including actor interactions with software in construction requirements use cases and post-construction incidents we give another definition:

Software narrative is a representation of software events and changes of state. Software Narratology is a discipline that studies such software narratives (software narrative science).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

The development of SPDL requires extending pattern-driven analysis approach to software problems such as software defect and software incident descriptions. Such a pattern language should help with accurate problem identification and problem resolution through software behaviour analysis and with choosing, for example, appropriate workaround patterns or, for a debugging strategy, unified debugging patterns. This can also be applicable to software security incident descriptions as well.

For this first part, we introduce a pattern called, Problem Summary. This is a very short description that summarizes the essence of the problem from a submitter point of view such as a software user or a technical support engineer. Whether it is correct or incorrect is another problem itself but the missing summary usually requires close reading of the whole problem description text and analyze together various relevant parts to reconstruct the summary.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -