I’m thinking big for a full-color book to celebrate the forthcoming anniversary of Memory Dump Analysis Anthology. Preliminary details:
- Dmitry Vostokov @ DumpAnalysis.org -
Not My Version is another basic pattern of DLL Hell variety. It is when we look at the component timestamps and paths and realize that that one of the modules from the production environment is older than we had during development and testing. The lmft WinDbg command will produce the necessary output. If there are many modules we might want to create a CAD graph (Component Age Diagram) to spot anomalies. Component version check is one of the basic troubleshooting and system administration activities that will be fully illustrated in the forthcoming book “Crash Dump Analysis for System Administrators and Support Engineers” (ISBN-13: 978-1-906717-02-5). Here is one example (module start and end load addresses are removed for visual clarity):
0:000> kL
Child-SP RetAddr Call Site
00000000`0012fed8 00000001`40001093 MyDLL!fnMyDLL
00000000`0012fee0 00000001`40001344 2DLLs+0×1093
00000000`0012ff10 00000000`773acdcd 2DLLs+0×1344
00000000`0012ff60 00000000`774fc6e1 kernel32!BaseThreadInitThunk+0xd
00000000`0012ff90 00000000`00000000 ntdll!RtlUserThreadStart+0×1d
0:000> lmft
module name
MyDLL C:\OLD\MyDLL.dll Wed Jun 18 14:49:13 2004
user32 C:\Windows\System32\user32.dll Thu Feb 15 05:22:33 2007
kernel32 C:\Windows\System32\kernel32.dll Thu Nov 02 11:14:48 2006
ntdll C:\Windows\System32\ntdll.dll Thu Nov 02 11:16:02 2006
2DLLs C:\2DLLs\2DLLs.exe Thu Jun 19 10:46:44 2008 (485A2B04)
uxtheme C:\Windows\System32\uxtheme.dll Thu Nov 02 11:15:07 2006
rpcrt4 C:\Windows\System32\rpcrt4.dll Tue Jul 17 05:21:15 2007
lpk C:\Windows\System32\lpk.dll Thu Nov 02 11:12:33 2006
oleaut32 C:\Windows\System32\oleaut32.dll Thu Dec 06 05:09:35 2007
usp10 C:\Windows\System32\usp10.dll Thu Nov 02 11:15:03 2006
ole32 C:\Windows\System32\ole32.dll Thu Nov 02 11:14:31 2006
advapi32 C:\Windows\System32\advapi32.dll Thu Nov 02 11:11:35 2006
gdi32 C:\Windows\System32\gdi32.dll Thu Feb 21 04:40:51 2008
msvcrt C:\Windows\System32\msvcrt.dll Thu Nov 02 11:13:37 2006
imm32 C:\Windows\System32\imm32.dll Thu Nov 02 11:13:15 2006
msctf C:\Windows\System32\msctf.dll Thu Nov 02 11:13:42 2006
This pattern should be checked when we have instances of Module Variety and, especially, Duplicated Module. Note that this pattern can also easily become an anti-pattern when applied to an unknown component: Alien Component.
- Dmitry Vostokov @ DumpAnalysis.org -
Following the introductory division of memory analysis into two broad categories I decided to plan yet another book with the following title and preliminary product details:
As you might have noticed, I prefer to put 128, 256 or 512 pages in my book announcements. What that would say about the author’s background? 
I would like to set 1024 pages for my memory dump analysis anthology series but 740 or 800 pages is the limit for POD technology I use.
- Dmitry Vostokov @ DumpAnalysis.org -
This is a must have book for system administrators of complex Windows server platforms and client workstations to understand and choose the best course of action to address system and application crashes, hangs, CPU spikes and memory leaks. It is also invaluable to general Windows users and technical support engineers.
- Dmitry Vostokov @ DumpAnalysis.org -
Due to demand from people that prefer ebooks I published Memory Dump Analysis Anthology, Volume 1 in a digital format that can be purchased in Crash Dump Analysis Store. This format has color pictures inside.
- Dmitry Vostokov @ DumpAnalysis.org -
I’m very proud to announce that it is finally available in both paperback and hardback. Why have I made available both editions? Because I personally prefer hardcover books. You can order the book today and it will be printed in 3-5 days (paperback) or 5-10 days (hardcover) and sent to you:
Memory Dump Analysis Anthology, Volume 1
Note: although listed on Amazon and other online bookstores it is not immediately available at these stores at the moment due to the late submission. I apologize for this. However, I expect that in a few weeks pre-orders taken there will be eventually fulfilled. In the mean time, if you want the book now, you can use the link above.
- Dmitry Vostokov @ DumpAnalysis.org -
Because Citrix CDF traces are ETW based it should be possible to process and analyze them using XPrerf and XPerfView from the latest Microsoft Windows Performance Tools:
Windows Performance Tools Kit, v.4.1.1
- Dmitry Vostokov @ DumpAnalysis.org -
Born out of 2008 April Fool’s joke (although I was thinking about it long before) the idea is to have a general framework and associated IDE for analysis activities for various tools that produce logs. Similar to planned GUI Monitor integrating WindowHistory, MesageHistory and ScreenHistory tools but also including Debugger Log Analyzer, Debugging Notebooks, various memory and trace log visualization subsystems. It also has the working title UTF (Unified Tools Framework).
Seems I’m very easy on inceptions All this will be covered in great detail in the forthcoming book which I tirelessly advertise:
DebugWare: The Art and Craft of Writing Troubleshooting Tools
- Dmitry Vostokov @ DumpAnalysis.org -
Ceteris Paribus means “with other things [being] the same” (Latin) and when applied to software troubleshooting and debugging means equal environment and configuration. My favorite example is troubleshooting an issue using two Citrix CDF traces (ETW based): one is for the problem and another for the expected behavior. Say we have a terminal services connectivity problem where a published application doesn’t start on the one particular server in Citrix farm. Here Ceteris Paribus means that the application, connection method, configuration, user name, and so on, are all the same for both traces.
Looks like I have used Latin to obfuscate something obvious but surely many engineers forget it when facing complex issues. This equally applies to debugging as well.
- Dmitry Vostokov @ DumpAnalysis.org -
Although the first volume has not been published yet (scheduled for 15th of April, 2008) the planning for the second volume has already begun. Preliminary information is:
Hardcover version is also planned. PDF version will be available for download too.
(*) subject to change
- Dmitry Vostokov @ DumpAnalysis.org -
It is very easy to become a publisher nowadays. Much easier than I thought. I registered myself as a publisher under the name of OpenTask which is my registered business name in Ireland. I also got the list of ISBN numbers and therefore can announce product details for the first volume of Memory Dump Analysis Anthology series:
Memory Dump Analysis Anthology, Volume 1
(*) subject to change
PDF file will be available for download too.
- Dmitry Vostokov @ DumpAnalysis.org -
SDT (Structured Debugging and Troubleshooting) is procedural (action-based). Once we get the description of the problem we jump to actions:
Whereas OODT is centered around objects (systems and customers are also objects):
Get objects from the problem description and problem environment
Interrogate them sending messages (could be an email at high levels :-)) like changing a registry key is a message to configuration management subsystem
OODT depends on troubleshooting domain and therefore finally we finally come to DDDT.
- Dmitry Vostokov @ DumpAnalysis.org -
Component identification is one of the main goals of post-mortem memory dump analysis and troubleshooting process in general. Using the definition of components as units of deployment and 3rd-party composition taken from Clemens Szyperski’s seminal book discussing component software in general and COM, CORBA, Java and .NET in particular (highly recommended book)
Component Software: Beyond Object-Oriented Programming (2nd Edition)
I would say that CBDT is centered around component isolation and replacement.
- Dmitry Vostokov @ DumpAnalysis.org -
OODT pronounced ”Oddity” is not a paradigm shift for support and software maintenance environments but a recognized way to solve problems using object-oriented techniques. In contrast to Structured Debugging and Troubleshooting methods (SDT) where engineers have sequence of questions and structure troubleshooting plans around them OODT is based on targeting specific objects, subsystems and systems (sending “messages” to them) and evaluating response and changes in their behaviour. I have to say more about it later after introducing CBDT (no easy pronunciation for it but any suggestions are welcome :-)).
Note: OODT doesn’t mean troubleshooting OO systems - it means applying OO techniques to troubleshooting
- Dmitry Vostokov @ DumpAnalysis.org -
It is difficult to analyze a problem when you have crash dumps and/or traces from various tracing tools and supporting information you have is incomplete or missing. After doing crash dump and trace analysis including ETW-based traces for more than 4 years I came up with this easy to remember 4WS questions to ask when you send or request traces and memory dumps:
What - What had happened or had been observed? Crash or hang, for example?
When - When did the problem happen if traces were recorded for hours?
Where - What server or workstation had been used for tracing or where memory dumps came from? For example, one trace is from a primary server and two others are from backup servers or one trace is from a client workstation and the other is from a server.
Why - Why did a customer or a support engineer request a dump or a trace? This could shed the light on various assumptions including presuppositions hidden in problem description.
Supporting information - needed to find a needle in a hay: process id, thread id, etc. Also, the answer to the following question is important: how dumps and traces were created?
Every trace or memory dump shall be accompanied by 4WS answers.
4WS rule can be applied to any troubleshooting because even the problem description itself is some kind of a trace.
- Dmitry Vostokov @ DumpAnalysis.org -
