Software Diagnostics Library

Fascinated by Kazimir Malevich’s Black Square I created the new art genre with the following two artistic installations:

A Pause before Crash

This is 1Mb of PAUSE instructions without the point of return:

_text SEGMENT

main PROC

DW 100000h DUP (90f3h)

main ENDP

_text ENDS

END

When launched it crashes:

0:000> kL
Child-SP          RetAddr           Call Site
00000000`0012ff58 00000000`7704be3d 1MbPause+0x201011
00000000`0012ff60 00000000`77256a51 kernel32!BaseThreadInitThunk+0xd
00000000`0012ff90 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

0:000> ub rip
1MbPause+0x201002:
00000001`40201002 f390            pause
00000001`40201004 f390            pause
00000001`40201006 f390            pause
00000001`40201008 f390            pause
00000001`4020100a f390            pause
00000001`4020100c f390            pause
00000001`4020100e f390            pause
00000001`40201010 cc              int     3

You can download the source code, PDB and 64-bit EXE from here:

1MbPause.zip

Do Nothing and Crash

This is 1Mb of NOP instructions without the point of return:

_text SEGMENT

main PROC

DB 100000h DUP (90h)

main ENDP

_text ENDS

END

When launched it crashes too:

0:000> kL
Child-SP          RetAddr           Call Site
00000000`0012ff58 00000000`7704be3d 1MbNop+0x101011
00000000`0012ff60 00000000`77256a51 kernel32!BaseThreadInitThunk+0xd
00000000`0012ff90 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

0:000> ub rip
1MbNop+0x101009:
00000001`40101009 90              nop
00000001`4010100a 90              nop
00000001`4010100b 90              nop
00000001`4010100c 90              nop
00000001`4010100d 90              nop
00000001`4010100e 90              nop
00000001`4010100f 90              nop
00000001`40101010 cc              int     3

You can download the source code, PDB and 64-bit EXE from here:

1MbNop.zip

The earliest opcodism binary was created on October 25th, 2006 that I now call Nothingness and Crash: The Smallest Program.

- Dmitry Vostokov @ DumpAnalysis.org -

This is a revised, edited, cross-referenced and thematically organized volume of selected DumpAnalysis.org blog posts about crash dump analysis and debugging written in October 2008 - June 2009 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software on Windows platforms and technical support and escalation engineers dealing with complex software issues. The third volume features:

- 15 new crash dump analysis patterns
- 29 new pattern interaction case studies
- Trace analysis patterns
- Updated checklist
- Fully cross-referenced with Volume 1 and Volume 2
- New appendixes

Product information:

• Title: Memory Dump Analysis Anthology, Volume 3
• Author: Dmitry Vostokov
• Language: English
• Product Dimensions: 22.86 x 15.24

• Paperback: 404 pages
• Publisher: Opentask (20 December 2009)
• ISBN-13: 978-1-906717-43-8

• Hardcover: 404 pages
• Publisher: Opentask (30 January 2010)
• ISBN-13: 978-1-906717-44-5

Back cover features 3D computer memory visualization image.

- Dmitry Vostokov @ DumpAnalysis.org -

Consider this example mapping (taken metaphorically from the mathematical notion of an injection) of one domain of knowledge to another:

This mapping between concepts and ideas was once called “bijectivism” but was trivially described either as one to one mapping between two domains (like physical vs. mathematical) or fusing different concepts together to get another emerging concept. I myself proposed the similar mapping and called it a metaphorical bijection.  

Now consider another mapping metaphorically equivalent to a mathematical notion of a surjection where all constituents of the second domain are covered metaphorically by the first domain:

What we strive for is to establish the complete bijective mapping and reorganize our knowledge of both domains to achieve that:

In diagrams above small boxes can represent sets of ideas, methods, etc. or individual ideas, methods, etc. The established metaphorical bijection can divide sets or combine them if needed. There can be several such bijections, of course, and we can use other methods of inquiry (for example, the scientific method) to choose between competing metaphorical bijections.

Useful mnemonic:

BEIS (B=I+S or to BE IS …)

Bijectionism Equals Injection + Surjection

Another mnemonic:

BET (B=T or to BE Transformation…)

Bijectionism Equals Transformation 

Note also the second letter of Alef-Beis or Alef-Bet, the letter of Light that has interpretation of Creation in Biblical Hebrew.   

More on this later as I need to come back to DebugWare patterns.

- Dmitry Vostokov @ DumpAnalysis.org -

The new contemporary movement of engineers resisting dump analysis automation (including automated debugging and perhaps automated software construction too)

Inspired by Luddite movement.

- Dmitry Vostokov @ DumpAnalysis.org -

This is a kind of a “faultomorphism”, a fault, a crash point and stack trace shape preserving map between two platforms (such as 32-bit and 64-bit). This new word was derived from the concatenation of platform and morphism. Here is an example:

; 64-bit crash dump

0: kd> r
Last set context:
rax=0000000063537852 rbx=0000000000000000 rcx=0000000000000009
rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
rip=fffffadf262760da rsp=fffffadf15973968 rbp=0000000070537852
 r8=fffffadf31614b00  r9=fffffadffe9fa7b0 r10=000000000000000a
r11=fffffadf31614bf0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz na po nc
cs=0010 ss=0018 ds=0000 es=0000 fs=0000 gs=0000 efl=00010206
rdbss!RxIsThisACscAgentOpen+0×30:
fffffadf`262760da f3a6 repe cmps byte ptr [rsi],byte ptr [rdi]

0: kd> kL 100
Child-SP          RetAddr           Call Site
fffffadf`15973968 fffffadf`2629e768 rdbss!RxIsThisACscAgentOpen+0x30
fffffadf`15973970 fffffadf`262988f5 rdbss!RxInitializeVNetRootParameters+0x31d
fffffadf`159739f0 fffffadf`2629bcfd rdbss!RxFindOrConstructVirtualNetRoot+0x180
fffffadf`15973ad0 fffffadf`26297a6c rdbss!RxCanonicalizeNameAndObtainNetRoot+0x223
fffffadf`15973b70 fffffadf`26272a77 rdbss!RxCommonCreate+0x470
fffffadf`15973c80 fffffadf`261be3e8 rdbss!RxFsdCommonDispatch+0x51c
fffffadf`15973d80 fffffadf`29314db3 mrxsmb!MRxSmbFsdDispatch+0x211
[...]

; 32-bit crash dump

0: kd> r
eax=00000000 ebx=b6a23a80 ecx=00000009 edx=00000000 esi=00000008 edi=b6a23a80
eip=b6a23a5f esp=b3ce800c ebp=b3ce801c iopl=0 nv up ei pl zr na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010246
rdbss!RxIsThisACscAgentOpen+0×38:
b6a23a5f f3a6 repe cmps byte ptr [esi],byte ptr es:[edi]

0: kd> kL 100
b3ce801c b6a2b431 rdbss!RxIsThisACscAgentOpen+0x38
b3ce803c b6a2bbf7 rdbss!RxInitializeVNetRootParameters+0x282
b3ce809c b6a2e6cd rdbss!RxFindOrConstructVirtualNetRoot+0xdc
b3ce80d0 b6a2ae15 rdbss!RxCanonicalizeNameAndObtainNetRoot+0x197
b3ce8134 b6a20d51 rdbss!RxCommonCreate+0x2c3
b3ce81cc b6a2acc2 rdbss!RxFsdCommonDispatch+0x353
b3ce81f4 b69ac317 rdbss!RxFsdDispatch+0xda
b3ce8214 804e13d9 mrxsmb!MRxSmbFsdDispatch+0x134
[...]

We can see that stack traces are almost the same, function offsets are very close and faulted instruction is the same up to an opcode. Not to mention that bugchecks are identical:

RDR_FILE_SYSTEM (27)
    If you see RxExceptionFilter on the stack then the 2nd and 3rd parameters are the
    exception record and context record. Do a .cxr on the 3rd parameter and then kb to
    obtain a more informative stack trace.
    The high 16 bits of the first parameter is the RDBSS bugcheck code, which is defined
    as follows:
     RDBSS_BUG_CHECK_CACHESUP  = 0xca550000,
     RDBSS_BUG_CHECK_CLEANUP   = 0xc1ee0000,
     RDBSS_BUG_CHECK_CLOSE     = 0xc10e0000,
     RDBSS_BUG_CHECK_NTEXCEPT  = 0xbaad0000

Therefore, we can also say that these crashes are platformorphic. Obviously, this stems from the fact that source code was identical or almost identical for both platforms.

- Dmitry Vostokov @ DumpAnalysis.org -

Do you remember memuons¹, the indivisible entities of memory? Their study is the domain of the new science called memuonics². According to the so called memophysical principle³,we have particle interpretation of memuons. This is called classical memuonics with classical memory field theory where memuons are “quanta” of memory. We can also ”quantize” memory fields and get quantum memory field theories where memuons are created and annihilated.

(1) The notion of memuons first appeared in the philosophy of memoidealism.

(2) Please don’t confuse memuonics with memiotics. The latter is computer memory semiotics.

(3) Memophysical principle - theories of memory-based universe need to take into account the current mainstream sciences including physics.

- Dmitry Vostokov @ DumpAnalysis.org -

Last week had some fearetical features. What’s it all about you should wait until my memoirs are published:

Crash Dump: A Software Engineering Autobiography, ISBN: 978-1906717193

If we break down fearetical linguistically we come with the following free morphemes:

fear e tical

According to Wikipedia, the last one is a currency unit subdivided into into 64, 32, 8 and 4. A coin weighing 15g (0xFg). It was replaced by another currency unit, the franc.

- Dmitry Vostokov @ DumpAnalysis.org -

New word - new nickname…

Mr. Heapocrat is a member of a powerful group called heap class and a pseudonym for a historian and journalist that Debugged! MZ/PE magazine editorial board has invited to write a history and current affairs column called “Heap Inquiries”.

- Dmitry Vostokov @ DumpAnalysis.org -

Every day a new religion appears on Earth and now my turn to announce Memory Religion with several tentative names:

Memorianism
Memorianity
Memoriandom

One of the attractive features of this religion is eternal immortality through memory of our universe. The current understanding of that employs memoidealistic notion of memuonic memory that comprises the philosophy of memoidealism.

I made a leap of faith yesterday and now consider myself as a true believer! Hope other people will follow soon. As every major religion it also has its testament book to be published:

Title: Memory Religion: A Testament
ISBN-13: 978-1906717476

- Dmitry Vostokov @ DumpAnalysis.org -

Once upon a time I was analyzing a memory dump and in the process become upset. I realized that the dump was vacuous and afterwards learnt that the customer forced the dump of a normal system with steady-state computational activities inside. At the same time computational precognition, the ability to see in advance what would happen with the system, always fascinated me from the very beginning of my memory dump analysis work. In computational precognition phrase, the word “computational” refers to computers and “precognition” refers to humans but this should not be the problem as the boundary between these two categories is not sharp and can be shifted in either direction. This is an opening of the new post series about investigation of occult, paranormal and supernatural in the realm of computable. In short, about psi-computation.

- Dmitry Vostokov @ DumpAnalysis.org -

I’m pleased to announce that OpenTask has submitted the book Dumps, Bugs and Debugging Forensics: The Adventures of Dr. Debugalov for printing and here is the link to TOC:

Table of Contents

- Dmitry Vostokov @ DumpAnalysis.org - 

DLL is also a recursive acronym for DLL List Landscape. OpenTask is going to publish soon the new full color book:

Title: DLL List Landscape: The Art from Computer Memory Space
ISBN-13: 978-1-906717-36-0

More details will be announced tomorrow.  

- Dmitry Vostokov @ DumpAnalysis.org -

I continued thinking about bugteria in memory dumps all the day yesterday and came to the conclusion that the study of crash dump analysis patterns needs its own name and the obvious choice was Bugteriology:

Bugteriology is the study of crash dump analysis patterns (bugteria). Its main subject is identification, classification and characterization of such patterns found in memory dumps (bugterial species).

I initially registered a domain for this purpose (later abandoned) pointing to crash dump analysis and debugging portal but I want to think through this idea and perhaps make it a subdomain of dumpanalysis.org with a page for easy online pattern classification and make it also an online supplement to forthcoming encyclopedia of crash dump analysis patterns.

- Dmitry Vostokov @ DumpAnalysis.org -

Yesterday was one of those days when I was in a good mood thinking about bugs. Suddenly a thought stroke me about the similar sounding words bacterium and bugterium (perhaps because I’m currently reading a theoretical biology book, Essays on Life Itself). I admit that it might be sounding the same only for a non-native English ear though. So the new definition was born:

Bugterium (pl. bugteria) - an instance of a memory dump analysis pattern found in a crash (memory, core) dump file.

Why a bugterium and not a cdarium? The motivation (with a hindsight) lies in the complexity of debugging (and life forms). While a bug is a complex thing (and a beast) and it takes sometimes days or weeks to chase and fix (kill) the one, a bugterium (bacterium) is of relatively smaller complexity and can be easily identified and dealt with by component removal or upgrade (massively killed). From software support perspective remember this bugtation No.14:

Crash dump analysis ”is anticipated with” joy, “performed with” eagerness, “and bragged about forever.”

Although the perceived simplicity of crash dump analysis is deceptive (bugtation No.2):

“It requires a very unusual mind to undertake the analysis of the obvious” crash.

Alfred North Whitehead, Science and the Modern World

- Dmitry Vostokov @ DumpAnalysis.org -

Paperback edition of Memory Dump Analysis Anthology, Volume 2 is finally available on Amazon and Barnes & Noble. Search Inside is also available on Amazon. In addition, I updated the list of recommended books:

Listmania! Crash Dump Analysis and Debugging

Hardcover edition will be available on Amazon and B&N in 2-3 weeks.

- Dmitry Vostokov @ DumpAnalysis.org -

I’ve just passed 5 year mark in my Citrix career and I say those years were really exciting! Incidentally at the same time, on 14th of October 2003 I started doing true memory dump analysis in WinDbg-sense of it despite the fact that I’d been doing debugging since 1987 and encountered the first symbolic assembly language debugger on PDP-11 RSX-11M Russian clone in 1988. Also 5 years ago I made my transition from software engineering world to software technical support and started my journey to become a manager.

I’ve put a timeline on a portal accessible from the main page or from the following link:

http://www.dumpanalysis.org/Timeline

The timeline will help me in the future to write my memoirs (Crash Dump: A Software Engineering Autobiography, ISBN: 978-1906717193).

Finally, to record the glorious Citrixofication of the world by various Citrixware, Opentask decided to publish a book where I plan to be an editor of various contributors and probably write a chapter or two myself:

Discovering Citrixware (ISBN: 978-1906717278) 

It is not about troubleshooting but more in a sense of discovery. More details will be announced later.

- Dmitry Vostokov @ DumpAnalysis.org -

“Everything is memory dump.”

I’m very excited to announce that Volume 2 is available in paperback, hardcover and digital editions:

Memory Dump Analysis Anthology, Volume 2

In one or two weeks paperback edition should also appear on Amazon and other bookstores. Amazon hardcover edition is planned to be available by the end of October.

I’m often asked when Volume 3 is available and I currently plan to release it in October - November, 2009. In the mean time I’m planning to concentrate on other publishing projects. 

- Dmitry Vostokov @ DumpAnalysis.org -

New cartoon and a word “bugxorcist” from Narasimha Vedala (click on it to enlarge):

Bugs At The Gate

- Dmitry Vostokov @ DumpAnalysis.org -

First, a definition for a new word that I coined today:  

Bugtation

noun
Date: 21st century
1. a modified quotation showing relation to debugging and troubleshooting

This is the first bugtation I would like to introduce and it is related to heisenbugs:

“There’s no such thing as” heisenbug;
“And what to us seems merest accident
Springs from the deepest source of” computation.

Friedrich Schiller, Early Dramas 

Deviations from original quotations are highlighted in blue. Welcome to the new literary art!

Note: if you notice any bugs in bugtations please let me know…

- Dmitry Vostokov @ DumpAnalysis.org -

New cartoon from Narasimha Vedala provides insight into the new ideology (click on it to enlarge):

Dawn of Debugism

Here I repeat 10 debugging commandments in scripture:

1. Thou shalt not underestimate bugs
2. Thou shalt walk the stack with thy colleagues
3. Thou shalt strive not to corrupt thine memory heap
4. Thou shalt share thine debugging knowledge
5. Thou shalt not overflow the buffer
6. Thou shalt not covet thy neighbour’s dump
7. Thou shalt not reverse engineer for profit
8. Thou shalt not attach debugger to thy neighbor’s wife
9. Thou shalt not commit adultery with bugs
10. Thou shalt not shalt thou to me

- Dmitry Vostokov @ DumpAnalysis.org -