Crash Dump Analysis

This is a revised, edited, cross-referenced and thematically organized volume of selected DumpAnalysis.org blog posts about crash dump analysis and debugging written in July 2009 - January 2010 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software on Windows platforms and technical support and escalation engineers dealing with complex software issues. The fourth volume features:

- 13 new crash dump analysis patterns
- 13 new pattern interaction case studies
- 10 new trace analysis patterns
- 6 new Debugware patterns and case study
- Workaround patterns
- Updated checklist
- Fully cross-referenced with Volume 1, Volume 2 and Volume 3
- New appendixes

Product information:

• Title: Memory Dump Analysis Anthology, Volume 4
• Author: Dmitry Vostokov
• Language: English
• Product Dimensions: 22.86 x 15.24

• Paperback: 410 pages
• Publisher: Opentask (30 March 2010)
• ISBN-13: 978-1-906717-86-5

• Hardcover: 410 pages
• Publisher: Opentask (30 April 2010)
• ISBN-13: 978-1-906717-87-2

Back cover features memory space art image: Internal Process Combustion.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Today found a CD in my old case (the link I give is for MP3 though):

Ancient Journeys: A Vision Of The New World

Here’s my version of track titles:

1. Discovering the Interface
2. Conquering the New Space
3. Frozen System
4. MS DOS  
5. Russian OS
6. The Journeys of RIP
7. The Spiking Threads
8. Not Enough Space
9. Against Bugs

- Dmitry Vostokov @ DumpAnalysis.org -

Highly recommended to listen during analysis of a complete memory dump from an isolated dead system to build tension resulting in a problem resolution in 21 minutes:

Rachmaninov: Symphonic Dances & the Isle of the Dead

- Dmitry Vostokov @ DumpAnalysis.org -

This is a revised, edited, cross-referenced and thematically organized volume of selected DumpAnalysis.org blog posts about crash dump analysis and debugging written in October 2008 - June 2009 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software on Windows platforms and technical support and escalation engineers dealing with complex software issues. The third volume features:

- 15 new crash dump analysis patterns
- 29 new pattern interaction case studies
- Trace analysis patterns
- Updated checklist
- Fully cross-referenced with Volume 1 and Volume 2
- New appendixes

Product information:

• Title: Memory Dump Analysis Anthology, Volume 3
• Author: Dmitry Vostokov
• Language: English
• Product Dimensions: 22.86 x 15.24

• Paperback: 404 pages
• Publisher: Opentask (20 December 2009)
• ISBN-13: 978-1-906717-43-8

• Hardcover: 404 pages
• Publisher: Opentask (30 January 2010)
• ISBN-13: 978-1-906717-44-5

Back cover features 3D computer memory visualization image.

- Dmitry Vostokov @ DumpAnalysis.org -

When listening for the first time to all Bach Cantatas from Complete Works, I found Aria Duetto “Wir danken, wir preisen” (”We thank you, we praise”) from BWV 134 (#4) very stimulating to think about multithreading while debugging or analyzing memory dumps and confirmed my feelings during second listening to all Cantatas.

Bach Edition: Complete Works (155 CD Box Set)

Whom do we thank and praise then? Of course, the creator of the correct and maintainable multithreaded code.

- Dmitry Vostokov @ DumpAnalysis.org -

Just finished listening twice to Rick Wakeman performance at Lincoln Cathedral. To paraphrase his words, when I do memory dump analysis I experience the feeling of power surge through the fingers. Highly recommended to get the feeling of being the debugging god back during bitter moments of software support:

At Lincoln Cathedral

Here is my version of track titles inspired by listening (with my comments in italics):

1. Process Mortality
2. Dance of the MIPS
3. Gifts from Help
4. THE ALL MIGHTY DEBUGGER
5. Problem solved
6. The DA TA Variations (remember DATA: Dump Analysis Trace Analysis)

- Dmitry Vostokov @ DumpAnalysis.org -

This is the enhanced version of Dump2Wave technology that allows to transform computational operations into audible artifacts.

Computational processes and threads are fiber bundled with native memory visualization techniques to create audio and visual images of powerful memory topoi. This opens the new era in music. The closure of analog -> digital -> analog enables visualization and auralization of finite and infinite (transfinite) digital data.

Stay tuned! More on this later…

- Dmitry Vostokov @ DumpAnalysis.org -

I used to be a fun of Yanni music in the late 1990s. Today I started listening again to some of his albums and recommend them for any debugging session. If you are new to this music there is a compilation album that I’m listening to while I’m writing this post:

In the Mirror

Here is my version of track titles inspired by listening (with my comments in italics): 

1. In the Memory Dump File
2. The Morning Session
3. Love for Debugging
4. A Debugger’s Dream 
5. Within Kernel
6. Forbidden Access
7. Once Upon a Second Chance
8. Chasing Bugs
9. The Main Thread [Special Debugging Version]  
10. Quiet Memory Analyst 
11. Debugging Joy (My Life is Debugging)
12. So Long My Debugger (My Only Friend on Virtual Memory Plains)
13. Before I Leave the Debugger 
14. End of Session (It wasn’t bad after all)
15. Face in the Memory Dump (after applying Natural Memory Visualization techniques: you can see pictures and various artifacts stored in memory buffers)

- Dmitry Vostokov @ DumpAnalysis.org -

Some fellow debuggers ask me what brand of headphones I use during debugging. It depends on the working environment. In the office I use STAX electrostatic headphones, one of the previous versions of their Basic System that I bought 4 years ago, similar to this one:

http://www.stax.co.jp/Export/SRS2050II.html

I learnt about STAX in 2000, Moscow, when I was obsessed with pure sound and rushed to the nearest dealer to buy the old version of SR-001:

http://www.stax.co.jp/Export/SR001mk2e.html

- Dmitry Vostokov @ DumpAnalysis.org -

If you like me totally immerse in complete memories of computation you probably would like a complete Beethoven collection that I bought last year and now listen to the second time:

Beethoven Edition: Complete Works

- Dmitry Vostokov @ DumpAnalysis.org -

I was a big fan of Kitaro music for many years during 90s. Today I started re-listening to some of his albums and recommend

Heaven & Earth: Music From The Motion Picture Soundtrack

to listen to when analyzing complete memory dumps from complex issues, looking at user (Earth) and kernel (Heaven) spaces and trying to figure out which module started the conflict.

- Dmitry Vostokov @ DumpAnalysis.org -

Reading stack traces like English verse (remeber to read from bottom to top):

0:01> ~8kL
ChildEBP RetAddr 
009ef258 7c827d0b ntdll!KiFastSystemCallRet
009ef25c 7c83d236 ntdll!NtWaitForSingleObject+0xc
009ef298 7c83d281 ntdll!RtlpWaitOnCriticalSection+0x1a3
009ef2b8 7c82dabf ntdll!RtlEnterCriticalSection+0xa8
009ef358 7c82dab1 ntdll!LdrpGetProcedureAddress+0x128
009ef374 77e764ea ntdll!LdrGetProcedureAddress+0x18
009ef5d8 7c34c456 kernel32!UnhandledExceptionFilter+0x46f
009ef5f4 7c34957c msvcr71!_XcptFilter+0x15f
009ef600 7c34246e msvcr71!_endthreadex+0xb7
009ef628 7c828752 msvcr71!_except_handler3+0x61
009ef64c 7c828723 ntdll!ExecuteHandler2+0x26
009ef6f4 7c82855e ntdll!ExecuteHandler+0x24
009ef6f4 7c82be3e ntdll!KiUserExceptionDispatcher+0xe
009efa00 7c82a319 ntdll!RtlpFindEntry+0x68
009efc2c 7c3416b3 ntdll!RtlAllocateHeap+0x606
009efc6c 7c3416db msvcr71!_heap_alloc+0xe0
009efc74 7c360947 msvcr71!_nh_malloc+0x10
009efc80 0285f893 msvcr71!operator new+0xb
009efca8 02852e38 SQLModule!ODBCDelete+0xf3
009efd54 0269acff Store!ProcessDeletes+0x3d
009eff38 0269badb Store!UpdateStore+0xe
009eff58 00323499 Common!WorkItem+0x15c
009eff84 7c349565 Common!WorkItemThread+0x339
009effb8 77e64829 msvcr71!_endthreadex+0xa0
009effec 00000000 kernel32!BaseThreadStart+0x34

The new thread started
To work through items
It got an item
Handled it to store
To run delete requests
Through Oh-Dee-Bee-See
It tried to alloc
But crashed in malloc
While browsing the heap
Exception was dispatched
And handler called at once
But couldn’t find a filter
And called default one
That filter needed help
And looked for its address
But halted in suspense
While entering crit sec.

- Dmitry Vostokov @ DumpAnalysis.org -

If you complain about heap corruption again or type !locks every day and see pages of output filled with columns the following album from Supertramp reminds us that

Some Things Never Change

Here is my track name interpretation:

1. It’s a Buggy World
2. You Code, I Debug
3. Let’s Debug Together
4. Live to Debug It
5. Some Bugs Never Disappear
6. Read My Bug Report Please
7. Sooner or Later I Fix It
8. Help Me Down that Code Path
9. And the Customer
10. Il Est De Mon Bug!
11. Where There’s a Bug

- Dmitry Vostokov @ DumpAnalysis.org -

Listening to étude No. 1 in C major written by Frédéric Chopin (Op. 10) an idea came to me about writing 16 debugging études (ISBN: 978-1906717575). It is surprising that there are many programming études out there but there are no debugging ones. Stay tuned and be in touch with this blog.

Draft definition:

Debugging étude is a composition of software with intentional defects (bugs) of considerable difficulty to find and fix, designed to provide practice material to perfect debugging techniques and problem-solving skills.

The idea actually came to me long time ago to create some sort of debugging excersises for training purposes. 

- Dmitry Vostokov @ DumpAnalysis.org -

Paperback edition of Memory Dump Analysis Anthology, Volume 2 is finally available on Amazon and Barnes & Noble. Search Inside is also available on Amazon. In addition, I updated the list of recommended books:

Listmania! Crash Dump Analysis and Debugging

Hardcover edition will be available on Amazon and B&N in 2-3 weeks.

- Dmitry Vostokov @ DumpAnalysis.org -

“Everything is memory dump.”

I’m very excited to announce that Volume 2 is available in paperback, hardcover and digital editions:

Memory Dump Analysis Anthology, Volume 2

In one or two weeks paperback edition should also appear on Amazon and other bookstores. Amazon hardcover edition is planned to be available by the end of October.

I’m often asked when Volume 3 is available and I currently plan to release it in October - November, 2009. In the mean time I’m planning to concentrate on other publishing projects. 

- Dmitry Vostokov @ DumpAnalysis.org -

The book is nearly finished and here is the final TOC:

Memory Dump Analysis Anthology, Volume 2: Table of Contents

- Dmitry Vostokov @ DumpAnalysis.org -

Album cover and songs list from Narasimha Vedala:

DUMPSTATIC
Dr. Debugalov Feat.
Assembly Crooks and
E.I.P. Wailers.

1. Mistah Dumpstatic Feat. E.I.P Wailers.
2. Attached to your pros-ass.
3. I put a dump on you.
4. Be my debugger.
5. When push comes to shove, call Debugalov.
6. Pop ECX Feat. Assembly Crooks.
7. You corrupted my memory Feat. E.I.P Wailers.
8. Dark side of the dump.
9. Sweet Dump o’mine.
10. Optimize your soul.
11. Load address blues Feat. Assembly Crooks.
12. Good bye, Kernel.

Some music for this album will be written with the help of Dump2Wave technology Stay statically tuned to further announcements.

- Dmitry Vostokov @ DumpAnalysis.org -

The Songs of Distant Earth is my favorite Mike Oldfield album. Highly recommended to keep optimism when doing remote debugging on different systems.

The Songs of Distant Earth

Here is my alternative track naming:

1. The Decision To Go Remote
2. Let There Be A Connection
3. Super System Crash
4. Connection Established
5. First Break In
6. The Sea Of Threads
7. Setting Breakpoints
8. Prayer For A Match
9. Lament For Users
10. The Kernel
11. Screensaver Starts
12. Tabular Output
13. The Shining Threads
14. Breakpoint Match
15. The Sunken Debugger
16. Contemplating Observations
17. A New Session

- Dmitry Vostokov @ DumpAnalysis.org -

Although the first bug (the real one) was found in Mark II (see Software bug) the first general-purpose electronic computer was ENIAC and surely it had defects like overflow. In 80s there was an electronic music group called Software. I have a few CDs and listen to them sometimes. One album is called Software-Visions and it has these tracks (one of them is called Xenix - Microsoft UNIX-based OS):

1. Software Visions
2. Secrets
3. Stranger
4. Realtime
5. Mainframe
6. Snobol
7. Xenix
8. Syntax
9. Overflow
10. Interface
11. Eniac

- Dmitry Vostokov @ DumpAnalysis.org -