Software Diagnostics Library

To paraphrase ”The Importance of Being Earnest“ I have made a Euclidean rigid motion on a plane by coming back from management line back to engineering line: my new title at Citrix Systems is Principal Dev Analysis Engineer. In the past before embarking on a blogger’s and writer’s career I wanted to become a Manager (and eventually became the one after being a Team Lead) and even had plans to enroll into one of business schools to get an MBA but now engineering path seems more natural to me at these times.

- Dmitry Vostokov @ DumpAnalysis.org -

Finally it has been published and available for orders from Amazon and other bookstores:

http://www.dumpanalysis.org/Debugged+Magazine

I had to increase the number of pages for the first issue from 16, planned originally, to 28 and this is reflected in the retail price of $10 (originally planned $8) but bookstores should sell it with a discount between 0% and 55%.

More information about the next issue should be ready by the end of the next week.

- Dmitry Vostokov @ DumpAnalysis.org - 

Previously announced Memory Analysis and Debugging Institute was registered in the Republic of Ireland (No. 382026) last week.

- Dmitry Vostokov @ DumpAnalysis.org - 

I’m very proud to announce that my first 2 volumes of Memory Dump Analysis Anthology are to be translated and published in the Republic of Korea this year:

http://www.opentask.com/korean-rights-mdaa-v1-v2

- Dmitry Vostokov @ DumpAnalysis.org -

While fighting the flu last week I forgot that on 26^th of March, 2006 I registered this domain. My excitement was so great that I couldn’t sleep the following night. I originally thought of domain names like crashdumpanalysis or memorydumpanalysis but was convinced by one of my colleagues that the shorter dumpanalysis is better.

- Dmitry Vostokov @ DumpAnalysis.org -

One day, last week, Dmitry was walking in Malahide Woods and thinking through his dangerous ideas about universal memory dumps and how to reconcile man-made PDB files with empirically discovered science files. Upon finding a problem resolution, Dmitry sat firmly on the ground and remained there happily for some time.

- Dmitry Vostokov @ DumpAnalysis.org -   

I was passing by a local bookstore on my way to the office a few days ago and bought this book:

Hitler’s Private Library: The Books That Shaped His Life

The title of the book prompted me to think about books that shaped my life and thought. Here is the list of 9 books:

Before school:

1. The Hobbit (in Russian translation)

At school before university:

2. The Feynman Lectures on Physics (in Russian translation)

3. A short book with the title if translated to English: ”Electron through the Eyes of a Chemist: Essays on Modern Quantum Chemistry” by I. Dmitriev. Seems not available in English.

4. Mathematics: The Loss of Certainty (in Russian translation)
 

At university:

5. C Programming Language (in Russian translation)

After university but before moving to Ireland:

6. Advanced CORBA Programming with C++

7. UML Distilled: A Brief Guide to the Standard Object Modeling Language

Last 3-4 years:

8. Becoming a Technical Leader: An Organic Problem-Solving Approach

9. Life Itself: A Comprehensive Inquiry into the Nature, Origin, and Fabrication of Life

- Dmitry Vostokov @ DumpAnalysis.org -

In December, 2000 I decided to apply for an Irish working visa after receiving an offer from Ericsson. Although offered compensation for a Senior Software Designer position was less than I had in Moscow at that time working for 2 companies simultaneously, I decided to accept the offer because of 3 primary reasons:

1. To learn spoken English

2. To work for one company only instead of many and dedicate free time to learning, reading and socializing

3. Working visa conditions such as the freedom to change an employer and virtually unlimited duration (permission to stay is renewed every 2 years)

If I had a US H-1B visa offer at the same time I would have definitely chosen the Irish one because I consider being tied to an employer in the case of H-1B as a kind of modern slavery. So Irish immigration is more progressive in this regard.  When in March, 2003 I got my redundancy in another Irish company I was calm because I knew that I could find another employer in Ireland and I didn’t have to leave the country like many engineers did leave USA during dot-com crash and what we see now when USA companies lay off H-1B workers. Therefore I had my working visa renewed 3-4 times and after 8 years I got today a stamp that allows me to stay indefinitely without any condition (practically until my passport expires). This is very good and allows me to proceed further with Memory Analysis and Debugging Institute and associated publishing activities. I also applied for Irish citizenship that if granted gives me freedom to visit other EU countries on demand and ease access to USA and Canada. Last year I got an invitation to Canada to participate in the development of Windows Driver Foundation exam but I had to abandon my visit because of simultaneous passport change and the need to renew my stay in Ireland that happened to coincide with the visit dates.

- Dmitry Vostokov @ DumpAnalysis.org -

The first week in February was extremely cold in Ireland with sub-zero temperatures and snowfall. This culminated on Sunday, 8th with even more snowfall that I haven’t seen in Ireland for 8 years. I woke up that morning and everything outside my apartment was white so I went out to take a few pictures:

- Dmitry Vostokov @ DumpAnalysis.org - 

I remember in my old days of PDP-11 system programming I learnt about the system call to spawn processes and wrote a program in assembly language that was spawning itself. This recursive spawning resulted in geometrical progression of running tasks and brought RSX-11M system to halt very quickly. Recently I observed the similar but non-recursive Process Factory pattern in one of memory dumps: explorer was relentlessly creating application.exe processes and at the time some effect was noticed there were more than 5,000 of them:

1: kd> !vm
[...]
5d20 application.exe 212 ( 848 Kb)
5d08 application.exe 212 ( 848 Kb)
5d04 application.exe 212 ( 848 Kb)
5cf8 application.exe 212 ( 848 Kb)
5cf0 application.exe 212 ( 848 Kb)
5ce8 application.exe 212 ( 848 Kb)
5cdc application.exe 212 ( 848 Kb)
5ccc application.exe 212 ( 848 Kb)
5cc8 application.exe 212 ( 848 Kb)
5cc0 application.exe 212 ( 848 Kb)
5ca8 application.exe 212 ( 848 Kb)
5c9c application.exe 212 ( 848 Kb)
5c98 application.exe 212 ( 848 Kb)
5c90 application.exe 212 ( 848 Kb)
5c88 application.exe 212 ( 848 Kb)
5c7c application.exe 212 ( 848 Kb)
5c70 application.exe 212 ( 848 Kb)
5c68 application.exe 212 ( 848 Kb)
5c64 application.exe 212 ( 848 Kb)
5c60 application.exe 212 ( 848 Kb)
5c50 application.exe 212 ( 848 Kb)
5c4c application.exe 212 ( 848 Kb)
5c44 application.exe 212 ( 848 Kb)
5c3c application.exe 212 ( 848 Kb)
5c34 application.exe 212 ( 848 Kb)
5c2c application.exe 212 ( 848 Kb)
5c24 application.exe 212 ( 848 Kb)
5c1c application.exe 212 ( 848 Kb)
5bf8 application.exe 212 ( 848 Kb)
5be0 application.exe 212 ( 848 Kb)
5bd4 application.exe 212 ( 848 Kb)
5bd0 application.exe 212 ( 848 Kb)
5ba4 application.exe 212 ( 848 Kb)
5b58 application.exe 212 ( 848 Kb)
5b50 application.exe 212 ( 848 Kb)
5b44 application.exe 212 ( 848 Kb)
5b38 application.exe 212 ( 848 Kb)
5b30 application.exe 212 ( 848 Kb)
5b04 application.exe 212 ( 848 Kb)
5af4 application.exe 212 ( 848 Kb)
5ad8 application.exe 212 ( 848 Kb)
5ad4 application.exe 212 ( 848 Kb)
5ac8 application.exe 212 ( 848 Kb)
5ac4 application.exe 212 ( 848 Kb)
5ab4 application.exe 212 ( 848 Kb)
5aa4 application.exe 212 ( 848 Kb)
5a9c application.exe 212 ( 848 Kb)
5a94 application.exe 212 ( 848 Kb)
5a8c application.exe 212 ( 848 Kb)
5a88 application.exe 212 ( 848 Kb)
5a74 application.exe 212 ( 848 Kb)
[...]

1: kd> !process 0 0
**** NT ACTIVE PROCESS DUMP ****
PROCESS 8b57f020  SessionId: none  Cid: 0004    Peb: 00000000  ParentCid: 0000
    DirBase: cffb3020  ObjectTable: e1003da0  HandleCount: 3932.
    Image: System

PROCESS 8a9f8d88  SessionId: none  Cid: 01b8    Peb: 7ffdf000  ParentCid: 0004
    DirBase: cffb3040  ObjectTable: e13e3f68  HandleCount: 111.
    Image: smss.exe

PROCESS 89f0d508  SessionId: 0  Cid: 01f0    Peb: 7ffd8000  ParentCid: 01b8
    DirBase: cffb3060  ObjectTable: e16bc370  HandleCount: 1292.
    Image: csrss.exe

PROCESS 89eea7c8  SessionId: 0  Cid: 0208    Peb: 7ffde000  ParentCid: 01b8
    DirBase: cffb3080  ObjectTable: e14b4160  HandleCount: 564.
    Image: winlogon.exe

[...]

PROCESS 8607c020  SessionId: 1  Cid: 44c8    Peb: 7ffdc000  ParentCid: 4cf8
    DirBase: cffb7080  ObjectTable: e3c9fd38  HandleCount: 25407.
    Image: explorer.exe

[...]

PROCESS 85e1d020  SessionId: 1  Cid: 538c    Peb: 7ffda000  ParentCid: 44c8
    DirBase: cffb8980  ObjectTable: e8065b20  HandleCount:  39.
    Image: application.exe

PROCESS 85c74610  SessionId: 1  Cid: 5394    Peb: 7ffd9000  ParentCid: 44c8
    DirBase: cffb89a0  ObjectTable: e6951878  HandleCount:  39.
    Image: application.exe

PROCESS 85c81020  SessionId: 1  Cid: 53a4    Peb: 7ffd7000  ParentCid: 44c8
    DirBase: cffb89c0  ObjectTable: e6d2f600  HandleCount:  39.
    Image: application.exe

PROCESS 85c6fb18  SessionId: 1  Cid: 53a8    Peb: 7ffd7000  ParentCid: 44c8
    DirBase: cffb89e0  ObjectTable: e54df078  HandleCount:  39.
    Image: application.exe

PROCESS 85c60020  SessionId: 1  Cid: 53bc    Peb: 7ffdf000  ParentCid: 44c8
    DirBase: cffb8a40  ObjectTable: e1214e90  HandleCount:  39.
    Image: application.exe

PROCESS 85c5d380  SessionId: 1  Cid: 53c8    Peb: 7ffde000  ParentCid: 44c8
    DirBase: cffb8a60  ObjectTable: e7baf638  HandleCount:  39.
    Image: application.exe

PROCESS 85c648b8  SessionId: 1  Cid: 53dc    Peb: 7ffde000  ParentCid: 44c8
    DirBase: cffb8a80  ObjectTable: e759d060  HandleCount:  39.
    Image: application.exe

PROCESS 85c62528  SessionId: 1  Cid: 53e0    Peb: 7ffde000  ParentCid: 44c8
    DirBase: cffb8aa0  ObjectTable: e3b8fa00  HandleCount:  39.
    Image: application.exe

PROCESS 85c59d88  SessionId: 1  Cid: 53e8    Peb: 7ffdc000  ParentCid: 44c8
    DirBase: cffb8ac0  ObjectTable: e31751e0  HandleCount:  39.
    Image: application.exe

PROCESS 85c46d88  SessionId: 1  Cid: 542c    Peb: 7ffd5000  ParentCid: 4d9c
    DirBase: cffb8b00  ObjectTable: e6fbc500  HandleCount: 136.
    Image: nlapplication.exe

PROCESS 85c3c020  SessionId: 1  Cid: 5464    Peb: 7ffdc000  ParentCid: 44c8
    DirBase: cffb8b40  ObjectTable: e218b948  HandleCount:  39.
    Image: application.exe

PROCESS 85c2a020  SessionId: 1  Cid: 546c    Peb: 7ffdb000  ParentCid: 44c8
    DirBase: cffb8b60  ObjectTable: e639a8d0  HandleCount:  39.
    Image: application.exe

PROCESS 85c202c8  SessionId: 1  Cid: 5474    Peb: 7ffd7000  ParentCid: 44c8
    DirBase: cffb8b80  ObjectTable: e517caa8  HandleCount:  39.
    Image: application.exe

PROCESS 85c1b020  SessionId: 1  Cid: 547c    Peb: 7ffd6000  ParentCid: 44c8
    DirBase: cffb8ba0  ObjectTable: e6c0cbc0  HandleCount:  39.
    Image: application.exe

PROCESS 85c1dd88  SessionId: 1  Cid: 5484    Peb: 7ffd5000  ParentCid: 44c8
    DirBase: cffb8bc0  ObjectTable: e4a42f68  HandleCount:  39.
    Image: application.exe

PROCESS 85d3ed88  SessionId: 1  Cid: 5488    Peb: 7ffd5000  ParentCid: 44c8
    DirBase: cffb8be0  ObjectTable: e68558f0  HandleCount:  39.
    Image: application.exe

[...]

We see that all created processes have the same parent process with PID 44c8 and when we inspect it we see many threads inside creating application.exe process:

1: kd> .process /r /p 8607c020
Implicit process is now 8607c020
Loading User Symbols

1: kd> !process 8607c020
PROCESS 8607c020  SessionId: 1  Cid: 44c8    Peb: 7ffdc000  ParentCid: 4cf8
    DirBase: cffb7080  ObjectTable: e3c9fd38  HandleCount: 25407.
    Image: explorer.exe
    VadRoot 88efec98 Vads 3445 Clone 0 Private 30423. Modified 71292. Locked 0.
    DeviceMap e3743340
    Token                             e29be5e0
    ElapsedTime                       00:54:31.359
    UserTime                          00:00:19.234
    KernelTime                        00:04:04.828
    QuotaPoolUsage[PagedPool]         1075132
    QuotaPoolUsage[NonPagedPool]      137800
    Working Set Sizes (now,min,max)  (15457, 50, 345) (61828KB, 200KB, 1380KB)
    PeakWorkingSetSize                48919
    VirtualSize                       585 Mb
    PeakVirtualSize                   978 Mb
    PageFaultCount                    123488
    MemoryPriority                    BACKGROUND
    BasePriority                      8
    CommitCharge                      49919

[...]

THREAD 84f25300  Cid 44c8.6288  Teb: 7ff8e000 Win32Thread: bc486830 READY
IRP List:
    88699110: (0006,0220) Flags: 00000884  Mdl: 00000000
Not impersonating
DeviceMap                 e3743340
Owning Process            8607c020       Image:         explorer.exe
Wait Start TickCount      1327981        Ticks: 29 (0:00:00:00.453)
Context Switch Count      145332                 LargeStack
UserTime                  00:00:00.000
KernelTime                00:00:00.093
Win32 Start Address SHLWAPI!SHCreateThread (0x77ec3ea5)
Start Address kernel32!BaseThreadStartThunk (0x7c8217ec)
Stack Init a98e4000 Current a98e3700 Base a98e4000 Limit a98e0000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0
ChildEBP RetAddr 
a98e3718 80833ec5 nt!KiSwapContext+0x26
a98e3744 80829bc0 nt!KiSwapThread+0x2e5
a98e378c 8087e0d8 nt!KeWaitForSingleObject+0x346
a98e37c4 8087e397 nt!ExpWaitForResource+0x30
a98e37e4 badff32a nt!ExAcquireResourceExclusiveLite+0x8d
a98e3808 badffe35 driverA+0x132a
a98e3824 bae00208 driverA+0x1e35
a98e3868 bae0e45a driverA+0x2208
a98e38a0 8081e095 driverA+0x1045a
a98e38b4 b972c73b nt!IofCallDriver+0x45
[...]
a98e38e8 b9b194e1 nt!IofCallDriver+0x45
[...]
a98e3940 b85cbf08 nt!IofCallDriver+0x45
a98e3968 b85bcfcc driverB!LowerDevicePassThrough+0x48
a98e398c b85bd63d driverB+0x6fcc
a98e3a24 b85cb167 driverB+0x763d
a98e3a34 b85cb1b7 driverB+0x15167
a98e3a5c 8081e095 driverB!DispatchPassThrough+0x48
a98e3a70 808fb13b nt!IofCallDriver+0x45
a98e3b58 80939c6a nt!IopParseDevice+0xa35
a98e3bd8 80935d9e nt!ObpLookupObjectName+0x5b0
a98e3c2c 808ece57 nt!ObOpenObjectByName+0xea
a98e3ca8 808ee0f1 nt!IopCreateFile+0x447
a98e3d04 808f1e31 nt!IoCreateFile+0xa3
a98e3d44 8088ad3c nt!NtOpenFile+0x27
a98e3d44 7c9485ec nt!KiFastCallEntry+0xfc (TrapFrame @ a98e3d64)
03bbda04 7c82bdf6 ntdll!KiFastSystemCallRet
03bbda2c 7c82dd9a kernel32!BasepSxsCreateStreams+0xe2
03bbda9c 7c82d895 kernel32!BasepSxsCreateProcessCsrMessage+0x136
03bbe2c4 7c8024a0 kernel32!CreateProcessInternalW+0x1943
03bbe2fc 7ca36750 kernel32!CreateProcessW+0×2c
03bbed80 7ca36b45 SHELL32!_SHCreateProcess+0×387
03bbedd4 7ca3617b SHELL32!CShellExecute::_DoExecCommand+0xb4
03bbede0 7ca35a76 SHELL32!CShellExecute::_TryInvokeApplication+0×49
03bbedf4 7ca3599f SHELL32!CShellExecute::ExecuteNormal+0xb1
03bbee08 7ca35933 SHELL32!ShellExecuteNormal+0×30
03bbee24 7ca452ff SHELL32!ShellExecuteExW+0×8d

1: kd> .thread 84e6a600
Implicit thread is now 84e6a600

1: kd> kv 100
[...]
03bbda04 7c82bdf6 001200a9 03bbda8c 03bbdb20 ntdll!KiFastSystemCallRet
03bbda2c 7c82dd9a 00000000 00000003 001200a9 kernel32!BasepSxsCreateStreams+0xe2
03bbda9c 7c82d895 00000000 00000000 03bbdc38 kernel32!BasepSxsCreateProcessCsrMessage+0x136
03bbe2c4 7c8024a0 00000000 01dafb9c 01dad904 kernel32!CreateProcessInternalW+0x1943
03bbe2fc 7ca36750 01dafb9c 01dad904 00000000 kernel32!CreateProcessW+0×2c
03bbed80 7ca36b45 00010098 00000000 01daffac SHELL32!_SHCreateProcess+0×387
[…]

1: kd> du /c 100 01dafb9c
01dafb9c  “C:\Program Files\App Package\Application.exe”

The difference of this pattern and similar Handle Leak or Zombie Processes is the fact that leaks usually happen when a process forgets to close handles but Process Factory creates active processes which are full resource containers and consume system resources, for example, they all have full handle table or consume GDI resources if they are GUI processes.

- Dmitry Vostokov @ DumpAnalysis.org -

The number of visits / month increased by 50% by the end of the year with more than 125,000 unique visitors in 2008 from 180 countries (150 in 2007) and almost 34% of them coming back. Here are the top 100 network locations out of almost 24,000:

Almost 66,000 Google search keywords (more than 100% increase since 2007) pointed to the portal and this blog with 100 most frequent:

Special thanks to 950 web sites that mention the portal and this blog with the first top 100:

Top 25 visiting countries:

- Dmitry Vostokov @ DumpAnalysis.org -

DumpAnalysis.org announces forthcoming 2010 as The Year of Dump Analysis.

Q&A 

Q. Why 2010?

A. Two reasons: 1) To do dump analysis effectively and efficiently an engineer needs some experience in debugging acquired in the previous year of debugging (perhaps after 7 debugging nights, 2009, 0×7D9); 2) 2010 is 0×7DA.

Q. What is the meaning of 7?

A. It is interpreted as Dump Analysis 7 days a week. Like what I do. Or from kernel pool tag perspective it is AD7: Analysis of Dumps 7 days a week. 

Q. What about the year 2011, 2012, 2013? 0×7DB, 0×7DC and 0×7DD?

A. Hmm, sounds like WinDbg commands db, dc and dd…

- Dmitry Vostokov @ DumpAnalysis.org -

Last week had some fearetical features. What’s it all about you should wait until my memoirs are published:

Crash Dump: A Software Engineering Autobiography, ISBN: 978-1906717193

If we break down fearetical linguistically we come with the following free morphemes:

fear e tical

According to Wikipedia, the last one is a currency unit subdivided into into 64, 32, 8 and 4. A coin weighing 15g (0xFg). It was replaced by another currency unit, the franc.

- Dmitry Vostokov @ DumpAnalysis.org -

I’ve been fascinated by Chemistry since the age of 13-14. At that time I noticed organic formulae on the blackboard of a higher school class and was curious about what they meant. So I asked my mother to bring me a book about Chemistry from a library and she brought a school textbook about Inorganic Chemistry. I read it in a few weeks and proceeded to reading a textbook about Organic Chemistry. At the same time I found in a local library 10 volumes of The Feynman Lectures on Physics (in Russian translation) and started reading the first volumes on classical mechanics and learnt about calculus. Another popular book about Quantum Chemistry raised my curiosity in Quantum Mechanics and Morris Kline’s The Loss of Certainty book (in Russian translation) made me interested in abstract mathematics and its logical and set-theoretical foundations including Gödel’s theorems and intuitionistic mathematics. All this happened before the age of 16 and in one evening when I was reading a Linear Algebra textbook an idea struck me to represent certain aspects of Inorganic Chemistry formalisms like Periodic Table and empirical formulas of chemical compounds as linear vector spaces of element vectors over the field of numbers.

Now OpenTask is going to publish its first popular science book called:

Vector Space Chemistry (ISBN: 978-1906717551) 

with a preface written after 25 years since the discovery of this mathematical model and formalization of Chemistry.

A note for cautious readers: I’m aware about over-excessive application of mathematics in sciences, especially after reading these books:

Fashionable Nonsense and Social Sciences as Sorcery

My book is just a popular science book that explains some chemical and abstract mathematical concepts and provides an example of using Mathematics as a modeling and formalization tool for Chemistry.

- Dmitry Vostokov @ DumpAnalysis.org -

New word - new nickname…

Mr. Heapocrat is a member of a powerful group called heap class and a pseudonym for a historian and journalist that Debugged! MZ/PE magazine editorial board has invited to write a history and current affairs column called “Heap Inquiries”.

- Dmitry Vostokov @ DumpAnalysis.org -

I’ve updated timeline widget with references to relevant blog posts and also added events that I forgot to add previously and ones that happened since my celebration of 5 years of memory dump analysis in October:

Memory Dump Analysis Portal Timeline

- Dmitry Vostokov @ DumpAnalysis.org -

It had always been my dream since I left Moscow State University to be associated with a research institute. Until yesterday it became a reality with the announcement of

Memory Analysis & Debugging Institute (MA_&D^I).

From: http://www.dumpanalysis.org/madinstitute-announcement

- Dmitry Vostokov @ DumpAnalysis.org -

Just noticed today that the number of posts in crash dump category became 512:

Of course, with this post, it is now 513.

 - Dmitry Vostokov @ DumpAnalysis.org -

I noticed that Amazon sells used copies of my Curriculum Vitae:

- Dmitry Vostokov @ DumpAnalysis.org -

I was thinking about the universal character of debugging for quite some time and finally the following bugtation provided an inspiration for a new book title to be published during the Year of Debugging:

Title: Breaking the Bug: Debugging as a Natural Phenomenon
ISBN-13: 978-1906717377

More product details will be announced later.

Actually I believe in the mystical nature of various debugging numbers and sequences. For example, the ISBN number of this book ends in 377 which is the octal base equivalent of 0n255 or 0xFF.

- Dmitry Vostokov @ DumpAnalysis.org -