Can computers debug?

Consider an application randomly crashing at different addresses or hanging sometimes. One day we are lucky to get this process postmortem memory dump:

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(f34.c6c): Access violation - code c0000005 (first/second chance not available)
eax=73726946 ebx=00403378 ecx=656c2070 edx=656c2074 esi=00403374 edi=00000004
eip=7d64d233 esp=0012ff24 ebp=0012ff4c iopl=0 nv up ei pl nz ac pe cy
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b  efl=00010217
ntdll!RtlpWaitOnCriticalSection+0xdf:
7d64d233 ff4014          inc     dword ptr [eax+14h]  ds:002b:7372695a=????????
  

Aha! It involves critical sections! Let’s see whether we have an instance of Critical Section Corruption pattern. The first disappointment comes when !locks command takes ages to finish so we break it:

0:000> !locks

Stopped scanning because of control-C

Scanned 154686373 critical sections

Next we try to list all of them but without any success:

0:000> !locks -v

CritSec at 00000000 could not be read
Perhaps the critical section was a global variable in a dll that was unloaded?

CritSec at 00000000 could not be read
Perhaps the critical section was a global variable in a dll that was unloaded?

CritSec at 00000000 could not be read
Perhaps the critical section was a global variable in a dll that was unloaded?

CritSec at 00000000 could not be read
Perhaps the critical section was a global variable in a dll that was unloaded?

[...]

Next we look at stack trace to find critical section address:

0:000> kv
ChildEBP RetAddr  Args to Child             
0012ff4c 7d628576 64726f77 00000004 00000000 ntdll!RtlpWaitOnCriticalSection+0xdf
0012ff6c 00401074 00403374 00403394 00000001 ntdll!RtlEnterCriticalSection+0xa8
0012ff7c 004011e9 00000001 004d2fc0 004d3030 application!wmain+0×74
0012ffc0 7d4e7d2a 00000000 00000000 7efde000 application!__tmainCRTStartup+0×10f
0012fff0 00000000 00401332 00000000 00000000 kernel32!BaseProcessStart+0×28

0:000> dt CRITICAL_SECTION 00403374
application!CRITICAL_SECTION
   +0×000 DebugInfo        : 0×73726946 _RTL_CRITICAL_SECTION_DEBUG
   +0×004 LockCount        : 1701585008
   +0×008 RecursionCount   : 1919251571
   +0×00c OwningThread     : 0×20666f20
   +0×010 LockSemaphore    : 0×64726f77
   +0×014 SpinCount        : 0×73

It looks corrupt indeed so let’s see if it has ASCII fragments:

0:000> db 00403374
00403374  46 69 72 73 70 20 6c 65-73 74 65 72 20 6f 66 20  Firsp lester of
00403384  77 6f 72 64 73 00 00 00-00 00 00 00 02 00 00 00  words………..
[…]

0:000> da 00403374
00403374 “Firsp lester of words”

Looks like garbled sentence “First letter of words”. Who wrote this? Sherlock would say: “Elementary, my dear Watson”, take the first letters, literally: “First letter of words”. Flow component or a component with similar name causes corruption at random addresses! We can’t believe this, run lm WinDbg command and to our astonishment we see Flows module:

0:000> lm
start    end        module name
00400000 00405000   application
00410000 004ab000   advapi32     
71c20000 71c32000   tsappcmp   
75490000 754f5000   usp10      
77ba0000 77bfa000   msvcrt     
78130000 781cb000   msvcr80    
7d4c0000 7d5f0000   kernel32  
7d600000 7d6f0000   ntdll     
7d800000 7d890000   gdi32      
7d8d0000 7d920000   secur32    
7d930000 7da00000   user32     
7da20000 7db00000   rpcrt4     
7dbc0000 7dbc9000   Flows
7dee0000 7df40000   imm32

Unloaded modules:
77b90000 77b98000   VERSION.dll
76920000 769e2000   USERENV.dll
71c40000 71c97000   NETAPI32.dll
771f0000 77201000   WINSTA.dll
770e0000 771e8000   SETUPAPI.dll
004e0000 00532000   SHLWAPI.dll
69500000 69517000   faultrep.dll

Checking the module information we see that it is the part of some unstable 3rd-party hookware and removing it solves the problem of elusive crashes. The problem solving power of Mind! The example is a bit contrived but my point here is that there are problems computers would never debug and troubleshoot. Answering the question of Dreyfus’ book “What computers still can’t do”: they still can’t debug…

- Dmitry Vostokov @ DumpAnalysis.org -

One Response to “Can computers debug?”

  1. Crash Dump Analysis » Blog Archive » Ruminations on Automated Debugging (Part 1) Says:

    […] Can computers debug? […]

Leave a Reply