Process crash - getting the dump manually

Sometimes customers have process crashes with exception dialogs but no dumps are saved due to some reason, for example, Dr. Watson limitation, NTSD doesn’t save dumps on Windows 2000, etc. One solution is to dump the process manually while it displays an error message. Customers and support engineers can use Microsoft userdump.exe for this purpose. Then looking at the dump we would see the exception because it is processed by an exception handler that either shows the error dialog or creates Windows Error Reporting process. Non-interactive services usually call NtRaiseHardError to let csrss.exe display a message. The following stack trace is from IE dump saved when WER error dialog box was shown:

0:000> k
ChildEBP RetAddr
0012973c 7c59a072 NTDLL!ZwWaitForSingleObject+0xb
00129764 7c57b3e9 KERNEL32!WaitForSingleObjectEx+0x71
00129774 00401b2f KERNEL32!WaitForSingleObject+0xf
0012a238 7918cd0e IEXPLORE!DwExceptionFilter+0×284
0012a244 03a3f0c3 mscoree!__CxxUnhandledExceptionFilter+0×46
0012a250 7c59bf8d msvcr71!__CxxUnhandledExceptionFilter+0×46
0012a984 715206e0 KERNEL32!UnhandledExceptionFilter+0×140
0012ee74 71520957 BROWSEUI!BrowserProtectedThreadProc+0×64
0012fef0 71762a0a BROWSEUI!SHOpenFolderWindow+0×1ec
0012ff10 00401ecd SHDOCVW!IEWinMain+0×108
0012ff60 00401f7d IEXPLORE!WinMainT+0×2dc
0012ffc0 7c5989a5 IEXPLORE!ModuleEntry+0×97
0012fff0 00000000 KERNEL32!BaseProcessStart+0×3d

If we disassemble DwExceptionFilter we would see CreateProcess call:

0:000> ub IEXPLORE!DwExceptionFilter+0x284
00401b0e call dword ptr [IEXPLORE!_imp__CreateProcessA (00401050)]
00401b14 test eax,eax
00401b16 je   IEXPLORE!DwExceptionFilter+0x2f6 (00401ba1)
00401b1c mov  dword ptr [ebp+7Ch],edi
00401b1f mov  edi,dword ptr [IEXPLORE!_imp__WaitForSingleObject (0040104c)]
00401b25 push 4E20h
00401b2a push dword ptr [ebp+68h]
00401b2d call edi

I already described WER processes in the previous post about post-mortem debugging so I won’t cover it here.

If we run !analyze -v command we are lucky because WinDbg will find the exception for us:

CONTEXT: 0012aa94 -- (.cxr 12aa94)
eax=00000000 ebx=00000000 ecx=00000000 edx=7283e058 esi=0271a60c edi=00000000
eip=35c5f973 esp=0012ad60 ebp=0012ad7c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00010246
35c5f973 8b01 mov eax,dword ptr [ecx] ds:0023:00000000=????????
0012ad7c 35c6042f 0012ae10 00000000 35c53390 componentA!InternalFoo+0x21
0012c350 779d7d5d 00000000 001ad114 00000000 componentA!InternalBar+0x157
0012c36c 77a2310e 02b23d5c 00000020 00000004 oleaut32!DispCallFunc+0x15d
0012c3fc 35cc8b60 024d2d94 02b23d5c 00000001 oleaut32!CTypeInfo2::Invoke+0x244

If you see several threads with UnhandledExceptionFilter - Multiple Exceptions pattern - you can set the exception context individually based on the first parameter of UnhandledExceptionFilter which is a pointer to _EXCEPTION_POINTERS structure and then use .cxr command:

0:000> ~*kv
. 0 Id: 1568.68c Suspend: 1 Teb: 7ffde000 Unfrozen
ChildEBP RetAddr Args to Child
0012a984 715206e0 0012a9ac 7800bdb5 0012a9b4 KERNEL32!UnhandledExceptionFilter+0×140 (FPO: [Non-Fpo])

0:000> dt _EXCEPTION_POINTERS 0012a9ac
+0×000 ep_xrecord : 0×12aa78
+0×004 ep_context : 0×12aa94
0:000> .cxr 0012aa94
eax=00000000 ebx=00000000 ecx=00000000 edx=7283e058 esi=0271a60c edi=00000000
eip=35c5f973 esp=0012ad60 ebp=0012ad7c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00010246
35c5f973 8b01 mov eax,dword ptr [ecx] ds:0023:00000000=????????

Another stack fragment comes from some Windows service and it shows the thread calling NtRaiseHardError:

0:000> ~*k
13 Id: 3624.16cc Suspend: 1 Teb: 7ffad000 Unfrozen
ChildEBP RetAddr
0148ed40 7c821b74 ntdll!KiFastSystemCallRet
0148ed44 77e99af9 ntdll!NtRaiseHardError+0xc
0148f3dc 77e84259 kernel32!UnhandledExceptionFilter+0×54b

0148f40c 7c82eeb2 kernel32!_except_handler3+0×61
0148f430 7c82ee84 ntdll!ExecuteHandler2+0×26
0148f4d8 7c82ecc6 ntdll!ExecuteHandler+0×24
0148f4d8 7c81e215 ntdll!KiUserExceptionDispatcher+0xe
0148f7e0 76133437 ntdll!RtlLengthSecurityDescriptor+0×2a
0148f80c 7613f33d serviceA!GetObjectSize+0×1c3
0148f8d0 77c70f3b serviceA!RpcGetObjectSize+0×1b
0148f8f8 77ce23f7 rpcrt4!Invoke+0×30
0148fcf8 77ce26ed rpcrt4!NdrStubCall2+0×299
0148fd14 77c709be rpcrt4!NdrServerCall2+0×19
0148fd48 77c7093f rpcrt4!DispatchToStubInCNoAvrf+0×38
0148fd9c 77c70865 rpcrt4!RPC_INTERFACE::DispatchToStubWorker+0×117
0148fdc0 77c734b1 rpcrt4!RPC_INTERFACE::DispatchToStub+0xa3

- Dmitry Vostokov -

2 Responses to “Process crash - getting the dump manually”

  1. Crash Dump Analysis » Blog Archive » Crash Dump Analysis Patterns (Part 72) Says:

    […] was that time when the dump should have been saved. See also Process crash - getting the dump manually post for another example and full […]

  2. Crash Dump Analysis » Blog Archive » WinDbg shortcuts: .exptr Says:

    […] looking at hidden exceptions and manual crash dumps we need information from this structure and this command provides a convenient way to see both […]

Leave a Reply

You must be logged in to post a comment.