Software Diagnostics Library

Crash Dump Analysis Patterns (Part 11)

One of mistakes beginners make is trusting WinDbg !analyze or kv commands displaying stack trace. WinDbg is only a tool, sometimes information necessary to get correct stack trace is missing and therefore some critical thought is required to distinguish between correct and incorrect stack traces. I call this pattern Incorrect Stack Trace. Incorrect stack traces usually

Have WinDbg warning: “Following frames may be wrong”
Don’t have the correct bottom frame like kernel32!BaseThreadStart (in user-mode)
Have function calls that don’t make any sense
Have strange looking disassembled function code or code that doesn’t make any sense from compiler perspective
Have ChildEBP and RetAddr addresses that don’t make any sense

Consider the following stack trace:

0:011> k ChildEBP RetAddr WARNING: Frame IP not in any known module. Following frames may be wrong. 0184e434 7c830b10 0×184e5bf 0184e51c 7c81f832 ntdll!RtlGetFullPathName_Ustr+0×15b 0184e5f8 7c83b1dd ntdll!RtlpLowFragHeapAlloc+0xc6a 00099d30 00000000 ntdll!RtlpLowFragHeapFree+0xa7

Here we have almost all attributes of the wrong stack trace. At the first glance it looks like some heap corruption happened (runtime heap alloc and free functions are present) but if you give it second thought you would see that low fragmentation heap Free function shouldn’t call low fragmentation heap Alloc function and the latter shoudn’t query full path name. That doesn’t make any sense.

What we should do here? Look at raw stack and try to build the correct stack trace ourselves. In our case this is very easy. We need to traverse stack frames from BaseThreadStart+0×34 until we don’t find any function call or reach the top. When functions are called (no optimization, most compilers) EBP registers are linked together as explained on slide 13 here:

Practical Foundations of Debugging (6.1)

0:011> !teb TEB at 7ffd8000 ExceptionList: 0184ebdc StackBase: 01850000 StackLimit: 01841000 SubSystemTib: 00000000 FiberData: 00001e00 ArbitraryUserPointer: 00000000 Self: 7ffd8000 EnvironmentPointer: 00000000 ClientId: 0000061c . 00001b60 RpcHandle: 00000000 Tls Storage: 00000000 PEB Address: 7ffdf000 LastErrorValue: 0 LastStatusValue: c0000034 Count Owned Locks: 0 HardErrorMode: 0

0:011> dds 01841000 01850000 01841000 00000000 … … … 0184eef0 0184ef0c 0184eef4 7615dff2 localspl!SplDriverEvent+0×21 0184eef8 00bc3e08 0184eefc 00000003 0184ef00 00000001 0184ef04 00000000 0184ef08 0184efb0 0184ef0c 0184ef30 0184ef10 7615f9d0 localspl!PrinterDriverEvent+0×46 0184ef14 00bc3e08 0184ef18 00000003 0184ef1c 00000000 0184ef20 0184efb0 0184ef24 00b852a8 0184ef28 00c3ec58 0184ef2c 00bafcc0 0184ef30 0184f3f8 0184ef34 7614a9b4 localspl!SplAddPrinter+0×5f3 0184ef38 00c3ec58 0184ef3c 00000003 0184ef40 00000000 0184ef44 0184efb0 0184ef48 00c117f8 … … … 0184ff28 00000000 0184ff2c 00000000 0184ff30 0184ff84 0184ff34 77c75286 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0×3a 0184ff38 0184ff4c 0184ff3c 77c75296 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0×4a 0184ff40 7c82f2fc ntdll!RtlLeaveCriticalSection 0184ff44 000de378 0184ff48 00097df0 0184ff4c 4d2fa200 0184ff50 ffffffff 0184ff54 ca5b1700 0184ff58 ffffffff 0184ff5c 8082d821 0184ff60 0184fe38 0184ff64 00097df0 0184ff68 000000aa 0184ff6c 80020000 0184ff70 0184ff54 0184ff74 80020000 0184ff78 000b0c78 0184ff7c 00a50180 0184ff80 0184fe38 0184ff84 0184ff8c 0184ff88 77c5778f RPCRT4!RecvLotsaCallsWrapper+0xd 0184ff8c 0184ffac 0184ff90 77c5f7dd RPCRT4!BaseCachedThreadRoutine+0×9d 0184ff94 0009c410 0184ff98 00000000 0184ff9c 00000000 0184ffa0 00097df0 0184ffa4 00097df0 0184ffa8 00015f90 0184ffac 0184ffb8 0184ffb0 77c5de88 RPCRT4!ThreadStartRoutine+0×1b 0184ffb4 00088258 0184ffb8 0184ffec 0184ffbc 77e6608b kernel32!BaseThreadStart+0×34 0184ffc0 00097df0 0184ffc4 00000000 0184ffc8 00000000 0184ffcc 00097df0 0184ffd0 8ad84818 0184ffd4 0184ffc4 0184ffd8 8980a700 0184ffdc ffffffff 0184ffe0 77e6b7d0 kernel32!_except_handler3 0184ffe4 77e66098 kernel32!`string’+0×98 0184ffe8 00000000 0184ffec 00000000 0184fff0 00000000 77c5de6d RPCRT4!ThreadStartRoutine 0184fff8 00097df0 0184fffc 00000000 01850000 00000008

Next we need to use custom k command and specify base pointer. In our case the last found stack address that links EBP pointers is 0184eef0:

0:011> k L=0184eef0 ChildEBP RetAddr WARNING: Frame IP not in any known module. Following frames may be wrong. 0184eef0 7615dff2 0×184e5bf 0184ef0c 7615f9d0 localspl!SplDriverEvent+0×21 0184ef30 7614a9b4 localspl!PrinterDriverEvent+0×46 0184f3f8 761482de localspl!SplAddPrinter+0×5f3 0184f424 74067c8f localspl!LocalAddPrinterEx+0×2e 0184f874 74067b76 SPOOLSS!AddPrinterExW+0×151 0184f890 01007e29 SPOOLSS!AddPrinterW+0×17 0184f8ac 01006ec3 spoolsv!YAddPrinter+0×75 0184f8d0 77c70f3b spoolsv!RpcAddPrinter+0×37 0184f8f8 77ce23f7 RPCRT4!Invoke+0×30 0184fcf8 77ce26ed RPCRT4!NdrStubCall2+0×299 0184fd14 77c709be RPCRT4!NdrServerCall2+0×19 0184fd48 77c7093f RPCRT4!DispatchToStubInCNoAvrf+0×38 0184fd9c 77c70865 RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0×117 0184fdc0 77c734b1 RPCRT4!RPC_INTERFACE::DispatchToStub+0xa3 0184fdfc 77c71bb3 RPCRT4!LRPC_SCALL::DealWithRequestMessage+0×42c 0184fe20 77c75458 RPCRT4!LRPC_ADDRESS::DealWithLRPCRequest+0×127 0184ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0×430 0184ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd

Stack traces make more sense now but we don’t see BaseThreadStart+0×34. By default WinDbg displays only certain amount of function calls (stack frames) so we need to specify stack frame count, for example, 100:

0:011> k L=0184eef0 100 ChildEBP RetAddr WARNING: Frame IP not in any known module. Following frames may be wrong. 0184eef0 7615dff2 0×184e5bf 0184ef0c 7615f9d0 localspl!SplDriverEvent+0×21 0184ef30 7614a9b4 localspl!PrinterDriverEvent+0×46 0184f3f8 761482de localspl!SplAddPrinter+0×5f3 0184f424 74067c8f localspl!LocalAddPrinterEx+0×2e 0184f874 74067b76 SPOOLSS!AddPrinterExW+0×151 0184f890 01007e29 SPOOLSS!AddPrinterW+0×17 0184f8ac 01006ec3 spoolsv!YAddPrinter+0×75 0184f8d0 77c70f3b spoolsv!RpcAddPrinter+0×37 0184f8f8 77ce23f7 RPCRT4!Invoke+0×30 0184fcf8 77ce26ed RPCRT4!NdrStubCall2+0×299 0184fd14 77c709be RPCRT4!NdrServerCall2+0×19 0184fd48 77c7093f RPCRT4!DispatchToStubInCNoAvrf+0×38 0184fd9c 77c70865 RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0×117 0184fdc0 77c734b1 RPCRT4!RPC_INTERFACE::DispatchToStub+0xa3 0184fdfc 77c71bb3 RPCRT4!LRPC_SCALL::DealWithRequestMessage+0×42c 0184fe20 77c75458 RPCRT4!LRPC_ADDRESS::DealWithLRPCRequest+0×127 0184ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0×430 0184ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd 0184ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0×9d 0184ffb8 77e6608b RPCRT4!ThreadStartRoutine+0×1b 0184ffec 00000000 kernel32!BaseThreadStart+0×34

Now stack trace looks much better.

- Dmitry Vostokov @ DumpAnalysis.org -

This entry was posted on Tuesday, April 3rd, 2007 at 3:09 pm and is filed under Crash Dump Analysis, Crash Dump Patterns, WinDbg Tips and Tricks. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

22 Responses to “Crash Dump Analysis Patterns (Part 11)”

Dmitry Vostokov Says:
July 25th, 2007 at 12:52 pm
I’ve published another case study for this pattern with complete source code:

http://www.dumpanalysis.org/blog/index.php/2007/07/25/reconstructing-stack-trace-manually/
abhi Says:
July 25th, 2007 at 11:03 pm
Hello,
My dump analysis looks like this. This is a hang dump file of a COM+ component on Windows server 2003 SP2.
Can you something obvious here. appreciate your help. Thanks

# 0 Id: f34.f40 Suspend: 1 Teb: 7ffdf000 Unfrozen ChildEBP RetAddr Args to Child 0007fcd4 7c827d0b 77e61d1e 00000078 00000000 ntdll!KiFastSystemCallRet 0007fcd8 77e61d1e 00000078 00000000 00000000 ntdll!NtWaitForSingleObject+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0007fd48 77e61c8d 00000078 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0x88 0007fd5c 77727b21 00000078 ffffffff ffffffff kernel32!WaitForSingleObject+0x12 0007fd78 77728683 0009a6b0 0007fdc7 00000000 ole32!CSurrogateProcessActivator::WaitForSurrogateTimeout+0x55 0007fd90 01001383 0007fdac 00000000 00000000 ole32!CoRegisterSurrogateEx+0x1dd 0007ff1c 010016a1 01000000 00000000 000924b0 dllhost!WinMain+0xe3 0007ffc0 77e6f23b 00000000 00000000 7ffdc000 dllhost!WinMainCRTStartup+0x182 0007fff0 00000000 0100151f 00000000 78746341 kernel32!ProcessIdToSessionId+0x209

1 Id: f34.f2c Suspend: 1 Teb: 7ffdd000 Unfrozen ChildEBP RetAddr Args to Child 00c2fe18 7c82783b 77c885ac 00000120 00c2ff74 ntdll!KiFastSystemCallRet 00c2fe1c 77c885ac 00000120 00c2ff74 00000000 ntdll!NtReplyWaitReceivePortEx+0xc 00c2ff84 77c88792 00c2ffac 77c8872d 0009fa58 rpcrt4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198 00c2ff8c 77c8872d 0009fa58 00000000 00000000 rpcrt4!RecvLotsaCallsWrapper+0xd 00c2ffac 77c7b110 0009cdc0 00c2ffec 77e64829 rpcrt4!BaseCachedThreadRoutine+0x9d 00c2ffb8 77e64829 0009fc80 00000000 00000000 rpcrt4!ThreadStartRoutine+0x1b WARNING: Stack unwind information not available. Following frames may be wrong. 00c2ffec 00000000 77c7b0f5 0009fc80 00000000 kernel32!GetModuleHandleA+0xdf

2 Id: f34.f94 Suspend: 1 Teb: 7ffda000 Unfrozen ChildEBP RetAddr Args to Child 00caff0c 7739c811 7739c844 00caff50 00000000 ntdll!KiFastSystemCallRet WARNING: Stack unwind information not available. Following frames may be wrong. 00caff2c 7768ffb3 00caff50 00000000 00000000 user32!PostMessageW+0x39 00caff6c 7768f366 00007530 77e61c96 000a26d8 ole32!CDllHost::STAWorkerLoop+0x72 00caff88 7768f2a2 00caffac 776bbab4 77792c30 ole32!CDllHost::WorkerThread+0xc8 00caff90 776bbab4 77792c30 00000000 000a26d8 ole32!DLLHostThreadEntry+0xd 00caffac 776b1704 00000000 00caffec 77e64829 ole32!CRpcThread::WorkerLoop+0x26 00caffb8 77e64829 000a26d8 00000000 00000000 ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x20 00caffec 00000000 776b16e4 000a26d8 00000000 kernel32!GetModuleHandleA+0xdf

3 Id: f34.f98 Suspend: 1 Teb: 7ffd8000 Unfrozen ChildEBP RetAddr Args to Child 00d7ff9c 7c826f4b 7c83d424 00000001 00d7ffb0 ntdll!KiFastSystemCallRet 00d7ffa0 7c83d424 00000001 00d7ffb0 00000000 ntdll!NtDelayExecution+0xc 00d7ffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpTimerThread+0x47 WARNING: Stack unwind information not available. Following frames may be wrong. 00d7ffec 00000000 7c83d3dd 00000000 00000000 kernel32!GetModuleHandleA+0xdf

4 Id: f34.f90 Suspend: 1 Teb: 7ffd7000 Unfrozen ChildEBP RetAddr Args to Child 00dbff70 7c8277db 7c839f38 0000017c 00dbffb0 ntdll!KiFastSystemCallRet 00dbff74 7c839f38 0000017c 00dbffb0 00dbffb4 ntdll!ZwRemoveIoCompletion+0xc 00dbffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWorkerThread+0x3d WARNING: Stack unwind information not available. Following frames may be wrong. 00dbffec 00000000 7c839efb 00000000 00000000 kernel32!GetModuleHandleA+0xdf

5 Id: f34.f70 Suspend: 1 Teb: 7ffd4000 Unfrozen ChildEBP RetAddr Args to Child 00fefee8 7c826f4b 77e41ed1 00000000 00feff2c ntdll!KiFastSystemCallRet 00fefeec 77e41ed1 00000000 00feff2c 00000000 ntdll!NtDelayExecution+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 00feff54 77e424ed 000003e8 00000000 00feffb8 kernel32!SleepEx+0x4d 00feff64 4a77de1e 000003e8 00000000 00000000 kernel32!Sleep+0xf 00feffb8 77e64829 00000000 00000000 00000000 comsvcs!PostData+0x1dd 00feffec 00000000 4a77dc41 00000000 00000000 kernel32!GetModuleHandleA+0xdf

6 Id: f34.f64 Suspend: 1 Teb: 7ffaf000 Unfrozen ChildEBP RetAddr Args to Child 0104fee8 7c827d0b 77e61d1e 000001ec 00000000 ntdll!KiFastSystemCallRet 0104feec 77e61d1e 000001ec 00000000 0104ff30 ntdll!NtWaitForSingleObject+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0104ff5c 77e61c8d 000001ec 00002710 00000000 kernel32!WaitForSingleObjectEx+0x88 0104ff70 4a77921b 000001ec 00002710 00037988 kernel32!WaitForSingleObject+0x12 0104ff84 77bcb530 4a8098d0 00000000 00000000 comsvcs!CSTAThreadPool::LoadBalanceThreadControlLoop+0x29 0104ffb8 77e64829 00037650 00000000 00000000 msvcrt!_endthreadex+0xa3 0104ffec 00000000 77bcb4bc 00037650 00000000 kernel32!GetModuleHandleA+0xdf

7 Id: f34.f7c Suspend: 1 Teb: 7ffae000 Unfrozen ChildEBP RetAddr Args to Child 0108fee0 7c827d0b 77e61d1e 000001ec 00000000 ntdll!KiFastSystemCallRet 0108fee4 77e61d1e 000001ec 00000000 0108ff28 ntdll!NtWaitForSingleObject+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0108ff54 77e61c8d 000001ec 00003a98 00000000 kernel32!WaitForSingleObjectEx+0x88 0108ff68 4a778cb6 000001ec 00003a98 00000000 kernel32!WaitForSingleObject+0x12 0108ff84 77bcb530 4a8098d0 00000000 00000000 comsvcs!CSTAThreadPool::KillThreadControlLoop+0x25 0108ffb8 77e64829 000376e8 00000000 00000000 msvcrt!_endthreadex+0xa3 0108ffec 00000000 77bcb4bc 000376e8 00000000 kernel32!GetModuleHandleA+0xdf

8 Id: f34.f80 Suspend: 1 Teb: 7ffad000 Unfrozen ChildEBP RetAddr Args to Child 010cfdcc 7c827cfb 77e6202c 00000003 010cfe1c ntdll!KiFastSystemCallRet 010cfdd0 77e6202c 00000003 010cfe1c 00000001 ntdll!NtWaitForMultipleObjects+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 010cfe78 7739bbd1 00000003 010cfea0 00000000 kernel32!WaitForMultipleObjectsEx+0xd2 010cfed4 7739ce36 00000002 010cff74 ffffffff user32!MsgWaitForMultipleObjectsEx+0xd7 010cfef0 4a77cb28 00000002 010cff74 00000000 user32!MsgWaitForMultipleObjects+0x1f 010cff84 77bcb530 000c14a0 00000000 00000000 comsvcs!CSTAThread::WorkerLoop+0x1f9 010cffb8 77e64829 00037780 00000000 00000000 msvcrt!_endthreadex+0xa3 010cffec 00000000 77bcb4bc 00037780 00000000 kernel32!GetModuleHandleA+0xdf

9 Id: f34.f68 Suspend: 1 Teb: 7ffac000 Unfrozen ChildEBP RetAddr Args to Child 0110fec8 7c8277db 77e5bea2 00000188 0110ff50 ntdll!KiFastSystemCallRet 0110fecc 77e5bea2 00000188 0110ff50 0110ff10 ntdll!ZwRemoveIoCompletion+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0110fef8 4a7ef40f 00000188 0110ff38 0110ff50 kernel32!GetQueuedCompletionStatus+0x29 0110ff74 4a7ef6aa 000addc0 00037b50 0110ffb8 comsvcs!WORK_QUEUE::WorkerLoop+0x66 0110ff84 77bcb530 000addc0 00000000 00000000 comsvcs!WORK_QUEUE::ThreadLoop+0x19 0110ffb8 77e64829 00036e40 00000000 00000000 msvcrt!_endthreadex+0xa3 0110ffec 00000000 77bcb4bc 00036e40 00000000 kernel32!GetModuleHandleA+0xdf

10 Id: f34.f54 Suspend: 1 Teb: 7ffab000 Unfrozen ChildEBP RetAddr Args to Child 0114fdcc 7c827cfb 77e6202c 00000003 0114fe1c ntdll!KiFastSystemCallRet 0114fdd0 77e6202c 00000003 0114fe1c 00000001 ntdll!NtWaitForMultipleObjects+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0114fe78 7739bbd1 00000003 0114fea0 00000000 kernel32!WaitForMultipleObjectsEx+0xd2 0114fed4 7739ce36 00000002 0114ff74 ffffffff user32!MsgWaitForMultipleObjectsEx+0xd7 0114fef0 4a77cb28 00000002 0114ff74 00000000 user32!MsgWaitForMultipleObjects+0x1f 0114ff84 77bcb530 000c45f0 00000000 00000000 comsvcs!CSTAThread::WorkerLoop+0x1f9 0114ffb8 77e64829 00037780 00000000 00000000 msvcrt!_endthreadex+0xa3 0114ffec 00000000 77bcb4bc 00037780 00000000 kernel32!GetModuleHandleA+0xdf

11 Id: f34.f5c Suspend: 1 Teb: 7ffaa000 Unfrozen ChildEBP RetAddr Args to Child 0118fdcc 7c827cfb 77e6202c 00000003 0118fe1c ntdll!KiFastSystemCallRet 0118fdd0 77e6202c 00000003 0118fe1c 00000001 ntdll!NtWaitForMultipleObjects+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0118fe78 7739bbd1 00000003 0118fea0 00000000 kernel32!WaitForMultipleObjectsEx+0xd2 0118fed4 7739ce36 00000002 0118ff74 ffffffff user32!MsgWaitForMultipleObjectsEx+0xd7 0118fef0 4a77cb28 00000002 0118ff74 00000000 user32!MsgWaitForMultipleObjects+0x1f 0118ff84 77bcb530 000c81c8 00000000 00000000 comsvcs!CSTAThread::WorkerLoop+0x1f9 0118ffb8 77e64829 00037780 00000000 00000000 msvcrt!_endthreadex+0xa3 0118ffec 00000000 77bcb4bc 00037780 00000000 kernel32!GetModuleHandleA+0xdf

12 Id: f34.f48 Suspend: 1 Teb: 7ffa9000 Unfrozen ChildEBP RetAddr Args to Child 011cfdcc 7c827cfb 77e6202c 00000003 011cfe1c ntdll!KiFastSystemCallRet 011cfdd0 77e6202c 00000003 011cfe1c 00000001 ntdll!NtWaitForMultipleObjects+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 011cfe78 7739bbd1 00000003 011cfea0 00000000 kernel32!WaitForMultipleObjectsEx+0xd2 011cfed4 7739ce36 00000002 011cff74 ffffffff user32!MsgWaitForMultipleObjectsEx+0xd7 011cfef0 4a77cb28 00000002 011cff74 00000000 user32!MsgWaitForMultipleObjects+0x1f 011cff84 77bcb530 000c8aa8 00000000 00000000 comsvcs!CSTAThread::WorkerLoop+0x1f9 011cffb8 77e64829 00037780 00000000 00000000 msvcrt!_endthreadex+0xa3 011cffec 00000000 77bcb4bc 00037780 00000000 kernel32!GetModuleHandleA+0xdf

13 Id: f34.f78 Suspend: 1 Teb: 7ffd5000 Unfrozen ChildEBP RetAddr Args to Child 00fafdcc 7c827cfb 77e6202c 00000003 00fafe1c ntdll!KiFastSystemCallRet 00fafdd0 77e6202c 00000003 00fafe1c 00000001 ntdll!NtWaitForMultipleObjects+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 00fafe78 7739bbd1 00000003 00fafea0 00000000 kernel32!WaitForMultipleObjectsEx+0xd2 00fafed4 7739ce36 00000002 00faff74 ffffffff user32!MsgWaitForMultipleObjectsEx+0xd7 00fafef0 4a77cb28 00000002 00faff74 00000000 user32!MsgWaitForMultipleObjects+0x1f 00faff84 77bcb530 000c7760 00000000 00000000 comsvcs!CSTAThread::WorkerLoop+0x1f9 00faffb8 77e64829 00037818 00000000 00000000 msvcrt!_endthreadex+0xa3 00faffec 00000000 77bcb4bc 00037818 00000000 kernel32!GetModuleHandleA+0xdf

14 Id: f34.f88 Suspend: 1 Teb: 7ffa8000 Unfrozen ChildEBP RetAddr Args to Child 0120fdcc 7c827cfb 77e6202c 00000003 0120fe1c ntdll!KiFastSystemCallRet 0120fdd0 77e6202c 00000003 0120fe1c 00000001 ntdll!NtWaitForMultipleObjects+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0120fe78 7739bbd1 00000003 0120fea0 00000000 kernel32!WaitForMultipleObjectsEx+0xd2 0120fed4 7739ce36 00000002 0120ff74 ffffffff user32!MsgWaitForMultipleObjectsEx+0xd7 0120fef0 4a77cb28 00000002 0120ff74 00000000 user32!MsgWaitForMultipleObjects+0x1f 0120ff84 77bcb530 000c92a8 00000000 00000000 comsvcs!CSTAThread::WorkerLoop+0x1f9 0120ffb8 77e64829 00037818 00000000 00000000 msvcrt!_endthreadex+0xa3 0120ffec 00000000 77bcb4bc 00037818 00000000 kernel32!GetModuleHandleA+0xdf

15 Id: f34.f8c Suspend: 1 Teb: 7ffa7000 Unfrozen ChildEBP RetAddr Args to Child 0124fdcc 7c827cfb 77e6202c 00000003 0124fe1c ntdll!KiFastSystemCallRet 0124fdd0 77e6202c 00000003 0124fe1c 00000001 ntdll!NtWaitForMultipleObjects+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0124fe78 7739bbd1 00000003 0124fea0 00000000 kernel32!WaitForMultipleObjectsEx+0xd2 0124fed4 7739ce36 00000002 0124ff74 ffffffff user32!MsgWaitForMultipleObjectsEx+0xd7 0124fef0 4a77cb28 00000002 0124ff74 00000000 user32!MsgWaitForMultipleObjects+0x1f 0124ff84 77bcb530 000c9928 00000000 00000000 comsvcs!CSTAThread::WorkerLoop+0x1f9 0124ffb8 77e64829 00037818 00000000 00000000 msvcrt!_endthreadex+0xa3 0124ffec 00000000 77bcb4bc 00037818 00000000 kernel32!GetModuleHandleA+0xdf

16 Id: f34.f74 Suspend: 1 Teb: 7ffa6000 Unfrozen ChildEBP RetAddr Args to Child 0128fdcc 7c827cfb 77e6202c 00000003 0128fe1c ntdll!KiFastSystemCallRet 0128fdd0 77e6202c 00000003 0128fe1c 00000001 ntdll!NtWaitForMultipleObjects+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0128fe78 7739bbd1 00000003 0128fea0 00000000 kernel32!WaitForMultipleObjectsEx+0xd2 0128fed4 7739ce36 00000002 0128ff74 ffffffff user32!MsgWaitForMultipleObjectsEx+0xd7 0128fef0 4a77cb28 00000002 0128ff74 00000000 user32!MsgWaitForMultipleObjects+0x1f 0128ff84 77bcb530 000ca8e0 00000000 00000000 comsvcs!CSTAThread::WorkerLoop+0x1f9 0128ffb8 77e64829 00037818 00000000 00000000 msvcrt!_endthreadex+0xa3 0128ffec 00000000 77bcb4bc 00037818 00000000 kernel32!GetModuleHandleA+0xdf

17 Id: f34.e24 Suspend: 1 Teb: 7ffa5000 Unfrozen ChildEBP RetAddr Args to Child 0268fd00 7c827d0b 77e61d1e 00000394 00000000 ntdll!KiFastSystemCallRet 0268fd04 77e61d1e 00000394 00000000 0268fd48 ntdll!NtWaitForSingleObject+0xc WARNING: Stack unwind information not available. Following frames may be wrong. 0268fd74 77e61c8d 00000394 00002710 00000000 kernel32!WaitForSingleObjectEx+0x88 0268fd88 4a756c3b 00000394 00002710 00000000 kernel32!WaitForSingleObject+0x12 0268ffb8 77e64829 000a7610 00000000 00000000 comsvcs!PingThread+0xf6 0268ffec 00000000 4a756b45 000a7610 00000000 kernel32!GetModuleHandleA+0xdf
Dmitry Vostokov Says:
July 26th, 2007 at 2:39 pm
Are you sure you have correct symbols applied? You have

WARNING: Stack unwind information not available.

always everywhere. Maybe the latest WinDbg will pick up them correctly. I had the similar problem with W2K3 R02 when I was using the previous version of WinDbg. I’m not an expert in COM+ hang dumps, it is better to ask MS support if !analyze -v -hang doesn’t point to the right direction.
There is no 3rd-party code visible in stack traces so it might be purely MS COM+ infrastructure problem or the problem in MS COM+ component.
Dmitry Vostokov Says:
October 15th, 2007 at 8:17 am
Sometimes incorrect stack trace is reported when symbols were not applied. Non-symbol gaps in stack traces can be the sign of this pattern too.

STACK_TEXT:
WARNING: Stack unwind information not available. Following frames may be wrong.
00b2f42c 091607aa mydll!foo+0×8338
00b2f4cc 7c83ab9e mydll!foo+0×8fe3
00b2f4ec 7c832d06 ntdll!RtlFindNextActivationContextSection+0×46
00b2f538 001a5574 ntdll!RtlFindActivationContextSectionString+0xe1
00b2f554 7c8302b3 0×1a5574
00b2f560 7c82f9c1 ntdll!RtlpFreeToHeapLookaside+0×22
00b2f640 7c832b7f ntdll!RtlFreeHeap+0×20e
001dd000 00080040 ntdll!LdrUnlockLoaderLock+0xad
001dd00c 0052005c 0×80040
001dd010 00470045 0×52005c
0052005c 00000000 0×470045
Steven Says:
November 27th, 2007 at 5:51 pm
Hi, i’m using windbg, because i AM a beginner and don’t have a clue about analyzing minidumps…

this is mine, i hope you maybe can get some information out of it, as for me i think it may be the video card…but it worked perfect on other computers. Memtest passed 11 times…i don’t know what the problem is

0: kd> !analyze -v

PAGE_FAULT_IN_NONPAGED_AREA (50)
Invalid system memory was referenced. This cannot be protected by try-except,
it must be protected by a Probe. Typically the address is just plain bad or it
is pointing at freed memory.
Arguments:
Arg1: e3421304, memory referenced.
Arg2: 00000000, value 0 = read operation, 1 = write operation.
Arg3: bf8afed3, If non-zero, the instruction address which referenced the bad memory
address.
Arg4: 00000001, (reserved)

Debugging Details:
——————

Could not read faulting driver name

READ_ADDRESS: e3421304

FAULTING_IP:
win32k!WatchdogDrvGetDirectDrawInfo+b1
bf8afed3 8b0e mov ecx,dword ptr [esi]

MM_INTERNAL_CODE: 1

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0×50

PROCESS_NAME: hl.exe

LAST_CONTROL_TRANSFER: from bf008af2 to bf8afed3

STACK_TEXT:
aac329c0 bf008af2 e5347080 e5657754 aac329fc win32k!WatchdogDrvGetDirectDrawInfo+0xb1
aac32a08 bf009033 00000001 e19bf680 aac32a90 dxg!vDdEnableDriver+0×8a
aac32a24 bf892b95 e5657008 00000001 e3fda008 dxg!DxDdEnableDirectDraw+0xbf
aac32a44 bf8ad4e5 00000000 e1856ad0 00000000 win32k!PDEVOBJ::bMakeSurface+0xe2
aac32a70 bf8adee5 e1856ad0 e59ec828 e4882d18 win32k!hCreateHDEV+0×3a9
aac32be8 bf8b4828 aac32cb0 e5a68400 00000001 win32k!DrvCreateMDEV+0×4dc
aac32cdc bf8b6a8d aac32cb0 e160c2d8 00000000 win32k!DrvChangeDisplaySettings+0×251
aac32d20 bf8b6980 00000000 0017ae60 00000000 win32k!xxxUserChangeDisplaySettings+0×141
aac32d48 805409ac 00000000 0017ae60 00000000 win32k!NtUserChangeDisplaySettings+0×4a
aac32d48 7c91eb94 00000000 0017ae60 00000000 nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
0012facc 00000000 00000000 00000000 00000000 0×7c91eb94

STACK_COMMAND: kb

FOLLOWUP_IP:
win32k!WatchdogDrvGetDirectDrawInfo+b1
bf8afed3 8b0e mov ecx,dword ptr [esi]

SYMBOL_STACK_INDEX: 0

SYMBOL_NAME: win32k!WatchdogDrvGetDirectDrawInfo+b1

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: win32k

IMAGE_NAME: win32k.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 45f0146f

FAILURE_BUCKET_ID: 0×50_win32k!WatchdogDrvGetDirectDrawInfo+b1

BUCKET_ID: 0×50_win32k!WatchdogDrvGetDirectDrawInfo+b1

Followup: MachineOwner
———
Dmitry Vostokov Says:
November 27th, 2007 at 6:38 pm
May be a problem related to DirectDraw dxg.sys and possibly Watchdog drivers. Is stack trace consistent among various minidumps from this computer? Check the date of dxg.sys and any updates from MS.
michaela Says:
May 22nd, 2008 at 7:29 am
Hi,

Can someone help me to explain whats happening on my server. It keep on crashing.

Windows Server 2003 Kernel Version 3790 (Service Pack 2) MP (8 procs) Free x86 compatible
Product: Server, suite: Enterprise TerminalServer SingleUserTS
Built by: 3790.srv03_sp2_rtm.070216-1710
Kernel base = 0×80800000 PsLoadedModuleList = 0×808a6ea8
Debug session time: Wed May 21 23:55:16.743 2008 (GMT+8)
System Uptime: 1 days 5:48:24.125
Loading Kernel Symbols
Loading User Symbols
Loading unloaded module list
*
* Bugcheck Analysis
*

Use !analyze -v to get detailed debugging information.

BugCheck C2, {7, 121a, 0, 8b6e6d00}

Probably caused by : Fs_Rec.SYS ( Fs_Rec!UdfsRecFsControl+63 )

Followup: MachineOwner
———

3: kd> !analyze -v;r;kv;lmtn;.logclose;q
*
* Bugcheck Analysis
*

BAD_POOL_CALLER (c2)
The current thread is making a bad pool request. Typically this is at a bad IRQL level or double freeing the same allocation, etc.
Arguments:
Arg1: 00000007, Attempt to free pool which was already freed
Arg2: 0000121a, (reserved)
Arg3: 00000000, Memory contents of the pool block
Arg4: 8b6e6d00, Address of the block of pool being deallocated

Debugging Details:
——————

POOL_ADDRESS: 8b6e6d00

FREED_POOL_TAG: Thre

BUGCHECK_STR: 0xc2_7_Thre

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: DRIVER_FAULT_SERVER_MINIDUMP

PROCESS_NAME: Rtvscan.exe

CURRENT_IRQL: 0

LAST_CONTROL_TRANSFER: from 808927bb to 80827c63

STACK_TEXT:
b86e18c0 808927bb 000000c2 00000007 0000121a nt!KeBugCheckEx+0×1b
b86e1928 8081e1b6 8b6e6d00 00000000 8b6e6af8 nt!ExFreePoolWithTag+0×477
b86e1954 f78037a1 8c8c8af8 8b95d030 b86e1988 nt!IopfCompleteRequest+0×180
b86e1964 f780309e 8c8c8af8 8a3aabd8 8c897730 Fs_Rec!UdfsRecFsControl+0×63
b86e1974 8081df65 8c8c8af8 8a3aabd8 8a3aabd8 Fs_Rec!FsRecFsControl+0×5a
b86e1988 808f785c 80a5a4d0 8b95d030 80a5a540 nt!IofCallDriver+0×45
b86e19d8 808220a4 8c8c8af8 b86e1c00 00000000 nt!IopMountVolume+0×1b4
b86e1a04 808f8910 b86e1c38 8b95d000 b86e1b40 nt!IopCheckVpbMounted+0×5c
b86e1afc 80937942 8b95d030 00000000 8a8d01e0 nt!IopParseDevice+0×3d4
b86e1b7c 80933a76 00000000 b86e1bbc 00000040 nt!ObpLookupObjectName+0×5b0
b86e1bd0 808ec76b 00000000 00000000 b86e1c01 nt!ObOpenObjectByName+0xea
b86e1d54 8088978c 05e7e2b4 05e7e28c 05e7e2d4 nt!NtQueryAttributesFile+0×11d
b86e1d54 7c8285ec 05e7e2b4 05e7e28c 05e7e2d4 nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
05e7e2d4 00000000 00000000 00000000 00000000 0×7c8285ec

STACK_COMMAND: kb

FOLLOWUP_IP:
Fs_Rec!UdfsRecFsControl+63
f78037a1 8bc6 mov eax,esi

SYMBOL_STACK_INDEX: 3

SYMBOL_NAME: Fs_Rec!UdfsRecFsControl+63

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: Fs_Rec

IMAGE_NAME: Fs_Rec.SYS

DEBUG_FLR_IMAGE_TIMESTAMP: 3e800074

FAILURE_BUCKET_ID: 0xc2_7_Thre_Fs_Rec!UdfsRecFsControl+63

BUCKET_ID: 0xc2_7_Thre_Fs_Rec!UdfsRecFsControl+63

Followup: MachineOwner
———

eax=f773713c ebx=0000121a ecx=00000000 edx=8b6e6d00 esi=f7737120 edi=000001ff
eip=80827c63 esp=b86e18a8 ebp=b86e18c0 iopl=0 nv up ei ng nz na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00000286
nt!KeBugCheckEx+0×1b:
80827c63 5d pop ebp
ChildEBP RetAddr Args to Child
b86e18c0 808927bb 000000c2 00000007 0000121a nt!KeBugCheckEx+0×1b (FPO: [Non-Fpo])
b86e1928 8081e1b6 8b6e6d00 00000000 8b6e6af8 nt!ExFreePoolWithTag+0×477 (FPO: [Non-Fpo])
b86e1954 f78037a1 8c8c8af8 8b95d030 b86e1988 nt!IopfCompleteRequest+0×180 (FPO: [Non-Fpo])
b86e1964 f780309e 8c8c8af8 8a3aabd8 8c897730 Fs_Rec!UdfsRecFsControl+0×63 (FPO: [Non-Fpo])
b86e1974 8081df65 8c8c8af8 8a3aabd8 8a3aabd8 Fs_Rec!FsRecFsControl+0×5a (FPO: [2,0,0])
b86e1988 808f785c 80a5a4d0 8b95d030 80a5a540 nt!IofCallDriver+0×45 (FPO: [Non-Fpo])
b86e19d8 808220a4 8c8c8af8 b86e1c00 00000000 nt!IopMountVolume+0×1b4 (FPO: [Non-Fpo])
b86e1a04 808f8910 b86e1c38 8b95d000 b86e1b40 nt!IopCheckVpbMounted+0×5c (FPO: [Non-Fpo])
b86e1afc 80937942 8b95d030 00000000 8a8d01e0 nt!IopParseDevice+0×3d4 (FPO: [Non-Fpo])
b86e1b7c 80933a76 00000000 b86e1bbc 00000040 nt!ObpLookupObjectName+0×5b0 (FPO: [Non-Fpo])
b86e1bd0 808ec76b 00000000 00000000 b86e1c01 nt!ObOpenObjectByName+0xea (FPO: [Non-Fpo])
b86e1d54 8088978c 05e7e2b4 05e7e28c 05e7e2d4 nt!NtQueryAttributesFile+0×11d (FPO: [Non-Fpo])
b86e1d54 7c8285ec 05e7e2b4 05e7e28c 05e7e2d4 nt!KiFastCallEntry+0xfc (FPO: [0,0] TrapFrame @ b86e1d64)
WARNING: Frame IP not in any known module. Following frames may be wrong.
05e7e2d4 00000000 00000000 00000000 00000000 0×7c8285ec
start end module name
80800000 80a56000 nt ntkrpamp.exe Sat Feb 17 13:48:00 2007 (45D69710)
80a56000 80a82000 hal halmacpi.dll Sat Feb 17 13:48:26 2007 (45D6972A)
b6e94000 b6ea6720 naveng naveng.sys Fri Feb 29 09:21:51 2008 (47C75E2F)
b6ea7000 b6f7ff80 navex15 navex15.sys Fri Feb 29 09:20:50 2008 (47C75DF2)
b734c000 b736a000 EraserUtilDrv10741 EraserUtilDrv10741.sys Fri Jan 18 10:44:27 2008 (4790128B)
b736a000 b73ca000 eeCtrl eeCtrl.sys Fri Jan 18 10:44:27 2008 (4790128B)
b7492000 b74f4000 SPBBCDrv SPBBCDrv.sys Thu Jan 11 08:09:29 2007 (45A58039)
b788e000 b78e6000 savrt savrt.sys Thu Sep 07 05:26:23 2006 (44FF3CFF)
b790e000 b7922000 Savrtpel Savrtpel.sys Thu Sep 07 05:26:26 2006 (44FF3D02)
b79ea000 b7a0c000 SYMEVENT SYMEVENT.SYS Wed Nov 08 11:38:37 2006 (4551513D)
b7a0c000 b7a29000 EraserUtilDrv10710 EraserUtilDrv10710.sys Thu Feb 01 08:54:39 2007 (45C13A4F)
b7e9a000 b7ec5000 RDPWD RDPWD.SYS Sat Feb 17 13:44:38 2007 (45D69646)
b8235000 b8240000 TDTCP TDTCP.SYS Sat Feb 17 13:44:32 2007 (45D69640)
b85a5000 b85bd000 mqac mqac.sys Sat Feb 17 13:44:18 2007 (45D69632)
b85e5000 b8642000 srv srv.sys Sat Feb 17 14:27:20 2007 (45D6A048)
b875a000 b87aa000 HTTP HTTP.sys Sat Feb 17 14:28:12 2007 (45D6A07C)
b89b2000 b89ca000 clusnet clusnet.sys Sat Feb 17 14:02:57 2007 (45D69A91)
b89f2000 b8a21000 RMCAST RMCAST.sys Sat Feb 17 13:57:43 2007 (45D69957)
b9511000 b951b000 ndisuio ndisuio.sys Sat Feb 17 13:58:25 2007 (45D69981)
b9551000 b9565000 dump_arcsas dump_arcsas.sys Thu Jul 19 08:33:04 2007 (469EB140)
b9587000 b959c000 Cdfs Cdfs.SYS Sat Feb 17 14:27:08 2007 (45D6A03C)
b95ce000 b95df000 Fips Fips.SYS Sat Feb 17 14:26:33 2007 (45D6A019)
b95df000 b9655000 mrxsmb mrxsmb.sys Sat Feb 17 14:28:15 2007 (45D6A07F)
b9655000 b9685000 rdbss rdbss.sys Sat Feb 17 14:27:37 2007 (45D6A059)
b9685000 b96af000 afd afd.sys Sat Feb 17 14:28:16 2007 (45D6A080)
b96af000 b96e0000 netbt netbt.sys Sat Feb 17 14:28:57 2007 (45D6A0A9)
b9708000 b9798000 tcpip tcpip.sys Sat Feb 17 14:28:05 2007 (45D6A075)
b9798000 b97b1000 ipsec ipsec.sys Sat Feb 17 14:29:28 2007 (45D6A0C8)
b9821000 b9838000 ClusDisk ClusDisk.sys Sat Feb 17 14:02:55 2007 (45D69A8F)
b9967000 b997c000 usbhub usbhub.sys Sat Feb 17 14:13:05 2007 (45D69CF1)
b9a1d000 b9a67000 update update.sys Sat Feb 17 14:28:59 2007 (45D6A0AB)
b9a67000 b9a9e000 rdpdr rdpdr.sys Sat Feb 17 13:51:00 2007 (45D697C4)
b9a9e000 b9ab0000 raspptp raspptp.sys Sat Feb 17 14:29:20 2007 (45D6A0C0)
b9ab0000 b9ac9000 ndiswan ndiswan.sys Sat Feb 17 14:29:22 2007 (45D6A0C2)
b9ac9000 b9add000 rasl2tp rasl2tp.sys Sat Feb 17 14:29:02 2007 (45D6A0AE)
b9add000 b9b04e00 e1000325 e1000325.sys Wed Oct 25 02:15:26 2006 (453E583E)
b9b05000 b9b30000 b57xp32 b57xp32.sys Tue Aug 29 07:10:05 2006 (44F377CD)
b9b30000 b9b45000 serial serial.sys Sat Feb 17 14:06:46 2007 (45D69B76)
b9b58000 b9b7f000 ks ks.sys Sat Feb 17 14:30:40 2007 (45D6A110)
b9b7f000 b9b93000 redbook redbook.sys Sat Feb 17 14:07:26 2007 (45D69B9E)
b9b93000 b9ba8000 cdrom cdrom.sys Sat Feb 17 14:07:48 2007 (45D69BB4)
b9ba8000 b9bd2000 USBPORT USBPORT.SYS Sat Feb 17 14:12:59 2007 (45D69CEB)
b9bd2000 b9bee000 VIDEOPRT VIDEOPRT.SYS Sat Feb 17 14:10:30 2007 (45D69C56)
b9bee000 b9d56000 ati2mtag ati2mtag.sys Thu Apr 06 11:03:52 2006 (44348518)
ba265000 ba26c000 dxgthk dxgthk.sys Tue Mar 25 15:05:52 2003 (3E7FFFD0)
ba26d000 ba275000 mouhid mouhid.sys Tue Mar 25 15:03:12 2003 (3E7FFF30)
ba275000 ba27cd80 usbccgp usbccgp.sys Sat Feb 17 14:13:08 2007 (45D69CF4)
ba27d000 ba285000 rasacd rasacd.sys Tue Mar 25 15:11:50 2003 (3E800136)
bac95000 bac9e000 hidusb hidusb.sys Tue Mar 25 15:10:17 2003 (3E8000D9)
bacd5000 bace0000 ptilink ptilink.sys Sat Feb 17 14:06:38 2007 (45D69B6E)
bace5000 bacef000 Dxapi Dxapi.sys Tue Mar 25 15:06:01 2003 (3E7FFFD9)
bad45000 bad50000 Msfs Msfs.SYS Sat Feb 17 13:50:33 2007 (45D697A9)
bad55000 bad5e000 watchdog watchdog.sys Sat Feb 17 14:11:45 2007 (45D69CA1)
bad75000 bad7e000 kbdhid kbdhid.sys Sat Feb 17 14:05:42 2007 (45D69B36)
bad85000 bad92000 wanarp wanarp.sys Sat Feb 17 13:59:17 2007 (45D699B5)
badd5000 bade0000 IPMI IPMI.sys Tue May 15 10:18:31 2007 (46491877)
bae35000 bae44000 termdd termdd.sys Sat Feb 17 13:44:32 2007 (45D69640)
bae65000 bae6f000 mouclass mouclass.sys Tue Mar 25 15:03:09 2003 (3E7FFF2D)
baf0d000 baf2c000 Mup Mup.sys Sat Feb 17 14:27:41 2007 (45D6A05D)
baf2c000 baf6b000 NDIS NDIS.sys Sat Feb 17 14:28:49 2007 (45D6A0A1)
baf6b000 bb000000 Ntfs Ntfs.sys Sat Feb 17 14:27:23 2007 (45D6A04B)
bf800000 bf9cf000 win32k win32k.sys Sat Feb 17 14:31:05 2007 (45D6A129)
bf9cf000 bf9e6000 dxg dxg.sys Sat Feb 17 14:14:39 2007 (45D69D4F)
bf9e6000 bfa29000 ati2dvag ati2dvag.dll Thu Apr 06 11:04:15 2006 (4434852F)
bfa29000 bfa68000 ati2cqag ati2cqag.dll Thu Apr 06 09:57:18 2006 (4434757E)
bfa68000 bfa9e000 atikvmag atikvmag.dll Thu Apr 06 10:25:29 2006 (44347C19)
bfa9e000 bfd280c0 ati3duag ati3duag.dll Thu Apr 06 10:47:35 2006 (44348147)
bfd29000 bfe3d180 ativvaxx ativvaxx.dll Thu Apr 06 10:40:57 2006 (44347FB9)
f720e000 f7222000 arcsas arcsas.sys Thu Jul 19 08:33:04 2007 (469EB140)
f7222000 f7241000 SCSIPORT SCSIPORT.SYS Sat Feb 17 14:28:41 2007 (45D6A099)
f7241000 f725e000 atapi atapi.sys Sat Feb 17 14:07:34 2007 (45D69BA6)
f725e000 f72be880 sddbus sddbus.sys Fri Aug 26 04:56:09 2005 (430E3069)
f72bf000 f72e9000 volsnap volsnap.sys Sat Feb 17 14:08:23 2007 (45D69BD7)
f72e9000 f7315000 dmio dmio.sys Sat Feb 17 14:10:44 2007 (45D69C64)
f7315000 f733c000 ftdisk ftdisk.sys Sat Feb 17 14:08:05 2007 (45D69BC5)
f733c000 f7352000 pci pci.sys Sat Feb 17 13:59:03 2007 (45D699A7)
f7352000 f7386000 ACPI ACPI.sys Sat Feb 17 13:58:47 2007 (45D69997)
f7487000 f7490000 WMILIB WMILIB.SYS Tue Mar 25 15:13:00 2003 (3E80017C)
f7497000 f74a6000 isapnp isapnp.sys Sat Feb 17 13:58:57 2007 (45D699A1)
f74a7000 f74b4000 PCIIDEX PCIIDEX.SYS Sat Feb 17 14:07:32 2007 (45D69BA4)
f74b7000 f74c7000 MountMgr MountMgr.sys Sat Feb 17 14:05:35 2007 (45D69B2F)
f74c7000 f74d2000 PartMgr PartMgr.sys Sat Feb 17 14:29:25 2007 (45D6A0C5)
f74d7000 f74e7000 disk disk.sys Sat Feb 17 14:07:51 2007 (45D69BB7)
f74e7000 f74f3000 Dfs Dfs.sys Sat Feb 17 13:51:17 2007 (45D697D5)
f74f7000 f7501000 crcdisk crcdisk.sys Sat Feb 17 14:09:50 2007 (45D69C2E)
f7577000 f7582000 TDI TDI.SYS Sat Feb 17 14:01:19 2007 (45D69A2F)
f7587000 f7590000 raspti raspti.sys Sat Feb 17 13:59:23 2007 (45D699BB)
f7597000 f75a0000 mssmbios mssmbios.sys Sat Feb 17 13:59:12 2007 (45D699B0)
f75a7000 f75b6000 intelppm intelppm.sys Sat Feb 17 13:48:30 2007 (45D6972E)
f75c7000 f75d1000 dump_diskdump dump_diskdump.sys Sat Feb 17 14:07:44 2007 (45D69BB0)
f75d7000 f75e1000 serenum serenum.sys Sat Feb 17 14:06:44 2007 (45D69B74)
f75e7000 f75f2000 kbdclass kbdclass.sys Sat Feb 17 14:05:39 2007 (45D69B33)
f75f7000 f7605000 NDProxy NDProxy.SYS Sat Feb 17 13:59:21 2007 (45D699B9)
f7607000 f7614000 netbios netbios.sys Sat Feb 17 13:58:29 2007 (45D69985)
f7617000 f7625000 msgpc msgpc.sys Sat Feb 17 13:58:37 2007 (45D6998D)
f7637000 f7641000 ibmhpa ibmhpa.sys Wed Aug 31 02:51:17 2005 (4314AAA5)
f7647000 f7654000 Npfs Npfs.SYS Sat Feb 17 13:50:36 2007 (45D697AC)
f7657000 f7665000 HIDCLASS HIDCLASS.SYS Tue Mar 25 15:10:17 2003 (3E8000D9)
f7687000 f7690000 ndistapi ndistapi.sys Sat Feb 17 13:59:19 2007 (45D699B7)
f76a7000 f76b6000 raspppoe raspppoe.sys Sat Feb 17 13:59:23 2007 (45D699BB)
f76b7000 f76c3000 vga vga.sys Sat Feb 17 14:10:30 2007 (45D69C56)
f76d7000 f76e8ac0 AACMgt AACMgt.sys Mon Nov 14 09:26:15 2005 (4377E7B7)
f76e9000 f7707000 storport storport.sys Sat Feb 17 14:07:48 2007 (45D69BB4)
f7707000 f770f000 kdcom kdcom.dll Tue Mar 25 15:08:00 2003 (3E800050)
f770f000 f7717000 BOOTVID BOOTVID.dll Tue Mar 25 15:07:58 2003 (3E80004E)
f7717000 f771e000 pciide pciide.sys Tue Mar 25 15:04:46 2003 (3E7FFF8E)
f771f000 f7726000 dmload dmload.sys Tue Mar 25 15:08:08 2003 (3E800058)
f775f000 f7767000 RDPCDD RDPCDD.sys Tue Mar 25 15:03:05 2003 (3E7FFF29)
f77df000 f77e3400 usbohci usbohci.sys Sat Feb 17 14:13:01 2007 (45D69CED)
f77e7000 f77edb80 usbehci usbehci.sys Sat Feb 17 14:12:56 2007 (45D69CE8)
f77ef000 f77f7000 pnpmem pnpmem.sys Sat Feb 17 13:59:03 2007 (45D699A7)
f77f7000 f77ff000 audstub audstub.sys Tue Mar 25 15:09:12 2003 (3E800098)
f77ff000 f7807000 Fs_Rec Fs_Rec.SYS Tue Mar 25 15:08:36 2003 (3E800074)
f7807000 f780e000 Null Null.SYS Tue Mar 25 15:03:05 2003 (3E7FFF29)
f780f000 f7816000 Beep Beep.SYS Tue Mar 25 15:03:04 2003 (3E7FFF28)
f7817000 f781d300 HIDPARSE HIDPARSE.SYS Sat Feb 17 14:12:35 2007 (45D69CD3)
f781f000 f7827000 mnmdd mnmdd.SYS Tue Mar 25 15:07:53 2003 (3E800049)
f7839000 f785f000 KSecDD KSecDD.sys Sat Feb 17 13:46:32 2007 (45D696B8)
f785f000 f7884000 fltmgr fltmgr.sys Sat Feb 17 13:51:08 2007 (45D697CC)
f7884000 f7897000 CLASSPNP CLASSPNP.SYS Sat Feb 17 14:28:16 2007 (45D6A080)
f7897000 f789a280 sddfltr sddfltr.sys Fri Aug 26 04:56:09 2005 (430E3069)
f7987000 f7988b00 sdddisk sdddisk.sys Fri Aug 26 04:56:08 2005 (430E3068)
f7999000 f799a280 swenum swenum.sys Sat Feb 17 14:05:56 2007 (45D69B44)
f799f000 f79a0580 USBD USBD.SYS Tue Mar 25 15:10:39 2003 (3E8000EF)
f7b2f000 f7bdf000 ql2300 ql2300.sys Fri Nov 05 06:05:34 2004 (418AA7AE)

Unloaded modules:
b7160000 b7173000 naveng.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b7173000 b724c000 navex15.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b7260000 b7273000 naveng.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b7273000 b734c000 navex15.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b77ad000 b77bf000 naveng.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b77bf000 b788e000 navex15.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b7a29000 b7a8a000 eeCtrl.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b77ad000 b77bf000 naveng.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b77bf000 b788e000 navex15.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
b9b45000 b9b58000 i8042prt.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
bad35000 bad43000 imapi.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
f75b7000 f75c0000 kbdhid.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
badf5000 badff000 Flpydisk.SYS
Timestamp: unavailable (00000000)
Checksum: 00000000
bae05000 bae10000 Fdc.SYS
Timestamp: unavailable (00000000)
Checksum: 00000000
molotov Says:
May 22nd, 2008 at 5:14 pm
michaela,

I’d suggest enabling Verifier (http://msdn.microsoft.com/en-us/library/ms792872.aspx) on all non-MS drivers. This should get you more information when the next bugcheck happens.
Crash Dump Analysis » Blog Archive » 10 Common Mistakes in Memory Analysis (Part 2) Says:
June 17th, 2008 at 2:20 pm
[…] would FreeHeap need a file name? See Incorrect Stack Trace pattern case study for semantic inconsistency. Why is this function on the stack […]
!analyze -v : Crash Dump Analysis Patterns (Part 11) Says:
September 18th, 2008 at 9:04 am
[…] Crash Dump Analysis Patterns (Part 11) […]
Crash Dump Analysis » Blog Archive » Null data pointer, incorrect stack trace, changed environment, hooked functions and coincidental symbolic information: pattern cooperation Says:
April 28th, 2009 at 5:56 pm
[…] Unfortunately due to the lack of gui_plugin symbols the stack trace is incorrect: […]
Crash Dump Analysis » Blog Archive » Inconsistent dump, blocked threads, wait chains, incorrect stack trace and process factory: pattern cooperation Says:
June 9th, 2009 at 11:14 pm
[…] about 45 threads are waiting that mutant 871d47d0 owned by thread 867322e0, like this one with incorrect stack trace because of inconsistent dump file […]
Crash Dump Analysis » Blog Archive » Invalid pointer, incorrect stack trace, multiple exceptions, insufficient memory and memory leak: pattern cooperation Says:
June 10th, 2009 at 2:55 pm
[…] warning about frames suggest that stack trace could be incorrect but backwards disassembly and preceding frames show that it is correct […]
Crash Dump Analysis » Blog Archive » The Importance of Symbols Says:
June 14th, 2009 at 12:46 am
[…] post looks at incorrect stack traces in more detail and provides an example and explanation of why WinDbg is not able to get them right […]
Crash Dump Analysis » Blog Archive » Virtualized process, incorrect stack trace, stack trace collection, multiple exceptions, optimized code and C++ exception: pattern cooperation Says:
July 8th, 2009 at 8:54 pm
[…] the shown above stack trace is incorrect but at the same time the first thread stack looks […]
Crash Dump Analysis » Blog Archive » WOW64 process, NULL data pointer, stack overflow, main thread, incorrect stack trace, nested exceptions, hidden exception, manual dump, multiple exceptions and virtualized system: pattern cooperation Says:
October 5th, 2009 at 10:20 pm
[…] there is a possibility of an exception in main GUI thread and also the stack trace itself looks incorrect, suddenly sending a Windows message without any kind of a message […]
Crash Dump Analysis » Blog Archive » Manual dump, virtualized process, stack trace collection, multiple exceptions, optimized code, wild code pointer, incorrect stack trace and hidden exception: pattern cooperation Says:
October 14th, 2009 at 7:46 pm
[…] 0×161dc2c so we might guess that this was an instance of wild code pointer or the case of incorrect stack trace. However using techniques to get exception context from hidden exceptions we get the following […]
Crash Dump Analysis » Blog Archive » Reflecting on 2008 (Part 1) Says:
January 27th, 2010 at 12:47 am
[…] dereferencing null debug windows crash dump analysis system_service_exception kernel32!pnlsuserinfo warning: frame ip not in any known module. following frames may be wrong. win32 error 0n2 previously announced volume is available in trade windbg script […]
Crash Dump Analysis » Blog Archive » Icons for Memory Dump Analysis Patterns (Part 19) Says:
April 7th, 2010 at 2:53 pm
[…] we introduce an icon for Incorrect Stack Trace […]
Crash Dump Analysis » Blog Archive » Incorrect stack trace, stack overflow, early crash dump, nested exception, problem exception handler and same vendor: pattern cooperation Says:
October 30th, 2010 at 11:59 pm
[…] Debugging Experts Magazine Online This case study centers on 3 process dump files (two first chance exception and one second chance exception). To recall the difference between them please read first chance exceptions explained series. When we get first and second chance exception dumps together we usually open a second chance exception dump first. However, in this case, the second chance exception dump had an incorrect stack trace: […]
Crash Dump Analysis » Blog Archive » Crash Dump Analysis Patterns (Part 114) Says:
November 9th, 2010 at 12:03 pm
[…] trace may or may not be included here and it might be incorrect, heuristic and not fully discernible automatically (requires raw stack semantic analysis) like in […]
Dmitry Vostokov Says:
February 17th, 2014 at 12:00 am
To check the correctness of some frames we can use the same method as described in Coincidental Symbolic Information pattern:

http://www.dumpanalysis.org/blog/index.php/2007/08/30/crash-dump-analysis-patterns-part-24/

We use backwards disassembly on a return address:

0286f430 690e6daa mshtml!CBase::PrivateInvokeEx+0×6d
WARNING: Stack unwind information not available. Following frames may be wrong.
0286f494 6915f5c5 jscript9!DllGetClassObject+0×18bb1

0:005> ub 690e6daa
jscript9!DllGetClassObject+0×18b9e:
690e6d97 ff7514 push dword ptr [ebp+14h]
690e6d9a ff7510 push dword ptr [ebp+10h]
690e6d9d 8b06 mov eax,dword ptr [esi]
690e6d9f 53 push ebx
690e6da0 ff75ec push dword ptr [ebp-14h]
690e6da3 ff7508 push dword ptr [ebp+8]
690e6da6 56 push esi
690e6da7 ff5020 call dword ptr [eax+20h]

Crash Dump Analysis Patterns (Part 11)

22 Responses to “Crash Dump Analysis Patterns (Part 11)”

Leave a Reply