Crash Dump Analysis Patterns (Part 246)

Visio was freezing after saving a diagram as a picture after we tried to close it. It eventually crashed with WER saving a crash dump file in LocalDumps folder. After a few such incidents Visio suggested to disable a 3rd-party plugin. We did that and double checked in Options \ Add-Ins dialog. Unfortunately, the same abnormal behaviour continued. When we looked at the crash dump stack trace we noticed Foreign Module Frame:

0:000> k
# ChildEBP RetAddr
00 0019cbac 746b1556 ntdll!NtWaitForMultipleObjects+0xc
01 0019cd40 746b1408 KERNELBASE!WaitForMultipleObjectsEx+0x136
02 0019cd5c 747ea02a KERNELBASE!WaitForMultipleObjects+0x18
03 0019d198 747e9ac6 kernel32!WerpReportFaultInternal+0x545
04 0019d1a8 747ccf09 kernel32!WerpReportFault+0x7a
05 0019d1b0 746c9f53 kernel32!BasepReportFault+0x19
06 0019d244 76fc2de5 KERNELBASE!UnhandledExceptionFilter+0x1b3
07 0019d2e8 76f8acd6 ntdll!LdrpLogFatalUserCallbackException+0x4d
08 0019d2f4 76f9d572 ntdll!KiUserCallbackExceptionHandler+0x26
09 0019d318 76f9d544 ntdll!ExecuteHandler2+0x26
0a 0019d3e0 76f8ad8f ntdll!ExecuteHandler+0x24
0b 0019d3e0 55403000 ntdll!KiUserExceptionDispatcher+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0c 0019d8d0 55402faa VISLIB!Ordinal1+0x24f3b
0d 0019d914 5b85c67e VISLIB!Ordinal1+0x24ee5
0e 0019d940 5b85c638 MSO!Ordinal2138+0x10a
0f 0019d950 5b8e7620 MSO!Ordinal2138+0xc4
10 0019d964 5b8e7602 MSO!Ordinal9998+0x3bc
11 0019d97c 5bc938a6 MSO!Ordinal9998+0x39e
12 0019dbb0 5c240add MSO!Ordinal7238+0x25bef
13 0019ddec 65598ed1 MSO!Ordinal2007+0x1766
14 0019de78 655c5eaa VisioPlugin!DllRegisterServer+0×43bf1
15 0019dfbc 555601db VisioPlugin!DllRegisterServer+0×70bca

16 0019dfe8 5555fe61 VISLIB!Ordinal1+0×182116
17 0019e028 55421b7c VISLIB!Ordinal1+0×181d9c
18 0019e070 5549f1a9 VISLIB!Ordinal1+0×43ab7
19 0019e090 5549ebba VISLIB!Ordinal1+0xc10e4
1a 0019e0c0 5540dd14 VISLIB!Ordinal1+0xc0af5
1b 0019e110 55426168 VISLIB!Ordinal1+0×2fc4f
1c 0019e134 55425446 VISLIB!Ordinal1+0×480a3
1d 0019e20c 5549eace VISLIB!Ordinal1+0×47381
1e 0019e264 5549e90e VISLIB!Ordinal1+0xc0a09
1f 0019e28c 6571fb03 VISLIB!Ordinal1+0xc0849
20 0019e334 6571f6cc mfc90u!CWnd::OnWndMsg+0×410
21 0019e354 553ef572 mfc90u!CWnd::WindowProc+0×24
22 0019e370 6571e2f2 VISLIB!Ordinal1+0×114ad
23 0019e3d8 6571e57e mfc90u!AfxCallWndProc+0xa3
24 0019e3fc 553ef518 mfc90u!AfxWndProc+0×37
25 0019e440 553ef4d9 VISLIB!Ordinal1+0×11453
26 0019e458 553ef49e VISLIB!Ordinal1+0×11414
27 0019e480 553ef338 VISLIB!Ordinal1+0×113d9
28 0019e49c 553ef2d6 VISLIB!Ordinal1+0×11273
29 0019e4c4 553ef107 VISLIB!Ordinal1+0×11211
2a 0019e528 75864923 VISLIB!Ordinal1+0×11042
2b 0019e554 75844790 user32!_InternalCallWinProc+0×2b
2c 0019e5fc 75844527 user32!UserCallWinProcCheckWow+0×1f0
2d 0019e638 71db7d40 user32!CallWindowProcW+0×97
2e 0019e6b8 71db7996 comctl32!CallNextSubclassProc+0×140
2f 0019e6d8 5b84d95a comctl32!DefSubclassProc+0×56
30 0019e720 5b84d7ad MSO!Ordinal6319+0×25e
31 0019e74c 71db7db8 MSO!Ordinal6319+0xb1
32 0019e7d0 71db7b61 comctl32!CallNextSubclassProc+0×1b8
33 0019e82c 75864923 comctl32!MasterSubclassProc+0xa1
34 0019e858 75844790 user32!_InternalCallWinProc+0×2b
35 0019e900 75844370 user32!UserCallWinProcCheckWow+0×1f0
36 0019e960 7584b179 user32!DispatchClientMessage+0xf0
37 0019e9a0 76f8ad66 user32!__fnDWORD+0×49
38 0019e9d8 75864dac ntdll!KiUserCallbackDispatcher+0×36
39 0019e9dc 75842ce8 user32!NtUserMessageCall+0xc
3a 0019ea68 758423ba user32!RealDefWindowProcWorker+0×148
3b 0019ea80 71f882ee user32!RealDefWindowProcW+0×5a
3c 0019eaa0 71f88145 uxtheme!DoMsgDefault+0×3a
3d 0019eab0 71f87bba uxtheme!OnDwpSysCommand+0×35
3e 0019eb1c 71f868d8 uxtheme!_ThemeDefWindowProc+0×6ca
3f 0019eb30 75842b66 uxtheme!ThemeDefWindowProcW+0×18
40 0019eb80 758415ee user32!DefWindowProcW+0×176
41 0019eb98 75851e3b user32!DefWindowProcWorker+0×2e
42 0019ec1c 758aa09b user32!DefFrameProcWorker+0xb7
43 0019ec34 55718ac5 user32!DefFrameProcW+0×1b
44 0019ec58 55708027 VISLIB!Ordinal1+0×33aa00
45 0019ec70 6571e3c1 VISLIB!Ordinal1+0×329f62
46 0019ec84 65725604 mfc90u!CWnd::Default+0×30
47 0019ec94 5549e617 mfc90u!CFrameWnd::OnSysCommand+0×50
48 0019ecb4 6571fd15 VISLIB!Ordinal1+0xc0552
49 0019ed64 6571f6cc mfc90u!CWnd::OnWndMsg+0×622
4a 0019ed84 553ef572 mfc90u!CWnd::WindowProc+0×24
4b 0019eda0 6571e2f2 VISLIB!Ordinal1+0×114ad
4c 0019ee08 6571e57e mfc90u!AfxCallWndProc+0xa3
4d 0019ee2c 553ef518 mfc90u!AfxWndProc+0×37
4e 0019ee70 553ef4d9 VISLIB!Ordinal1+0×11453
4f 0019ee88 553ef49e VISLIB!Ordinal1+0×11414
50 0019eeb0 553ef338 VISLIB!Ordinal1+0×113d9
51 0019eecc 553ef2d6 VISLIB!Ordinal1+0×11273
52 0019eef4 553ef107 VISLIB!Ordinal1+0×11211
53 0019ef58 75864923 VISLIB!Ordinal1+0×11042
54 0019ef84 75844790 user32!_InternalCallWinProc+0×2b
55 0019f02c 75844527 user32!UserCallWinProcCheckWow+0×1f0
56 0019f068 71db7d40 user32!CallWindowProcW+0×97
57 0019f0e8 71db7996 comctl32!CallNextSubclassProc+0×140
58 0019f108 5b84d95a comctl32!DefSubclassProc+0×56
59 0019f150 5b84d7ad MSO!Ordinal6319+0×25e
5a 0019f17c 71db7db8 MSO!Ordinal6319+0xb1
5b 0019f200 71db7b61 comctl32!CallNextSubclassProc+0×1b8
5c 0019f25c 75864923 comctl32!MasterSubclassProc+0xa1
5d 0019f288 75844790 user32!_InternalCallWinProc+0×2b
5e 0019f330 75844370 user32!UserCallWinProcCheckWow+0×1f0
5f 0019f390 7584b179 user32!DispatchClientMessage+0xf0
60 0019f3d0 76f8ad66 user32!__fnDWORD+0×49
61 0019f408 00000000 ntdll!KiUserCallbackDispatcher+0×36

Next, we applied lmv WinDbg command to the module name and followed its image path to rename it. After that, the problem disappeared. We call such modules Foreign because they were created not by the OS or the main process module vendors. Most likely these modules are either value-adding plugins or exposed Message Hooks.

- Dmitry Vostokov @ + -

Leave a Reply