Crash Dump Analysis Patterns (Part 181)

Very useful pattern for the analysis of memory dumps from terminal services environments is Incomplete Session. Normally the session processes include csrss.exe, winlogon.exe, wfshell.exe (in case of Citrix), explorer.exe and a few user defined processes such as winword.exe, for example:

0: kd> !session
Sessions on machine: 6
Valid Sessions: 0 1 3 5 6 8

0: kd> !sprocess 6
Dumping Session 6

_MM_SESSION_SPACE fffffa6009447000
_MMSESSION        fffffa6009447b40
PROCESS fffffa800fcee630
SessionId: 6  Cid: 1974    Peb: 7fffffd5000  ParentCid: 147c
DirBase: 158baf000  ObjectTable: fffff8801ef13b00  HandleCount: 532.
Image: csrss.exe

PROCESS fffffa800fc77040
SessionId: 6  Cid: 1ae4    Peb: 7fffffde000  ParentCid: 147c
DirBase: 15d2b4000  ObjectTable: fffff8802084b570  HandleCount: 238.
Image: winlogon.exe

PROCESS fffffa800fe61040
SessionId: 6  Cid: 1edc    Peb: 7efdf000  ParentCid: 1ec8
DirBase: 14df74000  ObjectTable: fffff88020f486e0  HandleCount: 313.
Image: wfshell.exe

PROCESS fffffa800ff5a660
SessionId: 6  Cid: 2054    Peb: 7fffffdf000  ParentCid: 1dbc
DirBase: 201a81000  ObjectTable: fffff88020dd56e0  HandleCount: 447.
Image: explorer.exe

PROCESS fffffa800fe28040
SessionId: 6  Cid: 1ce4    Peb: 7efdf000  ParentCid: 13a8
DirBase: 11f552000  ObjectTable: fffff8801fe96990  HandleCount: 1842.
Image: WINWORD.EXE

PROCESS fffffa800f119c10
SessionId: 6  Cid: 2074    Peb: 7efdf000  ParentCid: 2054
DirBase: 2d994f000  ObjectTable: fffff8801e76aec0  HandleCount: 673.
Image: iexplore.exe

If we compare with the last session #8 we see that the latter has only 2 processes:

0: kd> !sprocess 8
Dumping Session 8

_MM_SESSION_SPACE fffffa600bafc000
_MMSESSION        fffffa600bafcb40
PROCESS fffffa80103a4480
SessionId: 8  Cid: 2858    Peb: 7fffffdf000  ParentCid: 2660
DirBase: a04bb000  ObjectTable: fffff8801cb926a0  HandleCount: 534.
Image: csrss.exe

PROCESS fffffa801065b770
SessionId: 8  Cid: 2878    Peb: 7fffffdf000  ParentCid: 2660
DirBase: 5da40000  ObjectTable: fffff8801ce5e440  HandleCount: 235.
Image: winlogon.exe

Such anomalies may point to a disconnected session that failed to terminate due to some unresponsive session process or a session that is stuck in session initialization process launch sequence due to threads blocked in wait chains so process threads need to be analyzed.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

4 Responses to “Crash Dump Analysis Patterns (Part 181)”

  1. Marc Sherman Says:

    I think TS also preallocates sessions to make logon faster. In that case I believe such a session would only have two processes: csrss.exe and winlogon.exe.

  2. Dmitry Vostokov Says:

    If preallocation really happens this might be the case for RDP but not for ICA. Of course, when a user cannot connect or there is an error message or a hanging progress bar then we we should look at such incomplete session first. In real scenarious usually we see threads in session processes that are blocked in ALPC to session manager or terminal service or LSA, etc.

    Did you see prellocation in RDP dumps? What I know is that there are indeed a few listener threads in terminal service for incoming TS connections (but not processes in ICA, for example). I have seen less problem RDP complete dumps than ICA ones as former are usually sent to MS not to me :-) I’ll check that indeed

  3. Marc Sherman Says:

    IIRC, I’ve seen them on live TS machines and they were definitely RDP as opposed to ICA. I believe I’ve also seen them in TS dumps (at least once for sure).

  4. Dmitry Vostokov Says:

    In case of many terminal sessions on Windows we can dump processes sorted by session via !sprocess -4 to spot Incomplete Sessions.

Leave a Reply