Crash Dump Analysis Patterns (Part 181)
Very useful pattern for the analysis of memory dumps from terminal services environments is Incomplete Session. Normally the session processes include csrss.exe, winlogon.exe, wfshell.exe (in case of Citrix), explorer.exe and a few user defined processes such as winword.exe, for example:
0: kd> !session
Sessions on machine: 6
Valid Sessions: 0 1 3 5 6 8
0: kd> !sprocess 6
Dumping Session 6
_MM_SESSION_SPACE fffffa6009447000
_MMSESSION fffffa6009447b40
PROCESS fffffa800fcee630
SessionId: 6 Cid: 1974 Peb: 7fffffd5000 ParentCid: 147c
DirBase: 158baf000 ObjectTable: fffff8801ef13b00 HandleCount: 532.
Image: csrss.exe
PROCESS fffffa800fc77040
SessionId: 6 Cid: 1ae4 Peb: 7fffffde000 ParentCid: 147c
DirBase: 15d2b4000 ObjectTable: fffff8802084b570 HandleCount: 238.
Image: winlogon.exe
PROCESS fffffa800fe61040
SessionId: 6 Cid: 1edc Peb: 7efdf000 ParentCid: 1ec8
DirBase: 14df74000 ObjectTable: fffff88020f486e0 HandleCount: 313.
Image: wfshell.exe
PROCESS fffffa800ff5a660
SessionId: 6 Cid: 2054 Peb: 7fffffdf000 ParentCid: 1dbc
DirBase: 201a81000 ObjectTable: fffff88020dd56e0 HandleCount: 447.
Image: explorer.exe
PROCESS fffffa800fe28040
SessionId: 6 Cid: 1ce4 Peb: 7efdf000 ParentCid: 13a8
DirBase: 11f552000 ObjectTable: fffff8801fe96990 HandleCount: 1842.
Image: WINWORD.EXE
PROCESS fffffa800f119c10
SessionId: 6 Cid: 2074 Peb: 7efdf000 ParentCid: 2054
DirBase: 2d994f000 ObjectTable: fffff8801e76aec0 HandleCount: 673.
Image: iexplore.exe
If we compare with the last session #8 we see that the latter has only 2 processes:
0: kd> !sprocess 8
Dumping Session 8
_MM_SESSION_SPACE fffffa600bafc000
_MMSESSION fffffa600bafcb40
PROCESS fffffa80103a4480
SessionId: 8 Cid: 2858 Peb: 7fffffdf000 ParentCid: 2660
DirBase: a04bb000 ObjectTable: fffff8801cb926a0 HandleCount: 534.
Image: csrss.exe
PROCESS fffffa801065b770
SessionId: 8 Cid: 2878 Peb: 7fffffdf000 ParentCid: 2660
DirBase: 5da40000 ObjectTable: fffff8801ce5e440 HandleCount: 235.
Image: winlogon.exe
Such anomalies may point to a disconnected session that failed to terminate due to some unresponsive session process or a session that is stuck in session initialization process launch sequence due to threads blocked in wait chains so process threads need to be analyzed.
- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -
October 2nd, 2012 at 1:17 pm
I think TS also preallocates sessions to make logon faster. In that case I believe such a session would only have two processes: csrss.exe and winlogon.exe.
October 2nd, 2012 at 7:15 pm
If preallocation really happens this might be the case for RDP but not for ICA. Of course, when a user cannot connect or there is an error message or a hanging progress bar then we we should look at such incomplete session first. In real scenarious usually we see threads in session processes that are blocked in ALPC to session manager or terminal service or LSA, etc.
Did you see prellocation in RDP dumps? What I know is that there are indeed a few listener threads in terminal service for incoming TS connections (but not processes in ICA, for example). I have seen less problem RDP complete dumps than ICA ones as former are usually sent to MS not to me
I’ll check that indeed
October 3rd, 2012 at 1:40 pm
IIRC, I’ve seen them on live TS machines and they were definitely RDP as opposed to ICA. I believe I’ve also seen them in TS dumps (at least once for sure).
December 14th, 2016 at 3:42 pm
In case of many terminal sessions on Windows we can dump processes sorted by session via !sprocess -4 to spot Incomplete Sessions.