Crash Dump Analysis Patterns (Part 58a)
Wednesday, April 9th, 2008Among various memory leaks leading to Insufficient Memory pattern there is so called session pool leak briefly touched in the previous kernel pool leaks post. I decided to factor it out separately because it also involves GDI handles and structures allocated per user session that has the limit on how many of them can be created and this case should rather be called Handle Limit. Such leaks can result in poor visual application behavior after some time when drawing requests are not satisfied anymore. In severe cases, when the same bugs are present in a display driver, it can result in bugchecks like
BugCheck AB: SESSION_HAS_VALID_POOL_ON_EXIT
or, if a handle allocation request was not satisfied, it may result in a NULL pointer stored somewhere with the subsequent Invalid Pointer access:
SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)
CONTEXT: b791e010 -- (.cxr 0xffffffffb791e010)
eax=00000000 ebx=bc43d004 ecx=a233add8 edx=00000000 esi=bc430fff edi=00000000
eip=bfe7d380 esp=b791e3dc ebp=b791e480 iopl=0 nv up ei pl zr na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010246
DisplayDriver+0×3e380:
bfe7d380 8a4702 mov al,byte ptr [edi+2] ds:0023:00000002=??
Rather than providing examples from specific kernel dump files I created 3 Win32 applications in Visual C++ that simulate GDI leaks. All of them create GDI objects in a loop and select them into their current graphics device context (DC) on Windows Server 2003 x64 SP2. Before running them I got the following session paged pool statistics:
lkd> !poolused c
Sorting by Session Paged Pool Consumed
Pool Used:
NonPaged Paged
Tag Allocs Used Allocs Used
NV_x 0 0 5 14024704 UNKNOWN pooltag 'NV_x', please update pooltag.txt
BIG 0 0 257 3629056 Large session pool allocations (ntos\ex\pool.c) , Binary: nt!mm
NV 0 0 203 1347648 nVidia video driver
Ttfd 0 0 233 1053152 TrueType Font driver
Gh05 0 0 391 1050400 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gla1 0 0 348 785088 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gcac 0 0 25 640880 Gdi glyph cache
Gla5 0 0 631 323072 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gdrs 0 0 33 172288 Gdi GDITAG_DRVSUP
Gla: 0 0 212 139072 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gla4 0 0 487 116880 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Usti 0 0 148 97088 THREADINFO , Binary: win32k!AllocateW32Thread
Gla8 0 0 383 91920 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gla@ 0 0 339 70512 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gbaf 0 0 48 67584 UNKNOWN pooltag ‘Gbaf’, please update pooltag.txt
knlf 0 0 20 66496 UNKNOWN pooltag ‘knlf’, please update pooltag.txt
GDev 0 0 7 57344 Gdi pdev
Usqu 0 0 152 53504 Q , Binary: win32k!InitQEntryLookaside
Uscu 0 0 334 53440 CURSOR , Binary: win32k!_CreateEmptyCursorObject
Bmfd 0 0 21 50224 Font related stuff
Uspi 0 0 153 40000 PROCESSINFO , Binary: win32k!MapDesktop
Gfnt 0 0 47 39856 UNKNOWN pooltag ‘Gfnt’, please update pooltag.txt
Ggb 0 0 34 39088 Gdi glyph bits
Gh08 0 0 33 38656 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Ghab 0 0 228 32832 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Ovfl 0 0 1 32768 The internal pool tag table has overflowed - usually this is a result of nontagged allocations being made
Gpff 0 0 88 27712 Gdi physical font file
Gpfe 0 0 88 27600 UNKNOWN pooltag ‘Gpfe’, please update pooltag.txt
thdd 0 0 1 20480 DirectDraw/3D handle manager table
Gebr 0 0 17 19776 Gdi ENGBRUSH
Gh0@ 0 0 86 19264 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gsp 0 0 79 18960 Gdi sprite
HT40 0 0 2 16384 UNKNOWN pooltag ‘HT40′, please update pooltag.txt
Gpat 0 0 4 16192 UNKNOWN pooltag ‘Gpat’, please update pooltag.txt
Ggls 0 0 169 12944 Gdi glyphset
Glnk 0 0 371 11872 Gdi PFELINK
Gldv 0 0 9 11248 Gdi Ldev
Gffv 0 0 84 9408 Gdi FONTFILEVIEW
Gfsb 0 0 1 8192 Gdi font sustitution list
Uskt 0 0 2 7824 KBDTABLE , Binary: win32k!ReadLayoutFile
Gh04 0 0 7 5856 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gdcf 0 0 51 5712 UNKNOWN pooltag ‘Gdcf’, please update pooltag.txt
Gh0< 0 0 88 5632 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gglb 0 0 1 4096 Gdi temp buffer
Ustm 0 0 30 3360 TIMER , Binary: win32k!InternalSetTimer
Gspm 0 0 39 3120 UNKNOWN pooltag ‘Gspm’, please update pooltag.txt
Usac 0 0 16 3056 ACCEL , Binary: win32k!_CreateAcceleratorTable
Usqm 0 0 25 2800 QMSG , Binary: win32k!InitQEntryLookaside
Ghas 0 0 3 2592 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Uscl 0 0 20 2128 CLASS , Binary: win32k!ClassAlloc
Uswl 0 0 1 2032 WINDOWLIST , Binary: win32k!BuildHwndList
Gmul 0 0 19 1520 UNKNOWN pooltag ‘Gmul’, please update pooltag.txt
Dddp 0 0 8 1472 UNKNOWN pooltag ‘Dddp’, please update pooltag.txt
Ggdv 0 0 8 1472 Gdi GDITAG_GDEVICE
UsDI 0 0 4 1408 DEVICEINFO , Binary: win32k!CreateDeviceInfo
Vtfd 0 0 4 1312 Font file/context
Ushk 0 0 20 1280 HOTKEY , Binary: win32k!_RegisterHotKey
Gspr 0 0 3 1264 Gdi sprite grow range
Gtmw 0 0 13 1248 Gdi TMW_INTERNAL
Gxlt 0 0 8 1152 Gdi Xlate
Gpft 0 0 2 944 Gdi font table
Uspp 0 0 5 944 PNP , Binary: win32k!AllocateAndLinkHidTLCInf
Ussm 0 0 7 896 SMS , Binary: win32k!InitSMSLookaside
Gdbr 0 0 10 800 Gdi driver brush realization
Usdc 0 0 8 768 DCE , Binary: win32k!CreateCacheDC
Usct 0 0 12 768 CHECKPT , Binary: win32k!CkptRestore
Usim 0 0 2 736 IME , Binary: win32k!CreateInputContext
Usci 0 0 3 720 CLIENTTHREADINFO , Binary: win32k!InitSystemThread
Gh09 0 0 1 640 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Ussy 1 80 4 608 SYSTEM , Binary: win32k!xxxDesktopThread
Urdr 0 0 9 576 REDIRECT , Binary: win32k!SetRedirectionBitmap
Uswd 0 0 2 576 WINDOW , Binary: win32k!xxxCreateWindowEx
Uscb 0 0 3 544 CLIPBOARD , Binary: win32k!_ConvertMemHandle
Gcsl 0 0 1 496 Gdi string resource script names
Ustx 0 0 10 496 TEXT , Binary: win32k!NtUserDrawCaptionTemp
Ussw 0 0 1 496 SWP , Binary: win32k!_BeginDeferWindowPos
Gdev 0 0 2 480 Gdi GDITAG_DEVMODE
Usih 0 0 10 480 IMEHOTKEY , Binary: win32k!SetImeHotKey
Gdrv 0 0 1 368 UNKNOWN pooltag ‘Gdrv’, please update pooltag.txt
GVdv 0 0 1 320 UNKNOWN pooltag ‘GVdv’, please update pooltag.txt
Gmap 0 0 1 320 Gdi font map signature table
Uskb 0 0 2 288 KBDLAYOUT , Binary: win32k!xxxLoadKeyboardLayoutEx
Uskf 0 0 2 288 KBDFILE , Binary: win32k!LoadKeyboardLayoutFile
Uswe 0 0 2 224 WINEVENT , Binary: win32k!_SetWinEventHook
Gddf 0 0 2 224 Gdi ddraw driver heaps
Gddv 0 0 2 192 Gdi ddraw driver video memory list
GFil 0 0 2 192 Gdi engine descriptor list
Gdwd 0 0 2 96 Gdi watchdog support objects , Binary: win32k.sys
Usd9 0 0 1 80 DDE9 , Binary: win32k!xxxCsDdeInitialize
Gvds 0 0 1 64 UNKNOWN pooltag ‘Gvds’, please update pooltag.txt
GreA 0 0 1 64 UNKNOWN pooltag ‘GreA’, please update pooltag.txt
Usse 0 0 1 48 SECURITY , Binary: win32k!SetDisconnectDesktopSecu
Usvl 0 0 1 48 VWPL , Binary: win32k!VWPLAdd
Mdxg 1 112 0 0 UNKNOWN pooltag ‘Mdxg’, please update pooltag.txt
Gini 3 128 0 0 Gdi fast mutex
Usev 1 64 0 0 EVENT , Binary: win32k!xxxPollAndWaitForSingleO
Gdde 3 240 0 0 Gdi ddraw event
TOTAL 9 624 6256 24408704
The first application leaks fonts:
LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
int wmId, wmEvent;
PAINTSTRUCT ps;
HDC hdc;
switch (message)
{
case WM_PAINT:
hdc = BeginPaint(hWnd, &ps);
while (true)
{
HFONT hf = CreateFont(10, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, L"Arial");
SelectObject(ps.hdc, hf);
}
EndPaint(hWnd, &ps);
break;
We clearly see the leak as the greatly increased number of allocations for ”Gla:” pool tag:
Pool Used:
NonPaged Paged
Tag Allocs Used Allocs Used
NV_x 0 0 5 14024704 UNKNOWN pooltag 'NV_x', please update pooltag.txt
Gla: 0 0 10194 6687264 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
BIG 0 0 248 3690496 Large session pool allocations (ntos\ex\pool.c) , Binary: nt!mm
NV 0 0 203 1347648 nVidia video driver
Gh05 0 0 396 1057888 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Ttfd 0 0 226 1043264 TrueType Font driver
The second application leaks fonts and pens:
LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
int wmId, wmEvent;
PAINTSTRUCT ps;
HDC hdc;
switch (message)
{
case WM_PAINT:
hdc = BeginPaint(hWnd, &ps);
while (true)
{
HFONT hf = CreateFont(10, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, L"Arial");
HPEN hp = CreatePen(0, 10, RGB(10, 20, 30));
SelectObject(ps.hdc, hf);
SelectObject(ps.hdc, hp);
}
EndPaint(hWnd, &ps);
break;
We see that the roughly the same number of allocations is split between ”Gla:” and “Gh0@” pool tags:
Pool Used:
NonPaged Paged
Tag Allocs Used Allocs Used
NV_x 0 0 5 14024704 UNKNOWN pooltag 'NV_x', please update pooltag.txt
BIG 0 0 262 3874816 Large session pool allocations (ntos\ex\pool.c) , Binary: nt!mm
Gla: 0 0 5203 3413168 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
NV 0 0 203 1347648 nVidia video driver
Gh0@ 0 0 5077 1137248 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Ttfd 0 0 233 1053152 TrueType Font driver
The third program leaks fonts, pens and brushes:
LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
int wmId, wmEvent;
PAINTSTRUCT ps;
HDC hdc;
switch (message)
{
case WM_PAINT:
hdc = BeginPaint(hWnd, &ps);
while (true)
{
HFONT hf = CreateFont(10, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, L"Arial");
HPEN hp = CreatePen(0, 10, RGB(10, 20, 30));
HBRUSH hb = CreateSolidBrush(RGB(10, 20, 30));
SelectObject(ps.hdc, hf);
SelectObject(ps.hdc, hp);
SelectObject(ps.hdc, hb);
}
EndPaint(hWnd, &ps);
break;
Now we see that the same number of allocations is almost equally split between ”Gla:”, “Gh0@” and “Gla@” pool tags:
Pool Used:
NonPaged Paged
Tag Allocs Used Allocs Used
NV_x 0 0 5 14024704 UNKNOWN pooltag 'NV_x', please update pooltag.txt
BIG 0 0 262 3874816 Large session pool allocations (ntos\ex\pool.c) , Binary: nt!mm
Gla: 0 0 3539 2321584 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
NV 0 0 203 1347648 nVidia video driver
Ttfd 0 0 233 1053152 TrueType Font driver
Gh05 0 0 392 1052768 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gla1 0 0 353 796368 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gh0@ 0 0 3414 764736 Gdi Handle manager specific object types: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gla@ 0 0 3665 762320 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
Gcac 0 0 25 640880 Gdi glyph cache
When the certain amount of handles is reached all subsequent GDI Create calls fail and other applications start showing various visual defects. Print screen operation also fails with insufficient memory message.
- Dmitry Vostokov @ DumpAnalysis.org -
All this will be covered in great detail in the forthcoming book which I tirelessly advertise:
