NULL code pointer, changed environment, hooked functions and execution residue: pattern cooperation
Wednesday, February 4th, 2009After an upgrade to the new version of a productivity software package one unrelated application started to crash frequently. A crash dump was collected and the following stack trace pointed to a NULL code pointer:
0:000> r
eax=09680104 ebx=0013aefc ecx=0968a710 edx=0cdc0c0c esi=16a19058 edi=00000001
eip=00000000 esp=0013aea8 ebp=0013aeb8 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00200202
00000000 ?? ???
0:000> k 100
ChildEBP RetAddr
WARNING: Frame IP not in any known module. Following frames may be wrong.
0013aea4 096e28a0 0×0
0013aeb8 32e688bd dllC!Abort+0×10
0013aec8 32c82395 dllB+0×589e
0013aed8 32865718 dllB+0×18f1
[…]
0013b0c0 314de1ff dllB+0×4c6
0013b154 31293494 dllA!DllGetLCID+0×46d2d
0013b178 312af217 dllA!DllGetClassObject+0×4e896
[…]
0013f3d0 300e8721 dllA!DllGetClassObject+0×69e42
0013f578 300e7f5a application+0xcff5
[…]
0013ffc0 7c816ff7 application+0×51d5
0013fff0 00000000 kernel32!BaseProcessStart+0×23
To see if changed environment somehow affected this application the presence of any DLL hooks was checked. The following hooked functions were found in user32.dll:
0:000> !chkimg -lo 50 -d !user32 -v
Searching for module with expression: !user32
Will apply relocation fixups to file used for comparison
Will ignore NOP/LOCK errors
Will ignore patched instructions
Image specific ignores will be applied
Comparison image path: c:\mss\USER32.dll\45F02D7C90000\USER32.dll
No range specified
Scanning section: .text
Size: 389095
Range to scan: 7e411000-7e46ffe7
7e4188a6-7e4188aa 5 bytes - USER32!GetWindowLongW
[ 6a 08 68 e0 88:e9 55 77 a4 01 ]
7e418f9c-7e418fa0 5 bytes - USER32!GetSystemMetrics (+0×6f6)
[ 6a 10 68 00 90:e9 5f 70 b5 01 ]
7e41945d-7e419461 5 bytes - USER32!GetWindowLongA (+0×4c1)
[ 6a 10 68 78 9f:e9 f5 60 a2 01 ]
7e41b6ae-7e41b6b2 5 bytes - USER32!GetClientRect (+0×17a8)
[ 8b ff 55 8b ec:e9 4d 49 9f 01 ]
7e41b6d4-7e41b6d8 5 bytes - USER32!GetWindowRect (+0×26)
[ b8 74 11 00 00:e9 98 30 af 01 ]
7e41d60d-7e41d611 5 bytes - USER32!SetWindowLongA (+0×6aa)
[ 8b ff 55 8b ec:e9 ee 29 a5 01 ]
7e41d62b-7e41d62f 5 bytes - USER32!SetWindowLongW (+0×1e)
[ 6a 08 68 28 f5:e9 0e 0b b2 01 ]
7e41fc25-7e41fc29 5 bytes - USER32!CreateWindowExW (+0×738)
[ 8b ff 55 8b ec:e9 d6 03 b6 01 ]
7e41ff33-7e41ff37 5 bytes - USER32!CreateWindowExA (+0×30e)
Total bytes compared: 389095(100%)
Number of errors: 52
52 errors : !user32 (7e4188a6-7e42e8d5)
The hooking DLL was found to be from that upgraded package:
0:000> u 7e4188a6
USER32!GetWindowLongW:
7e4188a6 e95577a401 jmp 7fe60000
7e4188ab 41 inc ecx
7e4188ac 7ee8 jle USER32!_GetWindowLong+0xda (7e418896)
7e4188ae 0e push cs
7e4188af fd std
7e4188b0 ff ???
7e4188b1 ff8b4d08e816 dec dword ptr [ebx+16E8084Dh]
7e4188b7 fc cld
0:000> u 7fe60000
7fe60000 e9bb62b080 jmp hookA+0×62c0 (009662c0)
7fe60005 6a08 push 8
7fe60007 68e088417e push offset USER32!`string’+0×34 (7e4188e0)
7fe6000c e99c885bfe jmp USER32!GetWindowLongW+0×7 (7e4188ad)
7fe60011 0000 add byte ptr [eax],al
7fe60013 0000 add byte ptr [eax],al
7fe60015 0000 add byte ptr [eax],al
7fe60017 0000 add byte ptr [eax],al
0:000> lmv m hookA
start end module name
00960000 00976000 hookA (no symbols)
Loaded symbol image file: hookA.dll
Image path: C:\Program Files\CompanyA\hookA.dll
Image name: hookA.dll
[…]
ProductName: ProductA
[…]
Execution residue from hookA module was also found on the problem thread raw stack and it looks like real code (not a coincidental symbolic information):
0:000> !teb
TEB at 7ffdf000
ExceptionList: 0013f02c
StackBase: 00140000
StackLimit: 0010c000
SubSystemTib: 00000000
FiberData: 00001e00
ArbitraryUserPointer: 00000000
Self: 7ffdf000
EnvironmentPointer: 00000000
ClientId: 00000c38 . 00000840
RpcHandle: 00000000
Tls Storage: 00163268
PEB Address: 7ffdb000
LastErrorValue: 0
LastStatusValue: 0
Count Owned Locks: 0
HardErrorMode: 0
0:000> dds 0010c000 00140000
0010c000 00000000
0010c004 00000000
0010c008 00000000
[…]
00121f50 0096d7d1*** ERROR: Module load completed but symbols could not be loaded for hookA.dll
hookA+0xd7d1
00121f54 00009924
00121f58 00121fb4
00121f5c 00000000
00121f60 00121f9c
00121f64 0096d895 hookA+0xd895
00121f68 00121f78
00121f6c 00000000
00121f70 00122008
00121f74 00000000
00121f78 00000000
00121f7c 00000000
00121f80 00000000
00121f84 00121f6c
00121f88 000001fe
00121f8c 001220fc
00121f90 0096ec98 hookA+0xec98
00121f94 00970e48 hookA+0×10e48
00121f98 00121fb4
00121f9c 7e41b6a3 USER32!_GetClientRect+0×6e
00121fa0 00122008
00121fa4 fffffa01
[…]
0:000> u 0096d895
hookA+0xd895:
0096d895 8bc6 mov eax,esi
0096d897 8b4df0 mov ecx,dword ptr [ebp-10h]
0096d89a 64890d00000000 mov dword ptr fs:[0],ecx
0096d8a1 5f pop edi
0096d8a2 5e pop esi
0096d8a3 5b pop ebx
0096d8a4 8be5 mov esp,ebp
0096d8a6 5d pop ebp
0:000> ub 0096d895
hookA+0xd876:
0096d876 8b65e8 mov esp,dword ptr [ebp-18h]
0096d879 be0d000000 mov esi,0Dh
0096d87e c745fcffffffff mov dword ptr [ebp-4],0FFFFFFFFh
0096d885 eb05 jmp hookA+0xd88c (0096d88c)
0096d887 be72000000 mov esi,72h
0096d88c 8d55dc lea edx,[ebp-24h]
0096d88f 52 push edx
0096d890 e8fbfeffff call hookA+0xd790 (0096d790)
As was found the upgraded application had special DLL hooks to improve productivity and ease of use of GUI applications. Fortunately it was possible to disable that hook on a per-application basis and application crashes disappeared.
- Dmitry Vostokov @ DumpAnalysis.org -