Citrix Tools - everything you need for troubleshooting, optimization and analysis

It is based on my previous presentation with a few added slides which can be downloaded from Citrix support web site:

Selected Citrix Troubleshooting Tools

- Dmitry Vostokov @ DumpAnalysis.org -

• - CDFControl

• - ASC/AMC: Using Citrix Diagnostic Facility and the Access Suite Console for Tracing

• - tracelog: How to Collect System Startup Traces Using the Microsoft Utility Tracelog.exe

• - Windows Performance Tools Kit

- Dmitry Vostokov @ DumpAnalysis.org -

Here is another CAD from Windows 2000 server where the oldest driver is easily identified:

The following CAD diagram is created from lmt output in Module Variety pattern example:

- Dmitry Vostokov @ DumpAnalysis.org -

http://www.dumpanalysis.org/CDAPoster.html

It is also featured on http://windbg.org

I’m also working on the better version that will be released simultaneuosly with WDN book.

- Dmitry Vostokov @ DumpAnalysis.org -

http://windbg.org

- Dmitry Vostokov @ DumpAnalysis.org -

0:000> !stl
!stl [options] <varname>
  stl [options] <varname> - dumps an STL variable
  stl [options] -n <type-name> <address>
             currently works with string, wstring
             vector<string>, vector<wstring>
             list<string>, vector<wstring>
             (and pointer varieties therein)
   [options]
       -n <type-name> The name of the type. If the
               type has spaces, surround with
               parentheses ().
       -v      verbose output
       -V      extremely verbose output

If we have public symbols and know variable names we can simply dump their values, for example:

0:000> dv /i /V
prv local  @ecx @ecx            this = 0x0012fbdc
prv local  0012fbf8 @ebp-0x2c   MyName = class std::basic_string<char,std::char_traits<char>,std::allocator<char> >

0:000> !stl MyName
[da 0x12fbfc]
0012fbfc  "COMPANY__NAME"

We can also supply full STL type name:

0:000> !stl -n (std::basic_string<char,std::char_traits<char>,std::allocator<char> >) 0012fbf8
[da 0x12fbfc]
0012fbfc  "COMPANY__NAME"

Let’s dump this string type internal structure to be able to recognize it later in raw data:

0:000> dt -r -n std::basic_string<char,std::char_traits<char>,std::allocator<char> > 0012fbf8
application!std::basic_string<char,std::char_traits<char>,std::allocator<char> >
   +0x000 _Alval           : std::allocator<char>
   =00400000 npos             : 0x905a4d
   +0×004 _Bx              : std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Bxty
      +0×000 _Buf             : [16]  “COMPANY__NAME”
      +0×000 _Ptr             : 0×43415250  “”
   +0×014 _Mysize          : 0xd
   +0×018 _Myres           : 0xf

We see that for short strings less than 16 bytes std::basic_string<char> data starts from offset +4 and followed by the actual string size and its reserved size:

0:000> dd 0012fbf8
0012fbf8  00000000 43415250 45434954 53504d5f
0012fc08  41bf0033 0000000d 0000000f 41bf3b72
0012fc18  0012fc6c 0046107b 00000000 0012fc78
0012fc28  0041a441 00000000 41bf3b2e 00ed6380
0012fc38  00000003 00ed6128 00ed6128 00f41b00
0012fc48  00ed6128 41bf3b3e 0012fc3c 00000000
0012fc58  0000000f 00f41b98 00f469a0 00000000
0012fc68  014487c8 0012fcfc 00463fdd 00000002

For bigger strings implementation starts with a pointer from offset +4 to the actual string data and then followed by 12 bytes of garbage and then by the actual string size and its reserved size:

0:000> dt -r -n std::basic_string<char,std::char_traits<char>,std::allocator<char> >
application!std::basic_string<char,std::char_traits<char>,std::allocator<char> >
   +0x000 _Alval           : std::allocator<char>
   =00400000 npos             : Uint4B
   +0×004 _Bx              : std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Bxty
      +0×000 _Buf             : [16] Char
      +0×000 _Ptr             : Ptr32 Char
   +0×014 _Mysize          : Uint4B
   +0×018 _Myres           : Uint4B

0:000> dt -r -n std::basic_string<char,std::char_traits<char>,std::allocator<char> > 0012ff08
application!std::basic_string<char,std::char_traits<char>,std::allocator<char> >
   +0x000 _Alval           : std::allocator<char>
   =00400000 npos             : 0x905a4d
   +0×004 _Bx              : std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Bxty
      +0×000 _Buf             : [16]  “???”
      +0×000 _Ptr             : 0×00ed4ba0  “/h /c:100 /enum”
   +0×014 _Mysize          : 0×10
   +0×018 _Myres           : 0×1f

In such cases dpa or dpu commands help to show this additional dereference:

0:000> dpa 0012ff08
0012ff08  00ed2f90 "."
0012ff0c  00ed4ba0 “/h /c:100 /enum”
0012ff10  41eafd01
0012ff14  0012ffc0 “…”
0012ff18  0045890a “……U..SVWUj”
0012ff1c  00000010
0012ff20  0000001f
0012ff24  41bf3996
0012ff28  0012ffc0 “…”
0012ff2c  0044b528 “.E..}.”
0012ff30  00400000 “MZ.”

SDbgExt has commands to interrogate additional STL types.  

- Dmitry Vostokov @ DumpAnalysis.org -

PubForum, 2008

My presentation date, time and topic are:

Friday, May 9, 19:00 “Citrix Tools - everything you need for troubleshooting, optimization and analysis”

See you there

- Dmitry Vostokov @ DumpAnalysis.org -

0:000> lm
start    end        module name
00400000 00416000   Application
00470000 0050b000   advapi32
00520000 00572000   shlwapi
02340000 023cb000   oleaut32
04b80000 0523e000   System_Data_ni
1a400000 1a524000   urlmon
4dd60000 4df07000   GdiPlus
5f120000 5f12e000   ntlanman
5f860000 5f891000   netui1
5f8a0000 5f8b6000   netui0
637a0000 63d28000   System_Xml_ni
64890000 6498c000   System_Configuration_ni
64e70000 6515c000   System_Data
65ce0000 65ecc000   System_Web_Services_ni
71bd0000 71be1000   mpr           
71bf0000 71bf8000   ws2help
71c00000 71c17000   ws2_32
71c20000 71c32000   tsappcmp
71c40000 71c97000   netapi32
73070000 73097000   winspool
75e90000 75e97000   drprov
75ea0000 75eaa000   davclnt
76190000 761a2000   msasn1
761b0000 76243000   crypt32
76a80000 76a92000   atl
76b80000 76bae000   credui
76dc0000 76de8000   adsldpc
76df0000 76e24000   activeds
76f00000 76f08000   wtsapi32
76f10000 76f3e000   wldap32
771f0000 77201000   winsta
77670000 777a9000   ole32
77ba0000 77bfa000   msvcrt
78130000 781cb000   msvcr80
79000000 79046000   mscoree
79060000 790b6000   mscorjit
790c0000 79bf6000   mscorlib_ni
79e70000 7a3ff000   mscorwks
7a440000 7ac2a000   System_ni
7ade0000 7af7c000   System_Drawing_ni
7afd0000 7bc6c000   System_Windows_Forms_ni
7c340000 7c396000   msvcr71
7c8d0000 7d0ce000   shell32
7d4c0000 7d5f0000   kernel32
7d600000 7d6f0000   ntdll
7d800000 7d890000   gdi32
7d8d0000 7d920000   secur32
7d930000 7da00000   user32
7da20000 7db00000   rpcrt4
7dbd0000 7dcd3000   comctl32
7df50000 7dfc0000   uxtheme
7e020000 7e02f000   samlib

The similar address space reshuffling happens with ASLR-enabled applications with the difference that system modules are never remapped below 0×70000000.

 - Dmitry Vostokov @ DumpAnalysis.org -

• Title: Crash Dump Analysis for System Administrators
• Author: Dmitry Vostokov
• Publisher: Opentask (25 August 2008)
• Language: English
• Product Dimensions: 22.86 x 15.24
• ISBN-13: 978-1-906717-02-5
• Paperback: 180 pages

 - Dmitry Vostokov @ DumpAnalysis.org -

- Dmitry Vostokov @ DumpAnalysis.org -