Archive for June 12th, 2011

Crash Dump Analysis Patterns (Part 138)

Sunday, June 12th, 2011

Sometimes we don’t have symbols (No Component Symbols pattern) or have only a restricted set that we call No Data Types pattern. For example, in a base OS we have data types:

0:016> dt ntdll!*
          ntdll!LIST_ENTRY64
          ntdll!LIST_ENTRY32
          ntdll!_KUSER_SHARED_DATA
          ntdll!_KSYSTEM_TIME
          ntdll!_KSYSTEM_TIME
          ntdll!_NT_PRODUCT_TYPE
[...]

In the “private” version we don’t have them although the symbol file exists:

0:015> dt ntdll!*
0:015> !lmi ntdll
Loaded Module Info: [ntdll]
         Module: ntdll
   Base Address: 0000000076de0000
     Image Name: ntdll.dll
   Machine Type: 34404 (X64)
     Time Stamp: 4dcd9861 Fri May 13 21:45:21 2011
           Size: 17f000
       CheckSum: 188814
Characteristics: 2022  perf
Debug Data Dirs: Type  Size     VA  Pointer
             CODEVIEW    22, f72a8,   f66a8 RSDS - GUID: {05A648A7-625D-42E7-B736-7816F0CA1E0C}
               Age: 2, Pdb: ntdll.pdb
                CLSID     8, f72a0,   f66a0 [Data not mapped]
     Image Type: MEMORY   - Image read successfully from loaded memory.
    Symbol Type: PDB      - Symbols loaded successfully from symbol server.
                 c:\mss\ntdll.pdb\05A648A7625D42E7B7367816F0CA1E0C2\ntdll.pdb
    Load Report: public symbols , not source indexed
                 c:\mss\ntdll.pdb\05A648A7625D42E7B7367816F0CA1E0C2\ntdll.pdb

In such cases manually loading a proximate module might help: Coping with missing symbolic information (although I haven’t yet tested it on x64 systems). I also thought of naming the pattern as Private Modification but that would not cover many other cases where types were missing from the very beginning.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging | 1 Comment »

Crash Dump Analysis Patterns (Part 137)

Sunday, June 12th, 2011

After getting my hands on crash dumps from Windows Azure cloud platform I discerned a few patterns and the first one is just called Cloud Environment. The pattern covers both development (emulator, if it exists) and real (staging and deployment) environments. This pattern is best diagnosed by looking at specific infrastructure modules:

0:016> lm m Wa*
start             end                 module name
00000000`00b00000 00000000`00b0c000   WaWorkerHost
00000000`74fb0000 00000000`74fbd000   WaRuntimeProxy

0:016> lm m *Azure*
start             end                 module name
00000000`57cd0000 00000000`57d26000   Microsoft_WindowsAzure_StorageClient
00000000`58820000 00000000`5886c000   Microsoft_WindowsAzure_Diagnostics
00000000`5c750000 00000000`5c764000   Microsoft_WindowsAzure_ServiceRuntime

Development platform can be distinguished for now by looking at ntdll version: 

0:016> lmv m ntdll
start             end                 module name
00000000`76de0000 00000000`76f5f000   ntdll
    Loaded symbol image file: ntdll.dll
    Image path: D:\Windows\System32\ntdll.dll
    Image name: ntdll.dll
    Timestamp:        Fri May 13 21:45:21 2011 (4DCD9861)
    CheckSum:         00188814
    ImageSize:        0017F000
    File version:     6.0.6002.18446
    Product version:  6.0.6002.18446
    File flags:       0 (Mask 3F)
    File OS:          40004 NT Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04b0
    CompanyName:      Microsoft Corporation
    ProductName:      Microsoft® Windows® Operating System
    InternalName:     ntdll.dll
    OriginalFilename: ntdll.dll
    ProductVersion:   6.0.6002.18446
    FileVersion:      6.0.6002.18446 (rd_os_v1.110513-1321)
    FileDescription:  NT Layer DLL
    LegalCopyright:   © Microsoft Corporation. All rights reserved.

0:016> lmv m ntdll
start             end                 module name
00000000`775a0000 00000000`7774b000   ntdll
    Loaded symbol image file: ntdll.dll
    Image path: C:\Windows\System32\ntdll.dll
    Image name: ntdll.dll
    Timestamp:        Tue Jul 14 02:32:27 2009 (4A5BE02B)
    CheckSum:         001B1CB5
    ImageSize:        001AB000
    File version:     6.1.7600.16385
    Product version:  6.1.7600.16385
    File flags:       0 (Mask 3F)
    File OS:          40004 NT Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04b0
    CompanyName:      Microsoft Corporation
    ProductName:      Microsoft® Windows® Operating System
    InternalName:     ntdll.dll
    OriginalFilename: ntdll.dll
    ProductVersion:   6.1.7600.16385
    FileVersion:      6.1.7600.16385 (win7_rtm.090713-1255)
    FileDescription:  NT Layer DLL
    LegalCopyright:   © Microsoft Corporation. All rights reserved.

We see that real Windows Azure is currently RD OS V1 (at least in a datacenter chosen for West European affinity during deployment).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in Cloud Memory Dump Analysis, Crash Dump Analysis, Crash Dump Patterns, Debugging, Windows Azure | No Comments »