Crash Dump Analysis

Crash Dump Analysis Patterns (Part 174)

Sunday, May 20th, 2012

Activity Resonance pattern is observed when two products from different vendors compete in some functional domain such malware detection. In the example below ApplicationA and AVDriverA modules belong to Vendor A and AV-B module belongs to Vendor B. Both threads are spiking threads blocking all other activity in the system:

0: kd> !running

System Processors: (0000000000000003) Idle Processors: (0000000000000000) (0000000000000000) (0000000000000000) (0000000000000000)

Prcbs Current Next 0 fffff80001845e80 fffffa8004350060 ................ 1 fffff880009c4180 fffffa80028e7060 ................

0: kd> !thread fffffa8004350060 ff THREAD fffffa8004350060 Cid 14424.14b34 Teb: 000000007efdb000 Win32Thread: fffff900c1d32c30 RUNNING on processor 0 Not impersonating DeviceMap fffff8a00148fe80 Owning Process fffffa8003d6cb30 Image: ApplicationA.exe Attached Process N/A Image: N/A Wait Start TickCount 10568630 Ticks: 0 Context Switch Count 345 LargeStack UserTime 00:02:21.360 KernelTime 01:09:32.130 Win32 Start Address ApplicationA!mainCRTStartup (0×0000000000404c1b) Stack Init fffff88006c71db0 Current fffff88006c71670 Base fffff88006c72000 Limit fffff88006c6a000 Call 0 Priority 9 BasePriority 8 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`06c70ec0 fffff880`0197d53c AVDriverA+0×15d69 fffff880`06c70f10 fffff880`01988556 AVDriverA+0×1453c fffff880`06c70fd0 fffff880`019886a8 AVDriverA+0×1f556 fffff880`06c71000 fffff800`0198ebfd AVDriverA+0×1f6a8 fffff880`06c71060 fffff800`019bf4f2 nt! ?? ::NNGAKEGL::`string’+0×2a6fd fffff880`06c711e0 fffff800`019c3385 nt!PspCreateThread+0×246 fffff880`06c71460 fffff800`016d28d3 nt!NtCreateThreadEx+0×25d fffff880`06c71bb0 00000000`76e61d9a nt!KiSystemServiceCopyEnd+0×13 (TrapFrame @ fffff880`06c71c20) 00000000`0008e178 00000000`74990411 ntdll!ZwCreateThreadEx+0xa 00000000`0008e180 00000000`7497cf87 wow64!whNtCreateThreadEx+0×815 00000000`0008e350 00000000`748c2776 wow64!Wow64SystemServiceEx+0xd7 00000000`0008ec10 00000000`7497d07e wow64cpu!TurboDispatchJumpAddressEnd+0×2d 00000000`0008ecd0 00000000`7497c549 wow64!RunCpuSimulation+0xa 00000000`0008ed20 00000000`76e54956 wow64!Wow64LdrpInitialize+0×429 00000000`0008f270 00000000`76e51a17 ntdll!LdrpInitializeProcess+0×17e4 00000000`0008f760 00000000`76e3c32e ntdll! ?? ::FNODOBFM::`string’+0×29220 00000000`0008f7d0 00000000`00000000 ntdll!LdrInitializeThunk+0xe

0: kd> !thread fffffa80028e7060 ff THREAD fffffa80028e7060 Cid 0dc4.0e5c Teb: 000000007efa4000 Win32Thread: 0000000000000000 RUNNING on processor 1 Not impersonating DeviceMap fffff8a000008b30 Owning Process fffffa8002817060 Image: AV-B.exe Attached Process N/A Image: N/A Wait Start TickCount 10568617 Ticks: 13 (0:00:00:00.203) Context Switch Count 1763138 UserTime 00:04:26.765 KernelTime 03:09:31.140 Win32 Start Address AV-B (0×00000000004289f2) Stack Init fffff88003b88db0 Current fffff88003b88900 Base fffff88003b89000 Limit fffff88003b83000 Call 0 Priority 15 BasePriority 15 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`03b88660 fffff800`019919a9 nt!ObReferenceObjectSafe+0xf fffff880`03b88690 fffff800`01991201 nt!PsGetNextProcess+0×81 fffff880`03b886e0 fffff800`019dcef6 nt!ExpGetProcessInformation+0×774 fffff880`03b88830 fffff800`019dd949 nt!ExpQuerySystemInformation+0xfb4 fffff880`03b88be0 fffff800`016d28d3 nt!NtQuerySystemInformation+0×4d fffff880`03b88c20 00000000`76e6167a nt!KiSystemServiceCopyEnd+0×13 (TrapFrame @ fffff880`03b88c20) 00000000`0118e708 00000000`74987da7 ntdll!NtQuerySystemInformation+0xa 00000000`0118e710 00000000`74988636 wow64!whNT32QuerySystemProcessInformationEx+0×93 00000000`0118e760 00000000`7498a0e9 wow64!whNtQuerySystemInformation_SpecialQueryCase+0×466 00000000`0118e800 00000000`7497cf87 wow64!whNtQuerySystemInformation+0xf1 00000000`0118e840 00000000`748c2776 wow64!Wow64SystemServiceEx+0xd7 00000000`0118f100 00000000`7497d07e wow64cpu!TurboDispatchJumpAddressEnd+0×2d 00000000`0118f1c0 00000000`7497c549 wow64!RunCpuSimulation+0xa 00000000`0118f210 00000000`76e8e707 wow64!Wow64LdrpInitialize+0×429 00000000`0118f760 00000000`76e3c32e ntdll! ?? ::FNODOBFM::`string’+0×29364 00000000`0118f7d0 00000000`00000000 ntdll!LdrInitializeThunk+0xe

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns, Victimware, x64 Windows | No Comments »

Crash Dump Analysis Patterns (Part 173)

Sunday, May 20th, 2012

Value Adding Process is a frequently observed pattern in terminal services environments when you see one or several process names listed in each session but not necessarily required. They are usually running to provide some user experience enhancements. In such cases if observed functional problems correspond to the purpose of running additional processes we might want to eliminate them for testing and troubleshooting purposes.

0: kd> !sprocess 12 Dumping Session 12

_MM_SESSION_SPACE fffff8800e5d5000 _MMSESSION fffff8800e5d5b40 PROCESS fffffa8008d50b30 SessionId: 12 Cid: 0b04 Peb: 7fffffdc000 ParentCid: 1478 DirBase: 6bb77000 ObjectTable: fffff8a003f280b0 HandleCount: 158. Image: csrss.exe

PROCESS fffffa80030c7060 SessionId: 12 Cid: 1a48 Peb: 7fffffd8000 ParentCid: 1478 DirBase: 0a33c000 ObjectTable: fffff8a003c46c00 HandleCount: 179. Image: winlogon.exe

PROCESS fffffa8008250b30 SessionId: 12 Cid: 18c8 Peb: 7fffffdf000 ParentCid: 1a48 DirBase: 0350d000 ObjectTable: fffff8a0025b6840 HandleCount: 226. Image: LogonUI.exe

PROCESS fffffa8008b00530 SessionId: 12 Cid: 1508 Peb: 7fffffdf000 ParentCid: 02f0 DirBase: 02f65000 ObjectTable: fffff8a003b7e530 HandleCount: 197. Image: ExcitingFeatureX.exe

[...]

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns | No Comments »

Crash Dump Analysis Patterns (Part 20d)

Saturday, May 19th, 2012

This is a specialization of Insufficient Memory (kernel pool) pattern called Memory Leak (I/O completion packets). The currently unique diagnostics this pattern provides in comparison with other kernel pool tags is that the pool allocation entries show the leaking process:

0: kd> !poolused 3 Sorting by NonPaged Pool Consumed

Pool Used: NonPaged Paged Tag Allocs Frees Diff Used Allocs Frees Diff Used Icp 1294074 42875 1251199 96642976 0 0 0 0 I/O completion packets queue on a completion ports […]

0: kd> !poolfind Icp

Scanning large pool allocation table for Tag: Icp (fffffa8013e00000 : fffffa8014100000)

*fffffa800e188260 size: 50 previous size: 40 (Allocated) Icp Process: fffffa800899dc40 *fffffa800e1882e0 size: 50 previous size: 30 (Allocated) Icp Process: fffffa800899dc40 *fffffa800e188330 size: 50 previous size: 50 (Allocated) Icp Process: fffffa800899dc40 *fffffa800e188380 size: 50 previous size: 50 (Allocated) Icp Process: fffffa800899dc40 *fffffa800e1883d0 size: 50 previous size: 50 (Allocated) Icp Process: fffffa800899dc40 *fffffa800e188420 size: 50 previous size: 50 (Allocated) Icp Process: fffffa800899dc40 *fffffa800e188470 size: 50 previous size: 50 (Allocated) Icp Process: fffffa800899dc40 *fffffa800e1884c0 size: 50 previous size: 50 (Allocated) Icp Process: fffffa800899dc40

0: kd> !process fffffa800899dc40 1 PROCESS fffffa800899dc40 SessionId: 0 Cid: 43a4 Peb: 7efdf000 ParentCid: 0412 DirBase: 09d6b000 ObjectTable: fffff8a0046c8c10 HandleCount: 1068. Image: ServiceA.exe […]

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging | No Comments »

Crash Dump Analysis Patterns (Part 172)

Saturday, May 19th, 2012

Recently I observed a few occurrences of a rare No Current Thread pattern in a large set of process memory dumps:

0:???> k WARNING: The debugger does not have a current process or thread WARNING: Many commands will not work ^ Illegal thread error in ‘k’

0:???> ~ WARNING: The debugger does not have a current process or thread WARNING: Many commands will not work 0 Id: 95f4.6780 Suspend: 1 Teb: 7efdd000 Unfrozen

Setting a current thread helps:

0:???> ~0s WARNING: The debugger does not have a current process or thread WARNING: Many commands will not work eax=037d0010 ebx=0002bda0 ecx=03b1a010 edx=00000007 esi=037d0010 edi=03b069fc eip=0397939f esp=0018fd98 ebp=0018fdd8 iopl=0 nv up ei pl nz na po nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00200202 DllA+0×939f: 0397939f 8b10 mov edx,dword ptr [eax] ds:002b:037d0010=03b1a010

0:000> k ChildEBP RetAddr WARNING: Stack unwind information not available. Following frames may be wrong. 0018fdd8 03975257 DllA+0x939f 0018fdf8 03975577 DllA+0x5257 0018fe58 772bb9a0 DllA+0x5577 0018fe78 772d9b96 ntdll!LdrpCallInitRoutine+0x14 0018ff1c 772d9a38 ntdll!LdrShutdownProcess+0x1aa 0018ff30 752279f4 ntdll!RtlExitUserProcess+0x74 0018ff44 0040625d kernel32!ExitProcessStub+0x12 0018ff5c 012528e5 Application+0x625d 0018ff88 7522339a Application!foo+0xdc88f1 0018ff94 772bbf42 kernel32!BaseThreadInitThunk+0xe 0018ffd4 772bbf15 ntdll!__RtlUserThreadStart+0x70 0018ffec 00000000 ntdll!_RtlUserThreadStart+0x1b

However, EIP of the new current thread doesn’t point to any access violation and the dereferenced address is valid:

0:000> !address 037d0010 Usage: <unclassified> Allocation Base: 037d0000 Base Address: 037d0000 End Address: 038dd000 Region Size: 0010d000 Type: 00020000 MEM_PRIVATE State: 00001000 MEM_COMMIT Protect: 00000004 PAGE_READWRITE

Also, if we inspect the raw stack data we won’t find any hidden exceptions there. So we conclude that the missing thread was exceptional. Indeed, there is a saved exception context in the process memory dump:

0:000> .exr -1 ExceptionAddress: 08a9ae18 (<Unloaded_DllB.dll>+0x001cae18) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000001 NumberParameters: 1 Parameter[0]: 00000008

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging | No Comments »

Crash Dump Analysis Patterns (Part 14, Mac OS X)

Wednesday, May 9th, 2012

This is a Mac OS X / GDB counterpart to Spiking Thread pattern previously described for Windows platforms:

(gdb) info threads 4 0×00007fff85b542df in sqrt$fenv_access_off () 3 0×00007fff8616ee42 in __semwait_signal () 2 0×00007fff8616ee42 in __semwait_signal () * 1 0×00007fff8616ee42 in __semwait_signal ()

We notice a non-waiting thread and switch to it:

(gdb) thread 4 [Switching to thread 4 (core thread 3)] 0x00007fff85b542df in sqrt$fenv_access_off ()

(gdb) bt #0 0x00007fff85b542df in sqrt$fenv_access_off () #1 0×000000010cc85dc9 in thread_three (arg=0×7fff6c884ac0) #2 0×00007fff8fac68bf in _pthread_start () #3 0×00007fff8fac9b75 in thread_start ()

If we disassemble the return address for thread_three function to come back from sqrt call we see an infinite loop:

(gdb) disass 0x000000010cc85dc9 Dump of assembler code for function thread_three: 0x000000010cc85db0 <thread_three+0>: push %rbp 0×000000010cc85db1 <thread_three+1>: mov %rsp,%rbp 0×000000010cc85db4 <thread_three+4>: sub $0×10,%rsp 0×000000010cc85db8 <thread_three+8>: mov %rdi,-0×10(%rbp) 0×000000010cc85dbc <thread_three+12>: mov -0×10(%rbp),%ax 0×000000010cc85dc0 <thread_three+16>: movsd (%rax),%xmm0 0×000000010cc85dc4 <thread_three+20>: callq 0×10cc85eac <dyld_stub_sqrt> 0×000000010cc85dc9 <thread_three+25>: mov -0×10(%rbp),%rax 0×000000010cc85dcd <thread_three+29>: movsd %xmm0,(%rax) 0×000000010cc85dd1 <thread_three+33>: jmpq 0×10cc85dbc <thread_three+12> End of assembler dump.

Here’s the source code of the modeling application:

void * thread_one (void *arg)

{

while (1)

{

sleep (1);

}

return 0;

}

void * thread_two (void *arg)

{

while (1)

{

sleep (2);

}

return 0;

}

void * thread_three (void *arg)

{

while (1)

{

*(double*)arg=sqrt(*(double *)arg);

}

return 0;

}

int main(int argc, const char * argv[])

{

pthread_t threadID_one, threadID_two, threadID_three;

double result = 0xffffffff;

pthread_create (&threadID_one, NULL, thread_one, NULL);

pthread_create (&threadID_two, NULL, thread_two, NULL);

pthread_create (&threadID_three, NULL, thread_three,

&result);

pthread_join(threadID_three, NULL);

return 0;

}

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Forthcoming Training: Accelerated Mac OS X Core Dump Analysis

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Assembly Language, Core Dump Analysis, Crash Dump Analysis, Crash Dump Patterns, Debugging, GDB for WinDbg Users, Mac Crash Corner, Mac OS X, Software Defect Construction, x64 Mac OS X | No Comments »

Crash Dump Analysis Patterns (Part 6a, Mac OS X)

Thursday, May 3rd, 2012

This is a Mac OS X / GDB counterpart to NULL Pointer (code) pattern previously described for Windows platforms:

(gdb) bt #0 0×0000000000000000 in ?? () #1 0×000000010e8cce73 in bar (ps=0×7fff6e4cbac0) #2 0×000000010e8cce95 in foo (ps=0×7fff6e4cbac0) #3 0×000000010e8cced5 in main (argc=1, argv=0×7fff6e4cbb08)

(gdb) disass 0×000000010e8cce73-3 0×000000010e8cce73 Dump of assembler code from 0×10e8cce70 to 0×10e8cce73: 0×000000010e8cce70 : callq *0×8(%rdi) End of assembler dump.

(gdb) info r rdi rdi 0x7fff6e4cbac0 140735043910336

(gdb) x/2 0x7fff6e4cbac0 0x7fff6e4cbac0: 0x0000000a 0×00000000

(gdb) p/x *($rdi+8) $7 = 0×0

(gdb) bt #0 0x0000000000000000 in ?? () #1 0x000000010e8cce73 in bar (ps=0×7fff6e4cbac0) #2 0×000000010e8cce95 in foo (ps=0×7fff6e4cbac0) #3 0×000000010e8cced5 in main (argc=1, argv=0×7fff6e4cbb08)

(gdb) ptype MYSTRUCT type = struct _MyStruct_tag { int data; PFUNC pfunc; }

(gdb) print {MYSTRUCT}0×7fff6e4cbac0 $2 = {data = 10, pfunc = 0}

Here’s the source code of the modeling application:

typedef void (*PFUNC)(void);

typedef struct _MyStruct_tag

{

int data;

PFUNC pfunc;

} MYSTRUCT;

void bar(MYSTRUCT *ps)

{

ps->pfunc();

}

void foo(MYSTRUCT *ps)

{

bar(ps);

}

int main(int argc, const char * argv[])

{

MYSTRUCT pstruct = {10, NULL};

foo(&pstruct);

return 0;

}

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Forthcoming Training: Accelerated Mac OS X Core Dump Analysis

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Assembly Language, Core Dump Analysis, Crash Dump Analysis, Crash Dump Patterns, Debugging, GDB for WinDbg Users, Mac Crash Corner, Mac OS X, Software Defect Construction, x64 Mac OS X | No Comments »

Crash Dump Analysis Patterns (Part 53c)

Saturday, April 28th, 2012

This is a special variant of Blocked Thread pattern where we have a timeout value so a thread is potentially blocked only temporarily. For example, this main thread is blocked waiting for beep sound to finish after a minute:

0:000> kvL ChildEBP RetAddr Args to Child 0291f354 7c90d21a 7c8023f1 00000001 0291f388 ntdll!KiFastSystemCallRet 0291f358 7c8023f1 00000001 0291f388 7c90d27e ntdll!NtDelayExecution+0xc 0291f3b0 7c837beb 0000ea60 00000001 00000004 kernel32!SleepEx+0×61 0291f404 004952a2 00000370 0000ea60 004d6ae2 kernel32!Beep+0×1b3 0291f410 004d6ae2 00000370 0000ea60 004d6ed4 Application!DoBeep+0×16 […] 0291ffec 00000000 0045aad0 00e470a0 00000000 kernel32!BaseThreadStart+0×37

0:000> ? ea60/0n1000 Evaluate expression: 60 = 0000003c

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns | No Comments »

Software Behavior Pattern Prediction

Saturday, April 28th, 2012

Sometimes I hear voices saying that Linux, FreeBSD, and Mac OS X core dumps are uninteresting. This is not true. If you haven’t seen anything interesting there it just simply means you have only encountered a limited amount of abnormal software behaviour. The widespread usage of Windows OS means that most patterns have been diagnosed and described first and other OS are waiting their turn.

My goal is to have a pattern catalog with examples from different OS. For example, currently, all Mac OS X patterns I provide are just examples to existing Windows pattern names. All OS share the same structure and behavior, for example, structural memory analysis patterns and the same computational model. Although structural patterns are different from behavioral patterns I also plan to expand the structural list significantly especially in relation to forthcoming Windows malware analysis training. Regarding behavioral patterns it is possible to model and predict specific pattern examples for another OS by using already existing catalog.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Core Dump Analysis, Crash Dump Analysis, Crash Dump Patterns, Linux Crash Corner, Mac Crash Corner, Mac OS X, Malware Analysis, Malware Patterns, Pattern Models, Pattern Prediction, Pattern-Driven Debugging, Pattern-Driven Software Support, Software Behavior DNA, Software Behavior Patterns, Software Behavioral Genome, Software Diagnostics | No Comments »

Crash Dump Analysis Patterns (Part 171)

Saturday, April 28th, 2012

This is an example of Punctuated Memory Leak pattern somewhat similar to a large block allocation leak for process heap (see a modeling example). An application has some functionality and after each command its commited memory was increasing by 50 - 60 Mb. 3 process dumps were taken with one before failures and then after each failure:

// Before failures

0:000> !address -summary

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal Free 267 76c50000 ( 1.856 Gb) 92.79% <unclassified> 270 4d6f000 ( 77.434 Mb) 52.45% 3.78% Image 620 31bf000 ( 49.746 Mb) 33.70% 2.43% Stack 60 1400000 ( 20.000 Mb) 13.55% 0.98% ActivationContextData 48 35000 ( 212.000 kb) 0.14% 0.01% NlsTables 1 23000 ( 140.000 kb) 0.09% 0.01% TEB 20 14000 ( 80.000 kb) 0.05% 0.00% CsrSharedMemory 1 5000 ( 20.000 kb) 0.01% 0.00% PEB 1 1000 ( 4.000 kb) 0.00% 0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_PRIVATE 296 3bca000 ( 59.789 Mb) 40.50% 2.92% MEM_IMAGE 647 340c000 ( 52.047 Mb) 35.26% 2.54% MEM_MAPPED 78 23ca000 ( 35.789 Mb) 24.24% 1.75%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_FREE 267 76c50000 ( 1.856 Gb) 92.79% MEM_RESERVE 125 5006000 ( 80.023 Mb) 54.21% 3.91% MEM_COMMIT 896 439a000 ( 67.602 Mb) 45.79% 3.30%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal PAGE_EXECUTE_READ 125 1f2c000 ( 31.172 Mb) 21.12% 1.52% PAGE_READONLY 363 1ee5000 ( 30.895 Mb) 20.93% 1.51% PAGE_READWRITE 309 4c2000 ( 4.758 Mb) 3.22% 0.23% PAGE_WRITECOPY 43 6a000 ( 424.000 kb) 0.28% 0.02% PAGE_READWRITE|PAGE_GUARD 40 4b000 ( 300.000 kb) 0.20% 0.01% PAGE_EXECUTE_READWRITE 15 11000 ( 68.000 kb) 0.04% 0.00% PAGE_EXECUTE 1 1000 ( 4.000 kb) 0.00% 0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ---------- Free 6130000 5fb70000 ( 1.496 Gb) <unclassified> abf000 13d1000 ( 19.816 Mb) Image 75141000 879000 ( 8.473 Mb) Stack 3290000 fd000 (1012.000 kb) ActivationContextData 50000 4000 ( 16.000 kb) NlsTables 7efb0000 23000 ( 140.000 kb) TEB 7ef6f000 1000 ( 4.000 kb) CsrSharedMemory 7efe0000 5000 ( 20.000 kb) PEB 7efde000 1000 ( 4.000 kb)

// After the 1st failure

0:000> !address -summary

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal Free 267 7388c000 ( 1.805 Gb) 90.26% <unclassified> 272 8133000 ( 129.199 Mb) 64.80% 6.31% Image 614 31bf000 ( 49.746 Mb) 24.95% 2.43% Stack 60 1400000 ( 20.000 Mb) 10.03% 0.98% ActivationContextData 48 35000 ( 212.000 kb) 0.10% 0.01% NlsTables 1 23000 ( 140.000 kb) 0.07% 0.01% TEB 20 14000 ( 80.000 kb) 0.04% 0.00% CsrSharedMemory 1 5000 ( 20.000 kb) 0.01% 0.00% PEB 1 1000 ( 4.000 kb) 0.00% 0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_PRIVATE 297 6f8e000 ( 111.555 Mb) 55.95% 5.45% MEM_IMAGE 642 340c000 ( 52.047 Mb) 26.10% 2.54% MEM_MAPPED 78 23ca000 ( 35.789 Mb) 17.95% 1.75%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_FREE 267 7388c000 ( 1.805 Gb) 90.26% MEM_COMMIT 892 775e000 ( 119.367 Mb) 59.87% 5.83% MEM_RESERVE 125 5006000 ( 80.023 Mb) 40.13% 3.91%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal PAGE_READWRITE 314 38a3000 ( 56.637 Mb) 28.40% 2.77% PAGE_EXECUTE_READ 125 1f2c000 ( 31.172 Mb) 15.63% 1.52% PAGE_READONLY 363 1ee5000 ( 30.895 Mb) 15.49% 1.51% PAGE_WRITECOPY 34 4d000 ( 308.000 kb) 0.15% 0.01% PAGE_READWRITE|PAGE_GUARD 40 4b000 ( 300.000 kb) 0.15% 0.01% PAGE_EXECUTE_READWRITE 15 11000 ( 68.000 kb) 0.03% 0.00% PAGE_EXECUTE 1 1000 ( 4.000 kb) 0.00% 0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ---------- Free 94f4000 5c7ac000 ( 1.445 Gb) <unclassified> 6130000 33c4000 ( 51.766 Mb) Image 75141000 879000 ( 8.473 Mb) Stack 3290000 fd000 (1012.000 kb) ActivationContextData 50000 4000 ( 16.000 kb) NlsTables 7efb0000 23000 ( 140.000 kb) TEB 7ef6f000 1000 ( 4.000 kb) CsrSharedMemory 7efe0000 5000 ( 20.000 kb) PEB 7efde000 1000 ( 4.000 kb)

0:000> !address -f:VAR

BaseAddr EndAddr+1 RgnSize Type State Protect Usage ------------------------------------------------------------------------------------------- [...] 5e82000 5f70000 ee000 MEM_PRIVATE MEM_RESERVE <unclassified> 6130000 94f4000 33c4000 MEM_PRIVATE MEM_COMMIT PAGE_READWRITE <unclassified> 74220000 74221000 1000 MEM_IMAGE MEM_COMMIT PAGE_READONLY <unclassified> […]

0:000> ? 33c4000/0n1024 Evaluate expression: 53008 = 0000cf10

// After the 2nd failure

0:000> !address -summary

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal Free 268 704c8000 ( 1.755 Gb) 87.74% <unclassified> 273 b4f7000 ( 180.965 Mb) 72.05% 8.84% Image 614 31bf000 ( 49.746 Mb) 19.81% 2.43% Stack 60 1400000 ( 20.000 Mb) 7.96% 0.98% ActivationContextData 48 35000 ( 212.000 kb) 0.08% 0.01% NlsTables 1 23000 ( 140.000 kb) 0.05% 0.01% TEB 20 14000 ( 80.000 kb) 0.03% 0.00% CsrSharedMemory 1 5000 ( 20.000 kb) 0.01% 0.00% PEB 1 1000 ( 4.000 kb) 0.00% 0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_PRIVATE 298 a352000 ( 163.320 Mb) 65.03% 7.97% MEM_IMAGE 642 340c000 ( 52.047 Mb) 20.72% 2.54% MEM_MAPPED 78 23ca000 ( 35.789 Mb) 14.25% 1.75%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_FREE 268 704c8000 ( 1.755 Gb) 87.74% MEM_COMMIT 893 ab22000 ( 171.133 Mb) 68.14% 8.36% MEM_RESERVE 125 5006000 ( 80.023 Mb) 31.86% 3.91%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal PAGE_READWRITE 315 6c67000 ( 108.402 Mb) 43.16% 5.29% PAGE_EXECUTE_READ 125 1f2c000 ( 31.172 Mb) 12.41% 1.52% PAGE_READONLY 363 1ee5000 ( 30.895 Mb) 12.30% 1.51% PAGE_WRITECOPY 34 4d000 ( 308.000 kb) 0.12% 0.01% PAGE_READWRITE|PAGE_GUARD 40 4b000 ( 300.000 kb) 0.12% 0.01% PAGE_EXECUTE_READWRITE 15 11000 ( 68.000 kb) 0.03% 0.00% PAGE_EXECUTE 1 1000 ( 4.000 kb) 0.00% 0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ---------- Free c8c4000 593dc000 ( 1.394 Gb) <unclassified> 6130000 33c4000 ( 51.766 Mb) Image 75141000 879000 ( 8.473 Mb) Stack 3290000 fd000 (1012.000 kb) ActivationContextData 50000 4000 ( 16.000 kb) NlsTables 7efb0000 23000 ( 140.000 kb) TEB 7ef6f000 1000 ( 4.000 kb) CsrSharedMemory 7efe0000 5000 ( 20.000 kb) PEB 7efde000 1000 ( 4.000 kb)

0:000> !address -f:VAR

BaseAddr EndAddr+1 RgnSize Type State Protect Usage ------------------------------------------------------------------------------------------- 5e82000 5f70000 ee000 MEM_PRIVATE MEM_RESERVE <unclassified> 6130000 94f4000 33c4000 MEM_PRIVATE MEM_COMMIT PAGE_READWRITE <unclassified> 9500000 c8c4000 33c4000 MEM_PRIVATE MEM_COMMIT PAGE_READWRITE <unclassified> 74220000 74221000 1000 MEM_IMAGE MEM_COMMIT PAGE_READONLY <unclassified> […]

The name of this pattern comes from the process of discrete large memory allocations that happen after specific actions or events. Between them there is no visible or substantial increase in memory usage.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging | No Comments »

Crash Dump Analysis Patterns (Part 110, Mac OS X)

Sunday, April 22nd, 2012

This is a provisional Mac OS X example of Shared Buffer Overwrite pattern. Originally I wanted to construct a default C runtime heap corruption example using malloc / free functions. Unfortunately I couldn’t get heap corrupted easily as was possible in Windows Visual C++ environment by writing before or after allocated block. Desperately I printed allocated pointers and they all pointed to memory blocks laid out one after another without any headers in between (could be just a default Apple LLVM C runtime implementation and I have to check that with GCC). Therefore, any subsequent reallocation didn’t cause corruption either. So all this naturally fits into shared buffer overwrites or underwrites where corruption is only detectable when the overwritten data is used such as a pointer dereference.

int main(int argc, const char * argv[])

{

char *p1 = (char *) malloc (1024);

strcpy(p1, “Hello World!”);

printf(“p1 = %p\n”, p1);

printf(“*p1 = %s\n”, p1);

char *p2 = (char *) malloc (1024);

strcpy(p2, “Hello World!”);

printf(“p2 = %p\n”, p2);

printf(“*p2 = %s\n”, p2);

char *p3 = (char *) malloc (1024);

strcpy(p3, “Hello World!”);

printf(“p3 = %p\n”, p3);

printf(“*p3 = %s\n”, p3);

strcpy(p2-sizeof(p2), “Hello Crash!”);

strcpy(p3-sizeof(p3), “Hello Crash!”);

p2 = (char *)realloc(p2, 2048);

printf(“p2 = %p\n”, p2);

printf(“*p2 = %s\n”, p2);

char *p4 = (char *) malloc (1024);

strcpy(p4-sizeof(p4), “Hello Crash!”);

printf(“p4 = %p\n”, p4);

printf(“*p4 = %s\n”, p4);

p3 = (char *)realloc(p3, 2048);

printf(“p3 = %p\n”, p3);

printf(“*p3 = %s\n”, p3);

char *p5 = NULL; // to force a core dump

*p5 = 0;

free (p4);

free (p3);

free (p2);

free (p1);

return 0;

}

When we run the program above we get this output:

p1 = 0x7fc6d9000000 *p1 = Hello World! p2 = 0×7fc6d9001400 *p2 = Hello World! p3 = 0×7fc6d9001800 *p3 = Hello World! p2 = 0×7fc6d9001c00 *p2 = ash! p4 = 0×7fc6d9001400 *p4 = ash! p3 = 0×7fc6d9002400 *p3 = ash! Segmentation fault: 11 (core dumped)

Now is GDB output:

(gdb) x/1024bc p1 0x7fc6d9000000: 72 ‘H’ 101 ‘e’ 108 ‘l’ 108 ‘l’ 111 ‘o’ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9000008: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9000010: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ […] 0×7fc6d90003e8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d90003f0: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d90003f8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/32bc p1+1024-sizeof(p1) 0×7fc6d90003f8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9000400: 42 ‘*’ 112 ‘p’ 51 ‘3′ 32 ‘ ‘ 61 ‘=’ 32 ‘ ‘ 97 ‘a’ 115 ’s’ 0×7fc6d9000408: 104 ‘h’ 33 ‘!’ 10 ‘\n’ 100 ‘d’ 57 ‘9′ 48 ‘0′ 48 ‘0′ 50 ‘2′ 0×7fc6d9000410: 52 ‘4′ 48 ‘0′ 48 ‘0′ 10 ‘\n’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/2048bc p2 0×7fc6d9001c00: 97 ‘a’ 115 ’s’ 104 ‘h’ 33 ‘!’ 0 ‘\0′ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9001c08: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001c10: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ […] 0×7fc6d9001fe8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001ff0: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001ff8: 72 ‘H’ 101 ‘e’ 108 ‘l’ 108 ‘l’ 111 ‘o’ 32 ‘ ‘ 67 ‘C’ 114 ‘r’ 0×7fc6d9002000: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002008: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ […] 0×7fc6d90023e8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d90023f0: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d90023f8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/64bc p2-sizeof(p2) 0×7fc6d9001bf8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001c00: 97 ‘a’ 115 ’s’ 104 ‘h’ 33 ‘!’ 0 ‘\0′ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9001c08: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001c10: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001c18: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001c20: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001c28: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001c30: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/64bc p2+2048-sizeof(p2) 0×7fc6d90023f8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002400: 97 ‘a’ 115 ’s’ 104 ‘h’ 33 ‘!’ 0 ‘\0′ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9002408: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002410: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002418: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002420: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002428: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002430: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/1024bc p3 0×7fc6d9002400: 97 ‘a’ 115 ’s’ 104 ‘h’ 33 ‘!’ 0 ‘\0′ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9002408: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002410: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ […] 0×7fc6d90027e8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d90027f0: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d90027f8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/64bc p3-sizeof(p3) 0×7fc6d90023f8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002400: 97 ‘a’ 115 ’s’ 104 ‘h’ 33 ‘!’ 0 ‘\0′ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9002408: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002410: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002418: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002420: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002428: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002430: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/64bc p3+1024-sizeof(p3) 0×7fc6d90027f8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002800: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002808: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002810: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002818: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002820: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002828: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9002830: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/1024bc p4 0×7fc6d9001400: 97 ‘a’ 115 ’s’ 104 ‘h’ 33 ‘!’ 0 ‘\0′ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9001408: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001410: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ […] 0×7fc6d90017e8: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d90017f0: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d90017f8: 72 ‘H’ 101 ‘e’ 108 ‘l’ 108 ‘l’ 111 ‘o’ 32 ‘ ‘ 67 ‘C’ 114 ‘r’ (gdb) x/64bc p4-sizeof(p4) 0×7fc6d90013f8: 72 ‘H’ 101 ‘e’ 108 ‘l’ 108 ‘l’ 111 ‘o’ 32 ‘ ‘ 67 ‘C’ 114 ‘r’ 0×7fc6d9001400: 97 ‘a’ 115 ’s’ 104 ‘h’ 33 ‘!’ 0 ‘\0′ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9001408: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001410: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001418: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001420: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001428: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001430: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ (gdb) x/64bc p4+1024-sizeof(p4) 0×7fc6d90017f8: 72 ‘H’ 101 ‘e’ 108 ‘l’ 108 ‘l’ 111 ‘o’ 32 ‘ ‘ 67 ‘C’ 114 ‘r’ 0×7fc6d9001800: 97 ‘a’ 115 ’s’ 104 ‘h’ 33 ‘!’ 0 ‘\0′ 32 ‘ ‘ 87 ‘W’ 111 ‘o’ 0×7fc6d9001808: 114 ‘r’ 108 ‘l’ 100 ‘d’ 33 ‘!’ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001810: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001818: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001820: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001828: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0×7fc6d9001830: 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′ 0 ‘\0′

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Core Dump Analysis, Crash Dump Analysis, Crash Dump Patterns, Debugging, GDB for WinDbg Users, Mac Crash Corner, Mac OS X | No Comments »

Forthcoming 2nd edition of Memory Dump Analysis Anthology, Volume 1

Sunday, April 15th, 2012

After 4 years in print this bestselling title needs an update to address minor changes, include extra examples and reference additional research published in Volumes 2, 3, 4, 5 and 6.

Title: Memory Dump Analysis Anthology, Volume 1
Author: Dmitry Vostokov
Publisher: OpenTask (Summer 2012)
Language: English
Product Dimensions: 22.86 x 15.24

Paperback: 800 pages
ISBN-13: 978-1-908043-35-1

Hardcover: 800 pages
ISBN-13: 978-1-908043-36-8

The cover for both paperback and hardcover titles will also have a matte finish. We used A Memory Window artwork for the back cover.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Aesthetics of Memory Dumps, Announcements, AntiPatterns, Art, Assembly Language, Best Practices, Books, Bugchecks Depicted, C and C++, Complete Memory Dump Analysis, Computer Science, Crash Dump Analysis, Crash Dump Patterns, Crash Dumps for Dummies, Debugging, Debugging Methodology, Dr. Watson, Escalation Engineering, Fun with Crash Dumps, GDB for WinDbg Users, Hardware, Images of Computer Memory, Kernel Development, Mathematics of Debugging, Memiotics (Memory Semiotics), Memoretics, Memory Dump Analysis Methodology, Memory Space Art, Memory Space Music, Memory Visualization, Minidump Analysis, Multithreading, Pattern-Driven Debugging, Pattern-Driven Software Support, Publishing, Reference, Root Cause Analysis, Science of Memory Dump Analysis, Software Architecture, Software Behavior DNA, Software Behavior Patterns, Software Behavioral Genome, Software Diagnostics, Software Engineering, Software Technical Support, Stack Trace Collection, Testing, Tools, Troubleshooting Methodology, Vista, WinDbg Scripts, WinDbg Tips and Tricks, WinDbg for GDB Users, Windows 7, Windows Data Structures, Windows Server 2008, Windows System Administration, x64 Windows | No Comments »

Crash Dump Analysis Patterns (Part 13h)

Monday, April 9th, 2012

Allocated dynamic memory such as process heap can remain reserved after deallocation and its virtual memory region might become unavailable for usage. One example of this I encountered recently while debugging a .NET service. During a peak usage it reported various out-of-memory events but its managed heap was healthy and didn’t consume much. However, its process heap statistics showed a large reserved heap segment missing in a similar memory dump from a development environment. Remaining allocated entries in that heap segment contained a specific module hint that allowed us to suggest removing a 3rd-party product from a production environment.

In order to provide the proof of that possible scenario of reserved heap regions we created a special modeling application:

int _tmain(int argc, _TCHAR* argv[])
{
static char *pAlloc[1000000];
for (int i = 0; i < 1000000; i++)
{
pAlloc[i] = (char *)malloc (1000);
}
getc(stdin);
for (int i = 0; i < 1000000; i++)
{
free(pAlloc[i]);
}
getc(stdin);
return 0;
}

Here’s the debugging log:

0:001> .symfix c:\mss

0:001> .reload Reloading current modules .....

After allocation:

0:001> !heap -s LFH Key : 0x156356e0 Termination on corruption : ENABLED Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast (k) (k) (k) (k) length blocks cont. heap —————————————————————————– 00520000 00000002 1024 112 1024 8 1 1 0 0 LFH 007e0000 00001002 1019328 1012444 1019328 131 68 67 0 0 LFH —————————————————————————–

0:001> g (1588.14b0): Break instruction exception - code 80000003 (first chance) eax=7efda000 ebx=00000000 ecx=00000000 edx=770ff85a esi=00000000 edi=00000000 eip=7707000c esp=00f0f7e4 ebp=00f0f810 iopl=0 nv up ei pl zr na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 ntdll!DbgBreakPoint: 7707000c cc int 3

After deallocation:

0:001> !heap -s LFH Key : 0x156356e0 Termination on corruption : ENABLED Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast (k) (k) (k) (k) length blocks cont. heap —————————————————————————– 00520000 00000002 1024 112 1024 8 1 1 0 0 LFH 007e0000 00001002 1019328 73040 1019328 71365 419 165 0 0 LFH External fragmentation 97 % (419 free blocks) Virtual address fragmentation 92 % (165 uncommited ranges) —————————————————————————–

0:001> !address -summary --- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal Free 26 3fbe7000 (1019.902 Mb) 49.80% <unclassified> 752 3f8ec000 (1016.922 Mb) 98.92% 49.66% Image 41 76b000 ( 7.418 Mb) 0.72% 0.36% Stack 6 200000 ( 2.000 Mb) 0.19% 0.10% MemoryMappedFile 8 1af000 ( 1.684 Mb) 0.16% 0.08% TEB 2 2000 ( 8.000 kb) 0.00% 0.00% PEB 1 1000 ( 4.000 kb) 0.00% 0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_PRIVATE 734 3f8a2000 (1016.633 Mb) 98.89% 49.64% MEM_IMAGE 68 9b8000 ( 9.719 Mb) 0.95% 0.47% MEM_MAPPED 8 1af000 ( 1.684 Mb) 0.16% 0.08%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_FREE 26 3fbe7000 (1019.902 Mb) 49.80% MEM_RESERVE 374 3f6e8000 (1014.906 Mb) 98.72% 49.56% MEM_COMMIT 436 d21000 ( 13.129 Mb) 1.28% 0.64%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal PAGE_READWRITE 383 725000 ( 7.145 Mb) 0.69% 0.35% PAGE_EXECUTE_READ 10 414000 ( 4.078 Mb) 0.40% 0.20% PAGE_READONLY 29 1cd000 ( 1.801 Mb) 0.18% 0.09% PAGE_WRITECOPY 10 12000 ( 72.000 kb) 0.01% 0.00% PAGE_READWRITE|PAGE_GUARD 4 9000 ( 36.000 kb) 0.00% 0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ---------- Free 3f0c0000 33050000 ( 816.313 Mb) <unclassified> 158a1000 fcf000 ( 15.809 Mb) Image 1083000 3d1000 ( 3.816 Mb) Stack 200000 fd000 (1012.000 kb) MemoryMappedFile 7efe5000 fb000 (1004.000 kb) TEB 7efda000 1000 ( 4.000 kb) PEB 7efde000 1000 ( 4.000 kb)

We see that free memory available for allocation is only 816 Mb.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging | No Comments »

Forthcoming Introduction to Pattern-Driven Software Diagnostics

Monday, April 9th, 2012

Memory Dump Analysis Services organizes a free Webinar on Unified Software Diagnostics (USD) and the new scalable cost-effective software support model called Pattern-Driven Software Support devised to address various shortcomings in existing tiered software support organizations. Examples cover Windows, Mac OS and Linux.

Introduction to Pattern-Driven Software Diagnostics Logo

Date: 22nd of June, 2012
Time: 17:00 (BST) 12:00 (EST) 09:00 (PST)
Duration: 60 minutes

Space is limited.
Reserve your Webinar seat now at:
https://www3.gotomeeting.com/register/172771078

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Announcements, Best Practices, Crash Dump Analysis, Crash Dump Patterns, Debugging, Debugging Methodology, Debugging Trends, Economics, Escalation Engineering, Event Tracing for Windows (ETW), JIT Crash Analysis, JIT Memory Space Analysis, Malware Analysis, Malware Patterns, Memoretics, Memory Analysis Forensics and Intelligence, Memory Dump Analysis Methodology, Memory Dump Analysis Services, New Acronyms, New Debugging School, Pattern Prediction, Pattern-Driven Debugging, Pattern-Driven Software Support, Root Cause Analysis, Science of Memory Dump Analysis, Science of Software Tracing, Software Behavior DNA, Software Behavior Patterns, Software Behavioral Genome, Software Diagnostics, Software Engineering, Software Narratology, Software Problem Description Patterns, Software Problem Solving, Software Support Patterns, Software Technical Support, Software Trace Analysis, Software Troubleshooting Patterns, Software Victimology, Software and Economics, Structural Memory Patterns, Structural Trace Patterns, Systems Thinking, Testing, Tools, Trace Analysis Patterns, Training and Seminars, Troubleshooting Methodology, UI Problem Analysis Patterns, Unified Debugging Patterns, Unified Software Diagnostics, Victimware, Webinars, Workaround Patterns | No Comments »

Crash Dump Analysis Patterns (Part 170)

Thursday, April 5th, 2012

Address space-wide search for errors and status codes may show Coincidental Error Code pattern:

0:000> !heap -x -v c0000005 Search VM for address range c0000005 - c0000005 : 028690b8 (c0000005), [...]

0:000> dd 028690b8 l1 028690b8 c0000005

In such cases we need to check whether the addresses belong to volatile regions such as stack because it is possible to have such values as legitimate code and image data:

0:000> !address 028690b8 Usage: Image Allocation Base: 02700000 Base Address: 02869000 End Address: 02874000 Region Size: 0000b000 Type: 01000000 MEM_IMAGE State: 00001000 MEM_COMMIT Protect: 00000002 PAGE_READONLY More info: lmv m ModuleA More info: !lmi ModuleA More info: ln 0×28690b8

0:000> u 028690b8 ModuleA!ComputeB: 028690b8 050000c000 add eax,0C00000h [...]

Another example:

0:000> !heap -x -v c0000005 Search VM for address range 00000000c0000005 - 00000000c0000005 : 7feff63ab60 (c0000005),

0:000> !address 7feff63ab60 Usage: Image Allocation Base: 000007fe`ff460000 Base Address: 000007fe`ff635000 End Address: 000007fe`ff63c000 Region Size: 00000000`00007000 Type: 01000000 MEM_IMAGE State: 00001000 MEM_COMMIT Protect: 00000004 PAGE_READWRITE More info: lmv m ole32 More info: !lmi ole32 More info: ln 0×7feff63ab60

0:000> dp 7feff63ab60 000007fe`ff63ab60 00000000`c0000005 c0000194`00000001 000007fe`ff63ab70 00000001`00000000 00000000`c00000aa 000007fe`ff63ab80 80000002`00000001 00000001`00000000 000007fe`ff63ab90 00000000`c0000096 c000001d`00000001 000007fe`ff63aba0 00000001`00000000 00000000`80000003 000007fe`ff63abb0 c00000fd`00000001 00000001`00000000 000007fe`ff63abc0 00000000`c0000235 c0000006`00000001 000007fe`ff63abd0 00000001`00000000 00000000`c0000420

In the latter case the data structure suggests a table of errors:

0:000> ln 7feff63ab60 (000007fe`ff63ab60) ole32!gReportedExceptions

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Patterns, Debugging, x64 Windows | No Comments »

Crash Dump Analysis Patterns (Part 1, Mac OS X)

Wednesday, April 4th, 2012

The first Windows pattern called Multiple Exceptions in user mode now has Mac OS X equivalent. In the example below there are 3 threads and two of them experienced NULL Pointer (data) access violation exception:

(gdb) thread apply all bt full

Thread 3 (core thread 2): #0 0x00000001062ffe4e in thread_two (arg=0x0) at main.c:24 p = (int *) 0×0 #1 0×00007fff8abf58bf in _pthread_start () No symbol table info available. #2 0×00007fff8abf8b75 in thread_start () No symbol table info available.

Thread 2 (core thread 1): #0 0x00000001062ffe1e in thread_one (arg=0x0) at main.c:16 p = (int *) 0×0 #1 0×00007fff8abf58bf in _pthread_start () No symbol table info available. #2 0×00007fff8abf8b75 in thread_start () No symbol table info available.

Thread 1 (core thread 0): #0 0x00007fff854e0e42 in __semwait_signal () No symbol table info available. #1 0x00007fff8ababdea in nanosleep () No symbol table info available. #2 0x00007fff8ababc2c in sleep () No symbol table info available. #3 0x00000001062ffec3 in main (argc=1, argv=0x7fff65efeab8) at main.c:36 threadID_one = (pthread_t) 0×1063b4000 threadID_two = (pthread_t) 0×106581000

(gdb) thread 2 [Switching to thread 2 (core thread 1)] 0x00000001062ffe1e in thread_one (arg=0x0) at main.c:16 16 *p = 1;

(gdb) p/x p $1 = 0×0

(gdb) thread 3 [Switching to thread 3 (core thread 2)] 0x00000001062ffe4e in thread_two (arg=0x0) at main.c:24 24 *p = 2;

(gdb) p/x p $2 = 0×0

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Core Dump Analysis, Crash Dump Analysis, Crash Dump Patterns, Debugging, GDB for WinDbg Users, Mac Crash Corner, Mac OS X | No Comments »

Crash Dump Analysis Patterns (Part 25, Mac OS X)

Sunday, March 25th, 2012

This is a Mac OS X / GDB counterpart to Stack Trace pattern previously described for Windows platforms. Here we show a stack trace when symbols are not available and also how to apply symbols:

(gdb) bt #0 0×000000010d3b0e90 in ?? () #1 0×000000010d3b0ea9 in ?? () #2 0×000000010d3b0ec4 in ?? () #3 0×000000010d3b0e74 in ?? ()

(gdb) maintenance info sections Exec file: [...] Core file: `/cores/core.262', file type mach-o-le. 0×000000010d3b0000->0×000000010d3b1000 at 0×00001000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3b1000->0×000000010d3b2000 at 0×00002000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3b2000->0×000000010d3b3000 at 0×00003000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3b3000->0×000000010d3b4000 at 0×00004000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3b4000->0×000000010d3b5000 at 0×00005000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3b5000->0×000000010d3b6000 at 0×00006000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3b6000->0×000000010d3cb000 at 0×00007000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3cb000->0×000000010d3cc000 at 0×0001c000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3cc000->0×000000010d3cd000 at 0×0001d000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3cd000->0×000000010d3e2000 at 0×0001e000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3e2000->0×000000010d3e3000 at 0×00033000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d3e3000->0×000000010d3e4000 at 0×00034000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0×000000010d400000->0×000000010d500000 at 0×00035000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS […]

(gdb) add-symbol-file ~/Documents/Work/Test.sym 0×000000010d3b0000 add symbol table from file “/Users/DumpAnalysis/Documents/Work/Test.sym” at LC_SEGMENT.__TEXT = 0×10d3b0000 (y or n) y Reading symbols from /Users/DumpAnalysis/Documents/Work/Test.sym…done.

(gdb) bt #0 0x000000010d3b0e90 in bar () at main.c:15 #1 0x000000010d3b0ea9 in foo () at main.c:20 #2 0x000000010d3b0ec4 in main (argc=1, argv=0x7fff6cfafbf8) at main.c:25

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Forthcoming Training: Accelerated Mac OS X Core Dump Analysis

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Core Dump Analysis, Crash Dump Analysis, Crash Dump Patterns, Debugging, GDB for WinDbg Users, Mac Crash Corner, Mac OS X | No Comments »

Crash Dump Analysis Patterns (Part 6b, Mac OS X)

Sunday, March 25th, 2012

This is a Mac OS X / GDB counterpart to NULL Pointer (data) pattern previously described for Windows platforms:

(gdb) bt #0 0×000000010d3b0e90 in bar () at main.c:15 #1 0×000000010d3b0ea9 in foo () at main.c:20 #2 0×000000010d3b0ec4 in main (argc=1, argv=0×7fff6cfafbf8) at main.c:25

(gdb) disassemble Dump of assembler code for function bar: 0x000000010d3b0e80 <bar+0>: push %rbp 0×000000010d3b0e81 <bar+1>: mov %rsp,%rbp 0×000000010d3b0e84 <bar+4>: movq $0×0,-0×8(%rbp) 0×000000010d3b0e8c <bar+12>: mov -0×8(%rbp),%rax 0×000000010d3b0e90 <bar+16>: movl $0×1,(%rax) 0×000000010d3b0e96 <bar+22>: pop %bp 0×000000010d3b0e97 <bar+23>: retq End of assembler dump.

(gdb) p/x $rax $1 = 0×0

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Forthcoming Training: Accelerated Mac OS X Core Dump Analysis

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Assembly Language, Core Dump Analysis, Crash Dump Analysis, Crash Dump Patterns, Debugging, GDB for WinDbg Users, Mac Crash Corner, Mac OS X, Software Defect Construction, x64 Mac OS X | No Comments »

WinDbg shortcuts: !heap -x -v

Friday, March 23rd, 2012

The following command is useful for searching a process virtual space for any value references:

!heap -x -v <value> ”will search the entire virtual memory space of the current process for pointers to this” value (from WinDbg help).

Example:

0:000> !heap -x -v 6e412d82 Search VM for address range 6e412d82 - 6e412d82 : 778042bc (6e412d82),

0:000> dp 778042bc l1 778042bc 6e412d82

0:000> !heap -x -v c0000005 Search VM for address range c0000005 - c0000005 : 014df8d0 (c0000005), 014dfe8c (c0000005), 0155d908 (c0000005), 0155dd10 (c0000005), 0155ddc8 (c0000005), 0155dfa8 (c0000005), 0155dff0 (c0000005), 0155ea20 (c0000005), 6d000f9c (c0000005), 70d44054 (c0000005), 725c30d4 (c0000005), 7270d20c (c0000005), 7282ef74 (c0000005), 7449a878 (c0000005), 74511958 (c0000005), 74562ec4 (c0000005), 74563280 (c0000005), 74564fc8 (c0000005), 7456562c (c0000005), 74565748 (c0000005), 745664a8 (c0000005), 74566a30 (c0000005), 74566ad8 (c0000005), 747f6730 (c0000005), 747f682c (c0000005), 74861ef0 (c0000005), 7488743c (c0000005), 748aea68 (c0000005), 748b2830 (c0000005), 748c5118 (c0000005), 74935068 (c0000005), 749412a8 (c0000005), 7495caf0 (c0000005), 74a3a780 (c0000005), 74aa462c (c0000005), 74b19b68 (c0000005), 74b61060 (c0000005), 74b8fb44 (c0000005), 74b9d1c8 (c0000005), 74be1ad8 (c0000005), 74be72c8 (c0000005), 74c14b60 (c0000005), 74c83b84 (c0000005), 74c83b88 (c0000005), 74c83b9c (c0000005), 74c83ba0 (c0000005), 74c83ba4 (c0000005), 74c83ba8 (c0000005), 74c83bac (c0000005), 74c83bb0 (c0000005), 74c83bb4 (c0000005), 74c83bb8 (c0000005), 74c83bbc (c0000005), 74c83bc0 (c0000005), 74c83bc8 (c0000005), 74c83bcc (c0000005), 74c83bd0 (c0000005), 74c83bd4 (c0000005), 74c83bd8 (c0000005), 74c83bdc (c0000005), 74c83be0 (c0000005), 74c83be4 (c0000005), 74c83be8 (c0000005), 74c83bec (c0000005), 74c83bf0 (c0000005), 74c83bf4 (c0000005), 74c83bf8 (c0000005), 74c83bfc (c0000005), 74c83c00 (c0000005), 74c83c04 (c0000005), 74c83c08 (c0000005), 74c83c0c (c0000005), 74c83c10 (c0000005), 74c83c14 (c0000005), 74c83c18 (c0000005), 74c83c1c (c0000005), 74c83c20 (c0000005), 74c83c24 (c0000005), 74c83c28 (c0000005), 74c83c2c (c0000005), 74c83c34 (c0000005), 74c83c38 (c0000005), 74c83c3c (c0000005), 74c8c7ac (c0000005), 75019298 (c0000005), 750ff7b0 (c0000005), 751c1adc (c0000005), 751c2514 (c0000005), 7522c530 (c0000005), 752c311c (c0000005), 752d4734 (c0000005), 752d4ae8 (c0000005), 752d534c (c0000005), 752d7038 (c0000005), 752d7e9c (c0000005), 752eda04 (c0000005), 752edab0 (c0000005), 756d6624 (c0000005), 7571adc0 (c0000005), 7571addc (c0000005), 75723780 (c0000005), 757af774 (c0000005), 759c0f10 (c0000005), 76702360 (c0000005), 76703a30 (c0000005), 76d437ac (c0000005), 76d527ec (c0000005), 76dd0fa4 (c0000005), 77581f2c (c0000005), 777a33c0 (c0000005), 777c8b14 (c0000005),

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging, WinDbg Tips and Tricks | No Comments »

Meta-Memory Dump Patterns

Thursday, March 22nd, 2012

A page to reference all different kinds of patterns related to memory dumps as a whole and their properties is necessary, so I created this post:

I’ll update it as soon as I add more similar patterns.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging | No Comments »

Crash Dump Analysis Patterns (Part 168)

Tuesday, March 20th, 2012

This is a binary opposition counterpart to Early Crash Dump pattern called Late Crash Dump. It is usually saved after patterns representing a problem such as an exception thread stack trace are already gone. Most often we see one thread with process termination functions (Special Stack Trace pattern):

0:000> ~*k ChildEBP RetAddr 0037fcf0 770bd55c ntdll!ZwTerminateProcess+0x12 0037fd0c 750f79f4 ntdll!RtlExitUserProcess+0x85 0037fdf8 750f339a kernel32!ExitProcessStub+0x12 0037fe04 770a9ef2 kernel32!BaseThreadInitThunk+0xe 0037fe44 770a9ec5 ntdll!__RtlUserThreadStart+0x70 0037fe5c 00000000 ntdll!_RtlUserThreadStart+0x1b

0:000> ~*k ChildEBP RetAddr 0032faf0 77a9d55c ntdll!ZwTerminateProcess+0x12 0032fb0c 775579f4 ntdll!RtlExitUserProcess+0x85 0032fb20 74ac1720 kernel32!ExitProcessStub+0x12 0032fb28 74ac1a03 msvcr80!__crtExitProcess+0x14 0032fb64 74ac1a4b msvcr80!_cinit+0x101 0032fb74 01339bb3 msvcr80!exit+0xd 0032fbf8 7755339a App!__tmainCRTStartup+0x155 0032fc04 77a89ef2 kernel32!BaseThreadInitThunk+0xe 0032fc44 77a89ec5 ntdll!__RtlUserThreadStart+0x70 0032fc5c 00000000 ntdll!_RtlUserThreadStart+0x1b

However, sometimes, it is possible to see some execution residue left on a raw stack such as hidden exceptions, module hints, error codes and handled exceptions that might shed light on possible causes.

Another variant is when a memory dump is saved after a problem message box is dismissed or potentially disastrous exceptions such as access violations are handled and ignored to the fault in exception handling mechanism or severe corruption resuted in unresponsive process (hang).

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Sponsored link: Professional Software Debugging Services

/* Malware and Software Defects -> Victimware.org */

Memory Dump It

Posted in Crash Dump Analysis, Crash Dump Patterns, Debugging | No Comments »

Archive for the ‘Crash Dump Patterns’ Category

Crash Dump Analysis Patterns (Part 174)

Crash Dump Analysis Patterns (Part 173)

Crash Dump Analysis Patterns (Part 20d)

Crash Dump Analysis Patterns (Part 172)

Crash Dump Analysis Patterns (Part 14, Mac OS X)

Crash Dump Analysis Patterns (Part 6a, Mac OS X)

Crash Dump Analysis Patterns (Part 53c)

Software Behavior Pattern Prediction

Crash Dump Analysis Patterns (Part 171)

Crash Dump Analysis Patterns (Part 110, Mac OS X)

Forthcoming 2nd edition of Memory Dump Analysis Anthology, Volume 1

Crash Dump Analysis Patterns (Part 13h)

Forthcoming Introduction to Pattern-Driven Software Diagnostics

Crash Dump Analysis Patterns (Part 170)

Crash Dump Analysis Patterns (Part 1, Mac OS X)

Crash Dump Analysis Patterns (Part 25, Mac OS X)

Crash Dump Analysis Patterns (Part 6b, Mac OS X)

WinDbg shortcuts: !heap -x -v

Meta-Memory Dump Patterns

Crash Dump Analysis Patterns (Part 168)

Pages

Categories

Recent Comments

Archives

Blogroll

Contributors

Debugging Channels

Intelligence Analysis

Mac OS X

Magazines and Newspapers

Malware Analysis

Narratology

Related Links

Source Code

Meta

May 2012
M	T	W	T	F	S	S
« Apr
	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26	27
28	29	30	31