Archive for June 5th, 2012

Crash Dump Analysis Patterns (Part 60, Mac OS X)

Tuesday, June 5th, 2012

This is a Mac OS X / GDB counterpart to Execution Residue pattern previously described for Windows platforms:

(gdb) bt
#0 0x00007fff8616e82a in __kill ()
#1 0x00007fff8fab9a9c in abort ()
#2 0x000000010269dc29 in bar_5 ()
#3 0x000000010269dc39 in bar_4 ()
#4 0x000000010269dc49 in bar_3 ()
#5 0x000000010269dc59 in bar_2 ()
#6 0x000000010269dc69 in bar_1 ()
#7 0x000000010269dc79 in bar ()
#8 0x000000010269dca0 in main (argc=1, argv=0x7fff6229cb00)

(gdb) x $rsp
0x7fff6229ca38: 0x8fab9a9c

(gdb) x/1000a 0x7fff6229c000
0×7fff6229c000: 0×7fff8947b000 0×7fff8947b570
0×7fff6229c010: 0×4f3ee10c 0×7fff90cb0000
0×7fff6229c020: 0×7fff90cb04d0 0×4e938b16
[…]
0×7fff6229c5f0: 0×7fff622d8d80 0×10269d640
0×7fff6229c600: 0×7fff6229cad0 0×7fff622a460b
0×7fff6229c610: 0×100000000 0×269d000
0×7fff6229c620: 0×7fff6229c630 0×10269db59 <foo_8+9>
0×7fff6229c630: 0×7fff6229c640 0×10269db69 <foo_7+9>
0×7fff6229c640: 0×7fff6229c650 0×10269db79 <foo_6+9>
0×7fff6229c650: 0×7fff6229c660 0×10269db89 <foo_5+9>
0×7fff6229c660: 0×7fff6229c670 0×10269db99 <foo_4+9>
0×7fff6229c670: 0×7fff6229c680 0×10269dba9 <foo_3+9>
0×7fff6229c680: 0×7fff6229c690 0×10269dbb9 <foo_2+9>
0×7fff6229c690: 0×7fff6229c6a0 0×10269dbc9 <foo_1+9>
0×7fff6229c6a0: 0×7fff6229cac0 0×10269dbee <foo+30>
0×7fff6229c6b0: 0×0 0×0
0×7fff6229c6c0: 0×0 0×0
0×7fff6229c6d0: 0×0 0×0
0×7fff6229c6e0: 0×0 0×0
[…]
0×7fff6229c8d0: 0×7fff6229c960 0×7fff622b49cd
0×7fff6229c8e0: 0×10269f05c 0×0
0×7fff6229c8f0: 0×7fff622c465c 0×7fff8a31e5c0 <_Z21dyldGlobalLockReleasev>
0×7fff6229c900: 0×7fff8fab99eb <abort> 0×10269f05c
0×7fff6229c910: 0×101000000000000 0×7fff622d2110
0×7fff6229c920: 0×7fff622d8d80 0×10269f078
0×7fff6229c930: 0×7fff622daac8 0×18
0×7fff6229c940: 0×0 0×0
0×7fff6229c950: 0×10269e030 0×0
0×7fff6229c960: 0×7fff6229c980 0×7fff622a1922
0×7fff6229c970: 0×0 0×0
0×7fff6229c980: 0×7fff6229ca50 0×7fff8a31e716 <dyld_stub_binder_+13>
0×7fff6229c990: 0×1 0×7fff6229cb00
0×7fff6229c9a0: 0×7fff6229cb10 0xe223ea612ddc10b7
0×7fff6229c9b0: 0×8 0×0
0×7fff6229c9c0: 0xe223ea612ddc10b7 0×0
0×7fff6229c9d0: 0×0 0×0
0×7fff6229c9e0: 0×585f5f00474e414c 0×20435058005f4350
0×7fff6229c9f0: 0×0 0×0
0×7fff6229ca00: 0×0 0×0
0×7fff6229ca10: 0×0 0×0
0×7fff6229ca20: 0×0 0×0
0×7fff6229ca30: 0×7fff6229ca60 0×7fff8fab9a9c <abort+177>
0×7fff6229ca40: 0×0 0×0
0×7fff6229ca50: 0×7fffffffffdf 0×0
0×7fff6229ca60: 0×7fff6229ca70 0×10269dc29 <bar_5+9>
0×7fff6229ca70: 0×7fff6229ca80 0×10269dc39 <bar_4+9>
0×7fff6229ca80: 0×7fff6229ca90 0×10269dc49 <bar_3+9>
0×7fff6229ca90: 0×7fff6229caa0 0×10269dc59 <bar_2+9>
0×7fff6229caa0: 0×7fff6229cab0 0×10269dc69 <bar_1+9>
0×7fff6229cab0: 0×7fff6229cac0 0×10269dc79 <bar+9>
0×7fff6229cac0: 0×7fff6229cae0 0×10269dca0 <main+32>
0×7fff6229cad0: 0×7fff6229cb00 0×1
0×7fff6229cae0: 0×7fff6229caf0 0×10269db34 <start+52>
0×7fff6229caf0: 0×0 0×1
0×7fff6229cb00: 0×7fff6229cc48 0×0
0×7fff6229cb10: 0×7fff6229ccae 0×7fff6229ccca
[…]

Here’s the source code of the modeling application:

#define def_call(name,x,y) void name##_##x() { name##_##y(); }

#define def_final(name,x) void name##_##x() { }

#define def_final_abort(name,x) void name##_##x() { abort(); }

#define def_init(name,y) void name() { name##_##y(); }

#define def_init_alloc(name,y,size) void name() { int arr[size]; name##_##y(); *arr=0; }

def_final(foo,9)

def_call(foo,8,9)

def_call(foo,7,8)

def_call(foo,6,7)

def_call(foo,5,6)

def_call(foo,4,5)

def_call(foo,3,4)

def_call(foo,2,3)

def_call(foo,1,2)

def_init_alloc(foo,1,256)

def_final_abort(bar,5)

def_call(bar,4,5)

def_call(bar,3,4)

def_call(bar,2,3)

def_call(bar,1,2)

def_init(bar,1)

int main(int argc, const char * argv[])

{

    foo();

    bar();

} 

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Forthcoming Training: Accelerated Mac OS X Core Dump Analysis

Posted in Core Dump Analysis, Debugging, GDB for WinDbg Users, Mac Crash Corner, Mac OS X, Software Defect Construction, x64 Mac OS X | 2 Comments »

7 Habits of Highly Effective Diagnosticians (Part 0)

Tuesday, June 5th, 2012

Motivated by 7 Habits of Highly Effective Debuggers I would like to reflect on a distinction between diagnostics and problem solving as separate processes (although highly related). First, we reverse the precept from that article because stories such as software logs and traces are of primary importance to software diagnostics (and not only). And without diagnostics there is no effective debugging (treatment, problem solving, etc.)

The Principle Precept of Diagnostics

Stories NOT Statistics secure certainty.

Next parts will be about actual habits so please stay tuned. I would try to finish this list before the forthcoming Webinar on software diagnostics.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

Posted in 7 Habits, Core Dump Analysis, Crash Dump Analysis, Escalation Engineering, Software Diagnostics, Software Engineering, Software Narratology, Software Problem Solving, Software Technical Support, Software Trace Analysis | No Comments »