Software Diagnostics Library

Here we show the possible signs of the classical Thread Starvation.
Suppose we have two running threads with a priority 8:

0: kd> !running

System Processors 3 (affinity mask)
  Idle Processors 0

Prcbs  Current   Next   
  0    ffdff120  89a92020            O...............
  1    f7737120  89275020            W...............

0: kd> !thread 89a92020
THREAD 89a92020  Cid 11d8.27d8  Teb: 7ffd9000 Win32Thread: bc1e6860 RUNNING on processor 0
[...]
Priority 8 BasePriority 8 PriorityDecrement 0

0: kd> !thread 89275020
THREAD 89275020  Cid 1cd0.2510  Teb: 7ffa9000 Win32Thread: bc343180 RUNNING on processor 1
[...]
Priority 8 BasePriority 8 PriorityDecrement 0

If we have other threads ready with the same priority contending for the same processors other threads with less priority might starve (shown in red):

0: kd> !ready
Processor 0: Ready Threads at priority 8
    THREAD 894a1db0  Cid 1a98.25c0  Teb: 7ffde000 Win32Thread: bc19cea8 READY
    THREAD 897c4818  Cid 11d8.1c5c  Teb: 7ffa2000 Win32Thread: bc2c5ba8 READY
    THREAD 8911fd18  Cid 2730.03f4  Teb: 7ffd9000 Win32Thread: bc305830 READY
Processor 0: Ready Threads at priority 7
    THREAD 8a9e5ab0  Cid 0250.0470  Teb: 7ff9f000 Win32Thread: 00000000 READY
    THREAD 8a086838  Cid 0250.0654  Teb: 7ff93000 Win32Thread: 00000000 READY
    THREAD 8984b8b8  Cid 0250.1dc4  Teb: 7ff99000 Win32Thread: 00000000 READY
    THREAD 8912a4c0  Cid 0f4c.2410  Teb: 7ff81000 Win32Thread: 00000000 READY
    THREAD 89e5c570  Cid 0f4c.01c8  Teb: 00000000 Win32Thread: 00000000 READY
Processor 0: Ready Threads at priority 6
    THREAD 8a9353b0  Cid 1584.1598  Teb: 7ff8b000 Win32Thread: bc057698 READY
    THREAD 8aba2020  Cid 1584.15f0  Teb: 7ff9f000 Win32Thread: bc2a0ea8 READY
    THREAD 8aab17a0  Cid 1584.01a8  Teb: 7ff92000 Win32Thread: bc316ea8 READY
    THREAD 8a457020  Cid 1584.0634  Teb: 7ff8d000 Win32Thread: bc30fea8 READY
    THREAD 8a3d4020  Cid 1584.1510  Teb: 7ff8f000 Win32Thread: bc15b8a0 READY
    THREAD 8a5f5db0  Cid 1584.165c  Teb: 7ff9d000 Win32Thread: bc171be8 READY
    THREAD 8a297020  Cid 0f4c.0f54  Teb: 7ffde000 Win32Thread: bc20fda0 READY
    THREAD 8a126020  Cid 1584.175c  Teb: 7ffa9000 Win32Thread: 00000000 READY
    THREAD 8a548478  Cid 0250.07b0  Teb: 7ff9a000 Win32Thread: 00000000 READY
    THREAD 8a478020  Cid 0944.0988  Teb: 7ffd9000 Win32Thread: 00000000 READY
    THREAD 8986ad08  Cid 1d2c.1cf0  Teb: 7ffa8000 Win32Thread: bc285800 READY
    THREAD 897f4db0  Cid 1d2c.2554  Teb: 7ffdb000 Win32Thread: bc238e80 READY
    THREAD 89a2e618  Cid 1d2c.1de4  Teb: 7ffdd000 Win32Thread: bc203908 READY
Processor 0: Ready Threads at priority 0
    THREAD 8b184db0  Cid 0004.0008  Teb: 00000000 Win32Thread: 00000000 READY
Processor 1: Ready Threads at priority 8
    THREAD 89d89db0  Cid 1b10.20ac  Teb: 7ffd7000 Win32Thread: bc16e680 READY
    THREAD 891f24a8  Cid 1e2c.20d0  Teb: 7ffda000 Win32Thread: bc1b9ea8 READY
    THREAD 89214db0  Cid 1e2c.24d4  Teb: 7ffd7000 Win32Thread: bc24ed48 READY
    THREAD 89a28020  Cid 1b10.21b4  Teb: 7ffa7000 Win32Thread: bc25b3b8 READY
    THREAD 891e03b0  Cid 1a98.05c4  Teb: 7ffdb000 Win32Thread: bc228bb0 READY
    THREAD 891b0020  Cid 1cd0.0144  Teb: 7ffde000 Win32Thread: bc205ea8 READY
Processor 1: Ready Threads at priority 7
    THREAD 898367a0  Cid 0f4c.1cd4  Teb: 00000000 Win32Thread: 00000000 READY
    THREAD 8a1ac020  Cid 0f4c.1450  Teb: 00000000 Win32Thread: 00000000 READY
    THREAD 8aa1ab90  Cid 0f4c.11b0  Teb: 00000000 Win32Thread: 00000000 READY
    THREAD 89cc92e0  Cid 0f4c.1b34  Teb: 00000000 Win32Thread: 00000000 READY
    THREAD 89579020  Cid 0f4c.2220  Teb: 00000000 Win32Thread: 00000000 READY
Processor 1: Ready Threads at priority 6
    THREAD 8a487db0  Cid 1584.14bc  Teb: 7ffa2000 Win32Thread: bc304ea8 READY
    THREAD 8a3ce020  Cid 1584.0630  Teb: 7ff8e000 Win32Thread: bc293c20 READY
    THREAD 8a1b6db0  Cid 1584.1590  Teb: 7ff8c000 Win32Thread: bc310ea8 READY
    THREAD 8a1fe6e0  Cid 1584.15ec  Teb: 7ffa1000 Win32Thread: bc15bea8 READY
    THREAD 8ac0adb0  Cid 1584.156c  Teb: 7ff8a000 Win32Thread: bc153be8 READY
    THREAD 8b1e35a0  Cid 1584.15f4  Teb: 7ff9e000 Win32Thread: bc0567e8 READY
    THREAD 8a3288e8  Cid 1584.14b8  Teb: 7ff9a000 Win32Thread: bc2fbea8 READY
    THREAD 8a5056a0  Cid 1584.1518  Teb: 7ff91000 Win32Thread: bc337ea8 READY
    THREAD 891afdb0  Cid 1d2c.27e8  Teb: 7ffaf000 Win32Thread: bc217c18 READY
    THREAD 8a07d308  Cid 1d2c.2548  Teb: 7ffae000 Win32Thread: bc235750 READY
    THREAD 8a055d18  Cid 1584.17d0  Teb: 7ffd5000 Win32Thread: 00000000 READY
    THREAD 8ac0b770  Cid 0250.0268  Teb: 7ffde000 Win32Thread: bc2349d8 READY
    THREAD 8a0eeb40  Cid 1584.1560  Teb: 7ffdc000 Win32Thread: 00000000 READY

Here we should also analyze stack traces for running and ready threads with priority 8 and check kernel and user times. If we find anything common between them we should also check ready threads with lower priority to see if that commonality is unique to threads with priority 8. See also the similar pattern: Busy System and the similar starvation pattern resulted from realtime priority threads.

- Dmitry Vostokov @ DumpAnalysis.org + TraceAnalysis.org -

This entry was posted on Thursday, July 29th, 2010 at 4:48 pm and is filed under Crash Dump Analysis, Crash Dump Patterns, Debugging. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.