Comments on: Crash Dump Analysis Patterns (Part 14) https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/ Structural and Behavioral Patterns for Software Diagnostics, Forensics and Prognostics Wed, 06 May 2026 16:21:49 +0000 http://wordpress.org/?v=2.3.3 By: Dmitry Vostokov https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-741768 Dmitry Vostokov Sun, 28 Jun 2020 15:32:31 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-741768 !ready command is also useful: <p align="left">0: kd> !ready KSHARED_READY_QUEUE fffff8000da1a700: (00) **-------------------------------------------------------------- SharedReadyQueue fffff8000da1a700: Ready Threads at priority 8 THREAD ffff930ac224b080 Cid 0c14.0418 Teb: 000000ade0765000 Win32Thread: ffff930ac254b7a0 READY on processor 80000001 THREAD ffff930ac5dc3080 Cid 0c58.1af8 Teb: 00000056ece5b000 Win32Thread: 0000000000000000 READY on processor 80000000 THREAD ffff930abd933040 Cid 0004.00ec Teb: 0000000000000000 Win32Thread: 0000000000000000 READY on processor 80000001 THREAD ffff930ac1de4080 Cid 0c14.0d3c Teb: 000000ade0755000 Win32Thread: 0000000000000000 READY on processor 80000000 SharedReadyQueue fffff8000da1a700: Ready Threads at priority 5 THREAD ffff930ac1e36040 Cid 0bf8.0d68 Teb: 000000ce7c002000 Win32Thread: ffff930ac21f2bf0 READY on processor 80000000 SharedReadyQueue fffff8000da1a700: Ready Threads at priority 4 THREAD ffff930ac1b40080 Cid 081c.0ba4 Teb: 0000000000000000 Win32Thread: 0000000000000000 READY on processor 80000000 SharedReadyQueue fffff8000da1a700: Ready Threads at priority 0 THREAD ffff930abcfac080 Cid 0004.0038 Teb: 0000000000000000 Win32Thread: 0000000000000000 READY on processor 80000001 Processor 0: No threads in READY state Processor 1: Group Scheduling Queue <p align="left"> Scb: ffff930ac28a0778 Rank: 2 OQHistory: 0000000000015555 GenCycles: 706070 LTCycles: 21604816, MinTarget: 633600000, MaxTarget: 1267200000, RankTarget: 63360000 nt!_KSCB ffff930ac28a0778: Ready Threads at priority 9 THREAD ffff930ac62c70c0 Cid 158d0.14a8 Teb: 000000f09395c000 Win32Thread: 0000000000000000 READY on processor 1 !ready command is also useful:

0: kd> !ready
KSHARED_READY_QUEUE fffff8000da1a700: (00) **————————————————————–
SharedReadyQueue fffff8000da1a700: Ready Threads at priority 8
THREAD ffff930ac224b080 Cid 0c14.0418 Teb: 000000ade0765000 Win32Thread: ffff930ac254b7a0 READY on processor 80000001
THREAD ffff930ac5dc3080 Cid 0c58.1af8 Teb: 00000056ece5b000 Win32Thread: 0000000000000000 READY on processor 80000000
THREAD ffff930abd933040 Cid 0004.00ec Teb: 0000000000000000 Win32Thread: 0000000000000000 READY on processor 80000001
THREAD ffff930ac1de4080 Cid 0c14.0d3c Teb: 000000ade0755000 Win32Thread: 0000000000000000 READY on processor 80000000
SharedReadyQueue fffff8000da1a700: Ready Threads at priority 5
THREAD ffff930ac1e36040 Cid 0bf8.0d68 Teb: 000000ce7c002000 Win32Thread: ffff930ac21f2bf0 READY on processor 80000000
SharedReadyQueue fffff8000da1a700: Ready Threads at priority 4
THREAD ffff930ac1b40080 Cid 081c.0ba4 Teb: 0000000000000000 Win32Thread: 0000000000000000 READY on processor 80000000
SharedReadyQueue fffff8000da1a700: Ready Threads at priority 0
THREAD ffff930abcfac080 Cid 0004.0038 Teb: 0000000000000000 Win32Thread: 0000000000000000 READY on processor 80000001
Processor 0: No threads in READY state
Processor 1: Group Scheduling Queue

Scb: ffff930ac28a0778 Rank: 2 OQHistory: 0000000000015555
GenCycles: 706070 LTCycles: 21604816, MinTarget: 633600000, MaxTarget: 1267200000, RankTarget: 63360000
nt!_KSCB ffff930ac28a0778: Ready Threads at priority 9
THREAD ffff930ac62c70c0 Cid 158d0.14a8 Teb: 000000f09395c000 Win32Thread: 0000000000000000 READY on processor 1

]]> By: Dmitry Vostokov https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-741767 Dmitry Vostokov Sun, 28 Jun 2020 15:31:53 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-741767 Some very small value of ticks like Ticks: 1 is also a sign. Some very small value of ticks like Ticks: 1 is also a sign.

]]> By: Dmitry Vostokov https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-741638 Dmitry Vostokov Thu, 19 Sep 2013 14:54:25 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-741638 In kernel dumps we can see all running thread stacks with this command: !running -t For complete memory dumps user portion would be invalid if thread process contexts are different from the current process on the current CPU so manual thread inspection via !thread <addr> 3f is recommended. In kernel dumps we can see all running thread stacks with this command:

!running -t

For complete memory dumps user portion would be invalid if thread process contexts are different from the current process on the current CPU so manual thread inspection via !thread 3f is recommended.

]]> By: Dmitry Vostokov https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-384934 Dmitry Vostokov Sat, 03 Dec 2011 21:19:09 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-384934 How to use WinDbg scripts to get spiking threads from kernel and complete memory dumps: http://www.dumpanalysis.org/blog/index.php/2011/12/03/2-windbg-scripts-that-changed-the-world/ How to use WinDbg scripts to get spiking threads from kernel and complete memory dumps:
http://www.dumpanalysis.org/blog/index.php/2011/12/03/2-windbg-scripts-that-changed-the-world/

]]> By: Dmitry Vostokov https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-333006 Dmitry Vostokov Thu, 25 Aug 2011 10:23:10 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-333006 It is also useful to combine it with <a href="http://www.dumpanalysis.org/blog/index.php/2011/06/23/crash-dump-analysis-patterns-part-141/" rel="nofollow">Thread Age</a> pattern to see the magnitude of spike: Process Uptime: 0 days 17:35:06.000 If we look at thread CPU consumption we would see spike values relatively small compared to the overall process uptime: 0:000> !runaway f User Mode Time Thread Time 4:1560 0 days 0:00:08.034 7:7e0 0 days 0:00:03.338 11:1244 0 days 0:00:03.213 8:1510 0 days 0:00:02.324 3:86c 0 days 0:00:00.015 [...] However, compared to thread ages we see them much bigger: Elapsed Time Thread Time [...] 3:86c 0 days 17:35:05.213 4:1560 0 days 0:02:12.350 [...] 7:7e0 0 days 0:00:22.429 11:1244 0 days 0:00:17.655 It is also useful to combine it with Thread Age pattern to see the magnitude of spike:

Process Uptime: 0 days 17:35:06.000

If we look at thread CPU consumption we would see spike values relatively small compared to the overall process uptime:

0:000> !runaway f
User Mode Time
Thread Time
4:1560 0 days 0:00:08.034
7:7e0 0 days 0:00:03.338
11:1244 0 days 0:00:03.213
8:1510 0 days 0:00:02.324
3:86c 0 days 0:00:00.015
[…]

However, compared to thread ages we see them much bigger:

Elapsed Time
Thread Time
[…]
3:86c 0 days 17:35:05.213
4:1560 0 days 0:02:12.350
[…]
7:7e0 0 days 0:00:22.429
11:1244 0 days 0:00:17.655

]]> By: Dmitry Vostokov https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-242543 Dmitry Vostokov Mon, 14 Feb 2011 14:16:41 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-242543 Is some legacy dumps with missing !runaway information we can guess spiking threads if they do some sort of a peek message loop: <p align="left">0:012> !runaway ERROR: !runaway: extension exception 0x80004002. "Unable to get thread times - dumps may not have time information" <p align="left"> 6 Id: 136c.13bc Suspend: 1 Teb: 7ffd8000 Unfrozen ChildEBP RetAddr Args to Child 0198f914 5b2d887c 0198f938 00000000 00000000 USER32!PeekMessageA+0xfb 0198fa20 5b2d3b87 059d6704 00000001 00000000 ModuleA!Choose+0x177 [...] <p align="left">0:012> ~6s eax=000abbdd ebx=00000400 ecx=00000200 edx=00000abb esi=7ffd8000 edi=0066b3e0 eip=7e42a3e5 esp=0198f908 ebp=0198f914 iopl=0 nv up ei pl nz na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00040206 USER32!PeekMessageA+0xfb: 7e42a3e5 2b470c sub eax,dword ptr [edi+0Ch] ds:0023:0066b3ec=000abbdd Is some legacy dumps with missing !runaway information we can guess spiking threads if they do some sort of a peek message loop:

0:012> !runaway
ERROR: !runaway: extension exception 0×80004002.
“Unable to get thread times - dumps may not have time information”

6 Id: 136c.13bc Suspend: 1 Teb: 7ffd8000 Unfrozen
ChildEBP RetAddr Args to Child
0198f914 5b2d887c 0198f938 00000000 00000000 USER32!PeekMessageA+0xfb
0198fa20 5b2d3b87 059d6704 00000001 00000000 ModuleA!Choose+0×177
[…]

0:012> ~6s
eax=000abbdd ebx=00000400 ecx=00000200 edx=00000abb esi=7ffd8000 edi=0066b3e0
eip=7e42a3e5 esp=0198f908 ebp=0198f914 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00040206
USER32!PeekMessageA+0xfb:
7e42a3e5 2b470c sub eax,dword ptr [edi+0Ch] ds:0023:0066b3ec=000abbdd

]]> By: Crash Dump Analysis » Blog Archive » Case Study: Extremely Inconsitent Dump and CPU Spike https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-185920 Crash Dump Analysis » Blog Archive » Case Study: Extremely Inconsitent Dump and CPU Spike Wed, 22 Sep 2010 21:49:37 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-185920 [...] The both “running” threads were showing signs of a spiking thread: [...] […] The both “running” threads were showing signs of a spiking thread: […]

]]> By: Crash Dump Analysis » Blog Archive » Crash Dump Analysis Patterns (Part 106) https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-185199 Crash Dump Analysis » Blog Archive » Crash Dump Analysis Patterns (Part 106) Sun, 19 Sep 2010 21:42:29 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-185199 [...] level when we detect it using Task Manager, for example. Sometimes that process has only one spiking thread among many but there are cases when CPU consumption is spread among many threads. I call this [...] […] level when we detect it using Task Manager, for example. Sometimes that process has only one spiking thread among many but there are cases when CPU consumption is spread among many threads. I call this […]

]]> By: Crash Dump Analysis » Blog Archive » Insufficient kernel pool memory, spiking thread and data contents locality: pattern cooperation https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-184992 Crash Dump Analysis » Blog Archive » Insufficient kernel pool memory, spiking thread and data contents locality: pattern cooperation Sat, 18 Sep 2010 23:30:40 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-184992 [...] We also check CPU consumption and see two spiking threads: [...] […] We also check CPU consumption and see two spiking threads: […]

]]> By: Crash Dump Analysis » Blog Archive » Succession of Patterns (Part 2) https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-155820 Crash Dump Analysis » Blog Archive » Succession of Patterns (Part 2) Thu, 03 Jun 2010 23:20:54 +0000 https://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/#comment-155820 [...] where Wait Chains (executive resources) and Swarm of Shared Locks were probably resulted from a Spiking Thread. We have these resource [...] […] where Wait Chains (executive resources) and Swarm of Shared Locks were probably resulted from a Spiking Thread. We have these resource […]

]]>