I noticed that Visual C++ 2008 Express Edition sometimes starts spiking after building the project and accumulates significant amount of CPU time whent in the background. On my 2 processor machine this additional CPU spike while running Windows Media Player results in sound interruptions so I took this matter seriously and dumped VCExpress.exe process:
0:000:x86> !runaway ff
User Mode Time
Thread Time
0:b78 0 days 2:51:58.264
11:f1c 0 days 0:00:02.652
2:eb0 0 days 0:00:00.031
13:ed8 0 days 0:00:00.015
8:eac 0 days 0:00:00.015
16:8c0 0 days 0:00:00.000
15:fb8 0 days 0:00:00.000
14:e04 0 days 0:00:00.000
12:8c4 0 days 0:00:00.000
10:c1c 0 days 0:00:00.000
9:854 0 days 0:00:00.000
7:d4c 0 days 0:00:00.000
6:ce4 0 days 0:00:00.000
5:edc 0 days 0:00:00.000
4:ac0 0 days 0:00:00.000
3:a44 0 days 0:00:00.000
1:6dc 0 days 0:00:00.000
Kernel Mode Time
Thread Time
0:b78 0 days 0:00:44.039
11:f1c 0 days 0:00:00.358
2:eb0 0 days 0:00:00.062
8:eac 0 days 0:00:00.031
16:8c0 0 days 0:00:00.000
15:fb8 0 days 0:00:00.000
14:e04 0 days 0:00:00.000
13:ed8 0 days 0:00:00.000
12:8c4 0 days 0:00:00.000
10:c1c 0 days 0:00:00.000
9:854 0 days 0:00:00.000
7:d4c 0 days 0:00:00.000
6:ce4 0 days 0:00:00.000
5:edc 0 days 0:00:00.000
4:ac0 0 days 0:00:00.000
3:a44 0 days 0:00:00.000
1:6dc 0 days 0:00:00.000
Elapsed Time
Thread Time
0:b78 0 days 23:42:40.899
1:6dc 0 days 23:42:33.955
4:ac0 0 days 23:42:33.941
3:a44 0 days 23:42:33.941
2:eb0 0 days 23:42:33.941
5:edc 0 days 23:42:33.629
6:ce4 0 days 23:42:33.460
7:d4c 0 days 23:42:33.213
8:eac 0 days 23:42:31.676
9:854 0 days 23:41:18.544
10:c1c 0 days 23:41:18.539
11:f1c 0 days 23:40:25.753
13:ed8 0 days 23:40:14.118
12:8c4 0 days 23:40:14.118
14:e04 0 days 23:40:13.104
15:fb8 0 days 23:40:00.417
16:8c0 0 days 0:03:06.893
The stack was not interesting and looked the same as if we just launched the process without any spikes:
0:000:x86> kL
ChildEBP RetAddr
0040f8fc 7651e39b ntdll_77a40000!ZwWaitForMultipleObjects+0x15
0040f998 76628f86 kernel32!WaitForMultipleObjectsEx+0x11d
0040f9ec 6cf38523 user32!RealMsgWaitForMultipleObjectsEx+0x14d
0040fa30 6cfdb9bd msenv!EnvironmentMsgLoop+0x1ea
0040fa5c 6cfdb94d msenv!CMsoCMHandler::FPushMessageLoop+0x86
0040fa84 6cfdb8e9 msenv!SCM::FPushMessageLoop+0xb7
0040faa0 6cfdb8b8 msenv!SCM_MsoCompMgr::FPushMessageLoop+0x28
0040fac0 6cfdbe4e msenv!CMsoComponent::PushMsgLoop+0x28
0040fb58 6cfd7561 msenv!VStudioMainLogged+0x482
0040fb84 0124ee6a msenv!VStudioMain+0xc1
0040fbb0 01248f5e VCExpress!util_CallVsMain+0xff
0040fe8c 0124a3aa VCExpress!CVCExpressId::Run+0x6f6
0040fea8 01254e11 VCExpress!WinMain+0x74
0040ff38 7658eccb VCExpress!operator new[]+0x1a0
0040ff44 77abd24d kernel32!BaseThreadInitThunk+0xe
0040ff84 77abd45f ntdll_77a40000!__RtlUserThreadStart+0x23
0040ff9c 00000000 ntdll_77a40000!_RtlUserThreadStart+0x1b
I looked at the raw stack of the main thread to see if there are any traces (execution residue) of any foreign modules but it was densely populated by past execution histories:
0:000:x86> !wow64exts.info
PEB32: 0x7efde000
PEB64: 0x7efdf000
Wow64 information for current thread:
TEB32: 0x7efdd000
TEB64: 0x7efdb000
32 bit, StackBase : 0×410000
StackLimit : 0×3f4000
Deallocation: 0×310000
64 bit, StackBase : 0x24fd20
StackLimit : 0x245000
Deallocation: 0x210000
[...]
0:000:x86> dds 0×3f4000 0×410000
[…]

In order to see the execution residue of what was left between two WaitForMultipleObjectsEx calls I ran another instance of VCExpress.exe and reproduced the problem. Then I attached WinDbg to the process. The raw stack beyond the current stack trace portion looked like this (highlighted in blue):
0:000:x86> !wow64exts.info
PEB32: 0x7efde000
PEB64: 0x7efdf000
Wow64 information for current thread:
TEB32: 0x7efdd000
TEB64: 0x7efdb000
32 bit, StackBase : 0x140000
StackLimit : 0×12c000
Deallocation: 0×40000
64 bit, StackBase : 0x25fd20
StackLimit : 0x256000
Deallocation: 0x220000
[...]
0:000:x86> dds 0×12c000 0×140000
[…]
0013f810 0013f81c
0013f814 6d820e81 msenv!CEditView::CharIndexFromViewCol+0xd
0013f818 00000024
0013f81c 0013f844
0013f820 6d84bbca msenv!CEditView::GetSelectionSpan+0×92
0013f824 00000024
0013f828 0013f86c
0013f82c 6d84bafe msenv!CEditView::LocalSpanToBase+0×56
0013f830 00bb65a8
0013f834 00000024
0013f838 00000000
0013f83c 0013f8d4
0013f840 00000001
0013f844 0013f948
0013f848 00000001
0013f84c 6d84bb3f msenv!CEditView::CViewInterfaceWrapper::GetSelectionSpan+0×41
0013f850 00000000
0013f854 6d84b80f msenv!ATL::CComObject<CEditView>::Release+0×17
0013f858 00bb9488
0013f85c 00000000
0013f860 00000001
0013f864 0013f870
0013f868 6d84b871 msenv!CEditView::CViewInterfaceWrapper::Release+0×1c
0013f86c 00bb93e0
0013f870 0013f884
0013f874 6d91c8fc msenv!COutputWindow::GetSelectionSpan+0×3f
0013f878 00bb9b0c
0013f87c 079f565c
0013f880 00000000
0013f884 77a899fd ntdll_77a40000!ZwWaitForMultipleObjects+0×15
0013f888 7651e39b KERNEL32!WaitForMultipleObjectsEx+0×11d
0013f88c 00000001
0013f890 0013f8d4
0013f894 00000001
0013f898 00000001
0013f89c 00000000
0013f8a0 31c3f2b0
0013f8a4 00000000
0013f8a8 00000001
0013f8ac 0013f948
0013f8b0 00000024
0013f8b4 00000001
0013f8b8 00000000
0013f8bc 00000000
0013f8c0 00000030
0013f8c4 ffffffff
0013f8c8 ffffffff
0013f8cc 765315ef KERNEL32!WaitForMultipleObjectsEx+0×33
0013f8d0 00000000
0013f8d4 00000050
0013f8d8 00000000
0013f8dc 0013f914
0013f8e0 6d8386ed msenv!CMsoCMHandler::FContinueIdle+0×25
0013f8e4 0013f8f8
0013f8e8 00000000
0013f8ec 00000000
[…]
Then I cleared the portion of the raw stack from the limit to 0013f880 address:
0:000:x86> kL
ChildEBP RetAddr
0013f884 7651e39b ntdll_77a40000!ZwWaitForMultipleObjects+0×15
0013f998 76628f86 kernel32!WaitForMultipleObjectsEx+0×11d
0013f9ec 6cf38523 user32!RealMsgWaitForMultipleObjectsEx+0×14d
0013fa30 6cfdb9bd msenv!EnvironmentMsgLoop+0×1ea
0013fa5c 6cfdb94d msenv!CMsoCMHandler::FPushMessageLoop+0×86
0013fa84 6cfdb8e9 msenv!SCM::FPushMessageLoop+0xb7
0013faa0 6cfdb8b8 msenv!SCM_MsoCompMgr::FPushMessageLoop+0×28
0013fac0 6cfdbe4e msenv!CMsoComponent::PushMsgLoop+0×28
0013fb58 6cfd7561 msenv!VStudioMainLogged+0×482
0013fb84 0124ee6a msenv!VStudioMain+0xc1
0013fbb0 01248f5e VCExpress!util_CallVsMain+0xff
0013fe8c 0124a3aa VCExpress!CVCExpressId::Run+0×6f6
0013fea8 01254e11 VCExpress!WinMain+0×74
0013ff38 7658eccb VCExpress!operator new[]+0×1a0
0013ff44 77abd24d kernel32!BaseThreadInitThunk+0xe
0013ff84 77abd45f ntdll_77a40000!__RtlUserThreadStart+0×23
0013ff9c 00000000 ntdll_77a40000!_RtlUserThreadStart+0×1b
0:000:x86> f 0012c000 0013f884-4 0
Filled 0×13881 bytes
0:000:x86> dds 0×12c000 0×140000
[…]
0013f810 00000000
0013f814 00000000
0013f818 00000000
0013f81c 00000000
0013f820 00000000
0013f824 00000000
0013f828 00000000
0013f82c 00000000
0013f830 00000000
0013f834 00000000
0013f838 00000000
0013f83c 00000000
0013f840 00000000
0013f844 00000000
0013f848 00000000
0013f84c 00000000
0013f850 00000000
0013f854 00000000
0013f858 00000000
0013f85c 00000000
0013f860 00000000
0013f864 00000000
0013f868 00000000
0013f86c 00000000
0013f870 00000000
0013f874 00000000
0013f878 00000000
0013f87c 00000000
0013f880 00000000
0013f884 77a899fd ntdll_77a40000!ZwWaitForMultipleObjects+0×15
0013f888 7651e39b KERNEL32!WaitForMultipleObjectsEx+0×11d
0013f88c 00000001
0013f890 0013f8d4
0013f894 00000001
0013f898 00000001
0013f89c 00000000
0013f8a0 31c3f2b0
0013f8a4 00000000
0013f8a8 00000001
0013f8ac 0013f948
0013f8b0 00000024
0013f8b4 00000001
0013f8b8 00000000
0013f8bc 00000000
0013f8c0 00000030
0013f8c4 ffffffff
0013f8c8 ffffffff
0013f8cc 765315ef KERNEL32!WaitForMultipleObjectsEx+0×33
0013f8d0 00000000
0013f8d4 00000050
0013f8d8 00000000
0013f8dc 0013f914
0013f8e0 6d8386ed msenv!CMsoCMHandler::FContinueIdle+0×25
0013f8e4 0013f8f8
0013f8e8 00000000
0013f8ec 00000000
[…]
and put a breakpoint on the return address of msenv!EnvironmentMsgLoop (6cf38523):
0:000:x86> bp 6cf38523
0:000:x86> g
Breakpoint 0 hit
msenv!EnvironmentMsgLoop+0x1ea:
6cf38523 e92b90feff jmp msenv!EnvironmentMsgLoop+0x1ea (6cf38553)
Then I was able to see the execution residue for just one iteration of the message loop.
0:000:x86> dds 0x12c000 0x140000
[...]
- Dmitry Vostokov @ DumpAnalysis.org -