Skip to content
Sasha Goldshtein edited this page Jan 12, 2017 · 27 revisions

About msos

msos was born out of a need to debug .NET process dumps without the SOS.dll extension available. It quickly turned into more than that with a variety of commands and options that, in many cases, exceed what SOS.dll has to offer.

You should use msos when you don't have access to SOS.dll (e.g., when debugging Windows Phone dumps), when you need some of msos's advanced functionality, or simply when you don't want to deal with the overhead of the full-blown WinDbg debugger. At this time, msos offers commands (such as !hq) that are extremely difficult to implement using the SOS.dll extension. Other interesting commands include !waits (with deadlock detection), !clrstack -a with much better parameters and locals display that SOS, a mixed managed-native call stack with !mk, heap indexing for better performance, and even a mode that supports running arbitrary WinDbg commands on the same target.

This page serves as a reference; instead, you might prefer a gentle introduction in the form of a tutorial.

Getting Started

msos is a standalone command-line debugger. It can attach to a live process (in non-invasive mode) or open a dump file. You will need the x86 version of msos to debug x86 targets, and the x64 version to debug x64 targets. This is a limitation of the .NET debugging API.

msos is designed to be self-documenting. For example, try running msos --help from the command line to see the first-level command-line options. When inside msos, type help to get a list of commands or help cmdname to get information about a specific command's switches.

The command-line switches are case-sensitive, but the command verbs are not. For example, you can use !clrstack, !CLRStack, or !CLRSTACK with the same effect.

To open a dump file, use the -z switch:

msos -z myapp.dmp

To attach to a running process, use either the --pn or the --pid switches. If there is more than one process with the given name, you will be asked to specify the process id.

msos --pid 1008
msos --pn devenv

Note: if there are multiple versions of the CLR loaded into the target process, you can select the desired one using the --clrVersion switch.

Automation

You can run msos and ask it to execute a set of initial commands, separated by semicolons. This is useful for automatic analysis, or when you always begin a debugging session by displaying basic information. For example, if you always begin a session by looking at the list of modules followed by the current thread's stack, use this command line:

msos -z myapp.dmp -c "!lm; !clrstack"

You can include the q command so that the debugger exits without waiting for further input. This is especially useful when analyzing multiple dump files in a row, automatically, as part of a script.

You can also load initial commands from a file. Create a file with each command on a separate line, and pass that file to the -i switch:

msos -z myapp.dmp -i commands.txt

Finally, you can pipe msos's output to a file (instead of the console) by using the -o switch:

msos -z myapp.dmp -i commands.txt -o output.txt

Basic Crash Analysis

When you give msos a crash dump, it will automatically try to switch to the crashing thread, if there is one. The !pe (or !PrintException) command can be used to display the current exception. If you have the address of a .NET exception object, pass it to !pe as a parameter.

Inspecting Threads and Stacks

To display a list of all managed threads in the target, run !Threads. When available, additional information is displayed in the Special and Exception columns. The Lock# column indicates the number of locks the thread currently owns.

1> !Threads
2 CLR threads, 0 CLR thread pool threads, 1 background threads
MgdId  OSId   Lock#  Apt    Special              Exception
1      5980   1      MTA
2      8952   0      MTA    Finalizer

To see unmanaged (native) threads as well, use the !threads --native command. Thread entry points (start addresses) will also be displayed:

3> !threads --native
4 CLR threads, 2 CLR thread pool threads, 3 background threads
MgdId  OSId   Lock#  Apt    Special              Exception
1      2504   0      STA
2      872    0      MTA    Finalizer
3      3124   0      MTA                         ...stem.NullReferenceException
4      3600   0      MTA

OSId   MgdId  ExitCode   StartAddress
2504   1      active     00000000001a3e7e FileExplorer!COM+_Entry_Point <PERF> (FileExplorer+0x3e7e)
3368          active     00000000741e67ce clr!DebuggerRCThread::ThreadProcStatic
872    2      active     0000000074178104 clr!Thread::intermediateThreadProc
2792          active     000000007130c1f0 GdiPlus!DllRefCountSafeThreadThunk
3124   3      active     0000000074178104 clr!Thread::intermediateThreadProc
4076          active     0000000074105604 clr!ThreadpoolMgr::GateThreadStart
3600   4      active     0000000074178104 clr!Thread::intermediateThreadProc

When you open a dump file or attach to a live process, msos attempts to set the current thread to either the thread that encountered an exception (useful with crash dumps), or the first managed thread it can find. You can switch threads by using the ~ command and the managed thread id:

~ 7

To display the current thread's managed stack, run !CLRStack. If the -a switch is specified, this command displays method arguments and local variables for each frame. Due to optimizations, the values and names of some stack objects are not available. You can combine this output with !dso, which occasionally produces false positives.

1> !clrstack
SP                   IP                   Function
000000000020F180     0000000000000000     InlinedCallFrame
000000000020F17C     0000000073C8B7BF     DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)
000000000020F180     0000000000000000     InlinedCallFrame
000000000020F1E4     00000000743DCD64     System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Boolean, Boolean, Int32 ByRef)
000000000020F218     00000000743DCC6B     System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)
000000000020F238     0000000073C74EC8     System.IO.StreamReader.ReadBuffer()
000000000020F24C     0000000073BDE90F     System.IO.StreamReader.ReadLine()
000000000020F268     00000000743E2E5D     System.IO.TextReader+SyncTextReader.ReadLine()
000000000020F278     00000000742EA842     System.Console.ReadLine()
000000000020F280     00000000003D00C9     VSDebugging.Program.Main(System.String[])
000000000020F410     0000000000000000     GCFrame

Here is an example of a single frame with arguments and locals displayed. Types, addresses, names, sizes, and values are displayed when possible. Note that the local variable names could not be retrieved in this example.

0000000004F1F734     0000000000520A6A     FileExplorer.MainForm+<>c__DisplayClass1.<treeView1_AfterSelect>b__0(System.Object) [c:\Temp\crash1\FileExplorer\MainForm.cs:45,9]
 arg this 0000000004f1f74c = 00000000062daa84 (FileExplorer.MainForm+<>c__DisplayClass1, size 4)
 arg _ 0000000004f1f75c = 0000000000000000 (System.Object, size 4)
 lcl  0000000004f1f748 = <null> (System.String, size 4)
 lcl  0000000004f1f744 = 0000000000000000 (System.Object[], size 4)
 lcl  0000000004f1f758 = 0 (System.Int32, size 4)
 lcl  0000000004f1f754 = False (System.Boolean, size 1)

Occasionally, you might be interested in the native (unmanaged) stack as well as the managed part. This is where the !mk command comes in -- it can show you the mixed stack for the current thread, or, if you give it a Windows thread ID, it will display the mixed stack for that thread. For example:

1> !mk
Type       IP                   Function
Native     0000000076c3bfbc     user32!NtUserWaitMessage+0xc
Managed    00000000689458f8     System.Windows.Forms!System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr, Int32, Int32)+0x444
Managed    0000000068945389     System.Windows.Forms!System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext)+0x155
Managed    0000000068945202     System.Windows.Forms!System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext)+0x4a
Managed    00000000689215e1     System.Windows.Forms!System.Windows.Forms.Application.Run(System.Windows.Forms.Form)+0x31
Managed    0000000000520093     FileExplorer!FileExplorer.Program.Main() [c:\Temp\crash1\FileExplorer\MainForm.cs:22,9]
Special
Native     000000007409e670     clr!MethodDescCallSite::CallTargetWorker+0x152
Native     00000000741f2955     clr!RunMain+0x1aa
Native     00000000741f2891     clr!Assembly::ExecuteMainMethod+0x124
Native     00000000741aec8b     clr!SystemDomain::ExecuteMainMethod+0x63c
Native     00000000741aed2e     clr!ExecuteEXE+0x4c
Native     00000000741aee42     clr!_CorExeMainInternal+0xdc
Native     00000000741b2370     clr!_CorExeMain+0x4d
Native     000000007472ec94     mscoreei!_CorExeMain+0x10a
Native     00000000747abbcc     mscoree!_CorExeMain_Exported+0x8c
Native     0000000076af7c04     kernel32!BaseThreadInitThunk+0x24
Native     00000000770cad1f     ntdll!__RtlUserThreadStart+0x2f
Native     00000000770cacea     ntdll!_RtlUserThreadStart+0x1b

When there are many threads, going through each thread and displaying its call stack can be a very time-consuming process. Visual Studio has a Parallel Stacks window for this, and msos has !stacktree -- it displays a tree that aggregates all call stacks for all threads in the target, such that multiple threads that have the same call stack (or a part thereof) are displayed only once. For example:

> !stacktree
Stack tree for 50 threads:
+ System.Threading.ThreadHelper.ThreadStart() Threads: M22 [a0] M25 [a1] M21 [a2] M28 [a3] M29 [a4] M19 [a5] M32 [a6] 
  | System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
  | System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
  | System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
  | System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
  + Microsoft.VisualStudio.Shell.Connected.ConnectedUser.RegistryWatcher.MonitorRegistryKey() Threads: M25 [a7] M28 [a8] M29 [a9] 
    | System.Threading.WaitHandle.WaitAny(System.Threading.WaitHandle[], Int32, Boolean)
  + ..() Threads: M19 [a10] M32 [a11] 
    | System.Threading.Monitor.Wait(System.Object)
    | System.Threading.Monitor.Wait(System.Object, Int32, Boolean)
  + System.Management.MTAHelper.WorkerThread() Threads: M22 [a12] 
    | System.Threading.WaitHandle.WaitOne()
    | System.Threading.WaitHandle.WaitOne(Int32, Boolean)
    | System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle, Int64, Boolean, Boolean)
  + Microsoft.Build.BackEnd.NodeProviderInProc.InProcNodeThreadProc() Threads: M21 [a13] 
    | Microsoft.Build.BackEnd.InProcNode.Run(System.Exception ByRef)
    | System.Threading.WaitHandle.WaitAny(System.Threading.WaitHandle[], Int32, Boolean)
+ System.Threading.ThreadHelper.ThreadStart(System.Object) Threads: M6 [a14] M26 [a15] 
  | System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
  | System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
  | System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
  | System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
  + System.Threading.Tasks.ThreadPoolTaskScheduler.LongRunningThreadWork(System.Object) Threads: M6 [a16] 
    | System.Threading.Tasks.Task.ExecuteEntry(Boolean)
    | System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)
    | System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
    | System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
    | System.Threading.Tasks.Task.ExecutionContextCallback(System.Object)
    | System.Threading.Tasks.Task.Execute()
    | System.Threading.Tasks.Task.InnerInvoke()
    | Clide.Diagnostics.TracerManager.DoTrace()
    | System.Collections.Concurrent.BlockingCollection`1+<GetConsumingEnumerable>d__0[[System.__Canon, mscorlib]].MoveNext()
    | System.Collections.Concurrent.BlockingCollection`1[[System.__Canon, mscorlib]].TryTakeWithNoTimeValidation(System.__Canon ByRef, Int32, System.Threading.CancellationToken, System.Threading.CancellationTokenSource)
    | System.Threading.SemaphoreSlim.Wait(Int32, System.Threading.CancellationToken)
    | System.Threading.SemaphoreSlim.WaitUntilCountOrTimeout(Int32, UInt32, System.Threading.CancellationToken)
    | System.Threading.Monitor.Wait(System.Object, Int32, Boolean)
  + Microsoft.Internal.VisualStudio.PlatformUI.BackgroundDispatcher.ThreadProc(System.Object) Threads: M26 [a17] 
    | System.Windows.Threading.Dispatcher.Run()
    | System.Windows.Threading.Dispatcher.PushFrame(System.Windows.Threading.DispatcherFrame)
    | System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)
    | System.Windows.Threading.Dispatcher.GetMessage(System.Windows.Interop.MSG ByRef, IntPtr, Int32, Int32)
    | MS.Win32.UnsafeNativeMethods.GetMessageW(System.Windows.Interop.MSG ByRef, System.Runtime.InteropServices.HandleRef, Int32, Int32)
    | DomainNeutralILStubClass.IL_STUB_PInvoke(System.Windows.Interop.MSG ByRef, System.Runtime.InteropServices.HandleRef, Int32, Int32)
+ System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*) Threads: M10 [a18] 
  | Microsoft.ServiceBus.Common.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)
  | Microsoft.ServiceBus.Common.IOThreadScheduler+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
  | Microsoft.ServiceBus.Common.IOThreadTimer+TimerManager.OnWaitCallback(System.Object)
  | System.Threading.WaitHandle.WaitAny(System.Threading.WaitHandle[], Int32, Boolean)

!stacktree --native will construct a stack tree that contains both managed and native frames. It is also more time-consuming.

The !ThreadPool command displays general information about the CLR thread pool.

Synchronization

The !SyncBlk command displays all synchronization objects. These include Monitors (the C# lock statement acquires and releases a Monitor), Reader-Writer Locks, and other synchronization mechanisms. When available, ownership information is also displayed along with a list of threads waiting for the synchronization object. For example, in the following example, thread #1 owns the first monitor:

1> !syncblk
Address              Type       Locked   Owner(s)             Waiter(s)
0000000001fc3f34     Monitor    1        1
0000000001fc21f0     None       0

The !waits command runs an automatic analysis that detects deadlocks and displays thread wait chains, which help to understand which thread is waiting for which object, and what dependencies between threads are formed through synchronization mechanisms.

1> !waits
+ Thread 1
  | Monitor 0000000002687410 System.String
    + Thread 3
      | Monitor 00000000026873e0 System.String
        + Thread 1
        *** DEADLOCK!
+ Thread 2
+ Thread 3
  | Monitor 00000000026873e0 System.String
    + Thread 1
      | Monitor 0000000002687410 System.String
        + Thread 3
        *** DEADLOCK!
+ Thread 4

Memory and Objects

Use the !dumpheap --stat command to get an initial reading of what's going on with your managed heap. It displays statistics for each type indicating the number of objects from that type and their total size. Generally, object sizes do not include the sizes of any children. For example, the size of an array of strings doesn't include the size of the strings themselves. The !ObjSize command displays the size of the entire object graph rooted at a specific object.

The !dumpheap command accepts multiple advanced options to filter the output and to display individual objects. The --type switch is most flexible, and takes a regular expression that filters the types of objects to be displayed. For example (omit the --stat switch to get a list of objects, and not just statistics):

1> !dumpheap --type String --stat
Statistics:
MT                   Count      TotalSize  Class Name
0000000073cf4cf8     1          12         System.Collections.Generic.GenericEqualityComparer<System.String>
00000000738ec734     1          48         System.Collections.Generic.Dictionary<System.String,System.Globalization.CultureData>
0000000073cf4b04     2          56         System.Text.StringBuilder
0000000073cf4e04     1          60         System.Collections.Generic.Dictionary+Entry<System.String,System.Globalization.CultureData>[]
0000000073cb5738     18         680        System.String[]
0000000073cf21b4     165        5236       System.String
Total 188 objects

Other heap filtering options include --min, --max and --mt.

To display an individual object, use the !do (or !DumpObj) command. It displays the object's fields (unless you pass the --nofields switch), including any static and thread-static fields. Nested value types are displayed unless you pass the --norecurse switch. Nested reference types are never displayed.

1> !do 0000000001fc3984
Name:     System.Globalization.NumberFormatInfo
MT:       0000000073cf4e7c
Size:     132(0x84) bytes
Assembly: C:\WINDOWS\Microsoft.Net\assembly\GAC_32\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.
dll
Value:    1942965884
Fields:
Offset   Type                 VT  Attr       Value                Name
0        System.Int32[]       0   instance   33303612             numberGroupSizes
8        System.Int32[]       0   instance   33303628             currencyGroupSizes
10       System.Int32[]       0   instance   33303612             percentGroupSizes
18       System.String        0   instance   +                    positiveSign
20       System.String        0   instance   -                    negativeSign
28       System.String        0   instance   .                    numberDecimalSeparator
30       System.String        0   instance   ,                    numberGroupSeparator
38       System.String        0   instance   ,                    currencyGroupSeparator
40       System.String        0   instance   .                    currencyDecimalSeparator
48       System.String        0   instance   ☼                    currencySymbol
50       System.String        0   instance   <null>               ansiCurrencySymbol
58       System.String        0   instance   NaN                  nanSymbol
60       System.String        0   instance   Infinity             positiveInfinitySymbol
68       System.String        0   instance   -Infinity            negativeInfinitySymbol
70       System.String        0   instance   .                    percentDecimalSeparator
78       System.String        0   instance   ,                    percentGroupSeparator
80       System.String        0   instance   %                    percentSymbol
88       System.String        0   instance   %                    perMilleSymbol
90       System.Object[]      0   instance   33303556             nativeDigits
98       System.Int32         1   instance   0                    m_dataItem
a0       System.Int32         1   instance   2                    numberDecimalDigits
a8       System.Int32         1   instance   2                    currencyDecimalDigits
b0       System.Int32         1   instance   0                    currencyPositivePattern
b8       System.Int32         1   instance   0                    currencyNegativePattern
c0       System.Int32         1   instance   1                    numberNegativePattern
c8       System.Int32         1   instance   0                    percentPositivePattern
d0       System.Int32         1   instance   0                    percentNegativePattern
d8       System.Int32         1   instance   2                    percentDecimalDigits
e0       System.Int32         1   instance   1                    digitSubstitution
e8       System.Boolean       1   instance   True                 isReadOnly
ea       System.Boolean       1   instance   False                m_useUserOverride
ec       System.Boolean       1   instance   True                 m_isInvariant
ee       System.Boolean       1   instance   True                 validForParseAsNumber
f0       System.Boolean       1   instance   True                 validForParseAsCurrency
528      ...NumberFormatInfo  0   shared     static               invariantInfo
   >> Domain:Value  0000000000604ac8:<null> <<

To display arrays, use the !DumpArray command. It supports only simple arrays -- if you are working with other collections, use the !DumpCollection command, described below.

1> !dumparray 00000000024a2374
Name:  SampleConsoleApp.Outer[]
Size:  36(0x24) bytes
Array: Number of elements 2, Type SampleConsoleApp.Outer (value type)
[0] 00000000024a237c
[1] 00000000024a2388

To display value type instances, you need to instruct the !do command what the type of the object is. For example, if you just displayed an array of value types, the values you'll get from !DumpArray are object addresses. But you can view the individual objects using !do:

1> !do 00000000024a237c --type SampleConsoleApp.Outer
Name:     SampleConsoleApp.Outer
Size:     20(0x14) bytes
Assembly: C:\temp\SampleConsoleApp.exe
Fields:
Offset   Type                 VT  Attr       Value                Name
0        System.String        0   instance   Hello, World         Str
8        ...ConsoleApp.Inner  1   instance   00000000024a2380     Inn
4        System.Int32         1   instance   5                    Inn.X
c        System.Int32         1   instance   3                    Inn.Y

To display collections, use the !DumpCollection command. It supports arrays, lists, and dictionaries. Other collections can be displayed using the !hq command (discussed below).

1> !dumpcollection 0000000001fc1bb0
Type:   System.Collections.Generic.Dictionary<System.Type,System.Security.Policy.EvidenceTypeDescriptor>
Size:   10
Key                                      Value
[0000000001fc1d3c System.RuntimeType]    <null>
[0000000001fc1d58 System.RuntimeType]    <null>
[0000000001fc1d74 System.RuntimeType]    <null>
[0000000001fc1d90 System.RuntimeType]    <null>
[0000000001fc1dac System.RuntimeType]    <null>
[0000000001fc1dc8 System.RuntimeType]    <null>
[0000000001fc1de4 System.RuntimeType]    <null>
[0000000001fc1e00 System.RuntimeType]    <null>
[0000000001fc1e1c System.RuntimeType]    <null>
[0000000001fc1e38 System.RuntimeType]    <null>
Time: 3 ms, Memory start: 568.008kb, Memory end: 568.008kb, Memory delta: -0b

Use the !ObjSize command to display the size of the object graph rooted at a specific object. This includes all referenced objects.

1> !objsize 0000000001fc3984
0000000001fc3984 graph size is 24 objects, 574 bytes

Memory Leaks

The !memstats command provides basic information about the target's memory usage, virtual address spaces, CLR heaps, and external and internal fragmentation. There are multiple switches that configure which statistics to display:

  • --heap: The sizes of each heap and each generation (including the Large Object Heap) are displayed. Non-GC regions (such as the loader heap) are also shown.

  • --heapfrag: Fragmentation information for each heap segment is displayed.

  • --vm: A detailed address space report with each region marked as free/reserved/committed and accounted by usage (e.g. image, mapped file, private) is shown.

  • --vmstat: A summary of virtual memory usage and the size of the largest free block is displayed. Note that if the size of the largest free block is smaller than the GC segment size, you could be getting an OutOfMemoryException due to virtual address space fragmentation.

1> !memstats --vmstat
Virtual memory statistics:

PRIVATE    325.246mb
MAPPED     95.074mb
IMAGE      458.547mb

FREE       3.142gb
COMMIT     748.875mb
RESERVE    129.992mb

Largest free region size: 1.976gb

1> !memstats --heapfrag
Fragmentation statistics:
#    Base                 Size         Committed    Reserved     Fragmented   % Frag     SOH/LOH
0    0000000003df1000     15.995mb     15.996mb     15.996mb     1.624mb      10.15%     SOH
1    0000000004df1000     11.209mb     11.336mb     15.996mb     6.247mb      55.74%     LOH
2    000000000d641000     15.989mb     15.996mb     15.996mb     506.125kb    3.09%      SOH
3    00000000125b1000     13.683mb     13.809mb     15.996mb     1.693mb      12.37%     SOH
4    0000000018881000     14.520mb     14.648mb     15.996mb     12.901mb     88.85%     LOH
5    000000001a9a1000     15.400mb     15.527mb     15.996mb     10.592mb     68.78%     LOH
6    000000001b9a1000     10.803mb     10.930mb     15.996mb     3.915mb      36.24%     SOH
7    000000001c9a1000     11.981mb     11.984mb     15.996mb     10.121mb     84.47%     LOH
8    00000000217a1000     14.994mb     15.004mb     15.996mb     102.721kb    0.67%      SOH
9    00000000227a1000     12.000mb     12.066mb     15.996mb     22.650kb     0.18%      SOH
10   00000000237a1000     9.494mb      12.668mb     15.996mb     1.520mb      16.02%     SOH

Total size of free objects: 49.231mb

When chasing memory leaks, the hardest part is understanding why objects are being retained and not collected by the GC. If you need to inspect individual object references and figure out why they are not being collected, it is recommended that you build a heap index. A heap index is an internal msos data structure that collects information on object references, and makes certain operations much faster. The downside is that constructing the index takes time. However, the index can be stored to disk (in compressed form) and used repeatedly in future debugging sessions, or for multiple queries in a single session.

To construct a heap index, use the !bhi command. You can store it to a file or in-memory (if you do not intend to reuse it in a future debugging session). The --fast switch makes index construction faster at the expense of more accurate information pertaining to static variable roots.

Note: you can control the chunk size used by the heap index with the --chunkSize switch. This is an advanced option; use at your own risk. A larger chunk size means the index is smaller and faster to build, but subsequent operations using the index will be slower.

> !bhi -f C:\temp\heapindex --fast
Enumerating roots took 767 ms
Building chunks took 2229 ms
Average referencing chunks per chunk: 8.86
Max referencing chunks per chunk:     5339
Min referencing chunks per chunk:     1
Objects with missing chunk ids:       393
Building chunk index took 17480 ms
Total chunks: 212538, Chunk size: 1024
Memory covered by all segments: 908900496 bytes
Memory covered by all chunks:   217638912 bytes
Wrote index file of size 3.198mb
Saving index to disk took 5584 ms
Time: 26089 ms, Memory start: 569.141kb, Memory end: 37.921mb, Memory delta: +37.365mb

If you have an existing heap index that you would like to load, use the !lhi command. Note that if the index was created by a different version of msos, it will fail to load.

> !lhi -f c:\temp\heapindex
This heap index does not have detailed static root information. As a result, you will not see the
names of static variables referencing your objects, only their addresses. Recreate the index without
the --fast switch to get full information. This may be slower.
Time: 2303 ms, Memory start: 567.227kb, Memory end: 28.221mb, Memory delta: +27.667mb

There are two commands that use the heap index. First, the !refs command displays any object referencing the specified object or referenced by it. Note that if the object is not reachable from roots, i.e. dead from the GC's perspective, you will not see any references to it (even though there could be such references which are also dead).

>  !refs 00000001803f02a8
Note: unrooted (dead) objects will not have any referencing objects displayed.
Object 00000001803f02a8 (System.Runtime.InputQueue<System.ServiceModel.Channels.RequestContext>) is referenced by the following objects:
  00000001803f0210 (System.ServiceModel.Channels.TransportReplyChannelAcceptor+TransportReplyChannel)
  00000001c0e35b98 (System.Runtime.InputQueue+AsyncQueueReader<System.ServiceModel.Channels.RequestContext>)
Object 00000001803f02a8 (System.Runtime.InputQueue<System.ServiceModel.Channels.RequestContext>) references the following objects:
  00000001803f02e8 (System.Runtime.InputQueue+ItemQueue<System.ServiceModel.Channels.RequestContext>)
  00000001803f0310 (System.Collections.Generic.Queue<System.Runtime.InputQueue+IQueueReader<System.ServiceModel.Channels.RequestContext>>)
  00000001803f0360 (System.Collections.Generic.List<System.Runtime.InputQueue+IQueueWaiter<System.ServiceModel.Channels.RequestContext>>)
  00000001803f03a8 (System.Action<System.ServiceModel.Channels.RequestContext>)
  00000001803a7438 (System.Func<System.Action<System.AsyncCallback,System.IAsyncResult>>)
Time: 142 ms, Memory start: 28.228mb, Memory end: 29.475mb, Memory delta: +1.247mb

Even more useful is !paths, which relies on the heap index to determine which paths from roots lead to the specified object. It is usually much faster than the !GCRoot command, which performs full heap traversal and does not rely on a reverse index. You can control some aspects of the output, such as the maximum number of paths that will be displayed.

>  !paths 00000001803f0310 --max 1
00000000086fdcb8 -> 00000001802ea9c0 local var thread 76
        -> 00000001802ea9c0 System.ServiceModel.ServiceHost
        -> 00000001802eabc0 System.ServiceModel.Dispatcher.ChannelDispatcherCollection
        -> 00000001802eabe8 System.Collections.Generic.List<System.ServiceModel.Dispatcher.ChannelDispatcherBase>
        -> 00000001803aaeb8 System.ServiceModel.Dispatcher.ChannelDispatcherBase[]
        -> 00000001803aee28 System.ServiceModel.Dispatcher.ChannelDispatcher
        -> 00000001803ae878 System.ServiceModel.Channels.HttpsChannelListener
        -> 00000001803aebe0 System.ServiceModel.Channels.TransportReplyChannelAcceptor
        -> 00000001803f0210 System.ServiceModel.Channels.TransportReplyChannelAcceptor+TransportReplyChannel
        -> 00000001803f02a8 System.Runtime.InputQueue<System.ServiceModel.Channels.RequestContext>
        -> 00000001803f0310 System.Collections.Generic.Queue<System.Runtime.InputQueue+IQueueReader<System.ServiceModel.Channels.RequestContext>>

Total paths displayed: 1
Time: 373 ms, Memory start: 28.230mb, Memory end: 29.602mb, Memory delta: +1.372mb

Use the !GCRoot command to understand why an object is being retained (as explained above, if this command is too slow, use a heap index and the !paths command). Some duplicates might be displayed. In the following output, there is a static variable s_InvariantCultureInfo that references a CultureInfo object, which in turn references the NumberFormatInfo object (at address 0000000001fc3984) that was the target of our query.

1> !gcroot 0000000001fc3984
0000000002FC1430 -> 0000000001FC2594 static var System.Globalization.CultureInfo.s_InvariantCultureInfo
        -> 0000000001FC2594 System.Globalization.CultureInfo
        -> 0000000001FC3984 System.Globalization.NumberFormatInfo

If you are having trouble with finalization, e.g. objects are not being finalized quickly enough, you can review the contents of the f-reachable queue (objects ready for finalization) by using the !frq command:

1> !frq
Address              Size       Class Name
0000000001fc1b54     20         Microsoft.Win32.SafeHandles.SafePEFileHandle
0000000001fc21f0     44         System.Threading.ReaderWriterLock
0000000001fc235c     20         Microsoft.Win32.SafeHandles.SafeFileHandle
0000000001fc3b14     20         Microsoft.Win32.SafeHandles.SafeViewOfFileHandle
0000000001fc3b28     20         Microsoft.Win32.SafeHandles.SafeFileMappingHandle
Total 5 objects ready for finalization

Heap Queries

Heap queries are probably the most powerful feature msos currently has to offer. They have no parallel in SOS.dll, and offer a lot of room for customization and creativity. The !hq command takes an arbitrary C# expression and executes it in a context that gives you access to the current target's heap. You can locate specific objects, access their properties, aggregate statistics, filter, group, and sort the results, and so on.

The following utility methods are available:

  • AllObjects() returns a collection of all heap objects. Each object has special __Type and __Size properties. You can cast the returned objects to ulong to obtain their address. In addition, any fields of the original object are accessible directly. For example, !hq (from o in AllObjects() where o.__Type == "System.String" select (int)o.m_stringLength).Sum() displays the total length of all the strings found on the heap. Auto-generated properties can be accessed directly using the property name -- you don't (and can't) use the backing field (<PropName>k__BackingField) for this.

  • ObjectsOfType("...") is a convenience method that returns a collection of heap objects that have the specified type. The type is matched exactly, e.g. ObjectsOfType("System.String").

  • AllClasses() returns a collection of objects representing each type on the heap. These objects have special __Name, __Fields, __StaticFields, and __ThreadStaticFields properties.

  • Class("...") returns an object representing a specific type. You can access static fields by using their name, but you currently need to provide an app domain object to get the actual value. This will be addressed in the future.

  • Object(...) returns a dynamic object based on the specified address. This is useful when you discovered the object's address using some other command (e.g. !gcroot) but want to write code against it dynamically using C# expressions. Note that most commands display object addresses in hex, so prefix the object address with 0x when passing it to the Object method.

  • ObjectEn(...) returns an enumerable with a single dynamic object based on the specified address. Use this if your query is more conveniently expressed as a LINQ query over an enumerable.

There are two output formats currently supported: --tabular and --json. The tabular format attempts to render the results as a table, which works well when the number of columns is small and the values are not very wide. The JSON format works better for complex objects.

Example queries:

1> !hq --tabular (from obj in AllObjects()
        group obj by obj.__Type into g
        let size = g.Sum(o => (long)o.__Size)
        let count = g.Count()
        orderby size descending
        select new { Type = g.Key, Count = count, Size = size }
       ).Take(10)
Type                               Count                              Size
System.Object[]                    6                                  17308
System.String                      165                                5236
System.RuntimeType                 26                                 728
System.Char[]                      4                                  718
System.String[]                    18                                 680
System.Globalization.CultureDa...  2                                  616
System.Int32[]                     11                                 564
System.Collections.Generic.Dic...  3                                  468
System.Byte[]                      2                                  280
System.Threading.ThreadAbortEx...  2                                  168
Rows: 10

1> !hq --json (from c in ObjectsOfType("System.String") _
               where (int)c.m_stringLength > 100 _
               select new { Size = c.__Size, Value = c.ToString() }
              ).Take(2)
{
  Size = 256
  Value = System.Data.SqlClient.SqlConnection, System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
}
{
  Size = 320
  Value = System.Configuration.Internal.ConfigurationManagerInternal, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
}
Rows: 2
Time: 2697 ms, Memory start: 647.516kb, Memory end: 689.172kb, Memory delta: +41.656kb

To work your way through collections as part of a query, you can use the GetDictionaryItems() method on dictionaries and the GetItems() method on arrays and lists. They both return a collection of dynamic objects that you can work with. Occasionally, if you try to use LINQ to process the returned collections, you might hit a compiler error (e.g., CS1943) because the collection object is dynamic. You can work around this by inserting a cast to a concrete type (not dynamic). For example:

1> !hq --tabular from o in AllObjects()
       where o.__Type.Contains("List<VSDebugging.Person")
       from item in o._items.GetItems()
       select item.Name
Query compilation failed with 1 errors:
c:\Users\Sasha\AppData\Local\Temp\n3wsitxl.0.cs(48,104) : error CS1943: An expression of type 'dynamic' is not allowed in a subsequent from clause in a query expression with source type 'System.Collections.Generic.IEnumerable<dynamic>'.  Type inference failed in the call to 'SelectMany'.

1> !hq --tabular from o in AllObjects()
       where o.__Type.Contains("List<VSDebugging.Person")
       from item in (IEnumerable<dynamic>)o._items.GetItems()
       select item.Name
Mike
Rows: 1

Also note that if you compare some field to null, cast the field to proper type. Compare these two commands and their outputs:

1> !hq tabular from s in ObjectsOfType("People") where s._name == null select s
1> !hq tabular from s in ObjectsOfType("People") where (string)s._name == null select s

Tip: The preceding examples have long commands that are split across lines for readability. You can actually type multi-line commands into the msos prompt. To do so, end the line with a space followed by an underscore (_). The debugger will then prompt you for the next line. For example:

1> !dumpheap --stat --type _
>    System.Collections.Generic.Dictionary<.*
Statistics:
MT                   Count      TotalSize  Class Name
00000000738ec654     1          48         System.Collections.Generic.Dictionary<System.Type,System.Security.Policy.EvidenceTypeDescriptor>
00000000738ec734     1          48         System.Collections.Generic.Dictionary<System.String,System.Globalization.CultureData>
Total 2 objects

Command Aliases

You can define arbitrary command aliases to help your debugging experience. The .newalias, .listalias, .rmalias, and % commands perform alias-related management tasks. For example:

1> .newalias dh !dumpheap --stat
Time: 1 ms, Memory start: 594.141kb, Memory end: 594.234kb, Memory delta: +96b
1> .listalias
Name                 Command
dh                   !dumpheap --stat
Time: 6 ms, Memory start: 594.164kb, Memory end: 594.281kb, Memory delta: +120b
1> % dh
Alias 'dh' expanded to '!dumpheap --stat'
... command output omitted for brevity ...

Aliases can accept an arbitrary number of parameters by embedding $1, $2 etc. in the command text. For example:

1> .listalias
Name                 Command
dh                   !dumpheap --stat
Time: 1 ms, Memory start: 591.586kb, Memory end: 591.586kb, Memory delta: -0b
1> .rmalias dh
Time: 1 ms, Memory start: 591.672kb, Memory end: 591.578kb, Memory delta: -96b
1> .newalias dh !dumpheap --type $1 --stat
Time: 1 ms, Memory start: 591.852kb, Memory end: 591.961kb, Memory delta: +112b
1> % dh System.String$
Alias 'dh' expanded to '!dumpheap --type System.String$ --stat'
Statistics:
MT                   Count      TotalSize  Class Name
000007fef8a668f0     1286275    63714760   System.String
Total 1286275 objects
Time: 15211 ms, Memory start: 592.430kb, Memory end: 3.710mb, Memory delta: +3.132mb

Like any other command, aliases can be loaded from a file using the -i switch. For example, create a text file called alias.txt with the following contents (or use the sample alias file):

.newalias httprequests !hq --tabular from hc in ObjectsOfType("System.Web.HttpContext").Take($1) _
let elapsed = new TimeSpan(DateTime.Now.Ticks - (long)hc._utcTimestamp.dateData) _
let timeout = new TimeSpan((long)hc._timeout._ticks) _
select new { _
  hc.__Address, _
  Method = hc._request._httpMethod, _
  Code = hc._response._statusCode, _
  Elapsed = ((bool)hc._response._completed || (bool)hc._finishPipelineRequestCalled) ? "Finished" : elapsed.ToString() , _
  Timeout = (bool)hc._timeoutSet ? timeout.ToString() : "No timeout", _
  VirtualPath = hc._request._filePath._virtualPath _
}

Now, run msos on a dump file from an ASP.NET application and use the httprequests alias:

C:\> msos -z myapp.dmp --diag -i alias.txt
... initial output omitted for brevity ...
1> .listalias
Name                 Command
httprequests         !hq --tabular from hc in ObjectsOfType("System.Web.HttpContext").Take($1) let elapsed = new TimeSpan(DateTime.Now.Ticks - (long)hc._utcTimestamp.dateData) let timeout = new TimeSpan((long)hc._timeout._ticks) select new { hc.__Address, Method = hc._request._httpMethod, Code = hc._response._statusCode, Elapsed = ((bool)hc._response._completed || (bool)hc._finishPipelineRequestCalled) ? "Finished" : elapsed.ToString() , Timeout = (bool)hc._timeoutSet ? timeout.ToString() : "No timeout", VirtualPath = hc._request._filePath._virtualPath }
Time: 5 ms, Memory start: 593.891kb, Memory end: 593.695kb, Memory delta: -200b
1> % httprequests 2
Alias 'httprequests' expanded to '!hq --tabular from hc in ObjectsOfType("System.Web.HttpContext").Take(2) let elapsed = new TimeSpan(DateTime.Now.Ticks - (long)hc._utcTimestamp.dateData) let timeout = new TimeSpan((long)hc._timeout._ticks) select new { hc.__Address, Method = hc._request._httpMethod, Code = hc._response._statusCode, Elapsed = ((bool)hc._response._completed || (bool)hc._finishPipelineRequestCalled) ? "Finished" : elapsed.ToString() , Timeout
= (bool)hc._timeoutSet ? timeout.ToString() : "No timeout", VirtualPath = hc._request._filePath._virtualPath }'
__Address         Method            Code              Elapsed           Timeout           VirtualPath
0000000182093...  POST              400               Finished          24855.03:14:0...  /api/OrderService.svc
00000001820be...  POST              200               Finished          24855.03:14:0...  /API/LoginService.svc
Rows: 2
Time: 3012 ms, Memory start: 595.227kb, Memory end: 691.984kb, Memory delta: +96.758kb

By defining useful heap queries as aliases, you can save a lot of typing.

When your script and alias files start to get big, comments will become useful. It's possible to comment msos commands. For a single line comment use #, and for multi-line comments use # as well and end each line with _.

What's more, you can define full-blown functions (or classes) for use in your heap queries. Again, the sample alias file contains some examples. Suppose you often need to reference all the path-like strings that start with C:\ in your queries. You can define a helper with .define, list all available helpers with .listdefines, and remove helpers with .undefine:

1> .define IEnumerable<string> PathLikeStrings() { _
>    return from s in ObjectsOfType("System.String") _
>           where ((string)s).StartsWith(@"C:\") _
>           select (string)s; _
>    }
1> .listdefines
#      Body
0      IEnumerable<string> PathLikeStrings() { return from s in ObjectsOfType("System.String") where ((string)s).StartsWith(@"C:\") select (string)s; }
1> !hq --tabular PathLikeStrings().Take(10)
C:\Temp\API
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config
C:\Temp\API\OrderService.svc63477491502921577495
C:\Temp\
C:\Temp\
C:\Windows\TEMP\tmpDDBA.tmp
C:\Windows\TEMP\tmpDDBB.tmp
C:\Temp\Validator\
C:\Temp\Validator\
C:\Temp
Rows: 10
1> .undefine 0
1> .listdefines
You do not have any helper methods defined. Use .define to define some.

Hyperlinks

By default, msos uses hyperlink mode, in which some commands will output hyperlinks that help execute related commands easily and without typing them in full. You can turn hyperlink mode on and off by using the .hyperlinks command with the --enable and --disable switches.

Hyperlinks are displayed as blue strings in brackets. To follow the link, execute the alias inside the hyperlink. For example, if a hyperlink says [a0], type % a0 to follow the link. Here's a concrete example:

3> !do 0000000002a765a8
Name:     System.Windows.Forms.TreeNode
MT:       00007ffce8206ca8
Size:     168(0xa8) bytes
Assembly: C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Windows.Forms\v4.0_4.0.0.0__b77a5c561934
e089\System.Windows.Forms.dll
Value:    140724202925224
Fields:
Offset   Type                 VT  Attr       Value                Name
0        System.Object        0   instance   0000000000000000     __identity [a2]
10       ...rDrawPropertyBag  0   instance   0000000000000000     propBag [a3]
20       System.String        0   instance   Gadgets              text
30       System.String        0   instance   C:\Program Files\Windows Sidebar\Gadgets name
40       ...NodeImageIndexer  0   instance   0000000002a766a8     imageIndexer [a4]
50       ...NodeImageIndexer  0   instance   0000000002a766e0     selectedImageIndexer [a5]
60       ...NodeImageIndexer  0   instance   0000000000000000     stateImageIndexer [a6]
70       System.String        0   instance                        toolTipText
80       ...orms.ContextMenu  0   instance   0000000000000000     contextMenu [a7]
90       ...ContextMenuStrip  0   instance   0000000000000000     contextMenuStrip [a8]
a0       System.Object[]      0   instance   0                    children
b0       ...s.Forms.TreeNode  0   instance   0000000002a75168     parent [a9]
c0       ...s.Forms.TreeView  0   instance   0000000002775060     treeView [a10]
d0       ...eeNodeCollection  0   instance   0000000000000000     nodes [a11]
e0       System.Object        0   instance   0000000000000000     userData [a12]
f0       System.IntPtr        1   instance   12538608             handle
100      System.Int32         1   instance   0                    index
108      System.Int32         1   instance   0                    childCount
110      System.Boolean       1   instance   False                nodesCleared
112      System.Boolean       1   instance   False                expandOnRealization
114      System.Boolean       1   instance   False                collapseOnRealization
120      ...ized.BitVector32  1   instance   0000000002a76640     treeNodeState
90       System.UInt32        1   instance   0                    treeNodeState.data
1540     System.Int32         1   shared     static               insertMask
   >> Domain:Value  0000000000b884d0:35 <<

In the preceding output, hyperlinks help expand the object and further follow its references. For example, the imageIndexer field references an object that can be displayed by executing % a4:

3> % a4
Alias 'a4' expanded to '!do 0000000002a766a8'
Name:     System.Windows.Forms.TreeNode+TreeNodeImageIndexer
MT:       00007ffce82073d0
Size:     56(0x38) bytes
Assembly: C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Windows.Forms\v4.0_4.0.0.0__b77a5c561934
e089\System.Windows.Forms.dll
Value:    140724202927056
Fields:
Offset   Type                 VT  Attr       Value                Name
0        System.String        0   instance                        key
10       ....Forms.ImageList  0   instance   0000000000000000     imageList [a13]
20       System.Int32         1   instance   -1                   index
28       System.Boolean       1   instance   True                 useIntegerIndex
30       ...s.Forms.TreeNode  0   instance   0000000002a765a8     owner [a14]
40       ...er+ImageListType  1   instance   0                    imageListType

Hyperlink-enabled commands can end up creating a large number of temporary aliases. You can clear them (without affecting your existing aliases) by executing .clearalias --temporary.

DbgEng Mode

msos can forward commands to the DbgEng engine, which is what powers WinDbg and other debuggers from that family. Even though you can use these debuggers directly, it's sometimes convenient to run a few commands through msos itself. The .dec command runs a DbgEng command in a one-shot debugging session. The debugging session is closed immediately when the command completes, so you can't assume any state will remain. For example:

1> .dec ~
.  0  Id: 1808.11d0 Suspend: 0 Teb: 00007ff5`ff6ae000 Unfrozen
   1  Id: 1808.1308 Suspend: 0 Teb: 00007ff5`ff6ac000 Unfrozen
   2  Id: 1808.f00 Suspend: 0 Teb: 00007ff5`ff6a8000 Unfrozen
   3  Id: 1808.11ec Suspend: 0 Teb: 00007ff5`ff6a6000 Unfrozen
   4  Id: 1808.ec0 Suspend: 0 Teb: 00007ff5`ff4da000 Unfrozen

The .dem command enters a temporary DbgEng mode, which lets you run multiple DbgEng commands and maintain state. It also means you can load extensions. It's basically a mini-WinDbg mode inside msos. The prompt changes to dbgeng> as long as you're in DbgEng mode. For example:

1> .dem
Loading DAC from C:\Windows\Microsoft.NET\Framework64\v2.0.50727\mscordacwks.dll
Automatically loaded SOS Extension
CLRDLL: Loaded DLL C:\Windows\Microsoft.NET\Framework64\v2.0.50727\mscordacwks.dll
CLR DLL status: Loaded DLL C:\Windows\Microsoft.NET\Framework64\v2.0.50727\mscordacwks.dll
dbgeng> !Name2EE FileExplorer FileExplorer.MainForm
Module: 00007ff93fd92e38 (FileExplorer.exe)
Token: 0x0000000002000002
MethodTable: 00007ff93fd965d0
EEClass: 00007ff93fec2578
Name: FileExplorer.MainForm
dbgeng> .lastevent
Last event: 1808.11d0: Break instruction exception - code 80000003 (first/second chance not available)
  debugger time: Sun Jul 26 11:51:41.218 2015 (UTC + 0:00)
dbgeng> k 4
Child-SP          RetAddr           Call Site
00000000`0061d3b8 00007ff9`b31413ed ntdll!NtWaitForMultipleObjects+0xa
00000000`0061d3c0 00007ff9`b57f32b4 KERNELBASE!WaitForMultipleObjectsEx+0xe1
00000000`0061d6a0 00007ff9`b5615802 user32!MsgWaitForMultipleObjectsEx+0x141
00000000`0061d750 00007ff9`b56421c0 combase!CCliModalLoop::BlockFn+0x126
dbgeng> q
Exited DbgEng mode; you are now back in msos.

msos will try to load SOS.dll into the DbgEng session automatically. This doesn't always work because Microsoft doesn't index SOS.dll for all CLR versions, and doesn't make SOS.dll available at all for Windows Phone. Even if SOS.dll is not available, the DAC might be, and so you might be able to use sosex.

Miscellaneous Commands

Use the .paging --rows *n* command to configure paging. msos will then display up to n rows before pausing for user input. Turn off paging by running .paging --disable. Paging settings are ignored when the output is redirected to a file using msos' command-line -o switch.

Include the --diag switch when launching msos to get diagnostic information after each command is executed. Currently this information includes the time it took to execute each command, and the memory usage before and after the command's execution. This helps detect memory leaks and slow commands.

1> !hq --tabular (from i in Enumerable.Range(0, 100000000) select i).Count()
100000000
Rows: 1
Time: 6,258.01 ms, CPU time: 5,875.00 ms (93.88%)
Memory start: 595.680kb, Memory end: 635.359kb, Memory delta: +39.680kb

The !DumpDomain command lists all the application domains in the current target. If a specific domain id is passed, the modules loaded into that application domain are also displayed.

1> !dumpdomain
Id   Name                                     # Modules Application Base
1    DefaultDomain                            14       c:\windows\system32\inetsrv\
2    /LM/W3SVC/2/ROOT/12987421178073111222... 153      C:\MyService\API\
3    /LM/W3SVC/2/ROOT/API/V/12889123711201... 48       C:\MyService\Validator\
4    /LM/W3SVC/2/ROOT-3-129874220678904342    25       C:\MyService\

The !lm command lists the managed assemblies and native DLLs loaded into the target:

3> !lm
start                size       version              filename
00000000001a0000     8000       v1.0.0.00            C:\Temp\crash1\FileExplorer\bin\Debug\FileExplorer.exe
0000000062250000     d1000      v12.0.20806.33440    C:\Windows\Microsoft.NET\Framework\v4.0.30319\diasymreader.dll
0000000068370000     ee000      v4.0.30319.33440     C:\Windows\assembly\NativeImages_v4.0.30319_32\System.Configuration\814dd4620e79397b2463\System.Configuration.ni.dll
0000000068790000     c50000     v4.0.30319.34250     C:\Windows\assembly\NativeImages_v4.0.30319_32\System.Windows.Forms\592a40d8bc95bb64b2e8\System.Windows.Forms.ni.dll
... snipped for brevity ...

To display symbol load information for a specific module, use !lm --symstate <module name>. If symbols aren't loaded, it doesn't necessarily mean they cannot be found -- the debugger loads symbols on demand.

3> !lm --symstate FileExplorer.exe
Module:     C:\temp\crash1\FileExplorer\bin\Debug\FileExplorer.exe
PDB loaded: False
PDB name:   c:\Temp\crash1\FileExplorer\obj\Debug\FileExplorer.pdb
Debug mode: Default, IgnoreSymbolStoreSequencePoints, EnableEditAndContinue, DisableOptimizations

Use the db command to display a raw region of memory as hex bytes and ASCII characters. The -l and -c switches control how many bytes to display and how many columns to put in each row, respectively.

1> db 0000000001fc3984 -l 20 -c 10
0000000001fc3984  7c 4e cf 73 3c 2c fc 01 4c 2c   |N.s<,..L,
0000000001fc398e  fc 01 3c 2c fc 01 bc 26 fc 01   ..<,...&..

Use the q command to quit the debugger.

Symbol Loading

Currently, msos loads symbols from the location specified by the _NT_SYMBOL_PATH environment variable. Point it to the location of your symbols and to the Microsoft symbol server for best results. For example:

set _NT_SYMBOL_PATH=d:\mysymbols;\\myserver\moresymbols;srv*C:\symbols*http://msdl.microsoft.com/download/symbols
msos -z myapp.dmp

msos will complain if it needs but cannot find symbols, and will report when symbols are downloaded from the symbol server.