-
Notifications
You must be signed in to change notification settings - Fork 21
Home
msos was born out of a need to debug .NET process dumps without the SOS.dll extension available. It quickly turned into more than that with a variety of commands and options that, in many cases, exceed what SOS.dll has to offer.
You should use msos when you don't have access to SOS.dll (e.g., when debugging Windows Phone dumps), when you need some of msos's advanced functionality, or simply when you don't want to deal with the overhead of the full-blown WinDbg debugger. At this time, msos offers commands (such as !hq) that are extremely difficult to implement using the SOS.dll extension.
msos is a standalone command-line debugger. It can attach to a live process (in non-invasive mode) or open a dump file. You will need the x86 version of msos to debug x86 targets, and the x64 version to debug x64 targets. This is a limitation of the .NET debugging API.
msos is designed to be self-documenting. For example, try running msos --help from the command line to see the first-level command-line options. When inside msos, type help to get a list of commands or help cmdname to get information about a specific command's switches.
The command-line switches are case-sensitive, but the command verbs are not. For example, you can use !clrstack, !CLRStack, or !CLRSTACK with the same effect.
To open a dump file, use the -z switch:
msos -z myapp.dmp
To attach to a running process, use either the --pn or the --pid switches. If there is more than one process with the given name, you will be asked to specify the process id.
msos --pid 1008
msos --pn devenv
You can run msos and ask it to execute a set of initial commands, separated by semicolons. This is useful for automatic analysis, or when you always begin a debugging session by displaying basic information. For example, if you always begin a session by looking at the list of modules followed by the current thread's stack, use this command line:
msos -z myapp.dmp -c "!lm; !clrstack"
You can include the q command so that the debugger exits without waiting for further input. This is especially useful when analyzing multiple dump files in a row, automatically, as part of a script.
You can also load initial commands from a file. Create a file with each command on a separate line, and pass that file to the -i switch:
msos -z myapp.dmp -i commands.txt
Finally, you can pipe msos's output to a file (instead of the console) by using the -o switch:
msos -z myapp.dmp -i commands.txt -o output.txt
When you give msos a crash dump, it will automatically try to switch to the crashing thread and display the exception information along with its stack trace. The !pe (or !PrintException) command can be used to display the current exception again. If you have the address of a .NET exception object, pass it to !pe as a parameter.
To display a list of all managed threads in the target, run !Threads. When available, additional information is displayed in the Special and Exception columns. The Lock# column indicates the number of locks the thread currently owns.
1> !Threads
2 CLR threads, 0 CLR thread pool threads, 1 background threads
MgdId OSId Lock# Apt Special Exception
1 5980 1 MTA
2 8952 0 MTA Finalizer
msos attempts to set the current thread to either the thread that encountered an exception (useful with crash dumps), or the first managed thread it can find. You can switch threads by using the ~ command and the managed thread id:
~ 7
To display the current thread's managed stack, run !CLRStack. If the -a switch is specified, this command displays some of the local variables for each frame. Only local variables pointing to reference types (on the heap) are currently supported, and only variable addresses (not names) are available.
1> !clrstack
SP IP Function
000000000020F180 0000000000000000 InlinedCallFrame
000000000020F17C 0000000073C8B7BF DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)
000000000020F180 0000000000000000 InlinedCallFrame
000000000020F1E4 00000000743DCD64 System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Boolean, Boolean, Int32 ByRef)
000000000020F218 00000000743DCC6B System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)
000000000020F238 0000000073C74EC8 System.IO.StreamReader.ReadBuffer()
000000000020F24C 0000000073BDE90F System.IO.StreamReader.ReadLine()
000000000020F268 00000000743E2E5D System.IO.TextReader+SyncTextReader.ReadLine()
000000000020F278 00000000742EA842 System.Console.ReadLine()
000000000020F280 00000000003D00C9 VSDebugging.Program.Main(System.String[])
000000000020F410 0000000000000000 GCFrame
The !ThreadPool command displays general information about the CLR thread pool.
The !SyncBlk command displays all synchronization objects. These include Monitors (the C# lock statement acquires and releases a Monitor), Reader-Writer Locks, and other synchronization mechanisms. When available, ownership information is also displayed along with a list of threads waiting for the synchronization object. For example, in the following example, thread #1 owns the first monitor:
1> !syncblk
Address Type Locked Owner(s) Waiter(s)
0000000001fc3f34 Monitor 1 1
0000000001fc21f0 None 0
Use the !dumpheap --stat command to get an initial reading of what's going on with your managed heap. It displays statistics for each type indicating the number of objects from that type and their total size. Generally, object sizes do not include the sizes of any children. For example, the size of an array of strings doesn't include the size of the strings themselves. The !ObjSize command displays the size of the entire object graph rooted at a specific object.
The !dumpheap command accepts multiple advanced options to filter the output and to display individual objects. The --type switch is most flexible, and takes a regular expression that filters the types of objects to be displayed. For example (omit the --stat switch to get a list of objects, and not just statistics):
1> !dumpheap --type String --stat
Statistics:
MT Count TotalSize Class Name
0000000073cf4cf8 1 12 System.Collections.Generic.GenericEqualityComparer<System.String>
00000000738ec734 1 48 System.Collections.Generic.Dictionary<System.String,System.Globalization.CultureData>
0000000073cf4b04 2 56 System.Text.StringBuilder
0000000073cf4e04 1 60 System.Collections.Generic.Dictionary+Entry<System.String,System.Globalization.CultureData>[]
0000000073cb5738 18 680 System.String[]
0000000073cf21b4 165 5236 System.String
Total 188 objects
Other heap filtering options include --min, --max and --mt.
To display an individual object, use the !do (or !DumpObj) command. It displays the object's fields (unless you pass the --nofields switch), including any static and thread-static fields. Nested value types are displayed unless you pass the --norecurse switch. Nested reference types are never displayed.
1> !do 0000000001fc3984
Name: System.Globalization.NumberFormatInfo
MT: 0000000073cf4e7c
Size: 132(0x84) bytes
Assembly: C:\WINDOWS\Microsoft.Net\assembly\GAC_32\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.
dll
Value: 1942965884
Fields:
Offset Type VT Attr Value Name
0 System.Int32[] 0 instance 33303612 numberGroupSizes
8 System.Int32[] 0 instance 33303628 currencyGroupSizes
10 System.Int32[] 0 instance 33303612 percentGroupSizes
18 System.String 0 instance + positiveSign
20 System.String 0 instance - negativeSign
28 System.String 0 instance . numberDecimalSeparator
30 System.String 0 instance , numberGroupSeparator
38 System.String 0 instance , currencyGroupSeparator
40 System.String 0 instance . currencyDecimalSeparator
48 System.String 0 instance ☼ currencySymbol
50 System.String 0 instance <null> ansiCurrencySymbol
58 System.String 0 instance NaN nanSymbol
60 System.String 0 instance Infinity positiveInfinitySymbol
68 System.String 0 instance -Infinity negativeInfinitySymbol
70 System.String 0 instance . percentDecimalSeparator
78 System.String 0 instance , percentGroupSeparator
80 System.String 0 instance % percentSymbol
88 System.String 0 instance % perMilleSymbol
90 System.Object[] 0 instance 33303556 nativeDigits
98 System.Int32 1 instance 0 m_dataItem
a0 System.Int32 1 instance 2 numberDecimalDigits
a8 System.Int32 1 instance 2 currencyDecimalDigits
b0 System.Int32 1 instance 0 currencyPositivePattern
b8 System.Int32 1 instance 0 currencyNegativePattern
c0 System.Int32 1 instance 1 numberNegativePattern
c8 System.Int32 1 instance 0 percentPositivePattern
d0 System.Int32 1 instance 0 percentNegativePattern
d8 System.Int32 1 instance 2 percentDecimalDigits
e0 System.Int32 1 instance 1 digitSubstitution
e8 System.Boolean 1 instance True isReadOnly
ea System.Boolean 1 instance False m_useUserOverride
ec System.Boolean 1 instance True m_isInvariant
ee System.Boolean 1 instance True validForParseAsNumber
f0 System.Boolean 1 instance True validForParseAsCurrency
528 ...NumberFormatInfo 0 shared static invariantInfo
>> Domain:Value 0000000000604ac8:<null> <<
To display collections, use the !DumpCollection command. It supports arrays, lists, and dictionaries. Other collections can be displayed using the !hq command (discussed below).
1> !dumpcollection 0000000001fc1bb0
Type: System.Collections.Generic.Dictionary<System.Type,System.Security.Policy.EvidenceTypeDescriptor>
Size: 10
Key Value
[0000000001fc1d3c System.RuntimeType] <null>
[0000000001fc1d58 System.RuntimeType] <null>
[0000000001fc1d74 System.RuntimeType] <null>
[0000000001fc1d90 System.RuntimeType] <null>
[0000000001fc1dac System.RuntimeType] <null>
[0000000001fc1dc8 System.RuntimeType] <null>
[0000000001fc1de4 System.RuntimeType] <null>
[0000000001fc1e00 System.RuntimeType] <null>
[0000000001fc1e1c System.RuntimeType] <null>
[0000000001fc1e38 System.RuntimeType] <null>
Time: 3 ms, Memory start: 568.008kb, Memory end: 568.008kb, Memory delta: -0b
Use the !ObjSize command to display the size of the object graph rooted at a specific object. This includes all referenced objects.
1> !objsize 0000000001fc3984
0000000001fc3984 graph size is 24 objects, 574 bytes
The !EEHeap command provides basic information about the CLR's memory usage in the target. The sizes of each heap and each generation (including the Large Object Heap) are displayed. Non-GC regions (such as the loader heap) are also shown.
1> !EEHeap
GC regions:
Address Size Type Heap# Commit/Reserve
0000000001fc1000 68.000kb Ephemeral 0 Committed
0000000001fd2000 15.930mb Ephemeral 0 Reserved
0000000002fc1000 68.000kb LargeObject 0 Committed
0000000002fd2000 15.930mb LargeObject 0 Reserved
Gen Size
0 11.965kb
1 12b
2 12b
LOH 16.898kb
Total 28.887kb
Other regions:
...snipped...
After identifying objects that you suspect should have been collected, use the !GCRoot command to understand why they are being retained. Some duplicates might be displayed. In the following output, there is a static variable s_InvariantCultureInfo that references a CultureInfo object, which in turn references the NumberFormatInfo object (at address 0000000001fc3984) that was the target of our query.
1> !gcroot 0000000001fc3984
0000000002FC1430 -> 0000000001FC2594 static var System.Globalization.CultureInfo.s_InvariantCultureInfo
-> 0000000001FC2594 System.Globalization.CultureInfo
-> 0000000001FC3984 System.Globalization.NumberFormatInfo
If you are having trouble with finalization, e.g. objects are not being finalized quickly enough, you can review the contents of the f-reachable queue (objects ready for finalization) by using the !frq command:
1> !frq
Address Size Class Name
0000000001fc1b54 20 Microsoft.Win32.SafeHandles.SafePEFileHandle
0000000001fc21f0 44 System.Threading.ReaderWriterLock
0000000001fc235c 20 Microsoft.Win32.SafeHandles.SafeFileHandle
0000000001fc3b14 20 Microsoft.Win32.SafeHandles.SafeViewOfFileHandle
0000000001fc3b28 20 Microsoft.Win32.SafeHandles.SafeFileMappingHandle
Total 5 objects ready for finalization
Heap queries are probably the most powerful feature msos currently has to offer. They have no parallel in SOS.dll, and offer a lot of room for customization and creativity. The !hq command takes an arbitrary C# expression and executes it in a context that gives you access to the current target's heap. You can locate specific objects, access their properties, aggregate statistics, filter, group, and sort the results, and so on.
The following utility methods are available:
-
AllObjects()returns a collection of all heap objects. Each object has special__Typeand__Sizeproperties. You can cast the returned objects toulongto obtain their address. In addition, any fields of the original object are accessible directly. For example,!hq (from o in AllObjects() where o.__Type == "System.String" select (int)o.m_stringLength).Sum()displays the total length of all the strings found on the heap. Auto-generated properties can be accessed directly using the property name -- you don't (and can't) use the backing field (<PropName>k__BackingField) for this. -
ObjectsOfType("...")is a convenience method that returns a collection of heap objects that have the specified type. The type is matched exactly, e.g.ObjectsOfType("System.String"). -
AllClasses()returns a collection of objects representing each type on the heap. These objects have special__Name,__Fields,__StaticFields, and__ThreadStaticFieldsproperties. -
Class("...")returns an object representing a specific type. You can access static fields by using their name, but you currently need to provide an app domain object to get the actual value. This will be addressed in the future. -
Object(...)returns a dynamic object based on address. This is useful when you discovered the object's address using some other command (e.g.!gcroot) but want to write code against it dynamically using C# expressions. Note that most commands display object addresses in hex, so prefix the object address with0xwhen passing it to theObjectmethod.
Example query:
1> !hq (from obj in AllObjects()
group obj by obj.__Type into g
let size = g.Sum(o => (long)o.__Size)
let count = g.Count()
orderby size descending
select new { Type = g.Key, Count = count, Size = size }
).Take(10)
Type Count Size
System.Object[] 6 17308
System.String 165 5236
System.RuntimeType 26 728
System.Char[] 4 718
System.String[] 18 680
System.Globalization.CultureDa... 2 616
System.Int32[] 11 564
System.Collections.Generic.Dic... 3 468
System.Byte[] 2 280
System.Threading.ThreadAbortEx... 2 168
Rows: 10
To work your way through collections as part of a query, you can use the GetDictionaryItems() method on dictionaries and the GetItems() method on arrays and lists. They both return a collection of dynamic objects that you can work with. Occasionally, if you try to use LINQ to process the returned collections, you might hit a compiler error (e.g., CS1943) because the collection object is dynamic. You can work around this by inserting a cast to a concrete type (not dynamic). For example:
1> !hq from o in AllObjects()
where o.__Type.Contains("List<VSDebugging.Person")
from item in o._items.GetItems()
select item.Name
Query compilation failed with 1 errors:
c:\Users\Sasha\AppData\Local\Temp\n3wsitxl.0.cs(48,104) : error CS1943: An expression of type 'dynamic' is not allowed in a subsequent from clause in a query expression with source type 'System.Collections.Generic.IEnumerable<dynamic>'. Type inference failed in the call to 'SelectMany'.
1> !hq from o in AllObjects()
where o.__Type.Contains("List<VSDebugging.Person")
from item in (IEnumerable<dynamic>)o._items.GetItems()
select item.Name
Mike
Rows: 1
Include the --diag switch when launching msos to get diagnostic information after each command is executed. Currently this information includes the time it took to execute each command, and the memory usage before and after the command's execution. This helps detect memory leaks.
1> !hq (from i in Enumerable.Range(0, 100000000) select i).Count()
100000000
Rows: 1
Time: 1692 ms, Memory start: 89.262kb, Memory end: 116.945kb, Memory delta: +27.684kb
The !lm command lists the managed assemblies loaded into the target. Symbol load status is also displayed. If symbols aren't loaded, it doesn't necessarily mean they cannot be found -- the debugger loads symbols on demand.
1> !lm
start size symloaded filename
00000000738e0000 01095000 False C:\WINDOWS\Microsoft.Net\assembly\GAC_32\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
0000000000080000 00008000 False C:\Temp\VSDebugging\bin\Debug\VSDebugging.exe
Use the db command to display a raw region of memory as hex bytes and ASCII characters. The -l and -c switches control how many bytes to display and how many columns to put in each row, respectively.
1> db 0000000001fc3984 -l 20 -c 10
0000000001fc3984 7c 4e cf 73 3c 2c fc 01 4c 2c |N.s<,..L,
0000000001fc398e fc 01 3c 2c fc 01 bc 26 fc 01 ..<,...&..
Use the q command to quit the debugger.
Currently, msos loads symbols from the location specified by the _NT_SYMBOL_PATH environment variable. Point it to the location of your symbols and to the Microsoft symbol server for best results. For example:
set _NT_SYMBOL_PATH=d:\mysymbols;\\myserver\moresymbols;srv*C:\symbols*http://msdl.microsoft.com/download/symbols
msos -z myapp.dmp
msos will complain if it needs but cannot find symbols, and will report when symbols are downloaded from the symbol server.
TODO
Copyright (C) Sasha Goldshtein, 2015. All rights reserved.