This software demonstrates that issuing some NVMe commands from userland to NVMe device using Windows inbox NVMe driver (stornvme.sys)[1].
Specification that this software refered to is NVMe 1.3d[2] and NVMe 1.4b[6].
For StorNVMe on Windows 10 version 1903 and later, Supported NVMe Admin and NVM command sets and associated opcodes are shown in [8], and supported features are shown in [9].
Table 1. Command matrix (Admin Command Set)
Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed
| Opcode | Command name | Status | Note |
|---|---|---|---|
| 00h | Delete I/O Submission Queue | 1 | |
| 01h | Create I/O Submission Queue | 1 | |
| 02h | Get Log Page | See Table 4 | |
| 04h | Delete I/O Completion Queue | 1 | |
| 05h | Create I/O Completion Queue | 1 | |
| 06h | Identify | See Table 3 | |
| 08h | Abort | 1 | |
| 09h | Set Features | See Table 5 | |
| 0Ah | Get Features | See Table 5 | |
| 0Ch | Asynchronous Event Request | 1 | |
| 0Dh | Namespace Management | 2 | |
| 10h | Firmware Commit | ||
| 11h | Firmware Image Download | ||
| 14h | Device Self-test | x | |
| 15h | Namespace Attachment | 2 | |
| 18h | Keep Alive | Not listed in [8] | |
| 19h | Directive Send | 1 | |
| 1Ah | Directive Receive | 1 | |
| 1Ch | Virtualization Management | Not supported | |
| 1Dh | NVMe-MI Send | 2 | |
| 1Eh | NVMe-MI Receive | 2 | |
| 7Ch | Doorbell Buffer Config | Not supported | |
| 80h | Format NVM | x | Only for data drive. Need to support Format NVM command and cryptographic erase is supported along Format NVM (bit 2 of the FNA field in Identify Controller data structure) |
| 81h | Security Send | ||
| 82h | Security Receive | x | Getting TCG Level 0 Discovery Data for example in this tool |
| 84h | Sanitize | 2 | |
| 86h | Get LBA Status | Not supported | |
| C0h -- FFh | Vendor Specific |
Note 1: probably cannot issue this command because only "Internal Driver Use" is written in [8]
Note 2: can issue this command only in WinPE mode according to [8]
Table 2. Command matrix (NVM Command Set)
Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed
| Opcode | Command name | Status | Note |
|---|---|---|---|
| 00h | Flush | x | |
| 01h | Write | x | |
| 02h | Read | x | |
| 04h | Write Uncorrectable | 1 | |
| 05h | Compare | Can issue only in WinPE mode [8] | |
| 08h | Write Zeroes | 1 | |
| 09h | Dataset Management | x | Only Deallocate is supported |
| 0Ch | Verify | 1 | |
| 0Dh | Reservation Register | 1 | |
| 0Eh | Reservation Report | 1 | |
| 11h | Reservation Acquire | 1 | |
| 15h | Reservation Release | 1 | |
| 80h -- FFh | Vendor Specific |
Note 1: these command are stated as "Not supported" in [8]
CNS values from 10h to 13h are seemed to be able to issue only in WinPE mode, because these values are mandatory for controllers that support the Namespace Management capability in NVMe spec., but Namespace Management command can only be issued in WinPE mode according to [8].
CNS values 14h and 15h are seemed to be unable to issue, because these values are mandatory for controllers that support Virtualization Enhancements in NVMe spec., but Virtualization Management command is not supported according to [8].
Table 3. Controller or Namespace Structure (CNS)
Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed
| CNS | Description | Status | Note |
|---|---|---|---|
| 00h | Namespace | x | NSID = 1 |
| 01h | Controller | x | CNTID = 0 |
| 02h | Active Namespace ID list | returned data seems to be the data for Identify Namespace | |
| 03h | Namespace Identification Descriptor list | returned data seems to be the data for Identify Namespace | |
| 04h -- 0Fh | Reserved | ||
| 10h | Allocated Namespace ID list | Probably can issue only in WinPE mode [8] | |
| 11h | Specified allocated namespace | Probably can issue only in WinPE mode [8] | |
| 12h | List of controllers attached to specified namespace | Probably can issue only in WinPE mode [8] | |
| 13h | List of controllers exist in the NVM subsystem | Probably can issue only in WinPE mode [8] | |
| 14h | Primary Controller Capabilities | Probably cannot issue [8] | |
| 15h | Secondary controller list | Probably cannot issue [8] | |
| 16h -- 1Fh | Reserved | ||
| 20h -- FFh | Reserved |
Table 4. Log Identifier (LID)
Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed
| LID | Description | Status | Note |
|---|---|---|---|
| 01h | Error Information | x | Number of entries to be retrieved from devices depends on the value "ELPE" in Identify Controller data |
| 02h | SMART / Health Information | x | CNTID = 0, NSID = 0xFFFFFFFh |
| 03h | Firmware Slot Information | x | |
| 05h | Command Supported and Effects | x | |
| 06h | Device Self-test | x | |
| 07h | Telemetry Host-initiated | x | Getting header only. see also Note 1. |
| 08h | Telemetry Controller-initiated | x | Getting header only. |
| 09h -- 6Fh | Reserved | ||
| 70h | Discovery | For NVMe over Fabrics (NVMeoF) | |
| 71h -- 7Fh | Reserved for NVMeoF | ||
| 80h | Reservation Notification | Probably cannot issue [8] | |
| 81h | Sanitize Status | Probably can issue only in WinPE mode [8] | |
| 82h -- BFh | Reserved | ||
| C0h -- FFh | Vendor Specific |
Note 1: If you want to set Create Telemetry Host-Initiated Data to 1, you need Windows 10 version 2004 (build 10041) or later. And to do this, this tool uses IOCTL_STORAGE_GET_DEVICE_INTERNAL_LOG described in [9]. But I don't know products (SSDs) that can respond this type of request (command)...
According to [8], only Host Controlled Thermal Management is supported via Set Features command with IOCTL_STORAGE_SET_PROPERTY.
Table 5. Feature Identifier (FID)
Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed
| FID | Description | Status (Get/Set) | Note |
|---|---|---|---|
| 01h | Arbitration | x / | |
| 02h | Power Management | x / | |
| 03h | LBA Range Type | -- / | error code = 1117 (The request could not be performed because of an I/O device error.) |
| 04h | Temperature Threashold | x / | |
| 05h | Error Recovery | x / | |
| 06h | Volatile Write Cache | x / -- | error code = 1117 (The request could not be performed because of an I/O device error.) |
| 07h | Number of Queues | -- / | error code = 1117 (The request could not be performed because of an I/O device error.) |
| 08h | Interrupt Coalescing | x / | |
| 09h | Interrupt Vector Configuration | / | |
| 0Ah | Write Atomicity Normal | x / | |
| 0Bh | Asynchronous Event Configuration | x / | |
| 0Ch | Autonomous Power State Transition | -- / | error code = 1117 (The request could not be performed because of an I/O device error.) |
| 0Dh | Host Memory Buffer | -- / | error code = 1117 (The request could not be performed because of an I/O device error.) |
| 0Eh | Timestamp | -- / | error code = 1117 (The request could not be performed because of an I/O device error.) |
| 0Fh | Keep Alive Timer | / | |
| 10h | Host Controlled Thermal Management | x / x | "SV = 1" is not supported |
| 11h | Non-operational Power State Config | x / | |
| 12h -- 77h | Reserved | / | |
| 78h -- 7Fh | Refer to the NVMe-MI spec. | / | |
| 80h | Software Progress Marker | x / | |
| 81h | Host Identifier | / | |
| 82h | Reservation Notification Mask | / | 1 |
| 83h | Reservation Persistence | / | 1 |
| 84h -- BFh | Reserved | / | |
| C0h -- FFh | Vendor specific | / |
Note 1: FID 82h and 83h are seemed to be unable to issue, because commands related to Reservation are not supported according to [8].
Read command is issued via SCSI Pass-through[3].
Issue Read command with Starting Logical Block Address (SLBA) = 0 and Number of Logical Blocks (NLB) = 1.
Write command is issued via SCSI Pass-through[3].
Issue Write command with SLBA = 0, NLB = 1, and data is "DEADBEEF".
Dataset Management command is issued via SCSI Pass-through[3] with IOCTL_STORAGE_MANAGE_DATA_SET_ATTRIBUTES [4][5] (only Deallocate bit is set).
LBA region to be deallocated is SLBA = 0 and NLB = 1 (Sector 0).
An NVMe drive that supports Dataset Management command is also needed.
Supporting Dataset Management command can be checked by bit 2 of "Optional NVM Command Support (ONCS)" field in Identify Controller Data Structure.
If the bit is set to 1, the controller support Dataset Management command.
For further information, see "5.15.3 Identify Controller data structure (CNS 01h)" in the NVMe specification[2].
An NVMe drive that return fixed data pattern for reading deallocated logical block is recommended for checking the result of deallocation.
It can be checked by bit 2:0 of "Deallocate Logical Block Features (DLFEAT)" field in Identify Namespace Data Structure.
If it is 001b, the drive returns all bytes cleared to 00h for reading deallocated logical block.
If it is 010b, the drive returns all bytes set to FFh for reading deallocated logical block.
For further information, see "5.15.2 Identify Namespace data structure (CNS 00h)" in the NVMe specification[2].
Issuing "Write", "Dataset Management (Deallocate)", and "Format NVM" command to devices under management by file systems (e.g. FAT, NTFS) will corrupt your system.
These commands should be issued with highest care.
"Read" command does not have such a side effect... :-)
Confirmed on the following software environment:
- Operating system and device driver
- Windows 10 Professional 64bit (Version 2004, Build 21286.1000)
- stornvme.sys (version 10.0.21286.1000, WinBuild 160101.0800)
- Developping environment
- Microsoft Visual Studio Community 2019 (Version 16.7.6)
- Microsoft Visual C++ 2019
- Windows Driver Kit (10.0.19030.1000)
- Microsoft Visual Studio Community 2019 (Version 16.7.6)
Only tested with the NVMe drive directly attached to PC via PCIe.
It is ok to access to M.2 drives and M.2 drives that attached to M.2-PCIe converter Add-In-Card (AIC).
Also, it may be ok with U.2 drives.
But it may not work over protocol translations such as usb-nvme.
- Double click
NVMeTool.slnto start Visual Studio - Build program with Build -> Build Solution (or Control+Shift+B)
- Run program (F5)
Note that, when you run (debug) in Visual Studio, confirm that the command line argument is set to appropriate drive ID for your system.
You can find the settings of command line arguments from Debug menu -> NVMeTool Debug Properties -> Configuration Properties -> Debugging -> Command Arguments.
- Test commands, LIDs, FIDs, and CNSs in the NVMe specification and listed in [8]
- Test in WinPE mode
This software is released under the MIT License and Microsoft Public License (MS-PL), see LICENSE.
[1] Microsoft, "Working with NVMe drives", https://docs.microsoft.com/en-us/windows/desktop/fileio/working-with-nvme-devices, Retrieved May 14, 2019
[2] NVM Express, "NVM Express[TM] Base Specification", Revision 1.3d, March 20, 2019
[3] Microsoft, "SCSI Pass-Through Interface Tool", https://github.com/microsoft/Windows-driver-samples/tree/master/storage/tools/spti, Retrieved May 14, 2019.
[4] Mark E. Russinovich, et al., "Windows Internals, Part 2", 6th Edition, October 2012, Microsoft Press, ISBN: 978-0735665873
[5] Microsoft, "IOCTL_STORAGE_MANAGE_DATA_SET_ATTRIBUTES IOCTL", https://docs.microsoft.com/ja-jp/windows/desktop/api/winioctl/ni-winioctl-ioctl_storage_manage_data_set_attributes, Retrieved May 14, 2019
[6] NVM Express, "NVM Express[TM] Base Specification", Revision 1.4b, Sept. 21, 2020
[7] Microsoft, "NVMe Features Supported by StorNVMe", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/nvme-features-supported-by-stornvme, Retrieved June 25, 2020
[8] Microsoft, "StorNVMe Command Set Support", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/stornvme-command-set-support, Retrieved June 25, 2020
[9] Microsoft, "NVMe Features Supported by StorNVMe", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/stornvme-feature-support, Retrieved June 29, 2020