Skip to content

virtio fs

Jump to bottom Edit New page
WALDEMAR KOZACZUK edited this page Apr 1, 2021 · 3 revisions

Virtio-fs is a shared file system, providing efficient and performant access to a directory on the host. This has several benefits:

  • Dynamic reconfiguration. No need to repeat the image build step in order to change the guest file system contents. Just make the changes in the host directory directly (sometimes even while the guest is running!).
  • Faster builds. Since everything except the (small) kernel need not be included in the actual image, baking the image becomes faster (especially compared to ZFS).

An implementation consists of three parts:

  • The virtio device in the virtual machine monitor (vhost-user-fs in QEMU).
  • The guest driver for the above device.
  • The device backend on the host (virtiofsd).

Virtio-fs introduces the DAX window as a means for achieving high performance.

Status

Currently, virtio-fs is only supported on QEMU (since version 5.0). Included are an implementation for the device (vhost-user-fs), as well as virtiofsd, a separate component backing the device, serving the contents of the host directory. DAX support is not yet merged in stable QEMU.

OSv includes a virtio-fs device driver, allowing read-only access. The underlying host directory can be mounted both as a root or non-root file system. It also supports the DAX window.

How-to

Ensure you are using a version of QEMU with virtio-fs support (the more recent the better). Also, make sure the virtiofsd is in the $PATH. Depending on the distribution the virtiofsd may not be present in the PATH by default and you need to add it explicitly to the PATH: on Ubuntu, it is typically located under /usr/lib/qemu/virtiofsd, on Fedora - under /usr/libexec/virtiofsd.

NOTE: When using virtio-fs you need to execute scripts/run.py (actually qemu and virtiofsd) with administrative privileges (e.g. sudo). This is mandated by virtio-fs, at least for the time being.

Use virtio-fs as non-root file system

# Build your image as usual, no extra requirements.
# Run. This will mount the host directory at /path/to/host/dir to /virtiofs in
# the guest.
sudo ./scripts/run.py --mount-fs=virtiofs,/dev/virtiofs0,/virtiofs --virtio-fs-tag=myfs --virtio-fs-dir=/path/to/host/dir
# Or if virtiofsd not present in PATH
sudo PATH=/usr/libexec:$PATH ./scripts/run.py --mount-fs=virtiofs,/dev/virtiofs0,/virtiofs --virtio-fs-tag=myfs --virtio-fs-dir=/path/to/host/dir

Use virtio-fs as the root file system

# Build. This will build an image off the cli module, exporting the whole guest
# file system to build/export (the actual image does *not* contain the file
# system).
scripts/build fs=virtiofs export=all image=cli
# Run. No need to specify the file system type, the build step takes care of it.
sudo ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export

ARM (AArch64)

Please note that virtio-fs is also supported on AArch64 in both TCG (emulated mode on Intel) or with KVM on native ARM hardware. Here is how one can build and run a monitoring app:

./scripts/build -j$(nproc) fs=virtiofs export=all image=httpserver-monitoring-api.fg arch=aarch64
sudo PATH=/usr/libexec:$PATH ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export --api --arch=aarch64 -c 1

On native hardware, one does not need to specify the arch parameter. Please note that the DAX support (see below) does not work using the version of QEMU specified below on ARM.

Use DAX

QEMU

Since DAX support is not yet in stable QEMU, one currently needs the development version, virtio-fs QEMU (including the corresponding virtiofsd). A slightly outdated yet useful how-to is here. After building, make sure to place the resulting virtiofsd in the $PATH.

You can also use the script provided by OSv to build QEMU like so:

./scripts/download_and_build_qemu.sh virtio-fs-dev https://gitlab.com/virtio-fs/qemu.git

OSv

As mentioned, OSv already supports the DAX window. To use it, one only needs to alter how to run it (building the image is not concerned with the DAX window):

  • Use the QEMU built previously, using the --qemu-path option to run.py.
  • Modify the QEMU command line, using the --pass-args option to run.py.
# We need to specify the guest's memory size in the virtio-fs device options.
GUEST_MEM=2G
sudo ./scripts/run.py \
    -m ${GUEST_MEM} \
    --qemu-path=/path/to/built/qemu \
    --virtio-fs-dir=/path/to/host/dir \
    --virtio-fs-dax=1024M

You might also add the location of virtiofsd executable to the PATH.

References

Add a custom footer
Add a custom sidebar
Clone this wiki locally