-
Notifications
You must be signed in to change notification settings - Fork 2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: Add initial nomad-driver-virt driver plugin documentation. (#24094
) Co-authored-by: Aimee Ukasick <[email protected]>
- Loading branch information
Showing
7 changed files
with
395 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
~> Beta Feature: The Virt task driver is in beta and under active development. | ||
Refer to the [repo README](https://github.com/hashicorp/nomad-driver-virt) for | ||
the latest updates. Do not use this driver in production environments. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
--- | ||
layout: docs | ||
page_title: Client | ||
description: Nomad client requirements, capabilities, and configuration options for the Virt task driver. | ||
--- | ||
|
||
# Client | ||
|
||
@include 'virt-beta-callout.mdx' | ||
|
||
The `virt` task driver is not built into Nomad. You must [downloaded][releases] onto the client | ||
host in the configured plugin directory. The client host must also be capable of running hardware | ||
accelerated KVM virtual machines, which you can check by using [kvm-ok][] command. | ||
Additional requirements: | ||
|
||
- Linux | ||
- Commands `bridge-utils`, `cloudinit`, `dnsmasq`, and `iptables` | ||
- [libvirt daemon][] installed and running | ||
- QEMU system packages installed | ||
- Nomad client running as root | ||
|
||
Use the following command to install the package requirements: | ||
|
||
```console | ||
$ sudo apt-get update && \ | ||
sudo apt-get install -y \ | ||
bridge-utils \ | ||
cloud-init \ | ||
dnsmasq \ | ||
iptables \ | ||
libvirt-daemon-system \ | ||
qemu-system \ | ||
qemu-system-x86 \ | ||
qemu-utils | ||
``` | ||
|
||
## Client capabilities | ||
|
||
The `virt` task driver implements the following driver [capabilities][capabilities]. | ||
|
||
| Feature | Implementation | | ||
|----------------------|----------------| | ||
| `nomad alloc signal` | false | | ||
| `nomad alloc exec` | false | | ||
| filesystem isolation | image | | ||
| network isolation | host | | ||
| volume mounting | false | | ||
|
||
## Plugin configuration | ||
|
||
This example shows the default plugin configuration. | ||
|
||
```hcl | ||
plugin "nomad-driver-driver" { | ||
config { | ||
data_dir = "/var/lib/virt" | ||
image_paths = ["/var/lib/virt", "${ALLOC_DIR}"] | ||
emulator { | ||
uri = "qemu:///system" | ||
user = "" | ||
password = "" | ||
} | ||
} | ||
} | ||
``` | ||
|
||
- `data_dir` - (default: `"/var/lib/virt"`) - the directory for storing VM configuration files and | ||
intermediate files. | ||
|
||
- `image_paths` - (default: `["/var/lib/virt", "${ALLOC_DIR}"]`) - an allow-list of paths the driver | ||
is allowed to load an image from. | ||
|
||
- `emulator` - (block) | ||
- `uri` - (default: `"qemu:///system"`) - The `uri` specifies which hypervisor to connect to. | ||
- `user` - (default: `""`) - the username to use for authentication. | ||
- `password` - (default: `""`) - the password to use for authentication. | ||
|
||
## Client attributes | ||
|
||
When installed, the `virt` plugin provides the following node attributes that you can use as | ||
constraints when authoring jobs. | ||
|
||
``` | ||
driver.virt = true | ||
driver.virt.active = 0 | ||
driver.virt.emulator.version = 8002002 | ||
driver.virt.inactive = 0bytes | ||
driver.virt.libvirt.version = 10000000 | ||
driver.virt.network.default.bridge_name = virbr0 | ||
driver.virt.network.default.state = active | ||
``` | ||
|
||
[releases]: https://releases.hashicorp.com/nomad-driver-virt | ||
[kvm-ok]: https://manpages.ubuntu.com/manpages/bionic/man1/kvm-ok.1.html | ||
[libvirt daemon]: https://libvirt.org/daemons.html | ||
[capabilities]: https://developer.hashicorp.com/nomad/docs/concepts/plugins/task-drivers#task-driver-plugin-api |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
--- | ||
layout: docs | ||
page_title: 'Drivers: virt' | ||
description: >- | ||
The virt task driver leverages Linux capabilities and libvirt to run virtual | ||
machines. | ||
--- | ||
|
||
# Virt task driver | ||
|
||
@include 'virt-beta-callout.mdx' | ||
|
||
Name: `virt` | ||
|
||
The `virt` driver utilises the [libvirt][] API to run and manage virtual machines (VMs) via | ||
hypervisors such as [QEMU][]. It supports a wide variety of Nomad features including | ||
[workload identity][], [task templates][], and [service discovery][]. | ||
|
||
|
||
[libvirt]: https://libvirt.org/ | ||
[QEMU]: https://www.qemu.org/ | ||
[workload identity]: https://developer.hashicorp.com/nomad/docs/concepts/workload-identity | ||
[task templates]: https://developer.hashicorp.com/nomad/docs/job-specification/template | ||
[service discovery]: https://developer.hashicorp.com/nomad/docs/networking/service-discovery | ||
[github]: https://github.com/hashicorp/nomad-driver-virt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
--- | ||
layout: docs | ||
page_title: Installation | ||
description: Installation of the virt task driver. | ||
--- | ||
|
||
# Installation | ||
|
||
@include 'virt-beta-callout.mdx' | ||
|
||
Use the following instructions to download the task driver for use on your Nomad client host. | ||
|
||
<Tabs> | ||
<Tab heading="Manual installation" group="manual"> | ||
|
||
Download a [precompiled binary](https://releases.hashicorp.com/nomad-driver-virt/) and | ||
verify the binary using the available SHA-256 sums. Unzip the package. Make sure that the `nomad-driver-virt` binary is on your | ||
[plugin_dir](/nomad/docs/configuration#plugin_dir) path, specified by the client's config file, | ||
before continuing with the other guides. | ||
|
||
</Tab> | ||
<Tab heading="Ubuntu/Debian" group="manual"> | ||
|
||
Install the required packages. | ||
|
||
```shell-session | ||
$ sudo apt-get update && \ | ||
sudo apt-get install wget gpg coreutils | ||
``` | ||
|
||
Add the HashiCorp [GPG key][gpg-key]. | ||
|
||
```shell-session | ||
$ wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg | ||
``` | ||
|
||
Add the official HashiCorp Linux test repository. | ||
|
||
```shell-session | ||
$ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) test" | sudo tee /etc/apt/sources.list.d/hashicorp.list | ||
``` | ||
|
||
Update and install. | ||
|
||
```shell-session | ||
$ sudo apt-get update && sudo apt-get install nomad-driver-virt | ||
``` | ||
|
||
</Tab> | ||
<Tab heading="CentOS/RHEL" group="linux"> | ||
|
||
Install `yum-config-manager` to manage your repositories. | ||
|
||
```shell-session | ||
$ sudo yum install -y yum-utils | ||
``` | ||
|
||
Use `yum-config-manager` to add the official HashiCorp Linux repository. | ||
|
||
```shell-session | ||
$ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo | ||
``` | ||
|
||
Edit the repo file at `/etc/yum.repos.d/hashicorp.repo` and set `enabled=1` for `[hashicorp-test]`. | ||
|
||
Install. | ||
|
||
```shell-session | ||
$ sudo yum -y install nomad-driver-virt | ||
``` | ||
|
||
</Tab> | ||
</Tabs> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,155 @@ | ||
--- | ||
layout: docs | ||
page_title: Task Config | ||
description: Configure the virt task driver job specification parameters. | ||
--- | ||
|
||
# Task configuration | ||
|
||
@include 'virt-beta-callout.mdx' | ||
|
||
Task configuration for an virt task includes setting an `image` to provide the base VM image as | ||
well as other parameters required for bootstrapping a VM instance. | ||
|
||
- `image` - (string: required) - Path to an `.img` cloud image to base the VM disk on. The image should | ||
be located in an allowed path. The cloud image needs to include cloud-init, otherwise certain | ||
task features are not available. You may also downloaded via the [artifact block][]. | ||
|
||
- `use_thin_copy` - (bool: `false`) - Make a thin copy of the image using QEMU, and use it as the | ||
backing cloud image for the VM. | ||
|
||
- `hostname` - (string: `"nomad-<task-name>"`) - The hostname to assign to the VM. As this value is | ||
used as a network host name, it must be a valid DNS label according to RFC 1123. Nomad generates the default | ||
value using the Nomad task name. | ||
|
||
- `cmds` - (list(string): optional) - List of commands to execute on the VM once it is running. | ||
These commands can provide you with a quick way to start a process on the newly created | ||
VM. When used in conjunction with the template, this command list can be a powerful start up tool. | ||
|
||
- `default_user_password` - (string: `""`) - Initial password for the default user | ||
on the newly created VM. When set, the user must update the password on first connect. | ||
|
||
- `default_user_authorized_ssh_key` - (string: `""`) - SSH public key added to the SSH | ||
configuration for the default user of the cloud image distribution. | ||
|
||
- `user_data ` - (string: `""`) - Path to a cloud-init compliant user data file used as the user-data for the cloud-init configuration. | ||
|
||
- `primary_disk_size` - (int: required) - Disk space to assign to the VM. Must fit the VM's OS. | ||
|
||
Example: | ||
```hcl | ||
config { | ||
image = "local/focal-server-cloudimg-amd64.img" | ||
use_thing_copy = false | ||
hostname = "" | ||
cmds = ["python -m http.server 8000"] | ||
#default_user_password = "CHANGE-ME" | ||
default_user_authorized_ssh_key = "" | ||
user_data = "" | ||
primary_disk_size = 10000 | ||
} | ||
``` | ||
|
||
##### os | ||
|
||
The VM architecture and machine can be different from the client host. When this is the case, you must set the | ||
`os` block. By default, this information matches the client host where the driver | ||
plugin is running. | ||
|
||
- `arch` - (string: `""`) - The VM architecture. | ||
|
||
- `machine` - (string: `""`) - The machine type. | ||
|
||
Example: | ||
```hcl | ||
config { | ||
os { | ||
arch = "x86_64" | ||
machine = "pc-i440fx-2.9" | ||
} | ||
} | ||
``` | ||
|
||
##### network_interface | ||
|
||
You need to attach tasks to a libvirt network in order to access the internet and route | ||
from the local network. The `network_interface` block enables this as a list of network interfaces | ||
that the driver should attach to the VM. This feature only supports a single entry. | ||
|
||
- `bridge` - (block) - Attach the VM to a bridge network. `virbr0`, the default libvirt | ||
network, is a bridge network. | ||
|
||
- `name` - The name of the bridge interface to use. This relates to the output from commands | ||
such as `virsh net-info`. | ||
|
||
- `ports` - A list of port labels to expose on the host via mapping to the network | ||
interface. These labels must exist within the job specification [network block][]. | ||
|
||
Example: | ||
```hcl | ||
config { | ||
network_interface { | ||
bridge { | ||
name = "virbr0" | ||
ports = ["ssh", "http"] | ||
} | ||
} | ||
} | ||
``` | ||
|
||
### Full example | ||
|
||
The following job specification displays use of all the configuration options, as well as other | ||
blocks that are useful when deploying tasks via the virt driver. You need to modify parameters, such as the | ||
`default_user_authorized_ssh_key`, before use depending on your | ||
requirements and Nomad setup. | ||
|
||
```hcl | ||
job "virt-example" { | ||
group "virt-group" { | ||
network { | ||
mode = "host" | ||
provider = "nomad" | ||
port "ssh" { | ||
to = 22 | ||
} | ||
} | ||
task "virt-task" { | ||
driver = "nomad-driver-virt" | ||
artifact { | ||
source = "http://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img" | ||
destination = "local/focal-server-cloudimg-amd64.img" | ||
mode = "file" | ||
} | ||
config { | ||
image = "local/focal-server-cloudimg-amd64.img" | ||
primary_disk_size = 26000 | ||
#default_user_password = "CHANGE-ME" | ||
default_user_authorized_ssh_key = "ssh-ed25519 AAAAC3NzaC1lZDI..." | ||
network_interface { | ||
bridge { | ||
name = "virbr0" | ||
ports = ["ssh"] | ||
} | ||
} | ||
} | ||
resources { | ||
cores = 2 | ||
memory = 4096 | ||
} | ||
} | ||
} | ||
} | ||
``` | ||
|
||
[artifact block]: https://developer.hashicorp.com/nomad/docs/job-specification/artifact | ||
[network block]: https://developer.hashicorp.com/nomad/docs/job-specification/network |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.