Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOCS: CE-661 Refactor CNI plugin content #23707

Merged
merged 15 commits into from
Aug 6, 2024
Merged
328 changes: 90 additions & 238 deletions website/content/docs/install/index.mdx

Large diffs are not rendered by default.

47 changes: 2 additions & 45 deletions website/content/docs/install/production/requirements.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -103,52 +103,11 @@ $ echo "49152 65535" > /proc/sys/net/ipv4/ip_local_port_range

## Bridge Networking and `iptables`

Nomad's task group networks and Consul Connect integration use bridge networking and iptables to send traffic between containers. The Linux kernel bridge module has three "tunables" that control whether traffic crossing the bridge are processed by iptables. Some operating systems (RedHat, CentOS, and Fedora in particular) configure these tunables to optimize for VM workloads where iptables rules might not be correctly configured for guest traffic.

These tunables can be set to allow iptables processing for the bridge network as follows:

```shell-session
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-arptables
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
```

To preserve these settings on startup of a client node, add a file including the following to `/etc/sysctl.d/` or remove the file your Linux distribution puts in that directory.

```text
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
```
@include 'install/bridge-iptables.mdx'

## Cgroup Controllers

On Linux, Nomad uses cgroups to control access to resources like CPU and
memory. Nomad can support both [cgroups v2][] and the legacy [cgroups
v1][]. When Nomad clients start, they will determine the available cgroup
controllers and include the attribute `os.cgroups.version` in their
fingerprint.

Nomad can only use cgroups to control resources if all the required controllers
are available. If one or more required cgroups are unavailable, Nomad will
disable resource controls that require cgroups entirely. You will most often see
missing controllers on platforms used outside of datacenters, such as Raspberry
Pi or similar hobbyist computers.

On cgroups v2, you can verify that you have all required controllers as follows:

```shell-session
$ cat /sys/fs/cgroup/cgroup.controllers
cpuset cpu io memory pids
```

On legacy cgroups v1, you can look for this same list of required controllers as
directories under the directory `/sys/fs/cgroup`.

To enable missing cgroups, add the appropriate boot command line arguments. For
example, to enable the `cpuset` cgroup, you'll need to add `cgroup_cpuset=1
cgroup_enable=cpuset`. These arguments should be added wherever specified by
your bootloader.
@include 'install/cgroup-controllers.mdx'

## Hardening Nomad

Expand Down Expand Up @@ -260,5 +219,3 @@ in automated pipelines for [CLI operations][docs_cli], such as
[`nomad fmt`]: /nomad/docs/commands/fmt
[mTLS]: /nomad/tutorials/transport-security/security-enable-tls
[ephemeral disk migration]: /nomad/docs/job-specification/ephemeral_disk#migrat
[cgroups v1]: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/cgroups.html
[cgroups v2]: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html
204 changes: 76 additions & 128 deletions website/content/docs/networking/cni.mdx

Large diffs are not rendered by default.

217 changes: 73 additions & 144 deletions website/content/docs/networking/index.mdx

Large diffs are not rendered by default.

22 changes: 22 additions & 0 deletions website/content/partials/install/bridge-iptables.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
Nomad's task group networks and Consul Connect integration use bridge networking and iptables to send traffic between containers. The Linux kernel bridge module has three "tunables" that control whether iptables process traffic crossing the bridge. Some operating systems (RedHat, CentOS, and Fedora in particular) configure these tunables to optimize for VM workloads where iptables rules might not be correctly configured for guest traffic.

Ensure your Linux operating system distribution has been configured to allow
iptables to route container traffic through the bridge network. Set these tunables to allow iptables processing for the bridge network.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved

```shell-session
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-arptables
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
```

To preserve these settings on startup of a client node, add a file including the following to `/etc/sysctl.d/` or remove the file your Linux distribution puts in that directory.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved

<CodeBlockConfig filename="/etc/sysctl.d/bridge.conf">

```ini
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
```

</CodeBlockConfig>
20 changes: 20 additions & 0 deletions website/content/partials/install/cgroup-controllers.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
On Linux, Nomad uses cgroups to control access to resources like CPU and
memory. Nomad supports both [cgroups v2](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html) and the legacy [cgroups
v1](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/cgroups.html). When Nomad clients start, they determine the available cgroup controllers and include the attribute `os.cgroups.version` in their fingerprint.

Nomad can only use cgroups to control resources if all the required controllers
are available. If one or more required cgroups are not available, Nomad disables resource controls that require cgroups entirely. You most often see missing controllers on platforms used outside of datacenters, such as Raspberry Pi or similar hobbyist computers.

On cgroups v2, you can verify that you have all required controllers.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved

```shell-session
$ cat /sys/fs/cgroup/cgroup.controllers
cpuset cpu io memory pids
```

On legacy cgroups v1, look for this same list of required controllers as directories under the directory `/sys/fs/cgroup`.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved

To enable missing cgroups, add the appropriate boot command line arguments. For
example, to enable the `cpuset` cgroup, add `cgroup_cpuset=1 cgroup_enable=cpuset`. Add these arguments wherever specified by your bootloader.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved

Refer to the [cgroup controller requirements](/nomad/docs/install/production/requirements#hardening-nomad) for more details.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved
aimeeu marked this conversation as resolved.
Show resolved Hide resolved
12 changes: 12 additions & 0 deletions website/content/partials/install/install-cni-plugins.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Nomad uses CNI plugins to configure network namespaces when using the `bridge`
network mode. You must install the CNI plugins on all Linux Nomad client nodes using network namespaces. Refer to the [CNI Plugins Overview guide](https://www.cni.dev/plugins/current/) for details on individual plugins.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved

The following determines your operating system architecture, downloads the appropriate [1.5.1 release](https://github.com/containernetworking/plugins/releases/tag/v1.5.1), and then extracts the CNI plugin binaries into the `/opt/cni/bin` directory.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved


```shell-session
$ export ARCH_CNI=$( [ $(uname -m) = aarch64 ] && echo arm64 || echo amd64)
$ curl -L -o cni-plugins.tgz "https://github.com/containernetworking/plugins/releases/download/v1.5.1/cni-plugins-linux-${ARCH_CNI}"-v1.5.1.tgz && \
sudo mkdir -p /opt/cni/bin && \
sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz
aimeeu marked this conversation as resolved.
Show resolved Hide resolved
```
46 changes: 46 additions & 0 deletions website/content/partials/install/install-consul-cni-plugin.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
If you use the `transparent_proxy` block for Consul Service Mesh, then you must install the
`consul-cni` plugin on each client node to redirect inbound and outbound traffic for services to the Envoy proxy.

Install the CNI plugins before you install the Consul CNI plugin. The following commands assume that you have installed the CNI plugins.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved

Install the `consul-cni` plugin on each client node.

<Tabs>
<Tab heading="Ubuntu/Debian" group="ubuntu">

```shell-session
$ sudo apt-get install -y consul-cni
```

</Tab>
<Tab heading="RHEL/CentOS" group="rhel">

```shell-session
$ sudo yum -y install consul-cni
```

</Tab>
<Tab heading="Fedora" group="fedora">

```shell-session
$ sudo dnf -y install consul-cni
```

</Tab>
<Tab heading="Amazon Linux" group="amazonlinux">

```shell-session
$ sudo yum -y install consul-cni
```

</Tab>
<Tab heading="Manual" group="linux-manual">

```shell-session
$ export ARCH_CNI=$( [ $(uname -m) = aarch64 ] && echo arm64 || echo amd64)
$ curl -L -o consul-cni.zip "https://releases.hashicorp.com/consul-cni/1.5.1/consul-cni_1.5.1_linux_${ARCH_CNI}".zip && \
sudo unzip consul-cni.zip -d /opt/cni/bin -x LICENSE.txt
```

</Tab>
</Tabs>
2 changes: 2 additions & 0 deletions website/content/partials/install/manual-install.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

Download a [precompiled binary](https://releases.hashicorp.com/nomad/) and run it on your machine locally. Verify the binary using the available SHA-256 sums. After downloading Nomad, unzip the package. Make sure that the `nomad` binary is available on your `PATH` before continuing with the other guides.
aimeeu marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions website/content/partials/install/manual-path.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Permanently add a new location to your path by editing your shell's settings file, which is usually called something like `~/.bashrc`, where the part of the filename after the `.` and before `rc` is the name of your shell. In that file you should see a line that starts with `export PATH=`, followed by a colon-separated list of locations. Add the location of the Nomad binary to that list and save the file. Then reload your shell's configuration with the command `source ~/.bashrc`, replacing `bash` with the name of your shell.