add noswap to secretdir tmpfs

[chuckyz]

Description

This adds noswap mount option to the SecretDir mount.

Testing & Reproduction steps

• Create an allocation that triggers a secret dir creation

(having a hell of a time trying to get tests to run locally so I'm going to rely on GHA to run unit tests (fingers crossed))

Links

todo issue

Contributor Checklist

• Changelog Entry If this PR changes user-facing behavior, please generate and add a
  changelog entry using the make cl command.
• Documentation If the change impacts user-facing functionality such as the CLI, API, UI,
  and job configuration, please update the Nomad website documentation to reflect this. Refer to
  the website README for docs guidelines. Please also consider whether the
  change requires notes within the upgrade guide.

Reviewer Checklist

• Backport Labels Please add the correct backport labels as described by the internal
  backporting document.
• Commit Type Ensure the correct merge method is selected which should be "squash and merge"
  in the majority of situations. The main exceptions are long-lived feature branches or merges where
  history should be preserved.

[tgross]

Hi @chuckyz, nice to hear from you!

This is a great idea, but the noswap option was only introduced in Linux 6.4 and so we can't apply it blindly (many users have older kernels). For example, on Ubuntu 22.04:

$ uname -r
5.15.0-126-generic
$ mkdir /home/tim/secrets
$ sudo mount -t tmpfs -o size=100M,noswap tmpfs /home/tim/secrets
mount: /home/tim/secrets: wrong fs type, bad option, bad superblock on tmpfs, missing codepage or helper program, or other error.

$ sudo mount -t tmpfs -o size=100M tmpfs /home/tim/secrets
$ mount | grep secrets
tmpfs on /home/tim/secrets type tmpfs (rw,relatime,size=102400k,inode64)

This presents as follows in a build from this PR:

Recent Events:
Time                       Type              Description
2024-12-11T08:49:39-05:00  Not Restarting    Error was unrecoverable
2024-12-11T08:49:39-05:00  Task hook failed  task_dir: mount: invalid argument

But I'd love to find a way to get this in because it's a good incremental improvement to security. We just need a way to test for this option at runtime (maybe try with swap and then fallback to not using it? I'd like to avoid checking the kernel version directly because the various Frankenkernels out there make that less useful).

Also, once we've got that figured out can you run make cl to add a changelog item? Thanks!

[chuckyz]

Hey @tgross that sounds great.

Another option would potentially be ramfs, however then we'd (for safety/reliability) probably need to setup the bind-mount as read only (so folks don't accidentally, like, write a log into /secret or something and OOM a system) and it'd probably make the code base a bit tougher as ideally there'd be an option for tmpfs, or ramfs.

I personally strongly prefer the idea of just sticking with tmpfs.

I was going to call syscall.Uname, and parse out >= 6.4, sound good?

[tgross]

I was going to call syscall.Uname, and parse out >= 6.4, sound good?

If we do that, we won't be applying it on RHEL9, where the kernel version is 5.14 but they've apparently backported the noswap option (in a security patch, if you can believe that! This is what I was obliquely referring to as "frankenkernel"). So that's why I was suggesting we try with the noswap option and catch the error and retry if that fails.

[chuckyz]

Edit: I'm calling syscall.Mount twice 🤦

Gave this a shot, got tests mostly working locally. Would you consider maybe accepting syscall.MNT_DETACH in the unlink calls or sprinkling around time.Sleep() to make tests more consistent locally? Not sure why but I'm really having a hard time actually un-mounting during cleanup phases of tests.

Also I got fancy with this solution and then realized KISS and tried to make this way simpler (catch error once, then try again before returning a real error).

For reference I'm running tests on a Macbook M3 with the following VM setup managed by Orbstack using go 1.23.4:

Linux ubuntu 6.9.8-orbstack-00170-g7b4100b7ced4 #1 SMP Thu Jul 11 03:32:20 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

[tgross]

Looks great @chuckyz! One last item: can you run make cl to add a changelog entry. Something like "client: add noswap directive to secrets directory where supported on Linux"

[chuckyz]

Should be good to go!