amazon-cloudwatch-agent: let users specify configuration file paths

[commiterate]

Let users specify configuration file paths.

This is to allow dynamic configuration at runtime (mostly during boot since the systemd unit won't restart unless something runs nixos-rebuild switch/test).

Without this feature, users creating NixOS VM images need to create 1 image per agent configuration permutation if they want immutable images (i.e. don't run nixos-rebuild switch on boot).

This can result in a lot of near-identical images whose only difference is a few options in their CloudWatch Agent configuration (e.g. region option for multi-region AWS deployments). Since VM images are usually on the order of GBs, we end up with a lot of wasted space in Nix stores.

With this feature, users can write their own systemd units that dynamically create configurations (e.g. using information from IMDS, AWS SSM Parameter Store, or AWS Secrets Manager) and write them to a static path.

The tradeoff, however, is that mutable configuration files can cause drift if a change isn't followed by a systemctl restart amazon-cloudwatch-agent.service. nixos-rebuild won't work because we can't access files outside the Nix store in restricted evaluation mode (e.g. can't hash the file and feed it into restartTriggers).

Also doing some small cleanup.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

Maintainer Pings

@philipmw

[philipmw]

This makes sense to me, but I am concerned about someone specifying both a file and an inline config and having the inline config silently ignored. (I do see you document that the file takes precedence.) Do you think we should error out if both are specified?

[commiterate]

This makes sense to me, but I am concerned about someone specifying both a file and an inline config and having the inline config silently ignored. (I do see you document that the file takes precedence.) Do you think we should error out if both are specified?

I was imagining someone might define configuration as a default in some shareable NixOS config. Consumers can then override by redefining configuration or just using configurationFile. In the latter case, erroring out creates bad UX.

It looks like several other modules which offer both attribute set and path options for configurations (e.g. services.gitlab-runner, virtualisation.containerd) also make the path option take precedence over the attribute set rather than failing builds.

[commiterate]

I guess the other option is to do:

{
  lib,
  pkgs,
  config,
  ...
}:
let
  cfg = config.services.amazon-cloudwatch-agent;

  jsonFormat = pkgs.formats.json { };

  configurationFile =
    if (builtins.typeOf configuration == "path") then
      cfg.configuration
    else
      (jsonFormat.generate "amazon-cloudwatch-agent.json" cfg.configuration);

  # ...
in
{
  options.services.amazon-cloudwatch-agent = {
    configuration = lib.mkOption {
      type = lib.types.oneOf [
        lib.types.path
        jsonFormat.type
      ];
      default = { };
    };
  };
}

Not actually sure if there's another module that does this. Didn't see one from a cursory search.

https://github.com/search?q=repo%3ANixOS%2Fnixpkgs+oneOf+path+json+path%3A%2F%5Enixos%5C%2Fmodules%5C%2Fservices%5C%2F%2F&type=code

This isn't as nice IMO because the example option in the lib.mkOption function would only be one of the values, so the documentation won't have examples of both types.

It also gets messy in the future if the configuration format is in some transitional period. Fluent Bit has this problem since they have a legacy classic .conf format (this is custom) as well as the current YAML format. Having a type like configuration: jsonFormat.type | yamlFormat.type might make it impossible to know which attribute set serializer to use without some type discriminator (might be another field like configurationFormat).

Separating out each type a configuration might be (path, JSON, YAML, TOML) is probably going to be better on a documentation and dispatch level.

The most spartan approach would be to just drop any quality of life module options that serialize attribute sets and only support paths. That way, people need to supply their own derivations that produce a configuration file. This feels a bit heavy handed though and isn't the way most modules are designed.

[philipmw]

Sounds good to me; thanks for the thoughtful reply. I approve, though (for some reason) my approval isn't requested.

[commiterate]

Not sure if OfBorg waits until all checks pass before it requests maintainers for review (RIP aarch64-darwin) or some part of the maintainer metadata isn't quite right.

[commiterate]

Hmm since all checks passed and it still didn't request a review, might be because the input derivation for the package didn't change and NixOS modules don't actually care about the meta attribute (or at least OfBorg doesn't check it).

[arianvp]

you can use codeowners instead for modules. it's indeed not supported

[commiterate]

Apparently fetchFromGitHub now accepts a tag attribute. Going to swap to that.