UI: ACL permissions enabled by the 'Run Job' button

[Great-Stone]

Proposal

In Nomad 1.6.1, I have an ACL with permission to run a job for a specific Namespace, but the Run Job button is disabled in the UI. The CLI and API work fine.

Use-cases

If we enable Nomad with different ACL permissions for each user, we can validate and run jobs on the UI based on the user's permissions.

Attempted Solutions

Attempted ACLs

namespace "*" {
  policy = "write"
  capabilities = [
    "list-jobs",
    "parse-job",
    "read-job",
    "submit-job",
    "dispatch-job",
    "read-logs",
    "read-fs",
    "alloc-exec",
    "alloc-lifecycle",
    "csi-write-volume",
    "csi-mount-volume",
    "list-scaling-policies",
    "read-scaling-policy",
    "read-job-scaling",
    "scale-job"
  ]
  variables {
    path "*" {
      capabilities = ["read"]
    }
  }
}

[jrasell]

Hi @Great-Stone, would you be able to provide more details on your configuration and how to reproduce this? I have been unable to reproduce this locally on either 1.6.1 or 1.6.2-dev.

I ran a local Nomad agent using the dev mode with ACL's enabled and bootstrapped. I then created an ACL policy using the specification you posted and created an ACL token linked to this policy. I took the created ACL token and used this within the Nomad UI to sign in. When navigating to the jobs page, I can see the run button clearly present and am able to click it and trigger job runs. I also tested this in a different namespace, seeing as the policy you linked uses the wildcard namespace identifier.

[Great-Stone]

Hi @jrasell , I am late in replying. The Nomad version at the time was 1.6.1-ent. The only thing to add is that it happened when I associated the OIDC added in Nomad with the OIDC Provider in Vault. What I didn't check with Policy and Token alone is what I missed. I'll do a double check on that as well.

[Great-Stone]

I see this issue as related: #18382

[Great-Stone]

Follow the steps below to reproduce. As confirmed earlier, if I create a Policy and follow the instructions to create a Token directly, I have no problem either.

Run Vault - for OIDC provider

vault server -dev -dev-root-token-id=root

Run Nomad

nomad agent -server -dev -acl-enabled

Get Nomad Bootstrap Token

NOMAD_ADDR=http://localhost:4646
nomad acl bootstrap

Setup via Terraform

Don't forget to put the nomad token in locals.

///////////////
// Local Setup
locals {
  oidc_user   = toset(["ops", "dev"])
  vault_addr  = "http://localhost:8200"
  nomad_addr  = "http://localhost:4646"
  nomad_token = "<CHANGE_ME>"
}

//////////////////////////////
// Vault
//////////////////////////////
provider "vault" {
  address = local.vault_addr
  token   = "root"
}

resource "vault_auth_backend" "userpass" {
  type = "userpass"
  path = "userpass-nomad"
}

resource "vault_generic_endpoint" "user" {
  for_each = local.oidc_user

  path                 = "auth/${vault_auth_backend.userpass.path}/users/${each.key}"
  ignore_absent_fields = true
  data_json            = <<EOT
{
  "policies": ["default"],
  "password": "${each.key}"
}
EOT
}

resource "vault_identity_entity" "user" {
  for_each = local.oidc_user

  name     = each.key
  policies = ["default"]
  disabled = false
}

resource "vault_identity_group" "group" {
  for_each = local.oidc_user

  name              = "${each.key}group"
  type              = "internal"
  policies          = ["default"]
  member_entity_ids = [vault_identity_entity.user[each.key].id]
}

resource "vault_identity_entity_alias" "alias" {
  for_each = local.oidc_user

  name           = each.key
  mount_accessor = vault_auth_backend.userpass.accessor
  canonical_id   = vault_identity_entity.user[each.key].id
}

// Create a Vault OIDC client
resource "vault_identity_oidc_assignment" "nomad" {
  name = "nomad-assignment"
  entity_ids = [
    for entity in vault_identity_entity.user : entity.id
  ]
  group_ids = [
    for group in vault_identity_group.group : group.id
  ]
}

resource "vault_identity_oidc_key" "nomad" {
  name               = "nomad-key"
  algorithm          = "RS256"
  rotation_period    = 3600
  verification_ttl   = 7200
  allowed_client_ids = ["*"]
}

////////////
// localhost:10088 for tester
// https://medium.com/application-security/openid-connect-tester-for-developers-63427324faea
// https://gitlab.com/guenoledc-perso/idp-oidc-tester

resource "vault_identity_oidc_client" "nomad" {
  name = "nomad"
  redirect_uris = [
    "${local.nomad_addr}/oidc/callback",
    "${local.nomad_addr}/ui/settings/tokens",
    "http://localhost:4649/oidc/callback",
  ]
  assignments = [
    // "allow_all",
    vault_identity_oidc_assignment.nomad.name
  ]
  key              = vault_identity_oidc_key.nomad.name
  id_token_ttl     = 1800
  access_token_ttl = 3600
}

resource "vault_identity_oidc_scope" "user" {
  name     = "nuser"
  template = <<-EOF
  {
    "username": {{identity.entity.aliases.${vault_auth_backend.userpass.accessor}.name}},
    "contact": {
        "email": {{identity.entity.metadata.email}},
        "phone_number": {{identity.entity.metadata.phone_number}}
    },
    "groups": {{identity.entity.groups.names}}
  }
  EOF
  // "{\"username\":{{identity.entity.name}}}"

  description = "The user scope provides claims using Vault identity entity metadata"
}

resource "vault_identity_oidc_provider" "nomad" {
  name          = "nomad-oidc-provider"
  https_enabled = false
  issuer_host   = replace(local.vault_addr, "http://", "")
  allowed_client_ids = [
    vault_identity_oidc_client.nomad.client_id
  ]
  scopes_supported = [
    vault_identity_oidc_scope.user.name,
  ]
}

// Check a Vault well-known
data "http" "openid_configuration" {
  url = "${vault_identity_oidc_provider.nomad.issuer}/.well-known/openid-configuration"

  request_headers = {
    Accept = "application/json"
  }
}

data "http" "keys" {
  url = "${vault_identity_oidc_provider.nomad.issuer}/.well-known/keys"

  request_headers = {
    Accept = "application/json"
  }
}


//////////////////////////////
// Nomad
//////////////////////////////
provider "nomad" {
  address   = local.nomad_addr
  region    = "global"
  secret_id = local.nomad_token
}

resource "nomad_namespace" "ops" {
  name        = "ops"
  description = "Shared operation environment."
  meta = {
    owner = "Ops"
  }
}

resource "nomad_namespace" "dev" {
  name        = "dev"
  description = "Shared development environment."
  capabilities {
    enabled_task_drivers  = ["exec", "java", "docker"]
    disabled_task_drivers = ["raw_exec"]
  }
  meta = {
    owner = "Dev"
  }
}

resource "nomad_acl_policy" "ops" {
  name = "ops"

  rules_hcl = <<EOT
namespace "*" {
  policy = "write"
  capabilities = [
    "list-jobs",
    "parse-job",
    "read-job",
    "submit-job",
    "dispatch-job",
    "read-logs",
    "read-fs",
    "alloc-exec",
    "alloc-lifecycle",
    "csi-write-volume",
    "csi-mount-volume",
    "list-scaling-policies",
    "read-scaling-policy",
    "read-job-scaling",
    "scale-job"
  ]
  variables {
    path "dev" {
      capabilities = ["read"]
    }
  }
}
EOT
}

resource "nomad_acl_policy" "dev" {
  name = "dev"

  rules_hcl = <<EOT
namespace "dev" {
  policy = "write"
  capabilities = [
    "list-jobs",
    "parse-job",
    "read-job",
    "submit-job",
    "dispatch-job",
    "read-logs",
    "read-fs",
    "alloc-exec",
    "alloc-lifecycle",
    "csi-write-volume",
    "csi-mount-volume",
    "list-scaling-policies",
    "read-scaling-policy",
    "read-job-scaling",
    "scale-job"
  ]
  variables {
    path "dev" {
      capabilities = ["read"]
    }
  }
}
EOT
}

resource "nomad_acl_role" "ops" {
  name        = "ops-acl-role"
  description = "An ACL Role for cluster ops"

  policy {
    name = nomad_acl_policy.ops.name
  }
}

resource "nomad_acl_role" "dev" {
  name        = "dev-acl-role"
  description = "An ACL Role for cluster dev"

  policy {
    name = nomad_acl_policy.dev.name
  }
}

resource "nomad_acl_auth_method" "vault" {
  name           = "vault"
  type           = "OIDC"
  token_locality = "global"
  max_token_ttl  = "60m0s"
  default        = true

  config {
    oidc_discovery_url = vault_identity_oidc_provider.nomad.issuer
    oidc_client_id     = vault_identity_oidc_client.nomad.client_id
    oidc_client_secret = vault_identity_oidc_client.nomad.client_secret
    bound_audiences = [
      vault_identity_oidc_client.nomad.client_id
    ]
    oidc_scopes = [
      vault_identity_oidc_scope.user.name,
    ]
    allowed_redirect_uris = [
      "${local.nomad_addr}/oidc/callback",
      "${local.nomad_addr}/ui/settings/tokens",
      "http://localhost:4649/oidc/callback",
    ]
    list_claim_mappings = {
      groups = "roles"
    }
  }
}

// resource "nomad_acl_binding_rule" "ops" {
//   description = "ops rule"
//   auth_method = nomad_acl_auth_method.vault.name
//   selector    = "opsgroup in list.roles"
//   bind_type   = "management"
//   bind_name   = ""
// }

resource "nomad_acl_binding_rule" "ops" {
  description = "ops rule"
  auth_method = nomad_acl_auth_method.vault.name
  selector    = "opsgroup in list.roles"
  bind_type   = "role"
  bind_name   = nomad_acl_role.ops.name
}

resource "nomad_acl_binding_rule" "dev" {
  description = "dev rule"
  auth_method = nomad_acl_auth_method.vault.name
  selector    = "devgroup in list.roles"
  bind_type   = "role"
  bind_name   = nomad_acl_role.dev.name
}

and then,

terraform apply -auto-approve

Login via Vault OIDC on Nomad UI

I don't know why Policies is empty

Run button disabled and I can't click

Check the Policy

$ nomad acl token info 0d290704-aa2e-5329-5eda-210b038c70f5

Accessor ID  = 0d290704-aa2e-5329-5eda-210b038c70f5
Secret ID    = 3fb21144-025e-b1e5-a283-8b5917760014
Name         = OIDC-vault
Type         = client
Global       = true
Create Time  = 2023-09-21 23:49:34.029135 +0000 UTC
Expiry Time  = 2023-09-22 00:49:34.029135 +0000 UTC
Create Index = 26
Modify Index = 26
Policies     = []

Roles
ID                                    Name
1ad53db7-9be2-038b-fd06-44e3c9bc7fae  ops-acl-role

$ nomad acl role info 1ad53db7-9be2-038b-fd06-44e3c9bc7fae

ID           = 1ad53db7-9be2-038b-fd06-44e3c9bc7fae
Name         = ops-acl-role
Description  = An ACL Role for cluster ops
Policies     = ops
Create Index = 20
Modify Index = 20

$ nomad acl policy info ops

Name        = ops
Description = <none>
CreateIndex = 17
ModifyIndex = 17

Rules

namespace "*" {
  policy = "write"
  capabilities = [
    "list-jobs",
    "parse-job",
    "read-job",
    "submit-job",
    "dispatch-job",
    "read-logs",
    "read-fs",
    "alloc-exec",
    "alloc-lifecycle",
    "csi-write-volume",
    "csi-mount-volume",
    "list-scaling-policies",
    "read-scaling-policy",
    "read-job-scaling",
    "scale-job"
  ]
  variables {
    path "dev" {
      capabilities = ["read"]
    }
  }
}

[philrenaud]

Hi @Great-Stone — until #17770 , the UI did not recognize permissions granted by ACL Roles — only policies directly on the token. Now merged, this change should be coming in a release soon. Thank you for your patience!

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.