[Authz] Operator privileges

[elena-shostak]

Summary

This PR adds support for explicit indication whether endpoint is restricted to operator only users.

Context

1. If user has all operator privileges granted, but is not listed as operator in operator_users.yml, ES would throw an unauthorized error.
2. If user is listed as operator in operator_users.yml, but doesn't have necessary privileges granted, ES would throw an unauthorized error.
3. It’s not possible to determine if a user is operator via any ES API, i.e. _has_privileges.
4. If operator privileges are disabled we skip the the check for it, that's why we require to explicitly specify additional privileges to ensure that the route is protected even when operator privileges are disabled.

Checklist

• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios

Relates: #196271

How to test

1. Add your user to the operators list

   kibana/packages/kbn-es/src/serverless_resources/operator_users.yml

   Line 4 in 1bd8144

   auth_type: 'realm'

   or use existing user from the list to log in.
2. Run ES and Kibana serverless
3. Change any endpoint or create a new one with the following security config

      security: {
        authz: {
          requiredPrivileges: [ReservedPrivilegesSet.operator],
        },
      },

4. Check with enabled and disabled operator privileges (set xpack.security.operator_privileges.enabled)

Release Note

Added support for explicit indication whether endpoint is restricted to operator only users at the route definition level.

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[afharo]

Core changes LGTM

[azasypkin]

Caution

I don’t think we want to include these in our public setup or start contracts. Do we only need them for initAPIAuthorization? If that’s the case, and since this functionality is now exposed through Core, can we pass these two services directly to AuthorizationService.setup without exposing them elsewhere?

    const coreStartServices = core.getStartServices().then(([{ elasticsearch, security }]) => ({
      elasticsearch,
      security,
    }));

    this.authorizationSetup = this.authorizationService.setup({
      coreStartServices, // or split into `getCurrentUser` and `getClusterClient\getSecurity`
      ....
    });

[elena-shostak]

Addressed this, those methods are no longer exposed in our public contracts

[azasypkin]

Note

It's probably fine for now, but this is a hot path and if ever need to a high-performance endpoint to require operator privilege we might need to cache this response for some time.

[legrego]

IMO this call is too expensive to put in a hot path such as this. I think we should look into a way to cache this during startup. I recall SDHs where our frequent use of this endpoint elsewhere has caused performance issues on the cluster

[azasypkin]

I'm a bit concerned that if we determine this only once during startup, admins will have to restart Kibana whenever this functionality is enabled or disabled in Elasticsearch. If we want to avoid that risk, we could either cache the result near the invocation point or implement a periodic background job to handle it (which is probably too much comparing to simple cache). Do you think requiring a Kibana restart along with Elasticsearch would be a reasonable approach (similar to how it works for ES, although for the config that doesn't belong to Kibana)?

Alternatively, we can require explicit operator flag/config for Kibana as well, that we can compare with the response from ES at startup time and bail if they don't match... Not sure if I like this option though

[legrego]

I'd share your concerns if this was a different ES setting. Operator Privileges are only used by our orchestrated offerings, and are configured via files that reside on each node. I do not expect the same cluster to switch modes at runtime, and we explicitly do not support direct usage of this feature:

https://www.elastic.co/guide/en/elasticsearch/reference/current/operator-privileges.html

Since we are in control over the configuration of Operator Privileges, it seems like it'd be sufficient to check this once on startup, and require a restart in the event the cluster somehow switched between using Operator Privileges and not using Operator Privileges.

[elena-shostak]

We make a call to operator privileges config only on startup now

[azasypkin]

If operator privileges are disabled, we fall back to the superuser privileges check.

I know we discussed this here: #196271 (comment), but it seems we didn’t reach an agreement. This behavior deviates slightly from Elasticsearch’s approach, making it harder to reason about operator privileges across the Elastic project as a whole. This isn’t necessarily a bad thing, but we should explicitly state the benefits if we decide not to align with Elasticsearch’s treatment of operator privilege requirements.

Is there a reason we can’t or don't want to require developers to explicitly list additional privileges (beyond operator privileges) when exposing a Kibana endpoint to operator users? This would ensure privilege the same checks occur even if the operator user functionality is not enabled. The additional privilege could be superuser, but it could also be a role with fewer privileges if superuser access isn’t strictly necessary. This would discourage admins from granting superuser privileges to operator users unnecessarily.

The only potential issue I see with requiring additional privileges is if there’s a use case where no extra privilege check is needed at the API level because the handler solely relies on user-scoped Saved Objects or Elasticsearch services. If such a use case arises, we could explore solutions to handle it appropriately.

[elena-shostak]

Is there a reason we can’t or don't want to require developers to explicitly list additional privileges

I was for implementing it in the beginning and then thought to make it more feasible in terms of DX. That was a bit premature 😄

The additional privilege could be superuser, but it could also be a role with fewer privileges if superuser access isn’t strictly necessary. This would discourage admins from granting superuser privileges to operator users unnecessarily.

Agree, that makes sense

If there’s a use case where no extra privilege check is needed at the API level because the handler solely relies on user-scoped Saved Objects
Don't think such case has come up yet, but we can explore it when there is the need

Thanks for the feedback, will address the PR accordingly 👍

[elena-shostak]

@elasticmachine merge upstream

[elena-shostak]

@elasticmachine merge upstream

[azasypkin]

ACK: will review today

[azasypkin]

LGTM, tested locally and everything works as expected, thanks! Just left a few nits, comments, and questions.

[azasypkin]

note: It feels like adding getSecurityConfig and getCurrentUser to authz just to later pass it to the internal initAPIAuthorization makes things slightly more complex, as we need to update types in multiple places, mocks, and ensure it doesn’t leak into our contracts. Is there any reason we can’t just pass these two properties directly to initAPIAuthorization as additional arguments (and as the result I guess we won't need any changes in x-pack/packages/security/plugin_types_server at all?)?

[elena-shostak]

Pushed in 062ce7a 👍

[azasypkin]

optional nit: maybe we need to expose\export Privileges from the Core, but I'm fine with what you have already.

type Privileges = Array<Privilege | PrivilegeSet>;
export interface AuthzEnabled {
  requiredPrivileges: Privileges;
}

[azasypkin]

TIL: toSpliced is so handy!

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: b212b43

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/security-plugin-types-server	160	161	+1

Unknown metric groups

API count

id	before	after	diff
@kbn/core-security-common	20	21	+1
@kbn/security-plugin-types-common	125	126	+1
@kbn/security-plugin-types-server	281	282	+1
security	455	458	+3
total			+6

History

• 💔 Build #259958 failed 062ce7a
• 💚 Build #259835 succeeded b4ae823
• 💚 Build #259207 succeeded bfeb510
• 💔 Build #259142 failed 0516fd9
• 💔 Build #259076 failed 4b04f0e

[kibanamachine]

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12306196929

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation