⚠️ Re-define how related resources are configuring/found, add support for label selectors

[xrstf]

Summary

This PR significantly changes the approach the agent uses to locate related objects.

Old Approach

My original idea was:

1. For each related resource, the admin configures one (!) reference into the main object (by virtue of a path expression).
2. This one (!) reference is then evaluated equally on both sides of the sync (i.e. in the original main object in kcp, and in the potentially mutated copy of it on the service cluster).

The advantage of this approach is that mutations only happen in one place: the main object. When thinking about finding the related objects, the admin does not have to consider additional mutations/renamings, because that all happened already in the main object.

For example, if you had a certificate with spec.secretName=my-cert, you would mutate this spec in the main object to become spec.secretName=$clusterName-$remoteHashYaddaYadda, for example spec.secretName=1otjakf7q2tyhbii-4827da5-278adfe. And now when we want to locate it, we just follow the same path and get the correct names. Easy!

Label Selectors

The approach above works fine if you use references because that always yields a fixed, known value.

But now we want to add label selectors for related resources. This can be used when there is no reference (like spec.secretName) in the main object. This happens for operators that just take the name of the main object and append something like -credentials to it and use this, per convention, as the secret.

The Issue with secret information

The issue is that the name of the main object usually is somewhat encoded, to ensure many workspaces do not collide on the service cluster. There is also a chance that additional, "secret" information is embedded into them (because agents might sync special customers into special namespaces with special naming rules). Long story short, the main object's name is, like every part of the sync metadata, private to the service provider and should never leak into kcp workspaces.

This also means that the names of the related objects are also considered private and we must not re-use them on the destination side of the synchronisation. But now we're in a pickle, because if we follow the old approach, we would evaluate the same label selector on both sides of the sync. On the first iteration, this will ideally find the source object, but obviously cannot yet find the destination object (we haven't create it yet). So now the agent has to decide the destination object's name by itself.

This now means that the admin, in addition to a label selector, also has to specify the naming rules for the copy of the related object. And this means that the logic is not symmetric anymore:

• The label selector is actually applied on the origin side of the sync to find the source object.
• But on the destination side of the sync, the selector is ignored and the naming rules are used instead to create the destination namespace/name.

Asymmetry

This asymmetry is what is ultimately causing this API break. There is no sane way to express this in the old structure, without introducing a lot of extra validations (validations the OpenAPI schema cannot handle and would either require a webhook or loooots of CEL expressions) and need for documentation. Even I, who thought through all this, was confused when I began the implementation. By my own data structures no less. Because if you have naming rules, are they also used for reference-based related resources?

Since we do not have a conversion webhook and bumping API versions in Kubernetes is a major PITA, as I've just learned from kcp itself, I chose to commit an API break in this PR.

API Break

With this PR, we the configuration is extended like this. Previously this is the structure:

spec:
  related:
    - id: credentials
      # 1 reference
      reference:
        name:
          path: metadata.name
          regex: {pattern: ..., replacement: ...}
        # optional namespace
        namespace:
          path: metadata.foobar
          regex: {pattern: ..., replacement: ...}

With this change, all configuration was moved under a new "object" key and then restructured a bit:

spec:
  related:
    - id: credentials

      object:
        # these three are mutually exclusive
        # they all refer to the _name_ of the object only
        reference: {path: ..., regex: {...}}
        selector: {matchLabels: ..., matchExpressions: ..., rewrite: {regex: ..., template: ...}}
        template: {template: ...}

        # if necessary, the namespace can be configured the same way
        namespace:
          reference: {path: ..., regex: {...}}
          selector: {matchLabels: ..., matchExpressions: ..., rewrite: {regex: ..., template: ...}}
          template: {template: ...}

Label Selectors

...are currently not super useful, because you cannot use any dynamic values in them, like variables. The old approach of having a custom variable syntax ("foo$bar") is reaching its limits and so we plan on replacing it with Go template strings. But this will be done in another PR and that PR will then add support for templated strings to the selectors.

Multiple Related Objects

Since we now have support for label selectors, a single related resource can select many objects. And this is now supported by the agent. It's not super convenient yet (the annotation on the primary object that links to the related object(s) is a bit ugly with a counter at the end, but it works).

Related issue(s)

Fixes #31

Release Notes

* [BREAKING] Add support for finding related resources via label selectors.
* Add support for having one related resource match many objects.

[kcp-ci-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[xrstf]

/test all

[xrstf]

/test all

[xrstf]

/test all

[xrstf]

/test all

[xrstf]

/test all

[embik]

Just one teeny tiny thing.

[embik]

/kind api-change
/kind feature

[embik]

/approve

[kcp-ci-bot]

LGTM label has been added.

Git tree hash: 16ef992fad35e86a5fc39688019ade8abfe9790a

[kcp-ci-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: embik

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [embik]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment