From 05be439cff1539cebed62d490321a5d712d4e40a Mon Sep 17 00:00:00 2001 From: Vu Dinh Date: Mon, 11 Oct 2021 15:01:59 -0400 Subject: [PATCH] Update the EP to focus on generic constraint aspect only The EP is renamed generic constraint to focus on the constraint conponent only. The other components such as the mechanism to provide the cluster/bundle constraint will become its own EP. Signed-off-by: Vu Dinh --- enhancements/cluster-runtime-constraints.md | 159 ----------------- enhancements/generic-constraints.md | 181 ++++++++++++++++++++ 2 files changed, 181 insertions(+), 159 deletions(-) delete mode 100644 enhancements/cluster-runtime-constraints.md create mode 100644 enhancements/generic-constraints.md diff --git a/enhancements/cluster-runtime-constraints.md b/enhancements/cluster-runtime-constraints.md deleted file mode 100644 index 388bafcb..00000000 --- a/enhancements/cluster-runtime-constraints.md +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: Cluster Runtime Constraint -authors: - - "@dinhxuanvu" -reviewers: - - "@kevinrizza" -approvers: - - "@kevinrizza" -creation-date: 2021-08-26 -last-updated: 2020-09-06 -status: provisional ---- - -# cluster-runtime-constraint - -## Release Signoff Checklist - -- [ ] Enhancement is `implementable` -- [ ] Design details are appropriately documented from clear requirements -- [ ] Test plan is defined -- [ ] Graduation criteria for dev preview, tech preview, GA - -## Summary - -This proposal presents the specification and usage to enable runtime constraints to be considered during operator dependency resolution. This feature will ensure Operator Lifecycle Manage (OLM) to install operators that satisfy dependency requirements and runtime constraints if present. - -## Motivation - -At the moment, OLM resolves a set of operators which seem that they will work together based on dependency requirements that are specified by operator bundles. However, it is possible that those resolved operators will not work on the clusters due to cluster/runtime constraints. For example, if the cluster is running a specific Kubernetes version that is not compatible with the operators, the installed operators may fail to work properly on that given cluster. - -The cluster runtime constraints are often unknown by the operator authors and should be specified by cluster admins. However, currently, there is no mechanism for cluster admins to specify runtime constraints that OLM can understand and take under consideration during installation. - -### Goals -- Provide mechanism (including specification and guideline) for cluster admins to specify cluster runtime constraints -- Provide specification and guideline for author operators to specify runtime constraints/requirements if needed -- Block the updates/installations of operators based on cluster runtime constraints -- Allow the cluster runtime constraints to be generic and configurable by cluster admins - -### Non-Goals - -- Provide mechanism to retrieve/populate cluster runtime constraints automatically -- Allow cluster runtime constraints to be scoped -- Allow cluster runtime constraints to come from multiple/different sources -- Allow optional cluster runtime constraints - - -## Proposal - -To provide a generic solution for cluster runtime constraints, a library named [cel-go](https://github.com/google/cel-go) that provides expression evaluation capabilities is utilized in this proposal. - -### User Stories - -#### Constraints specified by cluster admin - -As a cluster admin, I would like to specify a list of cluster runtime constraints that are associated with a given cluster and all installable operators will meet these requirements. - -#### Constraint properties specified by operator author - -As an operator author, I would like to provide a list of runtime properties that may be needed to be considered before installation to ensure the operator working properly on the cluster. - -### Implementation Details - -#### Cluster Runtime Constraint Configmap - -Cluster admins can specified cluster runtime constraints in a configmap (named `olm-runtime-constraints`) in `olm` namespace using this format: - -```yaml= -apiVersion: v1 -kind: ConfigMap -metadata: - name: olm-runtime-constraints - namespace: olm -data: - properties: -``` - -Each constraint is specified using the following syntax: - -```json= -{ - "type": "olm.constraint", - "value": { - "evaluator": { - "id": "cel" - }, - "source": "properties.exists(p, p.type == \"certified\")", - "action": { - "id": "require" - } - } -} -``` - -The constraint `type` is `olm.constraint` and the `value` is information associated with constraint. The `value` struct has 3 fields: `evaluator`, `source` and `action`. - -The `evaluator` is a struct with `id` field to represent the language library that will be used to evaluate the expression. At the moment, only `cel-go` library is supported using `cel` as the identifier. More fields are potentially added to expand the supported expression languages/libraries in the future. - -The `source` field is the string of expression that will be evaluated during resolution. Currently, only `cel-go` expressions are supported. - -Finally, the `action` field is a struct with the `id` field to indicate the resolution action that this constraint will behave. For example, for `require` action, there must be at least one candidate that satisfies this constraint. At the moment, `require` and `conflict` actions are supported. More fields may be added into `action` struct to support more resolution actions in the future. - -#### Runtime Dependencies in Bundle - -The operator authors specify the cluster runtime requirements in `dependencies.yaml` using the following format: - -```yaml= -dependencies: - - - type: olm.constraint - value: - action: - id: require - evaluator: - id: cel - source: "properties.exists(p, p.type == \"certified\")" -``` - -#### Runtime Constraint Resolution - -The constraints that are specified in `olm-runtime-constraints` Configmap are converted into properties of a global existing virtual node in the cluster. All operators/candidates must satisfy those properties in order to be installed. - -If constraints are added to bundles, they will be evaluated and must be satisfied similarly to `olm.gvk.required` and `olm.package.required` properties. - -#### Compound Constraint - -It is possible to write `cel-go` expression to evaluate multiple different properties in one single expression that is specified in one constraint. For example: - -```json= -{ - "type": "olm.constraint", - "value": { - "evaluator": { - "id": "cel", - } - "source": 'properties.exists(p, p.type == "certified") && properties.exists(p, p.type == "new")', - "action": { - "id": "require" - } - } -} -``` - -The expression in `source` has two requirements that are linked in an AND (&&) logical operator. In order for that expression to be true, both requirements must be satisfied. This concept represents the compound constraint. - -### Risks and Mitigations - -### Prototype - -TBD - -## Drawbacks - -The [cel-go](https://github.com/google/cel-go) library is rather new and still under development. Even though cel-go has been used in other opensource projects such as Tekton, it has not yet reached stable release (v1). Breaking changes can be introduced to this library and potentially OLM behavior in the future. - -The complexity of the library is not yet fully evaluated which could lead to a poor user experience if the library is too difficult to use and utilized. - -One of the popular use cases is semver comparison which is not a supported value type in cel-go. Custom functions are required to support semver. - -## Alternatives diff --git a/enhancements/generic-constraints.md b/enhancements/generic-constraints.md new file mode 100644 index 00000000..5d12f9b2 --- /dev/null +++ b/enhancements/generic-constraints.md @@ -0,0 +1,181 @@ +--- +title: generic constraint +authors: + - "@dinhxuanvu" +reviewers: + - "@kevinrizza" + - "@joelandford" +approvers: + - "@kevinrizza" +creation-date: 2021-08-26 +last-updated: 2020-10-08 +status: provisional +--- + +# generic-constraint + +## Release Signoff Checklist + +- [ ] Enhancement is `implementable` +- [ ] Design details are appropriately documented from clear requirements +- [ ] Test plan is defined +- [ ] Graduation criteria for dev preview, tech preview, GA + +## Summary + +The dependency resolution in OLM needs to support constraints that are based on arbitrary properties from either bundles or clusters beyond the existing GVK or package properties. + +This enhancement proposes the adoption of [Common Expression Language +(CEL)](https://github.com/google/cel-go) to present the specification and usage of generic constraint that OLM can evaluate for dependency resolution at runtime. + +CEL is a sufficiently lightweight expression language that has been adopted in multiple opensource projects. It allows users to express operator constraints in both straight-forward and advanced cases without OLM having to dictate the evaluation methodology. + +## Motivation + +At the moment, OLM only supports 2 built-in types of constraint: +- GVK constraint (`olm.gvk.required`)* which depends on a specific GroupVersionKind information of a CRD/API. +- Package constraint (`olm.package.required`)* which depends a specific package name and version (or a range of version) + +Both types are specifically handled by the resolver in OLM and cannot be changed or expanded to support further use cases. In order to support more types of constraints, it is generically expected for OLM to add more specific built-in types to allow users to express those new constraints in the bundles. Over the time, it may become cumbersome and impractical to continue to add more specific types to the resolver. The need for OLM to support a generic constraint model becomes more apparent. With a generic constraint model, OLM no longer needs to carry any specific evaluation methodology besides the understanding of how to parse the constraint syntax. The users should be able to express declaratively the rules for a constraint that can be evaluated againts the dataset of arbitrary properties. + +* Note: The current GVK and package constraints are specified in `dependencies.yaml` using `olm.gvk` and `olm.package` type and they are coverted to `olm.gvk.required` or `olm.package.required` property (which is required constraint) to differentiate from the `olm.package` and `olm.gvk` property in the bundle. + +### Goals +- Provide specification and guideline to specify a constraint that contains declarative requirements/rules. +- Support the compound constraint (the constraint with mutliple requirements that is resolved into a single operator). +### Non-Goals +- Eliminate the existing built-in constraints. +- Support multiple expression languages other than CEL (though the syntax may allow the support for other languages for future use). +- Provide mechanism to provide cluster constraints. This feature can be addressed in a separate enhancement. + + +## Proposal + +The [Common Expression Language (CEL)](https://github.com/google/cel-go) is a great inline expression language that can be used to express multiple rules within a single expression. The rules are fully customizable and constructed specifically for users' needs and requirements without OLM needing to know the design methodology. It is important to recoginize the dependent relationship between constraints and properties. As a constraint is evaluated against a dataset of properties from a bundle or a given source, in order to design a constraint, there must be an understanding on what properties are available in a given a bundle or source and what information they present in order for a constraint to be evaluated propertly. + +The CEL library also supports [extension functions](https://github.com/google/cel-spec/blob/master/doc/langdef.md#extension-functions) which allows OLM to implement additional functions that may be necessary for certain operations. These functions then can be used within the CEL expression. For example, [semver](https://semver.org/) comparison is not built-in in CEL and it is often used in OLM as bundles are versioned in semver format. The semver functions can be added in the initial release and allow users to express semver evaluations in the constraint. However, extension functions should be treated and designed with care as they can be difficult to change in the future and potentially create version skew or breaking changes. + +All CEL expressions are [parsed and typechecked](https://github.com/google/cel-go#parse-and-check) before evaluation. The insulting programs can be cached for later evaluation to reduce unnecesary computation. Also, the CEL evaluation is [thread-safe and side-effect free](https://github.com/google/cel-go#evaluate). + +### User Stories + +#### Generic Constraint + +As an operator author, I would like to specify a constraint that has my own rules/requirements that are appliable to a set of known properties are from other bundles or from the cluster itself without having to tell OLM how to evaluate those rules. For example, if I know there is a property named certified in some bundles, I can express a constraint with a rule that says my operator to only be installed along with those certified bundles without having to tell OLM to support a new type of constraint for certified property. + +#### Compound Constraint + +As an operator author, I would like to specify a constraint that has multiple rules and requirements that must be resolved into a single dependent operator. For example, I would like to have a constraint for a dependent operator that belongs to a specific package and provides a specific API/GVK. + +### Design Details + +#### Generic Constraint Syntax + +The constraint syntax is designed to have similarity with the existing constraints that OLM supports which are GVK and package. It is intentional to keep the fundamental syntax with `type` and `value` the same so that it doesn't introduce any dramatic changes to the overall user experiences. + +Each constraint is specified using the following syntax: + +```json= +{ + "type":"olm.constraint", + "value":{ + "evaluator":{ + "id":"cel" + }, + "rule": "properties.exists(p, p.type == \"certified\")", + "message": "require to have certified property", + "action":{ + "id":"require" + } + } +} +``` + +The constraint `type` is `olm.constraint` and the `value` is information associated with constraint. The `value` struct has 4 fields: `evaluator`, `rule`, `message` and `action`. + +The `evaluator` is a struct with `id` field to represent the language library that will be used to evaluate the expression. At the moment, only CEL expression is supported using `cel` as the identifier. + +The `rule` field is the string that presents a CEL expression that will be evaluated during resolution against . Only CEL expression is supported in the initial release. + +The `message` field is an optional field that is accommodating the rule field to surface information regarding what this CEL rule is about. The message will be surfaced in resolution output when the rule evaluates to false. + +Finally, the `action` field is a struct with the `id` field to indicate the resolution action that this constraint will behave. For example, for `require` action, there must be at least one candidate that satisfies this constraint. This action can be used to indicate optional constraint in the future. For the initial release, `require` action is supported. + +#### Compound Constraint + +The CEL [syntax](https://github.com/google/cel-spec/blob/master/doc/langdef.md#syntax) supports a wide range of operators including logic operator such as AND and OR. As a result, a single CEL expression can have multiple rules for multiple conditions that are linked together by logic operators. These rules are evaluated against a dataset of multiple different properties from a bundle or any given source and the output is solved into a single bundle or operator that sastifies all of those rules within a single constraint. For example: + +```json= +{ + "type": "olm.constraint", + "value": { + "evaluator": { + "id": "cel", + } + "rule": 'properties.exists(p, p.type == "certified") && properties.exists(p, p.type == "stable")', + "message": 'require to have "certified" and "stable" properties', + "action": { + "id": "require" + } + } +} +``` + +The expression in `rule` has two requirements that are linked in an AND (&&) logical operator. In order for that expression to be true, both requirements must be satisfied. Therefore, the resolved operator will be a single bundle that has both certified and stable properties. This concept represents the compound constraint. + +### Risks and Mitigations + +#### User experience concern + +The CEL language is relatively new and requires a learning curve to understand. The complexity of the language may potentially create a difficult user experience (UX) especially when the users intend to write a complex expression for an advanced user case. Also, the current resolver doesn't surface the resolution information very clearly which leads to the difficulty on how the users can identify what goes wrong when the CEL expression is evaluated to `false`. + +Mitigation: The feature can be implemented under a feature flag to allow feedbacks to be gathered and then the UX can be evaluated based on those feedbacks in order to improve the UX if needed in future releases. + +Also, the constraint syntax can be simplified to reduce the complexity in order to improve the overall UX. (See Alternatives). + +For majority of use cases, the CEL expression should be simple and straight-forward especially with well-constructed examples and guidelines where users can make a few plug-in changes and ready to be used. + +The `message` field is introduced to allow specific information from the author who writes the expression to surface as resolution output. + +#### CEL language maturity and support + +The CEL language is relatively new and not yet at stable release (v1). There are potentially breaking changes that can be introduced. + +Migration: The CEL library can be spinned to a specific release to avoid introducing breaking changes if occurs. + +The language is being used in another well-known opensource projects including Tekton and Kubernetes ([apiserver/CRD](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/2876-crd-validation-expression-language/README.md)). This should improve the confidence on the usage of CEL language and these teams will potentially influence or impose against any major breaking changes that may come to CEL. + +If CEL becomes too instable in the future due to breaking changes, it is possible to gradually deprecate the support of the language and introduce another language that is more stable. The constraint syntax allows the support of other languages in the future via `evaluator` field. + +## Alternatives + +### Other languages choices + +While CEL is the selected language to be supoorted in the new OLM constraint type. It is not the intention to limit OLM to only support CEL expression. It possible to support multiple expression languages in the future releases if needed. + +Besides CEL, there are other languages that can be used such as: + +- [Rego](https://github.com/open-policy-agent/opa/tree/main/rego) (Open Policy Agent) +- [Expr](https://github.com/antonmedv/expr) +- [Starlark](https://github.com/bazelbuild/starlark) +- [Cue](https://github.com/cue-lang/cue) + +All of these languages can be supported in constraint type. The `evaluator` field is designed to support multiple evaluators/languages if needed. However, intrducing a new language to the constraint type should be evaluated carefully to ensure there is a real need for it. Providing the support for multiple languages can be overwhelming and potentially creates a fragmented user experiences and unnecessary maintainance effort in a long run. + +### Simplified constraint type + +The current proposed constraint syntax has most of information nested under `value` field. The complexity of a struct with nested fields can create a bad user experience. It is possible to introduce a constraint type that is specified for CEL + +```json= +{ + "type": "olm.constraint.cel", + "value": { + "rule": 'properties.exists(p, p.type == "certified") && properties.exists(p, p.type == "stable")', + "message": 'require to have "certified" and "stable" properties', + } +} +``` + +The identifier `type` for this constraint is `olm.constraint.cel` which means this is a CEL constraint that only supports CEL expression. The `value` field contains the `rule` field which houses the CEL expression and the `message` field which is the resolution output if the expression is evaluated as `false`. + +This simplified version of constraint type still satisfies the goals of having a generic constraint (though to lesser extend due to lacking of expandability that is provided by additional fields) and the compound constraint.