ILS-249 Add remove field to IBMLicensingDefinition

[alicenoknow]

This PR adds remove field to IBMLicensingDefinition. It should work similar to set field but for removing the annotations that may be already present in pod.

[tojoos]

Looks good 👍

[alicenoknow]

Is this change safe, i.e. do we have preserve unknown fields so that this new field won't be pruned by k8s when older LS CRD is loaded into the cluster?

we do have this setting already:

x-kubernetes-preserve-unknown-fields: true

the field is not pruned

[pgodowski]

When extending CRD, one must not add a new requied field. Where it would fail is the following scenario:

• cluster has already some CRs of Kind: IBMLicensingDefinition, before a new version of LicSvc is installed
• such CR has some fields in, but not remove (since such field was not available in older version of License Service)
• License Service is updated, thus CRD definition is upgraded
• K8s will re-validate all the existing CRs in the cluster, against the updated CRD schema - in this case all the already existing CRs will fail such validation
• moreover, if someone wants to add a new IBMLicensingDefinition CR using older schema when new LicSvc is installed, such operation also fail, since we require additional field.

Long story short: it is ok to add a new field, but not to mark it as required. Yet frankly, why do we actually require this new field in the first place? Perhaps we're just missing omitempty?

And then, given we now have both "set" and "remove" - shouldn't we actually say that both are optional?

[kflorianski-ibm]

I agree, both should now be optional. Do we want to enforce at least one is provided in the operand code (or log an error otherwise)?

[alicenoknow]

both fields are now optional, the operand will do the check for both set & remove empty during CR validation along with other checks and log IbmLicensingDefinition CR <name> is incomplete and cannot be processed

[pgodowski]

log IbmLicensingDefinition CR <name> is incomplete and cannot be processed

OK, but pls add more verbose log message, i.e. indicate that one of set or remove is required - to make sure those who look at the logs are clear what is wrong and what to do / fix

[alicenoknow]

Extracted to separate case with custom login message:

IbmLicensingDefinition CR <NAME> does not have either the set or remove sections and cannot be processed

[ibm-ci-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alicenoknow, kflorianski-ibm, tojoos

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [alicenoknow,kflorianski-ibm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment