[release-2.12] Add support for mcoa-managed observability capabilities

[periklis]

The following PR adds support to reconcile further observability capabilities via the multicluster-observability-addon. The present implementation adds:

1. Support of the addon as separate MCO operand.
2. Extends the MultiClusterObservability CRD spec by a new section named capabilities as discussed here (TODO: Add link to DDR here).
3. For logs data collection it adds support for platform and user workloads.
4. For tracing data collection/instrumentation it adds support for user workloads.

Special notes for your reviewer:
To the test the following PR one needs to publish an image of the referenced dependency PR and applying the following instructions:

1. Install the Advanced Cluster Management for Kubernetes from the OperatorHub.
2. Create a MultiClusterHub resource.
3. Set the MultiClusterHub resource to pause reconciliation via:
   kubectl -n open-cluster-management annotate mch multiclusterhub --overwrite mch-pause=true
4. Build and publish the MCO operator image from this branch:

make docker-build-local IMAGE_TAG_BASE=quay.io/$YOUR_ORG/multicluster-observability-operator

docker push quay.io/$YOUR_ORG/multicluster-observability-operator:latest

5. Edit the image for the multicluster-observability-operator deployment
   kubectl -n open-cluster-management edit deployment multicluster-observability-operator
6. Update the ClusterRole to add support for create/update/delete/patch for AddonDeploymentConfig resources:
   kubectl edit clusterrole open-cluster-management:multicluster-observability-operator:multicluster-observability-operator
7. Update the CRD using the one in this branch to add support for the capabilities section.
8. Finally create an MultiClusterObservability custom resource as the following:

apiVersion: observability.open-cluster-management.io/v1beta2
kind: MultiClusterObservability
metadata:
  annotations:
    mco-multicluster_observability_addon-image: quay.io/YOUR_ORG_HERE/multicluster-observability-addon:v0.0.1
  name: observability
spec:
  capabilities:
    platform:
      collection:
        logs:
          enabled: true
    userWorkloads:
      logs:
        collection:
          clusterLogForwarder:
            enabled: true
      traces:
        collection:
          instrumentation:
            enabled: true
          openTelemetryCollector:
            enabled: false
  observabilityAddonSpec: {}
  storageConfig:
    metricObjectStorage:
      name: thanos-object-storage
      key: thanos.yaml

Dependencies:

• Add management support for the multicluster-observability-operator multicluster-observability-addon#53

[openshift-ci]

Hi @periklis. Thanks for your PR.

I'm waiting for a stolostron member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[JoaoBraveCoding]

Overall looks good! I'm just concerned with how we will package the CRDs since without them users will not be able to create stanzas unless they install the operators

[sonarqubecloud]

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
75.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[periklis]

/test test-e2e

[philipgough]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: periklis, philipgough

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [periklis,philipgough]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment