Changes to operator hub tooling for 2.9 release (elastic#7056) (elast…

…ic#7059) I realised while looking at https://github.com/k8s-operatorhub/community-operators/tree/main/operators/elastic-cloud-eck/2.8.0/metadata that the format for the community operator submissions has changed to bundle format. I am not sure if a submission is still possible in legacy package format once the previously published versions have been (auto-) converted to bundles. This change - moves the manifest generation to bundle format - removes all the "extra" functions in the bundle PR submissions code - renaming of the CSV is not necessary if we just don't generate the version into the name to begin with. The registry seems to look at Kind anyway to identify CSVs - the image SHA replacement code was dead and duplicating the working code in the manifest generation - the *package.yaml thingy is no longer needed in bundle format - removes the bundle generate command which is not necessary if we template in bundle format to start with. (cherry picked from commit cc201b7) # Conflicts: # hack/operatorhub/config.yaml # hack/operatorhub/go.sum
pebrc · Jul 25, 2023 · 8be240b · 8be240b
1 parent 4b87c0e
commit 8be240b
Show file tree

Hide file tree

Showing 12 changed files with 87 additions and 888 deletions.
diff --git a/.buildkite/pipeline-release-redhat.yml b/.buildkite/pipeline-release-redhat.yml
@@ -60,7 +60,6 @@ steps:
       - |
         cd hack/operatorhub
         operatorhub generate-manifests
-        operatorhub bundle generate
         operatorhub bundle create-pr
     agents:
       image: docker.elastic.co/ci-agent-images/cloud-k8s-operator/buildkite-agent:abaeba8c

diff --git a/hack/operatorhub/README.md b/hack/operatorhub/README.md
@@ -6,8 +6,7 @@ These commands include
 
 - Pushing operator image to Quay.io
 - Publishing eck operator container image within redhat catalog
-- Generate Operator Lifecycle Manager format files
-- Generate Operator bundle metadata (wrapper for [OPM](https://github.com/operator-framework/operator-registry/tree/master/cmd/opm) command)
+- Generate Operator Lifecycle Manager registry bundle files
 - Publish draft pull requests to both https://github.com/redhat-openshift-ecosystem/certified-operators and https://github.com/k8s-operatorhub/community-operators
 
 ## Configuration
@@ -28,7 +27,6 @@ All commands require the configuration file `config.yaml` to be modified prior t
   * publish - publish operator container within redhat certification API
 * generate-manifests - generate operator lifecycle manager format files
 * bundle - parent command
-  * generate - generate operator metadata for publishing on openshift operator hub
   * create-pr - perform all git operations and create pull requests for community and certified operator repositories
 
 ## Vault Details
@@ -121,7 +119,6 @@ The `generate-manifests` command will extracts CRDs and RBAC definitions from di
 a new release to OperatorHub.  These files are written within the following 3 directories:
 
 - community-operators
-- upstream-community-operators
 - certified-operators
 
 ### Usage
@@ -153,28 +150,6 @@ To generate configuration based on yet unreleased YAML manifests:
 
 *This command's sub-commands all requires the output of the `generate-manifests` command to be run*
 
-### Generate sub-command
-
-The `bundle generate` command will perform the following tasks:
-1. Run the [opm](https://github.com/operator-framework/operator-registry/tree/master/cmd/opm) command on the output of `operatorhub` command to generate the operator metadata for operator hub publishing
-
-#### Usage
-
-- Ensure that the `generate-manifests` command has successfully ran
-- Run the `bundle generate` command, pointing to the output directory of the `generate-manifests` command
-
-With and without vault
-```shell
-./operatorhub bundle generate -d .
-```
-
-#### Flags
-
-| Parameter                        | Description                                                                                                                               | Environment Variable                | Default           |
-|----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------|-------------------|
-| `--dir`                          | Directory containing the output of the `generate-manifests` command.                                                                      | `OHUB_DIR`                          | `"./"`            |
-| `--conf`                         | Path to config.yaml file.                                                                                                                 | `OHUB_CONF`                         | `"./config.yaml"` |
-
 ### Create-PR sub-command
 
 The `bundle create-pr` command will perform the following tasks:

diff --git a/hack/operatorhub/cmd/bundle/bundle.go b/hack/operatorhub/cmd/bundle/bundle.go
@@ -6,36 +6,23 @@ package bundle
 
 import (
 	"fmt"
-	"path"
-	"path/filepath"
-	"strings"
 
 	"github.com/spf13/cobra"
 
 	"github.com/elastic/cloud-on-k8s/v2/hack/operatorhub/cmd/flags"
 	"github.com/elastic/cloud-on-k8s/v2/hack/operatorhub/internal/github"
-	"github.com/elastic/cloud-on-k8s/v2/hack/operatorhub/internal/opm"
 )
 
 // Command will return the bundle command
 func Command() *cobra.Command {
 	bundleCmd := &cobra.Command{
 		Use:   "bundle",
-		Short: "generate operator bundle metadata/create pull requests for new operator versions",
+		Short: "create pull requests for new operator versions",
 		Long: `Bundle and build operator metadata for publishing on openshift operator hub, and create pull requests to
 certified-operators, and community-operators repositories.`,
 		SilenceUsage: true,
 	}
 
-	generateCmd := &cobra.Command{
-		Use:          "generate",
-		Short:        "generate operator bundle metadata",
-		Long:         "Bundle and build operator metadata for publishing on openshift operator hub",
-		SilenceUsage: true,
-		PreRunE:      generateCmdPreRunE,
-		RunE:         doGenerate,
-	}
-
 	createPRCmd := &cobra.Command{
 		Use:   "create-pr",
 		Short: "create pull requests against community and certified repositories",
@@ -94,24 +81,11 @@ certified-operators and community-operators repositories.`,
 		"delete git temporary directory after script completes (OHUB_DELETE_TEMP_DIRECTORY)",
 	)
 
-	bundleCmd.AddCommand(generateCmd, createPRCmd)
+	bundleCmd.AddCommand(createPRCmd)
 
 	return bundleCmd
 }
 
-// generateCmdPreRunE are pre-run operations for the generate command
-func generateCmdPreRunE(cmd *cobra.Command, args []string) error {
-	if flags.Dir == "" {
-		return fmt.Errorf("directory containing output from operator hub release generator is required (%s)", flags.DirFlag)
-	}
-	if flags.Conf.MinSupportedOpenshiftVersion == "" {
-		return fmt.Errorf("minimum supported openshift version is required in configuration file")
-	} else if strings.Contains(flags.Conf.MinSupportedOpenshiftVersion, "-") {
-		return fmt.Errorf("minimum supported openshift version in configuration file should not be a range")
-	}
-	return nil
-}
-
 // createPRPreRunE are pre-run operations for the create pull request command
 func createPRPreRunE(cmd *cobra.Command, args []string) error {
 	if flags.Dir == "" {
@@ -134,23 +108,6 @@ func createPRPreRunE(cmd *cobra.Command, args []string) error {
 	return nil
 }
 
-// doGenerate will generate the operator bundle metadata
-func doGenerate(_ *cobra.Command, _ []string) error {
-	dir := filepath.Join(flags.Dir, "certified-operators", flags.Conf.NewVersion)
-	err := opm.GenerateBundle(opm.GenerateConfig{
-		LocalDirectory:  dir,
-		OutputDirectory: dir,
-	})
-	if err != nil {
-		return err
-	}
-	err = opm.EnsureAnnotations(path.Join(dir, "metadata", "annotations.yaml"), flags.Conf.MinSupportedOpenshiftVersion)
-	if err != nil {
-		return err
-	}
-	return opm.EnsureLabels(path.Join(flags.Dir, "bundle.Dockerfile"), flags.Conf.MinSupportedOpenshiftVersion)
-}
-
 // doCreatePR will execute a number of local, and potentially remote github operations
 // for each of the certified, and community github repositories.
 // 1. Clone the repository to a temporary directory

diff --git a/hack/operatorhub/cmd/flags/flags.go b/hack/operatorhub/cmd/flags/flags.go
@@ -76,22 +76,22 @@ var (
 
 // Config is the configuration that matches the config.yaml
 type Config struct {
-	NewVersion                   string `json:"newVersion"`
-	PrevVersion                  string `json:"prevVersion"`
-	StackVersion                 string `json:"stackVersion"`
-	MinSupportedOpenshiftVersion string `json:"minSupportedOpenShiftVersion"`
-	CRDs                         []struct {
+	NewVersion   string `json:"newVersion"`
+	PrevVersion  string `json:"prevVersion"`
+	StackVersion string `json:"stackVersion"`
+	CRDs         []struct {
 		Name        string `json:"name"`
 		DisplayName string `json:"displayName"`
 		Description string `json:"description"`
 	} `json:"crds"`
 	Packages []struct {
-		OutputPath          string `json:"outputPath"`
-		PackageName         string `json:"packageName"`
-		DistributionChannel string `json:"distributionChannel"`
-		OperatorRepo        string `json:"operatorRepo"`
-		UbiOnly             bool   `json:"ubiOnly"`
-		DigestPinning       bool   `json:"digestPinning"`
+		OutputPath                   string `json:"outputPath"`
+		PackageName                  string `json:"packageName"`
+		DistributionChannel          string `json:"distributionChannel"`
+		MinSupportedOpenShiftVersion string `json:"minSupportedOpenShiftVersion"`
+		OperatorRepo                 string `json:"operatorRepo"`
+		UbiOnly                      bool   `json:"ubiOnly"`
+		DigestPinning                bool   `json:"digestPinning"`
 	} `json:"packages"`
 }
 

diff --git a/hack/operatorhub/config.yaml b/hack/operatorhub/config.yaml
@@ -1,7 +1,6 @@
 newVersion: 2.9.0
 prevVersion: 2.8.0
 stackVersion: 8.9.0
-minSupportedOpenshiftVersion: v4.9
 crds:
   - name: elasticsearches.elasticsearch.k8s.elastic.co
     displayName: Elasticsearch Cluster
@@ -39,14 +38,10 @@ packages:
     distributionChannel: community-operators
     operatorRepo: docker.elastic.co/eck/eck-operator
     ubiOnly: false
-  - outputPath: upstream-community-operators
-    packageName: elastic-cloud-eck
-    distributionChannel: upstream-community-operators
-    operatorRepo: docker.elastic.co/eck/eck-operator
-    ubiOnly: false
   - outputPath: certified-operators
     packageName: elasticsearch-eck-operator-certified
     distributionChannel: certified-operators
+    minSupportedOpenshiftVersion: v4.9
     operatorRepo: registry.connect.redhat.com/elastic/eck-operator
     ubiOnly: true
     ## digestPinning should only be set to true for certified operator.

diff --git a/hack/operatorhub/go.mod b/hack/operatorhub/go.mod
@@ -8,9 +8,7 @@ require (
 	github.com/go-git/go-git/v5 v5.7.0
 	github.com/google/go-containerregistry v0.15.2
 	github.com/hashicorp/vault/api v1.9.2
-	github.com/operator-framework/operator-registry v1.28.0
 	github.com/otiai10/copy v1.12.0
-	github.com/sirupsen/logrus v1.9.3
 	github.com/spf13/cobra v1.7.0
 	github.com/spf13/viper v1.16.0
 	k8s.io/api v0.26.1
@@ -20,72 +18,35 @@ require (
 )
 
 require (
-	github.com/adrg/xdg v0.4.0 // indirect
-	github.com/bshuster-repo/logrus-logstash-hook v1.0.2 // indirect
-	github.com/cenkalti/backoff/v4 v4.1.3 // indirect
 	github.com/cloudflare/circl v1.3.3 // indirect
 	github.com/docker/cli v23.0.5+incompatible // indirect
 	github.com/docker/docker v23.0.5+incompatible // indirect
 	github.com/go-jose/go-jose/v3 v3.0.0 // indirect
-	github.com/go-logr/stdr v1.2.2 // indirect
-	github.com/google/go-cmp v0.5.9 // indirect
-	github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0 // indirect
-	github.com/mattn/go-sqlite3 v1.14.16 // indirect
-	github.com/moby/locker v1.0.1 // indirect
-	github.com/moby/sys/mountinfo v0.4.1 // indirect
 	github.com/pjbgf/sha1cd v0.3.0 // indirect
+	github.com/sirupsen/logrus v1.9.3 // indirect
 	github.com/skeema/knownhosts v1.1.1 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.10.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.10.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.10.0 // indirect
 	gotest.tools/v3 v3.4.0 // indirect
-	rsc.io/letsencrypt v0.0.3 // indirect
 )
 
 require (
-	github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
 	github.com/Masterminds/goutils v1.1.1 // indirect
 	github.com/Masterminds/semver/v3 v3.2.0 // indirect
 	github.com/Microsoft/go-winio v0.6.1 // indirect
-	github.com/Microsoft/hcsshim v0.8.25 // indirect
 	github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903 // indirect
 	github.com/acomagu/bufpipe v1.0.4 // indirect
-	github.com/antlr/antlr4/runtime/Go/antlr v1.4.10 // indirect
-	github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a // indirect
-	github.com/beorn7/perks v1.0.1 // indirect
-	github.com/blang/semver/v4 v4.0.0 // indirect
 	github.com/cenkalti/backoff/v3 v3.0.0 // indirect
-	github.com/cespare/xxhash/v2 v2.2.0 // indirect
-	github.com/containerd/cgroups v1.0.3 // indirect
-	github.com/containerd/containerd v1.5.18 // indirect
-	github.com/containerd/continuity v0.3.0 // indirect
 	github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
-	github.com/containerd/ttrpc v1.1.0 // indirect
-	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/docker/distribution v2.8.2+incompatible // indirect
 	github.com/docker/docker-credential-helpers v0.7.0 // indirect
-	github.com/docker/go-connections v0.4.0 // indirect
-	github.com/docker/go-metrics v0.0.1 // indirect
-	github.com/docker/go-units v0.5.0 // indirect
 	github.com/emirpasic/gods v1.18.1 // indirect
-	github.com/felixge/httpsnoop v1.0.3 // indirect
 	github.com/fsnotify/fsnotify v1.6.0 // indirect
 	github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
 	github.com/go-git/go-billy/v5 v5.4.1 // indirect
 	github.com/go-logr/logr v1.2.3 // indirect
-	github.com/go-openapi/jsonpointer v0.19.5 // indirect
-	github.com/go-openapi/jsonreference v0.20.0 // indirect
-	github.com/go-openapi/swag v0.19.14 // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
-	github.com/golang/protobuf v1.5.3 // indirect
-	github.com/google/cel-go v0.12.6 // indirect
-	github.com/google/gnostic v0.5.7-v3refs // indirect
 	github.com/google/gofuzz v1.1.0 // indirect
 	github.com/google/uuid v1.3.0 // indirect
-	github.com/gorilla/mux v1.8.0 // indirect
-	github.com/h2non/filetype v1.1.1 // indirect
-	github.com/h2non/go-is-svg v0.0.0-20160927212452-35e8c4b0612c // indirect
 	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
 	github.com/hashicorp/go-multierror v1.1.1 // indirect
@@ -99,80 +60,46 @@ require (
 	github.com/imdario/mergo v0.3.15 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
-	github.com/josharian/intern v1.0.0 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/kevinburke/ssh_config v1.2.0 // indirect
 	github.com/klauspost/compress v1.16.5 // indirect
 	github.com/magiconair/properties v1.8.7 // indirect
-	github.com/mailru/easyjson v0.7.6 // indirect
-	github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
 	github.com/mitchellh/copystructure v1.1.1 // indirect
 	github.com/mitchellh/go-homedir v1.1.0 // indirect
 	github.com/mitchellh/mapstructure v1.5.0 // indirect
 	github.com/mitchellh/reflectwalk v1.0.1 // indirect
-	github.com/moby/term v0.5.0 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
-	github.com/morikuni/aec v1.0.0 // indirect
-	github.com/onsi/gomega v1.24.1 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/opencontainers/image-spec v1.1.0-rc3 // indirect
-	github.com/operator-framework/api v0.17.4-0.20230223191600-0131a6301e42 // indirect
 	github.com/pelletier/go-toml/v2 v2.0.8 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
-	github.com/pmezard/go-difflib v1.0.0 // indirect
-	github.com/prometheus/client_golang v1.14.0 // indirect
-	github.com/prometheus/client_model v0.3.0 // indirect
-	github.com/prometheus/common v0.37.0 // indirect
-	github.com/prometheus/procfs v0.8.0 // indirect
 	github.com/ryanuber/go-glob v1.0.0 // indirect
 	github.com/sergi/go-diff v1.1.0 // indirect
 	github.com/shopspring/decimal v1.2.0 // indirect
 	github.com/spf13/afero v1.9.5 // indirect
 	github.com/spf13/cast v1.5.1 // indirect
 	github.com/spf13/jwalterweatherman v1.1.0 // indirect
 	github.com/spf13/pflag v1.0.5
-	github.com/stoewer/go-strcase v1.2.0 // indirect
-	github.com/stretchr/testify v1.8.3 // indirect
 	github.com/subosito/gotenv v1.4.2 // indirect
 	github.com/vbatts/tar-split v0.11.3 // indirect
 	github.com/xanzy/ssh-agent v0.3.3 // indirect
-	go.etcd.io/bbolt v1.3.6 // indirect
-	go.opencensus.io v0.24.0 // indirect
-	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.35.0 // indirect
-	go.opentelemetry.io/otel v1.10.0 // indirect
-	go.opentelemetry.io/otel/metric v0.31.0 // indirect
-	go.opentelemetry.io/otel/sdk v1.10.0 // indirect
-	go.opentelemetry.io/otel/trace v1.10.0 // indirect
-	go.opentelemetry.io/proto/otlp v0.19.0 // indirect
 	golang.org/x/crypto v0.9.0 // indirect
 	golang.org/x/mod v0.10.0 // indirect
 	golang.org/x/net v0.10.0 // indirect
-	golang.org/x/oauth2 v0.7.0 // indirect
 	golang.org/x/sync v0.2.0 // indirect
 	golang.org/x/sys v0.8.0 // indirect
-	golang.org/x/term v0.8.0 // indirect
 	golang.org/x/text v0.9.0 // indirect
 	golang.org/x/time v0.3.0 // indirect
 	golang.org/x/tools v0.9.1 // indirect
-	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
-	google.golang.org/grpc v1.55.0 // indirect
-	google.golang.org/protobuf v1.30.0 // indirect
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
-	gopkg.in/square/go-jose.v2 v2.6.0 // indirect
 	gopkg.in/warnings.v0 v0.1.2 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
-	k8s.io/apiserver v0.26.1 // indirect
 	k8s.io/client-go v0.26.1 // indirect
-	k8s.io/component-base v0.26.1 // indirect
 	k8s.io/klog/v2 v2.80.1 // indirect
-	k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 // indirect
 	k8s.io/utils v0.0.0-20221128185143-99ec85e7a448 // indirect
-	sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.35 // indirect
-	sigs.k8s.io/controller-runtime v0.14.4 // indirect
 	sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect
 	sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
 	sigs.k8s.io/yaml v1.3.0 // indirect