Allow Clusterctl upgrade plan/apply to use --config to upgrade to specific version

[robinAwallace]

What would you like to be added (User Story)?

Hello, Im not sure if this is a bug or a feature. But as a operator I would like to control which version to upgrade to using the --config flag, as you can do with init.

I know I can use arguments to control this behavior, but as there is a flag --config it would be nice to use it for upgrade plan/apply as well.

Detailed Description

When I run the plan command today with --config and with the debug flag I see that the config is picked up, but after fetching latest versions upstream.

Checking new release availability...
Potential override file SearchFile="/home/robin/.config/cluster-api/overrides/cluster-api/v1.8.1/metadata.yaml" Provider="cluster-api" Version="v1.8.1"
Fetching File="metadata.yaml" Provider="cluster-api" Type="CoreProvider" Version="v1.8.1"
Potential override file SearchFile="/home/robin/.config/cluster-api/overrides/bootstrap-kubeadm/v1.8.1/metadata.yaml" Provider="bootstrap-kubeadm" Version="v1.8.1"
Fetching File="metadata.yaml" Provider="kubeadm" Type="BootstrapProvider" Version="v1.8.1"
Potential override file SearchFile="/home/robin/.config/cluster-api/overrides/control-plane-kubeadm/v1.8.1/metadata.yaml" Provider="control-plane-kubeadm" Version="v1.8.1"
Fetching File="metadata.yaml" Provider="kubeadm" Type="ControlPlaneProvider" Version="v1.8.1"
Potential override file SearchFile="/home/robin/.config/cluster-api/overrides/cluster-api/v1.8.1/metadata.yaml" Provider="cluster-api" Version="v1.8.1"
Fetching File="metadata.yaml" Provider="cluster-api" Type="CoreProvider" Version="v1.8.1"
Potential override file SearchFile="/home/robin/.config/cluster-api/overrides/infrastructure-openstack/v0.10.4/metadata.yaml" Provider="infrastructure-openstack" Version="v0.10.4"
Fetching File="metadata.yaml" Provider="openstack" Type="InfrastructureProvider" Version="v0.10.4"

Latest release available for the v1beta1 API Version of Cluster API (contract):

NAME                       NAMESPACE                           TYPE                     CURRENT VERSION   NEXT VERSION
bootstrap-kubeadm          capi-kubeadm-bootstrap-system       BootstrapProvider        v1.7.4            v1.8.1
control-plane-kubeadm      capi-kubeadm-control-plane-system   ControlPlaneProvider     v1.7.4            v1.8.1
cluster-api                capi-system                         CoreProvider             v1.7.4            v1.8.1
infrastructure-openstack   capo-system                         InfrastructureProvider   v0.10.4           Already up to date

You can now apply the upgrade by executing the following command:

clusterctl upgrade apply --contract v1beta1

Using configuration File="/home/robin/Documents/capi/clusterctl-config.yaml"

/home/robin/Documents/capi/clusterctl-config.yaml:

providers:
  - name: "openstack"
    url: "https://github.com/kubernetes-sigs/cluster-api-provider-openstack/releases/v0.10.4/infrastructure-components.yaml"
    type: "InfrastructureProvider"
  - name: "cluster-api"
    url: "https://github.com/kubernetes-sigs/cluster-api/releases/v1.7.4/core-components.yaml"
    type: "CoreProvider"
  - name: "kubeadm"
    url: "https://github.com/kubernetes-sigs/cluster-api/releases/v1.7.4/bootstrap-components.yaml"
    type: "BootstrapProvider"
  - name: "kubeadm"
    url: "https://github.com/kubernetes-sigs/cluster-api/releases/v1.7.4/control-plane-components.yaml"
    type: ControlPlaneProvider

What I expect is that the version should be held back depending on my config file.

Anything else you would like to add?

clusterctl version
clusterctl version: &version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.4", GitCommit:"53f0c5375dbd0ba307db2b92d3e5fff6e3ee6d63", GitTreeState:"clean", BuildDate:"2024-07-09T15:58:50Z", GoVersion:"go1.22.5", Compiler:"gc", Platform:"linux/amd64"}

Label(s) to be applied

/kind feature
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.

[k8s-ci-robot]

This issue is currently awaiting triage.

If CAPI contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[fabriziopandini]

I'm not sure I agree with the idea of using the config file to control versions.

The config file is used to define custom provider repository and where they are located (and to provide variables).
Instead, available versions are not expected to defined in the config file, they are read from the provider repository itself.

Please note that the fact that the github providers has a version in its url its sort of "incidental" (this is how the GitHub url is, not a clusterctl feature; as a proof of this, you can replace version with latest and everything works the same)

[robinAwallace]

I'm not sure I agree with the idea of using the config file to control versions.

The config file is used to define custom provider repository and where they are located (and to provide variables). Instead, available versions are not expected to defined in the config file, they are read from the provider repository itself.

Okay sure, our thought was that it is a easy and gitops friendly way to know which version we are using/upgrading to. And it is a bit strange that it did not work, as it is a global flag specifying repos. 🙂

Please note that the fact that the github providers has a version in its url its sort of "incidental" (this is how the GitHub url is, not a clusterctl feature; as a proof of this, you can replace version with latest and everything works the same)

Well I guess that is true, but it is also common practice to link/use a specific release via its url hosted on github as I tried to do in my config.

Is there anyway you could reconsider? 🙏

[fabriziopandini]

Frankly speaking I don't think this is the way to go, there are better alternatives.

e.g. Most of the CAPI users use a local repository or a custom repository to control which version a users can access, because they validate/support only specific versions and usually ships custom templates.

[simonklb]

Please correct me if I'm wrong but when initializing a new CAPI management cluster, without specifying a config, a lookup is performed to determine the provider manifests.

The reason why we ended up here is that (before v1.7) when there was a new provider version released there can be a lag before a GitHub release has been created as well. This caused the lookup to fail with:

failed to create repo from provider url for provider \"openstack\": error creating the GitHub repository client: failed to get latest release: release not found for version v0.10.2, please retry later or set \"GOPROXY=off\" to get the current stable release: 404 Not Found

To solve this we now run clusterctl init with the --config flag and this works great. It also lets us pin which version we want to initialize the management cluster with as well as letting us replace it with a fork easily.

I see the hesitancy for allowing the upgrade versions to be provided via the configuration, but it feels surprising from a user perspective that the initialization utilize the config while the upgrade does not.

[fabriziopandini]

You are correct, there is no agreement on the idea of using the config file to control which version to upgrade to.
The config file must be used only to define custom provider repository and where they are located (and to provide variables + a few other things). The config file must not be used to provide a list of version

But as far as I remember the --config flag is a global flag that applies to all commands, upgrade included.
And AFAIK upgrade is using the config file for the scope it was designed for (define custom provider repository and where they are located).

Note: I'm not sure about the correlation between running clusterctl init with the --config flag and the error when a release was in progress, but for this I defer to folks who did the actual work
cc @Fedosin @chrischdi (might be others)

[chrischdi]

Note: I'm not sure about the correlation between running clusterctl init with the --config flag and the error when a release was in progress, but for this I defer to folks who did the actual work
cc @Fedosin @chrischdi (might be others)

Yes, this should have been fixed via

• 🌱 test: check for metadata.yaml when resolving releases to not try to use unreleased versions + avoid retry on 404 (also in clusterctl) #10618

And be part of clusterctl >= v1.8.0 and v1.7.3

[fabriziopandini]

/close
Based on the comment above, the issue with unreleased version should be now fixed, please report back if it is not working for you with newer versions

[k8s-ci-robot]

@fabriziopandini: Closing this issue.

In response to this:

/close
Based on the comment above, the issue with unreleased version should be now fixed, please report back if it is not working for you with newer versions

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.