Backport of ci: Single source of truth for Go version in CI and Dockerfile into release/1.16.x

[hc-github-team-consul-core]

Backport

This PR is auto-generated from #20058 to be assessed for backporting due to the inclusion of the label backport/1.16.

🚨

Warning automatic cherry-pick of commits failed. If the first commit failed,
you will see a blank no-op commit below. If at least one commit succeeded, you
will see the cherry-picked commits up to, not including, the commit where
the merge conflict occurred.

The person who merged in the original PR is:
@zalimeni
This person should manually cherry-pick the original PR into a new backport PR,
and close this one when the manual backport PR is merged in.

merge conflict error: POST https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict []

The below text is copied from the body of the original PR.

---

Description

To simplify Go version management and enable simpler conflict-less (CE + Ent) upgrades in the future, move all Go version management for CI, Docker builds, and related tooling to a single source of truth.

To do this, introduce a new .go-version file and reusable workflow to read it that all jobs using setup-go can depend on. Having a separate reusable job allows multiple dependent jobs to use the version after it's determined once. It also enables us to centralize control over the version in our own CI config rather than depending on the behavior of setup-go or other imported actions (see more below).

Make all reusable workflows require a Go version as input s.t. we can ensure it is always provided.

Rationale

We have two needs for our CI when it comes to Go versions:

• Using the most current version of Go that we support to take advantage of security patches and bugfixes. This should apply to our produced binaries and Docker containers at minimum; it should also apply to most tests to keep in sync w/ binaries and local dev.
• Proving backwards compatibility w/ our indicated minimum-required version; this primarily impacts consumers of our submodules and mitigates the risk of backwards-incompatible changes to repository code (e.g. using a new std lib function). This is accomplished by version-specific tests such as this for our published submodules.

When we upgrade to Go 1.21, we'll have the new toolchain directive available in addition to the go directive. More on how these work here; the salient point is that Go now attempts to install the correct toolchain for us when we update the go or toolchain directive and the local version of Go does not satisfy those versions. It also means that the go directive provides an enforced minimum version, not the suggested one.

However, this presents some rough edges in CI:

• setup-go does not yet support toolchain detection (see also Tar errors on cache restore after toolchain installation actions/setup-go#424)
• We don't want to install Go twice in every job due to a mismatch between the version installed by setup-go and the one required locally; this sort of mistake would be easy to overlook
• In addition to setup-go, we rely on hashicorp/action-go-build to produce binaries, which has a required input for the Go version that is not managed by setup-go; managing this separately from setup-go runs the risk of error when changing Go versions

Given these constraints, and the clarity provided by setting essential versions explicitly to avoid confusion or side-effect mistakes, this change proposes enforcing a consistently installed Go version throughout our workflows, still allowing for explicit overrides where we need a different version. It's easy to reverse if we want to go back to individual go-version-file declarations.

Futures

My hope is that we can move to calling go version to retrieve the toolchain version provided in Go 1.21+, as well as introduce a versioned go.work in the near future (which will bring several benefits, including removing submodule replaces, explicitly setting Go version via a single toolchain directive). The current change allows us to control the version from a single source of truth in CI, which is beneficial regardless of Go-native toolchain management changes in the future.

Testing & Reproduction steps

• CI continues to pass.
• Overrides for backwards compatibility tests still result in the desired version: Go 1.19 api and sdk tests
• Checked for missed updates (explicit Go versions are backwards compatibility tests noted above):

rg -t yaml 'go-version-file:' -B1
test-integrations.yml
101-          # Do not explicitly set Go version here, as it should depend on what Nomad declares.
102:          go-version-file: 'go.mod'
--
184-          # Do not explicitly set Go version here, as it should depend on what Vault declares.
185:          go-version-file: 'go.mod'

rg -t yaml 'go-version: ' | grep -v 'needs.get-go-version' | grep -v 'inputs.go-version'
go-tests.yml:      go-version: "1.19"
go-tests.yml:      go-version: "1.20"
go-tests.yml:      go-version: "1.19"
go-tests.yml:      go-version: "1.20"
reusable-get-go-version.yml:      go-version: ${{ steps.get-go-version.outputs.go-version }}

PR Checklist

• updated test coverage
• external facing docs updated
• appropriate backport labels added
• not a security concern

---

Overview of commits

• ea888cd

[hashicorp-cla]

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes

---

temp seems not to be a GitHub user.
You need a GitHub account to be able to sign the CLA. If you already have a GitHub account, please add the email address used for this commit to your account.

_{Have you signed the CLA already but the status is still pending? Recheck it.}

[github-team-consul-core-pr-approver]

Auto approved Consul Bot automated PR