Skip to content

Commit

Permalink
RE 2997 error building chainlink binary with wasmtime go dependency (#…
Browse files Browse the repository at this point in the history
…14583)

* Refactor .goreleaser.develop.yaml to use multi-platform native compilation

* Make required changes to get platform native builds working

* Programmatic goreleaser config generator, works with .goreleaser.develop.yaml

* Remove broken goreleaser makefile cmds

* Remove zig dep from shell.nix

* Remove macos-sdk, goreleaser exec, zig refs from goreleaser action

* Use no_unique_dist_dir config since we only build 1 target a time

* Remove qemu support

* Use ubuntu 24 for goreleaser base image

* Test split builds w/o merge

* Add sensible default for CHAINLINK_VERSION

* Set chainlink version in github action

* Merge ccip and regular builds together

* Use nightlies over snapshots

* Split and merge

* Correctly set release type

* Quote nullable var

* Pass down release type

* goreleaserpro -> goreleaser

* Set nightly version correctly

* Add fetch depth

* Make name more accurate

* Fix merge cmd

* Disable changelogs + archives unless prod

* Update develop config file

* Sign nightly images

* Handle prod image name prefix

* prod -> production

* Remove stale fixtures

* Add production config generation

* Correctly add ECR path to prod images

* Merge production + ccip production together

* Disable changelog on develop

* Remove env var shadowing and redundant templating

* Remove signing for develop builds

* Fix nightly version template

* Refactor build-sign-publish inputs

* Fix skippush condition, remove cosign signing

* Nuke cosign from goreleaser action in favor of gh artifact attestation

* Apply split+merge refactor to prod pipeline

* Run gomodtidy

* Remove useless test

* Update go.mod

* Remove push on release/** trigger

* Add fetch-depth 0 to image builds

* Use a separate workflow for goreleaser

* Update gomods

* TEST: goreleaser prod

* Add missing env

* Fix yaml extension

* Add missing fetch depth

* Fix incorrect manifest naming

* Configure skip_push for prod manifests

* Refactor artifact path handling in build-publish-goreleaser.yml

* Remove artifact attestation

* Update go.mod

* Remove test branch trigger
  • Loading branch information
HenryNguyen5 authored Oct 2, 2024
1 parent be774f0 commit 65f1d9d
Show file tree
Hide file tree
Showing 22 changed files with 1,409 additions and 1,467 deletions.
56 changes: 0 additions & 56 deletions .github/actions/goreleaser-build-sign-publish/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@ jobs:
permissions:
id-token: write
contents: read
env:
MACOS_SDK_VERSION: 12.3
steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
Expand All @@ -32,28 +30,11 @@ jobs:
role-to-assume: ${{ secrets.aws-role-arn }}
role-duration-seconds: ${{ secrets.aws-role-dur-sec }}
aws-region: ${{ secrets.aws-region }}
- name: Cache macos sdk
id: sdk-cache
uses: actions/cache@v3
with:
path: ${{ format('MacOSX{0}.sdk', env.MAC_SDK_VERSION) }}
key: ${{ runner.OS }}-${{ env.MAC_SDK_VERSION }}-macos-sdk-cache-${{ hashFiles('**/SDKSettings.json') }}
restore-keys: |
${{ runner.OS }}-${{ env.MAC_SDK_VERSION }}-macos-sdk-cache-
- name: Get macos sdk
if: steps.sdk-cache.outputs.cache-hit != 'true'
run: |
curl -L https://github.com/joseluisq/macosx-sdks/releases/download/${MACOS_SDK_VERSION}/MacOSX${MACOS_SDK_VERSION}.sdk.tar.xz > MacOSX${MACOS_SDK_VERSION}.sdk.tar.xz
tar -xf MacOSX${MACOS_SDK_VERSION}.sdk.tar.xz
- name: Build, sign, and publish
uses: ./.github/actions/goreleaser-build-sign-publish
with:
enable-docker-publish: "true"
enable-goreleaser-snapshot: "false"
docker-registry: ${{ secrets.aws-ecr-registry }}
goreleaser-exec: goreleaser
goreleaser-config: .goreleaser.yaml
macos-sdk-dir: ${{ format('MacOSX{0}.sdk', env.MAC_SDK_VERSION) }}
env:
GITHUB_TOKEN: ${{ secrets.gh-token }}
```
Expand All @@ -64,29 +45,8 @@ jobs:
- name: Build, sign, and publish image
uses: ./.github/actions/goreleaser-build-sign-publish
with:
enable-docker-publish: "true"
enable-goreleaser-snapshot: "true"
docker-registry: ${{ secrets.aws-ecr-registry }}
goreleaser-exec: goreleaser
goreleaser-config: .goreleaser.yaml
```
### image signing
```yaml
- name: Build, sign, and publish
uses: ./.github/actions/goreleaser-build-sign-publish
with:
enable-docker-publish: "true"
enable-goreleaser-snapshot: "false"
enable-cosign: "true"
docker-registry: ${{ secrets.aws-ecr-registry }}
goreleaser-exec: goreleaser
goreleaser-config: .goreleaser.yaml
cosign-password: ${{ secrets.cosign-password }}
cosign-public-key: ${{ secrets.cosign-public-key }}
cosign-private-key: ${{ secrets.cosign-private-key }}
macos-sdk-dir: MacOSX12.3.sdk
```
## customizing
Expand All @@ -98,22 +58,9 @@ Following inputs can be used as `step.with` keys
| Name | Type | Default | Description |
| ---------------------------- | ------ | ------------------ | ----------------------------------------------------------------------- |
| `goreleaser-version` | String | `~> v2` | `goreleaser` version |
| `zig-version` | String | `0.10.1` | `zig` version |
| `cosign-version` | String | `v2.2.2` | `cosign` version |
| `macos-sdk-dir` | String | `MacOSX12.3.sdk` | MacOSX sdk directory |
| `enable-docker-publish` | Bool | `true` | Enable publishing of Docker images / manifests |
| `docker-registry` | String | `localhost:5001` | Docker registry |
| `docker-image-name` | String | `chainlink` | Docker image name |
| `docker-image-tag` | String | `develop` | Docker image tag |
| `enable-goreleaser-snapshot` | Bool | `false` | Enable goreleaser build / release snapshot |
| `enable-goreleaser-split` | Bool | `false` | Enable goreleaser build using split and merge |
| `goreleaser-split-arch` | String | `""` | The arch to build the image with - amd64, arm64 |
| `goreleaser-exec` | String | `goreleaser` | The goreleaser executable, can invoke wrapper script |
| `goreleaser-config` | String | `.goreleaser.yaml` | The goreleaser configuration yaml |
| `enable-cosign` | Bool | `false` | Enable signing of Docker images |
| `cosign-public-key` | String | `""` | The public key to be used with cosign for verification |
| `cosign-private-key` | String | `""` | The private key to be used with cosign to sign the image |
| `cosign-password-key` | String | `""` | The password to decrypt the cosign private key needed to sign the image |

## testing

Expand All @@ -126,10 +73,7 @@ docker run -d --restart=always -p "127.0.0.1:5001:5000" --name registry registry
- run snapshot release, publish to local docker registry

```sh
GORELEASER_EXEC="<goreleaser-wrapper" \
GORELEASER_CONFIG=".goreleaser.yaml" \
ENABLE_GORELEASER_SNAPSHOT=true \
ENABLE_DOCKER_PUBLISH=true \
DOCKER_MANIFEST_EXTRA_ARGS="--insecure" \
./.github/actions/goreleaser-build-sign-publish/action_utils goreleaser_release
```
74 changes: 9 additions & 65 deletions .github/actions/goreleaser-build-sign-publish/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,67 +8,29 @@ inputs:
goreleaser-key:
description: The goreleaser key
required: false
zig-version:
description: The zig version
default: 0.10.1
required: false
cosign-version:
description: The cosign version
default: v2.4.0
required: false
macos-sdk-dir:
description: The macos sdk directory
default: MacOSX12.3.sdk
required: false
# publishing inputs
enable-docker-publish:
description: Enable publishing of docker images / manifests
default: "true"
required: false
docker-registry:
description: The docker registry
default: localhost:5001
required: false
docker-image-name:
description: The docker image name
default: chainlink
required: false
docker-image-tag:
description: The docker image tag
default: develop
required: false
# goreleaser inputs
goreleaser-exec:
description: "The goreleaser executable, can invoke wrapper script"
default: "goreleaser"
goreleaser-release-type:
description: The goreleaser release type, it can be either "nightly", "merge", "snapshot", "release"
default: "snapshot"
required: false
goreleaser-config:
description: "The goreleaser configuration yaml"
default: ".goreleaser.yaml"
required: false
enable-goreleaser-snapshot:
description: Enable goreleaser build / release snapshot
default: "false"
required: false
enable-goreleaser-split:
description: Enable goreleaser split and merge builds
default: "false"
required: false
goreleaser-split-arch:
description: The architecture to split the goreleaser build
required: false
# signing inputs
enable-cosign:
description: Enable signing of docker images
default: "false"
required: false
runs:
using: composite
steps:
- name: Setup docker buildx
uses: docker/setup-buildx-action@2b51285047da1547ffb1b2203d8be4c0af6b1f20 # v3.2.0
- name: Set up qemu
uses: docker/setup-qemu-action@68827325e0b33c7199eb31dd4e31fbe9023e06e3 # v3.0.0
- name: Setup go
uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0
with:
Expand All @@ -81,42 +43,24 @@ runs:
version: ${{ inputs.goreleaser-version }}
env:
GORELEASER_KEY: ${{ inputs.goreleaser-key }}
- name: Setup zig
uses: goto-bus-stop/setup-zig@7ab2955eb728f5440978d5824358023be3a2802d # v2.2.0
with:
version: ${{ inputs.zig-version }}
- name: Setup cosign
if: inputs.enable-cosign == 'true'
uses: sigstore/cosign-installer@4959ce089c160fddf62f7b42464195ba1a56d382 # v3.6.0
with:
cosign-release: ${{ inputs.cosign-version }}

- name: Login to docker registry
if: inputs.enable-docker-publish == 'true'
uses: docker/login-action@e92390c5fb421da1463c202d546fed0ec5c39f20 # v3.1.0
with:
registry: ${{ inputs.docker-registry }}
- name: Set goreleaser split env
if: inputs.enable-goreleaser-split == 'true'
shell: bash
run: |
echo "GOOS=linux" | tee -a $GITHUB_ENV
echo "GOARCH=${{ inputs.goreleaser-split-arch }}" | tee -a $GITHUB_ENV

- name: Install syft
uses: anchore/sbom-action/download-syft@61119d458adab75f756bc0b9e4bde25725f86a7a # v0.17.2

- name: Run goreleaser release
shell: bash
env:
ENABLE_GORELEASER_SNAPSHOT: ${{ inputs.enable-goreleaser-snapshot }}
ENABLE_GORELEASER_SPLIT: ${{ inputs.enable-goreleaser-split }}
ENABLE_DOCKER_PUBLISH: ${{ inputs.enable-docker-publish }}
GORELEASER_CONFIG: ${{ inputs.goreleaser-config }}
RELEASE_TYPE: ${{ inputs.goreleaser-release-type }}
IMAGE_PREFIX: ${{ inputs.docker-registry }}
IMAGE_NAME: ${{ inputs.docker-image-name }}
IMAGE_TAG: ${{ inputs.docker-image-tag }}
GORELEASER_EXEC: ${{ inputs.goreleaser-exec }}
GORELEASER_CONFIG: ${{ inputs.goreleaser-config }}
GORELEASER_KEY: ${{ inputs.goreleaser-key }}
GITHUB_TOKEN: ${{ github.token }}
MACOS_SDK_DIR: ${{ inputs.macos-sdk-dir }}
run: |
# https://github.com/orgs/community/discussions/24950
${GITHUB_ACTION_PATH}/action_utils goreleaser_release
${GITHUB_ACTION_PATH}/release.js
82 changes: 0 additions & 82 deletions .github/actions/goreleaser-build-sign-publish/action_utils

This file was deleted.

93 changes: 93 additions & 0 deletions .github/actions/goreleaser-build-sign-publish/release.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
#!/usr/bin/env node
const { execSync } = require("child_process");

function main() {
const goreleaserConfig = mustGetEnv("GORELEASER_CONFIG");
const releaseType = mustGetEnv("RELEASE_TYPE");
const command = constructGoreleaserCommand(releaseType, goreleaserConfig);

if (process.env.DRY_RUN) {
console.log(`Generated command: ${command}`);
console.log("Dry run enabled. Exiting without executing the command.");
return;
} else {
console.log(`Executing command: ${command}`);
execSync(command, { stdio: "inherit" });
}
}

main();

function constructGoreleaserCommand(releaseType, goreleaserConfig) {
const version = getVersion();
const flags = [];

checkReleaseType(releaseType);

let subCmd = "release";
const splitArgs = ["--split", "--clean"];

switch (releaseType) {
case "release":
flags.push(...splitArgs);
break;
case "nightly":
flags.push("--nightly", ...splitArgs);
break;
case "snapshot":
flags.push("--snapshot", ...splitArgs);
break;
case "merge":
flags.push("--merge");
subCmd = "continue";
break;
}

const flagsStr = flags.join(" ");
if (releaseType === "merge") {
return `CHAINLINK_VERSION=${version} goreleaser ${subCmd} ${flagsStr}`;
} else {
return `CHAINLINK_VERSION=${version} goreleaser ${subCmd} --config ${goreleaserConfig} ${flagsStr}`;
}
}

function checkReleaseType(releaseType) {
const VALID_RELEASE_TYPES = ["nightly", "merge", "snapshot", "release"];

if (!VALID_RELEASE_TYPES.includes(releaseType)) {
const validReleaseTypesStr = VALID_RELEASE_TYPES.join(", ");
console.error(
`Error: Invalid release type: ${releaseType}. Must be one of: ${validReleaseTypesStr}`
);
}
}

function mustGetEnv(key) {
const val = process.env[key];
if (!val || val.trim() === "") {
console.error(`Error: Environment variable ${key} is not set or empty.`);
process.exit(1);
}

return val.trim();
}

function getVersion() {
try {
const pkgPath = process.cwd() + "/package.json";
console.log("Looking for chainlink version in package.json at: ", pkgPath);
const packageJson = require(pkgPath);
if (!packageJson.version) {
console.error(
'Error: "version" field is missing or empty in package.json.'
);
process.exit(1);
}
console.log("Resolved version: ", packageJson.version);

return packageJson.version;
} catch (err) {
console.error(`Error reading package.json: ${err.message}`);
process.exit(1);
}
}
Loading

0 comments on commit 65f1d9d

Please sign in to comment.