Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: goreleaser develop build for local and CI #11847

Merged
merged 54 commits into from
Feb 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
5032a1a
add updated goreleaser templates
momentmaker Jan 22, 2024
011d042
update zig 0.11.0
momentmaker Jan 22, 2024
26b0489
update goreleaser develop
momentmaker Jan 22, 2024
fa9f57a
add back ldflags and rename goreleaser config file
momentmaker Jan 23, 2024
09893a6
update goreleaser build gha workflow and pin new versions
momentmaker Jan 23, 2024
8f66085
add back pre/post hooks and updated goreleaser Dockerfile to include …
momentmaker Jan 24, 2024
e29a738
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Jan 26, 2024
87c9226
comment out integration-tests
momentmaker Jan 26, 2024
6cab8d6
add back goreleaser_wrapper
momentmaker Jan 26, 2024
3b8154d
fix path in goreleaser_utils
momentmaker Jan 26, 2024
21edc15
add multi-line json output support
momentmaker Jan 26, 2024
43d13a9
save
momentmaker Jan 26, 2024
8dc6294
revert everthing and add go mod tidy
momentmaker Jan 26, 2024
9e01db7
revert zig and goreleaser config
momentmaker Jan 26, 2024
3862bc9
update
momentmaker Jan 26, 2024
ccffe98
zig 0.10.1
momentmaker Jan 26, 2024
7427726
update go.mod go version
momentmaker Jan 26, 2024
a465bd2
add -shared in goreleaser
momentmaker Jan 26, 2024
6b2fc08
use zig 0.11.0 and update bash fail
momentmaker Jan 27, 2024
91dc601
add to cc
momentmaker Jan 27, 2024
279124c
update name
momentmaker Jan 27, 2024
8455ff1
test
momentmaker Jan 27, 2024
3666188
test
momentmaker Jan 27, 2024
a37ae17
add LD_LIBRARY_PATH in dockerfile
momentmaker Jan 27, 2024
129d497
add _transform_path func
momentmaker Jan 27, 2024
33b4845
fix post-hook cp
momentmaker Jan 27, 2024
39710d4
remove post hooks
momentmaker Jan 27, 2024
6d38b56
use zig
momentmaker Jan 27, 2024
89af873
use older zig version
momentmaker Jan 27, 2024
d881f06
try zig 0.12.0-dev
momentmaker Feb 6, 2024
918b48f
add back posthook
momentmaker Feb 7, 2024
c401f38
use zig 0.10.1 again
momentmaker Feb 13, 2024
43007fc
update go.mod to 1.21.5
momentmaker Feb 13, 2024
87c7aff
try older zig version
momentmaker Feb 13, 2024
e5d286d
another older zig version
momentmaker Feb 13, 2024
1547a6c
add ldd_fix script with patchelf
momentmaker Feb 15, 2024
d620ce5
uncomment
momentmaker Feb 15, 2024
ba18508
add comments and remove unnecessary
momentmaker Feb 15, 2024
03122b7
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 15, 2024
2256176
update go.mod version
momentmaker Feb 16, 2024
b502110
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 16, 2024
7bb991a
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 16, 2024
5045eee
update go.mod to 1.21.7 for core/scripts
momentmaker Feb 16, 2024
8e3bb5e
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 16, 2024
eece142
update all go.mod to 1.21.7 and remove artifact outputs
momentmaker Feb 16, 2024
cd0e2d1
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 16, 2024
bb20bb6
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 16, 2024
86adaf2
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 19, 2024
839d189
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 20, 2024
5254c3a
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 20, 2024
e9b4d86
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 20, 2024
5ef1f31
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 20, 2024
ebbade6
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 20, 2024
e81abeb
Merge branch 'develop' into re-2202/update-goreleaser-workflow
momentmaker Feb 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions .github/actions/goreleaser-build-sign-publish/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,24 @@ description: A composite action that allows building and publishing signed chain
inputs:
goreleaser-version:
description: The goreleaser version
default: 1.15.2
default: 1.23.0
required: false
goreleaser-key:
description: The goreleaser key
required: false
zig-version:
description: The zig version
default: 0.10.1
required: false
cosign-version:
description: The cosign version
default: v1.13.1
default: v2.2.2
required: false
macos-sdk-dir:
description: The macos sdk directory
default: MacOSX12.3.sdk
required: false
# publising inputs
# publishing inputs
enable-docker-publish:
description: Enable publishing of docker images / manifests
default: "true"
Expand Down Expand Up @@ -75,16 +78,18 @@ runs:
- name: Setup goreleaser
uses: goreleaser/goreleaser-action@7ec5c2b0c6cdda6e8bbb49444bc797dd33d74dd8 # v5.0.0
with:
distribution: goreleaser
distribution: goreleaser-pro
install-only: true
version: ${{ inputs.goreleaser-version }}
env:
GORELEASER_KEY: ${{ inputs.goreleaser-key }}
- name: Setup zig
uses: goto-bus-stop/setup-zig@7ab2955eb728f5440978d5824358023be3a2802d # v2.2.0
with:
version: ${{ inputs.zig-version }}
- name: Setup cosign
if: inputs.enable-cosign == 'true'
uses: sigstore/cosign-installer@11086d25041f77fe8fe7b9ea4e48e3b9192b8f19 # v3.1.2
uses: sigstore/cosign-installer@9614fae9e5c5eddabb09f90a270fcb487c9f7149 # v3.3.0
with:
cosign-release: ${{ inputs.cosign-version }}
- name: Login to docker registry
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#!/usr/bin/env bash
set -x
set -euo pipefail

ENABLE_COSIGN=${ENABLE_COSIGN:-false}
ENABLE_GORELEASER_SNAPSHOT=${ENABLE_GORELEASER_SNAPSHOT:-false}
Expand Down Expand Up @@ -70,9 +71,6 @@ goreleaser_release() {
rm -rf cosign.pub
rm -rf cosign.key
fi

echo "metadata=$(cat dist/metadata.json)" >> "$GITHUB_OUTPUT"
echo "artifacts=$(cat dist/artifacts.json)" >> "$GITHUB_OUTPUT"
}

"$@"
5 changes: 3 additions & 2 deletions .github/workflows/goreleaser-build-publish-develop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ jobs:
enable-goreleaser-snapshot: "true"
goreleaser-exec: ./tools/bin/goreleaser_wrapper
goreleaser-config: .goreleaser.develop.yaml
# ISSUE: https://github.com/golang/go/issues/52690
zig-version: 0.11.0-dev.3380+7e0a02ee2 # TODO: update action to v0.11.x once released
goreleaser-key: ${{ secrets.GORELEASER_KEY }}
zig-version: 0.11.0
- name: Collect Metrics
if: always()
id: collect-gha-metrics
Expand All @@ -46,6 +46,7 @@ jobs:
hostname: ${{ secrets.GRAFANA_INTERNAL_HOST }}
this-job-name: push-chainlink-develop-goreleaser
continue-on-error: true

mercury-e2e-tests:
needs: [push-chainlink-develop-goreleaser]
runs-on:
Expand Down
12 changes: 5 additions & 7 deletions .goreleaser.develop.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ env:

before:
hooks:
- go mod tidy
- ./tools/bin/goreleaser_utils before_hook

# See https://goreleaser.com/customization/build/
Expand Down Expand Up @@ -62,6 +63,7 @@ dockers:
goarch: amd64
extra_files:
- tmp/linux_amd64/libs
- tools/bin/ldd_fix
build_flag_templates:
- "--platform=linux/amd64"
- "--pull"
Expand All @@ -84,6 +86,7 @@ dockers:
goarch: arm64
extra_files:
- tmp/linux_arm64/libs
- tools/bin/ldd_fix
build_flag_templates:
- "--platform=linux/arm64"
- "--pull"
Expand All @@ -106,6 +109,7 @@ dockers:
goarch: amd64
extra_files:
- tmp/linux_amd64/libs
- tools/bin/ldd_fix
build_flag_templates:
- "--platform=linux/amd64"
- "--pull"
Expand All @@ -129,6 +133,7 @@ dockers:
goarch: arm64
extra_files:
- tmp/linux_arm64/libs
- tools/bin/ldd_fix
build_flag_templates:
- "--platform=linux/arm64"
- "--pull"
Expand Down Expand Up @@ -170,13 +175,6 @@ docker_signs:
- artifacts: all
stdin: "{{ .Env.COSIGN_PASSWORD }}"

archives:
- rlcp: true
files:
- src: tmp/{{ .Os }}_{{ .Arch }}/libs/*
dst: libs
strip_parent: true

checksum:
name_template: "checksums.txt"

Expand Down
4 changes: 2 additions & 2 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
golang 1.21.5
golang 1.21.7
mockery 2.38.0
nodejs 16.16.0
postgres 13.3
helm 3.10.3
zig 0.10.1
zig 0.11.0
golangci-lint 1.55.2
protoc 25.1
3 changes: 2 additions & 1 deletion core/chainlink.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@ RUN apt-get update && apt-get install -y ca-certificates gnupg lsb-release curl
RUN curl https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
&& echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" |tee /etc/apt/sources.list.d/pgdg.list \
&& apt-get update && apt-get install -y postgresql-client-15 \
&& apt-get clean all
&& apt-get clean all \
&& rm -rf /var/lib/apt/lists/*

COPY --from=buildgo /go/bin/chainlink /usr/local/bin/

Expand Down
13 changes: 10 additions & 3 deletions core/chainlink.goreleaser.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,25 @@ FROM ubuntu:20.04
ARG CHAINLINK_USER=root
ARG TARGETARCH
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && apt-get install -y ca-certificates gnupg lsb-release curl
RUN apt-get update && apt-get install -y ca-certificates gnupg lsb-release curl patchelf

# Install Postgres for CLI tools, needed specifically for DB backups
RUN curl https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
&& echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" |tee /etc/apt/sources.list.d/pgdg.list \
&& apt-get update && apt-get install -y postgresql-client-15 \
&& apt-get clean all
&& apt-get clean all \
&& rm -rf /var/lib/apt/lists/*

COPY . /usr/local/bin/
COPY ./chainlink /usr/local/bin/
# Copy native libs if cgo is enabled
COPY ./tmp/linux_${TARGETARCH}/libs /usr/local/bin/libs

# Temp fix to patch correctly link the libwasmvm.so
COPY ./tools/bin/ldd_fix /usr/local/bin/ldd_fix
RUN chmod +x /usr/local/bin/ldd_fix
RUN /usr/local/bin/ldd_fix
momentmaker marked this conversation as resolved.
Show resolved Hide resolved
RUN apt-get remove -y patchelf

RUN if [ ${CHAINLINK_USER} != root ]; then \
useradd --uid 14933 --create-home ${CHAINLINK_USER}; \
fi
Expand Down
2 changes: 1 addition & 1 deletion core/scripts/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/smartcontractkit/chainlink/core/scripts

go 1.21.3
go 1.21.7

// Make sure we're working with the latest chainlink libs
replace github.com/smartcontractkit/chainlink/v2 => ../../
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/smartcontractkit/chainlink/v2

go 1.21.3
go 1.21.7

require (
github.com/Depado/ginprom v1.8.0
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/.tool-versions
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
golang 1.21.5
golang 1.21.7
k3d 5.4.6
kubectl 1.25.5
nodejs 18.13.0
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/smartcontractkit/chainlink/integration-tests

go 1.21.4
go 1.21.7

// Make sure we're working with the latest chainlink libs
replace github.com/smartcontractkit/chainlink/v2 => ../
Expand Down
27 changes: 27 additions & 0 deletions tools/bin/ldd_fix
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
# This script is used as a temp fix the ldd linking of cosm lib for binary
# Currently there is an issue with the go linker not working with zig
# https://github.com/ziglang/zig/issues/18922

momentmaker marked this conversation as resolved.
Show resolved Hide resolved
chainlink_path="/usr/local/bin/chainlink"
libs_path="/usr/local/bin/libs"

line=$(ldd ${chainlink_path} | grep "github.com/!cosm!wasm/wasmvm")

if [ -z "$line" ]; then
echo "Error: Path containing 'github.com/!cosm!wasm/wasmvm' not found in the ldd output."
exit 1
fi

path=$(echo "$line" | awk '{print $1}')

if [ -z "$path" ]; then
echo "Error: Failed to extract the path from the line."
exit 1
fi

trimmed_path=${path%.so*}.so
cosm_file=$(ls ${libs_path} | grep "\.so$" | head -n 1)

patchelf --remove-needed "${trimmed_path}" "$chainlink_path"
patchelf --add-needed "$cosm_file" "$chainlink_path"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪄

Loading