Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-1218] try modified compatibility pipeline #13443

Merged
merged 129 commits into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
a02b847
try modified compatibility pipeline
Tofel Jun 6, 2024
7567775
don't wait with matrix generation for image to be built
Tofel Jun 6, 2024
251e124
fix outputs names
Tofel Jun 6, 2024
16a0d5c
use images instead of tags
Tofel Jun 6, 2024
abc0573
fix array population
Tofel Jun 6, 2024
911b86a
construct array differently
Tofel Jun 6, 2024
c21c15c
fix & debug
Tofel Jun 6, 2024
bdc773d
fix echos
Tofel Jun 6, 2024
5d86e28
try running tests
Tofel Jun 6, 2024
857d58f
fix path
Tofel Jun 6, 2024
1d81541
fix paths and run command
Tofel Jun 6, 2024
c877e44
more tests + change names + slack notification fix
Tofel Jun 6, 2024
fa4e8e6
test ignored tags, add more tests
Tofel Jun 6, 2024
6a2fe66
fix slack + more tests
Tofel Jun 6, 2024
fe5ab3c
fix script
Tofel Jun 6, 2024
9afccf9
fix ignored tags array
Tofel Jun 6, 2024
3a4e7e6
add vrfv2plus tests, commenout out vrfv2 tests that always fail
Tofel Jun 7, 2024
8c2c224
disable vrfv2plus tests for besu and nethermind, modify pipeline trig…
Tofel Jun 7, 2024
9e711e3
adjust triggers
Tofel Jun 7, 2024
a336e8e
another trigger update
Tofel Jun 7, 2024
3343da5
not trigger should work as expected
Tofel Jun 7, 2024
2b7068a
fix again
Tofel Jun 7, 2024
de4cd5e
yet another fix + add keeper test
Tofel Jun 7, 2024
af5622b
debug 1
Tofel Jun 7, 2024
8fd0450
debug 2
Tofel Jun 7, 2024
9506ef3
fix output target
Tofel Jun 7, 2024
af40e32
try different condition
Tofel Jun 7, 2024
b304a19
here comes another one...
Tofel Jun 7, 2024
65a462a
fix da condition again, maybe forevah?
Tofel Jun 7, 2024
8ef215b
another attempt
Tofel Jun 7, 2024
09ce895
fix yaml
Tofel Jun 7, 2024
4a4f035
add go script to check whether new releases created N days ago, updat…
Tofel Jun 7, 2024
9379c33
a bit of debug
Tofel Jun 7, 2024
4375d7e
fix bash expansion
Tofel Jun 7, 2024
ef067ce
Merge branch 'develop' into tt_1218_update_compatibility_pipeline
Tofel Jun 10, 2024
c3092ff
return latest tag from go script + get correct ref when building chai…
Tofel Jun 10, 2024
74d5d13
fix workflow add depedency
Tofel Jun 10, 2024
7da1f56
fix chainlink version that's passed to build
Tofel Jun 10, 2024
7f8f63a
fix ... mistake
Tofel Jun 10, 2024
96d2a35
use correct chainlink image tag in tests
Tofel Jun 10, 2024
5ac021a
allow to pass evm implementations and image count in workflow dispatch
Tofel Jun 10, 2024
f145ef5
support for custom test/evm implementation/versions input
Tofel Jun 10, 2024
a525777
fix custom input parsing
Tofel Jun 10, 2024
54c98d7
use go script to get latest images, another fix of input decoding
Tofel Jun 10, 2024
16e9bce
fix workflow conditions
Tofel Jun 10, 2024
83e22ad
further fix the workflow
Tofel Jun 10, 2024
db59a0b
remove one more invalid comment
Tofel Jun 10, 2024
4205562
remove profile from latest_images script
Tofel Jun 10, 2024
d8ccbac
set aws region from secret
Tofel Jun 10, 2024
7007998
modify latest images script, so that it ignored tags don't count towa…
Tofel Jun 10, 2024
bb33465
add tests to compatibility script
Tofel Jun 10, 2024
c93ab94
refactor latest images and add tests
Tofel Jun 10, 2024
ba63672
add tests for new_release script
Tofel Jun 10, 2024
a65ac24
run only Link Billing vrfv2plus test
Tofel Jun 10, 2024
95c0702
fix latest images script
Tofel Jun 10, 2024
bd5a0c5
Add one more test
Tofel Jun 10, 2024
7e61079
fix vrfv2plus regex
Tofel Jun 10, 2024
0c03013
use semver matching when fetching latest images
Tofel Jun 11, 2024
c60c468
fix constraints + warning message
Tofel Jun 11, 2024
5a0e996
Merge branch 'develop' into tt_1218_update_compatibility_pipeline
Tofel Jun 12, 2024
ecaa68b
do not fail slack notification if not all products run
Tofel Jun 12, 2024
d63bda8
more tests + variable to force check if image exists
Tofel Jun 12, 2024
e762ee7
fix workflow lints
Tofel Jun 12, 2024
b1a4e32
remove scripts, use CLI tools from CTF
Tofel Jun 12, 2024
8a7b223
add go/bin to path
Tofel Jun 12, 2024
a93c1e1
fix path
Tofel Jun 12, 2024
00a42c7
Merge branch 'develop' into tt_1218_update_compatibility_pipeline
Tofel Jun 13, 2024
308390c
bump go-ethereum to test the pipeline
Tofel Jun 13, 2024
67ba30c
bump correct go.mod
Tofel Jun 13, 2024
c12020d
restore go-ethereum version, remove bash script, update compatibility…
Tofel Jun 13, 2024
ad6d7fd
fix pipeline
Tofel Jun 13, 2024
59375fa
Merge branch 'develop' into tt_1218_update_compatibility_pipeline
Tofel Jun 17, 2024
aaf285d
remove evm compatiblity json
Tofel Jun 17, 2024
89e4c0f
add changeset
Tofel Jun 17, 2024
b9a40ab
fix changeset
Tofel Jun 17, 2024
2b1b139
try success() instead of always()
Tofel Jun 17, 2024
4771464
fix commit selection for workflow_dispatch
Tofel Jun 17, 2024
80c7982
run tests only if build-chainlink succeeds
Tofel Jun 17, 2024
99cbd22
WIP: print aggregated results
Tofel Jun 17, 2024
a36a00e
try result parsing scripts
Tofel Jun 17, 2024
a7d3055
fix needs + script cmd
Tofel Jun 17, 2024
81aeabb
run small test
Tofel Jun 17, 2024
3daff12
run small test 2
Tofel Jun 17, 2024
bcf55fb
fix typos
Tofel Jun 17, 2024
a60895b
do not save empty results
Tofel Jun 17, 2024
5e9e921
fix file location + early exit
Tofel Jun 17, 2024
1ef0af9
do not panic if key is not present
Tofel Jun 17, 2024
fe010a7
add concurrency guard
Tofel Jun 17, 2024
26ee3fd
fix output file name
Tofel Jun 17, 2024
0cd766e
fix passing of table
Tofel Jun 17, 2024
7b96064
fix thread link
Tofel Jun 17, 2024
0349189
fix result printing
Tofel Jun 17, 2024
391c425
exit early if no results are found
Tofel Jun 17, 2024
b49d90b
try wrapping summary table in markdown
Tofel Jun 18, 2024
3635635
try what happens if there are no results
Tofel Jun 18, 2024
44bc213
fix slack notification part
Tofel Jun 18, 2024
35734ab
remove invalid character
Tofel Jun 18, 2024
8f7e8ab
Merge branch 'develop' into tt_1218_update_compatibility_pipeline
Tofel Jun 18, 2024
0d3754f
get results of current workflow
Tofel Jun 18, 2024
145dcbf
print results of cron, runlog and flux tests as well + test newer too…
Tofel Jun 18, 2024
8c4bb79
do not panic if there no results
Tofel Jun 18, 2024
e352e73
fix aggregated results printing condition
Tofel Jun 18, 2024
657bac6
do not output null, if there are no results
Tofel Jun 18, 2024
702c875
early exit for asciitable
Tofel Jun 18, 2024
57f5a45
Merge branch 'develop' into tt_1218_update_compatibility_pipeline
Tofel Jun 19, 2024
166320d
remove debug from the pipeline
Tofel Jun 19, 2024
9fad490
check if base64 input is valid
Tofel Jun 19, 2024
3aa96d7
fix the json validity check
Tofel Jun 19, 2024
d28f0ce
provide valid json example
Tofel Jun 19, 2024
d4212b9
try again with valid example
Tofel Jun 19, 2024
88cf892
fix indentation
Tofel Jun 19, 2024
963d62c
remove whitespaces before and after EOF
Tofel Jun 19, 2024
a9eb06e
use standalone tools in compat pipeline, modify notify action to also…
Tofel Jun 19, 2024
1b6741b
fix tool name + var name
Tofel Jun 19, 2024
c85322b
fix syntax + use newer asctiitable version
Tofel Jun 19, 2024
4477090
fix tool name
Tofel Jun 19, 2024
737a2d6
save results to input.json
Tofel Jun 19, 2024
fd7bd04
fix regex passed to notify job
Tofel Jun 19, 2024
7e94f51
another slack notification fix
Tofel Jun 19, 2024
d5e6f17
do not post anything for skipped tests
Tofel Jun 19, 2024
25b3868
do not post, if there are no results
Tofel Jun 19, 2024
371e143
remove incorrect line
Tofel Jun 19, 2024
b0330e1
bump asciitable version
Tofel Jun 19, 2024
111863d
remove comment
Tofel Jun 19, 2024
30b959b
Merge branch 'develop' into tt_1218_update_compatibility_pipeline
Tofel Jun 19, 2024
8dfa91b
remove unused action
Tofel Jun 20, 2024
34b6208
adjust live testnets workflow's regex to match new format expected by…
Tofel Jun 20, 2024
169a781
bump down go-ethereum in tests
Tofel Jun 20, 2024
3081925
bump down go-ethereum in core/scripts
Tofel Jun 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/pretty-dancers-pull.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": patch
---

#changed Expand EVM implementation compatibility pipeline
6 changes: 5 additions & 1 deletion .github/actions/build-chainlink-image/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,16 @@ inputs:
dep_evm_sha:
description: The chainlink-evm commit sha to use in go deps
required: false
check_image_exists:
description: "Check if the image exists in ECR before building"
required: false
default: 'false'

runs:
using: composite
steps:
- name: Check if image exists
if: ${{ inputs.dep_evm_sha != '' }}
if: ${{ inputs.dep_evm_sha != '' || inputs.check_image_exists == 'true'}}
id: check-image
uses: smartcontractkit/chainlink-github-actions/docker/image-exists@fc3e0df622521019f50d772726d6bf8dc919dd38 # v2.3.19
with:
Expand Down
85 changes: 50 additions & 35 deletions .github/actions/notify-slack-jobs-result/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ inputs:
slack_thread_ts:
description: "The Slack thread timestamp to post the message to, handy for keeping multiple related results in a single thread"
required: false
base64_parsed_results:
description: "Base64 encoded parsed results to use"
required: false

runs:
using: composite
Expand All @@ -33,43 +36,54 @@ runs:
shell: bash
id: test-results
run: |
# I feel like there's some clever, fully jq way to do this, but I ain't got the motivation to figure it out
echo "Querying test results at https://api.github.com/repos/${{inputs.github_repository}}/actions/runs/${{ inputs.workflow_run_id }}/jobs"

# we can get a maximum of 100 jobs per page, after that we need to start using pagination
PARSED_RESULTS=$(curl \
-H "Authorization: Bearer ${{ inputs.github_token }}" \
'https://api.github.com/repos/${{inputs.github_repository}}/actions/runs/${{ inputs.workflow_run_id }}/jobs?per_page=100' \
| jq -r --arg pattern "${{ inputs.github_job_name_regex }}" '.jobs[]
| select(.name | test($pattern)) as $job
| $job.steps[]
| select(.name == "Run Tests")
| { conclusion: (if .conclusion == "success" then ":white_check_mark:" else ":x:" end), cap: ("*" + ($job.name | capture($pattern).cap) + "*"), html_url: $job.html_url }')

if [ -n "${{ inputs.base64_parsed_results }}" ]; then
echo "Using base64 parsed results"
PARSED_RESULTS=$(echo "${{ inputs.base64_parsed_results }}" | base64 -d)
else
go install github.com/smartcontractkit/chainlink-testing-framework/tools/[email protected]
PATH=$PATH:$(go env GOPATH)/bin
export PATH

workflowresultparser -workflowRunID ${{ inputs.workflow_run_id }} -githubToken ${{ inputs.github_token }} -githubRepo "${{ inputs.github_repository }}" -jobNameRegex "${{ inputs.github_job_name_regex }}" -outputFile=output.json

if [ ! -f output.json ]; then
PARSED_RESULTS='""'
else
PARSED_RESULTS=$(cat output.json | jq -c "select(has(\"results\")) | .results[]")
fi

fi

echo "Parsed Results:"
echo $PARSED_RESULTS

ALL_SUCCESS=true
echo "Checking for failures"
echo "$PARSED_RESULTS" | jq -s | jq -r '.[] | select(.conclusion != ":white_check_mark:")'
for row in $(echo "$PARSED_RESULTS" | jq -s | jq -r '.[] | select(.conclusion != ":white_check_mark:")'); do
ALL_SUCCESS=false
break
done
echo "Success: $ALL_SUCCESS"

echo all_success=$ALL_SUCCESS >> $GITHUB_OUTPUT

FORMATTED_RESULTS=$(echo $PARSED_RESULTS | jq -s '[.[]
| {
conclusion: .conclusion,
cap: .cap,
html_url: .html_url
}
]
| map("{\"type\": \"section\", \"text\": {\"type\": \"mrkdwn\", \"text\": \"<\(.html_url)|\(.cap)>: \(.conclusion)\"}}")
| join(",")')


if [ "$PARSED_RESULTS" != '""' ]; then
echo "Checking for failures"
echo "$PARSED_RESULTS" | jq -s | jq -r '.[] | select(.conclusion != ":white_check_mark:")'
for row in $(echo "$PARSED_RESULTS" | jq -s | jq -r '.[] | select(.conclusion != ":white_check_mark:")'); do
ALL_SUCCESS=false
break
done
echo "Success: $ALL_SUCCESS"

echo all_success=$ALL_SUCCESS >> $GITHUB_OUTPUT

FORMATTED_RESULTS=$(echo $PARSED_RESULTS | jq -s '[.[]
| {
conclusion: .conclusion,
cap: .cap,
html_url: .html_url
}
]
| map("{\"type\": \"section\", \"text\": {\"type\": \"mrkdwn\", \"text\": \"<\(.html_url)|\(.cap)>: \(.conclusion)\"}}")
| join(",")')
else
echo "Nothing to post, no results found"
exit 0
fi

echo "Formatted Results:"
echo $FORMATTED_RESULTS

Expand All @@ -82,6 +96,7 @@ runs:
echo results=$CLEAN_RESULTS >> $GITHUB_OUTPUT
- name: Post Results
uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001 # v1.25.0
if: steps.test-results.outputs.results != ''
env:
SLACK_BOT_TOKEN: ${{ inputs.slack_bot_token }}
with:
Expand All @@ -108,4 +123,4 @@ runs:
]
}
]
}
}
Loading
Loading