Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create reusable workflow for E2E tests #13883

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
238 commits
Select commit Hold shift + click to select a range
8e21ca5
wip nightly pipeline
lukaszcl Jun 21, 2024
01d71a3
add reusable workflow
lukaszcl Jun 22, 2024
08eb8d0
on push
lukaszcl Jun 22, 2024
3b5a6e7
echo tests
lukaszcl Jun 22, 2024
2bf79bc
fix matrix
lukaszcl Jun 22, 2024
dfbdecf
fix
lukaszcl Jun 22, 2024
397ba14
add job name
lukaszcl Jun 22, 2024
6603e9b
run docker tests
lukaszcl Jun 22, 2024
e17499e
add reusable workflow
lukaszcl Jun 22, 2024
f255221
fix
lukaszcl Jun 22, 2024
1f3b5ca
fail if no test found
lukaszcl Jun 22, 2024
5901261
fix run selected workflow
lukaszcl Jun 22, 2024
d550f1d
Rename reusable workflow
lukaszcl Jun 22, 2024
04f2b32
Add nighly workflow
lukaszcl Jun 22, 2024
06f5922
try running k8s tests
lukaszcl Jun 22, 2024
099d654
cleanup
lukaszcl Jun 22, 2024
53858e0
add permissions
lukaszcl Jun 22, 2024
7db6612
fix
lukaszcl Jun 22, 2024
5611e17
test
lukaszcl Jun 22, 2024
f54a25b
Fix
lukaszcl Jun 22, 2024
a6ca1ff
fix
lukaszcl Jun 22, 2024
c2b411a
test remote runner
lukaszcl Jun 22, 2024
2c9ab53
Add test suite param
lukaszcl Jun 22, 2024
ada4954
Run all_tests_in_ocr_tests_go in parallel
lukaszcl Jun 22, 2024
3d587e8
Update e2e-tests.yml
lukaszcl Jun 22, 2024
fb95890
Add check-tests cmd
lukaszcl Jun 22, 2024
a82ad06
Unify tools into single cmd
lukaszcl Jun 22, 2024
79d3146
Add check tests step to workflow
lukaszcl Jun 22, 2024
0e6d314
show more info in annotation
lukaszcl Jun 22, 2024
cad8ebd
fix
lukaszcl Jun 22, 2024
880ae3b
Do not require check-tests
lukaszcl Jun 24, 2024
8079dab
fix go mod
lukaszcl Jun 24, 2024
2e112fc
temp: Disable checking test configuration
lukaszcl Jun 24, 2024
aa967ac
Add enable-check-test-configurations to reusable workflow. False by d…
lukaszcl Jun 24, 2024
acd558a
Add automation upgrade test to e2e list
lukaszcl Jun 24, 2024
f6c5049
Fix enable-check-test-configurations
lukaszcl Jun 24, 2024
74fb16d
Fix test cmd
lukaszcl Jun 24, 2024
27b3201
Make enable-check-test-configurations false by default
lukaszcl Jun 24, 2024
b7375e0
support test config override
lukaszcl Jun 24, 2024
be845a2
add more automation tests
lukaszcl Jun 24, 2024
b0acdda
Run automation upgrade tests on latest develop
lukaszcl Jun 24, 2024
73ed44e
Migrate automation-nightly-tests.yml to reusable workflow
lukaszcl Jun 24, 2024
8c19ba8
Update automation tests
lukaszcl Jun 24, 2024
092d8b0
Fix automation workflow
lukaszcl Jun 24, 2024
74d55e7
Fix
lukaszcl Jun 24, 2024
bbbbfc9
Fix
lukaszcl Jun 24, 2024
f4feb7f
Fx
lukaszcl Jun 24, 2024
5f44e12
Fix workflow_call inputs
lukaszcl Jun 24, 2024
c88a7f0
Select tests by github workflows
lukaszcl Jun 24, 2024
e7be8f1
Allow either test-ids or test-workflow in reusable workflow
lukaszcl Jun 24, 2024
ea39b14
Fix
lukaszcl Jun 24, 2024
a36fcee
Update workflow
lukaszcl Jun 24, 2024
0b02624
Add comment
lukaszcl Jun 24, 2024
8f7e27c
Fix
lukaszcl Jun 24, 2024
a4754da
Fix
lukaszcl Jun 24, 2024
3e0dcb7
Update e2e tests
lukaszcl Jun 24, 2024
da3b625
Add remote-runner-memory option
lukaszcl Jun 24, 2024
a780c75
Disable nightly run for now
lukaszcl Jun 24, 2024
52016f4
Fix output in k8s tests
lukaszcl Jun 24, 2024
f0b4a66
Add show test configuration step
lukaszcl Jun 24, 2024
d3bf504
Uncomment building test runner image
lukaszcl Jun 24, 2024
20e1409
Do not use json output for k8s tests
lukaszcl Jun 24, 2024
973cdff
test
lukaszcl Jun 24, 2024
104dbdc
Support custom runner version
lukaszcl Jun 24, 2024
1ff30f2
Fix test type env in remote runner
lukaszcl Jun 24, 2024
3d9820e
Update names and docs
lukaszcl Jun 25, 2024
9f0b76d
Add script to prepare test config override
lukaszcl Jun 26, 2024
b1f617b
Add CI test config override tools
lukaszcl Jun 26, 2024
d3cdd00
Add mask-secrets cmd and stop using reusable workflow
lukaszcl Jun 27, 2024
d388580
Use reusable workflow in run-selected-e2e-tests.yml
lukaszcl Jun 28, 2024
f917531
Remove unnecessary permissions
lukaszcl Jun 28, 2024
d8a330b
Add a sample test to quickly check CI workflows
lukaszcl Jun 28, 2024
48a40ae
Do not use mask cmd in reusable workflow
lukaszcl Jun 28, 2024
776f5a0
Clean up override test config cmd
lukaszcl Jun 28, 2024
265aac7
Fix masking of full chainlink image
lukaszcl Jun 28, 2024
535a49f
Finish "Set default test config override" step in reusable workflow
lukaszcl Jun 28, 2024
bb8f851
Remove env.CHAINLINK_IMAGE mask as it is not a scret
lukaszcl Jun 28, 2024
1742575
Fix reusable workflow
lukaszcl Jun 28, 2024
c7fa80e
Create create-default-e2e-config-override action
lukaszcl Jun 28, 2024
9861b6e
Fix
lukaszcl Jun 28, 2024
c67c854
Fix log targets in GHA and add test
lukaszcl Jun 28, 2024
494267c
Add create-default-e2e-config step to k8s tests
lukaszcl Jun 28, 2024
9194ff2
Fix ocr tests
lukaszcl Jun 28, 2024
a84b76a
Use sanitized id for artifact name
lukaszcl Jun 28, 2024
45b49ac
Update nightly workflow to test it
lukaszcl Jun 28, 2024
ff41f2e
Fail CI if test_config_base64_override is used
lukaszcl Jun 28, 2024
596e014
Use underscore for github inputs
lukaszcl Jun 28, 2024
3d1abd0
Fix
lukaszcl Jun 28, 2024
2cae0da
Fix inputs.chainlink_version
lukaszcl Jun 28, 2024
700f2f9
Remove test_config_base64_override from inputs until we are sure it d…
lukaszcl Jun 28, 2024
9b57657
Fix
lukaszcl Jun 28, 2024
0c88794
Remove unused action
lukaszcl Jun 28, 2024
9e04190
Fix
lukaszcl Jun 28, 2024
0c2979c
Update create config override cmd and action
lukaszcl Jun 28, 2024
5081809
Refactor client-compatibility-tests.yml to use actions/create-default…
lukaszcl Jun 28, 2024
e5a5782
Refactor on-demand-keeper-smoke-tests.yml
lukaszcl Jun 28, 2024
cd90826
Replace setup-create-base64-config with the new action in integration…
lukaszcl Jun 28, 2024
e4af6e0
Remove unused action
lukaszcl Jun 28, 2024
70493a8
Fix lint
lukaszcl Jun 28, 2024
2de3c68
Remove unused action
lukaszcl Jun 28, 2024
3692112
Fix handling selected_networks input
lukaszcl Jun 28, 2024
907cba8
Fix cache
lukaszcl Jun 28, 2024
cfe7de5
Do not run soak/ocr_test.go:^TestOCRSoak$ nightly
lukaszcl Jun 28, 2024
c3b1d4b
Enable nightly workflow to run some tests
lukaszcl Jun 28, 2024
94cdb4f
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jun 28, 2024
babde2b
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jun 28, 2024
2fccfbd
Remove config override input
lukaszcl Jun 28, 2024
a3eafeb
Remove temp test
lukaszcl Jun 28, 2024
71de825
Merge branch 'TT-1297-Create-nightly-pipeline-for-all-e2e-tests-in-co…
lukaszcl Jun 28, 2024
314f7e7
Fix setup go
lukaszcl Jun 28, 2024
69bee7c
go mod tidy in core/scripts
lukaszcl Jun 28, 2024
69453fb
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jun 28, 2024
ffb8bdd
Fix
lukaszcl Jun 28, 2024
6587671
Remove unready workflows
lukaszcl Jun 28, 2024
464c21d
Revert updates to core/scripts
lukaszcl Jun 28, 2024
a0470ba
Revert updates to go.mod
lukaszcl Jun 28, 2024
f23cee3
Remove unused script
lukaszcl Jun 28, 2024
5b0d11d
Add setup-gap id
lukaszcl Jul 1, 2024
210e4e8
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jul 1, 2024
74c1143
Remove unused steps
lukaszcl Jul 1, 2024
0e848db
test nightly workflow
lukaszcl Jul 1, 2024
63e2b2c
Add missing env vars in remote runner and fix lack of logs
lukaszcl Jul 1, 2024
d4a0a24
bump setup gap
lukaszcl Jul 1, 2024
f351eeb
Fix set-output
lukaszcl Jul 1, 2024
4f61a7a
update comment
lukaszcl Jul 1, 2024
5ceb64b
update test name
lukaszcl Jul 1, 2024
c3a8764
update cache
lukaszcl Jul 1, 2024
7a0466b
Fix empty output for k8s tests
lukaszcl Jul 1, 2024
2e7d11a
Add a test workflow with overrides
lukaszcl Jul 1, 2024
05efabc
Fix cache
lukaszcl Jul 1, 2024
ca08ff7
Resolve env vars in filter cmd
lukaszcl Jul 1, 2024
2728c0b
Fix
lukaszcl Jul 1, 2024
fbed49c
Rename test overrides to test inputs
lukaszcl Jul 1, 2024
594e948
Update input descriptions
lukaszcl Jul 1, 2024
3fb53ba
Fix
lukaszcl Jul 2, 2024
611f774
Check chainlink image exist in automation ondemand workflow
lukaszcl Jul 2, 2024
5127068
Add require-chainlink-image-versions-in-qa-ecr job to reusable workflow
lukaszcl Jul 2, 2024
b670bbb
Set no_cache for run-tests action
lukaszcl Jul 2, 2024
f40aaf3
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jul 9, 2024
bd77411
Fix
lukaszcl Jul 9, 2024
eea4c5c
Add test_secrets_override_key to run-selected-tests workflow
lukaszcl Jul 9, 2024
614cf92
Remove chainlink image and version from create test config cmd
lukaszcl Jul 9, 2024
28b0e4a
Update log
lukaszcl Jul 9, 2024
7961538
Remove chainlik image from create config tool
lukaszcl Jul 9, 2024
ec4f43e
Remove chainlik image from create config tool
lukaszcl Jul 9, 2024
0d3bef5
Set default upgrade version for config validation to pass when upgrad…
lukaszcl Jul 10, 2024
803006f
bump gha
lukaszcl Jul 10, 2024
6d05b2a
Fix
lukaszcl Jul 10, 2024
6993607
for test
lukaszcl Jul 10, 2024
f79e8f6
bump gha
lukaszcl Jul 10, 2024
f8b3458
Move e2e ci tool to CTF
lukaszcl Jul 10, 2024
d9f08f3
bump run-tests gha
lukaszcl Jul 10, 2024
3ebba32
bump ctf
lukaszcl Jul 10, 2024
cfca8ef
Allow to pass custom base64 test config override to reusable workflow
lukaszcl Jul 10, 2024
5a61d16
Update e2e-tests.yml
lukaszcl Jul 10, 2024
9a00468
Use DEFAULT_CHAINLINK_UPGRADE_IMAGE in workflow
lukaszcl Jul 10, 2024
a813ba3
Add run-name
lukaszcl Jul 10, 2024
1617eca
fix custom base64
lukaszcl Jul 10, 2024
56041ff
Fix workflow name
lukaszcl Jul 10, 2024
b1957af
Fix test inputs
lukaszcl Jul 10, 2024
ced6841
bump
lukaszcl Jul 10, 2024
a14f311
Remove default chainlink_version
lukaszcl Jul 10, 2024
96d7797
Bump
lukaszcl Jul 10, 2024
2714dda
Use CTF with no secret keys in toml
lukaszcl Jul 10, 2024
0ce31bf
add soak/ocr_test.go:^TestOCRv2Soak$
lukaszcl Jul 10, 2024
8f8e05b
update e2e tests
lukaszcl Jul 10, 2024
9206993
Fix setting pyroscope defaults
lukaszcl Jul 10, 2024
b8b6de3
Allow custom test secrets in on-demand-ocr-soak-test.yml
lukaszcl Jul 11, 2024
a93ba59
Bump CTF to mount test secrets in remote runner
lukaszcl Jul 11, 2024
ce40fc9
bump to use configmap
lukaszcl Jul 11, 2024
0355a00
bump ctf
lukaszcl Jul 11, 2024
b342d7c
bump
lukaszcl Jul 11, 2024
b8a1244
Use ctf latest release
lukaszcl Jul 11, 2024
b6dd446
Merge branch 'bump-ctf-to-mount-testsecrets' into TT-1397-Add-test_se…
lukaszcl Jul 11, 2024
ff1a086
Add test_secrets_override_base64 to other workflows that have testCon…
lukaszcl Jul 11, 2024
0240b8e
Do not fail workflow if no chainlink image found in setup-parse-base6…
lukaszcl Jul 11, 2024
504ea14
Merge branch 'develop' into TT-1397-Add-test_secrets_override_key-fla…
lukaszcl Jul 12, 2024
6a1d39d
Bump gha
lukaszcl Jul 15, 2024
dcd0fc0
Merge branch 'develop' into TT-1397-Add-test_secrets_override_key-fla…
lukaszcl Jul 15, 2024
a1f8aa7
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jul 15, 2024
ffe50f4
Test test-config-override-required
lukaszcl Jul 15, 2024
45da902
Bump CTF
lukaszcl Jul 15, 2024
b68c8fa
Bump ctf
lukaszcl Jul 15, 2024
ce4f6e5
Merge branch 'TT-1397-Add-test_secrets_override_key-flag-to-existing-…
lukaszcl Jul 15, 2024
c0b7904
Add integration-tests/load/vrfv2plus/vrfv2plus_test.go:^TestVRFV2Plus…
lukaszcl Jul 15, 2024
b945b1b
Bump ctf
lukaszcl Jul 16, 2024
5fd9378
Fix
lukaszcl Jul 16, 2024
6524eb7
Update test suite
lukaszcl Jul 16, 2024
294fc22
Add vrf tests to e2e-tests.yml
lukaszcl Jul 16, 2024
999d660
Cleanup
lukaszcl Jul 16, 2024
25208df
Add integration-tests/load/automationv2_1/automationv2_1_test.go to l…
lukaszcl Jul 16, 2024
0178385
Add rest of ocr soak tests
lukaszcl Jul 16, 2024
4b1f049
Add integration-tests/migration/upgrade_version_test.go:*
lukaszcl Jul 16, 2024
fbc13a8
Use snake case
lukaszcl Jul 16, 2024
738631b
Do not set default chainlink_upgrade_version to pass config validation
lukaszcl Jul 16, 2024
d89e96a
Use develop-plugins as default version for plugins tests
lukaszcl Jul 16, 2024
0748334
Fix
lukaszcl Jul 16, 2024
c1fc729
Add automation and keeper smoke tests
lukaszcl Jul 17, 2024
ee1ba9d
Add log poller test
lukaszcl Jul 17, 2024
e0f84e4
Add more log poller tests
lukaszcl Jul 17, 2024
d2a0b56
Clean up e2e tests
lukaszcl Jul 17, 2024
e8e4f0f
Fix
lukaszcl Jul 17, 2024
901ea42
Fix list of tests
lukaszcl Jul 17, 2024
3405ea1
Prioritise chainlink_version in test_inputs before the workflow input
lukaszcl Jul 17, 2024
a67341d
Fix
lukaszcl Jul 17, 2024
2536d2c
Move citool from CTF to Core repo
lukaszcl Jul 17, 2024
6985554
Fix import
lukaszcl Jul 17, 2024
622a465
Fix
lukaszcl Jul 17, 2024
fbaebcb
Fix check-tests cmd
lukaszcl Jul 17, 2024
ccec4bd
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jul 17, 2024
db8c9c9
bump run-tests
lukaszcl Jul 17, 2024
55b67ef
Add slack notification after tests
lukaszcl Jul 17, 2024
c8eda14
Add collect metrics step to reusable workflow
lukaszcl Jul 18, 2024
2ea6339
Fix tag
lukaszcl Jul 18, 2024
30e8e18
Fix job name in collect metrics steps
lukaszcl Jul 18, 2024
4b0bcae
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jul 18, 2024
43576f2
bump ctf
lukaszcl Jul 18, 2024
33b313b
fix collect metrics
lukaszcl Jul 18, 2024
6b266c2
Disallow passing test_config_override_base64 until TT-1392 is done
lukaszcl Jul 18, 2024
86ff615
Bump run-tests GHA
lukaszcl Jul 18, 2024
fc96b6d
Update channel for slack notification for nightly tests
lukaszcl Jul 18, 2024
bc1b071
Fix run-automation-ondemand-e2e-tests.yml
lukaszcl Jul 18, 2024
19e8585
Do not show base64 config override
lukaszcl Jul 18, 2024
da503b4
Fix lint
lukaszcl Jul 18, 2024
049d537
Fix config flag
lukaszcl Jul 18, 2024
dd4fcdf
Remove unused cmd
lukaszcl Jul 18, 2024
111501d
Fix go mod
lukaszcl Jul 18, 2024
fc6784f
Fix concurrent map writes
lukaszcl Jul 18, 2024
45c9d32
Flip cache
lukaszcl Jul 18, 2024
1775a99
Update automation workflow
lukaszcl Jul 18, 2024
db38119
revert MOD_CACHE_VERSION
lukaszcl Jul 18, 2024
270c507
Remove image keys from toml in automation-ondemand-tests.yml
lukaszcl Jul 18, 2024
0a16f68
Merge branch 'develop' into TT-1297-Create-nightly-pipeline-for-all-e…
lukaszcl Jul 18, 2024
33de583
Bump run-test GHA
lukaszcl Jul 18, 2024
dbccfc2
Fix reorg workflow according to Anirudh suggestion
lukaszcl Jul 18, 2024
23167f8
Fix
lukaszcl Jul 18, 2024
55b459a
Fix automation-ondemand-tests.yml
lukaszcl Jul 18, 2024
4c97265
Bump run-tests GHA in automation-ondemand-tests.yml
lukaszcl Jul 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 0 additions & 106 deletions .github/actions/setup-create-base64-config/action.yml

This file was deleted.

10 changes: 4 additions & 6 deletions .github/actions/setup-merge-base64-config/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,22 +24,20 @@ runs:
echo "BASE64_CONFIG_OVERRIDE=$BASE64_CONFIG_OVERRIDE" >> $GITHUB_ENV

decoded_toml=$(echo $BASE64_CONFIG_OVERRIDE | base64 -d)
CHAINLINK_IMAGE=$(echo "$decoded_toml" | dasel -r toml 'ChainlinkImage.image')
CHAINLINK_IMAGE=$(echo "$decoded_toml" | { dasel -r toml 'ChainlinkImage.image' 2>/dev/null || echo ''; })
echo ::add-mask::$CHAINLINK_IMAGE
CHAINLINK_VERSION=$(echo "$decoded_toml" | dasel -r toml 'ChainlinkImage.version')
CHAINLINK_VERSION=$(echo "$decoded_toml" | { dasel -r toml 'ChainlinkImage.version' 2>/dev/null || echo ''; })
NETWORKS=$(echo "$decoded_toml" | awk -F'=' '/^[[:space:]]*selected_networks[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2}' 2>/dev/null)

if [ -n "$CHAINLINK_IMAGE" ]; then
echo "CHAINLINK_IMAGE=$CHAINLINK_IMAGE" >> $GITHUB_ENV
else
echo "No Chainlink Image found in base64-ed config. Exiting"
exit 1
echo "No Chainlink Image found in base64-ed config"
fi
if [ -n "$CHAINLINK_VERSION" ]; then
echo "CHAINLINK_VERSION=$CHAINLINK_VERSION" >> $GITHUB_ENV
else
echo "No Chainlink Version found in base64-ed config. Exiting"
exit 1
echo "No Chainlink Version found in base64-ed config"
fi
if [ -n "$NETWORKS" ]; then
echo "NETWORKS=$NETWORKS" >> $GITHUB_ENV
Expand Down
8 changes: 3 additions & 5 deletions .github/actions/setup-parse-base64-config/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,23 +20,21 @@ runs:
shell: bash
run: |
decoded_toml=$(echo $BASE64_CONFIG_OVERRIDE | base64 -d)
CHAINLINK_IMAGE=$(echo "$decoded_toml" | dasel -r toml 'ChainlinkImage.image')
CHAINLINK_IMAGE=$(echo "$decoded_toml" | { dasel -r toml 'ChainlinkImage.image' 2>/dev/null || echo ''; })
echo ::add-mask::$CHAINLINK_IMAGE
CHAINLINK_VERSION=$(echo "$decoded_toml" | dasel -r toml 'ChainlinkImage.version')
CHAINLINK_VERSION=$(echo "$decoded_toml" | { dasel -r toml 'ChainlinkImage.version' 2>/dev/null || echo ''; })
NETWORKS=$(echo "$decoded_toml" | awk -F'=' '/^[[:space:]]*selected_networks[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2}' 2>/dev/null)
ETH2_EL_CLIENT=$(echo "$decoded_toml" | awk -F'=' '/^[[:space:]]*execution_layer[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2}' 2>/dev/null)

if [ -n "$CHAINLINK_IMAGE" ]; then
echo "CHAINLINK_IMAGE=$CHAINLINK_IMAGE" >> $GITHUB_ENV
else
echo "No Chainlink Image found in base64-ed config. Exiting"
exit 1
echo "No Chainlink Image found in base64-ed config"
fi
if [ -n "$CHAINLINK_VERSION" ]; then
echo "CHAINLINK_VERSION=$CHAINLINK_VERSION" >> $GITHUB_ENV
else
echo "No Chainlink Version found in base64-ed config. Exiting"
exit 1
fi
if [ -n "$NETWORKS" ]; then
echo "NETWORKS=$NETWORKS" >> $GITHUB_ENV
Expand Down
Loading
Loading