-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Use multiple EL clients with ocrv2 median smoke test (#11399)
* use multiple EL clients with ocrv2 median test * update ethereum network component to latest * try with latest implementation of eth clients * change WithTestLogger() to WithTestInstance() * fix geth backward compatibility * run ocrv2 eth2 tests in parallel * run also ocr1 and 2 vrf tests iwth multiple clients * try running multipl el client tests in parallel * fix go.sum * fix lint * 1. add matrix for running 4 smoke tests on different execution clients, 2. if for these tests no client is set in env var use geth * comment out failing nethermind & besu vrf tests in the CI * fix go.sum * do not use negative lookahead with go test * use faster eth2 network config * add comment to integration tests workflow * run only OCR and OCRv2 jobs with multiple clients, add on-demand jobs for VRF * use latest ctf with increased network timeout * slow down private eth networks a bit * update CTF dep to tagged version * disable nethermind ocr smoke tests in CI * fix compilation errors
- Loading branch information
Showing
26 changed files
with
303 additions
and
45 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| name: On Demand VRFV2 Smoke Test (Ethereum clients) | ||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| client: | ||
| description: Execution client to use | ||
| type: choice | ||
| options: | ||
| - "geth" | ||
| - "nethermind" | ||
| - "besu" | ||
| - "erigon" | ||
| chainlinkImage: | ||
| description: Container image location for the Chainlink nodes | ||
| required: true | ||
| default: public.ecr.aws/chainlink/chainlink | ||
| chainlinkVersion: | ||
| description: Container image version for the Chainlink nodes | ||
| required: true | ||
| default: "2.6.0" | ||
| configBase64: | ||
| description: TOML config in base64 (Needed when overriding config or providing contract addresses for existing env) | ||
| required: false | ||
| jobs: | ||
| vrfv2_smoke_test: | ||
| name: VRFV2 Smoke Test with ${{ inputs.client }} client | ||
| environment: integration | ||
| runs-on: ubuntu20.04-8cores-32GB | ||
| permissions: | ||
| checks: write | ||
| pull-requests: write | ||
| id-token: write | ||
| contents: read | ||
| env: | ||
| SELECTED_NETWORKS: "SIMULATED" | ||
| CONFIG: ${{ inputs.configBase64 }} | ||
| TEST_LOG_LEVEL: debug | ||
| REF_NAME: ${{ github.head_ref || github.ref_name }} | ||
| CHAINLINK_IMAGE: ${{ inputs.chainlinkImage }} | ||
| CHAINLINK_VERSION: ${{ inputs.chainlinkVersion }} | ||
| steps: | ||
| - name: Setup Push Tag | ||
| shell: bash | ||
| run: | | ||
| echo "### chainlink image used for this test run :link:" >>$GITHUB_STEP_SUMMARY | ||
| echo "\`${{ inputs.chainlinkVersion }}\`" >>$GITHUB_STEP_SUMMARY | ||
| echo "### chainlink-tests image tag for this test run :ship:" >>$GITHUB_STEP_SUMMARY | ||
| echo "\`${GITHUB_SHA}\`" >>$GITHUB_STEP_SUMMARY | ||
| - name: Checkout code | ||
| uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 | ||
| with: | ||
| fetch-depth: 0 | ||
| - name: Run Tests | ||
| uses: smartcontractkit/chainlink-github-actions/chainlink-testing-framework/run-tests@7d541cbbca52d45b8a718257af86d9cf49774d1f # v2.2.15 | ||
| with: | ||
| test_command_to_run: cd ./integration-tests && go test -timeout 30m -count=1 -json -run TestVRFv2Basic ./smoke/vrfv2_test.go 2>&1 | tee /tmp/gotest.log | gotestfmt | ||
| test_download_vendor_packages_command: cd ./integration-tests && go mod download | ||
| cl_repo: ${{ inputs.chainlinkImage }} | ||
| cl_image_tag: ${{ inputs.chainlinkVersion }} | ||
| aws_registries: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }} | ||
| artifacts_name: vrf-test-logs | ||
| artifacts_location: ./integration-tests/smoke/logs/ | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
| go_mod_path: ./integration-tests/go.mod | ||
| should_cleanup: false | ||
| QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }} | ||
| QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }} | ||
| QA_KUBECONFIG: "" | ||
| env: | ||
| ETH2_EL_CLIENT: ${{ inputs.client }} |
70 changes: 70 additions & 0 deletions
70
.github/workflows/on-demand-vrfv2plus-eth2-clients-test.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| name: On Demand VRFV2Plus Smoke Test (Ethereum clients) | ||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| client: | ||
| description: Execution client to use | ||
| type: choice | ||
| options: | ||
| - "geth" | ||
| - "nethermind" | ||
| - "besu" | ||
| - "erigon" | ||
| chainlinkImage: | ||
| description: Container image location for the Chainlink nodes | ||
| required: true | ||
| default: public.ecr.aws/chainlink/chainlink | ||
| chainlinkVersion: | ||
| description: Container image version for the Chainlink nodes | ||
| required: true | ||
| default: "2.6.0" | ||
| configBase64: | ||
| description: TOML config in base64 (Needed when overriding config or providing contract addresses for existing env) | ||
| required: false | ||
| jobs: | ||
| vrfv2plus_smoke_test: | ||
| name: VRFV2Plus Smoke Test with ${{ inputs.client }} client | ||
| environment: integration | ||
| runs-on: ubuntu20.04-8cores-32GB | ||
| permissions: | ||
| checks: write | ||
| pull-requests: write | ||
| id-token: write | ||
| contents: read | ||
| env: | ||
| SELECTED_NETWORKS: "SIMULATED" | ||
| CONFIG: ${{ inputs.configBase64 }} | ||
| TEST_LOG_LEVEL: debug | ||
| REF_NAME: ${{ github.head_ref || github.ref_name }} | ||
| CHAINLINK_IMAGE: ${{ inputs.chainlinkImage }} | ||
| CHAINLINK_VERSION: ${{ inputs.chainlinkVersion }} | ||
| steps: | ||
| - name: Setup Push Tag | ||
| shell: bash | ||
| run: | | ||
| echo "### chainlink image used for this test run :link:" >>$GITHUB_STEP_SUMMARY | ||
| echo "\`${{ inputs.chainlinkVersion }}\`" >>$GITHUB_STEP_SUMMARY | ||
| echo "### chainlink-tests image tag for this test run :ship:" >>$GITHUB_STEP_SUMMARY | ||
| echo "\`${GITHUB_SHA}\`" >>$GITHUB_STEP_SUMMARY | ||
| - name: Checkout code | ||
| uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 | ||
| with: | ||
| fetch-depth: 0 | ||
| - name: Run Tests | ||
| uses: smartcontractkit/chainlink-github-actions/chainlink-testing-framework/run-tests@7d541cbbca52d45b8a718257af86d9cf49774d1f # v2.2.15 | ||
| with: | ||
| test_command_to_run: cd ./integration-tests && go test -timeout 30m -count=1 -json -run ^TestVRFv2Plus$/^Link_Billing$ ./smoke/vrfv2plus_test.go 2>&1 | tee /tmp/gotest.log | gotestfmt | ||
| test_download_vendor_packages_command: cd ./integration-tests && go mod download | ||
| cl_repo: ${{ inputs.chainlinkImage }} | ||
| cl_image_tag: ${{ inputs.chainlinkVersion }} | ||
| aws_registries: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }} | ||
| artifacts_name: vrfplus-test-logs | ||
| artifacts_location: ./integration-tests/smoke/logs/ | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
| go_mod_path: ./integration-tests/go.mod | ||
| should_cleanup: false | ||
| QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }} | ||
| QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }} | ||
| QA_KUBECONFIG: "" | ||
| env: | ||
| ETH2_EL_CLIENT: ${{ inputs.client }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -84,4 +84,4 @@ go.work* | |
| tools/flakeytests/coverage.txt | ||
|
|
||
| .test_summary/ | ||
| .run.id | ||
| .run.id | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| package actions | ||
|
|
||
| import ( | ||
| "errors" | ||
|
|
||
| "github.com/rs/zerolog" | ||
|
|
||
| ctf_test_env "github.com/smartcontractkit/chainlink-testing-framework/docker/test_env" | ||
| ) | ||
|
|
||
| func EthereumNetworkConfigFromEnvOrDefault(l zerolog.Logger) (network ctf_test_env.EthereumNetwork, err error) { | ||
| chainConfig := ctf_test_env.EthereumChainConfig{ | ||
| SecondsPerSlot: 8, | ||
| SlotsPerEpoch: 4, | ||
| } | ||
|
|
||
| ethBuilder := ctf_test_env.NewEthereumNetworkBuilder() | ||
| network, err = ethBuilder. | ||
| WithExecClientFromEnvVar(). | ||
| WithEthereumChainConfig(chainConfig). | ||
| Build() | ||
|
|
||
| if errors.Is(err, ctf_test_env.ErrMissingExecClientEnvVar) { | ||
| l.Warn().Msg("No exec client env var set, will use old geth") | ||
| ethBuilder = ctf_test_env.NewEthereumNetworkBuilder() | ||
| network, err = ethBuilder. | ||
| WithConsensusType(ctf_test_env.ConsensusType_PoW). | ||
| WithExecutionLayer(ctf_test_env.ExecutionLayer_Geth). | ||
| Build() | ||
| } | ||
|
|
||
| return | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.