fix automation tests #11815
Merged
fix automation tests #11815
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I see that you haven't updated any CHANGELOG files. Would it make sense to do so? |
anirudhwarrier
force-pushed
the
f_toml_config-automation
branch
from
3466f29 to
e89d28e
Compare
anirudhwarrier
commented
Jan 22, 2024
anirudhwarrier
commented
Jan 23, 2024
anirudhwarrier
changed the title
|
SonarQube Quality Gate
|
Tofel
reviewed
Jan 23, 2024
| @@ -68,6 +73,8 @@ jobs: | |||
| TEST_ARGS: -test.timeout 720h | |||
| ENV_JOB_IMAGE: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com/chainlink-tests:${{ github.sha }} | |||
| INTERNAL_DOCKER_REPO: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com | |||
| TEST_TYPE: ${{ github.event.inputs.testType }} | |||
| TEST_TEST_TYPE: ${{ github.event.inputs.testType }} | |||
There was a problem hiding this comment.
maybe better to add forwarding of TEST_TYPE variable to CTF?
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jan 24, 2024
* WIP#1 * WIP#2 * WIP#3 * WIP#4 * WIP#5 * WIP#6 * wIP#7 - overrides and validations for vrfv2 * wIP#8 - overrides and validations for vrfv2lus * WIP#9 - adjust vrfv2/plus default configs * fix test config test, add logger to it * smoke tests should work now * fix replace, fix workflow * try to remove newlines from base64 * update ctf version, fix line wrap * use newer version of ctf, fix log poller and workflow issues * expand selected networks to array * fix vrfv2 smoke test, mask baseb4 override * define base64 override before masking it * a couple of masks more * set some content for BASE64_CONFIG_OVERRIDE so that it's masked * remove masks that break toml config * experiment with base64 #1 * fix ocr2 smoke test * fix lints * baseb4 experiment #2 * base64 experiment #3 * hardcode some image * exp#5 * fail config test on purpose * fix lints * run all smoke tests * add missing test config to ocr2 test * add missing loki_url * remove old config files, set config override for remote runner * fix compile errors * fix lint * make the vrfv2plus name in test matrix unique * fix loki env config, adjust vrfv2 and vrfv2plus tests and configs * add TOML-specific overriding of CL image/version in k8s * fixed automation load test * couple small fixes to how helm is created * remove multiple config files, store all config versions in single product-specific toml file * go.mod * restore deleted test, fix vrfv2 smoke tests * fix vrfv2plus config validation * properly fix vrfv2plus smoke test * fix loki basic auth passing * add add-mask again to base64 * remove obsolete configs, use latest ctf, adjust tomls * try another base64 hiding * go.sum * fix loki compile issue * try base64 env var with _SECRET suffix * try yet another variable name * restore old env var name * another obscuring attempt * another attempt * use newer CTF version, check what env vars are forwaded to k8s * go.sum * updated to latest ctf * try on demand autonation test with base64 input * modify all other test workflows so that they use TOML config * update reamdes, example envs and some more workflows * uncomment solana tests * update to latest ctf * update to latest ctf * go.mod * add MustCopy() method to TestConfig, fix failing vrvf2plus test * fix duration usages * fix lints * added examples of full configs for each product * use latest ctf * simplify some examples, correct client compatibility workflow * go.mod * copy TOML configs before executing test binary in compatibility job * use bash not sh to run the script, add missing patterns * chmod script and run it * add debug * print vars before checking their number * another debug attempt * try double quotes * store script and run that * fix script formatting * fix correct script 🤦 * fix zip command * fix upload artifact path * add some debug * update tracing readme, use latest ctf * latest ctf * lastest ctf * check what is the config missing * add missing pyrscope enabled var * remove debug * don't notify Adam if eth client compatibility test fail * latest ctf, faster eth2 chain by default * latest ctf, slots per epoch = 2 * go.sum * move convert_to_toml_array() bash function to a file and import it in GHA workflows instead of constantly defining before using * provide default configs to live testnet tests that use test binary * pass network config between jobs correctly * set .root_dir for live testnet job * try passing * try with defined output * define correct output for build-tests job * fix value name * debug * 2x base64 * try with shorter variable * add network config to dynamic config * include shared function * embed toml configs when build tag is present * test passing shared functions as outputs * update action version * remove newline from base64 * newer gha * fix log poller path, adjust client compatibility test to not copy tomls * debug * once more * do not use shared function * use correct network toml key for sepolia * fix dynamic toml * fix tomls * fix sepolia endpoint, uncomment slack notifications * remove outputs shared function from clients compatibility tests * fix alfajores once and for all * use newer gha to build tests * don't fail if there's no embedded config * remove tests we don't need anymore * use commit from main version of GHA repository instead of branch commit for build tests action * simplify config overriding * fix go.mod * remove ApplyOverride() and fix compile issue * fix lint * try base64 value extraction * try anothe approach * yet another attempt * yet another attempt 2 * fix base64 from input * one more time with final config * last test * make automation-ondemand test more secure, remove debug from on-demand-vrfv2-performance-test.yml * uncommet test execution in vrfv2 performance workflow * use private eth networks instead of geth in vrf load tests * update TOML keys related to Loki and Grafana * by default use pow, not pos * go.mod * latest ctf * remove references to deleted performance test suite * don't require Loki config when using remote runner * latest ctf * add Load config and figure out overrides * append loki config to ocr soak test on demand job * embed test configs and create .root_dir, when building test image * define an interface for global test config * move interface to testconfig * set pyroscope env for cron * fix check for remote running * bump CTF * use interfaces in public methods instead of TestConfig * small adjustments * fix test env builder * fix automation config type cast * use latest ctf * add special handling for slice override for Automation * update examples * slightly adjust vrfv2 configs, remove default empty values * move creation of base64 config override env var in smoke tests to a GHA action * fix compile error * debug for ocr test, camelcase for base64 action, two new actions to reduce loc * checkout repo before accessing actions * remove unnecessary test config instance from ocr test * mask before using input, more ocr debug * remove debug, fix reading named configurations from base64 override * mask input before using merge base64 action * fix integration tests workflow * update default.toml, add action for live testnet tests * download test binary after creating base64 env var * fix missing log targets * comment out reporting part * reuse live testnets specific action * fix network names * use dynamic network key * fix automation tests (#11815) * fix automation benchmark test action * update default toml * try newer run tests action * another try * look for .root_dir even deeper * Update integration-tests/benchmark/keeper_test.go * Update integration-tests/test.Dockerfile --------- Co-authored-by: Bartek Tofel <[email protected]> * fix chain config for eth2 compatibility tests * replace in-workflow toml generation for CL version upgrade tests with GH action * make test timeout configurable in client compatbility tests * prepare base64 config for log poller matrix tests --------- Co-authored-by: Ilja Pavlovs <[email protected]> Co-authored-by: skudasov <[email protected]> Co-authored-by: Gheorghe Strimtu <[email protected]> Co-authored-by: Anirudh Warrier <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.