E2E_TESTS_ON_GITHUB_CI.md

E2E Tests on GitHub CI

• E2E Tests on GitHub CI
  • Scheduled test workflows
    • PR E2E Tests
    • Nightly E2E Tests
    • Release E2E Tests
      • Integration (smoke) Tests
      • Client Compatibility Tests
  • On-Demand Workflows
    • Test workflows setup in CI
      • Configuration Overrides
      • Test Secrets

E2E tests are executed on GitHub CI using the E2E Tests Reusable Workflow or dedicated workflows.

Scheduled test workflows

These workflows are designed to run on every commit in a PR, nightly or before release (see triggers in e2e-tests.yaml).

PR E2E Tests

Tests triggered on every commit in a PR to ensure changes do not introduce regressions.

Workflow: integration-tests.yml

Nightly E2E Tests

Nightly E2E test runs.

Workflow: nightly-e2e-tests.yml

Release E2E Tests

E2E tests triggered on a release tag.

Integration (smoke) Tests

Workflow: integration-tests.yml

Client Compatibility Tests

Workflow: client-compatibility-tests.yml

On-Demand Workflows

These are dispatched parametrized workflows, that may be triggered manually for specific testing needs. For more details refer integration-tests README and per-product test run books.

Examples:

• Selected E2E Tests Workflow
• Client Compatibility Tests
• Chaos Tests
• OCR Soak Tests
• On-Demand Automation Tests
• CCIP Chaos Tests
• CCIP Load Tests
• VRFv2Plus Smoke Tests
• VRFv2Plus Performance Tests

Test workflows setup in CI

Most workflows may be triggered with default configs. Some, nevertheless, may be overridden.

Tip

Use gh CLI commands to run workflows from local machine.

Configuration Overrides

Caution

Test configurations should not keep any sensitive data or secrets.

1. Reference sources:
   1. Integration-Tests configurations;
   2. CTF Test Config.
2. Defaults and overrides should be stored (committed) in repository under ../integration-tests/testconfig/<product>/overrides/<override-name>.toml (see example here).
3. Use test_config_override_path to point to an override config. For example: test_config_override_path="testconfig/ocr2/overrides/base_sepolia.toml"

Test Secrets

Caution

Pay attention to never store/expose/commit your test secrets in repository.

Test secrets allow provisioning and override the sensitive data such as EOA's private key, RPCs, Docker registry links, etc.

Reference sources:

1. BASE64_CONFIG_OVERRIDE.
2. CTF Test Secrets documentation.
3. Guide on running GitHub workflows with your test secrets.