[TT-1218] try modified compatibility pipeline

[Tofel]

The "Client Compatibility Tests" workflow ensures compatibility between various EVM (Ethereum Virtual Machine) implementations and the Chainlink software. It is triggered on a regular schedule, by certain events, or manually. The workflow runs a suite of tests against different EVM versions to verify their compatibility with Chainlink.

Workflow Triggers:

1. Scheduled Runs:

   • Executes every Tuesday and Friday at 5:30 AM UTC (midnight + 30 minutes EST).
   • This ensures regular compatibility checks are conducted bi-weekly.

2. Push Events:

   • Triggered on push events that involve any tags (*).
   • Useful for testing releases and tagged versions automatically.

3. Merge Group and Pull Request Events:

   • Triggered when there is a pull request or a merge group event.
   • Ensures changes in pull requests are compatible before merging.

4. Manual Dispatch:

   • Can be manually triggered with specific inputs (chainlinkVersion, evmImplementations, latestVersionsNumber, and base64TestList).
   • Provides flexibility for developers to run tests on demand with customized settings.

Detailed Steps:

1. Check Dependency Bump:

   • This step checks if the go-ethereum dependency has been updated in the context of a pull request or a merge group event.
   • It fetches the base branch and compares the go.mod file for changes related to github.com/ethereum/go-ethereum.

2. Determine If Tests Should Run:

   • This step uses previous step's output and the event type to decide if the tests should proceed.
   • Runs Tests If:
     • The go-ethereum dependency was changed.
     • The event is a scheduled run.
     • The event is a manual workflow dispatch.
     • The event is triggered by a new tag.
   • Skips Tests If:
     • None of the above conditions are met.

3. Select Versions:

   • EVM Implementations:
     • Scheduled Events: Checks for new releases of EVM implementations (geth, besu, nethermind, erigon) within the last 4 days.
     • Manual Dispatch: Uses the evmImplementations input if provided; otherwise, defaults to a set list.
     • Other Events: Tests all four EVM implementations by default.
   • Chainlink Version:
     • Scheduled Events: Fetches the latest stable Chainlink version.
     • Manual Dispatch: Uses the chainlinkVersion input if provided; otherwise, defaults to the latest commit SHA.
     • Pull Requests: Uses the head commit SHA of the pull request.
     • Merge Queue: Uses the head commit SHA of the merge queue.
     • Tags: Uses the tag name as the version.
   • Latest Image Count:
     • Manual Dispatch: Uses the latestVersionsNumber input if provided.
     • Other Events: Defaults to 3.

Workflow Steps:

1. Check Dependency Bump:

   • Checks for changes in the go-ethereum dependency in pull requests or merge groups.
   • Outputs whether the dependency has changed.

2. Determine If Tests Should Run:

   • Decides if tests should run based on the trigger event and dependency changes.
   • Tests run if:
     • The go-ethereum dependency was changed.
     • The event is a scheduled run.
     • The event is a manual workflow dispatch.
     • A new tag was created.
   • Tests are skipped if none of the above conditions are met.

3. Select Versions:

   • EVM Implementations:
     • Scheduled Events: Checks for new releases within the last 4 days.
     • Manual Dispatch: Uses the provided evmImplementations or defaults to a predefined list.
     • Other Events: Tests all four EVM implementations (geth, besu, nethermind, erigon) by default.
   • Chainlink Version:
     • Scheduled Events: Fetches the latest stable Chainlink version.
     • Manual Dispatch: Uses the provided chainlinkVersion or defaults to the latest commit SHA.
     • Pull Requests: Uses the head commit SHA.
     • Merge Queue: Uses the merge queue's head commit SHA.
     • Tags: Uses the tag name.
   • Latest Image Count:
     • Manual Dispatch: Uses the provided latestVersionsNumber.
     • Other Events: Defaults to 3.
   • Outputs: evm_implementations (list of EVM implementations), chainlink_version (Chainlink version), latest_image_count (number of latest images).

4. Check ECR Images:

   • Verifies if required Docker images exist in the internal ECR.

5. Build Chainlink Image:

   • Builds the Chainlink Docker image using the selected version.

6. Get Latest EVM Implementation Images:

   • Retrieves the latest Docker images for specified EVM implementations from ECR.

7. Prepare Compatibility Matrix:

   • Prepares a test matrix outlining which tests to run against which EVM implementations.
   • Generates or decodes a provided base64 encoded test list.

8. Run Compatibility Tests:

   • Executes the compatibility tests defined in the test matrix.
   • Logs and publishes the results.

9. Notify Slack:

   • Posts a summary of test results to a designated Slack channel.
   • Starts a thread in Slack if any tests fail to notify the relevant team members.

10. Post Detailed Test Results to Slack:

    • Posts detailed results of each test to the Slack thread.

Last. but not least, now it also prints aggregated results in the summary:

[skudasov]

LGTM, but let's remove v1.13.9 go-ethereum bump, we are still on the old version

[cl-sonarqube-production]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube