-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'develop' into chore/release-2.7.1-to-develop
- Loading branch information
Showing
1,402 changed files
with
47,863 additions
and
37,556 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| name: Build Chainlink Image | ||
| description: A composite action that allows building and publishing the Chainlink image for integration testing | ||
|
|
||
| inputs: | ||
| tag_suffix: | ||
| description: The suffix to append to the image tag (usually blank or "-plugins") | ||
| default: "" | ||
| dockerfile: | ||
| description: The path to the Dockerfile to use (usually core/chainlink.Dockerfile or plugins/chainlink.Dockerfile) | ||
| default: core/chainlink.Dockerfile | ||
| git_commit_sha: | ||
| description: The git commit sha to use for the image tag | ||
| default: ${{ github.sha }} | ||
| GRAFANA_CLOUD_BASIC_AUTH: | ||
| description: "grafana cloud basic auth" | ||
| GRAFANA_CLOUD_HOST: | ||
| description: "grafana cloud hostname" | ||
| AWS_REGION: | ||
| description: "AWS region to use for ECR" | ||
| AWS_ROLE_TO_ASSUME: | ||
| description: "AWS role to assume for ECR" | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Check if image exists | ||
| id: check-image | ||
| uses: smartcontractkit/chainlink-github-actions/docker/image-exists@e865e376b8c2d594028c8d645dd6c47169b72974 # v2.2.16 | ||
| with: | ||
| repository: chainlink | ||
| tag: ${{ inputs.git_commit_sha }}${{ inputs.tag_suffix }} | ||
| AWS_REGION: ${{ inputs.AWS_REGION }} | ||
| AWS_ROLE_TO_ASSUME: ${{ inputs.AWS_ROLE_TO_ASSUME }} | ||
| - name: Build Image | ||
| if: steps.check-image.outputs.exists == 'false' | ||
| uses: smartcontractkit/chainlink-github-actions/chainlink-testing-framework/build-image@e865e376b8c2d594028c8d645dd6c47169b72974 # v2.2.16 | ||
| with: | ||
| cl_repo: smartcontractkit/chainlink | ||
| cl_ref: ${{ inputs.git_commit_sha }} | ||
| cl_dockerfile: ${{ inputs.dockerfile }} | ||
| push_tag: ${{ env.CHAINLINK_IMAGE }}:${{ inputs.git_commit_sha }}${{ inputs.tag_suffix }} | ||
| QA_AWS_REGION: ${{ inputs.AWS_REGION }} | ||
| QA_AWS_ROLE_TO_ASSUME: ${{ inputs.AWS_ROLE_TO_ASSUME }} | ||
| - name: Print Chainlink Image Built | ||
| shell: sh | ||
| run: | | ||
| echo "### Chainlink node image tag used for this test run :link:" >>$GITHUB_STEP_SUMMARY | ||
| echo "\`${GITHUB_SHA}\`" >>$GITHUB_STEP_SUMMARY |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,7 +31,7 @@ runs: | |
| current_version=$(head -n1 ./VERSION) | ||
| echo "current_version=${current_version}" | tee -a "$GITHUB_OUTPUT" | ||
| - name: Compare semantic versions | ||
| uses: smartcontractkit/chainlink-github-actions/[email protected].0 | ||
| uses: smartcontractkit/chainlink-github-actions/semver-compare@e865e376b8c2d594028c8d645dd6c47169b72974 # v2.2.16 | ||
| id: compare | ||
| with: | ||
| version1: ${{ steps.get-current-version.outputs.current_version }} | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,51 @@ | ||
| # Distributed Tracing | ||
|
|
||
| These config files are for an OTEL collector, grafana Tempo, and a grafana UI instance to run as containers on the same network. | ||
| As part of the LOOP plugin effort, we've added distributed tracing to the core node. This is helpful for initial development and maintenance of LOOPs, but will also empower product teams building on top of core. | ||
|
|
||
| ## Dev environment | ||
|
|
||
| One way to generate traces locally today is with the OCR2 basic smoke test. | ||
|
|
||
| 1. navigate to `.github/tracing/` and then run `docker compose --file local-smoke-docker-compose.yaml up` | ||
| 2. setup a local docker registry at `127.0.0.1:5000` (https://www.docker.com/blog/how-to-use-your-own-registry-2/) | ||
| 3. run `make build_push_plugin_docker_image` in `chainlink/integration-tests/Makefile` | ||
| 4. run `SELECTED_NETWORKS=SIMULATED CHAINLINK_IMAGE="127.0.0.1:5000/chainlink" CHAINLINK_VERSION="develop" go test -run TestOCRv2Basic ./smoke/ocr2_test.go` | ||
| 5. navigate to `localhost:3000/explore` in a web browser to query for traces | ||
|
|
||
| Core and the median plugins are instrumented with open telemetry traces, which are sent to the OTEL collector and forwarded to the Tempo backend. The grafana UI can then read the trace data from the Tempo backend. | ||
|
|
||
|
|
||
|
|
||
| ## CI environment | ||
|
|
||
| Another way to generate traces is by enabling traces for PRs. This will instrument traces for `TestOCRv2Basic` in the CI run. | ||
|
|
||
| A localhost client can send gRPC calls to the server. The gRPC server is instrumented with open telemetry traces, which are sent to the OTEL collector and forwarded to the Tempo backend. The grafana UI can then read the trace data from the Tempo backend. | ||
| 1. Cut a PR in the core repo | ||
| 2. Add the `enable tracing` label to the PR | ||
| 3. Navigate to `Integration Tests / ETH Smoke Tests ocr2-plugins (pull_request)` details | ||
| 4. Navigate to the summary of the integration tests | ||
| 5. After the test completes, the generated trace data will be saved as an artifact, currently called `trace-data` | ||
| 6. Download the artifact to this directory (`chainlink/.github/tracing`) | ||
| 7. `docker compose --file local-smoke-docker-compose.yaml up` | ||
| 8. Run `sh replay.sh` to replay those traces to the otel-collector container that was spun up in the last step. | ||
| 9. navigate to `localhost:3000/explore` in a web browser to query for traces | ||
|
|
||
| The artifact is not json encoded - each individual line is a well formed and complete json object. | ||
|
|
||
|
|
||
| ## Production and NOPs environments | ||
|
|
||
| In a production environment, we suggest coupling the lifecycle of nodes and otel-collectors. A best practice is to deploy the otel-collector alongside your node, using infrastructure as code (IAC) to automate deployments and certificate lifecycles. While there are valid use cases for using `Tracing.Mode = unencrypted`, we have set the default encryption setting to `Tracing.Mode = tls`. Externally deployed otel-collectors can not be used with `Tracing.Mode = unencrypted`. i.e. If `Tracing.Mode = unencrypted` and an external URI is detected for `Tracing.CollectorTarget` node configuration will fail to validate and the node will not boot. The node requires a valid encryption mode and collector target to send traces. | ||
|
|
||
| Once traces reach the otel-collector, the rest of the observability pipeline is flexible. We recommend deploying (through automation) centrally managed Grafana Tempo and Grafana UI instances to receive from one or many otel-collector instances. Always use networking best practices and encrypt trace data, especially at network boundaries. | ||
|
|
||
| ## Configuration | ||
| This folder contains the following config files: | ||
| * otel-collector-ci.yaml | ||
| * otel-collector-dev.yaml | ||
| * tempo.yaml | ||
| * grafana-datasources.yaml | ||
|
|
||
| These config files are for an OTEL collector, grafana Tempo, and a grafana UI instance to run as containers on the same network. | ||
| `otel-collector-dev.yaml` is the configuration for dev (i.e. your local machine) environments, and forwards traces from the otel collector to the grafana tempo instance on the same network. | ||
| `otel-collector-ci.yaml` is the configuration for the CI runs, and exports the trace data to the artifact from the github run. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.