-
Notifications
You must be signed in to change notification settings - Fork 41
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
18 changed files
with
330 additions
and
30 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
.PHONY: run | ||
run: | ||
mdbook serve -p 9999 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
## MDBook | ||
Install [mdbook](https://github.com/rust-lang/mdBook) | ||
``` | ||
cargo install mdbook | ||
``` | ||
Run it locally | ||
``` | ||
make run | ||
``` | ||
Open your browser [here](http://localhost:9999/) | ||
|
||
GitHub hosted [version](https://smartcontractkit.github.io/chainlink-testing-framework/overview.html) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
## CLI | ||
### Install | ||
``` | ||
go get github.com/smartcontractkit/chainlink-testing-framework/framework/cmd && \ | ||
go install github.com/smartcontractkit/chainlink-testing-framework/framework/cmd && \ | ||
mv ~/go/bin/cmd ~/go/bin/ctf | ||
``` | ||
### Usage | ||
``` | ||
ctf -h | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
# Anvil | ||
[Anvil](https://book.getfoundry.sh/anvil/) is a Foundry local EVM blockchain simulator | ||
|
||
Use `docker_cmd_params = ['--block-time=1', '...']` to provide more params | ||
|
||
## Configuration | ||
```toml | ||
[blockchain_a] | ||
chain_id = "31337" | ||
docker_cmd_params = [] | ||
image = "ghcr.io/gakonst/foundry:latest" | ||
port = "8545" | ||
pull_image = false | ||
type = "anvil" | ||
|
||
[blockchain_a.out] | ||
chain_id = "31337" | ||
use_cache = true | ||
|
||
[[blockchain_a.out.nodes]] | ||
docker_internal_http_url = "http://anvil-14411:8545" | ||
docker_internal_ws_url = "ws://anvil-14411:8545" | ||
http_url = "http://127.0.0.1:33955" | ||
ws_url = "ws://127.0.0.1:33955" | ||
``` | ||
|
||
## Usage | ||
```golang | ||
package my_test | ||
|
||
import ( | ||
"os" | ||
"testing" | ||
"github.com/smartcontractkit/chainlink-testing-framework/framework" | ||
"github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain" | ||
"github.com/stretchr/testify/require" | ||
) | ||
|
||
type Config struct { | ||
BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"` | ||
} | ||
|
||
func TestDON(t *testing.T) { | ||
in, err := framework.Load[Config](t) | ||
require.NoError(t, err) | ||
pkey := os.Getenv("PRIVATE_KEY") | ||
|
||
// deploy anvil blockchain simulator | ||
bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA) | ||
require.NoError(t, err) | ||
} | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# Blockchain components (Simulators) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
## Component caching | ||
|
||
We use component caching to accelerate test development and enforce idempotent test actions. | ||
|
||
Each component is designed to be pure with defined inputs/outputs. | ||
|
||
You can use an environment variable to skip deployment steps and use cached outputs if your infrastructure is already running (locally or remotely): | ||
|
||
``` | ||
export CTF_CONFIGS=smoke-cache.toml | ||
``` | ||
|
||
### Using remote components | ||
|
||
Because components are decoupled through outputs, you can use a cached config and switch outputs to any deployed infrastructure, such as staging. This allows you to reuse the same testing logic for behavior validation. | ||
|
||
Example: | ||
``` | ||
[blockchain_a.out] | ||
use_cache = true | ||
chain_id = '31337' | ||
[[blockchain_a.out.nodes]] | ||
ws_url = 'ws://127.0.0.1:33447' | ||
http_url = 'http://127.0.0.1:33447' | ||
docker_internal_ws_url = 'ws://anvil-3716a:8900' | ||
docker_internal_http_url = 'http://anvil-3716a:8900' | ||
``` | ||
Set flag `use_cache = true` on any component output and run your test again |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,94 @@ | ||
## Developing Components | ||
|
||
To build a scalable framework that enables the reuse of our product deployments (contracts or services in Docker), we need to establish a clear component structure. | ||
``` | ||
package mycomponent | ||
import ( | ||
"fmt" | ||
"github.com/smartcontractkit/chainlink-testing-framework/framework" | ||
) | ||
type Input struct { | ||
// inputs fields that component exposes for configuration | ||
... | ||
// outputs are embedded into inputs so framework can automatically save them | ||
Out *Output `toml:"out"` | ||
} | ||
type Output struct { | ||
UseCache bool `toml:"use_cache"` | ||
// outputs that will be dumped to config and cached | ||
} | ||
func NewComponent(input *Input) (*Output, error) { | ||
if input.Out.UseCache { | ||
return input.Out, nil | ||
} | ||
// component logic here | ||
// deploy a docker container(s) | ||
// or deploy a set of smart contracts | ||
input.Out = &Output{ | ||
UseCache: true, | ||
// other fields | ||
... | ||
} | ||
return out, nil | ||
} | ||
``` | ||
Each component can define inputs and outputs, following these rules: | ||
|
||
- Outputs should be included within inputs. | ||
- If your component is used for side effects output can be omitted. | ||
- `input.Out.UseCache` should be added if you'd like to use caching, see more [here](caching) | ||
|
||
### Docker components good practices for [testcontainers-go](https://golang.testcontainers.org/): | ||
|
||
An example [simple component](components/blockchain/anvil.go) | ||
|
||
An example of [complex component](components/clnode/clnode.go) | ||
|
||
An example of [composite component](components/node_set_extended/don.go) | ||
|
||
- Inputs should include at least `image`, `tag` and `pull_image` field | ||
``` | ||
Image string `toml:"image" validate:"required"` | ||
Tag string `toml:"tag" validate:"required"` | ||
PullImage bool `toml:"pull_image" validate:"required"` | ||
``` | ||
|
||
- `ContainerRequest` must contain labels, network and alias required for local observability stack and deployment isolation | ||
``` | ||
Labels: framework.DefaultTCLabels(), | ||
Networks: []string{framework.DefaultNetworkName}, | ||
NetworkAliases: map[string][]string{ | ||
framework.DefaultNetworkName: {containerName}, | ||
}, | ||
``` | ||
- In order to copy files into container use `framework.WriteTmpFile(data string, fileName string)` | ||
``` | ||
userSecretsOverridesFile, err := WriteTmpFile(in.Node.UserSecretsOverrides, "user-secrets-overrides.toml") | ||
if err != nil { | ||
return nil, err | ||
} | ||
``` | ||
- Output of docker component must contain all the URLs component exposes for access, both for internal docker usage and external test (host) usage | ||
``` | ||
host, err := framework.GetHost(c) | ||
if err != nil { | ||
return nil, err | ||
} | ||
mp, err := c.MappedPort(ctx, nat.Port(bindPort)) | ||
if err != nil { | ||
return nil, err | ||
} | ||
return &NodeOut{ | ||
UseCache: true, | ||
DockerURL: fmt.Sprintf("http://%s:%s", containerName, in.Node.Port), | ||
HostURL: fmt.Sprintf("http://%s:%s", host, mp.Port()), | ||
}, nil | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
## Test Configuration | ||
|
||
In our framework, components control the configuration. Each component defines an Input that you embed into your test configuration. We automatically ensure consistency between TOML and struct definitions during validation. | ||
|
||
An example of your component configuration: | ||
``` | ||
// Input is a blockchain network configuration params declared by the component | ||
type Input struct { | ||
Type string `toml:"type" validate:"required,oneof=anvil geth" envconfig:"net_type"` | ||
Image string `toml:"image" validate:"required"` | ||
Tag string `toml:"tag" validate:"required"` | ||
Port string `toml:"port" validate:"required"` | ||
ChainID string `toml:"chain_id" validate:"required"` | ||
DockerCmdParamsOverrides []string `toml:"docker_cmd_params"` | ||
Out *Output `toml:"out"` | ||
} | ||
``` | ||
|
||
How you use it in tests: | ||
``` | ||
type Config struct { | ||
BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"` | ||
} | ||
func TestDON(t *testing.T) { | ||
in, err := framework.Load[Config](t) | ||
require.NoError(t, err) | ||
// deploy component | ||
bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA) | ||
require.NoError(t, err) | ||
``` | ||
|
||
In `TOML`: | ||
``` | ||
[blockchain_a] | ||
chain_id = "31337" | ||
docker_cmd_params = ["--block-time=1"] | ||
image = "f4hrenh9it/foundry" | ||
port = "8500" | ||
tag = "latest" | ||
type = "anvil" | ||
``` | ||
|
||
### Best practices for configuration and validation | ||
- Avoid stateful types (e.g., loggers, clients) in your config. | ||
- All `input` fields should include validate: "required", ensuring consistency between TOML and struct definitions. | ||
- Add extra validation rules for URLs or "one-of" variants. Learn more here: go-playground/validator. | ||
|
||
### Overriding configuration | ||
To override any configuration, we merge multiple files into a single struct. | ||
|
||
You can specify multiple file paths using `CTF_CONFIGS=path1,path2,path3`. | ||
|
||
The framework will apply these configurations from right to left. |
File renamed without changes.
Empty file.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
## Chainlink Testing Framework Harness | ||
|
||
<!-- TOC --> | ||
* [CLI](./cmd/README.md) | ||
* [Components](./COMPONENTS.md) | ||
* [Configuration](./CONFIGURATION.md) | ||
* [Caching](./CACHING.md) | ||
* [Local Observability Stack](./cmd/observability/README.md) | ||
* [Examples](https://github.com/smartcontractkit/chainlink/tree/8e8597aa14c39c48ed4b3261f6080fa43b5d7cd0/e2e/capabilities) | ||
<!-- TOC --> | ||
|
||
This module includes the CTFv2 harness, a lightweight, modular, and data-driven framework designed for combining off-chain and on-chain components while implementing best practices for end-to-end system-level testing: | ||
|
||
- **Non-nil configuration**: All test variables must have defaults, automatic validation. | ||
- **Component isolation**: Components are decoupled via input/output structs, without exposing internal details. | ||
- **Modular configuration**: No arcane knowledge of framework settings is required; the config is simply a reflection of the components being used in the test. Components declare their own configuration—'what you see is what you get.' | ||
- **Replaceability and extensibility**: Since components are decoupled via outputs, any deployment component can be swapped with a real service without altering the test code. | ||
- **Caching**: any component can use cached configs to skip environment setup for faster test development | ||
- **Integrated observability stack**: use `ctf obs up` to spin up a local observability stack. | ||
|
||
|
||
### Environment variables (Tests, when using in Go code) | ||
| Name | Description | Possible values | Default | Required? | | ||
|:----------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------:|-------------------------:|:-------:|:------------------------:| | ||
| CTF_CONFIGS | Path(s) to test config files. <br/>Can be more than one, ex.: smoke.toml,smoke_1.toml,smoke_2.toml.<br/>First filepath will hold all the merged values | Any valid TOML file path | | ✅ | | ||
| CTF_LOG_LEVEL | Harness log level | `info`, `debug`, `trace` | `info` | 🚫 | | ||
| CTF_LOKI_STREAM | Streams all components logs to `Loki`, see params below | `true`, `false` | `false` | 🚫 | | ||
| LOKI_URL | URL to `Loki` push api, should be like`${host}/loki/api/v1/push` | URL | - | If you use `Loki` then ✅ | | ||
| LOKI_TENANT_ID | Streams all components logs to `Loki`, see params below | `true`, `false` | - | If you use `Loki` then ✅ | | ||
| TESTCONTAINERS_RYUK_DISABLED | Testcontainers-Go reaper container, removes all the containers after the test exit | `true`, `false` | `false` | 🚫 | | ||
| RESTY_DEBUG | Log all Resty client HTTP calls | `true`, `false` | `false` | 🚫 | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.