Skip to content

Commit

Permalink
try rebuild
Browse files Browse the repository at this point in the history
  • Loading branch information
skudasov committed Oct 30, 2024
1 parent 2776393 commit 112a680
Show file tree
Hide file tree
Showing 18 changed files with 330 additions and 30 deletions.
3 changes: 3 additions & 0 deletions book/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.PHONY: run
run:
mdbook serve -p 9999
12 changes: 12 additions & 0 deletions book/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
## MDBook
Install [mdbook](https://github.com/rust-lang/mdBook)
```
cargo install mdbook
```
Run it locally
```
make run
```
Open your browser [here](http://localhost:9999/)

GitHub hosted [version](https://smartcontractkit.github.io/chainlink-testing-framework/overview.html)
5 changes: 4 additions & 1 deletion book/book.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,7 @@ src = "src"
title = "Chainlink Testing Framework"

[build]
build-dir = "docs"
build-dir = "docs"

[output.html]
default-theme = "ayu"
Binary file removed book/mdbook
Binary file not shown.
12 changes: 10 additions & 2 deletions book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,16 @@
- [Libraries](./libraries.md)
- [Seth](./libs/seth.md)
- [WASP](./libs/wasp.md)
- [Framework](./framework.md)
- [Components](./components.md)
- [Havoc](./libs/havoc.md)
- [Framework](./framework/overview.md)
- [Getting Started](./framework/getting_started.md)
- [CLI](./framework/cli.md)
- [Components](framework/components/overview.md)
- [Blockchains](framework/components/blockchains/overview.md)
- [Anvil](framework/components/blockchains/anvil.md)
- [Secrets](./secrets.md)

---

- [Developing](./developing.md)
- [Releasing modules](./releasing_modules.md)
11 changes: 11 additions & 0 deletions book/src/framework/cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
## CLI
### Install
```
go get github.com/smartcontractkit/chainlink-testing-framework/framework/cmd && \
go install github.com/smartcontractkit/chainlink-testing-framework/framework/cmd && \
mv ~/go/bin/cmd ~/go/bin/ctf
```
### Usage
```
ctf -h
```
53 changes: 53 additions & 0 deletions book/src/framework/components/blockchains/anvil.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Anvil
[Anvil](https://book.getfoundry.sh/anvil/) is a Foundry local EVM blockchain simulator

Use `docker_cmd_params = ['--block-time=1', '...']` to provide more params

## Configuration
```toml
[blockchain_a]
chain_id = "31337"
docker_cmd_params = []
image = "ghcr.io/gakonst/foundry:latest"
port = "8545"
pull_image = false
type = "anvil"

[blockchain_a.out]
chain_id = "31337"
use_cache = true

[[blockchain_a.out.nodes]]
docker_internal_http_url = "http://anvil-14411:8545"
docker_internal_ws_url = "ws://anvil-14411:8545"
http_url = "http://127.0.0.1:33955"
ws_url = "ws://127.0.0.1:33955"
```

## Usage
```golang
package my_test

import (
"os"
"testing"
"github.com/smartcontractkit/chainlink-testing-framework/framework"
"github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain"
"github.com/stretchr/testify/require"
)

type Config struct {
BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"`
}

func TestDON(t *testing.T) {
in, err := framework.Load[Config](t)
require.NoError(t, err)
pkey := os.Getenv("PRIVATE_KEY")

// deploy anvil blockchain simulator
bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA)
require.NoError(t, err)
}
```

1 change: 1 addition & 0 deletions book/src/framework/components/blockchains/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Blockchain components (Simulators)
29 changes: 29 additions & 0 deletions book/src/framework/components/caching.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
## Component caching

We use component caching to accelerate test development and enforce idempotent test actions.

Each component is designed to be pure with defined inputs/outputs.

You can use an environment variable to skip deployment steps and use cached outputs if your infrastructure is already running (locally or remotely):

```
export CTF_CONFIGS=smoke-cache.toml
```

### Using remote components

Because components are decoupled through outputs, you can use a cached config and switch outputs to any deployed infrastructure, such as staging. This allows you to reuse the same testing logic for behavior validation.

Example:
```
[blockchain_a.out]
use_cache = true
chain_id = '31337'
[[blockchain_a.out.nodes]]
ws_url = 'ws://127.0.0.1:33447'
http_url = 'http://127.0.0.1:33447'
docker_internal_ws_url = 'ws://anvil-3716a:8900'
docker_internal_http_url = 'http://anvil-3716a:8900'
```
Set flag `use_cache = true` on any component output and run your test again
94 changes: 94 additions & 0 deletions book/src/framework/components/components.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
## Developing Components

To build a scalable framework that enables the reuse of our product deployments (contracts or services in Docker), we need to establish a clear component structure.
```
package mycomponent
import (
"fmt"
"github.com/smartcontractkit/chainlink-testing-framework/framework"
)
type Input struct {
// inputs fields that component exposes for configuration
...
// outputs are embedded into inputs so framework can automatically save them
Out *Output `toml:"out"`
}
type Output struct {
UseCache bool `toml:"use_cache"`
// outputs that will be dumped to config and cached
}
func NewComponent(input *Input) (*Output, error) {
if input.Out.UseCache {
return input.Out, nil
}
// component logic here
// deploy a docker container(s)
// or deploy a set of smart contracts
input.Out = &Output{
UseCache: true,
// other fields
...
}
return out, nil
}
```
Each component can define inputs and outputs, following these rules:

- Outputs should be included within inputs.
- If your component is used for side effects output can be omitted.
- `input.Out.UseCache` should be added if you'd like to use caching, see more [here](caching)

### Docker components good practices for [testcontainers-go](https://golang.testcontainers.org/):

An example [simple component](components/blockchain/anvil.go)

An example of [complex component](components/clnode/clnode.go)

An example of [composite component](components/node_set_extended/don.go)

- Inputs should include at least `image`, `tag` and `pull_image` field
```
Image string `toml:"image" validate:"required"`
Tag string `toml:"tag" validate:"required"`
PullImage bool `toml:"pull_image" validate:"required"`
```

- `ContainerRequest` must contain labels, network and alias required for local observability stack and deployment isolation
```
Labels: framework.DefaultTCLabels(),
Networks: []string{framework.DefaultNetworkName},
NetworkAliases: map[string][]string{
framework.DefaultNetworkName: {containerName},
},
```
- In order to copy files into container use `framework.WriteTmpFile(data string, fileName string)`
```
userSecretsOverridesFile, err := WriteTmpFile(in.Node.UserSecretsOverrides, "user-secrets-overrides.toml")
if err != nil {
return nil, err
}
```
- Output of docker component must contain all the URLs component exposes for access, both for internal docker usage and external test (host) usage
```
host, err := framework.GetHost(c)
if err != nil {
return nil, err
}
mp, err := c.MappedPort(ctx, nat.Port(bindPort))
if err != nil {
return nil, err
}
return &NodeOut{
UseCache: true,
DockerURL: fmt.Sprintf("http://%s:%s", containerName, in.Node.Port),
HostURL: fmt.Sprintf("http://%s:%s", host, mp.Port()),
}, nil
```
55 changes: 55 additions & 0 deletions book/src/framework/components/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
## Test Configuration

In our framework, components control the configuration. Each component defines an Input that you embed into your test configuration. We automatically ensure consistency between TOML and struct definitions during validation.

An example of your component configuration:
```
// Input is a blockchain network configuration params declared by the component
type Input struct {
Type string `toml:"type" validate:"required,oneof=anvil geth" envconfig:"net_type"`
Image string `toml:"image" validate:"required"`
Tag string `toml:"tag" validate:"required"`
Port string `toml:"port" validate:"required"`
ChainID string `toml:"chain_id" validate:"required"`
DockerCmdParamsOverrides []string `toml:"docker_cmd_params"`
Out *Output `toml:"out"`
}
```

How you use it in tests:
```
type Config struct {
BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"`
}
func TestDON(t *testing.T) {
in, err := framework.Load[Config](t)
require.NoError(t, err)
// deploy component
bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA)
require.NoError(t, err)
```

In `TOML`:
```
[blockchain_a]
chain_id = "31337"
docker_cmd_params = ["--block-time=1"]
image = "f4hrenh9it/foundry"
port = "8500"
tag = "latest"
type = "anvil"
```

### Best practices for configuration and validation
- Avoid stateful types (e.g., loggers, clients) in your config.
- All `input` fields should include validate: "required", ensuring consistency between TOML and struct definitions.
- Add extra validation rules for URLs or "one-of" variants. Learn more here: go-playground/validator.

### Overriding configuration
To override any configuration, we merge multiple files into a single struct.

You can specify multiple file paths using `CTF_CONFIGS=path1,path2,path3`.

The framework will apply these configurations from right to left.
File renamed without changes.
Empty file.
Empty file.
31 changes: 31 additions & 0 deletions book/src/framework/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
## Chainlink Testing Framework Harness

<!-- TOC -->
* [CLI](./cmd/README.md)
* [Components](./COMPONENTS.md)
* [Configuration](./CONFIGURATION.md)
* [Caching](./CACHING.md)
* [Local Observability Stack](./cmd/observability/README.md)
* [Examples](https://github.com/smartcontractkit/chainlink/tree/8e8597aa14c39c48ed4b3261f6080fa43b5d7cd0/e2e/capabilities)
<!-- TOC -->

This module includes the CTFv2 harness, a lightweight, modular, and data-driven framework designed for combining off-chain and on-chain components while implementing best practices for end-to-end system-level testing:

- **Non-nil configuration**: All test variables must have defaults, automatic validation.
- **Component isolation**: Components are decoupled via input/output structs, without exposing internal details.
- **Modular configuration**: No arcane knowledge of framework settings is required; the config is simply a reflection of the components being used in the test. Components declare their own configuration—'what you see is what you get.'
- **Replaceability and extensibility**: Since components are decoupled via outputs, any deployment component can be swapped with a real service without altering the test code.
- **Caching**: any component can use cached configs to skip environment setup for faster test development
- **Integrated observability stack**: use `ctf obs up` to spin up a local observability stack.


### Environment variables (Tests, when using in Go code)
| Name | Description | Possible values | Default | Required? |
|:----------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------:|-------------------------:|:-------:|:------------------------:|
| CTF_CONFIGS | Path(s) to test config files. <br/>Can be more than one, ex.: smoke.toml,smoke_1.toml,smoke_2.toml.<br/>First filepath will hold all the merged values | Any valid TOML file path | ||
| CTF_LOG_LEVEL | Harness log level | `info`, `debug`, `trace` | `info` | 🚫 |
| CTF_LOKI_STREAM | Streams all components logs to `Loki`, see params below | `true`, `false` | `false` | 🚫 |
| LOKI_URL | URL to `Loki` push api, should be like`${host}/loki/api/v1/push` | URL | - | If you use `Loki` then ✅ |
| LOKI_TENANT_ID | Streams all components logs to `Loki`, see params below | `true`, `false` | - | If you use `Loki` then ✅ |
| TESTCONTAINERS_RYUK_DISABLED | Testcontainers-Go reaper container, removes all the containers after the test exit | `true`, `false` | `false` | 🚫 |
| RESTY_DEBUG | Log all Resty client HTTP calls | `true`, `false` | `false` | 🚫 |
6 changes: 3 additions & 3 deletions book/src/libs/havoc.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ chaos.AddListener(logger)

Initialize `SingleLineGrafanaAnnotator` with your Grafana instance details and register it alongside `ChaosLogger`:

```
```golang
annotator := havoc.NewSingleLineGrafanaAnnotator(
"http://grafana-instance.com",
"grafana-access-token",
Expand All @@ -59,7 +59,7 @@ To create a chaos experiment, define the chaos object options, initialize a chao

Here is an example of creating and starting a PodChaos experiment:

```
```golang
package main

import (
Expand Down Expand Up @@ -108,7 +108,7 @@ func main() {

### Test Example

```
```golang
func TestChaosDON(t *testing.T) {
testDuration := time.Minute * 60

Expand Down
Loading

0 comments on commit 112a680

Please sign in to comment.