diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 174d4d017..56c528401 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -1,17 +1,29 @@ # Summary - [Overview](./overview.md) -- [Libraries](./libraries.md) - - [Seth](./libs/seth.md) - - [WASP](./libs/wasp.md) - - [Havoc](./libs/havoc.md) - [Framework](./framework/overview.md) + - [Overview](./framework/overview.md) - [Getting Started](./framework/getting_started.md) + - [First Test](./framework/first_test.md) + - [Connecting Chainlink Node](./framework/connecting_chainlink_node.md) + - [Connecting Chainlink Node (Multiple networks)]() + - [Basic NodeSet Environment](./framework/basic_environment.md) - [CLI](./framework/cli.md) + - [Configuration](./framework/configuration.md) + - [Observability Stack](framework/observability_stack.md) + - [Metrics]() + - [Logs](framework/logs.md) + - [Profiling](framework/profiling.md) + - [Traces]() + - [Debugger]() - [Components](framework/components/overview.md) - - [Blockchains](framework/components/blockchains/overview.md) - - [Anvil](framework/components/blockchains/anvil.md) + - [Blockchains](framework/components/blockchains/overview.md) + - [Anvil](framework/components/blockchains/anvil.md) - [Secrets](./secrets.md) +- [Libraries](./libraries.md) + - [Seth](./libs/seth.md) + - [WASP](./libs/wasp.md) + - [Havoc](./libs/havoc.md) --- diff --git a/book/src/framework/basic_environment.md b/book/src/framework/basic_environment.md new file mode 100644 index 000000000..e3dfda260 --- /dev/null +++ b/book/src/framework/basic_environment.md @@ -0,0 +1,75 @@ +# Chainlink Cluster (NodeSet) Environment Test + +Create a configuration file `smoke.toml` +```toml +funds_eth = 30.0 + +[blockchain_a] + chain_id = "31337" + image = "f4hrenh9it/foundry:latest" + port = "8545" + type = "anvil" + +[contracts] + +[data_provider] + port = 9111 + +[nodeset] + nodes = 5 + override_mode = "all" + + [[nodeset.node_specs]] + + [nodeset.node_specs.db] + image = "postgres:15.6" + pull_image = true + + [nodeset.node_specs.node] + image = "public.ecr.aws/chainlink/chainlink:v2.17.0" + pull_image = true + +``` + +Create a file `smoke_test.go` +```golang +package yourpackage_test + +import ( + "fmt" + "github.com/smartcontractkit/chainlink-testing-framework/framework" + "github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain" + "github.com/smartcontractkit/chainlink-testing-framework/framework/components/fake" + ns "github.com/smartcontractkit/chainlink-testing-framework/framework/components/simple_node_set" + "github.com/smartcontractkit/chainlink/e2e/capabilities/components/onchain" + "github.com/stretchr/testify/require" + "testing" +) + +type Config struct { + FundingETH float64 `toml:"funds_eth"` + BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"` + Contracts *onchain.Input `toml:"contracts" validate:"required"` + MockerDataProvider *fake.Input `toml:"data_provider" validate:"required"` + NodeSet *ns.Input `toml:"nodeset" validate:"required"` +} + +func TestNodeSet(t *testing.T) { + in, err := framework.Load[Config](t) + require.NoError(t, err) + + // deploy docker test environment + bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA) + require.NoError(t, err) + dp, err := fake.NewFakeDataProvider(in.MockerDataProvider) + require.NoError(t, err) + out, err := ns.NewSharedDBNodeSet(in.NodeSet, bc, dp.BaseURLDocker) + require.NoError(t, err) +} +``` + +Run it +```bash +go test -v -run TestNodeSet +``` + diff --git a/book/src/framework/cli.md b/book/src/framework/cli.md index 8ea0ec309..2594d3b34 100644 --- a/book/src/framework/cli.md +++ b/book/src/framework/cli.md @@ -1,11 +1,6 @@ ## CLI -### Install -``` -go get github.com/smartcontractkit/chainlink-testing-framework/framework/cmd && \ -go install github.com/smartcontractkit/chainlink-testing-framework/framework/cmd && \ -mv ~/go/bin/cmd ~/go/bin/ctf -``` -### Usage + +To keep documentation simple we provide CLI docs in "help" format ``` ctf -h ``` \ No newline at end of file diff --git a/book/src/framework/components/blockchains/anvil.md b/book/src/framework/components/blockchains/anvil.md index afe486461..b859b0c99 100644 --- a/book/src/framework/components/blockchains/anvil.md +++ b/book/src/framework/components/blockchains/anvil.md @@ -1,25 +1,33 @@ # Anvil [Anvil](https://book.getfoundry.sh/anvil/) is a Foundry local EVM blockchain simulator -Use `docker_cmd_params = ['--block-time=1', '...']` to provide more params - ## Configuration ```toml [blockchain_a] + # Blockchain node type, can be "anvil" or "geth" + type = "anvil" + # Chain ID chain_id = "31337" + # Anvil command line params, ex.: docker_cmd_params = ['--block-time=1', '...'] docker_cmd_params = [] + # Docker image and tag image = "ghcr.io/gakonst/foundry:latest" + # External port to expose port = "8545" + # Pulls the image every time if set to 'true', used like that in CI. Can be set to 'false' to speed up local runs pull_image = false - type = "anvil" + # Outputs are the results of deploying a component that can be used by another component [blockchain_a.out] chain_id = "31337" + # If 'use_cache' equals 'true' we skip component setup when we run the test and return the outputs use_cache = true [[blockchain_a.out.nodes]] + # URLs to access the node(s) inside docker network, used by other components docker_internal_http_url = "http://anvil-14411:8545" docker_internal_ws_url = "ws://anvil-14411:8545" + # URLs to access the node(s) on your host machine or in CI http_url = "http://127.0.0.1:33955" ws_url = "ws://127.0.0.1:33955" ``` diff --git a/book/src/framework/configuration.md b/book/src/framework/configuration.md new file mode 100644 index 000000000..af37a6b68 --- /dev/null +++ b/book/src/framework/configuration.md @@ -0,0 +1,12 @@ +# Configuration + +### Environment variables +| Name | Description | Possible values | Default | Required? | +|:----------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------:|-------------------------:|:-------:|:------------------------:| +| CTF_CONFIGS | Path(s) to test config files.
Can be more than one, ex.: smoke.toml,smoke_1.toml,smoke_2.toml.
First filepath will hold all the merged values | Any valid TOML file path | | ✅ | +| CTF_LOG_LEVEL | Harness log level | `info`, `debug`, `trace` | `info` | 🚫 | +| CTF_LOKI_STREAM | Streams all components logs to `Loki`, see params below | `true`, `false` | `false` | 🚫 | +| LOKI_URL | URL to `Loki` push api, should be like`${host}/loki/api/v1/push` | URL | - | If you use `Loki` then ✅ | +| LOKI_TENANT_ID | Streams all components logs to `Loki`, see params below | `true`, `false` | - | If you use `Loki` then ✅ | +| TESTCONTAINERS_RYUK_DISABLED | Testcontainers-Go reaper container, removes all the containers after the test exit | `true`, `false` | `false` | 🚫 | +| RESTY_DEBUG | Log all Resty client HTTP calls | `true`, `false` | `false` | 🚫 | diff --git a/book/src/framework/connecting_chainlink_node.md b/book/src/framework/connecting_chainlink_node.md new file mode 100644 index 000000000..556c2531a --- /dev/null +++ b/book/src/framework/connecting_chainlink_node.md @@ -0,0 +1,80 @@ +# Connecting Chainlink Node + +The Chainlink Testing Framework (CTF) is a modular, data-driven tool that lets you explicitly define and configure various Chainlink components. + +Let's spin up a simple component. + + +Create your configuration in `smoke.toml` +```toml +[blockchain_a] + chain_id = "31337" + image = "ghcr.io/gakonst/foundry:latest" + port = "8545" + type = "anvil" + +[cl_node] + data_provider_url = "http://example.com" + + [cl_node.db] + image = "postgres:15.6" + pull_image = true + + [cl_node.node] + image = "public.ecr.aws/chainlink/chainlink:v2.17.0" + pull_image = true +``` + +Create your test in `smoke_test.go` +```golang + +package capabilities_test + +import ( + "fmt" + "github.com/smartcontractkit/chainlink-testing-framework/framework" + "github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain" + "github.com/smartcontractkit/chainlink-testing-framework/framework/components/clnode" + "github.com/stretchr/testify/require" + "testing" +) + +type Config struct { + BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"` + CLNode *clnode.Input `toml:"cl_node" validate:"required"` +} + +func TestNode(t *testing.T) { + in, err := framework.Load[Config](t) + require.NoError(t, err) + + bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA) + require.NoError(t, err) + + networkCfg, err := clnode.NewNetworkCfgOneNetworkAllNodes(bc) + require.NoError(t, err) + in.CLNode.Node.TestConfigOverrides = networkCfg + + output, err := clnode.NewNodeWithDB(in.CLNode) + require.NoError(t, err) + + t.Run("test something", func(t *testing.T) { + fmt.Printf("node url: %s\n", output.Node.HostURL) + require.NotEmpty(t, output.Node.HostURL) + }) +} + + +``` + +Select your configuration by setting `CTF_CONFIGS=smoke.toml` and run it +```bash +go test -v -run TestNode +``` + +Summary: +- We defined configuration for `BlockchainNetwork` and `NodeWithDB` (Chainlink + PostgreSQL) +- We connected them together by creating common network config in `NewNetworkCfgOneNetworkAllNodes` +- We have a Chainlink node running, check `node url: ...` messages in logs to open UI + +You can learn more about [component design](./components/overview.md) or proceed with another example of [connecting Chainlink node](./connecting_chainlink_node.md) \ No newline at end of file diff --git a/book/src/framework/connecting_multiple_nodes.md b/book/src/framework/connecting_multiple_nodes.md new file mode 100644 index 000000000..ec18e26d2 --- /dev/null +++ b/book/src/framework/connecting_multiple_nodes.md @@ -0,0 +1,80 @@ +# Writing your first test + +The Chainlink Testing Framework (CTF) is a modular, data-driven tool that lets you explicitly define and configure various Chainlink components. + +Let's spin up a simple component. + + +Create your configuration in `smoke.toml` +```toml +[blockchain_a] + chain_id = "31337" + image = "ghcr.io/gakonst/foundry:latest" + port = "8545" + type = "anvil" + +[cl_node] + data_provider_url = "http://example.com" + + [cl_node.db] + image = "postgres:15.6" + pull_image = true + + [cl_node.node] + image = "public.ecr.aws/chainlink/chainlink:v2.17.0" + pull_image = true +``` + +Create your test in `smoke_test.go` +```golang + +package capabilities_test + +import ( + "fmt" + "github.com/smartcontractkit/chainlink-testing-framework/framework" + "github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain" + "github.com/smartcontractkit/chainlink-testing-framework/framework/components/clnode" + "github.com/stretchr/testify/require" + "testing" +) + +type Config struct { + BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"` + CLNode *clnode.Input `toml:"cl_node" validate:"required"` +} + +func TestNode(t *testing.T) { + in, err := framework.Load[Config](t) + require.NoError(t, err) + + bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA) + require.NoError(t, err) + + networkCfg, err := clnode.NewNetworkCfgOneNetworkAllNodes(bc) + require.NoError(t, err) + in.CLNode.Node.TestConfigOverrides = networkCfg + + output, err := clnode.NewNodeWithDB(in.CLNode) + require.NoError(t, err) + + t.Run("test something", func(t *testing.T) { + fmt.Printf("node url: %s\n", output.Node.HostURL) + require.NotEmpty(t, output.Node.HostURL) + }) +} + + +``` + +Select your configuration by setting `CTF_CONFIGS=smoke.toml` and run it +```bash +go test -v -run TestNode +``` + +Summary: +- We defined configuration for `BlockchainNetwork` and `NodeWithDB` (Chainlink + PostgreSQL) +- We connected them together by creating common network config in `NewNetworkCfgOneNetworkAllNodes` +- We have a Chainlink node running, check `node url: ...` messages in logs to open UI + +You can learn more about [component design](./components/overview.md) or proceed with another example of [connecting Chainlink node](./connecting_chainlink_node.md) \ No newline at end of file diff --git a/book/src/framework/debugger.md b/book/src/framework/debugger.md new file mode 100644 index 000000000..b0f56432b --- /dev/null +++ b/book/src/framework/debugger.md @@ -0,0 +1 @@ +# Debugger diff --git a/book/src/framework/first_test.md b/book/src/framework/first_test.md new file mode 100644 index 000000000..99916286c --- /dev/null +++ b/book/src/framework/first_test.md @@ -0,0 +1,55 @@ +# Writing your first test + +The Chainlink Testing Framework (CTF) is a modular, data-driven tool that lets you explicitly define and configure various Chainlink components. + +Let's spin up a simple component. + +Create your configuration in `smoke.toml` +```toml +[blockchain_a] + chain_id = "31337" + image = "ghcr.io/gakonst/foundry:latest" + port = "8545" + type = "anvil" +``` + +Create your test in `smoke_test.go` +```golang +package mymodule_test + +import ( + "github.com/smartcontractkit/chainlink-testing-framework/framework" + "github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain" + "github.com/stretchr/testify/require" + "testing" +) + +type Config struct { + BlockchainA *blockchain.Input `toml:"blockchain_a" validate:"required"` +} + +func TestMe(t *testing.T) { + in, err := framework.Load[Config](t) + require.NoError(t, err) + + bc, err := blockchain.NewBlockchainNetwork(in.BlockchainA) + require.NoError(t, err) + + t.Run("test something", func(t *testing.T) { + require.NotEmpty(t, bc.Nodes[0].HostHTTPUrl) + }) +} +``` + +Select your configuration by setting `CTF_CONFIGS=smoke.toml` and run it +```bash +go test -v -run TestMe +``` + +Summary: +- We defined configuration for `BlockchainNetwork` +- We've used one CTF component in test and checked if it's working + +You can learn more about [component design](./components/overview.md) or proceed with another example of [connecting Chainlink node](./connecting_chainlink_node.md) + +Learn more about [anvil](./components/blockchains/anvil.md) component. \ No newline at end of file diff --git a/book/src/framework/getting_started.md b/book/src/framework/getting_started.md index e69de29bb..b02f20f87 100644 --- a/book/src/framework/getting_started.md +++ b/book/src/framework/getting_started.md @@ -0,0 +1,35 @@ +# Getting started + +To start writing tests create a directory for your project with `go.mod` and pull the framework +``` +go get github.com/smartcontractkit/chainlink-testing-framework/framework@ac819d889f97e0f5c04aee3212454ad1f8b6f4ef +``` + +Then download the CLI (runs from directory where you have `go.mod`) +``` +go get github.com/smartcontractkit/chainlink-testing-framework/framework/cmd && \ +go install github.com/smartcontractkit/chainlink-testing-framework/framework/cmd && \ +mv ~/go/bin/cmd ~/go/bin/ctf +``` +More CLI [docs](./cli.md) + + +Spin up your local obserability stack +``` +ctf obs up +``` + +Create an `.envrc` file and put common parameters there (you can use [direnv](https://direnv.net/) to sync them more easily) +``` +export CTF_LOG_LEVEL=info +export CTF_LOKI_STREAM=true +export LOKI_TENANT_ID=promtail +export LOKI_URL=http://host.docker.internal:3030/loki/api/v1/push +export TESTCONTAINERS_RYUK_DISABLED=true +export RESTY_DEBUG=false + +export CTF_CONFIGS=smoke.toml +export PRIVATE_KEY="ac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80" +``` + +Now you are ready to write your [first test](./first_test.md) \ No newline at end of file diff --git a/book/src/framework/images/profiling.png b/book/src/framework/images/profiling.png new file mode 100644 index 000000000..b058bb5eb Binary files /dev/null and b/book/src/framework/images/profiling.png differ diff --git a/book/src/framework/logs.md b/book/src/framework/logs.md new file mode 100644 index 000000000..f724d89f1 --- /dev/null +++ b/book/src/framework/logs.md @@ -0,0 +1,17 @@ +# Logs + +We are using `Loki` for logging, check [localhost:3000](http://localhost:3000/explore?panes=%7B%22vYC%22:%7B%22datasource%22:%22P8E80F9AEF21F6940%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bjob%3D%5C%22ctf%5C%22,%20container%3D~%5C%22node0%5C%22%7D%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22P8E80F9AEF21F6940%22%7D,%22editorMode%22:%22code%22%7D%5D,%22range%22:%7B%22from%22:%22now-1h%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1) + +Queries: +- Particular node logs +```json +{job="ctf", container=~"node0"} +``` +- All nodes logs +```json +{job="ctf", container=~"node"} +``` +- Filter by log level +```json +{job="ctf", container=~"node"} |= "WARN|INFO|DEBUG" +``` diff --git a/book/src/framework/observability_stack.md b/book/src/framework/observability_stack.md new file mode 100644 index 000000000..5b4364fdd --- /dev/null +++ b/book/src/framework/observability_stack.md @@ -0,0 +1,12 @@ +# Local Observability Stack + +You can use a local observability stack, framework is connected to it by default +```bash +ctf obs up +``` +To remove it use +```bash +ctf obs down +``` + +Read more about how to check [logs](logs.md) and [profiles](profiling.md) \ No newline at end of file diff --git a/book/src/framework/overview.md b/book/src/framework/overview.md index 93deeb164..6cd205715 100644 --- a/book/src/framework/overview.md +++ b/book/src/framework/overview.md @@ -1,14 +1,5 @@ ## Chainlink Testing Framework Harness - -* [CLI](./cmd/README.md) -* [Components](./COMPONENTS.md) -* [Configuration](./CONFIGURATION.md) -* [Caching](./CACHING.md) -* [Local Observability Stack](./cmd/observability/README.md) -* [Examples](https://github.com/smartcontractkit/chainlink/tree/8e8597aa14c39c48ed4b3261f6080fa43b5d7cd0/e2e/capabilities) - - This module includes the CTFv2 harness, a lightweight, modular, and data-driven framework designed for combining off-chain and on-chain components while implementing best practices for end-to-end system-level testing: - **Non-nil configuration**: All test variables must have defaults, automatic validation. @@ -18,14 +9,3 @@ This module includes the CTFv2 harness, a lightweight, modular, and data-driven - **Caching**: any component can use cached configs to skip environment setup for faster test development - **Integrated observability stack**: use `ctf obs up` to spin up a local observability stack. - -### Environment variables (Tests, when using in Go code) -| Name | Description | Possible values | Default | Required? | -|:----------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------:|-------------------------:|:-------:|:------------------------:| -| CTF_CONFIGS | Path(s) to test config files.
Can be more than one, ex.: smoke.toml,smoke_1.toml,smoke_2.toml.
First filepath will hold all the merged values | Any valid TOML file path | | ✅ | -| CTF_LOG_LEVEL | Harness log level | `info`, `debug`, `trace` | `info` | 🚫 | -| CTF_LOKI_STREAM | Streams all components logs to `Loki`, see params below | `true`, `false` | `false` | 🚫 | -| LOKI_URL | URL to `Loki` push api, should be like`${host}/loki/api/v1/push` | URL | - | If you use `Loki` then ✅ | -| LOKI_TENANT_ID | Streams all components logs to `Loki`, see params below | `true`, `false` | - | If you use `Loki` then ✅ | -| TESTCONTAINERS_RYUK_DISABLED | Testcontainers-Go reaper container, removes all the containers after the test exit | `true`, `false` | `false` | 🚫 | -| RESTY_DEBUG | Log all Resty client HTTP calls | `true`, `false` | `false` | 🚫 | \ No newline at end of file diff --git a/book/src/framework/profiling.md b/book/src/framework/profiling.md new file mode 100644 index 000000000..2b864a2ad --- /dev/null +++ b/book/src/framework/profiling.md @@ -0,0 +1,6 @@ +# Profiling + +We are using [Pyroscope]() for profiling. + +Go to [localhost:4040](http://localhost:4040) and choose Chainlink application +![img.png](images/profiling.png) diff --git a/book/src/overview.md b/book/src/overview.md index 38c8619f2..f03a70e25 100644 --- a/book/src/overview.md +++ b/book/src/overview.md @@ -2,22 +2,3 @@ The Chainlink Testing Framework (CTF) is a blockchain development framework written in Go. Its primary purpose is to help chainlink developers create extensive integration, e2e, performance, and chaos tests to ensure the stability of the chainlink project. It can also be helpful to those who just want to use chainlink oracles in their projects to help test their contracts, or even for those that aren't using chainlink. - - -# Content - -1. [Libraries](#libraries) -2. [Releasing](#releasing) - -## Libraries - -CTF monorepository contains a set of libraries: - -- [Framework](framework.md) - Library to interact with different blockchains, create CL node jobs and use k8s and docker. -- [WASP](wasp.md) - Scalable protocol-agnostic load testing library for `Go` -- [Havoc](havoc.md) - Chaos testing library -- [Seth](seth.md) - Ethereum client library with transaction tracing and gas bumping - -## Releasing - -We follow [SemVer](https://semver.org/) and follow best Go practices for releasing our modules, please follow the [instruction](RELEASE.md) \ No newline at end of file