Skip to content

Commit

Permalink
examples, more readme.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Tofel committed Jan 4, 2024
1 parent 0e429ea commit a1c9f0a
Show file tree
Hide file tree
Showing 3 changed files with 285 additions and 19 deletions.
51 changes: 41 additions & 10 deletions config/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,20 @@ type GenericConfig[T any] interface {

`Validate()` should be used to ensure that the config is valid. `ApplyOverride()` should be used to apply overrides from another config. Some of the building blocks have also a `Default()` method that can be used to get default values.

## Working example

For a full working example making use of all the building blocks see [testconfig.go](../config/examples/testconfig.go). It provides methods for reading TOML, applying overrides and validating non-empty config blocks. It supports 4 levels of overrides, in order of precedence:
* `BASE64_CONFIG_OVERRIDE` env var
* `overrides.toml`
* `[product_name].toml`
* `default.toml`

All you need to do now to get the config is execute `func GetConfig(configurationName string, product string) (TestConfig, error)`. It will first look for folder with file `.root_dir` and from there it will look for config files in all subfolders, so that you can place the config files in whatever folder(s) work for you. It assumes that all configuration versions for a single product are kept in `[product_name].toml` under different configuration names (that can represent anything you want: a single test, a test type, a test group, etc).

It is advised to add `overrides.toml` to `.gitignore`.

## Network config (and default RPC endpoints)

Some more explanation is needed for the `NetworkConfig`:
```golang
type NetworkConfig struct {
Expand All @@ -45,19 +59,19 @@ It not only stores the configuration of selected networks and RPC endpoints and

## Providing custom values in the CI

Up to this point when we wanted to modify some dynamic tests parameters in the CI we would simply set env vars. That approach won't work anymore. The way to go around it is to build a TOML file, base64 it, mask it and then set is as `BASE64_CONFIG_OVERRIDE` env var that will be read by tests. Here's an example of a working snippet of how that could look:
Up to this point when we wanted to modify some dynamic tests parameters in the CI we would simply set env vars. That approach won't work anymore. The way to go around it is to build a TOML file, `base64` it, mask it and then set is as `BASE64_CONFIG_OVERRIDE` env var that will be read by tests. Here's an example of a working snippet of how that could look:
```bash
convert_to_toml_array() {
local IFS=','
local input_array=($1)
local toml_array_format="["
local IFS=','
local input_array=($1)
local toml_array_format="["

for element in "${input_array[@]}"; do
toml_array_format+="\"$element\","
done
for element in "${input_array[@]}"; do
toml_array_format+="\"$element\","
done

toml_array_format="${toml_array_format%,}]"
echo "$toml_array_format"
toml_array_format="${toml_array_format%,}]"
echo "$toml_array_format"
}

selected_networks=$(convert_to_toml_array "$SELECTED_NETWORKS")
Expand Down Expand Up @@ -122,4 +136,21 @@ BASE64_CONFIG_OVERRIDE=$(cat config.toml | base64 -w 0)
echo ::add-mask::$BASE64_CONFIG_OVERRIDE
```

`::add-mask::` has to be called only after env var has been set to it's final value, otherwise it won't be recognized and masked properly and secrets will be exposed in the logs.
`::add-mask::` has to be called only after env var has been set to it's final value, otherwise it won't be recognized and masked properly and secrets will be exposed in the logs.

## Providing custom values for local execution
For local execution it's best to put custom variables in `overrides.toml` file.

## Providing custom values in k8s
It's easy. All you need to do is:
* Create TOML file with these values
* Base64 it: `cat your.toml | base64`
* Set the base64 result as `BASE64_CONFIG_OVERRIDE` environment variable.

Both `BASE64_CONFIG_OVERRIDE` and `BASE64_NETWORK_CONFIG` will be automatically forwarded to k8s, when creating the environment programmatically via `environment.New()`.


# Known issues/limitations
* Slack configuration wasn't moved to TOML
* `TEST_LOG_LEVEL` als wasn't moved
* most of k8s-specific env variables were left untouched
237 changes: 237 additions & 0 deletions config/examples/testconfig.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
package examples

import (
"encoding/base64"
"fmt"
"os"
"strings"

"github.com/pelletier/go-toml/v2"
"github.com/pkg/errors"
"golang.org/x/text/cases"
"golang.org/x/text/language"

ctf_config "github.com/smartcontractkit/chainlink-testing-framework/config"
ctf_test_env "github.com/smartcontractkit/chainlink-testing-framework/docker/test_env"
k8s_config "github.com/smartcontractkit/chainlink-testing-framework/k8s/config"
"github.com/smartcontractkit/chainlink-testing-framework/logging"
"github.com/smartcontractkit/chainlink-testing-framework/utils/osutil"
)

type TestConfig struct {
ChainlinkImage *ctf_config.ChainlinkImageConfig `toml:"ChainlinkImage"`
Logging *ctf_config.LoggingConfig `toml:"Logging"`
Network *ctf_config.NetworkConfig `toml:"Network"`
Pyroscope *ctf_config.PyroscopeConfig `toml:"Pyroscope"`
PrivateEthereumNetwork *ctf_test_env.EthereumNetwork `toml:"PrivateEthereumNetwork"`
}

func GetConfig(configurationName string, product string) (TestConfig, error) {
logger := logging.GetTestLogger(nil)

configurationName = strings.ReplaceAll(configurationName, "/", "_")
configurationName = strings.ReplaceAll(configurationName, " ", "_")
configurationName = cases.Title(language.English, cases.NoLower).String(configurationName)
fileNames := []string{
"default.toml",
fmt.Sprintf("%s.toml", product),
"overrides.toml",
}

testConfig := TestConfig{}
maybeTestConfigs := []TestConfig{}

logger.Debug().Msgf("Will apply configuration named '%s' if it is found in any of the configs", configurationName)

for _, fileName := range fileNames {
logger.Debug().Msgf("Looking for config file %s", fileName)
filePath, err := osutil.FindFile(fileName, osutil.DEFAULT_STOP_FILE_NAME)

if err != nil && errors.Is(err, os.ErrNotExist) {
logger.Debug().Msgf("Config file %s not found", fileName)
continue
}
logger.Debug().Str("location", filePath).Msgf("Found config file %s", fileName)

content, err := readFile(filePath)
if err != nil {
return TestConfig{}, errors.Wrapf(err, "error reading file %s", filePath)
}

var readConfig TestConfig
err = toml.Unmarshal(content, &readConfig)
if err != nil {
return TestConfig{}, errors.Wrapf(err, "error unmarshaling config")
}

logger.Debug().Msgf("Successfully unmarshalled config file %s", fileName)
maybeTestConfigs = append(maybeTestConfigs, readConfig)

var someToml map[string]interface{}
err = toml.Unmarshal(content, &someToml)
if err != nil {
return TestConfig{}, err
}

if _, ok := someToml[configurationName]; !ok {
logger.Debug().Msgf("Config file %s does not contain configuration named '%s', skipping.", fileName, configurationName)
continue
}

marshalled, err := toml.Marshal(someToml[configurationName])
if err != nil {
return TestConfig{}, err
}

err = toml.Unmarshal(marshalled, &readConfig)
if err != nil {
return TestConfig{}, err
}

logger.Debug().Msgf("Configuration named '%s' read successfully.", configurationName)
maybeTestConfigs = append(maybeTestConfigs, readConfig)
}

configEncoded, isSet := os.LookupEnv(k8s_config.EnvBase64ConfigOverride)
if isSet && configEncoded != "" {
decoded, err := base64.StdEncoding.DecodeString(configEncoded)
if err != nil {
return TestConfig{}, err
}

var base64override TestConfig
err = toml.Unmarshal(decoded, &base64override)
if err != nil {
return TestConfig{}, errors.Wrapf(err, "error unmarshaling base64 config")
}

logger.Debug().Msgf("Applying base64 config override from environment variable %s", k8s_config.EnvBase64ConfigOverride)
maybeTestConfigs = append(maybeTestConfigs, base64override)
} else {
logger.Debug().Msg("Base64 config override from environment variable not found")
}

// currently we need to read that kind of secrets only for network configuration
testConfig.Network = &ctf_config.NetworkConfig{}
err := testConfig.Network.ApplySecrets()
if err != nil {
return TestConfig{}, errors.Wrapf(err, "error applying secrets to network config")
}

for i := range maybeTestConfigs {
err := testConfig.ApplyOverrides(&maybeTestConfigs[i])
if err != nil {
return TestConfig{}, errors.Wrapf(err, "error applying overrides to test config")
}
}

err = testConfig.Validate()
if err != nil {
return TestConfig{}, errors.Wrapf(err, "error validating test config")
}

return testConfig, nil
}

func (c *TestConfig) ApplyOverrides(from *TestConfig) error {
if from == nil {
return nil
}

if from.ChainlinkImage != nil {
if c.ChainlinkImage == nil {
c.ChainlinkImage = from.ChainlinkImage
} else {
err := c.ChainlinkImage.ApplyOverrides(from.ChainlinkImage)
if err != nil {
return errors.Wrapf(err, "error applying overrides to chainlink image config")
}
}
}

if from.Logging != nil {
if c.Logging == nil {
c.Logging = from.Logging
} else {
err := c.Logging.ApplyOverrides(from.Logging)
if err != nil {
return errors.Wrapf(err, "error applying overrides to logging config")
}
}
}

if from.Network != nil {
if c.Network == nil {
c.Network = from.Network
} else {
err := c.Network.ApplyOverrides(from.Network)
if err != nil {
return errors.Wrapf(err, "error applying overrides to network config")
}
}
}

if from.Pyroscope != nil {
if c.Pyroscope == nil {
c.Pyroscope = from.Pyroscope
} else {
err := c.Pyroscope.ApplyOverrides(from.Pyroscope)
if err != nil {
return errors.Wrapf(err, "error applying overrides to pyroscope config")
}
}
}

if from.PrivateEthereumNetwork != nil {
if c.PrivateEthereumNetwork == nil {
c.PrivateEthereumNetwork = from.PrivateEthereumNetwork
} else {
err := c.PrivateEthereumNetwork.ApplyOverrides(from.PrivateEthereumNetwork)
if err != nil {
return errors.Wrapf(err, "error applying overrides to private ethereum network config")
}
}
c.PrivateEthereumNetwork.EthereumChainConfig.GenerateGenesisTimestamp()
}

return nil
}

func (c *TestConfig) Validate() error {
if c.ChainlinkImage == nil {
return fmt.Errorf("chainlink image config must be set")
}
if err := c.ChainlinkImage.Validate(); err != nil {
return errors.Wrapf(err, "chainlink image config validation failed")
}
if err := c.Network.Validate(); err != nil {
return errors.Wrapf(err, "network config validation failed")
}
if c.Logging == nil {
return fmt.Errorf("logging config must be set")
}
if err := c.Logging.Validate(); err != nil {
return errors.Wrapf(err, "logging config validation failed")
}
if c.Pyroscope != nil {
if err := c.Pyroscope.Validate(); err != nil {
return errors.Wrapf(err, "pyroscope config validation failed")
}
}
if c.PrivateEthereumNetwork != nil {
if err := c.PrivateEthereumNetwork.Validate(); err != nil {
return errors.Wrapf(err, "private ethereum network config validation failed")
}
}

return nil
}

func readFile(filePath string) ([]byte, error) {
content, err := os.ReadFile(filePath)
if err != nil {
return nil, errors.Wrapf(err, "error reading file %s", filePath)
}

return content, nil
}
16 changes: 7 additions & 9 deletions networks/known_networks_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ func TestMain(m *testing.M) {
os.Exit(m.Run())
}

func TestMustGetSelectedNetworksFromEnv_MissingSelectedNetwork(t *testing.T) {
func TestMustGetSelectedNetworkConfig_MissingSelectedNetwork(t *testing.T) {
require.Panics(t, func() {
MustGetSelectedNetworkConfig(&config.NetworkConfig{})
})
}

func TestMustGetSelectedNetworksFromEnv_Missing_RpcHttpUrls(t *testing.T) {
func TestMustGetSelectedNetworkConfig_Missing_RpcHttpUrls(t *testing.T) {
networkName := "arbitrum_goerli"
testTOML := `
selected_networks = ["arbitrum_goerli"]
Expand All @@ -45,7 +45,7 @@ func TestMustGetSelectedNetworksFromEnv_Missing_RpcHttpUrls(t *testing.T) {
})
}

func TestMustGetSelectedNetworksFromEnv_Missing_RpcWsUrls(t *testing.T) {
func TestMustGetSelectedNetworkConfig_Missing_RpcWsUrls(t *testing.T) {
networkName := "arbitrum_goerli"
testTOML := `
selected_networks = ["arbitrum_goerli"]
Expand All @@ -66,7 +66,7 @@ func TestMustGetSelectedNetworksFromEnv_Missing_RpcWsUrls(t *testing.T) {
})
}

func TestMustGetSelectedNetworksFromEnv_Missing_WalletKeys(t *testing.T) {
func TestMustGetSelectedNetworkConfig_Missing_WalletKeys(t *testing.T) {
networkName := "arbitrum_goerli"
testTOML := `
selected_networks = ["arbitrum_goerli"]
Expand All @@ -87,7 +87,7 @@ func TestMustGetSelectedNetworksFromEnv_Missing_WalletKeys(t *testing.T) {
})
}

func TestMustGetSelectedNetworksFromEnv_DefaultUrlsFromSecret(t *testing.T) {
func TestMustGetSelectedNetworkConfig_DefaultUrlsFromSecret(t *testing.T) {
networkConfigTOML := `
[RpcHttpUrls]
arbitrum_goerli = ["https://devnet-1.mt/ABC/rpc/"]
Expand Down Expand Up @@ -119,9 +119,7 @@ func TestMustGetSelectedNetworksFromEnv_DefaultUrlsFromSecret(t *testing.T) {
require.Equal(t, []string{"1810868fc221b9f50b5b3e0186d8a5f343f892e51ce12a9e818f936ec0b651ed"}, networks[0].PrivateKeys)
}

//defaults and passed in config, passed in config should override defaults

func TestMustGetSelectedNetworksFromEnv_MultipleNetworks(t *testing.T) {
func TesMustGetSelectedNetworkConfig_MultipleNetworks(t *testing.T) {
testTOML := `
selected_networks = ["arbitrum_goerli", "optimism_goerli"]
Expand All @@ -148,7 +146,7 @@ func TestMustGetSelectedNetworksFromEnv_MultipleNetworks(t *testing.T) {
require.Equal(t, "Optimism Goerli", networks[1].Name)
}

func TestMustGetSelectedNetworksFromEnv_DefaultUrlsFromSecret_OverrideOne(t *testing.T) {
func TestMustGetSelectedNetworkConfig_DefaultUrlsFromSecret_OverrideOne(t *testing.T) {
networkConfigTOML := `
[RpcHttpUrls]
arbitrum_goerli = ["https://devnet-1.mt/ABC/rpc/"]
Expand Down

0 comments on commit a1c9f0a

Please sign in to comment.