Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup ten config library to replace our flags approach #2115

Merged
merged 29 commits into from
Nov 6, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
158c828
Initial spec for ten config
BedrockSquirrel Sep 10, 2024
73ccf36
WIP fixing after rebase
BedrockSquirrel Oct 22, 2024
c3511ac
WIP
BedrockSquirrel Oct 23, 2024
7194eab
WIP local network still buggered:
BedrockSquirrel Oct 23, 2024
000c78f
WIP some fixes
BedrockSquirrel Oct 24, 2024
648b8b4
fix local testnet launcher
BedrockSquirrel Oct 24, 2024
07623b1
tidying up
BedrockSquirrel Oct 26, 2024
ec2ec5e
Tidying and fixes
BedrockSquirrel Oct 29, 2024
eab6f52
Fix type cfg
BedrockSquirrel Oct 29, 2024
f1f61c9
Merge remote-tracking branch 'origin/main' into matt/config-viper
BedrockSquirrel Oct 29, 2024
8f5c248
add the host ID back in for now
BedrockSquirrel Oct 29, 2024
dc06319
Merge remote-tracking branch 'origin/main' into matt/config-viper
BedrockSquirrel Oct 30, 2024
39b6544
review comments
BedrockSquirrel Oct 30, 2024
4511e92
linting
BedrockSquirrel Oct 30, 2024
bb9ac7f
fix dockerfile
BedrockSquirrel Oct 30, 2024
552ef00
fix tests
BedrockSquirrel Oct 30, 2024
bf4d073
WIP fixes for node CLI
BedrockSquirrel Oct 31, 2024
35c1538
fix enclave DB flag
BedrockSquirrel Oct 31, 2024
8812b19
node launcher edb path
BedrockSquirrel Nov 1, 2024
1029750
node launcher: missed enclave rpc host
BedrockSquirrel Nov 1, 2024
26c9c37
node launcher: fix host db cfg
BedrockSquirrel Nov 1, 2024
987467f
Merge remote-tracking branch 'origin/main' into matt/config-viper
BedrockSquirrel Nov 1, 2024
4cd32e5
testnet launcher: turn on debug namespace by default
BedrockSquirrel Nov 1, 2024
276451d
fix noderunner test
BedrockSquirrel Nov 4, 2024
2d92ad9
Merge remote-tracking branch 'origin/main' into matt/config-viper
BedrockSquirrel Nov 4, 2024
ca7da4d
Merge remote-tracking branch 'origin/main' into matt/config-viper
BedrockSquirrel Nov 5, 2024
4a3362b
Merge remote-tracking branch 'origin/main' into matt/config-viper
BedrockSquirrel Nov 5, 2024
fd0a45e
review: add comment
BedrockSquirrel Nov 5, 2024
a046221
Merge remote-tracking branch 'origin/main' into matt/config-viper
BedrockSquirrel Nov 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
tidying up
  • Loading branch information
BedrockSquirrel committed Oct 26, 2024
commit 07623b1469e7443c3bffbbe0fd558f7b4149cccd
46 changes: 46 additions & 0 deletions go/config/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
All go processes in the TEN ecosystem will use the config mechanism here to load their configuration.

We have a TenConfig hierarchical structure that contains typed configuration fields used across all our systems.

They are defined in a single place, in the Config structs declared in `config.go`. These declarations include their type,
the mapstructure annotation that gives their yaml field name and any comments about their usage.

For example, `network.chainId` is a field that might be used by multiple processes, but `host.rpc.httpPort` is only used by the host processes.

This library provides a loading mechanism powered by viper, it reads 0-base-config.yaml to initialise the config and then
overrides fields with any other yaml files provided and finally overrides with environment variables.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this ok for production? ENV variable overrides should be disabled in enclave IMO? Or at least disabled for anything baked into the image


Environment variable keys are the same as the yaml keys but are uppercased and have dots replaced with underscores.
For example, `network.chainId` would be `NETWORK_CHAINID`, and `host.rpc.httpPort` would be `HOST_RPC_HTTPPORT`.

## Defining a new field
When defining a new field in the Config struct, there are a few things to keep in mind:

- Environment variable values are only applied when the field is defined in at least one of the yaml files. For this reason,
we want to make sure that all fields are defined in the base config file even if just with trivial or empty values.

- The ego enclave restricts the environment variables that can be accessed by the enclave process. This means that the enclave
process will not be able to access environment variables that are not whitelisted. This is a security feature of the ego enclave.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice


The enclave.json file used to produce the signed ego artifact allows environment variables to be specified, either with a hardcoded value
which we will use for fixed constants that are not allowed to change or with a 'fromHost' flag which will allow the enclave to access the
environment variable from the host process. This is useful for configuration values that are allowed to change between deployments.

So any configuration value that is expected to be set by an environment variable should be whitelisted in the enclave.json file.

The upshot of this is that defining a new field requires adding it in 2 or 3 places.
The nested Config struct declared from config.go, the 0-base-config.yaml file and, if field is used by the enclave, the enclave.json file.

## Loading the config

The loading method allows an ordered list of yaml files to be provided, values in later files will override values in earlier files.

In practice for production deployments, we expect that the 0-base-config.yaml will be the only file that is always provided, and the rest will be optional.

As a convention, we're using a prefixes for the files when lists are provided:
`0-base-config.yaml` - the base configuration that is always provided
`1-env-<env-name>.yaml` - environment-specific configuration
`2-node-<node-name>.yaml` - node-specific configuration (e.g. sequencer, validator in integration tests)
`3-network.yaml` - network-specific configuration (e.g. chainId, networkId)

The loader is not performing any verification on these conventions though, callers can provide files with any names and they are applied in the order provided.
2 changes: 1 addition & 1 deletion go/config2/config.go → go/config/config.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package config2
package config

import (
"fmt"
Expand Down
2 changes: 1 addition & 1 deletion go/config2/decoding.go → go/config/decoding.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package config2
package config

import (
"fmt"
Expand Down
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion go/config2/enclave.go → go/config/enclave.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package config2
package config

import "time"

Expand Down
2 changes: 1 addition & 1 deletion go/config2/host.go → go/config/host.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package config2
package config

import "time"

Expand Down
2 changes: 1 addition & 1 deletion go/config2/loader.go → go/config/loader.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package config2
package config

import (
"embed"
Expand Down
2 changes: 1 addition & 1 deletion go/config2/network.go → go/config/network.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package config2
package config

import (
"math/big"
Expand Down
2 changes: 1 addition & 1 deletion go/config2/node.go → go/config/node.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package config2
package config

import (
gethcommon "github.com/ethereum/go-ethereum/common"
Expand Down
Empty file removed go/config2/default/1-env-dev.yaml
Empty file.
Empty file.
Empty file removed go/config2/default/1-env-uat.yaml
Empty file.
11 changes: 0 additions & 11 deletions go/config2/main/example-config.yml

This file was deleted.

4 changes: 0 additions & 4 deletions go/config2/main/example-override.yml

This file was deleted.

7 changes: 0 additions & 7 deletions go/config2/main/main.go

This file was deleted.

4 changes: 2 additions & 2 deletions go/enclave/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (

gethcommon "github.com/ethereum/go-ethereum/common"
"github.com/ten-protocol/go-ten/go/common"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
)

// For now, this is the bridge between TenConfig and the config used internally by the enclave service.
Expand Down Expand Up @@ -74,7 +74,7 @@ type EnclaveConfig struct {
RPCTimeout time.Duration
}

func EnclaveConfigFromTenConfig(tenCfg *config2.TenConfig) *EnclaveConfig {
func EnclaveConfigFromTenConfig(tenCfg *config.TenConfig) *EnclaveConfig {
return &EnclaveConfig{
HostID: tenCfg.Node.ID,
HostAddress: tenCfg.Node.HostAddress,
Expand Down
4 changes: 2 additions & 2 deletions go/enclave/main/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ import (
"os"

"github.com/ten-protocol/go-ten/go/common/container"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
enclaveconfig "github.com/ten-protocol/go-ten/go/enclave/config"
enclavecontainer "github.com/ten-protocol/go-ten/go/enclave/container"
)

// Runs an Obscuro enclave as a standalone process.
func main() {
tenCfg, err := config2.LoadTenConfigForEnv("local")
tenCfg, err := config.LoadTenConfigForEnv("local")
if err != nil {
fmt.Println("Error loading ten config:", err)
os.Exit(1)
Expand Down
4 changes: 2 additions & 2 deletions go/host/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (

gethcommon "github.com/ethereum/go-ethereum/common"
"github.com/ten-protocol/go-ten/go/common"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
)

// For now, this is the bridge between TenConfig and the config used internally by the host service.
Expand Down Expand Up @@ -110,7 +110,7 @@ type HostConfig struct {
IsInboundP2PDisabled bool
}

func HostConfigFromTenConfig(tenCfg *config2.TenConfig) *HostConfig {
func HostConfigFromTenConfig(tenCfg *config.TenConfig) *HostConfig {
return &HostConfig{
ID: tenCfg.Node.ID,
PrivateKeyString: tenCfg.Node.PrivateKeyString,
Expand Down
4 changes: 2 additions & 2 deletions go/host/main/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ import (
"os"

"github.com/ten-protocol/go-ten/go/common/container"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
hostconfig "github.com/ten-protocol/go-ten/go/host/config"
hostcontainer "github.com/ten-protocol/go-ten/go/host/container"
)

// Runs an Obscuro host as a standalone process.
func main() {
tenCfg, err := config2.LoadTenConfigForEnv("local")
tenCfg, err := config.LoadTenConfigForEnv("local")
if err != nil {
fmt.Println("Error loading ten config:", err)
os.Exit(1)
Expand Down
6 changes: 3 additions & 3 deletions go/node/cmd/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (

gethcommon "github.com/ethereum/go-ethereum/common"
"github.com/ten-protocol/go-ten/go/common"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
)

var (
Expand Down Expand Up @@ -151,14 +151,14 @@ func validateNodeAction(action string) bool {
return false
}

func NodeCLIConfigToTenConfig(cliCfg *NodeConfigCLI) *config2.TenConfig {
func NodeCLIConfigToTenConfig(cliCfg *NodeConfigCLI) *config.TenConfig {
nodeType, err := common.ToNodeType(cliCfg.nodeType)
if err != nil {
fmt.Printf("Error converting node type: %v\n", err)
os.Exit(1)
}

tenCfg, err := config2.DefaultTenConfig()
tenCfg, err := config.DefaultTenConfig()
if err != nil {
fmt.Printf("Error loading default Ten config: %v\n", err)
os.Exit(1)
Expand Down
6 changes: 3 additions & 3 deletions go/node/docker_node.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,23 @@ import (
"strings"

"github.com/ethereum/go-ethereum/common"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"

"github.com/ten-protocol/go-ten/go/common/docker"
)

var _enclaveDataDir = "/enclavedata" // this is how the directory is references within the enclave container

type DockerNode struct {
cfg *config2.TenConfig
cfg *config.TenConfig
hostImage string
enclaveImage string
edgelessDBImage string
enclaveDebugMode bool
pccsAddr string // optional specified PCCS address
}

func NewDockerNode(cfg *config2.TenConfig, hostImage, enclaveImage, edgelessDBImage string, enclaveDebug bool, pccsAddr string) *DockerNode {
func NewDockerNode(cfg *config.TenConfig, hostImage, enclaveImage, edgelessDBImage string, enclaveDebug bool, pccsAddr string) *DockerNode {
return &DockerNode{
cfg: cfg,
hostImage: hostImage,
Expand Down
4 changes: 2 additions & 2 deletions go/node/network_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"os"
"path"

"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
)

// This is the location where the metadata will be stored
Expand All @@ -19,7 +19,7 @@ type NetworkConfig struct {
L1StartHash string // L1 block hash from which to process for L2 data (mgmt contract deploy block)
}

func WriteNetworkConfigToDisk(cfg *config2.TenConfig) error {
func WriteNetworkConfigToDisk(cfg *config.TenConfig) error {
n := NetworkConfig{
ManagementContractAddress: cfg.Network.L1.L1Contracts.ManagementContract.Hex(),
MessageBusAddress: cfg.Network.L1.L1Contracts.MessageBusContract.Hex(),
Expand Down
6 changes: 3 additions & 3 deletions integration/noderunner/in_memory_node.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (

"github.com/ethereum/go-ethereum/common"
"github.com/ten-protocol/go-ten/go/common/log"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
enclaveconfig "github.com/ten-protocol/go-ten/go/enclave/config"
hostconfig "github.com/ten-protocol/go-ten/go/host/config"
"github.com/ten-protocol/go-ten/go/node"
Expand All @@ -17,12 +17,12 @@ import (
)

type InMemNode struct {
tenCfg *config2.TenConfig
tenCfg *config.TenConfig
enclave *enclavecontainer.EnclaveContainer
host *hostcontainer.HostContainer
}

func NewInMemNode(cfg *config2.TenConfig) *InMemNode {
func NewInMemNode(cfg *config.TenConfig) *InMemNode {
return &InMemNode{
tenCfg: cfg,
}
Expand Down
4 changes: 2 additions & 2 deletions integration/noderunner/noderunner_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/ten-protocol/go-ten/go/common/profiler"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
"github.com/ten-protocol/go-ten/go/node"
"github.com/ten-protocol/go-ten/go/rpc"
"github.com/ten-protocol/go-ten/integration"
Expand Down Expand Up @@ -111,7 +111,7 @@ func TestCanStartStandaloneTenHostAndEnclave(t *testing.T) {
}

func createInMemoryNode() node.Node {
tenCfg, err := config2.DefaultTenConfig()
tenCfg, err := config.DefaultTenConfig()
if err != nil {
panic(err)
}
Expand Down
5 changes: 3 additions & 2 deletions integration/simulation/devnetwork/node.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ import (
"net/http"
"time"

"github.com/ten-protocol/go-ten/go/enclave/config"
hostconfig "github.com/ten-protocol/go-ten/go/host/config"
"github.com/ten-protocol/go-ten/lib/gethfork/node"

gethcommon "github.com/ethereum/go-ethereum/common"
Expand All @@ -21,7 +23,6 @@ import (
"github.com/ten-protocol/go-ten/go/common"
"github.com/ten-protocol/go-ten/go/common/log"
"github.com/ten-protocol/go-ten/go/common/metrics"
"github.com/ten-protocol/go-ten/go/config"
enclavecontainer "github.com/ten-protocol/go-ten/go/enclave/container"
"github.com/ten-protocol/go-ten/go/ethadapter"
hostcontainer "github.com/ten-protocol/go-ten/go/host/container"
Expand Down Expand Up @@ -126,7 +127,7 @@ func (n *InMemNodeOperator) createHostContainer() *hostcontainer.HostContainer {
p2pAddr := fmt.Sprintf("%s:%d", network.Localhost, p2pPort)
seqP2PAddr := fmt.Sprintf("%s:%d", network.Localhost, n.config.PortStart+integration.DefaultHostP2pOffset)

hostConfig := &config.HostConfig{
hostConfig := &hostconfig.HostConfig{
ID: n.l1Wallet.Address(),
IsGenesis: n.nodeType == common.Sequencer,
NodeType: n.nodeType,
Expand Down
4 changes: 2 additions & 2 deletions integration/simulation/network/socket.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"time"

"github.com/ten-protocol/go-ten/go/common"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
"github.com/ten-protocol/go-ten/go/host/l1"
"github.com/ten-protocol/go-ten/integration/noderunner"

Expand Down Expand Up @@ -96,7 +96,7 @@ func (n *networkOfSocketNodes) Create(simParams *params.SimParams, _ *stats.Stat
}
hostP2PAddress := fmt.Sprintf("127.0.0.1:%d", simParams.StartPort+integration.DefaultHostP2pOffset+i)

tenCfg, err := config2.LoadTenConfigForEnv("sim")
tenCfg, err := config.LoadTenConfigForEnv("sim")
if err != nil {
return nil, fmt.Errorf("unable to load TEN config: %w", err)
}
Expand Down
6 changes: 3 additions & 3 deletions testnet/launcher/docker.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
"github.com/sanity-io/litter"
common2 "github.com/ten-protocol/go-ten/go/common"
"github.com/ten-protocol/go-ten/go/common/retry"
"github.com/ten-protocol/go-ten/go/config2"
"github.com/ten-protocol/go-ten/go/config"
"github.com/ten-protocol/go-ten/go/node"
"github.com/ten-protocol/go-ten/go/obsclient"
"github.com/ten-protocol/go-ten/go/rpc"
Expand Down Expand Up @@ -48,7 +48,7 @@ func (t *Testnet) Start() error {
edgelessDBImage = "ghcr.io/edgelesssys/edgelessdb-sgx-1gb:v0.3.2"
}

sequencerCfg, err := config2.LoadTenConfigForEnv("local", "default/2-node-local-sequencer.yaml")
sequencerCfg, err := config.LoadTenConfigForEnv("local", "default/2-node-local-sequencer.yaml")
if err != nil {
return fmt.Errorf("unable to load sequencer config - %w", err)
}
Expand Down Expand Up @@ -76,7 +76,7 @@ func (t *Testnet) Start() error {
return fmt.Errorf("sequencer TEN node not healthy - %w", err)
}

validatorNodeCfg, err := config2.LoadTenConfigForEnv("local", "default/2-node-local-validator.yaml")
validatorNodeCfg, err := config.LoadTenConfigForEnv("local", "default/2-node-local-validator.yaml")
if err != nil {
return fmt.Errorf("unable to load validator config - %w", err)
}
Expand Down