Skip to content

Commit

Permalink
feat: Add multi node (validator) testnet (#4377)
Browse files Browse the repository at this point in the history
* add cmd multi-node

* read config

* minor

* init node

* done

* remove try

* updates

* updates

* make format

* updates

* minor

* rename and add info node

* add a reset command

* changelog

* lint

* use config.yml with more validators

* updates docs

* show log

* rename package model to bubblemodel

* Update ignite/cmd/testnet_multi_node.go

Co-authored-by: Julien Robert <[email protected]>

* nits

* Update ignite/cmd/bubblemodel/testnet_multi_node.go

Co-authored-by: Danilo Pantani <[email protected]>

* Update ignite/cmd/bubblemodel/testnet_multi_node.go

Co-authored-by: Danilo Pantani <[email protected]>

* Update ignite/cmd/bubblemodel/testnet_multi_node.go

Co-authored-by: Danilo Pantani <[email protected]>

* Update ignite/cmd/testnet_multi_node.go

Co-authored-by: Danilo Pantani <[email protected]>

* updates

* use lipgloss for View

* status bar

* nits

* Update ignite/cmd/bubblemodel/testnet_multi_node.go

Co-authored-by: Danilo Pantani <[email protected]>

* Update ignite/cmd/bubblemodel/testnet_multi_node.go

Co-authored-by: Danilo Pantani <[email protected]>

* remove ctx

* add comment

* use ports in ignite/pkg/availableport/availableport.go

* update errgroup

* Update changelog.md

Co-authored-by: Danilo Pantani <[email protected]>

* updates with v0.52

---------

Co-authored-by: Julien Robert <[email protected]>
Co-authored-by: Danilo Pantani <[email protected]>
  • Loading branch information
3 people authored Nov 8, 2024
1 parent 5523a52 commit b7e3ab3
Show file tree
Hide file tree
Showing 22 changed files with 1,215 additions and 8 deletions.
1 change: 1 addition & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
- [#4131](https://github.com/ignite/cli/pull/4131) Support `bytes` as data type in the `scaffold` commands
- [#4300](https://github.com/ignite/cli/pull/4300) Only panics the module in the most top function level
- [#4327](https://github.com/ignite/cli/pull/4327) Use the TxConfig from simState instead create a new one
- [#4377](https://github.com/ignite/cli/pull/4377) Add multi node (validator) testnet.
- [#4326](https://github.com/ignite/cli/pull/4326) Add `buf.build` version to `ignite version` command
- [#4362](https://github.com/ignite/cli/pull/4362) Scaffold `Makefile`
- [#4289](https://github.com/ignite/cli/pull/4289) Cosmos SDK v0.52 support
Expand Down
56 changes: 55 additions & 1 deletion docs/docs/03-CLI-Commands/01-cli-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -3666,7 +3666,7 @@ Start a testnet local

**Synopsis**

The commands in this namespace allow you to start your local testnet for development purposes. Currently there is only one feature to create a testnet from any state network (including mainnet).
The commands in this namespace allow you to start your local testnet for development purposes.


The "in-place" command is used to create and start a testnet from current local net state(including mainnet).
Expand All @@ -3675,9 +3675,12 @@ We can create a testnet from the local network state and mint additional coins f

During development, in-place allows you to quickly reboot the chain from a multi-node network state to a node you have full control over.

The "multi-node" initialization and start command is used to set up and launch a multi-node network, allowing you to enable, disable, and providing full interaction capabilities with the chain. The stake amount for each validator is defined in the config.yml file.

**SEE ALSO**

* [ignite testnet in-place](#ignite-testnet-in-place) - Create and start a testnet from current local net state
* [ignite testnet multi-node](#ignite-testnet-multi-node) - Initialize and provide multi-node on/off functionality


## ignite testnet in-place
Expand Down Expand Up @@ -3725,6 +3728,57 @@ ignite chain debug [flags]
-c, --config string path to Ignite config file (default: ./config.yml)
```

## ignite testnet multi-node

Initialize and start multiple nodes

**Synopsis**

The "multi-node" command allows developers to easily set up, initialize, and manage multiple nodes for a testnet environment. This command provides full flexibility in enabling or disabling each node as desired, making it a powerful tool for simulating a multi-node blockchain network during development.

By using the config.yml file, you can define validators with custom bonded amounts, giving you control over how each node participates in the network:

```
validators:
- name: alice
bonded: 100000000stake
- name: validator1
bonded: 100000000stake
- name: validator2
bonded: 200000000stake
- name: validator3
bonded: 300000000stake
```

Each validator's bonded stake can be adjusted according to your testing needs, providing a realistic environment to simulate various scenarios.

The multi-node command not only initializes these nodes but also gives you control over starting, stopping individual nodes. This level of control ensures you can test and iterate rapidly without needing to reinitialize the entire network each time a change is made. This makes it ideal for experimenting with validator behavior, network dynamics, and the impact of various configurations.

All initialized nodes will be stored under the `.ignite/local-chains/<appd>/testnet/` directory, which allows easy access and management.


Usage

```
ignite testnet multi-node [flags]
```

**Options**

```
-r, --reset-once reset the app state once on init
--node-dir-prefix dir prefix for node (default "validator")
-h, --help help for debug
-p, --path string path of the app (default ".")
```

**Options inherited from parent commands**

```
-c, --config string path to Ignite config file (default: ./config.yml)
```

**SEE ALSO**

* [ignite](#ignite) - Ignite CLI offers everything you need to scaffold, test, build, start testnet and launch your blockchain
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ import (
tea "github.com/charmbracelet/bubbletea"
"github.com/stretchr/testify/require"

cmdmodel "github.com/ignite/cli/v29/ignite/cmd/model"
"github.com/ignite/cli/v29/ignite/cmd/model/testdata"
cmdmodel "github.com/ignite/cli/v29/ignite/cmd/bubblemodel"
"github.com/ignite/cli/v29/ignite/cmd/bubblemodel/testdata"
"github.com/ignite/cli/v29/ignite/pkg/cliui/colors"
"github.com/ignite/cli/v29/ignite/pkg/cliui/icons"
cliuimodel "github.com/ignite/cli/v29/ignite/pkg/cliui/model"
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import (
tea "github.com/charmbracelet/bubbletea"
"github.com/stretchr/testify/require"

cmdmodel "github.com/ignite/cli/v29/ignite/cmd/model"
"github.com/ignite/cli/v29/ignite/cmd/model/testdata"
cmdmodel "github.com/ignite/cli/v29/ignite/cmd/bubblemodel"
"github.com/ignite/cli/v29/ignite/cmd/bubblemodel/testdata"
"github.com/ignite/cli/v29/ignite/pkg/cliui/colors"
"github.com/ignite/cli/v29/ignite/pkg/cliui/icons"
cliuimodel "github.com/ignite/cli/v29/ignite/pkg/cliui/model"
Expand Down
File renamed without changes.
269 changes: 269 additions & 0 deletions ignite/cmd/bubblemodel/testnet_multi_node.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,269 @@
package cmdmodel

import (
"bufio"
"context"
"fmt"
"os/exec"
"path/filepath"
"strconv"
"syscall"

tea "github.com/charmbracelet/bubbletea"
"github.com/charmbracelet/lipgloss"
"golang.org/x/sync/errgroup"

"github.com/ignite/cli/v29/ignite/services/chain"
)

// NodeStatus is an integer data type that represents the status of a node.
type NodeStatus int

const (
// Stopped indicates that the node is currently stopped.
Stopped NodeStatus = iota

// Running indicates that the node is currently running.
Running
)

// Make sure MultiNode implements tea.Model interface.
var _ tea.Model = MultiNode{}

// MultiNode represents a set of nodes, managing state and information related to them.
type MultiNode struct {
ctx context.Context
appd string
args chain.MultiNodeArgs

nodeStatuses []NodeStatus
pids []int // Store the PIDs of the running processes
numNodes int // Number of nodes
logs [][]string // Store logs for each node
}

// ToggleNodeMsg is a structure used to pass messages
// to enable or disable a node based on the node index.
type ToggleNodeMsg struct {
nodeIdx int
}

// UpdateStatusMsg defines a message that updates the status of a node by index.
type UpdateStatusMsg struct {
nodeIdx int
status NodeStatus
}

// UpdateLogsMsg is for continuously updating the chain logs in the View.
type UpdateLogsMsg struct{}

// UpdateDeemon returns a command that sends an UpdateLogsMsg.
// This command is intended to continuously refresh the logs displayed in the user interface.
func UpdateDeemon() tea.Cmd {
return func() tea.Msg {
return UpdateLogsMsg{}
}
}

// NewModel initializes the model.
func NewModel(ctx context.Context, chainname string, args chain.MultiNodeArgs) (MultiNode, error) {
numNodes, err := strconv.Atoi(args.NumValidator)
if err != nil {
return MultiNode{}, err
}
return MultiNode{
ctx: ctx,
appd: chainname + "d",
args: args,
nodeStatuses: make([]NodeStatus, numNodes), // initial states of nodes
pids: make([]int, numNodes),
numNodes: numNodes,
logs: make([][]string, numNodes), // Initialize logs for each node
}, nil
}

// Init implements the Init method of the tea.Model interface.
func (m MultiNode) Init() tea.Cmd {
return nil
}

// ToggleNode toggles the state of a node.
func ToggleNode(nodeIdx int) tea.Cmd {
return func() tea.Msg {
return ToggleNodeMsg{nodeIdx: nodeIdx}
}
}

// RunNode runs or stops the node based on its status.
func RunNode(nodeIdx int, start bool, m MultiNode) tea.Cmd {
var (
pid = &m.pids[nodeIdx]
args = m.args
appd = m.appd
)

return func() tea.Msg {
if start {
nodeHome := filepath.Join(args.OutputDir, args.NodeDirPrefix+strconv.Itoa(nodeIdx))
// Create the command to run in the background as a daemon
cmd := exec.Command(appd, "start", "--home", nodeHome)

// Start the process as a daemon
cmd.SysProcAttr = &syscall.SysProcAttr{
Setpgid: true, // Ensure it runs in a new process group
}

stdout, err := cmd.StdoutPipe() // Get stdout for logging
if err != nil {
fmt.Printf("Failed to start node %d: %v\n", nodeIdx+1, err)
return UpdateStatusMsg{nodeIdx: nodeIdx, status: Stopped}
}

err = cmd.Start() // Start the node in the background
if err != nil {
fmt.Printf("Failed to start node %d: %v\n", nodeIdx+1, err)
return UpdateStatusMsg{nodeIdx: nodeIdx, status: Stopped}
}

*pid = cmd.Process.Pid // Store the PID

// Create an errgroup with context
g, gCtx := errgroup.WithContext(m.ctx)
g.Go(func() error {
scanner := bufio.NewScanner(stdout)
for scanner.Scan() {
select {
case <-gCtx.Done():
// Handle context cancellation
return gCtx.Err()
default:

line := scanner.Text()
// Add log line to the respective node's log slice
m.logs[nodeIdx] = append(m.logs[nodeIdx], line)
// Keep only the last 5 lines
if len(m.logs[nodeIdx]) > 5 {
m.logs[nodeIdx] = m.logs[nodeIdx][len(m.logs[nodeIdx])-5:]
}
}
}
if err := scanner.Err(); err != nil {
return err
}
return nil
})

// Goroutine to handle stopping the node if context is canceled
g.Go(func() error {
<-gCtx.Done() // Wait for context to be canceled

// Stop the daemon process if context is canceled
if *pid != 0 {
err := syscall.Kill(-*pid, syscall.SIGTERM) // Stop the daemon process
if err != nil {
fmt.Printf("Failed to stop node %d: %v\n", nodeIdx+1, err)
} else {
*pid = 0 // Reset PID after stopping
}
}

return gCtx.Err()
})

return UpdateStatusMsg{nodeIdx: nodeIdx, status: Running}
}
// Use kill to stop the node process by PID
if *pid != 0 {
err := syscall.Kill(-*pid, syscall.SIGTERM) // Stop the daemon process
if err != nil {
fmt.Printf("Failed to stop node %d: %v\n", nodeIdx+1, err)
} else {
*pid = 0 // Reset PID after stopping
}
}
return UpdateStatusMsg{nodeIdx: nodeIdx, status: Stopped}
}
}

// StopAllNodes stops all nodes.
func (m *MultiNode) StopAllNodes() {
for i := 0; i < m.numNodes; i++ {
if m.nodeStatuses[i] == Running {
RunNode(i, false, *m)() // Stop node
}
}
}

// Update handles messages and updates the model.
func (m MultiNode) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
switch msg := msg.(type) {
case tea.KeyMsg:
switch msg.String() {
case "q":
m.StopAllNodes() // Stop all nodes before quitting
return m, tea.Quit
default:
// Check for numbers from 1 to numNodes
for i := 0; i < m.numNodes; i++ {
if msg.String() == fmt.Sprintf("%d", i+1) {
return m, ToggleNode(i)
}
}
}

case ToggleNodeMsg:
if m.nodeStatuses[msg.nodeIdx] == Running {
return m, RunNode(msg.nodeIdx, false, m) // Stop node
}
return m, RunNode(msg.nodeIdx, true, m) // Start node

case UpdateStatusMsg:
m.nodeStatuses[msg.nodeIdx] = msg.status
return m, UpdateDeemon()
case UpdateLogsMsg:
return m, UpdateDeemon()
}

return m, nil
}

// View renders the interface.
func (m MultiNode) View() string {
// Define styles for the state
runningStyle := lipgloss.NewStyle().Foreground(lipgloss.Color("2")) // green
stoppedStyle := lipgloss.NewStyle().Foreground(lipgloss.Color("1")) // red
tcpStyle := lipgloss.NewStyle().Foreground(lipgloss.Color("3")) // yellow
grayStyle := lipgloss.NewStyle().Foreground(lipgloss.Color("8")) // gray
purpleStyle := lipgloss.NewStyle().Foreground(lipgloss.Color("5")) // purple
statusBarStyle := lipgloss.NewStyle().Background(lipgloss.Color("0")) // Status bar style
blueStyle := lipgloss.NewStyle().Foreground(lipgloss.Color("45")).Background(lipgloss.Color("0")) //blue

Check failure on line 239 in ignite/cmd/bubblemodel/testnet_multi_node.go

View workflow job for this annotation

GitHub Actions / Lint Go code

commentFormatting: put a space between `//` and comment text (gocritic)

statusBar := blueStyle.Render("Press q to quit | Press 1-4 to ") + statusBarStyle.Render(runningStyle.Render("start")) + blueStyle.Render("/") + statusBarStyle.Render(stoppedStyle.Render("stop")) + blueStyle.Render(" corresponding node")
output := statusBar + "\n\n"

// Add node control section
output += purpleStyle.Render("Node Control:")
for i := 0; i < m.numNodes; i++ {
status := stoppedStyle.Render("[Stopped]")
if m.nodeStatuses[i] == Running {
status = runningStyle.Render("[Running]")
}

tcpAddress := tcpStyle.Render(fmt.Sprintf("tcp://127.0.0.1:%d", m.args.ListPorts[i]))
nodeGray := grayStyle.Render("--node")
nodeNumber := purpleStyle.Render(fmt.Sprintf("%d.", i+1))

output += fmt.Sprintf("\n%s Node %d %s %s %s:\n", nodeNumber, i+1, status, nodeGray, tcpAddress)
output += " [\n"
if m.logs != nil {
for _, line := range m.logs[i] {
output += " " + line + "\n"
}
}

output += " ]\n\n"
}

output += grayStyle.Render("\nPress q to quit.\n")
return output
}
2 changes: 1 addition & 1 deletion ignite/cmd/chain_debug.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
tea "github.com/charmbracelet/bubbletea"
"github.com/spf13/cobra"

cmdmodel "github.com/ignite/cli/v29/ignite/cmd/model"
cmdmodel "github.com/ignite/cli/v29/ignite/cmd/bubblemodel"
chainconfig "github.com/ignite/cli/v29/ignite/config/chain"
"github.com/ignite/cli/v29/ignite/pkg/chaincmd"
"github.com/ignite/cli/v29/ignite/pkg/cliui"
Expand Down
2 changes: 1 addition & 1 deletion ignite/cmd/chain_serve.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
tea "github.com/charmbracelet/bubbletea"
"github.com/spf13/cobra"

cmdmodel "github.com/ignite/cli/v29/ignite/cmd/model"
cmdmodel "github.com/ignite/cli/v29/ignite/cmd/bubblemodel"
"github.com/ignite/cli/v29/ignite/pkg/cliui"
uilog "github.com/ignite/cli/v29/ignite/pkg/cliui/log"
cliuimodel "github.com/ignite/cli/v29/ignite/pkg/cliui/model"
Expand Down
Loading

0 comments on commit b7e3ab3

Please sign in to comment.