Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply feedback to Run a Full Node page #253

Merged
merged 9 commits into from
Aug 6, 2024
Merged

Apply feedback to Run a Full Node page #253

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
2c3327f
wip
zachfc Aug 5, 2024
8b4320c
reframe
zachfc Aug 5, 2024
b87783a
context
zachfc Aug 5, 2024
2ed1c73
update intros
zachfc Aug 5, 2024
a5cef28
intros
zachfc Aug 5, 2024
57b9d46
wip
zachfc Aug 5, 2024
331e70f
feedback
zachfc Aug 6, 2024
6fb6279
fix links
zachfc Aug 6, 2024
f3c51ac
feedback plus typo fix
zachfc Aug 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pages/infrastructure_providers-validators/_meta.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"hardware_requirements": "Hardware Requirements",
"required_node_configs": "Required Node Configs",
"how_to_set_up_full_node": "Set Up a Full Node",
"running_full_node": "Run a Full Node",
"running_full_node": "Optimize Your Full Node",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated title and reframed the page -- now about configuring an existing node for success

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we rename the file name to reduce confusion?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do in the next round of changes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we should update the filename accordingly

Suggested change
"running_full_node": "Optimize Your Full Node",
"optimize_full_node": "Optimize Your Full Node",

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if/when you do this can you add a redirect to next.config.js? some paths are indexed in google/etc and used in other places, we should try to keep the original paths

"running_a_validator": "Running a Validator",
"snapshots": "Snapshots",
"full_node_streaming": "Full Node Streaming",
Expand Down
2 changes: 1 addition & 1 deletion pages/infrastructure_providers-validators/how_to_set_up_full_node.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Set Up a Full Node
Installing and running a full node allows your system to participate in a dYdX Chain network.
Installing and running a full node allows you to read orderbook and on-chain data from a network, as well as place, confirm and cancel orders directly on that network.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add reasons a person would want a full node
Improved from "participate in a network"

Please verify that "read orderbook and on-chain data" is accurate

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm


> Code snippets on this page use example values. Replace them with your own. See the [Network Configuration](../infrastructure_providers-network/network_constants.mdx) section of the documentation for network constants and other resources you need to configure a full node.

Expand Down
61 changes: 37 additions & 24 deletions pages/infrastructure_providers-validators/running_full_node.md
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update the filename to optimize_full_node.md to match the comment above

Original file line number Diff line number Diff line change
@@ -1,29 +1,32 @@
# Run a Full Node
Running a full node allows your system to participate in a dYdX Chain network. The recommended settings and best practices on this page help ensure your node stays healthy and up to date with the network.
# Optimize Your Full Node
Optimizing your full node helps keep it online, up-to-date, and operating quickly. Faster nodes have an advantage over slower nodes because they tend to receive new data first and they minimize the time between placing and resolving orders.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adds reasons a person would want to optimize

> Code snippets on this page use example values. Replace them with your own. See the [Network Configuration](../infrastructure_providers-network/network_constants.mdx) section of the documentation for network constants and other resources you need to configure a full node.

## Prerequisites
You need a running, non-validating full node that is connected to network.
You need a running, non-validating full node that is connected to a network.

- If you created a system service for your node by following the instructions on the previous page, [Set Up a Full Node](../infrastructure_providers-validators/how_to_set_up_full_node.md), start your node with the following command:
```bash
stystemctl start dydxprotocold
```
- To start your node with Cosmovisor, include the flag `--non-validating-full-node=true`. Cosmovisor may prompt you to configure additional variables in your environment or include them in your command.
- To start your node with Cosmovisor, you must include the flag `--non-validating-full-node=true`. The flag disables the functionality intended for validator nodes and enables additional logic for reading data. Cosmovisor may prompt you to configure additional variables in your environment or include them in your command.
```bash
cosmovisor run start --non-validating-full-node=true
```
> The flag `--non-validating-full-node` is required. It disables the functionality intended for validator nodes and enables additional logic for reading data.
- To start your node directly with the `dydxprotocold` binary, include the flag `--non-validating-full-node=true`. `dydxprotocold` may prompt you to configure additional variables in your environment or include them in your command.
- To start your node directly with the `dydxprotocold` binary, you must include the flag `--non-validating-full-node=true`. The flag disables the functionality intended for validator nodes and enables additional logic for reading data. `dydxprotocold` may prompt you to configure additional variables in your environment or include them in your command.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is basically duplicate info across Cosmovisor route and the direct binary route.

can we dedup this?

```bash
dydxprotocold run start --non-validating-full-node=true
```
> The flag `--non-validating-full-node` is required. It disables the functionality intended for validator nodes and enables additional logic for reading data.

## Connect to Healthy Peers
To keep your full node up to date, connect to healthy peers with the latest state of the network. Request a list of healthy peers for your deployment from a [Live Peer Node](../infrastructure_providers-network/resources.mdx#live-peer-node-providers) provider.
## Customize Node Settings
The following recommended settings and best practices help ensure that your node stays healthy and up to date with the network.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: noticed that we use hyphenated version above, so I think we should be consistent? no preference on which way, but just want consistency

Suggested change
The following recommended settings and best practices help ensure that your node stays healthy and up to date with the network.
The following recommended settings and best practices help ensure that your node stays healthy and up-to-date with the network.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good callout, I actually messed up a different one. It should be hyphened when its a compound modifier before a noun (up-to-date addrbook.json) but not hyphenated when used as part of a sentence (up to date with the network), so planning to leave this one as is and fix a spot where I hyphenated when I shouldn't have

### Save a List of Persistent Peers
Save a list of healthy peers to more quickly and reliably read the latest data from the network.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please verify this is accurate

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my suggestion needs better grammar and word-smithing, but the idea is to explain why persistent peering is useful

Suggested change
Save a list of healthy peers to more quickly and reliably read the latest data from the network.
Save a list of healthy, stable nodes as persistent peers to more quickly and reliably reconnect to the network. In case your node disconnects and restarts, your node will try to connect directly with peers specified by persistent peers configuration. This allows your node to skip the step connecting to the seed node and finding new peers to connect to.

Request a list of healthy peers for your deployment from a [Live Peer Node](../infrastructure_providers-network/resources.mdx#live-peer-node-providers) provider.

From the list of healthy peers that you retrieve from peer node provider, choose any 5 for your node to query for the latest state. Add a comma-separated list of those peer addresses to the `persistent_peers` field in your `config.toml`, like in the following example:

Expand All @@ -33,28 +36,38 @@ From the list of healthy peers that you retrieve from peer node provider, choose
persistent_peers=83c299de2052db247f08422b6592e1383dd7a104@136.243.36.60:23856,[email protected]:26656,[email protected]:26656,[email protected]:23856,[email protected]:26656
```

## Save an Address Book
Download the latest `addrbook.json` file, which stores configuration details that help your node efficiently connect to peers in its network. Download an up-to-date address book file for your deployment from an [Address Book](../infrastructure_providers-network/resources.mdx#address-book-providers) provider.
### Save an Address Book File
Save an address book file of trusted nodes operated by a provider to more quickly and reliably connect to the network.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Save an Address Book File
### Replace an Address Book File


Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please verify this is accurate

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

take a look at the following docs to get a better understanding

Suggested change
Save an address book file of trusted nodes operated by a provider to more quickly and reliably connect to the network.
Alternatively, replace your node's local address book with the latest address book provided by trusted providers to reconnect to the network. This allows your node to have the latest peer address info when trying to re-connect with peers and the network

Save the `addrbook.json` file in your `/.dydxprotocol/config` directory.

## Restore Your Node
> If you followed the procedure on the previous page, [Set Up a Full Node](../infrastructure_providers-validators/how_to_set_up_full_node.md), you already have a snapshot installed.
Download an up-to-date `addrbook.json` file for your deployment from an [Address Book](../infrastructure_providers-network/resources.mdx#address-book-providers) provider.

Your full node needs a backup plan to replay the history of the network in case it falls out of sync. You can prepare a backup for your node in one of two ways.

### Snapshot
You can use a **snapshot** stored on the system that your node runs on. A snapshot contains a compressed copy of the application state at the time the snapshot was taken. If your node falls out of sync, a snapshot allows the node to recover to that saved state before replaying the rest of the history of the network. This speeds up the syncing process because you avoid replaying the entire history of the network, instead starting from your stored application state snapshot. To use a snapshot to back up your full node, install a snapshot for your deployment from a [Snapshot Service](../infrastructure_providers-network/resources.mdx#snapshot-service).

### State Sync
You can use **state sync**, a set of configuration settings that allow your node to retrieve a snapshot from the network. If your node falls out of sync, it queries a state sync node for a verified, recent snapshot of the application state. This speeds up the syncing process because you avoid replaying the entire history of the network, instead starting from the network's most recent application state snapshot. To use state sync to back up your full node, follow the instructions for your deployment from a [State Sync](../infrastructure_providers-network/resources.mdx#state-sync-service) service.
Save the `addrbook.json` file in your `/.dydxprotocol/config` directory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Save the `addrbook.json` file in your `/.dydxprotocol/config` directory.
Replace the `addrbook.json` file in your `/.dydxprotocol/config` directory with the newly downloaded `addrbook.json` file in the above step.

## Optimize Pruning Settings
### Configure a Pruning Strategy
In general, dYdX recommends the following pruning setting, configured in your `app.toml` file:

```bash
# app.toml
pruning = "everything" # 2 latest states will be kept; pruning at 10 block intervals
```

However, if you want to use your node to query historical data, configure a custom pruning strategy to retain more states. Retaining more states increases storage requirements.
However, if you want to use your node to query historical data, configure a custom pruning strategy to retain more states. Retaining more states increases storage requirements.

## Prepare to Restore Your Node
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reframed this section from "Restore your Node" -- the page is about optimizing your node to keep a healthy connection, so this section is about configuring your system so that you can restore right away/ automatically if you need to. There should be procedures on another page and we should link to them from here when they're done

Prepare to restore your node quickly in case it falls out of sync.

Your full node can fall out of sync with the rest of the network for a variety of reasons, including a bad software upgrade, unexpected node crashes, or human operational error. To re-sync with the network, your full node must replay the history of the network.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Your full node can fall out of sync with the rest of the network for a variety of reasons, including a bad software upgrade, unexpected node crashes, or human operational error. To re-sync with the network, your full node must replay the history of the network.
Your full node can fall out of sync with the rest of the network for a variety of reasons, including a bad software upgrade, unexpected node crashes, or human operational error. To re-sync with the network, your full node needs to replay the history of the network, which can be a lengthy process.


You can speed up the re-syncing process significantly by providing your node with a snapshot. A snapshot contains a compressed copy of the application state at the time the snapshot was taken. If your node falls out of sync, a snapshot allows it to recover to that saved state before replaying the rest of the history of the network, saving time.

### Save a Snapshot on Your System
You can use a snapshot saved on the system on which your node runs to restore your node.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From our convo, i think this approach is about proactively downloading snapshots from snapshot providers and saving them on your system to reduce the time for your node to use the latest snapshot. I think this text can use clarification to get this point across.


To use a snapshot to back up your full node, install a snapshot for your deployment from a [Snapshot Service](../infrastructure_providers-network/resources.mdx#snapshot-service).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be made more clear. Currently, the word "back up" and "install" are confusing.

The main point to get across here is:

  1. you should download snapshots from snapshot service providers
  2. then use the downloaded snapshot to restore/overwrite your node state (https://docs.dydx.exchange/infrastructure_providers-validators/how_to_set_up_full_node#step-8-use-a-snapshot-as-your-nodes-initial-state)


> If you followed the procedure on the previous page, [Set Up a Full Node](../infrastructure_providers-validators/how_to_set_up_full_node.md), you already have a snapshot installed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we remove this?

the snapshot outlined in the "set up" doc is for the initial start up. IMO, this is about acquiring a new snapshot (ie snapshot at a higher height)


### Configure Your Node's State Sync
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is more automated than manually downloading snapshots, so would it make sense to recommend this over the manual snapshot route? if so, can we move this section to be above the other option?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes a lot of sense. Also, from our discussion, it seems like practically you'd want to do a little more than save one snapshot (you'd want your own process to update the saved snapshot). I'll reorder like you suggested, and longer term I wonder if it makes sense to take this out. Could seem like were recommending saving one snapshot. I think if we remove the snapshot section it should be in another PR that also addresses the step in Set Up a Node

You can use state sync, a configuration setting that allows your node to retrieve a snapshot from the network, to restore your node.
Copy link
Contributor Author

@zachfc zachfc Aug 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to create a procedure for doing this in another PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

State sync is a process that automates the state syncing process by automatically fetching snapshots.

I think it's worth reading this doc and summarizing it in 1-2 sentences here for a high-level overview of this approach. I found this part to be sufficient:

... state sync, which allows a new node to join a network by fetching a snapshot of the application state at a recent height instead of fetching and replaying all historical blocks. This can reduce the time needed to sync with the network from days to minutes.

..., the good news is that the upcoming Cosmos SDK 0.40 release will include automatic support for state sync, and developers only need to enable it in their applications to make use of it. We’ll explain how in this article, but first some background.


To use state sync to back up your full node, follow the instructions for your deployment from a [State Sync](../infrastructure_providers-network/resources.mdx#state-sync-service) service.
Loading