Skip to content

Commit

Permalink
feat: rename all JSON to YAML (#1650)
Browse files Browse the repository at this point in the history 
## Description:
<!-- Describe this change, how it works, and the motivation behind it.
-->
This PR helps to standardize the examples to YAML from JSON. 

## Is this change user facing?
YES

## References (if applicable):
<!-- Add relevant Github Issues, Discord threads, or other helpful
information. -->
ethpandaops/ethereum-package#332

Co-authored-by: Derek <[email protected]>
Co-authored-by: Gyanendra Mishra <[email protected]>
  • Loading branch information
@barnabasbusa @leeederek @h4ck3rk3y
3 people authored Nov 1, 2023
1 parent f172cd7 commit 219ac7a
Show file tree
Hide file tree
Showing 6 changed files with 92 additions and 87 deletions.
7 changes: 5 additions & 2 deletions docs/docs/best-practices.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,14 @@ Passing [package parameters][package-parameterization] via the CLI can get hairy
```bash
kurtosis run github.com/user/repo '{"some_param":5,"some_other_param":"My value"}'
```
1. **When you have a large number of arguments:** put them in a `.json` file and use `--args-file` to slot them into the `kurtosis run` command. E.g.:
1. **When you have a large number of arguments:** put them in a `.yaml` or `.json` file and use `--args-file` to slot them into the `kurtosis run` command. E.g.:
```bash
kurtosis run github.com/user/repo --args-file my-params.yaml
```
or if you prefer JSON:
```bash
kurtosis run github.com/user/repo --args-file my-params.json
```

Choosing the right wait
-----------------------
Kurtosis has three different types of waits. Described here are the three, with tips on when to use each:
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/cli-reference/run.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,10 @@ run(plan, some_parameter = struct(some_property = "Property value"))

`kurtosis run` has additional flags that can further modify its behaviour:

1. The `--args-file` flag can be used to send in a JSON/YAML file, from a local file through the filepath or from remote using the URL, as an argument to the Kurtosis Package. Note that if you pass in package arguments as CLI arguments and via the flag, the CLI arguments will be the one used.
1. The `--args-file` flag can be used to send in a YAML/JSON file, from a local file through the filepath or from remote using the URL, as an argument to the Kurtosis Package. Note that if you pass in package arguments as CLI arguments and via the flag, the CLI arguments will be the one used.
For example:
```bash
kurtosis run github.com/kurtosis-tech/ethereum-package --args-file "devnet-5.json"
kurtosis run github.com/kurtosis-tech/ethereum-package --args-file "devnet-5.yaml"
```
or
```bash
Expand Down
63 changes: 30 additions & 33 deletions docs/docs/get-started/quickstart.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,37 +48,34 @@ Kurtosis environment definitions, referred to here on as [Kurtosis packages][pac

You're now going to create a file to house the parameters that you will pass in at runtime when your system starts up. Check out the README for the [ethereum-package][ethereum-package] to see the full list of configuration options and flags available for use.

Create a file titled: `network_params.json` in your home directory, populate it with the below contents, and save your file:
```json
{
"participants": [{
"el_client_type": "geth",
"el_client_image": "ethereum/client-go:latest",
"el_client_log_level": "",
"el_extra_params": [],
"cl_client_type": "lighthouse",
"cl_client_image": "sigp/lighthouse:latest",
"cl_client_log_level": "",
"beacon_extra_params": [],
"validator_extra_params": [],
"builder_network_params": null,
"count": 2
}],
"network_params": {
"network_id": "3151908",
"deposit_contract_address": "0x4242424242424242424242424242424242424242",
"seconds_per_slot": 12,
"slots_per_epoch": 32,
"num_validator_keys_per_node": 64,
"preregistered_validator_keys_mnemonic": "giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete",
"deneb_for_epoch": 500
},
"verifications_epoch_limit": 5,
"global_client_log_level": "info",
"mev_type": "none",
// The default value bootstraps a bunch of services like grafana, prometheus, el/cl forkmons, explorers
"additional_services": [],
}
Create a file titled: `network_params.yaml` in your home directory, populate it with the below contents, and save your file:
```yaml
participants:
- el_client_type: geth
el_client_image: ethereum/client-go:latest
el_client_log_level: ''
el_extra_params: []
cl_client_type: lighthouse
cl_client_image: sigp/lighthouse:latest
cl_client_log_level: ''
beacon_extra_params: []
validator_extra_params: []
builder_network_params:
count: 2
network_params:
network_id: '3151908'
deposit_contract_address: '0x4242424242424242424242424242424242424242'
seconds_per_slot: 12
slots_per_epoch: 32
num_validator_keys_per_node: 64
preregistered_validator_keys_mnemonic: 'giant issue aisle success illegal bike spike
question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy
very lucky have athlete'
deneb_for_epoch: 500
verifications_epoch_limit: 5
global_client_log_level: info
additional_services: []

```
The arrays in the `participant` object enables you to define the specific Ethereum client types and respective image tags you want to use in your network, alongside any extra parameters for the validator, beacon, or builder as well as some useful flags for the verbosity of log lines. In this example you'll be using the `latest` image tags for the Geth and Lighthouse clients and have specified `2` nodes to be spun up.

Expand All @@ -90,7 +87,7 @@ Spin up your system!
--------------------
Great! You're now ready to bring up your own network. Simply run:
```console
kurtosis run github.com/kurtosis-tech/ethereum-package --args-file ~/network_params.json --enclave eth-network
kurtosis run github.com/kurtosis-tech/ethereum-package --args-file ~/network_params.yaml --enclave eth-network
```

Kurtosis will then begin to spin up your private Ethereum testnet by interpreting the instructions in the Kurtosis package, validating the plan to ensure there are no conflicts or obvious errors, and then finally executes the plan (read more about multi-phase runs [here][multi-phase-runs-reference]). Kurtosis first spins up an isolated, ephemeral environment on your machine called an [enclave][enclaves-reference] where all the services and [files artifacts][files-artifacts-reference] for your system will reside in. Then, those services will be bootstrapped and required files generated to start up the system.
Expand Down Expand Up @@ -158,7 +155,7 @@ Why Kurtosis packages - from a consumer's perspective
Kurtosis was built to make building distributed systems as easy as building a single server app. Kurtosis aims to achieve this by bridging the environment definition author-consumer divide. Tactically, this means making it dead simple for a consumer (like yourself) to pick up an environment definition, spin it up, and deploy it the way you want, where you want - all without needing to know specialized knowledge about how the system works or how to use Kubernetes or Docker.

Specifically, this guide showed you:
- ***The power of parameterizability***: as a consumer of the environment definition, having both the knowledge and means to configure the system to spin up the way you need it is incredibly valuable - a big reason why Kurtosis packages are meant to be parameterized. In this guide, you created the `network_params.json` file which contained your preferences for how the network should look and passed them in to Kurtosis with relative ease. The author of the package need only define the arguments and flags available for a consumer, and Kurtosis handles the rest once those are passed in at runtime.
- ***The power of parameterizability***: as a consumer of the environment definition, having both the knowledge and means to configure the system to spin up the way you need it is incredibly valuable - a big reason why Kurtosis packages are meant to be parameterized. In this guide, you created the `network_params.yaml` file which contained your preferences for how the network should look and passed them in to Kurtosis with relative ease. The author of the package need only define the arguments and flags available for a consumer, and Kurtosis handles the rest once those are passed in at runtime.
- ***Portable and easy to wield***: a major contributor to the author-consumer divide comes from the knowledge gap between the author and consumer regarding the infrastruture and tools needed to instantiate a system. Understanding how Kubernetes works, what Bash script to use at which step, and working with Docker primitivies are all pain points we believe Kurtosis alleviates. In this guide, you installed Kurtosis and ran a single command to get your system up and running. This same command will work anywhere, over Docker or on Kubernetes, locally or on remote infrastructure. We believe this portability and ease of use are requirements for bridging the author-consumer divide.

There are many other reasons why we believe Kurtosis is the right tool for bridging the author-consumer divide. Check out the [next guide][how-to-set-up-postgres-guide] to experience the workflow for a package author and how Kurtosis improves the developer experience for an environment definition author.
Expand Down
39 changes: 23 additions & 16 deletions docs/docs/guides/how-to-compose-your-own-testnet.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,18 +42,19 @@ To begin, create and `cd` into a directory to hold your files:
```bash
mkdir my-testnet && cd my-testnet
```
Next, create a file called `network_params.json` in that folder with the following contents:
```json
{
"preregistered_validator_keys_mnemonic": "giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete",
"num_validator_keys_per_node": 64,
"network_id": "3151908",
"deposit_contract_address": "0x4242424242424242424242424242424242424242",
"seconds_per_slot": 12,
"genesis_delay": 10,
"capella_fork_epoch": 2,
"deneb_fork_epoch": 500
}
Next, create a file called `network_params.yaml` in that folder with the following contents:
```yaml
preregistered_validator_keys_mnemonic: 'giant issue aisle success illegal bike spike
question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy
very lucky have athlete'
num_validator_keys_per_node: 64
network_id: '3151908'
deposit_contract_address: '0x4242424242424242424242424242424242424242'
seconds_per_slot: 12
genesis_delay: 10
capella_fork_epoch: 2
deneb_fork_epoch: 500

```
The contents above will be used to define the specific parameters with which to start the network with.
Expand All @@ -66,10 +67,13 @@ Awesome. You have just created the beginnings of your first [Kurtosis package](.
### 2. Import dependencies
Now that you have a local project to house your definition and some parameters to start the network with, its time to actually build the network. First, create a Starlark file called `main.star` and add the following three lines:
```python
import yaml
# main.star
geth = import_module("github.com/kurtosis-tech/geth-package/lib/geth.star")
lighthouse = import_module("github.com/kurtosis-tech/lighthouse-package/lib/lighthouse.star")
network_params = json.decode(read_file("./network_params.json"))
with open("./network_params.yaml") as stream:
network_params = yaml.safe_load(stream)
```

In the first two lines, you're using [Locators](../concepts-reference/locators.md) to import in `geth.star` and `lighthouse.star` files from Github, making them available to use in your testnet definition. These files themselves are environment definitions that can be used to bootstrap and start up a Geth execution layer client and a Lighthouse consensus layer client as part of your testnet - which is exactly what you will do next.
Expand All @@ -78,7 +82,7 @@ In the first two lines, you're using [Locators](../concepts-reference/locators.m
Feel free to check out the [`geth.star`](https://github.com/kurtosis-tech/geth-package/blob/main/lib/geth.star) and [`lighthouse.star`](https://github.com/kurtosis-tech/lighthouse-package/blob/main/lib/lighthouse.star) to understand how they work. At a high level, the definition instructs Kurtosis to generate genesis data, set up pre-funded accounts, and then launches the client using the client container images.
:::

Finally, we are converting the local `network_params.json` file into a format that can be used in your environment definition using [`json.decode()`](https://bazel.build/rules/lib/core/json#decode) and [`read_file()`](../starlark-reference/read-file.md).
Finally, we are converting the local `network_params.yaml` file into a format that can be used in your environment definition using [`yaml.safe_load()`](https://pyyaml.org/wiki/PyYAMLDocumentation#Loader). This is then saved into a variable called `network_params` which you will use later on.

### 3. Define how your testnet gets built
Now that you have all the necessary dependencies, you can start writing the function that will instantiate the network. Within your `main.star` file, add the following 3 new lines:
Expand All @@ -97,9 +101,12 @@ With some execution layer genesis data in hand, you will now bootstrap the node!

```python
# main.star
import yaml
geth = import_module("github.com/kurtosis-tech/geth-package/lib/geth.star")
lighthouse = import_module("github.com/kurtosis-tech/lighthouse-package/lib/lighthouse.star")
network_params = json.decode(read_file("./network_params.json"))
with open("./network_params.yaml") as stream:
network_params = yaml.safe_load(stream)
def run(plan):
# Generate genesis, note EL and the CL needs the same timestamp to ensure that timestamp based forking works
Expand Down Expand Up @@ -175,7 +182,7 @@ Otherwise, we've got some neat ideas for what you can do next. If you need a han
* Deploy your node in a Kubernetes cluster for collaborative work and scale it out to multiple nodes! Check out our docs for how to do so [here](https://docs.kurtosis.com/k8s/).
* Simulate MEV workflows by importing the [MEV Package](https://github.com/kurtosis-tech/mev-package) into your testnet definition. The MEV package deploys and configures the Flashbots suite of products to your local Ethereum testnet and includes: MEV-Boost, MEV-Flood, and MEV-relay, and any required dependencies (postgres & redis). Here's a full example of this set up [here](https://github.com/kurtosis-tech/geth-lighthouse-mev-package).
* Connect other infrastructure (oracles, relayers, etc) to the network by adding more to your `main.star` file! Remember, this is an environment definition and you can import any pre-existing packages that you may find useful. Here are a [few examples](https://github.com/kurtosis-tech/awesome-kurtosis/tree/main)
* Deploy your dApp onto the local network! Hardhat can be used to do so by using the given RPC URL & the `network_id` defined in the `network_params.json` you wrote at the beginning. In your case, the `network_id` should be: `3151908`. A more thorough example of this workflow can also be found [here](./how-to-local-eth-testnet.md).
* Deploy your dApp onto the local network! Hardhat can be used to do so by using the given RPC URL & the `network_id` defined in the `network_params.yaml` you wrote at the beginning. In your case, the `network_id` should be: `3151908`. A more thorough example of this workflow can also be found [here](./how-to-local-eth-testnet.md).

We're currently building out more components of [`eth-kurtosis`](https://github.com/kurtosis-tech/eth-kurtosis/tree/main), which serves as an index of plug-and-play components for Ethereum private testnets. We're building support for more clients - so let us know if there's something you would love to see added to the index!

Expand Down
Loading

0 comments on commit 219ac7a

Please sign in to comment.