Skip to content

Commit

Permalink
Issue/1756 mithril script updates (#24)
Browse files Browse the repository at this point in the history
* syntactic bug in entrypoint.sh cardano-community#1757

* Remove uneeded UPDATE_CHECK=N exports now that it is a default of the image ENV variables

* export G_ACCOUNT so guild-deploy.sh inherits it during forked builds

* Set defaults for G_ACCOUNT and GUILD_DEPLOY_BRANCH to simplify manual docker build commands

* Fix bug where updating cncli.sh cardano-community#1756

mithril-client binary command/subcommand changes replace snapshot with cardano-db cardano-community#1759

* Add reference to MITHRIL_DOWNLOAD for snapshot sync in regular docs

* export each line of mithril.env for the signer
use tee for logging to not hide issues/errors

* Fix for PARTY_ID used in verify_signer_request and verify_signer_signature

* Move sanchonet to RELEASE=testing

* SANCHONET/PREVIEW - adjustments to support sanchonet and preview versions of mithril for guild-deploy.sh downloading.

* SANCHONET/PREVIEW - adjustments to support sanchonet and preview versions of mithril for entrypoint.sh .

* linting and indentation consistentcy for entrypoint.sh

* Workflows to update mithril and node prerelease versions in preview branch, and mithril unstable version in sanchonet branch.

* Workflows to rebase preview and sanchonet branches on push to alpha, or workflow dispatch.

* Docker Image workflow determine cnversion to use based on the guild_deploy_branch name.
- When branch preview/sanchonet use prerelease node version.
- For any other branch use the original logic for latest (stable) node version.

---------

Co-authored-by: RdLrT <[email protected]>
  • Loading branch information
TrevorBenson and rdlrt authored Apr 27, 2024
1 parent a8743cb commit eabd93f
Show file tree
Hide file tree
Showing 18 changed files with 898 additions and 425 deletions.
8 changes: 6 additions & 2 deletions .github/workflows/docker_bin.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,14 @@ jobs:
id: set_short_sha
run: |
echo "short_sha=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
- name: Get CNVERSION
- name: Set CNVersion
id: set_cnversion
run: |
echo "cnversion=$(cat files/docker/node/release-versions/cardano-node-latest.txt)" >> $GITHUB_OUTPUT
if [[ "${{ steps.set_guild_deploy_branch.outputs.guild_deploy_branch }}" != "preview" && "${{ steps.set_guild_deploy_branch.outputs.guild_deploy_branch }}" != "sanchonet" ]]; then
echo "cnversion=$(cat files/docker/node/release-versions/cardano-node-latest.txt)" >> "$GITHUB_OUTPUT"
else
echo "cnversion=$(cat files/docker/node/release-versions/cardano-node-prerelease.txt)" >> "$GITHUB_OUTPUT"
fi
build_production:
needs: set_environment_vars
if: needs.set_environment_vars.outputs.testing == 'false' && needs.set_environment_vars.outputs.guild_deploy_branch == 'master'
Expand Down
51 changes: 51 additions & 0 deletions .github/workflows/mithril-prerelease.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Get Mithril pre-release version
on:
workflow_dispatch:
push:
branches:
- alpha
schedule:
- cron: '0 */8 * * *'

jobs:
get-version:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
ref: preview
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
echo "GitHub Actor: ${GITHUB_ACTOR}" >> "$GITHUB_STEP_SUMMARY"
echo "GitHub Email: ${GITHUB_ACTOR}@users.noreply.github.com" >> "$GITHUB_STEP_SUMMARY"
- name: Fetch Mithril release and pre-release versions
run: |
mithril_release="$(curl -s https://api.github.com/repos/input-output-hk/mithril/releases/latest | jq -r '.tag_name')"
mithril_prerelease="$(curl -s https://api.github.com/repos/input-output-hk/mithril/releases | jq -r '.[] | select(.prerelease == true) | select(.tag_name | endswith("-pre")) | .tag_name' | head -n 1)"
if [[ "${mithril_release}-pre" == "${mithril_prerelease}" ]]; then
echo "${mithril_release}" > files/docker/node/release-versions/mithril-prerelease.txt
echo "PRERELEASE_VERSION=${mithril_release}" >> $GITHUB_ENV
echo "Pre-release version: ${mithril_release}" >> $GITHUB_STEP_SUMMARY
echo "Pre-release == Release: True" >> $GITHUB_STEP_SUMMARY
else
echo "${mithril_prerelease}" > files/docker/node/release-versions/mithril-prerelease.txt
echo "PRERELEASE_VERSION=${mithril_prerelease}" >> $GITHUB_ENV
echo "Pre-release version: ${mithril_prerelease}" >> $GITHUB_STEP_SUMMARY
echo "Pre-release == Release: False" >> $GITHUB_STEP_SUMMARY
fi
- name: Check for changes
id: git-check
run: |
export MODIFIED=$([ -z "`git status --porcelain`" ] && echo "false" || echo "true")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
echo "Repository Modified: ${MODIFIED}" >> "$GITHUB_STEP_SUMMARY"
- name: Commit latest pre-release versions
if: steps.git-check.outputs.MODIFIED == 'true'
run: |
echo "Modified files: $(git diff --name-only $(jq -r '.sha' "$GITHUB_OUTPUT"))" >> "$GITHUB_STEP_SUMMARY"
git add files/docker/node/release-versions/mithril-prerelease.txt
git commit -am "New mithril pre-release versions: pre-release ${PRERELEASE_VERSION}"
git push
44 changes: 44 additions & 0 deletions .github/workflows/mithril-unstable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Get Mithril unstable version
on:
workflow_dispatch:
push:
branches:
- alpha
schedule:
- cron: '0 */8 * * *'

jobs:
get-version:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
ref: sanchonet
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
echo "GitHub Actor: ${GITHUB_ACTOR}" >> "$GITHUB_STEP_SUMMARY"
echo "GitHub Email: ${GITHUB_ACTOR}@users.noreply.github.com" >> "$GITHUB_STEP_SUMMARY"
- name: Fetch Mithril unstable version
run: |
curl -sL https://api.github.com/repos/input-output-hk/mithril/releases | jq -r '.[] | select(.prerelease == true and .tag_name == "unstable") | .tag_name' | head -1 > files/docker/node/release-versions/mithril-unstable.txt
- name: Assign unstable version
run: |
export UNSTABLE_VERSION=$(cat files/docker/node/release-versions/mithril-unstable.txt)
echo "UNSTABLE_VERSION=${UNSTABLE_VERSION}" >> $GITHUB_ENV
echo "Unstable version: ${UNSTABLE_VERSION}" >> $GITHUB_STEP_SUMMARY
- name: Check for changes
id: git-check
run: |
export MODIFIED=$([ -z "`git status --porcelain`" ] && echo "false" || echo "true")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
echo "Repository Modified: ${MODIFIED}" >> "$GITHUB_STEP_SUMMARY"
- name: Commit latest unstable versions
if: steps.git-check.outputs.MODIFIED == 'true'
run: |
echo "Modified files: $(git diff --name-only $(jq -r '.sha' "$GITHUB_OUTPUT"))" >> "$GITHUB_STEP_SUMMARY"
git add files/docker/node/release-versions/mithril-unstable.txt
git commit -am "New mithril unstable versions: unstable ${UNSTABLE_VERSION}"
git push
42 changes: 42 additions & 0 deletions .github/workflows/node-prerelease.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Get Node pre-release version
on:
workflow_dispatch:
push:
branches:
- alpha
schedule:
- cron: '0 */8 * * *'

jobs:
get-version:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
ref: preview
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
echo "GitHub Actor: ${GITHUB_ACTOR}" >> "$GITHUB_STEP_SUMMARY"
echo "GitHub Email: ${GITHUB_ACTOR}@users.noreply.github.com" >> "$GITHUB_STEP_SUMMARY"
- name: Fetch node pre-release version
run: |
curl -sL https://api.github.com/repos/IntersectMBO/cardano-node/releases | jq -r '.[] | select(.prerelease == true) | .tag_name' | head -1 > files/docker/node/release-versions/cardano-node-prerelease.txt
- name: Assigns pre-release version
run: |
VERSION=$(cat files/docker/node/release-versions/cardano-node-prerelease.txt)
- name: Check for changes
id: git-check
run: |
export MODIFIED=$([ -z "`git status --porcelain`" ] && echo "false" || echo "true")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
echo "Repository Modified: ${MODIFIED}" >> "$GITHUB_STEP_SUMMARY"
- name: Commit latest pre-release versions
if: steps.git-check.outputs.MODIFIED == 'true'
run: |
echo "Modified files: $(git diff --name-only $(jq -r '.sha' "$GITHUB_OUTPUT"))" >> "$GITHUB_STEP_SUMMARY"
git add files/docker/node/release-versions/cardano-node-prerelease.txt
git commit -am "New mithril pre-release versions: pre-release ${PRERELEASE_VERSION}"
git push
40 changes: 40 additions & 0 deletions .github/workflows/rebase-preview.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Rebase preview branch
on:
workflow_dispatch:
push:
branches:
- alpha

jobs:
rebase:
runs-on: ubuntu-latest
steps:
- name: Sleep for automatic pre-release / unstable version updates to complete
run: sleep 90s
- uses: actions/checkout@v2
with:
ref: preview
fetch-depth: 0
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
echo "GitHub Actor: ${GITHUB_ACTOR}" >> "$GITHUB_STEP_SUMMARY"
echo "GitHub Email: ${GITHUB_ACTOR}@users.noreply.github.com" >> "$GITHUB_STEP_SUMMARY"
- name: Fetch alpha
run: |
git fetch origin alpha
- name: Get current SHA
run: |
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Rebase on alpha
run: |
git rebase origin/alpha
- name: Check for changes
id: git-check
run: |
export MODIFIED=$([ "$(git rev-parse HEAD)" != "$(jq -r '.sha' "$GITHUB_OUTPUT")" ] && echo "true" || echo "false")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
- name: Push changes
if: steps.git-check.outputs.MODIFIED == 'true'
run: git push --force-with-lease
40 changes: 40 additions & 0 deletions .github/workflows/rebase-sanchonet.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Rebase sanchonet branch
on:
workflow_dispatch:
push:
branches:
- alpha
jobs:
rebase:
runs-on: ubuntu-latest
steps:
- name: Sleep for automatic pre-release / unstable version updates to complete
run: sleep 90s
- uses: actions/checkout@v2
with:
ref: sanchonet
fetch-depth: 0
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
echo "GitHub Actor: ${GITHUB_ACTOR}" >> "$GITHUB_STEP_SUMMARY"
echo "GitHub Email: ${GITHUB_ACTOR}@users.noreply.github.com" >> "$GITHUB_STEP_SUMMARY"
- name: Fetch alpha
run: |
git fetch origin alpha
- name: Get current SHA
run: |
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Rebase on alpha
run: |
git rebase origin/alpha
- name: Check for changes
id: git-check
run: |
export MODIFIED=$([ "$(git rev-parse HEAD)" != "$(jq -r '.sha' "$GITHUB_OUTPUT")" ] && echo "true" || echo "false")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
echo "Repository Modified: ${MODIFIED}" >> "$GITHUB_STEP_SUMMARY"
- name: Push changes
if: steps.git-check.outputs.MODIFIED == 'true'
run: git push --force-with-lease
2 changes: 1 addition & 1 deletion docs/Build/node-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ POOL_NAME="GUILD"

#### Start the node

To test starting the node in interactive mode, we will make use of pre-built script `cnode.sh`. This script automatically determines whether to start the node as a relay or block producer (if the required pool keys are present in the `$CNODE_HOME/priv/pool/<POOL_NAME>` as mentioned above). The script contains a user-defined variable `CPU_CORES` which determines the number of CPU cores the node will use upon start-up:
To test starting the node in interactive mode, we will make use of pre-built script `cnode.sh`. This script automatically determines whether to start the node as a relay or block producer (if the required pool keys are present in the `$CNODE_HOME/priv/pool/<POOL_NAME>` as mentioned above). If the `<MITHRIL_DOWNLOAD>` variable is set to 'Y' it will download the latest snapshot from a Mithril aggregator to speed up the blockchain synchronization. The script contains a user-defined variable `CPU_CORES` which determines the number of CPU cores the node will use upon start-up:

```bash
######################################
Expand Down
41 changes: 34 additions & 7 deletions docs/Scripts/mithril-client.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,37 @@
`mithril-client.sh` is a script to manage the Mithril client, a tool used to set up the Mithril client environment and manage downloading Mithril snapshots and stake distributions. The main features include:

- **environment** - Creates a new `mithril.env` file with all the necessary environment variables for the Mithril client.
- **snapshot** - Download, list all or show a specific available Mithril snapshot.
- **cardano-db** - Download, list all or show a specific available Mithril snapshot.
- **stake-distribution** - Download or list available Mithril stake distributions.
- **-u** - Skip script update check.

## Usage

```bash
Usage: bash [-u] <command> <subcommand> [<sub arg>]
A script to run Cardano Mithril Client

-u Skip script update check overriding UPDATE_CHECK value in env (must be first argument to script)

Commands:
environment Manage mithril environment file
setup Setup mithril environment file
override Override default variable in the mithril environment file
update Update mithril environment file
cardano-db Interact with Cardano DB
download Download Cardano DB from Mithril snapshot
snapshot Interact with Mithril snapshots
list List available Mithril snapshots
json List availble Mithril snapshots in JSON format
show Show details of a Mithril snapshot
json Show details of a Mithril snapshot in JSON format
stake-distribution Interact with Mithril stake distributions
download Download latest stake distribution
list List available stake distributions
json Output latest Mithril snapshot in JSON format

```

## Preparing a Relay or Block Producer Node

To prepare a relay or block producer node, you should follow these steps:
Expand All @@ -18,7 +45,7 @@ To prepare a relay or block producer node, you should follow these steps:
2. **Download the latest Mithril snapshot:** Once the environment file is set up, you can download the latest Mithril snapshot by running the script with the `snapshot download` command. This snapshot contains the latest state of the Cardano blockchain db from a Mithril Aggregator.

```bash
./mithril-client.sh snapshot download
./mithril-client.sh cardano-db download
```

## Investigating Available Snapshots
Expand All @@ -28,16 +55,16 @@ You can investigate the available snapshots by using the `snapshot list` and `sn
- **List all available Mithril snapshots:** You can list all available Mithril snapshots by running the script with the `snapshot list` command. Add `json` at the end to get the output in JSON format.

```bash
./mithril-client.sh snapshot list
./mithril-client.sh snapshot list json
./mithril-client.sh cardano-dbsnapshot list
./mithril-client.sh cardano-dbsnapshot list json
```

- **Show details of a specific Mithril snapshot:** You can show details of a specific Mithril snapshot by running the script with the `snapshot show <DIGEST>` command, where `<DIGEST>` is the digest of the snapshot. Add `json` at the end to get the output in JSON format.

```bash
./mithril-client.sh snapshot show <DIGEST>
./mithril-client.sh snapshot show <DIGEST> json
./mithril-client.sh snapshot show json <DIGEST>
./mithril-client.sh cardano-dbsnapshot show <DIGEST>
./mithril-client.sh cardano-dbsnapshot show <DIGEST> json
./mithril-client.sh cardano-dbsnapshot show json <DIGEST>
```

## Managing Stake Distributions
Expand Down
11 changes: 6 additions & 5 deletions docs/Scripts/mithril-relay.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,13 @@ loadbalancer. It provides functionalities such as:
## Usage

```bash
Usage: mithril-relay.sh [-d] [-l]
bash [-d] [-l] [-u] [-h]
A script to setup Cardano Mithril relays

Options:
-d Install squid and configure as a relay
-l Install nginx and configure as a load balancer
-h Show this help text
-d Install squid and configure as a relay
-l Install nginx and configure as a load balancer
-u Skip update check
-h Show this help text
```

# Description
Expand Down
17 changes: 11 additions & 6 deletions docs/Scripts/mithril-signer.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,17 @@ environment file to contain variables specific to the Mithril Signer.
## Usage

```bash
Usage: mithril-signer.sh [-d] [-u]

Options:
-d Deploy mithril-signer as a systemd service
-u Update mithril environment file
-h Show this help text
Usage: bash [-d] [-D] [-e] [-k] [-r] [-s] [-u] [-h]
A script to setup, run and verify Cardano Mithril Signer

-d Deploy mithril-signer as a systemd service
-D Run mithril-signer as a daemon
-e Update mithril environment file
-k Stop signer using SIGINT
-r Verify signer registration
-s Verify signer signature
-u Skip update check
-h Show this help text
```

# Description
Expand Down
Loading

0 comments on commit eabd93f

Please sign in to comment.