diff --git a/docs/Build/node-cli.md b/docs/Build/node-cli.md index 8ec0f5718..c41075c4b 100644 --- a/docs/Build/node-cli.md +++ b/docs/Build/node-cli.md @@ -63,7 +63,7 @@ POOL_NAME="GUILD" #### Start the node -To test starting the node in interactive mode, we will make use of pre-built script `cnode.sh`. This script automatically determines whether to start the node as a relay or block producer (if the required pool keys are present in the `$CNODE_HOME/priv/pool/` as mentioned above). The script contains a user-defined variable `CPU_CORES` which determines the number of CPU cores the node will use upon start-up: +To test starting the node in interactive mode, we will make use of pre-built script `cnode.sh`. This script automatically determines whether to start the node as a relay or block producer (if the required pool keys are present in the `$CNODE_HOME/priv/pool/` as mentioned above). If the `` variable is set to 'Y' it will download the latest snapshot from a Mithril aggregator to speed up the blockchain synchronization. The script contains a user-defined variable `CPU_CORES` which determines the number of CPU cores the node will use upon start-up: ```bash ###################################### diff --git a/docs/Scripts/mithril-client.md b/docs/Scripts/mithril-client.md index 2e4b1f960..32beab383 100644 --- a/docs/Scripts/mithril-client.md +++ b/docs/Scripts/mithril-client.md @@ -1,10 +1,37 @@ `mithril-client.sh` is a script to manage the Mithril client, a tool used to set up the Mithril client environment and manage downloading Mithril snapshots and stake distributions. The main features include: - **environment** - Creates a new `mithril.env` file with all the necessary environment variables for the Mithril client. -- **snapshot** - Download, list all or show a specific available Mithril snapshot. +- **cardano-db** - Download, list all or show a specific available Mithril snapshot. - **stake-distribution** - Download or list available Mithril stake distributions. - **-u** - Skip script update check. +## Usage + +```bash +Usage: bash [-u] [] +A script to run Cardano Mithril Client + +-u Skip script update check overriding UPDATE_CHECK value in env (must be first argument to script) + +Commands: +environment Manage mithril environment file + setup Setup mithril environment file + override Override default variable in the mithril environment file + update Update mithril environment file +cardano-db Interact with Cardano DB + download Download Cardano DB from Mithril snapshot + snapshot Interact with Mithril snapshots + list List available Mithril snapshots + json List availble Mithril snapshots in JSON format + show Show details of a Mithril snapshot + json Show details of a Mithril snapshot in JSON format +stake-distribution Interact with Mithril stake distributions + download Download latest stake distribution + list List available stake distributions + json Output latest Mithril snapshot in JSON format + +``` + ## Preparing a Relay or Block Producer Node To prepare a relay or block producer node, you should follow these steps: @@ -18,7 +45,7 @@ To prepare a relay or block producer node, you should follow these steps: 2. **Download the latest Mithril snapshot:** Once the environment file is set up, you can download the latest Mithril snapshot by running the script with the `snapshot download` command. This snapshot contains the latest state of the Cardano blockchain db from a Mithril Aggregator. ```bash - ./mithril-client.sh snapshot download + ./mithril-client.sh cardano-db download ``` ## Investigating Available Snapshots @@ -28,16 +55,16 @@ You can investigate the available snapshots by using the `snapshot list` and `sn - **List all available Mithril snapshots:** You can list all available Mithril snapshots by running the script with the `snapshot list` command. Add `json` at the end to get the output in JSON format. ```bash - ./mithril-client.sh snapshot list - ./mithril-client.sh snapshot list json + ./mithril-client.sh cardano-dbsnapshot list + ./mithril-client.sh cardano-dbsnapshot list json ``` - **Show details of a specific Mithril snapshot:** You can show details of a specific Mithril snapshot by running the script with the `snapshot show ` command, where `` is the digest of the snapshot. Add `json` at the end to get the output in JSON format. ```bash - ./mithril-client.sh snapshot show - ./mithril-client.sh snapshot show json - ./mithril-client.sh snapshot show json + ./mithril-client.sh cardano-dbsnapshot show + ./mithril-client.sh cardano-dbsnapshot show json + ./mithril-client.sh cardano-dbsnapshot show json ``` ## Managing Stake Distributions diff --git a/docs/Scripts/mithril-relay.md b/docs/Scripts/mithril-relay.md index a7713274a..01454779f 100644 --- a/docs/Scripts/mithril-relay.md +++ b/docs/Scripts/mithril-relay.md @@ -7,12 +7,13 @@ loadbalancer. It provides functionalities such as: ## Usage ```bash -Usage: mithril-relay.sh [-d] [-l] +bash [-d] [-l] [-u] [-h] +A script to setup Cardano Mithril relays -Options: - -d Install squid and configure as a relay - -l Install nginx and configure as a load balancer - -h Show this help text +-d Install squid and configure as a relay +-l Install nginx and configure as a load balancer +-u Skip update check +-h Show this help text ``` # Description diff --git a/docs/Scripts/mithril-signer.md b/docs/Scripts/mithril-signer.md index 1c846d433..4e28d4e6c 100644 --- a/docs/Scripts/mithril-signer.md +++ b/docs/Scripts/mithril-signer.md @@ -5,12 +5,17 @@ environment file to contain variables specific to the Mithril Signer. ## Usage ```bash -Usage: mithril-signer.sh [-d] [-u] - -Options: - -d Deploy mithril-signer as a systemd service - -u Update mithril environment file - -h Show this help text +Usage: bash [-d] [-D] [-e] [-k] [-r] [-s] [-u] [-h] +A script to setup, run and verify Cardano Mithril Signer + +-d Deploy mithril-signer as a systemd service +-D Run mithril-signer as a daemon +-e Update mithril environment file +-k Stop signer using SIGINT +-r Verify signer registration +-s Verify signer signature +-u Skip update check +-h Show this help text ``` # Description diff --git a/files/docker/node/addons/entrypoint.sh b/files/docker/node/addons/entrypoint.sh index e338c0f0a..5915223f7 100755 --- a/files/docker/node/addons/entrypoint.sh +++ b/files/docker/node/addons/entrypoint.sh @@ -6,6 +6,7 @@ trap 'killall -s SIGINT cardano-node' SIGINT SIGTERM head -n 8 ~/.scripts/banner.txt +# shellcheck disable=SC1090 . ~/.bashrc > /dev/null 2>&1 echo "NETWORK: $NETWORK $POOL_NAME $TOPOLOGY"; @@ -17,29 +18,29 @@ echo "NODE: $HOSTNAME - Port:$CNODE_PORT - $POOL_NAME"; cardano-node --version; if [[ "${ENABLE_BACKUP}" == "Y" ]] || [[ "${ENABLE_RESTORE}" == "Y" ]]; then - [[ ! -d "${CNODE_HOME}"/backup/$NETWORK-db ]] && mkdir -p $CNODE_HOME/backup/$NETWORK-db - dbsize=$(du -s $CNODE_HOME/db | awk '{print $1}') - bksizedb=$(du -s $CNODE_HOME/backup/$NETWORK-db 2>/dev/null | awk '{print $1}') - if [[ "${ENABLE_RESTORE}" == "Y" ]] && [[ "$dbsize" -lt "$bksizedb" ]]; then - echo "Backup Started" - cp -rf "${CNODE_HOME}"/backup/"${NETWORK}"-db/* "${CNODE_HOME}"/db 2>/dev/null - echo "Backup Finished" - fi - - if [[ "${ENABLE_BACKUP}" == "Y" ]] && [[ "$dbsize" -gt "$bksizedb" ]]; then - echo "Restore Started" - cp -rf "${CNODE_HOME}"/db/* "${CNODE_HOME}"/backup/"${NETWORK}"-db/ 2>/dev/null - echo "Restore Finished" - fi + [[ ! -d "${CNODE_HOME}"/backup/$NETWORK-db ]] && mkdir -p $CNODE_HOME/backup/$NETWORK-db + dbsize=$(du -s $CNODE_HOME/db | awk '{print $1}') + bksizedb=$(du -s $CNODE_HOME/backup/$NETWORK-db 2>/dev/null | awk '{print $1}') + if [[ "${ENABLE_RESTORE}" == "Y" ]] && [[ "$dbsize" -lt "$bksizedb" ]]; then + echo "Backup Started" + cp -rf "${CNODE_HOME}"/backup/"${NETWORK}"-db/* "${CNODE_HOME}"/db 2>/dev/null + echo "Backup Finished" + fi + + if [[ "${ENABLE_BACKUP}" == "Y" ]] && [[ "$dbsize" -gt "$bksizedb" ]]; then + echo "Restore Started" + cp -rf "${CNODE_HOME}"/db/* "${CNODE_HOME}"/backup/"${NETWORK}"-db/ 2>/dev/null + echo "Restore Finished" + fi fi # Customisation customise () { -find /opt/cardano/cnode/files -name "*config*.json" -print0 | xargs -0 sed -i 's/127.0.0.1/0.0.0.0/g' > /dev/null 2>&1 -grep -i ENABLE_CHATTR /opt/cardano/cnode/scripts/cntools.sh >/dev/null && sed -E -i 's/^#?ENABLE_CHATTR=(true|false)?/ENABLE_CHATTR=false/g' /opt/cardano/cnode/scripts/cntools.sh > /dev/null 2>&1 -grep -i ENABLE_DIALOG /opt/cardano/cnode/scripts/cntools.sh >/dev/null && sed -E -i 's/^#?ENABLE_DIALOG=(true|false)?/ENABLE_DIALOG=false/' /opt/cardano/cnode/scripts/cntools.sh >> /opt/cardano/cnode/scripts/cntools.sh -find /opt/cardano/cnode/files -name "*config*.json" -print0 | xargs -0 sed -i 's/\"hasEKG\": 12788,/\"hasEKG\": [\n \"0.0.0.0\",\n 12788\n],/g' > /dev/null 2>&1 -return 0 + find /opt/cardano/cnode/files -name "*config*.json" -print0 | xargs -0 sed -i 's/127.0.0.1/0.0.0.0/g' > /dev/null 2>&1 + grep -i ENABLE_CHATTR /opt/cardano/cnode/scripts/cntools.sh >/dev/null && sed -E -i 's/^#?ENABLE_CHATTR=(true|false)?/ENABLE_CHATTR=false/g' /opt/cardano/cnode/scripts/cntools.sh > /dev/null 2>&1 + grep -i ENABLE_DIALOG /opt/cardano/cnode/scripts/cntools.sh >/dev/null && sed -E -i 's/^#?ENABLE_DIALOG=(true|false)?/ENABLE_DIALOG=false/' /opt/cardano/cnode/scripts/cntools.sh > /dev/null 2>&1 + find /opt/cardano/cnode/files -name "*config*.json" -print0 | xargs -0 sed -i 's/\"hasEKG\": 12788,/\"hasEKG\": [\n \"0.0.0.0\",\n 12788\n],/g' > /dev/null 2>&1 + return 0 } load_configs () { diff --git a/files/docker/node/dockerfile_bin b/files/docker/node/dockerfile_bin index b9aec97a4..9782d4c14 100644 --- a/files/docker/node/dockerfile_bin +++ b/files/docker/node/dockerfile_bin @@ -44,7 +44,6 @@ RUN set -x && apt update \ && mkdir -p /root/.local/bin \ && wget https://raw.githubusercontent.com/${G_ACCOUNT}/guild-operators/${GUILD_DEPLOY_BRANCH}/scripts/cnode-helper-scripts/guild-deploy.sh \ && export SUDO='N' \ - && export UPDATE_CHECK='N' \ && export SKIP_DBSYNC_DOWNLOAD='Y' \ && export G_ACCOUNT=${G_ACCOUNT} \ && chmod +x ./guild-deploy.sh && ./guild-deploy.sh -b ${GUILD_DEPLOY_BRANCH} -s p \ @@ -57,7 +56,6 @@ RUN set -x && apt update \ && rm -rf /var/lib/apt/lists/* RUN set -x && export SUDO='N' \ - && export UPDATE_CHECK='N' \ && export SKIP_DBSYNC_DOWNLOAD='Y' \ && export G_ACCOUNT=${G_ACCOUNT} \ && ./guild-deploy.sh -b ${GUILD_DEPLOY_BRANCH} -s dcmowx \ diff --git a/scripts/cnode-helper-scripts/cnode.sh b/scripts/cnode-helper-scripts/cnode.sh index 865bdb2b9..720f88ade 100755 --- a/scripts/cnode-helper-scripts/cnode.sh +++ b/scripts/cnode-helper-scripts/cnode.sh @@ -65,9 +65,9 @@ pre_startup_sanity() { mithril_snapshot_download() { [[ -z "${MITHRIL_CLIENT}" ]] && MITHRIL_CLIENT="${CNODE_HOME}"/scripts/mithril-client.sh if [[ ! -f "${MITHRIL_CLIENT}" ]] || [[ ! -e "${MITHRIL_CLIENT}" ]]; then - echo "ERROR: Could not locate mithril-client.sh script or script is not executable. Skipping mithril snapshot download!!" + echo "ERROR: Could not locate mithril-client.sh script or script is not executable. Skipping mithril cardano-db snapshot download!!" else - "${MITHRIL_CLIENT}" snapshot download + "${MITHRIL_CLIENT}" cardano-db download fi } diff --git a/scripts/cnode-helper-scripts/guild-deploy.sh b/scripts/cnode-helper-scripts/guild-deploy.sh index a746257b2..d69a8c6b5 100755 --- a/scripts/cnode-helper-scripts/guild-deploy.sh +++ b/scripts/cnode-helper-scripts/guild-deploy.sh @@ -638,6 +638,7 @@ populate_cnode() { updateWithCustomConfig "mithril-client.sh" updateWithCustomConfig "mithril-relay.sh" updateWithCustomConfig "mithril-signer.sh" + updateWithCustomConfig "mithril.library" find "${CNODE_HOME}/scripts" -name '*.sh' -exec chmod 755 {} \; 2>/dev/null chmod -R 700 "${CNODE_HOME}"/priv 2>/dev/null diff --git a/scripts/cnode-helper-scripts/mithril-client.sh b/scripts/cnode-helper-scripts/mithril-client.sh index 54c9a140c..9c59716b0 100755 --- a/scripts/cnode-helper-scripts/mithril-client.sh +++ b/scripts/cnode-helper-scripts/mithril-client.sh @@ -2,7 +2,7 @@ # shellcheck disable=SC2086 #shellcheck source=/dev/null -. "$(dirname $0)"/env offline +. "$(dirname $0)"/mithril.library ###################################### # User Variables - Change as desired # @@ -15,9 +15,6 @@ # Do NOT modify code below # ###################################### -U_ID=$(id -u) -G_ID=$(id -g) - ##################### # Functions # ##################### @@ -25,8 +22,8 @@ G_ID=$(id -g) usage() { cat <<-EOF - Usage: $(basename "$0") - Script to run Cardano Mithril Client + Usage: $(basename "$0") [-u] [] + A script to run Cardano Mithril Client -u Skip script update check overriding UPDATE_CHECK value in env (must be first argument to script) @@ -35,12 +32,13 @@ usage() { setup Setup mithril environment file override Override default variable in the mithril environment file update Update mithril environment file - snapshot Interact with Mithril snapshots - download Download latest Mithril snapshot - list List available Mithril snapshots - json List availble Mithril snapshots in JSON format - show Show details of a Mithril snapshot - json Show details of a Mithril snapshot in JSON format + cardano-db Interact with Cardano DB + download Download Cardano DB from Mithril snapshot + snapshot Interact with Mithril snapshots + list List available Mithril snapshots + json List availble Mithril snapshots in JSON format + show Show details of a Mithril snapshot + json Show details of a Mithril snapshot in JSON format stake-distribution Interact with Mithril stake distributions download Download latest stake distribution list List available stake distributions @@ -50,62 +48,10 @@ EOF } SKIP_UPDATE=N -[[ $1 = "-u" ]] && SKIP_UPDATE=Y && shift +[[ $1 = "-u" ]] && export SKIP_UPDATE=Y && shift ## mithril environment subcommands -environment_setup() { - local env_file="${CNODE_HOME}/mithril/mithril.env" - - if [[ -f "$env_file" ]]; then - if [[ "$UPDATE_ENVIRONMENT" != "Y" ]]; then - echo "Error: $env_file already exists. To update it, set UPDATE_ENVIRONMENT to 'Y'." >&2 - return 1 - else - echo "Updating $env_file..." - fi - else - echo "Creating $env_file..." - fi - - if [[ ! -d "${CNODE_HOME}/mithril/data-stores" ]]; then - sudo mkdir -p "${CNODE_HOME}"/mithril/data-stores - sudo chown -R "$U_ID":"$G_ID" "${CNODE_HOME}"/mithril 2>/dev/null - fi - if [[ -n "${POOL_NAME}" ]] && [[ "${POOL_NAME}" != "CHANGE_ME" ]]; then - export ERA_READER_ADDRESS=https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/era.addr - export ERA_READER_VKEY=https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/era.vkey - bash -c "cat <<-'EOF' > ${CNODE_HOME}/mithril/mithril.env - KES_SECRET_KEY_PATH=${POOL_DIR}/${POOL_HOTKEY_SK_FILENAME} - OPERATIONAL_CERTIFICATE_PATH=${POOL_DIR}/${POOL_OPCERT_FILENAME} - NETWORK=${NETWORK_NAME,,} - RELEASE=${RELEASE} - AGGREGATOR_ENDPOINT=https://aggregator.${RELEASE}-${NETWORK_NAME,,}.api.mithril.network/aggregator - RUN_INTERVAL=60000 - DB_DIRECTORY=${CNODE_HOME}/db - CARDANO_NODE_SOCKET_PATH=${CARDANO_NODE_SOCKET_PATH} - CARDANO_CLI_PATH=${HOME}/.local/bin/cardano-cli - DATA_STORES_DIRECTORY=${CNODE_HOME}/mithril/data-stores - STORE_RETENTION_LIMITS=5 - ERA_READER_ADAPTER_TYPE=cardano-chain - ERA_READER_ADAPTER_PARAMS=$(jq -nc --arg address "$(wget -q -O - "${ERA_READER_ADDRESS}")" --arg verification_key "$(wget -q -O - "${ERA_READER_VKEY}")" '{"address": $address, "verification_key": $verification_key}') - GENESIS_VERIFICATION_KEY=$(wget -q -O - https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/genesis.vkey) - PARTY_ID=$(cat ${POOL_DIR}/${POOL_ID_FILENAME}) - SNAPSHOT_DIGEST=latest - EOF" - else - bash -c "cat <<-'EOF' > ${CNODE_HOME}/mithril/mithril.env - NETWORK=${NETWORK_NAME,,} - RELEASE=${RELEASE} - AGGREGATOR_ENDPOINT=https://aggregator.${RELEASE}-${NETWORK_NAME,,}.api.mithril.network/aggregator - DB_DIRECTORY=${CNODE_HOME}/db - GENESIS_VERIFICATION_KEY=$(wget -q -O - https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/genesis.vkey) - SNAPSHOT_DIGEST=latest - EOF" - fi - chown $USER:$USER "${CNODE_HOME}"/mithril/mithril.env -} - environment_override() { local var_to_override="$1" local new_value="$2" @@ -121,96 +67,36 @@ environment_override() { sed -i "s|^${var_to_override}=.*|${var_to_override}=${new_value}|" "$env_file" } -pre_startup_sanity() { - if [[ ${UPDATE_CHECK} = Y && ${SKIP_UPDATE} != Y ]]; then - - echo "Checking for script updates..." - - # Check availability of checkUpdate function - if [[ ! $(command -v checkUpdate) ]]; then - echo -e "\nCould not find checkUpdate function in env, make sure you're using official guild docos for installation!" - exit 1 - fi - - # check for env update - ENV_UPDATED=${BATCH_AUTO_UPDATE} - checkUpdate "${PARENT}"/env N N N - case $? in - 1) ENV_UPDATED=Y ;; - 2) exit 1 ;; - esac - - # check for cncli.sh update - checkUpdate "${PARENT}"/cncli.sh ${ENV_UPDATED} - case $? in - 1) $0 "-u" "$@"; exit 0 ;; # re-launch script with same args skipping update check - 2) exit 1 ;; - esac - fi - - REQUIRED_PARAMETERS="Y" - if [[ ! -f "${CNODE_HOME}"/mithril/mithril.env ]]; then - echo "INFO: Mithril environment file not found, creating environment file.." - environment_setup && echo "INFO: Mithril environment file created successfully!!" - elif [[ "${UPDATE_ENVIRONMENT}" == "Y" ]]; then - echo "INFO: Updating mithril environment file.." - environment_setup && echo "INFO: Mithril environment file updated successfully!!" - fi +mithril_init() { + [[ ! -f "${CNODE_HOME}"/mithril/mithril.env ]] && generate_environment_file . "${CNODE_HOME}"/mithril/mithril.env - [[ -z "${NETWORK}" ]] && echo "ERROR: The NETWORK must be set before calling mithril-client!!" && REQUIRED_PARAMETERS="N" - [[ -z "${RELEASE}" ]] && echo "ERROR: Failed to set RELEASE variable, please check NETWORK variable in env file!!" && REQUIRED_PARAMETERS="N" - [[ -z "${CNODE_HOME}" ]] && echo "ERROR: The CNODE_HOME must be set before calling mithril-client!!" && REQUIRED_PARAMETERS="N" - [[ ! -d "${CNODE_HOME}" ]] && echo "ERROR: The CNODE_HOME directory does not exist, please check CNODE_HOME variable in env file!!" && REQUIRED_PARAMETERS="N" - [[ -z "${AGGREGATOR_ENDPOINT}" ]] && echo "ERROR: The AGGREGATOR_ENDPOINT must be set before calling mithril-client!!" && REQUIRED_PARAMETERS="N" - [[ -z "${GENESIS_VERIFICATION_KEY}" ]] && echo "ERROR: The GENESIS_VERIFICATION_KEY must be set before calling mithril-client!!" && REQUIRED_PARAMETERS="N" - [[ ! -x "${MITHRILBIN}" ]] && echo "ERROR: The MITHRILBIN variable does not contain an executable file, please check MITHRILBIN variable in env file!!" && REQUIRED_PARAMETERS="N" - [[ "${REQUIRED_PARAMETERS}" != "Y" ]] && exit 1 - export GENESIS_VERIFICATION_KEY - DOWNLOAD_SNAPSHOT="N" - REMOVE_DB_DIR="N" } -set_defaults() { - [[ -z "${MITHRILBIN}" ]] && MITHRILBIN="${HOME}"/.local/bin/mithril-client - if [[ -z "${NETWORK_NAME}" ]]; then - echo "ERROR: The NETWORK_NAME must be set before mithril-client can download snapshots!!" - exit 1 - else - case "${NETWORK_NAME,,}" in - mainnet|preprod|guild) - RELEASE="release" - ;; - preview) - RELEASE="pre-release" - ;; - *) - echo "ERROR: The NETWORK_NAME must be set to Mainnet, PreProd, Preview, Guild before mithril-client can download snapshots!!" - exit 1 - esac - fi - pre_startup_sanity -} check_db_dir() { # If the DB directory does not exist then set DOWNLOAD_SNAPSHOT to Y if [[ ! -d "${DB_DIRECTORY}" ]]; then echo "INFO: The db directory does not exist.." DOWNLOAD_SNAPSHOT="Y" - # If the DB directory is empty then set DOWNLOAD_SNAPSHOT to Y and REMOVE_DB_DIR to Y + # If the DB directory is empty then set DOWNLOAD_SNAPSHOT to Y elif [[ -d "${DB_DIRECTORY}" ]] && [[ -z "$(ls -A "${DB_DIRECTORY}")" ]] && [[ $(du -cs "${DB_DIRECTORY}"/* 2>/dev/null | awk '/total$/ {print $1}') -eq 0 ]]; then echo "INFO: The db directory is empty.." - REMOVE_DB_DIR="Y" DOWNLOAD_SNAPSHOT="Y" else - echo "INFO: The db directory is not empty.." + echo "INFO: The db directory is not empty, skipping Cardano DB download.." fi } -remove_db_dir() { - # Mithril client errors if the db folder already exists, so remove it if it is empty - if [[ "${REMOVE_DB_DIR}" == "Y" ]]; then - echo "INFO: Removing empty db directory to prepare for snapshot download.." - rmdir "${DB_DIRECTORY}" +cleanup_db_directory() { + echo "WARNING: Download failure, cleaning up DB directory.." + # Safety check to prevent accidental deletion of system files + if [[ -z "${DB_DIRECTORY}" ]]; then + echo "ERROR: DB_DIRECTORY is unset or null." + elif [[ -n "${DB_DIRECTORY}" && "${DB_DIRECTORY}" != "/" && "${DB_DIRECTORY}" != "${CNODE_HOME}" ]]; then + # :? Safety check to prevent accidental deletion of system files, even though initial if block should already prevent this + rm -rf "${DB_DIRECTORY:?}/"* + else + echo "INFO: Skipping cleanup of DB directory: ${DB_DIRECTORY}." fi } @@ -219,18 +105,26 @@ remove_db_dir() { download_snapshot() { if [[ "${DOWNLOAD_SNAPSHOT}" == "Y" ]]; then echo "INFO: Downloading latest mithril snapshot.." - "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} snapshot download --download-dir ${CNODE_HOME} ${SNAPSHOT_DIGEST} + trap 'cleanup_db_directory' INT + if ! "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} cardano-db download --download-dir ${CNODE_HOME} --genesis-verification-key ${GENESIS_VERIFICATION_KEY} ${SNAPSHOT_DIGEST} ; then + cleanup_db_directory + exit 1 + fi else - echo "INFO: Skipping snapshot download.." + echo "INFO: Skipping Cardano DB download.." fi } list_snapshots() { - if [[ $1 == "json" ]]; then - "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} snapshot list --json - else - "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} snapshot list - fi + local json_flag="" + + for arg in "$@"; do + if [[ $arg == "json" ]]; then + json_flag="--json" + fi + done + + "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} cardano-db snapshot list $json_flag } show_snapshot() { @@ -250,7 +144,7 @@ show_snapshot() { exit 1 fi - "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} snapshot show $digest $json_flag + "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} cardano-db snapshot show $digest $json_flag } ## mithril-stake-distribution subcommands @@ -265,31 +159,39 @@ download_stake_distribution() { } list_stake_distributions() { - if [[ $1 == "json" ]]; then - "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} mithril-stake-distribution list --json - else - "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} mithril-stake-distribution list - fi + local json_flag="" + + for arg in "$@"; do + if [[ $arg == "json" ]]; then + json_flag="--json" + fi + done + + "${MITHRILBIN}" -v --aggregator-endpoint ${AGGREGATOR_ENDPOINT} mithril-stake-distribution list $json_flag + } ##################### -# Execution # +# Execution/Main # ##################### +update_check "$@" + +set_defaults + # Parse command line options case $1 in environment) - set_defaults case $2 in setup) - environment_setup + generate_environment_file ;; override) environment_override $3 $4 ;; update) - UPDATE_ENVIRONMENT="Y" - environment_setup + export UPDATE_ENVIRONMENT="Y" + generate_environment_file ;; *) echo "Invalid environment subcommand: $2" >&2 @@ -298,36 +200,38 @@ case $1 in ;; esac ;; - snapshot) - set_defaults + cardano-db) + mithril_init case $2 in download) check_db_dir - remove_db_dir download_snapshot ;; - list) - case $3 in - json) - list_snapshots json + snapshot) + case $3 in + list) + case $4 in + json) + list_snapshots json + ;; + *) + list_snapshots + ;; + esac + ;; + show) + show_snapshot $4 $5 ;; *) - list_snapshots + echo "Invalid snapshot subcommand: $3" >&2 + usage + exit 1 ;; esac - ;; - show) - show_snapshot $3 $4 - ;; - *) - echo "Invalid snapshot subcommand: $2" >&2 - usage - exit 1 - ;; esac ;; stake-distribution) - set_defaults + mithril_init case $2 in download) download_stake_distribution @@ -350,7 +254,7 @@ case $1 in esac ;; *) - echo "Invalid command: $1" >&2 + echo "Invalid $(basename "$0") command: $1" >&2 usage exit 1 ;; diff --git a/scripts/cnode-helper-scripts/mithril-relay.sh b/scripts/cnode-helper-scripts/mithril-relay.sh index 6c104fddf..0207856f4 100755 --- a/scripts/cnode-helper-scripts/mithril-relay.sh +++ b/scripts/cnode-helper-scripts/mithril-relay.sh @@ -1,4 +1,8 @@ #!/bin/bash +# shellcheck disable=SC2086 +#shellcheck source=/dev/null + +. "$(dirname $0)"/mithril.library ###################################### # User Variables - Change as desired # @@ -26,16 +30,18 @@ RELAY_LISTENING_IP=() usage() { cat <<-EOF - $(basename "$0") [-d] [-l] + $(basename "$0") [-d] [-l] [-u] [-h] + A script to setup Cardano Mithril relays - Cardano Mithril relay wrapper script!! -d Install squid and configure as a relay -l Install nginx and configure as a load balancer + -u Skip update check -h Show this help text EOF } + generate_nginx_conf() { sudo bash -c "cat > /etc/nginx/nginx.conf <<'EOF' worker_processes 1; @@ -123,76 +129,103 @@ generate_squid_conf() { EOF" } +deploy_nginx_load_balancer() { + # Install nginx and configure load balancing + echo -e "\nInstalling nginx load balancer" + sudo apt-get update + sudo apt-get install -y nginx + + # Read the listening IP addresses from user input + while true; do + read -r -p "Enter the IP address of a relay: " ip + RELAY_LISTENING_IP+=("${ip}") + read -r -p "Are there more relays? (y/n) " yn + case ${yn} in + [Nn]*) break ;; + *) continue ;; + esac + done + + # Read the listening IP for the load balancer + read -r -p "Enter the IP address of the load balancer (press Enter to use default 127.0.0.1): " SIDECAR_LISTENING_IP + SIDECAR_LISTENING_IP=${SIDECAR_LISTENING_IP:-127.0.0.1} + echo "Using IP address ${SIDECAR_LISTENING_IP} for the load balancer configuration." + + # Read the listening port from user input + read -r -p "Enter the relay's listening port (press Enter to use default 3132): " RELAY_LISTENING_PORT + RELAY_LISTENING_PORT=${RELAY_LISTENING_PORT:-3132} + echo "Using port ${RELAY_LISTENING_PORT} for relay's listening port." + + # Generate the nginx configuration file + generate_nginx_conf + # Restart nginx and check status + echo -e "\nStarting Mithril relay sidecar (nginx load balancer)" + sudo systemctl restart nginx + sudo systemctl status nginx + +} + +deploy_squid_proxy() { + # Install squid and make a backup of the config file + echo -e "\nInstalling squid proxy" + sudo apt-get update + sudo apt-get install -y squid + sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.bak + + # Read the listening IP addresses from user input + while true; do + read -r -p "Enter the IP address of your Block Producer: " ip + BLOCK_PRODUCER_IP+=("${ip}") + read -r -p "Are there more block producers? (y/n) " yn + case ${yn} in + [Nn]*) break ;; + *) continue ;; + esac + done + + # Read the listening port from user input + read -r -p "Enter the relay's listening port (press Enter to use default 3132): " RELAY_LISTENING_PORT + RELAY_LISTENING_PORT=${RELAY_LISTENING_PORT:-3132} + echo "Using port ${RELAY_LISTENING_PORT} for relay's listening port." + generate_squid_conf + + # Restart squid and check status + echo -e "\nStarting Mithril relay (squid proxy)" + sudo systemctl restart squid + sudo systemctl status squid + + # Inform the user to create the appropriate firewall rule + for ip in "${RELAY_LISTENING_IP[@]}"; do + echo "Create the appropriate firewall rule: sudo ufw allow from ${ip} to any port ${RELAY_LISTENING_PORT} proto tcp" + done +} + +stop_relays() { + echo " Stopping squid proxy and nginx load balancers.." + sudo systemctl stop squid 2>/dev/null + sudo systemctl stop nginx 2>/dev/null + sleep 5 + exit 0 +} + +##################### +# Execution/Main # +##################### + # Parse command line arguments -while getopts :dlh opt; do +while getopts :dlsuh opt; do case ${opt} in d) - # Install squid and make a backup of the config file - echo -e "\nInstalling squid proxy" - sudo apt-get update - sudo apt-get install -y squid - sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.bak - - # Read the listening IP addresses from user input - while true; do - read -r -p "Enter the IP address of your Block Producer: " ip - BLOCK_PRODUCER_IP+=("${ip}") - read -r -p "Are there more block producers? (y/n) " yn - case ${yn} in - [Nn]*) break ;; - *) continue ;; - esac - done - - # Read the listening port from user input - read -r -p "Enter the relay's listening port (press Enter to use default 3132): " RELAY_LISTENING_PORT - RELAY_LISTENING_PORT=${RELAY_LISTENING_PORT:-3132} - echo "Using port ${RELAY_LISTENING_PORT} for relay's listening port." - generate_squid_conf - - # Restart squid and check status - echo -e "\nStarting Mithril relay (squid proxy)" - sudo systemctl restart squid - sudo systemctl status squid - - # Inform the user to create the appropriate firewall rule - for ip in "${RELAY_LISTENING_IP[@]}"; do - echo "Create the appropriate firewall rule: sudo ufw allow from ${ip} to any port ${RELAY_LISTENING_PORT} proto tcp" - done + INSTALL_SQUID_PROXY=Y ;; l) - # Install nginx and configure load balancing - echo -e "\nInstalling nginx load balancer" - sudo apt-get update - sudo apt-get install -y nginx - - # Read the listening IP addresses from user input - while true; do - read -r -p "Enter the IP address of a relay: " ip - RELAY_LISTENING_IP+=("${ip}") - read -r -p "Are there more relays? (y/n) " yn - case ${yn} in - [Nn]*) break ;; - *) continue ;; - esac - done - - # Read the listening IP for the load balancer - read -r -p "Enter the IP address of the load balancer (press Enter to use default 127.0.0.1): " SIDECAR_LISTENING_IP - SIDECAR_LISTENING_IP=${SIDECAR_LISTENING_IP:-127.0.0.1} - echo "Using IP address ${SIDECAR_LISTENING_IP} for the load balancer configuration." - - # Read the listening port from user input - read -r -p "Enter the relay's listening port (press Enter to use default 3132): " RELAY_LISTENING_PORT - RELAY_LISTENING_PORT=${RELAY_LISTENING_PORT:-3132} - echo "Using port ${RELAY_LISTENING_PORT} for relay's listening port." - - # Generate the nginx configuration file - generate_nginx_conf - # Restart nginx and check status - echo -e "\nStarting Mithril relay sidecar (nginx load balancer)" - sudo systemctl restart nginx - sudo systemctl status nginx + INSTALL_NGINX_LOAD_BALANCER=Y + ;; + u) + export SKIP_UPDATE='Y' + ;; + s) + STOP_RELAYS=Y ;; h) usage @@ -220,3 +253,15 @@ if [[ ${OPTIND} -eq 1 ]]; then usage exit 1 fi + +[[ "${STOP_RELAYS}" == "Y" ]] && stop_relays + +update_check "$@" + +if [[ ${INSTALL_SQUID_PROXY} = Y ]]; then + deploy_squid_proxy +fi + +if [[ ${INSTALL_NGINX_LOAD_BALANCER} = Y ]]; then + deploy_nginx_load_balancer +fi diff --git a/scripts/cnode-helper-scripts/mithril-signer.sh b/scripts/cnode-helper-scripts/mithril-signer.sh index 9670460c3..82c0f27c0 100755 --- a/scripts/cnode-helper-scripts/mithril-signer.sh +++ b/scripts/cnode-helper-scripts/mithril-signer.sh @@ -2,7 +2,7 @@ # shellcheck disable=SC2086 #shellcheck source=/dev/null -. "$(dirname $0)"/env offline +. "$(dirname $0)"/mithril.library ###################################### # User Variables - Change as desired # @@ -16,9 +16,6 @@ # Do NOT modify code below # ###################################### -U_ID=$(id -u) -G_ID=$(id -g) - ##################### # Functions # ##################### @@ -26,102 +23,36 @@ G_ID=$(id -g) usage() { cat <<-EOF - Usage: $(basename "$0") [-d] [-u] + Usage: $(basename "$0") [-d] [-D] [-e] [-k] [-r] [-s] [-u] [-h] + A script to setup, run and verify Cardano Mithril Signer - Cardano Mithril signer wrapper script !! -d Deploy mithril-signer as a systemd service - -u Update mithril environment file + -D Run mithril-signer as a daemon + -e Update mithril environment file + -k Stop signer using SIGINT + -r Verify signer registration + -s Verify signer signature + -u Skip update check -h Show this help text EOF } -set_defaults() { - [[ -z "${MITHRILBIN}" ]] && MITHRILBIN="${HOME}"/.local/bin/mithril-signer - if [[ -z "${POOL_NAME}" ]] || [[ "${POOL_NAME}" == "CHANGE_ME" ]]; then - echo "ERROR: The POOL_NAME must be set before deploying mithril-signer as a systemd service!!" - exit 1 - else - case "${NETWORK_NAME,,}" in - mainnet|preprod|guild) - RELEASE="release" - ;; - preview) - RELEASE="pre-release" - ;; - *) - echo "ERROR: The NETWORK_NAME must be set to Mainnet, PreProd, Preview, or Guild before mithril-signer can be deployed!!" - exit 1 - esac - fi -} - -pre_startup_sanity() { - [[ ! -f "${MITHRILBIN}" ]] && MITHRILBIN="$(command -v mithril-signer)" - if [[ ! -S "${CARDANO_NODE_SOCKET_PATH}" ]]; then - echo "ERROR: Could not locate socket file at ${CARDANO_NODE_SOCKET_PATH}, the node may not have completed startup !!" - exit 1 - fi +mithril_init() { + [[ ! -f "${CNODE_HOME}"/mithril/mithril.env ]] && generate_environment_file + for line in $(cat "${CNODE_HOME}"/mithril/mithril.env); do + export "${line}" + done # Move logs to archive - [[ -f "${LOG_DIR}"/mithril-signer.log ]] && mv "${LOG_DIR}"/mithril-signer.log "${LOG_DIR}"/archive/ -} - -get_relay_endpoint() { - read -r -p "Enter the IP address of the relay endpoint: " RELAY_ENDPOINT_IP - read -r -p "Enter the port of the relay endpoint (press Enter to use default 3132): " RELAY_PORT - RELAY_PORT=${RELAY_PORT:-3132} - echo "Using RELAY_ENDPOINT=${RELAY_ENDPOINT_IP}:${RELAY_PORT} for the Mithril signer relay endpoint." -} - -generate_environment_file() { - if [[ ! -d "${CNODE_HOME}/mithril/data-stores" ]]; then - sudo mkdir -p "${CNODE_HOME}"/mithril/data-stores - sudo chown -R "$U_ID":"$G_ID" "${CNODE_HOME}"/mithril 2>/dev/null - fi - # Inquire about the relay endpoint - read -r -p "Are you using a relay endpoint? (y/n, press Enter to use default y): " ENABLE_RELAY_ENDPOINT - ENABLE_RELAY_ENDPOINT=${ENABLE_RELAY_ENDPOINT:-y} - if [[ "${ENABLE_RELAY_ENDPOINT}" == "y" ]]; then - get_relay_endpoint - else - echo "Using a naive Mithril configuration without a mithril relay." - fi - - # Generate the full set of environment variables required by Mithril signer use case - export ERA_READER_ADDRESS=https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/era.addr - export ERA_READER_VKEY=https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/era.vkey - sudo bash -c "cat <<-'EOF' > ${CNODE_HOME}/mithril/mithril.env - KES_SECRET_KEY_PATH=${POOL_DIR}/${POOL_HOTKEY_SK_FILENAME} - OPERATIONAL_CERTIFICATE_PATH=${POOL_DIR}/${POOL_OPCERT_FILENAME} - NETWORK=${NETWORK_NAME,,} - RELEASE=${RELEASE} - AGGREGATOR_ENDPOINT=https://aggregator.${RELEASE}-${NETWORK_NAME,,}.api.mithril.network/aggregator - RUN_INTERVAL=60000 - DB_DIRECTORY=${CNODE_HOME}/db - CARDANO_NODE_SOCKET_PATH=${CARDANO_NODE_SOCKET_PATH} - CARDANO_CLI_PATH=${HOME}/.local/bin/cardano-cli - DATA_STORES_DIRECTORY=${CNODE_HOME}/mithril/data-stores - STORE_RETENTION_LIMITS=5 - ERA_READER_ADAPTER_TYPE=cardano-chain - ERA_READER_ADAPTER_PARAMS=$(jq -nc --arg address "$(wget -q -O - "${ERA_READER_ADDRESS}")" --arg verification_key "$(wget -q -O - "${ERA_READER_VKEY}")" '{"address": $address, "verification_key": $verification_key}') - GENESIS_VERIFICATION_KEY=$(wget -q -O - https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/genesis.vkey) - PARTY_ID=$(cat ${POOL_DIR}/${POOL_ID_FILENAME}) - SNAPSHOT_DIGEST=latest - EOF" && sudo chown $USER:$USER "${CNODE_HOME}"/mithril/mithril.env - - if [[ "${ENABLE_RELAY_ENDPOINT}" == "y" ]]; then - sudo bash -c "echo RELAY_ENDPOINT=http://${RELAY_ENDPOINT_IP}:${RELAY_PORT} >> ${CNODE_HOME}/mithril/mithril.env" - fi + [[ -d "${LOG_DIR}"/archive ]] || mkdir -p "${LOG_DIR}"/archive + [[ -f "${LOG_DIR}"/$(basename "${0::-3}").log ]] && mv "${LOG_DIR}/$(basename "${0::-3}")".log "${LOG_DIR}"/archive/ ; touch "${LOG_DIR}/$(basename "${0::-3}")".log } deploy_systemd() { - echo "Creating ${CNODE_VNAME}-mithril-signer systemd service environment file.." - if [[ ! -f "${CNODE_HOME}"/mithril/mithril.env ]]; then - generate_environment_file && echo "Mithril environment file created successfully!!" - fi + echo "Creating ${CNODE_VNAME}-$(basename "${0::-3}") systemd service environment file.." - echo "Deploying ${CNODE_VNAME}-mithril-signer as systemd service.." - sudo bash -c "cat <<-'EOF' > /etc/systemd/system/${CNODE_VNAME}-mithril-signer.service + echo "Deploying ${CNODE_VNAME}-$(basename "${0::-3}") as systemd service.." + sudo bash -c "cat <<-'EOF' > /etc/systemd/system/${CNODE_VNAME}-$(basename "${0::-3}").service [Unit] Description=Cardano Mithril signer service StartLimitIntervalSec=0 @@ -136,35 +67,116 @@ deploy_systemd() { RestartSec=60 User=${USER} EnvironmentFile=${CNODE_HOME}/mithril/mithril.env - ExecStart=/bin/bash -l -c \"exec ${HOME}/.local/bin/mithril-signer -vv\" + ExecStart=/bin/bash -l -c \"exec ${HOME}/.local/bin/$(basename "${0::-3}") -vv\" KillSignal=SIGINT SuccessExitStatus=143 StandardOutput=syslog StandardError=syslog - SyslogIdentifier=${CNODE_VNAME}-mithril-signer + SyslogIdentifier=${CNODE_VNAME}-$(basename "${0::-3}") TimeoutStopSec=5 KillMode=mixed [Install] WantedBy=multi-user.target - EOF" && echo "${CNODE_VNAME}-mithril-signer.service deployed successfully!!" && sudo systemctl daemon-reload && sudo systemctl enable ${CNODE_VNAME}-mithril-signer.service + EOF" && echo "${CNODE_VNAME}-$(basename "${0::-3}").service deployed successfully!!" && sudo systemctl daemon-reload && sudo systemctl enable ${CNODE_VNAME}-"$(basename "${0::-3}")".service +} + +stop_signer() { + CNODE_PID=$(pgrep -fn "$(basename ${CNODEBIN}).*.--port ${CNODE_PORT}" 2>/dev/null) # env was only called in offline mode + kill -2 ${CNODE_PID} 2>/dev/null + # touch clean "${CNODE_HOME}"/db/clean # Disabled as it's a bit hacky, but only runs when SIGINT is passed to node process. Should not be needed if node does it's job + echo " Sending SIGINT to $(basename "${0::-3}") process.." + sleep 5 + exit 0 } -################### -# Execution # -################### + +user_interrupt_received() { + echo " SIGINT received, stopping $(basename "${0::-3}").." |tee -a "${LOG_DIR}/$(basename "${0::-3}")".log 2>&1 + stop_signer + +} + +verify_signer_registration() { + set -e + + if [ -z "$AGGREGATOR_ENDPOINT" ] || [ -z "$PARTY_ID" ]; then + echo ">> ERROR: Required environment variables AGGREGATOR_ENDPOINT and/or PARTY_ID are not set." + exit 1 + fi + + CURRENT_EPOCH=$(curl -s "$AGGREGATOR_ENDPOINT/epoch-settings" -H 'accept: application/json' | jq -r '.epoch') + SIGNERS_REGISTERED_RESPONSE=$(curl -s "$AGGREGATOR_ENDPOINT/signers/registered/$CURRENT_EPOCH" -H 'accept: application/json') + + if echo "$SIGNERS_REGISTERED_RESPONSE" | grep -q "$PARTY_ID"; then + echo ">> Congrats, your signer node is registered!" + else + echo ">> Oops, your signer node is not registered. Party ID not found among the signers registered at epoch ${CURRENT_EPOCH}." + fi + +} + +verify_signer_signature() { + set -e + + if [ -z "$AGGREGATOR_ENDPOINT" ] || [ -z "$PARTY_ID" ]; then + echo ">> ERROR: Required environment variables AGGREGATOR_ENDPOINT and/or PARTY_ID are not set." + exit 1 + fi + + CERTIFICATES_RESPONSE=$(curl -s "$AGGREGATOR_ENDPOINT/certificates" -H 'accept: application/json') + CERTIFICATES_COUNT=$(echo "$CERTIFICATES_RESPONSE" | jq '. | length') + + echo "$CERTIFICATES_RESPONSE" | jq -r '.[] | .hash' | while read -r HASH; do + RESPONSE=$(curl -s "$AGGREGATOR_ENDPOINT/certificate/$HASH" -H 'accept: application/json') + SIGNER_COUNT=$(echo "$RESPONSE" | jq '.metadata.signers | length') + for (( i=0; i < SIGNER_COUNT; i++ )); do + PARTY_ID_RESPONSE=$(echo "$RESPONSE" | jq -r ".metadata.signers[$i].party_id") + if [[ "$PARTY_ID_RESPONSE" == "$PARTY_ID" ]]; then + echo ">> Congrats, you have signed this certificate: $AGGREGATOR_ENDPOINT/certificate/$HASH !" + exit 1 + fi + done + done + + echo ">> Oops, your party id was not found in the last ${CERTIFICATES_COUNT} certificates. Please try again later." + +} + + +##################### +# Execution / Main # +##################### # Parse command line options -while getopts :duh opt; do +while getopts :dDekrsuh opt; do case ${opt} in - d ) DEPLOY_SYSTEMD="Y" ;; - u ) UPDATE_ENVIRONMENT="Y" ;; + d ) + DEPLOY_SYSTEMD="Y" ;; + D ) + SIGNER_DAEMON="Y" + ;; + e ) + export UPDATE_ENVIRONMENT="Y" + ;; + k ) + STOP_SIGNER="Y" + ;; + r ) + VERIFY_REGISTRATION="Y" + ;; + s ) + VERIFY_SIGNATURE="Y" + ;; + u ) + export SKIP_UPDATE="Y" + ;; h) usage exit 0 ;; \?) - echo "Invalid option: -${OPTARG}" >&2 + echo "Invalid $(basename "$0") option: -${OPTARG}" >&2 usage exit 1 ;; @@ -176,32 +188,46 @@ while getopts :duh opt; do esac done -# Check if env file is missing in current folder (no update checks as will mostly run as daemon), source env if present -[[ ! -f "$(dirname $0)"/env ]] && echo -e "\nCommon env file missing, please ensure latest guild-deploy.sh was run and this script is being run from ${CNODE_HOME}/scripts folder! \n" && exit 1 -. "$(dirname $0)"/env -case $? in - 1) echo -e "ERROR: Failed to load common env file\nPlease verify set values in 'User Variables' section in env file or log an issue on GitHub" && exit 1;; - 2) clear ;; -esac +[[ "${STOP_SIGNER}" == "Y" ]] && stop_signer + +# Check for updates +update_check "$@" # Set defaults and do basic sanity checks set_defaults + + #Deploy systemd if -d argument was specified -if [[ "${UPDATE_ENVIRONMENT}" == "Y" && "${DEPLOY_SYSTEMD}" == "Y" ]]; then - generate_environment_file && echo "Environment file updated successfully" && deploy_systemd && echo "Mithril signer service successfully deployed" && exit 0 - exit 2 -elif [[ "${UPDATE_ENVIRONMENT}" == "Y" ]]; then - generate_environment_file && echo "Environment file updated successfully" && exit 0 - exit 2 -elif [[ "${DEPLOY_SYSTEMD}" == "Y" ]]; then - deploy_systemd && echo "Mithril signer service successfully deployed" && exit 0 - exit 2 -fi - -pre_startup_sanity - -# Run Mithril Signer Server -echo "Sourcing the Mithril Signer environment file.." -. "${CNODE_HOME}"/mithril/mithril.env -echo "Starting Mithril Signer Server.." -"${MITHRILBIN}" -vvv >> "${LOG_DIR}"/mithril-signer.log 2>&1 +if [[ "${UPDATE_ENVIRONMENT}" == "Y" ]]; then + generate_environment_file + exit 0 +else + mithril_init + if [[ "${DEPLOY_SYSTEMD}" == "Y" ]]; then + if deploy_systemd ; then + echo "Mithril signer Systemd service successfully deployed" + exit 0 + else + echo "Failed to deploy Mithril signer Systemd service" + exit 2 + fi + elif [[ "${VERIFY_REGISTRATION}" == "Y" ]]; then + # Verify signer registration + echo "Verifying Mithril Signer registration.." + verify_signer_registration + exit 0 + elif [[ "${VERIFY_SIGNATURE}" == "Y" ]]; then + # Verify signer signature + echo "Verifying Mithril Signer signature.." + verify_signer_signature + exit 0 + elif [[ "${SIGNER_DAEMON}" == "Y" ]]; then + # Run Mithril Signer Server + echo "Starting Mithril Signer Server.." + trap 'user_interrupt_received' INT + if ! "${MITHRILBIN}" -vv | tee -a "${LOG_DIR}/$(basename "${0::-3}")".log 2>&1 ; then + echo "Failed to start Mithril Signer Server" | tee -a "${LOG_DIR}/$(basename "${0::-3}")".log 2>&1 + exit 1 + fi + fi +fi \ No newline at end of file diff --git a/scripts/cnode-helper-scripts/mithril.library b/scripts/cnode-helper-scripts/mithril.library new file mode 100644 index 000000000..4a471ef18 --- /dev/null +++ b/scripts/cnode-helper-scripts/mithril.library @@ -0,0 +1,196 @@ +#!/usr/bin/env bash +# shellcheck disable=SC2034,SC2086,SC2230,SC2206,SC2140,SC2059,SC2154 +#shellcheck source=/dev/null + +###################################### +# Do NOT modify code below # +###################################### + +. "$(dirname $0)"/env offline + +U_ID=$(id -u) +G_ID=$(id -g) +MITHRILBIN="${HOME}"/.local/bin/$(basename "${0::-3}") + +compare_versions() { + local min_version=$1 + local test_version=$2 + if [[ $(printf '%s\n' "$min_version" "$test_version" | sort -V | head -n1) = "$min_version" ]]; then + return 0 + else + return 1 + fi +} + +set_node_minimum_version() { + response_file=$(mktemp) + status_code=$(curl -s -o "$response_file" -w "%{http_code}" https://raw.githubusercontent.com/input-output-hk/mithril/${MITHRIL_LATEST_VERSION}/networks.json) + + if [[ "$status_code" -eq 404 ]]; then + NODE_MINIMUM_VERSION="" + else + NODE_MINIMUM_VERSION=$(jq -r ".${NETWORK}.\"cardano-minimum-version\".\"mithril-signer\"" "$response_file") + fi + rm -f "$response_file" +} + +update_check() { + # Check availability of checkUpdate function + if [[ ! $(command -v checkUpdate) ]]; then + echo -e "\nCould not find checkUpdate function in env, make sure you're using official guild docos for installation!" + exit 1 + fi + # Check if flag is set by script to skip update check + [[ ${SKIP_UPDATE} == Y ]] && return 0 + # Check if flag is set by user as a global (container environments etc.) to skip update check + if [[ ${UPDATE_CHECK} = Y ]]; then + echo "Checking for script updates..." + # check for env update + ENV_UPDATED=${BATCH_AUTO_UPDATE} + checkUpdate "${PARENT}"/env N N N + case $? in + 1) ENV_UPDATED=Y ;; + 2) exit 1 ;; + esac + # check for mithril.library update + checkUpdate "${PARENT}"/mithril.library N N N + # borrow ENV_UPDATED for mithril.library updates + case $? in + 1) ENV_UPDATED=Y ;; + 2) exit 1 ;; + esac + # check the script update + checkUpdate "${PARENT}"/"$(basename "$0")" ${ENV_UPDATED} + case $? in + 1) echo ""; $0 "-u" "$@"; exit 0 ;; # re-launch script with same args skipping update check + 2) exit 1 ;; + esac + fi +} + + +set_defaults() { + MITHRIL_LATEST_VERSION=$(curl -s https://raw.githubusercontent.com/cardano-community/guild-operators/alpha/files/docker/node/release-versions/mithril-latest.txt) + set_node_minimum_version + NODE_CURRENT_VERSION=$(cardano-node --version | awk 'NR==1{print $2}') + + [[ -z "${MITHRILBIN}" ]] && MITHRILBIN="${HOME}"/.local/bin/"$(basename "${0::-3}")" + if [[ $(basename "${0::-3}") == "mithril-signer" ]] && { [[ -z "${POOL_NAME}" ]] || [[ "${POOL_NAME}" == "CHANGE_ME" ]]; }; then + echo "ERROR: The POOL_NAME must be set before deploying mithril-signer as a systemd service!!" + exit 1 + else + case "${NETWORK_NAME,,}" in + mainnet|preprod|guild) + RELEASE="release" + ;; + preview|sanchonet) + RELEASE="pre-release" + ;; + *) + echo "ERROR: The NETWORK_NAME must be set to mainnet, preprod, preview, or sanchonet before $(basename "${0::-3}") can be deployed!!" + exit 1 + esac + fi + AGGREGATOR_ENDPOINT=https://aggregator.${RELEASE}-${NETWORK_NAME,,}.api.mithril.network/aggregator + GENESIS_VERIFICATION_KEY=$(wget -q -O - https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/genesis.vkey) +} + +create_data_stores_directory() { + if [[ ! -d "${CNODE_HOME}/mithril/data-stores" ]]; then + sudo mkdir -p "${CNODE_HOME}"/mithril/data-stores + sudo chown -R "$U_ID":"$G_ID" "${CNODE_HOME}"/mithril 2>/dev/null + fi +} + +set_env_file_ownership() { + chown $USER:$USER "${CNODE_HOME}"/mithril/mithril.env +} + + +check_mithril_environment_file_exists() { + local env_file="${CNODE_HOME}/mithril/mithril.env" + + if [[ -f "$env_file" ]]; then + if [[ "$UPDATE_ENVIRONMENT" != "Y" ]]; then + echo "Error: $env_file already exists. To update it, set UPDATE_ENVIRONMENT to 'Y'." >&2 + return 1 + else + echo "Updating $env_file..." + fi + else + echo "Creating $env_file..." + fi +} + +get_relay_endpoint() { + read -r -p "Enter the IP address of the relay endpoint: " RELAY_ENDPOINT_IP + read -r -p "Enter the port of the relay endpoint (press Enter to use default 3132): " RELAY_PORT + RELAY_PORT=${RELAY_PORT:-3132} + echo "Using RELAY_ENDPOINT=${RELAY_ENDPOINT_IP}:${RELAY_PORT} for the Mithril signer relay endpoint." +} + +update_mithril_environment_for_signer() { + echo "Info: Setting all environment variables, supporting the Mithril signer use case." + # Inquire about the relay endpoint + read -r -p "Are you using a relay endpoint? (y/n, press Enter to use default y): " ENABLE_RELAY_ENDPOINT + ENABLE_RELAY_ENDPOINT=${ENABLE_RELAY_ENDPOINT:-y} + if [[ "${ENABLE_RELAY_ENDPOINT}" == "y" ]]; then + get_relay_endpoint + else + echo "Using a naive Mithril configuration without a mithril relay." + fi + # Generate the full set of environment variables required by Mithril signer use case + export ERA_READER_ADDRESS=https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/era.addr + export ERA_READER_VKEY=https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/era.vkey + sudo bash -c "cat <<-'EOF' > ${CNODE_HOME}/mithril/mithril.env + KES_SECRET_KEY_PATH=${POOL_DIR}/${POOL_HOTKEY_SK_FILENAME} + OPERATIONAL_CERTIFICATE_PATH=${POOL_DIR}/${POOL_OPCERT_FILENAME} + NETWORK=${NETWORK_NAME,,} + RELEASE=${RELEASE} + AGGREGATOR_ENDPOINT=https://aggregator.${RELEASE}-${NETWORK_NAME,,}.api.mithril.network/aggregator + RUN_INTERVAL=60000 + DB_DIRECTORY=${CNODE_HOME}/db + CARDANO_NODE_SOCKET_PATH=${CARDANO_NODE_SOCKET_PATH} + CARDANO_CLI_PATH=${HOME}/.local/bin/cardano-cli + DATA_STORES_DIRECTORY=${CNODE_HOME}/mithril/data-stores + STORE_RETENTION_LIMITS=5 + ERA_READER_ADAPTER_TYPE=cardano-chain + ERA_READER_ADAPTER_PARAMS=$(jq -nc --arg address "$(wget -q -O - "${ERA_READER_ADDRESS}")" --arg verification_key "$(wget -q -O - "${ERA_READER_VKEY}")" '{"address": $address, "verification_key": $verification_key}') + GENESIS_VERIFICATION_KEY=$(wget -q -O - https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/genesis.vkey) + PARTY_ID=$(cat ${POOL_DIR}/${POOL_ID_FILENAME}-bech32) + SNAPSHOT_DIGEST=latest + EOF" + + if [[ "${ENABLE_RELAY_ENDPOINT}" == "y" ]]; then + sudo bash -c "echo RELAY_ENDPOINT=http://${RELAY_ENDPOINT_IP}:${RELAY_PORT} >> ${CNODE_HOME}/mithril/mithril.env" + fi +} + +update_mithril_environment_for_client() { + echo "Info: Setting minimal environment variables supporting only the Mithril client use case." + bash -c "cat <<-'EOF' > ${CNODE_HOME}/mithril/mithril.env + NETWORK=${NETWORK_NAME,,} + RELEASE=${RELEASE} + AGGREGATOR_ENDPOINT=https://aggregator.${RELEASE}-${NETWORK_NAME,,}.api.mithril.network/aggregator + DB_DIRECTORY=${CNODE_HOME}/db + GENESIS_VERIFICATION_KEY=$(wget -q -O - https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/${RELEASE}-${NETWORK_NAME,,}/genesis.vkey) + SNAPSHOT_DIGEST=latest + EOF" +} + +component_environment_setup() { + check_mithril_environment_file_exists + + if [[ -n "${POOL_NAME}" ]] && [[ "${POOL_NAME}" != "CHANGE_ME" ]] && [[ "$(basename "$0")" == "mithril-signer.sh" ]]; then + update_mithril_environment_for_signer + else + update_mithril_environment_for_client + fi +} + +generate_environment_file() { + create_data_stores_directory + component_environment_setup + set_env_file_ownership +} +