Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(zkstack_cli): Add the ability to run the server from Docker without building it #3314

Open
wants to merge 49 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
5224856
refactor: wrap server run shell command in function
manuelmauro Nov 21, 2024
b7e8386
refactor: name arg module consistently with command module
manuelmauro Nov 21, 2024
c6eb6c6
refactor: use module folder for server command
manuelmauro Nov 21, 2024
c3dccee
refactor: create module for wait subcommand
manuelmauro Nov 21, 2024
159ebdf
refactor: create module for build subcommand
manuelmauro Nov 21, 2024
9d72c07
refactor: create module for run subcommand
manuelmauro Nov 21, 2024
c85ba08
refactor: create module for server run args
manuelmauro Nov 21, 2024
80f15ee
refactor: clippy
manuelmauro Nov 21, 2024
52bbf71
feat: add debug execution mode for server
manuelmauro Nov 22, 2024
a513136
feat: draft docker execution mode for server
manuelmauro Nov 22, 2024
ba7314e
feat: remove hardcoded volume folders
manuelmauro Nov 22, 2024
9676335
feat: draft option to specify docker image's tag
manuelmauro Nov 22, 2024
ccb1740
chore: update autocompletion files
manuelmauro Nov 25, 2024
fc4a31b
Merge branch 'main' into manuel-run-server-from-docker
manuelmauro Nov 25, 2024
c4e0bc9
feat: add utils to get list of github tags
manuelmauro Nov 25, 2024
f8b5ea3
feat: add selector for docker image tag
manuelmauro Nov 25, 2024
8bc6966
fix: remove temporary test
manuelmauro Nov 25, 2024
477a11c
feat: deduplicate cargo run command
manuelmauro Nov 25, 2024
27138a2
refactor: use const for repo url
manuelmauro Nov 25, 2024
a0e3c38
refactor: remove /config volume from dockerized server
manuelmauro Nov 26, 2024
711f3b4
Merge branch 'main' into manuel-run-server-from-docker
manuelmauro Nov 26, 2024
4061cbf
feat: add mode/tag to server run subcommands
manuelmauro Nov 26, 2024
debb507
fix: fix docker volumes/paths
manuelmauro Nov 26, 2024
d80eaa5
feat: add option to run dockerized server in genesis subcommand
manuelmauro Nov 26, 2024
5b81995
chore: regenerate autocomplete files
manuelmauro Nov 26, 2024
eef40fd
chore: update autocompletion files
manuelmauro Nov 27, 2024
fa85308
refactor: bail instead of unwrapping
manuelmauro Nov 27, 2024
f990a0a
refactor: improve execution mode enum
manuelmauro Nov 27, 2024
03a615e
refactor: clippy
manuelmauro Nov 27, 2024
70c17a6
feat: add execution mode options to missing subcommands
manuelmauro Nov 27, 2024
221deb7
chore: regenerate autocompletion files
manuelmauro Nov 27, 2024
5b1854c
refactor: add message const for select tag prompt
manuelmauro Nov 27, 2024
7711a3c
refactor: wrap docker run command in a function
manuelmauro Nov 27, 2024
9087f0a
feat: use folder for config files in docker
manuelmauro Nov 27, 2024
2a2e712
Merge branch 'main' into manuel-run-server-from-docker
manuelmauro Nov 27, 2024
f3ec2ee
refactor: clippy
manuelmauro Nov 27, 2024
54b8141
docs: improve CLI help
manuelmauro Nov 28, 2024
1939883
chore: update autocompletion files
manuelmauro Nov 28, 2024
bbe5b7d
feat: add port mapping to server's docker run
manuelmauro Nov 29, 2024
698e558
feat: expose all required ports from the server
manuelmauro Nov 29, 2024
fc211f9
feat: adjust host to execution mode
manuelmauro Nov 29, 2024
f054118
fix: fix debug execution mode
manuelmauro Nov 29, 2024
1bb08b3
refactor: clippy
manuelmauro Nov 29, 2024
ff1418c
Merge branch 'main' into manuel-run-server-from-docker
manuelmauro Nov 29, 2024
a58cf27
chore: update lock file
manuelmauro Nov 29, 2024
55b0a6c
fix: adjust host only when required
manuelmauro Nov 30, 2024
c09b95e
fix: remove unwrapping
manuelmauro Nov 30, 2024
282d05f
Merge branch 'main' into manuel-run-server-from-docker
manuelmauro Dec 3, 2024
2180bd9
chore: update cargo lock
manuelmauro Dec 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
refactor: improve execution mode enum
  • Loading branch information
manuelmauro committed Nov 27, 2024
commit f990a0ae25fe8896df91838f1f06bcf4c622c827
25 changes: 9 additions & 16 deletions zkstack_cli/crates/common/src/server.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use std::{ffi::OsStr, path::PathBuf};

use anyhow::{bail, Context};
use anyhow::Context;
use xshell::{cmd, Shell};

use crate::cmd::Cmd;
Expand All @@ -21,12 +21,14 @@ pub enum ServerMode {
}

/// Possible execution modes.
#[derive(Debug, Default)]
#[derive(Clone, Debug, Default)]
pub enum ExecutionMode {
#[default]
Release,
Debug,
Docker,
Docker {
tag: String,
},
}

impl Server {
Expand All @@ -52,7 +54,6 @@ impl Server {
secrets_path: P,
contracts_path: P,
mut additional_args: Vec<String>,
tag: Option<String>,
) -> anyhow::Result<()>
where
P: AsRef<OsStr>,
Expand All @@ -79,7 +80,6 @@ impl Server {
additional_args,
execution_mode,
server_mode,
tag,
)
.await?;

Expand Down Expand Up @@ -116,7 +116,6 @@ async fn run_server<P>(
additional_args: Vec<String>,
execution_mode: ExecutionMode,
server_mode: ServerMode,
tag: Option<String>,
) -> anyhow::Result<()>
where
P: AsRef<OsStr>,
Expand Down Expand Up @@ -144,14 +143,9 @@ where
contracts_path,
additional_args,
),
ExecutionMode::Docker => {
let Some(tag) = tag else {
bail!("Undefined docker image tag");
};

Cmd::new(cmd!(
shell,
"docker run
ExecutionMode::Docker { tag } => Cmd::new(cmd!(
Deniallugo marked this conversation as resolved.
Show resolved Hide resolved
shell,
"docker run
--platform linux/amd64
--net=host
-v {genesis_path}:/genesis.yaml
Expand All @@ -166,8 +160,7 @@ where
--secrets-path /secrets.yaml
--contracts-config-path /contracts.yaml
{additional_args...}"
))
}
)),
};

// If we are running server in normal mode
Expand Down
27 changes: 14 additions & 13 deletions zkstack_cli/crates/zkstack/src/commands/args/server/run.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use clap::{Parser, ValueEnum};
use common::server::ExecutionMode;
use serde::{Deserialize, Serialize};

use crate::{
Expand All @@ -10,27 +11,29 @@ use crate::{
};

#[derive(Clone, Debug, Default, Serialize, Deserialize, ValueEnum)]
pub enum ExecutionMode {
pub enum Mode {
#[default]
Release,
Debug,
Docker,
}

impl From<ExecutionMode> for common::server::ExecutionMode {
fn from(mode: ExecutionMode) -> Self {
match mode {
ExecutionMode::Debug => Self::Debug,
ExecutionMode::Release => Self::Release,
ExecutionMode::Docker => Self::Docker,
impl Mode {
pub fn as_execution_mode(self, tag: Option<String>) -> ExecutionMode {
match self {
Mode::Debug => ExecutionMode::Debug,
Mode::Release => ExecutionMode::Release,
Mode::Docker => ExecutionMode::Docker {
tag: tag.unwrap_or("latest".to_string()),
},
}
}
}

#[derive(Debug, Serialize, Deserialize, Parser)]
pub struct RunServerArgs {
#[arg(long, default_value = "release")]
pub mode: ExecutionMode,
pub mode: Mode,
#[arg(long)]
pub tag: Option<String>,
#[arg(long, help = MSG_SERVER_COMPONENTS_HELP)]
Expand All @@ -51,16 +54,15 @@ pub struct RunServerArgs {

impl RunServerArgs {
pub async fn fill_values_with_prompt(self) -> RunServerArgsFinal {
let tag = if let ExecutionMode::Docker = self.mode {
let tag = if let Mode::Docker = self.mode {
self.tag
.or(select_tag().await.ok().or(Some("latest".to_string())))
} else {
None
};

RunServerArgsFinal {
mode: self.mode,
tag,
mode: self.mode.as_execution_mode(tag),
components: self.components,
genesis: self.genesis,
additional_args: self.additional_args,
Expand All @@ -69,10 +71,9 @@ impl RunServerArgs {
}
}

#[derive(Debug, Serialize, Deserialize, Parser)]
#[derive(Debug)]
pub struct RunServerArgsFinal {
pub mode: ExecutionMode,
pub tag: Option<String>,
pub components: Option<Vec<String>>,
pub genesis: bool,
pub additional_args: Vec<String>,
Expand Down
18 changes: 8 additions & 10 deletions zkstack_cli/crates/zkstack/src/commands/chain/args/genesis/mod.rs
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
use anyhow::Context;
use clap::Parser;
use common::{db::DatabaseConfig, Prompt};
use common::{db::DatabaseConfig, server::ExecutionMode, Prompt};
use config::ChainConfig;
use serde::{Deserialize, Serialize};
use slugify_rs::slugify;
use url::Url;

use crate::{
commands::args::run::ExecutionMode,
commands::args::run::Mode,
defaults::{generate_db_names, DBNames, DATABASE_SERVER_URL},
messages::{
msg_server_db_name_prompt, msg_server_db_url_prompt, MSG_SERVER_DB_NAME_HELP,
Expand All @@ -21,7 +21,7 @@ pub mod server;
#[derive(Debug, Clone, Serialize, Deserialize, Parser, Default)]
pub struct GenesisArgs {
#[arg(long, default_value = "release")]
pub mode: ExecutionMode,
pub mode: Mode,
#[arg(long)]
pub tag: Option<String>,
#[clap(long, help = MSG_SERVER_DB_URL_HELP)]
Expand All @@ -36,7 +36,7 @@ pub struct GenesisArgs {

impl GenesisArgs {
pub async fn fill_values_with_prompt(self, config: &ChainConfig) -> GenesisArgsFinal {
let tag = if let ExecutionMode::Docker = self.mode {
let tag = if let Mode::Docker = self.mode {
self.tag
.or(select_tag().await.ok().or(Some("latest".to_string())))
} else {
Expand All @@ -49,8 +49,7 @@ impl GenesisArgs {
GenesisArgsFinal {
server_db: DatabaseConfig::new(DATABASE_SERVER_URL.clone(), server_name),
dont_drop: self.dont_drop,
mode: self.mode,
tag,
mode: self.mode.as_execution_mode(tag),
}
} else {
let server_db_url = self.server_db_url.unwrap_or_else(|| {
Expand All @@ -66,11 +65,11 @@ impl GenesisArgs {
}),
separator = "_"
);

GenesisArgsFinal {
server_db: DatabaseConfig::new(server_db_url, server_db_name),
dont_drop: self.dont_drop,
mode: self.mode,
tag,
mode: self.mode.as_execution_mode(tag),
}
}
}
Expand Down Expand Up @@ -104,10 +103,9 @@ impl GenesisArgs {
}
}

#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone)]
pub struct GenesisArgsFinal {
pub server_db: DatabaseConfig,
pub dont_drop: bool,
pub mode: ExecutionMode,
pub tag: Option<String>,
}
Original file line number Diff line number Diff line change
@@ -1,52 +1,33 @@
use clap::{Parser, ValueEnum};
use clap::Parser;
use common::server::ExecutionMode;
use serde::{Deserialize, Serialize};

use crate::utils::docker::select_tag;

#[derive(Clone, Debug, Default, Serialize, Deserialize, ValueEnum)]
pub enum ExecutionMode {
#[default]
Release,
Debug,
Docker,
}

impl From<ExecutionMode> for common::server::ExecutionMode {
fn from(mode: ExecutionMode) -> Self {
match mode {
ExecutionMode::Debug => Self::Debug,
ExecutionMode::Release => Self::Release,
ExecutionMode::Docker => Self::Docker,
}
}
}
use crate::{commands::args::run::Mode, utils::docker::select_tag};

#[derive(Clone, Debug, Serialize, Deserialize, Parser)]
pub struct GenesisServerArgs {
#[arg(long, default_value = "release")]
pub mode: ExecutionMode,
pub mode: Mode,
#[arg(long)]
pub tag: Option<String>,
}

impl GenesisServerArgs {
pub async fn fill_values_with_prompt(self) -> GenesisServerArgsFinal {
let tag = if let ExecutionMode::Docker = self.mode {
let tag = if let Mode::Docker = self.mode {
self.tag
.or(select_tag().await.ok().or(Some("latest".to_string())))
} else {
None
};

GenesisServerArgsFinal {
mode: self.mode,
tag,
mode: self.mode.as_execution_mode(tag),
}
}
}

#[derive(Debug, Serialize, Deserialize, Parser)]
#[derive(Debug)]
pub struct GenesisServerArgsFinal {
pub mode: ExecutionMode,
pub tag: Option<String>,
}
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ pub struct InitConfigsArgs {
pub no_port_reallocation: bool,
}

#[derive(Debug, Serialize, Deserialize, Clone)]
#[derive(Debug, Clone)]
pub struct InitConfigsArgsFinal {
pub genesis_args: GenesisArgsFinal,
pub l1_rpc_url: String,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use url::Url;

use crate::{
commands::{
args::run::ExecutionMode,
args::run::Mode,
chain::args::genesis::{GenesisArgs, GenesisArgsFinal},
},
defaults::LOCAL_RPC_URL,
Expand Down Expand Up @@ -45,8 +45,8 @@ pub struct InitArgs {
impl InitArgs {
pub fn get_genesis_args(&self) -> GenesisArgs {
GenesisArgs {
mode: ExecutionMode::Release,
tag: None,
mode: Mode::Release, // TODO
tag: None, // TODO
server_db_url: self.server_db_url.clone(),
server_db_name: self.server_db_name.clone(),
dev: self.dev,
Expand Down Expand Up @@ -96,7 +96,7 @@ impl InitArgs {
}
}

#[derive(Debug, Serialize, Deserialize, Clone)]
#[derive(Debug, Clone)]
pub struct InitArgsFinal {
pub forge_args: ForgeScriptArgs,
pub genesis_args: GenesisArgsFinal,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ pub async fn genesis(
spinner.finish();

let spinner = Spinner::new(MSG_STARTING_GENESIS_SPINNER);
run_server_genesis(config, shell, args.mode.into(), args.tag).await?;
run_server_genesis(config, shell, args.mode.into()).await?;
spinner.finish();

Ok(())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ pub async fn run(args: GenesisServerArgs, shell: &Shell) -> anyhow::Result<()> {
.context(MSG_CHAIN_NOT_INITIALIZED)?;

let spinner = Spinner::new(MSG_STARTING_GENESIS_SPINNER);
run_server_genesis(&chain_config, shell, args.mode.into(), args.tag).await?;
run_server_genesis(&chain_config, shell, args.mode.into()).await?;
spinner.finish();
logger::outro(MSG_GENESIS_COMPLETED);

Expand All @@ -38,7 +38,6 @@ pub async fn run_server_genesis(
chain_config: &ChainConfig,
shell: &Shell,
execution_mode: ExecutionMode,
tag: Option<String>,
) -> anyhow::Result<()> {
let server = Server::new(None, chain_config.link_to_code.clone(), false);
server
Expand All @@ -52,7 +51,6 @@ pub async fn run_server_genesis(
SecretsConfig::get_path_with_base_path(&chain_config.configs),
ContractsConfig::get_path_with_base_path(&chain_config.configs),
vec![],
tag,
)
.await
.context(MSG_FAILED_TO_RUN_SERVER_ERR)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use types::L1Network;
use url::Url;

use crate::{
commands::{args::run::ExecutionMode, chain::args::genesis::GenesisArgs},
commands::{args::run::Mode, chain::args::genesis::GenesisArgs},
defaults::LOCAL_RPC_URL,
messages::{
MSG_DEPLOY_ECOSYSTEM_PROMPT, MSG_DEPLOY_ERC20_PROMPT, MSG_DEV_ARG_HELP,
Expand Down Expand Up @@ -106,8 +106,8 @@ pub struct EcosystemInitArgs {
impl EcosystemInitArgs {
pub fn get_genesis_args(&self) -> GenesisArgs {
GenesisArgs {
mode: ExecutionMode::Release,
tag: None,
mode: Mode::Release, // TODO
tag: None, // TODO
server_db_url: self.server_db_url.clone(),
server_db_name: self.server_db_name.clone(),
dev: self.dev,
Expand Down
3 changes: 1 addition & 2 deletions zkstack_cli/crates/zkstack/src/commands/server/run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,14 @@ pub(super) async fn run_server(
server
.run(
shell,
args.mode.into(),
args.mode,
server_mode,
GenesisConfig::get_path_with_base_path(&chain_config.configs),
WalletsConfig::get_path_with_base_path(&chain_config.configs),
GeneralConfig::get_path_with_base_path(&chain_config.configs),
SecretsConfig::get_path_with_base_path(&chain_config.configs),
ContractsConfig::get_path_with_base_path(&chain_config.configs),
vec![],
args.tag,
)
.await
.context(MSG_FAILED_TO_RUN_SERVER_ERR)
Expand Down
Loading