Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node state for the RPC server #67

Merged
merged 154 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
154 commits
Select commit Hold shift + click to select a range
6bd6170
Add dockerfile for executable node
IAvecilla Jan 10, 2024
b2fb9ab
Add compose file for testing purposes
IAvecilla Jan 10, 2024
e768bc6
Add entrypoint for node consensus dockerfile
IAvecilla Jan 12, 2024
e6b6487
Add some comments and improve compose test file
IAvecilla Jan 12, 2024
4ca7fdc
Add new makefile commands to run dockerized consensus node
IAvecilla Jan 12, 2024
28df58b
Update readme with docs to run consensus node in docker
IAvecilla Jan 12, 2024
3e4b881
Delete unnecesary building dependency in compose file
IAvecilla Jan 15, 2024
febcafc
Rename docker image
IAvecilla Jan 15, 2024
b2e7e9c
Set container names manually in compose file
IAvecilla Jan 15, 2024
5e27e6e
Separate config directory for nodes running in docker
IAvecilla Jan 15, 2024
6e2236a
Fix node configuration for docker consensus example
IAvecilla Jan 15, 2024
810a4b5
Improve command to run a node in a container
IAvecilla Jan 15, 2024
4088c0a
Generate the node configs in release mode
IAvecilla Jan 15, 2024
4475489
Fix docker cleanup to force deletion
IAvecilla Jan 15, 2024
689b6d5
Remove unnecesary copies to container
IAvecilla Jan 15, 2024
99dc691
Add target dir to docker ignore
IAvecilla Jan 16, 2024
a3b270d
Move every docker related config file to the project root
IAvecilla Jan 16, 2024
f93890f
Fix typo in README
IAvecilla Jan 17, 2024
f3893cf
Fix typo in makefile command
IAvecilla Jan 17, 2024
1c721c8
Make the path to makefile be the same for local and docker
IAvecilla Jan 17, 2024
8cac863
disabled clipply lint
pompon0 Jan 17, 2024
d24c8bc
fixed lint, updated deps
pompon0 Jan 17, 2024
9520c4c
Change file name for the docker entrypoint and add comment to the script
IAvecilla Jan 18, 2024
54c72ce
Move version to latest for rust image
IAvecilla Jan 18, 2024
13e61e1
Change name of command and dir generation for node config
IAvecilla Jan 18, 2024
3b8d299
Add command to stop dockerized nodes
IAvecilla Jan 18, 2024
2d09c1d
Add example file with local address for node configuration
IAvecilla Jan 18, 2024
c287c56
Update README with new updates
IAvecilla Jan 18, 2024
e66cec9
Write a general overview on the README
IAvecilla Jan 18, 2024
eb7077e
Add tonic as dependency
IAvecilla Jan 19, 2024
058418e
Create test proto file with helathcheck endpoint
IAvecilla Jan 19, 2024
f323873
Add basic rpc server implementation
IAvecilla Jan 19, 2024
d0d9b13
Add basic rpc client implementation to use with CLI
IAvecilla Jan 19, 2024
a332327
Run RPC server for the standalone node
IAvecilla Jan 19, 2024
8e2d83c
Update docker config files to expose rpc server
IAvecilla Jan 19, 2024
96e18b7
Replace tonic crate for jsonrpsee
IAvecilla Jan 23, 2024
6b57f2a
Delete protobuff related files
IAvecilla Jan 23, 2024
c0aaa9b
Delete unnecesary RPC client
IAvecilla Jan 23, 2024
88a5b23
Add new jsonrpsee server and health endpoint
IAvecilla Jan 23, 2024
890bdee
Add inner docs for new rpc module
IAvecilla Jan 23, 2024
8f806bc
quick and dirty k8s deployment scripts for test framework
ElFantasma Jan 23, 2024
7592041
Merge branch 'main' into rpc_test_layer
IAvecilla Jan 23, 2024
23b8ea4
Merge branch 'rpc_test_layer' into k8s_deploy_script
IAvecilla Jan 23, 2024
438a2b8
Change name for generated config folders
IAvecilla Jan 24, 2024
19d7fcc
Update localnet config bin
IAvecilla Jan 24, 2024
b49af96
Delete old docker entrypoint file
IAvecilla Jan 24, 2024
2fd8061
Update k8s deployment manifest adding node env var
IAvecilla Jan 24, 2024
758ff9e
Add new stage in dockerfile to generate executor binary
IAvecilla Jan 25, 2024
addacc8
Build correct target in compose file
IAvecilla Jan 25, 2024
e263eb6
Update Makefile with all new targets
IAvecilla Jan 25, 2024
9503f05
Add addresses example file for docker config
IAvecilla Jan 25, 2024
8bcdb88
Remove docker config dir from clean command
IAvecilla Jan 25, 2024
2323211
Added some utility targets to Makefile
ElFantasma Jan 25, 2024
2720e74
Make rpc port a CLI argument for the executor
IAvecilla Jan 25, 2024
af1caaa
Change param type to run the server
IAvecilla Jan 25, 2024
77591f3
Merge branch 'rpc_test_layer' into k8s_deploy_script
IAvecilla Jan 25, 2024
24cddea
Add config generation for k8s node deployer
IAvecilla Jan 25, 2024
2cfaff4
Add makefile target to clean k8s config
IAvecilla Jan 25, 2024
4ed83cc
Improve config generation struct
IAvecilla Jan 26, 2024
00a008a
Use new config app functions to build localent and k8s config
IAvecilla Jan 26, 2024
4ce804b
Add kube as dependency for tools crate
IAvecilla Jan 26, 2024
b723715
Fix node config generation
IAvecilla Jan 29, 2024
83589a1
Add k8s-openapi as dependency
IAvecilla Jan 29, 2024
6248277
Add subcommands to generate config and deploy pods
IAvecilla Jan 29, 2024
87d9def
Override entrypoint to run node in k8s pod
IAvecilla Jan 29, 2024
251052f
Add makefile target to run k8s pods
IAvecilla Jan 29, 2024
09484fd
Merge branch 'main' into rpc_test_layer
IAvecilla Jan 30, 2024
ac145ee
Merge branch 'rpc_test_layer' into k8s_deploy_script
IAvecilla Jan 30, 2024
10ba6d6
Update nodes argument type to generate nodes config
IAvecilla Jan 30, 2024
991a8ac
Fix deployer config generation after merge
IAvecilla Jan 30, 2024
926b387
Fix clippy and format warnings
IAvecilla Jan 30, 2024
5fca6b8
Improve RPC server API to scale it easier
IAvecilla Jan 30, 2024
0726aff
Fix concurrency management for rpc server
IAvecilla Jan 30, 2024
21fb1c6
Add missing docs to solve clippy warnings
IAvecilla Jan 30, 2024
41ec845
Revert change for executor new flag
IAvecilla Jan 30, 2024
7ba8cb3
shutting down server when context is cancelled
pompon0 Jan 31, 2024
d859f39
Removed k8s yml files and modularized kube.rs access
ElFantasma Feb 1, 2024
95ec2b7
Update consensus docker example with new rpc ports
IAvecilla Feb 1, 2024
d697cc1
Add new endpoint to get peers of the node
IAvecilla Feb 1, 2024
7fe6ed9
Fix format and linter
IAvecilla Feb 1, 2024
57f600a
Update cargo.lock
IAvecilla Feb 1, 2024
8ffa0c0
Delete unnecesary features in added crates
IAvecilla Feb 1, 2024
44c979e
Added k8s module
ElFantasma Feb 1, 2024
5257592
Add old crate versions to deny.toml file
IAvecilla Feb 2, 2024
75cd2cf
Use new app config api for test in tools crate
IAvecilla Feb 2, 2024
958954f
Merge branch 'rpc_test_layer' into k8s_deploy_script
IAvecilla Feb 2, 2024
b0803dc
Deploying seed peers in a different phase
ElFantasma Feb 5, 2024
3a249dc
Retrieving seed peers IPs and using it in the rest of the peers
ElFantasma Feb 5, 2024
eb7512a
Add necessary features in kube and jsonrpsee dependencies
IAvecilla Feb 6, 2024
b587174
Deploy service to communicate with k8s pods
IAvecilla Feb 6, 2024
0f1e075
Add sanity test to check node is running in a pod
IAvecilla Feb 6, 2024
3ba75b1
Add service deployment to deployer binary
IAvecilla Feb 6, 2024
8e49847
Add bash script for setup and test run
IAvecilla Feb 6, 2024
ea6c3b9
Make executor argument optional
IAvecilla Feb 6, 2024
d612889
Busy waiting for pods to start to obtain their IPs
ElFantasma Feb 7, 2024
75b21ed
Add makefile targets to run tests inside kubernetes
IAvecilla Feb 8, 2024
5d599a8
Add new binary to run test scenarios
IAvecilla Feb 8, 2024
cf48dff
Add Dockerfile and kubernetes config files
IAvecilla Feb 8, 2024
7b5d264
Fix format
IAvecilla Feb 8, 2024
6bd5394
Revert change to have node ips as directory names for local config
IAvecilla Feb 8, 2024
1d58431
Making cli argument optional and forwarding arguments in entrypoint …
ElFantasma Feb 9, 2024
960ff13
Merge branch 'main' into k8s_deploy_script
ElFantasma Feb 9, 2024
c864f51
Fixed compiler warnings
ElFantasma Feb 9, 2024
b5b5ee6
Consensus node now can obtain it's public address from an ENV VAR. Al…
ElFantasma Feb 14, 2024
702a587
Added config RPC endpoint
ElFantasma Feb 15, 2024
025203f
Merge branch 'k8s_deploy_script' into k8s_node_communication_test
IAvecilla Feb 15, 2024
1ef6877
Corrected k8s_entrypoint.sh script to forward cli arguments
ElFantasma Feb 15, 2024
8b9d149
Update config files to run tester in k8s pod
IAvecilla Feb 15, 2024
0d1511d
Make tester a CLI tool to generate config and run the tests
IAvecilla Feb 15, 2024
91632d0
Update Makefile with new tester commands
IAvecilla Feb 15, 2024
b97e025
Add clap and tracing dependencies to the tester crate
IAvecilla Feb 15, 2024
1a2dc84
Remove service creation
IAvecilla Feb 15, 2024
aeae5fb
Ignore binaries for bin tools generated in docker
IAvecilla Feb 15, 2024
61fa5b7
Move makefile targets of tester to new makefile inside that crate
IAvecilla Feb 15, 2024
6480c23
Fix tester entrypoint to run the tests
IAvecilla Feb 15, 2024
636ab28
Add function to deploy pod for tests inside rust
IAvecilla Feb 15, 2024
a15b7bf
Update Dockerfile with new commands
IAvecilla Feb 15, 2024
1ae8e6b
Add readme to run tests pod
IAvecilla Feb 15, 2024
85b66d0
Delete outdated kubernetes yaml and dockerfile
IAvecilla Feb 15, 2024
5e8b011
Several corrections on naming and coding style
ElFantasma Feb 15, 2024
6ef448e
Fix missing documentation
IAvecilla Feb 16, 2024
4cf876f
Fix deny toml file
IAvecilla Feb 16, 2024
c62f0b4
Fixed deny.toml for kube.rs
ElFantasma Feb 16, 2024
2e89af8
Fixed clippy suggestions
ElFantasma Feb 19, 2024
8c2f79b
Merge branch 'k8s_deploy_script' into k8s_node_communication_test
IAvecilla Feb 19, 2024
fb0c345
Added amount of seed nodes as cli argument
ElFantasma Feb 19, 2024
3b1256f
Merge branch 'main' into k8s_deploy_script
ElFantasma Feb 19, 2024
23a7761
Added brief description for k8s deployment in README.md
ElFantasma Feb 20, 2024
7dbf270
Unified node_name and node_id values
ElFantasma Feb 20, 2024
53b805d
Add node storage as parameter for the rpc server
IAvecilla Feb 20, 2024
c72b9ca
Create new endpoint to check last view of the node
IAvecilla Feb 20, 2024
7f489fe
Merge branch 'k8s_deploy_script' into k8s_node_communication_test
IAvecilla Feb 21, 2024
9b309c4
Remove unwraps and improve error handling
IAvecilla Feb 21, 2024
f15d2bf
Change hardcoded deployment json to deployment strcut
IAvecilla Feb 21, 2024
04f1a2f
Remove unwraps in k8s module
IAvecilla Feb 21, 2024
93824c0
Add anyhow as dependency for tests crate
IAvecilla Feb 21, 2024
be03154
Get pods from consensus docker image
IAvecilla Feb 22, 2024
7e4c33d
Improve error logging for the tester
IAvecilla Feb 22, 2024
573d238
Do general cleanup of k8s new functions
IAvecilla Feb 22, 2024
abab39a
Use of filter_map and add context to errors
IAvecilla Feb 22, 2024
9bf46f5
Clean test-suite image in the clean target
IAvecilla Feb 22, 2024
37c74eb
Add doc comment explaining the unwraps usage
IAvecilla Feb 22, 2024
8b6f54d
Remove unnecesary clones
IAvecilla Feb 22, 2024
473b00c
Merge branch 'k8s_node_communication_test' into rpc_execution_connection
IAvecilla Feb 23, 2024
3442526
Remove RPCMethod trait
IAvecilla Feb 23, 2024
a8364b9
Merge main into rpc_execution_connection
IAvecilla Mar 7, 2024
d6b904a
Merge branch 'main' into rpc_execution_connection
IAvecilla Mar 7, 2024
c366ecc
Update comments and delete serialize for view number
IAvecilla Mar 7, 2024
f3569cd
Delete unnused rpc endpoints
IAvecilla Mar 7, 2024
dc8289b
Add endpoint to get last commited block for a replica
IAvecilla Mar 8, 2024
d37329f
Fix unnecesary iter conversion
IAvecilla Mar 11, 2024
1d72090
Merge branch 'main' into rpc_execution_connection
IAvecilla Mar 15, 2024
b4f6b57
Remove old changes
IAvecilla Mar 15, 2024
11a5950
Merge branch 'main' into rpc_execution_connection
IAvecilla Mar 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
262 changes: 136 additions & 126 deletions node/Cargo.lock

Large diffs are not rendered by default.

12 changes: 4 additions & 8 deletions node/tests/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,9 @@
//! This is a simple test for the RPC server. It checks if the server is running and can respond to.
use anyhow::{ensure, Context};
use clap::{Parser, Subcommand};
use jsonrpsee::{core::client::ClientT, http_client::HttpClientBuilder, rpc_params, types::Params};
use jsonrpsee::{core::client::ClientT, http_client::HttpClientBuilder, rpc_params};
use std::{fs, io::Write, net::SocketAddr, path::PathBuf, str::FromStr};
use zksync_consensus_tools::{
k8s,
rpc::methods::{health_check::HealthCheck, RPCMethod},
};
use zksync_consensus_tools::{k8s, rpc::methods::health_check};

/// Command line arguments.
#[derive(Debug, Parser)]
Expand Down Expand Up @@ -80,12 +77,11 @@ pub async fn sanity_test() {
let socket = SocketAddr::from_str(socket).unwrap();
let url = format!("http://{}", socket);
let rpc_client = HttpClientBuilder::default().build(url).unwrap();
let params = Params::new(None);
let response: serde_json::Value = rpc_client
.request(HealthCheck::method(), rpc_params!())
.request(health_check::method(), rpc_params!())
.await
.unwrap();
assert_eq!(response, HealthCheck::callback(params).unwrap());
assert_eq!(response, health_check::callback().unwrap());
}
}

Expand Down
17 changes: 8 additions & 9 deletions node/tools/src/bin/deployer.rs
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
//! Deployer for the kubernetes cluster.
use clap::Parser;
use std::collections::HashMap;
use zksync_consensus_roles::{node, validator};
use zksync_consensus_roles::node::SecretKey;
use zksync_consensus_roles::validator;
use zksync_consensus_tools::k8s::ConsensusNode;
use zksync_consensus_tools::{k8s, AppConfig};

/// K8s namespace for consensus nodes.
const NAMESPACE: &str = "consensus";

/// Command line arguments.
#[derive(Debug, Parser)]
#[command(name = "deployer")]
Expand Down Expand Up @@ -35,7 +33,7 @@ fn generate_consensus_nodes(nodes: usize, seed_nodes_amount: Option<usize>) -> V
// Each node will have `gossip_peers` outbound peers.
let peers = 2;

let node_keys: Vec<node::SecretKey> = (0..nodes).map(|_| node::SecretKey::generate()).collect();
let node_keys: Vec<SecretKey> = (0..nodes).map(|_| SecretKey::generate()).collect();

let default_config = AppConfig::default_for(setup.genesis.clone());

Expand Down Expand Up @@ -65,7 +63,7 @@ fn generate_consensus_nodes(nodes: usize, seed_nodes_amount: Option<usize>) -> V
async fn deploy(nodes_amount: usize, seed_nodes_amount: Option<usize>) -> anyhow::Result<()> {
let mut consensus_nodes = generate_consensus_nodes(nodes_amount, seed_nodes_amount);
let client = k8s::get_client().await?;
k8s::create_or_reuse_namespace(&client, NAMESPACE).await?;
k8s::create_or_reuse_namespace(&client, k8s::DEFAULT_NAMESPACE).await?;

let seed_nodes = &mut HashMap::new();
let mut non_seed_nodes = HashMap::new();
Expand All @@ -81,12 +79,13 @@ async fn deploy(nodes_amount: usize, seed_nodes_amount: Option<usize>) -> anyhow

// Deploy seed peer(s)
for node in seed_nodes.values_mut() {
node.deploy(&client, NAMESPACE).await?;
node.deploy(&client, k8s::DEFAULT_NAMESPACE).await?;
}

// Fetch and complete node addrs into seed nodes
for node in seed_nodes.values_mut() {
node.fetch_and_assign_pod_ip(&client, NAMESPACE).await?;
node.fetch_and_assign_pod_ip(&client, k8s::DEFAULT_NAMESPACE)
.await?;
}

// Build a vector of (PublicKey, SocketAddr) to provide as gossip_static_outbound
Expand All @@ -106,7 +105,7 @@ async fn deploy(nodes_amount: usize, seed_nodes_amount: Option<usize>) -> anyhow
// Deploy the rest of the nodes
for node in non_seed_nodes.values_mut() {
node.config.gossip_static_outbound.extend(peers.clone());
node.deploy(&client, NAMESPACE).await?;
node.deploy(&client, k8s::DEFAULT_NAMESPACE).await?;
}

Ok(())
Expand Down
4 changes: 2 additions & 2 deletions node/tools/src/k8s.rs
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ pub async fn get_consensus_nodes_address(client: &Client) -> anyhow::Result<Vec<
"No consensus pods found in the k8s cluster"
);
let mut node_rpc_addresses: Vec<SocketAddr> = Vec::new();
for pod in pods.into_iter() {
for pod in pods {
let pod_spec = pod.spec.as_ref().context("Failed to get pod spec")?;
let pod_container = pod_spec
.containers
Expand All @@ -218,7 +218,7 @@ pub async fn get_consensus_nodes_address(client: &Client) -> anyhow::Result<Vec<
.context("Failed to get ports of container")?
.iter()
.find_map(|port| {
let port: u16 = port.container_port.try_into().ok()?;
let port = port.container_port.try_into().ok()?;
(port != config::NODES_PORT).then_some(port)
})
.context("Failed parsing container port")?;
Expand Down
6 changes: 3 additions & 3 deletions node/tools/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -144,9 +144,9 @@ async fn main() -> anyhow::Result<()> {
rpc_addr.set_port(rpc_addr.port() + 100);
}

// cloning configuration to let RPCServer show it
// TODO this should be queried in real time instead, to reflect any possible change in config
let rpc_server = RPCServer::new(rpc_addr, configs.app.clone());
// Create the RPC server with the executor's storage.
let node_storage = executor.block_store.clone();
let rpc_server = RPCServer::new(rpc_addr, node_storage);

// Initialize the storage.
scope::run!(ctx, |ctx, s| async {
Expand Down
47 changes: 0 additions & 47 deletions node/tools/src/rpc/methods/config.rs

This file was deleted.

31 changes: 12 additions & 19 deletions node/tools/src/rpc/methods/health_check.rs
Original file line number Diff line number Diff line change
@@ -1,23 +1,16 @@
//! Health check method for RPC server.
use super::RPCMethod;
use jsonrpsee::types::{error::ErrorCode, Params};
use jsonrpsee::core::RpcResult;

/// Health check method for RPC server.
pub struct HealthCheck;

impl RPCMethod for HealthCheck {
/// Health check response for /health endpoint.
fn callback(_params: Params) -> Result<serde_json::Value, ErrorCode> {
Ok(serde_json::json!({"health": true}))
}
/// Health check response for /health endpoint.
pub fn callback() -> RpcResult<serde_json::Value> {
Ok(serde_json::json!({"health": true}))
}

/// Health check method name.
fn method() -> &'static str {
"health_check"
}
/// Health check method name.
pub fn method() -> &'static str {
"health_check"
}

/// Method path for GET requests.
fn path() -> &'static str {
"/health"
}
/// Method path for GET requests.
pub fn path() -> &'static str {
"/health"
}
34 changes: 34 additions & 0 deletions node/tools/src/rpc/methods/last_commited_block.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
//! Peers method for RPC server.
use anyhow::Context;
use jsonrpsee::{
core::RpcResult,
types::{error::ErrorCode, ErrorObjectOwned},
};
use std::sync::Arc;
use zksync_consensus_storage::BlockStore;

/// Last view response for /last_view endpoint.
pub fn callback(node_storage: Arc<BlockStore>) -> RpcResult<serde_json::Value> {
let sub = &mut node_storage.subscribe();
let state = sub.borrow().clone();
let last_commited_block_header = state
.last
.context("Failed to get last state")
.map_err(|_| ErrorObjectOwned::from(ErrorCode::InternalError))?
.header()
.number
.0;
Ok(serde_json::json!({
"last_commited_block": last_commited_block_header
}))
}

/// Last view method name.
pub fn method() -> &'static str {
"last_commited_block"
}

/// Method path for GET requests.
pub fn path() -> &'static str {
"/last_commited_block"
}
34 changes: 34 additions & 0 deletions node/tools/src/rpc/methods/last_view.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
//! Peers method for RPC server.
use anyhow::Context;
use jsonrpsee::{
core::RpcResult,
types::{error::ErrorCode, ErrorObjectOwned},
};
use std::sync::Arc;
use zksync_consensus_storage::BlockStore;

/// Last view response for /last_view endpoint.
pub fn callback(node_storage: Arc<BlockStore>) -> RpcResult<serde_json::Value> {
let sub = &mut node_storage.subscribe();
let state = sub.borrow().clone();
let last_view = state
.last
.context("Failed to get last state")
.map_err(|_| ErrorObjectOwned::from(ErrorCode::InternalError))?
.view()
.number
.0;
Ok(serde_json::json!({
"last_view": last_view
}))
}

/// Last view method name.
pub fn method() -> &'static str {
"last_view"
}

/// Method path for GET requests.
pub fn path() -> &'static str {
"/last_view"
}
16 changes: 2 additions & 14 deletions node/tools/src/rpc/methods/mod.rs
Original file line number Diff line number Diff line change
@@ -1,15 +1,3 @@
use jsonrpsee::types::{error::ErrorCode, Params};

/// Trait to implement for new RPC methods.
pub trait RPCMethod {
/// Method response logic when called.
fn callback(params: Params) -> Result<serde_json::Value, ErrorCode>;
/// Method name.
fn method() -> &'static str;
/// Method path for GET requests.
fn path() -> &'static str;
}

pub(crate) mod config;
pub mod health_check;
pub(crate) mod peers;
pub mod last_commited_block;
pub mod last_view;
40 changes: 0 additions & 40 deletions node/tools/src/rpc/methods/peers.rs

This file was deleted.

Loading
Loading