Ehtereum<->Centauri integration test; hyperspace-ethereum CLI (#51)

* scripts: add docker-compose to execute ethereum devnet

* add cosmos chain

* Diamond pattern (EIP-2535)

Use [ERC-2535: Diamonds, Multi-Facet Proxy](https://eips.ethereum.org/EIPS/eip-2535) to make IBC contracts more flexible in terms of upgradability and deployment

Instead of one contract (OwnableIBCHandler) that inherits all other contract handlers (IBCClientHandler, IBCConnectionHandler, ...), we now deploy each of the handlers separately and register their functions in the Diamond contract. It allows to deploy as many contracts as possible not depending on the code size anymore

* Improve ICS Ethereum light client integration

Refactored the light-clients/icsxx-ethereum and hyperspace/ethereum modules to improve ICS Ethereum light client integration. The integration tests are updated to accommodate these changes. Also, a new module 'verify' was added to the project to streamline verifying processes. The hyperspace module's dependencies and Cargo.toml file also saw updates. These changes were necessary to refine the function and increase efficiency in the codebase, making it easier to manage and navigate through. Finally, an empty file named jwt.rs got added to manage Json Web Tokens but awaits population.

* Add Ethereum-Cosmos tests and correctly decode light client state

New test file `ethereum_cosmos.rs` was added under `hyperspace/testsuite/tests` to test integration of Ethereum and Cosmos. Fixed light client state decoding in Ethereum `ibc_provider.rs`. Update client event creation was included along with a create client event handler. Further, additional log filtering for key Ethereum events useful for the IBC (Inter-Blockchain Communication) was added.

Additionally, couple of methods were moved out of the `#[cfg(test)]` normalization blocks, making them available outside of testing. Some TODO comments were also addressed, that mainly handled function implementations and debugging of values.

The integration tests include behaviors such as setting up clients, connection and channel setup, tests with different packet timeouts and connection delays.

* Remove deprecated IBCClientHandler and repurpose ICS23 light client

ICS23 light client was repurposed to handle chain state retrieval and updating.
The IBCClientHandler contract, which used ICS23 light client, is removed due to redundancy.
IBCClientHandler was initially used for delegate calls to contracts implementing `IIBCClient`.
However, with architectural improvements and updates, this operation is no longer necessary, hence removing the contract for better lean code management.
Subsequently, the ICS23 client is equipped with prefixing and key formatting methods to retrieve consensus data effectively.
This enhances the performance of the IBC client and simplifies operations involving Ethermint and Substrate light clients.

In addition to the file changes above:
- Introduced the function parse_ethereum_events to handle parsing of Ethereum events.
- Functions on ibc_provider.rs have been refactored to accommodate the above changes.
- Improved error handling and log message content.
- Updated address indexing in events for better searchability.
- Tests have been improved to suit the updated codebase.

* disable verification

* return client error in eth relayer instead of palic or unwrap. (#48)

* "Add dynamic handling of client consensus state and improve logging"

This commit adds dynamic handling of client consensus state in ibc_provider.rs and improves logging across different modules. Previously, for certain IBC events the provider was not returning if the Ethereum IBC event was not found, leading to undesired behaviors. Now, after each IBC event processing, an appropriate response is always returned, improving the robustness. Additionally, incorrect log entries have been amended, while superfluous log entries have been streamlined, enhancing debugging capabilities. Finally, erroneous static client consensus handling has been replaced by dynamic queries, increasing flexibility and correctness.

* Use client error instead of unwrap and expect in ibc provider (#49)

* use client error in ibc provider

* more replacement of unwrap to ClientError

* replace in query_client_update_time_and_height

* remove few unwrap and use ClientError

* update 1 more fn query_client_id_from_tx_hash to replace unwrap

* more changes to return ClientError instead of unwrap or expect

* remove all panic, unwrap and expect and return ClientError ibc provider

* Improve Ethereum provider functionality and update dependencies

This update includes several changes to improve the functionality of the Ethereum provider, as well as an update of dependencies. Specific changes include the removal of unnecessary log info calls and the addition of debug level logging to capture events. Additionally, the calculation of the 'from' and 'to' values for block processing has been streamlined. Dependencies for the Ethereum provider, such as 'anyhow', have also been updated to the latest versions. The update also includes a fix to ensure that the mock Ethereum client has the correct commitment sequences.

This was necessary to ensure the Ethereum provider operates optimally, with relevant debug info being captured. The dependency updates ensure compatibility and security, and the mock client fix ensures accurate testing results.

* update sync-committee deps

* add script for generating ABI and update ABI

* Add commands for ethereum client

* add a default config

* remove `prev_state` from client

* update configs; clarify jwtsecret not found error

* Client error across hyperspace eth (#50)

* update chain.rs to provide more info into client error

* update client_state_from_abi_token, replaced all unwrap

* update consensus_state_from_abi_token

* update tm_header_from_abi_token

---------

Signed-off-by: Vladislav <[email protected]>
Co-authored-by: Vladislav <[email protected]>

* disable parachain chains

* disable storage layout

* generate ics20-bank ABI

* add a check for empty client state

* little refactor

* Remove unused BLS verification related code

---------

Signed-off-by: Vladislav <[email protected]>
Co-authored-by: Blas Rodriguez Irizar <[email protected]>
Co-authored-by: rust.dev <[email protected]>

algorithms/beefy/prover/Cargo.toml

@@ -8,7 +8,7 @@ authors = ["Seun Lanlege <[email protected]>", "David Salami <wizdave97@gmai
 targets = ["x86_64-unknown-linux-gnu"]
 
 [build-dependencies]
-anyhow = "1.0.66"
+anyhow = "1.0.75"
 once_cell = "1.16.0"
 subxt-codegen = { package = "codegen", path = "../../../utils/subxt/codegen" }
 tokio = { version = "1.19.2", features = ["macros", "rt-multi-thread"] }

config/centauri-local.toml

@@ -0,0 +1,19 @@
+type = "cosmos"
+name = "centauri"
+rpc_url = "http://127.0.0.1:26657/"
+grpc_url = "http://127.0.0.1:9090/"
+websocket_url = "ws://127.0.0.1:26657/websocket"
+chain_id = "centauri-testnet-1"
+client_id = "07-tendermint-0"
+connection_id = "connection-18"
+account_prefix = "centauri"
+fee_denom = "stake"
+fee_amount = "4000"
+gas_limit = 9223372036854775806
+store_prefix = "ibc"
+max_tx_size = 200000
+wasm_code_id = "4c7504846b7c35e1b7dfc62f9347fcb98fec95aecede9822a745ce14d6973ced"
+channel_whitelist = [["channel-8", "transfer"]]
+mnemonic = "sense state fringe stool behind explain area quit ugly affair develop thumb clinic weasel choice atom gesture spare sea renew penalty second upon peace"
+skip_optional_client_updates = true
+max_packets_to_process = 200

config/ethereum-local.toml

@@ -0,0 +1,17 @@
+type = "ethereum"
+http_rpc_url = "http://localhost:8545/"
+ws_rpc_url = "ws://127.0.0.1:8546/"
+beacon_rpc_url = "http://localhost:3500/"
+private_key_path = "hyperspace/ethereum/keys/0x73db010c3275eb7a92e5c38770316248f4c644ee"
+max_block_weight = 1
+name = "ethereum_client"
+client_id = "08-wasm-22"
+connection_id = "connection-0"
+channel_whitelist = [["channel-0", "transfer"]]
+commitment_prefix = "6962632f"
+diamond_address = "0x4a976e25cde9d022007f7f02889901b382038237"
+tendermint_address = "0xa6915519ec2cbc4e7eea7912673c885fc1f156ca"
+bank_address = "0x87b47d11cdc84e3c3cb70b34dde1235c0862b68e"
+diamond_facets = [["IBCClient", "0x10b57a97dad42b87185f39fc653939c89abda030"], ["IBCConnection", "0x163f4a54af24d13151f231fb74ef5cbc642d05a7"], ["IBCChannelHandshake", "0x5078d1a1240bcbca0f67ba25f07c66b85ab86d37"], ["IBCPacket", "0xc0a2f84324ddf94f4d5af8b507ffefb9016fcdaa"], ["IBCQuerier", "0x98f6789f22e1066dcd128ca99a859acf4821f834"], ["DiamondCutFacet", "0xc1e3b960d3db95ea5a4f2b7c91ba28b2222d5484"], ["DiamondLoupeFacet", "0x4dccf8ac3a04b9d7f294047ebc6481ed9d8fbab6"], ["OwnershipFacet", "0x0fce292ea5fbf749d8ef31412dbc8ac5841ba66d"]]
+client_type = "07-tendermint"
+jwt_secret_path = "/Users/vmark/work/eth-pos-devnet/execution/jwtsecret"

contracts/pallet-ibc/src/light_clients.rs

@@ -51,8 +51,6 @@ use icsxx_ethereum::{
 	client_state::ETHEREUM_CLIENT_STATE_TYPE_URL,
 	consensus_state::ETHEREUM_CONSENSUS_STATE_TYPE_URL,
 };
-#[cfg(feature = "ethereum")]
-use icsxx_ethereum::{BlsVerify, EthereumError, EthereumPublicKey, EthereumSignature};
 use prost::Message;
 use sp_core::{crypto::ByteArray, ed25519, H256};
 use sp_runtime::{
@@ -221,17 +219,6 @@ impl beefy_client_primitives::HostFunctions for HostFunctionsManager {
 	}
 }
 
-#[cfg(feature = "ethereum")]
-impl BlsVerify for HostFunctionsManager {
-	fn verify(
-		_public_keys: &[&EthereumPublicKey],
-		_msg: &[u8],
-		_signature: &EthereumSignature,
-	) -> Result<(), EthereumError> {
-		unimplemented!()
-	}
-}
-
 #[derive(Clone, Debug, PartialEq, Eq, ClientDef)]
 pub enum AnyClient {
 	Grandpa(ics10_grandpa::client_def::GrandpaClient<HostFunctionsManager>),
@@ -345,6 +332,13 @@ impl AnyConsensusState {
 			inner: Box::new(inner),
 		}))
 	}
+
+	pub fn unpack_recursive(&self) -> &Self {
+		match self {
+			Self::Wasm(wasm_state) => wasm_state.inner.unpack_recursive(),
+			c => c,
+		}
+	}
 }
 
 #[derive(Clone, Debug, ClientMessage)]
@@ -389,7 +383,8 @@ impl AnyClientMessage {
 			},
 			#[cfg(feature = "ethereum")]
 			Self::Ethereum(m) => match m {
-				icsxx_ethereum::client_message::ClientMessage::Header(h) => None, /* TODO: header height */
+				icsxx_ethereum::client_message::ClientMessage::Header(h) =>
+					Some(Height::new(0, h.inner.execution_payload.block_number)),
 				icsxx_ethereum::client_message::ClientMessage::Misbehaviour(_) => None,
 			},
 			#[cfg(test)]
@@ -548,10 +543,20 @@ impl From<AnyClientMessage> for Any {
 				value: msg.encode_vec().expect("encode_vec failed"),
 			},
 			#[cfg(feature = "ethereum")]
-			AnyClientMessage::Ethereum(msg) => Any {
-				type_url: ETHEREUM_CLIENT_MESSAGE_TYPE_URL.to_string(),
-				value: msg.encode_vec().expect("encode_vec failed"),
+			AnyClientMessage::Ethereum(msg) => match msg {
+				icsxx_ethereum::client_message::ClientMessage::Header(h) => Any {
+					type_url: ETHEREUM_HEADER_TYPE_URL.to_string(),
+					value: h.encode_vec().expect("encode_vec failed"),
+				},
+				icsxx_ethereum::client_message::ClientMessage::Misbehaviour(m) => Any {
+					type_url: ETHEREUM_MISBEHAVIOUR_TYPE_URL.to_string(),
+					value: m.encode_vec().expect("encode_vec failed"),
+				},
 			},
+			// AnyClientMessage::Ethereum(msg) => Any {
+			// 	type_url: ETHEREUM_CLIENT_MESSAGE_TYPE_URL.to_string(),
+			// 	value: msg.encode_vec().expect("encode_vec failed"),
+			// },
 			#[cfg(test)]
 			AnyClientMessage::Mock(_msg) => panic!("MockHeader can't be serialized"),
 		}

hyperspace/Cargo.toml

@@ -6,7 +6,7 @@ authors = ["Seun Lanlege <[email protected]>", "David Salami <wizdave97@gmai
 
 [dependencies]
 tokio = { version = "1.19.2", features = ["macros", "sync", "time", "rt-multi-thread"] }
-clap = { version = "3.2.22", features = ["derive"] }
+clap = { version = "4.4.2", features = ["derive"] }
 anyhow = "1.0.66"
 hyperspace-core = { path = "./core" }
 toml = "0.7.3"

hyperspace/core/Cargo.toml

@@ -28,7 +28,7 @@ env_logger = "0.9.0"
 hex = "0.4.3"
 tokio = { version = "1.19.2", features = ["macros", "rt-multi-thread", "fs"] }
 codec = { version = "3.0.0", package = "parity-scale-codec" }
-clap = { version = "3.2.22", features = ["derive"] }
+clap = { version = "4.4.2", features = ["derive"] }
 toml = "0.7.3"
 serde = "1.0.144"
 thiserror = "1.0.31"
@@ -57,7 +57,7 @@ sp-keystore = { git = "https://github.com/paritytech/substrate", branch = "polka
 sp-core = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.39" }
 sp-beefy = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.39" }
 subxt = { git = "https://github.com/paritytech/subxt",  rev = "2a4da618a033bb82f768e4ef67b093b371f8b492", features = ["substrate-compat"] }
-subxt-generated = { path = "../../utils/subxt/generated" }
+subxt-generated = { path = "../../utils/subxt/generated", optional = true }
 frame-system = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.39" }
 frame-support = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.39", default-features = false }
 prost = { version = "0.11", default-features = false }
@@ -79,6 +79,7 @@ sp-keyring = { git = "https://github.com/paritytech/substrate", branch = "polkad
 build-metadata-from-ws = []
 #near = ["dep:near"]
 cosmos = ["dep:cosmos"]
+parachain = ["subxt-generated"]
 ethereum = ["dep:ethereum"]
 testing = ["primitives/testing", "parachain/testing", "cosmos/testing"]
 default = ["cosmos", "ethereum"]

hyperspace/core/src/chain.rs

@@ -14,18 +14,15 @@
 
 #![allow(unreachable_patterns)]
 
-use crate::{
-	chains,
-	substrate::{
-		default::DefaultConfig, ComposableConfig, PicassoKusamaConfig, PicassoRococoConfig,
-	},
-};
+use crate::chains;
 use async_trait::async_trait;
 #[cfg(feature = "cosmos")]
 use cosmos::client::{CosmosClient, CosmosClientConfig};
 #[cfg(feature = "ethereum")]
 use ethereum::client::EthereumClient;
 #[cfg(feature = "ethereum")]
+use ethereum::cmd::EthereumCmd;
+#[cfg(feature = "ethereum")]
 use ethereum::config::EthereumClientConfig;
 use futures::Stream;
 #[cfg(any(test, feature = "testing"))]
@@ -96,15 +93,18 @@ impl From<String> for AnyError {
 }
 
 chains! {
+	#[cfg(feature = "parachain")]
 	Parachain(ParachainClientConfig, ParachainClient<DefaultConfig>),
-	// Dali(ParachainClientConfig, ParachainClient<DaliConfig>),
+	#[cfg(feature = "parachain")]
 	Composable(ParachainClientConfig, ParachainClient<ComposableConfig>),
+	#[cfg(feature = "parachain")]
 	PicassoRococo(ParachainClientConfig, ParachainClient<PicassoRococoConfig>),
+	#[cfg(feature = "parachain")]
 	PicassoKusama(ParachainClientConfig, ParachainClient<PicassoKusamaConfig>),
 	#[cfg(feature = "cosmos")]
-	Cosmos(CosmosClientConfig, CosmosClient<DefaultConfig>),
+	Cosmos(CosmosClientConfig, CosmosClient<()>),
 	#[cfg(feature = "ethereum")]
-	Ethereum(EthereumClientConfig, EthereumClient),
+	Ethereum(EthereumClientConfig, EthereumClient, EthereumCmd),
 }
 
 fn wrap_any_msg_into_wasm(msg: Any, code_id: Bytes) -> Result<Any, anyhow::Error> {

hyperspace/core/src/command.rs

@@ -13,7 +13,7 @@
 // limitations under the License.
 
 use crate::{
-	chain::{AnyConfig, Config, CoreConfig},
+	chain::{AnyCmd, AnyConfig, Config, CoreConfig},
 	fish, relay, Mode,
 };
 use anyhow::{anyhow, Result};
@@ -51,6 +51,13 @@ pub enum Subcommand {
 	CreateConnection(Cmd),
 	#[clap(name = "create-channel", about = "Creates a channel on the specified port")]
 	CreateChannel(Cmd),
+	#[clap(name = "client", about = "Client commands")]
+	Client {
+		#[command(subcommand)]
+		client: AnyCmd,
+		#[command(flatten)]
+		cmd: Cmd,
+	},
 }
 
 #[derive(Debug, Clone, Parser)]

hyperspace/core/src/events.rs

@@ -100,11 +100,11 @@ pub async fn parse_events(
 						Height::new(proof_height.revision_number, proof_height.revision_height);
 					let client_state_proof =
 						CommitmentProofBytes::try_from(client_state_response.proof).ok();
-
 					let client_state = client_state_response
 						.client_state
 						.map(AnyClientState::try_from)
 						.ok_or_else(|| Error::Custom("Client state is empty".to_string()))??;
+					log::info!(target: "hyperspace_ethereum", "ph={proof_height}, client_state.latest_height()={}", client_state.latest_height());
 					let consensus_proof = source
 						.query_client_consensus(
 							open_init.height(),
@@ -288,8 +288,7 @@ pub async fn parse_events(
 								channel_id,
 								open_init.port_id.clone()
 							))
-						})?)
-						.expect("Channel end decoding should not fail");
+						})?)?;
 					let counterparty = channel_end.counterparty();
 
 					let connection_response = source

hyperspace/core/src/lib.rs

@@ -21,6 +21,7 @@ pub mod logging;
 mod macros;
 pub mod packets;
 pub mod queue;
+#[cfg(feature = "parachain")]
 pub mod substrate;
 mod utils;
 

hyperspace/core/src/macros.rs

@@ -16,7 +16,7 @@
 macro_rules! chains {
 	($(
         $(#[$($meta:meta)*])*
-		$name:ident($config:path, $client:path),
+		$name:ident($config:path, $client:path$(, $cmd:path)?),
 	)*) => {
 		#[derive(Debug, Serialize, Deserialize, Clone)]
 		#[serde(tag = "type", rename_all = "snake_case")]
@@ -45,7 +45,7 @@ macro_rules! chains {
 			)*
 		}
 
-		#[derive(Clone)]
+		#[derive(Clone, Debug)]
 		pub enum AnyAssetId {
 			$(
 				$(#[$($meta)*])*
@@ -72,6 +72,17 @@ macro_rules! chains {
 			Other(String),
 		}
 
+		#[derive(Debug, Clone, clap::Parser)]
+		pub enum AnyCmd {
+			$(
+				$(#[$($meta)*])*
+				$(
+				#[command(subcommand)]
+				$name($cmd),
+				)?
+			)*
+		}
+
 		impl From<anyhow::Error> for AnyError {
 			fn from(e: anyhow::Error) -> Self {
 				Self::Other(e.to_string())
@@ -1042,7 +1053,7 @@ macro_rules! chains {
 				let maybe_code_id = match self {
 					$(
 						$(#[$($meta)*])*
-						Self::$name(chain) => Option::<CodeId>::None,//chain.wasm_code_id.as_ref(),
+						Self::$name(chain) => chain.wasm_code_id.as_ref(),
 					)*
 				};
 				let maybe_code_id =
@@ -1056,7 +1067,7 @@ macro_rules! chains {
 					$(
 						$(#[$($meta)*])*
 						Self::$name(chain) => {
-							// chain.wasm_code_id = Some(code_id);
+							chain.wasm_code_id = Some(code_id);
 						},
 					)*
 				}

hyperspace/core/src/packets.rs

@@ -37,7 +37,7 @@ use ibc::{
 	},
 	Height,
 };
-use ibc_proto::google::protobuf::Any;
+use ibc_proto::{google::protobuf::Any, ibc::core::channel::v1::QueryNextSequenceReceiveResponse};
 use pallet_ibc::light_clients::AnyClientState;
 use primitives::{
 	error::Error, find_suitable_proof_height_for_client, packet_info_to_packet,
@@ -133,9 +133,15 @@ pub async fn query_ready_and_timed_out_packets(
 			},
 		};
 
-		let next_sequence_recv = sink
-			.query_next_sequence_recv(sink_height, &sink_port_id, &sink_channel_id)
-			.await?;
+		// TODO: query_next_sequence_recv
+		// let next_sequence_recv = sink
+		// 	.query_next_sequence_recv(sink_height, &sink_port_id, &sink_channel_id)
+		// 	.await?;
+		let next_sequence_recv = QueryNextSequenceReceiveResponse {
+			next_sequence_receive: 0,
+			proof: vec![1],
+			proof_height: None,
+		};
 
 		let source_client_state_on_sink =
 			sink.query_client_state(sink_height, source.client_id()).await?;
@@ -229,6 +235,9 @@ pub async fn query_ready_and_timed_out_packets(
 						Error::Custom(format!("Packet height not found for packet {packet:?}"))
 					})?;
 
+
+				    log::info!("sink_height = {sink_height:?}, timeout_height = {:?}", packet.timeout_height);
+
 					if packet.timed_out(&sink_timestamp, sink_height) {
 						timeout_packets_count.fetch_add(1, Ordering::SeqCst);
 						// so we know this packet has timed out on the sink, we need to find the maximum
@@ -306,7 +315,7 @@ pub async fn query_ready_and_timed_out_packets(
 					// creation height on source chain
 					if packet_height > latest_source_height_on_sink.revision_height {
 						// Sink does not have client update required to prove recv packet message
-						log::debug!(target: "hyperspace", "Skipping packet {:?} as sink does not have client update required to prove recv packet message", packet);
+						log::debug!(target: "hyperspace", "Skipping packet as sink does not have client update required to prove recv packet message: {:?}", packet);
 						recv_packets_count.fetch_add(1, Ordering::SeqCst);
 						return Ok(None)
 					}
@@ -324,7 +333,7 @@ pub async fn query_ready_and_timed_out_packets(
 					{
 						proof_height
 					} else {
-						log::trace!(target: "hyperspace", "Skipping packet {:?} as no proof height could be found", packet);
+						log::trace!(target: "hyperspace", "Skipping packet as no proof height could be found: {:?}", packet);
 						return Ok(None)
 					};
 

hyperspace/cosmos/src/client.rs

@@ -401,12 +401,13 @@ where
 		let mut xs = Vec::new();
 		let heightss = (from.value()..=to.value()).collect::<Vec<_>>();
 		let client = Arc::new(self.clone());
-		let to = self.rpc_call_delay().as_millis();
+		let delay_to = self.rpc_call_delay().as_millis();
 		for heights in heightss.chunks(5) {
 			let mut join_set = JoinSet::<Result<Result<_, Error>, Elapsed>>::new();
 			for height in heights.to_owned() {
 				let client = client.clone();
-				let duration = Duration::from_millis(rand::thread_rng().gen_range(0..to) as u64);
+				let duration =
+					Duration::from_millis(rand::thread_rng().gen_range(0..delay_to) as u64);
 				let fut = async move {
 					log::trace!(target: "hyperspace_cosmos", "Fetching header at height {:?}", height);
 					let latest_light_block =
@@ -430,7 +431,6 @@ where
 						true => UpdateType::Optional,
 						false => UpdateType::Mandatory,
 					};
-
 					Ok((
 						Header {
 							signed_header: latest_light_block.signed_header,

hyperspace/cosmos/src/provider.rs

@@ -308,6 +308,9 @@ where
 			Path::ClientState(ClientStatePath(client_id.clone())).to_string().into_bytes();
 		let (q, proof) = self.query_path(path_bytes.clone(), at, true).await?;
 		let client_state = Any::decode(&*q.value)?;
+		if client_state.type_url.is_empty() || client_state.value.is_empty() {
+			return Err(Error::Custom(format!("empty client state for height {at}")))
+		}
 		Ok(QueryClientStateResponse {
 			client_state: Some(client_state),
 			proof,