Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type info to oracle data #1184

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,14 @@ multiple oracle inputs, this means any programs that were compiled and used need
runtime
- In [#1179](https://github.com/entropyxyz/entropy-core/pull/1179) the format of TDX Quote input data has
been changed.
- In [#1184](https://github.com/entropyxyz/entropy-core/pull/1184/) The ```OracleData``` mapping now holds a
struct ```OracleInfo``` which includes the ```oracle_data``` and the ```oracle_type```
Comment on lines +29 to +30
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For non-code blocks there should only be a single tick used.

Suggested change
- In [#1184](https://github.com/entropyxyz/entropy-core/pull/1184/) The ```OracleData``` mapping now holds a
struct ```OracleInfo``` which includes the ```oracle_data``` and the ```oracle_type```
- In [#1184](https://github.com/entropyxyz/entropy-core/pull/1184/) The `OracleData` mapping now
holds a struct `OracleInfo` which includes the `oracle_data` and the `oracle_type`


### Added
- Protocol message versioning ([#1140](https://github.com/entropyxyz/entropy-core/pull/1140))
- CLI command to get oracle headings ([#1170](https://github.com/entropyxyz/entropy-core/pull/1170))
- Add TSS endpoint to get TDX quote ([#1173](https://github.com/entropyxyz/entropy-core/pull/1173))
- Add type info to oracle data ([#1184](https://github.com/entropyxyz/entropy-core/pull/1184))

### Changed
- Use correct key rotation endpoint in OCW ([#1104](https://github.com/entropyxyz/entropy-core/pull/1104))
Expand Down
Binary file modified crates/client/entropy_metadata.scale
Binary file not shown.
24 changes: 19 additions & 5 deletions crates/client/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ use base64::prelude::{Engine, BASE64_STANDARD};
use entropy_protocol::RecoverableSignature;
use entropy_shared::HashingAlgorithm;
use futures::stream::StreamExt;
use serde::{Deserialize, Serialize};
use sp_core::{
sr25519::{self, Signature},
Pair,
Expand Down Expand Up @@ -446,20 +447,33 @@ pub async fn request_attestation(
Ok(user::request_attestation(api, rpc, attestee).await?)
}

/// Return type for getting oracle headings.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct OracleDataReturn {
pub oracle_heading: String,
pub oracle_type: String,
}
/// Get oracle data headings
/// This is useful for program developers to know what oracle data is available
pub async fn get_oracle_headings(
api: &OnlineClient<EntropyConfig>,
_rpc: &LegacyRpcMethods<EntropyConfig>,
) -> Result<Vec<String>, ClientError> {
rpc: &LegacyRpcMethods<EntropyConfig>,
) -> Result<Vec<OracleDataReturn>, ClientError> {
let storage_address = entropy::storage().oracle().oracle_data_iter();
let mut iter = api.storage().at_latest().await?.iter(storage_address).await?;
let block_hash = rpc
.chain_get_block_hash(None)
.await?
.ok_or(ClientError::ChainFetch("Failed to get block hash"))?;
let mut iter = api.storage().at(block_hash).iter(storage_address).await?;
let mut headings = Vec::new();
while let Some(Ok(kv)) = iter.next().await {
// Key is: storage_address || 128 bit hash || key
let mut input = &kv.key_bytes[32 + 16 + 1..];
let heading = String::decode(&mut input)?;
headings.push(heading);
let oracle_heading = String::decode(&mut input)?;
headings.push(OracleDataReturn {
oracle_heading,
oracle_type: std::str::from_utf8(&kv.value.oracle_type)?.to_string(),
});
}
Ok(headings)
}
2 changes: 2 additions & 0 deletions crates/client/src/errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -123,4 +123,6 @@ pub enum ClientError {
Codec(#[from] parity_scale_codec::Error),
#[error("Attestation request: {0}")]
AttestationRequest(#[from] AttestationRequestError),
#[error("Chain Fetch: {0}")]
ChainFetch(&'static str),
}
11 changes: 7 additions & 4 deletions crates/client/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use crate::{
change_endpoint, change_threshold_accounts, get_oracle_headings, register, remove_program,
request_attestation, store_program,
substrate::query_chain,
update_programs,
update_programs, OracleDataReturn,
};

use entropy_shared::{QuoteContext, QuoteInputData};
Expand Down Expand Up @@ -284,7 +284,10 @@ async fn test_get_oracle_headings() {
current_block = rpc.chain_get_header(Some(finalized_head)).await.unwrap().unwrap().number;
}

let headings = get_oracle_headings(&api, &rpc).await.unwrap();

assert_eq!(headings, vec!["block_number_entropy".to_string()]);
let oracle_headings = get_oracle_headings(&api, &rpc).await.unwrap();
let mock_data = OracleDataReturn {
oracle_heading: "block_number_entropy".to_string(),
oracle_type: "u32".to_string(),
};
assert_eq!(oracle_headings, vec![mock_data]);
}
8 changes: 6 additions & 2 deletions crates/test-cli/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -573,8 +573,12 @@ pub async fn run_command(
}
},
CliCommand::GetOracleHeadings => {
let headings = get_oracle_headings(&api, &rpc).await?;
Ok(serde_json::to_string_pretty(&headings)?)
let oracles_data = get_oracle_headings(&api, &rpc).await?;
if cli.json {
Ok(serde_json::to_string_pretty(&oracles_data)?)
} else {
Ok(format!("{:?}", oracles_data))
}
},
CliCommand::GetTdxQuote { tss_endpoint, output_filename } => {
let quote_bytes =
Expand Down
10 changes: 6 additions & 4 deletions crates/threshold-signature-server/src/helpers/substrate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ use crate::{
entropy::{
self,
runtime_types::{
bounded_collections::bounded_vec::BoundedVec, pallet_programs::pallet::ProgramInfo,
bounded_collections::bounded_vec::BoundedVec, pallet_oracle::module::OracleInfo,
pallet_programs::pallet::ProgramInfo,
},
},
EntropyConfig,
Expand Down Expand Up @@ -70,9 +71,10 @@ pub async fn get_oracle_data(
for program_oracle_data in program_oracle_datas {
let oracle_data_call =
entropy::storage().oracle().oracle_data(BoundedVec(program_oracle_data));
let oracle_info =
query_chain(api, rpc, oracle_data_call, None).await?.unwrap_or(BoundedVec(vec![]));
oracle_infos.push(oracle_info.0);
let oracle_info = query_chain(api, rpc, oracle_data_call, None)
.await?
.unwrap_or(OracleInfo { oracle_data: BoundedVec(vec![]), oracle_type: vec![] });
oracle_infos.push(oracle_info.oracle_data.0);
}
Ok(oracle_infos)
}
Expand Down
2 changes: 1 addition & 1 deletion pallets/oracle/src/benchmarking.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ benchmarks! {
}: {
Oracle::<T>::on_initialize(50u32.into());
} verify {
assert_eq!(OracleData::<T>::get(BoundedVec::try_from("block_number_entropy".encode()).unwrap()).unwrap()[0], 50);
assert_eq!(OracleData::<T>::get(BoundedVec::try_from("block_number_entropy".encode()).unwrap()).unwrap().oracle_data[0], 50);
}

impl_benchmark_test_suite!(Oracle, crate::mock::new_test_ext(), crate::mock::Test);
Expand Down
19 changes: 16 additions & 3 deletions pallets/oracle/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@

use frame_support::pallet_prelude::*;
use frame_system::pallet_prelude::*;
use sp_std::vec::Vec;

#[cfg(test)]
mod mock;
Expand Down Expand Up @@ -55,13 +56,22 @@ pub mod module {
type WeightInfo: WeightInfo;
}

/// Oracle Info for oracle data
#[derive(Clone, Encode, Decode, Eq, PartialEqNoBound, RuntimeDebug, TypeInfo)]
#[scale_info(skip_type_params(T))]
pub struct OracleInfo<T: Config> {
pub oracle_data: BoundedVec<u8, T::MaxOracleValueLength>,
pub oracle_type: Vec<u8>,
}

/// Storage for oracle info to be passed to programs.
#[pallet::storage]
#[pallet::getter(fn oracle_data)]
pub type OracleData<T: Config> = StorageMap<
_,
Blake2_128Concat,
BoundedVec<u8, T::MaxOracleKeyLength>,
BoundedVec<u8, T::MaxOracleValueLength>,
OracleInfo<T>,
OptionQuery,
>;

Expand Down Expand Up @@ -92,8 +102,11 @@ pub mod module {
OracleData::<T>::insert(
BoundedVec::try_from("block_number_entropy".encode())
.expect("Key fits in bounded vec; qed"),
BoundedVec::try_from(block_number.encode())
.expect("Block number fits in bounded vec; qed"),
OracleInfo {
oracle_data: BoundedVec::try_from(block_number.encode())
.expect("Block number fits in bounded vec; qed"),
oracle_type: "u32".as_bytes().to_vec(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we not need to say something like scale encoded u32? Or are we assuming that oracle_data types will always be scale encoded and implement Encode / Decode?

It seems strange that elsewhere we use JSON for program aux data / config, but use scale here. Using both would mean programs need to have dependencies for both json and scale.

If i just read u32 i would expect to use either u32::to_be_bytes() or u32::to_le_bytes().

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ye, I chose scale because it plays really nicely with substrate, I mean Im open for other optinons here, but pretty much we should force all oracle info data types to be the same type either scale or json (im still in favour of scale in substrate)

},
);
T::WeightInfo::on_initialize()
}
Expand Down
9 changes: 5 additions & 4 deletions pallets/oracle/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,11 @@ fn test_set_block_number() {

<Oracle as OnInitialize<u64>>::on_initialize(50);

assert_eq!(
let oracle_info =
Oracle::oracle_data(BoundedVec::try_from("block_number_entropy".encode()).unwrap())
.unwrap()[0],
50
);
.unwrap();

assert_eq!(u32::decode(&mut oracle_info.oracle_data.as_ref()).unwrap(), 50u32);
assert_eq!(std::str::from_utf8(&oracle_info.oracle_type).unwrap().to_string(), "u32");
});
}
Loading