Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZSA integration (step 8): Merge zsa-issued-assets into zsa-integration-consensus with bug fixes #29

Open
wants to merge 36 commits into
base: zsa-integration-consensus
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
1af120e
Defines and implements the issued asset state types
arya2 Nov 12, 2024
cc8bc0d
Adds issued assets to the finalized state
arya2 Nov 12, 2024
c7116f3
Validates issuance actions and burns before committing blocks to a no…
arya2 Nov 12, 2024
bb62c67
Adds `issued_assets` fields on `ChainInner` and `ContextuallyValidate…
arya2 Nov 12, 2024
3d00b81
Adds issued assets map to non-finalized chains
arya2 Nov 12, 2024
2daf84f
adds new column family to list of state column families
arya2 Nov 12, 2024
c6c099b
Updates AssetState, AssetStateChange, IssuedAssetsOrChange, & Semanti…
arya2 Nov 14, 2024
9e0e043
Fixes tests by computing an `IssuedAssetsChange` for conversions to C…
arya2 Nov 14, 2024
8f26a89
fixes finalization checks
arya2 Nov 14, 2024
e063729
Adds documentation to types and methods in `asset_state` module, fixe…
arya2 Nov 15, 2024
6aad8da
Merge branch 'zsa-issued-assets' into zsa-integration-state
dmidem Nov 27, 2024
f0b64ad
Fix compilation errors that appeared after the previous merge
dmidem Nov 27, 2024
bc0c8e6
Avoid using NonEmpty in orchard_zsa/issuance
dmidem Nov 27, 2024
17f3ee6
Fix BurnItem serialization/deserializartioon errors (use LE instead o…
dmidem Nov 27, 2024
3f96af0
Make a minor fix and add FIXME comment in orchard_flavor_ext.rs
dmidem Nov 27, 2024
5524480
Fix the sign of burn value in SupplyChange::add in orchard_zsa/asset_…
dmidem Nov 27, 2024
8096da4
Fix the 'transactions must have only one burn item per asset base' er…
dmidem Nov 27, 2024
20fd58d
Use NoteValue from the orchard crate for BurnItem amount instead of u…
dmidem Nov 27, 2024
4932495
Use BurnItem::from instead of try_from
dmidem Nov 27, 2024
89be470
Fix a compilation error for the previous commit ('Use BurnItem::from …
dmidem Nov 27, 2024
c3daec9
Fix a compilation error for the previous commit ('Use BurnItem::from …
dmidem Nov 27, 2024
a8668d6
Modify ValueCommitment::with_asset to accept value as a NoteValue ins…
dmidem Dec 3, 2024
e31f24c
Adds TODOs
arya2 Nov 28, 2024
2a5aebd
Adds state request/response variants for querying asset states
arya2 Nov 28, 2024
f7b43a9
Adds a `getassetstate` RPC method
arya2 Nov 28, 2024
e35ae57
Adds snapshot test
arya2 Nov 28, 2024
c278758
Addesses some FIXMEs and replaces a couple others with TODOs.
arya2 Nov 29, 2024
d144774
Removes `issued_assets_change` field from `SemanticallyVerifiedBlock`
arya2 Nov 29, 2024
727e3e3
Temporarily disable specific Clippy checks for Rust 1.83.0 compatibility
dmidem Dec 5, 2024
9a8c032
Disable clippy warning about doc comment for empty line
dmidem Dec 5, 2024
fb512d9
Update Orchard ZSA consensus tests to calculate and check asset supply
dmidem Dec 6, 2024
977af42
Rename ZSA workflow tests (including file, constant and variable name…
dmidem Dec 6, 2024
29af613
Add amount method to BurnItem and make BurnItem pub (visible for othe…
dmidem Dec 6, 2024
2b7926a
Fix Orchard ZSA workflow tests to make it compilable with getblocktem…
dmidem Dec 6, 2024
73c804f
Fix clippy error
dmidem Dec 6, 2024
6bd4284
Add rust-toolchain.toml with Rust version 1.82.0 to avoid clippy erro…
dmidem Dec 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions zebra-chain/src/orchard/orchard_flavor_ext.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,11 @@ use orchard::{note_encryption::OrchardDomainCommon, orchard_flavor};

use crate::{
orchard::ValueCommitment,
orchard_zsa,
serialization::{ZcashDeserialize, ZcashSerialize},
};

#[cfg(feature = "tx-v6")]
use crate::orchard_zsa::{Burn, NoBurn};
use crate::orchard_zsa::{Burn, BurnItem, NoBurn};

use super::note;

Expand Down Expand Up @@ -59,8 +58,9 @@ pub trait OrchardFlavorExt: Clone + Debug {
+ Default
+ ZcashDeserialize
+ ZcashSerialize
// FIXME: consider using AsRef instead of Into, to avoid a redundancy
+ Into<ValueCommitment>
+ AsRef<[orchard_zsa::BurnItem]>
+ AsRef<[BurnItem]>
+ TestArbitrary;
}

Expand Down
6 changes: 1 addition & 5 deletions zebra-chain/src/orchard_zsa/arbitrary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,9 @@ impl Arbitrary for BurnItem {
// FIXME: move arb_asset_to_burn out of BundleArb in orchard
// as it does not depend on flavor (we pinned it here OrchardVanilla
// just for certainty, as there's no difference, which flavor to use)
// FIXME: consider to use BurnItem(asset_base, value.try_into().expect("Invalid value for Amount"))
// instead of filtering non-convertable values
// FIXME: should we filter/protect from including native assets into burn here?
BundleArb::<orchard::orchard_flavor::OrchardVanilla>::arb_asset_to_burn()
.prop_filter_map("Conversion to Amount failed", |(asset_base, value)| {
BurnItem::try_from((asset_base, value)).ok()
})
.prop_map(|(asset_base, value)| BurnItem::from((asset_base, value)))
.boxed()
}

Expand Down
8 changes: 6 additions & 2 deletions zebra-chain/src/orchard_zsa/asset_state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ pub struct AssetState {
pub struct AssetStateChange {
/// Whether the asset should be finalized such that no more of it can be issued.
pub should_finalize: bool,
// FIXME: is this a correct comment?
/// Whether the asset should be finalized such that no more of it can be issued.
pub includes_issuance: bool,
/// The change in supply from newly issued assets or burned assets, if any.
Expand All @@ -50,6 +51,7 @@ impl Default for SupplyChange {
}
}

// FIXME: can we reuse some functions from orchard crate?s
impl SupplyChange {
/// Applies `self` to a provided `total_supply` of an asset.
///
Expand Down Expand Up @@ -81,7 +83,8 @@ impl SupplyChange {
// Burn amounts MUST not be 0
// TODO: Reference ZIP
0.. => signed.try_into().ok().map(Self::Issuance),
..0 => signed.try_into().ok().map(Self::Burn),
// FIXME: (-signed) - is this a correct fix?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is correct, thank you for catching it!

..0 => (-signed).try_into().ok().map(Self::Burn),
})
{
*self = result;
Expand Down Expand Up @@ -208,6 +211,7 @@ impl AssetStateChange {
})
.unwrap_or_default()
.into_iter()
// FIXME: We use 0 as a value - is that correct?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's correct but overly complicated, it should be refactored.

.map(|asset_base| Self::new(asset_base, SupplyChange::Issuance(0), true));

supply_changes.chain(finalize_changes)
Expand Down Expand Up @@ -238,7 +242,7 @@ impl AssetStateChange {
/// Accepts an [`BurnItem`] and returns an iterator of asset bases and issued asset state changes
/// that should be applied to those asset bases when committing the provided burn to the chain state.
fn from_burn(burn: &BurnItem) -> (AssetBase, Self) {
Self::new(burn.asset(), SupplyChange::Burn(burn.amount()), false)
Self::new(burn.asset(), SupplyChange::Burn(burn.raw_amount()), false)
}

/// Updates and returns self with the provided [`AssetStateChange`] if
Expand Down
40 changes: 23 additions & 17 deletions zebra-chain/src/orchard_zsa/burn.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ use std::io;
use halo2::pasta::pallas;

use crate::{
amount::Amount,
block::MAX_BLOCK_BYTES,
orchard::ValueCommitment,
serialization::{
Expand Down Expand Up @@ -36,28 +37,27 @@ const AMOUNT_SIZE: u64 = 8;
// FIXME: is this a correct way to calculate (simple sum of sizes of components)?
const BURN_ITEM_SIZE: u64 = ASSET_BASE_SIZE + AMOUNT_SIZE;

// FIXME: Define BurnItem (or, even Burn/NoBurn) in Orchard and reuse it here?
/// Orchard ZSA burn item.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub struct BurnItem(AssetBase, u64);
pub struct BurnItem(AssetBase, NoteValue);

impl BurnItem {
/// Returns [`AssetBase`] being burned.
pub fn asset(&self) -> AssetBase {
self.0
}

/// Returns [`u64`] representing amount being burned.
pub fn amount(&self) -> u64 {
self.1
/// Returns the raw [`u64`] amount being burned.
pub fn raw_amount(&self) -> u64 {
self.1.inner()
}
}

// Convert from burn item type used in `orchard` crate
impl TryFrom<(AssetBase, NoteValue)> for BurnItem {
type Error = crate::amount::Error;

fn try_from(item: (AssetBase, NoteValue)) -> Result<Self, Self::Error> {
Ok(Self(item.0, item.1.inner()))
impl From<(AssetBase, NoteValue)> for BurnItem {
fn from(item: (AssetBase, NoteValue)) -> Self {
Self(item.0, item.1)
}
}

Expand All @@ -66,20 +66,18 @@ impl ZcashSerialize for BurnItem {
let BurnItem(asset_base, amount) = self;

asset_base.zcash_serialize(&mut writer)?;
writer.write_all(&amount.to_be_bytes())?;
writer.write_all(&amount.to_bytes())?;

Ok(())
}
}

impl ZcashDeserialize for BurnItem {
fn zcash_deserialize<R: io::Read>(mut reader: R) -> Result<Self, SerializationError> {
let asset_base = AssetBase::zcash_deserialize(&mut reader)?;
let mut amount_bytes = [0; 8];
reader.read_exact(&mut amount_bytes)?;
Ok(Self(
AssetBase::zcash_deserialize(&mut reader)?,
u64::from_be_bytes(amount_bytes),
))
Ok(Self(asset_base, NoteValue::from_bytes(amount_bytes)))
}
}

Expand All @@ -98,7 +96,7 @@ impl serde::Serialize for BurnItem {
S: serde::Serializer,
{
// FIXME: return a custom error with a meaningful description?
(self.0.to_bytes(), &self.1).serialize(serializer)
(self.0.to_bytes(), &self.1.inner()).serialize(serializer)
}
}

Expand All @@ -114,7 +112,7 @@ impl<'de> serde::Deserialize<'de> for BurnItem {
// FIXME: duplicates the body of AssetBase::zcash_deserialize?
Option::from(AssetBase::from_bytes(&asset_base_bytes))
.ok_or_else(|| serde::de::Error::custom("Invalid orchard_zsa AssetBase"))?,
amount,
NoteValue::from_raw(amount),
))
}
}
Expand Down Expand Up @@ -166,7 +164,15 @@ impl From<Burn> for ValueCommitment {
burn.0
.into_iter()
.map(|BurnItem(asset, amount)| {
ValueCommitment::with_asset(pallas::Scalar::zero(), amount, &asset)
ValueCommitment::with_asset(
pallas::Scalar::zero(),
// FIXME: consider to use TryFrom and return an error instead of using "expect"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a u64 so the conversion should be infallible.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. But to make it work, the new ValueCommitment::with_asset constructor I added earlier should accept the value as a NoteValue instead of Amount (which is i64-based). I've fixed that and pushed the change into the zsa-integration-state branch.

amount
.inner()
.try_into()
.expect("should convert Burn amount to i64"),
&asset,
)
})
.sum()
}
Expand Down
4 changes: 2 additions & 2 deletions zebra-chain/src/orchard_zsa/issuance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ impl IssueData {
}

/// Returns issuance actions
pub fn actions(&self) -> &NonEmpty<IssueAction> {
self.0.actions()
pub fn actions(&self) -> impl Iterator<Item = &IssueAction> {
self.0.actions().iter()
}
}

Expand Down
25 changes: 21 additions & 4 deletions zebra-state/src/service/check/issuance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ pub fn valid_burns_and_issuance(
) -> Result<IssuedAssets, ValidateContextError> {
let mut issued_assets = HashMap::new();

// FIXME: Do all checks (for duplication, existence etc.) need to be performed per tranaction, not per
// the entire block?
for (issued_assets_change, transaction) in semantically_verified
.issued_assets_changes
.iter()
Expand All @@ -41,7 +43,11 @@ pub fn valid_burns_and_issuance(
asset_state(finalized_state, parent_chain, &issued_assets, &asset_base)
.ok_or(ValidateContextError::InvalidBurn)?;

if asset_state.total_supply < burn.amount() {
// FIXME: The check that we can't burn an asset before we issued it is implicit -
// through the check it total_supply < burn.amount (the total supply is zero if the
// asset is not issued). May be validation functions from the orcharfd crate need to be
// reused in a some way?
if asset_state.total_supply < burn.raw_amount() {
return Err(ValidateContextError::InvalidBurn);
} else {
// Any burned asset bases in the transaction will also be present in the issued assets change,
Expand All @@ -50,6 +56,13 @@ pub fn valid_burns_and_issuance(
}
}

// FIXME: Not sure: it looks like semantically_verified.issued_assets_changes is already
// filled with burn and issuance items in zebra-consensus, see Verifier::call function in
// zebra-consensus/src/transaction.rs (it uses from_burn and from_issue_action AssetStateChange
// methods from ebra-chain/src/orchard_zsa/asset_state.rs). Can't it cause a duplication?
// Can we collect all change items here, not in zebra-consensus (and so we don't need
// SemanticallyVerifiedBlock::issued_assets_changes at all), or performing part of the
// checks in zebra-consensus is important for the consensus checks order in a some way?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't it cause a duplication?

It wouldn't be duplicated since the previous loop isn't mutating the asset states, it's just loading them into memory.

Can we collect all change items here, not in zebra-consensus

Yep, I'll try to make this change.

for (asset_base, change) in issued_assets_change.iter() {
let asset_state =
asset_state(finalized_state, parent_chain, &issued_assets, &asset_base)
Expand All @@ -59,9 +72,13 @@ pub fn valid_burns_and_issuance(
.apply_change(change)
.ok_or(ValidateContextError::InvalidIssuance)?;

issued_assets
.insert(asset_base, updated_asset_state)
.expect("transactions must have only one burn item per asset base");
// FIXME: Is it correct to do nothing if the issued_assets aready has asset_base? Now it'd be
// replaced with updated_asset_state in this case (where the duplicated value is added to
// the supply). Block may have two burn records for the same asset but in different
// transactions - it's allowed, that's why the check has been removed. On the other
// hand, there needs to be a check that denies duplicated burn records for the same
// asset inside the same transaction.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does need to check that there are no duplicate burns in a single transaction somewhere, but that could be validated during deserialization or semantic validation, and this should update the asset state correctly otherwise.

I think we should update the Burn type to be Burn(HashMap<AssetBase, NoteValue>) and return an error during deserialization if any asset base is burned twice in the same transaction.

Copy link
Author

@dmidem dmidem Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea to use a HashMap to ensure the uniqueness of AssetBase at the type level (the uniqueness would be validated in the deserialization function). However, I see two downsides:

  1. Using a HashMap makes the burn.rs module code more bloated, as, for example, we can't directly take advantage of the automatic vector serialization provided by ZcashSerialize and ZcashDeserialize and have to implement it manually. But this isn't a major issue - I attempted to implement it today.

  2. A more serious problem is that HashMap does not preserve the order of items. If we deserialize the transaction and then serialize it back or calculate the hash, the result would differ because the order of the burn items would change (most likely different, to be precise). To resolve this issue, we would need to add a new requirement to the specification that burn items must be sorted in the transaction by the bytes of AssetBase, for example.

In my opinion, we could continue using a Vec for now but perform uniqueness validation within/after the deserializer. And we already have a burn validation function in the orchard crate. I will add this check.

issued_assets.insert(asset_base, updated_asset_state);
}
}

Expand Down