Skip to content
This repository has been archived by the owner on Mar 24, 2023. It is now read-only.

Merkle tree misc fixes #47

Merged
merged 21 commits into from
Jul 2, 2020
Merged

Merkle tree misc fixes #47

merged 21 commits into from
Jul 2, 2020

Conversation

adlerjohn
Copy link
Member

@adlerjohn adlerjohn commented Jul 2, 2020

  • Change binary Merkle tree to use scheme in RFC 6962, i.e. unbalanced tree.
  • Split up computing the root and Merkle proofs into different subsections
  • Example diagrams. (Need Merkle tree diagrams #49)
  • Define zero tree base case.
  • Fix wording on leaf messages -> leaf data.
  • Merkle multiproof format. (Spec more Merkle proofs format #48)
  • Remove the annotated Merkle tree abstraction since it isn't used elsewhere in the spec except for NMT. Define the domain separation prefixing for base Merkle trees.

Rendered:

@adlerjohn adlerjohn added bug Something isn't working documentation Improvements or additions to documentation labels Jul 2, 2020
@adlerjohn adlerjohn added this to the Pre-implementation draft milestone Jul 2, 2020
@adlerjohn adlerjohn self-assigned this Jul 2, 2020
@adlerjohn adlerjohn linked an issue Jul 2, 2020 that may be closed by this pull request
@adlerjohn adlerjohn requested review from liamsi and musalbas July 2, 2020 15:03
@adlerjohn adlerjohn marked this pull request as ready for review July 2, 2020 15:03
@adlerjohn
Copy link
Member Author

I think I'll relegate making nice diagrams and the Merkle multiproof format to issues for later, since they're not critical atm.

specs/data_structures.md Show resolved Hide resolved
specs/data_structures.md Outdated Show resolved Hide resolved
specs/data_structures.md Outdated Show resolved Hide resolved
specs/data_structures.md Outdated Show resolved Hide resolved
@adlerjohn adlerjohn marked this pull request as draft July 2, 2020 15:52
@adlerjohn adlerjohn marked this pull request as ready for review July 2, 2020 15:58
@adlerjohn adlerjohn requested a review from liamsi July 2, 2020 15:58
```C++
node.n_min = d.namespaceID
node.n_max = d.namespaceID
node.v = h(0x00, serialize(d))
Copy link
Member

@liamsi liamsi Jul 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last nitpick: You implicitly assuming that data d is a structure as well namely struct{namespaceID, raw_data}. And that last line is still ambiguous to what gets hashed in the end: if h(0x00, d.raw_data) or h(0x00, d.namespaceID||d.raw_data)) (or whatever the result of serialize(d) is).

Note: as far as I understand both h(0x00, d.raw_data) or h(0x00, d.namespaceID||d.raw_data)) correctly define a NMT but I think we should be explicit here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, the serialization of shares is defined as a special case: https://github.com/lazyledger/lazyledger-specs/blob/adlerjohn-merkle_tree_fixes/specs/data_structures.md#share-serialization.

Shares canonically serialized using only the raw share data, i.e. serialize(share) = serialize(share.rawData).

Copy link
Member

@liamsi liamsi Jul 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I missed that. But that doesn't address that you implicitly assume that d has to have a structure (namely two fields) but I guess the important part is covered in the spec (how the shares end up in the tree ...).

Copy link
Member

@liamsi liamsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏 thanks🙏

@adlerjohn adlerjohn merged commit ec317ae into master Jul 2, 2020
@adlerjohn adlerjohn deleted the adlerjohn-merkle_tree_fixes branch July 2, 2020 16:29
@@ -228,100 +229,129 @@ Merkle trees are used to authenticate various pieces of data across the LazyLedg

## Binary Merkle Tree

Binary Merkle trees are constructed in the usual fashion, with leaves being hashed once to get leaf node values and internal node values being the hash of the concatenation of their children. The specific mechanism for hashing leaves for leaf nodes and children for internal nodes may be different (see: [annotated Merkle trees](#annotated-merkle-tree)), but for plain binary Merkle trees are the same.
Binary Merkle trees are constructed in the same fashion as described in [Certificate Transparency (RFC-6962)](https://tools.ietf.org/html/rfc6962). Leaves are hashed once to get leaf node values and internal node values are the hash of the concatenation of their children (either leaf nodes or other internal nodes).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be helpful to specify how it's the same as RFC 6962. Specifically, that the tree is unbalanced when there isn't 2^x leaves. The next sentence is describing properties of all Merkle trees.

```

If a compact Merkle root is needed, the root level (which consists of root fields and a root value) can be hashed once.
Note that rather than duplicating the last node if there are an odd number of nodes (the [Bitcoin design](https://github.com/bitcoin/bitcoin/blob/5961b23898ee7c0af2626c46d5d70e80136578d3/src/consensus/merkle.cpp#L9-L43)), trees are allowed to be imbalanced. In other words, the height of each leaf may be different. For an example, see Section 2.1.3 of [Certificate Transparency (RFC-6962)](https://tools.ietf.org/html/rfc6962).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah nvm, it's specified here.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Merkle trees misc cleanup and fixes
3 participants