Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADR 006: Non-interactive Defaults, Wrapped Txs, and Subtree Root Message Inclusion Checks #673

Merged
merged 41 commits into from
Oct 3, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
889e564
init
evan-forbes Aug 30, 2022
9b853f0
flesh out PrepoareProposal
evan-forbes Aug 31, 2022
5ad18ba
add processproposal
evan-forbes Sep 1, 2022
152be10
linter
evan-forbes Sep 1, 2022
764fc74
remove unused images
evan-forbes Sep 1, 2022
dd0ac52
fix a bunch of wording
evan-forbes Sep 1, 2022
12afde0
better wording
evan-forbes Sep 1, 2022
e95584c
remove todo
evan-forbes Sep 1, 2022
813400a
missing word
evan-forbes Sep 1, 2022
d370eeb
spelling
evan-forbes Sep 1, 2022
7cee154
Merge branch 'evan/adr-003-NID' of https://github.com/celestiaorg/cel…
evan-forbes Sep 1, 2022
a161fbb
fix word
evan-forbes Sep 2, 2022
27ee1da
be more explicit with splitting shares api
evan-forbes Sep 2, 2022
a74e1ec
move comment to emphasize
evan-forbes Sep 4, 2022
502ba93
add word
evan-forbes Sep 4, 2022
edfac9e
add comment explaining the namespaces and messages used in the diagram
evan-forbes Sep 4, 2022
c91e169
user merkle mountain range to describe subtree root commitment
evan-forbes Sep 4, 2022
7dc68b2
msw -> mss
evan-forbes Sep 4, 2022
2402006
Merge branch 'evan/adr-003-NID' of github.com:celestiaorg/lazyledger-…
evan-forbes Sep 4, 2022
c8959f8
update wording and implementation to reflect the usage of
evan-forbes Sep 4, 2022
66e811b
wording
evan-forbes Sep 4, 2022
3373aba
Apply suggestions from code review
evan-forbes Sep 4, 2022
98f10e1
change var name of nextAlignedPowerOfTwo
evan-forbes Sep 6, 2022
a60787d
Merge branch 'evan/adr-003-NID' of https://github.com/celestiaorg/cel…
evan-forbes Sep 6, 2022
70f62c6
change var name
evan-forbes Sep 6, 2022
63b0ff3
remove strictly from comments
evan-forbes Sep 6, 2022
01b30f0
fix: typo
evan-forbes Sep 23, 2022
c162833
fix: pls -> please
evan-forbes Sep 23, 2022
1f47929
Merge branch 'evan/adr-003-NID' of https://github.com/celestiaorg/cel…
evan-forbes Sep 23, 2022
e8842c8
fix: better wording
evan-forbes Sep 23, 2022
0d22a53
fix: point to figure in comment
evan-forbes Sep 23, 2022
ccea1f5
fix: remove redudundant word "ideally"
evan-forbes Sep 23, 2022
e3f9831
fix: better wording
evan-forbes Sep 23, 2022
540af06
Merge branch 'evan/adr-003-NID' of https://github.com/celestiaorg/cel…
evan-forbes Sep 23, 2022
e706bec
fix: remove statement for reviewers
evan-forbes Sep 23, 2022
703384a
fix: add links to issues and PRs
evan-forbes Sep 23, 2022
185e157
fix: use permalinks to the specs
evan-forbes Sep 23, 2022
69826cc
fix: better wording
evan-forbes Sep 23, 2022
eb745c8
fix: remove comments meant for reviewer
evan-forbes Sep 27, 2022
1f6bda1
Merge branch 'main' into evan/adr-003-NID
evan-forbes Sep 27, 2022
dd66693
fix: rename to adr006
evan-forbes Sep 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions docs/architecture/ADR-003-Non-interactive-defaults.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# ADR 003: Non-interactive Defaults, Wrapped Transactions, and Subtree Root Message Inclusion Checks

## Intro

Currently, when checking for message inclusion, validators recreate the share commitment from the messages in the block and compare those with what are signed over in the `MsgPayForData` transactions also in that block. If any commitment is not found in one of the PFD transactions, or if there is a commitment that doesn't have a corresponding message, then they reject that block.

While this functions as a message inclusion check, the light client has to assume that 2/3's of the voting power is honest in order to be assured that both the messages they are interested in and the rest of the messages paid for in that block are actually included.

To fix this, the spec outlines the “non-interactive default rules”. These involve a few additional message layout rules that allow for commitments to messages to consist entirely of sub tree roots of the data root, and for those sub tree roots to be generated only from the message itself (so that the user can sign something “non-interactively”). NOTE: MODIFIED FROM THE SPEC


- Messages begin at a location aligned with the largest power of 2 that is not larger than the message length or k.
- If the largest power of 2 of a given message spans multiple rows it must begin at the start of a row (this can occur if a message is longer than k shares or if the block producer decides to start a message partway through a row and it cannot fit).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question] I'm confused by this rule's application in example-full-block.png for ns=2. ns=2 is a message that spans 7 shares. The largest power of 2 less than the message length (7) or k (8) is 4. The message is aligned with index 4.

If the largest power of 2 of a given message spans multiple rows

The largest power of 2 for this message (4) doesn't span multiple rows but this message does because the block producer decided to start this message partway through the row and it cannot fit. If ns=2 is indeed supposed to be split across rows, then should this read:

Suggested change
- If the largest power of 2 of a given message spans multiple rows it must begin at the start of a row (this can occur if a message is longer than k shares or if the block producer decides to start a message partway through a row and it cannot fit).
- If the largest power of 2 of a given message is larger than k then it must begin at the start of a row.

If ns=2 is supposed to start at a new row, I think the current language makes sense.

example-full-block

example-full-block

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still thinking about this, but I think the first rule might be the only one we need. The we can just go into more detail of a what an "aligned power of two" actually means


We can always create a commitment to the data that is a subtree root of the data root while only knowing the data in that message. Below illustrates how we can break a message up into two different subtree roots, the first for first four shares, the second consisting of the last two shares.

![before](./assets/subtree-root.png "Subtree Root based commitments")

In practice this means that we end up adding padding between messages (zig-zag hatched share). Padding consists of a the namespace of the message before it, with all zeros for data.
evan-forbes marked this conversation as resolved.
Show resolved Hide resolved

![before](./assets/before.png "before")
![after](./assets/after.png "after")
![example](./assets/example-full-block.png "example")

evan-forbes marked this conversation as resolved.
Show resolved Hide resolved

Not only does doing this allow for easy trust minimized message inclusion checks for specific messages by light clients, but also allows for the creation of message inclusion fraud proofs for all messages in the block.


## Alternative Approaches

Arranging the messages in the block to maximize for fees is an NP-hard problem because each change to the square potentially affects the rest of the messages in the square. There will likely be many different strategies we could use to quickly and efficiently fill the square in a valid way.
evan-forbes marked this conversation as resolved.
Show resolved Hide resolved

## Decision

## Detailed Design

## Status

{Deprecated|Proposed|Accepted|Declined}
evan-forbes marked this conversation as resolved.
Show resolved Hide resolved

## Consequences

### Positive

### Negative

### Neutral

## References
Binary file added docs/architecture/assets/after.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/architecture/assets/before.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/architecture/assets/example-full-block.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/architecture/assets/extra.drawio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/architecture/assets/multi-row-str.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/architecture/assets/subtree-root.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/architecture/assets/subtreeroot.drawio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.