Update nep-0508.md #517
Merged
Update nep-0508.md #517
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Creating a PR so that we can discuss changes... This only covers additions upto `Reference Implementation` section.
wacban
approved these changes
Nov 14, 2023
neps/nep-0508.md
Outdated
| * Merkle Patricia Trie is the undelying data structure for the protocol state. | ||
| * Minimal epoch gap between two resharding events is X. | ||
| * Epoch is atleast 6 hrs long for resharding to complete. |
neps/nep-0508.md
Outdated
| * A validator should be able to recover in case they go offline during resharding. | ||
| * For now, our aim is at least allowing a validator to join back after resharding is finished. | ||
| * Resharding should work efficiently within the limits of the current hardware requirements for nodes. | ||
| * Resharding should be fault tolerant. A validator should be able to recover in case they go offline during resharding. |
There was a problem hiding this comment.
Can you add that recovery will be a manual process requiring attention from the node operator?
There was a problem hiding this comment.
Change the fault tolerant point here
neps/nep-0508.md
Outdated
|
|
||
| ### New shard layout | ||
|
|
||
| The first release of the resharding v2 will contain a new shard layout where one of the existing shard will be split into two smaller shards. Furthermore additional reshardings can be scheduled in subsequent neard releases without additional NEPs unless the need for it arises. A new shard layout will be determined and will be scheduled and executed in the production networks. Resharding will typically happen by splitting one of the existing shards into two smaller shards. The new shard layout will be created by adding a new boundary account. The new boundary account will be determined by analysing the storage and gas usage within the shard and selecting a point that will divide the shard roughly in half in accordance to the mentioned metrics. Other metrics can also be used. | ||
| The first release of the resharding v2 will contain a new shard layout where shard 3 will be split into two smaller shards. We chose shard 3 for this purpose as that was the largest of the 4 shards. Furthermore additional reshardings can be scheduled in subsequent neard releases without additional NEPs unless the need for it arises. A new shard layout can be determined and will be scheduled and executed with the next protocol upgrade. Resharding will typically happen by splitting one of the existing shards into two smaller shards. The new shard layout will be created by adding a new boundary account that will be determined by analysing the storage and gas usage metrics within the shard and selecting a point that will divide the shard roughly in half in accordance to the mentioned metrics. Other metrics can also be used based on requirements. |
There was a problem hiding this comment.
@walnut-the-cat suggested earlier to not specify which shard we'll be splitting in the NEP
added 2 commits
November 14, 2023 17:39
wacban
approved these changes
Nov 14, 2023
neps/nep-0508.md
Outdated
| * When generating proofs for incoming receipts. The proof for each of the children shards contains only the receipts of the shard but it's generated on the parent shard layout and so may not be verified. | ||
|
|
||
| In this NEP we propose that resharding should be rolled out first, before any real dependency on state sync is added. We can then safely roll out the resharding logic and solve the abovementioned issues separately. | ||
| In this NEP we propose that resharding should be rolled out first, before any real dependency on state sync is added. We can then safely roll out the resharding logic and solve the above mentioned issues separately. We believe atleast some of the issues can be mitigated by the new implementation of post-state root and chunk execution design. |
There was a problem hiding this comment.
It's not longer post state root, it's "new pre state root" :D
There was a problem hiding this comment.
What? We went back to pre state root?
neps/nep-0508.md
Outdated
| * The resharding process is still not fully automated. Analyzing shard data, determining the split boundary, and triggering an actual shard split all need to be manually curated by a person. | ||
| * During resharding, a node is expected to do more work as it will first need to copy a lot of data around the then will have to apply changes twice (for the current shard and the future shard). | ||
| * The resharding process, as of now, is not fully automated. Analyzing shard data, determining the split boundary, and triggering an actual shard split all need to be manually curated and tracked. | ||
| * During resharding, a node is expected to require more resources as it will first need to copy state data from the parent shard to the child shard, the then will have to apply trie and flat state changes twice, once for the parent shard and once for the child shards. |
neps/nep-0508.md
Outdated
|
|
||
| [The changelog section provides historical context for how the NEP developed over time. Initial NEP submission should start with version 1.0.0, and all subsequent NEP extensions must follow [Semantic Versioning](https://semver.org/). Every version should have the benefits and concerns raised during the review. The author does not need to fill out this section for the initial draft. Instead, the assigned reviewers (Subject Matter Experts) should create the first version during the first technical review. After the final public call, the author should then finalize the last version of the decision context.] | ||
|
|
||
| ### 1.0.0 - Initial Version |
There was a problem hiding this comment.
I thought this should stay here and be filled in when reviewed. No clue if that is how it works actually.
There was a problem hiding this comment.
Oh right, sorry, I just took a look at some other NEPs and they didn't have this, so maybe I thought it shouldn't be there. My bad, will add in
added 2 commits
November 14, 2023 20:37
frol
added
WG-protocol
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Creating a PR so that we can discuss changes...