Merge feature branch feature/combine-domain-payloads-and-domain-storage-maps to master branch
#3700
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit: 1. Combines all domain (non-atree) payloads into one account (non-atree) payload per account. 2. Combines all domain (atree) storage maps into one account (atree) storage map per account. 3. Uses on-the-fly (OTF) migration when an account is modified (write ops). Currently, accounts store data on-chain under pre-defined domains, such as "storage". Each domain requires domain payload (8-byte non-atree payload) and domain storage map payload (atree payload). Also, each payload requires ~2 mtrie nodes (~2x96 byte overhead). New domains were added in Cadence 1.0 and domain payloads count increased to 150 million on Sept. 4 (was 80 million pre-spork). Nearly 25% of total payloads on-chain are 8-byte domain payloads. Each account on mainnet has an average of ~4 domain payloads and ~4 domain storage maps. This commit creates 1 account (non-atree) payload and 1 account (atree) storage map per account, eliminating all domain (non-atree) payloads and domain storage maps for that given account. Based on preliminary estimates using Sept. 17, 2024 mainnet state, this approach can: - eliminate mtrie nodes: -425 million (-28.5%) - reduce payload count: -174 million (also -28.5%) This commit also includes on-the-fly migration so we can see improvements to accounts that have write activity without requiring downtime. Given this, we won't see the full benefits/impact until all accounts (including idle accounts) are eventually migrated (e.g. using full migration or other means).
This is for completeness.
…ap) into separate type
…to bastian/refactor-account-storage
…ce-domain-regsiters Combine domain payloads and provide on-the-fly migration
This commit replaces domain string, such as "storage", "public", "private", etc. with domain enum (integer) as the key for AccountStorageMap. Also, this uses CBOR to store the domain integer in shortest form. This optimization further improves storage efficiency of domain payloads of PR 3664.
…mbine-domain-payloads Sync `feature/combine-domain-payloads` branch with master
This commit renames function convertKeyToDomain to convertAccountStorageMapKeyToStorageDomain.
…ount-storage-map-key Replace domain string with enum for AccountStorageMap key
…' into bastian/refactor-account-storage
Currently, tests for register reads only check sequence of register reading. However, same register can be read multiple times and all register reads are included in the sequence. This commit updates tests to explicitly checks unique registers being read in addition to sequence of registers being read.
Co-authored-by: Bastian Müller <[email protected]>
[Account Storage Maps] Add more tests for register reads for GetDomainStorageMap()
…' into fxamacker/reduce-register-reads-for-new-accounts
…for-new-accounts [Account Storage Maps] Reduce storage register reads when using `StorageFormatV2Enabled`
…ling-return-value [Account Storage Maps] Add return value to account migration scheduling functions
…ain-payloads-to-master-branch
Cadence Benchstat comparisonThis branch with compared with the base branch onflow:master commit d555d17 Collapsed results for better readability
|
|
@onflow/cadence could you please have a final look? |
…tion Add `Storage.AccountStorageFormat()`
SupunS
approved these changes
Nov 28, 2024
turbolent
deleted the
fxamacker/merge-feature-branch-combine-domain-payloads-to-master-branch
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates #3584
This PR merges feature branch
feature/combine-domain-payloads-and-domain-storage-mapstomasterbranch since account storage v2 is behind a feature flag.Special thanks to @turbolent 🙌 for adding feature flag, many other improvements, and PR reviews!
masterbranchFiles changedin the Github PR explorer