Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge feature branch feature/combine-domain-payloads-and-domain-storage-maps to master branch #3700

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
94e8cc2
Combine domain payloads and provide OTF migration
fxamacker Oct 31, 2024
124aef7
Merge branch 'master' into fxamacker/use-atree-map-to-replace-domain-…
fxamacker Oct 31, 2024
7429be0
Lint
fxamacker Oct 31, 2024
e617908
Update go.mod
fxamacker Oct 31, 2024
5843373
Export DomainRegisterMigration.MigrateAccount()
fxamacker Nov 4, 2024
658f6c1
Skip domain register migration for no domain accts
fxamacker Nov 4, 2024
3f8513e
Rename storagemap.go to domain_storagemap.go
fxamacker Nov 5, 2024
ec7fc82
refactor current account storage format (payload per domain storage m…
turbolent Nov 5, 2024
6099a06
try to add storage V2
turbolent Nov 6, 2024
f14ac1c
Merge branch 'fxamacker/use-atree-map-to-replace-domain-regsiters' in…
turbolent Nov 6, 2024
3b83c83
align V1 and V2 further
turbolent Nov 6, 2024
e827275
fix and use account storage V2
turbolent Nov 6, 2024
1e1e0af
re-enable test
turbolent Nov 6, 2024
dae042f
bring back health check
turbolent Nov 6, 2024
b5e97e0
put new storage format behind feature flag
turbolent Nov 6, 2024
e28e8c9
Refactor to use a type alias
fxamacker Nov 7, 2024
b13fdba
Merge pull request #3664 from onflow/fxamacker/use-atree-map-to-repla…
fxamacker Nov 12, 2024
7660d65
Merge branch 'master' into fxamacker/sync-feature-branch-combine-doma…
fxamacker Nov 12, 2024
bc2b313
Replace domain string with enum for AccountStorageMap
fxamacker Nov 12, 2024
cc95223
Merge pull request #3676 from onflow/fxamacker/sync-feature-branch-co…
fxamacker Nov 12, 2024
88c31e6
Rename function convertKeyToDomain
fxamacker Nov 12, 2024
f68c5eb
Add comments and warnings about StorageDomain
fxamacker Nov 12, 2024
0abfa9a
Merge pull request #3677 from onflow/fxamacker/use-domain-enum-as-acc…
fxamacker Nov 13, 2024
f43803a
Merge branch 'feature/combine-domain-payloads-and-domain-storage-maps…
turbolent Nov 13, 2024
d579591
improve naming
turbolent Nov 13, 2024
99710ce
ensure account is not tested for V1 registers if V2 is disabled
turbolent Nov 14, 2024
4ac1120
simplify V1/V2 check
turbolent Nov 14, 2024
1eea55e
revert part of 4ac1120b654cc5b38c442c6ef3d203c6fe20c3af
turbolent Nov 14, 2024
0178bcb
test storage format v1 and v2
turbolent Nov 14, 2024
3b2af1c
improve comment
turbolent Nov 15, 2024
b761a7d
Apply suggestions from code review
turbolent Nov 15, 2024
4f93a4a
adjust names based on b761a7df880b8bce671338f261d8424c98b7d51e
turbolent Nov 15, 2024
e827dcd
simplify domain register migration: only migrate one account specific…
turbolent Nov 14, 2024
f23eca0
try to run manual migration
turbolent Nov 15, 2024
dfca497
bring back logic to automatically migrate accounts with writes on commit
turbolent Nov 15, 2024
16130ed
Fix storage test
fxamacker Nov 15, 2024
d3c76d5
allow scheduling migrations of accounts to V2
turbolent Nov 15, 2024
6d95444
simplify ordered maps to simple maps. sort keys on commit
turbolent Nov 15, 2024
9593335
optimize commit for single map
turbolent Nov 16, 2024
0dae060
gate initializaton and use of account storage V2 behind feature flag
turbolent Nov 16, 2024
3fa8ad1
simplify logic
turbolent Nov 16, 2024
c7b02e3
remove TODO
turbolent Nov 16, 2024
29aea54
fix suggestion
turbolent Nov 18, 2024
4addf32
refactor slab index writing
turbolent Nov 18, 2024
4fd610a
refactor slab index register reading/writing to separate file and imp…
turbolent Nov 18, 2024
c795bfc
Reduce register reads with StorageFormatV2Enabled
fxamacker Nov 19, 2024
caaf254
Merge pull request #3678 from onflow/bastian/refactor-account-storage
turbolent Nov 20, 2024
f562727
Add tests for register reads for GetDomainStorageMap()
fxamacker Nov 20, 2024
3a71bf0
Add tests for register reads for GetDomainStorageMap()
fxamacker Nov 25, 2024
9af92f6
Update unit tests to check unique register reads
fxamacker Nov 25, 2024
c7cf012
Refactor getAccountStorageFormat()
fxamacker Nov 25, 2024
426a548
Use refactored getCachedAccountFormat()
fxamacker Nov 25, 2024
42d6b26
Rename storageFormat constant
fxamacker Nov 25, 2024
1568414
Refactor Storage.GetDomainStorageMap()
fxamacker Nov 25, 2024
c8a6176
add return value to migration scheduling functions
turbolent Nov 25, 2024
7772eb0
Add test helper function
fxamacker Nov 25, 2024
ba76c11
Extract new function NewPersistentSlabStorage()
fxamacker Nov 25, 2024
b51053d
Refactor Storage.isV1Account()
fxamacker Nov 25, 2024
157d322
Merge pull request #3694 from onflow/fxamacker/add-register-reads-tests
fxamacker Nov 25, 2024
9b3df51
Merge branch 'feature/combine-domain-payloads-and-domain-storage-maps…
fxamacker Nov 25, 2024
5646c24
Merge pull request #3683 from onflow/fxamacker/reduce-register-reads-…
fxamacker Nov 26, 2024
8e40b73
Merge pull request #3695 from onflow/bastian/account-migration-schedu…
turbolent Nov 26, 2024
1b79b41
Bump atree version to v0.8.1
fxamacker Nov 27, 2024
bcd2acb
Merge pull request #3699 from onflow/fxamacker/bump-atree-version-to-…
turbolent Nov 27, 2024
4ee0719
Merge branch 'master' into fxamacker/merge-feature-branch-combine-dom…
fxamacker Nov 27, 2024
d002625
Add Storage.AccountStorageFormat()
fxamacker Nov 27, 2024
427957d
Merge pull request #3701 from onflow/fxamacker/add-storageformat-func…
fxamacker Nov 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions cmd/decode-state-values/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -234,8 +234,13 @@ type interpreterStorage struct {

var _ interpreter.Storage = &interpreterStorage{}

func (i interpreterStorage) GetStorageMap(_ common.Address, _ common.StorageDomain, _ bool) *interpreter.StorageMap {
panic("unexpected GetStorageMap call")
func (i interpreterStorage) GetDomainStorageMap(
_ *interpreter.Interpreter,
_ common.Address,
_ common.StorageDomain,
_ bool,
) *interpreter.DomainStorageMap {
panic("unexpected GetDomainStorageMap call")
}

func (i interpreterStorage) CheckHealth() error {
Expand Down
5 changes: 5 additions & 0 deletions common/address.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
package common

import (
"bytes"
"encoding/hex"
goErrors "errors"
"fmt"
Expand Down Expand Up @@ -112,6 +113,10 @@ func (a Address) HexWithPrefix() string {
return fmt.Sprintf("0x%x", [AddressLength]byte(a))
}

func (a Address) Compare(other Address) int {
return bytes.Compare(a[:], other[:])
}

// HexToAddress converts a hex string to an Address after
// ensuring that the hex string starts with the prefix 0x.
func HexToAddressAssertPrefix(h string) (Address, error) {
Expand Down
15 changes: 15 additions & 0 deletions common/storagedomain.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,19 @@ import (
"github.com/onflow/cadence/errors"
)

// StorageDomain is used to store domain values on chain.
//
// !!! *WARNING* !!!
//
// Only add new StorageDomain by:
// - appending to the end.
//
// Only remove StorageDomain by:
// - replacing existing StorageDomain with a placeholder `_`.
//
// DO *NOT* REPLACE EXISTING STORAGEDOMAIN!
// DO *NOT* REMOVE EXISTING STORAGEDOMAIN!
// DO *NOT* INSERT NEW STORAGEDOMAIN IN BETWEEN!
type StorageDomain uint8

const (
Expand Down Expand Up @@ -54,6 +67,8 @@ const (
// StorageDomainAccountCapability is the storage domain which
// records active account capability controller IDs
StorageDomainAccountCapability

// Append new StorageDomain here (if needed).
)

var AllStorageDomains = []StorageDomain{
Expand Down
6 changes: 3 additions & 3 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ require (
github.com/kr/pretty v0.3.1
github.com/leanovate/gopter v0.2.9
github.com/logrusorgru/aurora/v4 v4.0.0
github.com/onflow/atree v0.8.0
github.com/onflow/atree v0.8.1
github.com/rivo/uniseg v0.4.4
github.com/schollz/progressbar/v3 v3.13.1
github.com/stretchr/testify v1.9.0
github.com/stretchr/testify v1.10.0
github.com/texttheater/golang-levenshtein/levenshtein v0.0.0-20200805054039-cae8b0eaed6c
github.com/tidwall/pretty v1.2.1
github.com/turbolent/prettier v0.0.0-20220320183459-661cc755135d
Expand Down Expand Up @@ -54,7 +54,7 @@ require (
github.com/rogpeppe/go-internal v1.9.0 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/zeebo/assert v1.3.0 // indirect
github.com/zeebo/blake3 v0.2.3 // indirect
github.com/zeebo/blake3 v0.2.4 // indirect
golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect
golang.org/x/sync v0.8.0 // indirect
golang.org/x/sys v0.26.0 // indirect
Expand Down
14 changes: 6 additions & 8 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ github.com/k0kubun/pp v3.0.1+incompatible h1:3tqvf7QgUnZ5tXO6pNAZlrvHgl6DvifjDrd
github.com/k0kubun/pp v3.0.1+incompatible/go.mod h1:GWse8YhT0p8pT4ir3ZgBbfZild3tgzSScAn6HmfYukg=
github.com/k0kubun/pp/v3 v3.2.0 h1:h33hNTZ9nVFNP3u2Fsgz8JXiF5JINoZfFq4SvKJwNcs=
github.com/k0kubun/pp/v3 v3.2.0/go.mod h1:ODtJQbQcIRfAD3N+theGCV1m/CBxweERz2dapdz1EwA=
github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.2.0 h1:4ZexSFt8agMNzNisrsilL6RClWDC5YJnLHNIfTy4iuc=
github.com/klauspost/cpuid/v2 v2.2.0/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
github.com/kodova/html-to-markdown v1.0.1 h1:MJxQAnqxtss3DaPnm72DRV65HZiMQZF3DUAfEaTg+14=
Expand Down Expand Up @@ -75,8 +74,8 @@ github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db h1:62I3jR2Em
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db/go.mod h1:l0dey0ia/Uv7NcFFVbCLtqEBQbrT4OCwCSKTEv6enCw=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/onflow/atree v0.8.0 h1:qg5c6J1gVDNObughpEeWm8oxqhPGdEyGrda121GM4u0=
github.com/onflow/atree v0.8.0/go.mod h1:yccR+LR7xc1Jdic0mrjocbHvUD7lnVvg8/Ct1AA5zBo=
github.com/onflow/atree v0.8.1 h1:DAnPnL9/Ks3LaAnkQVokokTBG/znTW0DJfovDtJDhLI=
github.com/onflow/atree v0.8.1/go.mod h1:FT6udJF9Q7VQTu3wknDhFX+VV4D44ZGdqtTAE5iztck=
github.com/onflow/crypto v0.25.0 h1:BeWbLsh3ZD13Ej+Uky6kg1PL1ZIVBDVX+2MVBNwqddg=
github.com/onflow/crypto v0.25.0/go.mod h1:C8FbaX0x8y+FxWjbkHy0Q4EASCDR9bSPWZqlpCLYyVI=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
Expand All @@ -98,8 +97,8 @@ github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/texttheater/golang-levenshtein/levenshtein v0.0.0-20200805054039-cae8b0eaed6c h1:HelZ2kAFadG0La9d+4htN4HzQ68Bm2iM9qKMSMES6xg=
github.com/texttheater/golang-levenshtein/levenshtein v0.0.0-20200805054039-cae8b0eaed6c/go.mod h1:JlzghshsemAMDGZLytTFY8C1JQxQPhnatWqNwUXjggo=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
Expand All @@ -108,11 +107,10 @@ github.com/turbolent/prettier v0.0.0-20220320183459-661cc755135d h1:5JInRQbk5UBX
github.com/turbolent/prettier v0.0.0-20220320183459-661cc755135d/go.mod h1:Nlx5Y115XQvNcIdIy7dZXaNSUpzwBSge4/Ivk93/Yog=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
github.com/zeebo/assert v1.1.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/blake3 v0.2.3 h1:TFoLXsjeXqRNFxSbk35Dk4YtszE/MQQGK10BH4ptoTg=
github.com/zeebo/blake3 v0.2.3/go.mod h1:mjJjZpnsyIVtVgTOSpJ9vmRE4wgDeyt2HU3qXvvKCaQ=
github.com/zeebo/blake3 v0.2.4 h1:KYQPkhpRtcqh0ssGYcKLG1JYvddkEA8QwCM/yBqhaZI=
github.com/zeebo/blake3 v0.2.4/go.mod h1:7eeQ6d2iXWRGF6npfaxl2CU+xy2Fjo2gxeyZGCRUjcE=
github.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo=
github.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4=
go.opentelemetry.io/otel v1.8.0 h1:zcvBFizPbpa1q7FehvFiHbQwGzmPILebO0tyqIR5Djg=
Expand Down
Loading
Loading