Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Cadence 1.0] Improve Migration #5479

Merged
merged 52 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
76bd405
test deployment migration, on filtered payloads
turbolent Feb 28, 2024
8876cdd
pass logger
turbolent Feb 28, 2024
977d6d0
fail on contract update registrations with zero address
turbolent Feb 28, 2024
8907501
add missing address for FungibleTokenSwitchboard contract
turbolent Feb 28, 2024
aec38a3
improve error: report any invalid address for given chain
turbolent Feb 28, 2024
cdb5817
refactor payload extraction into common function
turbolent Feb 29, 2024
9362929
fix tests: pass logger
turbolent Feb 29, 2024
96a8ad1
fix tests
turbolent Feb 29, 2024
1fc2d25
Add logs to merge
janezpodhostnik Feb 29, 2024
547cf64
Merge branch 'feature/stable-cadence' into bastian/improve-migration-2
turbolent Feb 29, 2024
c90a1cd
Merge branch 'bastian/improve-migration-2' into janez/add-logs-to-merge
turbolent Feb 29, 2024
cf232f0
Merge pull request #5484 from onflow/janez/add-logs-to-merge
turbolent Feb 29, 2024
1a966ce
check deployment only writes to authorizer account
turbolent Feb 29, 2024
46f26d6
check migrated payloads are only for migration account
turbolent Feb 29, 2024
5c6375d
name all migrations
turbolent Feb 29, 2024
050dd9f
fix allocation
turbolent Feb 29, 2024
c61927c
remove unnecessary coverage report
turbolent Mar 1, 2024
0e378d1
support multiple expected addresses, do not assume all deployments on…
turbolent Mar 1, 2024
a50113a
test deployment
turbolent Mar 1, 2024
5598890
Merge branch 'feature/stable-cadence' into bastian/improve-migration-2
turbolent Mar 1, 2024
2fdbf62
Merge branch 'feature/stable-cadence' into bastian/improve-migration-2
turbolent Mar 1, 2024
f19e509
update API usage, new parameters location range and transfer elements…
turbolent Mar 1, 2024
579abb8
Merge branch 'bastian/improve-migration-2' into bastian/optimize-depl…
turbolent Mar 4, 2024
2e9bbef
Merge pull request #5470 from onflow/bastian/optimize-deployment-migr…
turbolent Mar 4, 2024
947ebee
Add migration diff util to compare Cadence values
fxamacker Mar 4, 2024
ee743ad
Merge branch 'bastian/improve-migration-2' into fxamacker/add-diff-re…
fxamacker Mar 4, 2024
6b66b53
Merge branch 'feature/stable-cadence' into bastian/improve-migration-2
turbolent Mar 4, 2024
d8a5fa6
Enable migration diff for more Cadence migrations
fxamacker Mar 4, 2024
d977b35
Add Uint64AtreeValue to StorageMap key type assertion
fxamacker Mar 4, 2024
4d03d8d
Add old and new static types in migration diff
fxamacker Mar 5, 2024
80e915c
Log error in DiffState instead of returning error
fxamacker Mar 5, 2024
d3c3afe
Merge pull request #5499 from onflow/fxamacker/add-diff-report-to-mig…
fxamacker Mar 5, 2024
59b8ecf
Merge branch 'feature/stable-cadence' into bastian/improve-migration-2
turbolent Mar 6, 2024
a51ec20
add flag to disable sorting of payloads
turbolent Mar 7, 2024
3ec0ba1
either deploy or update the burner contract, depending on network
turbolent Mar 7, 2024
19d2228
clean up
turbolent Mar 7, 2024
6c0259c
log addresses for which contract updates are staged
turbolent Mar 7, 2024
f3d56ff
debug log account statistics: payload counts and byte size
turbolent Mar 7, 2024
8be637d
fix address of burner contract
turbolent Mar 7, 2024
0cd52f2
add option to prune random beacon history (for development purposes)
turbolent Mar 7, 2024
1b2fddd
improve messages
turbolent Mar 8, 2024
0b4d0cf
enable zero copy for payload decoding
turbolent Mar 8, 2024
89c8456
Merge branch 'auto-update-onflow-cadence-v1.0.0-preview.10' into bast…
turbolent Mar 8, 2024
4704d68
Merge branch 'feature/stable-cadence' into bastian/improve-migration-2
turbolent Mar 8, 2024
bd4636d
add missing argument
turbolent Mar 8, 2024
aff00e9
remove debug log
turbolent Mar 8, 2024
ddfa6d5
remove TODO
turbolent Mar 8, 2024
b12ca8c
lint
turbolent Mar 8, 2024
0c17a58
test that migration of bootstrapped TN state succeeds
turbolent Mar 8, 2024
3f10906
add test case for program loading error
turbolent Mar 8, 2024
c2d79eb
program loading errors should include the parsing/checking error, but…
turbolent Mar 8, 2024
afeb436
avoid account info computation if it is not logged
turbolent Mar 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 48 additions & 2 deletions cmd/util/cmd/execution-state-extract/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,11 @@ var (
flagNoReport bool
flagValidateMigration bool
flagAllowPartialStateFromPayloads bool
flagSortPayloads bool
flagPrune bool
flagLogVerboseValidationError bool
flagDiffMigration bool
flagLogVerboseDiff bool
flagStagedContractsFile string
flagInputPayloadFileName string
flagOutputPayloadFileName string
Expand Down Expand Up @@ -81,12 +85,24 @@ func init() {
Cmd.Flags().BoolVar(&flagLogVerboseValidationError, "log-verbose-validation-error", false,
"log entire Cadence values on validation error (atree migration)")

Cmd.Flags().BoolVar(&flagDiffMigration, "diff", false,
"compare Cadence values and log diff (migration)")

Cmd.Flags().BoolVar(&flagLogVerboseDiff, "log-verbose-diff", false,
"log entire Cadence values on diff (requires --diff flag)")

Cmd.Flags().StringVar(&flagStagedContractsFile, "staged-contracts", "",
"Staged contracts CSV file")

Cmd.Flags().BoolVar(&flagAllowPartialStateFromPayloads, "allow-partial-state-from-payload-file", false,
"allow input payload file containing partial state (e.g. not all accounts)")

Cmd.Flags().BoolVar(&flagSortPayloads, "sort-payloads", true,
"sort payloads (generate deterministic output; disable only for development purposes)")

Cmd.Flags().BoolVar(&flagPrune, "prune", false,
"prune the state (for development purposes)")

// If specified, the state will consist of payloads from the given input payload file.
// If not specified, then the state will be extracted from the latest checkpoint file.
// This flag can be used to reduce total duration of migrations when state extraction involves
Expand Down Expand Up @@ -139,6 +155,10 @@ func run(*cobra.Command, []string) {
log.Fatal().Msg("--extract-payloads-by-address requires --output-payload-filename to be specified")
}

if flagValidateMigration && flagDiffMigration {
log.Fatal().Msg("Both --validate and --diff are enabled, please specify only one (or none) of these")
}

if len(flagBlockHash) > 0 {
blockID, err := flow.HexStringToIdentifier(flagBlockHash)
if err != nil {
Expand Down Expand Up @@ -241,11 +261,19 @@ func run(*cobra.Command, []string) {
}

if flagValidateMigration {
log.Warn().Msgf("atree migration validation flag is enabled and will increase duration of migration")
log.Warn().Msgf("--validate flag is enabled and will increase duration of migration")
}

if flagLogVerboseValidationError {
log.Warn().Msgf("atree migration has verbose validation error logging enabled which may increase size of log")
log.Warn().Msgf("--log-verbose-validation-error flag is enabled which may increase size of log")
}

if flagDiffMigration {
log.Warn().Msgf("--diff flag is enabled and will increase duration of migration")
}

if flagLogVerboseDiff {
log.Warn().Msgf("--log-verbose-diff flag is enabled which may increase size of log")
}

var inputMsg string
Expand Down Expand Up @@ -287,6 +315,14 @@ func run(*cobra.Command, []string) {
// TODO:
evmContractChange := migrations.EVMContractChangeNone

var burnerContractChange migrations.BurnerContractChange
switch chainID {
case flow.Emulator:
burnerContractChange = migrations.BurnerContractChangeDeploy
case flow.Testnet, flow.Mainnet:
burnerContractChange = migrations.BurnerContractChangeUpdate
}

stagedContracts, err := migrations.StagedContractsFromCSV(flagStagedContractsFile)
if err != nil {
log.Fatal().Err(err).Msgf("error loading staged contracts: %s", err.Error())
Expand All @@ -299,12 +335,17 @@ func run(*cobra.Command, []string) {
flagOutputDir,
flagNWorker,
!flagNoMigration,
flagDiffMigration,
flagLogVerboseDiff,
chainID,
evmContractChange,
burnerContractChange,
stagedContracts,
flagInputPayloadFileName,
flagOutputPayloadFileName,
exportedAddresses,
flagSortPayloads,
flagPrune,
)
} else {
err = extractExecutionState(
Expand All @@ -314,11 +355,16 @@ func run(*cobra.Command, []string) {
flagOutputDir,
flagNWorker,
!flagNoMigration,
flagDiffMigration,
flagLogVerboseDiff,
chainID,
evmContractChange,
burnerContractChange,
stagedContracts,
flagOutputPayloadFileName,
exportedAddresses,
flagSortPayloads,
flagPrune,
)
}

Expand Down
Loading
Loading