Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Db suggestions #1

Open
wants to merge 22 commits into
base: LedgerEntryStorage
Choose a base branch
from

Conversation

tamirms
Copy link

@tamirms tamirms commented Jan 20, 2023

  1. The NORMAL synchronous mode is recommended while using WAL, see https://sqlite.org/pragma.html#pragma_synchronous
  2. I discovered there is a common interface for testing.T and testing.B which can be used to simplify creating test dbs https://pkg.go.dev/testing#TB
  3. It is not necessary to commit a read only transaction so we can reduce boilerplate by doing a defer tx.rollback on read functions. For the updater, it seems like we should always rollback on errors because I don't think we will continue to use the updater after encountering an error.

@2opremio
Copy link
Owner

About the updater, I was only rolling back on database errors (or that was the intention). If the error is not related to the DB, it could be retried.

@tamirms
Copy link
Author

tamirms commented Jan 20, 2023

If the error is not related to the DB, it could be retried.

I think the non db errors are encoding / decoding errors:

	encodedKey, err := encodeLedgerKey(l.buffer, key)
	if err != nil {
		return err
	}
	// safe since we cast to string right away
	encodedEntry, err := l.buffer.UnsafeMarshalBinary(&entry)
	if err != nil {
		return err
	}

so if you retry I don't think the error will go away.

2opremio pushed a commit that referenced this pull request Feb 15, 2023
* feat: initial steps towards config

* feat: create Secret type and use rpassword to read secret inputs

* feat: validate secret key

* fix: clippy

* feat: initial config resolution

* feat: add toml support

* feat: add `rm` and `ls`

* feat: add `args-file` option to `invoke`

Currently only tests raw string

* chore: fmt

* fix: deploy test contract once and invoke twice

* fix: test and arg type

* feat: add slip10

* feat: add `identity generate` command

* chore: rename location to args

* start the addition of the network command (#1)

* Update cmd/soroban-cli/src/config/network/mod.rs

Co-authored-by: Chad Ostrowski <[email protected]>

* chore: make Wasm type for tests

* feat: config Args and incorporate into other commands

* fix: pass tests

* chore: replace wrap args with config::args

* feat: add wasm::Args

* chore: use secret::Args and add `as`

Note that since as is a keyword you need to add `r#` as a prefix to tell the parser.

* chore: remove unneeded file

* chore: update strkey to newest version

* fix: secret key args

* feat: add test secret key test file

* chore: config --> config_locator

* chore: rename location --> config_locator

Add needed descriptions

* fix: network_path needs network_dir

* fix: make --identity a flag, not subcommand

* feat: simplify config consumption

First of all, no longer allow setting `--config-dir`. Only allow using
the workspace config (the default) or the global config by passing
`--global` when adding new configs.

Then, when _consuming_ configs in commands like `invoke`, `read`, etc
(that is, in all parts of the CLI that are not nested under `config`),
don't allow specifying `--global`. In addition to being hard to
interpret (what does it mean to invoke _globally_?), this specificity
introduced potential for user error: if I add a global identity called
"ahalabs", and I call `invoke` with `--identity ahalabs` but forget to
also include `--global`, I would have gotten errors.

Instead, the new code always uses the same behavior when _reading_
configs:

1. Check local config files for an identity/network with the given name
2. If not found, check global config files for that name

This introduces the possibility that a user may add both a local and a
global config with the same name, and could become confused that the
global is not being used. To help prevent such confusion, this also adds
some `println` statements to tell the CLI user which config is being
used, and to warn when adding one with a duplicate name.

* fix: clippy and merge errors

* feat: incorporate config args to simplify commands

* feat!: remove defaults for now

This isn't implemented and currently doesn't make sense for networks anyway. Also could be a footgun to users accidentally using a default identity.

* feat: use XDG's `.config/soroban` for global config

* chore: refactor/cleanup

* feat: remove unneeded collision checks since lookup starts local

* chore: add tests

Still need to finish global tests

* chore: clippy

* feat: XDG_CONFIG_HOME env allows a global custom location for config

* fix: make secret-key take a CLI arg for now

* chore: remove seed_phrase arguments and logic for now

* fix: sort ls output

* chore: address some initial feedback

* chore: improve clarity

* chore(tests): new Sandbox instance; multiple cmds to share temp dir

* feat: add support for seed phrases

* feat: add generate command

* chore: update to released version of rs-sep5

---------

Co-authored-by: Jonathon Hammond <[email protected]>
Co-authored-by: Chad Ostrowski <[email protected]>
Co-authored-by: Tsachi Herman <[email protected]>
2opremio pushed a commit that referenced this pull request Feb 15, 2023
commit 8dfb7f3
Author: Willem Wyndham <[email protected]>
Date:   Tue Feb 7 14:33:08 2023 -0500

    chore(cli): clippy (stellar#398)

    * chore(cli): clippy

    * chore: convert to binary expression

    ---------

    Co-authored-by: Tsachi Herman <[email protected]>

commit 518cafc
Author: Willem Wyndham <[email protected]>
Date:   Tue Feb 7 10:12:25 2023 -0500

    feat: Config command (stellar#275)

    * feat: initial steps towards config

    * feat: create Secret type and use rpassword to read secret inputs

    * feat: validate secret key

    * fix: clippy

    * feat: initial config resolution

    * feat: add toml support

    * feat: add `rm` and `ls`

    * feat: add `args-file` option to `invoke`

    Currently only tests raw string

    * chore: fmt

    * fix: deploy test contract once and invoke twice

    * fix: test and arg type

    * feat: add slip10

    * feat: add `identity generate` command

    * chore: rename location to args

    * start the addition of the network command (#1)

    * Update cmd/soroban-cli/src/config/network/mod.rs

    Co-authored-by: Chad Ostrowski <[email protected]>

    * chore: make Wasm type for tests

    * feat: config Args and incorporate into other commands

    * fix: pass tests

    * chore: replace wrap args with config::args

    * feat: add wasm::Args

    * chore: use secret::Args and add `as`

    Note that since as is a keyword you need to add `r#` as a prefix to tell the parser.

    * chore: remove unneeded file

    * chore: update strkey to newest version

    * fix: secret key args

    * feat: add test secret key test file

    * chore: config --> config_locator

    * chore: rename location --> config_locator

    Add needed descriptions

    * fix: network_path needs network_dir

    * fix: make --identity a flag, not subcommand

    * feat: simplify config consumption

    First of all, no longer allow setting `--config-dir`. Only allow using
    the workspace config (the default) or the global config by passing
    `--global` when adding new configs.

    Then, when _consuming_ configs in commands like `invoke`, `read`, etc
    (that is, in all parts of the CLI that are not nested under `config`),
    don't allow specifying `--global`. In addition to being hard to
    interpret (what does it mean to invoke _globally_?), this specificity
    introduced potential for user error: if I add a global identity called
    "ahalabs", and I call `invoke` with `--identity ahalabs` but forget to
    also include `--global`, I would have gotten errors.

    Instead, the new code always uses the same behavior when _reading_
    configs:

    1. Check local config files for an identity/network with the given name
    2. If not found, check global config files for that name

    This introduces the possibility that a user may add both a local and a
    global config with the same name, and could become confused that the
    global is not being used. To help prevent such confusion, this also adds
    some `println` statements to tell the CLI user which config is being
    used, and to warn when adding one with a duplicate name.

    * fix: clippy and merge errors

    * feat: incorporate config args to simplify commands

    * feat!: remove defaults for now

    This isn't implemented and currently doesn't make sense for networks anyway. Also could be a footgun to users accidentally using a default identity.

    * feat: use XDG's `.config/soroban` for global config

    * chore: refactor/cleanup

    * feat: remove unneeded collision checks since lookup starts local

    * chore: add tests

    Still need to finish global tests

    * chore: clippy

    * feat: XDG_CONFIG_HOME env allows a global custom location for config

    * fix: make secret-key take a CLI arg for now

    * chore: remove seed_phrase arguments and logic for now

    * fix: sort ls output

    * chore: address some initial feedback

    * chore: improve clarity

    * chore(tests): new Sandbox instance; multiple cmds to share temp dir

    * feat: add support for seed phrases

    * feat: add generate command

    * chore: update to released version of rs-sep5

    ---------

    Co-authored-by: Jonathon Hammond <[email protected]>
    Co-authored-by: Chad Ostrowski <[email protected]>
    Co-authored-by: Tsachi Herman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants