Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LevelDB with 2 databases #126

Merged
merged 30 commits into from
Mar 26, 2024
Merged

LevelDB with 2 databases #126

merged 30 commits into from
Mar 26, 2024

Conversation

oliverbundalo
Copy link

@oliverbundalo oliverbundalo commented Feb 20, 2024

Description

Addes storageV2 with leveldb having 2 databases with perf tests.

Changes include

  • Bugfix (non-breaking change that solves an issue)
  • Hotfix (change that solves an urgent issue, and requires immediate attention)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (change that is not backwards-compatible and/or changes current functionality)

Breaking changes

Please complete this section if any breaking changes have been made, otherwise delete it

Checklist

  • I have assigned this PR to myself
  • I have added at least 1 reviewer
  • I have added the relevant labels
  • I have updated the official documentation
  • I have added sufficient documentation in code

Testing

  • I have tested this code with the official test suite
  • I have tested this code manually

Manual tests

Please complete this section if you ran manual tests for this functionality, otherwise delete it

Documentation update

Please link the documentation update PR in this section if it's present, otherwise delete it

Additional comments

Please post additional comments in this section if you have them, otherwise delete it

Copy link

github-actions bot commented Feb 20, 2024

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@oliverbundalo
Copy link
Author

I have read the CLA Document and I hereby sign the CLA

Copy link

@Stefan-Ethernal Stefan-Ethernal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving some initial thoughts, though I haven't taken a deep look into the entire PR.

blockchain/storagev2/mdbx/mdbx_perf_test.go Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
blockchain/storagev2/mdbx/mdbx.go Outdated Show resolved Hide resolved
e2e-polybft/e2e/storage_test.go Outdated Show resolved Hide resolved
blockchain/storagev2/utils.go Show resolved Hide resolved
blockchain/storagev2/storage.go Outdated Show resolved Hide resolved
blockchain/storagev2/storage.go Outdated Show resolved Hide resolved
blockchain/storagev2/mdbx/mdbx_test.go Outdated Show resolved Hide resolved
blockchain/storagev2/mdbx/mdbx.go Outdated Show resolved Hide resolved
blockchain/storagev2/leveldb/leveldb_test.go Outdated Show resolved Hide resolved
blockchain/storagev2/leveldb/leveldb_test.go Show resolved Hide resolved
blockchain/storagev2/leveldb/leveldb_perf_test.go Outdated Show resolved Hide resolved
blockchain/storagev2/leveldb/leveldb_test.go Show resolved Hide resolved
blockchain/storagev2/mdbx/mdbx.go Outdated Show resolved Hide resolved
@@ -0,0 +1,101 @@
//nolint:stylecheck

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we revert this?

Suggested change
//nolint:stylecheck

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do then this error appears
ST1003: should not use ALL_CAPS in Go names; use CamelCase instead (stylecheck)
HEAD_HASH = uint8(2) | LOOKUP_INDEX

I think we should keep it, those are constants. It's better that way and it's already used in couple of files.

blockchain/storagev2/storage.go Outdated Show resolved Hide resolved
@oliverbundalo oliverbundalo merged commit 1c1d7f6 into develop Mar 26, 2024
8 of 10 checks passed
@oliverbundalo oliverbundalo deleted the feat/state-db-refactoring branch March 26, 2024 14:42
}

func getKey(n uint64, h types.Hash) []byte {
return append(append(make([]byte, 0, 40), common.EncodeUint64ToBytes(n)...), h.Bytes()...)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: maybe code bellow will optimize memory allocation a little bit

a, b := common.EncodeUint64ToBytes(n), h.Bytes()
return append(append(make([]byte, 0, len(a) + len(b)), a...), b...)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thanks, will change this through some other PR.

}

var tableMapper = map[uint8][]byte{
// Main DB

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we use suffixes instead of prefixes?
Why lookup DB do not have suffixes at all?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we use suffixes instead of prefixes?
With suffixes write into main DB is sequential, with prefixes is random.

Why lookup DB do not have suffixes at all?
Because there is no need for suffixes, data is already unique.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature performance-improvements Performance improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants