Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Conflict Detection in Parallelization by Considering Slots to Reduce False Positives #7923

Merged
merged 15 commits into from
Feb 17, 2025

Conversation

matkt
Copy link
Contributor

@matkt matkt commented Nov 26, 2024

PR description

This update improve conflict detection during transaction parallelization by checking storage slot changes. By considering modified slots in addition to addresses, we improve the accuracy of conflict detection.

Key Changes:

  • Added storage slot consideration in conflict detection.
  • Reduced false positives by more accurately identifying conflicts between transactions and block state.

Fixed Issue(s)

Thanks for sending a pull request! Have you done the following?

  • Checked out our contribution guidelines?
  • Considered documentation and added the doc-change-required label to this PR if updates are required.
  • Considered the changelog and included an update if required.
  • For database changes (e.g. KeyValueSegmentIdentifier) considered compatibility and performed forwards and backwards compatibility tests

Locally, you can run these tests to catch failures early:

  • unit tests: ./gradlew build
  • acceptance tests: ./gradlew acceptanceTest
  • integration tests: ./gradlew integrationTest
  • reference tests: ./gradlew ethereum:referenceTests:referenceTests

Signed-off-by: Karim Taam <[email protected]>
Signed-off-by: Karim Taam <[email protected]>
@ahamlat
Copy link
Contributor

ahamlat commented Nov 26, 2024

This new conflict detection improved the number of parallelized transactions from ~40% to ~60%.
The numbers in the charts below are TPS.

Before this PR (24.10.0 running with --Xbonsai-parallel-tx-processing-enabled=true)
image

With this PR (running with --Xbonsai-parallel-tx-processing-enabled=true)
image

With more transactions executed in parallel, we noticed around 20% performance improvement compared to the last parallelization implementation. The numbers below are from instances running both EL and CL and are not using high spec flag.

image

We can notice also almost twice better performance when comparing to besu 24.10.0 with Teku as a CL

image

I don't see any overhead on the resource usage level.

@matkt
Copy link
Contributor Author

matkt commented Nov 26, 2024

we need to do stability tests before merging

Signed-off-by: Karim Taam <[email protected]>
- copy storageKeyHashLookup when cloning accumulator

Signed-off-by: Karim Taam <[email protected]>
Copy link
Contributor

@macfarla macfarla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changelog entry?

Signed-off-by: Karim Taam <[email protected]>
@matkt matkt force-pushed the feature/para-evm-2 branch from 68a5c15 to 85bfd20 Compare December 4, 2024 16:31
@matkt matkt force-pushed the feature/para-evm-2 branch from 788f628 to 9985e73 Compare January 7, 2025 19:37
Copy link
Contributor

@ahamlat ahamlat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, comparing with the first version of the parallelization, this PR has better performances even on 4 Cores, without more overhead. Conflict detection takes even less time than the first version.

Signed-off-by: Karim Taam <[email protected]>
@matkt matkt enabled auto-merge (squash) February 17, 2025 11:55
@matkt matkt disabled auto-merge February 17, 2025 11:56
Signed-off-by: Karim Taam <[email protected]>
@matkt matkt enabled auto-merge (squash) February 17, 2025 11:57
@matkt
Copy link
Contributor Author

matkt commented Feb 17, 2025

changelog entry?

done

Signed-off-by: Karim Taam <[email protected]>
@matkt matkt force-pushed the feature/para-evm-2 branch from 0a23a9b to 552bc52 Compare February 17, 2025 13:14
@matkt matkt merged commit 41a0647 into hyperledger:main Feb 17, 2025
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants