Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

services/horizon: Remove Horizon Submission Queue #5039

Merged
merged 32 commits into from
Sep 18, 2023

Conversation

aditya1702
Copy link
Contributor

@aditya1702 aditya1702 commented Sep 5, 2023

Closes #4996

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with name of package that is most changed in the PR, ex.
    services/friendbot, or all or doc if the changes are broad or impact many
    packages.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated any docs (developer docs, .md
    files, etc... affected by this change). Take a look in the docs folder for a given service,
    like this one.

Release planning

  • I've updated the relevant CHANGELOG (here for Horizon) if
    needed with deprecations, added features, breaking changes, and DB schema changes.
  • I've decided if this PR requires a new major/minor version according to
    semver, or if it's mainly a patch change. The PR is targeted at the next
    release branch if it's not a patch change.

What

Horizon manages an internal submission queue. This intent of this is to protect/isolate clients from potentially receiving error messages on transaction submission when multiple transactions are submitted very quickly in succession. If transactions (from the same source account) are submitted with sequence numbers out-of-order, then this submission queue will hold those transactions in-memory so that it can re-order them in line with their sequence numbers, allowing all of them to succeed when they are actually submitted to core

This PR:

  • removes this queue and routes transactions directly to core for submission.
  • removes the tests related to the queue.

Why

Recently, core has released a change that prevents the submission of multiple transactions in a ledger for a given account. With this change, the Horizon submission queue is even less likely to provide any benefit to end-users, and if anything cause more confusion. Transactions would be queued up in-memory, only to have only one of them be able to be submitted when they were finally submitted to core.

Known limitations

N/A

@aditya1702 aditya1702 changed the title Remove Horizon Queue: Remove history_transactions_filtered_tmp table Remove Horizon Submission Queue (temp) (WIP) Sep 5, 2023
@aditya1702 aditya1702 changed the title Remove Horizon Submission Queue (temp) (WIP) Remove Horizon Submission Queue Sep 5, 2023
Copy link
Contributor

@Shaptic Shaptic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loving that there's so much code going away 🥳 Left a couple of notes where I'm not sure about behavior.

Are there DB tables that can be dropped as a result of this PR?

services/horizon/internal/txsub/system.go Show resolved Hide resolved
@tamirms
Copy link
Contributor

tamirms commented Sep 14, 2023

I think ErrNoAccount can be removed since it seems to not be referenced anywhere

@tamirms
Copy link
Contributor

tamirms commented Sep 14, 2023

I think we can simplify OpenSubmissionList by removing all the error return values from the function signatures. I noticed that none of the functions actually returned any errors except for Add() which tries to validate the transaction hash. I don't think it's necessary to have this validation because we have already validated the transaction hash before invoking txsub

Copy link
Contributor

@tamirms tamirms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@aditya1702 aditya1702 requested a review from sreuland September 15, 2023 20:59
@aditya1702 aditya1702 merged commit d41faf8 into stellar:master Sep 18, 2023
36 checks passed
@aditya1702 aditya1702 deleted the remove-submission-queue-2 branch September 20, 2023 19:34
2opremio pushed a commit that referenced this pull request Oct 16, 2023
* Remove horizon submission queue - 1

* Update system.go

* Update system.go

* remove WaitUntilAccountSequence

* Update system.go

* Remove tests related to WaitUntilAccountSequence - 1

* Update system_test.go

* Remove history_transactions_filtered_tmp - 1

* Fix failing tests in system_test.go

* Revert "Remove history_transactions_filtered_tmp - 1"

This reverts commit d1c6959.

* Revert "Fix failing tests in system_test.go"

This reverts commit 767a9f9.

* Remove sequenceNumber from checkTxAlreadyExists

* Undo removing waitUntilAccountSequence

* Small change - 1

* Small changes - 2

* fix failing unit tests

* Add some comments - 1

* Small changes - 3

* Small changes - 4

* Fix failing tests - 1

* Use defer for sys.finish

* Revert "Use defer for sys.finish"

This reverts commit b2321be.

* Small changes - 5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

services/horizon: remove internal transaction submission queue
4 participants