Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TxThrottler: dont throttle unless lag #14789

Merged
merged 13 commits into from
Feb 9, 2024
Merged

TxThrottler: dont throttle unless lag #14789

merged 13 commits into from
Feb 9, 2024

Conversation

ejortegau
Copy link
Contributor

@ejortegau ejortegau commented Dec 15, 2023 •
edited by deepthi
Loading

Description

This PR addresses issue #14768. It does so by ensuring that the transaction throttler checks for current lag in the shard before deciding whether to throttle the request or not.

The issue mentioned above is a bug causing artificially high throttling when the TxThrottler is enabled. Backporting of this is desirable as it will correct that undesirable behaviour in previous versions, not only in the next one. However, it seems I cannot add the backport to label to the PR.

Related Issue(s)

#14768

Checklist

  • "Backport to:" labels have been added if this change should be back-ported to release branches
  • If this change is to be back-ported to previous releases, a justification is included in the PR description
  • Tests were added or are not required
  • Did the new or modified tests pass consistently locally and on CI?
  • Documentation was added or is not required

Deployment Notes

N/A

@vitess-bot vitess-bot
Copy link
Contributor

vitess-bot bot commented Dec 15, 2023

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

  • Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

  • Apply the release notes (needs details) label if users need to know about this change.
  • New features should be documented.
  • There should be some code comments as to why things are implemented the way they are.
  • There should be a comment at the top of each new or modified test to explain what the test does.

New flags

  • Is this flag really necessary?
  • Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
  • vtctl command output order should be stable and awk-able.

@vitess-bot vitess-bot bot added NeedsBackportReason If backport labels have been applied to a PR, a justification is required NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsWebsiteDocsUpdate What it says labels Dec 15, 2023
@github-actions github-actions bot added this to the v19.0.0 milestone Dec 15, 2023
@ejortegau ejortegau force-pushed the ejortegau/txthrottler_dont_throttle_unless_lag branch from 7979573 to 231c2f6 Compare December 15, 2023 10:34
@ejortegau ejortegau marked this pull request as ready for review December 15, 2023 10:43
@ejortegau ejortegau marked this pull request as draft December 15, 2023 11:08
@ejortegau ejortegau marked this pull request as ready for review December 18, 2023 16:24
@shlomi-noach shlomi-noach removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says NeedsIssue A linked issue is missing for this Pull Request labels Jan 16, 2024
@deepthi deepthi changed the title Ejortegau/txthrottler dont throttle unless lag TxThrottler: dont throttle unless lag Jan 16, 2024
@codecov Codecov
Copy link

codecov bot commented Jan 18, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (main@6b481a7). Click here to learn what that means.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main   #14789   +/-   ##
=======================================
  Coverage        ?   39.75%           
=======================================
  Files           ?     1639           
  Lines           ?   380078           
  Branches        ?        0           
=======================================
  Hits            ?   151084           
  Misses          ?   212777           
  Partials        ?    16217

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ejortegau ejortegau force-pushed the ejortegau/txthrottler_dont_throttle_unless_lag branch from 276a9f8 to 3c2acbb Compare January 24, 2024 13:13
@deepthi deepthi added Backport to: release-17.0 and removed NeedsBackportReason If backport labels have been applied to a PR, a justification is required labels Jan 31, 2024
deepthi
deepthi reviewed Jan 31, 2024
View reviewed changes
go/vt/vttablet/tabletserver/txthrottler/tx_throttler.go Outdated Show resolved Hide resolved
go/vt/vttablet/tabletserver/txthrottler/tx_throttler.go Outdated Show resolved Hide resolved
go/vt/throttler/throttler.go Outdated Show resolved Hide resolved
go/vt/throttler/throttler.go Outdated Show resolved Hide resolved
go/vt/throttler/throttler.go Show resolved Hide resolved
@ejortegau
Address PR comments
57797ef 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
@ejortegau
Address More PR comments
fde32b8 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
@ejortegau ejortegau requested a review from deepthi February 5, 2024 09:09
@frouioui frouioui modified the milestones: v19.0.0, v20.0.0 Feb 6, 2024
deepthi
deepthi approved these changes Feb 8, 2024
View reviewed changes
Copy link
Member

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more comment that needs to be addressed and then we can merge this. Thank you for your patience with the prolonged review process on this PR.

@ejortegau
Address PR comments
80bdff5 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
@deepthi deepthi merged commit 2b25639 into vitessio:main Feb 9, 2024
102 checks passed
This was referenced Feb 9, 2024
Closed
Merged
@vitess-bot vitess-bot bot mentioned this pull request Feb 9, 2024
Merged
@frouioui
Copy link
Member

frouioui commented Feb 9, 2024

It feels like this should have been backported to release-19.0. Adding the label now and re-triggering the event manually.

@frouioui frouioui added the Backport to: release-19.0 Needs to be back ported to release-19.0 label Feb 9, 2024
vitess-bot pushed a commit that referenced this pull request Feb 9, 2024
@vitess-bot
TxThrottler: dont throttle unless lag (#14789)
c7c902a 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
@vitess-bot vitess-bot bot mentioned this pull request Feb 9, 2024
Merged
frouioui pushed a commit that referenced this pull request Feb 9, 2024
@ejortegau @frouioui
TxThrottler: dont throttle unless lag (#14789)
91de40c 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
frouioui pushed a commit that referenced this pull request Feb 9, 2024
@vitess-bot @ejortegau
[release-18.0] TxThrottler: dont throttle unless lag (#14789) (#15190)
f359bb9 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Eduardo J. Ortega U <[email protected]>
frouioui pushed a commit that referenced this pull request Feb 9, 2024
@vitess-bot
[release-19.0] TxThrottler: dont throttle unless lag (#14789) (#15196)
2024e38 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
harshit-gangal pushed a commit that referenced this pull request Mar 22, 2024
@vitess-bot @ejortegau
[release-17.0] TxThrottler: dont throttle unless lag (#14789) (#15189)
a19be92 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Eduardo J. Ortega U <[email protected]>
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request May 16, 2024
@ejortegau @timvaillancourt
TxThrottler: dont throttle unless lag (vitessio#14789)
bb2c0e1 
Signed-off-by: Eduardo J. Ortega U <[email protected]>
@timvaillancourt timvaillancourt mentioned this pull request May 16, 2024
Merged
4 tasks
timvaillancourt added a commit to slackhq/vitess that referenced this pull request May 21, 2024
@timvaillancourt @Juneezee
@shlomi-noach @ejortegau @deepthi
slack-vitess-r15.0.5: backport required Transaction Throttler PRs, …
4a92ae7 
…pt. 3 + ci fixes (#351)

* txthrottler: add metrics for topoWatcher and healthCheckStreamer (vitessio#13153)

Signed-off-by: Tim Vaillancourt <[email protected]>

* Replace deprecated `github.com/golang/mock` with `go.uber.org/mock` (vitessio#13512)

Signed-off-by: Eng Zer Jun <[email protected]>
Signed-off-by: Shlomi Noach <[email protected]>
Co-authored-by: Shlomi Noach <[email protected]>

* Per workload TxThrottler metrics (vitessio#13526)

Signed-off-by: Eduardo J. Ortega U <[email protected]>

* tx throttler: healthcheck all cells if `--tx-throttler-healthcheck-cells` is undefined (vitessio#12477)

Signed-off-by: Tim Vaillancourt <[email protected]>
Co-authored-by: Shlomi Noach <[email protected]>

* Add dry-run/monitoring-only mode for TxThrottler (vitessio#13604)

Signed-off-by: Eduardo J. Ortega U <[email protected]>
Signed-off-by: Eduardo J. Ortega U. <[email protected]>

* `txthrottler`: remove `txThrottlerConfig` struct, rely on `tabletenv` (vitessio#13624)

Signed-off-by: Tim Vaillancourt <[email protected]>

* tx throttler: remove unused topology watchers (vitessio#14412)

Signed-off-by: deepthi <[email protected]>

* tx_throttler: delete topo watcher metric instead of deprecating (vitessio#14445)

Signed-off-by: deepthi <[email protected]>

* TxThrottler: dont throttle unless lag (vitessio#14789)

Signed-off-by: Eduardo J. Ortega U <[email protected]>

* go test -v

Signed-off-by: Tim Vaillancourt <[email protected]>

* add mutex to flaky parseFlags()

Signed-off-by: Tim Vaillancourt <[email protected]>

* revert tweaks for flaky tests

Signed-off-by: Tim Vaillancourt <[email protected]>

* fix protojson err

Signed-off-by: Tim Vaillancourt <[email protected]>

* make vtadmin_web_proto_types

Signed-off-by: Tim Vaillancourt <[email protected]>

* remove debug t.Logf(...)

Signed-off-by: Tim Vaillancourt <[email protected]>

---------

Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Eng Zer Jun <[email protected]>
Signed-off-by: Shlomi Noach <[email protected]>
Signed-off-by: Eduardo J. Ortega U <[email protected]>
Signed-off-by: Eduardo J. Ortega U. <[email protected]>
Signed-off-by: deepthi <[email protected]>
Co-authored-by: Eng Zer Jun <[email protected]>
Co-authored-by: Shlomi Noach <[email protected]>
Co-authored-by: Eduardo J. Ortega U <[email protected]>
Co-authored-by: Deepthi Sigireddi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shlomi-noach shlomi-noach shlomi-noach left review comments

@harshit-gangal harshit-gangal harshit-gangal approved these changes

@deepthi deepthi deepthi approved these changes

Assignees

@deepthi deepthi

Labels
Backport to: release-19.0 Needs to be back ported to release-19.0 Component: Query Serving Type: Bug
Projects
None yet
Milestone
v20.0.0
Development

Successfully merging this pull request may close these issues.
5 participants
@ejortegau @frouioui @deepthi @shlomi-noach @harshit-gangal