Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

boost: load query configs in vtgate #15582

Closed
wants to merge 127 commits into from
Closed

boost: load query configs in vtgate #15582

wants to merge 127 commits into from

Conversation

benetis
Copy link

@benetis benetis commented Mar 27, 2024

No description provided.

GuptaManan100 and others added 30 commits July 15, 2021 08:23
@GuptaManan100
fixed function call in default value of a column and added tests
d4bf38c 
Signed-off-by: GuptaManan100 <[email protected]>
@GuptaManan100
specify collate earlier in command because parser for now expects it so
2c416fa 
Signed-off-by: GuptaManan100 <[email protected]>
@harshit-gangal
Merge pull request vitessio#8476 from planetscale/function-call-defau…
94d3d54 
…lt-11.0

Fix for function calls in DEFAULT value of CREATE TABLE statement in release-11.0
@shlomi-noach
Fixing multiple issues related to onlineddl/lifecycle: - stale migrat…
0d56514 
…ion analysis did not update completed_timestamp. As result lifecycle would not run - retroactively updating completed_timestamp for existing NULL entries - liveness_timestamp now always set with started_timestamp, so that it is never NULL even if migration didn't report liveness. This is essential for detecting and marking stale migrations - when garbage collecting artifacts, RENAME statement would create collissions between artifacts of same migration. This is now solved by assigning distinct timestamps to artifacts of same migration - on the safe side, though there is no evidence this ever happened, whenever an artifact is added, we set cleanup_timestamp to be NULL

Signed-off-by: Shlomi Noach <[email protected]>
@deepthi
Merge pull request vitessio#8517 from planetscale/onlineddl-stale-mig…
b0a094d 
…ration-compeleted-timestamp-release-11

Backport: Fixing multiple issues related to onlineddl/lifecycle
@rohit-nayak-ps
Change local example to use v2 vreplication flows
35e0dd4 
Signed-off-by: Rohit Nayak <[email protected]>
@shlomi-noach
Merge pull request vitessio#8527 from planetscale/rn-local-example-v2…
02f2a7a 
…-11.0

Change local example to use v2 vreplication flows
@GuptaManan100
default should not add paranthesis for boolean values
3e2206d 
Signed-off-by: GuptaManan100 <[email protected]>
@systay
Merge pull request vitessio#8531 from planetscale/default-bool-11
0abe768 
boolean values should not be parenthesised in default clause - release 11
@shlomi-noach
OnlineDDL: SHOW VITESS_MIGRATION ... LOGS
b8f1195 
Signed-off-by: Shlomi Noach <[email protected]>
@shlomi-noach
Removed ShowMigrationLogs primitive. Now using plain Send
8351c4e 
Signed-off-by: Shlomi Noach <[email protected]>
@shlomi-noach
Merge pull request vitessio#8532 from planetscale/onlineddl-show-vite…
810c792 
…ss-migration-log-release-11

SHOW VITESS_MIGRATION '...' LOGS, retain logs for 24 hours
@rohit-nayak-ps
Add metric to measure how often a picker doesn't find tablets to stre…
5497bad 
…am from

Signed-off-by: Rohit Nayak <[email protected]>
@rohit-nayak-ps
Ignore SBR statements from pt-table-checksum
bad4a9b 
Signed-off-by: Rohit Nayak <[email protected]>
@harshit-gangal @GuptaManan100
check for topo.Error before clearing the cache value
5687384 
Signed-off-by: Harshit Gangal <[email protected]>
@GuptaManan100 @harshit-gangal
added an end to end test for testing the change and fixed consul startup
1e6cf0f 
for testing

Signed-off-by: GuptaManan100 <[email protected]>
Co-authored-by: harshit-gangal <[email protected]>
@harshit-gangal @GuptaManan100
reload test has both the tests
b528506 
Signed-off-by: Harshit Gangal <[email protected]>
@harshit-gangal @GuptaManan100
check for topo.Error and not *topo.Error, fix unit test to send the c…
d226399 
…orrect error type

Signed-off-by: Harshit Gangal <[email protected]>
@GuptaManan100
changed default server port in consul setup to allow running multiple…
2444bd7 
… servers simultaneously

Signed-off-by: GuptaManan100 <[email protected]>
@harshit-gangal @GuptaManan100
added bind address to consul args
b8d4ab0 
Signed-off-by: Harshit Gangal <[email protected]>
@harshit-gangal @GuptaManan100
increase timeout
78486ee 
Signed-off-by: Harshit Gangal <[email protected]>
@rohit-nayak-ps
Fix vreplication error metric
cb71762 
Signed-off-by: Rohit Nayak <[email protected]>
@rohit-nayak-ps
Return from throttler goroutine if context is cancelled to prevent go…
3723a87 
…routine leaks

Signed-off-by: Rohit Nayak <[email protected]>
@shlomi-noach @rohit-nayak-ps
even more check for ctx.Done()
b9478f0 
Signed-off-by: Shlomi Noach <[email protected]>
@rohit-nayak-ps
Add BIT datatype to list of byte comparable types
807c67b 
Signed-off-by: Rohit Nayak <[email protected]>
@rohit-nayak-ps
Delete consistent lookup test for BIT datatypes since it is now compa…
75df633 
…rable

Signed-off-by: Rohit Nayak <[email protected]>
@rohit-nayak-ps
Log error output from CreateKeyspace, which seems to randomly fail in CI
0c4e5e5 
Signed-off-by: Rohit Nayak <[email protected]>
@rohit-nayak-ps
Truncate logs and db log messages to prevent huge failing queries spa…
8bd4e35 
…mming the logs. Fix logic for incrementing count of recurring error messages.

Signed-off-by: Rohit Nayak <[email protected]>
@harshit-gangal
Merge pull request vitessio#8533 from planetscale/topo-qs-fix-11
a63bceb 
[11.0] query serving to continue when topo server restarts
@rohit-nayak-ps
Merge pull request vitessio#8536 from planetscale/rn-8403-release-11
cc2de83 
Backports of vitessio#8403 vitessio#8483 vitessio#8489 vitessio#8401 vitessio#8521 vitessio#8396 from main into release 11.0
DeathBorn and others added 20 commits January 26, 2023 14:46
@DeathBorn
backport partial: Update for new errcheck violations after rebase; pl…
07effb3 
…anetscale@81a2396

Signed-off-by: Vilius Okockis <[email protected]>
@frouioui @DeathBorn
Added a mutex for vstreamer manager errors
dcd0e5a 
Signed-off-by: Florent Poinsard <[email protected]>
@rohit-nayak-ps @DeathBorn
Fix issue where sending events received from source into the target c…
0c19762 
…hannel might block the vstream thread if target channel goes away: context was not being checked then. Fix health stream goroutine leak. (vitessio#10571)

Signed-off-by: Rohit Nayak <[email protected]>
Signed-off-by: Matt Lord <[email protected]>
@rohit-nayak-ps @mattlord @DeathBorn
Backport fix: Accomodate missing changes for vitessio@8037f43
49d066b 
VStreamer: fix deadlock when there are a lot of vschema changes at the same time as binlog events (vitessio#11325)

* Don't block on vschema channel in case of heavy vschema changes and vstream load

Signed-off-by: Rohit Nayak <[email protected]>

* Prevent VStreamer engine deadlocks during state transitions

The VStreamer engine is somewhat unusual in two ways:
  1. It is open and running on replica tablets rather than only
     running on primary tablets.
  2. It has no controllers so the main engine mutex is widely shared.

Because of this, when a tablet has open vstreams (direct binary log
streams) performing work and a state transition starts, it can
deadlock with the tabletmanager's state lock when checking if
the engine is open or not.

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Rohit Nayak <[email protected]>

Signed-off-by: Rohit Nayak <[email protected]>
Signed-off-by: Matt Lord <[email protected]>
Co-authored-by: Matt Lord <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@systay @DeathBorn
Backport fix: Merge pull request vitessio#9145 from planetscale/Norma…
11ba806 
…lizeHexValuesInQueries vitessio@322ac09 using vitessio#9163 as a guide

Merge pull request vitessio#9118 from planetscale/NormalizeHexValuesInQueries

Ensure that hex query predicates are normalized for planner cache

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@mattlord @DeathBorn
Backport to v11
573b2fd 
Take MySQL column type into account in vstreamer

This is required when we need to match MySQL behavior for data
that requires column type information as well. For example, the
binlog event metadata makes no distinction between events for a
BINARY(4) column and events for a CHAR(4) column with a binary
collation like utf8mb4_bin. So we need to know the underlying
MySQL column type in order to handle them disctinctly -- MySQL
pads (fixed length) binary columns on the right side with null
bytes, but it does NOT do that for (fixed lengthed) CHARo
columns, regardless of the collation.

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@mattlord @DeathBorn
Backport to v11
70c1bb7 
Detect and handle MySQL 8 specifically in vstreamer tests

And use ToLower when looking for BINARY types to be safe.

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@harshit-gangal @DeathBorn
Backport fix: vitessio#10260
49d5ba3 
fix: copy bindvariables in logstats

Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@aquarapid @DeathBorn
Quick fix for schema tracker not recognizing RENAME for schema reloading
3cf20dd 
Signed-off-by: Jacques Grove <[email protected]>
@mvh-stripe @DeathBorn
Backport to v11 vitessio#9324
de77e50 
Fix schema engine reload so it accounts for swapped tables during online DDL

Signed-off-by: Harish Mallipeddi <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@rohit-nayak-ps @DeathBorn
Backport to v11 vitessio#9324
fc9c4b9 
Reinstate lastChange logic to handle multiple DDLs that happen within a second: for instance if a column is altered immediately after creation then we get test failures due to incorrect FIELD events which are based on the previous DDL. Unit tests: reload schema when tablets are added/deleted since the test schema engine is an external singleton

Signed-off-by: Rohit Nayak <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@rohit-nayak-ps @DeathBorn
Backport to v11 vitessio#9324
8a1032b 
Address review comments

Signed-off-by: Rohit Nayak <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@deepthi @DeathBorn
Backport to v11 vitessio#8634 reset table size stats when a table is …
862e317 
…dropped

schema engine: reset table size stats when a table is dropped

Signed-off-by: deepthi <[email protected]>
Signed-off-by: Vilius Okockis <[email protected]>
@zhePin @vbalys
adding continueAfterCopyWithOwner flag
4d028a5 
Signed-off-by: zhe <[email protected]>
@mattlord @vbalys
Externalize VIndexes properly when not stopping after copy
b4e50ec 
We did not check for this config state at externalize.
When -stop_after_copy=false was used for the lookup vindex
materialization the workflow state will be Running when
it's healthy.

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Vaidas Balys <[email protected]>
@mattlord @vbalys
Update tests now that we check BinlogSource when externalizing lookup…
5d8fa9a 
… vindexes

Signed-off-by: Matt Lord <[email protected]>
@paukste3000
redis api and cache key gen
ff8aa34
@benetis
boost: draft for vtgate boost query checks (#8)
83dcd06
@paukste3000
Merge pull request #9 from vinted/hackaton_liu_redis_api
bb85bb2 
Caching API
@benetis
boost: load query filters configs and check for boost
67a1690
@vitess-bot vitess-bot
Copy link
Contributor

vitess-bot bot commented Mar 27, 2024

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

  • Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

  • Apply the release notes (needs details) label if users need to know about this change.
  • New features should be documented.
  • There should be some code comments as to why things are implemented the way they are.
  • There should be a comment at the top of each new or modified test to explain what the test does.

New flags

  • Is this flag really necessary?
  • Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
  • vtctl command output order should be stable and awk-able.

@vitess-bot vitess-bot bot added NeedsBackportReason If backport labels have been applied to a PR, a justification is required NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsWebsiteDocsUpdate What it says labels Mar 27, 2024
@benetis benetis closed this Mar 27, 2024
@github-actions github-actions bot added this to the v20.0.0 milestone Mar 27, 2024
@benetis benetis deleted the vinted/zygis/is-boosted-checks branch March 27, 2024 15:37
@benetis benetis restored the vinted/zygis/is-boosted-checks branch March 27, 2024 15:37
@benetis
Copy link
Author

benetis commented Mar 27, 2024

Mistake, sorry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GuptaManan100 GuptaManan100 Awaiting requested review from GuptaManan100 GuptaManan100 will be requested when the pull request is marked ready for review GuptaManan100 is a code owner

@frouioui frouioui Awaiting requested review from frouioui frouioui will be requested when the pull request is marked ready for review frouioui is a code owner

@rohit-nayak-ps rohit-nayak-ps Awaiting requested review from rohit-nayak-ps rohit-nayak-ps will be requested when the pull request is marked ready for review rohit-nayak-ps is a code owner

@deepthi deepthi Awaiting requested review from deepthi deepthi will be requested when the pull request is marked ready for review deepthi is a code owner

@mattlord mattlord Awaiting requested review from mattlord mattlord will be requested when the pull request is marked ready for review mattlord is a code owner

@harshit-gangal harshit-gangal Awaiting requested review from harshit-gangal harshit-gangal will be requested when the pull request is marked ready for review harshit-gangal is a code owner

@systay systay Awaiting requested review from systay systay will be requested when the pull request is marked ready for review systay is a code owner

@shlomi-noach shlomi-noach Awaiting requested review from shlomi-noach shlomi-noach will be requested when the pull request is marked ready for review shlomi-noach is a code owner

@timvaillancourt timvaillancourt Awaiting requested review from timvaillancourt timvaillancourt will be requested when the pull request is marked ready for review timvaillancourt is a code owner

@arthurschreiber arthurschreiber Awaiting requested review from arthurschreiber arthurschreiber will be requested when the pull request is marked ready for review arthurschreiber is a code owner

@notfelineit notfelineit Awaiting requested review from notfelineit notfelineit will be requested when the pull request is marked ready for review notfelineit is a code owner

@ajm188 ajm188 Awaiting requested review from ajm188 ajm188 will be requested when the pull request is marked ready for review ajm188 is a code owner

@vmg vmg Awaiting requested review from vmg vmg will be requested when the pull request is marked ready for review vmg is a code owner

Assignees
No one assigned
Labels
NeedsBackportReason If backport labels have been applied to a PR, a justification is required NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsWebsiteDocsUpdate What it says
Projects
None yet
Milestone
v20.0.0
Development

Successfully merging this pull request may close these issues.