Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VReplication/OnlineDDL: reordering enum values #15103

Merged
merged 8 commits into from
Feb 25, 2024

Conversation

shlomi-noach
Copy link
Contributor

@shlomi-noach shlomi-noach commented Feb 1, 2024

Description

Fixes #15349

This PR builds a test case where an enum's values are reordered, from ('red', 'green', 'blue') to ('blue', 'green', 'red'), and observes how vreplication (in this case via OnlineDDL) handles it.

The behavior before this PR (see test output) is that vcopier will copy enum values by text, and vplayer by ordinal value.

As of this PR, and solved specifically for Online DDL (the problem is irrelevant for other vreplication workflows), vplayer applies enum column changes by writing the logical (string representation, e.g. 'blue') of the column.

The change is actually quite small. A prior work on converting enums to textual columns has already laid the necessary foundation: making vplayer aware of the enum's schema. This is already supported by setting vreplication's Filter.Rule.ConvertEnumToText map. Previously, we only populated it, as mentioned, when converting enum to text. The only change needed in this PR was to populate it also when an enum remains an enum. Whether the enum's schema is change or not, this ensures applying binlog events from the original table to the target table by enum textual value.

Backport

As this is a corruption bugfix, I believe this should be backported to all supported versions.

Related Issue(s)

Checklist

  • "Backport to:" labels have been added if this change should be back-ported to release branches
  • If this change is to be back-ported to previous releases, a justification is included in the PR description
  • Tests were added or are not required
  • Did the new or modified tests pass consistently locally and on CI?
  • Documentation was added or is not required

Deployment Notes

@github-actions github-actions bot added this to the v19.0.0 milestone Feb 1, 2024
Signed-off-by: Shlomi Noach <[email protected]>
Copy link

codecov bot commented Feb 1, 2024

Codecov Report

Attention: Patch coverage is 0% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 65.42%. Comparing base (d8f771c) to head (4fd765e).

Files Patch % Lines
go/vt/vttablet/onlineddl/vrepl.go 0.00% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #15103      +/-   ##
==========================================
+ Coverage   65.39%   65.42%   +0.02%     
==========================================
  Files        1561     1561              
  Lines      193524   193526       +2     
==========================================
+ Hits       126563   126609      +46     
+ Misses      66961    66917      -44     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@shlomi-noach
Copy link
Contributor Author

@frouioui frouioui modified the milestones: v19.0.0, v20.0.0 Feb 6, 2024
Signed-off-by: Shlomi Noach <[email protected]>
…to text or remains an ENUM

Signed-off-by: Shlomi Noach <[email protected]>
@@ -133,3 +133,248 @@ func TestGetSharedColumns(t *testing.T) {
})
}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This added test is mostly irrelevant to the changes in this PR, but while writing the PR I noticed how this was missing.

@@ -120,8 +120,8 @@ const (
)

const (
EnumReorderStrategyReject int = iota
EnumReorderStrategyAllow
EnumReorderStrategyAllow int = iota
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Subsequently to the main change in this PR, it's now fine to reorder enum values, and the default schemadiff hints enum-reordering strategy is now EnumReorderStrategyAllow

@shlomi-noach shlomi-noach marked this pull request as ready for review February 25, 2024 10:05
@shlomi-noach shlomi-noach requested a review from a team February 25, 2024 10:05
Copy link
Contributor

@dbussink dbussink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, great that tracking the schema is already solved. That makes this much easier!

@shlomi-noach
Copy link
Contributor Author

I believe we should backport this to all supported versions, as this is a bugfix in OnlineDDL/VReplication.

@shlomi-noach shlomi-noach added Backport to: release-17.0 Backport to: release-19.0 Needs to be back ported to release-19.0 labels Feb 25, 2024
@shlomi-noach shlomi-noach merged commit 47e1375 into vitessio:main Feb 25, 2024
105 checks passed
@shlomi-noach shlomi-noach deleted the onlineddl-vrepl-enum-reorder branch February 25, 2024 16:08
vitess-bot pushed a commit that referenced this pull request Feb 25, 2024
shlomi-noach pushed a commit that referenced this pull request Feb 25, 2024
#15352)

Signed-off-by: Shlomi Noach <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
shlomi-noach added a commit that referenced this pull request Feb 25, 2024
#15350)

Signed-off-by: Shlomi Noach <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Shlomi Noach <[email protected]>
shlomi-noach added a commit that referenced this pull request Feb 25, 2024
#15351)

Signed-off-by: Shlomi Noach <[email protected]>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Shlomi Noach <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backport to: release-19.0 Needs to be back ported to release-19.0 Component: Online DDL Online DDL (vitess/native/gh-ost/pt-osc) Component: VReplication Type: Bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OnlineDDL/VReplication: inconsistency between vcopier & vplayer when copying shuffled enum values
4 participants