Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release of v18.0.0-rc1 #14136

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/code_freeze.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ jobs:
steps:
- name: Fail if Code Freeze is enabled
run: |
exit 1
exit 0
465 changes: 465 additions & 0 deletions changelog/18.0/18.0.0/changelog.md

Large diffs are not rendered by default.

303 changes: 303 additions & 0 deletions changelog/18.0/18.0.0/release_notes.md
harshit-gangal marked this conversation as resolved.
Show resolved Hide resolved

Large diffs are not rendered by default.

70 changes: 35 additions & 35 deletions changelog/18.0/18.0.0/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,12 @@
- [Updated to node v18.16.0](#update-node)
- **[Deprecations and Deletions](#deprecations-and-deletions)**
- [Deprecated Flags](#deprecated-flags)
- [Deleted Flags](#deleted-flags)
- [Deprecated Stats](#deprecated-stats)
- [Deleted Flags](#deleted-flags)
- [Deleted `V3` planner](#deleted-v3)
- [Deleted `k8stopo`](#deleted-k8stopo)
- [Deleted `vtgr`](#deleted-vtgr)
- [Deleted `query_analyzer`](#deleted-query_analyzer)
- [Deprecated VTBackup stat `DurationByPhase`](#deprecated-vtbackup-stat-duration-by-phase)
- [Deprecated VDiff v1](#deprecated-vdiff-v1)
- **[New Stats](#new-stats)**
- [VTGate Vindex unknown parameters](#vtgate-vindex-unknown-parameters)
- [VTBackup stat `Phase`](#vtbackup-stat-phase)
Expand Down Expand Up @@ -54,14 +52,14 @@ and API mode or [migrate your existing data from v2 to v3](https://etcd.io/docs/
#### <a id="new-flag-toggle-ers"/>VTOrc flag `--allow-emergency-reparent`

VTOrc has a new flag `--allow-emergency-reparent` that allows the users to toggle the ability of VTOrc to run emergency
reparent operations. The users that want VTOrc to fix the replication issues, but don't want it to run any reparents
should start using this flag. By default, VTOrc will be able to run `EmergencyReparentShard`. The users must specify the
reparent operations. Users that want VTOrc to fix the replication issues, but don't want it to run any reparents
should start using this flag. By default, VTOrc will be able to run `EmergencyReparentShard`. Users must specify the
flag to `false` to change the behaviour.

#### <a id="new-flag-errant-gtid-convert"/>VTOrc flag `--change-tablets-with-errant-gtid-to-drained`

VTOrc has a new flag `--change-tablets-with-errant-gtid-to-drained` that allows users to choose whether VTOrc should change the
tablet type of tablets with errant GTIDs to `DRAINED`. By default, the flag is false.
tablet type of tablets with errant GTIDs to `DRAINED`. By default, it is disabled.

This feature allows users to configure VTOrc such that any tablet that encounters errant GTIDs is automatically taken out of the
serving graph. These tablets can then be inspected for what the errant GTIDs are, and once fixed, they can rejoin the cluster.
Expand All @@ -71,19 +69,19 @@ serving graph. These tablets can then be inspected for what the errant GTIDs are
Running `EmergencyReparentShard` from the vtctldclient has a new sub-flag `--wait-for-all-tablets` that makes `EmergencyReparentShard` wait
for a response from all the tablets. Originally `EmergencyReparentShard` was meant only to be run when a primary tablet is unreachable.
We have realized now that there are cases when the replication is broken but all the tablets are reachable. In these cases, it is advisable to
call `EmergencyReparentShard` with `--wait-for-all-tablets` so that it doesn't ignore one of the tablets.
call `EmergencyReparentShard` with `--wait-for-all-tablets` so that it does not ignore one of the tablets.

#### <a id="new-vtgate-streaming-sesion"/>VTGate GRPC stream execute session flag `--grpc-send-session-in-streaming`

This flag enables transaction support on `StreamExecute` api.
One enabled, VTGate `StreamExecute` grpc api will send session as the last packet in the response.
Once enabled, VTGate `StreamExecute` gRPC api will send session as the last packet in the response.
The client should enable it only when they have made the required changes to expect such a packet.

It is disabled by default.

### <a id="foreign-keys"/>Experimental Foreign Key Support

A new field `foreignKeyMode` has been added to the Vschema. This field can be provided for each keyspace. The Vtgate flag `--foreign_key_mode` has been deprecated in favour of this field.
A new field `foreignKeyMode` has been added to the VSchema. This field can be provided for each keyspace. The VTGate flag `--foreign_key_mode` has been deprecated in favour of this field.

There are 3 foreign key modes now supported in Vitess -
1. `FK_UNMANAGED` -
Expand All @@ -94,27 +92,27 @@ There are 3 foreign key modes now supported in Vitess -
This enables seamless integration of VReplication with foreign keys.
For more details on what operations Vitess takes please refer to the [design document for foreign keys](https://github.com/vitessio/vitess/issues/12967).
3. `FK_DISALLOW` -
In this mode Vitess explicitly disallows any DDL statements that try to create a foreign key constraint. This mode is equivalent to running Vtgates with the flag `--foreign_key_mode=disallow`.
In this mode Vitess explicitly disallows any DDL statements that try to create a foreign key constraint. This mode is equivalent to running VTGate with the flag `--foreign_key_mode=disallow`.

#### Upgrade process

After upgrading from v17 to v18, the users should specify the correct foreign key mode for all their keyspaces in the Vschema using the new property. Once this change has taken effect, the deprecated flag `--foreign_key_mode` can be dropped from all the Vtgates.
After upgrading from v17 to v18, the users should specify the correct foreign key mode for all their keyspaces in the VSchema using the new property.
Once this change has taken effect, the deprecated flag `--foreign_key_mode` can be dropped from all the VTGates.

### <a id="vtadmin"/>VTAdmin

#### <a id="updated-node"/>vtadmin-web updated to node v18.16.0 (LTS)

Building vtadmin-web now requires node >= v18.16.0 (LTS). Breaking changes from v16 to v18 are listed
Building `vtadmin-web` now requires node >= v18.16.0 (LTS). Breaking changes from v16 to v18 are listed
in https://nodejs.org/en/blog/release/v18.0.0, but none apply to VTAdmin. Full details on v18.16.0 are listed
here https://nodejs.org/en/blog/release/v18.16.0.
on https://nodejs.org/en/blog/release/v18.16.0.

### <a id="deprecations-and-deletions"/>Deprecations and Deletions

#### <a id="deprecated-flags"/>Deprecated Command Line Flags

Throttler related `vttablet` flags:

- `--enable-lag-throttler` is now removed after being deprecated in `v17.0`
- `--throttle_threshold` is deprecated and will be removed in `v19.0`
- `--throttle_metrics_query` is deprecated and will be removed in `v19.0`
- `--throttle_metrics_threshold` is deprecated and will be removed in `v19.0`
Expand All @@ -135,11 +133,29 @@ Cache related `vtgate` flags:
- `--gate_query_cache_lfu` is deprecated and will be removed in `v19.0`. The query cache always uses a LFU implementation now.
- `--gate_query_cache_size` is deprecated and will be removed in `v19.0`. This option only applied to LRU caches, which are now unsupported.

VTGate flag:
VTGate flags:

- `--schema_change_signal_user` is deprecated and will be removed in `v19.0`
- `--foreign_key_mode` is deprecated and will be removed in `v19.0`. For more detail read the [foreign keys](#foreign-keys) section.

VDiff v1:

[VDiff v2 was added in Vitess 15.0](https://vitess.io/blog/2022-11-22-vdiff-v2/) and marked as GA in 16.0.
The [legacy v1 client command](https://vitess.io/docs/18.0/reference/vreplication/vdiffv1/) is now deprecated in Vitess 18.0 and will be **removed** in 19.0.
Please switch all of your usage to the [new VDiff client](https://vitess.io/docs/18.0/reference/vreplication/vdiff/) command ASAP.


#### <a id="deprecated-stats"/>Deprecated Stats

The following `EmergencyReparentShard` stats are deprecated in `v18.0` and will be removed in `v19.0`:
- `ers_counter`
- `ers_success_counter`
- `ers_failure_counter`

These metrics are replaced by [new reparenting stats introduced in `v18.0`](#vtctld-and-vtorc-reparenting-stats).

VTBackup stat `DurationByPhase` is deprecated. Use the binary-valued `Phase` stat instead.

#### <a id="deleted-flags"/>Deleted Command Line Flags

Flags in `vtcombo`:
Expand All @@ -163,6 +179,7 @@ Flags in `vttablet`:
- `--queryserver-config-stream-pool-prefill-parallelism`
- `--queryserver-config-transaction-pool-prefill-parallelism`
- `--queryserver-config-schema-change-signal-interval`
- `--enable-lag-throttler`

Flags in `vtctld`:
- `--vtctld_show_topology_crud`
Expand All @@ -172,15 +189,6 @@ Flags in `vtorc`:
- `--lock-shard-timeout`
- `--orc_web_dir`

#### <a id="deprecated-stats"/>Deprecated Stats

The following Emergency Reparent Shard stats are deprecated in `v18.0` and will be removed in `v19.0`:
- `ers_counter`
- `ers_success_counter`
- `ers_failure_counter`

These metrics are replaced by [new reparenting stats introduced in `v18.0`](#vtctld-and-vtorc-reparenting-stats).

#### <a id="deleted-v3"/>Deleted `v3` planner

The `Gen4` planner has been the default planner since Vitess 14. The `v3` planner was deprecated in Vitess 15 and has now been removed in this release.
Expand All @@ -198,14 +206,6 @@ The `vtgr` has been deprecated in Vitess 17, also see https://github.com/vitessi

The undocumented `query_analyzer` binary has been removed in Vitess 18, see https://github.com/vitessio/vitess/issues/14054.

#### <a id="deprecated-vdiff-v1"/>Deprecated VDiff v1

[VDiff v2 was added in Vitess 15.0](https://vitess.io/blog/2022-11-22-vdiff-v2/) and marked as GA in 16.0. The [legacy v1 client command](https://vitess.io/docs/18.0/reference/vreplication/vdiffv1/) is now deprecated in Vitess 18.0 and will be **removed** in 19.0. Please switch all of your usage to the [new VDiff client](https://vitess.io/docs/18.0/reference/vreplication/vdiff/) command ASAP.

#### <a id="deprecated-vtbackup-stat-duration-by-phase"/>Deprecated VTbackup stat `DurationByPhase`

VTBackup stat `DurationByPhase` is deprecated. Use the binary-valued `Phase` stat instead.

### <a id="new-stats"/>New stats

#### <a id="vtgate-vindex-unknown-parameters"/>VTGate Vindex unknown parameters
Expand Down Expand Up @@ -250,16 +250,16 @@ vtbackup_restore_count{component="BackupStorage",implementation="S3",operation="
#### <a id="vtctld-and-vtorc-reparenting-stats"/>VTCtld and VTOrc reparenting stats

New VTCtld and VTorc stats were added to measure frequency of reparents by keyspace/shard:
- `emergency_reparent_counts` - Number of times Emergency Reparent Shard has been run. It is further subdivided by the keyspace, shard and the result of the operation.
- `planned_reparent_counts` - Number of times Planned Reparent Shard has been run. It is further subdivided by the keyspace, shard and the result of the operation.
- `emergency_reparent_counts` - Number of times `EmergencyReparentShard` has been run. It is further subdivided by the keyspace, shard and the result of the operation.
- `planned_reparent_counts` - Number of times `PlannedReparentShard` has been run. It is further subdivided by the keyspace, shard and the result of the operation.

Also, the `reparent_shard_operation_timings` stat was added to provide per-operation timings of reparent operations.

### <a id="vttablet"/>VTTablet

#### <a id="vttablet-new-rpc-reset-sequences"/>New ResetSequences rpc

A new vttablet RPC `ResetSequences` has been added, which is being used by `MoveTables` and `Migrate` for workflows
A new VTTablet RPC `ResetSequences` has been added, which is being used by `MoveTables` and `Migrate` for workflows
where a `sequence` table is being moved (https://github.com/vitessio/vitess/pull/13238). This has an impact on the
Vitess upgrade process from an earlier version if you need to use such a workflow before the entire cluster is upgraded.

Expand Down
2 changes: 2 additions & 0 deletions changelog/18.0/README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
## v18.0
* **[18.0.0](18.0.0)**
* [Changelog](18.0.0/changelog.md)
* [Release Notes](18.0.0/release_notes.md)
2 changes: 1 addition & 1 deletion docker/release.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash
set -ex

vt_base_version='v18.0.0-SNAPSHOT'
vt_base_version='v18.0.0-rc1'
debian_versions='bullseye bookworm'
default_debian_version='bookworm'

Expand Down
20 changes: 10 additions & 10 deletions examples/compose/docker-compose.beginners.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ services:
- "3306"

vtctld:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
ports:
- "15000:$WEB_PORT"
- "$GRPC_PORT"
Expand All @@ -81,7 +81,7 @@ services:
condition: service_healthy

vtgate:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
ports:
- "15099:$WEB_PORT"
- "$GRPC_PORT"
Expand Down Expand Up @@ -111,7 +111,7 @@ services:
condition: service_healthy

schemaload:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
command:
- sh
- -c
Expand Down Expand Up @@ -144,12 +144,12 @@ services:
environment:
- KEYSPACES=$KEYSPACE
- GRPC_PORT=15999
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
volumes:
- .:/script

vttablet100:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
ports:
- "15100:$WEB_PORT"
- "$GRPC_PORT"
Expand Down Expand Up @@ -181,7 +181,7 @@ services:
retries: 15

vttablet101:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
ports:
- "15101:$WEB_PORT"
- "$GRPC_PORT"
Expand Down Expand Up @@ -213,7 +213,7 @@ services:
retries: 15

vttablet102:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
ports:
- "15102:$WEB_PORT"
- "$GRPC_PORT"
Expand Down Expand Up @@ -245,7 +245,7 @@ services:
retries: 15

vttablet103:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
ports:
- "15103:$WEB_PORT"
- "$GRPC_PORT"
Expand Down Expand Up @@ -277,7 +277,7 @@ services:
retries: 15

vtorc:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
command: ["sh", "-c", "/script/vtorc-up.sh"]
depends_on:
- vtctld
Expand Down Expand Up @@ -307,7 +307,7 @@ services:
retries: 15

vreplication:
image: vitess/lite:${VITESS_TAG:-latest}
image: vitess/lite:v18.0.0-rc1
volumes:
- ".:/script"
environment:
Expand Down
Loading
Loading