Tiny Weights

[vmg]

Description

This is a new optimization pattern for the execution engine. The idea is to speed up comparison operations by embedding "tiny weights" inside sqltypes.Value during execution. A tiny weight is a 4-byte compressed form of the full weight string for the value (see: evalengine.TinyWeighter for a detailed description and implementation). Since we actually have 4 spare bytes inside sqltypes.Value, we can inject the weight there without increasing the allocation cost for our in-memory rows, and any further comparison operators will automatically make use of them. This makes e.g. sorting wildly more efficient, because most comparisons during the sort can be performed by comparing two uint32 integers, instead of doing a full collation-aware comparison.

Of course, two tiny weight strings can collide (as they're essentially a lossy form of the weight string), but this is perfectly safe because we always fall back to a full comparison of the two values whenever their tiny weight strings are identical.

The arewefastyet benchmark results are not wildly impressive because OLTP is actually a pathological case for this example. The strings that OLTP uses in sort queries in the benchmark are all numerical strings, so their alphabet is very reduced (10 possible characters), making the 4 byte string collide quite often. The improvement on these OLTP Distinct-sorted queries is just 15%.

goos: linux
goarch: amd64
pkg: vitess.io/vitess/go/vt/vtgate/endtoend
cpu: 13th Gen Intel(R) Core(TM) i9-13900K
                       │ baseline.txt │          sha-2e689b2faf.txt          │
                       │    sec/op    │    sec/op     vs base                │
OLTP/DistinctRanges-16    565.4µ ± 6%   479.3µ ± 31%  -15.22% (p=0.000 n=10)

If we were to craft a different benchmark with string columns that contain arbitrary UTF8 data, the improvement gets all the way to ~40% because all the comparisons during sorting are performed with the tiny weight strings.

The global improvement in arewefast is pretty good, particularly for latency:

https://benchmark.vitess.io/compare?ltag=369b6a1e55aecd98c3cf6d4366cfbcee0477c474&rtag=946eb31e74187866a4a7414ca5df1954435681da

Again I wouldn't pay much attention to OLTP here because it's not representative of real world data (:cry:), but the speed up for real queries that include SORT BY or DISTINCT will be significant.

cc @dbussink @systay

Related Issue(s)

Checklist

• "Backport to:" labels have been added if this change should be back-ported
• Tests were added or are not required
• Did the new or modified tests pass consistently locally and on the CI
• Documentation was added or is not required

Deployment Notes

[vitess-bot]

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

• Ensure that the Pull Request has a descriptive title.
• Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

• Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

• Apply the release notes (needs details) label if users need to know about this change.
• New features should be documented.
• There should be some code comments as to why things are implemented the way they are.
• There should be a comment at the top of each new or modified test to explain what the test does.

New flags

• Is this flag really necessary?
• Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

• Each item in Jobs should be named in order to mark it as required.
• If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

• Protobuf changes should be wire-compatible.
• Changes to _vt tables and RPCs need to be backward compatible.
• RPC changes should be compatible with vitess-operator
• If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
• vtctl command output order should be stable and awk-able.

[vitess-bot]

Hello! 👋

This Pull Request is now handled by arewefastyet. The current HEAD and future commits will be benchmarked.

You can find the performance comparison on the arewefastyet website.

[systay]

we don't want to do this in the evalengine.NullsafeCompare instead?

[vmg]

I opted to wire up the comparison on the relevant callsites because there's a lot of places that don't use tiny weights right now. I think it'll be sensible to move it into NullsafeCompare once we wire up tiny weight generation in more paths.

[vmg]

This was actually huge in arewefast.