CodeGen: Rewrite dot product lowering using a dedicated IR instruction #1512
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This will help optimize lowering of vector.dot and vector.normalize
This exposes vdpps on X64 and allows to compute a 3-wide dot product for two vectors, returning the result as a number.
This is useful for vector.dot, vector.magnitude and vector.normalize.
This is using existing instructions and scalar adds to have a baseline. This is still faster than the original implementation of vector. ops.
We now support scalar version of faddp opcode which can add the first two floats of the vector into the first scalar of the destination.
This results in about the same performance as a naive version on M2, but uses fewer registers and is what clang generates for a similar source.
aviralg
approved these changes
Nov 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of doing the dot product related math in scalar IR, we lift the computation into a dedicated IR instruction.
On x64, we can use VDPPS which was more or less tailor made for this purpose. This is better than manual scalar lowering that requires reloading components from memory; it's not always a strict improvement over the shuffle+add version (which we never had), but this can now be adjusted in the IR lowering in an optimal fashion (maybe even based on CPU vendor, although that'd create issues for offline compilation).
On A64, we can either use naive adds or paired adds, as there is no dedicated vector-wide horizontal instruction until SVE. Both run at about the same performance on M2, but paired adds require fewer instructions and temporaries.
I've measured this using mesh-normal-vector benchmark, changing the benchmark to just report the time of the second loop inside
calculate_normals, testing master vs #1504 vs this PR, also increasing the grid size to 400 for more stable timings.On Zen 4 (7950X), this PR is comfortably ~8% faster vs master, while I see neutral to negative results in #1504.
On M2 (base), this PR is ~28% faster vs master, while #1504 is only about ~10% faster.
If I measure the second loop in
calculate_tangent_spaceinstead, I get:On Zen 4 (7950X), this PR is ~12% faster vs master, while #1504 is ~3% faster
On M2 (base), this PR is ~24% faster vs master, while #1504 is only about ~13% faster.
Note that the loops in question are not quite optimal, as they store and reload various vectors to dictionary values due to inappropriate use of locals. The underlying gains in individual functions are thus larger than the numbers above; for example, changing the
calculate_normalsloop to use a local variable to store the normalized vector (but still saving the result to dictionary value), I get a ~24% performance increase from this PR on Zen4 vs master instead of just 8% (#1504 is ~15% slower in this setup).