Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.16](backport #6192) Use xxhash instead of sha256 for hashing AST nodes #6250

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Dec 9, 2024

What does this PR do?

We currently use sha256 for hashing AST nodes when generating configuration. This is used both for checking equality between AST instances and for avoiding recomputing inputs unnecessarily. This PR changes this to xxHash, which is much faster. It also changes the interface, letting the caller supply their own hasher instance, and avoiding needing to allocate one for each Node in the tree.

Of note is that I'm using the xxhash.Digest struct as an argument instead of the generic hash.Hash64 interface from the standard library. The reason is that the former has an optimized WriteString method, which avoid needing to cast the string to a byte slice. I also don't expect to need to actually supply different hash implementations to this method.

Note also that there's already fairly exhaustive tests for this, some unexpected behaviour included.

Why is it important?

It's a pretty significant performance improvement. See the benchstat results using the benchmark from #6180:

goos: linux
goarch: amd64
pkg: github.com/elastic/elastic-agent/internal/pkg/agent/application/coordinator
cpu: 13th Gen Intel(R) Core(TM) i7-13700H
                                      │ bench_main.txt │           bench_hash.txt            │
                                      │     sec/op     │    sec/op     vs base               │
Coordinator_generateComponentModel-20     37.99m ± 24%   34.29m ± 14%  -9.73% (p=0.009 n=10)

                                      │ bench_main.txt │           bench_hash.txt            │
                                      │      B/op      │     B/op      vs base               │
Coordinator_generateComponentModel-20     34.22Mi ± 0%   31.53Mi ± 0%  -7.88% (p=0.000 n=10)

                                      │ bench_main.txt │           bench_hash.txt            │
                                      │   allocs/op    │  allocs/op   vs base                │
Coordinator_generateComponentModel-20      810.7k ± 0%   713.0k ± 0%  -12.05% (p=0.000 n=10)

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool

Related issues


This is an automatic backport of pull request #6192 done by [Mergify](https://mergify.com).

@mergify mergify bot requested a review from a team as a code owner December 9, 2024 17:07
@mergify mergify bot requested review from pkoutsovasilis and swiatekm and removed request for a team December 9, 2024 17:07
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Dec 9, 2024
Copy link
Contributor Author

mergify bot commented Dec 9, 2024

Cherry-pick of 9c13110 has failed:

On branch mergify/bp/8.16/pr-6192
Your branch is up to date with 'origin/8.16'.

You are currently cherry-picking commit 9c1311073b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   NOTICE.txt
	new file:   changelog/fragments/1733238171-use-xxhash-for-ast-hashing.yaml
	modified:   internal/pkg/agent/transpiler/utils.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   go.mod
	both modified:   internal/pkg/agent/transpiler/ast.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@swiatekm swiatekm force-pushed the mergify/bp/8.16/pr-6192 branch 3 times, most recently from 12bfb7a to 065c9c3 Compare December 10, 2024 11:55
@swiatekm swiatekm enabled auto-merge (squash) December 10, 2024 11:56
# Conflicts:
#	internal/pkg/agent/transpiler/ast.go

(cherry picked from commit 9c13110)

# Conflicts:
#	go.mod
#	internal/pkg/agent/transpiler/ast.go

# Conflicts:
#	internal/pkg/agent/transpiler/ast.go
@swiatekm swiatekm force-pushed the mergify/bp/8.16/pr-6192 branch from 065c9c3 to 65d4e5b Compare December 10, 2024 12:17
Copy link

Quality Gate failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 40%)

See analysis details on SonarQube

@swiatekm swiatekm merged commit 535051f into 8.16 Dec 10, 2024
13 of 14 checks passed
@swiatekm swiatekm deleted the mergify/bp/8.16/pr-6192 branch December 10, 2024 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport conflicts There is a conflict in the backported pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant