Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Solving indexing regression #5064

Merged
merged 1 commit into from
Jun 4, 2024
Merged

Solving indexing regression #5064

merged 1 commit into from
Jun 4, 2024

Conversation

fulmicoton
Copy link
Contributor

It was caused by the output compression.

Closes #5049

@fulmicoton fulmicoton requested a review from guilload June 3, 2024 01:38
guilload
guilload approved these changes Jun 3, 2024
View reviewed changes
Copy link
Member

@guilload guilload left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you think sooner or later, somebody is going to ask to put it back?

@fulmicoton
Copy link
Contributor Author

@guilload It was apparently introduced in #2958.
It was apparently motivated by large aggregations.

I can open an issue to readd it and run it in the cpu intensive thread pool

@fulmicoton
Copy link
Contributor Author

@guilload i ended it not removing but put it behind an environment variable. I also added support for zstd.

@rdettai @fulmicoton
Fix indexing regression by disabling payload compression
cba636c 
This disables default compression.
After this PR, compression is only enabled if the
`QW_MINIMUM_COMPRESSION_SIZE` environment variable is set.

This PR also enables ZSTD compression, and sets the compression
quality to "Fast".

Closes #5049
@fulmicoton fulmicoton force-pushed the issue/5049-indexing-regression branch from ccf4f47 to cba636c Compare June 4, 2024 02:17
@fulmicoton fulmicoton enabled auto-merge (squash) June 4, 2024 02:17
@fulmicoton fulmicoton disabled auto-merge June 4, 2024 02:48
@fulmicoton fulmicoton merged commit 5b90428 into main Jun 4, 2024
5 checks passed
@fulmicoton fulmicoton deleted the issue/5049-indexing-regression branch June 4, 2024 02:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guilload guilload guilload approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Ingest v2 Regression observed in main
3 participants
@fulmicoton @guilload @rdettai