Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

limit search thread pool size #5304

Closed
wants to merge 3 commits into from
Closed

limit search thread pool size #5304

wants to merge 3 commits into from

Conversation

PSeitz
Copy link
Contributor

@PSeitz PSeitz commented Aug 8, 2024

Don't take all threads for search, reserve one thread for incoming requests. Otherwise there may not be enough CPU left to answer health checks
This behavior can be overruled by manually setting the search thread pool size via QW_SEARCH_THREAD_POOL_SIZE.

improve logging:
log actual indices that are being queried on leaf nodes instead of the patterns from the user request
reduce env var spam

Related to #5305

Copy link

github-actions bot commented Aug 8, 2024

On SSD:

Average search latency is 1.03x that of the reference (lower is better).
Ref run id: 2754, ref commit: e911e99
Link

On GCS:

Average search latency is 1.03x that of the reference (lower is better).
Ref run id: 2755, ref commit: e911e99
Link

@PSeitz PSeitz requested review from fulmicoton and fmassot August 9, 2024 06:58
@PSeitz PSeitz changed the title add env parameter for thread pool size limit search thread pool size Aug 14, 2024
@fmassot
Copy link
Contributor

fmassot commented Aug 26, 2024

I suggest closing this PR as we are not reproducing the health checks issue with heavy CPU queries

@PSeitz
Copy link
Contributor Author

PSeitz commented Aug 27, 2024

Yes, I'll move the logging improvements to a new PR

@PSeitz PSeitz closed this Aug 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants