New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

search nodes unresponsive with expensive aggregation #5305

Open

PSeitz opened this issue Aug 8, 2024 · 1 comment

Labels

bug

Contributor

PSeitz commented Aug 8, 2024

As reported:

If I do an aggregation over a large time span, the searchers will end up being killed by kubernetes for missing health checks. I'd assume because the CPU usage is too high and they are mostly un-responsive

PSeitz added the bug label

PSeitz mentioned this issue

limit search thread pool size #5304

Closed

Contributor Author

PSeitz commented Aug 14, 2024

It seems there are two issues

Judging from the logs, the aggregation request is sent 50 times.
The search thread pool takes all CPUs. This may not leave enough resources to answer health checks.

#5304 takes all threads except one for the search thread pool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment