remove hits beyond max requested hit #5180
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
due to the 1st case of what's described in #3650 , it can happen that we return up to
start_offset + 2 * max_hits
documents instead ofmax_hits
. This happens when a split first fails, and then succeed, but its result is just concatenated instead of doing a proper top-k.The fix consist in dropping the tail hits at the same time as the hits that should be omitted due to start_offset are removed.
How was this PR tested?
tested with a modified s3 that errors on a fraction of requests. Without the patch, i often get too many results (and sometime a double error so no response), with the patch, i either get the right number of docs, or said error.