Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support PITs (points in time) and search_after #872

Open
abartov opened this issue Jan 31, 2023 · 4 comments
Open

Support PITs (points in time) and search_after #872

abartov opened this issue Jan 31, 2023 · 4 comments

Comments

@abartov
Copy link

abartov commented Jan 31, 2023

Paging through large result sets (in my case, about 40k results) is memory expensive when just increase the max result window. The recommended method by Elastic is using PITs and search_after, but I don't see support for that in Chewy.

It would be great to be able to use search_after, or even to benefit from it when requesting it with a flag, and simply calling .page() to get a particular results page.

@BagpipesJohnson
Copy link

We also would like to be able to use point_in_time. If there is a way to do this without forking chewy ourselves, that would be huge.

https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html

@abartov
Copy link
Author

abartov commented May 19, 2023

Perhaps @konalegi or someone could at least review this feature request and indicate whether support is planned?

@konalegi
Copy link

Added to my todo list to take a look, I cannot give quick answer right now.

@BagpipesJohnson
Copy link

Is there any way that we can add to the request body ourselves?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants