Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to opt out of caching for certain requests #25

Open
mattiapomelli opened this issue Aug 23, 2023 · 0 comments
Open

Allow to opt out of caching for certain requests #25

mattiapomelli opened this issue Aug 23, 2023 · 0 comments

Comments

@mattiapomelli
Copy link

Caching is awesome to save on tokens, but there are few scenarios where someone might want to opt out of it.

E.g. allowing to re-generate a response if the user doesn't like it. In this case the prompt is the same as a previous request (so the response would be taken from the cache) but we actually want to obtain a fresh new result.

I would then suggest to allow to opt out of caching on a request basis. Maybe using a specific header for this?

@mattiapomelli mattiapomelli changed the title Allow to disable caching for certain requests Allow to opt out of caching for certain requests Aug 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant