You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Caching is awesome to save on tokens, but there are few scenarios where someone might want to opt out of it.
E.g. allowing to re-generate a response if the user doesn't like it. In this case the prompt is the same as a previous request (so the response would be taken from the cache) but we actually want to obtain a fresh new result.
I would then suggest to allow to opt out of caching on a request basis. Maybe using a specific header for this?
The text was updated successfully, but these errors were encountered:
mattiapomelli
changed the title
Allow to disable caching for certain requests
Allow to opt out of caching for certain requests
Aug 23, 2023
Caching is awesome to save on tokens, but there are few scenarios where someone might want to opt out of it.
E.g. allowing to re-generate a response if the user doesn't like it. In this case the prompt is the same as a previous request (so the response would be taken from the cache) but we actually want to obtain a fresh new result.
I would then suggest to allow to opt out of caching on a request basis. Maybe using a specific header for this?
The text was updated successfully, but these errors were encountered: