We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is your feature request related to a problem? Please describe. Anthropic, in particular the models Claude 3.5 Sonnet and Claude 3 Haiku support prompt caching or context caching. Support for Claude 3 Opus is coming soon. We should enable users to use that feature through Haystack to reduce costs and latency. https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching https://www.anthropic.com/news/prompt-caching
Describe the solution you'd like We need to implement support for Anthropic's cache_control parameter in messages.
cache_control
The text was updated successfully, but these errors were encountered:
related issue #1003
Sorry, something went wrong.
To have a full benefit of prompt caching in Haystack pipelines we need to resolve #1011
ChatPromptBuilder
ChatMessage
Support for prompt caching added in https://pypi.org/project/anthropic-haystack/ 1.1.0 release. See our example on how to leverage prompt caching in a "chat with a long research paper" use case.
vblagoje
Successfully merging a pull request may close this issue.
Is your feature request related to a problem? Please describe.
Anthropic, in particular the models Claude 3.5 Sonnet and Claude 3 Haiku support prompt caching or context caching. Support for Claude 3 Opus is coming soon. We should enable users to use that feature through Haystack to reduce costs and latency.
https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
https://www.anthropic.com/news/prompt-caching
Describe the solution you'd like
We need to implement support for Anthropic's
cache_control
parameter in messages.The text was updated successfully, but these errors were encountered: