Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for bedrock mistral models and claude messages api #7543

Merged
merged 3 commits into from
Apr 23, 2024

Conversation

tstadel
Copy link
Member

@tstadel tstadel commented Apr 12, 2024

Related Issues

  • fixes #issue-number

Proposed Changes:

  • support Mistal AI model on Bedrock
  • support Anthropic Messages API on Bedrock to support latest models such as Claude v3

How did you test it?

  • added tests

Notes for the reviewer

  • all currently available Anthropic models on Bedrock are compatible with Messages API. However Claude v3 models are not compatible with the old Text Completion API. So best is to make Messages API the default. If you still want to use Text Completion API you can do so setting model_kwargs param use_messages_api: False.

Checklist

@github-actions github-actions bot added topic:tests topic:DX Developer Experience labels Apr 12, 2024
@github-actions github-actions bot added the type:documentation Improvements on the docs label Apr 12, 2024
@@ -27,7 +27,7 @@ repos:
- id: ruff

- repo: https://github.com/codespell-project/codespell
rev: v2.2.5
rev: b8ecc9b3acf31690c3cb2fc5bb03a3fbbbc2d7a3
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version supporting inline ignores has been merged but not released yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you super badly need this? Let's not "pollute" this PR with such details, wdyt?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vblagoje if we don't bump that, we can't use inline ignores such as the one in line 262 of haystack/nodes/prompt/invocation_layer/amazon_bedrock.py. So we would need to globally ignore "tral" for spell corrections which is worse than just bumping codespell from my view.

@tstadel
Copy link
Member Author

tstadel commented Apr 20, 2024

@vblagoje would also be nice to have in next release

@vblagoje
Copy link
Member

@tstadel will review and include 🙏

Copy link
Member

@vblagoje vblagoje left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, why do we need to change codespell to a specific version, can we go forward without this change and minimal impact to new code?

@@ -27,7 +27,7 @@ repos:
- id: ruff

- repo: https://github.com/codespell-project/codespell
rev: v2.2.5
rev: b8ecc9b3acf31690c3cb2fc5bb03a3fbbbc2d7a3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you super badly need this? Let's not "pollute" this PR with such details, wdyt?

@sjrl sjrl added this to the 1.25.3 milestone Apr 22, 2024
@ArzelaAscoIi
Copy link
Member

@tstadel can we merge this ?

@tstadel
Copy link
Member Author

tstadel commented Apr 23, 2024

@vblagoje if you're ok with my comment under #7543 (comment) feel free to merge.

@vblagoje
Copy link
Member

Ok let's merge this one @tstadel @ArzelaAscoIi

@tstadel tstadel merged commit 2634b41 into v1.x Apr 23, 2024
53 checks passed
@tstadel tstadel deleted the feat/bedrock_mistral_and_claude_messages_api_support branch April 23, 2024 13:17
vblagoje pushed a commit that referenced this pull request Apr 23, 2024
* feat: support for bedrock mistral models and claude messages api

* feat: support for bedrock mistral models and claude messages api

* fix mypy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic:DX Developer Experience topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants