Add support for Amazon Bedrock models #35
Conversation
There was a problem hiding this comment.
Thank you so much @Cerrix !!! I just added a couple minor comments. Bare minimum let's change the requirements.txt file and improve the import error handling adding the environment variable warnings -- if you have time, would love to have multiple messages functionality as I added the comment (not required though).
Thanks again for your contribution!
requirements.txt
Outdated
| gdown>=5.2.0 | ||
| boto3>=1.34.133 |
There was a problem hiding this comment.
can we make this optional? i.e., without adding this to requirements.txt, can we just leave it to the import error handler (like we have for e.g., anthropic.py / openai.py etc., saying people who want to use bedrock models should install boto?
There was a problem hiding this comment.
For sure we can, I'm already managing the import error :) let me change that
| from botocore.config import Config | ||
|
|
||
| except ImportError: | ||
| raise ImportError("If you'd like to use Amazon Bedrock models, please install the boto3 package by running `pip install boto3`") |
There was a problem hiding this comment.
can we add the related environment variables that would be needed here? i.e. AWS_ACCESS_KEY_ID etc,
There was a problem hiding this comment.
Lines 63/68 handles the variables needed. But if you prefer I can move them also here
|
|
||
| return response | ||
|
|
||
| def generate( |
There was a problem hiding this comment.
this is great! Do you want to add support for multiple inputs to this engine? Like here?
There was a problem hiding this comment.
yes I can do! Can you provide me a simple example to test the implementation out?
Regarding to images only Anthropic models in Bedrock support vision so I'll enable that only for them!
|
Hi @mertyg thank you so much for the comments! I modified the requirements and commented out the other two comments (for env variables please check if you need them also at line 6, they are already specified at line 63/68). I'm happy to implement multiple inputs support but can you please provide me an example to test that out? I'll enable the vision capabilities only for Anthropic models (currently they are the only one to support vision in Amazon bedrock). thank you! |
|
It would be excellent to see this pushed through opening up quite a lot of models today and in the future. |
|
Hi, I would definitely make use of the bedrock version, do you have an estimated date for this branch ? thx |
Overview
This PR adds support for Amazon Bedrock, allowing TextGrad users to leverage a variety of foundation models from AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API.
Key Changes
get_enginefunctionbedrock-<model_name_id>(here the model id list)Usage
To use a Bedrock model, call the
get_enginefunction with the Bedrock model name ID prefixed by "bedrock-". For example:Authentication
The PR supports two authentication methods:
Supported Models
This PR adds support for the following model families:
A complete list of supported model IDs will be added to the documentation.
Testing
Feedback
I welcome any feedback or suggestions for improvement. Please let me know if any additional changes or clarifications are needed.