BedrockEmbeddings with Mistral or Anthropic models

[kamilcglr]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

BedrockEmbeddings(model_id="mistral.mixtral-8x7b-instruct-v0:1", model_kwargs={"max_tokens": 250}),

Description

Hello, is BedrockEmbeddings supposed to work with Mistral or Anthropic models ?.

When I use BedrockEmbeddings(model_id="mistral.mixtral-8x7b-instruct-v0:1", model_kwargs={"max_tokens" : 250}) I have the following error: "ValueError: Error raised by inference endpoint: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: # : required key [prompt] not found# : extraneous key [inputText] is not permitted, please reformat your input and try again. "

System Info

langchain==0.1.16
langchain-community==0.0.32
langchain-core==0.1.42
langchain-mistralai==0.1.2
langchain-text-splitters==0.0.1

macOS 14.3.1

python 3.9

[dosubot[bot]]

It looks like the issue you're encountering is due to a mismatch in the expected input format for the BedrockEmbeddings function. The error message indicates that the model is expecting a key named prompt but received inputText instead, which it does not recognize.

To fix this, you should adjust your input to include a prompt key with the text you want the model to process, and ensure that you do not include an inputText key, as it's not expected by the model's API.

The BedrockEmbeddings class is designed for computing embeddings with models like Mistral or Anthropic, and it expects input in a specific format. For document embeddings, you should provide a list of strings, and for query embeddings, a single string is expected.

Make sure your input data is correctly formatted according to these requirements. If you continue to face issues or have any more questions, feel free to ask for further assistance.

Sources

• Bedrock Mistral not working as agent
• libs/community/langchain_community/llms/bedrock.py
• libs/community/langchain_community/embeddings/bedrock.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[cktricky]

For anyone else that stumbles on this error and because documentation is non-obvious on this...

This error ultimately happens because we're trying to invoke embeddings on a model that does not support it and so the prompt error is a red herring. What is REALLY happening is that certain models do not support embedding on Bedrock (and so then the request doesn't make sense to AWS's API).

Amazon's Titan series LLMs and the Cohere LLMs on Bedrock offer embedding modalities - most others do not (as of now). Make sure you compare the models on Bedrock so that you know which models support embedding.

[luisandino]

thanks, it is still not functioning, using the following:

1. langchain 0.3.4
2. langchain_aws.BedRockEmbeddings
3. llama3.1