Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ChatBedrock not working with latest nova models (amazon.nova-pro-v1:0 model) #308

Open
aashipandya opened this issue Dec 18, 2024 · 2 comments

Comments

@aashipandya
Copy link

aashipandya commented Dec 18, 2024

I am trying to use the new Amazon Bedrock model amazon.nova-pro-v1:0 with the ChatBedrock integration provided by LangChain-AWS. However, I encounter a validation error when I attempt to invoke the model.

Code to Reproduce
import boto3
from langchain_aws import ChatBedrock
import os

AWS credentials -
aws_access_key = "XYZ"
aws_secret_key = "XYZ"
region_name = "us-east-1"

Bedrock Runtime Client -
model_name = "amazon.nova-pro-v1:0"
bedrock_client = boto3.client(
service_name="bedrock-runtime",
region_name=region_name,
aws_access_key_id=aws_access_key,
aws_secret_access_key=aws_secret_key
)
llm = ChatBedrock(client=bedrock_client, model_id=model_name, model_kwargs=dict(temperature=0))
response = llm.invoke("Create a list of 3 pop songs")
print(response)

Error Output
ValidationError: 1 validation error for ChatBedrockConverse
Value error, Could not load credentials to authenticate with AWS client.
Please check that credentials in the specified profile name are valid.

Although If i try same credentials & same model with client.converse method it works.

response = bedrock_client.converse(
modelId=model_name,
messages=messages,
inferenceConfig={"temperature": 0.0}
)

Environment
• LangChain-AWS version: 0.2.9
• Python version: 3.10
• AWS SDK (boto3) version: 1.35.83

I am not sure if the issue is with credentials or ChatBedrock.

@mlrana
Copy link

mlrana commented Dec 19, 2024

I have also tried chatbedrock with nova models
throwing me this error
An error occurred (ValidationException) when calling the Converse operation: Invocation of model ID amazon.nova-micro-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.

Though chatbedrock is working with claude models.

@anujj-ti
Copy link

I am also facing similar issues.

It frequently gives this error:

An error occurred (ModelErrorException) when calling the Converse operation: The system encountered an unexpected error during processing. Try your request again.

My code snippet:


            client = ChatBedrock(
                model_id=model_id,
                temperature=temperature,
                max_tokens=max_tokens,
            )

            if tools:
                client = client.bind_tools(
                    tools=[tool.get_schema() for tool in tools],
                    tool_choice=tool_choice
                )

            response: AIMessage = await client.ainvoke(messages)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants