The langchain and llama3.3 70b running agent cannot invoke the tool

[iamrraj]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import boto3
from langchain_community.llms.bedrock import Bedrock
bedrock_client = boto3.client(
            service_name="bedrock-runtime",
            region_name="us-east-1"
)
        
# Initialize Bedrock LLM
llm = Bedrock(
    model_id=DEFAULT_TEXT_LLAMA,
    client=bedrock_client,
    model_kwargs={'max_gen_len':512}
)


from langchain.agents import AgentType, initialize_agent
agent = initialize_agent(toolkit.get_tools(), llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)

qa_chain = agent.invoke({"input": "How far are these cities to Tokyo? New York City"})



Entering new AgentExecutor chain...
---------------------------------------------------------------------------
ValidationException                       Traceback (most recent call last)
File /opt/homebrew/lib/python3.11/site-packages/langchain_community/llms/bedrock.py:547, in BedrockBase._prepare_input_and_invoke(self, prompt, system, messages, stop, run_manager, **kwargs)
    546 try:
--> 547     response = self.client.invoke_model(**request_options)
    549     text, body, usage_info = LLMInputOutputAdapter.prepare_output(
    550         provider, response
    551     ).values()

File /opt/homebrew/lib/python3.11/site-packages/botocore/client.py:569, in ClientCreator._create_api_method.<locals>._api_call(self, *args, **kwargs)
    568 # The "self" in this scope is referring to the BaseClient.
--> 569 return self._make_api_call(operation_name, kwargs)

File /opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1023, in BaseClient._make_api_call(self, operation_name, api_params)
   1022     error_class = self.exceptions.from_code(error_code)
-> 1023     raise error_class(parsed_response, operation_name)
   1024 else:

ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: #: required key [prompt] not found#: extraneous key [inputText] is not permitted, please reformat your input and try again.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[36], line 1
----> 1 qa_chain = agent.invoke({"input": "How far are these cities to Tokyo? New York City"})
...
--> 554     raise ValueError(f"Error raised by bedrock service: {e}")
    556 if stop is not None:
    557     text = enforce_stop_tokens(text, stop)

ValueError: Error raised by bedrock service: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: #: required key [prompt] not found#: extraneous key [inputText] is not permitted, please reformat your input and try again.
Ou

### Description

I'm encountering an issue while using the LangChain library with AWS Bedrock LLama 3 integration. When invoking the agent with the input key, I receive a ValidationException from the Bedrock service, stating that the request is malformed due to missing the prompt key and an extraneous inputText key.

Observations:
The Bedrock client appears to expect the key prompt in the request, but LangChain's AgentType.ZERO_SHOT_REACT_DESCRIPTION seems to default to using input.
Changing the input dictionary to use the prompt key instead of input results in another error indicating that the input key is missing.
Steps Taken:
Tried switching the key from input to prompt in the agent.invoke method.
Confirmed that the Bedrock client is initialized properly and matches the required configurations.
Verified the toolkit setup to ensure compatibility with the agent.
Expected Behavior:
The invoke method should send a correctly formatted request to the Bedrock client, adhering to its API specifications.

Actual Behavior:
The request is rejected as malformed, either due to a missing prompt key or an unexpected inputText key.



### System Info

System Information
------------------
> OS:  Darwin
> OS Version:  Darwin Kernel Version 24.0.0: Mon Aug 12 20:52:31 PDT 2024; root:xnu-11215.1.10~2/RELEASE_ARM64_T6030
> Python Version:  3.11.9 (main, Apr  2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)]

Package Information
-------------------
> langchain_core: 0.3.28
> langchain: 0.3.13
> langchain_community: 0.3.13
> langsmith: 0.1.142
> langchain_aws: 0.2.10
> langchain_openai: 0.1.25
> langchain_text_splitters: 0.3.4
> langchainhub: 0.1.21

Optional packages not installed
-------------------------------
> langserve

Other Dependencies
------------------
> aiohttp: 3.10.3
> async-timeout: Installed. No version info available.
> boto3: 1.35.90
> dataclasses-json: 0.6.7
> httpx: 0.27.2
> httpx-sse: 0.4.0
> jsonpatch: 1.33
> numpy: None
> openai: 1.58.1
> orjson: 3.10.7
> packaging: 23.2
> pydantic: 2.9.2
> pydantic-settings: 2.6.1
> PyYAML: 6.0.1
> requests: 2.32.3
> requests-toolbelt: 1.0.0
> SQLAlchemy: 2.0.36
> tenacity: 8.2.3
> tiktoken: 0.7.0
> types-requests: 2.32.0.20241016
> typing-extensions: 4.12.2