Update ChatAnthropic class langchain-anthropic with new function calling features available in Anthropic API
#21871
Replies: 4 comments 14 replies
-
|
@ccurme It looks like you are the most active developer for langchain-anthropic. Any idea about when this will be implemented? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tommasodelorenzo, thanks for the ping on this.
Can you clarify what you mean by vision inputs? If you mean a model calling tools in response to an image input, this should be supported: import base64
from typing import Literal
import httpx
from langchain_anthropic import ChatAnthropic
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
image_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
@tool
def weather_tool(weather: Literal["sunny", "cloudy", "rainy"]) -> None:
"""Describe the weather"""
pass
image_data = base64.b64encode(httpx.get(image_url).content).decode("utf-8")
model = ChatAnthropic(model="claude-3-sonnet-20240229").bind_tools([weather_tool])
message = HumanMessage(
content=[
{"type": "text", "text": "describe the weather in this image"},
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/jpeg",
"data": image_data,
},
},
],
)
response = model.invoke([message])
print(response.tool_calls)If you mean processing images as the result of a tool call, there's an open PR for that that should be merged shortly. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tommasodelorenzo (cc @ccurme) about your first point, are you able to stream tool calls already? I am using langchain-anthropic 0.1.19 and I am not able to do it (I am using AgentExecutor and it just stops when it needs to call a tool). Would be great if you could let me know! Thanks! (: |
Beta Was this translation helpful? Give feedback.
-
|
Streaming tool calls is released: from langchain_anthropic import ChatAnthropic
def magic_function(input: int) -> int:
"""Applies a magic function to an input."""
return input + 2
llm = ChatAnthropic(
model="claude-3-haiku-20240307",
).bind_tools([magic_function])
full = None
for chunk in llm.stream("What is magic_function(3)?"):
full = chunk if full is None else full + chunk
print(chunk.content)print(full.tool_calls)@Thimows the default behavior of |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @ccurme! I am actually already using stream events (astream_events) with ChatOpenAI and everything is working perfectly. However, I want to implement ChatAnthropic as well. Everything seems to work well, however when the agent needs to call a tool the event stream just stops (no errors). Do you know if I am overlooking something? If you could point me in the right direction that would be immensely helpful. Thank you so much (: Example of the behavior (it stops when it needs to call a tool): |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Thimows, this may be a bug related to your tool having no arguments (if I'm reading that correctly). Created an issue to track: #23911 |
Beta Was this translation helpful? Give feedback.
-
|
@ccurme Ah that was it! The tool does not have arguments indeed. When I try a different tool that does have arguments everything works as expected. Hopefully this can be fixed in a future update. Thanks again! (: |
Beta Was this translation helpful? Give feedback.
-
|
It works for me! |
Beta Was this translation helpful? Give feedback.
-
|
Just opened an issue about it #24136 |
Beta Was this translation helpful? Give feedback.
-
|
I have some issues with the current version as well (langchain = "0.2.7", langchain-anthropic = "0.1.20"). I use different models, and when using Anthropic with tool calling, the output of the Is this behaviour expected? I expected that LangChain handles it so that the LLMs are interchangeable. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a lot @ccurme for the swift reply, I'll have a look at the proposed solution and provide some feedback asap |
Beta Was this translation helpful? Give feedback.
-
|
@ccurme the problem persists, here you can find an example code to reproduce it from langchain_anthropic import ChatAnthropic
from langchain_openai import ChatOpenAI
import asyncio
from langgraph.graph import END, START, StateGraph
from typing import Annotated, Sequence, TypedDict
from langchain_core.messages import BaseMessage
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages
from pydantic import BaseModel as BaseModel
from pydantic import Field as Field
from langchain_core.output_parsers import StrOutputParser
class AgentState(TypedDict):
# The add_messages function defines how an update should be processed
# Default is to replace. add_messages says "append"
messages: Annotated[Sequence[BaseMessage], add_messages]
class cited_answer(BaseModel):
"""Answer the user question based only on the given sources, and cite the sources used."""
answer: str = Field(
...,
description="The answer to the user question, which is based only on the given sources.",
)
citations: list[int] = Field(
...,
description="The integer IDs of the SPECIFIC sources which justify the answer.",
)
followup_questions: list[str] = Field(
...,
description="Generate up to 3 follow-up questions that could be asked based on the answer given or context provided.",
)
def create_graph():
workflow = StateGraph(AgentState)
workflow.add_node("generate_answer", generate_answer)
workflow.add_edge(START, "generate_answer")
workflow.add_edge("generate_answer", END)
return workflow.compile()
prompt = """
System: Your name is Quivr. You're a helpful assistant. Today's date is November 19, 2024. - When answering use markdown. Use markdown code blocks for code snippets.
- Answer in a concise and clear manner.
- If no preferred language is provided, answer in the same language as the language used by the user.
- You must use ONLY the provided context to answer the question. Do not use any prior knowledge or external information, even if you are certain of the answer.
- Do not apologize when providing an answer.
- Don't cite the source id in the answer objects, but you can use the source to answer the question.
System:
- You have access to the following internal reasoning to provide an answer: ['The current system prompt already contains the exact same instruction "Responses will be provided in French." Therefore, no changes are needed to the prompt as it already captures the user\'s requirement clearly and concisely.', "The instruction about providing responses in French is a language requirement and doesn't require any specific tools. The available tool (tavily search engine) is not necessary for language-based instructions. Therefore, no tools need to be activated."]
- You have access to the following files to answer the user question (limited to first 20 files): None
- You have access to the following context to answer the user question: Filename: https://aipure.ai/products/phospho
Source: 0
Phospho is an open-source text analytics platform for LLM apps that helps monitor, analyze, and improve AI-powered products. https://phospho.ai/ It provides developers and product teams with tools to monitor LLM app interactions, extract insights from text data, and continuously improve model performance. Phospho is an all-in-one text analytics platform for LLM apps that helps developers and product managers monitor user interactions, detect issues, extract insights, and continuously improve their AI products. Phospho is an open-source text analytics platform for LLM (Large Language Model) apps. Latest AI Tools Similar to Phospho ---------------------------------- LLMChat is a privacy-focused web application that allows users to interact with multiple AI language models using their own API keys, enhanced with plugins and personalized memory features. Popular AI Tools Like Phospho -----------------------------
Filename: https://aipure.ai/products/phospho/howto
Source: 1
Phospho is an open-source text analytics platform for LLM apps that helps monitor, analyze, and improve AI-powered products.
Filename: https://blog.phospho.ai/how-to-use-phospho-ai-to-improve-your-ai-software-in-2024/
Human: Original task: Tell me about the startup Phospho AI
Rephrased and contextualized task: ['What is Phospho AI and what does it do?']
"""
def magic_function(input: int) -> int:
"""Applies a magic function to an input."""
return input + 2
llm = ChatAnthropic(
model="claude-3-5-sonnet-20240620",
).bind_tools([cited_answer])
llm = ChatOpenAI(
model="gpt-4o",
).bind_tools([cited_answer])
def generate_answer(state: AgentState):
messages = state["messages"]
user_question = messages[0].content
chain = StrOutputParser() | llm
response = chain.invoke(user_question)
return {**state, "messages": [response]}
version = "v2"
metadata = {}
data = {
"messages": [("user", prompt)]
}
chain = create_graph()
async def process_stream():
full = None
async for event in chain.astream_events(data, version=version, config={"metadata": metadata}):
if event["event"] == "on_chat_model_stream":
#print("\n",event)
chunk = event["data"]["chunk"]
print("\n",chunk)
full = chunk if full is None else full + chunk
print(chunk.content)
asyncio.run(process_stream())When run with OpenAI, the output is consistent and looks like while when running with Anthropic the first chunks are as expected but then the output of the tool call is returned differently, as and the partial_json cannot be streamed anymore, thus breaking any code using the streamed output |
Beta Was this translation helpful? Give feedback.
-
|
(This snippet could be much simpler, but I tried to mimic as mush as possible what we use in our library quivr) |
Beta Was this translation helpful? Give feedback.
-
|
If you want the text content of a message: parser = StrOutputParser()
parser.invoke(chunk)If you want the tool calls from a message: chunk.tool_callsIf you want partial JSON tool calls from a message (see more on streaming tool calls here): chunk.tool_call_chunksThese should be formatted consistently across providers. Also, I see you're defining |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @ccurme. Does this mean that I cannot stream the output of tool calls from Anthropic (since I only get the partial JSON / |
Beta Was this translation helpful? Give feedback.
-
Checked
Feature request
Update the
ChatAnthropicclass to work with the newest function calling features. Notably streaming, force tool use and vision inputs.Motivation
Streaming and force tool use were a long waited feature for agents based on tools calling with Anthropic models.
Proposal (If applicable)
I tried setting
default_headers = {"anthropic-beta": "tools-2024-05-16"}but it does not seem to work.Beta Was this translation helpful? Give feedback.
All reactions