Structured Output Fails with Anthropic Model on Vertex AI

[abhayk05]

Bug: Structured Output Fails with Anthropic Model on Vertex AI

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Description

When using the with_structured_output function with the Anthropic model on Google Cloud Vertex AI, the structured output functionality doesn't work as expected. While it functions correctly with other models (including Gemini Pro, Gemini Flash, and OpenAI models), the Anthropic model on Vertex AI fails to use the PydanticToolsParser and instead defaults to the ToolsOutputParser.

Expected Behavior

The with_structured_output function should use the PydanticToolsParser and return a structured Joke object.

Actual Behavior

The function appears to use the ToolsOutputParser instead, and the response is None.

Additional Context

• This issue is specific to the Anthropic model on Vertex AI.
• The same code works correctly with other models, including:
  • ChatVertexAI with "gemini-1.5-flash"
  • ChatOpenAI with "gpt-4"
  • ChatAnthropic with "claude-3-5-sonnet-2024"

Steps to Reproduce

1. Set up a Pydantic model for structured output:

from pydantic import BaseModel, Field
class Joke(BaseModel):
setup: str = Field(description="The setup of the joke")
punchline: str = Field(description="The punchline to the joke")```



2. Initialize the Anthropic model on Vertex AI:
```python
AntrhropicOnVertexllm = ChatAnthropicVertex(
model_name="claude-3-5-sonnet@20240620",
location=VERTEX_LOCATION,
project=GCP_PROJECT,
temperature=TEMPRATURE
)```


3. Attempt to use structured output:
```python
structured_llm = AntrhropicOnVertexllm.with_structured_output(Joke)
response = structured_llm.invoke("Tell me a joke about horse.")
print(response)```



Output of the print: `None`

### Other Models Tested (Working Correctly)


```python
ChatVertexAI with "gemini-1.5-flash" works perfectly fine
llmVertexflash = ChatVertexAI(
location=VERTEX_LOCATION,
project=GCP_PROJECT,
temperature=TEMPRATURE,
model="gemini-1.5-flash"
)

ChatOpenAI and ChatAnthropic model also works fine
openAimodel = ChatOpenAI(model="gpt-4", temperature=0)
Anthropicmodel = ChatAnthropic(model="claude-3-5-sonnet-2024")```



## System Info

- langchain==0.2.3
- langchain-anthropic==0.1.19
- langchain-community==0.2.4
- langchain-core==0.3.10
- langchain-google-vertexai==2.0.5
- langchain-openai==0.1.8
- langchain-text-splitters==0.2.1

Python version: 3.11.9


![Error Screenshot](https://github.com/user-attachments/assets/20e02ead-3450-4e78-a891-fa70549da566)