Structured Output Fails with ChatTogether Model

[mail2mhossain]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_together import ChatTogether
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
from decouple import config

TOGETHER_API_KEY = config("TOGETHER_API_KEY")

llm = ChatTogether(
    together_api_key=TOGETHER_API_KEY,
    model= "meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo",
    temperature=0,
)


class Joke(BaseModel):
    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")

chain = llm.with_structured_output(Joke) 


results = chain.invoke("Tell me a joke about horse.")
print(results)

Description

I am trying to get a structured output from ChatTogether but failed. Getting following error:

openai.BadRequestError: Error code: 400 - {'error': {'message': 'meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo is not supported for JSON mode/function calling', 'type': 'invalid_request_error', 'param': None, 'code': 'constraints_model'}}

System Info

langchain Version: 0.3.2
langchain-together Version: 0.2.0
openai Version: 1.45.1

Windows 11

Python Version: 3.11.9