How to retrieve response_metadata when using structured output?

[mzw4]

Hi! Hopefully a simple question. If I have a model defined like so:

class Plan(BaseModel):
    steps: List[str]

model = llm.with_structured_output(Plan)
plan = llm.invoke('something')

and I want to see the response_metadata when I invoke it, how can I do that?

Normally it's on the message object, but since Plan is a custom model it only contains the fields I specified.

message = llm.invoke('something')
message.response_metadata

[darthShana]

here is what i would do

template = """Please provide a JSON response in the following format to the question provided, mark the json as ```json:
            {format_instructions}
            ---
            Question: 
             .....
            """

    parser = JsonOutputParser(pydantic_object=VehicleListing)

    prompt = PromptTemplate(
            template=template,
            input_variables=["page_content", "images"],
            partial_variables={"format_instructions": parser.get_format_instructions()}
        )
        chain = prompt | self.chat

SO we are asking the model to format the results but not restricting its full response
So then i would

        output_from_claude = chain.invoke({"foo": "goo"})
        message_json = parse_json_markdown(output_from_claude.content)
        message = VehicleListing.parse_obj(message_json)

now i have the message and the full output from the llm

[dsobczynski88]

Hello, thanks for responding to this. I actually had the same question. Just a clarification, what is self.chat referring to in this context? Just trying to reproduce the response. Thanks.

[mzw4]

Good idea, thanks! I'll do it that way. It'd be a nicer interface if all responses returned a Message object as a standard and content or another field contained the structured output instead changing the response type altogether.

[dsobczynski88]

Could you show me a snippet if you were able to get this to work? Thanks very much.

[mzw4]

I don't think I ended up doing that way actually. Instead I used the param include_raw=True then you get the response metadata in the response in addition to the Pydantic object

model.with_structured_output(schema=output_type, include_raw=True)

[dsobczynski88]

Thanks so much for the reply, I will give it a try!! Have a great day.