How to get OpenAI with_structured_output token usage with Pydantic Schema

[zahidekilic]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from typing import TypedDict
from langchain_openai import ChatOpenAI

# Define the structured output schema
class UserInput(TypedDict):
    info_request: str
    analysis_request: str
    other: str

# Initialize the ChatOpenAI instance
co = ChatOpenAI(
    api_key=GPT_API_KEY, 
    model="gpt-4o-mini", 
    temperature=0
)

# Configure the model for structured output
structured_co = co.with_structured_output(UserInput, include_raw=True, method="json_mode")
structured_co2 = co.with_structured_output(UserInput)

try:
    prompt = "I want to perform analysis but don’t know how to do it"
    
    # Invoke the model
    AI_Response = structured_co.invoke(input=prompt)
    AI_Response_2 = structured_co2.invoke(input=prompt)

    # Parse the responses
    response = AI_Response['parsed'] 
    print("**1111111**")
    print(response)
    print("**22222**")
    print(AI_Response_2)

except Exception as e:
    print(f"An error occurred: {e}")

Description

Outputs:

Here are the outputs I received:

**1111111**
info_request='A situation where the user requests information, explanation, or a report (e.g., asking whether an analysis can be performed).' analysis_request='A situation where the user requests data processing, analysis, or visualization (e.g., requesting to perform operations on existing data/files).' other='Other content that does not fit into the above categories (e.g., a file upload request, general conversations).'
**22222**
info_request='I want to perform analysis but don’t know how to do it' analysis_request='' other=''

---

System Info

Problem:

I want to achieve the output in the second format (structured output matching the user input), but when I use the second method, I am unable to access input_tokens or token usage details.

---

Request:

Can you provide guidance or an example that demonstrates how to:

1. Get structured outputs in the desired format (like the second example).
2. Access token usage information such as input_tokens at the same time?

Thank you!