using the citations for the langchain agent using the queryengine tool from llamaindex

[avs-qualgenai]

If I have used the queryenginetool from llamaindex and then used the langchain tool method to convert it into the langchain tool and then used it in the langchain agent. How can we cite the answers here ?

[dosubot[bot]]

Hey @avs-qualgenai! I'm here to help you out with any bugs, questions, or contributions you have. Let's tackle this together!

To cite answers when using the queryengine tool from llamaindex, converting it into a LangChain tool, and then using it in a LangChain agent, you can use function-calling to have the model specify which of the provided documents it's referencing. Here's a simplified approach:

1. Define a Pydantic model for the cited answer:

from pydantic import BaseModel, Field
from typing import List

class CitedAnswer(BaseModel):
    """Answer the user question based only on the given sources, and cite the sources used."""
    answer: str = Field(
        ...,
        description="The answer to the user question, which is based only on the given sources.",
    )
    citations: List[int] = Field(
        ...,
        description="The integer IDs of the SPECIFIC sources which justify the answer.",
    )

2. Bind the model to the language model tool:

structured_llm = llm.with_structured_output(CitedAnswer)

3. Format the documents with IDs and create a chain to process the question:

def format_docs_with_id(docs: List[Document]) -> str:
    formatted = [
        f"Source ID: {i}\nArticle Title: {doc.metadata['title']}\nArticle Snippet: {doc.page_content}"
        for i, doc in enumerate(docs)
    ]
    return "\n\n" + "\n\n".join(formatted)

class State(TypedDict):
    question: str
    context: List[Document]
    answer: CitedAnswer

def generate(state: State):
    formatted_docs = format_docs_with_id(state["context"])
    messages = prompt.invoke({"question": state["question"], "context": formatted_docs})
    response = structured_llm.invoke(messages)
    return {"answer": response}

graph_builder = StateGraph(State).add_sequence([retrieve, generate])
graph_builder.add_edge(START, "retrieve")
graph = graph_builder.compile()

4. Invoke the chain with a question to get the answer and citations:

result = graph.invoke({"question": "How fast are cheetahs?"})

This setup allows the model to provide an answer along with citations indicating which sources were used to justify the answer ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}