Return Source Documents in LECL

[drew-wks]

How do I get source documents to pass through the LECL pipeline? This was a argument in the LLMChain, but now I'm lost. Chat GPT and I have spent about 3 hours with Harrison's example, but honestly these do little for instruction. It crams a bunch of other concepts into each example, so I can't follow. Also there's the whole model vs llm, promptTemplate versus ChatPromptTemplate, Runnable this and iterable that. It's soooo confusing.

Can anyone suggest how to adapt this code to pull the sources through? If it's easier to do it with PromptTemplate, that's fine.

import openai
from langchain.prompts import load_prompt
from langchain.chat_models import ChatOpenAI
from langchain.schema.runnable import RunnablePassthrough, RunnableMap
from langchain.schema.output_parser import StrOutputParser


openai.api_key = os.environ['OPENAI_API_KEY']
model = ChatOpenAI()

prompt_template = load_prompt("generic_prompt_template.json")

rag_dates_lcel = (
    {"context": retriever, "question": RunnablePassthrough()}  # metadata passes
    | prompt_template
    | model
)

and the generic_prompt_template.json file looks like this:

{
    "_type": "prompt",
    "input_variables": ["context", "question"],
    "template": "Use the following pieces of context to answer the users question. Include any relevant conditions, requirements or standards in your response. The most recent documents should be given more weight in resolving contradictions.\nIf you don't know the answer, just say I am unable to find an answer in the USCG Auxiliary documents. Don't try to make up an answer.\n----------------\n{context}\n\nQuestion: {question}"
}

[dosubot[bot]]

🤖

Hi @drew-wks! I'm Dosu, an AI bot here to assist with your LangChain inquiries. I'm here to help you solve bugs, answer your questions, and guide you in becoming a contributor while we wait for a human maintainer. Let's get started!

Based on the issues and discussions I found in the LangChain repository, it seems like you can use the RetrievalQAWithSourcesChain class to return source documents. You can set the return_source_documents parameter to True during the chain's construction. Here's an example of how you can do this:

from langchain.chains import RetrievalQAWithSourcesChain
from langchain import OpenAI

# Create a document search object with source metadata
docsearch = Chroma.from_texts(texts, embeddings, metadatas=[{"source": f"{i}-pl"} for i in range(len(texts))])

# Create a chain with the document search object and specify that source documents should be returned
chain = RetrievalQAWithSourcesChain.from_chain_type(OpenAI(temperature=0), chain_type="stuff", retriever=docsearch.as_retriever(), return_source_documents=True)

# Use the chain to answer a question and return the source documents
result = chain({"question": "What did the president say about Justice Breyer"}, return_only_outputs=True)

# The result will include the answer and the source documents
print(result)

In this example, the result will be a dictionary that includes the answer to the question and the source documents used to generate the answer. The source documents are identified by the "source" metadata key in the document search object. You can verify the number of source documents returned by printing len(result['source_documents']). The first source document can be accessed with result['source_documents'][0].

You can also use the RetrievalQA class to return source documents by setting return_source_documents=True when constructing the chain.

Regarding the RunnablePassthrough and RunnableMap, these are used to pass through inputs unchanged or with additional keys. They can handle both synchronous and asynchronous operations. For example, RunnablePassthrough can be used to pass the input through while adding some keys to the output using the assign method.

I hope this helps! If you have any other questions, feel free to ask.

Sources

• DOC: ContextualCompressionRetriever - is it possible to retain sources
• Conversational Retrieval QA with sources cannot return source
• Retrieval QA and prompt templates
• libs/langchain/langchain/document_loaders/evernote.py
• libs/core/langchain_core/runnables/passthrough.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[drew-wks]

@dosu-bot: How do I return source documents using LangChain Expression Language? I am familiar with how to do it using the legacy chains.

[dosubot[bot]]

🤖

To return source documents using the LangChain Expression Language (LECL) in the LangChain Python framework, you can utilize the PromptTemplate class. This class is designed to define the structure of the prompts that will be used in the language model.

Here's an example of how you can use it:

EXAMPLE_PROMPT = PromptTemplate(
    template="Content: {page_content}\nSource: {source}",
    input_variables=["page_content", "source"],
)

In this example, page_content and source are the input variables that you can replace with the actual content and source of the document you want to return. The PromptTemplate will format the string according to the provided template and input variables.

This example is based on the usage of PromptTemplate in the libs/langchain/langchain/chains/qa_with_sources/map_reduce_prompt.py file in the LangChain repository.

As for the exact syntax and usage of the LangChain Expression Language (LECL), I wasn't able to find specific information within the repository. The same goes for the differences between using the legacy chains and the LECL for returning source documents.

If you could provide more details about your use case or specific issues you're encountering, I might be able to give a more precise answer.

Sources

• libs/langchain/langchain/chains/qa_with_sources/map_reduce_prompt.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[VerganiCodes]

@dosu-bot i couldn't understand your answer;
i'm having the same issue as @drew-wks i think; my application uses the RAG cookbook to make a simple chain with LCEL:
chain = (
{"context": vectorstore.as_retriever(), "question": RunnablePassthrough()}
| prompt
| model
| StrOutputParser()
)
how can i make it spit out both the answer and the context it used to generate it ?

[wangpengfei2048]

@dosu-bot i couldn't understand your answer;
i'm having the same issue as @drew-wks, my application uses the RAG cookbook to make a simple chain with LCEL:
chain = (
{"context": vectorstore.as_retriever(), "question": RunnablePassthrough()}
| prompt
| model
| StrOutputParser()
)

and ask it with the code:
res = cl.Message(content="")
async for chunk in runnable.astream(
question,
config=RunnableConfig(callbacks=[cl.LangchainCallbackHandler()]),
):
await res.stream_token(chunk)
await res.send()

but the result "res" is "Message", I cant get the source_documents

[RobertHH-IS]

Also interested in this

[jfahrenkrug]

I ran into the same thing and this works well for me:

    rag_chain = (
        RunnablePassthrough.assign(source_documents=condense_question | retriever)
        | RunnablePassthrough.assign(context=lambda inputs: format_docs(inputs["source_documents"]) if inputs["source_documents"] else "")
        | RunnablePassthrough.assign(prompt=qa_prompt)
        | RunnablePassthrough.assign(response=lambda inputs: llm(inputs["prompt"].messages))
    )

It's called like this:

    response_dict = rag_chain.invoke({"question": question, "chat_history": chat_history})
    ai_msg = response_dict["response"]
    source_documents = response_dict["source_documents"]

The way that helped me understand how to do it was this:

1. You initially pass a dictionary into the chain (in my case with the keys question and chat_history).
2. Every time you use RunnablePassthrough.assign, you can ADD stuff to that dictionary and then pass that on to the next step.
3. RunnablePassthrough.assign always RETURNS a dictionary.

This is what happens in my code example:

1. We use RunnablePassthrough.assign to add a new source_documents key to the dictionary. Its value is the result of calling the condense_question function (defined elsewhere) that builds and returns a condenser chain. Its condensed result is passed into our retriever (also defined elsewhere).
2. We use RunnablePassthrough.assign to add a new context key to the dictionary. Its value is the result of calling a format_docs method (defined elsewhere) that combines the source_documents into a single context string.
3. We use RunnablePassthrough.assign to add a new prompt key to the dictionary. Its value is the result of calling qa_prompt, which is defined as qa_prompt = ChatPromptTemplate.from_messages(...).
4. We use RunnablePassthrough.assign one more time to add a new response key to the dictionary. Its value is the result of actually calling the llm with the messages from our prompt.
5. The chain returns a dictionary with all the keys we've added along the way. The response key contains the LLM's response as an AIMessage, and the source_documents key contains the source documents.

I'm sure this can be done in a more concise way, but this worked for me and I can understand it :)

[mingjun1120]

Can I have a look on your full code so that I can have clearer picture?

[MathisBeom]

I agree, since there is no integration example, it would be really appreciated if you could share your code for people facing the same problem, thanks!

[cloudhuang]

From the documentation

from langchain_core.runnables import RunnableParallel

rag_chain_from_docs = (
    RunnablePassthrough.assign(context=(lambda x: format_docs(x["context"])))
    | prompt
    | llm
    | StrOutputParser()
)

rag_chain_with_source = RunnableParallel(
    {"context": retriever, "question": RunnablePassthrough()}
).assign(answer=rag_chain_from_docs)

rag_chain_with_source.invoke("What is Task Decomposition")

this approach can return the source documents:

{'context': [Document(page_content='Fig. 1. Overview of a LLM-powered autonomous agent system.\nComponent One: Planning#\nA complicated task usually involves many steps. An agent needs to know what they are and plan ahead.\nTask Decomposition#\nChain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to “think step by step” to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model’s thinking process.', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'start_index': 1585}),
  Document(page_content='Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.\nTask decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'start_index': 2192}),
  Document(page_content='The AI assistant can parse user input to several tasks: [{"task": task, "id", task_id, "dep": dependency_task_ids, "args": {"text": text, "image": URL, "audio": URL, "video": URL}}]. The "dep" field denotes the id of the previous task which generates a new resource that the current task relies on. A special tag "-task_id" refers to the generated text image, audio and video in the dependency task with id as task_id. The task MUST be selected from the following options: {{ Available Task List }}. There is a logical relationship between tasks, please note their order. If the user input can\'t be parsed, you need to reply empty JSON. Here are several cases for your reference: {{ Demonstrations }}. The chat history is recorded as {{ Chat History }}. From this chat history, you can find the path of the user-mentioned resources for your task planning.', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'start_index': 17804}),
  Document(page_content='Fig. 11. Illustration of how HuggingGPT works. (Image source: Shen et al. 2023)\nThe system comprises of 4 stages:\n(1) Task planning: LLM works as the brain and parses the user requests into multiple tasks. There are four attributes associated with each task: task type, ID, dependencies, and arguments. They use few-shot examples to guide LLM to do task parsing and planning.\nInstruction:', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'start_index': 17414}),
  Document(page_content='Resources:\n1. Internet access for searches and information gathering.\n2. Long Term memory management.\n3. GPT-3.5 powered Agents for delegation of simple tasks.\n4. File output.\n\nPerformance Evaluation:\n1. Continuously review and analyze your actions to ensure you are performing to the best of your abilities.\n2. Constructively self-criticize your big-picture behavior constantly.\n3. Reflect on past decisions and strategies to refine your approach.\n4. Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps.', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'start_index': 29630}),
  Document(page_content="(3) Task execution: Expert models execute on the specific tasks and log results.\nInstruction:\n\nWith the input and the inference results, the AI assistant needs to describe the process and results. The previous stages can be formed as - User Input: {{ User Input }}, Task Planning: {{ Tasks }}, Model Selection: {{ Model Assignment }}, Task Execution: {{ Predictions }}. You must first answer the user's request in a straightforward manner. Then describe the task process and show your analysis and model inference results to the user in the first person. If inference results contain a file path, must tell the user the complete file path.", metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'start_index': 19373})],
 'question': 'What is Task Decomposition',
 'answer': 'Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. It involves transforming big tasks into multiple manageable tasks, allowing for a more systematic and organized approach to problem-solving. Thanks for asking!'}

[mingjun1120]

What if I want to add chat history into it also? How should I modify the code?

[dylanlap98]

this is exactly what I am trying to figure out now. Not sure where to go from here. Examples using this doc returning source are scarce.. 😵‍💫

[gyi123]

I tried this follwoing the document, but I only get stream endpoint working. The invoke endpoint just have 'question' and 'answer', no 'context' element.
Any idea?

[bhargav-11]

why langchain has to do this ?
Messing up everythig while it was easy to understand when everything was at one place and now everything is mess.

[vijay-ravi]

Damn. I will continue to use the legacy chain for this one. Seems overcomplicated with LCEL lol.

[MathisBeom]

How can this be achieved with the legacy chain tho ? Do you have any examples please ?

[drew-wks]

My repo has some some functions that may help with legacy approach: drew-wks/ASK/ASK_inference.py

rag(query, retriever)  # Shows how to retrieve info from your vectorstore
create_short_source_list(response)  # Displays list of source documents 
create_long_source_list(response)  # Displays contents of the source document

The UI is located here: https://uscg-auxiliary-ask.streamlit.app/