Finding the most relevant document from ConversationalRetrievalChain

[vp-82]

Hi,

I'm using the ConversationalRetrievalChain together with a vector store to retrieve information from a PDF:

llm = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo-16k-0613')
question_generator = LLMChain(llm=llm,
                              prompt=GERMAN_QG_PROMPT)
doc_chain = load_qa_with_sources_chain(llm,
                                      chain_type="stuff",
                                      prompt=GERMAN_QA_PROMPT,
                                      document_prompt=GERMAN_DOC_PROMPT
                                       )
chain = ConversationalRetrievalChain(
    retriever=doc_store.as_retriever(),
    question_generator=question_generator,
    combine_docs_chain=doc_chain,
    return_generated_question=False,
    return_source_documents=True
)

result1 = chain({"question": query1, "chat_history": chat_history})

My question is: in the result, I get four documents (I think that's the default for k of the retriever) - how do I know which of the sources was the most relevant for the generated answer? I would like to show the most relevant page where the information was found; unfortunately, the first entry in the list is not always the best one.