GraphCypherQAChain how are schema and question instantiated under the hood

[rmccorkl]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

CYPHER_GENERATION_TEMPLATE = """Task:Generate Cypher statement to query a graph database.
Instructions:
Use only the provided relationship types and properties in the schema.
Do not use any other relationship types or properties that are not provided.
Schema:
{schema}
Note: Do not include any explanations or apologies in your responses.
Do not respond to any questions that might ask anything else than for you to construct a Cypher statement.
Do not include any text except the generated Cypher statement.
Examples: Here are a few examples of generated Cypher statements for particular questions:
Always use 'n' as the Return variable match 
# The question is: "Match all nodes of type PrincipleNode."
MATCH (n:PrincipleNode) RETURN n

The question is:
{question}"""



CYPHER_GENERATION_PROMPT = PromptTemplate(
    input_variables=["schema", "question"], template=CYPHER_GENERATION_TEMPLATE
)


chain = GraphCypherQAChain.from_llm(
        ChatOpenAI(temperature=0, model=model),
        graph=graph,
        verbose=True,
        cypher_prompt=CYPHER_GENERATION_PROMPT,
        return_direct=True,
        top_k = TOP_K
    )
    
    reponse = chain.invoke("Use 'n' as the Return variable match all nodes with label of ProcedureNode.")

Description

@DosuBot

How is the "question" and in-particular the "schema" variables injected into the prompt, as input variables they are declared but I don't understand how they are initialise. Obviously the invoke method initialises the question but what retrieves the schema. It seems this is done underneath the hood, ultimately i like to print out the finished prompt to check what has been constructed ? The reason being I suspect I'm generating large prompts because of the schema which is making expensive API calls. The verbose parameter does not print the prompt hence I would like to debug what is being created under the hood

System Info

langchain==0.2.5
langchain-community==0.2.5
langchain-core==0.2.7
langchain-experimental==0.0.61
langchain-google-vertexai==1.0.5
langchain-openai==0.1.8
langchain-text-splitters==0.2.1

[dosubot[bot]]

The "schema" variable in the GraphCypherQAChain class is retrieved and initialized through the construct_schema function, which filters and formats the schema based on included or excluded types. This function processes the structured schema from the GraphStore and returns a formatted string representing the schema. Here is the relevant code snippet:

def construct_schema(
    structured_schema: Dict[str, Any],
    include_types: List[str],
    exclude_types: List[str],
) -> str:
    """Filter the schema based on included or excluded types"""

    def filter_func(x: str) -> bool:
        return x in include_types if include_types else x not in exclude_types

    filtered_schema: Dict[str, Any] = {
        "node_props": {
            k: v
            for k, v in structured_schema.get("node_props", {}).items()
            if filter_func(k)
        },
        "rel_props": {
            k: v
            for k, v in structured_schema.get("rel_props", {}).items()
            if filter_func(k)
        },
        "relationships": [
            r
            for r in structured_schema.get("relationships", [])
            if all(filter_func(r[t]) for t in ["start", "end", "type"])
        ],
    }

    # Format node properties
    formatted_node_props = []
    for label, properties in filtered_schema["node_props"].items():
        props_str = ", ".join(
            [f"{prop['property']}: {prop['type']}" for prop in properties]
        )
        formatted_node_props.append(f"{label} {{{props_str}}}")

    # Format relationship properties
    formatted_rel_props = []
    for rel_type, properties in filtered_schema["rel_props"].items():
        props_str = ", ".join(
            [f"{prop['property']}: {prop['type']}" for prop in properties]
        )
        formatted_rel_props.append(f"{rel_type} {{{props_str}}}")

    # Format relationships
    formatted_rels = [
        f"(:{el['start']})-[:{el['type']}]->(:{el['end']})"
        for el in filtered_schema["relationships"]
    ]

    return "\n".join(
        [
            "Node properties are the following:",
            ",".join(formatted_node_props),
            "Relationship properties are the following:",
            ",".join(formatted_rel_props),
            "The relationships are the following:",
            ",".join(formatted_rels),
        ]
    )

To print out the finished prompt for debugging purposes, you can modify the test_graph_cypher_qa_chain function to include print statements. Here is an example of how you can do this:

def test_graph_cypher_qa_chain() -> None:
    template = """You are a nice chatbot having a conversation with a human.

    Schema:
    {schema}

    Previous conversation:
    {chat_history}

    New human question: {question}
    Response:"""

    prompt = PromptTemplate(
        input_variables=["schema", "question", "chat_history"], template=template
    )

    memory = ConversationBufferMemory(memory_key="chat_history")
    readonlymemory = ReadOnlySharedMemory(memory=memory)
    prompt1 = (
        "You are a nice chatbot having a conversation with a human.\n\n    "
        "Schema:\n    Node properties are the following:\n\nRelationship "
        "properties are the following:\n\nThe relationships are the "
        "following:\n\n\n    "
        "Previous conversation:\n    \n\n    New human question: "
        "Test question\n    Response:"
    )

    prompt2 = (
        "You are a nice chatbot having a conversation with a human.\n\n    "
        "Schema:\n    Node properties are the following:\n\nRelationship "
        "properties are the following:\n\nThe relationships are the "
        "following:\n\n\n    "
        "Previous conversation:\n    Human: Test question\nAI: foo\n\n    "
        "New human question: Test new question\n    Response:"
    )

    llm = FakeLLM(queries={prompt1: "answer1", prompt2: "answer2"})
    chain = GraphCypherQAChain.from_llm(
        cypher_llm=llm,
        qa_llm=FakeLLM(),
        graph=FakeGraphStore(),
        verbose=True,
        return_intermediate_steps=False,
        cypher_llm_kwargs={"prompt": prompt, "memory": readonlymemory},
        memory=memory,
    )
    
    # Run the chain and print the prompts for debugging
    print("Running first question...")
    chain.run("Test question")
    print(f"Prompt after first question: {prompt1}")
    
    print("Running second question...")
    chain.run("Test new question")
    print(f"Prompt after second question: {prompt2}")
    
    # If we get here without a key error, that means memory
    # was used properly to create prompts.
    assert True

This will print the prompts used in the GraphCypherQAChain to the console, allowing you to debug what is being created ^[1][2].

To continue talking to Dosu, mention @dosu.