Chain invoke and passing input variables is inconsistent

[rmccorkl]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

base_dir=os.environ["BASE_PATH"]
schema_file=os.environ["SCHEMA"]
def load_schema_data():
    schema = os.path.join(base_dir, schema_file)
    if os.path.exists(schema):
        with open(schema, 'r') as file:
            schema_data = file.read()
    else:
        schema_data = graph.get_schema
    return schema_data

CYPHER_GENERATION_TEMPLATE = """Task:Generate Cypher statement to query a graph database.
Instructions:
Use only the provided relationship types and properties in the schema.
Do not use any other relationship types or properties that are not provided.

Note: Do not include any explanations or apologies in your responses.
Do not respond to any questions that might ask anything else than for you to construct a Cypher statement.
Do not include any text except the generated Cypher statement.
Examples: Here are a few examples of generated Cypher statements for particular questions:
Always use 'n' as the Return variable match 
# The question is: "Match all nodes of type PrincipleNode."
MATCH (n:PrincipleNode) RETURN n

The question is:
{question}

The schema is:
{schema}
"""

CYPHER_GENERATION_PROMPT = PromptTemplate(
    input_variables=["schema", "question"], template=CYPHER_GENERATION_TEMPLATE
)

chain = GraphCypherQAChain.from_llm(
        ChatOpenAI(temperature=0, model=model),
        graph=graph, verbose=True,
        cypher_prompt=CYPHER_GENERATION_PROMPT,
        return_direct=True,
        top_k = TOP_K
    )

graph_schema=load_schema_data()
    print(graph_schema)
    question = "Use 'n' as the Return variable match all nodes with label of PrincipleNode."
    try:
        principle_nodes = chain.invoke( input={ 'schema': graph_schema, 'question': question})
    except Exception as e:
        print(f"Error: {e}")

Description

@dosubot Have a lack of understanding why the following code does not work

The inputs from the invoke, should be mapped to the input_variables of the CYPHER_GENERATION_PROMPT as schema and question but it fails with the following error

> Entering new GraphCypherQAChain chain...
Error: Missing some input keys: {'query'}
Traceback (most recent call last):
  File "/Users/rmccorkl/Code/Python/AI/VectorisedPDFDuel/runSCFImportNeo4jGraph.py", line 82, in <module>
    Step_02_process_policys_principles(df)
  File "/Users/rmccorkl/Code/Python/AI/VectorisedPDFDuel/src/SCF_KG_Neo4jmethods.py", line 144, in Step_02_process_policys_principles
    nodes_properties = [node['n'] for node in principle_nodes['result']]

In response , changing question to query the program works

try:
        principle_nodes = chain.invoke( input={ 'schema': graph_schema, 'query': question})
    except Exception as e:
        print(f"Error: {e}")

@dosubot Why are the inputs names in the dict not mapped one for one to the input_variables of the prompt, and is the schema actual being passed

System Info

langchain==0.2.5
langchain-community==0.2.5
langchain-core==0.2.7
langchain-experimental==0.0.61
langchain-google-vertexai==1.0.5
langchain-openai==0.1.8
langchain-pinecone==0.1.1
langchain-text-splitters==0.2.1

MAC M2 Sonoma 14.5

[dosubot[bot]]

The issue you're encountering is due to the GraphCypherQAChain class expecting the input key query instead of question. This is why changing question to query in your input dictionary resolves the error.

The GraphCypherQAChain class defines its input key as query:

input_key: str = "query"  #: :meta private:

This means that the chain expects the input dictionary to have a key named query. The input_keys property method confirms this:

@property
def input_keys(self) -> List[str]:
    return [self.input_key]

Therefore, when you pass the input dictionary to the invoke method, it should look like this:

principle_nodes = chain.invoke(input={'schema': graph_schema, 'query': question})

This ensures that the input keys are correctly mapped to what the GraphCypherQAChain expects.

Regarding the schema being passed correctly, the PromptTemplate class handles input variables by maintaining a list of the names of the variables the prompt template expects. In your case, the CYPHER_GENERATION_PROMPT is correctly set up with input_variables=["schema", "question"]. However, since the GraphCypherQAChain expects query instead of question, you need to ensure that the input dictionary matches this expectation.

To summarize:

• The GraphCypherQAChain expects the input key query.
• Ensure your input dictionary uses query instead of question.

This adjustment should resolve the input key mapping issue you're experiencing ^[1].

To continue talking to Dosu, mention @dosu.

[rmccorkl]

@dosubot @dosu But then how is the input 'query' translated /mapped to the variable 'question' in underlying structure in CYPHER_GENERATION_TEMPLATE which contains 'question' variable but not 'query', are the input_variables being populated correctly or is this being ignore ?

[dosubot[bot]]

The GraphCypherQAChain class expects the input key query, but the CYPHER_GENERATION_TEMPLATE uses the variable question. This discrepancy is handled internally by the chain. When you pass the input dictionary with the key query, the chain's internal logic maps this to the appropriate variable in the prompt template.

Here's how it works:

1. Input Key Mapping: The GraphCypherQAChain class expects the input key query. This is defined in the class as:

   input_key: str = "query"  #: :meta private:

2. Prompt Template Variables: The CYPHER_GENERATION_PROMPT is set up with input_variables=["schema", "question"]. This means the prompt template expects variables named schema and question.

3. Internal Mapping: When you invoke the chain with input={'schema': graph_schema, 'query': question}, the chain internally maps the query input to the question variable in the prompt template. This mapping is handled by the chain's _call method, which extracts the query input and uses it to populate the question variable in the prompt template.

Here is a simplified flow of how the mapping works:

• You provide the input dictionary: {'schema': graph_schema, 'query': question}.
• The chain extracts the query input and maps it to the question variable in the prompt template.
• The prompt template is then formatted with the schema and question variables.

This ensures that the input keys are correctly mapped to the variables expected by the prompt template, even though the names differ.

To summarize, the GraphCypherQAChain class handles the mapping of the query input to the question variable internally, ensuring that the prompt template is populated correctly with the provided inputs ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[rmccorkl]

@dosu Thanks for the explanation and how the discrepancy is handled by the chain , I have one last question is there anyway to print and debug the prompt to ensure values are being passed/mapped correctly , which would confirm the explanation. I can't find a way to debug the dynamic prompt ?

[dosubot[bot]]

Yes, you can print and debug the dynamic prompt to ensure the values are being passed and mapped correctly. Here is an example of how you can do it using Python and the LangChain framework:

from langchain_core.prompts.base import BasePromptTemplate
from langchain_core.prompt_values import StringPromptValue
from langchain.globals import set_debug

class MyPromptTemplate(BasePromptTemplate[StringPromptValue]):
    input_variables = ["schema", "question"]

    def format_prompt(self, **kwargs) -> StringPromptValue:
        schema = kwargs.get("schema", "")
        question = kwargs.get("question", "")
        # Here you can add more complex logic if needed
        return StringPromptValue(f"The schema is {schema} and the question is {question}")

# Enable debugging to print intermediate steps
set_debug(True)

# Define the input variables
input_variables = {
    "schema": "ExampleSchema",
    "question": "What is the capital of France?"
}

# Create an instance of the custom prompt template
template = MyPromptTemplate()

# Render the template with the input variables
rendered_template = template.format_prompt(**input_variables)

# Print the rendered template for debugging
print(rendered_template.value)

This code will help you see the final output of the template after the variables have been substituted, allowing you to debug and ensure the values are being passed and mapped correctly. The BasePromptTemplate class from LangChain provides a structured way to handle input variables and format the prompt. Additionally, setting the global debug flag with set_debug(True) will cause all LangChain components to print the inputs they receive and outputs they generate, providing a comprehensive view for debugging ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[rmccorkl]

Many Thanks the following is invaluable

from langchain.globals import set_debug