Improving Custom Agent Speed?

[Radvian]

I'm doing a passion project in which I try to create a knowledge base for a website, and create a Conversational Retrieval QA Chain so we can ask questions about the knowledge base.

This is very much akin to this: https://python.langchain.com/en/latest/modules/chains/index_examples/chat_vector_db.html

And I have no problem to implement it.

Now, I want to go to the next step. Instead of creating a chain, I want to create an agent.

I want to do something like this:

• First, always ask the user for their name.
• If the user asks a particular question that is not in the knowledge base, don't reply with "I don't know", but search Wikipedia about it
• If the user asks about X, send an image link (from google search images) of 'X' item
• and few other custom tools

And, I also don't have any problem in creating tools and combining them into a unified agent.

However, my problem is this (and this is expected): the agent performs slower, much slower than if I just use the chain.

If I only use Retrieval QA Chain, and enables streaming, it can start typing the output in less than 5 seconds.

If I use the complete agent, with a few custom tools, it starts to type the answer / output within 10-20 seconds.

I know this is to be expected, since the agent needs to think about which tools to use. But any ideas on how to make this faster? In the end, I want to deploy this whole project in a streamlit and wants this to perform like a 'functional' chatbot, with almost-instant reply, and not having to wait 10-20 seconds before the 'agent' starts typing.

Any ideas are welcome, apologies if this is a stupid question. Thank you!