long-term memory is expensive.

[punksta]

Hi, amazing work!

As I can tell, long term memory requires sending all memories to gpt-4 to fetch the more relevant memories, for each input.

If it stoes many memories, the cost of this feature will be very high.

Is there a way of optimizing this? I am thinking about using a cheaper model.

[ian-kent]

Hey 👋 thanks!

I'm looking at options for improving this but I haven't had a lot of time to work on it. There's definitely optimisations that can be made, but any pre-filtering done on the data (which ultimately is the only solution) means letting something that isn't GPT determine how relevant a memory is.

It's worth trying with GPT-3.5 for memory recall though (you can replace openai.GPT4 in the memory module code), other parts of GPTChat (commands, plugins) don't work particularly well but memory recall might be ok.

[Suppenterrine]

Hello there!
I saw your reddit post the other day where you mentioned this project. In the comments below I found the hint to use Vector Databases for Memory Storing.
I'm coding a kind of similiar project like you, but in Javascript.
I just wanted to share this video to you, because it helped me to integrate a vector db (I never worked with on before). Maybe it'll be useful to you: https://www.youtube.com/watch?v=tp0bQNDtLPc

And don't quit your project, it's a perfect way of learning because it keeps you motivated.