Add Support for Embeddings to Reduce Token Usage And reduce the risk of abnormal stop.

[Rain-Of-Stars]

Which API Provider are you using?

OpenAI Compatible

Which Model are you using?

deepseek

What happened?

I propose adding a feature using an embedding system to reduce tokens in requests. This could help manage and optimize the daily token. Leveraging embeddings may maintain or enhance functionality while reducing token usage and costs.

Steps to reproduce

Now the context consumption is too large, and multiple rounds of dialogue will overflow tokens.

Relevant API REQUEST output

No response

Additional context

No response

[mrubens]

@Rain-Of-Stars sounds great - any thoughts on the best way to do it?