Pre-call hooks?

[twardoch]

In a sense, LiteLLM (the proxy) is a way to create an OpenAI-compatible endpoint. I'd like to deploy my own OpenAI-compatible endpoint on some serverless provider, but in such a way that whenever I prompt my own endpoint, "something happens" and only then I'm routing you the LLM.

For example, I might want to do some prompt pruning or do my own code-based choice of the right model to call.

Or maybe I want to call my own RAG. So basically I might want an OAI-compatible endpoint that calls something that's beyond just the LLM. A simple way to do it might be be exposing some pre-call hooks. My endpoint accepts a prompt but then it actually calls my own RAG and only with the enriched prompt it calls the LLM.

Is this already possible?

[twardoch]

Ok, found it at https://litellm.vercel.app/docs/observability/custom_callback#callback-functions

[ishaan-jaff]

Hi @twardoch can we hop on a call sometime this week to learn how we can improve litellm for you ? What's the best email to send an invite to ?

If it's easier here's my cal: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat