[Feature]: gzip upstream requests?

[Manouchehri]

The Feature

Only some providers support this, like Google Vertex AI.

echo '{
    "contents": [{
      "role": "user",
      "parts": [{
        "text": "tell me a good joke"
      }]
    }],
    "generationConfig": {
      "temperature": 0.0,
      "maxOutputTokens": 100,
      "seed": 1337
    }
  }' | gzip | curl -v "https://gateway.ai.cloudflare.com/v1/0399b10e77ac6668c80404a5ff49eb37/litellm-test/google-vertex-ai/v1/projects/adroit-crow-413218/locations/us-east4/publishers/google/models/gemini-1.5-pro-001:generateContent" \
    -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Encoding: gzip" \
    -H "Content-Type: application/json" \

    --data-binary @-

OpenAI does not support it. I have not tested Azure OpenAI or Amazon Bedrock.

Motivation, pitch

I have a feeling this will likely be a negative perf outcome for folks like us who are running LiteLLM in a datacentre, but it might be a win for others who self-host LiteLLM locally on-prem.

Related to #3964.

Twitter / LinkedIn details

https://www.linkedin.com/in/davidmanouchehri/