Awesome Serverless GPU

List of where to run code on GPUs for AI, inference, predictions that are serverless.
Serverless is defined as pay-as-you-go, scale-to-zero, minimal infrastructure configuration.

Serverless GPU is a reletively new and fast evolving field. New services are appearing and disappearing frequently.
I will do my best to keep the list updated, and soon include benchmarks.

Common weaknessses of serverless GPU at the moment is very long cold starts, and configuration that are less easy to use than the more mature field of serverless on CPUs.

Inference

Bring your own model

True serverless inference

• Inferless.com
• Replicate.com
• Runpod.io
• Modelz.ai
• Banana.dev (Shutting down March 31st 2024)
• Beam.cloud
• Mystic.ai
• Modal.com
• Baseten.co
• Covalent.xyz

Predefined models

True serverless with a limited set of models

• Cloudflare AI
• OpenAI
• Mistral
• Sievedata.com
• Together.ai
• Anyscale.com
• Fireworks.ai
• Fal.ai

Not serverless inference

Needs dedicated server, but works with your own model

• Together.ai
• Sievedata.com
• Anyscale.com
• Runpod.io
• Huggingface.co
• Lepton.ai
• Vast.ai
• Lambdalabs.com
• Paperspace

Dev on GPUs

Flexible on-demand GPU providers

• Brev.dev
• Colab
• Kaggle.com
• Modal.com
• Beam.cloud

Predefined models over API

Speech to text

• OpenAI
• Amazon, Azure, Google
• Gladia.io
• Deepgram.com
• Speechmatics.com
• Sieve ASR
• Fal.ai Whisper

Text to speech

• OpenAI
• Amazon, Azure, Google
• Speechify.com
• Elevenlabs.io
• Unrealspeech.com
• Play.ht
• Murf.ai
• Deepgram.com
• Resemble.ai

Image generation

• OpenAI Dall-e
• Dreamstudio
• Stablediffusionapi.com
• Modelslab.com
• Bria.ai
• Fal.ai

Workflow platforms

• Leap

AI Agents for Websites

• Chatflow
• OneAI
• SiteGPT
• Chatbase