Add mistralrs mention

EricLBuehler · Mar 6, 2024 · e7c392c · e7c392c
1 parent 92893f8
commit e7c392c
Showing 1 changed file with 7 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -23,6 +23,13 @@ X-LoRA is easily applied to any HuggingFace Transformers model. Please see our w
 
 See the [examples](examples) folder for some examples of how to get started with X-LoRA.
 
+## Efficent Inference Support
+[Mistral.rs](https://github.com/EricLBuehler/mistral.rs) is an inference framework which supports X-LoRA! To use it, follow the installation instructions and run the following command to start up an X-LoRA inference platform!
+
+`./mistralrs-server --port 1234 x-lora-mistral -o ordering.json`
+
+Base and X-LoRA Huggingface model IDs may be specified through command line switches to use your own models. Please see the Github page for further details.
+
 ## Installation
 Pending a pip release, run the following command to install X-LoRA.