Skip to content

Commit

Permalink
[projects][vllm] Reenable vLLM
Browse files Browse the repository at this point in the history
  • Loading branch information
sasha0552 authored Aug 10, 2024
1 parent d6f1850 commit 84eb924
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 10 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ work/
airootfs/root/customize_airootfs.sh
airootfs/root/customize_airootfs/scripts/1000-automatic.sh
airootfs/root/customize_airootfs/scripts/1000-comfyui.sh
airootfs/root/customize_airootfs/scripts/1000-invokeai.sh
airootfs/root/customize_airootfs/scripts/1000-llamacpp.sh
airootfs/root/customize_airootfs/scripts/1000-sillytavern.sh
airootfs/root/customize_airootfs/scripts/1000-vllm.sh
Expand Down
16 changes: 16 additions & 0 deletions airootfs/root/customize_airootfs/scripts/1000-vllm.sh.jinja2
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/sh
set -eu

{% if platform == "cuda" %}
# use patched vllm
export PIP_EXTRA_INDEX_URL="https://sasha0552.github.io/pascal-pkgs-ci/"

# install vllm
pipx install vllm

# remove triton
pipx runpip vllm uninstall -y triton

# install patched triton
pipx runpip vllm install triton-pascal
{% endif %}

This file was deleted.

0 comments on commit 84eb924

Please sign in to comment.