v0.4.23

Installation

pip install openllm==0.4.23

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.4.23

Usage

All available models: openllm models

To start a LLM: python -m openllm start HuggingFaceH4/zephyr-7b-beta

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P -v $PWD/data:$HOME/.cache/huggingface/ ghcr.io/bentoml/openllm:0.4.23 start HuggingFaceH4/zephyr-7b-beta

Find more information about this release in the CHANGELOG.md

What's Changed

• chore: cleanup unused prompt templates by @aarnphm in #713
• feat(generation): add support for eos_token_id by @aarnphm in #714
• fix(ci): tests by @aarnphm in #715
• refactor: delete unused code by @aarnphm in #716
• chore(logger): fix logger and streamline style by @aarnphm in #717
• chore(strategy): compact and add stubs by @aarnphm in #718
• chore(types): append additional types change by @aarnphm in #719
• fix(base-image): update base image to include cuda for now by @aarnphm in #720

Full Changelog: v0.4.22...v0.4.23