Skip to content

v0.0.0
Compare
Choose a tag to compare
@mdegans mdegans released this 23 May 22:43
· 4 commits to main since this release
v0.0.0
0246432

This is the first release of weave. It has only basic features but it supports:

Inference backends:

  • Metal inference on MacOS.
  • CUDA inference on Linux.
  • OpenAI (all models supporting the chat completions API, including GPT-4o)

Settings:

  • Settings for OpenAI are available in the GUI including sampling settings, system prompt, and an initial set of example messages to bootstrap the story writing.
  • Settings for drama_llama are not yet available in the GUI. Right now they default to whatever the crate default for drama_llama is. As of writing that's greedy sampling with no repetition penalty. That's not great, so the defaults will be changed shortly and a GUI added.

Notes:

  • The MacOS release is tested, including with LLaMA 3 70b, but the binary is not signed or notarized so it MacOS will complain until you tell Gatekeeper to allow it to run in Settings. Future releases will likely be signed and notarized so it will "just work".
  • The Linux release is completely untested. I don't have Linux+NVIDIA machine that isn't headless available, so if you do, feel free to test it and file and issue if it doesn't work. It will likely also work via X11 forwarding if you have that setup. It will also likely work in WSL.
Assets 4
Loading