Skip to content

Commit

Permalink
small fixes to llamafile
Browse files Browse the repository at this point in the history
  • Loading branch information
anakin87 committed Jun 25, 2024
1 parent 40fadb0 commit acd6b84
Showing 1 changed file with 8 additions and 4 deletions.
12 changes: 8 additions & 4 deletions integrations/llamafile.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,14 @@ toc: true
### **Table of Contents**

- [Overview](#overview)
- [Download and run the model](#download-and-run-the-model)
- [Usage](#usage)
- [Download and run the model](#download-and-run-models)
- [Generative models](#generative-models)
- [Embedding models](#embedding-models)
- [Usage](#usage-with-haystack)

## Overview

[llamafile](https://github.com/Mozilla-Ocho/llamafile) is a project that aims to make open LLMs accessible to developers and users.
[llamafile](https://github.com/Mozilla-Ocho/llamafile) is a project by Mozilla that aims to make open LLMs accessible to developers and users.

To run LLMs locally, simply download a single-file executable ("llamafile") that contains both the model and the inference engine and runs locally on most computers.

Expand Down Expand Up @@ -151,4 +153,6 @@ result = rag_pipe.run({"text_embedder":{"text": query},
print(result["generator"]["replies"][0])

# According to the documents, the best food in the world is pizza.
```
```

For a fun use case, explore this notebook: [Quizzes and Adventures with Character Codex and llamafile](https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/charactercodex_llamafile.ipynb).

0 comments on commit acd6b84

Please sign in to comment.