diff --git a/integrations/fastrag.md b/integrations/fastrag.md index c8af45b7..b7e7fcb0 100644 --- a/integrations/fastrag.md +++ b/integrations/fastrag.md @@ -11,9 +11,26 @@ repo: https://github.com/IntelLabs/fastRAG type: Custom Component report_issue: https://github.com/IntelLabs/fastRAG/issues logo: /logos/intel-labs.png +version: Haystack 2.0 --- -fast**RAG** is a research framework, that extends [Haystack](https://github.com/deepset-ai/haystack), with abilities to build ***efficient*** and ***optimized*** retrieval augmented generative pipelines (with emphasis on ***Intel hardware***), incorporating state-of-the-art LLMs and Information Retrieval modules. +fast**RAG** is a research framework for ***efficient*** and ***optimized*** retrieval augmented generative pipelines, +incorporating state-of-the-art LLMs and Information Retrieval. fastRAG is designed to empower researchers and developers +with a comprehensive tool-set for advancing retrieval augmented generation. + +Comments, suggestions, issues and pull-requests are welcomed! ❤️ + +> **IMPORTANT** +> +> Now compatible with Haystack v2+. Please report any possible issues you find. + +## 📣 Updates + +- **2024-05**: fastRAG V3 is Haystack 2.0 compatible 🔥 +- **2023-12**: Gaudi2 and ONNX runtime support; Optimized Embedding models; Multi-modality and Chat demos; [REPLUG](https://arxiv.org/abs/2301.12652) text generation. +- **2023-06**: ColBERT index modification: adding/removing documents. +- **2023-05**: [RAG with LLM and dynamic prompt synthesis example](https://github.com/IntelLabs/fastRAG/blob/main/examples/rag-prompt-hf.ipynb). +- **2023-04**: Qdrant `DocumentStore` support. ## Key Features @@ -21,9 +38,9 @@ fast**RAG** is a research framework, that extends [Haystack](https://github.com/ - **Optimized for Intel Hardware**: Leverage [Intel extensions for PyTorch (IPEX)](https://github.com/intel/intel-extension-for-pytorch), [🤗 Optimum Intel](https://github.com/huggingface/optimum-intel) and [🤗 Optimum-Habana](https://github.com/huggingface/optimum-habana) for *running as optimal as possible* on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators. - **Customizable**: fastRAG is built using [Haystack](https://github.com/deepset-ai/haystack) and HuggingFace. All of fastRAG's components are 100% Haystack compatible. -## Components +## 🚀 Components -For a brief overview of the various unique components in fastRAG refer to the [Components Overview]([components.md](https://github.com/IntelLabs/fastRAG/blob/main/components.md)) page. +For a brief overview of the various unique components in fastRAG refer to the [Components Overview](https://github.com/IntelLabs/fastRAG/blob/main/components.md) page.
@@ -43,6 +60,10 @@ For a brief overview of the various unique components in fastRAG refer to the [C + + + + @@ -80,7 +101,7 @@ For a brief overview of the various unique components in fastRAG refer to the [C
ONNX Runtime Running LLMs with optimized ONNX-runtime
OpenVINORunning quantized LLMs using OpenVINO
Llama-CPP Running RAG Pipelines with LLMs on a Llama CPP backend
-## Installation +## 📍 Installation Preliminary requirements: @@ -92,20 +113,77 @@ To set up the software, clone the project and run the following, preferably in a ```bash git clone https://github.com/IntelLabs/fastRAG.git cd fastrag +pip install . ``` -There are several dependencies to consider, depending on your specific usage: +## Usage + +You can import components from fastRAG and use them in a Haystack pipeline: + +```python +from haystack.components.retrievers.in_memory import InMemoryBM25Retriever +from haystack.components.builders.prompt_builder import PromptBuilder +from haystack.components.rankers import TransformersSimilarityRanker + +from fastrag.generators.openvino import OpenVINOGenerator + +prompt_template = """ +Given these documents, answer the question. +Documents: +{% for doc in documents %} + {{ doc.content }} +{% endfor %} +Question: {{query}} +Answer: +""" + +openvino_compressed_model_path = "path/to/quantized/model" + +generator = OpenVINOGenerator( + model="microsoft/phi-2", + compressed_model_dir=openvino_compressed_model_path, + device_openvino="CPU", + task="text-generation", + generation_kwargs={ + "max_new_tokens": 100, + } +) + +pipe = Pipeline() + +pipe.add_component("retriever", InMemoryBM25Retriever(document_store=store)) +pipe.add_component("ranker", ransformersSimilarityRanker()) +pipe.add_component("prompt_builder", PromptBuilder(template=prompt_template)) +pipe.add_component("llm", generator) + +pipe.connect("retriever.documents", "ranker.documents") +pipe.connect("ranker", "prompt_builder.documents") +pipe.connect("prompt_builder", "llm") + +query = "Who is the main villan in Lord of the Rings?" +answer_result = pipe.run({ + "prompt_builder": { + "query": query + }, + "retriever": { + "query": query + }, + "ranker": { + "query": query, + "top_k": 1 + } +}) + +print(answer_result["llm"]["replies"][0]) +#' Sauron\n' +``` -Basic installation: +For more examples, check out [Example Use Cases](https://github.com/IntelLabs/fastRAG/blob/main/examples.md). -```bash -pip install . -``` +## License -fastRAG with Intel-optimized backend: +The code is licensed under the [Apache 2.0 License](https://github.com/IntelLabs/fastRAG/blob/main/LICENSE). -```bash -pip install .[intel] -``` +## Disclaimer -Other installation options can be found [here](https://github.com/IntelLabs/fastRAG#round_pushpin-installation). +This is not an official Intel product.