-
Notifications
You must be signed in to change notification settings - Fork 86
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add gradient * update logo * updating integration page for gradient * Update gradient.md * Update gradient.md --------- Co-authored-by: Tuana Celik <[email protected]>
- Loading branch information
1 parent
6304699
commit b77a04a
Showing
2 changed files
with
131 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,131 @@ | ||
--- | ||
layout: integration | ||
name: Gradient | ||
description: Gradient AI is a self-managed cloud service for Large Language Models, offering fine-tuning and inference of open-source embedding and generative models. | ||
authors: | ||
- name: Mateusz Haligowski | ||
- name: deepset | ||
socials: | ||
github: deepset-ai | ||
twitter: deepset_ai | ||
linkedin: deepset-ai | ||
pypi: https://pypi.org/project/gradient-haystack/ | ||
repo: https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/gradient | ||
type: Model Provider | ||
report_issue: https://github.com/deepset-ai/haystack-core-integrations/issues | ||
logo: /logos/gradient.png | ||
version: Haystack 2.0 | ||
--- | ||
# Gradient AI integration | ||
|
||
[![PyPI - Version](https://img.shields.io/pypi/v/gradient-haystack.svg)](https://pypi.org/project/gradient-haystack) | ||
|
||
----- | ||
|
||
**Table of Contents** | ||
|
||
- [Gradient AI integration](#gradient-ai-integration) | ||
- [Installation](#installation) | ||
- [Usage](#usage) | ||
- [Examples](#examples) | ||
- [License](#license) | ||
|
||
## Installation | ||
Use `pip` to install the integration: | ||
|
||
```console | ||
pip install gradient-haystack | ||
``` | ||
## Usage | ||
Once installed, you will have access to a Generator and two Embedder objects. | ||
- `GradientDocumentEmbedder`: Use this component to create embeddings of documents. This is commonly used in indexing pipelines to store documents and their embeddings in document stores. | ||
- `GradientTextEmbedder`: Use this component to create embeddings of text, such as queries. This is commonly used as the first component of query pipelines to create embeddings of the query. | ||
- `GradientGenerator`: Use this component to query generative models with Gradient. This is commonly used in query pipelines to generate responses to queries. | ||
|
||
### Use the GradientDocumentEmbedder | ||
You can use embedding models with `GradientDocumentEmbedder`` to create embeddings of your documents. This is commonly used in indexing pipelines to write documents and their embeddings into a document store. | ||
|
||
```python | ||
import os | ||
|
||
from haystack import Pipeline | ||
from haystack.document_stores import InMemoryDocumentStore | ||
from haystack.components.writers import DocumentWriter | ||
|
||
from gradient_haystack.embedders.gradient_document_embedder import GradientDocumentEmbedder | ||
|
||
os.environ["GRADIENT_ACCESS_TOKEN"] = "Your Gradient Access Token" | ||
os.environ["GRADIENT_WORKSPACE_ID"] = "Your Gradient Workspace id: " | ||
|
||
documents = [ | ||
Document(content="My name is Jean and I live in Paris."), | ||
Document(content="My name is Mark and I live in Berlin."), | ||
Document(content="My name is Giorgio and I live in Rome."), | ||
] | ||
|
||
indexing_pipeline = Pipeline() | ||
indexing_pipeline.add_component(instance=GradientDocumentEmbedder(), name="document_embedder") | ||
indexing_pipeline.add_component(instance=DocumentWriter(document_store=InMemoryDocumentStore()), name="document_writer") | ||
indexing_pipeline.connect("document_embedder", "document_writer") | ||
indexing_pipeline.run({"document_embedder": {"documents": documents}}) | ||
``` | ||
|
||
### Use the GradientTextEmbedder and GradientGenerator | ||
You can use embedding models with `GradientTextEmbedder` and generative models with `GradientGenerator`. These two are commonly used together in a query pipeline such as a retrievel-augmented generative (RAG) pipeline such as the one below. | ||
|
||
```python | ||
from haystack.components.builders.answer_builder import AnswerBuilder | ||
from haystack.components.builders.prompt_builder import PromptBuilder | ||
from haystack.components.retrievers import InMemoryEmbeddingRetriever | ||
|
||
from gradient_haystack.embedders.gradient_text_embedder import GradientTextEmbedder | ||
from gradient_haystack.generator.base import GradientGenerator | ||
|
||
from getpass import getpass | ||
|
||
prompt_template = """ | ||
Given these documents, answer the question.\nDocuments: | ||
{% for doc in documents %} | ||
{{ doc.content }} | ||
{% endfor %} | ||
\nQuestion: {{question}} | ||
\nAnswer: | ||
""" | ||
|
||
gradient_access_token = os.environ.get("GRADIENT_ACCESS_TOKEN") | ||
|
||
retriever = InMemoryEmbeddingRetriever(document_store) | ||
prompt_builder = PromptBuilder(template=prompt_template) | ||
embedder = GradientTextEmbedder(access_token=gradient_access_token) | ||
generator = GradientGenerator(access_token=gradient_access_token, base_model_slug="llama2-7b-chat") | ||
|
||
rag_pipeline = Pipeline() | ||
rag_pipeline.add_component(instance=embedder, name="text_embedder") | ||
rag_pipeline.add_component( | ||
instance=retriever, name="retriever" | ||
) | ||
rag_pipeline.add_component(instance=prompt_builder, name="prompt_builder") | ||
rag_pipeline.add_component(instance=generator, name="llm") | ||
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder") | ||
|
||
rag_pipeline.connect("text_embedder", "retriever") | ||
rag_pipeline.connect("retriever", "prompt_builder.documents") | ||
rag_pipeline.connect("prompt_builder", "llm") | ||
rag_pipeline.connect("llm.replies", "answer_builder.replies") | ||
rag_pipeline.connect("retriever", "answer_builder.documents") | ||
|
||
rag_pipeline.run( | ||
{ | ||
"text_embedder": {"text": question}, | ||
"prompt_builder": {"question": question}, | ||
"answer_builder": {"query": question}, | ||
} | ||
) | ||
``` | ||
|
||
## Examples | ||
You can find a full code example showing how to use the integration in [this Colab](https://colab.research.google.com/drive/1kE_NAKKgZztQJMbgm2esyTVkAxlrpGtd#scrollTo=coE-fMtTJ-Pp). | ||
|
||
## License | ||
|
||
`gradient-haystack` is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.