Skip to content

Commit

Permalink
add NvidiaRanker to NVIDIA integrations (#260)
Browse files Browse the repository at this point in the history
  • Loading branch information
mattf authored Aug 28, 2024
1 parent 9ebc97e commit ed1892f
Showing 1 changed file with 27 additions and 0 deletions.
27 changes: 27 additions & 0 deletions integrations/nvidia.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ This integration introduces the following components:

- `NvidiaGenerator`: A component for generating text using generative models provided by NVIDIA AI Foundation Endpoints and NVIDIA Inference Microservices.

- `NvidiaRanker`: A component for ranking documents, using [NVIDIA NIMs](https://ai.nvidia.com).

## Use the components on their own:

### `NvidiaTextEmbedder`:
Expand Down Expand Up @@ -110,6 +112,31 @@ print(result["meta"])
# ['The Golden Gate Bridge was built in 1937 and was completed and opened to the public on May 29, 1937....'[{'role': 'assistant', 'finish_reason': 'stop'}]
```


### `NvidiaRanker`:

```python
from haystack_integrations.components.rankers.nvidia import NvidiaRanker
from haystack import Document
from haystack.utils import Secret

ranker = NvidiaRanker(
api_key=Secret.from_env_var("NVIDIA_API_KEY"),
)
ranker.warm_up()

query = "What is the capital of Germany?"
documents = [
Document(content="Berlin is the capital of Germany."),
Document(content="The capital of Germany is Berlin."),
Document(content="Germany's capital is Berlin."),
]

result = ranker.run(query, documents, top_k=1)
print(result["documents"][0].content)
# The capital of Germany is Berlin.
```

## Use NVIDIA components in Haystack pipelines

### Indexing pipeline
Expand Down

0 comments on commit ed1892f

Please sign in to comment.