Add Python 3.13 and fastembed 0.6 support #1102
Conversation
In `test_examples_included_in_prompts` and `test_examples_included_in_pr≈ompts_2`
we make an HTP call to GET /api/models/qdrant/all-MiniLM-L6-v2-onnx to download
an embedding model to look up the most similar user intent to the user saying "Hi".
Our pyproject.toml has:
```
fastembed = [
{ version = ">=0.2.2, <0.5", python = ">=3.9,<3.10" },
{ version = ">=0.5.0, <=0.6.0", python = ">=3.10, <3.14" },
]
```
In Python 3.9.21 poetry instaled fastembed 0.2.7, giving a similarity of [(0.9502324275672436, 'hi')]
In Python 3.11.11 we use fastembed 0.6.0, giving a similairty of [(0.7792390137910843, 'hi')]
Prior to this commit, the threshold was set at 0.8. This means for Python 3.9.21
(fastembed 0.2.7) no LLM call was made (0.95>0.8). For fastembed 0.6.0 the same embedding
model gives 0.78 < 0.8. There's an issue here since the same embedding model should give
the same results regardless of the API/toolkit we use to call it.
But a bigger issue is that we should be mocking the fastembed call anyway
so we can return a value above, below, and exactly at the threshold and test each.
We shouldn't need internet access for unit-tests. Will file an issue for this.
tgasser-nv
changed the title
tgasser-nv
force-pushed
the
chore/upgrade-fastembed
branch
from
236e190 to
734a991
Compare
tgasser-nv
force-pushed
the
chore/upgrade-fastembed
branch
from
a358169 to
c321c1e
Compare
Signed-off-by: Tim Gasser <200644301+tgasser-nv@users.noreply.github.com>
tgasser-nv
changed the title
Resolving dependencies...
tgasser-nv
changed the title
| @@ -42,7 +42,7 @@ def colang_1_config(): | |||
| dialog: | |||
| user_messages: | |||
| embeddings_only: True | |||
| embeddings_only_similarity_threshold: 0.8 | |||
| embeddings_only_similarity_threshold: 0.99 | |||
There was a problem hiding this comment.
Where did this change come from? Related to the fast-embed update?
There was a problem hiding this comment.
We're downloading and calling live production embedding models in our unit-tests (this needs to be fixed). The same all-minilm-l6-v2 model gives two different similarities for the same user string and colang rule (see this commit message) for fastembed 0.2.7 and 0.6.0. If the threshold isn't updated the test fails on Python 3.9 but passes everywhere else.
We should be mocking these embedding model calls rather than running it. Calling other services from a test makes it an integration test (which we should also add).
There was a problem hiding this comment.
How about opening a separate PR(s) for all the embedding and indexing issues and address them there. Let's not include this change in this PR.
Just a note so that we don't forget, the poetry.lock file is updated to test the changes work, which is great, but we should delete it and only do poetry lock --no-update before merging this PR.
| fastembed = ">=0.2.2, <0.4.1" | ||
| fastembed = [ | ||
| { version = ">=0.2.2, <0.5", python = ">=3.9,<3.10" }, | ||
| { version = ">=0.5.0, <=0.6.0", python = ">=3.10, <3.14" }, |
There was a problem hiding this comment.
Cannot we support the other fastembed versions?
{ version = ">=0.2.2, <=0.6.0", python = ">=3.10, <3.14" },
|
My bad, I didn’t realize it had been merged. Here are some comments that apply to similar PRs:
There seems to be a dependency resolution bug where Poetry is not properly enforcing python version specific constraints. We can open a PR to revert changes made to the lock file. |
Description
Upgrading fastembed to 0.6.0 and adding Python 3.13 support (tested at Python 3.13.2).
Note spacy does not support Python 3.13 (explosion/spaCy#13658). Once compile-from-source or wheels are available we'll add.
Checklist