refactor: move vector search capabilities to core package #39

PatrykWyzgowski · 2024-09-25T12:10:15Z

No description provided.

github-actions · 2024-09-25T12:14:15Z

🧪 Test coverage: 92.00%

Code Coverage Summary

Filename                                                                                                Stmts    Miss  Cover    Missing
----------------------------------------------------------------------------------------------------  -------  ------  -------  --------------------------------
packages/ragbits-core/src/ragbits/core/__init__.py                                                          0       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                               0       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/base.py                                                   4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/__init__.py                                                     4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/base.py                                                        33       4  87.88%   33, 52, 88, 97
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                     25       4  84.00%   8-9, 54, 85
packages/ragbits-core/src/ragbits/core/llms/local.py                                                       24      10  58.33%   8-9, 42-47, 57, 70-71
packages/ragbits-core/src/ragbits/core/llms/types.py                                                        8       2  75.00%   24, 28
packages/ragbits-core/src/ragbits/core/llms/clients/__init__.py                                             4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/base.py                                                23       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/exceptions.py                                          14       6  57.14%   7-8, 17, 26-27, 36
packages/ragbits-core/src/ragbits/core/llms/clients/litellm.py                                             50      10  80.00%   10-11, 70, 108, 122-127
packages/ragbits-core/src/ragbits/core/llms/clients/local.py                                               37      12  67.57%   11-12, 62-70, 92-103
packages/ragbits-core/src/ragbits/core/prompt/__init__.py                                                   2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/base.py                                                      18       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                   34       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                    99       2  97.98%   105, 109
packages/ragbits-core/src/ragbits/core/vector_store/__init__.py                                             0       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_store/base.py                                                12       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py                                      51       4  92.16%   9-10, 58, 100
packages/ragbits-core/src/ragbits/core/vector_store/in_memory.py                                           18       0  100.00%
packages/ragbits-core/tests/unit/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/llms/__init__.py                                                           0       0  100.00%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                      63       0  100.00%
packages/ragbits-core/tests/unit/prompts/__init__.py                                                        0       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_parsers.py                                                   65       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_prompt.py                                                   112       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chromadb_store.py                                      69       4  94.20%   31, 34, 39, 44
packages/ragbits-core/tests/unit/vector_stores/test_simple_vector_store.py                                 16       0  100.00%
packages/ragbits-dev-kit/src/ragbits/dev_kit/__init__.py                                                    0       0  100.00%
packages/ragbits-dev-kit/src/ragbits/dev_kit/prompt_lab/__init__.py                                         0       0  100.00%
packages/ragbits-dev-kit/src/ragbits/dev_kit/prompt_lab/discovery/__init__.py                               2       0  100.00%
packages/ragbits-dev-kit/src/ragbits/dev_kit/prompt_lab/discovery/prompt_discovery.py                      33       2  93.94%   56-57
packages/ragbits-dev-kit/tests/unit/discovery/__init__.py                                                   0       0  100.00%
packages/ragbits-dev-kit/tests/unit/discovery/prompt_classes_for_tests.py                                  30       0  100.00%
packages/ragbits-dev-kit/tests/unit/discovery/test_prompt_discovery.py                                     18       0  100.00%
packages/ragbits-dev-kit/tests/unit/discovery/ragbits_tests_pkg_with_prompts/__init__.py                    2       1  50.00%   3
packages/ragbits-dev-kit/tests/unit/discovery/ragbits_tests_pkg_with_prompts/prompts/__init__.py            3       2  33.33%   2-4
packages/ragbits-dev-kit/tests/unit/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt1.py       14       0  100.00%
packages/ragbits-dev-kit/tests/unit/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt2.py       14       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/__init__.py                                    2       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                      45       1  97.78%   94
packages/ragbits-document-search/src/ragbits/document_search/documents/__init__.py                          0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/document.py                         55       3  94.55%   52, 95, 124
packages/ragbits-document-search/src/ragbits/document_search/documents/element.py                          30       1  96.67%   34
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py                          46       9  80.43%   13-14, 58, 87, 106, 109, 120-122
packages/ragbits-document-search/src/ragbits/document_search/ingestion/__init__.py                          0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py               20       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/base.py                   14       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/dummy.py                  11       1  90.91%   29
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured.py           30       5  83.33%   83-94, 98
packages/ragbits-document-search/src/ragbits/document_search/retrieval/__init__.py                          0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/base.py                   5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/noop.py                   5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/base.py                    6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                    7       0  100.00%
packages/ragbits-document-search/tests/__init__.py                                                          0       0  100.00%
packages/ragbits-document-search/tests/helpers.py                                                           3       0  100.00%
packages/ragbits-document-search/tests/integration/__init__.py                                              0       0  100.00%
packages/ragbits-document-search/tests/integration/test_unstructured.py                                    38      24  36.84%   22-29, 37-43, 51-57, 65-72
packages/ragbits-document-search/tests/unit/__init__.py                                                     0       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_processor.py                                     17       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                                        41       0  100.00%
packages/ragbits-document-search/tests/unit/test_documents.py                                              13       0  100.00%
packages/ragbits-document-search/tests/unit/test_elements.py                                               15       0  100.00%
packages/ragbits-document-search/tests/unit/test_gcs_source.py                                             14       0  100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                              20       0  100.00%
TOTAL                                                                                                    1338     107  92.00%

Diff against main

Filename                                                                      Stmts    Miss  Cover
--------------------------------------------------------------------------  -------  ------  --------
packages/ragbits-core/src/ragbits/core/vector_store/__init__.py                   0       0  +100.00%
packages/ragbits-core/src/ragbits/core/vector_store/base.py                     +12       0  +100.00%
packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py           +51      +4  +92.16%
packages/ragbits-core/src/ragbits/core/vector_store/in_memory.py                +18       0  +100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chromadb_store.py           +69      +4  +94.20%
packages/ragbits-core/tests/unit/vector_stores/test_simple_vector_store.py      +16       0  +100.00%
TOTAL                                                                          +166      +8  +100.00%

Results for commit: c1cbdc8

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

github-actions · 2024-09-25T12:14:22Z

Trivy scanning results.

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: JWT (jwt-token)
════════════════════════════════════════
JWT token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA:80
────────────────────────────────────────
78 >>> encoded = jwt.encode({"some": "payload"}, "secret", algorithm="HS256")
79 >>> print(encoded)
80 [ *********************************************************************************************************
81 >>> jwt.decode(encoded, "secret", algorithms=["HS256"])
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Slack (slack-web-hook)
════════════════════════════════════════
Slack Webhook
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/proxy/_types.py:1288
────────────────────────────────────────
1286 alert_to_webhook_url: Optional[Dict] = Field(
1287 None,
1288 [ description="Mapping of alert type to webhook url. e.g. alert_to_webhook_url: {'budget_alerts': '*****************************************************************************'}",
1289 )
────────────────────────────────────────

akonarski-ds

lgtm

…to-core-package

mhordynski

lgtm

…to-core-package

PatrykWyzgowski added 3 commits September 25, 2024 12:11

refactor: Movnig python code from document-search to core

88a7f98

Fix import in chromadb unit tests

7c54d01

Applying AAA convention for tests and fix-ups.

81af745

PatrykWyzgowski linked an issue Sep 25, 2024 that may be closed by this pull request

refactor: move vector_search capabilities to core package #14

Closed

fix(document-search): test_elements

1c46550

PatrykWyzgowski added 2 commits September 25, 2024 14:18

fix: update locks and tomls

5e56899

Applied isort modifications

48b87f9

PatrykWyzgowski requested a review from mhordynski September 25, 2024 12:43

akonarski-ds approved these changes Sep 26, 2024

View reviewed changes

Merge branch 'main' into 14-refactor-move-vector_search-capabilities-…

5f53c78

…to-core-package

mhordynski approved these changes Oct 7, 2024

View reviewed changes

micpst approved these changes Oct 7, 2024

View reviewed changes

Merge branch 'main' into 14-refactor-move-vector_search-capabilities-…

c1cbdc8

…to-core-package

PatrykWyzgowski force-pushed the 14-refactor-move-vector_search-capabilities-to-core-package branch from 36a734a to c1cbdc8 Compare October 7, 2024 13:45

PatrykWyzgowski merged commit a186751 into main Oct 7, 2024
3 checks passed

micpst deleted the 14-refactor-move-vector_search-capabilities-to-core-package branch October 7, 2024 14:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: move vector search capabilities to core package #39

refactor: move vector search capabilities to core package #39

PatrykWyzgowski commented Sep 25, 2024

github-actions bot commented Sep 25, 2024 •

edited

Loading

github-actions bot commented Sep 25, 2024 •

edited

Loading

akonarski-ds left a comment

mhordynski left a comment

refactor: move vector search capabilities to core package #39

refactor: move vector search capabilities to core package #39

Conversation

PatrykWyzgowski commented Sep 25, 2024

github-actions bot commented Sep 25, 2024 • edited Loading

Code Coverage Summary

Diff against main

github-actions bot commented Sep 25, 2024 • edited Loading

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

akonarski-ds left a comment

Choose a reason for hiding this comment

mhordynski left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 25, 2024 •

edited

Loading

github-actions bot commented Sep 25, 2024 •

edited

Loading