Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: document search sources string resolver #264

Merged
merged 13 commits into from
Jan 22, 2025

Conversation

maciejklimek
Copy link
Contributor

Solves #221

Copy link
Contributor

github-actions bot commented Dec 27, 2024

badge

Code Coverage Summary

Filename                                                                                                      Stmts    Miss  Cover    Missing
----------------------------------------------------------------------------------------------------------  -------  ------  -------  ------------------------------------------------------
packages/__init__.py                                                                                              0       0  100.00%
packages/ragbits-cli/src/ragbits/cli/__init__.py                                                                 31       4  87.10%   73-74, 81-82
packages/ragbits-cli/src/ragbits/cli/_utils.py                                                                   23       4  82.61%   45, 62-64
packages/ragbits-cli/src/ragbits/cli/state.py                                                                    55       8  85.45%   48-49, 61, 64-65, 108, 115-116
packages/ragbits-conversations/src/ragbits/conversations/__init__.py                                              0       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/__init__.py                                      0       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/__init__.py                          3       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/base.py                             10       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/llm.py                              29       1  96.55%   79
packages/ragbits-conversations/tests/unit/history/test_llm_compressor.py                                         64       0  100.00%
packages/ragbits-core/src/ragbits/core/__init__.py                                                                6       2  66.67%   8-9
packages/ragbits-core/src/ragbits/core/cli.py                                                                     6       0  100.00%
packages/ragbits-core/src/ragbits/core/config.py                                                                 17       0  100.00%
packages/ragbits-core/src/ragbits/core/options.py                                                                17       0  100.00%
packages/ragbits-core/src/ragbits/core/types.py                                                                   9       0  100.00%
packages/ragbits-core/src/ragbits/core/audit/__init__.py                                                         74       5  93.24%   42-45, 52-54
packages/ragbits-core/src/ragbits/core/audit/base.py                                                             44       1  97.73%   119
packages/ragbits-core/src/ragbits/core/audit/cli.py                                                              69       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                                     4       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/base.py                                                        20       2  90.00%   56, 69
packages/ragbits-core/src/ragbits/core/embeddings/exceptions.py                                                  17       7  58.82%   7-8, 17, 26-27, 36, 45
packages/ragbits-core/src/ragbits/core/embeddings/litellm.py                                                     38      18  52.63%   78-112
packages/ragbits-core/src/ragbits/core/embeddings/noop.py                                                         8       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/__init__.py                                                           4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/base.py                                                              66       8  87.88%   61, 73, 174, 193-201, 205
packages/ragbits-core/src/ragbits/core/llms/exceptions.py                                                        17       7  58.82%   7-8, 17, 26-27, 36, 45
packages/ragbits-core/src/ragbits/core/llms/factory.py                                                           12       3  75.00%   30, 41, 51
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                           95      29  69.47%   94, 100-121, 165, 175, 204-229, 251-256, 267
packages/ragbits-core/src/ragbits/core/llms/local.py                                                             56      27  51.79%   9-12, 66-74, 86-87, 108-119, 140-156
packages/ragbits-core/src/ragbits/core/llms/mock.py                                                              29       9  68.97%   63-74
packages/ragbits-core/src/ragbits/core/metadata_stores/__init__.py                                                3       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/base.py                                                   11       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/exceptions.py                                              4       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/in_memory.py                                              16       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/__init__.py                                                         2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/_cli.py                                                            44      21  52.27%   25-33, 47-49, 63-65, 73-75, 89-97
packages/ragbits-core/src/ragbits/core/prompt/base.py                                                            20       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                         35       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                         126       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/__init__.py                                               2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/prompt_discovery.py                                      33       2  93.94%   54-55
packages/ragbits-core/src/ragbits/core/utils/__init__.py                                                          0       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/_pyproject.py                                                       38       1  97.37%   113
packages/ragbits-core/src/ragbits/core/utils/config_handling.py                                                  72       8  88.89%   16, 54-55, 62-63, 152-154
packages/ragbits-core/src/ragbits/core/utils/decorators.py                                                       29       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/dict_transformations.py                                             72       3  95.83%   24, 27, 108
packages/ragbits-core/src/ragbits/core/vector_stores/__init__.py                                                  3       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/_cli.py                                                     51       4  92.16%   63, 85, 91, 121
packages/ragbits-core/src/ragbits/core/vector_stores/base.py                                                     40       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/chroma.py                                                   59       1  98.31%   77
packages/ragbits-core/src/ragbits/core/vector_stores/in_memory.py                                                38       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/qdrant.py                                                   66       1  98.48%   81
packages/ragbits-core/tests/cli/__init__.py                                                                       0       0  100.00%
packages/ragbits-core/tests/cli/test_cli_trace_handler.py                                                        48       3  93.75%   30, 43, 56
packages/ragbits-core/tests/cli/test_vector_store.py                                                            105       0  100.00%
packages/ragbits-core/tests/integration/vector_stores/test_vector_store.py                                       31       0  100.00%
packages/ragbits-core/tests/unit/__init__.py                                                                      0       0  100.00%
packages/ragbits-core/tests/unit/test_options.py                                                                 21       0  100.00%
packages/ragbits-core/tests/unit/audit/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/audit/test_cli.py                                                              102       0  100.00%
packages/ragbits-core/tests/unit/audit/test_trace.py                                                             93       3  96.77%   14, 17, 20
packages/ragbits-core/tests/unit/embeddings/test_from_config.py                                                  14       0  100.00%
packages/ragbits-core/tests/unit/llms/__init__.py                                                                 0       0  100.00%
packages/ragbits-core/tests/unit/llms/test_from_config.py                                                        16       0  100.00%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                            70       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/__init__.py                                                         0       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_get_default_llm.py                                            12       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/__init__.py                                                      0       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/test_from_config.py                                             11       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/test_in_memory.py                                               22       0  100.00%
packages/ragbits-core/tests/unit/prompts/__init__.py                                                              0       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_parsers.py                                                         65       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_prompt.py                                                         165       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/__init__.py                                                    0       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/prompt_classes_for_tests.py                                   30       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/test_prompt_discovery.py                                      18       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/__init__.py                     2       1  50.00%   3
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/__init__.py             3       2  33.33%   2-4
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt1.py        14       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt2.py        14       0  100.00%
packages/ragbits-core/tests/unit/utils/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/utils/test_config_handling.py                                                   65       2  96.92%   27-28
packages/ragbits-core/tests/unit/utils/test_decorators.py                                                        26       2  92.31%   17, 39
packages/ragbits-core/tests/unit/utils/test_dict_transformations.py                                              69       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_find.py                                                    13       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_config.py                                               9       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_instace.py                                             37       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/__init__.py                                                        0       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chroma.py                                                    61       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_from_config.py                                               42       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_in_memory.py                                                 84       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_qdrant.py                                                    40       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/__init__.py                                          2       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                           119       4  96.64%   266-273, 276
packages/ragbits-document-search/src/ragbits/document_search/cli.py                                              28       1  96.43%   78
packages/ragbits-document-search/src/ragbits/document_search/documents/__init__.py                                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/document.py                               61       2  96.72%   99, 146
packages/ragbits-document-search/src/ragbits/document_search/documents/element.py                                78      12  84.62%   87, 167-174, 183-185
packages/ragbits-document-search/src/ragbits/document_search/documents/exceptions.py                             11       5  54.55%   7-8, 17, 26-27
packages/ragbits-document-search/src/ragbits/document_search/documents/source_resolver.py                        20       5  75.00%   5, 51-52, 55-56
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py                               180      15  91.67%   166, 204, 246, 383-386, 390-391, 421, 428-429, 443-445
packages/ragbits-document-search/src/ragbits/document_search/ingestion/__init__.py                                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py                     33       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/__init__.py           5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/base.py              25       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/batched.py           18       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/distributed.py       30       8  73.33%   8-9, 36, 64-71
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/sequential.py        13       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py                      3       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/base.py                         19       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/dummy.py                        20       7  65.00%   33, 54-60
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/__init__.py         4       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/default.py         46       4  91.30%   98, 103-104, 137
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/images.py          50      19  62.00%   73-80, 87-99, 111, 124
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/pdf.py             19       6  68.42%   23, 35-43
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/utils.py           38      11  71.05%   71, 82-83, 98-101, 110, 121-123
packages/ragbits-document-search/src/ragbits/document_search/retrieval/__init__.py                                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py                     5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/base.py                         9       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/llm.py                         25       4  84.00%   47-50
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/noop.py                         6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/prompts.py                     16       1  93.75%   52
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                      3       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/base.py                         17       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/litellm.py                      16       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                          9       0  100.00%
packages/ragbits-document-search/tests/__init__.py                                                                0       0  100.00%
packages/ragbits-document-search/tests/helpers.py                                                                 3       0  100.00%
packages/ragbits-document-search/tests/cli/test_search.py                                                        56       0  100.00%
packages/ragbits-document-search/tests/integration/__init__.py                                                    0       0  100.00%
packages/ragbits-document-search/tests/integration/test_rerankers.py                                             15       6  60.00%   18-38
packages/ragbits-document-search/tests/integration/test_sources.py                                               23      10  56.52%   22-32, 40-45
packages/ragbits-document-search/tests/integration/test_unstructured.py                                          48      10  79.17%   52-58, 71-77
packages/ragbits-document-search/tests/unit/test_config.py                                                       57       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_processor.py                                           17       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                                             233       1  99.57%   466
packages/ragbits-document-search/tests/unit/test_documents.py                                                    13       0  100.00%
packages/ragbits-document-search/tests/unit/test_elements.py                                                     19       0  100.00%
packages/ragbits-document-search/tests/unit/test_local_file_source.py                                            13       0  100.00%
packages/ragbits-document-search/tests/unit/test_processing_strategies.py                                        25       0  100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                                    41       0  100.00%
packages/ragbits-document-search/tests/unit/test_rephrasers.py                                                   26       0  100.00%
packages/ragbits-document-search/tests/unit/test_rerankers.py                                                    51       1  98.04%   23
packages/ragbits-document-search/tests/unit/test_source_discriminator.py                                         35       0  100.00%
packages/ragbits-document-search/tests/unit/test_sources.py                                                      52       8  84.62%   20-21, 56-61
packages/ragbits-document-search/tests/unit/testprojects/project_with_instance_factory/__init__.py                0       0  100.00%
packages/ragbits-document-search/tests/unit/testprojects/project_with_instance_factory/factories.py              20       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/__init__.py                                                    0       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/base.py                                                       15       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/openai_moderation.py                                          19       5  73.68%   29-33
packages/ragbits-guardrails/tests/unit/test_openai_moderation.py                                                 35       0  100.00%
TOTAL                                                                                                          4633     334  92.79%

Diff against main

Filename                                                                                     Stmts    Miss  Cover
-----------------------------------------------------------------------------------------  -------  ------  -------
packages/ragbits-document-search/src/ragbits/document_search/_main.py                           +4       0  +0.12%
packages/ragbits-document-search/src/ragbits/document_search/documents/source_resolver.py      +20      +5  +75.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py              +58      -1  +4.78%
packages/ragbits-document-search/tests/unit/test_document_search.py                           +145      +1  -0.43%
packages/ragbits-document-search/tests/unit/test_sources.py                                    +27      +8  -15.38%
TOTAL                                                                                         +254     +13  +0.12%

Results for commit: e8aa52c

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

@micpst micpst linked an issue Dec 27, 2024 that may be closed by this pull request
@micpst micpst changed the title Feat/221 document search sources string resolver feat: document search sources string resolver Dec 27, 2024
@maciejklimek maciejklimek force-pushed the feat/221-document-search-sources-string-resolver branch 2 times, most recently from d3b1e38 to 4505521 Compare December 30, 2024 14:43
Copy link
Contributor

github-actions bot commented Dec 30, 2024

Trivy scanning results.

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: JWT (jwt-token)
════════════════════════════════════════
JWT token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA:80
────────────────────────────────────────
78 >>> encoded = jwt.encode({"some": "payload"}, "secret", algorithm="HS256")
79 >>> print(encoded)
80 [ *********************************************************************************************************
81 >>> jwt.decode(encoded, "secret", algorithms=["HS256"])
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/llms/huggingface/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

Total: 1 (MEDIUM: 0, HIGH: 0, CRITICAL: 1)

CRITICAL: HuggingFace (hugging-face-access-token)
════════════════════════════════════════
Hugging Face Access Token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/llms/huggingface/huggingface_llms_metadata/hf_text_generation_models.txt:36162
────────────────────────────────────────
36160 mncai/Llama2-7B-Active_3rd-floor-LoRA-dim64_epoch4
36161 ajcdp/CM
36162 [ Nagharjun17/*************************************
36163 BigSalmon/InformalToFormalLincoln114Paraphrase
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Slack (slack-web-hook)
════════════════════════════════════════
Slack Webhook
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/proxy/_types.py:1314
────────────────────────────────────────
1312 alert_to_webhook_url: Optional[Dict] = Field(
1313 None,
1314 [ bhook_url: {'budget_alerts': '*****************************************************************************'}`",
1315 )
────────────────────────────────────────

@ludwiktrammer
Copy link
Collaborator

This PR should probably also include some documentation

@ludwiktrammer
Copy link
Collaborator

@maciejklimek Did you intend to push another commit? Because your comments indicate that you changed things but I don't see the changes here.

@maciejklimek
Copy link
Contributor Author

@maciejklimek Did you intend to push another commit? Because your comments indicate that you changed things but I don't see the changes here.

Yeah, sorry for that, but I just really didn't have enough capacity to take care of this.

@ludwiktrammer
Copy link
Collaborator

@maciejklimek No problem then :) I just thought that maybe you simply forgot to do a push

@kdziedzic68 kdziedzic68 force-pushed the feat/221-document-search-sources-string-resolver branch from b2a9ee3 to ce8453a Compare January 22, 2025 12:37
@kdziedzic68 kdziedzic68 merged commit 839271e into main Jan 22, 2025
6 checks passed
@kdziedzic68 kdziedzic68 deleted the feat/221-document-search-sources-string-resolver branch January 22, 2025 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat(document-search): document-search sources string resolver
3 participants