-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Simplification of Document Search Evaluation interface #258
base: main
Are you sure you want to change the base?
Conversation
Code Coverage Summary
Diff against main
Results for commit: 5cd3f31 Minimum allowed coverage is ♻️ This comment has been updated with latest results |
Trivy scanning results. .venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0) MEDIUM: JWT (jwt-token) .venv/lib/python3.10/site-packages/litellm/llms/huggingface/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)Total: 1 (MEDIUM: 0, HIGH: 0, CRITICAL: 1) CRITICAL: HuggingFace (hugging-face-access-token) .venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0) MEDIUM: Slack (slack-web-hook) |
@@ -64,11 +65,11 @@ class DocumentSearchWithIngestionPipeline(DocumentSearchPipeline): | |||
|
|||
def __init__(self, config: DictConfig | None = None) -> None: | |||
super().__init__(config) | |||
self.config.vector_store.config.index_name = str(uuid.uuid4()) | |||
# self.config.vector_store.config.index_name = str(uuid.uuid4()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is it commented?
args: | ||
callback_type: neptune.integrations.optuna:NeptuneCallback | ||
project: deepsense-ai/ragbits | ||
#callbacks: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is it commented?
module = sys.modules[__name__] | ||
|
||
|
||
def pipeline_factory(pipeline_config: DictConfig) -> EvaluationPipeline: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe we can leverage new WithConstructionConfig
class here, same applies for merics and dataloaders factories
""" | ||
Function running evaluation for all datasets and evaluation tasks defined in config. | ||
""" | ||
config = OmegaConf.create( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we have to wrap these configs with hydra?
No description provided.