Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(document-search): document-search sources string resolver #221

Open
mhordynski opened this issue Dec 2, 2024 · 0 comments · May be fixed by #264
Open

feat(document-search): document-search sources string resolver #221

mhordynski opened this issue Dec 2, 2024 · 0 comments · May be fixed by #264
Assignees
Labels
feature New feature or request
Milestone

Comments

@mhordynski
Copy link
Member

mhordynski commented Dec 2, 2024

Feature description

Add a string resolver to identify which sources the user wants to ingest into a document search.

Here’s an example:

await document_search.ingest("gcs://my_bucket/*")  # Include all files from the GCS bucket "my_bucket"
await document_search.ingest("s3://my_bucket/*")  # Include all files from the S3 bucket "my_bucket"

Each Source can register a new "protocol" in the SourceResolver, thus user-defined sources also can be auto-resolved.

Motivation

Currently, users are forced to delve into our documentation to figure out the different sources. Additionally, the Source.list_source API is not streamlined and varies between different implementations.

Additional context

No response

@mhordynski mhordynski added the feature New feature or request label Dec 2, 2024
@mhordynski mhordynski changed the title feat: document-search sources string resolver feat(document-search): document-search sources string resolver Dec 2, 2024
@mhordynski mhordynski moved this to Backlog in ragbits Dec 2, 2024
@mhordynski mhordynski moved this from Backlog to Ready in ragbits Dec 2, 2024
@mhordynski mhordynski added this to the Ragbits 0.6 milestone Dec 2, 2024
@maciejklimek maciejklimek self-assigned this Dec 4, 2024
@mhordynski mhordynski moved this from Ready to In Progress in ragbits Dec 5, 2024
@maciejklimek maciejklimek linked a pull request Dec 27, 2024 that will close this issue
@micpst micpst linked a pull request Dec 27, 2024 that will close this issue
@ludwiktrammer ludwiktrammer modified the milestones: Ragbits 0.6, Ragbits 0.7 Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

3 participants