Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(document-search): add gcs source to the DocumentMeta #6

Closed
mhordynski opened this issue Sep 12, 2024 · 0 comments · Fixed by #25
Closed

feat(document-search): add gcs source to the DocumentMeta #6

mhordynski opened this issue Sep 12, 2024 · 0 comments · Fixed by #25
Assignees
@akotyla
Labels
document search Changes to the document search package feature New feature or request integrations Changes in vendor integration
Milestone
Ragbits 0.1

Comments

@mhordynski
Copy link
Member

Feature description

Right now DocumentMeta supports only one source which is LocalFileSource.
The proposed improvement is to add new type of the source: GoogleCloudSource, which will indicate that a document is stored in GCS bucket. New type of source should allow to load the document from remote using asynchronous google-cloud libraries.

Motivation

GCS is one of the top storage providers, so it will be common to load documents from this source.

Additional context

https://pypi.org/project/gcloud-aio-storage/ handles async operations on GCS well
@mhordynski mhordynski added feature New feature or request document search Changes to the document search package labels Sep 12, 2024
@mhordynski mhordynski moved this to Backlog in ragbits Sep 13, 2024
@micpst micpst added the integrations Changes in vendor integration label Sep 13, 2024
@micpst micpst added this to the Document search release milestone Sep 13, 2024
@mhordynski mhordynski linked a pull request Sep 23, 2024 that will close this issue
feat(document-search): add gcs source to the DocumentMeta #25
Merged
@mhordynski mhordynski modified the milestones: Ragbits Document Search 0.1, Ragbits 0.1 Sep 23, 2024
@github-project-automation github-project-automation bot moved this from In review to Done in ragbits Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@akotyla akotyla

Labels
document search Changes to the document search package feature New feature or request integrations Changes in vendor integration
Projects
ragbits
Status: Done
Milestone
Ragbits 0.1
Development

Successfully merging a pull request may close this issue.

feat(document-search): add gcs source to the DocumentMeta deepsense-ai/ragbits
3 participants
@mhordynski @micpst @akotyla