Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding AstraDB as a DocumentStore #144

Merged
merged 80 commits into from
Jan 11, 2024
Merged

Adding AstraDB as a DocumentStore #144

merged 80 commits into from
Jan 11, 2024

Conversation

hc33brackles
Copy link
Contributor

This is the PR discussed with Masci and Tuana which adds Astra using the JSON API as a DocumentStore for Haystack built for Haystack 2.0.

All unit tests passing.

Working examples.

Looking forward to feedback. :-) --Nick

@hc33brackles hc33brackles requested a review from a team as a code owner December 22, 2023 14:53
@hc33brackles hc33brackles requested review from vblagoje and removed request for a team December 22, 2023 14:53
@CLAassistant
Copy link

CLAassistant commented Dec 22, 2023

CLA assistant check
All committers have signed the CLA.

@masci masci requested review from masci and removed request for vblagoje December 22, 2023 15:06
@masci masci mentioned this pull request Dec 23, 2023
10 tasks
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @hc33brackles Thank you for opening this PR. It looks quite good already! 👍 I left some comments after reviewing the code. However, I didn't have the time to run the examples yet. Please let me know if you have any questions regarding the review. 🙂

integrations/astra/examples/example.py Outdated Show resolved Hide resolved
integrations/astra/examples/example.py Outdated Show resolved Hide resolved
integrations/astra/examples/pipeline_example.py Outdated Show resolved Hide resolved
integrations/astra/pyproject.toml Outdated Show resolved Hide resolved
integrations/astra/pyproject.toml Outdated Show resolved Hide resolved
integrations/astra/src/astra_store/document_store.py Outdated Show resolved Hide resolved
integrations/astra/src/astra_store/document_store.py Outdated Show resolved Hide resolved
integrations/astra/src/astra_store/document_store.py Outdated Show resolved Hide resolved
integrations/astra/tests/test_document_store.py Outdated Show resolved Hide resolved
@julian-risch
Copy link
Member

@hc33brackles By the way, in order to run the other CI steps, the Contributor License Agreement needs to be signed by all contributors. You can find the check above: #144 (comment)

@hc33brackles
Copy link
Contributor Author

@julian-risch - we are still responding to some comments but all contributors have signed!

@masci masci removed their request for review January 8, 2024 08:20
@mounaTay
Copy link
Contributor

mounaTay commented Jan 8, 2024

@julian-risch We just pushed all the needed fixes and added our CI work as well.
Please let us know if you think we should add more changes to this PR
An FYI about the CI, it requires setting some git secrets to be able to connect to the datastax's database

@hc33brackles
Copy link
Contributor Author

Mouna is working on the linting issues, but there might be something more underlying with the way we changed the astra secrets to be handled is causing some of the errors in the python components. @julian-risch - I think if you input the ASTRA_DB_APPLICATION_TOKEN and ASTRA_DB_ID to the test repo as repo secrets these tests will pass.

@julian-risch julian-risch self-requested a review January 9, 2024 08:48
@mounaTay mounaTay force-pushed the main branch 8 times, most recently from 099a730 to 3cb6d0d Compare January 11, 2024 18:32
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks all very good to me! 👍 We just need to skip the tests when the API secret tokens are not available. I suggested the required changes so that you can commit them directly from the UI. I couldn't commit to your branch directly.
Thank you @mounaTay for your help running the tests locally and confirming the availability of the keys was an issue.
Once the changes I suggested with this review are committed the CI should pass and I will approve and merge the PR.

Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Great job everyone involved!

@julian-risch julian-risch merged commit 0b55a52 into deepset-ai:main Jan 11, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants