Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core[minor],langchain[patch]: Move base indexing interface and logic to core #20667

Merged
merged 15 commits into from
Apr 24, 2024

Conversation

eyurtsev
Copy link
Collaborator

This PR moves the interface and the logic to core.

The following changes to namespaces:

indexes -> indexing
indexes._api -> indexing.api

Testing code is intentionally duplicated for now since it's testing different
implementations of the record manager (in-memory vs. SQL).

Common logic will need to be pulled out into the test client.

A follow up PR will move the SQL based implementation outside of LangChain.

Copy link

vercel bot commented Apr 19, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
langchain ⬜️ Ignored (Inspect) Visit Preview Apr 24, 2024 5:02pm

@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. 🤖:refactor A large refactor of a feature(s) or restructuring of many files labels Apr 19, 2024
@eyurtsev eyurtsev mentioned this pull request Apr 23, 2024
Copy link
Member

@efriis efriis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implementation good - some questions about tests

from langchain_core.pydantic_v1 import root_validator
from langchain_core.vectorstores import VectorStore

from langchain.indexes.base import NAMESPACE_UUID, RecordManager
NAMESPACE_UUID = uuid.UUID(int=1984)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i know this is just a migration but a comment of what this is doing would be great

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be worth keeping some form of the langchain ones here just to confirm we don't break anything for existing users for now (maybe with a note at top to remove test file for 0.2)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we removing these

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I think I meant to remove some of the unit tests but not all of them (duplicated coverage with test_indexing in core). I can revert for now and just keep the duplicated code

@dosubot dosubot bot added the lgtm PR looks good. Use to confirm that a PR is ready for merging. label Apr 23, 2024
@eyurtsev eyurtsev merged commit d8aa72f into master Apr 24, 2024
96 checks passed
@eyurtsev eyurtsev deleted the eugene/indexing branch April 24, 2024 17:18
hinthornw pushed a commit that referenced this pull request Apr 26, 2024
…to core (#20667)

This PR moves the interface and the logic to core.

The following changes to namespaces:


`indexes` -> `indexing`
`indexes._api` -> `indexing.api`


Testing code is intentionally duplicated for now since it's testing
different
implementations of the record manager (in-memory vs. SQL).

Common logic will need to be pulled out into the test client.


A follow up PR will move the SQL based implementation outside of
LangChain.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm PR looks good. Use to confirm that a PR is ready for merging. 🤖:refactor A large refactor of a feature(s) or restructuring of many files size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants