Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

langchain_community: Checking text property first in neo4j to avoid duplicate nodes #17381

Closed
wants to merge 9 commits into from
Closed

langchain_community: Checking text property first in neo4j to avoid duplicate nodes #17381

wants to merge 9 commits into from

Conversation

abhimalamkar
Copy link
Contributor

@abhimalamkar abhimalamkar commented Feb 11, 2024 •
edited
Loading

Checking text property first in neo4j to avoid duplicate nodes

@dev2049
@vowelparrot

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Feb 11, 2024
@vercel Vercel
Copy link

vercel bot commented Feb 11, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
langchain ⬜️ Ignored (Inspect) Visit Preview Feb 14, 2024 7:00am

@dosubot dosubot bot added Ɑ: vector store Related to vector store module 🤖:improvement Medium size change to existing code to handle new use-cases labels Feb 11, 2024
@tomasonjo
Copy link
Contributor

tomasonjo commented Feb 12, 2024 •
edited
Loading

Merging on long text properties is expensive and not good. If you want, you can change the id calculation by using a cheap hash function

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Feb 12, 2024
@tomasonjo
Copy link
Contributor

All you need to do is to change this one line: https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/vectorstores/neo4j_vector.py#L429

No need to introduce new params or anything

@abhimalamkar
Copy link
Contributor Author

abhimalamkar commented Feb 13, 2024 •
edited
Loading

@tomasonjo This diff now also handles creation of relationships to parent and child nodes.

@tomasonjo
Copy link
Contributor

Not really, this is way too specific and not general to be merged in

@abhimalamkar
Copy link
Contributor Author

abhimalamkar commented Feb 13, 2024 •
edited
Loading

@tomasonjo This does not break any existing functionality but allow the ability to create relationships with other exiting nodes while creating new docs.

No worries, I reverted my changes to just introduce text hash as ids

@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Feb 13, 2024
@baskaryan
Copy link
Collaborator

believe this was resolved in #18846, let me know if i'm missing something!

@baskaryan baskaryan closed this Mar 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
🤖:improvement Medium size change to existing code to handle new use-cases size:XS This PR changes 0-9 lines, ignoring generated files. Ɑ: vector store Related to vector store module
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@abhimalamkar @tomasonjo @baskaryan