Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

checkpoint: add DuckDB store #2154

Merged
merged 18 commits into from
Oct 23, 2024
Merged

checkpoint: add DuckDB store #2154

merged 18 commits into from
Oct 23, 2024

Conversation

vbarda
Copy link
Collaborator

@vbarda vbarda commented Oct 21, 2024

No description provided.

@vbarda vbarda requested a review from nfcampos October 21, 2024 21:47
)
)

if PutOp in grouped_ops:
Copy link
Contributor

@eyurtsev eyurtsev Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we worried about race conditions?

We have write operations mixed with read operations?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mm, that's a good question. don't think we're currently calling them together anywhere, but it's probably good to avoid that -- i can add a warning?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this exposed to end users?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes but we're not showing that anywhere in the docs. i.e. the APIs that are actually meant to be used are .put() / .search() / .get, which just call .batch() under the hood

Base automatically changed from vb/add-duckdb-checkpointer to main October 23, 2024 21:11
) -> list[tuple[str, Sequence]]:
inserts: list[PutOp] = []
deletes: list[PutOp] = []
for _, op in put_ops:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this was copied over from the Postgres implementation.

I'm wondering if there's a scenario where breaking the ordering of DELETE and INSERT in the original list of put_ops causes an issue.

This method accepts an arbitrary list of put_ops and returns a reordered list of queries.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is a valid concern, kind of related to Eugene's questions above. i think we'll need to rethink this a bit for both postgres and duckdb, but for now all of the publicly documented methods that we show in the docs are single ops https://github.com/langchain-ai/langgraph/blob/main/libs/checkpoint/langgraph/store/base/__init__.py#L260

libs/checkpoint-duckdb/langgraph/store/duckdb/base.py Outdated Show resolved Hide resolved
@vbarda vbarda enabled auto-merge (squash) October 23, 2024 22:14
@vbarda vbarda merged commit 62a5ec5 into main Oct 23, 2024
58 checks passed
@vbarda vbarda deleted the vb/add-duckdb-store branch October 23, 2024 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants