Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(google-genai): Context Caching #7169

Merged
merged 8 commits into from
Dec 4, 2024
Merged

feat(google-genai): Context Caching #7169

merged 8 commits into from
Dec 4, 2024

Conversation

kwei-zhang
Copy link
Contributor

Fixes #5841

Implemented conetxt caching feature for google-genai, we now allow user to caching a file and create gen-ai model base on the cached content

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Nov 7, 2024
@vercel Vercel
Copy link

vercel bot commented Nov 7, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-docs ✅ Ready (Inspect) Visit Preview Dec 4, 2024 3:09am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ⬜️ Ignored (Inspect) Dec 4, 2024 3:09am

@dosubot dosubot bot added the auto:improvement Medium size change to existing code to handle new use-cases label Nov 7, 2024
@kwei-zhang kwei-zhang marked this pull request as draft November 7, 2024 11:45
@kwei-zhang kwei-zhang force-pushed the google-genai-context-caching branch from 8e039e2 to b065a2f Compare November 7, 2024 23:26
@kwei-zhang
Copy link
Contributor Author

Hi @jacoblee93, we've implemented the foundational structure for context caching with files. Could you take a look and let us know if the code aligns with our intended design? Thank you!

@kwei-zhang kwei-zhang marked this pull request as ready for review November 13, 2024 17:30
@kwei-zhang kwei-zhang force-pushed the google-genai-context-caching branch from b065a2f to 6658acb Compare November 13, 2024 17:30
@jacoblee93
Copy link
Collaborator

Looks reasonable to me - @afirstenberg can you have a look? It looks similar to some work you've done on Vertex.

We will also want to write up some docs!

@jacoblee93 jacoblee93 added the close PRs that need one or two touch-ups to be ready label Nov 17, 2024
@chaunguyenm
Copy link

Looks reasonable to me - @afirstenberg can you have a look? It looks similar to some work you've done on Vertex.

We will also want to write up some docs!

We will be working on the docs now, but please keep us updated if there's any suggestion on the design or the tests. Thank you both!

@chaunguyenm
Copy link

@jacoblee93 @afirstenberg We currently add a wrapper around GoogleAIFileManager and GoogleAICacheManager to support context caching, but the wrapper doesn't provide additional functionalities. This is because we are not sure if we can rely on users installing and using google/generative-ai/server package directly. Do you have a suggestion on how we can structure this? Thanks a lot.

jacoblee93
jacoblee93 reviewed Dec 3, 2024
View reviewed changes
libs/langchain-google-genai/src/tests/context_caching.int.test.ts

const filename = fileURLToPath(import.meta.url);
const dirname = path.dirname(filename);
const pathToVideoFile = path.join(dirname, "/data/Sherlock_Jr_FullMovie.mp4");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use a smaller file for this test? Maybe one of the files that already exist in the repo?

@jacoblee93
Copy link
Collaborator

Thank you! Removed the intermediate class you added in favor of just the raw managers

It wasn't exported anyway

@jacoblee93 jacoblee93 removed the close PRs that need one or two touch-ups to be ready label Dec 4, 2024
@dosubot dosubot bot added the lgtm PRs that are ready to be merged as-is label Dec 4, 2024
@jacoblee93 jacoblee93 changed the title google-genai [feature]: Context Caching feat(google-genai): Context Caching Dec 4, 2024
@jacoblee93 jacoblee93 merged commit 5f62174 into langchain-ai:main Dec 4, 2024
28 checks passed
@jacoblee93
Copy link
Collaborator

Also renamed to useCachedContent

Thanks again, will go live shortly!

syntaxsec pushed a commit to aks-456/langchainjs that referenced this pull request Dec 13, 2024
@kwei-zhang @jacoblee93 @syntaxsec
feat(google-genai): Context Caching (langchain-ai#7169)
16e3d54 
Co-authored-by: Chau Nguyen <[email protected]>
Co-authored-by: jacoblee93 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chaunguyenm chaunguyenm chaunguyenm left review comments

@jacoblee93 jacoblee93 jacoblee93 approved these changes

Assignees
No one assigned
Labels
auto:improvement Medium size change to existing code to handle new use-cases lgtm PRs that are ready to be merged as-is size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

google-common google-genai [feature]: Context Caching
3 participants
@kwei-zhang @jacoblee93 @chaunguyenm