langchain: Add PebbloRetrievalQA chain with Identity & Semantic Enforcement support

[Raj725]

• Description: PebbloRetrievalQA chain introduces identity enforcement using vector-db metadata filtering
• Dependencies: None
• Issue: None
• Documentation: Adding documentation for PebbloRetrievalQA chain in a separate PR(docs: Added providers page for Pebblo and docs for PebbloRetrievalQA #20746)
• Unit tests: New unit-tests added

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		May 15, 2024 1:08pm

[Raj725]

cc: @eyurtsev

[eyurtsev]

Hi @Raj725, could you move this into community?

The name of the retriever isn't tied conceptually to what it does, but to Pebblo. Is it provider specific or is it meant to be generic?

[Raj725]

Hi @Raj725, could you move this into community?

The name of the retriever isn't tied conceptually to what it does, but to Pebblo. Is it provider specific or is it meant to be generic?

@eyurtsev
I tried to move it to the community library, but now it's not allowing subclassing Chain. It's failing in linting.
Importing from the langchain library is not allowed

snippet from libs/community/scripts/lint_imports.sh :

# Initialize a variable to keep track of errors
errors=0

# make sure not importing from langchain or langchain_experimental
git --no-pager grep '^from langchain\.' . && errors=$((errors+1))
git --no-pager grep '^from langchain_experimental\.' . && errors=$((errors+1))

[Raj725]

Hi @Raj725, could you move this into community?
The name of the retriever isn't tied conceptually to what it does, but to Pebblo. Is it provider specific or is it meant to be generic?

@eyurtsev I tried to move it to the community library, but now it's not allowing subclassing Chain. It's failing in linting. Importing from the langchain library is not allowed

snippet from libs/community/scripts/lint_imports.sh :

# Initialize a variable to keep track of errors
errors=0

# make sure not importing from langchain or langchain_experimental
git --no-pager grep '^from langchain\.' . && errors=$((errors+1))
git --no-pager grep '^from langchain_experimental\.' . && errors=$((errors+1))

@eyurtsev We want to extend Chain, but "community" doesn't allow imports from "langchain". Can you please suggest an alternative approach?

[eyurtsev]

@Raj725 we're in the process of inverting the dependency between langchain and langchain-community (~2 weeks timeline)

---

Why do you need to use Chain rather than using LCEL for composition?

Chain is a legacy object

[eyurtsev]

@Raj725 any chance you could provide a bit of context for what this code is supposed to do ? Why should it be merged it into langchain/langchain-community?

[Raj725]

@Raj725 any chance you could provide a bit of context for what this code is supposed to do ? Why should it be merged it into langchain/langchain-community?

Context:
Sure. We are introducing PebbloRetrievalQA chain as a retrieval counterpart to Pebblo Safe DocumentLoader that @hwchase17 helped to merge into langchain-community sometime back.

For more context, Pebblo is an opensource project for Data Visibility & Governance for LangChain apps(LinkedIn Post by Langchain).

Both Pebblo Safe DocumentLoader and PebbloRetrievalQA work in tandem to add metadata tags on the loader with corresponding filters/enforcement on the retrieval side. Given that the Pebblo SafeLoader is already in langchain-community, our common customer base will be better served with the PebbloRetrievalQA chain, also in the same pkg.

References:

1. https://medium.com/@sridhar_ramaswamy/identity-enabled-rag-using-pebblo-4d608ed28c31
2. https://medium.com/@sridhar_ramaswamy/introducing-pebblo-data-visibility-governance-for-gen-ai-apps-086ca8a62d10

[Raj725]

@Raj725 we're in the process of inverting the dependency between langchain and langchain-community (~2 weeks timeline)

@eyurtsev Thanks for sharing the timeline. We can wait for dependency inversion.

Why do you need to use Chain rather than using LCEL for composition?

Chain is a legacy object

We would like to offer a governance and security solution for our existing customers who are using legacy chain-based solutions. We are also exploring LCEL.

[Raj725]

@Raj725 we're in the process of inverting the dependency between langchain and langchain-community (~2 weeks timeline)

@eyurtsev It looks like dependency inversion is complete; the lint error is gone now. However, it's still throwing the following error in the import check.

Can you please guide me on the right way to import "langchain" modules into "langchain-community"?

tests/unit_tests/chains/test_pebblo_retrieval.py:16: in <module>
    from langchain_community.chains import PebbloRetrievalQA
langchain_community/chains/__init__.py:7: in <module>
    from langchain_community.chains.pebblo_retrieval.base import PebbloRetrievalQA
langchain_community/chains/pebblo_retrieval/base.py:9: in <module>
    from langchain.chains.base import Chain
E   ModuleNotFoundError: No module named 'langchain'

[eyurtsev]

The dependency inversion will only happen for the 0.2 release of langchain -- still a few weeks away

[Raj725]

@eyurtsev Could you please review this PR? All checks are passing now.

[eyurtsev]

Suggested change

"PebbloRetrievalQA": "langchain.chains.pebblo_retrieval.base",

[eyurtsev]

It should use a hard-coded list of string literals otherwise some dev tools won't be able to pick it up