refactor langchain.retrievers.self_query

[leo-gan]

Issue: all classes in langchain.retrievers.self_query namespace belong to the community package.

Following issue:
We cannot just move these classes to community because of the dependency on the Visitor class and other classes from the langchain.chains.query_constructor namespace.

What is interesting is that langchain.chains.query_constructor artifacts are used ONLY in the langchain.retrievers.self_query artifacts.
It happens that langchain.chains.query_constructor artifacts do not depend on the langchain.chains. The langchain.chains.query_constructor is quite independent. Because of that we can move langchain.chains.query_constructor under langchain_core as the langchain_core.sql_constructor
The reason for renaming query_constructor into sql_constructor is simple. It IS the SQL constructor. Moreover, the query_.. is the ambiguous name in the LLM context because we use query in retrievers and many other places not as the SQL queries but as the queries in prompts. sql is unambiguous name here.

NOTE: The langchain.chains.query_constructor.base.load_query_constructor_chain is not moved! This is the only dependency to chains. It is not important because Chains will be deprecated soon.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 22, 2024 6:53pm

[leo-gan]

@baskaryan Conflicts resolved. Any comments?

[leo-gan]

@baskaryan Hey Bagatur, let me know if you have other plans for extracting community code from langchain.

[leo-gan]

@efriis Merging conflicts resolved. Please, take a look.

[leo-gan]

@baskaryan Please let me know if this PR makes sense. If it makes sense, then I resolve conflicts. I've resolved branch conflicts several times, but nobody has reviewed PR.

[eyurtsev]

@cbornet

[eyurtsev]

We should keep as much of the provider agnostic code in langchain and not move it, so that providers that move to partner packages are able to re-use it without importing from community

[leo-gan]

@eyurtsev Do you mean to keep the from __future__ import annotations line?

[eyurtsev]

OK I think this PR is blocked on release 0.2.0 that will break dependency on langchain-community, and will make it possible for langchain-community to import these base chains form langchain (rather than langchain-core)

The issue is that these are higher order chains and it's not clear that they should live in langchain-core.

[leo-gan]

OK I think this PR is blocked on release 0.2.0 that will break dependency on langchain-community, and will make it possible for langchain-community to import these base chains form langchain (rather than langchain-core)

The issue is that these are higher order chains and it's not clear that they should live in langchain-core.

@eyurtsev Do you mean to wait with this PR till the langchain 0.2.0 release?

[leo-gan]

Converted to Draft. It could be a part of 0.2.0 langchain.

[eyurtsev]

Re-opened temporarily to make it easier to reference. I'm going to make a similar change, but slightly different name spaces

[cbornet]

This will need to be rebased carefully as there have been changes done to the self query retriever since the beginning of this PR.

[eyurtsev]

@cbornet yeah was mostly using for reference. @baskaryan doing it here: #20849

---

@leo-gan / @cbornet, context is that we're in the process of reversing the dependency between community and langchain

So langchain-community will start depending on langchain (rather than vice versa). So we'll be doing some acrobatics during the next week or so