Merge branch 'master' into bagatur/json_mode_standard

langchain-ai · Aug 22, 2024 · bed7919 · bed7919
2 parents 9d2d45a + 7d13a2f
commit bed7919
Show file tree

Hide file tree

Showing 215 changed files with 17,073 additions and 5,642 deletions.
diff --git a/.github/scripts/check_diff.py b/.github/scripts/check_diff.py
@@ -68,6 +68,13 @@ def dependents_graph() -> dict:
 
                     if "langchain" in dep:
                         dependents[dep].add(pkg_dir)
+
+    # remove huggingface from dependents because of CI instability
+    # specifically in huggingface jobs
+    # https://github.com/langchain-ai/langchain/issues/25558
+    for k in dependents:
+        if "libs/partners/huggingface" in dependents[k]:
+            dependents[k].remove("libs/partners/huggingface")
     return dependents
 
 

diff --git a/README.md b/README.md
@@ -14,18 +14,20 @@
 
 Looking for the JS/TS library? Check out [LangChain.js](https://github.com/langchain-ai/langchainjs).
 
-To help you ship LangChain apps to production faster, check out [LangSmith](https://smith.langchain.com). 
-[LangSmith](https://smith.langchain.com) is a unified developer platform for building, testing, and monitoring LLM applications. 
+To help you ship LangChain apps to production faster, check out [LangSmith](https://smith.langchain.com).
+[LangSmith](https://smith.langchain.com) is a unified developer platform for building, testing, and monitoring LLM applications.
 Fill out [this form](https://www.langchain.com/contact-sales) to speak with our sales team.
 
 ## Quick Install
 
 With pip:
+
 ```bash
 pip install langchain
 ```
 
 With conda:
+
 ```bash
 conda install langchain -c conda-forge
 ```
@@ -36,22 +38,25 @@ conda install langchain -c conda-forge
 
 For these applications, LangChain simplifies the entire application lifecycle:
 
-- **Open-source libraries**:  Build your applications using LangChain's open-source [building blocks](https://python.langchain.com/v0.2/docs/concepts#langchain-expression-language-lcel), [components](https://python.langchain.com/v0.2/docs/concepts), and [third-party integrations](https://python.langchain.com/v0.2/docs/integrations/platforms/).
-Use [LangGraph](/docs/concepts/#langgraph) to build stateful agents with first-class streaming and human-in-the-loop support.
+- **Open-source libraries**: Build your applications using LangChain's open-source [building blocks](https://python.langchain.com/v0.2/docs/concepts#langchain-expression-language-lcel), [components](https://python.langchain.com/v0.2/docs/concepts), and [third-party integrations](https://python.langchain.com/v0.2/docs/integrations/platforms/).
+  Use [LangGraph](/docs/concepts/#langgraph) to build stateful agents with first-class streaming and human-in-the-loop support.
 - **Productionization**: Inspect, monitor, and evaluate your apps with [LangSmith](https://docs.smith.langchain.com/) so that you can constantly optimize and deploy with confidence.
 - **Deployment**: Turn your LangGraph applications into production-ready APIs and Assistants with [LangGraph Cloud](https://langchain-ai.github.io/langgraph/cloud/).
 
 ### Open-source libraries
+
 - **`langchain-core`**: Base abstractions and LangChain Expression Language.
 - **`langchain-community`**: Third party integrations.
   - Some integrations have been further split into **partner packages** that only rely on **`langchain-core`**. Examples include **`langchain_openai`** and **`langchain_anthropic`**.
 - **`langchain`**: Chains, agents, and retrieval strategies that make up an application's cognitive architecture.
 - **[`LangGraph`](https://langchain-ai.github.io/langgraph/)**: A library for building robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph. Integrates smoothly with LangChain, but can be used without it.
 
 ### Productionization:
+
 - **[LangSmith](https://docs.smith.langchain.com/)**: A developer platform that lets you debug, test, evaluate, and monitor chains built on any LLM framework and seamlessly integrates with LangChain.
 
 ### Deployment:
+
 - **[LangGraph Cloud](https://langchain-ai.github.io/langgraph/cloud/)**: Turn your LangGraph applications into production-ready APIs and Assistants.
 
 ![Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.](docs/static/svg/langchain_stack_062024.svg "LangChain Architecture Overview")
@@ -76,15 +81,17 @@ Use [LangGraph](/docs/concepts/#langgraph) to build stateful agents with first-c
 And much more! Head to the [Tutorials](https://python.langchain.com/v0.2/docs/tutorials/) section of the docs for more.
 
 ## 🚀 How does LangChain help?
+
 The main value props of the LangChain libraries are:
+
 1. **Components**: composable building blocks, tools and integrations for working with language models. Components are modular and easy-to-use, whether you are using the rest of the LangChain framework or not
 2. **Off-the-shelf chains**: built-in assemblages of components for accomplishing higher-level tasks
 
-Off-the-shelf chains make it easy to get started. Components make it easy to customize existing chains and build new ones. 
+Off-the-shelf chains make it easy to get started. Components make it easy to customize existing chains and build new ones.
 
 ## LangChain Expression Language (LCEL)
 
-LCEL is the foundation of many of LangChain's components, and is a declarative way to compose chains. LCEL was designed from day 1 to support putting prototypes in production, with no code changes, from the simplest “prompt + LLM” chain to the most complex chains.
+LCEL is a key part of LangChain, allowing you to build and organize chains of processes in a straightforward, declarative manner. It was designed to support taking prototypes directly into production without needing to alter any code. This means you can use LCEL to set up everything from basic "prompt + LLM" setups to intricate, multi-step workflows.
 
 - **[Overview](https://python.langchain.com/v0.2/docs/concepts/#langchain-expression-language-lcel)**: LCEL and its benefits
 - **[Interface](https://python.langchain.com/v0.2/docs/concepts/#runnable-interface)**: The standard Runnable interface for LCEL objects
@@ -123,7 +130,6 @@ Please see [here](https://python.langchain.com) for full documentation, which in
 - [🦜🕸️ LangGraph](https://langchain-ai.github.io/langgraph/): Create stateful, multi-actor applications with LLMs. Integrates smoothly with LangChain, but can be used without it.
 - [🦜🏓 LangServe](https://python.langchain.com/docs/langserve): Deploy LangChain runnables and chains as REST APIs.
 
-
 ## 💁 Contributing
 
 As an open-source project in a rapidly developing field, we are extremely open to contributions, whether it be in the form of a new feature, improved infrastructure, or better documentation.

diff --git a/cookbook/Multi_modal_RAG_google.ipynb b/cookbook/Multi_modal_RAG_google.ipynb
@@ -445,7 +445,7 @@
     "\n",
     "\n",
     "def plt_img_base64(img_base64):\n",
-    "    \"\"\"Disply base64 encoded string as image\"\"\"\n",
+    "    \"\"\"Display base64 encoded string as image\"\"\"\n",
     "    # Create an HTML img tag with the base64 string as the source\n",
     "    image_html = f'<img src=\"data:image/jpeg;base64,{img_base64}\" />'\n",
     "    # Display the image by rendering the HTML\n",

diff --git a/cookbook/langgraph_self_rag.ipynb b/cookbook/langgraph_self_rag.ipynb
@@ -336,7 +336,7 @@
     "    # Create a prompt template with format instructions and the query\n",
     "    prompt = PromptTemplate(\n",
     "        template=\"\"\"You are generating questions that is well optimized for retrieval. \\n \n",
-    "        Look at the input and try to reason about the underlying sematic intent / meaning. \\n \n",
+    "        Look at the input and try to reason about the underlying semantic intent / meaning. \\n \n",
     "        Here is the initial question:\n",
     "        \\n ------- \\n\n",
     "        {question} \n",
@@ -643,7 +643,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3.11.1 64-bit",
    "language": "python",
    "name": "python3"
   },
@@ -657,7 +657,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.11.1"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "1a1af0ee75eeea9e2e1ee996c87e7a2b11a0bebd85af04bb136d915cefc0abce"
+   }
   }
  },
  "nbformat": 4,

diff --git a/cookbook/rag-locally-on-intel-cpu.ipynb b/cookbook/rag-locally-on-intel-cpu.ipynb
@@ -647,7 +647,7 @@
    "metadata": {},
    "source": [
     "**Now we see the results are correct as it is mentioned in earnings release.** <br>\n",
-    "**To further automate, we will create a chain that will take input as question and retriever so that we don't need to retrieve documents seperately**"
+    "**To further automate, we will create a chain that will take input as question and retriever so that we don't need to retrieve documents separately**"
    ]
   },
   {
@@ -734,9 +734,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "rag-on-intel",
+   "display_name": "Python 3.11.1 64-bit",
    "language": "python",
-   "name": "rag-on-intel"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -748,7 +748,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.9"
+   "version": "3.11.1"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "1a1af0ee75eeea9e2e1ee996c87e7a2b11a0bebd85af04bb136d915cefc0abce"
+   }
   }
  },
  "nbformat": 4,

diff --git a/docs/Makefile b/docs/Makefile
@@ -18,7 +18,7 @@ for dir; do \
     if find "$$dir" -maxdepth 1 -type f \( -name "pyproject.toml" -o -name "setup.py" \) | grep -q .; then \
         echo "$$dir"; \
     fi \
-done' sh {} + | grep -vE "airbyte|ibm|couchbase" | tr '\n' ' ')
+done' sh {} + | grep -vE "airbyte|ibm|couchbase|databricks" | tr '\n' ' ')
 
 PORT ?= 3001
 

diff --git a/docs/api_reference/create_api_rst.py b/docs/api_reference/create_api_rst.py
@@ -318,7 +318,7 @@ def _construct_doc(
 
         index_autosummary += f"""
 :ref:`{package_namespace}_{module}`
-{'^' * (len(module) + 5)}
+{'^' * (len(package_namespace) + len(module) + 8)}
 """
 
         if classes:
@@ -448,7 +448,6 @@ def _construct_doc(
 """
         docs.append((f"{module}.rst", module_doc))
     docs.append(("index.rst", index_doc + index_autosummary))
-
     return docs
 
 
@@ -530,56 +529,30 @@ def _build_index(dirs: List[str]) -> None:
     ordered = ["core", "langchain", "text-splitters", "community", "experimental"]
     main_ = [dir_ for dir_ in ordered if dir_ in dirs]
     integrations = sorted(dir_ for dir_ in dirs if dir_ not in main_)
-    main_headers = [
-        " ".join(custom_names.get(x, x.title()) for x in dir_.split("-"))
-        for dir_ in main_
-    ]
-    integration_headers = [
-        " ".join(
-            custom_names.get(x, x.title().replace("ai", "AI").replace("db", "DB"))
-            for x in dir_.split("-")
-        )
-        for dir_ in integrations
-    ]
-    main_tree = "\n".join(
-        f"{header_name}<{dir_.replace('-', '_')}/index>"
-        for header_name, dir_ in zip(main_headers, main_)
-    )
-    main_grid = "\n".join(
-        f'- header: "**{header_name}**"\n  content: "{_package_namespace(dir_).replace("_", "-")}: {_get_package_version(_package_dir(dir_))}"\n  link: {dir_.replace("-", "_")}/index.html'
-        for header_name, dir_ in zip(main_headers, main_)
-    )
-    integration_tree = "\n".join(
-        f"{header_name}<{dir_.replace('-', '_')}/index>"
-        for header_name, dir_ in zip(integration_headers, integrations)
-    )
-
-    integration_grid = ""
-    integrations_to_show = [
-        "openai",
-        "anthropic",
-        "google-vertexai",
-        "aws",
-        "huggingface",
-        "mistralai",
-    ]
-    for header_name, dir_ in sorted(
-        zip(integration_headers, integrations),
-        key=lambda h_d: integrations_to_show.index(h_d[1])
-        if h_d[1] in integrations_to_show
-        else len(integrations_to_show),
-    )[: len(integrations_to_show)]:
-        integration_grid += f'\n- header: "**{header_name}**"\n  content: {_package_namespace(dir_).replace("_", "-")} {_get_package_version(_package_dir(dir_))}\n  link: {dir_.replace("-", "_")}/index.html'
-    doc = f"""# LangChain Python API Reference
+    doc = """# LangChain Python API Reference
 
 Welcome to the LangChain Python API reference. This is a reference for all 
 `langchain-x` packages. 
 
 For user guides see [https://python.langchain.com](https://python.langchain.com).
 
 For the legacy API reference hosted on ReadTheDocs see [https://api.python.langchain.com/](https://api.python.langchain.com/).
+"""
 
-## Base packages
+    if main_:
+        main_headers = [
+            " ".join(custom_names.get(x, x.title()) for x in dir_.split("-"))
+            for dir_ in main_
+        ]
+        main_tree = "\n".join(
+            f"{header_name}<{dir_.replace('-', '_')}/index>"
+            for header_name, dir_ in zip(main_headers, main_)
+        )
+        main_grid = "\n".join(
+            f'- header: "**{header_name}**"\n  content: "{_package_namespace(dir_).replace("_", "-")}: {_get_package_version(_package_dir(dir_))}"\n  link: {dir_.replace("-", "_")}/index.html'
+            for header_name, dir_ in zip(main_headers, main_)
+        )
+        doc += f"""## Base packages
 
 ```{{gallery-grid}}
 :grid-columns: "1 2 2 3"
@@ -594,8 +567,37 @@ def _build_index(dirs: List[str]) -> None:
 
 {main_tree}
 ```
+"""
+    if integrations:
+        integration_headers = [
+            " ".join(
+                custom_names.get(x, x.title().replace("ai", "AI").replace("db", "DB"))
+                for x in dir_.split("-")
+            )
+            for dir_ in integrations
+        ]
+        integration_tree = "\n".join(
+            f"{header_name}<{dir_.replace('-', '_')}/index>"
+            for header_name, dir_ in zip(integration_headers, integrations)
+        )
 
-## Integrations
+        integration_grid = ""
+        integrations_to_show = [
+            "openai",
+            "anthropic",
+            "google-vertexai",
+            "aws",
+            "huggingface",
+            "mistralai",
+        ]
+        for header_name, dir_ in sorted(
+            zip(integration_headers, integrations),
+            key=lambda h_d: integrations_to_show.index(h_d[1])
+            if h_d[1] in integrations_to_show
+            else len(integrations_to_show),
+        )[: len(integrations_to_show)]:
+            integration_grid += f'\n- header: "**{header_name}**"\n  content: {_package_namespace(dir_).replace("_", "-")} {_get_package_version(_package_dir(dir_))}\n  link: {dir_.replace("-", "_")}/index.html'
+        doc += f"""## Integrations
 
 ```{{gallery-grid}}
 :grid-columns: "1 2 2 3"
@@ -612,7 +614,6 @@ def _build_index(dirs: List[str]) -> None:
 
 {integration_tree}
 ```
-
 """
     with open(HERE / "reference.md", "w") as f:
         f.write(doc)

diff --git a/docs/api_reference/guide_imports.json b/docs/api_reference/guide_imports.json