-
Notifications
You must be signed in to change notification settings - Fork 16k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs:
integrations
reference updates 16 (#26059)
Added missed provider pages and links. Fixed inconsistent formatting. Co-authored-by: Erick Friis <[email protected]>
- Loading branch information
Showing
4 changed files
with
166 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
# Apache Software Foundation | ||
|
||
>[The Apache Software Foundation (Wikipedia)](https://en.wikipedia.org/wiki/The_Apache_Software_Foundation) | ||
> is a decentralized open source community of developers. The software they | ||
> produce is distributed under the terms of the Apache License, a permissive | ||
> open-source license for free and open-source software (FOSS). The Apache projects | ||
> are characterized by a collaborative, consensus-based development process | ||
> and an open and pragmatic software license, which is to say that it | ||
> allows developers, who receive the software freely, to redistribute | ||
> it under non-free terms. Each project is managed by a self-selected | ||
> team of technical experts who are active contributors to the project. | ||
## Apache AGE | ||
|
||
>[Apache AGE](https://age.apache.org/) is a `PostgreSQL` extension that provides | ||
> graph database functionality. `AGE` is an acronym for `A Graph Extension`, and | ||
> is inspired by Bitnine’s fork of `PostgreSQL 10`, `AgensGraph`, which is | ||
> a multimodal database. The goal of the project is to create single | ||
> storage that can handle both relational and graph model data so that users | ||
> can use standard ANSI SQL along with `openCypher`, the Graph query language. | ||
> The data elements `Apache AGE` stores are nodes, edges connecting them, and | ||
> attributes of nodes and edges. | ||
See more about [integrating with Apache AGE](/docs/integrations/graphs/apache_age). | ||
|
||
## Apache Cassandra | ||
|
||
>[Apache Cassandra](https://cassandra.apache.org/) is a NoSQL, row-oriented, | ||
> highly scalable and highly available database. Starting with version 5.0, | ||
> the database ships with vector search capabilities. | ||
See more about [integrating with Apache Cassandra](/docs/integrations/providers/cassandra/). | ||
|
||
## Apache Doris | ||
|
||
>[Apache Doris](https://doris.apache.org/) is a modern data warehouse for | ||
> real-time analytics. It delivers lightning-fast analytics on real-time data at scale. | ||
> | ||
>Usually `Apache Doris` is categorized into OLAP, and it has showed excellent | ||
> performance in ClickBench — a Benchmark For Analytical DBMS. Since it has | ||
> a super-fast vectorized execution engine, it could also be used as a fast vectordb. | ||
See more about [integrating with Apache Doris](/docs/integrations/providers/apache_doris/). | ||
|
||
## Apache Kafka | ||
|
||
>[Apache Kafka](https://github.com/apache/kafka) is a distributed messaging system | ||
> that is used to publish and subscribe to streams of records. | ||
See more about [integrating with Apache Kafka](/docs/integrations/memory/kafka_chat_message_history). | ||
|
||
|
||
## Apache Spark | ||
|
||
>[Apache Spark](https://spark.apache.org/) is a unified analytics engine for | ||
> large-scale data processing. It provides high-level APIs in Scala, Java, | ||
> Python, and R, and an optimized engine that supports general computation | ||
> graphs for data analysis. It also supports a rich set of higher-level | ||
> tools including `Spark SQL` for SQL and DataFrames, `pandas API on Spark` | ||
> for pandas workloads, `MLlib` for machine learning, | ||
> `GraphX` for graph processing, and `Structured Streaming` for stream processing. | ||
See more about [integrating with Apache Spark](/docs/integrations/providers/spark). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# Spark | ||
|
||
>[Apache Spark](https://spark.apache.org/) is a unified analytics engine for | ||
> large-scale data processing. It provides high-level APIs in Scala, Java, | ||
> Python, and R, and an optimized engine that supports general computation | ||
> graphs for data analysis. It also supports a rich set of higher-level | ||
> tools including `Spark SQL` for SQL and DataFrames, `pandas API on Spark` | ||
> for pandas workloads, `MLlib` for machine learning, | ||
> `GraphX` for graph processing, and `Structured Streaming` for stream processing. | ||
## Document loaders | ||
|
||
### PySpark | ||
|
||
It loads data from a `PySpark` DataFrame. | ||
|
||
See a [usage example](/docs/integrations/document_loaders/pyspark_dataframe). | ||
|
||
```python | ||
from langchain_community.document_loaders import PySparkDataFrameLoader | ||
``` | ||
|
||
## Tools/Toolkits | ||
|
||
### Spark SQL toolkit | ||
|
||
Toolkit for interacting with `Spark SQL`. | ||
|
||
See a [usage example](/docs/integrations/tools/spark_sql). | ||
|
||
```python | ||
from langchain_community.agent_toolkits import SparkSQLToolkit, create_spark_sql_agent | ||
from langchain_community.utilities.spark_sql import SparkSQL | ||
``` | ||
|
||
#### Spark SQL individual tools | ||
|
||
You can use individual tools from the Spark SQL Toolkit: | ||
- `InfoSparkSQLTool`: tool for getting metadata about a Spark SQL | ||
- `ListSparkSQLTool`: tool for getting tables names | ||
- `QueryCheckerTool`: tool uses an LLM to check if a query is correct | ||
- `QuerySparkSQLTool`: tool for querying a Spark SQL | ||
|
||
```python | ||
from langchain_community.tools.spark_sql.tool import InfoSparkSQLTool | ||
from langchain_community.tools.spark_sql.tool import ListSparkSQLTool | ||
from langchain_community.tools.spark_sql.tool import QueryCheckerTool | ||
from langchain_community.tools.spark_sql.tool import QuerySparkSQLTool | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters