diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 0000000000..e69de29bb2 diff --git a/404.html b/404.html new file mode 100644 index 0000000000..94f8f7b134 --- /dev/null +++ b/404.html @@ -0,0 +1,2131 @@ + + + +
+ + + + + + + + + + + + + + + + +The API supports combining multiple API instances into a single logical embeddings index. An example configuration is shown below.
+cluster:
+ shards:
+ - http://127.0.0.1:8002
+ - http://127.0.0.1:8003
+
This configuration aggregates the API instances above as index shards. Data is evenly split among each of the shards at index time. Queries are run in parallel against each shard and the results are joined together. This method allows horizontal scaling and supports very large index clusters.
+This method is only recommended for data sets in the 1 billion+ records. The ANN libraries can easily support smaller data sizes and this method is not worth the additional complexity. At this time, new shards can not be added after building the initial index.
+See the link below for a detailed example covering distributed embeddings clusters.
+Notebook | +Description | ++ |
---|---|---|
Distributed embeddings cluster | +Distribute an embeddings index across multiple data nodes | ++ |
Configuration is set through YAML. In most cases, YAML keys map to fields names in Python. The example in the previous section gave a full-featured example covering a wide array of configuration options.
+Each section below describes the available configuration settings.
+The configuration parser expects a top level embeddings
key to be present in the YAML. All embeddings configuration is supported.
The following example defines an embeddings index.
+path: index path
+writable: true
+
+embeddings:
+ path: vector model
+ content: true
+
Three top level settings are available to control where indexes are saved and if an index is a read-only index.
+path: string
+
Path to save and load the embeddings index. Each API instance can only access a single index at a time.
+writable: boolean
+
Determines if the input embeddings index is writable (true) or read-only (false). This allows serving a read-only index.
+Cloud storage settings can be set under a cloud
top level configuration group.
Pipelines are loaded as top level configuration parameters. Pipeline names are automatically detected in the YAML configuration and created upon startup. All pipelines are supported.
+The following example defines a series of pipelines. Note that entries below are the lower-case names of the pipeline class.
+caption:
+
+extractor:
+ path: model path
+
+labels:
+
+summary:
+
+tabular:
+
+translation:
+
Under each pipeline name, configuration settings for the pipeline can be set.
+Workflows are defined under a top level workflow
key. Each key under the workflow
key is the name of the workflow. Under that is a tasks
key with each task definition.
The following example defines a workflow.
+workflow:
+ sumtranslate:
+ tasks:
+ - action: summary
+ - action: translation
+
Schedules a workflow using a cron expression.
+workflow:
+ index:
+ schedule:
+ cron: 0/10 * * * * *
+ elements: ["api params"]
+ tasks:
+ - task: service
+ url: api url
+ - action: index
+
tasks: list
+
Expects a list of workflow tasks. Each element defines a single workflow task. All task configuration is supported.
+A shorthand syntax for creating tasks is supported. This syntax will automatically map task strings to an action:value
pair.
Example below.
+workflow:
+ index:
+ tasks:
+ - action1
+ - action2
+
Each task element supports the following additional arguments.
+action: string|list
+
Both single and multi-action tasks are supported.
+The action parameter works slightly different when passed via configuration. The parameter(s) needs to be converted into callable method(s). If action is a pipeline that has been defined in the current configuration, it will use that pipeline as the action.
+There are three special action names index
, upsert
and search
. If index
or upsert
are used as the action, the task will collect workflow data elements and load them into defined the embeddings index. If search
is used, the task will execute embeddings queries for each input data element.
Otherwise, the action must be a path to a callable object or function. The configuration parser will resolve the function name and use that as the task action.
+task: string
+
Optionally sets the type of task to create. For example, this could be a file
task or a retrieve
task. If this is not specified, a generic task is created. The list of workflow tasks can be found here.
args: list
+
Optional list of static arguments to pass to the workflow task. These are combined with workflow data to pass to each __call__
.
While the API is extremely flexible and complex logic can be executed through YAML-driven workflows, some may prefer to create an endpoint in Python. API extensions define custom Python endpoints that interact with txtai applications.
+See the link below for a detailed example covering how to create an API extension.
+Notebook | +Description | ++ |
---|---|---|
Custom API Endpoints | +Extend the API with custom endpoints | ++ |
+
+txtai has a full-featured API, backed by FastAPI, that can optionally be enabled for any txtai process. All functionality found in txtai can be accessed via the API.
+The following is an example configuration and startup script for the API.
+Note: This configuration file enables all functionality. For memory-bound systems, splitting pipelines into multiple instances is a best practice.
+# Index file path
+path: /tmp/index
+
+# Allow indexing of documents
+writable: True
+
+# Enbeddings index
+embeddings:
+ path: sentence-transformers/nli-mpnet-base-v2
+
+# Extractive QA
+extractor:
+ path: distilbert-base-cased-distilled-squad
+
+# Zero-shot labeling
+labels:
+
+# Similarity
+similarity:
+
+# Text segmentation
+segmentation:
+ sentences: true
+
+# Text summarization
+summary:
+
+# Text extraction
+textractor:
+ paragraphs: true
+ minlength: 100
+ join: true
+
+# Transcribe audio to text
+transcription:
+
+# Translate text between languages
+translation:
+
+# Workflow definitions
+workflow:
+ sumfrench:
+ tasks:
+ - action: textractor
+ task: url
+ - action: summary
+ - action: translation
+ args: ["fr"]
+ sumspanish:
+ tasks:
+ - action: textractor
+ task: url
+ - action: summary
+ - action: translation
+ args: ["es"]
+
Assuming this YAML content is stored in a file named config.yml, the following command starts the API process.
+CONFIG=config.yml uvicorn "txtai.api:app"
+
Uvicorn is a full-featured production-ready server. See the Uvicorn deployment guide for more on configuration options.
+The default port for the API is 8000. See the uvicorn link above to change this.
+txtai has a number of language bindings which abstract the API (see links below). Alternatively, code can be written to connect directly to the API. Documentation for a live running instance can be found at the /docs
url (i.e. http://localhost:8000/docs). The following example runs a workflow using cURL.
curl \
+ -X POST "http://localhost:8000/workflow" \
+ -H "Content-Type: application/json" \
+ -d '{"name":"sumfrench", "elements": ["https://github.com/neuml/txtai"]}'
+
The API service command specified earlier starts a Uvicorn server as a HTTP service on port 8000. To run a HTTPS service, consider the following options.
+TLS Proxy Server. Recommended choice. With this configuration, the txtai API service runs as a HTTP service only accessible on the localhost/local network. The proxy server handles all encryption and redirects requests to local services. See this example configuration for more.
+Uvicorn SSL Certificate. Another option is setting the SSL certificate on the Uvicorn service. This works in simple situations but gets complex when hosting multiple txtai or other related services.
+A local instance can be instantiated. In this case, a txtai application runs internally, without any network connections, providing the same consolidated functionality. This enables running txtai in Python with configuration.
+The configuration above can be run in Python with:
+from txtai.app import Application
+
+# Load and run workflow
+app = Application(config.yml)
+app.workflow("sumfrench", ["https://github.com/neuml/txtai"])
+
See this link for a full list of methods.
+The API can be containerized and run. This will bring up an API instance without having to install Python, txtai or any dependencies on your machine!
+See this section for more information.
+The following programming languages have bindings with the txtai API:
+See the link below for a detailed example covering how to use the API.
+Notebook | +Description | ++ |
---|---|---|
API Gallery | +Using txtai in JavaScript, Java, Rust and Go | ++ |
Base API template. The API is an extended txtai application, adding the ability to cluster API instances together.
+Downstream applications can extend this base template to add/modify functionality.
+ +txtai/api/base.py
class API(Application):
+ """
+ Base API template. The API is an extended txtai application, adding the ability to cluster API instances together.
+
+ Downstream applications can extend this base template to add/modify functionality.
+ """
+
+ def __init__(self, config, loaddata=True):
+ super().__init__(config, loaddata)
+
+ # Embeddings cluster
+ self.cluster = None
+ if self.config.get("cluster"):
+ self.cluster = Cluster(self.config["cluster"])
+
+ # pylint: disable=W0221
+ def search(self, query, limit=None, weights=None, index=None, request=None):
+ # When search is invoked via the API, limit is set from the request
+ # When search is invoked directly, limit is set using the method parameter
+ limit = self.limit(request.query_params.get("limit") if request and hasattr(request, "query_params") else limit)
+ weights = self.weights(request.query_params.get("weights") if request and hasattr(request, "query_params") else weights)
+ index = request.query_params.get("index") if request and hasattr(request, "query_params") else index
+
+ if self.cluster:
+ return self.cluster.search(query, limit, weights, index)
+
+ return super().search(query, limit, weights, index)
+
+ def batchsearch(self, queries, limit=None, weights=None, index=None):
+ if self.cluster:
+ return self.cluster.batchsearch(queries, self.limit(limit), weights, index)
+
+ return super().batchsearch(queries, limit, weights, index)
+
+ def add(self, documents):
+ """
+ Adds a batch of documents for indexing.
+
+ Downstream applications can override this method to also store full documents in an external system.
+
+ Args:
+ documents: list of {id: value, text: value}
+
+ Returns:
+ unmodified input documents
+ """
+
+ if self.cluster:
+ self.cluster.add(documents)
+ else:
+ super().add(documents)
+
+ return documents
+
+ def index(self):
+ """
+ Builds an embeddings index for previously batched documents.
+ """
+
+ if self.cluster:
+ self.cluster.index()
+ else:
+ super().index()
+
+ def upsert(self):
+ """
+ Runs an embeddings upsert operation for previously batched documents.
+ """
+
+ if self.cluster:
+ self.cluster.upsert()
+ else:
+ super().upsert()
+
+ def delete(self, ids):
+ """
+ Deletes from an embeddings index. Returns list of ids deleted.
+
+ Args:
+ ids: list of ids to delete
+
+ Returns:
+ ids deleted
+ """
+
+ if self.cluster:
+ return self.cluster.delete(ids)
+
+ return super().delete(ids)
+
+ def reindex(self, config, function=None):
+ """
+ Recreates this embeddings index using config. This method only works if document content storage is enabled.
+
+ Args:
+ config: new config
+ function: optional function to prepare content for indexing
+ """
+
+ if self.cluster:
+ self.cluster.reindex(config, function)
+ else:
+ super().reindex(config, function)
+
+ def count(self):
+ """
+ Total number of elements in this embeddings index.
+
+ Returns:
+ number of elements in embeddings index
+ """
+
+ if self.cluster:
+ return self.cluster.count()
+
+ return super().count()
+
+ def limit(self, limit):
+ """
+ Parses the number of results to return from the request. Allows range of 1-250, with a default of 10.
+
+ Args:
+ limit: limit parameter
+
+ Returns:
+ bounded limit
+ """
+
+ # Return between 1 and 250 results, defaults to 10
+ return max(1, min(250, int(limit) if limit else 10))
+
+ def weights(self, weights):
+ """
+ Parses the weights parameter from the request.
+
+ Args:
+ weights: weights parameter
+
+ Returns:
+ weights
+ """
+
+ return float(weights) if weights else weights
+
add(self, documents)
+
+
+Adds a batch of documents for indexing.
+Downstream applications can override this method to also store full documents in an external system.
+ +Parameters:
+Name | +Type | +Description | +Default | +
---|---|---|---|
documents |
+ + | list of {id: value, text: value} |
+ required | +
Returns:
+Type | +Description | +
---|---|
+ | unmodified input documents |
+
txtai/api/base.py
def add(self, documents):
+ """
+ Adds a batch of documents for indexing.
+
+ Downstream applications can override this method to also store full documents in an external system.
+
+ Args:
+ documents: list of {id: value, text: value}
+
+ Returns:
+ unmodified input documents
+ """
+
+ if self.cluster:
+ self.cluster.add(documents)
+ else:
+ super().add(documents)
+
+ return documents
+
batchsearch(self, queries, limit=None, weights=None, index=None)
+
+
+Finds documents most similar to the input queries. This method will run either an index search +or an index + database search depending on if a database is available.
+ +Parameters:
+Name | +Type | +Description | +Default | +
---|---|---|---|
queries |
+ + | input queries |
+ required | +
limit |
+ + | maximum results |
+ None |
+
weights |
+ + | hybrid score weights, if applicable |
+ None |
+
index |
+ + | index name, if applicable |
+ None |
+
Returns:
+Type | +Description | +
---|---|
list of {id |
+ value, score: value} per query for index search, list of dict per query for an index + database search |
+
txtai/api/base.py
def batchsearch(self, queries, limit=None, weights=None, index=None):
+ if self.cluster:
+ return self.cluster.batchsearch(queries, self.limit(limit), weights, index)
+
+ return super().batchsearch(queries, limit, weights, index)
+
count(self)
+
+
+Total number of elements in this embeddings index.
+ +Returns:
+Type | +Description | +
---|---|
+ | number of elements in embeddings index |
+
txtai/api/base.py
def count(self):
+ """
+ Total number of elements in this embeddings index.
+
+ Returns:
+ number of elements in embeddings index
+ """
+
+ if self.cluster:
+ return self.cluster.count()
+
+ return super().count()
+
delete(self, ids)
+
+
+Deletes from an embeddings index. Returns list of ids deleted.
+ +Parameters:
+Name | +Type | +Description | +Default | +
---|---|---|---|
ids |
+ + | list of ids to delete |
+ required | +
Returns:
+Type | +Description | +
---|---|
+ | ids deleted |
+
txtai/api/base.py
def delete(self, ids):
+ """
+ Deletes from an embeddings index. Returns list of ids deleted.
+
+ Args:
+ ids: list of ids to delete
+
+ Returns:
+ ids deleted
+ """
+
+ if self.cluster:
+ return self.cluster.delete(ids)
+
+ return super().delete(ids)
+
index(self)
+
+
+Builds an embeddings index for previously batched documents.
+ +txtai/api/base.py
def index(self):
+ """
+ Builds an embeddings index for previously batched documents.
+ """
+
+ if self.cluster:
+ self.cluster.index()
+ else:
+ super().index()
+
limit(self, limit)
+
+
+Parses the number of results to return from the request. Allows range of 1-250, with a default of 10.
+ +Parameters:
+Name | +Type | +Description | +Default | +
---|---|---|---|
limit |
+ + | limit parameter |
+ required | +
Returns:
+Type | +Description | +
---|---|
+ | bounded limit |
+
txtai/api/base.py
def limit(self, limit):
+ """
+ Parses the number of results to return from the request. Allows range of 1-250, with a default of 10.
+
+ Args:
+ limit: limit parameter
+
+ Returns:
+ bounded limit
+ """
+
+ # Return between 1 and 250 results, defaults to 10
+ return max(1, min(250, int(limit) if limit else 10))
+
reindex(self, config, function=None)
+
+
+Recreates this embeddings index using config. This method only works if document content storage is enabled.
+ +Parameters:
+Name | +Type | +Description | +Default | +
---|---|---|---|
config |
+ + | new config |
+ required | +
function |
+ + | optional function to prepare content for indexing |
+ None |
+
txtai/api/base.py
def reindex(self, config, function=None):
+ """
+ Recreates this embeddings index using config. This method only works if document content storage is enabled.
+
+ Args:
+ config: new config
+ function: optional function to prepare content for indexing
+ """
+
+ if self.cluster:
+ self.cluster.reindex(config, function)
+ else:
+ super().reindex(config, function)
+
search(self, query, limit=None, weights=None, index=None, request=None)
+
+
+Finds documents most similar to the input query. This method will run either an index search +or an index + database search depending on if a database is available.
+ +Parameters:
+Name | +Type | +Description | +Default | +
---|---|---|---|
query |
+ + | input query |
+ required | +
limit |
+ + | maximum results |
+ None |
+
weights |
+ + | hybrid score weights, if applicable |
+ None |
+
index |
+ + | index name, if applicable |
+ None |
+
Returns:
+Type | +Description | +
---|---|
list of {id |
+ value, score: value} for index search, list of dict for an index + database search |
+
txtai/api/base.py
def search(self, query, limit=None, weights=None, index=None, request=None):
+ # When search is invoked via the API, limit is set from the request
+ # When search is invoked directly, limit is set using the method parameter
+ limit = self.limit(request.query_params.get("limit") if request and hasattr(request, "query_params") else limit)
+ weights = self.weights(request.query_params.get("weights") if request and hasattr(request, "query_params") else weights)
+ index = request.query_params.get("index") if request and hasattr(request, "query_params") else index
+
+ if self.cluster:
+ return self.cluster.search(query, limit, weights, index)
+
+ return super().search(query, limit, weights, index)
+
upsert(self)
+
+
+Runs an embeddings upsert operation for previously batched documents.
+ +txtai/api/base.py
def upsert(self):
+ """
+ Runs an embeddings upsert operation for previously batched documents.
+ """
+
+ if self.cluster:
+ self.cluster.upsert()
+ else:
+ super().upsert()
+
weights(self, weights)
+
+
+Parses the weights parameter from the request.
+ +Parameters:
+Name | +Type | +Description | +Default | +
---|---|---|---|
weights |
+ + | weights parameter |
+ required | +
Returns:
+Type | +Description | +
---|---|
+ | weights |
+
txtai/api/base.py
def weights(self, weights):
+ """
+ Parses the weights parameter from the request.
+
+ Args:
+ weights: weights parameter
+
+ Returns:
+ weights
+ """
+
+ return float(weights) if weights else weights
+
{"use strict";/*!
+ * escape-html
+ * Copyright(c) 2012-2013 TJ Holowaychuk
+ * Copyright(c) 2015 Andreas Lubbe
+ * Copyright(c) 2015 Tiancheng "Timothy" Gu
+ * MIT Licensed
+ */var Wa=/["'&<>]/;Vn.exports=Ua;function Ua(e){var t=""+e,r=Wa.exec(t);if(!r)return t;var o,n="",i=0,s=0;for(i=r.index;i