Skip to content

Commit

Permalink
[native] Add Prestissimo documentation page to Developer Guide
Browse files Browse the repository at this point in the history
Adding a new documentation page for Prestissimo to the Developer Guide,
and adding some of the configuration parameters for remote function
execution.
  • Loading branch information
pedroerp committed Aug 15, 2023
1 parent 6f332a3 commit 1753d68
Showing 1 changed file with 95 additions and 0 deletions.
95 changes: 95 additions & 0 deletions presto-docs/src/main/sphinx/develop/presto-native.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
===========================
Presto Native - Prestissimo
===========================

Prestissimo is a C++ drop-in replacement for Presto workers based on the Velox
library. It implements the same RESTful endpoints as Java workers using the
Proxygen C++ HTTP framework. Since communication with the Java coordinator and
across workers is only done using the REST endpoints, Prestissimo does not use
JNI and it is commonly deployed in such a way as to avoid having a JVM process
on worker nodes.

Prestissimo's codebase is located at `presto-native-execution
<https://github.com/prestodb/presto/tree/master/presto-native-execution>`_.


Endpoints
---------

HTTP endpoints related to tasks are registered to Proxygen in
`TaskResource.cpp`. Important endpoints implemented include:

* POST: v1/task: This processes a `TaskUpdateRequest`
* GET: v1/task: This returns a serialized `TaskInfo` (used for comprehensive
metrics, may be reported less frequently)
* GET: v1/task/status: This returns
a serialized `TaskStatus` (used for query progress tracking, must be reported
frequently)

Other HTTP endpoints include:

* POST: v1/memory
* Reports memory, but no assignments are adjusted unlike in Java workers.
* GET: v1/info
* GET: v1/status

The request/response flow of Prestissimo is identical to Java workers. The
tasks or new splits are registered via `TaskUpdateRequest`. Resource
utilization and query progress are sent to the coordinator via task endpoints.


Remote Function Execution
-------------------------

Prestissimo supports remote execution of scalar functions. This feature is
useful for cases when the function code is not written in C++, or if for
security or flexibility reasons the function code cannot be linked to the same
executable as the main engine.

Remote function signatures need to be provided using a JSON file, following
the format implemented by `JsonFileBasedFunctionNamespaceManager
<https://github.com/prestodb/presto/blob/master/presto-function-namespace-managers/src/main/java/com/facebook/presto/functionNamespace/json/JsonFileBasedFunctionNamespaceManager.java>`_.
The following properties allow the configuration of remote function execution:

``remote-function-server.signature.files.directory.path``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``string``
* **Default value:** ``""``

The local filesystem path where JSON files containing remote function
signatures are located. If not empty, the Presto native worker will
recursively search, open, parse, and register function definitions from
these JSON files.


``remote-function-server.thrift.address``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``string``
* **Default value:** ``""``

The location (ip address or hostname) that hosts the remote function
server, if any remote functions were registered using
``remote-function-server.signature.files.directory.path``.
If not specified, falls back to the loopback interface (``::1``)

``remote-function-server.thrift.port``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``0``

The port that hosts the remote function server. If not specified and remote
functions are trying to be registered, an exception is thrown.

``remote-function-server.thrift.uds-path``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``string``
* **Default value:** ``""``

The UDS (unix domain socket) path to communicate with a local remote
function server. If specified, takes precedence over
``remote-function-server.thrift.address`` and
``remote-function-server.thrift.port``.

0 comments on commit 1753d68

Please sign in to comment.