Skip to content

Commit

Permalink
Add presto_cpp/properties-session.rst
Browse files Browse the repository at this point in the history
  • Loading branch information
steveburnett committed Oct 17, 2024
1 parent 8c69bcb commit 44a124a
Show file tree
Hide file tree
Showing 4 changed files with 254 additions and 246 deletions.
1 change: 1 addition & 0 deletions presto-docs/src/main/sphinx/presto-cpp.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Note: Presto C++ is in active development. See :doc:`Limitations </presto_cpp/li
presto_cpp/features
presto_cpp/limitations
presto_cpp/properties
presto_cpp/properties-session

Overview
========
Expand Down
240 changes: 1 addition & 239 deletions presto-docs/src/main/sphinx/presto_cpp/features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -217,242 +217,4 @@ Users can enable collection of worker level metrics by setting the property:
* **Default value:** ``false``

When true, the default behavior is a no-op. There is a prior setup that must be done before enabling this flag. To enable
metrics collection in Prometheus Data Format refer `here <https://github.com/prestodb/presto/tree/master/presto-native-execution#build-prestissimo>`_.

Session Properties
------------------

The following are the native session properties for C++ based Presto.

``driver_cpu_time_slice_limit_ms``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``1000``

Native Execution only. Defines the maximum CPU time in milliseconds that a driver thread
is permitted to run before it must yield to other threads,facilitating fair CPU usage across
multiple threads.

A positive value enforces this limit, ensuring threads do not monopolize CPU resources.

Negative values are considered invalid and are treated as a request to use the system default setting,
which is ``1000`` ms in this case.

Note: Setting the property to ``0`` allows a thread to run indefinitely
without yielding, which is not recommended in a shared environment as it can lead to
resource contention.

``legacy_timestamp``
^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``true``

Native Execution only. Use legacy TIME and TIMESTAMP semantics.

``native_aggregation_spill_memory_threshold``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``0``

Native Execution only. Specifies the maximum memory in bytes
that a final aggregation operation can utilize before it starts spilling to disk.
If set to ``0``, there is no limit, allowing the aggregation to consume unlimited memory resources,
which may impact system performance.

``native_debug_validate_output_from_operators``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``false``

If set to ``true``, then during the execution of tasks, the output vectors of every operator are validated for consistency.
It can help identify issues where a malformed vector causes failures or crashes, facilitating the debugging of operator output issues.

Note: This is an expensive check and should only be used for debugging purposes.

``native_debug_disable_expression_with_peeling``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``false``

If set to ``true``, disables the optimization in expression evaluation to peel common dictionary layer from inputs.

This should only be used for debugging purposes.

``native_debug_disable_common_sub_expressions``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``false``

If set to ``true``, disables the optimization in expression evaluation to reuse cached results for common sub-expressions.

This should only be used for debugging purposes.

``native_debug_disable_expression_with_memoization``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``false``

If set to ``true``, disables the optimization in expression evaluation to reuse cached results between subsequent
input batches that are dictionary encoded and have the same alphabet(underlying flat vector).

This should only be used for debugging purposes.

``native_debug_disable_expression_with_lazy_inputs``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``false``

If set to ``true``, disables the optimization in expression evaluation to delay loading of lazy inputs unless required.

This should only be used for debugging purposes.

``native_selective_nimble_reader_enabled``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``false``

Temporary flag to control whether selective Nimble reader should be used in this
query or not.

``native_join_spill_enabled``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``true``

Native Execution only. Enable join spilling on native engine.

``native_join_spill_memory_threshold``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``0``

Native Execution only. Specifies the maximum memory, in bytes, that a hash join operation can use before starting to spill to disk.
A value of ``0`` indicates no limit, permitting the join operation to use unlimited memory resources, which might affect overall system performance.

``native_join_spiller_partition_bits``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``2``

Native Execution only. Specifies the number of bits (N)
used to calculate the spilling partition number for hash join and RowNumber operations.
The partition number is determined as ``2`` raised to the power of N, defining how data is partitioned during the spill process.

``native_max_spill_file_size``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``0``

Specifies the maximum allowed spill file size in bytes. If set to ``0``, there is no limit on the spill file size,
allowing spill files to grow as large as necessary based on available disk space.
Use ``native_max_spill_file_size`` to manage disk space usage during operations that require spilling to disk.

``native_max_spill_level``
^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``4``

Native Execution only. The maximum allowed spilling level for hash join build.
``0`` is the initial spilling level, ``-1`` means unlimited.

``native_order_by_spill_memory_threshold``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``0``

Native Execution only. Specifies the maximum memory, in bytes, that the `ORDER BY` operation can utilize before starting to spill data to disk.
If set to ``0``, there is no limit on memory usage, potentially leading to large memory allocations for sorting operations.
Use this threshold to manage memory usage more efficiently during `ORDER BY` operations.

``native_row_number_spill_enabled``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``true``

Native Execution only. Enable row number spilling on native engine.

``native_simplified_expression_evaluation_enabled``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``false``

Native Execution only. Enable simplified path in expression evaluation.

``native_expression.max_array_size_in_reduce``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``integer``
* **Default value:** ``100000``

Native Execution only. The `reduce <https://prestodb.io/docs/current/functions/array.html#reduce-array-T-initialState-S-inputFunction-S-T-S-outputFunction-S-R-R>`_ function will throw an error if it encounters an array of size greater than this value.

``native_spill_compression_codec``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``varchar``
* **Default value:** ``none``

Native Execution only. Specifies the compression CODEC used to compress spilled data.
Supported compression CODECs are: ZLIB, SNAPPY, LZO, ZSTD, LZ4, and GZIP.
Setting this property to ``none`` disables compression.

``native_spill_file_create_config``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``varchar``
* **Default value:** ``""``

Native Execution only. Specifies the configuration parameters used to create spill files.
These parameters are provided to the underlying file system, allowing for customizable spill file creation based on the requirements of the environment.
The format and options of these parameters are determined by the capabilities of the underlying file system
and may include settings such as file location, size limits, and file system-specific optimizations.

``native_spill_write_buffer_size``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``bigint``
* **Default value:** ``1048576``

Native Execution only. The maximum size in bytes to buffer the serialized spill data before writing to disk for IO efficiency.
If set to ``0``, buffering is disabled.

``native_topn_row_number_spill_enabled``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``true``

Native Execution only. Enable topN row number spilling on native engine.

``native_window_spill_enabled``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``true``

Native Execution only. Enable window spilling on native engine.

``native_writer_spill_enabled``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* **Type:** ``boolean``
* **Default value:** ``true``

Native Execution only. Enable writer spilling on native engine.
metrics collection in Prometheus Data Format refer `here <https://github.com/prestodb/presto/tree/master/presto-native-execution#build-prestissimo>`_.
Loading

0 comments on commit 44a124a

Please sign in to comment.