From f360c846947ee9b6b049f239a9f628507634601a Mon Sep 17 00:00:00 2001 From: ShuNing Date: Wed, 6 Dec 2023 16:11:51 +0800 Subject: [PATCH] resource_control: update resource control about background example (#15561) --- releases/release-7.4.0.md | 1 + system-variables.md | 9 +++++++ tidb-resource-control.md | 50 +++++++++++++++++++++++++++++---------- 3 files changed, 47 insertions(+), 13 deletions(-) diff --git a/releases/release-7.4.0.md b/releases/release-7.4.0.md index c27e18e365f86..9d2c9686473fd 100644 --- a/releases/release-7.4.0.md +++ b/releases/release-7.4.0.md @@ -296,6 +296,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.4/quick-start-with- | [`tidb_cloud_storage_uri`](/system-variables.md#tidb_cloud_storage_uri-new-in-v740) | Newly added | Specifies the cloud storage URI to enable [Global Sort](/tidb-global-sort.md). | | [`tidb_opt_enable_hash_join`](/system-variables.md#tidb_opt_enable_hash_join-new-in-v740) | Newly added | Controls whether the optimizer will select hash joins for tables. The value is `ON` by default. If set to `OFF`, the optimizer avoids selecting a hash join of a table unless there is no other execution plan available. | | [`tidb_opt_objective`](/system-variables.md#tidb_opt_objective-new-in-v740) | Newly added | This variable controls the objective of the optimizer. `moderate` maintains the default behavior in versions prior to TiDB v7.4.0, where the optimizer tries to use more information to generate better execution plans. `determinate` mode tends to be more conservative and makes the execution plan more stable. | +| [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) | Newly added | Explicitly specifies the task type for the current session, which is identified and controlled by [Resource Control](/tidb-resource-control.md). For example: `SET @@tidb_request_source_type = "background"`. | | [`tidb_schema_version_cache_limit`](/system-variables.md#tidb_schema_version_cache_limit-new-in-v740) | Newly added | This variable limits how many historical schema versions can be cached in a TiDB instance. The default value is `16`, which means that TiDB caches 16 historical schema versions by default. | | [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Newly added | This variable is an instance-level system variable. You can use it to control the service scope of TiDB nodes under the [TiDB distributed execution framework](/tidb-distributed-execution-framework.md). When you set `tidb_service_scope` of a TiDB node to `background`, the TiDB distributed execution framework schedules that TiDB node to execute background tasks, such as [`ADD INDEX`](/sql-statements/sql-statement-add-index.md) and [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md). | | [`tidb_session_alias`](/system-variables.md#tidb_session_alias-new-in-v740) | Newly added | Controls the value of the `session_alias` column in the logs related to the current session. | diff --git a/system-variables.md b/system-variables.md index d28900c8140a2..4f3034d765efb 100644 --- a/system-variables.md +++ b/system-variables.md @@ -4717,6 +4717,15 @@ SHOW WARNINGS; - For uncommitted read-only transactions with modified data, the commit of these transactions is rejected. - After the read-only mode is enabled, all users (including the users with the `SUPER` privilege) cannot execute the SQL statements that might write data unless the user is explicitly granted the `RESTRICTED_REPLICA_WRITER_ADMIN` privilege. +### tidb_request_source_type New in v7.4.0 + +- Scope: SESSION +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: String +- Default value: `""` +- Possible values: `"ddl"`, `"stats"`, `"br"`, `"lightning"`, `"background"` +- This variable is used to explicitly specify the task type for the current session, which is identified and controlled by [Resource Control](/tidb-resource-control.md). For example: `SET @@tidb_request_source_type = "background"`. + ### tidb_retry_limit - Scope: SESSION | GLOBAL diff --git a/tidb-resource-control.md b/tidb-resource-control.md index d6f6358be9d2e..7662d1871829f 100644 --- a/tidb-resource-control.md +++ b/tidb-resource-control.md @@ -74,7 +74,7 @@ Request Unit (RU) is a unified abstraction unit in TiDB for system resources, wh Write - 1 storage write batch consumes 1 RU for each replica + 1 storage write batch consumes 1 RU 1 storage write request consumes 1 RU @@ -89,13 +89,11 @@ Request Unit (RU) is a unified abstraction unit in TiDB for system resources, wh -Currently, TiFlash resource control only considers SQL CPU, which is the CPU time consumed by the execution of pipeline tasks for queries, and read request payload. - > **Note:** > > - Each write operation is eventually replicated to all replicas (by default, TiKV has 3 replicas). Each replication operation is considered a different write operation. -> - In addition to queries executed by users, RU can be consumed by background tasks, such as automatic statistics collection. > - The preceding table lists only the resources involved in RU calculation for TiDB Self-Hosted clusters, excluding the network and storage consumption. For TiDB Serverless RUs, see [TiDB Serverless Pricing Details](https://www.pingcap.com/tidb-cloud-serverless-pricing-details/). +> - Currently, TiFlash resource control only considers SQL CPU, which is the CPU time consumed by the execution of pipeline tasks for queries, and read request payload. ## Estimate RU consumption of SQL statements @@ -422,6 +420,7 @@ TiDB supports the following types of background tasks: - `br`: perform backup and restore tasks using [BR](/br/backup-and-restore-overview.md). PITR is not supported. - `ddl`: control the resource usage during the batch data write back phase of Reorg DDLs. - `stats`: the [collect statistics](/statistics.md#collect-statistics) tasks that are manually executed or automatically triggered by TiDB. +- `background`: a reserved task type. You can use the [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) system variable to specify the task type of the current session as `background`. @@ -431,35 +430,60 @@ TiDB supports the following types of background tasks: - `br`: perform backup and restore tasks using [BR](https://docs.pingcap.com/tidb/stable/backup-and-restore-overview). PITR is not supported. - `ddl`: control the resource usage during the batch data write back phase of Reorg DDLs. - `stats`: the [collect statistics](/statistics.md#collect-statistics) tasks that are manually executed or automatically triggered by TiDB. +- `background`: a reserved task type. You can use the [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) system variable to specify the task type of the current session as `background`. -By default, the task types that are marked as background tasks are empty, and the management of background tasks is disabled. This default behavior is the same as that of versions prior to TiDB v7.4.0. To manage background tasks, you need to manually modify the background task types of the `default` resource group. +By default, the task types that are marked as background tasks are `""`, and the management of background tasks is disabled. To enable background task management, you need to manually modify the background task type of the `default` resource group. After a background task is identified and matched, Resource Control is automatically performed. This means that when system resources are insufficient, the background tasks are automatically reduced to the lowest priority to ensure the execution of foreground tasks. + +> **Note:** +> +> Currently, background tasks for all resource groups are bound to the `default` resource group. You can manage background task types globally through `default`. Binding background tasks to other resource groups is currently not supported. #### Examples -1. Create the `rg1` resource group and set `br` and `stats` as background tasks. +1. Modify the `default` resource group and mark `br` and `ddl` as background tasks. ```sql - CREATE RESOURCE GROUP IF NOT EXISTS rg1 RU_PER_SEC = 500 BACKGROUND=(TASK_TYPES='br,stats'); + ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES='br,ddl'); ``` -2. Change the `rg1` resource group to set `br` and `ddl` as background tasks. +2. Change the `default` resource group to revert the background task type to its default value. ```sql - ALTER RESOURCE GROUP rg1 BACKGROUND=(TASK_TYPES='br,ddl'); + ALTER RESOURCE GROUP `default` BACKGROUND=NULL; ``` -3. Restore the background tasks of `rg1` resource group to the default value. In this case, the background task types follow the configuration of the `default` resource group. +3. Change the `default` resource group to set the background task type to empty. In this case, all tasks of this resource group are not treated as background tasks. ```sql - ALTER RESOURCE GROUP rg1 BACKGROUND=NULL; + ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES=""); ``` -4. Change the `rg1` resource group to set the background task types to empty. In this case, all tasks of this resource group are not treated as background tasks. +4. View the background task type of the `default` resource group. ```sql - ALTER RESOURCE GROUP rg1 BACKGROUND=(TASK_TYPES=""); + SELECT * FROM information_schema.resource_groups WHERE NAME="default"; + ``` + + The output is as follows: + + ``` + +---------+------------+----------+-----------+-------------+---------------------+ + | NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND | + +---------+------------+----------+-----------+-------------+---------------------+ + | default | UNLIMITED | MEDIUM | YES | NULL | TASK_TYPES='br,ddl' | + +---------+------------+----------+-----------+-------------+---------------------+ + ``` + +5. To explicitly mark tasks in the current session as the background type, you can use `tidb_request_source_type` to explicitly specify the task type. The following is an example: + + ``` sql + SET @@tidb_request_source_type="background"; + /* Add background task type */ + ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES="background"); + /* Execute LOAD DATA in the current session */ + LOAD DATA INFILE "s3://resource-control/Lightning/test.customer.aaaa.csv" ``` ## Disable resource control