Skip to content

Commit

Permalink
resource_control: update resource control about background example (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
nolouch authored Dec 6, 2023
1 parent 7d3813d commit f360c84
Show file tree
Hide file tree
Showing 3 changed files with 47 additions and 13 deletions.
1 change: 1 addition & 0 deletions releases/release-7.4.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.4/quick-start-with-
| [`tidb_cloud_storage_uri`](/system-variables.md#tidb_cloud_storage_uri-new-in-v740) | Newly added | Specifies the cloud storage URI to enable [Global Sort](/tidb-global-sort.md). |
| [`tidb_opt_enable_hash_join`](/system-variables.md#tidb_opt_enable_hash_join-new-in-v740) | Newly added | Controls whether the optimizer will select hash joins for tables. The value is `ON` by default. If set to `OFF`, the optimizer avoids selecting a hash join of a table unless there is no other execution plan available. |
| [`tidb_opt_objective`](/system-variables.md#tidb_opt_objective-new-in-v740) | Newly added | This variable controls the objective of the optimizer. `moderate` maintains the default behavior in versions prior to TiDB v7.4.0, where the optimizer tries to use more information to generate better execution plans. `determinate` mode tends to be more conservative and makes the execution plan more stable. |
| [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) | Newly added | Explicitly specifies the task type for the current session, which is identified and controlled by [Resource Control](/tidb-resource-control.md). For example: `SET @@tidb_request_source_type = "background"`. |
| [`tidb_schema_version_cache_limit`](/system-variables.md#tidb_schema_version_cache_limit-new-in-v740) | Newly added | This variable limits how many historical schema versions can be cached in a TiDB instance. The default value is `16`, which means that TiDB caches 16 historical schema versions by default. |
| [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Newly added | This variable is an instance-level system variable. You can use it to control the service scope of TiDB nodes under the [TiDB distributed execution framework](/tidb-distributed-execution-framework.md). When you set `tidb_service_scope` of a TiDB node to `background`, the TiDB distributed execution framework schedules that TiDB node to execute background tasks, such as [`ADD INDEX`](/sql-statements/sql-statement-add-index.md) and [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md). |
| [`tidb_session_alias`](/system-variables.md#tidb_session_alias-new-in-v740) | Newly added | Controls the value of the `session_alias` column in the logs related to the current session. |
Expand Down
9 changes: 9 additions & 0 deletions system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -4717,6 +4717,15 @@ SHOW WARNINGS;
- For uncommitted read-only transactions with modified data, the commit of these transactions is rejected.
- After the read-only mode is enabled, all users (including the users with the `SUPER` privilege) cannot execute the SQL statements that might write data unless the user is explicitly granted the `RESTRICTED_REPLICA_WRITER_ADMIN` privilege.
### tidb_request_source_type <span class="version-mark">New in v7.4.0</span>
- Scope: SESSION
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Type: String
- Default value: `""`
- Possible values: `"ddl"`, `"stats"`, `"br"`, `"lightning"`, `"background"`
- This variable is used to explicitly specify the task type for the current session, which is identified and controlled by [Resource Control](/tidb-resource-control.md). For example: `SET @@tidb_request_source_type = "background"`.
### tidb_retry_limit
- Scope: SESSION | GLOBAL
Expand Down
50 changes: 37 additions & 13 deletions tidb-resource-control.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ Request Unit (RU) is a unified abstraction unit in TiDB for system resources, wh
</tr>
<tr>
<td rowspan="3">Write</td>
<td>1 storage write batch consumes 1 RU for each replica</td>
<td>1 storage write batch consumes 1 RU</td>
</tr>
<tr>
<td>1 storage write request consumes 1 RU</td>
Expand All @@ -89,13 +89,11 @@ Request Unit (RU) is a unified abstraction unit in TiDB for system resources, wh
</tbody>
</table>

Currently, TiFlash resource control only considers SQL CPU, which is the CPU time consumed by the execution of pipeline tasks for queries, and read request payload.

> **Note:**
>
> - Each write operation is eventually replicated to all replicas (by default, TiKV has 3 replicas). Each replication operation is considered a different write operation.
> - In addition to queries executed by users, RU can be consumed by background tasks, such as automatic statistics collection.
> - The preceding table lists only the resources involved in RU calculation for TiDB Self-Hosted clusters, excluding the network and storage consumption. For TiDB Serverless RUs, see [TiDB Serverless Pricing Details](https://www.pingcap.com/tidb-cloud-serverless-pricing-details/).
> - Currently, TiFlash resource control only considers SQL CPU, which is the CPU time consumed by the execution of pipeline tasks for queries, and read request payload.
## Estimate RU consumption of SQL statements

Expand Down Expand Up @@ -422,6 +420,7 @@ TiDB supports the following types of background tasks:
- `br`: perform backup and restore tasks using [BR](/br/backup-and-restore-overview.md). PITR is not supported.
- `ddl`: control the resource usage during the batch data write back phase of Reorg DDLs.
- `stats`: the [collect statistics](/statistics.md#collect-statistics) tasks that are manually executed or automatically triggered by TiDB.
- `background`: a reserved task type. You can use the [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) system variable to specify the task type of the current session as `background`.
</CustomContent>
Expand All @@ -431,35 +430,60 @@ TiDB supports the following types of background tasks:
- `br`: perform backup and restore tasks using [BR](https://docs.pingcap.com/tidb/stable/backup-and-restore-overview). PITR is not supported.
- `ddl`: control the resource usage during the batch data write back phase of Reorg DDLs.
- `stats`: the [collect statistics](/statistics.md#collect-statistics) tasks that are manually executed or automatically triggered by TiDB.
- `background`: a reserved task type. You can use the [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) system variable to specify the task type of the current session as `background`.
</CustomContent>
By default, the task types that are marked as background tasks are empty, and the management of background tasks is disabled. This default behavior is the same as that of versions prior to TiDB v7.4.0. To manage background tasks, you need to manually modify the background task types of the `default` resource group.
By default, the task types that are marked as background tasks are `""`, and the management of background tasks is disabled. To enable background task management, you need to manually modify the background task type of the `default` resource group. After a background task is identified and matched, Resource Control is automatically performed. This means that when system resources are insufficient, the background tasks are automatically reduced to the lowest priority to ensure the execution of foreground tasks.
> **Note:**
>
> Currently, background tasks for all resource groups are bound to the `default` resource group. You can manage background task types globally through `default`. Binding background tasks to other resource groups is currently not supported.
#### Examples
1. Create the `rg1` resource group and set `br` and `stats` as background tasks.
1. Modify the `default` resource group and mark `br` and `ddl` as background tasks.
```sql
CREATE RESOURCE GROUP IF NOT EXISTS rg1 RU_PER_SEC = 500 BACKGROUND=(TASK_TYPES='br,stats');
ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES='br,ddl');
```
2. Change the `rg1` resource group to set `br` and `ddl` as background tasks.
2. Change the `default` resource group to revert the background task type to its default value.
```sql
ALTER RESOURCE GROUP rg1 BACKGROUND=(TASK_TYPES='br,ddl');
ALTER RESOURCE GROUP `default` BACKGROUND=NULL;
```
3. Restore the background tasks of `rg1` resource group to the default value. In this case, the background task types follow the configuration of the `default` resource group.
3. Change the `default` resource group to set the background task type to empty. In this case, all tasks of this resource group are not treated as background tasks.
```sql
ALTER RESOURCE GROUP rg1 BACKGROUND=NULL;
ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES="");
```
4. Change the `rg1` resource group to set the background task types to empty. In this case, all tasks of this resource group are not treated as background tasks.
4. View the background task type of the `default` resource group.
```sql
ALTER RESOURCE GROUP rg1 BACKGROUND=(TASK_TYPES="");
SELECT * FROM information_schema.resource_groups WHERE NAME="default";
```
The output is as follows:
```
+---------+------------+----------+-----------+-------------+---------------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND |
+---------+------------+----------+-----------+-------------+---------------------+
| default | UNLIMITED | MEDIUM | YES | NULL | TASK_TYPES='br,ddl' |
+---------+------------+----------+-----------+-------------+---------------------+
```
5. To explicitly mark tasks in the current session as the background type, you can use `tidb_request_source_type` to explicitly specify the task type. The following is an example:
``` sql
SET @@tidb_request_source_type="background";
/* Add background task type */
ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES="background");
/* Execute LOAD DATA in the current session */
LOAD DATA INFILE "s3://resource-control/Lightning/test.customer.aaaa.csv"
```
## Disable resource control
Expand Down

0 comments on commit f360c84

Please sign in to comment.