Skip to content

Commit bcab97e

Browse files
authored
dm: update doc for error handling (pingcap#7894)
1 parent 330da0b commit bcab97e

10 files changed

+357
-199
lines changed

dm/dm-faq.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,7 @@ if the DDL is not needed, you can use a filter rule with \"*\" schema-pattern to
188188

189189
The reason for this type of error is that the TiDB parser cannot parse DDL statements sent by the upstream, such as `ALTER EVENT`, so `sql-skip` does not take effect as expected. You can add [binlog event filters](/dm/dm-key-features.md#binlog-event-filter) in the configuration file to filter those statements and set `schema-pattern: "*"`. Starting from DM v2.0.1, DM pre-filters statements related to `EVENT`.
190190

191-
Since DM v2.0, `handle-error` replaces `sql-skip`. You can use `handle-error` instead to avoid this issue.
191+
Since DM v6.0, `binlog` replaces `sql-skip` and `handle-error`. You can use the `binlog` command instead to avoid this issue.
192192

193193
## Why do `REPLACE` statements keep appearing in the downstream when DM is replicating?
194194

@@ -368,7 +368,7 @@ In this case, you can check the environment variable `https_proxy` (note that it
368368
To solve this issue, check whether `https_proxy` is mandatory. If not, cancel the setting. Otherwise, add the environment variable setting `https_proxy="" ./dmctl --master-addr "x.x.x.x:8261"` before the oringial dmctl commands.
369369

370370
> **Note:**
371-
>
371+
>
372372
> The environment variables related to `proxy` include `http_proxy`, `https_proxy`, and `no_proxy`. If the connection error persists after you perform the above steps, check whether the configuration parameters of `http_proxy` and `no_proxy` are correct.
373373
374374
## How to handle the returned error when executing start-relay command for DM versions from 2.0.2 to 2.0.6?

dm/dm-manage-schema.md

+88-37
Original file line numberDiff line numberDiff line change
@@ -30,76 +30,83 @@ However, during the migration with [optimistic mode sharding DDL support](/dm/fe
3030

3131
In addition, in some scenarios (such as when the downstream table has more columns than the upstream table), `schema-D` might be inconsistent with `schema-B` and `schema-I`.
3232

33-
To support the scenarios mentioned above and handle other migration interruptions caused by schema inconsistency, DM provides the `operate-schema` command to obtain, modify, and delete the `schema-I` table schema maintained in DM.
33+
To support the scenarios mentioned above and handle other migration interruptions caused by schema inconsistency, DM provides the `binlog-schema` command to obtain, modify, and delete the `schema-I` table schema maintained in DM.
34+
35+
> **Note:**
36+
>
37+
> The `binlog-schema` command is supported only in DM v6.0 or later versions. For earlier versions, you must use the `operate-schema` command.
3438
3539
## Command
3640

3741
{{< copyable "shell-regular" >}}
3842

3943
```bash
40-
help operate-schema
44+
help binlog-schema
4145
```
4246

4347
```
44-
`get`/`set`/`remove` the schema for an upstream table.
48+
manage or show table schema in schema tracker
4549
4650
Usage:
47-
dmctl operate-schema <operate-type> <-s source ...> <task-name | task-file> <-d database> <-t table> [schema-file] [--flush] [--sync] [flags]
51+
dmctl binlog-schema [command]
52+
53+
Available Commands:
54+
delete delete table schema structure
55+
list show table schema structure
56+
update update tables schema structure
4857
4958
Flags:
50-
-d, --database string database name of the table
51-
--flush flush the table info and checkpoint immediately
52-
-h, --help help for operate-schema
53-
--sync sync the table info to master to resolve shard ddl lock, only for optimistic mode now
54-
-t, --table string table name
59+
-h, --help help for binlog-schema
5560
5661
Global Flags:
5762
-s, --source strings MySQL Source ID.
63+
64+
Use "dmctl binlog-schema [command] --help" for more information about a command.
5865
```
5966

6067
> **Note:**
6168
>
62-
> - Because a table schema might change during data migration, to obtain a predictable table schema, currently the `operate-schema` command can be used only when the data migration task is in the `Paused` state.
69+
> - Because a table schema might change during data migration, to obtain a predictable table schema, currently the `binlog-schema` command can be used only when the data migration task is in the `Paused` state.
6370
> - To avoid data loss due to mishandling, it is **strongly recommended** to get and backup the table schema firstly before you modify the schema.
6471
6572
## Parameters
6673

67-
* `operate-type`:
68-
- Required.
69-
- Specifies the type of operation on the schema. The optional values are `get`, `set`, and `remove`.
70-
* `-s`:
74+
* `delete`: Deletes the table schema.
75+
* `list`: Lists the table schema.
76+
* `update`: Updates the table schema.
77+
* `-s` or `--source`:
7178
- Required.
7279
- Specifies the MySQL source that the operation is applied to.
73-
* `task-name | task-file`:
74-
- Required.
75-
- Specifies the task name or task file path.
76-
* `-d`:
77-
- Required.
78-
- Specifies the name of the upstream database the table belongs to.
79-
* `-t`:
80-
- Required.
81-
- Specifies the name of the upstream table corresponding to the table.
82-
* `schema-file`:
83-
- Required when the operation type is `set`. Optional for other operation types.
84-
- The table schema file to be set. The file content should be a valid `CREATE TABLE` statement.
85-
* `--flush`:
86-
- Optional.
87-
- Writes the schema to the checkpoint so that DM can load it after restarting the task.
88-
- The default value is `true`.
89-
* `--sync`:
90-
- Optional. Only used when an error occurs in the optimistic sharding DDL mode.
91-
- Updates the optimistic sharding metadata with this schema.
9280

9381
## Usage example
9482

9583
### Get the table schema
9684

85+
To get the table schema, run the `binlog-schema list` command:
86+
87+
```bash
88+
help binlog-schema list
89+
```
90+
91+
```
92+
show table schema structure
93+
94+
Usage:
95+
dmctl binlog-schema list <task-name> <database> <table> [flags]
96+
97+
Flags:
98+
-h, --help help for list
99+
100+
Global Flags:
101+
-s, --source strings MySQL Source ID.
102+
```
103+
97104
If you want to get the table schema of the ``` `db_single`.`t1` ``` table corresponding to the `mysql-replica-01` MySQL source in the `db_single` task, run the following command:
98105

99106
{{< copyable "shell-regular" >}}
100107

101108
```bash
102-
operate-schema get -s mysql-replica-01 task_single -d db_single -t t1
109+
binlog-schema list -s mysql-replica-01 task_single db_single t1
103110
```
104111

105112
```
@@ -117,7 +124,32 @@ operate-schema get -s mysql-replica-01 task_single -d db_single -t t1
117124
}
118125
```
119126

120-
### Set the table schema
127+
### Update the table schema
128+
129+
To update the table schema, run the `binlog-schema update` command:
130+
131+
{{< copyable "shell-regular" >}}
132+
133+
```bash
134+
help binlog-schema update
135+
```
136+
137+
```
138+
update tables schema structure
139+
140+
Usage:
141+
dmctl binlog-schema update <task-name> <database> <table> [schema-file] [flags]
142+
143+
Flags:
144+
--flush flush the table info and checkpoint immediately (default true)
145+
--from-source use the schema from upstream database as the schema of the specified tables
146+
--from-target use the schema from downstream database as the schema of the specified tables
147+
-h, --help help for update
148+
--sync sync the table info to master to resolve shard ddl lock, only for optimistic mode now (default true)
149+
150+
Global Flags:
151+
-s, --source strings MySQL Source ID.
152+
```
121153

122154
If you want to set the table schema of the ``` `db_single`.`t1` ``` table corresponding to the `mysql-replica-01` MySQL source in the `db_single` task as follows:
123155

@@ -152,7 +184,26 @@ operate-schema set -s mysql-replica-01 task_single -d db_single -t t1 db_single.
152184
}
153185
```
154186

155-
### Delete table schema
187+
### Delete the table schema
188+
189+
To delete the table schema, run the `binlog-schema delete` command:
190+
191+
```bash
192+
help binlog-schema delete
193+
```
194+
195+
```
196+
delete table schema structure
197+
198+
Usage:
199+
dmctl binlog-schema delete <task-name> <database> <table> [flags]
200+
201+
Flags:
202+
-h, --help help for delete
203+
204+
Global Flags:
205+
-s, --source strings MySQL Source ID.
206+
```
156207

157208
> **Note:**
158209
>
@@ -167,7 +218,7 @@ If you want to delete the table schema of the ``` `db_single`.`t1` ``` table cor
167218
{{< copyable "shell-regular" >}}
168219

169220
```bash
170-
operate-schema remove -s mysql-replica-01 task_single -d db_single -t t1
221+
binlog-schema delete -s mysql-replica-01 task_single db_single t1
171222
```
172223

173224
```

dm/dm-manage-source.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -94,14 +94,14 @@ The following is an example of the returned result:
9494

9595
> **Note:**
9696
>
97-
> The `get-config` command is only supported in DM v2.0.1 and later versions.
97+
> The `config` command is only supported in DM v6.0 and later versions. For earlier versions, you must use the `get-config` command.
9898
99-
If you know the `source-id`, you can run `dmctl --master-addr <master-addr> get-config source <source-id>` to get the data source configuration.
99+
If you know the `source-id`, you can run `dmctl --master-addr <master-addr> config source <source-id>` to get the data source configuration.
100100

101101
{{< copyable "" >}}
102102

103103
```bash
104-
get-config source mysql-replica-01
104+
config source mysql-replica-01
105105
```
106106

107107
```

dm/dmctl-introduction.md

+59-52
Original file line numberDiff line numberDiff line change
@@ -41,26 +41,29 @@ Usage:
4141
dmctl [command]
4242
4343
Available Commands:
44-
check-task Checks the configuration file of the task.
45-
config Commands to import/export config.
46-
get-config Gets the configuration.
47-
handle-error `skip`/`replace`/`revert` the current error event or a specific binlog position (binlog-pos) event.
48-
help Gets help about any command.
49-
list-member Lists member information.
50-
offline-member Offlines member which has been closed.
51-
operate-leader `evict`/`cancel-evict` the leader.
52-
operate-schema `get`/`set`/`remove` the schema for an upstream table.
53-
operate-source `create`/`update`/`stop`/`show` upstream MySQL/MariaDB source.
54-
pause-relay Pauses DM-worker's relay unit.
55-
pause-task Pauses a specified running task.
56-
purge-relay Purges relay log files of the DM-worker according to the specified filename.
57-
query-status Queries task status.
58-
resume-relay Resumes DM-worker's relay unit.
59-
resume-task Resumes a specified paused task.
60-
show-ddl-locks Shows un-resolved DDL locks.
61-
start-task Starts a task as defined in the configuration file.
62-
stop-task Stops a specified task.
63-
unlock-ddl-lock Unlocks DDL lock forcefully.
44+
binlog manage or show binlog operations
45+
binlog-schema manage or show table schema in schema tracker
46+
check-task Checks the configuration file of the task
47+
config manage config operations
48+
decrypt Decrypts cipher text to plain text
49+
encrypt Encrypts plain text to cipher text
50+
help Gets help about any command
51+
list-member Lists member information
52+
offline-member Offlines member which has been closed
53+
operate-leader `evict`/`cancel-evict` the leader
54+
operate-source `create`/`update`/`stop`/`show` upstream MySQL/MariaDB source
55+
pause-relay Pauses DM-worker's relay unit
56+
pause-task Pauses a specified running task or all (sub)tasks bound to a source
57+
purge-relay Purges relay log files of the DM-worker according to the specified filename
58+
query-status Queries task status
59+
resume-relay Resumes DM-worker's relay unit
60+
resume-task Resumes a specified paused task or all (sub)tasks bound to a source
61+
shard-ddl-lock maintain or show shard-ddl locks information
62+
start-relay Starts workers pulling relay log for a source
63+
start-task Starts a task as defined in the configuration file
64+
stop-relay Stops workers pulling relay log for a source
65+
stop-task Stops a specified task or all (sub)tasks bound to a source
66+
transfer-source Transfers a upstream MySQL/MariaDB source to a free worker
6467
6568
Flags:
6669
-h, --help Help for dmctl.
@@ -91,36 +94,40 @@ export DM_MASTER_ADDR="172.16.30.14:8261"
9194

9295
```
9396
Available Commands:
94-
check-task check-task <config-file> [--error count] [--warn count]
95-
config commands to import/export config
96-
get-config get-config <task | master | worker | source> <name> [--file filename]
97-
handle-error handle-error <task-name | task-file> [-s source ...] [-b binlog-pos] <skip/replace/revert> [replace-sql1;replace-sql2;]
98-
list-member list-member [--leader] [--master] [--worker] [--name master-name/worker-name ...]
99-
offline-member offline-member <--master/--worker> <--name master-name/worker-name>
100-
operate-leader operate-leader <operate-type>
101-
operate-schema operate-schema <operate-type> <-s source ...> <task-name | task-file> <-d database> <-t table> [schema-file]
102-
operate-source operate-source <operate-type> [config-file ...] [--print-sample-config]
103-
pause-relay pause-relay <-s source ...>
104-
pause-task pause-task [-s source ...] <task-name | task-file>
105-
purge-relay purge-relay <-s source> <-f filename> [--sub-dir directory]
106-
query-status query-status [-s source ...] [task-name | task-file] [--more]
107-
resume-relay resume-relay <-s source ...>
108-
resume-task resume-task [-s source ...] <task-name | task-file>
109-
show-ddl-locks show-ddl-locks [-s source ...] [task-name | task-file]
110-
start-task start-task [-s source ...] [--remove-meta] <config-file>
111-
stop-task stop-task [-s source ...] <task-name | task-file>
112-
unlock-ddl-lock unlock-ddl-lock <lock-ID>
113-
114-
Special Commands:
115-
--encrypt Encrypts plaintext to ciphertext.
116-
--decrypt Decrypts ciphertext to plaintext.
117-
118-
Global Options:
119-
--V Prints version and exit.
120-
--config Path to configuration file.
121-
--master-addr Master API server addr.
122-
--rpc-timeout RPC timeout, default is 10m.
123-
--ssl-ca Path of file that contains list of trusted SSL CAs for connection.
124-
--ssl-cert Path of file that contains X509 certificate in PEM format for connection.
125-
--ssl-key Path of file that contains X509 key in PEM format for connection.
97+
binlog manage or show binlog operations
98+
binlog-schema manage or show table schema in schema tracker
99+
check-task Checks the configuration file of the task
100+
config manage config operations
101+
decrypt Decrypts cipher text to plain text
102+
encrypt Encrypts plain text to cipher text
103+
help Gets help about any command
104+
list-member Lists member information
105+
offline-member Offlines member which has been closed
106+
operate-leader `evict`/`cancel-evict` the leader
107+
operate-source `create`/`update`/`stop`/`show` upstream MySQL/MariaDB source
108+
pause-relay Pauses DM-worker's relay unit
109+
pause-task Pauses a specified running task or all (sub)tasks bound to a source
110+
purge-relay Purges relay log files of the DM-worker according to the specified filename
111+
query-status Queries task status
112+
resume-relay Resumes DM-worker's relay unit
113+
resume-task Resumes a specified paused task or all (sub)tasks bound to a source
114+
shard-ddl-lock maintain or show shard-ddl locks information
115+
start-relay Starts workers pulling relay log for a source
116+
start-task Starts a task as defined in the configuration file
117+
stop-relay Stops workers pulling relay log for a source
118+
stop-task Stops a specified task or all (sub)tasks bound to a source
119+
transfer-source Transfers a upstream MySQL/MariaDB source to a free worker
120+
121+
Flags:
122+
--config string Path to config file.
123+
-h, --help help for dmctl
124+
--master-addr string Master API server address, this parameter is required when interacting with the dm-master
125+
--rpc-timeout string RPC timeout, default is 10m. (default "10m")
126+
-s, --source strings MySQL Source ID.
127+
--ssl-ca string Path of file that contains list of trusted SSL CAs for connection.
128+
--ssl-cert string Path of file that contains X509 certificate in PEM format for connection.
129+
--ssl-key string Path of file that contains X509 key in PEM format for connection.
130+
-V, --version Prints version and exit.
131+
132+
Use "dmctl [command] --help" for more information about a command.
126133
```

0 commit comments

Comments
 (0)