Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TiDB can't close connection immediately after killing client process #57531

Open
pcqz opened this issue Nov 20, 2024 · 8 comments
Open

TiDB can't close connection immediately after killing client process #57531

pcqz opened this issue Nov 20, 2024 · 8 comments
Assignees
Labels
affects-8.5 This bug affects the 8.5.x(LTS) versions. report/customer Customers have encountered this bug. severity/minor sig/sql-infra SIG: SQL Infra type/bug The issue is confirmed as a bug.

Comments

@pcqz
Copy link

pcqz commented Nov 20, 2024

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

Run a tidb instance locally, and execute select sleep(300); in a connection.
Then kill this client process ps aux | grep -i mysql | grep -v grep | awk '{print $2}' | xargs kill

2. What did you expect to see? (Required)

The connection is closed immediately like mysql.

3. What did you see instead (Required)

The connection is still in processlist and closed after 300s of execution.

4. What is your TiDB version? (Required)

v8.2.0

@pcqz pcqz added the type/bug The issue is confirmed as a bug. label Nov 20, 2024
@pcqz
Copy link
Author

pcqz commented Nov 20, 2024

/report customer

@yibin87
Copy link
Contributor

yibin87 commented Nov 25, 2024

The sleep command can be killed by "kill" command. It seems the killing of client doesn't send kill signal to the target sleep process. I think this should be a sql infra issue.

@yibin87
Copy link
Contributor

yibin87 commented Nov 25, 2024

/sig infra

Copy link

ti-chi-bot bot commented Nov 25, 2024

@yibin87: The label(s) sig/infra cannot be applied, because the repository doesn't have them.

In response to this:

/sig infra

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@yibin87
Copy link
Contributor

yibin87 commented Nov 25, 2024

/sig sql-infra

@ti-chi-bot ti-chi-bot bot added the sig/sql-infra SIG: SQL Infra label Nov 25, 2024
@yibin87
Copy link
Contributor

yibin87 commented Nov 25, 2024

/remove-sig execution

@Defined2014
Copy link
Contributor

Defined2014 commented Nov 28, 2024

The biggest different is MySQL could sense the tcp connect closed which TiDB can't.

MySQL, it replied a response after the kill processed:
Image

If we want do it, we can monitor the fd of conn in the background and terminate SQL when a read signal is received and EOF is read.

A very hack way to acheive it

diff --git a/pkg/server/conn.go b/pkg/server/conn.go
index e201ff3cb4..05c6e5085a 100644
--- a/pkg/server/conn.go
+++ b/pkg/server/conn.go
@@ -1375,6 +1375,14 @@ func (cc *clientConn) dispatch(ctx context.Context, data []byte) error {
                        data = data[:len(data)-1]
                        dataStr = string(hack.String(data))
                }
+               go func(cc *clientConn) {
+                       b := make([]byte, 100)
+                       if idx := strings.Index(dataStr, "sleep"); idx >= 0 {
+                               if _, err := cc.bufReadConn.Conn.Read(b); err != nil {
+                                       killQuery(cc, false, false)
+                               }
+                       }
+               }(cc)
                return cc.handleQuery(ctx, dataStr)
        case mysql.ComFieldList:
                return cc.handleFieldList(ctx, dataStr)

@ti-chi-bot ti-chi-bot bot added the affects-8.5 This bug affects the 8.5.x(LTS) versions. label Dec 2, 2024
@Defined2014 Defined2014 added severity/minor and removed severity/moderate affects-8.5 This bug affects the 8.5.x(LTS) versions. labels Dec 9, 2024
@ti-chi-bot ti-chi-bot bot added the affects-8.5 This bug affects the 8.5.x(LTS) versions. label Dec 19, 2024
@dveeden
Copy link
Contributor

dveeden commented Dec 20, 2024

This is what go-sql-driver/mysql uses: https://github.com/go-sql-driver/mysql/blob/v1.8.1/conncheck.go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-8.5 This bug affects the 8.5.x(LTS) versions. report/customer Customers have encountered this bug. severity/minor sig/sql-infra SIG: SQL Infra type/bug The issue is confirmed as a bug.
Projects
None yet
Development

No branches or pull requests

5 participants