Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] mysql cluster 8.0.33/5.7.42 create failed #5595

Closed
JashBook opened this issue Oct 23, 2023 · 0 comments · Fixed by #5631
Closed

[BUG] mysql cluster 8.0.33/5.7.42 create failed #5595

JashBook opened this issue Oct 23, 2023 · 0 comments · Fixed by #5631
Assignees
@xuriwuyun
Labels
bug kind/bug Something isn't working severity/major Great chance user will encounter the same problem
Milestone
Release 0.7.0

Comments

@JashBook
Copy link
Collaborator

Describe the bug
mysql cluster 8.0.33/5.7.42 create failed.

kbcli version
Kubernetes: v1.26.7-gke.500
KubeBlocks: 0.7.0-beta.9
kbcli: 0.7.0-beta.9

To Reproduce
Steps to reproduce the behavior:

  1. create mysql cluster
 kbcli cluster create  asmysql-8-0 --cluster-definition mysql --termination-policy=WipeOut --cluster-version=mysql-8.0.33 --set replicas=2

kbcli cluster create  asmysql-5-7 --cluster-definition mysql --termination-policy=WipeOut --cluster-version=mysql-5.7.42 --set replicas=2
  1. See error
kubectl get cluster
NAME          CLUSTER-DEFINITION   VERSION        TERMINATION-POLICY   STATUS     AGE
asmysql-5-7   mysql                mysql-5.7.42   WipeOut              Updating   8m17s
asmysql-8-0   mysql                mysql-8.0.33   WipeOut              Updating   2m10s

describe cluster


➜  ~ kubectl describe cluster asmysql-5-7
Name:         asmysql-5-7
Namespace:    default
Labels:       clusterdefinition.kubeblocks.io/name=mysql
              clusterversion.kubeblocks.io/name=mysql-5.7.42
Annotations:  <none>
API Version:  apps.kubeblocks.io/v1alpha1
Kind:         Cluster
Metadata:
  Creation Timestamp:  2023-10-23T11:42:38Z
  Finalizers:
    cluster.kubeblocks.io/finalizer
  Generation:        1
  Resource Version:  13495898
  UID:               4611e6e1-2065-4d3d-be86-9ccf98a181c7
Spec:
  Affinity:
    Pod Anti Affinity:     Preferred
    Tenancy:               SharedNode
  Cluster Definition Ref:  mysql
  Cluster Version Ref:     mysql-5.7.42
  Component Specs:
    Component Def Ref:  mysql
    Monitor:            false
    Name:               mysql
    No Create PDB:      false
    Replicas:           2
    Resources:
      Limits:
        Cpu:     1
        Memory:  1Gi
      Requests:
        Cpu:               1
        Memory:            1Gi
    Service Account Name:  kb-asmysql-5-7
    Switch Policy:
      Type:  Noop
    Volume Claim Templates:
      Name:  data
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:  20Gi
  Monitor:
  Resources:
    Cpu:     0
    Memory:  0
  Storage:
    Size:              0
  Termination Policy:  WipeOut
Status:
  Cluster Def Generation:  2
  Components:
    Mysql:
      Phase:            Updating
      Pods Ready:       false
      Pods Ready Time:  2023-10-23T11:43:21Z
      Replication Set Status:
        Primary:
          Pod:  Unknown
  Conditions:
    Last Transition Time:  2023-10-23T11:42:38Z
    Message:               The operator has started the provisioning of Cluster: asmysql-5-7
    Observed Generation:   1
    Reason:                PreCheckSucceed
    Status:                True
    Type:                  ProvisioningStarted
    Last Transition Time:  2023-10-23T11:43:24Z
    Message:               Successfully applied for resources
    Observed Generation:   1
    Reason:                ApplyResourcesSucceed
    Status:                True
    Type:                  ApplyResources
    Last Transition Time:  2023-10-23T11:43:24Z
    Message:               pods are not ready in Components: [mysql], refer to related component message in Cluster.status.components
    Reason:                ReplicasNotReady
    Status:                False
    Type:                  ReplicasReady
    Last Transition Time:  2023-10-23T11:43:24Z
    Message:               pods are unavailable in Components: [mysql], refer to related component message in Cluster.status.components
    Reason:                ComponentsNotReady
    Status:                False
    Type:                  Ready
  Observed Generation:     1
  Phase:                   Updating
Events:
  Type     Reason                    Age                    From                Message
  ----     ------                    ----                   ----                -------
  Normal   PreCheckSucceed           8m53s                  cluster-controller  The operator has started the provisioning of Cluster: asmysql-5-7
  Normal   ComponentPhaseTransition  8m52s                  cluster-controller  Create a new component
  Normal   ComponentPhaseTransition  8m10s                  cluster-controller  component is Running
  Warning  ApplyResourcesFailed      8m8s (x11 over 8m10s)  cluster-controller  the number of primary pod is not equal to 1, primary pods: [asmysql-5-7-mysql-0 asmysql-5-7-mysql-1], emptyRole pods: []
  Normal   ApplyResourcesSucceed     8m7s (x2 over 8m53s)   cluster-controller  Successfully applied for resources
  Normal   AllReplicasReady          8m7s                   cluster-controller  all pods of components are ready, waiting for the probe detection successful
  Normal   ClusterReady              8m7s                   cluster-controller  Cluster: asmysql-5-7 is ready, current phase is Running
  Normal   Running                   8m7s                   cluster-controller  Cluster: asmysql-5-7 is ready, current phase is Running
  Normal   ComponentPhaseTransition  8m7s                   cluster-controller  component is Updating
  Warning  ReplicasNotReady          8m7s                   cluster-controller  pods are not ready in Components: [mysql], refer to related component message in Cluster.status.components
  Warning  ComponentsNotReady        8m7s                   cluster-controller  pods are unavailable in Components: [mysql], refer to related component message in Cluster.status.components

logs pod check role

kubectl logs asmysql-5-7-mysql-0 kb-checkrole
{"level":"error","ts":1698061397.3134859,"caller":"binding/operation_volume_protection.go:182","msg":"unmarshal volume protection spec error","raw spec":"","error":"unexpected end of JSON input","stacktrace":"github.com/apecloud/kubeblocks/lorry/binding.(*operationVolumeProtection).initVolumes\n\t/src/lorry/binding/operation_volume_protection.go:182\ngithub.com/apecloud/kubeblocks/lorry/binding.init.1\n\t/src/lorry/binding/operation_volume_protection.go:103\nruntime.doInit1\n\t/usr/local/go/src/runtime/proc.go:6740\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:6707\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:249"}
{"level":"info","ts":1698061397.313626,"caller":"binding/operation_volume_protection.go:104","msg":"init volumes to monitor failed","error":"unexpected end of JSON input"}
{"level":"info","ts":1698061397.3137121,"caller":"binding/operation_volume_protection.go:106","msg":"succeed to init volume protection, pod: asmysql-5-7-mysql-0, spec: {\"highWatermark\":\"0\",\"volumes\":[]}"}
2023-10-23T11:43:17Z	INFO	Mysql	Initializing MySQL binding
2023-10-23T11:43:17Z	INFO	HA	HA starting
2023-10-23T11:43:17Z	INFO	HA	pod selector: app.kubernetes.io/instance=asmysql-5-7,app.kubernetes.io/managed-by=kubeblocks,apps.kubeblocks.io/component-name=mysql
2023-10-23T11:43:17Z	INFO	HA	podlist: 2
2023-10-23T11:43:17Z	INFO	HA	no switchOver [asmysql-5-7-mysql-switchover] setting
2023-10-23T11:43:17Z	ERROR	HA	new pinger failed	{"error": "lookup asmysql-5-7-mysql-0.asmysql-5-7-mysql-headless.default.svc on 10.3.240.10:53: no such host"}
github.com/apecloud/kubeblocks/lorry/highavailability.(*Ha).IsPodReady
	/src/lorry/highavailability/ha.go:421
github.com/apecloud/kubeblocks/lorry/highavailability.(*Ha).Start
	/src/lorry/highavailability/ha.go:220
2023-10-23T11:43:17Z	INFO	HA	Waiting for dns resolution to be ready
2023-10-23T11:43:20Z	ERROR	HA	new pinger failed	{"error": "lookup asmysql-5-7-mysql-0.asmysql-5-7-mysql-headless.default.svc on 10.3.240.10:53: no such host"}
github.com/apecloud/kubeblocks/lorry/highavailability.(*Ha).IsPodReady
	/src/lorry/highavailability/ha.go:421
github.com/apecloud/kubeblocks/lorry/highavailability.(*Ha).Start
	/src/lorry/highavailability/ha.go:224
2023-10-23T11:43:20Z	INFO	HA	Waiting for dns resolution to be ready
2023-10-23T11:43:21Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:21Z	INFO	Mysql	DB startup ready
2023-10-23T11:43:21Z	INFO	Mysql	send event: map[event:Success operation:checkRole originalRole: role:primary]
2023-10-23T11:43:21Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":"primary"}}
2023-10-23T11:43:21Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"9.663586ms\",\"end-time\":\"2023-10-23T11:43:21.625203651Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:21.615540071Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"primary\"}"}
2023-10-23T11:43:22Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:22Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"primary","role":"primary"}}
2023-10-23T11:43:22Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"611.822µs\",\"end-time\":\"2023-10-23T11:43:22.156443973Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:22.155832157Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"primary\",\"role\":\"primary\"}"}
2023-10-23T11:43:23Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:23Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"primary","role":"primary"}}
2023-10-23T11:43:23Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"582.523µs\",\"end-time\":\"2023-10-23T11:43:23.156046814Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:23.155464297Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"primary\",\"role\":\"primary\"}"}
2023-10-23T11:43:23Z	INFO	HA	ping failed, socket: permission denied, but temporarily return true
2023-10-23T11:43:23Z	INFO	HA	dns resolution is ready
2023-10-23T11:43:23Z	INFO	HA	cluster: &{asmysql-5-7-mysql default 2 0xc000634080 0xc0002da2d0 [{ asmysql-5-7-mysql-0   3306  107a3602-bdac-4f49-9455-1da5fb51a052 0xc0002d4480} { asmysql-5-7-mysql-1  10.0.0.73 3306  775edaf6-64d6-402d-a448-6f6dcf4b661a 0xc0002d4d80}] <nil> map[] 0xc0003b4580}
2023-10-23T11:43:23Z	INFO	HA	The database cluster is initialized.
2023-10-23T11:43:23Z	INFO	HA	pod selector: app.kubernetes.io/instance=asmysql-5-7,app.kubernetes.io/managed-by=kubeblocks,apps.kubeblocks.io/component-name=mysql
2023-10-23T11:43:23Z	INFO	HA	podlist: 2
2023-10-23T11:43:23Z	INFO	HA	lock expired: map[acquire-time:1698061395 extra: leader:asmysql-5-7-mysql-1 renew-time:1698061395 ttl:5], now: 1698061403
2023-10-23T11:43:23Z	INFO	HA	no switchOver [asmysql-5-7-mysql-switchover] setting
2023-10-23T11:43:24Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:24Z	INFO	Mysql	send event: map[event:Success operation:checkRole originalRole:primary role:]
2023-10-23T11:43:24Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"primary","role":""}}
2023-10-23T11:43:24Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"7.693853ms\",\"end-time\":\"2023-10-23T11:43:24.162393269Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:24.154699423Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"primary\",\"role\":\"\"}"}
2023-10-23T11:43:25Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:25Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:25Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"90.248µs\",\"end-time\":\"2023-10-23T11:43:25.158104441Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:25.158014193Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:26Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:26Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:26Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"67.384µs\",\"end-time\":\"2023-10-23T11:43:26.149668349Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:26.149600972Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:27Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:27Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:27Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"138.033µs\",\"end-time\":\"2023-10-23T11:43:27.149621642Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:27.149483609Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:28Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:28Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:28Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"113.749µs\",\"end-time\":\"2023-10-23T11:43:28.156038912Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:28.155925167Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:29Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:29Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:29Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"94.15µs\",\"end-time\":\"2023-10-23T11:43:29.151097456Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:29.151003305Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:30Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:30Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:30Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"72.276µs\",\"end-time\":\"2023-10-23T11:43:30.148139988Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:30.14806771Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:31Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:31Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:31Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"77.584µs\",\"end-time\":\"2023-10-23T11:43:31.150856998Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:31.150779408Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:32Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:32Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:32Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"70.304µs\",\"end-time\":\"2023-10-23T11:43:32.147649471Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:32.147579172Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:33Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:33Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:33Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"144.397µs\",\"end-time\":\"2023-10-23T11:43:33.159405836Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:33.159261442Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:33Z	INFO	HA	pod selector: app.kubernetes.io/instance=asmysql-5-7,app.kubernetes.io/managed-by=kubeblocks,apps.kubeblocks.io/component-name=mysql
2023-10-23T11:43:33Z	INFO	HA	podlist: 2
2023-10-23T11:43:33Z	INFO	HA	lock expired: map[acquire-time:1698061395 extra: leader:asmysql-5-7-mysql-1 renew-time:1698061395 ttl:5], now: 1698061413
2023-10-23T11:43:33Z	INFO	HA	no switchOver [asmysql-5-7-mysql-switchover] setting
2023-10-23T11:43:34Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:34Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:34Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"92.449µs\",\"end-time\":\"2023-10-23T11:43:34.149810012Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:34.149717565Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:35Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:35Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:35Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"75.771µs\",\"end-time\":\"2023-10-23T11:43:35.153530563Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:35.153454794Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:36Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:36Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:36Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"75.784µs\",\"end-time\":\"2023-10-23T11:43:36.151383583Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:36.151307801Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:37Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:37Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:37Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"56.758µs\",\"end-time\":\"2023-10-23T11:43:37.151520725Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:37.151463968Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:38Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:38Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:38Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"183.522µs\",\"end-time\":\"2023-10-23T11:43:38.149378245Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:38.149194728Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:39Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:39Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:39Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"55.329µs\",\"end-time\":\"2023-10-23T11:43:39.150321615Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:39.150266286Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:40Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:40Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:40Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"117.035µs\",\"end-time\":\"2023-10-23T11:43:40.152065612Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:40.151948579Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:41Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:41Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:41Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"181.966µs\",\"end-time\":\"2023-10-23T11:43:41.149100691Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:41.148918732Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:42Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:42Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:42Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"91.473µs\",\"end-time\":\"2023-10-23T11:43:42.148924485Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:42.148833019Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:43Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:43Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:43Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"52.186µs\",\"end-time\":\"2023-10-23T11:43:43.156073821Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:43.156021638Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:43Z	INFO	HA	pod selector: app.kubernetes.io/instance=asmysql-5-7,app.kubernetes.io/managed-by=kubeblocks,apps.kubeblocks.io/component-name=mysql
2023-10-23T11:43:43Z	INFO	HA	podlist: 2
2023-10-23T11:43:43Z	INFO	HA	lock expired: map[acquire-time:1698061395 extra: leader:asmysql-5-7-mysql-1 renew-time:1698061395 ttl:5], now: 1698061423
2023-10-23T11:43:43Z	INFO	HA	no switchOver [asmysql-5-7-mysql-switchover] setting
2023-10-23T11:43:44Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:44Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:44Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"102.12µs\",\"end-time\":\"2023-10-23T11:43:44.151689604Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:44.151587486Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:45Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:45Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:45Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"78.492µs\",\"end-time\":\"2023-10-23T11:43:45.154690559Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:45.154612067Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:46Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:46Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:46Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"61.515µs\",\"end-time\":\"2023-10-23T11:43:46.151048635Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:46.150987125Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:47Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:47Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
2023-10-23T11:43:47Z	INFO	grpc	No event detected	{"meta": "{\"duration\":\"96.975µs\",\"end-time\":\"2023-10-23T11:43:47.152883344Z\",\"operation\":\"checkRole\",\"start-time\":\"2023-10-23T11:43:47.152786367Z\"}", "data": "{\"event\":\"Success\",\"operation\":\"checkRole\",\"originalRole\":\"\",\"role\":\"\"}"}
2023-10-23T11:43:48Z	INFO	grpc	role probe request	{"type": "mysql"}
2023-10-23T11:43:48Z	INFO	Mysql	operation called	{"operation": "checkRole", "result": {"event":"Success","operation":"checkRole","originalRole":"","role":""}}
...

Expected behavior
mysql cluster 8.0.33/5.7.42 create success.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.
@JashBook JashBook added kind/bug Something isn't working severity/major Great chance user will encounter the same problem labels Oct 23, 2023
@JashBook JashBook added this to the Release 0.7.0 milestone Oct 23, 2023
@xuriwuyun xuriwuyun linked a pull request Oct 26, 2023 that will close this issue
fix: enable ha for mysql replication #5631
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xuriwuyun xuriwuyun

Labels
bug kind/bug Something isn't working severity/major Great chance user will encounter the same problem
Projects
None yet
Milestone
Release 0.7.0
Development

Successfully merging a pull request may close this issue.

fix: enable ha for mysql replication apecloud/kubeblocks
3 participants
@xuriwuyun @JashBook @apecloud-bot