Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] orc mysql cluster switchover error: component mysql does not support switchover #8839

Closed
JashBook opened this issue Jan 21, 2025 · 2 comments
Closed

[BUG] orc mysql cluster switchover error: component mysql does not support switchover #8839

JashBook opened this issue Jan 21, 2025 · 2 comments
Assignees
@kubeJocker
Labels
kind/bug Something isn't working
Milestone
Release 0.9.3

Comments

@JashBook
Copy link
Collaborator

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. create orc cluster
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: orcm-pvaoym
  namespace: default
  annotations:
    "kubeblocks.io/extra-env": '{"ORC_TOPOLOGY_PASSWORD":"orchestrator","ORC_META_PASSWORD":"orchestrator","ORC_META_DATABASE":"orchestrator"}'
spec:
  clusterDef: orchestrator
  topology: raft
  terminationPolicy: WipeOut
  services:
    - name: orchestrator
      componentSelector: orchestrator
      spec:
        ports:
          - name: orc-http
            port: 80
  componentSpecs:
    - name: orchestrator
      serviceVersion: 3.2.6
      env:
        - name: ORC_TOPOLOGY_PASSWORD
          value: orchestrator
        - name: ORC_META_PASSWORD
          value: orchestrator
        - name: ORC_META_DATABASE
          value: orchestrator
      replicas: 3
      resources:
        requests:
          cpu: 500m
          memory: 0.5Gi
        limits:
          cpu: 500m
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
  1. create mysql cluster with orc
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: asmysql-pvaoym
  namespace: default
  
spec:
  clusterDef: mysql
  topology: orc
  terminationPolicy: WipeOut
  componentSpecs:
    - name: mysql
      serviceVersion: 8.0.39
      disableExporter: true
      replicas: 2      
      resources:
        limits:
          cpu: 500m
          memory: 0.5Gi
        requests:
          cpu: 500m
          memory: 0.5Gi      
      volumeClaimTemplates:
        - name: data
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
      serviceRefs:
      - name: orchestrator
        namespace: default
        clusterServiceSelector:
          cluster: orcm-pvaoym
          service:
            component: orchestrator
            service: orchestrator
            port: orc-http

kbcli cluster list-instances asmysql-pvaoym
NAME                     NAMESPACE   CLUSTER          COMPONENT   STATUS    ROLE        ACCESSMODE   AZ              CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)   STORAGE     NODE                      CREATED-TIME                 
asmysql-pvaoym-mysql-0   default     asmysql-pvaoym   mysql       Running   primary     ReadWrite    cn-shanghai-a   500m / 500m          512Mi / 512Mi           data:20Gi   172.16.0.43/172.16.0.43   Jan 21,2025 16:02 UTC+0800   
asmysql-pvaoym-mysql-1   default     asmysql-pvaoym   mysql       Running   secondary   Readonly     cn-shanghai-a   500m / 500m          512Mi / 512Mi           data:20Gi   172.16.0.43/172.16.0.43   Jan 21,2025 16:02 UTC+0800
  1. switchover
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: asmysql-pvaoym-switchover
spec:
  clusterName: asmysql-pvaoym
  type: Switchover
  switchover:
  - componentName: mysql
    instanceName: '*'
  1. See error
kubectl get ops -l app.kubernetes.io/instance=asmysql-pvaoym 
NAME                          TYPE         CLUSTER          STATUS    PROGRESS   AGE
asmysql-pvaoym-expose-kvklv   Expose       asmysql-pvaoym   Succeed   1/1        8m56s
asmysql-pvaoym-switchover     Switchover   asmysql-pvaoym   Failed    -/-        5m26s

describe ops

kubectl describe ops asmysql-pvaoym-switchover 
Name:         asmysql-pvaoym-switchover
Namespace:    default
Labels:       app.kubernetes.io/instance=asmysql-pvaoym
              operations.kubeblocks.io/ops-type=Switchover
Annotations:  <none>
API Version:  operations.kubeblocks.io/v1alpha1
Kind:         OpsRequest
Metadata:
  Creation Timestamp:  2025-01-21T08:09:35Z
  Finalizers:
    opsrequest.kubeblocks.io/finalizer
  Generation:  1
  Owner References:
    API Version:     apps.kubeblocks.io/v1
    Kind:            Cluster
    Name:            asmysql-pvaoym
    UID:             57819eb3-441b-4637-a4b6-6b8b305b9a27
  Resource Version:  33523720
  UID:               e52b8dc6-0708-41c4-8d20-3b95c49338e9
Spec:
  Cluster Name:                    asmysql-pvaoym
  Enqueue On Force:                false
  Pre Condition Deadline Seconds:  0
  Switchover:
    Component Name:  mysql
    Instance Name:   *
  Type:              Switchover
Status:
  Completion Timestamp:  2025-01-21T08:09:36Z
  Conditions:
    Last Transition Time:  2025-01-21T08:09:36Z
    Message:               wait for the controller to process the OpsRequest: asmysql-pvaoym-switchover in Cluster: asmysql-pvaoym
    Reason:                WaitForProgressing
    Status:                True
    Type:                  WaitForProgressing
    Last Transition Time:  2025-01-21T08:09:36Z
    Message:               this cluster component mysql does not support switchover
    Reason:                ValidateFailed
    Status:                False
    Type:                  Validated
  Phase:                   Failed
  Progress:                -/-
Events:
  Type     Reason              Age    From                    Message
  ----     ------              ----   ----                    -------
  Normal   WaitForProgressing  5m36s  ops-request-controller  wait for the controller to process the OpsRequest: asmysql-pvaoym-switchover in Cluster: asmysql-pvaoym
  Warning  ValidateFailed      5m36s  ops-request-controller  this cluster component mysql does not support switchover

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.
@JashBook JashBook added the kind/bug Something isn't working label Jan 21, 2025
@JashBook JashBook mentioned this issue Jan 21, 2025
Open
@JashBook
Copy link
Collaborator Author

dup #8800

@github-actions github-actions bot added this to the Release 0.9.3 milestone Jan 21, 2025
@JashBook
Copy link
Collaborator Author

Custom switchover failed

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: mysql-switchover
spec:
  clusterName: asmysql-pvaoym
  type: Custom
  custom:
    components:
      - componentName: mysql
        parameters:
          - name: candidate
            value: asmysql-pvaoym-mysql-1
    opsDefinitionName: mysql-orc-switchover

kubectl get ops asmysql-pvaoym-switchover-2
NAME                          TYPE     CLUSTER          STATUS    PROGRESS   AGE
asmysql-pvaoym-switchover-2   Custom   asmysql-pvaoym   Succeed   1/1        2m12s

kubectl describe ops asmysql-pvaoym-switchover-2 
Name:         asmysql-pvaoym-switchover-2
Namespace:    default
Labels:       app.kubernetes.io/instance=asmysql-pvaoym
              operations.kubeblocks.io/ops-type=Custom
Annotations:  <none>
API Version:  operations.kubeblocks.io/v1alpha1
Kind:         OpsRequest
Metadata:
  Creation Timestamp:  2025-01-21T08:22:40Z
  Finalizers:
    opsrequest.kubeblocks.io/finalizer
  Generation:  2
  Owner References:
    API Version:     apps.kubeblocks.io/v1
    Kind:            Cluster
    Name:            asmysql-pvaoym
    UID:             57819eb3-441b-4637-a4b6-6b8b305b9a27
  Resource Version:  33533128
  UID:               96f2d315-a9fd-405e-9e6a-2a6403685526
Spec:
  Cluster Name:  asmysql-pvaoym
  Custom:
    Components:
      Component Name:  mysql
      Parameters:
        Name:                      candidate
        Value:                     asmysql-pvaoym-mysql-1
    Max Concurrent Components:     0
    Ops Definition Name:           mysql-orc-switchover
  Enqueue On Force:                false
  Pre Condition Deadline Seconds:  0
  Type:                            Custom
Status:
  Cluster Generation:    4
  Completion Timestamp:  2025-01-21T08:22:47Z
  Components:
    Mysql:
      Phase:  Running
      Progress Details:
        Action Name:  switchover
        Action Tasks:
          Namespace:   default
          Object Key:  Job/96f2d315-asmysql-pvaoym-swi-mysql-switchover-0
          Status:      Succeed
        End Time:      2025-01-21T08:22:47Z
        Message:       the action "switchover" of the component "mysql" is Succeed
        Start Time:    2025-01-21T08:22:40Z
        Status:        Succeed
  Conditions:
    Last Transition Time:  2025-01-21T08:22:40Z
    Message:               wait for the controller to process the OpsRequest: asmysql-pvaoym-switchover-2 in Cluster: asmysql-pvaoym
    Reason:                WaitForProgressing
    Status:                True
    Type:                  WaitForProgressing
    Last Transition Time:  2025-01-21T08:22:40Z
    Message:               OpsRequest: asmysql-pvaoym-switchover-2 is validated
    Reason:                ValidateOpsRequestPassed
    Status:                True
    Type:                  Validated
    Last Transition Time:  2025-01-21T08:22:40Z
    Message:               Start to handle MysqlOrcSwitchover on the Cluster: asmysql-pvaoym
    Reason:                MysqlOrcSwitchoverStarting
    Status:                True
    Type:                  CustomOperation
    Last Transition Time:  2025-01-21T08:22:47Z
    Message:               Successfully processed the OpsRequest: asmysql-pvaoym-switchover-2 in Cluster: asmysql-pvaoym
    Reason:                OpsRequestProcessedSuccessfully
    Status:                True
    Type:                  Succeed
  Phase:                   Succeed
  Progress:                1/1
  Start Timestamp:         2025-01-21T08:22:40Z
Events:
  Type    Reason                           Age                    From                    Message
  ----    ------                           ----                   ----                    -------
  Normal  WaitForProgressing               2m15s                  ops-request-controller  wait for the controller to process the OpsRequest: asmysql-pvaoym-switchover-2 in Cluster: asmysql-pvaoym
  Normal  ValidateOpsRequestPassed         2m15s                  ops-request-controller  OpsRequest: asmysql-pvaoym-switchover-2 is validated
  Normal  MysqlOrcSwitchoverStarting       2m15s                  ops-request-controller  Start to handle MysqlOrcSwitchover on the Cluster: asmysql-pvaoym
  Normal  Processing                       2m15s (x2 over 2m15s)  ops-request-controller  Start to processing action "switchover" of the component mysql
  Normal  Succeed                          2m8s                   ops-request-controller  the action "switchover" of the component "mysql" is Succeed
  Normal  OpsRequestProcessedSuccessfully  2m8s                   ops-request-controller  Successfully processed the OpsRequest: asmysql-pvaoym-switchover-2 in Cluster: asmysql-pvaoym

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kubeJocker kubeJocker

Labels
kind/bug Something isn't working
Projects
None yet
Milestone
Release 0.9.3
Development

No branches or pull requests
3 participants
@kubeJocker @JashBook @shanshanying