Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]ob primary/standby cluster created failed #7025

Closed
ahjing99 opened this issue Apr 11, 2024 · 2 comments
Closed

[BUG]ob primary/standby cluster created failed #7025

ahjing99 opened this issue Apr 11, 2024 · 2 comments
Assignees
@shanshanying
Labels
kind/bug Something isn't working Stale
Milestone
Release 0.8.5

Comments

@ahjing99
Copy link
Collaborator

➜ ~ kbcli version
Kubernetes: v1.27.8-gke.1067004
KubeBlocks: 0.9.0-beta.5
kbcli: 0.9.0-beta.1

# Add Helm repo 
helm repo add kubeblocks-addons https://apecloud.github.io/helm-charts
# If github is not accessible or very slow for you, please use following repo instead
helm repo add kubeblocks-addons https://jihulab.com/api/v4/projects/150246/packages/helm/stable
# Update helm repo
helm repo update

# Enable oceanbase 
helm upgrade -i oceanbase-ce kubeblocks-addons/oceanbase-ce --version 0.9.0 -n kb-system
  1. Create cluster
    kubectl apply -f cluster.yaml
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
  name: oceanbase-cluster-repl
  namespace: default
  annotations:
    "kubeblocks.io/extra-env": "{\"TENANT_NAME\":\"obtenant\",\"ZONE_COUNT\":\"1\",\"OB_CLUSTERS_COUNT\":\"2\",\"TENANT_CPU\":\"2\",\"TENANT_MEMORY\":\"2G\",\"TENANT_DISK\":\"5G\"}"
spec:
  clusterDefinitionRef: ob-ce
  clusterVersionRef: ob-ce-4.2.0.0-100010032023083021
  terminationPolicy: Delete
  affinity:
    podAntiAffinity: Preferred
    topologyKeys:
      - kubernetes.io/hostname
    tenancy: SharedNode
  tolerations:
    - key: kb-data
      operator: Equal
      value: "true"
      effect: NoSchedule
  componentSpecs:
    - name: ob-ce-0
      componentDefRef: ob-ce-repl
      replicas: 1
      resources:
        limits:
          cpu: "3"
          memory: "8Gi"
        requests:
          cpu: "3"
          memory: "8Gi"
      volumeClaimTemplates:
        - name: data-file
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 50Gi
        - name: data-log
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 50Gi
        - name: log
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 10Gi
        - name: workdir
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
    - name: ob-ce-1
      componentDefRef: ob-ce-repl
      replicas: 1
      resources:
        limits:
          cpu: "3"
          memory: "8Gi"
        requests:
          cpu: "3"
          memory: "8Gi"
      volumeClaimTemplates:
        - name: data-file
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 50Gi
        - name: data-log
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 50Gi
        - name: log
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 10Gi
        - name: workdir
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
  1. create failed
➜  ~ k describe cluster oceanbase-cluster-repl
Name:         oceanbase-cluster-repl
Namespace:    default
Labels:       clusterdefinition.kubeblocks.io/name=ob-ce
              clusterversion.kubeblocks.io/name=ob-ce-4.2.0.0-100010032023083021
Annotations:  kubeblocks.io/extra-env:
                {"TENANT_NAME":"obtenant","ZONE_COUNT":"1","OB_CLUSTERS_COUNT":"2","TENANT_CPU":"2","TENANT_MEMORY":"2G","TENANT_DISK":"5G"}
API Version:  apps.kubeblocks.io/v1alpha1
Kind:         Cluster
Metadata:
  Creation Timestamp:  2024-04-11T08:59:31Z
  Finalizers:
    cluster.kubeblocks.io/finalizer
  Generation:  1
  Managed Fields:
    API Version:  apps.kubeblocks.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubeblocks.io/extra-env:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:affinity:
          .:
          f:podAntiAffinity:
          f:tenancy:
          f:topologyKeys:
            .:
            v:"kubernetes.io/hostname":
        f:clusterDefinitionRef:
        f:clusterVersionRef:
        f:componentSpecs:
        f:terminationPolicy:
        f:tolerations:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2024-04-11T08:59:31Z
    API Version:  apps.kubeblocks.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"cluster.kubeblocks.io/finalizer":
        f:labels:
          .:
          f:clusterdefinition.kubeblocks.io/name:
          f:clusterversion.kubeblocks.io/name:
    Manager:      manager
    Operation:    Update
    Time:         2024-04-11T08:59:31Z
    API Version:  apps.kubeblocks.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:clusterDefGeneration:
        f:components:
          .:
          f:ob-ce-0:
            .:
            f:phase:
            f:podsReady:
          f:ob-ce-1:
            .:
            f:phase:
            f:podsReady:
        f:conditions:
        f:observedGeneration:
        f:phase:
    Manager:         manager
    Operation:       Update
    Subresource:     status
    Time:            2024-04-11T08:59:33Z
  Resource Version:  2751662
  UID:               9057a94a-523b-4299-9b36-b1451ebde49d
Spec:
  Affinity:
    Pod Anti Affinity:  Preferred
    Tenancy:            SharedNode
    Topology Keys:
      kubernetes.io/hostname
  Cluster Definition Ref:  ob-ce
  Cluster Version Ref:     ob-ce-4.2.0.0-100010032023083021
  Component Specs:
    Component Def Ref:  ob-ce-repl
    Monitor:            false
    Name:               ob-ce-0
    Replicas:           1
    Resources:
      Limits:
        Cpu:     3
        Memory:  8Gi
      Requests:
        Cpu:     3
        Memory:  8Gi
    Volume Claim Templates:
      Name:  data-file
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:  50Gi
      Name:           data-log
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:  50Gi
      Name:           log
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:  10Gi
      Name:           workdir
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:    20Gi
    Component Def Ref:  ob-ce-repl
    Monitor:            false
    Name:               ob-ce-1
    Replicas:           1
    Resources:
      Limits:
        Cpu:     3
        Memory:  8Gi
      Requests:
        Cpu:     3
        Memory:  8Gi
    Volume Claim Templates:
      Name:  data-file
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:  50Gi
      Name:           data-log
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:  50Gi
      Name:           log
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:  10Gi
      Name:           workdir
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:   20Gi
  Termination Policy:  Delete
  Tolerations:
    Effect:    NoSchedule
    Key:       kb-data
    Operator:  Equal
    Value:     true
Status:
  Cluster Def Generation:  2
  Components:
    ob-ce-0:
      Phase:       Creating
      Pods Ready:  false
    ob-ce-1:
      Phase:       Creating
      Pods Ready:  false
  Conditions:
    Last Transition Time:  2024-04-11T08:59:31Z
    Message:               The operator has started the provisioning of Cluster: oceanbase-cluster-repl
    Observed Generation:   1
    Reason:                PreCheckSucceed
    Status:                True
    Type:                  ProvisioningStarted
    Last Transition Time:  2024-04-11T08:59:31Z
    Message:               Successfully applied for resources
    Observed Generation:   1
    Reason:                ApplyResourcesSucceed
    Status:                True
    Type:                  ApplyResources
    Last Transition Time:  2024-04-11T08:59:31Z
    Message:               pods are not ready in Components: [ob-ce-0 ob-ce-1], refer to related component message in Cluster.status.components
    Reason:                ReplicasNotReady
    Status:                False
    Type:                  ReplicasReady
    Last Transition Time:  2024-04-11T08:59:31Z
    Message:               pods are unavailable in Components: [ob-ce-0 ob-ce-1], refer to related component message in Cluster.status.components
    Reason:                ComponentsNotReady
    Status:                False
    Type:                  Ready
  Observed Generation:     1
  Phase:                   Creating
Events:
  Type    Reason                    Age                From                Message
  ----    ------                    ----               ----                -------
  Normal  PreCheckSucceed           15m                cluster-controller  The operator has started the provisioning of Cluster: oceanbase-cluster-repl
  Normal  ApplyResourcesSucceed     15m                cluster-controller  Successfully applied for resources
  Normal  ComponentPhaseTransition  15m (x2 over 15m)  cluster-controller  component is Creating

kb-logs:
2024-04-11T09:05:02.240Z	ERROR	Reconciler error	{"controller": "replicatedstatemachine", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "ReplicatedStateMachine", "ReplicatedStateMachine": {"name":"oceanbase-cluster-repl-ob-ce-1","namespace":"default"}, "namespace": "default", "name": "oceanbase-cluster-repl-ob-ce-1", "reconcileID": "63b70c20-9c0b-4f18-a697-0187b742b490", "error": "Pod \"oceanbase-cluster-repl-ob-ce-1-0\" is invalid: [spec.containers[0].image: Required value, spec.containers[1].image: Required value]"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227

report-kubeblocks-2024-04-11-17-15-26.zip
report-cluster-oceanbase-cluster-repl-2024-04-11-17-15-13.zip
@ahjing99 ahjing99 added kind/bug Something isn't working severity/major Great chance user will encounter the same problem labels Apr 11, 2024
@ahjing99 ahjing99 added this to the Release 0.9.0 milestone Apr 11, 2024
@ahjing99 ahjing99 removed the severity/major Great chance user will encounter the same problem label Apr 11, 2024
@github-actions GitHub Actions
Copy link

This issue has been marked as stale because it has been open for 30 days with no activity

@github-actions github-actions bot added the Stale label May 13, 2024
@ahjing99 ahjing99 modified the milestones: Release 0.9.0, Release 0.9.1 Jul 8, 2024
@github-actions github-actions bot modified the milestones: Release 0.9.1, Release 0.9.2 Aug 8, 2024
@github-actions github-actions bot modified the milestones: Release 0.9.2, Release 0.8.5 Oct 8, 2024
@shanshanying
Copy link
Contributor

fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shanshanying shanshanying

Labels
kind/bug Something isn't working Stale
Projects
None yet
Milestone
Release 0.8.5
Development

No branches or pull requests
2 participants
@ahjing99 @shanshanying