Failed NAB reconciles again after DPA update

[stillalearner]

Description:

When I created a NAB CR in a non-admin namespace, with some enforcedBackupSpec value mismatch to what is set already in DPA by admin (for example snapshotMoveData), NAB validation fails at first, But on updating the DPA enforcedBackupSpec value to match the value with the NAB backup spec, It reconciles and start running again.

Steps to reproduce:

1. Create a DPA with Non ADmin enabled with some enforcedBackupSpec value set , in this case snapshotMoveData : true

DPA:

`apiVersion: v1
items:

• apiVersion: oadp.openshift.io/v1alpha1
  kind: DataProtectionApplication
  metadata:
  creationTimestamp: "2025-01-22T11:52:33Z"
  generation: 15
  name: ts-dpa
  namespace: openshift-adp
  resourceVersion: "306326"
  uid: f8d32ddf-c943-4bc5-a6a9-5b4b75454723
  spec:
  backupLocations:
  • velero:
    credential:
    key: cloud
    name: cloud-credentials-gcp
    default: true
    objectStorage:
    bucket: oadp108491vsrf7
    prefix: velero
    provider: gcp
    configuration:
    nodeAgent:
    enable: true
    uploaderType: kopia
    velero:
    defaultPlugins:
    • csi
    • gcp
    • openshift
      nonAdmin:
      enable: true
      enforceBackupSpec:
      snapshotMoveData: true
      unsupportedOverrides:
      tech-preview-ack: "true"
      status:
      conditions:
  • lastTransitionTime: "2025-01-22T11:52:33Z"
    message: Reconcile complete
    reason: Complete
    status: "True"
    type: Reconciled
    `

2. Go to Non admin namespace as non admin user, and create a Non Admin Backup CR (lets say backuptest1)

`$ oc get nab backuptest1 -o yaml
apiVersion: oadp.openshift.io/v1alpha1
kind: NonAdminBackup
metadata:
creationTimestamp: "2025-01-22T15:23:12Z"
finalizers:

• nonadminbackup.oadp.openshift.io/finalizer
  generation: 1
  labels:
  app.kubernetes.io/created-by: oadp-operator
  app.kubernetes.io/instance: nonadminbackup-sample
  app.kubernetes.io/managed-by: kustomize
  app.kubernetes.io/name: nonadminbackup
  app.kubernetes.io/part-of: oadp-operator
  name: backuptest1
  namespace: sns-3
  resourceVersion: "307039"
  uid: 2dad0964-f8a3-4f6d-b30d-beac62ccc363
  spec:
  backupSpec:
  defaultVolumesToFsBackup: true
  metadata: {}
  snapshotMoveData: false
  status:
  conditions:
• lastTransitionTime: "2025-01-22T15:23:12Z"
  message: NonAdminBackup spec.backupSpec.snapshotMoveData field value is enforced
  by admin user, can not override it
  reason: InvalidBackupSpec
  status: "False"
  type: Accepted
  phase: BackingOff
  `

3. Update the DPA to match the snapshotMoveData value as "false"

Actual Results

Same NonAdminBackup CR starts running again:

`$ oc get nab backuptest1 -o yaml
apiVersion: oadp.openshift.io/v1alpha1
kind: NonAdminBackup
metadata:
creationTimestamp: "2025-01-22T15:23:12Z"
finalizers:

• nonadminbackup.oadp.openshift.io/finalizer
  generation: 1
  labels:
  app.kubernetes.io/created-by: oadp-operator
  app.kubernetes.io/instance: nonadminbackup-sample
  app.kubernetes.io/managed-by: kustomize
  app.kubernetes.io/name: nonadminbackup
  app.kubernetes.io/part-of: oadp-operator
  name: backuptest1
  namespace: sns-3
  resourceVersion: "307490"
  uid: 2dad0964-f8a3-4f6d-b30d-beac62ccc363
  spec:
  backupSpec:
  defaultVolumesToFsBackup: true
  metadata: {}
  snapshotMoveData: false
  status:
  conditions:
• lastTransitionTime: "2025-01-22T15:24:07Z"
  message: backup accepted
  reason: BackupAccepted
  status: "True"
  type: Accepted
• lastTransitionTime: "2025-01-22T15:24:07Z"
  message: Created Velero Backup object
  reason: BackupScheduled
  status: "True"
  type: Queued
  phase: Created
  queueInfo:
  estimatedQueuePosition: 1
  veleroBackup:
  nacuuid: sns-3-backuptest1-148d6cd5-3aa9-4a2b-b777-b82dc754178b
  name: sns-3-backuptest1-148d6cd5-3aa9-4a2b-b777-b82dc754178b
  namespace: openshift-adp
  status:
  expiration: "2025-02-21T15:24:07Z"
  formatVersion: 1.1.0
  phase: InProgress
  progress:
  totalItems: 150
  startTimestamp: "2025-01-22T15:24:07Z"
  version: 1
  `

and it completes succssfully

`status:
conditions:

• lastTransitionTime: "2025-01-22T15:24:07Z"
  message: backup accepted
  reason: BackupAccepted
  status: "True"
  type: Accepted
• lastTransitionTime: "2025-01-22T15:24:07Z"
  message: Created Velero Backup object
  reason: BackupScheduled
  status: "True"
  type: Queued
  phase: Created
  queueInfo:
  estimatedQueuePosition: 0
  veleroBackup:
  nacuuid: sns-3-backuptest1-148d6cd5-3aa9-4a2b-b777-b82dc754178b
  name: sns-3-backuptest1-148d6cd5-3aa9-4a2b-b777-b82dc754178b
  namespace: openshift-adp
  status:
  completionTimestamp: "2025-01-22T15:24:19Z"
  expiration: "2025-02-21T15:24:07Z"
  formatVersion: 1.1.0
  hookStatus: {}
  phase: Completed
  progress:
  itemsBackedUp: 153
  totalItems: 153
  startTimestamp: "2025-01-22T15:24:07Z"
  version: 1
  `

Expected Results:

Once a NAB is marked as FAILED, It should remain in Failed state.

[stillalearner]

LOGS:

$ oc logs non-admin-controller-84c87b8f47-rpfh6 | grep backuptest1 2025-01-22T15:24:07Z DEBUG Accepted NAB Create event {"NonAdminBackupPredicate": {"name":"backuptest1","namespace":"sns-3"}} 2025-01-22T15:24:07Z DEBUG NonAdminBackup Reconcile start {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3"} 2025-01-22T15:24:07Z DEBUG Executing nab creation/update path {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3"} 2025-01-22T15:24:07Z DEBUG NonAdminBackup Phase already initialized {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3", "currentPhase": "BackingOff"} 2025-01-22T15:24:07Z DEBUG NonAdminBackup Spec is valid {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3"} 2025-01-22T15:24:07Z DEBUG Rejected NAB Update event {"NonAdminBackupPredicate": {"name":"backuptest1","namespace":"sns-3"}} 2025-01-22T15:24:07Z DEBUG NonAdminBackup condition set to Accepted {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3"} 2025-01-22T15:24:07Z DEBUG NonAdminBackup - Status Updated with UUID reference {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3"} 2025-01-22T15:24:07Z DEBUG Finalizer exists on the NonAdminBackup object {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3", "finalizer": "nonadminbackup.oadp.openshift.io/finalizer"} 2025-01-22T15:24:07Z DEBUG Rejected NAB Update event {"NonAdminBackupPredicate": {"name":"backuptest1","namespace":"sns-3"}} 2025-01-22T15:24:07Z INFO VeleroBackup with label not found, creating one {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3", "UUID": "sns-3-backuptest1-148d6cd5-3aa9-4a2b-b777-b82dc754178b"} 2025-01-22T15:24:07Z INFO VeleroBackup successfully created {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3"} 2025-01-22T15:24:07Z DEBUG Rejected NAB Update event {"NonAdminBackupPredicate": {"name":"backuptest1","namespace":"sns-3"}} 2025-01-22T15:24:07Z DEBUG Rejected Backup Update event: no changes to the CompletionTimestamp in the VeleroBackup object {"VeleroBackupQueuePredicate": {"name":"sns-3-backuptest1-148d6cd5-3aa9-4a2b-b777-b82dc754178b","namespace":"openshift-adp"}} 2025-01-22T15:24:07Z DEBUG Accepted Backup Update event {"VeleroBackupPredicate": {"name":"sns-3-backuptest1-148d6cd5-3aa9-4a2b-b777-b82dc754178b","namespace":"openshift-adp"}} 2025-01-22T15:24:07Z DEBUG NonAdminBackup - Exit after Status Update {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3"} 2025-01-22T15:24:07Z DEBUG NonAdminBackup Reconcile exit {"controller": "nonadminbackup", "controllerGroup": "oadp.openshift.io", "controllerKind": "NonAdminBackup", "NonAdminBackup": {"name":"backuptest1","namespace":"sns-3"}, "namespace": "sns-3", "name": "backuptest1", "reconcileID": "caf397c8-60e1-4c8f-beb1-0c4a6b621bb3"} 2025-01-22T15:24:07Z DEBUG Rejected Backup Update event: no changes to the CompletionTimestamp in the VeleroBackup object {"VeleroBackupQueuePredicate": {"name":"sns-3-backuptest1-148d6cd5-3aa9-4a2b-b777-b82dc754178b","namespace":"openshift-adp"}}