autoPromotionEnabled: false not respected by ArgoCD

[TrondT]

I am trying out Argo Rollout with ArgoCD.

The Canary strategy seems fine, however the BlueGreen strategy, used with "autoPromotionEnabled: false", ArgoCD/Argo Rollout promotes the preview upon "sync", without any confirmation. It seems like the "autoPromotionEnabled: false" is disregarded.

If I test Argo Rollout in minikube, using helm to deploy, I can verify that the Rollout waits for Promote action, either via dashboard or "kubectl argo rollouts promote APP", as expected.

Like this:

Name:            bluegreen
Namespace:       default
Status:          ॥ Paused
Message:         BlueGreenPause
Strategy:        BlueGreen
Images:          app01:BLUE (preview)
                 app01:GREEN (stable, active)
Replicas:
  Desired:       1
  Current:       2
  Updated:       1
  Ready:         1
  Available:     1

NAME                                   KIND        STATUS        AGE    INFO
⟳ bluegreen                            Rollout     ॥ Paused      3d2h   
├──# revision:4                                                                  
│  └──⧉ bluegreen-69d7475c7b           ReplicaSet  ✔ Healthy     2d23h  preview
│     └──□ bluegreen-69d7475c7b-w9pt6  Pod         ✔ Running     2d23h  ready:1/1
├──# revision:3                                                                  
│  └──⧉ bluegreen-88b5dfc59            ReplicaSet  ✔ Healthy     3d2h   stable,active
│     └──□ bluegreen-88b5dfc59-8bs6g   Pod         ✔ Running     3d     ready:1/1
└──# revision:2                                                                  
   └──⧉ bluegreen-55dbcf8d6d           ReplicaSet  • ScaledDown  3d

But, in our cluster, after an ArgoCD "sync", the preview is fully promoted without any questions with this result:

Name:            app01
Namespace:       default
Status:          ✔ Healthy
Strategy:        BlueGreen
Images:          <image>/app01:<hash> (stable, active)
Replicas:
 Desired:       1
 Current:       1
 Updated:       1
 Ready:         1
 Available:     1

NAME                                              KIND        STATUS        AGE    INFO
⟳ app01                            Rollout     ✔ Healthy     3d3h   
├──# revision:5                                                                    
│  └──⧉ app01-6554b77c84           ReplicaSet  ✔ Healthy     80m    stable,active
│     └──□ app01-6554b77c84-89bkn  Pod         ✔ Running     80m    ready:1/1
├──# revision:4                                                                    
│  └──⧉ app01-7b4c5d8fbd           ReplicaSet  • ScaledDown  2d23h  
├──# revision:3                                                                    
│  └──⧉ app01-7bc958b84f           ReplicaSet  • ScaledDown  3d1h   
├──# revision:2                                                                    
│  └──⧉ app01-5dc9fb95cb           ReplicaSet  • ScaledDown  3d2h   
└──# revision:1                                                                    
  └──⧉ app01-5cf46dd4b5           ReplicaSet  • ScaledDown  3d3h

I have search the Argo Rollout logs and found, in minikube, these log events occurs:

time="2023-09-22T13:30:39Z" level=info msg="Adding pause reason BlueGreenPause with start time 2023-09-22T13:30:39Z" namespace=default rollout=bluegreen
time="2023-09-22T13:30:39Z" level=info msg="Event(v1.ObjectReference{Kind:\"Rollout\", Namespace:\"default\", Name:\"bluegreen\", UID:\"2d7020d7-9ece-481f-ba85-478d77acce5b\", APIVersion:\"argoproj.io/v1alpha1\", ResourceVersion:\"730096\", FieldPath:\"\"}): type: 'Normal' reason: 'RolloutPaused' Rollout is paused (BlueGreenPause)"
time="2023-09-22T13:30:39Z" level=info msg="Rollout is paused (BlueGreenPause)" event_reason=RolloutPaused namespace=default rollout=bluegreen
time="2023-09-22T13:30:39Z" level=info msg="Patched: {\"status\":{\"controllerPause\":true,\"message\":\"BlueGreenPause\",\"pauseConditions\":[{\"reason\":\"BlueGreenPause\",\"startTime\":\"2023-09-22T13:30:39Z\"}],\"phase\":\"Paused\"}}" generation=5 namespace=default resourceVersion=730096 rollout=bluegreen
time="2023-09-22T13:30:39Z" level=info msg="persisted to informer" generation=5 namespace=default resourceVersion=730121 rollout=bluegreen

While in our cluster, no such events are seen. Which component is responsible for Pausing the Deployment?

Extract of my Rollout object:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  annotations:
    rollout.argoproj.io/revision: '5'
  creationTimestamp: '2023-09-22T09:52:26Z'
  generation: 16
  labels:
    ...
  name: app01
  namespace: default
  resourceVersion: '17837373'
  uid: 140c6c92-c460-4044-8ab3-9f7cf970a5bf
spec:
  replicas: 1
  revisionHistoryLimit: 4
  selector:
    matchLabels:
      app.kubernetes.io/instance: bg
      app.kubernetes.io/name: app01
  strategy:
    blueGreen:
      activeService: app01
      autoPromotionEnabled: false
      previewService: app01-preview
...

I would expect ArgoCD/Argo-Rollout to respect the "autoPromotionEnabled: false" flag, thus not promoting the "preview" until told so.

I know about the Analysis concept in BlueGreen, and will probably adopt to that in the future.

So, my question is: how is it possible to not automatically promote "preview" when ArgoCD/Argo-Rollout is used?

[kingbj940429]

I experienced same issue.

I solved this issue by changing ServerSideApply at kind: ApplicationSet to ServerSideApply=false

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: app-demo
  namespace: continuous-system
spec:
  generators:
    - list:
        elements:
          - deployTier: dev
            targetRevision: feature-0b154b
  template:
    metadata:
      name: "app-demo-{{deployTier}}"
      namespace: continuous-system
    spec:
      project: pf
      destination:
        namespace: insight
        server: "https://kubernetes.default.svc"
      source:
        chart: app-demo
        repoURL: "https://privates/repository/helm-hosted"
        targetRevision: "{{targetRevision}}"
        plugin:
          name: avp-helm
          env:
            - name: HELM_ARGS
              value: -f values-{{deployTier}}.yaml
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        retry:
          limit: 3
          backoff:
            duration: 5s
            factor: 2
            maxDuration: 3m
        syncOptions:
          - ServerSideApply=false #-- to false