Backup pod restarts without doing backup

[Svenum]

Version
quay.io/ansible/galaxy-operator:2024.02.29

Describe the bug
If I create an Backup CRD. the pod comes up but restarts the hole time.

To Reproduce

1. kubectl apply -f backup.yml
   backup.yml:

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: galaxybackup-backup-claim
  namespace: galaxy
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: nfs-backup-csi
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: galaxybackup-cr
  namespace: galaxy
data:
  galaxy_backup.yaml: |
    apiVersion: galaxy.ansible.com/v1beta1
    kind: GalaxyBackup
    metadata:
      name: galaxybackup
      namespace: galaxy
    spec:
      backup_storage_class: nfs-backup-csi
      deployment_name: galaxy
      backup_pvc: galaxybackup-backup-claim
      backup_pvc_namespace: galaxy

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: galaxybackup
  namespace: galaxy
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: galaxybackup
  namespace: galaxy
rules:
- apiGroups: ["galaxy.ansible.com"]
  resources: ["galaxybackups"]
  verbs: ["get", "watch", "list","create","patch","update","delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: galaxybackup
  namespace: galaxy
subjects:
- kind: ServiceAccount
  name: galaxybackup
roleRef:
  kind: Role
  name: galaxybackup
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: galaxybackup
  namespace: galaxy
spec:
  schedule: "0 4 * * *"
  successfulJobsHistoryLimit: 1
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: bkp
            image: bitnami/kubectl:latest
            imagePullPolicy: IfNotPresent
            volumeMounts:
            - name: galaxybackup-cr
              mountPath: /tmp/galaxy_backup.yaml
              subPath: galaxy_backup.yaml
            command:
            - /bin/sh
            - -c
            args:
            - "kubectl apply -f /tmp/galaxy_backup.yaml && kubectl wait --for condition=BackupComplete --timeout=600s -f /tmp/galaxy_backup.yaml ; kubectl delete -f /tmp/galaxy_backup.yaml"
          restartPolicy: Never
          serviceAccountName: galaxybackup
          volumes:
          - name: galaxybackup-cr
            configMap:
              name: galaxybackup-cr

2. kubectl create job --from=cronjob/galaxybackup -n galaxy manual-backup

Expected behavior
backup-manager pod must be created, backup the galaxy, terminate

Additional context
Error message in operator:

2024-03-13T07:44:51.553313959Z stdout F --------------------------- Ansible Task StdOut -------------------------------
2024-03-13T07:44:51.553318318Z stdout F
2024-03-13T07:44:51.553321602Z stdout F TASK [common : Get the imagePullSecrets from "galaxy"] *************************
2024-03-13T07:44:51.553324551Z stdout F ESC[1;30mtask path: /opt/ansible/roles/common/tasks/main.yml:20ESC[0m
2024-03-13T07:44:51.553295876Z stderr F {"level":"info","ts":1710315891.5531833,"logger":"logging_event_handler","msg":"[playbook task start]","name":"galaxy","namespace":"galaxy","gvk":"galaxy.ansible.com/v1beta1, Kind=Galaxy","event_type":"playbook_on_task_start","job":"2962438046381893477","EventData.Name":"common : Get the imagePullSecrets from \"galaxy\""}
2024-03-13T07:44:51.553327629Z stdout F
2024-03-13T07:44:51.553349704Z stdout F -------------------------------------------------------------------------------
...skipping...
lient.py\", line 192, in __call_api\n    return_data = self.deserialize(response_data, response_type)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\", line 264, in deserialize\n    return self.__deserialize(data, response_type)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\", line 303, in __
deserialize\n    return self.__deserialize_model(data, klass)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\", line 639, in __deserialize_model\n    kwargs[attr] = self.__deserialize(value, attr_type)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\", line 303, in __deserialize\n    return sel
f.__deserialize_model(data, klass)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\", line 639, in __deserialize_model\n    kwargs[attr] = self.__deserialize(value, attr_type)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\", line 280, in __deserialize\n    return [self.__deserialize(sub_data,
sub_kls)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\", line 280, in <listcomp>\n    return [self.__deserialize(sub_data, sub_kls)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\", line 303, in __deserialize\n    return self.__deserialize_model(data, klass)\n  File \"/usr/local/lib/python3.
8/site-packages/kubernetes/client/api_client.py\", line 641, in __deserialize_model\n    instance = klass(**kwargs)\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/models/v1_pod_condition.py\", line 76, in __init__\n    self.type = type\n  File \"/usr/local/lib/python3.8/site-packages/kubernetes/client/models/v1_pod_condition.py\", line 219,
 in type\n    raise ValueError(\nValueError: Invalid value for `type` (PodReadyToStartContainers), must be one of ['ContainersReady', 'Initialized', 'PodScheduled', 'Ready']\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}ESC[0m
2024-03-13T08:40:13.651663203Z stdout F
2024-03-13T08:40:13.651666505Z stdout F -------------------------------------------------------------------------------
2024-03-13T08:40:13.651611602Z stderr F {"level":"error","ts":1710319213.6514914,"logger":"logging_event_handler","msg":"","name":"galaxybackup","namespace":"galaxy","gvk":"galaxy.ansible.com/v1beta1, Kind=GalaxyBackup","event_type":"runner_on_failed","job":"775119205517794129","EventData.Task":"Create directory for backup","EventData.TaskArgs":"","EventData.F
ailedTaskPath":"/opt/ansible/roles/backup/tasks/postgres.yml:66","error":"[playbook task failed]"}
2024-03-13T08:40:13.833998397Z stderr P {"level":"error","ts":1710319213.8334985,"logger":"runner","msg":"\u001b[0;34mansible-playbook 2.9.27\u001b[0m\r\n\u001b[0;34m  config file = /etc/ansible/ansible.cfg\u001b[0m\r\n\u001b[0;34m  configured module search path = ['/usr/share/ansible/openshift']\u001b[0m\r\n\u001b[0;34m  ansible python module location = /usr/
local/lib/python3.8/site-packages/ansible\u001b[0m\r\n\u001b[0;34m  executable location = /usr/local/bin/ansible-playbook\u001b[0m\r\n\u001b[0;34m  python version = 3.8.12 (default, Sep 16 2021, 10:46:05) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]\u001b[0m\r\n\u001b[0;34mUsing /etc/ansible/ansible.cfg as config file\u001b[0m\r\n\u001b[0;34mSkipping callback 'actio
nable', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'awx_display', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'counter_enabled', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'debug', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback
 'dense', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'dense', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'full_skip', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'json', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'minimal',
 as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'null', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'oneline', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'selective', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'skippy', as we a
lready have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'stderr', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'unixy', as we already have a stdout callback.\u001b[0m\n\u001b[0;34mSkipping callback 'yaml', as we already have a stdout callback.\u001b[0m\n\r\nPLAYBOOK: 6e617a9cce9144fd9e8a12974374efcd *************
************************\n\u001b[0;34m1 plays in /tmp/ansible-operator/runner/galaxy.ansible.com/v1beta1/GalaxyBackup/galaxy/galaxybackup/project/6e617a9cce9144fd9e8a12974374efcd\u001b[0m\n\u001b[1;35m[WARNING]: Found variable using reserved name: no_log\u001b[0m\n\r\nPLAY [localhost] ***************************************************************\n\u001b[0;34
mMETA: ran handlers\u001b[0m\n\r\nTASK [backup : Obtain custom resource information] *****************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/main.yml:3\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"custom_resource\": {\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"kind\": \"GalaxyBackup\", \"metadata\": {\"a
nnotations\": {\"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"galaxy.ansible.com/v1beta1\\\",\\\"kind\\\":\\\"GalaxyBackup\\\",\\\"metadata\\\":{\\\"annotations\\\":{},\\\"name\\\":\\\"galaxybackup\\\",\\\"namespace\\\":\\\"galaxy\\\"},\\\"spec\\\":{\\\"backup_pvc\\\":\\\"galaxybackup-backup-claim\\\",\\\"backup_pvc_namespace\\
\":\\\"galaxy\\\",\\\"backup_storage_class\\\":\\\"nfs-backup-csi\\\",\\\"deployment_name\\\":\\\"galaxy\\\"}}\\n\"}, \"creationTimestamp\": \"2024-03-13T08:31:16Z\", \"generation\": 1, \"managedFields\": [{\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:kubectl.ku
bernetes.io/last-applied-configuration\": {}}}, \"f:spec\": {\".\": {}, \"f:backup_pvc\": {}, \"f:backup_pvc_namespace\": {}, \"f:backup_storage_class\": {}, \"f:deployment_name\": {}, \"f:no_log\": {}}}, \"manager\": \"kubectl-client-side-apply\", \"operation\": \"Update\", \"time\": \"2024-03-13T08:31:16Z\"}, {\"apiVersion\": \"galaxy.ansible.com/v1beta1\",
\"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\".\": {}, \"f:conditions\": {}}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-03-13T08:39:21Z\"}], \"name\": \"galaxybackup\", \"namespace\": \"galaxy\", \"resourceVersion\": \"18232842\", \"uid\": \"ef364c33-9370-4542-b955-a638fdd5100c\
"}, \"spec\": {\"backup_pvc\": \"galaxybackup-backup-claim\", \"backup_pvc_namespace\": \"galaxy\", \"backup_storage_class\": \"nfs-backup-csi\", \"deployment_name\": \"galaxy\", \"no_log\": true}, \"status\": {\"conditions\": [{\"lastTransitionTime\": \"2024-03-13T08:31:33Z\", \"reason\": \"\", \"status\": \"False\", \"type\": \"Successful\"}, {\"lastTransiti
onTime\": \"2024-03-13T08:39:21Z\", \"reason\": \"Failed\", \"status\": \"False\", \"type\": \"Failure\"}, {\"lastTransitionTime\": \"2024-03-13T08:39:21Z\", \"reason\": \"Running\", \"status\": \"True\", \"type\": \"Running\"}]}}, \"custom_resource_status\": {\"conditions\": [{\"lastTransitionTime\": \"2024-03-13T08:31:33Z\", \"reason\": \"\", \"status\": \"F
alse\", \"type\": \"Successful\"}, {\"lastTransitionTime\": \"2024-03-13T08:39:21Z\", \"reason\": \"Failed\", \"status\": \"False\", \"type\": \"Failure\"}, {\"lastTransitionTime\": \"2024-03-13T08:39:21Z\", \"reason\": \"Running\", \"status\": \"True\", \"type\": \"Running\"}]}}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : include_tasks] ****************
**********************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/main.yml:9\u001b[0m\n\u001b[0;34mstatically imported: /opt/ansible/roles/backup/tasks/error_handling.yml\u001b[0m\n\u001b[0;36mincluded: /opt/ansible/roles/backup/tasks/init.yml for localhost\u001b[0m\n\r\nTASK [backup : Delete any existing management pod] ************
*****************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:3\u001b[0m\n\u001b[0;33mchanged: [localhost] => {\"changed\": true, \"duration\": 35, \"method\": \"delete\", \"result\": {\"apiVersion\": \"v1\", \"kind\": \"Pod\", \"metadata\": {\"creationTimestamp\": \"2024-03-13T08:39:13Z\", \"deletionGracePeriodSeconds\": 30, \"deletionT
imestamp\": \"2024-03-13T08:39:54Z\", \"labels\": {\"app.kubernetes.io/component\": \"backup-manager\", \"app.kubernetes.io/instance\": \"galaxy-backup-manager-galaxybackup\", \"app.kubernetes.io/managed-by\": \"galaxy-operator\", \"app.kubernetes.io/name\": \"galaxy-backup-manager\", \"app.kubernetes.io/part-of\": \"galaxy\"}, \"managedFields\": [{\"apiVersio
n\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/instance\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/name\": {}, \"f:app.kubernetes.io/part-of\": {}}, \"f:ownerReferences\": {\".\": {}, \"k:{\\\"uid\\\":\\\"ef364c33-9370-45
42-b955-a638fdd5100c\\\"}\": {}}}, \"f:spec\": {\"f:containers\": {\"k:{\\\"name\\\":\\\"galaxybackup-backup-manager\\\"}\": {\".\": {}, \"f:command\": {}, \"f:image\": {}, \"f:imagePullPolicy\": {}, \"f:name\": {}, \"f:resources\": {}, \"f:terminationMessagePath\": {}, \"f:terminationMessagePolicy\": {}, \"f:volumeMounts\": {\".\": {}, \"k:{\\\"mountPath\\\":
\\\"/backups\\\"}\": {\".\": {}, \"f:mountPath\": {}, \"f:name\": {}}, \"k:{\\\"mountPath\\\":\\\"/var/lib/pulp\\\"}\": {\".\": {}, \"f:mountPath\": {}, \"f:name\": {}}}}}, \"f:dnsPolicy\": {}, \"f:enableServiceLinks\": {}, \"f:restartPolicy\": {}, \"f:schedulerName\": {}, \"f:securityContext\": {}, \"f:terminationGracePeriodSeconds\": {}, \"f:volumes\": {\".\
": {}, \"k:{\\\"name\\\":\\\"file-storage\\\"}\": {\".\": {}, \"f:name\": {}, \"f:persistentVolumeClaim\": {\".\": {}, \"f:claimName\": {}}}, \"k:{\\\"name\\\":\\\"galaxybackup-backup\\\"}\": {\".\": {}, \"f:name\": {}, \"f:persistentVolumeClaim\": {\".\": {}, \"f:claimName\": {}}}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"
2024-03-13T08:39:13Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:conditions\": {\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\\"type\\\":\\\"Initialized\\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:la
stTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\\"type\\\":\\\"PodReadyToStartContainers\\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\\"type\\\":\\\"Ready\\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}}, \"f:co
ntainerStatuses\": {}, \"f:hostIP\": {}, \"f:hostIPs\": {}, \"f:phase\": {}, \"f:podIP\": {}, \"f:podIPs\": {\".\": {}, \"k:{\\\"ip\\\":\\\"10.255.1.120\\\"}\": {\".\": {}, \"f:ip\": {}}}, \"f:startTime\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-03-13T08:39:15Z\"}], \"name\": \"galaxybackup-backup-ma
nager\", \"namespace\": \"galaxy\", \"ownerReferences\": [{\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"kind\": \"GalaxyBackup\", \"name\": \"galaxybackup\", \"uid\": \"ef364c33-9370-4542-b955-a638fdd5100c\"}], \"resourceVersion\": \"18232862\", \"uid\": \"89141d01-df6f-4393-9d77-048715939a29\"}, \"spec\": {\"containers\": [{\"command\": [\"sleep\", \"inf
inity\"], \"image\": \"postgres:13\", \"imagePullPolicy\": \"Always\", \"name\": \"galaxybackup-backup-manager\", \"resources\": {}, \"terminationMessagePath\": \"/dev/termination-log\", \"terminationMessagePolicy\": \"File\", \"volumeMounts\": [{\"mountPath\": \"/backups\", \"name\": \"galaxybackup-backup\"}, {\"mountPath\": \"/var/lib/pulp\", \"name\": \"fil
e-storage\"}, {\"mountPath\": \"/var/run/secrets/kubernetes.io/serviceaccount\", \"name\": \"kube-api-access-8ggjl\", \"readOnly\": true}]}], \"dnsPolicy\": \"ClusterFirst\", \"enableServiceLinks\": true, \"nodeName\": \"stlconfmn400165\", \"preemptionPolicy\": \"PreemptLowerPriority\", \"priority\": 0, \"restartPolicy\": \"Never\", \"schedulerName\": \"defaul
t-scheduler\", \"securityContext\": {}, \"serviceAccount\": \"default\", \"serviceAccountName\": \"default\", \"terminationGracePeriodSeconds\": 30, \"tolerations\": [{\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/not-ready\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}, {\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/unreachable
\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}], \"volumes\": [{\"name\": \"galaxybackup-backup\", \"persistentVolumeClaim\": {\"claimName\": \"galaxybackup-backup-claim\"}}, {\"name\": \"file-storage\", \"persistentVolumeClaim\": {\"claimName\": \"galaxy-file-storage\"}}, {\"name\": \"kube-api-access-8ggjl\", \"projected\": {\"defaultMode\": 420, \
"sources\": [{\"serviceAccountToken\": {\"expirationSeconds\": 3607, \"path\": \"token\"}}, {\"configMap\": {\"items\": [{\"key\": \"ca.crt\", \"path\": \"ca.crt\"}], \"name\": \"kube-root-ca.crt\"}}, {\"downwardAPI\": {\"items\": [{\"fieldRef\": {\"apiVersion\": \"v1\", \"fieldPath\": \"metadata.namespace\"}, \"path\": \"namespace\"}]}}]}}]}, \"status\": {\"c
onditions\": [{\"lastProbeTime\": null, \"lastTransitionTime\": \"2024-03-13T08:39:15Z\", \"status\": \"True\", \"type\": \"PodReadyToStartContainers\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2024-03-13T08:39:13Z\", \"status\": \"True\", \"type\": \"Initialized\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2024-03-13T08:39:15Z\", \"stat
us\": \"True\", \"type\": \"Ready\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2024-03-13T08:39:15Z\", \"status\": \"True\", \"type\": \"ContainersReady\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2024-03-13T08:39:13Z\", \"status\": \"True\", \"type\": \"PodScheduled\"}], \"containerStatuses\": [{\"containerID\": \"containerd://85b6450e2
fab02c797e9b9c8675de1988a2e1b4d0a75e2791ff0d5d124fe767d\", \"image\": \"docker.io/library/postgres:13\", \"imageID\": \"docker.io/library/postgres@sha256:337b6fff599687159acf150b7f32bd62b761608a39f749ac2100efacc9cd986c\", \"lastState\": {}, \"name\": \"galaxybackup-backup-manager\", \"ready\": true, \"restartCount\": 0, \"started\": true, \"state\": {\"running
\": {\"startedAt\": \"2024-03-13T08:39:15Z\"}}}], \"hostIP\": \"10.247.0.212\", \"hostIPs\": [{\"ip\": \"10.247.0.212\"}], \"phase\": \"Running\", \"podIP\": \"10.255.1.120\", \"podIPs\": [{\"ip\": \"10.255.1.120\"}], \"qosClass\": \"BestEffort\", \"startTime\": \"2024-03-13T08:39:13Z\"}}}\u001b[0m\n\r\nTASK [backup : Check provided PVC exists] ***************
***********************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:13\u001b[0m\n\u001b[0;32mok: [localhost] => {\"api_found\": true, \"changed\": false, \"resources\": [{\"apiVersion\": \"v1\", \"kind\": \"PersistentVolumeClaim\", \"metadata\": {\"annotations\": {\"pv.kubernetes.io/bind-completed\": \"yes\", \"pv.kubernetes.io/bound-by-
controller\": \"yes\", \"volume.beta.kubernetes.io/storage-provisioner\": \"nfs.csi.k8s.io\", \"volume.kubernetes.io/storage-provisioner\": \"nfs.csi.k8s.io\"}, \"creationTimestamp\": \"2024-03-13T08:24:07Z\", \"finalizers\": [\"kubernetes.io/pvc-protection\"], \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:spec\"
: {\"f:accessModes\": {}, \"f:resources\": {\"f:requests\": {\".\": {}, \"f:storage\": {}}}, \"f:storageClassName\": {}, \"f:volumeMode\": {}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2024-03-13T08:24:07Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {},
 \"f:pv.kubernetes.io/bind-completed\": {}, \"f:pv.kubernetes.io/bound-by-controller\": {}, \"f:volume.beta.kubernetes.io/storage-provisioner\": {}, \"f:volume.kubernetes.io/storage-provisioner\": {}}}, \"f:spec\": {\"f:volumeName\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"time\": \"2024-03-13T08:24:07Z\"}, {\"apiVersion\": \"v1\", \"fieldsTyp
e\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:accessModes\": {}, \"f:capacity\": {\".\": {}, \"f:storage\": {}}, \"f:phase\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-03-13T08:24:07Z\"}], \"name\": \"galaxybackup-backup-claim\", \"namespace\": \"galaxy\", \"resourceVersion\": \"18226813\", \"ui
d\": \"cb8f5189-e445-4870-b343-4fc7fc37ad28\"}, \"spec\": {\"accessModes\": [\"ReadWriteMany\"], \"resources\": {\"requests\": {\"storage\": \"10Gi\"}}, \"storageClassName\": \"nfs-backup-csi\", \"volumeMode\": \"Filesystem\", \"volumeName\": \"pvc-cb8f5189-e445-4870-b343-4fc7fc37ad28\"}, \"status\": {\"accessModes\": [\"ReadWriteMany\"], \"capacity\": {\"stor
age\": \"10Gi\"}, \"phase\": \"Bound\"}}]}\u001b[0m\n\r\nTASK [backup : Set error message] **********************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:24\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [backup : Set ap
iVersion and kind variables] ******************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/error_handling.yml:3\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [backup : Determine the timestamp] ****************************************\r\n\u001b
[1;30mtask path: /opt/ansible/roles/backup/tasks/error_handling.yml:8\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [backup : Emit ocp event with error] **************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/error_handling.yml:1
2\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [backup : Fail early if pvc is defined but does not exist] ****************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:31\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_r
eason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [backup : Set default pvc name] ****************************************
2024-03-13T08:40:13.834059236Z stderr P ***\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:39\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"_default_backup_pvc\": \"galaxy-backup-claim\"}, \"changed\": false}\u001b[0m\n\r\nTASK [Set PVC to use for backup] ***********************************************\r\n\u001b[1;30mtask
path: /opt/ansible/roles/backup/tasks/init.yml:44\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"backup_claim\": \"galaxybackup-backup-claim\"}, \"changed\": false}\u001b[0m\n\r\nTASK [Create persistent volume claim for backup] *******************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:48\u001b[0m\n\u001
b[0;36mskipping: [localhost] => (item=backup)  => {\"ansible_loop_var\": \"item\", \"changed\": false, \"item\": \"backup\", \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [Remove ownerReferences from backup-claim pvc to avoid garbage collection] ***\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:57\u001b[0m\n\u001b[0
;32mok: [localhost] => {\"changed\": false, \"method\": \"patch\", \"result\": {\"apiVersion\": \"v1\", \"kind\": \"PersistentVolumeClaim\", \"metadata\": {\"annotations\": {\"pv.kubernetes.io/bind-completed\": \"yes\", \"pv.kubernetes.io/bound-by-controller\": \"yes\", \"volume.beta.kubernetes.io/storage-provisioner\": \"nfs.csi.k8s.io\", \"volume.kubernetes.
io/storage-provisioner\": \"nfs.csi.k8s.io\"}, \"creationTimestamp\": \"2024-03-13T08:24:07Z\", \"finalizers\": [\"kubernetes.io/pvc-protection\"], \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:spec\": {\"f:accessModes\": {}, \"f:resources\": {\"f:requests\": {\".\": {}, \"f:storage\": {}}}, \"f:storageClassName\
": {}, \"f:volumeMode\": {}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2024-03-13T08:24:07Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:pv.kubernetes.io/bind-completed\": {}, \"f:pv.kubernetes.io/bound-by-controller\": {}, \"f:volume.beta.kubern
etes.io/storage-provisioner\": {}, \"f:volume.kubernetes.io/storage-provisioner\": {}}}, \"f:spec\": {\"f:volumeName\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"time\": \"2024-03-13T08:24:07Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:accessModes\": {}, \"f:capacity\": {\".\": {}, \"f:storage\":
 {}}, \"f:phase\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-03-13T08:24:07Z\"}], \"name\": \"galaxybackup-backup-claim\", \"namespace\": \"galaxy\", \"resourceVersion\": \"18226813\", \"uid\": \"cb8f5189-e445-4870-b343-4fc7fc37ad28\"}, \"spec\": {\"accessModes\": [\"ReadWriteMany\"], \"resources\": {\
"requests\": {\"storage\": \"10Gi\"}}, \"storageClassName\": \"nfs-backup-csi\", \"volumeMode\": \"Filesystem\", \"volumeName\": \"pvc-cb8f5189-e445-4870-b343-4fc7fc37ad28\"}, \"status\": {\"accessModes\": [\"ReadWriteMany\"], \"capacity\": {\"storage\": \"10Gi\"}, \"phase\": \"Bound\"}}}\u001b[0m\n\r\nTASK [backup : Get Galaxy custom resource object] ********
**********************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:67\u001b[0m\n\u001b[0;32mok: [localhost] => {\"api_found\": true, \"changed\": false, \"resources\": [{\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"kind\": \"Galaxy\", \"metadata\": {\"creationTimestamp\": \"2024-03-13T07:44:47Z\", \"generation\": 1, \"managedFields\
": [{\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:spec\": {\".\": {}, \"f:api\": {\".\": {}, \"f:log_level\": {}, \"f:replicas\": {}}, \"f:content\": {\".\": {}, \"f:log_level\": {}, \"f:replicas\": {}}, \"f:file_storage_access_mode\": {}, \"f:file_storage_size\": {}, \"f:force_drop_db\": {}, \"f:gunicorn_api
_workers\": {}, \"f:gunicorn_content_workers\": {}, \"f:gunicorn_timeout\": {}, \"f:haproxy_timeout\": {}, \"f:hostname\": {}, \"f:image_pull_policy\": {}, \"f:ingress_type\": {}, \"f:loadbalancer_port\": {}, \"f:loadbalancer_protocol\": {}, \"f:nginx_client_max_body_size\": {}, \"f:nginx_proxy_connect_timeout\": {}, \"f:nginx_proxy_read_timeout\": {}, \"f:ngi
nx_proxy_send_timeout\": {}, \"f:no_log\": {}, \"f:postgres_configuration_secret\": {}, \"f:route_tls_termination_mechanism\": {}, \"f:storage_type\": {}, \"f:web\": {\".\": {}, \"f:replicas\": {}}, \"f:worker\": {\".\": {}, \"f:replicas\": {}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2024-03-13T07:44:47Z\"}, {\"apiVersion\
": \"galaxy.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\".\": {}, \"f:adminPasswordSecret\": {}, \"f:conditions\": {}, \"f:containerTokenSecret\": {}, \"f:databaseConfigurationSecret\": {}, \"f:dbFieldsEncryptionSecret\": {}, \"f:deployedImage\": {}, \"f:storagePersistentVolumeClaim\": {}, \"f:storageType\": {}}}, \"mana
ger\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-03-13T07:50:02Z\"}], \"name\": \"galaxy\", \"namespace\": \"galaxy\", \"resourceVersion\": \"18213358\", \"uid\": \"08f3dec1-ea8a-41ec-b926-1edb975abb82\"}, \"spec\": {\"api\": {\"log_level\": \"INFO\", \"replicas\": 2}, \"content\": {\"log_level\": \"INFO\",
 \"replicas\": 2}, \"file_storage_access_mode\": \"ReadWriteOnce\", \"file_storage_size\": \"8Gi\", \"force_drop_db\": false, \"gunicorn_api_workers\": 2, \"gunicorn_content_workers\": 2, \"gunicorn_timeout\": 90, \"haproxy_timeout\": \"180s\", \"hostname\": \"galaxy-kube.shared-services.non-prod.azr-non-prod.swm.de\", \"image_pull_policy\": \"IfNotPresent\",
\"ingress_type\": \"ingress\", \"loadbalancer_port\": 80, \"loadbalancer_protocol\": \"http\", \"nginx_client_max_body_size\": \"10m\", \"nginx_proxy_connect_timeout\": \"120s\", \"nginx_proxy_read_timeout\": \"120s\", \"nginx_proxy_send_timeout\": \"120s\", \"no_log\": false, \"postgres_configuration_secret\": \"galaxy-postgres-configuration\", \"route_tls_te
rmination_mechanism\": \"Edge\", \"storage_type\": \"File\", \"web\": {\"replicas\": 2}, \"worker\": {\"replicas\": 2}}, \"status\": {\"adminPasswordSecret\": \"galaxy-admin-password\", \"conditions\": [{\"lastTransitionTime\": \"2024-03-13T07:49:19+00:00\", \"message\": \"All Postgres tasks ran successfully\", \"reason\": \"DatabaseTasksFinished\", \"status\"
: \"True\", \"type\": \"Database-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:51+00:00\", \"message\": \"All API tasks ran successfully\", \"reason\": \"ApiTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-API-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:50:02+00:00\", \"message\": \"All tasks ran successfully\", \"reason\": \"OperatorFi
nishedExecution\", \"status\": \"True\", \"type\": \"Galaxy-Operator-Finished-Execution\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:25+00:00\", \"message\": \"All Galaxy-web tasks ran successfully\", \"reason\": \"WebTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Web-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:32+00:00\", \"message\":
\"All Galaxy-content tasks ran successfully\", \"reason\": \"ContentTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Content-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:37+00:00\", \"message\": \"All Galaxy-worker tasks ran successfully\", \"reason\": \"WorkerTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Worker-Ready\"}, {\"lastT
ransitionTime\": \"2024-03-13T07:48:17+00:00\", \"message\": \"Checking routes\", \"reason\": \"CheckingRoutes\", \"status\": \"True\", \"type\": \"Galaxy-Routes-Ready\"}], \"containerTokenSecret\": \"galaxy-container-auth\", \"databaseConfigurationSecret\": \"galaxy-postgres-configuration\", \"dbFieldsEncryptionSecret\": \"galaxy-db-fields-encryption\", \"dep
loyedImage\": \"quay.io/ansible/galaxy-ng:latest\", \"storagePersistentVolumeClaim\": \"galaxy-file-storage\", \"storageType\": \"File\"}}]}\u001b[0m\n\r\nTASK [backup : Set Galaxy object] **********************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:75\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_fact
s\": {\"_galaxy\": {\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"kind\": \"Galaxy\", \"metadata\": {\"creationTimestamp\": \"2024-03-13T07:44:47Z\", \"generation\": 1, \"managedFields\": [{\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:spec\": {\".\": {}, \"f:api\": {\".\": {}, \"f:log_level\": {}, \"f:rep
licas\": {}}, \"f:content\": {\".\": {}, \"f:log_level\": {}, \"f:replicas\": {}}, \"f:file_storage_access_mode\": {}, \"f:file_storage_size\": {}, \"f:force_drop_db\": {}, \"f:gunicorn_api_workers\": {}, \"f:gunicorn_content_workers\": {}, \"f:gunicorn_timeout\": {}, \"f:haproxy_timeout\": {}, \"f:hostname\": {}, \"f:image_pull_policy\": {}, \"f:ingress_type\
": {}, \"f:loadbalancer_port\": {}, \"f:loadbalancer_protocol\": {}, \"f:nginx_client_max_body_size\": {}, \"f:nginx_proxy_connect_timeout\": {}, \"f:nginx_proxy_read_timeout\": {}, \"f:nginx_proxy_send_timeout\": {}, \"f:no_log\": {}, \"f:postgres_configuration_secret\": {}, \"f:route_tls_termination_mechanism\": {}, \"f:storage_type\": {}, \"f:web\": {\".\":
 {}, \"f:replicas\": {}}, \"f:worker\": {\".\": {}, \"f:replicas\": {}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2024-03-13T07:44:47Z\"}, {\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\".\": {}, \"f:adminPasswordSecret\": {}, \"f:conditions\": {}, \"f:containerT
okenSecret\": {}, \"f:databaseConfigurationSecret\": {}, \"f:dbFieldsEncryptionSecret\": {}, \"f:deployedImage\": {}, \"f:storagePersistentVolumeClaim\": {}, \"f:storageType\": {}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-03-13T07:50:02Z\"}], \"name\": \"galaxy\", \"namespace\": \"galaxy\",
\"resourceVersion\": \"18213358\", \"uid\": \"08f3dec1-ea8a-41ec-b926-1edb975abb82\"}, \"spec\": {\"api\": {\"log_level\": \"INFO\", \"replicas\": 2}, \"content\": {\"log_level\": \"INFO\", \"replicas\": 2}, \"file_storage_access_mode\": \"ReadWriteOnce\", \"file_storage_size\": \"8Gi\", \"force_drop_db\": false, \"gunicorn_api_workers\": 2, \"gunicorn_content
_workers\": 2, \"gunicorn_timeout\": 90, \"haproxy_timeout\": \"180s\", \"hostname\": \"galaxy-kube.shared-services.non-prod.azr-non-prod.swm.de\", \"image_pull_policy\": \"IfNotPresent\", \"ingress_type\": \"ingress\", \"loadbalancer_port\": 80, \"loadbalancer_protocol\": \"http\", \"nginx_client_max_body_size\": \"10m\", \"nginx_proxy_connect_timeout\": \"12
0s\", \"nginx_proxy_read_timeout\": \"120s\", \"nginx_proxy_send_timeout\": \"120s\", \"no_log\": false, \"postgres_configuration_secret\": \"galaxy-postgres-configuration\", \"route_tls_termination_mechanism\": \"Edge\", \"storage_type\": \"File\", \"web\": {\"replicas\": 2}, \"worker\": {\"replicas\": 2}}, \"status\": {\"adminPasswordSecret\": \"galaxy-admin
-password\", \"conditions\": [{\"lastTransitionTime\": \"2024-03-13T07:49:19+00:00\", \"message\": \"All Postgres tasks ran successfully\", \"reason\": \"DatabaseTasksFinished\", \"status\": \"True\", \"type\": \"Database-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:51+00:00\", \"message\": \"All API tasks ran successfully\", \"reason\": \"ApiTasksFin
ished\", \"status\": \"True\", \"type\": \"Galaxy-API-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:50:02+00:00\", \"message\": \"All tasks ran successfully\", \"reason\": \"OperatorFinishedExecution\", \"status\": \"True\", \"type\": \"Galaxy-Operator-Finished-Execution\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:25+00:00\", \"message\": \"All Galaxy
-web tasks ran successfully\", \"reason\": \"WebTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Web-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:32+00:00\", \"message\": \"All Galaxy-content tasks ran successfully\", \"reason\": \"ContentTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Content-Ready\"}, {\"lastTransitionTime\": \"20
24-03-13T07:49:37+00:00\", \"message\": \"All Galaxy-worker tasks ran successfully\", \"reason\": \"WorkerTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Worker-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:48:17+00:00\", \"message\": \"Checking routes\", \"reason\": \"CheckingRoutes\", \"status\": \"True\", \"type\": \"Galaxy-Routes-Ready\"}],
\"containerTokenSecret\": \"galaxy-container-auth\", \"databaseConfigurationSecret\": \"galaxy-postgres-configuration\", \"dbFieldsEncryptionSecret\": \"galaxy-db-fields-encryption\", \"deployedImage\": \"quay.io/ansible/galaxy-ng:latest\", \"storagePersistentVolumeClaim\": \"galaxy-file-storage\", \"storageType\": \"File\"}}}, \"changed\": false}\u001b[0m\n\r
\nTASK [backup : Set apiVersion] *************************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:79\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"api_version\": \"galaxy.ansible.com/v1beta1\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set user specified spec] ***********************
*****************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:83\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"cr_spec\": {\"api\": {\"log_level\": \"INFO\", \"replicas\": 2}, \"content\": {\"log_level\": \"INFO\", \"replicas\": 2}, \"file_storage_access_mode\": \"ReadWriteOnce\", \"file_storage_size\": \"8Gi\", \"force
_drop_db\": false, \"gunicorn_api_workers\": 2, \"gunicorn_content_workers\": 2, \"gunicorn_timeout\": 90, \"haproxy_timeout\": \"180s\", \"hostname\": \"galaxy-kube.shared-services.non-prod.azr-non-prod.swm.de\", \"image_pull_policy\": \"IfNotPresent\", \"ingress_type\": \"ingress\", \"loadbalancer_port\": 80, \"loadbalancer_protocol\": \"http\", \"nginx_clie
nt_max_body_size\": \"10m\", \"nginx_proxy_connect_timeout\": \"120s\", \"nginx_proxy_read_timeout\": \"120s\", \"nginx_proxy_send_timeout\": \"120s\", \"no_log\": false, \"postgres_configuration_secret\": \"galaxy-postgres-configuration\", \"route_tls_termination_mechanism\": \"Edge\", \"storage_type\": \"File\", \"web\": {\"replicas\": 2}, \"worker\": {\"rep
licas\": 2}}}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set galaxy status] **********************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:87\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"status\": {\"adminPasswordSecret\": \"galaxy-admin-password\", \"conditions\": [{\"lastTransitionT
ime\": \"2024-03-13T07:49:19+00:00\", \"message\": \"All Postgres tasks ran successfully\", \"reason\": \"DatabaseTasksFinished\", \"status\": \"True\", \"type\": \"Database-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:51+00:00\", \"message\": \"All API tasks ran successfully\", \"reason\": \"ApiTasksFinished\", \"status\": \"True\", \"type\": \"Galax
y-API-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:50:02+00:00\", \"message\": \"All tasks ran successfully\", \"reason\": \"OperatorFinishedExecution\", \"status\": \"True\", \"type\": \"Galaxy-Operator-Finished-Execution\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:25+00:00\", \"message\": \"All Galaxy-web tasks ran successfully\", \"reason\": \"Web
TasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Web-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:32+00:00\", \"message\": \"All Galaxy-content tasks ran successfully\", \"reason\": \"ContentTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Content-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:49:37+00:00\", \"message\": \"All Ga
laxy-worker tasks ran successfully\", \"reason\": \"WorkerTasksFinished\", \"status\": \"True\", \"type\": \"Galaxy-Worker-Ready\"}, {\"lastTransitionTime\": \"2024-03-13T07:48:17+00:00\", \"message\": \"Checking routes\", \"reason\": \"CheckingRoutes\", \"status\": \"True\", \"type\": \"Galaxy-Routes-Ready\"}], \"containerTokenSecret\": \"galaxy-container-aut
h\", \"databaseConfigurationSecret\": \"galaxy-postgres-configuration\", \"dbFieldsEncryptionSecret\": \"galaxy-db-fields-encryption\"
2024-03-13T08:40:13.834076685Z stderr P , \"deployedImage\": \"quay.io/ansible/galaxy-ng:latest\", \"storagePersistentVolumeClaim\": \"galaxy-file-storage\", \"storageType\": \"File\"}}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set galaxy admin secret if found] *******************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/in
it.yml:91\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"admin_password_secret\": \"galaxy-admin-password\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set galaxy database configuration secret if found] **************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:98\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansibl
e_facts\": {\"postgres_configuration_secret\": \"galaxy-postgres-configuration\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set galaxy storage type if found] *******************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:105\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"storage_type\": \"File\"}, \"
changed\": false}\u001b[0m\n\r\nTASK [backup : Set galaxy file storage claim if found] *************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:112\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"storage_claim\": \"galaxy-file-storage\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set galaxy object stor
age secret if found] **********************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:119\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [backup : Set DB fields encryption secret if found] ***********************\r\n\u001b[1;30mtask path: /opt/an
sible/roles/backup/tasks/init.yml:126\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"db_fields_encryption_secret\": \"galaxy-db-fields-encryption\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set galaxy signing secret if found] *****************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:133\u001b[0m\
n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [backup : Set container token secret if found] ****************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:140\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"container_token_secret
\": \"galaxy-container-auth\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set SSO secret if found] ****************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:147\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [backu
p : Get PVC information] ********************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:154\u001b[0m\n\u001b[0;32mok: [localhost] => {\"api_found\": true, \"changed\": false, \"resources\": [{\"apiVersion\": \"v1\", \"kind\": \"PersistentVolumeClaim\", \"metadata\": {\"annotations\": {\"pv.kubernetes.io/bind-com
pleted\": \"yes\", \"pv.kubernetes.io/bound-by-controller\": \"yes\", \"volume.beta.kubernetes.io/storage-provisioner\": \"nfs.csi.k8s.io\", \"volume.kubernetes.io/storage-provisioner\": \"nfs.csi.k8s.io\"}, \"creationTimestamp\": \"2024-03-13T07:45:17Z\", \"finalizers\": [\"kubernetes.io/pvc-protection\"], \"labels\": {\"app.kubernetes.io/component\": \"stora
ge\", \"app.kubernetes.io/instance\": \"galaxy-storage-galaxy\", \"app.kubernetes.io/managed-by\": \"galaxy-operator\", \"app.kubernetes.io/name\": \"galaxy-storage\", \"app.kubernetes.io/part-of\": \"galaxy\"}, \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernete
s.io/component\": {}, \"f:app.kubernetes.io/instance\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/name\": {}, \"f:app.kubernetes.io/part-of\": {}}}, \"f:spec\": {\"f:accessModes\": {}, \"f:resources\": {\"f:requests\": {\".\": {}, \"f:storage\": {}}}, \"f:volumeMode\": {}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\",
 \"time\": \"2024-03-13T07:45:17Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:pv.kubernetes.io/bind-completed\": {}, \"f:pv.kubernetes.io/bound-by-controller\": {}, \"f:volume.beta.kubernetes.io/storage-provisioner\": {}, \"f:volume.kubernetes.io/storage-provisioner\": {}}}, \"f:
spec\": {\"f:volumeName\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"time\": \"2024-03-13T07:45:17Z\"}, {\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:accessModes\": {}, \"f:capacity\": {\".\": {}, \"f:storage\": {}}, \"f:phase\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"
status\", \"time\": \"2024-03-13T07:45:17Z\"}], \"name\": \"galaxy-file-storage\", \"namespace\": \"galaxy\", \"resourceVersion\": \"18211268\", \"uid\": \"18a866a1-8115-4a4f-a677-d7eca335f992\"}, \"spec\": {\"accessModes\": [\"ReadWriteOnce\"], \"resources\": {\"requests\": {\"storage\": \"8Gi\"}}, \"storageClassName\": \"nfs-csi\", \"volumeMode\": \"Filesyst
em\", \"volumeName\": \"pvc-18a866a1-8115-4a4f-a677-d7eca335f992\"}, \"status\": {\"accessModes\": [\"ReadWriteOnce\"], \"capacity\": {\"storage\": \"8Gi\"}, \"phase\": \"Bound\"}}]}\u001b[0m\n\r\nTASK [backup : Set storage claim access mode] **********************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:162\u001b[0m\n\u0
01b[0;32mok: [localhost] => {\"ansible_facts\": {\"pvc_access_mode\": \"ReadWriteOnce\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set user provided postgres image] *******************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:171\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \
"Conditional result was False\"}\u001b[0m\n\r\nTASK [backup : Set Postgres image URL] *****************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:177\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"_postgres_image\": \"postgres:13\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Create manage
ment pod from templated deployment config] *********\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/init.yml:181\u001b[0m\n\u001b[0;33mchanged: [localhost] => {\"changed\": true, \"duration\": 5, \"method\": \"create\", \"result\": {\"apiVersion\": \"v1\", \"kind\": \"Pod\", \"metadata\": {\"creationTimestamp\": \"2024-03-13T08:40:05Z\", \"labels\":
 {\"app.kubernetes.io/component\": \"backup-manager\", \"app.kubernetes.io/instance\": \"galaxy-backup-manager-galaxybackup\", \"app.kubernetes.io/managed-by\": \"galaxy-operator\", \"app.kubernetes.io/name\": \"galaxy-backup-manager\", \"app.kubernetes.io/part-of\": \"galaxy\"}, \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fiel
dsV1\": {\"f:metadata\": {\"f:labels\": {\".\": {}, \"f:app.kubernetes.io/component\": {}, \"f:app.kubernetes.io/instance\": {}, \"f:app.kubernetes.io/managed-by\": {}, \"f:app.kubernetes.io/name\": {}, \"f:app.kubernetes.io/part-of\": {}}, \"f:ownerReferences\": {\".\": {}, \"k:{\\\"uid\\\":\\\"ef364c33-9370-4542-b955-a638fdd5100c\\\"}\": {}}}, \"f:spec\": {\
"f:containers\": {\"k:{\\\"name\\\":\\\"galaxybackup-backup-manager\\\"}\": {\".\": {}, \"f:command\": {}, \"f:image\": {}, \"f:imagePullPolicy\": {}, \"f:name\": {}, \"f:resources\": {}, \"f:terminationMessagePath\": {}, \"f:terminationMessagePolicy\": {}, \"f:volumeMounts\": {\".\": {}, \"k:{\\\"mountPath\\\":\\\"/backups\\\"}\": {\".\": {}, \"f:mountPath\":
 {}, \"f:name\": {}}, \"k:{\\\"mountPath\\\":\\\"/var/lib/pulp\\\"}\": {\".\": {}, \"f:mountPath\": {}, \"f:name\": {}}}}}, \"f:dnsPolicy\": {}, \"f:enableServiceLinks\": {}, \"f:restartPolicy\": {}, \"f:schedulerName\": {}, \"f:securityContext\": {}, \"f:terminationGracePeriodSeconds\": {}, \"f:volumes\": {\".\": {}, \"k:{\\\"name\\\":\\\"file-storage\\\"}\":
 {\".\": {}, \"f:name\": {}, \"f:persistentVolumeClaim\": {\".\": {}, \"f:claimName\": {}}}, \"k:{\\\"name\\\":\\\"galaxybackup-backup\\\"}\": {\".\": {}, \"f:name\": {}, \"f:persistentVolumeClaim\": {\".\": {}, \"f:claimName\": {}}}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2024-03-13T08:40:05Z\"}, {\"apiVersion\": \"v1\",
 \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:conditions\": {\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\\"type\\\":\\\"Initialized\\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:typ
e\": {}}, \"k:{\\\"type\\\":\\\"PodReadyToStartContainers\\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}, \"k:{\\\"type\\\":\\\"Ready\\\"}\": {\".\": {}, \"f:lastProbeTime\": {}, \"f:lastTransitionTime\": {}, \"f:status\": {}, \"f:type\": {}}}, \"f:containerStatuses\": {}, \"f:hostIP\": {}, \"f:host
IPs\": {}, \"f:phase\": {}, \"f:podIP\": {}, \"f:podIPs\": {\".\": {}, \"k:{\\\"ip\\\":\\\"10.255.1.121\\\"}\": {\".\": {}, \"f:ip\": {}}}, \"f:startTime\": {}}}, \"manager\": \"k3s\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-03-13T08:40:07Z\"}], \"name\": \"galaxybackup-backup-manager\", \"namespace\": \"galaxy\", \"ownerRefere
nces\": [{\"apiVersion\": \"galaxy.ansible.com/v1beta1\", \"kind\": \"GalaxyBackup\", \"name\": \"galaxybackup\", \"uid\": \"ef364c33-9370-4542-b955-a638fdd5100c\"}], \"resourceVersion\": \"18233143\", \"uid\": \"34ddd1d6-b183-4520-af98-5a86f37c9b7e\"}, \"spec\": {\"containers\": [{\"command\": [\"sleep\", \"infinity\"], \"image\": \"postgres:13\", \"imagePull
Policy\": \"Always\", \"name\": \"galaxybackup-backup-manager\", \"resources\": {}, \"terminationMessagePath\": \"/dev/termination-log\", \"terminationMessagePolicy\": \"File\", \"volumeMounts\": [{\"mountPath\": \"/backups\", \"name\": \"galaxybackup-backup\"}, {\"mountPath\": \"/var/lib/pulp\", \"name\": \"file-storage\"}, {\"mountPath\": \"/var/run/secrets/
kubernetes.io/serviceaccount\", \"name\": \"kube-api-access-k5qp7\", \"readOnly\": true}]}], \"dnsPolicy\": \"ClusterFirst\", \"enableServiceLinks\": true, \"nodeName\": \"stlconfmn400165\", \"preemptionPolicy\": \"PreemptLowerPriority\", \"priority\": 0, \"restartPolicy\": \"Never\", \"schedulerName\": \"default-scheduler\", \"securityContext\": {}, \"service
Account\": \"default\", \"serviceAccountName\": \"default\", \"terminationGracePeriodSeconds\": 30, \"tolerations\": [{\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/not-ready\", \"operator\": \"Exists\", \"tolerationSeconds\": 300}, {\"effect\": \"NoExecute\", \"key\": \"node.kubernetes.io/unreachable\", \"operator\": \"Exists\", \"tolerationSeconds
\": 300}], \"volumes\": [{\"name\": \"galaxybackup-backup\", \"persistentVolumeClaim\": {\"claimName\": \"galaxybackup-backup-claim\"}}, {\"name\": \"file-storage\", \"persistentVolumeClaim\": {\"claimName\": \"galaxy-file-storage\"}}, {\"name\": \"kube-api-access-k5qp7\", \"projected\": {\"defaultMode\": 420, \"sources\": [{\"serviceAccountToken\": {\"expirat
ionSeconds\": 3607, \"path\": \"token\"}}, {\"configMap\": {\"items\": [{\"key\": \"ca.crt\", \"path\": \"ca.crt\"}], \"name\": \"kube-root-ca.crt\"}}, {\"downwardAPI\": {\"items\": [{\"fieldRef\": {\"apiVersion\": \"v1\", \"fieldPath\": \"metadata.namespace\"}, \"path\": \"namespace\"}]}}]}}]}, \"status\": {\"conditions\": [{\"lastProbeTime\": null, \"lastTra
nsitionTime\": \"2024-03-13T08:40:07Z\", \"status\": \"True\", \"type\": \"PodReadyToStartContainers\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2024-03-13T08:40:05Z\", \"status\": \"True\", \"type\": \"Initialized\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2024-03-13T08:40:07Z\", \"status\": \"True\", \"type\": \"Ready\"}, {\"lastProb
eTime\": null, \"lastTransitionTime\": \"2024-03-13T08:40:07Z\", \"status\": \"True\", \"type\": \"ContainersReady\"}, {\"lastProbeTime\": null, \"lastTransitionTime\": \"2024-03-13T08:40:05Z\", \"status\": \"True\", \"type\": \"PodScheduled\"}], \"containerStatuses\": [{\"containerID\": \"containerd://cbf23043766c35c7976f3b047a8f70c3f95978436a1c213abb6f5f8a79
0844b9\", \"image\": \"docker.io/library/postgres:13\", \"imageID\": \"docker.io/library/postgres@sha256:337b6fff599687159acf150b7f32bd62b761608a39f749ac2100efacc9cd986c\", \"lastState\": {}, \"name\": \"galaxybackup-backup-manager\", \"ready\": true, \"restartCount\": 0, \"started\": true, \"state\": {\"running\": {\"startedAt\": \"2024-03-13T08:40:07Z\"}}}],
 \"hostIP\": \"10.247.0.212\", \"hostIPs\": [{\"ip\": \"10.247.0.212\"}], \"phase\": \"Running\", \"podIP\": \"10.255.1.121\", \"podIPs\": [{\"ip\": \"10.255.1.121\"}], \"qosClass\": \"BestEffort\", \"startTime\": \"2024-03-13T08:40:05Z\"}}}\u001b[0m\n\r\nTASK [backup : include_tasks] **************************************************\r\n\u001b[1;30mtask path:
 /opt/ansible/roles/backup/tasks/main.yml:11\u001b[0m\n\u001b[0;36mincluded: /opt/ansible/roles/backup/tasks/postgres.yml for localhost\u001b[0m\n\r\nTASK [backup : Check for specified PostgreSQL configuration] *******************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/postgres.yml:3\u001b[0m\n\u001b[0;32mok: [localhost] => {\"api_found\": t
rue, \"changed\": false, \"resources\": [{\"apiVersion\": \"v1\", \"data\": {\"database\": \"Z2FsYXh5\", \"host\": \"cGdzbWZ5MDEuc2hhcmVkLXNlcnZpY2VzLm5vbi1wcm9kLmF6ci1ub24tcHJvZC5zd20uZGU=\", \"password\": \"M2pQZ2JGZnl5V3dQYXdQdGZDcWZ6ZmhQdHlrNEJL\", \"port\": \"NTQzMg==\", \"sslmode\": \"cHJlZmVy\", \"type\": \"dW5tYW5hZ2Vk\", \"username\": \"Z2FsYXh5\"}, \
"immutable\": true, \"kind\": \"Secret\", \"metadata\": {\"creationTimestamp\": \"2024-03-13T07:44:47Z\", \"managedFields\": [{\"apiVersion\": \"v1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:data\": {\".\": {}, \"f:database\": {}, \"f:host\": {}, \"f:password\": {}, \"f:port\": {}, \"f:sslmode\": {}, \"f:type\": {}, \"f:username\": {}}, \"f:immutable\
": {}, \"f:type\": {}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2024-03-13T07:44:47Z\"}], \"name\": \"galaxy-postgres-configuration\", \"namespace\": \"galaxy\", \"resourceVersion\": \"18210991\", \"uid\": \"a9254676-afef-43ab-935b-1d3207ffab89\"}, \"type\": \"Opaque\"}]}\u001b[0m\n\r\nTASK [backup : Check for default Postgre
SQL configuration] *********************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/postgres.yml:11\u001b[0m\n\u001b[0;32mok: [localhost] => {\"censored\": \"the output has been hidden due to the fact that 'no_log: true' was specified for this result\", \"changed\": false}\u001b[0m\n\r\nTASK [backup : Set PostgreSQL configuration] **************
*********************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/postgres.yml:19\u001b[0m\n\u001b[0;32mok: [localhost] => {\"censored\": \"the output has been hidden due to the fact that 'no_log: true' was specified for this result\", \"changed\": false}\u001b[0m\n\r\nTASK [backup : Store Database Configuration] *********************************
**\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/postgres.yml:24\u001b[0m\n\u001b[0;32mok: [localhost] => {\"censored\": \"the output has been hidden due to the fact that 'no_log: true' was specified for this result\", \"changed\": false}\u001b[0m\n\r\nTASK [backup : Default label selector to custom resource generated postgres] ***\r\n\u001b[1;30mt
ask path: /opt/ansible/roles/backup/tasks/postgres.yml:35\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"postgres_label_selector\": \"app.kubernetes.io/instance=postgres-galaxy\"}, \"changed\": false}\u001b[0m\n\r\nTASK [backup : Get the postgres pod information] *******************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/back
up/tasks/postgres.yml:41\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was
2024-03-13T08:40:13.834243904Z stderr F  False\"}\u001b[0m\n\r\nTASK [backup : Set the resource pod name as a variable.] ***********************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/postgres.yml:53\u001b[0m\n\u001b[0;36mskipping: [localhost] => {\"changed\": false, \"skip_reason\": \"Conditional result was False\"}\u001b[0m\n\r\nTASK [Dete
rmine the timestamp for the backup once for all nodes] ***************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup/tasks/postgres.yml:58\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"now\": \"2024-03-13-084012\"}, \"changed\": false}\u001b[0m\n\r\nTASK [Set backup directory name] ***********************************************\r\n\u00
1b[1;30mtask path: /opt/ansible/roles/backup/tasks/postgres.yml:62\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"_backup_dir\": \"/backups/openshift-backup-2024-03-13-084012\"}, \"changed\": false}\u001b[0m\n\r\nTASK [Create directory for backup] *********************************************\r\n\u001b[1;30mtask path: /opt/ansible/roles/backup
/tasks/postgres.yml:66\u001b[0m\n\u001b[0;31mfatal: [localhost]: FAILED! => {\"changed\": false, \"module_stderr\": \"/usr/local/lib/python3.8/site-packages/kubernetes/client/apis/__init__.py:10: DeprecationWarning: The package kubernetes.client.apis is renamed and deprecated, use kubernetes.client.api instead (please note that the trailing s was removed).\\n
 warnings.warn(\\nTraceback (most recent call last):\\n  File \\\"/opt/ansible/.ansible/tmp/ansible-tmp-1710319212.6376235-9955-62319315908609/AnsiballZ_k8s_exec.py\\\", line 102, in <module>\\n    _ansiballz_main()\\n  File \\\"/opt/ansible/.ansible/tmp/ansible-tmp-1710319212.6376235-9955-62319315908609/AnsiballZ_k8s_exec.py\\\", line 94, in _ansiballz_main\\
n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\\n  File \\\"/opt/ansible/.ansible/tmp/ansible-tmp-1710319212.6376235-9955-62319315908609/AnsiballZ_k8s_exec.py\\\", line 40, in invoke_module\\n    runpy.run_module(mod_name='ansible_collections.kubernetes.core.plugins.modules.k8s_exec', init_globals=None, run_name='__main__', alter_sys=True)\\n  Fil
e \\\"/usr/lib64/python3.8/runpy.py\\\", line 207, in run_module\\n    return _run_module_code(code, init_globals, run_name, mod_spec)\\n  File \\\"/usr/lib64/python3.8/runpy.py\\\", line 97, in _run_module_code\\n    _run_code(code, mod_globals, init_globals,\\n  File \\\"/usr/lib64/python3.8/runpy.py\\\", line 87, in _run_code\\n    exec(code, run_globals)\\
n  File \\\"/tmp/ansible_k8s_exec_payload_kg537nmt/ansible_k8s_exec_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s_exec.py\\\", line 245, in <module>\\n  File \\\"/tmp/ansible_k8s_exec_payload_kg537nmt/ansible_k8s_exec_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s_exec.py\\\", line 241, in main\\n  File \\\"/tmp/ansib
le_k8s_exec_payload_kg537nmt/ansible_k8s_exec_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s_exec.py\\\", line 173, in execute_module\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py\\\", line 23483, in read_namespaced_pod\\n    return self.read_namespaced_pod_with_http_info(name, namespace, **kwargs
)  # noqa: E501\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py\\\", line 23570, in read_namespaced_pod_with_http_info\\n    return self.api_client.call_api(\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\\\", line 348, in call_api\\n    return self.__call_api(resource_path, method,\
\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\\\", line 192, in __call_api\\n    return_data = self.deserialize(response_data, response_type)\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\\\", line 264, in deserialize\\n    return self.__deserialize(data, response_type)\\n  File \\\"/u
sr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\\\", line 303, in __deserialize\\n    return self.__deserialize_model(data, klass)\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\\\", line 639, in __deserialize_model\\n    kwargs[attr] = self.__deserialize(value, attr_type)\\n  File \\\"/usr/local/lib/py
thon3.8/site-packages/kubernetes/client/api_client.py\\\", line 303, in __deserialize\\n    return self.__deserialize_model(data, klass)\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\\\", line 639, in __deserialize_model\\n    kwargs[attr] = self.__deserialize(value, attr_type)\\n  File \\\"/usr/local/lib/python3.8/site-pa
ckages/kubernetes/client/api_client.py\\\", line 280, in __deserialize\\n    return [self.__deserialize(sub_data, sub_kls)\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\\\", line 280, in <listcomp>\\n    return [self.__deserialize(sub_data, sub_kls)\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_
client.py\\\", line 303, in __deserialize\\n    return self.__deserialize_model(data, klass)\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py\\\", line 641, in __deserialize_model\\n    instance = klass(**kwargs)\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/models/v1_pod_condition.py\\\", line 76, in
__init__\\n    self.type = type\\n  File \\\"/usr/local/lib/python3.8/site-packages/kubernetes/client/models/v1_pod_condition.py\\\", line 219, in type\\n    raise ValueError(\\nValueError: Invalid value for `type` (PodReadyToStartContainers), must be one of ['ContainersReady', 'Initialized', 'PodScheduled', 'Ready']\\n\", \"module_stdout\": \"\", \"msg\": \"M
ODULE FAILURE\\nSee stdout/stderr for the exact error\", \"rc\": 1}\u001b[0m\n\r\nPLAY RECAP *********************************************************************\r\n\u001b[0;31mlocalhost\u001b[0m                  : \u001b[0;32mok=30  \u001b[0m \u001b[0;33mchanged=2   \u001b[0m unreachable=0    \u001b[0;31mfailed=1   \u001b[0m \u001b[0;36mskipped=12  \u001b[0m
 rescued=0    ignored=0   \r\n\n","job":"775119205517794129","name":"galaxybackup","namespace":"galaxy","error":"exit status 2"}

I think ValueError: Invalid value for type (PodReadyToStartContainers), must be one of ['ContainersReady', 'Initialized', 'PodScheduled', 'Ready'] is the important line.