K8SPG-703: add internal.percona.com/keep-job finalizer for backups

[pooknull]

https://perconadev.atlassian.net/browse/K8SPG-703

DESCRIPTION

Problem:
Backup jobs may be deleted before they are finished (e.g. via ttlSecondsAfterFinished), causing pg-backup to be stuck in the Running state.

Solution:
Add internal.percona.com/keep-job finalizer to backup jobs to prevent them from being deleted while they are running. The finalizer should be automatically removed when the backup finishes or fails.

CHECKLIST

Jira

• Is the Jira ticket created and referenced properly?
• Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
• Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

• Is an E2E test/test case added for the new feature/change?
• Are unit tests added where appropriate?

Config/Logging/Testability

• Are all needed new/changed options added to default YAML files?
• Are all needed new/changed options added to the Helm Chart?
• Did we add proper logging messages for operator actions?
• Did we ensure compatibility with the previous version or cluster upgrade process?
• Does the change support oldest and newest supported PG version?
• Does the change support oldest and newest supported Kubernetes version?

[hors]

@pooknull please check tests

[JNKPercona]

Test name	Status
backup-enable-disable	passed
custom-extensions	passed
custom-tls	passed
demand-backup	passed
finalizers	passed
init-deploy	passed
monitoring	passed
monitoring-pmm3	passed
one-pod	passed
operator-self-healing	passed
pitr	passed
scaling	passed
scheduled-backup	passed
self-healing	failure
sidecars	passed
start-from-backup	passed
tablespaces	passed
telemetry-transfer	passed
upgrade-consistency	passed
upgrade-minor	passed
users	passed
We run 21 out of 21

commit: 94b3bad
image: perconalab/percona-postgresql-operator:PR-1134-94b3bad43

[gkech]

Why don't we assert that the specific finalizer we created for this PR, If it is set, and instead we are checking if the job finalizers are not empty? If we decide at some point to have more finalizers for some reason, what would happen to this logic?

[egegunes]

I think we need to address @gkech's comment