Changes to the "app.kubernetes.io/managed-by" label breaks Helm upgrades

[sonnysideup]

Overview

I'm attempting to upgrade our Weaviate Helm chart version but unfortunately the changes are invalid and so Kubernetes rejects them.

Current version: 16.8.1
Desired version: 16.8.6
Error Message:

  Error: UPGRADE FAILED: an error occurred while rolling back the release. original upgrade error: cannot patch "weaviate" with kind StatefulSet: StatefulSet.apps "weaviate" is invalid: spec: Forbidden: updates to statefulset spec for fields
 other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden: no Secret with the name "weaviate-aws" found     

#193, which was merged into release v16.8.2, breaks our ability to perform a helm upgrade. Specifically, changing the StatefulSet.spec.volumeClaimTemplates[*].metadata.labels is strictly forbidden by Kubernetes; offending change here.

I would recommend a couple of things here:

1. Start using a named template inside_helpers.tpl to apply your labels to resources. It seems like you have to manually update labels on all your resources instead of using a macro.
2. Provide some guidance on upgrade strategy for this particular bug.

[sonnysideup]

ping!

[phlegx]

I think I'm facing the same issues:

│ Error: cannot patch "weaviate" with kind StatefulSet: StatefulSet.apps "weaviate" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden

How can we solve this? Should we destroy and recreate the statefulset? I see not much other options tbh. @sonnysideup did you maybe find a solution here in the meanwhile?