Re-consider --dry-run=server for previews

[blampe]

Placeholder for v5 wishlist.

Server-side dry-run has a number of surprising edge cases, and we might want to consider client-side previews for v5. Keeping in mind that a lot of users rely on previews as a CI test, so false positives (where the preview succeeds but update fails) can be very disruptive.

#3053 (review)

[rquitales]

We may also want to reconsider this for Helm chart templating (#3247).

[blampe]

We may also want to reconsider this for Helm chart templating (#3247).

It's interesting because the Helm docs seem to recommend server-side apply when previewing charts that use this lookup method. It sounds like it might not even work client-side?

Keep in mind that Helm is not supposed to contact the Kubernetes API Server during a helm template|install|upgrade|delete|rollback --dry-run operation. To test lookup against a running cluster, helm template|install|upgrade|delete|rollback --dry-run=server should be used instead to allow cluster connection.

[EronWright]

@blampe note that Chart v4 uses server-side dry-run, to support the lookup method. Lookup is similar to a Pulumi resource get method, in that it assumes the object exists even during preview. I don't see an issue with reading from the cluster during preview, of course mutations are to be avoided.

Another area that would be affected is the CSA-to-SSA migration logic. I observe that the provider skips doing migration during preview, but then may see a conflict message (during preview, not in the update). See thread.

[EronWright]

Also, ArgoCD has various "diff" strategies, including "server-side dry-run":
https://argo-cd.readthedocs.io/en/stable/user-guide/diff-strategies/