Function controller overwrites function's deployment's 3rd party annotations

[kwiatekus]

Description

Function controller is propagating annotations only from function spec into function's deployment object (spec.template.metadata.annotations.). It ignores any 3rd party annotations

Expected result

In case a 3rd party tool is adding annotations to workloads (for example to enforce workload restart to bump istio container version) function controller should merge existing annotations with user defined ones via function API.

Actual result

In case a 3rd party tool is adding annotations to workloads (for example to enforce workload restart to bump istio container version) function controller is not merging annotations coming from istio and user. It reconciles function deployment and keeps only user defined annotations originating from function crspec.annotaions

Steps to reproduce

1. Install serverless on k3d
2. Create a function
3. Manually edit function's deployment to add a random "foo=bar" annotation in the spec.template.metadata.annotation
4. Observe that function controller reverted the annotation on the target deployment
5. Repeat the steps with serverless controller scaled down to 0 to see that annotation is propagated to pod correctly

Attachments
here is how istio uses annotations to enforce workload restart.