Observability for Argo-rollouts

[bwplotka]

Hi 👋🏽 I recently with @AnaisUrlichs we started to look on Argo project and rollout parts and we are curious what is the best way to gather some observability about the rollout argo rollout performs e.g:

• Alerts: Let's say we want to automate rollouts completely and have Prometheus alert if rollout was paused or rolled back etc. Or if we want to have dashboards for plenty of rollouts and versions currently being rolled on etc. 🤗

• Statistics about latency and error rates of rollouts. e.g from the moment of new version being applied to the moment of rollout being completed

Is there some easy way to gather those? 🤗

[kostis-codefresh]

@bwplotka you can see all metrics here https://argoproj.github.io/argo-rollouts/features/controller-metrics/

Maybe you can use a combination of those (e.g.rollout_reconcile_error) for failed rollbacks?

I think there is another open issue for adding additional metrics to the controller, but I cannot seem to find it now.

It would be also great if you have suggestions on different Graphana dashboards for developers/SREs/non-technical stuff etc.

[bwplotka]

Yes, that page was actually very helpful. We will talk about some panels on PromCon2021. We can share our dashboards too then. (: