Central Gateway Metrics

[david-martin]

Use Case

I want to view metrics from my gateways to provide me with a single place to observe and monitor when using a multi-cluster gateways

Tasks

• Update the local setup to (optionally) install a federated metrics solution that exposes metrics in the hub in prometheus format (using thanos & prometheus) #304
• configure a data source for federated metrics #305
• Configure the local setup to scrape metrics from Istio gateways in each spoke cluster #306
• Create an example Grafana dashboard that shows traffic data, leveraging Istio metrics #307
• Create an example Grafana dashboard that shows SRE focused metrics for the controller #308

Notes

There is an add-on for OCM however there are known issues with running on plain k8s that would have to be worked through. See stolostron/multicluster-observability-operator#1193 for more details.

There's also an open question about the desire to have that add-on work with plain k8s.

UPDATE At this time it would be prudent to install a minimal stack with thanos & prometheus ( & grafana in the hub) to satisfy the minimum requirements. This will serve as an reference of how to leverage prometheus metrics from Istio, knowing that the federation stack and the gateway provider may actually be different for users.

These problems would need to be bottomed out before going with this add-on as the approach for local-setup.
Alternatively:

- maintain a fork of this add-on that works with k8s
- maintain a solution that is stripped down to the minimal requirements i.e. thanos, minio, prometheus (no observatorium or grafana). There is an example of installing & configuring thanos & promtheus in https://github.com/david-martin/multi-cluster-rollouts

Regarding scraping metrics from Istio, further info is available at https://istio.io/latest/docs/ops/integrations/prometheus/

Out of scope

• Configuring or hooking in to the metrics stack in OCP clusters
• Configuring federation of multiple OSSM gateways metrics centrally

[philbrookes]

This issue is stale because it has been open for 30 days with no activity.

[david-martin]

Given that all tasks are marked as completely, I'm going to close this epic