diff --git a/CODEOWNERS b/CODEOWNERS index ed995e22..bfe6ee3e 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -8,7 +8,7 @@ * @a-thaler @lilitgh @rakesh-garimella @hisarbalik @skhalash @chrkl @lindnerby # Service binding example -/service-binding/ @Abd4llA @joek @rakesh-garimella @venturasr @lilitgh @k15r @nachtmaar @anishj0shi @montaro @abbi-gaurav @marcobebway @radufa @sayanh +/service-binding/ @rakesh-garimella @lilitgh @k15r @nachtmaar @marcobebway # Orders service example /orders-service/ @m00g3n @pPrecel @@ -19,11 +19,19 @@ # Custom function runtime image example /custom-serverless-runtime-image/ @m00g3n @pPrecel @dbadura @kwiatekus @moelsayed @cortey +# Observability +/prometheus/ @a-thaler @rakesh-garimella @skhalash @chrkl @lindnerby @dennis-ge @lilitgh +/kiali/ @a-thaler @rakesh-garimella @skhalash @chrkl @lindnerby @dennis-ge @lilitgh +/loki/ @a-thaler @rakesh-garimella @skhalash @chrkl @lindnerby @dennis-ge @lilitgh +/trace-demo/ @a-thaler @rakesh-garimella @skhalash @chrkl @lindnerby @dennis-ge @lilitgh +/monitoring-custom-metrics/ @a-thaler @rakesh-garimella @skhalash @chrkl @lindnerby @dennis-ge @lilitgh +/monitoring-custom-alerts/ @a-thaler @rakesh-garimella @skhalash @chrkl @lindnerby @dennis-ge @lilitgh + # Scaling functions to zero with KEDA /scale-to-zero-with-kedam @m00g3n @pPrecel @dbadura @kwiatekus @moelsayed @cortey # All .md files -*.md @klaudiagrz @mmitoraj @majakurcius @alexandra-simeonova @NHingerl @grego952 +*.md @mmitoraj @majakurcius @NHingerl @grego952 # Config file for MILV - milv.config.yaml -milv.config.yaml @m00g3n @aerfio @pPrecel @magicmatatjahu +milv.config.yaml @m00g3n @pPrecel diff --git a/monitoring-custom-metrics/README.md b/monitoring-custom-metrics/README.md index 3c49b55c..6ccb2b24 100644 --- a/monitoring-custom-metrics/README.md +++ b/monitoring-custom-metrics/README.md @@ -21,13 +21,19 @@ This example shows how to expose custom metrics to Prometheus with a Golang serv export KYMA_EXAMPLE_NS="{namespace}" ``` -2. Deploy the service: +2. Ensure that your Namespace has Istio sidecar injection enabled. This example assumes that the metrics are exposed in a strict mTLS mode: + + ```bash + kubectl label namespace ${KYMA_EXAMPLE_NS} istio-injection=enabled + ``` + +3. Deploy the service: ```bash kubectl apply -f deployment/deployment.yaml -n $KYMA_EXAMPLE_NS ``` -3. Deploy the ServiceMonitor: +4. Deploy the ServiceMonitor: ```bash kubectl apply -f deployment/service-monitor.yaml @@ -44,7 +50,7 @@ This example shows how to expose custom metrics to Prometheus with a Golang serv All the **sample-metrics** endpoints appear as the [`Targets`](http://localhost:9090/targets#job-sample-metrics) list. 2. Use either `cpu_temperature_celsius` or `hd_errors_total` in the **expression** field [here](http://localhost:9090/graph). -3. Click the **Execute** button to check the values scrapped by Prometheus. +3. Click the **Execute** button to check the values scraped by Prometheus. ### Cleanup diff --git a/monitoring-custom-metrics/deployment/deployment.yaml b/monitoring-custom-metrics/deployment/deployment.yaml index 96060f0f..276eb417 100644 --- a/monitoring-custom-metrics/deployment/deployment.yaml +++ b/monitoring-custom-metrics/deployment/deployment.yaml @@ -14,8 +14,6 @@ spec: example: monitoring-custom-metrics template: metadata: - annotations: - traffic.sidecar.istio.io/includeInboundPorts: "8080" labels: app: sample-metrics example: monitoring-custom-metrics @@ -29,6 +27,9 @@ spec: memory: 100Mi requests: memory: 32Mi + ports: + - name: http + containerPort: 8080 --- kind: Service apiVersion: v1 diff --git a/prometheus/README.md b/prometheus/README.md new file mode 100644 index 00000000..62538a83 --- /dev/null +++ b/prometheus/README.md @@ -0,0 +1,123 @@ +# Install a custom kube-prometheus-stack in Kyma + +## Overview + +The Kyma monitoring stack brings limited configuration options in contrast to the upstream [`kube-prometheus-stack`](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack) chart. Modifications might be reset at the next upgrade cycle. + +As an alternative, you can install the upstream chart with all customization options in parallel. This tutorial outlines how to set up such installation in co-existence to the Kyma monitoring stack. + +> **CAUTION:** +- This tutorial describes a basic setup that you should not use in production. Typically, a production setup needs further configuration, like optimizing the amount of data to scrape and the required resource footprint of the installation. To achieve qualities like [high availability](https://prometheus.io/docs/introduction/faq/#can-prometheus-be-made-highly-available), [scalability](https://prometheus.io/docs/introduction/faq/#i-was-told-prometheus-doesnt-scale), or [durable long-term storage](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage), you need a more advanced setup. +- This example uses the latest Grafana version, which is under AGPL-3.0 and might not be free of charge for commercial usage. + +## Prerequisites + +- Kyma as the target deployment environment. +- Kubectl > 1.22.x +- Helm 3.x + +## Installation + +### Preparation +1. If you cluster was installed manually using the Kyma CLI, you must assure that the Kyma monitoring stack running in your cluster is limited to detection of Kubernetes resources only in the `kyma-system` Namespace. To rule out that there are any side effects with the additional custom stack, run: + ```bash + kyma deploy --component monitoring --value monitoring.prometheusOperator.namespaces.releaseNamespace=true + ``` + +1. Export your Namespace as a variable. Replace the `{namespace}` placeholder in the following command and run it: + + ```bash + export KYMA_NS="{namespace}" + ``` +1. If you haven't created the Namespace yet, now is the time to do so: + ```bash + kubectl create namespace $KYMA_NS + ``` + >**Note**: This Namespace must have **no** Istio sidecar injection enabled; that is, there must be no `istio-injection` label present on the Namespace. The Helm chart deploys jobs that will not succeed when Isto sidecar injection is enabled. + +1. Export the Helm release name that you want to use. It can be any name, but be aware that all resources in the cluster will be prefixed with that name. Replace the `{release-name}` placeholder in the following command and run it: + ```bash + export HELM_RELEASE="{release-name}" + ``` + +1. Update your Helm installation with the required Helm repository: + + ```bash + helm repo add prometheus-community https://prometheus-community.github.io/helm-charts + helm repo update + ``` + +### Install the kube-prometheus-stack + +1. Run the Helm upgrade command, which installs the chart if it's not present yet. At the end of the command, change the Grafana admin password to some value of your choice. + ```bash + helm upgrade --install -n ${KYMA_NS} ${HELM_RELEASE} prometheus-community/kube-prometheus-stack -f https://raw.githubusercontent.com/kyma-project/examples/main/prometheus/values.yaml --set grafana.adminPassword=myPwd + ``` + +2. You can use the [values.yaml](./values.yaml) provided with this tutorial, which contains customized settings deviating from the default settings, or create your own one. +The provided `values.yaml` covers the following adjustments: +- Parallel operation to a Kyma monitoring stack +- Client certificate injection to support scraping of workload secured with Istio strict mTLS +- Active scraping of workload annotated with @prometheus.io/scrape + +### Activate scraping of Istio metrics & Grafana dashboards + +1. To configure Prometheus for scraping of the Istio-specific metrics from any istio-proxy running in the cluster, deploy a PodMonitor, which scrapes any Pod that has a port with name `.*-envoy-prom` exposed. + + ```bash + kubectl -n ${KYMA_NS} apply -f https://raw.githubusercontent.com/kyma-project/examples/main/prometheus/istio/podmonitor-istio-proxy.yaml + ``` + +2. Deploy a ServiceMonitor definition for the central metrics of the `istiod` deployment: + + ```bash + kubectl -n ${KYMA_NS} apply -f https://raw.githubusercontent.com/kyma-project/examples/main/prometheus/istio/servicemonitor-istiod.yaml + ``` + +3. Get the latest versions of the Istio-specific dashboards. + Grafana is configured to load dashboards dynamically from ConfigMaps in the cluster, so Istio-specific dashboards can be applied as well. + Either follow the [Istio quick start instructions](https://istio.io/latest/docs/ops/integrations/grafana/#option-1-quick-start), or take the prepared ones with the following command: + + ```bash + kubectl -n ${KYMA_NS} apply -f https://raw.githubusercontent.com/kyma-project/examples/main/prometheus/istio/configmap-istio-grafana-dashboards.yaml + kubectl -n ${KYMA_NS} apply -f https://raw.githubusercontent.com/kyma-project/examples/main/prometheus/istio/configmap-istio-services-grafana-dashboards.yaml + ``` + + > **NOTE:** This setup collects all Istio metrics on a Pod level, which can lead to cardinality issues. Because metrics are only needed on service level, for setups having a bigger amount of workloads deployed, it is recommended to use a setup based on federation as described in the [Istio documentation](https://istio.io/latest/docs/ops/best-practices/observability/#using-prometheus-for-production-scale-monitoring). + +### Verify the installation + +1. You should see several Pods coming up in the Namespace, especially Prometheus and Alertmanager. Assure that all Pods have the "Running" state. +2. Browse the Prometheus dashboard and verify that all "Status->Targets" are healthy. The following command exposes the dashboard on `http://localhost:9090`: + ```bash + kubectl -n ${KYMA_NS} port-forward $(kubectl -n ${KYMA_NS} get service -l app=kube-prometheus-stack-prometheus -oname) 9090 + ``` +3. Browse the Grafana dashboard and verify that the dashboards are showing data. The user `admin` is preconfigured in the Helm chart; the password was provided in your `helm install` command. The following command exposes the dashboard on `http://localhost:3000`: + ```bash + kubectl -n ${KYMA_NS} port-forward svc/${HELM_RELEASE}-grafana 3000:80 + ``` + +### Deploy a custom workload and scrape it + +Follow the tutorial [monitoring-custom-metrics](./../monitoring-custom-metrics/), but use the steps above to verify that the metrics are collected. + +### Scrape workload via annotations + +Instead of defining a ServiceMonitor per workload for setting up custom metric scraping, you can use a simplified way based on annotations. The used [values.yaml](./values.yaml) defines an `additionalScrapeConfig`, which scrapes all Pods and services that have the following annotations: + +```yaml +prometheus.io/scrape: "true" # mandatory to enable automatic scraping +prometheus.io/scheme: https # optional, default is "http" if no Istio sidecar is used. When using a sidecar (Pod has label `security.istio.io/tlsMode=istio`), the default is "https". Use "https" to scrape workloads using Istio client certificates. Will only work for annotated services (not Pods) +prometheus.io/port: "1234" # optional, configure the port under which the metrics are exposed +prometheus.io/path: /myMetrics # optional, configure the path under which the metrics are exposed +``` + +You can try it out by removing the ServiceMonitor from the previous example and instead providing the annotations to the Service manifest. + +### Cleanup + +To remove the installation from the cluster, call Helm: + +```bash +helm delete -n ${KYMA_NS} ${HELM_RELEASE} +``` diff --git a/prometheus/istio/configmap-istio-grafana-dashboards.yaml b/prometheus/istio/configmap-istio-grafana-dashboards.yaml new file mode 100644 index 00000000..0ff00a43 --- /dev/null +++ b/prometheus/istio/configmap-istio-grafana-dashboards.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + labels: + grafana_dashboard: "1" + name: istio-grafana-dashboards +data: + istio-performance-dashboard.json: | + {"annotations":{"list":[{"builtIn":1,"datasource":"-- Grafana --","enable":true,"hide":true,"iconColor":"rgba(0, 211, 255, 1)","name":"Annotations & Alerts","type":"dashboard"}]},"editable":false,"gnetId":null,"graphTooltip":0,"links":[],"panels":[{"collapsed":true,"gridPos":{"h":1,"w":24,"x":0,"y":0},"id":21,"panels":[{"content":"The charts on this dashboard are intended to show Istio main components cost in terms of resources utilization under steady load.\n\n- **vCPU / 1k rps:** shows vCPU utilization by the main Istio components normalized by 1000 requests/second. When idle or low traffic, this chart will be blank. The curve for istio-proxy refers to the services sidecars only.\n- **vCPU:** vCPU utilization by Istio components, not normalized.\n- **Memory:** memory footprint for the components. Telemetry and policy are normalized by 1k rps, and no data is shown when there is no traffic. For ingress and istio-proxy, the data is per instance.\n- **Bytes transferred / sec:** shows the number of bytes flowing through each Istio component.\n\n\n","gridPos":{"h":6,"w":24,"x":0,"y":1},"id":19,"links":[],"mode":"markdown","timeFrom":null,"timeShift":null,"title":"Performance Dashboard README","transparent":true,"type":"text"}],"title":"Performance Dashboard Notes","type":"row"},{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":1},"id":6,"panels":[],"title":"vCPU Usage","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":8,"w":12,"x":0,"y":2},"id":4,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"(sum(irate(container_cpu_usage_seconds_total{pod=~\"istio-ingressgateway-.*\",container=\"istio-proxy\"}[1m])) / (round(sum(irate(istio_requests_total{source_workload=\"istio-ingressgateway\", reporter=\"source\"}[1m])), 0.001)/1000))","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"istio-ingressgateway","refId":"A"},{"expr":"(sum(irate(container_cpu_usage_seconds_total{namespace!=\"istio-system\",container=\"istio-proxy\"}[1m]))/ (round(sum(irate(istio_requests_total[1m])), 0.001)/1000))/ (sum(irate(istio_requests_total{source_workload=\"istio-ingressgateway\"}[1m])) >bool 10)","format":"time_series","intervalFactor":1,"legendFormat":"istio-proxy","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"vCPU / 1k rps","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":8,"w":12,"x":12,"y":2},"id":7,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(rate(container_cpu_usage_seconds_total{pod=~\"istio-ingressgateway-.*\",container=\"istio-proxy\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"istio-ingressgateway","refId":"A"},{"expr":"sum(rate(container_cpu_usage_seconds_total{namespace!=\"istio-system\",container=\"istio-proxy\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"istio-proxy","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"vCPU","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":10},"id":13,"panels":[],"title":"Memory and Data Rates","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":8,"w":12,"x":0,"y":11},"id":902,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(container_memory_working_set_bytes{pod=~\"istio-ingressgateway-.*\"}) / count(container_memory_working_set_bytes{pod=~\"istio-ingressgateway-.*\",container!=\"POD\"})","format":"time_series","intervalFactor":1,"legendFormat":"per istio-ingressgateway","refId":"A"},{"expr":"sum(container_memory_working_set_bytes{namespace!=\"istio-system\",container=\"istio-proxy\"}) / count(container_memory_working_set_bytes{namespace!=\"istio-system\",container=\"istio-proxy\"})","format":"time_series","intervalFactor":1,"legendFormat":"per istio proxy","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Memory Usage","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"bytes","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":8,"w":12,"x":12,"y":11},"id":11,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(irate(istio_response_bytes_sum{source_workload=\"istio-ingressgateway\", reporter=\"source\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"istio-ingressgateway","refId":"A"},{"expr":"sum(irate(istio_response_bytes_sum{source_workload_namespace!=\"istio-system\", reporter=\"source\"}[1m])) + sum(irate(istio_request_bytes_sum{source_workload_namespace!=\"istio-system\", reporter=\"source\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"istio-proxy","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Bytes transferred / sec","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"Bps","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":19},"id":17,"panels":[],"title":"Istio Component Versions","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":8,"w":24,"x":0,"y":20},"id":15,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(istio_build) by (component, tag)","format":"time_series","intervalFactor":1,"legendFormat":"{{ component }}: {{ tag }}","refId":"A"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Istio Components by Version","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":31},"id":71,"panels":[],"title":"Proxy Resource Usage","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":0,"y":32},"id":72,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(container_memory_working_set_bytes{container=\"istio-proxy\"})","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Total (k8s)","refId":"A","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Memory","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"bytes","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":6,"y":32},"id":73,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(rate(container_cpu_usage_seconds_total{container=\"istio-proxy\"}[1m]))","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Total (k8s)","refId":"A","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"vCPU","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":12,"y":32},"id":702,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(container_fs_usage_bytes{container=\"istio-proxy\"})","format":"time_series","intervalFactor":2,"legendFormat":"Total (k8s)","refId":"A","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Disk","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"bytes","label":"","logBase":1,"max":null,"min":null,"show":true},{"decimals":null,"format":"none","label":"","logBase":1024,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":39},"id":69,"panels":[],"title":"Istiod Resource Usage","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":0,"y":40},"id":5,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"process_virtual_memory_bytes{app=\"istiod\"}","format":"time_series","instant":false,"intervalFactor":2,"legendFormat":"Virtual Memory","refId":"I","step":2},{"expr":"process_resident_memory_bytes{app=\"istiod\"}","format":"time_series","intervalFactor":2,"legendFormat":"Resident Memory","refId":"H","step":2},{"expr":"go_memstats_heap_sys_bytes{app=\"istiod\"}","format":"time_series","hide":true,"intervalFactor":2,"legendFormat":"heap sys","refId":"A"},{"expr":"go_memstats_heap_alloc_bytes{app=\"istiod\"}","format":"time_series","hide":true,"intervalFactor":2,"legendFormat":"heap alloc","refId":"D"},{"expr":"go_memstats_alloc_bytes{app=\"istiod\"}","format":"time_series","intervalFactor":2,"legendFormat":"Alloc","refId":"F","step":2},{"expr":"go_memstats_heap_inuse_bytes{app=\"istiod\"}","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Heap in-use","refId":"E","step":2},{"expr":"go_memstats_stack_inuse_bytes{app=\"istiod\"}","format":"time_series","intervalFactor":2,"legendFormat":"Stack in-use","refId":"G","step":2},{"expr":"sum(container_memory_working_set_bytes{container=~\"discovery|istio-proxy\", pod=~\"istiod-.*\"})","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Total (k8s)","refId":"C","step":2},{"expr":"container_memory_working_set_bytes{container=~\"discovery|istio-proxy\", pod=~\"istiod-.*\"}","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"{{ container }} (k8s)","refId":"B","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Memory","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"bytes","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":6,"y":40},"id":602,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(rate(container_cpu_usage_seconds_total{container=~\"discovery|istio-proxy\", pod=~\"istiod-.*\"}[1m]))","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Total (k8s)","refId":"A","step":2},{"expr":"sum(rate(container_cpu_usage_seconds_total{container=~\"discovery|istio-proxy\", pod=~\"istiod-.*\"}[1m])) by (container)","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"{{ container }} (k8s)","refId":"B","step":2},{"expr":"irate(process_cpu_seconds_total{app=\"istiod\"}[1m])","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"pilot (self-reported)","refId":"C","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"vCPU","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":12,"y":40},"id":74,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"process_open_fds{app=\"istiod\"}","format":"time_series","hide":true,"instant":false,"interval":"","intervalFactor":2,"legendFormat":"Open FDs (pilot)","refId":"A"},{"expr":"container_fs_usage_bytes{ container=~\"discovery|istio-proxy\", pod=~\"istiod-.*\"}","format":"time_series","intervalFactor":2,"legendFormat":"{{ container }}","refId":"B","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Disk","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"bytes","label":"","logBase":1,"max":null,"min":null,"show":true},{"decimals":null,"format":"none","label":"","logBase":1024,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":18,"y":40},"id":402,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":false,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"go_goroutines{app=\"istiod\"}","format":"time_series","intervalFactor":2,"legendFormat":"Number of Goroutines","refId":"A","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Goroutines","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":"","logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}}],"refresh":"10s","schemaVersion":18,"style":"dark","tags":[],"templating":{"list":[]},"time":{"from":"now-5m","to":"now"},"timepicker":{"refresh_intervals":["5s","10s","30s","1m","5m","15m","30m","1h","2h","1d"],"time_options":["5m","15m","1h","6h","12h","24h","2d","7d","30d"]},"timezone":"","title":"Istio Performance Dashboard","uid":"vu8e0VWZk","version":22} + pilot-dashboard.json: | + {"annotations":{"list":[{"builtIn":1,"datasource":"-- Grafana --","enable":true,"hide":true,"iconColor":"rgba(0, 211, 255, 1)","name":"Annotations & Alerts","type":"dashboard"}]},"editable":false,"gnetId":null,"graphTooltip":1,"links":[],"panels":[{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":0},"id":60,"panels":[],"title":"Deployed Versions","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":5,"w":24,"x":0,"y":1},"id":56,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(istio_build{component=\"pilot\"}) by (tag)","format":"time_series","intervalFactor":1,"legendFormat":"{{ tag }}","refId":"A"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Pilot Versions","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":6},"id":62,"panels":[],"title":"Resource Usage","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":0,"y":7},"id":5,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"process_virtual_memory_bytes{app=\"istiod\"}","format":"time_series","instant":false,"intervalFactor":2,"legendFormat":"Virtual Memory","refId":"I","step":2},{"expr":"process_resident_memory_bytes{app=\"istiod\"}","format":"time_series","intervalFactor":2,"legendFormat":"Resident Memory","refId":"H","step":2},{"expr":"go_memstats_heap_sys_bytes{app=\"istiod\"}","format":"time_series","hide":true,"intervalFactor":2,"legendFormat":"heap sys","refId":"A"},{"expr":"go_memstats_heap_alloc_bytes{app=\"istiod\"}","format":"time_series","hide":true,"intervalFactor":2,"legendFormat":"heap alloc","refId":"D"},{"expr":"go_memstats_alloc_bytes{app=\"istiod\"}","format":"time_series","intervalFactor":2,"legendFormat":"Alloc","refId":"F","step":2},{"expr":"go_memstats_heap_inuse_bytes{app=\"istiod\"}","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Heap in-use","refId":"E","step":2},{"expr":"go_memstats_stack_inuse_bytes{app=\"istiod\"}","format":"time_series","intervalFactor":2,"legendFormat":"Stack in-use","refId":"G","step":2},{"expr":"container_memory_working_set_bytes{container=~\"discovery\", pod=~\"istiod-.*|istio-pilot-.*\"}","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Discovery (container)","refId":"B","step":2},{"expr":"container_memory_working_set_bytes{container=~\"istio-proxy\", pod=~\"istiod-.*|istio-pilot-.*\"}","format":"time_series","intervalFactor":1,"legendFormat":"Sidecar (container)","refId":"C"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Memory","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"bytes","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":6,"y":7},"id":6,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(irate(container_cpu_usage_seconds_total{container=\"discovery\", pod=~\"istiod-.*|istio-pilot-.*\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"Discovery (container)","refId":"A"},{"expr":"irate(process_cpu_seconds_total{app=\"istiod\"}[1m])","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Discovery (process)","refId":"C","step":2},{"expr":"sum(irate(container_cpu_usage_seconds_total{container=\"istio-proxy\", pod=~\"istiod-.*|istio-pilot-.*\"}[1m]))","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Sidecar (container)","refId":"B","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"CPU","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":12,"y":7},"id":7,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"container_fs_usage_bytes{container=\"discovery\", pod=~\"istiod-.*|istio-pilot-.*\"}","format":"time_series","intervalFactor":2,"legendFormat":"Discovery","refId":"B","step":2},{"expr":"container_fs_usage_bytes{container=\"istio-proxy\", pod=~\"istiod-.*|istio-pilot-.*\"}","format":"time_series","intervalFactor":1,"legendFormat":"Sidecar","refId":"A"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Disk","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"bytes","label":"","logBase":1,"max":null,"min":null,"show":true},{"decimals":null,"format":"none","label":"","logBase":1024,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":7,"w":6,"x":18,"y":7},"id":4,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":false,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"go_goroutines{app=\"istiod\"}","format":"time_series","intervalFactor":2,"legendFormat":"Number of Goroutines","refId":"A","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Goroutines","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":"","logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":14},"id":58,"panels":[],"title":"Pilot Push Information","type":"row"},{"aliasColors":{},"bars":true,"dashLength":10,"dashes":false,"datasource":"Prometheus","description":"Shows the rate of pilot pushes","fill":1,"gridPos":{"h":8,"w":8,"x":0,"y":15},"id":622,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":false,"linewidth":1,"links":[],"nullPointMode":"null as zero","paceLength":10,"percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":true,"steppedLine":false,"targets":[{"expr":"sum(irate(pilot_xds_pushes{type=\"cds\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"Cluster","refId":"C"},{"expr":"sum(irate(pilot_xds_pushes{type=\"eds\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"Endpoints","refId":"D"},{"expr":"sum(irate(pilot_xds_pushes{type=\"lds\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"Listeners","refId":"A"},{"expr":"sum(irate(pilot_xds_pushes{type=\"rds\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"Routes","refId":"E"},{"expr":"sum(irate(pilot_xds_pushes{type=\"sds\"}[1m]))","interval":"","legendFormat":"Secrets","refId":"B"},{"expr":"sum(irate(pilot_xds_pushes{type=\"nds\"}[1m]))","interval":"","legendFormat":"Nametables","refId":"F"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Pilot Pushes","tooltip":{"shared":false,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":["total"]},"yaxes":[{"format":"ops","label":null,"logBase":1,"max":null,"min":"0","show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","description":"Captures a variety of pilot errors","fill":1,"gridPos":{"h":8,"w":8,"x":8,"y":15},"id":67,"legend":{"avg":false,"current":false,"hideEmpty":true,"hideZero":true,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(pilot_xds_cds_reject{app=\"istiod\"}) or (absent(pilot_xds_cds_reject{app=\"istiod\"}) - 1)","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Rejected CDS Configs","refId":"C"},{"expr":"sum(pilot_xds_eds_reject{app=\"istiod\"}) or (absent(pilot_xds_eds_reject{app=\"istiod\"}) - 1)","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Rejected EDS Configs","refId":"D"},{"expr":"sum(pilot_xds_rds_reject{app=\"istiod\"}) or (absent(pilot_xds_rds_reject{app=\"istiod\"}) - 1)","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Rejected RDS Configs","refId":"A"},{"expr":"sum(pilot_xds_lds_reject{app=\"istiod\"}) or (absent(pilot_xds_lds_reject{app=\"istiod\"}) - 1)","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Rejected LDS Configs","refId":"B"},{"expr":"sum(rate(pilot_xds_write_timeout{app=\"istiod\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"Write Timeouts","refId":"F"},{"expr":"sum(rate(pilot_total_xds_internal_errors{app=\"istiod\"}[1m]))","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Internal Errors","refId":"H"},{"expr":"sum(rate(pilot_total_xds_rejects{app=\"istiod\"}[1m]))","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Config Rejection Rate","refId":"E"},{"expr":"sum(rate(pilot_xds_push_context_errors{app=\"istiod\"}[1m]))","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Push Context Errors","refId":"K"},{"expr":"sum(rate(pilot_xds_write_timeout{app=\"istiod\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"Push Timeouts","refId":"G"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Pilot Errors","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","description":"Shows the total time it takes to push a config update to a proxy","fill":1,"gridPos":{"h":8,"w":8,"x":16,"y":15},"id":624,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"histogram_quantile(0.5, sum(rate(pilot_proxy_convergence_time_bucket[1m])) by (le))","format":"time_series","intervalFactor":1,"legendFormat":"p50 ","refId":"A"},{"expr":"histogram_quantile(0.9, sum(rate(pilot_proxy_convergence_time_bucket[1m])) by (le))","format":"time_series","intervalFactor":1,"legendFormat":"p90","refId":"B"},{"expr":"histogram_quantile(0.99, sum(rate(pilot_proxy_convergence_time_bucket[1m])) by (le))","format":"time_series","intervalFactor":1,"legendFormat":"p99","refId":"C"},{"expr":"histogram_quantile(0.999, sum(rate(pilot_proxy_convergence_time_bucket[1m])) by (le))","format":"time_series","intervalFactor":1,"legendFormat":"p99.9","refId":"D"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Proxy Push Time","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"s","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":8,"w":12,"x":0,"y":23},"id":45,"legend":{"avg":false,"current":false,"hideEmpty":true,"hideZero":true,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null as zero","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"pilot_conflict_inbound_listener{app=\"istiod\"}","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Inbound Listeners","refId":"B"},{"expr":"pilot_conflict_outbound_listener_http_over_current_tcp{app=\"istiod\"}","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Outbound Listeners (http over current tcp)","refId":"A"},{"expr":"pilot_conflict_outbound_listener_tcp_over_current_tcp{app=\"istiod\"}","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Outbound Listeners (tcp over current tcp)","refId":"C"},{"expr":"pilot_conflict_outbound_listener_tcp_over_current_http{app=\"istiod\"}","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"Outbound Listeners (tcp over current http)","refId":"D"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Conflicts","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":8,"w":12,"x":12,"y":23},"id":47,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"avg(pilot_virt_services{app=\"istiod\"})","format":"time_series","intervalFactor":1,"legendFormat":"Virtual Services","refId":"A"},{"expr":"avg(pilot_services{app=\"istiod\"})","format":"time_series","intervalFactor":1,"legendFormat":"Services","refId":"B"},{"expr":"sum(pilot_xds{app=\"istiod\"}) by (pod)","format":"time_series","intervalFactor":1,"legendFormat":"Connected Endpoints {{pod}}","refId":"E"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"ADS Monitoring","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"gridPos":{"h":1,"w":24,"x":0,"y":31},"id":64,"panels":[],"title":"Envoy Information","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","description":"Shows details about Envoy proxies in the mesh","fill":1,"gridPos":{"h":8,"w":8,"x":0,"y":32},"id":40,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(irate(envoy_cluster_upstream_cx_total{cluster_name=\"xds-grpc\"}[1m]))","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"XDS Connections","refId":"C"},{"expr":"sum(irate(envoy_cluster_upstream_cx_connect_fail{cluster_name=\"xds-grpc\"}[1m]))","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"XDS Connection Failures","refId":"A"},{"expr":"sum(increase(envoy_server_hot_restart_epoch[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"Envoy Restarts","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Envoy Details","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"ops","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"ops","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":8,"w":8,"x":8,"y":32},"id":41,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(envoy_cluster_upstream_cx_active{cluster_name=\"xds-grpc\"})","format":"time_series","intervalFactor":2,"legendFormat":"XDS Active Connections","refId":"C","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"XDS Active Connections","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","description":"Shows the size of XDS requests and responses","fill":1,"gridPos":{"h":8,"w":8,"x":16,"y":32},"id":42,"legend":{"avg":false,"current":false,"hideEmpty":false,"hideZero":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"max(rate(envoy_cluster_upstream_cx_rx_bytes_total{cluster_name=\"xds-grpc\"}[1m]))","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"XDS Response Bytes Max","refId":"D"},{"expr":"quantile(0.5, rate(envoy_cluster_upstream_cx_rx_bytes_total{cluster_name=\"xds-grpc\"}[1m]))","format":"time_series","hide":false,"intervalFactor":1,"legendFormat":"XDS Response Bytes Average","refId":"B"},{"expr":"max(rate(envoy_cluster_upstream_cx_tx_bytes_total{cluster_name=\"xds-grpc\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"XDS Request Bytes Max","refId":"A"},{"expr":"quantile(.5, rate(envoy_cluster_upstream_cx_tx_bytes_total{cluster_name=\"xds-grpc\"}[1m]))","format":"time_series","intervalFactor":1,"legendFormat":"XDS Request Bytes Average","refId":"C"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"XDS Requests Size","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"Bps","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"ops","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"datasource":null,"gridPos":{"h":1,"w":24,"x":0,"y":40},"id":626,"panels":[],"title":"Webhooks","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":null,"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":0,"y":41},"hiddenSeries":false,"id":629,"legend":{"avg":false,"current":false,"hideEmpty":false,"hideZero":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(rate(galley_validation_passed[1m]))","interval":"","legendFormat":"Validations (Success)","refId":"A"},{"expr":"sum(rate(galley_validation_failed[1m]))","interval":"","legendFormat":"Validation (Failure)","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Configuration Validation","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":null,"description":"","fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":12,"y":41},"hiddenSeries":false,"id":630,"legend":{"avg":false,"current":false,"hideZero":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(rate(sidecar_injection_success_total[1m]))","interval":"","legendFormat":"Injections (Success)","refId":"A"},{"expr":"sum(rate(sidecar_injection_failure_total[1m]))","interval":"","legendFormat":"Injections (Failure)","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Sidecar Injection","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}}],"refresh":"5s","schemaVersion":18,"style":"dark","tags":[],"templating":{"list":[{"current":{"selected":true,"text":"default","value":"default"},"hide":0,"includeAll":false,"label":null,"multi":false,"name":"datasource","options":[],"query":"prometheus","queryValue":"","refresh":1,"regex":"","skipUrlSync":false,"type":"datasource"}]},"time":{"from":"now-5m","to":"now"},"timepicker":{"refresh_intervals":["5s","10s","30s","1m","5m","15m","30m","1h","2h","1d"],"time_options":["5m","15m","1h","6h","12h","24h","2d","7d","30d"]},"timezone":"browser","title":"Istio Control Plane Dashboard","uid":"3--MLVZZk","version":11} + diff --git a/prometheus/istio/configmap-istio-services-grafana-dashboards.yaml b/prometheus/istio/configmap-istio-services-grafana-dashboards.yaml new file mode 100644 index 00000000..a57e32d9 --- /dev/null +++ b/prometheus/istio/configmap-istio-services-grafana-dashboards.yaml @@ -0,0 +1,895 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + labels: + grafana_dashboard: "1" + name: istio-services-grafana-dashboards +data: + istio-extension-dashboard.json: | + {"annotations":{"list":[{"builtIn":1,"datasource":"-- Grafana --","enable":true,"hide":true,"iconColor":"rgba(0, 211, 255, 1)","name":"Annotations & Alerts","type":"dashboard"}]},"editable":false,"gnetId":null,"graphTooltip":0,"links":[],"panels":[{"collapsed":false,"datasource":"Prometheus","gridPos":{"h":1,"w":24,"x":0,"y":0},"id":3,"panels":[],"title":"Wasm VMs","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","description":"","fieldConfig":{"defaults":{"custom":{"align":null},"links":[],"mappings":[],"thresholds":{"mode":"absolute","steps":[{"color":"green","value":null},{"color":"red","value":80}]}},"overrides":[]},"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":0,"y":1},"hiddenSeries":false,"id":2,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"alertThreshold":true},"percentage":false,"pluginVersion":"7.2.1","pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"avg(envoy_wasm_envoy_wasm_runtime_null_active)","interval":"","legendFormat":"native","refId":"A"},{"expr":"avg(envoy_wasm_envoy_wasm_runtime_v8_active)","interval":"","legendFormat":"v8","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Active","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"$$hashKey":"object:123","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"$$hashKey":"object:124","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fieldConfig":{"defaults":{"custom":{},"links":[]},"overrides":[]},"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":12,"y":1},"hiddenSeries":false,"id":6,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"alertThreshold":true},"percentage":false,"pluginVersion":"7.2.1","pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"avg(envoy_wasm_envoy_wasm_runtime_null_created)","interval":"","legendFormat":"native","refId":"A"},{"expr":"avg(envoy_wasm_envoy_wasm_runtime_v8_created)","interval":"","legendFormat":"v8","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Created","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"$$hashKey":"object:68","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"$$hashKey":"object:69","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"datasource":"Prometheus","gridPos":{"h":1,"w":24,"x":0,"y":9},"id":7,"panels":[],"title":"Wasm Module Remote Load","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fieldConfig":{"defaults":{"custom":{},"links":[]},"overrides":[]},"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":8,"x":0,"y":10},"hiddenSeries":false,"id":11,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"alertThreshold":true},"percentage":false,"pluginVersion":"7.2.1","pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"avg(envoy_wasm_remote_load_cache_entries)","interval":"","legendFormat":"entries","refId":"A"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Cache Entry","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"$$hashKey":"object:178","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"$$hashKey":"object:179","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fieldConfig":{"defaults":{"custom":{},"links":[]},"overrides":[]},"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":8,"x":8,"y":10},"hiddenSeries":false,"id":8,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"alertThreshold":true},"percentage":false,"pluginVersion":"7.2.1","pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"avg(envoy_wasm_remote_load_cache_hits)","interval":"","legendFormat":"hits","refId":"A"},{"expr":"avg(envoy_wasm_remote_load_cache_misses)","interval":"","legendFormat":"misses","refId":"B"},{"expr":"avg(envoy_wasm_remote_load_cache_negative_hits)","interval":"","legendFormat":"negative hits","refId":"C"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Cache Visit","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"$$hashKey":"object:233","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"$$hashKey":"object:234","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fieldConfig":{"defaults":{"custom":{},"links":[]},"overrides":[]},"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":8,"x":16,"y":10},"hiddenSeries":false,"id":10,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"alertThreshold":true},"percentage":false,"pluginVersion":"7.2.1","pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"avg(envoy_wasm_remote_load_fetch_failures)","interval":"","legendFormat":"failures","refId":"A"},{"expr":"avg(envoy_wasm_remote_load_fetch_successes)","interval":"","legendFormat":"successes","refId":"B"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Remote Fetch","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"$$hashKey":"object:288","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"$$hashKey":"object:289","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"collapsed":false,"datasource":"Prometheus","gridPos":{"h":1,"w":24,"x":0,"y":18},"id":71,"panels":[],"title":"Proxy Resource Usage","type":"row"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fieldConfig":{"defaults":{"custom":{}},"overrides":[]},"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":0,"y":19},"hiddenSeries":false,"id":72,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","options":{"alertThreshold":true},"percentage":false,"pluginVersion":"7.2.1","pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(container_memory_working_set_bytes{container=\"istio-proxy\"})","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Total (k8s)","refId":"A","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Memory","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"$$hashKey":"object:396","format":"bytes","label":null,"logBase":1,"max":null,"min":null,"show":true},{"$$hashKey":"object:397","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fieldConfig":{"defaults":{"custom":{}},"overrides":[]},"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":12,"y":19},"hiddenSeries":false,"id":73,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","options":{"alertThreshold":true},"percentage":false,"pluginVersion":"7.2.1","pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(rate(container_cpu_usage_seconds_total{container=\"istio-proxy\"}[1m]))","format":"time_series","hide":false,"intervalFactor":2,"legendFormat":"Total (k8s)","refId":"A","step":2}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"vCPU","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"$$hashKey":"object:447","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"$$hashKey":"object:448","format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}}],"refresh":false,"schemaVersion":26,"style":"dark","tags":[],"templating":{"list":[]},"time":{"from":"now-5m","to":"now"},"timepicker":{"refresh_intervals":["10s","30s","1m","5m","15m","30m","1h","2h","1d"]},"timezone":"","title":"Istio Wasm Extension Dashboard","uid":"7PAV7ctGz","version":17} + istio-mesh-dashboard.json: | + {"annotations":{"list":[{"builtIn":1,"datasource":"-- Grafana --","enable":true,"hide":true,"iconColor":"rgba(0, 211, 255, 1)","name":"Annotations & Alerts","type":"dashboard"}]},"editable":false,"gnetId":null,"graphTooltip":0,"id":null,"links":[],"panels":[{"content":"
\n
\n Istio\n
\n
\n Istio is an open platform that provides a uniform way to secure,\n connect, and \n monitor microservices.\n
\n Need help? Join the Istio community.\n
\n
","gridPos":{"h":3,"w":24,"x":0,"y":0},"height":"50px","id":13,"links":[],"mode":"html","style":{"font-size":"18pt"},"title":"","transparent":true,"type":"text"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"datasource":"Prometheus","format":"ops","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":0,"y":3},"id":20,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":true,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"round(sum(irate(istio_requests_total{reporter=\"source\"}[1m])), 0.001)","intervalFactor":1,"refId":"A","step":4}],"thresholds":"","title":"Global Request Volume","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"avg"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"datasource":"Prometheus","format":"percentunit","gauge":{"maxValue":100,"minValue":80,"show":false,"thresholdLabels":false,"thresholdMarkers":false},"gridPos":{"h":3,"w":6,"x":6,"y":3},"id":21,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":true,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"sum(rate(istio_requests_total{reporter=\"source\", response_code!~\"5.*\"}[1m])) / sum(rate(istio_requests_total{reporter=\"source\"}[1m]))","format":"time_series","intervalFactor":1,"refId":"A","step":4}],"thresholds":"95, 99, 99.5","title":"Global Success Rate (non-5xx responses)","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"avg"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"datasource":"Prometheus","format":"ops","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":12,"y":3},"id":22,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":true,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"sum(irate(istio_requests_total{reporter=\"source\", response_code=~\"4.*\"}[1m]))","format":"time_series","intervalFactor":1,"refId":"A","step":4}],"thresholds":"","title":"4xxs","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"avg"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"datasource":"Prometheus","format":"ops","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":18,"y":3},"id":23,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":true,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"sum(irate(istio_requests_total{reporter=\"source\", response_code=~\"5.*\"}[1m]))","format":"time_series","intervalFactor":1,"refId":"A","step":4}],"thresholds":"","title":"5xxs","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"avg"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"none","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":0,"y":6},"id":113,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"max(pilot_k8s_cfg_events{type=\"VirtualService\", event=\"add\"}) - (max(pilot_k8s_cfg_events{type=\"VirtualService\", event=\"delete\"}) or max(up * 0))","format":"time_series","intervalFactor":1,"refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"Virtual Services","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"current"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"none","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":6,"y":6},"id":114,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"max(pilot_k8s_cfg_events{type=\"DestinationRule\", event=\"add\"}) - (max(pilot_k8s_cfg_events{type=\"DestinationRule\", event=\"delete\"}) or max(up * 0))","format":"time_series","intervalFactor":1,"refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"Destination Rules","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"current"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"none","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":12,"y":6},"id":115,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"max(pilot_k8s_cfg_events{type=\"Gateway\", event=\"add\"}) - (max(pilot_k8s_cfg_events{type=\"Gateway\", event=\"delete\"}) or max(up * 0))","format":"time_series","intervalFactor":1,"refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"Gateways","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"current"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"none","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":18,"y":6},"id":116,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"max(pilot_k8s_cfg_events{type=\"WorkloadEntry\", event=\"add\"}) - (max(pilot_k8s_cfg_events{type=\"WorkloadEntry\", event=\"delete\"}) or max(up * 0))","format":"time_series","intervalFactor":1,"refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"Workload Entries","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"current"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"none","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":0,"y":6},"id":117,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"max(pilot_k8s_cfg_events{type=\"ServiceEntry\", event=\"add\"}) - (max(pilot_k8s_cfg_events{type=\"ServiceEntry\", event=\"delete\"}) or max(up * 0))","format":"time_series","intervalFactor":1,"refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"Service Entries","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"current"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"none","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":6,"y":6},"id":90,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"max(pilot_k8s_cfg_events{type=\"PeerAuthentication\", event=\"add\"}) - (max(pilot_k8s_cfg_events{type=\"PeerAuthentication\", event=\"delete\"}) or max(up * 0))","format":"time_series","intervalFactor":1,"refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"PeerAuthentication Policies","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"current"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"none","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":12,"y":6},"id":91,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"max(pilot_k8s_cfg_events{type=\"RequestAuthentication\", event=\"add\"}) - (max(pilot_k8s_cfg_events{type=\"RequestAuthentication\", event=\"delete\"}) or max(up * 0))","format":"time_series","intervalFactor":1,"refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"RequestAuthentication Policies","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"current"},{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"none","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":3,"w":6,"x":18,"y":6},"id":92,"interval":null,"links":[],"options":{"colorMode":"value","graphMode":"area","justifyMode":"auto","orientation":"horizontal","reduceOptions":{"calcs":["lastNotNull"],"fields":"","values":false},"textMode":"auto"},"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true},"tableColumn":"","targets":[{"expr":"max(pilot_k8s_cfg_events{type=\"AuthorizationPolicy\", event=\"add\"}) - (max(pilot_k8s_cfg_events{type=\"AuthorizationPolicy\", event=\"delete\"}) or max(up * 0))","format":"time_series","intervalFactor":1,"refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"Authorization Policies","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"current"},{"columns":[],"datasource":"Prometheus","fontSize":"100%","gridPos":{"h":21,"w":24,"x":0,"y":9},"hideTimeOverride":false,"id":73,"links":[],"pageSize":null,"repeat":null,"repeatDirection":"v","scroll":true,"showHeader":true,"sort":{"col":5,"desc":true},"styles":[{"alias":"Workload","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"link":false,"linkTargetBlank":false,"linkTooltip":"Workload dashboard","linkUrl":"/dashboard/db/istio-workload-dashboard?var-namespace=${__cell_3:raw}&var-workload=${__cell_2:raw}","pattern":"destination_workload","preserveFormat":false,"sanitize":false,"thresholds":[],"type":"hidden","unit":"short"},{"alias":"","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Time","thresholds":[],"type":"hidden","unit":"short"},{"alias":"Requests","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Value #A","thresholds":[],"type":"number","unit":"ops"},{"alias":"P50 Latency","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Value #B","thresholds":[],"type":"number","unit":"s"},{"alias":"P90 Latency","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Value #C","thresholds":[],"type":"number","unit":"s"},{"alias":"P99 Latency","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Value #D","thresholds":[],"type":"number","unit":"s"},{"alias":"Success Rate","colorMode":"cell","colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Value #E","thresholds":[".95"," 1.00"],"type":"number","unit":"percentunit"},{"alias":"Workload","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"link":true,"linkTooltip":"$__cell dashboard","linkUrl":"/dashboard/db/istio-workload-dashboard?var-workload=${__cell_2:raw}&var-namespace=${__cell_3:raw}","pattern":"destination_workload_var","thresholds":[],"type":"number","unit":"short"},{"alias":"Service","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"link":true,"linkTooltip":"$__cell dashboard","linkUrl":"/dashboard/db/istio-service-dashboard?var-service=${__cell_1:raw}","pattern":"destination_service","thresholds":[],"type":"string","unit":"short"},{"alias":"","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"destination_workload_namespace","thresholds":[],"type":"hidden","unit":"short"}],"targets":[{"expr":"label_join(sum(rate(istio_requests_total{reporter=\"source\", response_code=\"200\"}[1m])) by (destination_workload, destination_workload_namespace, destination_service), \"destination_workload_var\", \".\", \"destination_workload\", \"destination_workload_namespace\")","format":"table","hide":false,"instant":true,"intervalFactor":1,"legendFormat":"{{ destination_workload}}.{{ destination_workload_namespace }}","refId":"A"},{"expr":"label_join((histogram_quantile(0.50, sum(rate(istio_request_duration_milliseconds_bucket{reporter=\"source\"}[1m])) by (le, destination_workload, destination_workload_namespace)) / 1000) or histogram_quantile(0.50, sum(rate(istio_request_duration_seconds_bucket{reporter=\"source\"}[1m])) by (le, destination_workload, destination_workload_namespace)), \"destination_workload_var\", \".\", \"destination_workload\", \"destination_workload_namespace\")","format":"table","hide":false,"instant":true,"intervalFactor":1,"legendFormat":"{{ destination_workload}}.{{ destination_workload_namespace }}","refId":"B"},{"expr":"label_join((histogram_quantile(0.90, sum(rate(istio_request_duration_milliseconds_bucket{reporter=\"source\"}[1m])) by (le, destination_workload, destination_workload_namespace)) / 1000) or histogram_quantile(0.90, sum(rate(istio_request_duration_seconds_bucket{reporter=\"source\"}[1m])) by (le, destination_workload, destination_workload_namespace)), \"destination_workload_var\", \".\", \"destination_workload\", \"destination_workload_namespace\")","format":"table","hide":false,"instant":true,"intervalFactor":1,"legendFormat":"{{ destination_workload }}.{{ destination_workload_namespace }}","refId":"C"},{"expr":"label_join((histogram_quantile(0.99, sum(rate(istio_request_duration_milliseconds_bucket{reporter=\"source\"}[1m])) by (le, destination_workload, destination_workload_namespace)) / 1000) or histogram_quantile(0.99, sum(rate(istio_request_duration_seconds_bucket{reporter=\"source\"}[1m])) by (le, destination_workload, destination_workload_namespace)), \"destination_workload_var\", \".\", \"destination_workload\", \"destination_workload_namespace\")","format":"table","hide":false,"instant":true,"intervalFactor":1,"legendFormat":"{{ destination_workload }}.{{ destination_workload_namespace }}","refId":"D"},{"expr":"label_join((sum(rate(istio_requests_total{reporter=\"source\", response_code!~\"5.*\"}[1m])) by (destination_workload, destination_workload_namespace) / sum(rate(istio_requests_total{reporter=\"source\"}[1m])) by (destination_workload, destination_workload_namespace)), \"destination_workload_var\", \".\", \"destination_workload\", \"destination_workload_namespace\")","format":"table","hide":false,"instant":true,"interval":"","intervalFactor":1,"legendFormat":"{{ destination_workload }}.{{ destination_workload_namespace }}","refId":"E"}],"timeFrom":null,"title":"HTTP/GRPC Workloads","transform":"table","type":"table"},{"columns":[],"datasource":"Prometheus","fontSize":"100%","gridPos":{"h":18,"w":24,"x":0,"y":30},"hideTimeOverride":false,"id":109,"links":[],"pageSize":null,"repeatDirection":"v","scroll":true,"showHeader":true,"sort":{"col":5,"desc":true},"styles":[{"alias":"Workload","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"link":false,"linkTargetBlank":false,"linkTooltip":"$__cell dashboard","linkUrl":"/dashboard/db/istio-workload-dashboard?var-namespace=${__cell_3:raw}&var-workload=${__cell_2:raw}","pattern":"destination_workload","preserveFormat":false,"sanitize":false,"thresholds":[],"type":"hidden","unit":"short"},{"alias":"Bytes Sent","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Value #A","thresholds":[""],"type":"number","unit":"Bps"},{"alias":"Bytes Received","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Value #B","thresholds":[],"type":"number","unit":"Bps"},{"alias":"","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"Time","thresholds":[],"type":"hidden","unit":"short"},{"alias":"Workload","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"link":true,"linkTooltip":"$__cell dashboard","linkUrl":"/dashboard/db/istio-workload-dashboard?var-namespace=${__cell_3:raw}&var-workload=${__cell_2:raw}","pattern":"destination_workload_var","thresholds":[],"type":"string","unit":"short"},{"alias":"","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"pattern":"destination_workload_namespace","thresholds":[],"type":"hidden","unit":"short"},{"alias":"Service","colorMode":null,"colors":["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat":"YYYY-MM-DD HH:mm:ss","decimals":2,"link":true,"linkTooltip":"$__cell dashboard","linkUrl":"/dashboard/db/istio-service-dashboard?var-service=${__cell_1:raw}","pattern":"destination_service","thresholds":[],"type":"number","unit":"short"}],"targets":[{"expr":"label_join(sum(rate(istio_tcp_received_bytes_total{reporter=\"source\"}[1m])) by (destination_workload, destination_workload_namespace, destination_service), \"destination_workload_var\", \".\", \"destination_workload\", \"destination_workload_namespace\")","format":"table","hide":false,"instant":true,"intervalFactor":1,"legendFormat":"{{ destination_workload }}","refId":"A"},{"expr":"label_join(sum(rate(istio_tcp_sent_bytes_total{reporter=\"source\"}[1m])) by (destination_workload, destination_workload_namespace, destination_service), \"destination_workload_var\", \".\", \"destination_workload\", \"destination_workload_namespace\")","format":"table","hide":false,"instant":true,"intervalFactor":1,"legendFormat":"{{ destination_workload }}","refId":"B"}],"timeFrom":null,"title":"TCP Workloads","transform":"table","type":"table"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","fill":1,"gridPos":{"h":9,"w":24,"x":0,"y":48},"id":111,"legend":{"alignAsTable":false,"avg":false,"current":false,"max":false,"min":false,"rightSide":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"links":[],"nullPointMode":"null","percentage":false,"pointradius":5,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"sum(istio_build) by (component, tag)","format":"time_series","intervalFactor":1,"legendFormat":"{{ component }}: {{ tag }}","refId":"A"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Istio Components by Version","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":false}],"yaxis":{"align":false,"alignLevel":null}}],"refresh":"5s","schemaVersion":18,"style":"dark","tags":[],"templating":{"list":[{"current":{"selected":true,"text":"default","value":"default"},"hide":0,"includeAll":false,"label":null,"multi":false,"name":"datasource","options":[],"query":"prometheus","queryValue":"","refresh":1,"regex":"","skipUrlSync":false,"type":"datasource"}]},"time":{"from":"now-5m","to":"now"},"timepicker":{"refresh_intervals":["5s","10s","30s","1m","5m","15m","30m","1h","2h","1d"],"time_options":["5m","15m","1h","6h","12h","24h","2d","7d","30d"]},"timezone":"browser","title":"Istio Mesh Dashboard","uid":"G8wLrJIZk","version":5} + istio-service-dashboard.json: "{\"annotations\":{\"list\":[{\"builtIn\":1,\"datasource\":\"-- + Grafana --\",\"enable\":true,\"hide\":true,\"iconColor\":\"rgba(0, 211, 255, 1)\",\"name\":\"Annotations + & Alerts\",\"type\":\"dashboard\"}]},\"editable\":false,\"gnetId\":null,\"graphTooltip\":0,\"iteration\":1595591291797,\"links\":[],\"panels\":[{\"collapsed\":true,\"gridPos\":{\"h\":1,\"w\":24,\"x\":0,\"y\":0},\"id\":106,\"panels\":[{\"content\":\"
\\nSERVICE: $service\\n
\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"gridPos\":{\"h\":3,\"w\":24,\"x\":0,\"y\":1},\"id\":89,\"links\":[],\"mode\":\"html\",\"options\":{\"content\":\"
\\nSERVICE: $service\\n
\",\"mode\":\"html\"},\"pluginVersion\":\"7.1.0\",\"title\":\"\",\"transparent\":true,\"type\":\"text\"},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"rgba(245, + 54, 54, 0.9)\",\"rgba(237, 129, 40, 0.89)\",\"rgba(50, 172, 45, 0.97)\"],\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"ops\",\"gauge\":{\"maxValue\":100,\"minValue\":0,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":true},\"gridPos\":{\"h\":4,\"w\":6,\"x\":0,\"y\":4},\"id\":12,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"round(sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\"}[5m])), + 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"refId\":\"A\",\"step\":4}],\"thresholds\":\"\",\"title\":\"Client + Request Volume\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"current\"},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"rgba(50, + 172, 45, 0.97)\",\"rgba(237, 129, 40, 0.89)\",\"rgba(245, 54, 54, 0.9)\"],\"datasource\":\"Prometheus\",\"decimals\":null,\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"percentunit\",\"gauge\":{\"maxValue\":100,\"minValue\":80,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":false},\"gridPos\":{\"h\":4,\"w\":6,\"x\":6,\"y\":4},\"id\":14,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\",response_code!~\\\"5.*\\\"}[5m])) + / sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\"}[5m]))\",\"format\":\"time_series\",\"intervalFactor\":1,\"refId\":\"A\"}],\"thresholds\":\"95, + 99, 99.5\",\"title\":\"Client Success Rate (non-5xx responses)\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"avg\"},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":4,\"w\":6,\"x\":12,\"y\":4},\"hiddenSeries\":false,\"id\":87,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":false,\"hideZero\":false,\"max\":false,\"min\":false,\"rightSide\":true,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le)) / 1000) or histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le))\",\"format\":\"time_series\",\"interval\":\"\",\"intervalFactor\":1,\"legendFormat\":\"P50\",\"refId\":\"A\"},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le)) / 1000) or histogram_quantile(0.90, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"P90\",\"refId\":\"B\"},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le)) / 1000) or histogram_quantile(0.99, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"P99\",\"refId\":\"C\"}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Client + Request Duration\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"s\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"#299c46\",\"rgba(237, + 129, 40, 0.89)\",\"#d44a3a\"],\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"Bps\",\"gauge\":{\"maxValue\":100,\"minValue\":0,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":true},\"gridPos\":{\"h\":4,\"w\":6,\"x\":18,\"y\":4},\"id\":84,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"sum(irate(istio_tcp_received_bytes_total{reporter=~\\\"$qrep\\\", + destination_service=~\\\"$service\\\"}[1m]))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"\",\"refId\":\"A\"}],\"thresholds\":\"\",\"title\":\"TCP + Received Bytes\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"avg\"},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"rgba(245, + 54, 54, 0.9)\",\"rgba(237, 129, 40, 0.89)\",\"rgba(50, 172, 45, 0.97)\"],\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"ops\",\"gauge\":{\"maxValue\":100,\"minValue\":0,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":true},\"gridPos\":{\"h\":4,\"w\":6,\"x\":0,\"y\":8},\"id\":97,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"round(sum(irate(istio_requests_total{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\"}[5m])), + 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"refId\":\"A\",\"step\":4}],\"thresholds\":\"\",\"title\":\"Server + Request Volume\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"current\"},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"rgba(50, + 172, 45, 0.97)\",\"rgba(237, 129, 40, 0.89)\",\"rgba(245, 54, 54, 0.9)\"],\"datasource\":\"Prometheus\",\"decimals\":null,\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"percentunit\",\"gauge\":{\"maxValue\":100,\"minValue\":80,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":false},\"gridPos\":{\"h\":4,\"w\":6,\"x\":6,\"y\":8},\"id\":98,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"sum(irate(istio_requests_total{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\",response_code!~\\\"5.*\\\"}[5m])) + / sum(irate(istio_requests_total{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\"}[5m]))\",\"format\":\"time_series\",\"intervalFactor\":1,\"refId\":\"A\"}],\"thresholds\":\"95, + 99, 99.5\",\"title\":\"Server Success Rate (non-5xx responses)\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"avg\"},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":4,\"w\":6,\"x\":12,\"y\":8},\"hiddenSeries\":false,\"id\":99,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":false,\"hideZero\":false,\"max\":false,\"min\":false,\"rightSide\":true,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le)) / 1000) or histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le))\",\"format\":\"time_series\",\"interval\":\"\",\"intervalFactor\":1,\"legendFormat\":\"P50\",\"refId\":\"A\"},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le)) / 1000) or histogram_quantile(0.90, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"P90\",\"refId\":\"B\"},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le)) / 1000) or histogram_quantile(0.99, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\",destination_service=~\\\"$service\\\"}[1m])) + by (le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"P99\",\"refId\":\"C\"}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Server + Request Duration\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"s\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"#299c46\",\"rgba(237, + 129, 40, 0.89)\",\"#d44a3a\"],\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"Bps\",\"gauge\":{\"maxValue\":100,\"minValue\":0,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":true},\"gridPos\":{\"h\":4,\"w\":6,\"x\":18,\"y\":8},\"id\":100,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"sum(irate(istio_tcp_sent_bytes_total{reporter=~\\\"$qrep\\\", + destination_service=~\\\"$service\\\"}[1m]))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"\",\"refId\":\"A\"}],\"thresholds\":\"\",\"title\":\"TCP + Sent Bytes\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"avg\"}],\"title\":\"General\",\"type\":\"row\"},{\"collapsed\":true,\"gridPos\":{\"h\":1,\"w\":24,\"x\":0,\"y\":1},\"id\":104,\"panels\":[{\"content\":\"
\\nCLIENT WORKLOADS\\n
\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"gridPos\":{\"h\":3,\"w\":24,\"x\":0,\"y\":2},\"id\":45,\"links\":[],\"mode\":\"html\",\"options\":{\"content\":\"
\\nCLIENT WORKLOADS\\n
\",\"mode\":\"html\"},\"pluginVersion\":\"7.1.0\",\"title\":\"\",\"transparent\":true,\"type\":\"text\"},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":0,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":0,\"y\":5},\"hiddenSeries\":false,\"id\":25,\"legend\":{\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null + as zero\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_requests_total{connection_security_policy=\\\"mutual_tls\\\",destination_service=~\\\"$service\\\",reporter=~\\\"$qrep\\\",source_workload=~\\\"$srcwl\\\",source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace, response_code), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace }} : {{ response_code }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_requests_total{connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", reporter=~\\\"$qrep\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[5m])) by (source_workload, source_workload_namespace, + response_code), 0.001)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace }} : {{ response_code }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Requests By Source And Response Code\",\"tooltip\":{\"shared\":false,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[\"total\"]},\"yaxes\":[{\"format\":\"ops\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":12,\"y\":5},\"hiddenSeries\":false,\"id\":26,\"legend\":{\"avg\":false,\"current\":false,\"hideEmpty\":true,\"hideZero\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\",response_code!~\\\"5.*\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace) / sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\",response_code!~\\\"5.*\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace) / sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Success Rate (non-5xx responses) By Source\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"percentunit\",\"label\":null,\"logBase\":1,\"max\":\"1.01\",\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"description\":\"\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":0,\"y\":11},\"hiddenSeries\":false,\"id\":27,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"hideZero\":false,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le)) / 1000) or histogram_quantile(0.50, + sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le)) / 1000) or histogram_quantile(0.90, + sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"(histogram_quantile(0.95, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le)) / 1000) or histogram_quantile(0.95, + sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le)) / 1000) or histogram_quantile(0.99, + sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le)) / 1000) or histogram_quantile(0.50, + sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"(histogram_quantile(0.90, sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le)) / 1000) or histogram_quantile(0.90, + sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"(histogram_quantile(0.95, sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le)) / 1000) or histogram_quantile(0.95, + sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"(histogram_quantile(0.99, sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le)) / 1000) or histogram_quantile(0.99, + sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Request Duration By Source\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"s\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":8,\"y\":11},\"hiddenSeries\":false,\"id\":28,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + \ P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + \ P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Request Size By Source\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"decbytes\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":16,\"y\":11},\"hiddenSeries\":false,\"id\":68,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + \ P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + \ P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Response + Size By Source\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"decbytes\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":0,\"y\":17},\"hiddenSeries\":false,\"id\":80,\"legend\":{\"avg\":false,\"current\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_tcp_received_bytes_total{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace), 0.001)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace}} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_tcp_received_bytes_total{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace}}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Bytes + Received from Incoming TCP Connection\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"Bps\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":12,\"y\":17},\"hiddenSeries\":false,\"id\":82,\"legend\":{\"avg\":false,\"current\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_tcp_sent_bytes_total{connection_security_policy=\\\"mutual_tls\\\", + reporter=~\\\"$qrep\\\", destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace), + 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace}} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_tcp_sent_bytes_total{connection_security_policy!=\\\"mutual_tls\\\", + reporter=~\\\"$qrep\\\", destination_service=~\\\"$service\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace), + 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace}}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Bytes + Sent to Incoming TCP Connection\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"Bps\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true}],\"yaxis\":{\"align\":false,\"alignLevel\":null}}],\"title\":\"Client + Workloads\",\"type\":\"row\"},{\"collapsed\":true,\"gridPos\":{\"h\":1,\"w\":24,\"x\":0,\"y\":2},\"id\":102,\"panels\":[{\"content\":\"
\\nSERVICE WORKLOADS\\n
\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"gridPos\":{\"h\":3,\"w\":24,\"x\":0,\"y\":3},\"id\":69,\"links\":[],\"mode\":\"html\",\"options\":{\"content\":\"
\\nSERVICE WORKLOADS\\n
\",\"mode\":\"html\"},\"pluginVersion\":\"7.1.0\",\"title\":\"\",\"transparent\":true,\"type\":\"text\"},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":0,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":0,\"y\":6},\"hiddenSeries\":false,\"id\":90,\"legend\":{\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null + as zero\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_requests_total{connection_security_policy=\\\"mutual_tls\\\",destination_service=~\\\"$service\\\",reporter=\\\"destination\\\",destination_workload=~\\\"$dstwl\\\",destination_workload_namespace=~\\\"$dstns\\\"}[5m])) + by (destination_workload, destination_workload_namespace, response_code), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} : {{ response_code + }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_requests_total{connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", reporter=\\\"destination\\\", destination_workload=~\\\"$dstwl\\\", + destination_workload_namespace=~\\\"$dstns\\\"}[5m])) by (destination_workload, + destination_workload_namespace, response_code), 0.001)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} : {{ response_code + }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Requests By Destination Workload And Response Code\",\"tooltip\":{\"shared\":false,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[\"total\"]},\"yaxes\":[{\"format\":\"ops\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":12,\"y\":6},\"hiddenSeries\":false,\"id\":91,\"legend\":{\"avg\":false,\"current\":false,\"hideEmpty\":true,\"hideZero\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"sum(irate(istio_requests_total{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\",response_code!~\\\"5.*\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[5m])) + by (destination_workload, destination_workload_namespace) / sum(irate(istio_requests_total{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[5m])) + by (destination_workload, destination_workload_namespace)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"sum(irate(istio_requests_total{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\",response_code!~\\\"5.*\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[5m])) + by (destination_workload, destination_workload_namespace) / sum(irate(istio_requests_total{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[5m])) + by (destination_workload, destination_workload_namespace)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Success Rate (non-5xx responses) By Destination Workload\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"percentunit\",\"label\":null,\"logBase\":1,\"max\":\"1.01\",\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"description\":\"\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":0,\"y\":12},\"hiddenSeries\":false,\"id\":94,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"hideZero\":false,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.50, + sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.90, + sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"(histogram_quantile(0.95, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.95, + sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.99, + sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.50, + sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.90, + sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"(histogram_quantile(0.95, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.95, + sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.99, + sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Request Duration By Service Workload\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"s\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":8,\"y\":12},\"hiddenSeries\":false,\"id\":95,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_request_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_request_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_request_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_request_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_request_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_request_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_request_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_request_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Request Size By Service Workload\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"decbytes\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":16,\"y\":12},\"hiddenSeries\":false,\"id\":96,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_response_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_response_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_response_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_response_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_response_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_response_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_response_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_response_bytes_bucket{reporter=\\\"destination\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace }} P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Response + Size By Service Workload\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"decbytes\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":0,\"y\":18},\"hiddenSeries\":false,\"id\":92,\"legend\":{\"avg\":false,\"current\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_tcp_received_bytes_total{reporter=\\\"destination\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace), 0.001)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace}} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_tcp_received_bytes_total{reporter=\\\"destination\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_service=~\\\"$service\\\", + destination_workload=~\\\"$dstwl\\\", destination_workload_namespace=~\\\"$dstns\\\"}[1m])) + by (destination_workload, destination_workload_namespace), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{ destination_workload_namespace}}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Bytes + Received from Incoming TCP Connection\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"Bps\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":12,\"y\":18},\"hiddenSeries\":false,\"id\":93,\"legend\":{\"avg\":false,\"current\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_tcp_sent_bytes_total{connection_security_policy=\\\"mutual_tls\\\", + reporter=\\\"destination\\\", destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", + destination_workload_namespace=~\\\"$dstns\\\"}[1m])) by (destination_workload, + destination_workload_namespace), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{destination_workload_namespace }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_tcp_sent_bytes_total{connection_security_policy!=\\\"mutual_tls\\\", + reporter=\\\"destination\\\", destination_service=~\\\"$service\\\", destination_workload=~\\\"$dstwl\\\", + destination_workload_namespace=~\\\"$dstns\\\"}[1m])) by (destination_workload, + destination_workload_namespace), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_workload }}.{{destination_workload_namespace }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Bytes + Sent to Incoming TCP Connection\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"Bps\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true}],\"yaxis\":{\"align\":false,\"alignLevel\":null}}],\"title\":\"Service + Workloads\",\"type\":\"row\"}],\"refresh\":\"1m\",\"schemaVersion\":26,\"style\":\"dark\",\"tags\":[],\"templating\":{\"list\":[{\"current\":{\"selected\":true,\"text\":\"default\",\"value\":\"default\"},\"hide\":0,\"includeAll\":false,\"label\":null,\"multi\":false,\"name\":\"datasource\",\"options\":[],\"query\":\"prometheus\",\"queryValue\":\"\",\"refresh\":1,\"regex\":\"\",\"skipUrlSync\":false,\"type\":\"datasource\"},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":false,\"label\":\"Service\",\"multi\":false,\"name\":\"service\",\"options\":[],\"query\":\"label_values(destination_service)\",\"refresh\":1,\"regex\":\"\",\"skipUrlSync\":false,\"sort\":0,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{\"selected\":true,\"text\":\"destination\",\"value\":\"destination\"},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":false,\"label\":\"Reporter\",\"multi\":true,\"name\":\"qrep\",\"options\":[],\"query\":\"label_values(reporter)\",\"refresh\":1,\"regex\":\"\",\"skipUrlSync\":false,\"sort\":1,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Client + Cluster\",\"multi\":true,\"name\":\"srccluster\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total{reporter=~\\\"$qrep\\\", + destination_service=\\\"$service\\\"}) by (source_cluster) or sum(istio_tcp_sent_bytes_total{reporter=~\\\"$qrep\\\", + destination_service=~\\\"$service\\\"}) by (source_cluster))\",\"refresh\":1,\"regex\":\"/.*cluster=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":2,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Client + Workload Namespace\",\"multi\":true,\"name\":\"srcns\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total{reporter=~\\\"$qrep\\\", + destination_service=\\\"$service\\\"}) by (source_workload_namespace) or sum(istio_tcp_sent_bytes_total{reporter=~\\\"$qrep\\\", + destination_service=~\\\"$service\\\"}) by (source_workload_namespace))\",\"refresh\":1,\"regex\":\"/.*namespace=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":3,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Client + Workload\",\"multi\":true,\"name\":\"srcwl\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total{reporter=~\\\"$qrep\\\", + destination_service=~\\\"$service\\\", source_workload_namespace=~\\\"$srcns\\\"}) + by (source_workload) or sum(istio_tcp_sent_bytes_total{reporter=~\\\"$qrep\\\", + destination_service=~\\\"$service\\\", source_workload_namespace=~\\\"$srcns\\\"}) + by (source_workload))\",\"refresh\":1,\"regex\":\"/.*workload=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":4,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Service + Workload Cluster\",\"multi\":true,\"name\":\"dstcluster\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total{reporter=\\\"destination\\\", + destination_service=\\\"$service\\\"}) by (destination_cluster) or sum(istio_tcp_sent_bytes_total{reporter=\\\"destination\\\", + destination_service=~\\\"$service\\\"}) by (destination_cluster))\",\"refresh\":1,\"regex\":\"/.*cluster=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":2,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Service + Workload Namespace\",\"multi\":true,\"name\":\"dstns\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total{reporter=\\\"destination\\\", + destination_service=\\\"$service\\\"}) by (destination_workload_namespace) or + sum(istio_tcp_sent_bytes_total{reporter=\\\"destination\\\", destination_service=~\\\"$service\\\"}) + by (destination_workload_namespace))\",\"refresh\":1,\"regex\":\"/.*namespace=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":3,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Service + Workload\",\"multi\":true,\"name\":\"dstwl\",\"options\":[],\"query\":\"query_result( + sum(istio_requests_total{reporter=\\\"destination\\\", destination_service=~\\\"$service\\\", + destination_cluster=~\\\"$dstcluster\\\", destination_workload_namespace=~\\\"$dstns\\\"}) + by (destination_workload) or sum(istio_tcp_sent_bytes_total{reporter=\\\"destination\\\", + destination_service=~\\\"$service\\\", destination_cluster=~\\\"$dstcluster\\\", + destination_workload_namespace=~\\\"$dstns\\\"}) by (destination_workload))\",\"refresh\":1,\"regex\":\"/.*workload=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":4,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false}]},\"time\":{\"from\":\"now-5m\",\"to\":\"now\"},\"timepicker\":{\"refresh_intervals\":[\"5m\",\"15m\",\"30m\",\"1h\",\"2h\",\"1d\"],\"time_options\":[\"5m\",\"15m\",\"1h\",\"6h\",\"12h\",\"24h\",\"2d\",\"7d\",\"30d\"]},\"timezone\":\"\",\"title\":\"Istio + Service Dashboard\",\"uid\":\"LJ_uJAvmk\",\"version\":1}\n" + istio-workload-dashboard.json: "{\"annotations\":{\"list\":[{\"builtIn\":1,\"datasource\":\"-- + Grafana --\",\"enable\":true,\"hide\":true,\"iconColor\":\"rgba(0, 211, 255, 1)\",\"name\":\"Annotations + & Alerts\",\"type\":\"dashboard\"}]},\"editable\":false,\"gnetId\":null,\"graphTooltip\":0,\"iteration\":1531345461465,\"links\":[],\"panels\":[{\"collapsed\":true,\"gridPos\":{\"h\":1,\"w\":24,\"x\":0,\"y\":0},\"id\":95,\"panels\":[{\"content\":\"
\\nWORKLOAD: $workload.$namespace\\n
\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"gridPos\":{\"h\":3,\"w\":24,\"x\":0,\"y\":1},\"id\":89,\"links\":[],\"mode\":\"html\",\"options\":{\"content\":\"
\\nWORKLOAD: $workload.$namespace\\n
\",\"mode\":\"html\"},\"pluginVersion\":\"7.1.0\",\"title\":\"\",\"transparent\":true,\"type\":\"text\"},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"rgba(245, + 54, 54, 0.9)\",\"rgba(237, 129, 40, 0.89)\",\"rgba(50, 172, 45, 0.97)\"],\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"ops\",\"gauge\":{\"maxValue\":100,\"minValue\":0,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":true},\"gridPos\":{\"h\":4,\"w\":8,\"x\":0,\"y\":4},\"id\":12,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"round(sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\",destination_workload_namespace=~\\\"$namespace\\\",destination_workload=~\\\"$workload\\\"}[5m])), + 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"refId\":\"A\",\"step\":4}],\"thresholds\":\"\",\"title\":\"Incoming + Request Volume\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"current\"},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"rgba(50, + 172, 45, 0.97)\",\"rgba(237, 129, 40, 0.89)\",\"rgba(245, 54, 54, 0.9)\"],\"datasource\":\"Prometheus\",\"decimals\":null,\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"percentunit\",\"gauge\":{\"maxValue\":100,\"minValue\":80,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":false},\"gridPos\":{\"h\":4,\"w\":8,\"x\":8,\"y\":4},\"id\":14,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\",destination_workload_namespace=~\\\"$namespace\\\",destination_workload=~\\\"$workload\\\",response_code!~\\\"5.*\\\"}[5m])) + / sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\",destination_workload_namespace=~\\\"$namespace\\\",destination_workload=~\\\"$workload\\\"}[5m]))\",\"format\":\"time_series\",\"intervalFactor\":1,\"refId\":\"A\"}],\"thresholds\":\"95, + 99, 99.5\",\"title\":\"Incoming Success Rate (non-5xx responses)\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"avg\"},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":4,\"w\":8,\"x\":16,\"y\":4},\"hiddenSeries\":false,\"id\":87,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":false,\"hideZero\":false,\"max\":false,\"min\":false,\"rightSide\":true,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\",destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\"}[1m])) by (le)) / 1000) or + histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\",destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\"}[1m])) by (le))\",\"format\":\"time_series\",\"interval\":\"\",\"intervalFactor\":1,\"legendFormat\":\"P50\",\"refId\":\"A\"},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\",destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\"}[1m])) by (le)) / 1000) or + histogram_quantile(0.90, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\",destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\"}[1m])) by (le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"P90\",\"refId\":\"B\"},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\",destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\"}[1m])) by (le)) / 1000) or + histogram_quantile(0.99, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\",destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\"}[1m])) by (le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"P99\",\"refId\":\"C\"}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Request + Duration\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"s\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"#299c46\",\"rgba(237, + 129, 40, 0.89)\",\"#d44a3a\"],\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"Bps\",\"gauge\":{\"maxValue\":100,\"minValue\":0,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":true},\"gridPos\":{\"h\":4,\"w\":12,\"x\":0,\"y\":8},\"id\":84,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"sum(irate(istio_tcp_sent_bytes_total{reporter=~\\\"$qrep\\\", + destination_workload_namespace=~\\\"$namespace\\\", destination_workload=~\\\"$workload\\\"}[1m])) + + sum(irate(istio_tcp_received_bytes_total{reporter=~\\\"$qrep\\\", destination_workload_namespace=~\\\"$namespace\\\", + destination_workload=~\\\"$workload\\\"}[1m]))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"\",\"refId\":\"A\"}],\"thresholds\":\"\",\"title\":\"TCP + Server Traffic\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"avg\"},{\"cacheTimeout\":null,\"colorBackground\":false,\"colorValue\":false,\"colors\":[\"#299c46\",\"rgba(237, + 129, 40, 0.89)\",\"#d44a3a\"],\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"format\":\"Bps\",\"gauge\":{\"maxValue\":100,\"minValue\":0,\"show\":false,\"thresholdLabels\":false,\"thresholdMarkers\":true},\"gridPos\":{\"h\":4,\"w\":12,\"x\":12,\"y\":8},\"id\":85,\"interval\":null,\"links\":[],\"options\":{\"colorMode\":\"value\",\"graphMode\":\"area\",\"justifyMode\":\"auto\",\"orientation\":\"horizontal\",\"reduceOptions\":{\"calcs\":[\"lastNotNull\"],\"fields\":\"\",\"values\":false},\"textMode\":\"auto\"},\"mappingType\":1,\"mappingTypes\":[{\"name\":\"value + to text\",\"value\":1},{\"name\":\"range to text\",\"value\":2}],\"maxDataPoints\":100,\"nullPointMode\":\"connected\",\"nullText\":null,\"postfix\":\"\",\"postfixFontSize\":\"50%\",\"prefix\":\"\",\"prefixFontSize\":\"50%\",\"rangeMaps\":[{\"from\":\"null\",\"text\":\"N/A\",\"to\":\"null\"}],\"sparkline\":{\"fillColor\":\"rgba(31, + 118, 189, 0.18)\",\"full\":true,\"lineColor\":\"rgb(31, 120, 193)\",\"show\":true},\"tableColumn\":\"\",\"targets\":[{\"expr\":\"sum(irate(istio_tcp_sent_bytes_total{reporter=~\\\"$qrep\\\", + source_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$workload\\\"}[1m])) + + sum(irate(istio_tcp_received_bytes_total{reporter=~\\\"$qrep\\\", source_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$workload\\\"}[1m]))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"\",\"refId\":\"A\"}],\"thresholds\":\"\",\"title\":\"TCP + Client Traffic\",\"type\":\"singlestat\",\"valueFontSize\":\"80%\",\"valueMaps\":[{\"op\":\"=\",\"text\":\"N/A\",\"value\":\"null\"}],\"valueName\":\"avg\"}],\"title\":\"General\",\"type\":\"row\"},{\"collapsed\":true,\"gridPos\":{\"h\":1,\"w\":24,\"x\":0,\"y\":1},\"id\":93,\"panels\":[{\"content\":\"
\\nINBOUND WORKLOADS\\n
\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"gridPos\":{\"h\":3,\"w\":24,\"x\":0,\"y\":13},\"id\":45,\"links\":[],\"mode\":\"html\",\"options\":{\"content\":\"
\\nINBOUND WORKLOADS\\n
\",\"mode\":\"html\"},\"pluginVersion\":\"7.1.0\",\"title\":\"\",\"transparent\":true,\"type\":\"text\"},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":0,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":0,\"y\":16},\"hiddenSeries\":false,\"id\":25,\"legend\":{\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null + as zero\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_requests_total{connection_security_policy=\\\"mutual_tls\\\", + destination_workload_namespace=~\\\"$namespace\\\", destination_workload=~\\\"$workload\\\", + reporter=~\\\"$qrep\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace, response_code), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace }} : {{ response_code }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_requests_total{connection_security_policy!=\\\"mutual_tls\\\", + destination_workload_namespace=~\\\"$namespace\\\", destination_workload=~\\\"$workload\\\", + reporter=~\\\"$qrep\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace, response_code), 0.001)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace }} : {{ response_code }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Requests By Source And Response Code\",\"tooltip\":{\"shared\":false,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[\"total\"]},\"yaxes\":[{\"format\":\"ops\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":12,\"y\":16},\"hiddenSeries\":false,\"id\":26,\"legend\":{\"avg\":false,\"current\":false,\"hideEmpty\":true,\"hideZero\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload_namespace=~\\\"$namespace\\\", + destination_workload=~\\\"$workload\\\",response_code!~\\\"5.*\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[5m])) by (source_workload, source_workload_namespace) + / sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload_namespace=~\\\"$namespace\\\", destination_workload=~\\\"$workload\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload_namespace=~\\\"$namespace\\\", + destination_workload=~\\\"$workload\\\",response_code!~\\\"5.*\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[5m])) by (source_workload, source_workload_namespace) + / sum(irate(istio_requests_total{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_workload_namespace=~\\\"$namespace\\\", destination_workload=~\\\"$workload\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[5m])) + by (source_workload, source_workload_namespace)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Success Rate (non-5xx responses) By Source\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"percentunit\",\"label\":null,\"logBase\":1,\"max\":\"1.01\",\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"description\":\"\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":0,\"y\":22},\"hiddenSeries\":false,\"id\":27,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"hideZero\":false,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le)) / 1000) or histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le)) / 1000) or histogram_quantile(0.90, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"(histogram_quantile(0.95, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le)) / 1000) or histogram_quantile(0.95, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le)) / 1000) or histogram_quantile(0.99, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le)) / 1000) or histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"(histogram_quantile(0.90, sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le)) / 1000) or histogram_quantile(0.90, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"(histogram_quantile(0.95, sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le)) / 1000) or histogram_quantile(0.95, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"(histogram_quantile(0.99, sum(irate(istio_request_duration_milliseconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le)) / 1000) or histogram_quantile(0.99, sum(irate(istio_request_duration_seconds_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Request Duration By Source\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"s\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":8,\"y\":22},\"hiddenSeries\":false,\"id\":28,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + \ P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + \ P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, sum(irate(istio_request_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Incoming + Request Size By Source\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"decbytes\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":16,\"y\":22},\"hiddenSeries\":false,\"id\":68,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + \ P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + \ P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", connection_security_policy!=\\\"mutual_tls\\\", + destination_workload=~\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, sum(irate(istio_response_bytes_bucket{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload=~\\\"$workload\\\", + destination_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$srcwl\\\", + source_workload_namespace=~\\\"$srcns\\\"}[1m])) by (source_workload, source_workload_namespace, + le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{source_workload}}.{{source_workload_namespace}} + P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Response + Size By Source\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"decbytes\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":0,\"y\":28},\"hiddenSeries\":false,\"id\":80,\"legend\":{\"avg\":false,\"current\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_tcp_received_bytes_total{reporter=~\\\"$qrep\\\", + connection_security_policy=\\\"mutual_tls\\\", destination_workload_namespace=~\\\"$namespace\\\", + destination_workload=~\\\"$workload\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace), 0.001)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace}} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_tcp_received_bytes_total{reporter=~\\\"$qrep\\\", + connection_security_policy!=\\\"mutual_tls\\\", destination_workload_namespace=~\\\"$namespace\\\", + destination_workload=~\\\"$workload\\\", source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace}}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Bytes + Received from Incoming TCP Connection\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"Bps\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":12,\"y\":28},\"hiddenSeries\":false,\"id\":82,\"legend\":{\"avg\":false,\"current\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_tcp_sent_bytes_total{connection_security_policy=\\\"mutual_tls\\\", + reporter=~\\\"$qrep\\\", destination_workload_namespace=~\\\"$namespace\\\", destination_workload=~\\\"$workload\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace}} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_tcp_sent_bytes_total{connection_security_policy!=\\\"mutual_tls\\\", + reporter=~\\\"$qrep\\\", destination_workload_namespace=~\\\"$namespace\\\", destination_workload=~\\\"$workload\\\", + source_workload=~\\\"$srcwl\\\", source_workload_namespace=~\\\"$srcns\\\"}[1m])) + by (source_workload, source_workload_namespace), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + source_workload }}.{{ source_workload_namespace}}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Bytes + Sent to Incoming TCP Connection\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"Bps\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true}],\"yaxis\":{\"align\":false,\"alignLevel\":null}}],\"title\":\"Inbound + Workloads\",\"type\":\"row\"},{\"collapsed\":true,\"gridPos\":{\"h\":1,\"w\":24,\"x\":0,\"y\":2},\"id\":91,\"panels\":[{\"content\":\"
\\nOUTBOUND SERVICES\\n
\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"gridPos\":{\"h\":3,\"w\":24,\"x\":0,\"y\":14},\"id\":69,\"links\":[],\"mode\":\"html\",\"options\":{\"content\":\"
\\nOUTBOUND SERVICES\\n
\",\"mode\":\"html\"},\"pluginVersion\":\"7.1.0\",\"title\":\"\",\"transparent\":true,\"type\":\"text\"},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":0,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":0,\"y\":17},\"hiddenSeries\":false,\"id\":70,\"legend\":{\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null + as zero\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_requests_total{destination_principal=~\\\"spiffe.*\\\", + source_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$workload\\\", + reporter=\\\"source\\\", destination_service=~\\\"$dstsvc\\\"}[5m])) by (destination_service, + response_code), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} : {{ response_code }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_requests_total{destination_principal!~\\\"spiffe.*\\\", + source_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$workload\\\", + reporter=\\\"source\\\", destination_service=~\\\"$dstsvc\\\"}[5m])) by (destination_service, + response_code), 0.001)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} : {{ response_code }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Outgoing + Requests By Destination And Response Code\",\"tooltip\":{\"shared\":false,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[\"total\"]},\"yaxes\":[{\"format\":\"ops\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":12,\"y\":17},\"hiddenSeries\":false,\"id\":71,\"legend\":{\"avg\":false,\"current\":false,\"hideEmpty\":true,\"hideZero\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"sum(irate(istio_requests_total{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$workload\\\",response_code!~\\\"5.*\\\", destination_service=~\\\"$dstsvc\\\"}[5m])) + by (destination_service) / sum(irate(istio_requests_total{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$workload\\\", destination_service=~\\\"$dstsvc\\\"}[5m])) + by (destination_service)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"sum(irate(istio_requests_total{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$workload\\\",response_code!~\\\"5.*\\\", destination_service=~\\\"$dstsvc\\\"}[5m])) + by (destination_service) / sum(irate(istio_requests_total{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$workload\\\", destination_service=~\\\"$dstsvc\\\"}[5m])) + by (destination_service)\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Outgoing + Success Rate (non-5xx responses) By Destination\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"percentunit\",\"label\":null,\"logBase\":1,\"max\":\"1.01\",\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"description\":\"\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":0,\"y\":23},\"hiddenSeries\":false,\"id\":72,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"hideZero\":false,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le)) / 1000) or histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le)) / 1000) or histogram_quantile(0.90, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"(histogram_quantile(0.95, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le)) / 1000) or histogram_quantile(0.95, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le)) / 1000) or histogram_quantile(0.99, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"(histogram_quantile(0.50, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le)) / 1000) or histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"(histogram_quantile(0.90, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le)) / 1000) or histogram_quantile(0.90, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"(histogram_quantile(0.95, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le)) / 1000) or histogram_quantile(0.95, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"(histogram_quantile(0.99, + sum(irate(istio_request_duration_milliseconds_bucket{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le)) / 1000) or histogram_quantile(0.99, sum(irate(istio_request_duration_seconds_bucket{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload=~\\\"$workload\\\", + source_workload_namespace=~\\\"$namespace\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Outgoing + Request Duration By Destination\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"s\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":8,\"y\":23},\"hiddenSeries\":false,\"id\":73,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_request_bytes_bucket{reporter=\\\"source\\\", connection_security_policy=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_request_bytes_bucket{reporter=\\\"source\\\", connection_security_policy=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_request_bytes_bucket{reporter=\\\"source\\\", connection_security_policy=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_request_bytes_bucket{reporter=\\\"source\\\", connection_security_policy=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_request_bytes_bucket{reporter=\\\"source\\\", connection_security_policy!=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_request_bytes_bucket{reporter=\\\"source\\\", connection_security_policy!=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_request_bytes_bucket{reporter=\\\"source\\\", connection_security_policy!=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_request_bytes_bucket{reporter=\\\"source\\\", connection_security_policy!=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Outgoing + Request Size By Destination\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"decbytes\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":8,\"x\":16,\"y\":23},\"hiddenSeries\":false,\"id\":74,\"legend\":{\"alignAsTable\":false,\"avg\":false,\"current\":false,\"hideEmpty\":true,\"max\":false,\"min\":false,\"rightSide\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_response_bytes_bucket{reporter=\\\"source\\\", connection_security_policy=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P50 (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_response_bytes_bucket{reporter=\\\"source\\\", connection_security_policy=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P90 (\U0001F510mTLS)\",\"refId\":\"B\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_response_bytes_bucket{reporter=\\\"source\\\", connection_security_policy=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P95 (\U0001F510mTLS)\",\"refId\":\"C\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_response_bytes_bucket{reporter=\\\"source\\\", connection_security_policy=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P99 (\U0001F510mTLS)\",\"refId\":\"D\",\"step\":2},{\"expr\":\"histogram_quantile(0.50, + sum(irate(istio_response_bytes_bucket{reporter=\\\"source\\\", connection_security_policy!=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P50\",\"refId\":\"E\",\"step\":2},{\"expr\":\"histogram_quantile(0.90, + sum(irate(istio_response_bytes_bucket{reporter=\\\"source\\\", connection_security_policy!=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P90\",\"refId\":\"F\",\"step\":2},{\"expr\":\"histogram_quantile(0.95, + sum(irate(istio_response_bytes_bucket{reporter=\\\"source\\\", connection_security_policy!=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P95\",\"refId\":\"G\",\"step\":2},{\"expr\":\"histogram_quantile(0.99, + sum(irate(istio_response_bytes_bucket{reporter=\\\"source\\\", connection_security_policy!=\\\"mutual_tls\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service, le))\",\"format\":\"time_series\",\"hide\":false,\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} P99\",\"refId\":\"H\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Response + Size By Destination\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"decbytes\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":false}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":0,\"y\":29},\"hiddenSeries\":false,\"id\":76,\"legend\":{\"avg\":false,\"current\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_tcp_received_bytes_total{connection_security_policy=\\\"mutual_tls\\\", + reporter=\\\"source\\\", source_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$workload\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_tcp_received_bytes_total{connection_security_policy!=\\\"mutual_tls\\\", + reporter=\\\"source\\\", source_workload_namespace=~\\\"$namespace\\\", source_workload=~\\\"$workload\\\", + destination_service=~\\\"$dstsvc\\\"}[1m])) by (destination_service), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Bytes + Sent on Outgoing TCP Connection\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"Bps\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true}],\"yaxis\":{\"align\":false,\"alignLevel\":null}},{\"aliasColors\":{},\"bars\":false,\"dashLength\":10,\"dashes\":false,\"datasource\":\"Prometheus\",\"fieldConfig\":{\"defaults\":{\"custom\":{}},\"overrides\":[]},\"fill\":1,\"fillGradient\":0,\"gridPos\":{\"h\":6,\"w\":12,\"x\":12,\"y\":29},\"hiddenSeries\":false,\"id\":78,\"legend\":{\"avg\":false,\"current\":false,\"max\":false,\"min\":false,\"show\":true,\"total\":false,\"values\":false},\"lines\":true,\"linewidth\":1,\"links\":[],\"nullPointMode\":\"null\",\"percentage\":false,\"pluginVersion\":\"7.1.0\",\"pointradius\":5,\"points\":false,\"renderer\":\"flot\",\"seriesOverrides\":[],\"spaceLength\":10,\"stack\":false,\"steppedLine\":false,\"targets\":[{\"expr\":\"round(sum(irate(istio_tcp_sent_bytes_total{reporter=\\\"source\\\", + connection_security_policy=\\\"mutual_tls\\\", source_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$workload\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }} (\U0001F510mTLS)\",\"refId\":\"A\",\"step\":2},{\"expr\":\"round(sum(irate(istio_tcp_sent_bytes_total{reporter=\\\"source\\\", + connection_security_policy!=\\\"mutual_tls\\\", source_workload_namespace=~\\\"$namespace\\\", + source_workload=~\\\"$workload\\\", destination_service=~\\\"$dstsvc\\\"}[1m])) + by (destination_service), 0.001)\",\"format\":\"time_series\",\"intervalFactor\":1,\"legendFormat\":\"{{ + destination_service }}\",\"refId\":\"B\",\"step\":2}],\"thresholds\":[],\"timeFrom\":null,\"timeRegions\":[],\"timeShift\":null,\"title\":\"Bytes + Received from Outgoing TCP Connection\",\"tooltip\":{\"shared\":true,\"sort\":0,\"value_type\":\"individual\"},\"type\":\"graph\",\"xaxis\":{\"buckets\":null,\"mode\":\"time\",\"name\":null,\"show\":true,\"values\":[]},\"yaxes\":[{\"format\":\"Bps\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":\"0\",\"show\":true},{\"format\":\"short\",\"label\":null,\"logBase\":1,\"max\":null,\"min\":null,\"show\":true}],\"yaxis\":{\"align\":false,\"alignLevel\":null}}],\"title\":\"Outbound + Services\",\"type\":\"row\"}],\"refresh\":\"1m\",\"schemaVersion\":26,\"style\":\"dark\",\"tags\":[],\"templating\":{\"list\":[{\"current\":{\"selected\":true,\"text\":\"default\",\"value\":\"default\"},\"hide\":0,\"includeAll\":false,\"label\":null,\"multi\":false,\"name\":\"datasource\",\"options\":[],\"query\":\"prometheus\",\"queryValue\":\"\",\"refresh\":1,\"regex\":\"\",\"skipUrlSync\":false,\"type\":\"datasource\"},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":false,\"label\":\"Namespace\",\"multi\":false,\"name\":\"namespace\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total) + by (destination_workload_namespace) or sum(istio_tcp_sent_bytes_total) by (destination_workload_namespace))\",\"refresh\":1,\"regex\":\"/.*_namespace=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":0,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":false,\"label\":\"Workload\",\"multi\":false,\"name\":\"workload\",\"options\":[],\"query\":\"query_result((sum(istio_requests_total{destination_workload_namespace=~\\\"$namespace\\\"}) + by (destination_workload) or sum(istio_requests_total{source_workload_namespace=~\\\"$namespace\\\"}) + by (source_workload)) or (sum(istio_tcp_sent_bytes_total{destination_workload_namespace=~\\\"$namespace\\\"}) + by (destination_workload) or sum(istio_tcp_sent_bytes_total{source_workload_namespace=~\\\"$namespace\\\"}) + by (source_workload)))\",\"refresh\":1,\"regex\":\"/.*workload=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":1,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{\"selected\":true,\"text\":\"destination\",\"value\":\"destination\"},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":false,\"label\":\"Reporter\",\"multi\":true,\"name\":\"qrep\",\"options\":[],\"query\":\"label_values(reporter)\",\"refresh\":1,\"regex\":\"\",\"skipUrlSync\":false,\"sort\":2,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Inbound + Workload Namespace\",\"multi\":true,\"name\":\"srcns\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total{reporter=~\\\"$qrep\\\", + destination_workload=\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\"}) + by (source_workload_namespace) or sum(istio_tcp_sent_bytes_total{reporter=~\\\"$qrep\\\", + destination_workload=\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\"}) + by (source_workload_namespace))\",\"refresh\":1,\"regex\":\"/.*namespace=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":2,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Inbound + Workload\",\"multi\":true,\"name\":\"srcwl\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total{reporter=~\\\"$qrep\\\", + destination_workload=\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload_namespace=~\\\"$srcns\\\"}) by (source_workload) or sum(istio_tcp_sent_bytes_total{reporter=~\\\"$qrep\\\", + destination_workload=\\\"$workload\\\", destination_workload_namespace=~\\\"$namespace\\\", + source_workload_namespace=~\\\"$srcns\\\"}) by (source_workload))\",\"refresh\":1,\"regex\":\"/.*workload=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":3,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false},{\"allValue\":null,\"current\":{},\"datasource\":\"Prometheus\",\"definition\":\"\",\"hide\":0,\"includeAll\":true,\"label\":\"Destination + Service\",\"multi\":true,\"name\":\"dstsvc\",\"options\":[],\"query\":\"query_result(sum(istio_requests_total{reporter=\\\"source\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\"}) + by (destination_service) or sum(istio_tcp_sent_bytes_total{reporter=\\\"source\\\", + source_workload=~\\\"$workload\\\", source_workload_namespace=~\\\"$namespace\\\"}) + by (destination_service))\",\"refresh\":1,\"regex\":\"/.*destination_service=\\\"([^\\\"]*).*/\",\"skipUrlSync\":false,\"sort\":4,\"tagValuesQuery\":\"\",\"tags\":[],\"tagsQuery\":\"\",\"type\":\"query\",\"useTags\":false}]},\"time\":{\"from\":\"now-5m\",\"to\":\"now\"},\"timepicker\":{\"refresh_intervals\":[\"5m\",\"15m\",\"30m\",\"1h\",\"2h\",\"1d\"],\"time_options\":[\"5m\",\"15m\",\"1h\",\"6h\",\"12h\",\"24h\",\"2d\",\"7d\",\"30d\"]},\"timezone\":\"\",\"title\":\"Istio + Workload Dashboard\",\"uid\":\"UbsSZTDik\",\"version\":1}\n" + diff --git a/prometheus/istio/podmonitor-istio-proxy.yaml b/prometheus/istio/podmonitor-istio-proxy.yaml new file mode 100644 index 00000000..1489575a --- /dev/null +++ b/prometheus/istio/podmonitor-istio-proxy.yaml @@ -0,0 +1,13 @@ +apiVersion: monitoring.coreos.com/v1 +kind: PodMonitor +metadata: + name: istio-proxy +spec: + selector: + matchLabels: + security.istio.io/tlsMode: 'istio' + namespaceSelector: + any: true + podMetricsEndpoints: + - port: http-envoy-prom + path: /stats/prometheus diff --git a/prometheus/istio/servicemonitor-istiod.yaml b/prometheus/istio/servicemonitor-istiod.yaml new file mode 100644 index 00000000..caf93cd1 --- /dev/null +++ b/prometheus/istio/servicemonitor-istiod.yaml @@ -0,0 +1,13 @@ +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + name: istiod +spec: + selector: + matchExpressions: + - {key: istio, operator: In, values: [pilot]} + namespaceSelector: + matchNames: + - istio-system + endpoints: + - port: http-monitoring diff --git a/prometheus/values.yaml b/prometheus/values.yaml new file mode 100644 index 00000000..7a5083f7 --- /dev/null +++ b/prometheus/values.yaml @@ -0,0 +1,176 @@ +####### This block is required to run in parallel with a Kyma monitoring stack +prometheusOperator: + # exclude the kyma-system Namespace from watching to avoid conflict with the Kyma monitoring stack + denyNamespaces: + - kyma-system + # re-use the kublet service definition of the Kyma monitoring stack. + kubeletService: + enabled: false + +# change the port of the node-export to be different from the one used by the Kyma monitoring stack +prometheus-node-exporter: + service: + port: 9101 + targetPort: 9101 + annotations: +####### This block disables prometheus annotations on monitoring components in order to be able to enable scraping via annotation for all workloads + prometheus.io/scrape: "false" + +kube-state-metrics: + prometheusScrape: false + +####### This block disables control plane components which are not reachable from within the Gardener data plane + +# Disable scraping of etcd +kubeEtcd: + enabled: false + +# Disable scraping of kubeControllerManager +kubeControllerManager: + enabled: false + +# Disable scraping of kubeProxy +kubeProxy: + enabled: false + +# Disable scraping of kubeScheduler +kubeScheduler: + enabled: false + +####### This block is required to enable scraping of endpoints with Istio strict mTLS, see also https://istio.io/latest/docs/ops/integrations/prometheus/#tls-settings +prometheus: + prometheusSpec: + podMetadata: + labels: + sidecar.istio.io/inject: "true" + annotations: + traffic.sidecar.istio.io/includeInboundPorts: "" # do not intercept any inbound ports + traffic.sidecar.istio.io/includeOutboundIPRanges: "" # do not intercept any outbound traffic + proxy.istio.io/config: | + # configure an env variable `OUTPUT_CERTS` to write certificates to the given folder + proxyMetadata: + OUTPUT_CERTS: /etc/istio-output-certs + sidecar.istio.io/userVolumeMount: '[{"name": "istio-certs", "mountPath": "/etc/istio-output-certs"}]' # mount the shared volume at sidecar proxy + + # Additional volumes on the output StatefulSet definition. + volumes: + - emptyDir: + medium: Memory + name: istio-certs + + # Additional VolumeMounts on the output StatefulSet definition. + volumeMounts: + - mountPath: /etc/prometheus/secrets/istio.default/ + name: istio-certs + +####### This block is needed to also use ServiceMonitors, which are not deployed as part of the chart + + # Disable required Helm release labels on ServiceMonitors + serviceMonitorSelectorNilUsesHelmValues: false + # Disable required Helm release labels on PodMonitors + podMonitorSelectorNilUsesHelmValues: false + # Disable required Helm release labels on Probes + probeSelectorNilUsesHelmValues: false + # Disable required Helm release labels on Rules + ruleSelectorNilUsesHelmValues: false + +####### This block enabled scrape discovery based on typical prometheus annotations + + additionalScrapeConfigs: + # Scrape config for service endpoints. + # + # The relabeling allows the actual service scrape endpoint to be configured + # via the following annotations: + # + # * `prometheus.io/scrape`: Only scrape services that have a value of `true` + # * `prometheus.io/scheme`: If the metrics endpoint is secured then you will need + # to set this to `https` & most likely set the `tls_config` of the scrape config. + # * `prometheus.io/path`: If the metrics path is not `/metrics` override this. + # * `prometheus.io/port`: If the metrics are exposed on a different port to the + # service then set this appropriately. + - job_name: 'kubernetes-service-endpoints' + + kubernetes_sd_configs: + - role: endpoints + + relabel_configs: + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape] + action: keep + regex: true + - source_labels: [__meta_kubernetes_pod_label_security_istio_io_tlsMode] + action: replace + target_label: __scheme__ + replacement: https + regex: (istio) + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme] + action: replace + target_label: __scheme__ + regex: (https?) + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path] + action: replace + target_label: __metrics_path__ + regex: (.+) + - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port] + action: replace + target_label: __address__ + regex: ([^:]+)(?::\d+)?;(\d+) + replacement: $1:$2 + - action: labelmap + regex: __meta_kubernetes_service_label_(.+) + - source_labels: [__meta_kubernetes_namespace] + action: drop + regex: kyma-system|kube-system|compass-system|kyma-integration + - source_labels: [__meta_kubernetes_namespace] + action: replace + target_label: kubernetes_namespace + - source_labels: [__meta_kubernetes_service_name] + action: replace + target_label: kubernetes_name + - source_labels: [__meta_kubernetes_service_name] + action: drop + regex: 'node-exporter' + tls_config: + ca_file: /etc/prometheus/secrets/istio.default/root-cert.pem + cert_file: /etc/prometheus/secrets/istio.default/cert-chain.pem + insecure_skip_verify: true + key_file: /etc/prometheus/secrets/istio.default/key.pem + + # Example scrape config for pods + # + # The relabeling allows the actual pod scrape endpoint to be configured via the + # following annotations: + # + # * `prometheus.io/scrape`: Only scrape pods that have a value of `true` + # * `prometheus.io/scheme`: If the metrics endpoint is secured then you will need + # to set this to `https` & most likely set the `tls_config` of the scrape config. + # * `prometheus.io/path`: If the metrics path is not `/metrics` override this. + # * `prometheus.io/port`: Scrape the pod on the indicated port instead of the default of `9102`. + - job_name: 'kubernetes-pods' + + kubernetes_sd_configs: + - role: pod + + relabel_configs: + - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape] + action: keep + regex: true + - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path] + action: replace + target_label: __metrics_path__ + regex: (.+) + - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port] + action: replace + regex: ([^:]+)(?::\d+)?;(\d+) + replacement: $1:$2 + target_label: __address__ + - action: labelmap + regex: __meta_kubernetes_pod_label_(.+) + - source_labels: [__meta_kubernetes_namespace] + action: drop + regex: kyma-system|kube-system|compass-system|kyma-integration + - source_labels: [__meta_kubernetes_namespace] + action: replace + target_label: kubernetes_namespace + - source_labels: [__meta_kubernetes_pod_name] + action: replace + target_label: kubernetes_pod_name