diff --git a/source/operations/monitoring/collect-minio-metrics-using-prometheus.rst b/source/operations/monitoring/collect-minio-metrics-using-prometheus.rst index 9b3c7c5b..cba9ca95 100644 --- a/source/operations/monitoring/collect-minio-metrics-using-prometheus.rst +++ b/source/operations/monitoring/collect-minio-metrics-using-prometheus.rst @@ -21,6 +21,9 @@ The procedure on this page documents the following: - Configuring a Prometheus service to scrape and display metrics from a MinIO deployment - Configuring an Alert Rule on a MinIO Metric to trigger an AlertManager action +These instructions use :ref:`version 2 metrics. ` +For more about metrics API versions, see :ref:`Metrics and alerts. ` + .. admonition:: Prerequisites :class: note @@ -32,8 +35,6 @@ The procedure on this page documents the following: - An :mc:`mc` installation on your local host configured to :ref:`access ` the MinIO deployment - These instructions use :ref:`version 2 metrics. ` - For more about metrics API versions, see :ref:`Metrics and alerts. ` Configure Prometheus to Collect and Alert using MinIO Metrics ------------------------------------------------------------- diff --git a/source/operations/monitoring/grafana.rst b/source/operations/monitoring/grafana.rst index 82f81ea5..88d0c92e 100644 --- a/source/operations/monitoring/grafana.rst +++ b/source/operations/monitoring/grafana.rst @@ -22,10 +22,12 @@ Prerequisites .. admonition:: Grafana dashboards use metrics version 2 :class: note - The MinIO Grafana dashboards are designed for use with :ref:`metrics version 2 `. + The MinIO Grafana dashboards use :ref:`metrics version 2 `. For more about metrics API versions, see :ref:`Metrics and alerts. ` - + Version 3 metrics require creating your own dashboard. + For more information about dashboards, see `the Grafana documentation. `__ + MinIO Grafana Dashboard ----------------------- diff --git a/source/operations/monitoring/healthcheck-probe.rst b/source/operations/monitoring/healthcheck-probe.rst index 5e2c205e..01fd60a9 100644 --- a/source/operations/monitoring/healthcheck-probe.rst +++ b/source/operations/monitoring/healthcheck-probe.rst @@ -33,8 +33,9 @@ A response code of ``200 OK`` indicates the MinIO server is online and functional. Any other HTTP codes indicate an issue with reaching the server, such as a transient network issue or potential downtime. -The healthcheck probe alone cannot determine if a MinIO server is offline - only that the current host machine cannot reach the server. -Consider configuring a Prometheus :ref:`alert ` using the :ref:`metrics v3 ` ``minio_cluster_health_nodes_offline_count`` or :ref:`v2 ` ``minio_cluster_nodes_offline_total`` metrics to detect whether one or more MinIO nodes are offline. +The healthcheck probe alone cannot determine if a MinIO server is offline. +Instead, the probe determines whether the current host machine can reach the server. +Consider configuring a Prometheus :ref:`alert ` using ``minio_cluster_health_nodes_offline_count`` for :ref:`metrics v3 ` or ``minio_cluster_nodes_offline_total`` for :ref:`metrics v2 ` to detect whether one or more MinIO nodes are offline. .. _minio-cluster-write-quorum: diff --git a/source/operations/monitoring/metrics-and-alerts.rst b/source/operations/monitoring/metrics-and-alerts.rst index c8af188c..79174a01 100644 --- a/source/operations/monitoring/metrics-and-alerts.rst +++ b/source/operations/monitoring/metrics-and-alerts.rst @@ -12,19 +12,24 @@ Metrics and alerts :local: :depth: 2 -.. admonition:: Metrics version 3 + +MinIO publishes metrics using the :prometheus-docs:`Prometheus Data Model `. +You can use any scraping tool to pull metrics data from MinIO for further analysis and alerting. + +Starting with MinIO Server :minio-release:`RELEASE.2024-07-15T19-02-30Z` and MinIO Client :mc-release:`RELEASE.2024-07-11T18-01-28Z`, metrics version 3 provides additional endpoints. +MinIO recommends version 3 for new deployments. + +.. admonition:: Metrics version 2 :class: note - Starting with MinIO Server :minio-release:`RELEASE.2024-07-15T19-02-30Z` and MinIO Client :mc-release:`RELEASE.2024-07-11T18-01-28Z`, metrics version 3 provides additional metrics scraping endpoints. - MinIO recommends version 3 for new deployments. + Existing deployments can continue to use version 2 :ref:`metrics ` and :ref:`Grafana dashboards `. - Existing deployments can continue to use :ref:`metrics version 2 ` and :ref:`the v2 Grafana dashboards `. -MinIO publishes cluster and node metrics using the :prometheus-docs:`Prometheus Data Model `. -You can use any scraping tool to pull metrics data from MinIO for further analysis and alerting. +Version 3 Endpoints +------------------- For metrics version 3, all metrics are available under the base ``/minio/metrics/v3`` endpoint. -You can scrape the base endpoint to collect all metrics in a single scraping operation, or append an optional path to scrape a single category. +You can scrape the base endpoint to collect all metrics in a single operation, or append an optional path to return a specific category. For example, the following endpoint returns audit metrics: @@ -102,7 +107,7 @@ MinIO provides the following scraping endpoints, relative to the base URL: ``/system/process`` -For a complete list of metrics for each endpoint, see :ref:`Available Metrics `. +For a complete list of metrics for each endpoint, see :ref:`Available version 3 etrics `. .. cond:: k8s @@ -123,8 +128,7 @@ For a complete list of metrics for each endpoint, see :ref:`Available Metrics ` - :ref:`Audit Metrics ` @@ -137,6 +141,7 @@ Each metric includes a label for the MinIO server which generated that metric. - :ref:`Scanner Metrics ` - :ref:`System Metrics ` +Many metrics include labels identifying the resource which generated that metric and other relevant details. .. _minio-available-v3-api-metrics: diff --git a/source/operations/monitoring/metrics-v2.rst b/source/operations/monitoring/metrics-v2.rst index 30115c0b..a6946bfd 100644 --- a/source/operations/monitoring/metrics-v2.rst +++ b/source/operations/monitoring/metrics-v2.rst @@ -14,6 +14,9 @@ Metrics version 2 MinIO publishes cluster and node metrics using the :prometheus-docs:`Prometheus Data Model `. You can use any scraping tool to pull metrics data from MinIO for further analysis and alerting. +Version 3 Endpoints +------------------- + Metrics version 2 provides metrics organized into three categories: - :ref:`Cluster Metrics ` @@ -31,7 +34,7 @@ For example, scraping the following endpoint returns all cluster metrics: The base endpoint alone, ``/minio/v2/metrics/``, returns cluster metrics. For more flexible scraping and a wider range of metrics, use :ref:`metrics version 3. ` -Existing deployments can continue to use :ref:`metrics version 2 ` and :ref:`the v2 Grafana dashboards `. + Existing deployments can continue to use version 2 :ref:`metrics ` and :ref:`Grafana dashboards `. MinIO Grafana dashboard diff --git a/source/reference/minio-mc-admin/mc-admin-prometheus-generate.rst b/source/reference/minio-mc-admin/mc-admin-prometheus-generate.rst index 89473e40..5da567c1 100644 --- a/source/reference/minio-mc-admin/mc-admin-prometheus-generate.rst +++ b/source/reference/minio-mc-admin/mc-admin-prometheus-generate.rst @@ -21,7 +21,7 @@ The :mc:`mc admin prometheus generate` command generates a metrics scraping conf For more complete documentation on using MinIO with Prometheus, see :ref:`How to monitor MinIO server with Prometheus ` -Starting with MinIO Server :minio-release:`RELEASE.2024-07-15T19-02-30Z` and MinIO Client :mc-release:`RELEASE.2024-07-11T18-01-28Z`, :ref:`metrics version 3 (v3) ` provides additional metrics. +Starting with MinIO Server :minio-release:`RELEASE.2024-07-15T19-02-30Z` and MinIO Client :mc-release:`RELEASE.2024-07-11T18-01-28Z`, :ref:`metrics version 3 (v3) ` provides additional endpoints and metrics. To generate a v3 scrape configuration use the ``--api_version v3`` option. MinIO recommends new deployments use :ref:`version 3 (v3) `. @@ -56,7 +56,7 @@ Existing deployments can continue to use :ref:`metrics version 2 --api_version v3] + [--bucket ] .. include:: /includes/common-minio-mc.rst :start-after: start-minio-syntax @@ -76,14 +76,16 @@ Parameters To generate a scrape configuration for :ref:`v3 metrics `, include an ``--api-version v3`` parameter. ``v3`` is the only accepted value. - + Omit ``--api-version`` to generate a :ref:`v2 metrics ` configuration. .. mc-cmd:: --bucket :optional: + Only valid for v3 metrics. + For v3 metric types that return bucket-level metrics, specify a bucket name. - Use with :mc-cmd:`~mc admin prometheus generate --api-version`. + Requires :mc-cmd:`~mc admin prometheus generate --api-version`. ``--bucket`` works for the following v3 metric types: @@ -125,7 +127,7 @@ Parameters - ``resource`` If not specified, a ``v2`` command returns cluster metrics. - Cluster metrics also include rollups of certain node metrics. + Cluster metrics include rollups of certain node metrics. Global flags @@ -163,10 +165,12 @@ The output resembles the following: static_configs: - targets: ['localhost:9000'] -Generate a v3 cluster metrics config -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Use :mc-cmd:`mc admin prometheus generate --api-version v3 ` to generate a scrape configuration that collects v3 cluster metrics for a MinIO deployment: +Generate a v3 metrics config for another type +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To generate a configuration for another metric type, specify the type. +The following generates a scrape configuration for v3 cluster metrics: .. code-block:: shell :class: copyable @@ -189,6 +193,7 @@ The output resembles the following: To generate a configuration for a :mc-cmd:`different metric type `, replace ``cluster`` with the desired type. + Generate a v3 bucket replication metrics config ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -237,6 +242,7 @@ The output resembles the following: static_configs: - targets: [`localhost:9000`] + Generate a default metrics v2 config ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -261,6 +267,10 @@ The output resembles the following: static_configs: - targets: ['localhost:9000'] + +Generate a v2 config for other metric types +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + To generate a configuration for another metric type, specify the type. The following generates a scrape configuration for v2 bucket metrics: @@ -268,3 +278,4 @@ The following generates a scrape configuration for v2 bucket metrics: :class: copyable mc admin prometheus generate ALIAS bucket + diff --git a/source/reference/minio-mc-admin/mc-admin-prometheus-metrics.rst b/source/reference/minio-mc-admin/mc-admin-prometheus-metrics.rst index 48e734c3..df344ae3 100644 --- a/source/reference/minio-mc-admin/mc-admin-prometheus-metrics.rst +++ b/source/reference/minio-mc-admin/mc-admin-prometheus-metrics.rst @@ -23,7 +23,7 @@ The output includes additional information about each metric, such as if its val For more complete documentation on using MinIO with Prometheus, see :ref:`How to monitor MinIO server with Prometheus ` -Starting with MinIO Server :minio-release:`RELEASE.2024-07-15T19-02-30Z` and MinIO Client :mc-release:`RELEASE.2024-07-11T18-01-28Z`, :ref:`metrics version 3 (v3) ` provides additional metrics. +Starting with MinIO Server :minio-release:`RELEASE.2024-07-15T19-02-30Z` and MinIO Client :mc-release:`RELEASE.2024-07-11T18-01-28Z`, :ref:`metrics version 3 (v3) ` provides additional endpoints and metrics. To print v3 metrics use the ``--api_version v3`` option. MinIO recommends new deployments use :ref:`version 3 (v3) `. @@ -58,7 +58,7 @@ Existing deployments can continue to use :ref:`metrics version 2 --api_version v3] + [--bucket ] .. include:: /includes/common-minio-mc.rst @@ -85,8 +85,8 @@ Parameters .. mc-cmd:: --bucket :optional: + Requires :mc-cmd:`~mc admin prometheus metrics --api-version`. For v3 metric types that return bucket-level metrics, specify a bucket name. - Use with :mc-cmd:`~mc admin prometheus metrics --api-version`. ``--bucket`` works for the following v3 metric types: @@ -128,7 +128,7 @@ Parameters - ``resource`` If not specified, a ``v2`` command returns cluster metrics. - Cluster metrics also include rollups of certain node metrics. + Cluster metrics include rollups of certain node metrics. Global flags @@ -162,10 +162,10 @@ The following prints all scanner metrics for a deployment: mc admin prometheus metrics ALIAS scanner --api-version v3 -Print v3 bucket replication metrics -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Print v3 API or bucket replication metrics +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Certain metric types accept a :mc-cmd:`~mc admin prometheus metrics --bucket` parameter to specify the bucket for which to print metrics. +Certain v3 metric types accept a :mc-cmd:`~mc admin prometheus metrics --bucket` parameter to specify the bucket for which to print metrics. The following example prints v3 replication metrics for bucket ``mybucket``: .. code-block:: shell @@ -175,10 +175,11 @@ The following example prints v3 replication metrics for bucket ``mybucket``: - Replace ``ALIAS`` with the :mc-cmd:`alias ` of the MinIO deployment. -To print API metrics for a bucket, replace ``replication`` with ``api``. +To print API metrics for the bucket, replace ``replication`` with ``api``. -Print v2 metrics -~~~~~~~~~~~~~~~~ + +Print v2 cluster metrics +~~~~~~~~~~~~~~~~~~~~~~~~ By default, :mc-cmd:`mc admin prometheus metrics` prints v2 cluster metrics: @@ -189,10 +190,16 @@ By default, :mc-cmd:`mc admin prometheus metrics` prints v2 cluster metrics: - Replace ``ALIAS`` with the :mc-cmd:`alias ` of the MinIO deployment. -To print another type of metrics, specify the desired :mc-cmd:`~mc admin prometheus metrics TYPE`. + +Print other types of v2 metrics +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To print another type of v2 metrics, specify the desired :mc-cmd:`~mc admin prometheus metrics TYPE`. The following example prints v2 bucket metrics: .. code-block:: shell :class: copyable mc admin prometheus metrics ALIAS bucket + +Accepted values are ``bucket``, ``cluster``, ``node``, and ``resource``.