Multiple SERVER Releases (#1141)

MinIO SERVER RELEASE.2024-01-05T22-17-24Z - added new metrics to github.com/minio/minio for later sync MinIO SERVER RELEASE.2024-01-28T22-35-53Z - MinIO preallocates memory, mc update compresses binary in transit MinIO SERVER RELEASE.2024-02-06T21-36-22Z - MinIO adds condition key for restricting STS AssumeRoleWithWebIdentity duration at policy level Closes #1124 , Partially addresses #1116 Partially Addresses #1105 --------- Co-authored-by: Daryl White <[email protected]>
minio · Mar 11, 2024 · 67e81ce · 67e81ce
1 parent 54584b2
commit 67e81ce
Show file tree

Hide file tree

Showing 11 changed files with 179 additions and 44 deletions.
diff --git a/source/administration/identity-access-management/policy-based-access-control.rst b/source/administration/identity-access-management/policy-based-access-control.rst
@@ -675,6 +675,40 @@ MinIO supports the following condition keys for all supported
 
 For additional keys supported by a specific S3 action, see the reference documentation for that action.
 
+MinIO Extended Condition Keys
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+MinIO extends the S3 standard condition keys with the following extended key:
+
+``sts:DurationSeconds``
+
+   .. versionadded:: MinIO SERVER RELEASE.2024-02-06T21-36-22Z
+
+   Specify a time in seconds to limit the duration of *all* Security Token Service credentials generated by :ref:`minio-sts-assumerolewithwebidentity`.
+
+   This value overrides the ``DurationSeconds`` field specified to the client.
+
+   For example:
+
+   .. code-block:: json
+
+      {
+         "Version": "2012-10-17",
+         "Statement": [
+            {
+                  "Effect": "Allow",
+                  "Action": [
+                     "sts:AssumeRoleWithWebIdentity"
+                  ],
+                  "Condition": {
+                     "NumericLessThanEquals": {
+                        "sts:DurationSeconds": "300"
+                     }
+                  }
+            }
+         ]
+      }
+
 .. _minio-policy-mc-admin-actions:
 
 ``mc admin`` Policy Action Keys

diff --git a/source/operations/checklists/hardware.rst b/source/operations/checklists/hardware.rst
@@ -189,6 +189,10 @@ The following table provides general guidelines for allocating memory for use by
    * - More than 1 Pebibyte (Pi)
      - 128GiB
 
+.. important::
+
+   Starting with :minio-release:`RELEASE.2024-01-28T22-35-53Z`, MinIO preallocates 2GiB of memory per node in distributed setups and 1GiB of memory for a single-node setup.
+
 .. _minio-hardware-checklist-storage:
 
 Storage

diff --git a/source/operations/data-recovery/recover-after-site-failure.rst b/source/operations/data-recovery/recover-after-site-failure.rst
@@ -92,7 +92,7 @@ This procedure assumes a *total loss* of one or more peer sites versus replicati
    All healthy peers in the site replication configuration update for the new peer automatically.
    You can use the :mc-cmd:`mc admin replicate info` command to verify the new site replication configuration.
 
-#. Resynchronize the new peer with :mc:`mc admin replicate resync`.
+#. Resynchronize the new peer with :mc-cmd:`mc admin replicate resync`.
 
    .. code-block:: shell
       :class: copyable

diff --git a/source/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.rst b/source/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.rst
@@ -104,6 +104,16 @@ Storage Requirements
    :start-after: start-storage-requirements-desc
    :end-before: end-storage-requirements-desc
 
+Memory Requirements
+~~~~~~~~~~~~~~~~~~~
+
+.. versionchanged:: RELEASE.2024-01-28T22-35-53Z
+
+   MinIO pre-allocates 2GiB of system memory at startup.
+
+MinIO recommends a *minimum* of 32GiB of memory per host.
+See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.
+
 Time Synchronization
 ~~~~~~~~~~~~~~~~~~~~
 

diff --git a/source/operations/install-deploy-manage/deploy-minio-single-node-multi-drive.rst b/source/operations/install-deploy-manage/deploy-minio-single-node-multi-drive.rst
@@ -33,6 +33,16 @@ Storage Requirements
    :start-after: start-storage-requirements-desc
    :end-before: end-storage-requirements-desc
 
+Memory Requirements
+~~~~~~~~~~~~~~~~~~~
+
+.. versionchanged:: RELEASE.2024-01-28T22-35-53Z
+
+   MinIO pre-allocates 2GiB of system memory at startup.
+
+MinIO recommends a *minimum* of 32GiB of memory per host.
+See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.
+
 .. _deploy-minio-standalone-multidrive:
 
 Deploy Single-Node Multi-Drive MinIO

diff --git a/source/operations/install-deploy-manage/deploy-minio-single-node-single-drive.rst b/source/operations/install-deploy-manage/deploy-minio-single-node-single-drive.rst
@@ -11,14 +11,8 @@ Deploy MinIO: Single-Node Single-Drive
    :depth: 2
 
 The procedures on this page cover deploying MinIO in a Single-Node Single-Drive (SNSD) configuration for early development and evaluation.
-|SNSD| deployments provide no added reliability or availability beyond what the underlying storage volume implements (RAID, LVM, ZFS, etc.).
-
-Starting with :minio-release:`RELEASE.2022-06-02T02-11-04Z`, MinIO implements a zero-parity erasure coded backend for single-node single-drive deployments.
-This feature allows access to :ref:`erasure coding dependent features <minio-erasure-coding>` without the requirement of multiple drives.
-This mode **requires** accessing stored objects through the S3 API, and does **not** support direct access to objects through the filesystem/POSIX interface.
-
-MinIO only starts in |SNSD| mode if the storage volume or path is empty *or* only contain files generated by a previous |SNSD| deployment.
-See the documentation on :ref:`SNSD behavior with pre-existing data <minio-snsd-pre-existing-data>` for more information.
+|SNSD| deployments use a zero-parity erasure coded backend that provides no added reliability or availability beyond what the underlying storage volume implements.
+These deployments are best suited for local testing and evaluation, or for small-scale data workloads that do not have availability or performance requirements.
 
 .. cond:: container
 
@@ -67,6 +61,37 @@ The following table lists the possible storage volume states and MinIO behavior:
 
        .. versionchanged:: RELEASE.2022-10-29T06-21-33Z
 
+Prerequisites
+-------------
+
+Storage Requirements
+~~~~~~~~~~~~~~~~~~~~
+
+The following requirements summarize the :ref:`minio-hardware-checklist-storage` section of MinIO's hardware recommendations:
+
+Use Local Storage
+   Direct-Attached Storage (DAS) has significant performance and consistency advantages over networked storage (:abbr:`NAS (Network Attached Storage)`, :abbr:`SAN (Storage Area Network)`, :abbr:`NFS (Network File Storage)`).
+   MinIO strongly recommends flash storage (NVMe, SSD) for primary or "hot" data.
+
+Use XFS-Formatting for Drives
+   MinIO strongly recommends provisioning XFS formatted drives for storage.
+   MinIO uses XFS as part of internal testing and validation suites, providing additional confidence in performance and behavior at all scales.
+
+Persist Drive Mounting and Mapping Across Reboots
+   Use ``/etc/fstab`` to ensure consistent drive-to-mount mapping across node reboots.
+
+   Non-Linux Operating Systems should use the equivalent drive mount management tool.
+
+Memory Requirements
+~~~~~~~~~~~~~~~~~~~
+
+.. versionchanged:: RELEASE.2024-01-28T22-35-53Z
+
+   MinIO pre-allocates 2GiB of system memory at startup.
+
+MinIO recommends a *minimum* of 32GiB of memory per host.
+See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.
+
 .. _deploy-minio-standalone:
 
 Deploy Single-Node Single-Drive MinIO
@@ -83,9 +108,6 @@ The following procedure deploys MinIO consisting of a single MinIO server and a
    MinIO cannot provide consistency guarantees if the underlying storage
    volumes are NFS or a similar network-attached storage volume. 
 
-   For deployments that *require* using network-attached storage, use
-   NFSv4 for best results.
-
 .. cond:: linux
 
    .. include:: /includes/linux/steps-deploy-minio-single-node-single-drive.rst

diff --git a/source/operations/install-deploy-manage/deploy-minio-tenant.rst b/source/operations/install-deploy-manage/deploy-minio-tenant.rst
@@ -362,6 +362,7 @@ Settings marked with an asterisk :guilabel:`*` are *required*:
    * - :guilabel:`Memory per Node [Gi]`
      - Specify the total amount of memory (RAM) to allocate per MinIO server pod. 
        See :ref:`minio-hardware-checklist-memory` for guidance on setting this value.
+       MinIO **requires** a minimum of 2GiB of memory per worker.
 
        The Kubernetes cluster *must* have worker nodes with sufficient free RAM to match the pod request.
 

diff --git a/source/operations/monitoring/metrics-and-alerts.rst b/source/operations/monitoring/metrics-and-alerts.rst
@@ -46,6 +46,20 @@ MinIO provides scraping endpoints for the following metric groups:
       Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
       For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
 
+   .. tab-item:: Resource Metrics
+
+      .. versionadded:: RELEASE.2023-10-07T15-07-38Z 
+
+      You can scrape :ref:`resource metrics <minio-available-resource-metrics>` using the following URL endpoint:
+
+      .. code-block:: shell
+         :class: copyable
+
+         http://HOSTNAME:PORT/minio/v2/metrics/resource
+
+      Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
+      For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
+
 
 MinIO by default requires authentication for scraping the metrics endpoints.
 Use the :mc-cmd:`mc admin prometheus generate` command to generate the necessary bearer tokens. 
@@ -110,6 +124,7 @@ Each metric includes a label for the MinIO server which generated that metric.
 
 - :ref:`Cluster Metrics <minio-available-cluster-metrics>`
 - :ref:`Bucket Metrics <minio-available-bucket-metrics>`
+- :ref:`Resource Metrics <minio-available-resource-metrics>`
 
 .. _minio-available-cluster-metrics:
 
@@ -121,6 +136,11 @@ Each metric includes a label for the MinIO server which generated that metric.
 .. include:: /includes/common-metrics-bucket.md
    :parser: myst_parser.sphinx_
 
+.. _minio-available-resource-metrics:
+
+.. include:: /includes/common-metrics-resource.md
+   :parser: myst_parser.sphinx_
+
 .. toctree::
    :titlesonly:
    :hidden:

diff --git a/source/operations/monitoring/minio-logging.rst b/source/operations/monitoring/minio-logging.rst
@@ -320,51 +320,66 @@ MinIO audit logs resemble the following JSON document:
 
    {
       "version": "1",
-      "deploymentid": "bc0e4d1e-bacc-42eb-91ad-2d7f3eacfa8d",
-      "time": "2019-08-12T21:34:37.187817748Z",
+      "deploymentid": "8ca2b7ad-20cf-4d07-9efb-28b2f519f4a5",
+      "time": "2024-02-29T19:39:25.744431903Z",
+      "event": "",
+      "trigger": "incoming",
       "api": {
-         "name": "PutObject",
-         "bucket": "testbucket",
-         "object": "hosts",
+         "name": "CompleteMultipartUpload",
+         "bucket": "data",
+         "object": "test-data.csv",
          "status": "OK",
          "statusCode": 200,
-         "timeToFirstByte": "366333ns",
-         "timeToResponse": "16438202ns"
+         "rx": 267,
+         "tx": 358,
+         "txHeaders": 387,
+         "timeToFirstByte": "2096989ns",
+         "timeToFirstByteInNS": "2096989",
+         "timeToResponse": "2111986ns",
+         "timeToResponseInNS": "2111986"
       },
       "remotehost": "127.0.0.1",
-      "requestID": "15BA4A72C0C70AFC",
-      "userAgent": "MinIO (linux; amd64) minio-go/v6.0.32 mc/2019-08-12T18:27:13Z",
+      "requestID": "17B86CB0ED88EBE9",
+      "userAgent": "MinIO (linux; amd64) minio-go/v7.0.67 mc/RELEASE.2024-02-24T01-33-20Z",
+      "requestPath": "/data/test-data.csv",
+      "requestHost": "minio.example.net:9000",
+      "requestQuery": {
+         "uploadId": "OGNhMmI3YWQtMjBjZi00ZDA3LTllZmItMjhiMmY1MTlmNGE1LmU3MjNlNWI4LTNiYWYtNDYyNy1hNzI3LWMyNDE3NTVjMmMzNw"
+      },
       "requestHeader": {
-         "Authorization": "AWS4-HMAC-SHA256 Credential=minio/20190812/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length,Signature=d3f02a6aeddeb29b06e1773b6a8422112890981269f2463a26f307b60423177c",
-         "Content-Length": "686",
+         "Accept-Encoding": "zstd,gzip",
+         "Authorization": "AWS4-HMAC-SHA256 Credential=minioadmin/20240229/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=ccb3acdc1763509a88a7e4a3d7fe431ef0ee5ca3f66ccb430d5a09326e87e893",
+         "Content-Length": "267",
          "Content-Type": "application/octet-stream",
-         "User-Agent": "MinIO (linux; amd64) minio-go/v6.0.32 mc/2019-08-12T18:27:13Z",
-         "X-Amz-Content-Sha256": "STREAMING-AWS4-HMAC-SHA256-PAYLOAD",
-         "X-Amz-Date": "20190812T213437Z",
-         "X-Amz-Decoded-Content-Length": "512"
+         "User-Agent": "MinIO (linux; amd64) minio-go/v7.0.67 mc/RELEASE.2024-02-24T01-33-20Z",
+         "X-Amz-Content-Sha256": "d61969719ee94f43c4e87044229b7a13b54cab320131e9a77259ad0c9344f6d3",
+         "X-Amz-Date": "20240229T193925Z"
       },
       "responseHeader": {
          "Accept-Ranges": "bytes",
-         "Content-Length": "0",
-         "Content-Security-Policy": "block-all-mixed-content",
-         "ETag": "a414c889dc276457bd7175f974332cb0-1",
-         "Server": "MinIO/DEVELOPMENT.2019-08-12T21-28-07Z",
-         "Vary": "Origin",
-         "X-Amz-Request-Id": "15BA4A72C0C70AFC",
+         "Content-Length": "358",
+         "Content-Type": "application/xml",
+         "ETag": "1d9fdc88af5e74f5eac0a3dd750ce58e-2",
+         "Server": "MinIO",
+         "Strict-Transport-Security": "max-age=31536000; includeSubDomains",
+         "Vary": "Origin,Accept-Encoding",
+         "X-Amz-Id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
+         "X-Amz-Request-Id": "17B86CB0ED88EBE9",
+         "X-Content-Type-Options": "nosniff",
          "X-Xss-Protection": "1; mode=block"
       },
       "tags": {
-         "objectErasureMap": {
-            "object": {
+         "objectLocation": {
+               "name": "Mousepad Template-v03final.jpg",
                "poolId": 1,
-               "setId": 10,
+               "setId": 1,
                "disks": [
-                  "http://server01/mnt/pool1/disk01",
-                  "http://server02/mnt/pool1/disk02",
-                  "http://server03/mnt/pool1/disk03",
-                  "http://server04/mnt/pool1/disk04"
+                  "/mnt/drive-1",
+                  "/mnt/drive-2",
+                  "/mnt/drive-3",
+                  "/mnt/drive-4"
                ]
-            }
          }
-      }
-   }
+      },
+      "accessKey": "minioadmin"
+   }
diff --git a/source/reference/minio-mc-admin/mc-admin-update.rst b/source/reference/minio-mc-admin/mc-admin-update.rst
@@ -88,3 +88,15 @@ Syntax
    :optional:
 
    Pass this flag to confirm the update and bypass the confirmation prompt.
+
+Behavior
+--------
+
+Binary Compression 
+~~~~~~~~~~~~~~~~~~
+
+.. versionchanged:: RELEASE.2024-01-28T22-35-53Z
+
+   :mc-cmd:`mc admin update` compresses the binary before sending to all nodes in the deployment.
+
+This feature does not apply to :ref:`systemctl managed deployments <minio-upgrade-systemctl>`.
diff --git a/sync-minio-server-docs.sh b/sync-minio-server-docs.sh
@@ -6,7 +6,14 @@ set -x
 function main() {
    curl --retry 10 -Ls https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/list.md | csplit - /"# Bucket Metrics"/
    mv xx00 source/includes/common-metrics-cluster.md
-   mv xx01 source/includes/common-metrics-bucket.md
+
+   # Kludgy. Does csplit again on the Bucket Metrics file fragment
+   # Tried to get smart using `grep '^# [A-Za-z]` to get line numbers but got stuck
+
+   cat xx01 | csplit - /"# Resource Metrics"/
+
+   mv xx00 source/includes/common-metrics-bucket.md
+   mv xx01 source/includes/common-metrics-resource.md
 }
 
 main "$@"