diff --git a/docs/cosi_driver/deployment.md b/docs/cosi_driver/deployment.md new file mode 100644 index 0000000..f06af74 --- /dev/null +++ b/docs/cosi_driver/deployment.md @@ -0,0 +1,101 @@ +# Overview + +The HPE COSI Driver for Kubernetes is deployed by using industry standard means, a Helm chart. + +[TOC] + +## Delivery Vehicles + +Helm is currently the only supported delivery vehicle for the deployment of the HPE COSI Driver. + +### Helm + +[Helm](https://helm.sh) is the package manager for Kubernetes. Software is being delivered in a format designated as a "chart". Helm is a [standalone CLI](https://helm.sh/docs/intro/install/) that interacts with the Kubernetes API server using your `KUBECONFIG` file. + +The official [Helm chart](https://github.com/hpe-storage/co-deployments/tree/master/helm/charts/hpe-cosi-driver) for the HPE COSI Driver for Kubernetes is hosted on [Artifact Hub](https://artifacthub.io/packages/helm/hpe-storage/hpe-cosi-driver). The chart supports Helm 3 from version 1.0.0 of the HPE COSI Driver. In an effort to avoid duplicate documentation, please see the chart for instructions on how to deploy the COSI driver using Helm. + +- Go to the chart on [Artifact Hub](https://artifacthub.io/packages/helm/hpe-storage/hpe-cosi-driver). + +## Add an HPE Storage Backend + +Once the COSI driver is deployed, you must create a `Secret` with the following details before you can use the [COSI API resources](using.md). + +### Secret Parameters + +All parameters are mandatory and described below. + +| Parameter | Description | +| ------------------- | ------------| +| accessKey | The access key of the S3 user with bucket creation, bucket-tagging and deletion permissions. +| secretKey | The secret key for the S3 user who has bucket creation, bucket-tagging and deletion permissions. +| endpoint | The S3 frontend network DNS subdomains address of the backend object storage system; that is, an HPE Alletra Storage MP X10000 system. +| glcpUserClientId | The HPE Green Lake API client ID. +| glcpUserSecretKey | The HPE Green Lake API client secret. +| dsccZone | The fully qualified domain name (FQDN) of the HPE Data Services Cloud Console zone. +| clusterSerialNumber | The backend storage system cluster serial number. + +!!! note + The Kubernetes compute nodes where the HPE COSI Driver is allowed to run need to be able to access the Data Services Cloud Console zone specified. + +Example `Secret` manifest named "hpe-object-backend.yaml": + +```yaml fct_label="HPE COSI Driver v1.0.0" +apiVersion: v1 +kind: Secret +metadata: + name: hpe-object-backend + namespace: default +stringData: + accessKey: testuser + secretKey: testkey + endpoint: http://192.168.1.100:8080 + glcpUserClientId: 00000000-0000-0000-0000-000000000000 + glcpUserSecretKey: 00000000000000000000000000000000 + dsccZone: us1.data.cloud.hpe.com + clusterSerialNumber: 0000000000 +``` + +Create the `Secret`. + +```text +kubectl create -f hpe-object-backend.yaml +``` + +!!! tip "See Also" + The COSI source code repository contains a parameterized script that can assist in creating a correctly formatted `Secret`. See [github.com/hpe-storage/cosi-driver/scripts/cosi_secret](https://github.com/hpe-storage/cosi-driver/tree/main/scripts/cosi_secret) for more details. + +### Creating and Locating Resources + +1. To create the S3 user: + * Follow the steps in the HPE documentation to [create an access policy](https://support.hpe.com/hpesc/docDisplay?docId=sd00004219en_us&page=objstr_access_policies_create_dscc.html). + - Choose _All Buckets_. + - Add custom actions for the policy: `CreateBucket`, `DeleteBucket`, `PutBucketTagging`. + * To create the user, refer to the [HPE documentation](https://support.hpe.com/hpesc/docDisplay?docId=sd00004219en_us&page=objstr_users_create_dscc.html) for this purpose and select the access policy created in the previous step. + - Save the user name and password. These will be used as the S3 access key and S3 secret key respectively in the COSI secret. +2. To create the HPE Green Lake API client ID and secret, refer to the following [HPE documentation](https://support.hpe.com/hpesc/public/docDisplay?docId=a00120892en_us&page=GUID-23E6EE78-AAB7-472C-8D16-7169938BE628.html). +3. To locate the Data Services Cloud Console zone FQDN: + * Log into HPE Data Services Cloud Console. + * On the _Services_ page, click _My Services_ to view all services available in your workspace. + * Select the service that your HPE Alletra Storage MP X10000 device is assigned to and click _Launch_. + * After the service is launched, save the value of the URL from the browser. E.g.: `https://console-us1.data.cloud.hpe.com`. + * After dropping the prefix `https://console-`, the Data Services Cloud Console zone FQDN value to be used in the `Secret` should have the following format: `us1.data.cloud.hpe.com`. + * Supported Data Services Cloud Console zone FQDNs as of January 2025 are: + - us1.data.cloud.hpe.com + - jp1.data.cloud.hpe.com + - eu1.data.cloud.hpe.com + - uk1.data.cloud.hpe.com +4. To locate the S3 endpoint: + * Log into HPE Data Services Cloud Console. + * Launch the service that your HPE Alletra Storage MP X10000 device is assigned to. + * Select _Data Ops Manager_. + * From the menu on the left, select _Systems_. From the list click on the name of the system you want to use for COSI operations. + * Click on the _Networking_ tab. Under the _Frontend Network_ section, save the value of the _Network DNS Subdomains_ field. + * The S3 endpoint can be constructed from the _Network DNS Subdomains_ value by using the format: `http://`. +5. To locate the cluster serial number of the HPE Alletra Storage MP X10000 system, refer to the following [HPE documentation](https://support.hpe.com/hpesc/public/docDisplay?docId=a00120892en_us&page=GUID-616CE4D4-C31A-4BFE-8F41-887C2B0B9046.html). + +!!! tip + In a real world scenario it's more practical to name the `Secret` something that makes sense for the organization. It could be the hostname of the backend or the role it carries; i.e., "hpe-alletra-sanjose-prod". + +## Next Steps + +Next you need to create [a BucketClass](using.md#configure_a_bucketclass). diff --git a/docs/cosi_driver/diagnostics.md b/docs/cosi_driver/diagnostics.md new file mode 100644 index 0000000..bc1901a --- /dev/null +++ b/docs/cosi_driver/diagnostics.md @@ -0,0 +1,174 @@ +# Introduction + +It's recommended to familiarize yourself with inspecting workloads on Kubernetes. This [cheat sheet](https://kubernetes.io/docs/reference/kubectl/cheatsheet/#interacting-with-running-pods) is very useful, and you should have it readily available. + +## Sanity Checks + +Once the COSI driver has been deployed using Helm, the list of pods shown below is representative of what a healthy system would look like after installation. If any of the workload deployments lists anything but `Running`, proceed to inspect the logs for the problematic workload. + +```text +kubectl get pods +NAME READY STATUS RESTARTS AGE +hpe-cosi-provisioner-559b458788-5jc56 2/2 Running 0 8d +objectstorage-controller-7dff56f8fc-r5tdq 1/1 Running 0 8d +``` + +Once the COSI API objects `BucketClaim`, `Bucket` and `BucketAccess` have been created in the cluster, use the `kubectl describe` command to check the status and events, which will have information useful to help diagnose any issue. If any of the resources' statuses shows anything but `true` or if there are any warning events, inspect the logs of the COSI driver and sidecar. + +Describing a `BucketClaim`. + +```text fct_label="BucketClaim" +kubectl describe bucketclaim my-first-bucketclaim +Name: my-first-bucketclaim +Namespace: default +Labels: +Annotations: +API Version: objectstorage.k8s.io/v1alpha1 +Kind: BucketClaim +Metadata: + Creation Timestamp: 2024-09-17T04:34:55Z + Finalizers: + cosi.objectstorage.k8s.io/bucketclaim-protection + Generation: 1 + Resource Version: 110670 + UID: 99dde004-4f8d-4a20-900b-e5d61e3facb9 +Spec: + Bucket Class Name: hpe-standard-object + Protocols: + s3 +Status: + Bucket Name: bc199dde004-4f8d-4a20-900b-e5d61e3facb9 + Bucket Ready: true +Events: +``` + +Describing a `Bucket`. + +```text fct_label="Bucket" +kubectl describe bucket bc199dde004-4f8d-4a20-900b-e5d61e3facb9 +Name: bc199dde004-4f8d-4a20-900b-e5d61e3facb9 +Namespace: +Labels: +Annotations: +API Version: objectstorage.k8s.io/v1alpha1 +Kind: Bucket +Metadata: + Creation Timestamp: 2024-09-17T04:34:55Z + Finalizers: + cosi.objectstorage.k8s.io/bucket-protection + cosi.objectstorage.k8s.io/bucketaccess-bucket-protection + Generation: 1 + Resource Version: 111014 + UID: e7fb69f6-c20e-4abf-a522-f1ae765b3b6a +Spec: + Bucket Claim: + Name: my-first-bucketclaim + Namespace: default + UID: 99dde004-4f8d-4a20-900b-e5d61e3facb9 + Bucket Class Name: hpe-standard-object + Deletion Policy: Delete + Driver Name: cosi.hpe.com + Parameters: + Bucket Tags: mytag=myvalue, mytag2=, mytag3=myvalue3, + Cosi User Secret Name: hpe-object-backend + Cosi User Secret Namespace: default + Protocols: + s3 +Status: + Bucket ID: bc199dde004-4f8d-4a20-900b-e5d61e3facb9 + Bucket Ready: true +Events: +``` + +Describing a `BucketAccess`. + +```text fct_label="BucketAccess" +kubectl describe bucketaccess hpe-standard-access +Name: hpe-standard-access +Namespace: default +Labels: +Annotations: +API Version: objectstorage.k8s.io/v1alpha1 +Kind: BucketAccess +Metadata: + Creation Timestamp: 2024-09-17T04:38:00Z + Finalizers: + cosi.objectstorage.k8s.io/bucketaccess-protection + Generation: 1 + Resource Version: 111017 + UID: 37fd517f-27ac-45fa-b369-57e645967366 +Spec: + Bucket Access Class Name: hpe-standard-access + Bucket Claim Name: my-first-bucket-claim + Credentials Secret Name: my-first-access-secret + Protocol: s3 +Status: + Access Granted: true + Account ID: ba-37fd517f-27ac-45fa-b369-57e645967366 +Events: +``` + +!!! tip + It is useful to check that the network connection between the Kubernetes cluster hosting the HPE COSI driver and HPE Data Services Cloud Console, as well as between the HPE Alletra Storage MP X 10000 system and HPE Data Services Cloud Console is intact. A poor network connection is a common cause of failure while creating or deleting `Bucket` and `BucketAccess` resources. + +## Logging + +Log files associated with the HPE COSI Driver posts data to the standard output stream and can be accessed using options under [kubectl logs](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_logs/). + +If the logs need to be retained for long term, use a standard logging solution for Kubernetes, such as Fluentd. Alternatively, it's advised to use the log collector script, at regular intervals to preserve logs. + +### COSI Driver Logs + +```text +kubectl logs -f deploy/hpe-objectstorage-provisioner -c hpe-cosi-driver +``` + +### COSI Sidecar Logs + +```text +kubectl logs -f deploy/hpe-objectstorage-provisioner -c hpe-cosi-provisioner-sidecar +``` + +### COSI Controller Logs + +```text +kubectl logs -f deploy/objectstorage-controller +``` + +### Log Level of Sidecar + +You can control the log level for the COSI Sidecar using the `.containers.sideCar.verbosityLevel` field in [`values.yaml`](https://github.com/hpe-storage/co-deployments/blob/master/helm/values/cosi-driver/v1.0.0/values.yaml) of the Helm chart. The values are generally small positive integers. + +```yaml +containers: + sideCar: + # Verbosity level: Small postive integer values are generally recommended. + # Ref.: https://pkg.go.dev/k8s.io/klog/v2#V + # containers.sideCar.verbosityLevel -- Specifies the verbosity of the logs that will be printed by the sidecar container + verbosityLevel: 5 +``` + +### Log Collector + +The log collector script `hpe-logcollector.sh` can be used to collect the logs from any node that has `kubectl` access to the cluster. Please see the script's associated [documentation](https://github.com/hpe-storage/cosi-driver/tree/main/scripts/log_collection) for more details on usage and troubleshooting. + +Download the script and provide execute permissions: + +```text +wget https://raw.githubusercontent.com/hpe-storage/cosi-driver/main/scripts/log_collection/hpe-logcollector.sh +chmod +x hpe-logcollector.sh +``` + +Usage: + +```text +./hpe-logcollector.sh -h +Collect HPE storage diagnostic logs using kubectl. + +Usage: + hpe-logcollector.sh [-h|--help] [-n|--namespace NAMESPACE] +Options: +-h|--help Print this usage text +-n|--namespace NAMESPACE Collect logs from HPE COSI Driver Deployment in Namespace + NAMESPACE (default: default) +``` diff --git a/docs/cosi_driver/examples/using/my-cosi-app.yaml b/docs/cosi_driver/examples/using/my-cosi-app.yaml new file mode 100644 index 0000000..0a7c699 --- /dev/null +++ b/docs/cosi_driver/examples/using/my-cosi-app.yaml @@ -0,0 +1,89 @@ +apiVersion: batch/v1 +kind: Job +metadata: + name: my-cosi-app +spec: + template: + spec: + restartPolicy: Never + initContainers: + - name: install-boto3 + image: python:3.9 + command: ["sh", "-c", "pip install boto3 -t /app && cp /scripts/sample_pod.py /app"] + env: + - name: http_proxy + value: "" + - name: https_proxy + value: "" + volumeMounts: + - name: app-volume + mountPath: /app + - name: script-volume + mountPath: /scripts + containers: + - name: my-cosi-app + image: python:3.9 + command: ["python", "-u", "/app/sample_pod.py"] + volumeMounts: + - name: app-volume + mountPath: /app + - name: secret-volume + mountPath: /data/cosi + env: + - name: PYTHONPATH + value: "/app" + volumes: + - name: app-volume + emptyDir: {} + - name: script-volume + configMap: + name: test-object-script + - name: secret-volume + secret: + secretName: my-first-access-secret +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: test-object-script +data: + sample_pod.py: | + #!/usr/bin/env python3 + + import json + import boto3 + import sys + + try: + # load the bucket info + print("Loading bucket info...") + with open('/data/cosi/BucketInfo', 'r') as f: + bi = json.load(f) + + # create an s3 client session + print("Creating S3 client session...") + session = boto3.Session( + aws_access_key_id=bi['spec']['secretS3']['accessKeyID'], + aws_secret_access_key=bi['spec']['secretS3']['accessSecretKey'], + ) + + s3 = session.resource( + bi['spec']['protocols'][0], # set protocol + endpoint_url=bi['spec']['secretS3']['endpoint'], # set endpoint + ) + + # create an object + print("Creating object in S3 bucket...") + s3.Object(bi['spec']['bucketName'], 'sample.txt').put(Body='Hello, World!') + print('Object created') + + # read the object + print('Testing the get-object operation on bucket: ' + bi['spec']['bucketName'] + ', object body: ' + s3.Object(bi['spec']['bucketName'], 'sample.txt').get()['Body'].read().decode('utf-8')) + + # Exit gracefully with zero exit code + print("Script completed successfully.") + sys.exit(0) + + except Exception as e: + print(f"An error occurred: {e}", file=sys.stderr) + sys.exit(1) \ No newline at end of file diff --git a/docs/cosi_driver/img/cosi_driver_architecture-1.0.0.png b/docs/cosi_driver/img/cosi_driver_architecture-1.0.0.png new file mode 100644 index 0000000..381242f Binary files /dev/null and b/docs/cosi_driver/img/cosi_driver_architecture-1.0.0.png differ diff --git a/docs/cosi_driver/index.md b/docs/cosi_driver/index.md new file mode 100644 index 0000000..40e2f15 --- /dev/null +++ b/docs/cosi_driver/index.md @@ -0,0 +1,84 @@ +# Introduction + +A Container Object Storage Interface ([COSI](https://github.com/kubernetes-sigs/container-object-storage-interface-spec)) Driver for HPE Alletra Storage MP X10000. The HPE COSI Driver for Kubernetes allows you to use the HPE Object Storage Provider (OSP) to perform bucket management operations on storage resources. The COSI architecture allows you to integrate HPE Alletra Storage MP X10000 Object Storage with a COSI-compatible containerized application running on the Kubernetes cluster. The COSI driver follows the gRPC [specification](https://github.com/kubernetes-sigs/container-object-storage-interface/blob/main/proto/cosi.proto) provided by Kubernetes. + +![HPE COSI Driver for Kubernetes architecture](img/cosi_driver_architecture-1.0.0.png) + +!!! tip + The HPE COSI Driver for Kubernetes is vendor-specific and works only with the HPE Alletra Storage MP X10000 OSP. + +## Table of Contents + +[TOC] + +## Features and Capabilities + +Below is the official table for COSI features that HPE has officially tested and validated against the [platform matrix](#compatibility_and_support). + +| Feature | K8s maturity | Since K8s version | HPE COSI Driver | +|----------------------------------------|--------------|-------------------|-----------------| +| Bucket Creation | Alpha | 1.25 | 1.0.0 | +| Bucket Deletion | Alpha | 1.25 | 1.0.0 | +| Bucket Tagging | Alpha | 1.25 | 1.0.0 | +| Granting Bucket Access | Alpha | 1.25 | 1.0.0 | +| Revoking Bucket Access | Alpha | 1.25 | 1.0.0 | + +Refer to the [official table](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) of feature gates in the Kubernetes docs to determine the availability of alpha features. File any issues, questions or feature requests [here](https://github.com/hpe-storage/cosi-driver/issues). You may also join the HPE Slack community to chat with people close to this project on the `#Alletra` and `#Kubernetes` channels. Sign up at [slack.hpedev.io](https://slack.hpedev.io/) and log in at [hpedev.slack.com](https://hpedev.slack.com). + +!!! tip + Familiarize yourself with the basic requirements below for running the COSI driver on your Kubernetes cluster before you install it. HPE strongly recommends you install the COSI driver with a [Helm chart](deployment.md#helm). + +## Compatibility and Support + +HPE has tested the following combinations and included them as part of the official support services for the first COSI driver release. + + +### HPE COSI Driver for Kubernetes v1.0.0 + +Release highlights: + +* Support for Kubernetes v1.25 to v1.31. +* Implementation of bucket creation, configuration (bucket tagging), lifecycle and access management. +* A log collector script that can be used to collect logs from any node. + + + + + + + + + + + + + + + + + + + + + + + + + + +
Kubernetesv1.25-v1.31
Helm Chartv1.0.0 on ArtifactHub
Platforms + HPE Alletra Storage MP X10000 +
HPE Alletra Storage MP X10000 OS + R1 +
ProtocolsS3
Release notesv1.0.0 on GitHub
+ +### Release Archive + +HPE does not currently have any archived releases of the HPE COSI Driver. + +## Known Limitations + +* The HPE COSI driver can be deployed only in the `default` namespace due to a [bug](https://github.com/kubernetes-sigs/container-object-storage-interface-provisioner-sidecar/issues/140) in creating events in the COSI API objects when deployed in non-default namespaces. +* Creating `BucketClaim` or `BucketAccess` objects in parallel can cause failures in the COSI driver. A [bug](https://github.com/kubernetes-sigs/container-object-storage-interface-api/issues/101) has been filed to address this issue. +* A warning event is created in the `Bucket` or `BucketAccess` resources when an error occurs, and has a life-span of one hour. During this period, if the error is resolved the Status will show `Bucket Ready: true` or `Access Granted: true` in the `Bucket` or `BucketAccess` respectively, but the warning event will persist till an hour lapses. A [bug](https://github.com/kubernetes-sigs/container-object-storage-interface-api/issues/103) has been raised to resolve this ambiguity. +* Recreation of `BucketClaim` or `BucketAccess` objects doesn't work intermittently, as gRPC request is not sent to the COSI driver. This [pull request](https://github.com/kubernetes-retired/container-object-storage-interface-api/pull/86) will address the issue. diff --git a/docs/cosi_driver/using.md b/docs/cosi_driver/using.md new file mode 100644 index 0000000..3529ac5 --- /dev/null +++ b/docs/cosi_driver/using.md @@ -0,0 +1,292 @@ +# Overview + +At this point the HPE COSI Driver and HPE Alletra Storage MP X10000 should be installed and configured. + +!!! important + All examples below assumes there's a `Secret` with the necessary credentials to connect to the backend object storage system and HPE Data Services Cloud Console. Learn how to create the `Secret` in the [Deployment section](deployment.md#add_an_hpe_storage_backend). + +[TOC] + +## Configure a BucketClass + +The `BucketClass` defines a set of properties that can be used to configure buckets. The `BucketClass` has a variety of fields that you can configure. An example of `BucketClass` is below. `BucketClasses` are clustered resources normally created by Kubernetes administrators. + +```yaml +kind: BucketClass +apiVersion: objectstorage.k8s.io/v1alpha1 +metadata: + name: hpe-standard-object +driverName: cosi.hpe.com +deletionPolicy: Delete +parameters: + cosiUserSecretName: hpe-object-backend + cosiUserSecretNamespace: default +``` + +!!! info "Clarification of `deletionPolicy`" + The `deletionPolicy` is used to delete a bucket. It can be set to `Delete` or `Retain`, depending on whether the actual bucket in HPE Alletra Storage MP X10000 should be removed or retained. The associated COSI API resources `BucketClaim` and `Bucket` will be deleted regardless of the `deletionPolicy`. The `Bucket` resource can only be removed once all `BucketAccess` resources have been removed. + +The `kubectl create` command can be used to create this resource. + +```text +kubectl create -f hpe-standard-object.yaml +``` + +!!! important "Important" + The [`Secret`](deployment.md#add_an_hpe_storage_backend) should be created before referring to it in the BucketClass. + +### Optional BucketClass Parameters + +Optional `BucketClass` `.parameters`. + +| Parameter | String | Description | +| ----------------------------- | -------------- | ----------- | +| bucketTags | Text | The tags to be applied on a bucket. The key-value pairs must be comma separated. In a tag, the key is required while the value is optional. Example: `.parameters.bucketTags: mytag1=myval1, mytag2, mytag3=myval3`. | + +## Create a BucketClaim + +The `BucketClaim` resource will be used to create a `Bucket` in COSI according to the properties specified in the `BucketClass` it refers to. Examples of Greenfield and Brownfield provisioning have been provided below. `BucketClaims` are namespaced resources normally created by applications and Kubernetes users. + +### Greenfield Bucket Provisioning + +You can use the following manifest to create a new bucket on the object storage system. + +```yaml +kind: BucketClaim +apiVersion: objectstorage.k8s.io/v1alpha1 +metadata: + name: my-first-bucketclaim +spec: + bucketClassName: hpe-standard-object + protocols: + - s3 +``` + +The `kubectl create` command can be used to create this resource. + +```text +kubectl create -f my-first-bucketclaim.yaml +``` + +The `kubectl create` command will create the `BucketClaim` and `Bucket` on the cluster. Check the `Status` and `Events` field of the resources' description to verify if the bucket was created successfully. + +```text +kubectl get bucketclaim,bucket +NAME AGE +bucketclaim/my-first-bucketclaim 10d + +NAME AGE +bucket/bc199dde004-4f8d-4a20-900b-e5d61e3facb9 10d +``` + +!!! important "Important" + The [`Secret`](deployment.md#add_an_hpe_storage_backend) and [`BucketClass`](using.md#configure_a_bucketclass) should be created before provisioning a new bucket. + +The `Bucket` created by COSI can be deleted by deleting the associated `BucketClaim` using the `kubectl delete` command. + +```text +kubectl delete bucketclaim my-first-bucketclaim +``` + +### Brownfield Bucket Provisioning + +To map an existing bucket to a `BucketClaim` and `Bucket`, refer the following manifests. In the examples provided here the name of the existing bucket is "brownfield-bucket-name". + +```yaml +apiVersion: objectstorage.k8s.io/v1alpha1 +kind: Bucket +metadata: + name: my-existing-bucket +spec: + bucketClaim: {} + bucketClassName: hpe-standard-object + deletionPolicy: Delete + driverName: cosi.hpe.com + existingBucketID: my-existing-bucket + parameters: + cosiUserSecretName: hpe-object-backend + cosiUserSecretNamespace: default + protocols: + - s3 +``` + +!!! note + A `Bucket` is a clustered resource and is normally created by a Kubernetes administrator. + +Create a `BucketClaim` referring to the pre-existing `Bucket`. + +```yaml +apiVersion: objectstorage.k8s.io/v1alpha1 +kind: BucketClaim +metadata: + name: my-brownfield-bucketclaim +spec: + bucketClassName: hpe-standard-object + existingBucketName: my-existing-bucket + protocols: + - s3 +``` + +The `kubectl create` command can be used to create this resource. + +```text +kubectl create -f my-existing-bucket.yaml -f my-brownfield-bucketclaim.yaml +``` + +The above command will create the `BucketClaim` and `Bucket` on the cluster that maps to the existing bucket. Check the `Status` and `Events` field of the resources' description to verify if the mapping was successful. + +```text +kubectl get bucketclaim,bucket +NAME AGE +bucketclaim/my-brownfield-bucketclaim 72s + +NAME AGE +bucket/my-existing-bucket 5s +``` + +## BucketAccessClass Configuration + +The `BucketAccessClass` defines a set of properties that can be used to request bucket-specific access credentials. The `BucketAccessClass` has a variety of fields that can be configured according to the user's preferences. An example has been provided below. `BucketAccessClasses` are clustered resources and normally created by a Kubernetes administrator. + +```yaml +kind: BucketAccessClass +apiVersion: objectstorage.k8s.io/v1alpha1 +metadata: + name: hpe-standard-access +driverName: cosi.hpe.com +authenticationType: Key +parameters: + cosiUserSecretName: hpe-object-backend + cosiUserSecretNamespace: default +``` + +!!! important + The `.parameters.cosiUserSecretName` must reference the same `Secret` used to provision the `BucketClaim` and it must exist prior to creating a `BucketAccessClass` and `BucketClass`. See [add an HPE storage backend for more details](deployment.md#add_an_hpe_storage_backend). + +The `kubectl create` command can be used to create this resource. + +```text +kubectl create -f hpe-standard-access.yaml +``` + +## BucketAccess Configuration + +The `BucketAccess` resource can be used to create the s3 protocol user credentials to access a particular bucket according to the properties specified in the `BucketAccessClass`. The `BucketAccess` has a variety of fields that can be configured according to the user's preferences. An example has been provided below. `BucketAccess` resources are namespaced and normally created by Kubernetes users to grant `BucketClaim` access to a workload. + +```yaml +kind: BucketAccess +apiVersion: objectstorage.k8s.io/v1alpha1 +metadata: + name: my-first-bucketaccess +spec: + bucketAccessClassName: hpe-standard-access + credentialsSecretName: my-first-access-secret + bucketClaimName: my-first-bucketclaim + protocol: s3 +``` + +The `kubectl create` command can be used to create this resource. + +```text +kubectl create -f my-first-bucketaccess.yaml +``` + +The `kubectl create` will create the `BucketAccess` and a `Secret` in the `default` namespace of the cluster. Check the `Status` and `Events` field of the `BucketAccess` resource's description to verify if the access was granted successfully. + +```text +kubectl get bucketaccess,secret +NAME AGE +bucketaccess/my-first-bucketaccess 10d + +NAME TYPE DATA AGE +secret/my-first-access-secret Opaque 1 10d +``` + +To inspect the `Secret`, retrieve the content with `kubectl`, decode with `base64` and present with `jq`. + +```yaml +kubectl get secret my-first-access-secret -o jsonpath='{.data.BucketInfo}' | base64 --decode | jq +{ + "metadata": { + "name": "bc-37fd517f-27ac-45fa-b369-57e645967366", + "creationTimestamp": null + }, + "spec": { + "bucketName": "bc199dde004-4f8d-4a20-900b-e5d61e3facb9", + "authenticationType": "Key", + "secretS3": { + "endpoint": "http://192.168.1.100:8080", + "region": "us-east-1", + "accessKeyID": "user_ba-37fd517f-27ac-45fa-b369-57e645967366", + "accessSecretKey": "qLe3DM0Gyoy5I7AMRqRN1Dv2gNWvvq+9EX2Qul+5tWXxgtL5DJ14i+K5UEdh+Orp" + }, + "secretAzure": null, + "protocols": [ + "s3" + ] + } +} +``` + +!!! tip + The `Secret` generated here can be used by any COSI-enabled containerized application on the Kubernetes cluster. + +The credentials in this `Secret` can be revoked by deleting the associated `BucketAccess` using the `kubectl delete` command. + +```text +kubectl delete bucketaccess my-first-bucketaccess +``` + +### Attaching an Access Secret to an Object Storage Workload + +The `Secret` specified in `credentialsSecretName` of the `BucketAccess` can be mounted as a volume to any containerized application running in a `Pod` on the Kubernetes cluster. The s3 user credentials and other relevant information will be found in the `mountPath` to access the bucket. The workload running in the `Pod` can read this information to create an s3 client, read and write objects to the bucket using the library functions available in the [`aws` sdk package](https://aws.amazon.com/developer/tools/) for the programming language of your choice. For more details, refer to [official Kubernetes Enhancement Proposals documentation](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1979-object-storage-support#attaching-bucket-information-to-pods) on using the COSI driver with a `Pod`. + +As an example, let us create a `Job` that runs a Python script to put and get an object in the `Bucket` using the mounted `Secret`. + +The Python `aws` package `boto3` will be installed in the init container. The Python script is stored in a `ConfigMap` and is mounted to the init container, which will copy it to `PYTHONPATH=/app` where `boto3` is installed. `my-first-access-secret` is mounted as a volume to the main container `my-cosi-app` in the `mountPath: /data/cosi` from where the script will read the bucket-specific s3 user credentials to create the s3 client session. + +```yaml +{% include "cosi_driver/examples/using/my-cosi-app.yaml" %} +``` + +!!! tip + You may need to set `http_proxy` and `https_proxy` environment variables of the init container to `pip install` the `boto3` package. + +Create the `Job` and `ConfigMap`. + +```text +kubectl create -f {{ config.site_url }}cosi_driver/examples/using/my-cosi-app.yaml +``` + +Wait for the `Job` to be completed. + +```text fct_label="Job" +kubectl get job +NAME COMPLETIONS DURATION AGE +my-cosi-app 1/1 15s 2m5s +``` + +The status of the `Pod` can also be tracked till completion. + +```text fct_label="Pod" +kubectl get pods +NAME READY STATUS RESTARTS AGE +my-cosi-app-tlr7l 0/1 Completed 0 2m9s +``` + +Once the `Job` has completed, the results of the object operations tested in the script can be found in the logs. + +```text +kubectl logs jobs/my-cosi-app +Defaulted container "my-cosi-app" out of: my-cosi-app, install-boto3 (init) +Loading bucket info... +Creating S3 client session... +Creating object in S3 bucket... +Object created +Testing the get-object operation on bucket: bc169822df7-532e-43a2-978d-90631719f78e, object body: Hello, World! +Script completed successfully. +``` + +## Further Reading + +The [official Kubernetes documentation](https://kubernetes.io/blog/2022/09/02/cosi-kubernetes-object-storage-management/) contains comprehensive documentation on how the COSI API resources work together. diff --git a/docs/legal/support/index.md b/docs/legal/support/index.md index 19e768d..372ca97 100644 --- a/docs/legal/support/index.md +++ b/docs/legal/support/index.md @@ -2,6 +2,8 @@ Software components documented on SCOD are generally covered with valid support contract on the HPE product being used. Terms and conditions may be found in the support contract. Please reach out to your official HPE representative or HPE partner for any uncertainties. +[TOC] + ## CSI Info Metrics Provider support The HPE CSI Info Metrics Provider for Prometheus is supported by HPE when used with HPE storage arrays on valid support contracts. Send email to [support@nimblestorage.com](mailto:support@nimblestorage.com) to get started with any issue that requires assistance. Engage your HPE representative for other means to contact HPE Storage support directly. @@ -33,3 +35,15 @@ Limited to the HPE Alletra Storage MP B10000, Alletra 9000 and Primera and 3PAR | **Warranty and Foundation Care** | 800-633-3600 | | **Proactive Care (PC)** | 866-211-5211 | | **Datacenter Care (DC)** | 888-751-2149 | + +## Object Storage Providers + +### HPE Alletra Storage MP X10000 + +Best effort support is available for the Object Storage Provider (OSP) HPE Alletra Storage MP X10000 with All-inclusive Single or Multi-System software and an active HPE Pointnext support agreement. Since HPE Pointnext support for the OSP is best effort only, any other support levels like Warranty, Foundation Care, Proactive Care, Proactive Care Advanced and Datacenter Care or other support levels do not apply. Best effort response times are based on local standard business days and working hours. If your location is outside the customary service zone, response time may be longer. + +| HPE Alletra Storage MP X10000 Hardware Contract Type | Phone Number | +| ----------------------------------------------------------------------------- | ------------ | +| **Warranty and Foundation Care** | 800-633-3600 | +| **Proactive Care (PC)** | 866-211-5211 | +| **Datacenter Care (DC)** | 888-751-2149 | diff --git a/mkdocs.yml b/mkdocs.yml index 76918eb..4d8b966 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -2,14 +2,14 @@ site_name: 'SCOD.HPEDEV.IO' site_description: 'Reference documentation for container orchestrator integration for HPE storage' site_url: 'https://scod.hpedev.io/' site_author: 'Hewlett Packard Enterprise' -theme: +theme: name: readthedocs custom_dir: hpe_theme/ analytics: gtag: 'G-PC28RTKKTW' repo_name: 'hpe-storage/scod' repo_url: 'https://github.com/hpe-storage/scod' -copyright: 'Copyright 2020-2024 Hewlett Packard Enterprise Development LP' +copyright: 'Copyright 2020-2025 Hewlett Packard Enterprise Development LP' extra_css: - css/hpedev.css use_directory_urls: false @@ -40,7 +40,7 @@ plugins: 'partners/rancher_labs/index.md': 'csi_driver/partners/suse_rancher/index.md' 'partners/amazon_eks_anywhere/index.md': 'csi_driver/partners/amazon_eks_anywhere/index.md' 'partners/canonical/index.md': 'csi_driver/partners/canonical/index.md' - 'partners/cohesity/index.md': 'csi_driver/partners/cohesity/index.md' + 'partners/cohesity/index.md': 'csi_driver/partners/cohesity/index.md' 'partners/commvault/index.md': 'csi_driver/partners/commvault/index.md' 'partners/kasten/index.md': 'csi_driver/partners/kasten/index.md' 'partners/mirantis/index.md': 'csi_driver/partners/mirantis/index.md' @@ -73,7 +73,7 @@ nav: - 'HPE Ezmeral Runtime Enterprise': 'csi_driver/partners/hpe_ezmeral/install.md' - 'Amazon EKS Anywhere': 'csi_driver/partners/amazon_eks_anywhere/index.md' - 'Canonical': 'csi_driver/partners/canonical/index.md' - - 'Cohesity': 'csi_driver/partners/cohesity/index.md' + - 'Cohesity': 'csi_driver/partners/cohesity/index.md' - 'Commvault': 'csi_driver/partners/commvault/index.md' - 'Veeam Kasten': 'csi_driver/partners/kasten/index.md' - 'Mirantis': 'csi_driver/partners/mirantis/index.md' @@ -82,6 +82,11 @@ nav: - 'SUSE Rancher': 'csi_driver/partners/suse_rancher/index.md' - 'Tanzu TKGI': 'csi_driver/partners/tkgi/index.md' - 'VMware': 'csi_driver/partners/vmware/index.md' + - HPE COSI DRIVER FOR KUBERNETES: + - 'Overview': 'cosi_driver/index.md' + - 'Deployment': 'cosi_driver/deployment.md' + - 'Using': 'cosi_driver/using.md' + - 'Diagnostics': 'cosi_driver/diagnostics.md' - HPE GREENLAKE FOR FILE STORAGE: - 'Overview': 'filex_csi_driver/index.md' - 'Deployment': 'filex_csi_driver/deployment.md'