Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add minio support for custom creds #25

Merged
merged 8 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 0 additions & 139 deletions charts/all/minio/templates/aws-minio-dep.yaml

This file was deleted.

34 changes: 17 additions & 17 deletions charts/all/minio/templates/secret-eso.yaml
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# ---
# apiVersion: "external-secrets.io/v1beta1"
# kind: ExternalSecret
# metadata:
# name: minio-secret-eso
# spec:
# refreshInterval: 15s
# secretStoreRef:
# name: {{ .Values.secretStore.name }}
# kind: {{ .Values.secretStore.kind }}
# target:
# name: minio-secret
# template:
# type: Opaque
# dataFrom:
# - extract:
# key: {{ .Values.minioRootCreds.key }}
---
apiVersion: "external-secrets.io/v1beta1"
kind: ExternalSecret
metadata:
name: minio-secret-eso
spec:
refreshInterval: 15s
secretStoreRef:
name: {{ .Values.secretStore.name }}
kind: {{ .Values.secretStore.kind }}
target:
name: minio-tenant-secret
template:
type: Opaque
dataFrom:
- extract:
key: {{ .Values.minioRootCreds.key }}
24 changes: 7 additions & 17 deletions charts/all/minio/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,31 +12,21 @@ minioRootCreds:

# Begin global parameters





operator:
operator:
env:
- name: "MINIO_OPERATOR_RUNTIME"
value: "OpenShift"
# securityContext:
# runAsUser: 1000770000
# runAsGroup: 1000770000
# fsGroup: 1000770000
# containerSecurityContext:
# runAsUser: 1000770000
# runAsGroup: 1000770000
# console:
# securityContext:
# runAsUser: 1000770000
# containerSecurityContext:
# runAsUser: 1000770000

tenant:
secrets:
name: minio-tenant-secret
existingSecret:
name: enabled
tenant:
name: my-minio-tenant
configuration:
name: minio-tenant-secret
existingSecret: true
pools:
- servers: 1
name: mypool
Expand Down
3 changes: 3 additions & 0 deletions dev-scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# dev-scripts

See detailed explaination on the [docs site](https://opendatahub-io-contrib.github.io/genai-llm-rag-pattern/dev-scripts)
5 changes: 5 additions & 0 deletions dev-scripts/argo-env.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh
ARGO_CMD=`oc get secrets -A -o jsonpath='{range .items[*]}{"oc get -n "}{.metadata.namespace}{" routes; oc -n "}{.metadata.namespace}{" extract secrets/"}{.metadata.name}{" --to=-\\n"}{end}' | grep gitops-cluster`
CMD=`echo $ARGO_CMD | sed 's|- oc|-;oc|g'`

eval $CMD
26 changes: 26 additions & 0 deletions docs/deploying.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: Deploying the RAG pattern.
---

## Assumptions

### GPUs

The current demonstration relies on [`flash-attention`](https://github.com/Dao-AILab/flash-attention) to decrease memory consumption for the LLM models. Today support to this limited to specific Nvidia GPUs which this system can work with. GPUs which are known to be good include:

- Nvidia L40S
- Nvidia A100
- Nvidia H100/H200

Note: The V100 GPUs are not supported.

### GPU pool management (WIP)

The pattern today allows GPU pools to be managed for scale-out computing via [MCAD](https://github.com/project-codeflare/multi-cluster-app-dispatcher/) and [Instascale](https://github.com/project-codeflare/instascale). It is important to note that this is designed primarily to manage scaling for batch workloads.

This works where:

1. The cluster auto-scaler is enabled (e.g. using the assisted installer into your own tenancy on AWS / GCP)
1. Clusters managed via OpenShift Cluster Manager (e.g. ROSA, ARO and OSD)

### Manual setup steps on OSD.
15 changes: 15 additions & 0 deletions docs/dev-scripts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
title: Development scripts
---

# Development scripts

These scripts are useful for development and automation where the gap has not been completely closed.

## `argo-env.sh`

Two argoCD deployments are created by the validated patterns operator. The depending on your identity and RBAC setup you may not get access with `cluster-admin` or similar.

Running (pre-authenticated with `oc`) `sh argo-env.sh` will provide the default admin passwords for each argo instance.

##
13 changes: 5 additions & 8 deletions values-secret.yaml.template
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,13 @@ version: "2.0"
# automatically generated inside the vault this should not really matter)

secrets:
- name: minio_root
- name: minio_tenant_root
vaultPrefixes:
- global
fields:
- name: minio_root_user
value: ''
- name: 'config.env'
value: |-
export MINIO_ROOT_USER=ROOTUSERNAME
export MINIO_ROOT_PASSWORD=ROOTUSERPASSWORD
onMissingValue: prompt
vaultPolicy: validatedPatternDefaultPolicy
- name: minio_root_password
value: ''
onMissingValue: prompt
vaultPolicy: validatedPatternDefaultPolicy

Loading