This repository is a work-in-progress and is not recommended for use in a POC until complete.
Vanilla configurations for a RHOAI instance to deploy a GenAI POC. This will deploy a variety of different required services including an LLM interface (AnythingLLM), a local S3 bucket, and other services needed for a GenAI POC on RHOAI.
Use the RHDP catalog item named "GenAI POC RHOAI 2025." This catalog item will deploy an OpenShift cluster on AWS with the following configurations:
- publicly trusted TLS keys
- some basic users
- an NVIDIA GPU node
- A few operator installations
- NVIDIA GPU Operator
- OpenShift Pipelines
- OpenShift Serverless (for KServe)
- OpenShift Service Mesh (for KServe)
- OpenShift GitOps
OpenShift GitOps is then wired up to use this repository by applying the bootstrap ApplicationSet.
One of the key elements for how this repository scaffolds your serving environment is that you need to move the model from wherever you trained it into the cluster. The synchronization from a source S3 bucket into the cluster is automated, and credentials need to be created in the cluster. An example manifest has been provided. You will need to modify the values to represent your actual model source, whether it be an existing S3 bucket in Amazon or the S3 compatible endpoint to IBM COS, where your models should be delivered after the alignment tuning process. You can apply this manifest, after updating with your values, using either the command line or by copying and pasting it into the OpenShift console's "➕" button in the top right.