Intro

This lab has been created for RH Summit 2023 Compliance-as-code interactive session by Tommer Amber.

This session provides a demonstration of how to manage compliance policies for multiple Kubernetes clusters across disciplines, including configuration management, image vulnerability detection, continuous integration and continuous delivery (CI/CD) pipelines, and cluster misconfiguration mutation. During this session, you’ll get an overview of how to use with Red Hat Advanced Cluster Management for Kubernetes, Red Hat Advanced Cluster Security for Kubernetes, ArgoCD, and Kyverno to:

• Design end-to-end solutions for multicloud environments.
• Manage Red Hat OpenShift (Kubernetes) clusters using GitOps.
• Implement security best practices in Red Hat OpenShift and Kubernetes.
• Deploy and manage popular open source tools for microservices and security.

Prerequisits

0. ACM Hub Cluster available
1. Install:
   • Git
   • Helm
   • Python3
   • Ansible
   • Ansible module for kubernetes - ansible-galaxy collection install kubernetes.core
   • Relevant python modules - pip3 install openshift pyyaml kubernetes --user
   • oc

HowTo

0. Fork this project

1. SSH Clone (A must for the automation)

   1.1. Create SSH Key on you machine

   ssh-keygen
   cat ~/.ssh/id_rsa.pub

   1.2. Copy the public key and create new repo SSH key here - https://github.com/settings/ssh/new

   1.3. Clone the repo

   export USERNAME=<YOUR_GIT_REPO_USERNAME>
   git clone [email protected]:$USERNAME/Compliance-As-Code.git
   cd Compliance-As-Code
   export BASE=$(pwd)

   1.4. Change the Playbook to point to your git forked repo

   sed -i "s,tommeramber,$USERNAME,g" Part-0-Ansible-Playbook/playbook.yaml

2. Login to Hub cluster from your local machine

3. Register/Import or provision Openshift clusters to ACM

4. Run the Ansible playbook - Automation Explainer

cd Part-0-Ansible-Playbook
ansible-playbook playbook.yaml

5. Login to the new ArgoCD instance and see that (almost) everything is ready

6. Label the managed clusters in ACM to get security policies by ACM

How everything is installed on the managed clusters?

The ArgoCD is synced with our Git Repo, and it deploys ACM policies on ACM, they are based on ACM's built-in Governance feature.

ACM Policies defines for ACM what is the desired state of K8s objects on the managed clusters (or the Hub itself) and in case they are adsent - it (ACM) will create them on the relevant cluster.

ACM relies on managed clusters labels to decide which policy should be installed on which cluster.

By integrate kyverno policies inside ACM policies, ACM deploys the kyverno policies compliance policies on the managed clusters that has kyverno installed.

Label-to-Policy Legend

Policy Name	Explainer	Requires Label	Type	Source
Install Kyverno	Install kyverno on the managed cluster/s	install-kyverno=true	ACM Policy	Link
Install Kyverno's Policy Reporter	Install policy-reporter UI on the managed cluster/s	install-kyverno=true	ACM Policy	Tommer Amber Original
Install ACS	Install ACS on Hub Cluster	None	ACM Policy	Link
Link ACS to Managed Cluster	Link ACS to all managed clusters, requires a manual operation described here	None	ACM Policy	Link
Etcd Backup	Deploy a CronJob on that managed cluster that will backup Etcd periodically	etcd-backup=true	ACM Policy	Link
Etcd Encryption	Enable etcd encryption for your cluster to provide an additional layer of data security (data at rest)	enforce-etcd-encryption=true	ACM Policy	Link
Validate Kyverno Reports	Display in ACM any drift from Kyverno policies on managed clusters	install-kyverno=true	ACM Policy	Link
Disallow "Latest" Tag for a Pod	Validates that the image specifies a tag and that it is not called latest. * Requires Kyverno to be installed on the managed cluster.	kyverno-disallow-latest-tag=true	Kyverno Policy	Link
Add Quota by default to new Namespaces	Control quota limits in new namespaces. * Requires Kyverno to be installed on the managed cluster. * Can be edited to allow namespace exceptions	kyverno-add-default-quota=true	Kyverno Policy	Link

So what's up with all the "app-of-apps"?

Our Ansible Automation "manually" applyed the initial app-of-apps YAML file on our ACM Openshift-GitOps namespace.

This YAML points our ArgoCD instance to look at this Helm Chart directory in the git repo (in your case it's the forked repo), and from that point forward ArgoCD will apply all the other YAMLs for us.

This Helm Chart directory actually holds ArgoCD Applications & ApplicationSets, that once applied (by our ArgoCD itself after the initial app-of-apps creation by the Ansible Automation) - will generate all the YAMLs in the Part-2-Compliance-as-GitOps directory one by one:

Policy Name	Explainer	Created by
install-security-tools	1. Install ACS on ACM 2. Install Kyverno + Policy-Reporter	ArgoCD Application
acm-policies	Install our ACM-natvie custom policies that ACM uses to enforce/inform roles on the managed clusters	ArgoCD ApplicationSet
acs-policies	Import our managed cluster from ACM to ACS	ArgoCD ApplicationSet
kyverno-policies	Apply our Kyverno-k8s-native custom policies that will be installed on our managed clusters that has a specific label. You can see the required label in the README.md	ArgoCD ApplicationSet
acm-argo-objects	Link the YAMLs our Ansible Automation generated & applied to link between our ACM and ArgoCD instance; From that point our ArgoCD literally manages itself and every change to our Argo instance will be applied by our ArgoCD	ArgoCD Application