Skip to content

Commit

Permalink
Add operator project
Browse files Browse the repository at this point in the history
Signed-off-by: Michael Edgar <[email protected]>
  • Loading branch information
MikeEdgar committed Jul 8, 2024
1 parent 5f974da commit 1f07818
Show file tree
Hide file tree
Showing 76 changed files with 3,607 additions and 288 deletions.
File renamed without changes.
File renamed without changes.
35 changes: 33 additions & 2 deletions .github/workflows/integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,41 @@ jobs:
#
# See: https://quarkus.io/guides/tests-with-coverage#coverage-for-integration-tests
#
mvn verify -Pdocker -B --no-transfer-progress \
mvn verify -P container-image -B --no-transfer-progress \
-Dquarkus.package.write-transformed-bytecode-to-build-output=true \
-Dquarkus.docker.buildx.platform=linux/amd64,linux/arm64
- name: Build Operator Bundle Image
uses: docker/build-push-action@v6
with:
context: operator/target/bundle/console-operator/
platforms: linux/amd64,linux/arm64
provenance: false
push: false
file: operator/target/bundle/console-operator/bundle.Dockerfile
tags: |
streamshub/console-operator-bundle:latest
- name: Build Operator Catalog
run: |
curl -L -o opm https://github.com/operator-framework/operator-registry/releases/download/v1.43.1/linux-amd64-opm
chmod +x opm
sudo cp -v opm /usr/bin/
rm -vf opm
VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout | tr '[:upper:]' '[:lower:]')
operator/bin/generate-catalog.sh ${VERSION}
- name: Build Operator Catalog Image
uses: docker/build-push-action@v6
with:
context: operator/target/
platforms: linux/amd64,linux/arm64
provenance: false
push: false
file: operator/target/catalog.Dockerfile
tags: |
streamshub/console-operator-catalog:latest
- name: Archive Results
uses: actions/upload-artifact@v4
if: failure()
Expand All @@ -60,7 +91,7 @@ jobs:
## Save the context information for use in Sonar analysis
- name: Save Build Context
run: |
mkdir -v target
mkdir -vp target
echo "$GITHUB_CONTEXT" > target/build-context.json
env:
GITHUB_CONTEXT: ${{ toJson(github) }}
Expand Down
37 changes: 35 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build and Push API Image
- name: Build and Push API and Operator Image
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
Expand All @@ -55,7 +55,7 @@ jobs:
export QUARKUS_CONTAINER_IMAGE_ADDITIONAL_TAGS=latest
export GIT_REVISION=$(git rev-parse --short release)
# Build and push the release images using the commit tagged in `release:prepare`
mvn -B -P docker release:perform '-Drelease.arguments=-Dquarkus.docker.buildx.platform=linux/amd64,linux/arm64'
mvn -B -P container-image release:perform --no-transfer-progress '-Drelease.arguments=-Dquarkus.docker.buildx.platform=linux/amd64,linux/arm64'
- name: Login to Quay
uses: docker/login-action@v3
Expand All @@ -64,6 +64,39 @@ jobs:
username: "${{ secrets.IMAGE_REPO_USERNAME }}"
password: "${{ secrets.IMAGE_REPO_PASSWORD }}"

- name: Build and Push Operator Bundle Image
uses: docker/build-push-action@v6
with:
context: operator/target/bundle/console-operator/
platforms: linux/amd64,linux/arm64
provenance: false
push: true
file: operator/target/bundle/console-operator/bundle.Dockerfile
tags: |
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-operator-bundle:${{steps.metadata.outputs.current-version}}
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-operator-bundle:latest
- name: Build Operator Catalog
run: |
curl -L -o opm https://github.com/operator-framework/operator-registry/releases/download/v1.43.1/linux-amd64-opm
chmod +x opm
sudo cp -v opm /usr/bin/
rm -vf opm
VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout | tr '[:upper:]' '[:lower:]')
operator/bin/generate-catalog.sh ${VERSION}
- name: Build and Push Operator Catalog Image
uses: docker/build-push-action@v6
with:
context: operator/target/
platforms: linux/amd64,linux/arm64
provenance: false
push: true
file: operator/target/catalog.Dockerfile
tags: |
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-operator-catalog:${{steps.metadata.outputs.current-version}}
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-operator-catalog:latest
- name: Build UI
working-directory: ui
run: |
Expand Down
119 changes: 119 additions & 0 deletions .github/workflows/snapshot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
name: Publish Snapshot

on:
pull_request:
branches: [ 'main' ]
types: [ closed ]
paths-ignore:
- '.github/project.yml'

jobs:
release:
runs-on: ubuntu-latest
name: snapshot
if: ${{github.event.pull_request.merged == true}}

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Retrieve Project Metadata
uses: radcortez/project-metadata-action@603060a0627d4f04d37d20331ebd472377fd088b
id: metadata
with:
github-token: ${{secrets.GITHUB_TOKEN}}
metadata-file-path: '.github/project.yml'

- name: Setup JDK
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'adopt'

- name: Cache Maven Packages
uses: actions/cache@v4
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build and Push API and Operator Image
run: |
export QUARKUS_CONTAINER_IMAGE_REGISTRY="${{ secrets.IMAGE_REPO_HOSTNAME }}"
export QUARKUS_CONTAINER_IMAGE_GROUP="${{ secrets.IMAGE_REPO_NAMESPACE }}"
export QUARKUS_CONTAINER_IMAGE_USERNAME="${{ secrets.IMAGE_REPO_USERNAME }}"
export QUARKUS_CONTAINER_IMAGE_PASSWORD="${{ secrets.IMAGE_REPO_PASSWORD }}"
export QUARKUS_CONTAINER_IMAGE_PUSH="true"
export QUARKUS_CONTAINER_IMAGE_ADDITIONAL_TAGS=snapshot-${{github.base_ref}}
export GIT_REVISION=$(git rev-parse --short release)
# Build and push the snapshot images
mvn -B -P container-image verify --no-transfer-progress -DskipTests -Dquarkus.docker.buildx.platform=linux/amd64,linux/arm64
- name: Login to Quay
uses: docker/login-action@v3
with:
registry: "${{ secrets.IMAGE_REPO_HOSTNAME }}"
username: "${{ secrets.IMAGE_REPO_USERNAME }}"
password: "${{ secrets.IMAGE_REPO_PASSWORD }}"

- name: Build and Push Operator Bundle Image
uses: docker/build-push-action@v6
with:
context: operator/target/bundle/console-operator/
platforms: linux/amd64,linux/arm64
provenance: false
push: true
file: operator/target/bundle/console-operator/bundle.Dockerfile
tags: |
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-operator-bundle:${{steps.metadata.outputs.next-version}}
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-operator-bundle:snapshot-${{github.base_ref}}
- name: Build Operator Catalog
run: |
curl -L -o opm https://github.com/operator-framework/operator-registry/releases/download/v1.43.1/linux-amd64-opm
chmod +x opm
sudo cp -v opm /usr/bin/
rm -vf opm
VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout | tr '[:upper:]' '[:lower:]')
operator/bin/generate-catalog.sh ${VERSION}
- name: Build and Push Operator Catalog Image
uses: docker/build-push-action@v6
with:
context: operator/target/
platforms: linux/amd64,linux/arm64
provenance: false
push: true
file: operator/target/catalog.Dockerfile
tags: |
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-operator-catalog:${{steps.metadata.outputs.next-version}}
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-operator-catalog:snapshot-${{github.base_ref}}
- name: Build UI
working-directory: ui
run: |
npm ci --omit=dev
export BACKEND_URL=http://example
export CONSOLE_METRICS_PROMETHEUS_URL=http://example
export NEXTAUTH_SECRET=examplesecret
export LOG_LEVEL=info
export CONSOLE_MODE=read-only
npm run build
- name: Build and Push UI Image
uses: docker/build-push-action@v6
with:
context: ui/
platforms: linux/amd64,linux/arm64
provenance: false
push: true
tags: |
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-ui:${{steps.metadata.outputs.next-version}}
${{ secrets.IMAGE_REPO_HOSTNAME }}/${{ secrets.IMAGE_REPO_NAMESPACE }}/console-ui:snapshot-${{github.base_ref}}
File renamed without changes.
29 changes: 24 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,46 @@

include *compose.env

CONSOLE_API_IMAGE ?= quay.io/streamshub/console-api:latest
CONSOLE_UI_IMAGE ?= quay.io/streamshub/console-ui:latest
IMAGE_REGISTRY ?= quay.io
IMAGE_GROUP ?= streamshub
VERSION = $(shell mvn help:evaluate -Dexpression=project.version -q -DforceStdout | tr '[:upper:]' '[:lower:]')

CONSOLE_API_IMAGE ?= $(IMAGE_REGISTRY)/$(IMAGE_GROUP)/console-api:$(VERSION)
CONSOLE_OPERATOR_IMAGE ?= $(IMAGE_REGISTRY)/$(IMAGE_GROUP)/console-operator:$(VERSION)
CONSOLE_OPERATOR_BUNDLE_IMAGE ?= $(IMAGE_REGISTRY)/$(IMAGE_GROUP)/console-operator-bundle:$(VERSION)
CONSOLE_OPERATOR_CATALOG_IMAGE ?= $(IMAGE_REGISTRY)/$(IMAGE_GROUP)/console-operator-catalog:$(VERSION)
CONSOLE_UI_IMAGE ?= $(IMAGE_REGISTRY)/$(IMAGE_GROUP)/console-ui:$(VERSION)

CONSOLE_UI_NEXTAUTH_SECRET ?= $(shell openssl rand -base64 32)
CONSOLE_METRICS_PROMETHEUS_URL ?=
CONTAINER_RUNTIME ?= $(shell which podman || which docker)

container-image-api:
mvn package -f api/pom.xml -Pdocker -DskipTests -Dquarkus.container-image.image=$(CONSOLE_API_IMAGE)
mvn package -am -pl api -Pcontainer-image -DskipTests -Dquarkus.container-image.image=$(CONSOLE_API_IMAGE)

container-image-api-push: container-image-api
$(CONTAINER_RUNTIME) push $(CONSOLE_API_IMAGE)

container-image-operator:
mvn package -am -pl operator -Pcontainer-image -DskipTests -Dquarkus.container-image.image=$(CONSOLE_OPERATOR_IMAGE)
operator/bin/generate-catalog.sh $(VERSION)
$(CONTAINER_RUNTIME) build -t $(CONSOLE_OPERATOR_BUNDLE_IMAGE) -f operator/target/bundle/console-operator/bundle.Dockerfile
$(CONTAINER_RUNTIME) build -t $(CONSOLE_OPERATOR_CATALOG_IMAGE) -f operator/target/catalog.Dockerfile

container-image-operator-push: container-image-operator
$(CONTAINER_RUNTIME) push $(CONSOLE_OPERATOR_IMAGE)
$(CONTAINER_RUNTIME) push $(CONSOLE_OPERATOR_BUNDLE_IMAGE)
$(CONTAINER_RUNTIME) push $(CONSOLE_OPERATOR_CATALOG_IMAGE)

container-image-ui:
$(CONTAINER_RUNTIME) build -t $(CONSOLE_UI_IMAGE) ./ui -f ./ui/Dockerfile

container-image-ui-push: container-image-ui
$(CONTAINER_RUNTIME) push $(CONSOLE_UI_IMAGE)

container-images: container-image-api container-image-ui
container-images: container-image-api container-image-ui container-image-operator

container-images-push: container-image-api-push container-image-ui-push
container-images-push: container-image-api-push container-image-ui-push container-image-operator-push

compose-up:
> compose-runtime.env
Expand Down
34 changes: 27 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
# Console for Apache Kafka® on Kubernetes
# Console for Apache Kafka<sup>®</sup> on Kubernetes

This project is a web console designed to facilitate interactions with Apache Kafka® instances on Kubernetes, leveraging the [Strimzi](https://strimzi.io) Cluster Operator.
It is composed of two main parts:
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0) [![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=streamshub_console&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=streamshub_console) [![Coverage](https://sonarcloud.io/api/project_badges/measure?project=streamshub_console&metric=coverage)](https://sonarcloud.io/summary/new_code?id=streamshub_console)

- a REST API backend developed with Java and [Quarkus](https://quarkus.io/)
- a user interface (UI) built with [Next.js](https://nextjs.org/) and [PatternFly](https://patternfly.org)
This project is a web console designed to facilitate interactions with Apache Kafka<sup>®</sup> instances on Kubernetes, leveraging the [Strimzi](https://strimzi.io) Cluster Operator.
It is composed of three main parts:

- a [REST API](./api) backend developed with Java and [Quarkus](https://quarkus.io/)
- a [user interface (UI)](./ui) built with [Next.js](https://nextjs.org/) and [PatternFly](https://patternfly.org)
- a Kubernetes [operator](./operator) developed with Java and [Quarkus](https://quarkus.io/)

#### Roadmap / Goals

Expand All @@ -27,7 +30,7 @@ Please refer to the [installation README](./install/README.md) file for detailed
### Run locally

Running the console locally requires the use of a remote or locally-running Kubernetes cluster that hosts the Strimzi Kafka operator
and any Apache Kafka® clusters that will be accessed from the console. To get started, you will need to provide a console configuration
and any Apache Kafka<sup>®</sup> clusters that will be accessed from the console. To get started, you will need to provide a console configuration
file and credentials to connect to the Kubernetes cluster where Strimzi and Kafka are available.

1. Using the [console-config-example.yaml](./console-config-example.yaml) file as an example, create your own configuration
Expand Down Expand Up @@ -71,7 +74,24 @@ file and credentials to connect to the Kubernetes cluster where Strimzi and Kafk
## Contributing

We welcome contributions of all forms. Please see the [CONTRIBUTING](./CONTRIBUTING.md) file for how to get started.
Join us in enhancing the capabilities of this console for Apache Kafka® on Kubernetes.
Join us in enhancing the capabilities of this console for Apache Kafka<sup>®</sup> on Kubernetes.

## Releasing

### Milestones
Each release requires an open milestone that includes the issues/pull requests that are part of the release. All issues in the release milestone must be closed. The name of the milestone must match the version number to be released.

### Configuration
The release action flow requires that the following secrets are configured in the repository:
* `IMAGE_REPO_HOSTNAME` - the host (optionally including a port number) of the image repository where images will be pushed
* `IMAGE_REPO_NAMESPACE` - namespace/library/user where the image will be pushed
* `IMAGE_REPO_USERNAME` - user name for authentication to server `IMAGE_REPO_HOSTNAME`
* `IMAGE_REPO_PASSWORD` - password for authentication to server `IMAGE_REPO_HOSTNAME`
These credentials will be used to push the release image to the repository configured in the `.github/workflows/release.yml` workflow.

### Performing the Release
Releases are performed by modifying the `.github/project.yml` file, setting `current-version` to the release version and `next-version` to the next SNAPSHOT. Open a pull request with the changed `project.yml` to initiate the pre-release workflows. At this phase, the project milestone will be checked and it will be verified that no issues for the release milestone are still open. Additionally, the project's integration test will be run.
Once approved and the pull request is merged, the release action will execute. This action will execute the Maven release plugin to tag the release commit, build the application artifacts, create the build image, and push the image to (currently) quay.io. If successful, the action will push the new tag to the Github repository and generate release notes listing all of the closed issues included in the milestone. Finally, the milestone will be closed.

## License

Expand Down
2 changes: 1 addition & 1 deletion api/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Please check the project's [contributing guide](../CONTRIBUTING.md) first.

## Prerequisites

Ensure you have Kubernetes and Strimzi Cluster Operator installed on your system. Either [minikube](https://minikube.sigs.k8s.io/) or [OpenShifot Local](https://developers.redhat.com/products/openshift-local) are good options.
Ensure you have Kubernetes and Strimzi Cluster Operator installed on your system. Either [minikube](https://minikube.sigs.k8s.io/) or [OpenShift Local](https://developers.redhat.com/products/openshift-local) are good options.
One option to get started is to follow [Strimzi's Quick Starts](https://strimzi.io/quickstarts/).

You will also need a working installation of:
Expand Down
Loading

0 comments on commit 1f07818

Please sign in to comment.