Working with Kabanero Collections

Note	This repository contains the guide documentation source. To view the guide in published form, view it on the website.

What you will learn

You will learn how to create a local Kabanero Collection Hub for your organisation. After learning about the general structure of a Collection, you will apply some changes for an application stack. You will then build, test, and publish your own customised Kabanero Collection.

This guide does not cover working with Tekton pipelines, which is the subject of a separate guide.

Prerequisites

1. Docker must be installed.

2. Appsody must be installed.

3. To build a Kabanero Collection, yq must be installed for processing YAML files.

4. (Optional) To build a Codewind index file that can be used for local application development with VS Code, Eclipse, or Eclipse Che, you must also install python3 and pyyaml for processing YAML files.

Getting started

A Kabanero Collection Hub contains a set of Kabanero Collections that organisations can use to develop, build, and deploy containerized, microservice-based applications.

A Kabanero Collection includes an Appsody stack, together with build and deployment artifacts, such as Tekton pipelines. Stacks consist of base container images and project templates for different language runtimes and framework, such as Microprofile with Open Liberty, or Node.js with Express.

Creating a local Kabanero Collection Hub

The public Kabanero Collection Hub contains all the latest collections from the Kabanero project. Before working with Kabanero Collections, you must build the Kabanero Collection Hub locally by cloning the public GitHub repository.

Run the following commands to clone the public Kabanero Collection Hub and push a copy of your repository (private-org) to your GitHub Enterprise organisation (https://github.acme.com/my_org/collections.git):

git clone https://github.com/kabanero-io/collections.git
cd collections
git remote add private-org https://github.acme.com/my_org/collections.git
git push -u private-org

Understanding the file structure of a Kabanero Collection

Now that you have a local clone of the Kabanero Collection Hub, you will learn about the file structure so that you are familiar with the contents and configurable options.

Kabanero Collections are categorized into one of the following collection types:

• stable collections meet a set of predefined technical requirements.

• incubator collections are actively being worked on to meet the requirements for a stable collection.

• experimental collections are just that! You can use them for proof of concept work or to try out specific scenarios, but these collections are not being actively worked on.

Although three categories are available, Kabanero builds only stable or incubator collections.

In a Kabanero Collection GitHub repository, collections are contained in a folder that matches the collection category. For example, stable/, incubator/, or experimental/. Each folder contains a common/ directory for pipelines, and a collection folder for each collection in that category. For example:

<category_folder>
├── common/
|   ├── pipelines/
|   |   ├── <common-pipeline-1>/
|   |   |       └── [pipeline files that make up a full tekton pipeline used with all collections in this category]
|   |   └── <common-pipeline-n>/
|   |           └── [pipeline files that make up a full tekton pipeline used with all collections in this category]
├── collection-1/
|   ├── [collection files]
├── collection-2/
|   └── [collection files]

Each collection has a standard file system structure, which is represented in the following diagram:

collection-1
├── README.md
├── stack.yaml
├── collection.yaml
├── image/
|   ├── config/
|   |   └── app-deploy.yaml
|   ├── project/
|   |   ├── [files that provide the technology components of the stack]
|   |   └── Dockerfile
│   ├── Dockerfile-stack
|   └── LICENSE
├── pipelines/
|   ├── my-pipeline-1/
|   |       └── [pipeline files that make up the full tekton pipeline]
└── templates/
    ├── my-template-1/
    |       └── [example files as a starting point for the application, E.g. "hello world"]
    └── my-template-2/
            └── [example files as a starting point for a more complex application]

When you build a collection, the build processes some files in this structure to generate a Docker image for the collection. Other files, such as templates and pipelines, are compressed and stored as tar files in an Appsody repository. The Docker images are used by the Appsody CLI to generate a development Docker container for local application development.

By modifying the files in a collection you can customize a collection for your organisation. The following list describes each file and its purpose:

README.md

Describes the contents of the collection and how it should be used.

stack.yaml

Defines the different attributes of the stack and which template the stack should use by default.

collection.yaml

Defines the different attributes of the collection and which container image and pipeline the collection should use by default.

app-deploy.yaml

Defines the configuration for deploying an Appsody project that uses the Appsody Operator. The Appsody Operator can install, upgrade, remove, and monitor application deployments on Kubernetes clusters.

Dockerfile

Defines the deployment Docker image that is created by the appsody build command. The Dockerfile contains the content from the stack and the application that is created by a developer, which is typically based on one of the templates. The image can be used to run the final application in a test or production environment where the Appsody CLI is not present.

Dockerfile-stack

Defines the development Docker image for the stack, exposed ports, and a set of Appsody environment variables that can be used during local application development.

LICENSE

Details the license terms for the Collection.

pipelines/

This directory contains Tekton pipeline information for a Collection. The pipeline information defines kubernetes-style resources for declaring CI/CD pipelines. A Collection can have multiple pipelines.

templates/

This directory contains pre-configured templates for applications that can be used with a stack image. These templates help a developer get started with a development project.

Creating and modifying Kabanero Collections

Setting up a local build environment

In addition to the tools that are defined in the pre-requisites section of this guide, to correctly build a Kabanero Collection, set the following environment variables by running export <ENVIRONMENT_VARIABLE=option> on the command line :

IMAGE_REGISTRY_ORG=kabanero

Defines the organization for images

CODEWIND_INDE=false

Defines whether to build the Codewind index file for application development in VS Code, Eclipse, or Eclipse Che. If you want to build and test a collection for use with Codewind in an IDE, change this value to true.

If you plan to manually release collections, you must also export the following environment variables:

IMAGE_REGISTRY_PUBLISH=false

Defines whether to publish images to an image registry

IMAGE_REGISTRY=<registry>

Defines the image registry

IMAGE_REGISTRY_USERNAME=<registry_username>

Defines the user name for the image registry IMAGE_REGISTRY_PASSWORD=<registry_password> Defines the password for the image registry

You are now ready to build a Kabanero Collection.

Building a Kabanero Collection

To build all the incubator collections, run the following command from the root directory of your local Kabanero Collections repository:

 ./ci/build.sh
 ```

When the build completes, you can find the deployment Docker images in your local registry by running the `docker images` command.

Other collection assets can be found in the `$PWD/ci/assets/` directory.

The build process also adds your local Kabanero Collection to the Appsody repository list.

== Testing a Kabanero Collection locally

First, make sure that your local Kabanero index is correctly added to the Appsody repository list by running `appsody repo list`.
The output is similar to the following example:


If the `kabanero-index-local` repository is not in the list, add it manually by running the following command:

appsody repo add kabanero-index-list file://$PWD/ci/assets/kabanero-index-local.YAML

Draft comment: Do we need to set the repo as the default?

You can now test your updated collection.

To test the collections using local docker images, rather than pulling them from docker hub, set the following environment variable:

export APPSODY_PULL_POLICY=IFNOTPRESENT

To create a new project that is based on your updated collection, run:

appsody init whatevercollectionwechoose (and if we set a default we can avoid specifying the index-repo/stack)

Draft comment: We should have a quick sample and appsody run to prove it works correctly.

Draft comment: We should have something better here for testing pipelines.

Testing your collection would not be complete without testing your pipelines. Working with pipelines is covered in a separate guide.

== Testing your Kabanero Collection with a test GIT release

Draft comment: Is this more a Todd test? ->-> unedited

Before creating a final production release for use with your Kabanero installation, you may wish to create a test release on a test GIT repository.

Use these instructions to create a GIT release manually from your local build.
Once all the artifacts are uploaded to the GIT Release, go to the assets inside that release and find the file kabanero-index.yaml and copy its URL. e.g. https://github.com/kabanero-io/collections/releases/download/v0.2.0.beta2/kabanero-index.yaml
Within your Kabanero environment create a sample.yaml based on this sample yaml e.g. https://raw.githubusercontent.com/kabanero-io/kabanero-operator/master/config/samples/full.yaml
In the yaml update the URL for the kabanero-index.yaml to tghe one from the release. e.g. https://github.com/kabanero-io/collections/releases/download/v0.2.0.beta2/kabanero-index.yaml
Save the yaml and then apply it to your Kabanero instance kubectl apply -f sample.yaml
This will update the collections and pipelines in your environment. New collections and pipeline should get activated as a result of this step.

== Releasing Kabanero Collections

When you are happy with the changes to your Kabanero Collection, push the changes back to your GIT repository:

git commit -a -m "Test Kabanero Collection created" git push -u private-org

You can use Jenkins or Travis to trigger events based on webhooks. For example, you can set up a webhook that triggers a Travis build
of your collections when a GIT merge takes place, providing an additional build test.

Draft comment: Annotated tags?

It is good practice to create release tags in GIT for versions of your collections. Create a GIT tag for your
test collection:

git tag v0.1.0 -m "Test collection, version 0.1.0"

Push the tags to your GIT repository by running `git push --tags`.

A further webhook can be set up to trigger a Travis build that generates a GIT release, pushing the images to the image
repository.

image:https://github.com/kabanero-io/draft-guide-collection-nodejs/raw/master/resources/codewind-workspace.png[Diagram
shows the VS Code project view]