Skip to content

Latest commit

 

History

History
278 lines (194 loc) · 11.5 KB

deploy.md

File metadata and controls

278 lines (194 loc) · 11.5 KB
Raw

Deploying Cloud Foundry on a Kubernetes cluster

Prerequisites

Required Tools

You need the following CLIs on your system to be able to run the script:

Make sure that your Kubernetes config (e.g, ~/.kube/config) is pointing to the cluster you intend to deploy cf-for-k8s to.

Kubernetes Cluster Requirements

❗ This project is in it's early stages of development and hence the resource requirements are subject to change in the future. This document and the release notes will be updated accordingly. ❗

To deploy cf-for-k8s as is, the cluster should:

  • be running Kubernetes version within range 1.16.x to 1.18.x
  • have a minimum of 5 nodes
  • have a minimum of 4 CPU, 15GB memory per node
  • support LoadBalancer services
  • support metrics-server
    • Most IaaSes come with metrics-server, but if yours does not come with it or if you're using kind, you will need to include config-optional/add-metrics-server-components.yml when you render the K8s template.
  • defines a default StorageClass

Requirements for pushing source-code based apps to Cloud Foundry foundation

To be able to push source-code based apps to your cf-for-k8s installation, you will need to add OCI compliant registry (e.g. hub.docker.com) to the configuration.

Under the hood, cf-for-k8s uses Cloud Native buildpacks to detect and build the app source code into an oci compliant image and pushes the app image to the registry. Though cf-for-k8s has been tested with Google Container Registry and Dockerhub.com, it should work for any external OCI compliant registry.

Currently, we have tested the following two container registries:

  • Docker Hub:

    1. Create an account in hub.docker.com. Note down the user name and password you used during signup.
    2. Create a repository in your account. Note down the repository name.

  • Google Container Registry:

    1. Create a GCP Service Account with Storage/Storage Admin role.
    2. Create a Service Key JSON and download it to the machine from which you will install cf-for-k8s (referred to, below, as path-to-kpack-gcr-service-account).

Known Issues

This project is in it's early stages of development and hence there are features missing. For a list of the known issues, take a look at the GitHub issues tagged 'known-issue'.

Requirements to use an external database

cf-for-k8s can be configured to use an external database.

Steps to deploy

  1. Clone and initialize this git repository:

    git clone https://github.com/cloudfoundry/cf-for-k8s.git
cd cf-for-k8s
TMP_DIR=<your-tmp-dir-path> ; mkdir -p ${TMP_DIR}

  2. Create a "CF Installation Values" file and configure it:

    You can either: a) auto-generate the installation values or b) create the values by yourself.

    Option A - Use the included hack-script to generate the install values

    NOTE: The script requires the BOSH CLI in installed on your machine. The BOSH CLI is an handy tool to generate self signed certs and passwords.

    ./hack/generate-values.sh -d <cf-domain> > ${TMP_DIR}/cf-values.yml

    Replace <cf-domain> with your registered DNS domain name for your CF installation.

    Option B - Create the install values by hand

    1. Clone file sample-cf-install-values.yml from this directory as a starting point.

      cp sample-cf-install-values.yml ${TMP_DIR}/cf-values.yml

    2. Open the file and change the system_domain and app_domain to your desired domain address.

    3. Generate certificates for the above domains and paste them in crt, key, ca values

      • IMPORTANT Your certificates must include a subject alternative name entry for the internal *.cf-system.svc.cluster.local domain in addition to your chosen external domain.

  3. To enable Cloud Native buildpacks feature, configure access to an external registry in cf-values.yml:

    You can choose any of the cloud provider container registries, such as hub.docker.com, Google container registry, Azure container registry and so on. Below are examples for dockerhub or google container registry.

    1. To configure Dockerhub.com

      Add the following registry config block to the end of cf-values.yml file.

      app_registry:
   hostname: https://index.docker.io/v1/
   repository_prefix: "<my_username>"
   username: "<my_username>"
   password: "<my_password>"
      1. Update <my_username> with your docker username.
      2. Update <my_password> with your docker password.

    2. To configure a Google Container Registry, add the following registry config block to the end of cf-values.yml file.

      app_registry:
   hostname: gcr.io
   repository_prefix: gcr.io/<gcp_project_id>/cf-workloads
   username: _json_key
   password: |
   <contents_of_service_account_json>
      1. Update the gcp_project_id portion to your GCP Project Id.
      2. Change contents_of_service_account_json to be the entire contents of your GCP Service Account JSON.

  4. Run the following commands to install Cloud Foundry on your Kubernetes cluster.

    i. Render the final K8s template to raw K8s configuration

    ytt -f config -f ${TMP_DIR}/cf-values.yml > ${TMP_DIR}/cf-for-k8s-rendered.yml

    cf-for-k8s uses ytt to create and maintain reusable YAML templates. You can visit the ytt playground to learn more about it's templating features. In the above command, ytt can take a folder e.g. config or file via -f. See all options by running ytt help.

    ii. Install using kapp and pass the above K8s configuration file

    kapp deploy -a cf -f ${TMP_DIR}/cf-for-k8s-rendered.yml -y

    cf-for-k8s uses kapp to manage it's lifecycle. kapp will first show you a list of resources it plans to install on the cluster and then will attempt to install those resources. kapp will not exit untill all resources are installed and their status is running. See all options by running kapp help.

    Once you run the command, it should take about 10 minutes depending on your cluster bandwidth, size. kapp will provide updates on pending resource creations in the cluster and will wait until all resources are created and running. Here is a sample snippet from kapp output:

    4:08:19PM: ---- waiting on 1 changes [0/1 done] ----
4:08:19PM: ok: reconcile serviceaccount/cc-kpack-registry-service-account (v1) namespace: cf-workloads-staging
4:08:19PM: ---- waiting complete [5/10 done] ----
...

  5. Configure DNS on your IaaS provider to point the wildcard subdomain of your system domain and the wildcard subdomain of all apps domains to point to external IP of the Istio Ingress Gateway service. You can retrieve the external IP of this service by running

    kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[*].ip}'

    OR in certain environments, the external ip may be surfaced as a hostname instead. In that case use:

    kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[*].hostname}'

    If you used a single DNS record for both system_domain and app_domains, then have it resolve to the Ingress Gateway's external IP

    e.g.

    # sample A record in Google cloud DNS. The IP address below is the address of Ingress gateway's external IP
Domain         Record Type  TTL  IP Address
*.<cf-domain>  A            30   35.111.111.111

Validate the deployment

  1. Target your CF CLI to point to the new CF instance

    cf api --skip-ssl-validation https://api.<cf-domain>

    Replace <cf-domain> with your desired domain address.

  2. Login using the admin credentials for key cf_admin_password in ${TMP_DIR}/cf-values.yml

    cf auth admin <cf-values.yml.cf-admin_password>
# or using yq: cf auth admin "$(yq -r '.cf_admin_password' ${TMP_DIR}/cf-values.yml)"

  3. Create an org/space for your app:

    cf create-org test-org
cf create-space -o test-org test-space
cf target -o test-org -s test-space

  4. Deploy a source code based app:

    cf push test-node-app -p tests/smoke/assets/test-node-app
    Pushing app test-node-app to org test-org / space test-space as admin...
Getting app info...
Creating app with these attributes...

name: test-node-app
path: /Users/pivotal/workspace/cf-for-k8s/tests/smoke/assets/test-node-app
routes: test-node-app.<cf-domain>

Creating app test-node-app...
Mapping routes...
Comparing local files to remote cache...
Packaging files to upload...
Uploading files...

.... logs omitted for brevity


Waiting for app to start...

name: test-node-app
requested state: started
isolation segment: placeholder
routes: test-node-app.<cf-domain>
last uploaded: Tue 17 Mar 19:24:21 PDT 2020
stack:
buildpacks:

type: web
instances: 1/1
memory usage: 1024M
state since cpu memory disk details
#0 running 2020-03-18T02:24:51Z 0.0% 0 of 1G 0 of 1G

  5. Validate the app is reachable over https

    # for self-signed certs, use -k to allow insecure server connections when using SSL
curl -k https://test-node-app.<cf-domain>
    Hello World

Delete the cf-for-k8s deployment

You can delete the cf-for-k8s deployment by running the following command.

# Assuming that you ran `kapp deploy -a cf...`
kapp delete -a cf

Additional resources

Use the following resources to enable additional features in cf-for-k8s.