deploy_oss.prolific

Install bosh-bootloader and dependencies

### What?
[BOSH](http://bosh.io/docs/) is an open source tool for deployment, release engineering, lifecycle management, and monitoring of distributed systems. It is what we use to deploy and manage Cloud Foundry.

While it is under the [Cloud Foundry Foundation umbrella](https://www.cloudfoundry.org/projects/#bosh), it is a distinct product. BOSH can be used to deploy just about any software and Cloud Foundry can (inadvisably) be deployed by something other than BOSH.

Since Cloud Foundry is a maintenance and monitoring system itself, it sometimes confuses people that there would be another maintenance and monitoring system underneath it. Think of it as the "who watches the watchmen" of [high availability](https://docs.cloudfoundry.org/concepts/high-availability.html). Or maybe "turtles all the way down". "Watchmen all the way down." Anyway...

There have been many ways to deploy BOSH over the years, but the latest, greatest, and under active development-est, is  **[bosh-bootloader (a.k.a. bbl)](https://github.com/cloudfoundry/bosh-bootloader)**.

`bbl` stands up the BOSH Director (VM and persistent disk). The BOSH Director deploys and monitors Cloud Foundry VMs and processes. And Cloud Foundry is used to deploy and monitor applications and services in the form of containerized workloads.

You'll use `bbl` to deploy your BOSH Director in a later story, but for now I've told you all of this just so you'll install it (and a few of its dependencies) on your workstation.

This isn't the most exciting first story, but all good things to those who wait, right?

### How?
Install or update the following dependencies on your local machine:
* **Golang** (run `brew install go`)
* **Terraform** is a tool for building, changing, and versioning infrastructure safely and efficiently (run `brew install terraform`)

The following dependencies can be installed through the Cloud Foundry [homebrew tap](https://github.com/cloudfoundry/homebrew-tap)
* **BOSH CLI V2** is written in Golang (the original was written in Ruby) and has an expanded feature set (`brew tap cloudfoundry/tap && brew install cloudfoundry/tap/bosh-cli`)
* **bbl** is a tool for paving infrastructure in your desired IAAS as well as deploying a BOSH director (run `brew install bbl`)
* **cf cli** is a tool for interacting with a Cloud Foundry deployment (run `brew install cf-cli`)

### Expected Result
* Run `go version` => "go version go1.10.2 darwin/amd64" or above
* `terraform --version` => "Terraform v0.11.7" or above
* `bosh -v` => "version 4.0.1-...." or above
* `bbl version` => "bbl 6.7.2 (darwin/amd64)" or above
* `cf version` => "cf version 6.37.0+...." or above

### Resources
[Docs: BOSH V2 CLI](https://bosh.io/docs/cli-v2.html)
[Docs: CF CLI](http://docs.cloudfoundry.org/cf-cli/)
[Docs: What is Cloud Foundry?](https://www.cloudfoundry.org/platform/)
[Docs: What is Terraform?](https://www.terraform.io/intro/index.html)

### Relevant Teams and Repos
**BOSH:** [cloudfoundry/bosh](https://github.com/cloudfoundry/bosh)
**BOSH:** [cloudfoundry/bosh-cli](https://github.com/cloudfoundry/bosh-cli)
**Infrastructure:** [cloudfoundry/bosh-bootloader](https://github.com/cloudfoundry/bosh-bootloader)
**CLI:** [cloudfoundry/cli](https://github.com/cloudfoundry/cli)

L: bbl

---

Install Google Cloud SDK

### What?
You're going to deploy your BOSH Director to [Google Cloud Platform (GCP)](https://cloud.google.com). Although [Amazon Web Services (AWS)](https://aws.amazon.com/what-is-aws/) was historically Pivotal Cloud Foundry's go-to IAAS, we've decided to migrate as much as possible to  GCP. GCP is significantly cheaper than AWS and managing costs is significantly easier—no more reserved instances! We also have an excellent relationship with GCP PMs and Engineers and can escalate issues directly through our contacts.

The `gcloud` CLI manages authentication, local configuration, developer workflow, and interactions with the Cloud Platform APIs. Download and install the [Google SDK](https://cloud.google.com/sdk) to get `gcloud`, as you'll need it to `bbl up`.

This is another "fun-sized" story (you know you'll miss them once you hit the 6-hour ones....)

### How?
Follow **[these instructions](https://cloud.google.com/sdk/docs/downloads-interactive)** to install Google Cloud SDK on your workstation.

Once you've done that, run `gcloud init` in your terminal.

You should have received an invitation to join a GCP project from your facilitator or ask+cf@pivotal.io. Use your @pivotal.io email address and the name of your assigned project to configure the tool (the project name is in the GCP dashboard menu bar and the URL).

### Expected Result
Run `gcloud version` => "Google Cloud SDK 203.0.0 ..." or higher

Run `gcloud config list` =>
```
[core]
account = your-email@pivotal.io
disable_usage_reporting = False
project = PROJECT_ID

Your active configuration is: [default]
```

### Resources
[Docs: Comparison GCP <> AWS](https://cloud.google.com/docs/compare/aws/)
[Internal Doc: The Great GCP Migration of 2016](https://docs.google.com/document/d/1ze6znVK32UlpsmGXHmMn4ZAAAvuUCK0yVPXhXH74-n0/edit#heading=h.x5ivrrqjuddi)

L: bbl

---

Create a GCP service account

### What?
We're going to create a **[service account](https://cloud.google.com/compute/docs/access/service-accounts)** with `gcloud`, which `bbl` can use to set up infrastructure on GCP. Service accounts authenticate applications running on your virtual machine instances to other Google Cloud Platform services, like APIs.

### How?
Create a new directory in your workspace and `cd` into it. You're going to be generating a few files over the next couple stories and it'll be good to put them somewhere logical.

Then, follow these steps to set up your GCP service account, replacing ACCOUNT_NAME with a name of your choice and PROJECT_ID with your assigned project's name:
1. Run the following command to create a new GCP service account:
```
gcloud iam service-accounts create ACCOUNT_NAME
```
1. Create a key for that account to access your project:
```
gcloud iam service-accounts keys create "service-account.key.json" --iam-account "ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com"
```
1. Add the "editor" [Identity and Access Management (IAM) role](https://cloud.google.com/compute/docs/access/iam) to your service account for your project:
```
gcloud projects add-iam-policy-binding PROJECT_ID --member 'serviceAccount:ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com' --role 'roles/editor'
```

### Expected Result
You should now have a `service-account.key.json` in your workspace.

Verify that the account is listed in your GCP dashboard under Menu > IAM & Admin > Service Accounts.

### Resources
[Docs: GCP Service Accounts](https://cloud.google.com/compute/docs/access/service-accounts)
[Docs: Identity and Access Management (IAM) Roles](https://cloud.google.com/compute/docs/access/iam)
[Docs: Comparison GCP <> AWS](https://cloud.google.com/docs/compare/aws/)

L: bbl

---

Pave your IaaS and stand up a BOSH Director with `bbl up`

### What?
As previously discussed, **[bbl](https://github.com/cloudfoundry/bosh-bootloader)** is a command line utility for standing up a BOSH Director (the VM and persistent disk) on an IAAS.
In the past, you would have used Terraform (or on AWS, a CloudFormation template) to create the networks and VMs and [bosh create-env](https://bosh.io/docs/init-google.html) to set up your Director.
Under the hood that's what `bbl` uses too.

#### Digging in: What does `bbl` do, actually?
`bbl`'s responsbilities can be broken down into the following things:
1. IaaS paving
2. Bootstrapping a BOSH director and jumpbox
3. Uploading a cloud-config to the BOSH director

[Let's break it down.](https://giphy.com/gifs/L253rIFQkyXSg/tile)

**IaaS Paving**
Before you deploy a BOSH director or CF (or anything else), you'll need to set up some back IaaS resources. This varies a bit by IaaS (AWS, GCP, etc.), but most of the time that includes the following things:
- A private network for deploying your VMs. The network will typically include a subnet for each **availability zone** (or AZ) you're hoping to use. Each subnet is defined by a specific range of IP addresses that it's allowed to use. For example, if you want to use 3 AZs, you'll end up with three subnets defined by IP ranges 10.0.32.0/24, 10.0.48.0/24, and 10.0.64.0/24. Each subnet would live in a different AZ, so that if one AZ were to experience issues, software deployed to the other AZs would continue to function.
- Routing rules. These are closely related to your network and subnet definitions, but they express how traffic **from your VMs** gets to the public internet, or even to other VMs. For example, you'll see rules that say "Traffic to 10.0.0.0/16" -- meaning, all private IP addresses -- "should get routed to the gateway for my private network."
- Firewall rules. These allow or disallow network traffic to your VMs from certain IP addresses. Most commonly, we want to block all traffic from public IP addresses, and then allow a few exceptions so that users can, you know, use the platform and apps deployed there.
- External IP addresses. Most VMs will use **private IPs** from your `10.0.X.Y` ranges, but a few VMs need to be available on the public internet. In our example, these will eventually be used by load balancers and the BOSH jumpbox.
You might see a few other things that are specific to a certain IaaS -- instance groups on GCP and security groups on AWS, for example -- the resources listed above are the most common across platforms.

**Bootstraping a BOSH Director and Jumpbox**
All that IaaS-paving was done for the purpose of getting a BOSH director running. HOWEVER, for super security fun time reasons, we want to deploy the director a private IP -- that means that any IP address outside of our private network will fail to send traffic to it.

Well, in that case, how do _I_ access the BOSH director from my local machine? That's where the jumpbox becomes your friend. The jumpbox is a pretty lazy VM: all it does is hang out on both a public and private IP address, and it acts as the bridge between the outside world (i.e. you) and your private network. Any time you want to communicate with the BOSH director, you have to go through the jumpbox. If that sounds like a lot of work, you wouldn't be wrong, but have no fear: `bbl` is on the case.

After running `bbl up` below, if you run `bbl print-env`, you'll see a lot of output. One of the most important lines you'll see will say something like:
```
export BOSH_ALL_PROXY=ssh+socks5://jumpbox@<some IP address>:22?private-key=/path/to/some/bosh_jumpbox_private.key
```
This line of `bash` would set environment variables that the BOSH CLI can use to discover the location and credentials for your jumpbox, so that you can tunnel through the jumpbox and issue CLI commands against the director.

So, to **actually** set all of the variables, you'd need to run:
```
eval "$(bbl print-env)"
```
from the bbl state directory.

**Uploading a cloud-config to the BOSH director**
The point of a [cloud-config](https://bosh.io/docs/cloud-config) is to hold IaaS- and environment-specific information about your deployment, so that your deployment manifest -- in this case, cf-deployment, which you'll use soon -- doesn't have to know about things like your IaaS credentials or identifiers for resources on your IaaS.

If you were setting up your environment without bbl, after paving your IaaS with Terraform, and [bootstrapping a bosh director with the `create-env` command](https://bosh.io/docs/init-google.html), you'd need to build a cloud-config by hand based on the output from your Terraform work. `bbl` bails you out of this work by constructing that cloud-config for you, and uploading it to your BOSH director.

### How?
Make sure you're in the directory that you created in the last step, with the `service-account.key.json` file generated by `gcloud`.

Run

_Note: if you specified a different default region for `gcloud init`, change "us-east1" below. `gcloud config list` will list your previously-entered defaults._

```
bbl up \
  --name FRIENDLY_ENV_NAME \
  --gcp-region us-east1 \
  --gcp-service-account-key service-account.key.json \
  --iaas gcp
```

> If you don't supply a friendly name, bbl will pick a random name for you.

...and...*<<drumroll, please>>*...it might fail. Sad panda. But if it does, the error is straightforward, so follow its instructions to fix the problem.

**Pro-tip:** if you're using iTerm, hold down the command-⌘ key and click on the link to open it directly from the prompt.

### Expected Result
If your `bbl up` succeeds, it will have printed the list of steps it has taken to spin up your brand-spanking new BOSH Director and generated a whole slew of files and directories that represent bbl's internal state in the current working directory. Those directories and files correspond, more or less, to the different steps outlined above. Just as importantly, they all have some interface for customizing your BOSH setup. No worries, we won't make you do that here.
- `bbl-state.json`. This file is very important as it contains credentials and other metadata related to your BOSH director and infrastructure.
- `terraform`. This directory includes the terraform templates that bbl used to pave your IaaS.
- `jumpbox-deployment`, `create-jumpbox.sh`, and `delete-jumpbox.sh` -- for bootstrapping a jumpbox. The shell scripts actually contain the `create-env` commands that bbl used to set up your jumpbox.
- `bosh-deployment`, `create-director.sh`, and `delete-director.sh` -- for bootstrapping a BOSH director. The shell scripts actually contain the `create-env` commands that bbl used to set up your director.
- `cloud-config`, which includes the cloud-config bbl generates for you.

**Financially important:** Please do not expose your GCP credentials by pushing `bbl-state.json` to Github. You'd be surprised how quickly bitcoin mining VMs pop up. This costs the company beaucoup moolah.


### Resources
[Docs: Using bosh create-env](https://bosh.io/docs/cli-v2.html#create-env)
[Docs: cloud-configs](https://bosh.io/docs/cloud-config)
[Docs: What is cf-deployment?](https://github.com/cloudfoundry/cf-deployment/blob/master/texts/deployment-guide.md)
[Video: Creating Cloud Environments](https://drive.google.com/file/d/1ltNggSltyLdRs8orh-qCzZTM_EAx3x7h/view?usp=sharing)
  - Note: your onboarding week facilitator may schedule this talk in-person if you're in SF

L: bbl

---

Connect to your BOSH environment

### What?
Time to target the Director VM you just `bbl up`ed.

### How?
To set the BOSH environment, run the following commands:

`bbl print-env` to see the environment variables you'll set to use your new BOSH.

`eval "$(bbl print-env)"` to export them. **Note: you will definitely need to include the double-quotes for this command to work properly.** Once you've run it, you should be able to start working with your bosh director. If you ever have a hard time connecting to your BOSH director, your first step should be to re-run `eval "$(bbl print-env)"`.

### Expected Results
When you run `bosh env`, it should print something like:

![image](https://www.pivotaltracker.com/file_attachments/77821681/download)

Annnd that's it!

# Congratulations, you've successfully deployed BOSH! #bigpatontheback
L: bbl

---

[RELEASE] Deploy BOSH Director to GCP with Bosh Bootloader ⇧

---

Learn you some DNS, TLS, and Load Balancing

I'll admit, this story is a little goofy, since there isn't a ton to do. Instead, we're just going to introduce a handful of interrelaled concepts, so that you'll feel a bit more informed as you go through the process of setting up everything needed to make your Cloud Foundry addressable from the public internet.

Let's start of with some vocabulary:

* An **IP address** is number associated with a computer, and allows other computers to find them. We typically think of two kinds of IP addresses: **public** and **private** IPs. Public IPs are unique, and anybody on the internet can try to communicate with the computer associated to a public IP. Private IPs are not accessible by just any computer -- you have to be on the same private network to communicate with a computer on a private IP. In most Cloud Foundry deployments, the majority of VM instances use private IPs.
* **Load balancers** are computers with the sole job of distributing requests across a set of other computers (sometimes referred to as "backends" or "backend servers"). In CF deployments, we typically create load balancers with public IPs, and then configure them to route traffic to components that have private IPs.
* A **domain** is a series of strings separated by the `.` character that serves as an alias for an IP address, or set of IP addresses. For example, `run.pivotal.io` is a domain that maps to the IP address(s) of the load balancers on PWS. We use domains for a few reasons. One is that IP addresses are hard for humans to remember, but `google.com` is pretty easy. We also use domains to group computers together and even do some load balancing. For example, `google.com` does not resolve to a single IP address (one poor computer could never handle that kind of load!).
  * A **subdomain** is a term you'll see as well. It's a relative term that describes the relationship of one domain to another -- for example, `my-site.example.com` is a subdomain of `example.com`.
* **DNS** stands for "Domain Name System," which translates domains IP addresses. We'll explain how it works in a second.
* **TLS** stands for "Transport Layer Security". You'll often also hear it referred to as **SSL**, which stands for "Secure Socket Layer," even though the term "SSL" is deprecated. Anywho, TLS is a system for encrypting your communications and validating that you're communicating with the right computer on the internet. To prove its identity, a computer will present a **certificate** to the machine making the request.

Why introduce all this now? Because you'll work on these bit-by-bit over the next few stories. Here's how.

Most components in CF are deployed to private IPs. As such, for a user to access the platform, or any app deployed to it, we need some way of getting traffic into the private network for CF. For that, we typically set up load balancers; as mentioned before, load balancers are mostly deployed to public IPs, so they're accessible from the internet. But how does a user know where to find the load balancer? We could just share the public IP around, but that's hard for people to remember, and looks janky on a marketing webpage. So, we create a domain for our system and have it (and some of its subdomains) resolve to our load balancers.

Ok, so we've got domains that resolve to load balancers, which are deployed to public IPs and route traffic to our private IPs. But, how does a computer verify the identity of the load balancers once it sends them some traffic using its domain? The load balancer presents all clients with its TLS certificate, which essentially states "I am in fact `example.com`, and I have the certificate to prove it." This is important: the certificate includes the domain in its "metadata."

### Side note: How does DNS work exactly?
As part of these stories, you'll pick a subdomain of `onboarding.cf-app.com` -- like `my-env.onboarding.cf-app.com` -- and use it for your environment. That means that if anyone tries to make an HTTP request to `my-env.onboarding.cf-app.com`, a process on their computer, called a **DNS resolver**, will make a series of queries to computers on the internet, called **DNS servers**, essentially asking, "Do you know this guy, `my-env.onboarding.cf-app.com`?"

The simplest way to manage such a request is to create an **A record.** These records map a domain, like `onboarding.cf`, to a specific IP address like `1.2.3.4`. In this setup, when a user runs `curl onboarding.cf`, their DNS resolver first queries its nearest DNS server for `.cf`. The DNS server redirects your computer to _another_ DNS server that knows about `.cf` domains. Then your computer asks that server about `onboarding.cf`. That DNS server has the A record, and returns an answer: `1.2.3.4`

However, Cloud Foundry requires a slightly more complex setup, because we want to resolve all queries that _end_ with `onboarding.cf` -- for example, `api.onboarding.cf` or `my-app.onboarding.cf`. We also often want to reserve certain subdomains like `bosh.onboarding.cf` or `ssh.onboarding.cf`.

To achieve that, we first set up a different kind of record, an **NS record**, which maps a domain to other DNS servers. It essentially says, "If you're looking for information about domains that end with `onboarding.cf`, go ask this other set of DNS servers." Your DNS resolver will then query those servers instead. This is a useful way to hook up different DNS providers or piggy-pack on existing domains. You're going to leverage this functionality in order to register your subdomain of `onboarding.cf-app.com` on GCP.

In a few stories, you'll use bbl to create load balancers that have IP addresses associated with them. The same  bbl command will _also_ create a few DNS records:
- An NS record that defines the Google DNS servers. You'll take that list of servers to our "master" DNS account on GCP to configure it to defer to your DNS records.
- An A record for a few reserved subdomains like `ssh.my-env.onboarding.cf-app.com`. This DNS record maps the `ssh.` subdomain to a load balancer for SSH connections to CF Apps (you'll get to explore this feature more later).
- An A record for `*.my-env.onboarding.cf-app.com`. This record maps **all** other subdomains of `my-env.onboarding.cf-app.com` to a load balancer that delegates traffic to the router. This **wildcard domain** enables apps that get pushed to the platform to piggyback on the `my-env.onboarding.cf-app.com` domain, so that you can make subdomains on the fly like `my-app.my-env.onboarding.cf-app.com`.

To get an idea of how this works, try running `dig +trace api.run.pivotal.io` to visualize how each part of the domain gets resolved.

### Resources
[Blog post: What is a domain name?](https://www.lifewire.com/what-is-a-domain-name-2483189)
[Blog post: SSL vs TLS, what's the difference?](https://luxsci.com/blog/ssl-versus-tls-whats-the-difference.html)
[Tutorial: How SSL and TLS encryption works](http://computer.howstuffworks.com/encryption4.htm)
[Blog post: What is a Domain Name Server?](https://www.lifewire.com/what-is-a-dns-server-817513)
[Video: How DNS works](https://www.youtube.com/watch?v=GlZC4Jwf3xQ)
[Video: A Tour of TLS](https://www.youtube.com/watch?v=yzz3bcnWf7M)

L: bbl

---

Choose a domain and create TLS certificates

### What?
Ok, we're getting closer to deploying. Before you can go prime time with BOSH and CF, you'll need a way to make your platform addressable from the internet -- specifically, by getting a domain name for your Cloud Foundry deployment. In the next story, we're going to use `bbl` to create a load balancer for our soon-to-be Cloud Foundry deployment. This load balancer requires a TLS certificate, which will include your domain in its metadata. So, the order of operations here is:
1. Choose a domain
2. Create a TLS certificate that includes that domain as the "Common Name"
3. Create load balancers and configure them with your TLS certificate

In this story, you'll do steps 1 and 2. The last step will come in the next story.

(If you want some more information on DNS, jump down to the "Side note" below.)

### How?
We're going to use [OpenSSL](https://www.openssl.org/) to generate a self-signed TLS certificate. In real life, you'd get one from a trusted Certificate Authority like [Let's Encrypt](https://letsencrypt.org/), but for this exercise self-signed is sufficient.

So, first, choose a subdomain of `onboarding.cf-app.com` -- something like `my-env.onboarding.cf-app.com`, but feel free to be creative and call it whatever you like. From here on out, we're going to call this domain your **system domain**, because this is the domain that the CF system will use for itself, including things like the CF API (AKA Cloud Controller).

Next, we'll make a TLS certificate. To do this, run `openssl req` with a few arguments:

* `-x509` outputs a x509 structure, a standard that defines the format of public key certificates.
* `-newkey rsa:2048` generates a new RSA key of 2048 bits in size.
* `-keyout` and `-out` arguments set output file paths (I generally use `key.pem` and `cert.pem`, respectively)
* The `-nodes` argument sets it to not encrypt private keys, meaning you won't have to enter a PEM passphrase.

**Caution:** when you run OpenSSL, it will ask you to enter values like your Country and State. The only one that really matters (and it matters a great deal) is the "Common Name," which you should fill with your system domain.

### Expected Result
You should have two new `.pem` files, one containing your private key and one containing your cert.

When you run `openssl x509 -noout -subject -in cert.pem` the information returned matches what you entered.

### Resources
[Blog post: Signed vs Self-Signed Certificates](https://www.thoughtco.com/signed-vs-self-signed-certificates-3469534)
[How to: Get Common Name from TLS cert](https://unix.stackexchange.com/questions/103461/get-common-name-cn-from-ssl-certificate)

L: bbl

---

Create load balancers
### What
We've been alluding to load balancers for a while, but now we'll actually get around to creating them with bbl.

A load balancer is some machine -- or set of machines -- that serve as the entry point to the system and delegate traffic to other components. Importantly, one of their jobs is to make sure that traffic is distributed evenly among VMs that do similar work. The easiest load balancer to understand is the HTTP load balancer, which balances traffic to the [CF routers](http://docs.cloudfoundry.org/concepts/architecture/router.html).

Most public IaaSes provide load balancing solutions, but some do not. For example, if you were deploying to vSphere, you'd need to come up with your own load balancing solution: most people use [HAProxy](https://github.com/cloudfoundry-incubator/haproxy-boshrelease) or hardware load balancers for this.

In this story, you'll update bbl's "plan" with the `--lb-type` flag. When set to `cf`, this flag will cause bbl to make a few different load balancers for you:
- HTTP and Websocket load balancers that delegate to the HTTP router, commonly known as GoRouter
- SSH load balancer the delegates to the SSH proxy
- TCP load balancer that delegates to the TCP routers
- Credhub load balancer that delegates directly to Credhub

The other thing you'll do in this command is configure the load balancers with the TLS certificate you made in the previous story.

Finally, if you provide the `--lb-domain` flag, bbl will also create DNS records for each of the domains associated with your load balancers, like A records for `*.my-env.onboarding.cf-app.com` and `ssh.my-env.onboarding.cf-app.com`

### How
To set up load balancers with bbl, we're going to leverage bbl's "plan" feature. Remember how you saw all those directories and files in your working directory after running `bbl up`? Under the hood, bbl first laid out a "plan" of all the stuff it was going to do, and then it did them. We're going to ask bbl to update its plan to include load balancers, and then we're going to run `bbl up` again to effect those changes on GCP.

Run
```
bbl plan \
  --lb-type cf \
  --lb-cert PATH_TO_TLS_CERT --lb-key PATH_TO_TLS_KEY \
  --lb-domain YOUR_DOMAIN \
  --gcp-service-account-key service-account.key.json
```
You should be able to see that a handful of files in your bbl state directory have been updates -- specifically, your terraform and cloud-configs directory.

Next, run
```
bbl up --gcp-service-account-key service-account.key.json
```


### Expected Results
Network Services > Load balancing to see the load balancer that `bbl` generated on GCP.

Network Services > Cloud DNS to see the [Cloud DNS Zone](https://cloud.google.com/dns/overview) and [record sets](https://cloud.google.com/dns/overview#supported_dns_record_types) that `bbl` generated on GCP.

### Resources
[Docs: Types of load balancing](https://cloud.google.com/compute/docs/load-balancing/#types_of_load_balancing)
[Docs: HAProxy](http://www.haproxy.org/)

L: bbl

---

Set domain nameservers

### What
So far, you've created a TLS certificate, configured your domain, and then used `bbl` to create both Load Balancers (which are configured with that certificate), as well as DNS records for your system domain -- which is itself a subdomain of `onboarding.cf-app.com`.

If you look back at our earlier description of DNS, you'll recall that you need a series of NS records to guide the resolver to more specific subdomains. The CF R&D Org has an NS record for `cf-app.com`, and the Onboarding Staff has created the NS record for `onboarding.cf-app.com`. You yourself created a series of A records for `*.my-env.onboarding.cf-app.com` and such -- all that's left to do is create one last NS record for `my-env.onboarding.cf-app.com`.

### How
**First, find the nameservers for the DNS records that bbl created for you.**
1. Open the GCP console in your browser.
1. In the hamburger menu (the three-line icon in the top left corner), select `Network services` > `Cloud DNS`.
1. Click on the zone created by `bbl` (it often has `bbl-` prepended to the zone name).
1. The first entry in the table of DNS records should be an NS record for your system domain. In the `Data` column, you should see four values of the form `ns-cloud-b1.googledomains.com.`. Copy that list.

**Next, create an NS record in the Onboarding DNS account to link up your new DNS zone to the exisiting zone for `onboarding.cf-app.com`.**
1. Next to the Google Cloud Platform logo, you should see the name of a GCP project. Click on the dropdown and select the project called `CF-Onboarding-dns`.
1. In the hamburger menu (the three-line icon in the top left corner), select `Network services` > `Cloud DNS`.
1. Choose the zone called `onboarding-wildcard-domain`. At this point, you should see NS and SOA records for `onboarding.cf-app.com.`, and possibly NS records for some subdomains that were never cleaned up (if you see any such records, notice that they've also got four values from `googledomains.com`. We're going to make a new record just like that.)
1. Click `Add record set`
1. Fill out the form. The `DNS Name` should match your system domain, the `Resource Record Type` should be `NS`, and the `Name server` list should have one value for each of the values you copied earlier. (By default, the TTL is set at 5 minutes. The TTL, or Time to Live, indicates how long a DNS client is allowed to cache the DNS lookup result. A longer TTL means you don't need to query the DNS server as much, but if you make a change to your DNS record clients will be slower to pick up the change.)
1. Finish by clicking `Create`

### Expected Result
DNS updates can take a minute to propogate, so go grab a cup of coffee or a snack. When you get back, go to your GCP Cloud DNS entry. Find the IP associated with `*.your-domain.com`. Running `dig api.your-domain.com` should return the same IP address.


L: bbl
---

[RELEASE] Set up load balancers

---

Update number of compilation workers in cloud-config

### What?
GCP creates fresh projects with surprisingly low CPU quotas, which means we generally have to reduce the number of machines we use if we want to deploy CF on a fresh project. [Compilation workers](https://bosh.io/docs/deployment-manifest.html#compilation) temporarily increase your CPU usage beyond what will ultimately be needed by CF once it's up and running, so since we have flexibility on how many we need this is where we have wiggle room to slip in under the barebones quota.

To do this, we edit our **[cloud config](http://bosh.io/docs/cloud-config.html)**, a YAML file that defines the IaaS-specific configuration used by the BOSH Director and all of its deployments. It allows us to separate this configuration out into its own file and keep deployment manifests IaaS-agnostic.

To deploy Cloud Foundry you'll be taking advantage of the [cloud config](http://bosh.io/docs/cloud-config.html) that `bbl` generated for you during `bbl up`.

### How?
Check your CPU quota in the GCP Console by navigating to Home > IAM & Admin > Quotas.  There are a lot of line items in this list, you can trim it down by setting the `Metric` drop-down to `CPUs` (only) and the setting the `Location` drop-down to `US-East1` (only).  Currently, we have quite a few more CPUs than the bare minimum of 24 (check the "Limit" column).  This means we do not need to change the cloud config but knowing how to do so for the future is a good thing, so read on.

Run `bosh cloud-config > cloud-config.yml`

Then, open the new `cloud-config.yml` file in an editor and locate `compilation: workers:`. If you were to have 24 CPU quota on GCP, then you would want to reduce the number of compilation machines to 2; if you have something way higher than 24 CPU (as we have), you can leave it alone.

Then, if you edited the file, run `bosh update-cloud-config cloud-config.yml` to apply your changes.

### Expected Result
When you run `bosh -d cf deploy` a couple stories down, it doesn't fail with a reached CPU quota error.

### Resources
[Docs: What is a cloud config?](http://bosh.io/docs/cloud-config.html)
[Forum question: What's the difference between > and >>?](http://askubuntu.com/questions/382793/whats-is-the-difference-between-and-in-shell-command)
[YAML Validator](http://codebeautify.org/yaml-validator)

L: cf-deployment

---

Select a stemcell for your BOSH deployment

### What?
The **[stemcell](http://bosh.cloudfoundry.org/docs/stemcell.html)** is the foundation of every VM the [BOSH Director](https://bosh.io/docs/bosh-components.html#director) deploys. It is a versioned Operating System [image](https://docs.docker.com/engine/userguide/storagedriver/imagesandcontainers/#images-and-layers) wrapped with [IaaS](https://en.wikipedia.org/wiki/Cloud_computing#Infrastructure_as_a_service_.28IaaS.29)-specific packaging.

A typical stemcell contains a bare bones OS skeleton with a few common utilities pre-installed, some configuration files to securely configure the OS by default, and a BOSH Agent. The [BOSH Agent](https://bosh.io/docs/bosh-components.html#agent) is there to listen for instructions from the Director and to carry them out.

Stemcells are distributed as [tarballs](https://bosh.io/docs/build-stemcell.html#tarball-structure). You need to upload a stemcell for your BOSH Director to use when it deploys your Cloud Foundry VMs.

### How?

Available stemcells **[are listed at bosh.io](http://bosh.io/stemcells)**. Identify the appropriate stemcell for the IaaS  ([Google KVM](https://en.wikipedia.org/wiki/Google_Compute_Engine#Machine_Types)) and OS ([Ubuntu Xenial](https://en.wikipedia.org/wiki/Ubuntu_version_history#Ubuntu_16.04_LTS_(Xenial_Xerus))) you'll be using to deploy. You'll also need to pick a specific version of the stemcell; to find the version number, you'll need to look at [cf-deployment.yml](https://github.com/cloudfoundry/cf-deployment/blob/master/cf-deployment.yml) (you'll learn more about this file in the next story), and check in the `stemcells` section. You'll also notice that some stemcells on bosh.io have both a "Light" and "Full" version. A full stemcell includes the entire OS image while a light stemcell contains an ID or URI pointing to a public OS image which has already been uploaded on that IaaS. For public clouds like GCP you'll almost always use the light version.

Now that we've selected which stemcell to upload, we can upload the stemcell tarball to our BOSH Director:

```
bosh upload-stemcell --sha1 $STEMCELL_SHA1 \
  https://bosh.io/d/stemcells/bosh-google-kvm-ubuntu-xenial-go_agent?v=$STEMCELL_VERSION
```

Note: the SHA1 value should be listed next to the stemcell download link on bosh.io.

### Expected Result

If all went according to plan, the stemcell should be visible via:

```
bosh stemcells
```

### Resources
[Docs: What is a stemcell?](http://bosh.cloudfoundry.org/docs/stemcell.html)
[Docs: What are light stemcells?](https://bosh.io/docs/build-stemcell.html#light-stemcells)
[Docs: Tarball structure](https://bosh.io/docs/build-stemcell.html#tarball-structure)

### Relevant Repos and Teams
**BOSH:** [cloudfoundry/bosh](https://github.com/cloudfoundry/bosh)
**BOSH:** [cloudfoundry/bosh-agent](https://github.com/cloudfoundry/bosh-agent)
L: cf-deployment

---

Deploy CF using BOSH + cf-deployment

### What?
[Cf-deployment](https://github.com/cloudfoundry/cf-deployment/blob/master/README.md#purpose) is the supported way to deploy CF in the OSS world. It consists primarily of a deployment manifest, as well as **operations files**, or ops-files, that transform the manifest to create a slightly different deployment.

The standard cf-deployment is fairly hefty, so we're going to scale ours down to a more reasonable size using an [operations file](https://bosh.io/docs/cli-ops-files.html) that reduces the number of VMs we deploy. Using an operations file to make small adjustments like this means we can edit our deployment without changing the core `cf-deployment.yml` manifest.

### How?
Clone the **[cf-deployment repo](https://github.com/cloudfoundry/cf-deployment)** to your workstation (we assume you put it in `~/workspace`, next to your onboarding directory).
```
bosh -d cf deploy ../cf-deployment/cf-deployment.yml -v system_domain=<YOUR_DOMAIN> --non-interactive -o ../cf-deployment/operations/use-compiled-releases.yml -o ../cf-deployment/operations/scale-to-one-az.yml
```

This one step is part of a step-by-step procedure listed in [these instructions](https://github.com/cloudfoundry/cf-deployment/blob/master/texts/deployment-guide.md), but this backlog has already set you up with everything you need.

Let's discuss the options we used:
- We added `--non-interactive` flag. Normally, after initial preprocessing steps, BOSH will present the "manifest changes", differences between the currently-deployed manifest (currently nothing) and the deploying manifest. Without this flag, you'll be prompted ~5 minutes after the `bosh deploy` command to approve the manifest changes.
- We added `-o operations/use-compiled-releases.yml`, to skip compiling all of your releases.
- We added `-o operations/scale-to-one-az.yml` to the deploy command. This ops-file will cut your deployment size in half, so you'll deploy faster (and your deployment will cost less money).


### Expected Result
Wait for the BOSH deploy to complete. It will take a while...like 20 minutes or so...you might want to check out the next "intermission" story, or go grab a snack, or play game of ping pong.
Once the deploy is finished, run `bosh vms`. All of them (currently 15) should have a status of "running".

### Resources
[Docs: What is a BOSH release?](https://bosh.io/docs/release.html)
[cf-deployment Github README](https://github.com/cloudfoundry/cf-deployment/blob/master/README.md)

L: cf-deployment

---

Entertain yourself during the CF deploy "intermission"

### What?
You have some time to kill while your CF deploys. Keep an eye on it (it could flake out at any moment), but otherwise here's some light reading:

* [How the Cloud Balances its load](https://docs.cloudfoundry.org/concepts/overview.html#balances) on the official [overview of Cloud Foundry](https://docs.cloudfoundry.org/concepts/overview.html)™
* [PCF Linux Containers Overview by Onsi](https://docs.google.com/a/pivotal.io/document/d/1QNcmQCrHZNAr17ULoYis_sDPNV0X4IUO1grFDBlQdO8/edit?usp=sharing)—I really recommend this one, even if you don't get to it now.
* The [Diego design notes](https://github.com/cloudfoundry/diego-design-notes): a comprehensive, up-to-date, and (most importantly) readable overview of Cloud Foundry components and their interaction. Plus, that diagram (diagram, *so* hot right now).

Some good videos:
* [Pivotal Cloud Foundry Overview with Onsi Fakhouri](https://www.youtube.com/watch?v=7APZD0me1nU)
* [CF Container to Container Networking with Jay Dunkelberger](https://docs.google.com/file/d/0B9Srlx6zCCQtdXVjSG5JZklUYmc/view)
* [Diego Runtime Overview with Eric Malm](https://www.youtube.com/watch?v=iv5EpheLLh0)
* [The Illustrated Children's Guide to Kubernetes](https://www.youtube.com/watch?v=4ht22ReBjno)

And some basic tutorials, specific to topics you might be wondering about right now:
* [Basic Bash scripting tutorial](http://ryanstutorials.net/bash-scripting-tutorial/)
* [Basic Linux/Bash tutorial](http://ryanstutorials.net/linuxtutorial/)
* [Basic Linux Networking reference](http://www.penguintutor.com/linux/basic-network-reference)

### Expected Result
Mastery of the material.

(I'm joking)

L: cf-deployment

---

Login to CF CLI using creds from Credhub

### What?
We're almost there. The final step is to target your newly deployed Cloud Foundry.

### How?
First, point the CF CLI at your deployed cloudfoundry. To do this, run `cf api api.YOUR_DOMAIN.com --skip-ssl-validation`.

Next, before you can login to the CF CLI, you'll need to get your admin password, which is stored in Credhub. When you used `bbl` to create your BOSH setup, it colocated a Credhub server with the BOSH director, so we can use `bbl` to get a handful of environment variables that will help you target the Credhub that was created as part of your BOSH setup.

Make sure you've got your bbl env set up (`eval "$(bbl print-env)"`). If you run `env | grep CREDHUB`, you can look at all of the environment variables that get used to access Credhub.

Next, make sure you've got the credhub CLI installed. You can do that with `brew install cloudfoundry/tap/credhub-cli`.

Next, you can use the credhub CLI to search for the "path" that the variable is stored at:
```
credhub find -n cf_admin_password
```

The result should include a path that looks like `/bosh-bbl-env-baikal-2018-06-04t22-02z/cf/cf_admin_password`. Copy that path, and use the credhub CLI one more time to fetch the value of the credential:
```
credhub get -n /bosh-bbl-env-baikal-2018-06-04t22-02z/cf/cf_admin_password
```

The output should include a field called `value` -- that's your CF admin password.

Finally, provide your credentials via `cf login` (hint: if it is not already apparent, your username is "admin").

### Expected Result
`cf login` works.

### Troubleshooting
* Do the domain names associated with your TLS cert, your load balancer, and your system_domain all match the domain you registered on GCP?
* Are the nameservers associated with your domain the same as those associated with your load balancer? (run `bbl lbs` to check)
* When you `dig` or `ping` your CC api endpoint, is the IP you hit what you expect it to be?
* If you were redeploying, did you run the `bosh -n interpolate` step before running `bosh -d cf deploy`?

### Resources
[Forum question: Get Common Name from TLS cert](https://unix.stackexchange.com/questions/103461/get-common-name-cn-from-ssl-certificate)

L: cf-deployment

---

Dive into CLIs

### What?
A CLI , or **Command-line interface** , is a way to issue commands via the terminal. Many Cloud Foundry products don't have any GUIs (Graphical User Interfaces, a website for example), instead the way users interact with our products is via many (many) CLIs.  You've been copying and pasting a lot of CLI commands as part of this onboarding, but for those of us who haven't used CLIs before, let us dig in a little more.

Not all CLIs are the same, but after you start using them you will find a lot of commonality.

Here are some of the Cloud Foundry CLI's you will be using in this onboarding:
- [gcloud](https://cloud.google.com/pubsub/docs/quickstart-cli)
- [bbl](https://github.com/cloudfoundry/bosh-bootloader)
- [credhub](https://github.com/cloudfoundry-incubator/credhub-cli)
- [bosh](https://bosh.io/docs/cli-v2/)
- [cf](https://github.com/cloudfoundry/cli)

Now that you have all of these CLIs set up already from prior stories, let's dig into them.

**How to get help!**

There are many common ways to get help text about the CLI you want to use. Unfortunately, nothing is completely standardized, so you might have to try a few options first.

In your terminal type `bbl --help` and see the results. It will show everything that's available via the bbl cli.

Typically you can append `--help`, `-help`, `-h`, or `help` and get help text, but nothing is standardized, so you might need to try each to find out which one works.

For example all of the following are valid commands to get help for the `bbl` cli.
```
bbl --help
bbl -h
bbl help
```

The `help` results will typically return a list of commands you can run with the CLI. If there is a specific CLI command you want to find out more about you can append the same help words at the end of the command. For example `bbl env-id --help` will give help text about the `env-id` command.

**Reading CLI usage syntax**
One of the common features of help text, is showing the CLI usage. The usage is written in a common syntax.

For example here is the usage for the `bosh deploy` command: `bosh [OPTIONS] deploy [DEPLOY-OPTIONS] PATH`

- `[Square brackets]` indicate an optional parameter
- `CAPITALS` indicate a value (or values) that the user needs to input and change for their needs. When you replace the text it should not be capitalized.
- `--words-prepended-with-dashes` indicates a flag to change how the command is run. The flags usage should be explained in the help text.
-  `--words-prepended-with-dashes-with-equal-sign-after=VALUE` indicates an option that the user assigns a value to. Its usage should be explained in the help text.

Let's try using a flag. Run `bosh instances`.  `this command runs fine when you have only 1 deployment`

You will get the helpful error message `expected argument for flag -d, --deployment`. This is telling you that you must use the `-d` flag to specify a bosh deployment, in order to return the bosh instances.
Given this, let's try running `bosh -d YOUR-DEPLOYMENT-NAME instances`. Success! (If you're having trouble getting your deployment name try `bosh deployments --help`.

**The Linux CLI**
If you've been using the terminal, you've been using a CLI. Commands like `cd` (change directory), `ls` (list), and `cp` (copy) are all apart of the linux and mac CLIs.
In order to get help for the linux CLI you need to prepend the command with `man`, which stands for manual.

For example `man cp` results in the following manual for `cp`
```
NAME
     cp -- copy files

SYNOPSIS
     cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
     cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory

DESCRIPTION
     In the first synopsis form, the cp utility copies the contents of the source_file to the
     target_file.  In the second synopsis form, the contents of each named source_file is copied
     to the destination target_directory.  The names of the files themselves are not changed.  If
     cp detects an attempt to copy a file to itself, the copy will fail.
```
Manual pages can be a little bit dense, so always feel free to google things like `linux terminal copy one file to a different folder` and you should be able to find some help from stack overflow.


### How?
**Let's practice using CLIs**

0. Look at the help text for credhub
0. Add a `value` credential type  with any contents you want to credhub
___
0. Look at the help text for `bosh` and for `bosh logs`
0. Use `bosh logs` to download the logs for all diego-cell instances
___
0. Use `mkdir` to make a new temp file `learning-clis` in the `/tmp` directory.
0. Use `vim` to make a new file `hello-world.sh` in your new directory. In the file write `echo "hello world"`
0. Use `chmod` to make the file `hello-world.sh` executable
0. Run your new script!

### Expected Result
You should be able to add a credential to credhub. You should see something like the following output:
```
id: 27b47c13-d427-4758-8857-8c176a754490
name: /my-super-cool-value
type: value
value: <redacted>
version_created_at: "2019-03-22T23:51:13Z"
```
You should be able to get bosh logs and see them downloaded on your computer

```
Task 3678 | 23:52:24 | Fetching logs for diego-cell/a21fbda8-0a3d-41cb-9f16-6ff3a3286cd2 (0): Finding and packing log files (00:00:05)

Task 3678 Started  Fri Mar 22 23:52:24 UTC 2019
Task 3678 Finished Fri Mar 22 23:52:29 UTC 2019
Task 3678 Duration 00:00:05
Task 3678 done

Downloading resource '070ca5d5-699b-4b1c-55e8-81efe931fc68' to '/tmp/cf.diego-cell.0-20190322-165231-227811.tgz'...

#############################################################  99.94% 8.54 MiB/s
Succeeded
```

Your hello-world.sh script should print out `hello world`.

### Resources
[linux command line primer](https://lifehacker.com/a-command-line-primer-for-beginners-5633909)
[how to make a new directory](https://www.techwalla.com/articles/how-to-make-a-new-directory-in-linux)
[how to make a file executable and run it](https://askubuntu.com/questions/229589/how-to-make-a-file-e-g-a-sh-script-executable-so-it-can-be-run-from-termina)

---

Party on, Garth

## You just deployed Cloud Foundry
![party](https://media.giphy.com/media/yoJC2GnSClbPOkV0eA/giphy.gif)

L: cf-deployment

---

[RELEASE] Deploy CF with cf-deployment ⇧