Skip to content

Commit

Permalink
Add self-hosting documentation (#419)
Browse files Browse the repository at this point in the history 
* Update README.md

* Add files via upload

* remove extra line break

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* fix formatting

* change link format from html to markdown

* missed a link

* Run prettier

* run prettier with yarn instead of npx?

* fix codeblock formatting

* run prettier formatting

* adding self-hosting docs

* revise language based on feedback, add instructions for runner application installation

* start adding QEMU

* add images to vm setup docs, separate out qemu types

* add new qemu dir

* finish up Windows and add macos

* run prettier on readme

* add images to windows, cleanup formatting, fix spelling errors

* change organasation structure and fix typos

* re-roganize, add cloud-init details, dogfood testing

* Fix spellign errors, dogfooding, add advanced qemu config guides

* testing some style stuff that the linter doesnt like

* testing some style stuff that the linter doesnt like

* changing the syle to abuse titles less

* changing the syle to abuse titles less

* changing the syle to abuse titles less

* changing the syle to abuse titles less

* changing the syle to abuse titles less

* cleanup dangling :

* Formatting pass, add new diagrams

* center images and lint

* proof-reading and docgfood pass on k8s docs

* add info on creating secrets, minor formatting

* move k8s to new branch to unblock the rest of the docs

* re-center images

* add paddig above ssh image

* change image indent so linter wont wreck the spacing

* remove text artifact inserted by linter

* center images

* add disclaimers to advanced configs, add more examples to windows

* remove text artifact introduced by linter and minor formatting fix

* add missing image file

* Update docs/12-self-hosting/01-overview.mdx

Co-authored-by: Gabriel Le Breton <[email protected]>

* Update docs/12-self-hosting/03-host-creation/03-QEMU/01-overview.mdx

Co-authored-by: Gabriel Le Breton <[email protected]>

* Update docs/12-self-hosting/03-host-creation/03-QEMU/01-overview.mdx

Co-authored-by: Gabriel Le Breton <[email protected]>

* Update docs/12-self-hosting/03-host-creation/03-QEMU/03-linux-liveiso.mdx

Co-authored-by: Gabriel Le Breton <[email protected]>

* fix issue with metal diagram failing to render as a svg, its a png now

* remove accidental file addition

* fix formatting

* Update docs/12-self-hosting/03-host-creation/03-QEMU/03-linux-liveiso.mdx

Co-authored-by: Gabriel Le Breton <[email protected]>

* add power usage blurb

* add doublequotes to VNC invocation

* add sections to overview

* add not on hackintosh legality

* add note about remote access security to MacOS

* fix typo in link

* fix linux path for multipass

---------

Co-authored-by: David Finol <[email protected]>
Co-authored-by: Max Roby <[email protected]>
Co-authored-by: Gabriel Le Breton <[email protected]>
  • Loading branch information
@cloudymax @davidmfinol @GabLeRoux
4 people authored Nov 16, 2023
1 parent 4747e1c commit cd8711e
Show file tree
Hide file tree
Showing 88 changed files with 3,225 additions and 590 deletions.
97 changes: 97 additions & 0 deletions docs/12-self-hosting/01-overview.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
toc_max_heading_level: 4
---

# About Self-Hosting

Projects often encounter a constraint or requirement which makes free-tier hosted CI/CD runners
insufficient for their needs. In these cases hosting your own CI/CD runner can be a viable
alternative to premium-tier services or subscriptions. Self-hosting may also provide access to
resources that are simply not available on many CI/CD services such as GPUs, faster drives, and
newer CPU models.

This guide will cover basic methods for hosting CI/CD runners on Bare-Metal, Virtual Machines, or
using Cloud Runner. Containerized hosts will not be discussed because of their inherent reliance on
insecure practices such as
[Docker-in-Docker](http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/),
[Privileged Containers](https://www.trendmicro.com/en_th/research/19/l/why-running-a-privileged-container-in-docker-is-a-bad-idea.html)
and the additional tooling required to mitigate those risks such as
[Kaniko](https://github.com/GoogleContainerTools/kaniko) or
[Kata Containers](https://katacontainers.io/).

## 📚 Prerequisite Knowledge

Users of this guide should already be familiar with the Linux command-line, Shell scripting, and
have a basic grasp of CI/CD concepts. For users who are not familiar with these concepts, we have
included some resources for you to start your learning journey below.

- [Techworld with Nana](https://www.youtube.com/@TechWorldwithNana)
- [DevOps Toolkit](https://www.youtube.com/@DevOpsToolkit)
- [Introduction to Bash Scripting](https://itsfoss.com/bash-scripting-tutorial/)

## 📋 Constraints

There are many ways to self-host CI/CD runners, and which one is best for you will depend on your
own situation and constraints. For the purpose of this guide we will make the following assumptions:

- 💻 User already has their own hardware
- 💸 Budget for new hardware, software, or services is $0
- 🛠️ FOSS tools should be prioritized where possible
- 📜 We define `Self-Hosting` in this context to refer to a user taking responsibility for the
operating-system level configuration and life-cycle-management of a given compute resource (metal,
on-prem, cloud VM, VPS etc...)

## ⚠️ Security Disclaimer

This guide strives to maintain a balance between convenience and security for the sake of usability.
The examples included in this guide are intended for use with on-prem hardware without public IP
addresses accessible from external networks. Security is a constantly moving target which requires
continuous effort to maintain. Users should conduct their own security review before using the
following techniques on production or public systems.

## ⚡️ Power Costs

Hosting your own runners also comes with an increase in power consumption. This will vary based on
the hardware you use and the prices of energy in your area. Below are some useful resources for
discovering the potential energy costs of self-hosting.

- https://outervision.com/power-supply-calculator
- https://energyusecalculator.com/electricity_computer.htm

## 💻 System Requirements

This guide is tested on devices which meet the following requirements:

- x86 or amd64 processor
- Ubuntu 22.04 LTS Server or Debian 12 Bookworm
- Root access to the operating system
- Network connectivity

## 📎 Quick Links:

### Host Creation

"Host Creation" in this context is the process of installing an operating system onto a piece of
physical hardware, or the creation and configuration of virtualised compute resources.

- [Bare-Metal](./03-host-creation/02-bare-metal.mdx)
- [Virtual Machines using Multipass](./03-host-creation/02-multipass.mdx)
- [Virtual Machines using QEMU](./03-host-creation/03-QEMU/01-overview.mdx)

### Host Provisioning

"Provisioning" here refers to the process of installing additional resources onto, and the
configuration of your host beyond installing the base operating-system. Both manual and declarative
workflows are supported.

- [Manual Ubuntu 22.04 Setup](./04-host-provisioning/02-ubuntu-setup.mdx)
- [Manual Debian 12 Setup](./04-host-provisioning/01-debian-setup.mdx)
- [Declarative provisioning via Cloud-Init](./04-host-provisioning/03-cloud-init/01-about.mdx)

### Runner Application Installation

Once your host has been provisioned, you will then need to install the appropriate runner
application. The guides below will walk you through that process.

- [Github Actions](./05-runner-application-installation/02-github-actions.mdx)
- [GitLab Pipelines](./05-runner-application-installation/01-gitlab-pipelines.mdx)
129 changes: 129 additions & 0 deletions docs/12-self-hosting/02-host-types.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
import Virtualisation from '/assets/images/Virtualization.drawio.png';
import Metal from '/assets/images/Metal.drawio.png';
import Docker from '/assets/images/DockerHost.drawio.png';
import Kubernetes from '/assets/images/kubernetes.drawio.png';
import Layers from '/assets/images/k8s-layers.drawio.png';
import Layer0 from '/assets/images/k8s-layer0.drawio.png';
import Layer1 from '/assets/images/k8s-layer1.drawio.png';
import Layer2 from '/assets/images/k8s-layer2.drawio.png';
import Layers01 from '/assets/images/k8s-layers01.drawio.png';
import Layers012 from '/assets/images/k8s-layers012.drawio.png';

# Types of Hosts

## Bare-Metal

"Bare Metal" means that your host OS is running directly on a piece of hardware without any
virtualisation. This reduces the complexity of deployment at the cost of increased time and effort
for re-provisioning the host.

<div style={{ textAlign: 'center' }}>
<a target="\_blank" href={require('/assets/images/Metal.drawio.png').default}>
<img src={Metal} width="500" />
</a>
</div>
<br />

## Virtual Machines

Virtual Machines are a software-defined layer of abstraction atop a Bare-Metal host which makes
deployments more consistent and easier to manage declaratively. This greatly reduces the difficulty
of re-deployment and creates the conditions required for securely running multiple guests within the
same physical host. Virtual Machines can also be used to create hosts that run different operating
systems (Windows, MacOS) or architectures (ARM) than the host machine. This added functionality
comes at the cost of added complexity, a slight performance penalty, and you need to already have a
Bare-Metal host on which to run the VMs.

<div style={{ textAlign: 'center' }}>
<a target="\_blank" href={require('/assets/images/Virtualization.drawio.png').default}>
<img src={Virtualisation} width="500" />
</a>
</div>
<br />

Additional Reading:

- [A Study of Performance and Security Across the Virtualization Spectrum](https://repository.tudelft.nl/islandora/object/uuid:34b3732e-2960-4374-94a2-1c1b3f3c4bd5/datastream/OBJ/download) -
Vincent van Rijn
- [Hyper-converged infrastructure](https://en.wikipedia.org/wiki/Hyper-converged_infrastructure) -
Wikipedia
- [Rethinking the PC](https://www.computerworld.com/article/3518849/rethinking-the-pc-why-virtual-machines-should-replace-operating-systems.html) -
Rob Enderle

## Containers

Containers are built on 'cgroups' (control groups), which are a feature of the Linux kernel that
limits monitors, and isolates the resource usage of a collection of processes. This means that
running containers on Linux is very lightweight form of virtualisation. However, on other operating
systems which do not use the Linux kernel, a Linux virtual machine or translation-layer must be
created to run containers. The manner by which each Operating System resolves this issue varies
greatly as shown below. Because of this variance, the self-hosting documentation targets Linux as a
means of avoiding excess complexity.

<div style={{ textAlign: 'center' }}>
<a target="\_blank" href={require('/assets/images/DockerHost.drawio.png').default}>
<img src={Docker} width="500" />
</a>
</div>
<br />

Additional Reading:

- [The Mental Model Of Docker Container Shipping](https://bernhardwenzel.com/2022/the-mental-model-of-docker-container-shipping/) -
Bernhard Wenzel
- [Why is Docker-in-Docker considered bad?](https://devops.stackexchange.com/questions/676/why-is-docker-in-docker-considered-bad)
- [Why it is recommended to run only one process in a container?](https://devops.stackexchange.com/questions/447/why-it-is-recommended-to-run-only-one-process-in-a-container)

## Kubernetes (Cloud Runner)

Kubernetes is somewhat of a combination of all other host types. Since it is an API, it must be
installed on an existing host (called a "Node") which is usually either a VM or physical device. A
Kubernetes "Cluster" is usually made up of 3 or more nodes - though you can have as few as one, or
as many 5,000 per cluster.

<br />
<div style={{ textAlign: 'center' }}>
<a target="\_blank" href={require('/assets/images/k8s-layer0.drawio.png').default}>
<img src={Layer0} width="700" />
</a>
<br />
</div>
<br />

Once installed, Kubernetes creates
[standardised interfaces](https://matt-rickard.com/kubernetes-interfaces) to control the hardware &
software components of the underlying nodes (networking, storage, GPUs, CPU cores etc...) as well as
a distributed key-value store which facilitates communication between all nodes in the cluster.

<br />
<div style={{ textAlign: 'center' }}>
<a target="\_blank" href={require('/assets/images/k8s-layers01.drawio.png').default}>
<img src={Layers01} width="750" />
</a>
<br />
</div>
<br />

With the underlying hardware abstracted into a generic pool of resources, Kubernetes is then able to
re-compose those assets into isolated environments called "Namespaces" where it deploys
containerised workloads in groups called "Pods". This layer of Kubernetes is very similar to a
typical container host but with many more features for multi-tenancy, security, and life-cycle
management.

<br />
<div style={{ textAlign: 'center' }}>
<a target="\_blank" href={require('/assets/images/k8s-layers012.drawio.png').default}>
<img src={Layers012} width="800" />
</a>
</div>
<br />

Additional Reading:

- [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) - kubernetes.io
- [A visual guide to Kubernetes networking fundamentals](https://opensource.com/article/22/6/kubernetes-networking-fundamentals) -
Nived Velayudhan
- [Thinking about the complexity of the Kubernetes ecosystem](https://erkanerol.github.io/post/complexity-of-kubernetes/) -
Erkan Erol
- [Ephemeral, Idempotent and Immutable Infrastructure ](https://cloudnativenow.com/topics/ephemeral-idempotent-and-immutable-infrastructure/) -
Marc Hornbeek
41 changes: 41 additions & 0 deletions docs/12-self-hosting/03-host-creation/02-bare-metal.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import Metal from '/assets/images/Metal.drawio.png';

# Bare-Metal

The Host is the computer that will execute the runner application. This can be a desktop computer,
laptop, Virtual Machine, or VPS from a cloud provider.

<div style={{ textAlign: 'center' }}>
<a target="\_blank" href={require('/assets/images/Metal.drawio.png').default}>
<img src={Metal} width="500" />
</a>
</div>
<br />

## If your host is a local machine:

For a local machine you will need to perform a clean installation of the operating system. This
means creating a bootable USB drive from an ISO file, booting the machine from the USB drive, and
installing the OS. Links to download an official Live ISO file and installation guides are provided
below. If you would like to create a custom ISO, try [PXEless](https://github.com/cloudymax/pxeless)
or [Cubic](https://github.com/PJ-Singh-001/Cubic).

### Ubuntu

- Download the Ubuntu 22.04 LTS
[server installer](https://ftp.snt.utwente.nl/pub/os/linux/ubuntu-releases/22.04.3/ubuntu-22.04.3-live-server-amd64.iso)
- [Guide: Install Ubuntu 22.04 LTS on a local machine](https://ostechnix.com/install-ubuntu-server/)

### Debian

- Download the Debian 12
[installation image](https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.1.0-amd64-DVD-1.iso)
- [Guide: Install Debian on a local system](https://www.linuxtechi.com/how-to-install-debian-11-bullseye/)

## If your host is a virtual-machine:

If you are using a VPS or VM, the OS should already be installed and admin user should already
exist. Follow the appropriate guide in the provisioning section for your operating-system.

- [Ubuntu 22.04](../04-host-provisioning/02-ubuntu-setup.mdx)
- [Debian 12](../04-host-provisioning/01-debian-setup.mdx)
Loading

0 comments on commit cd8711e

Please sign in to comment.