Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add terraform for oci oke #3

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 28 additions & 38 deletions README.org
Original file line number Diff line number Diff line change
Expand Up @@ -4,76 +4,66 @@
a Flux + Terraform infrastructure repo
#+end_quote

* Todo

- [x] add Flux for cluster repo
- [x] get talosconfig
- [ ] access Kubernetes APIServer
- [ ] verify Ceph disk allocation

* Prerequisites

Install OpenTofu

#+begin_src tmux :session ":tofu "
brew install opentofu
brew install opentofu oci-cli
#+end_src

#+RESULTS:

Create a .tfvars file

#+begin_src hcl
tenancy_ocid = "TENANCY OCID : https://cloud.oracle.com/tenancy"
user_ocid = "YOUR USER OCID : https://cloud.oracle.com/identity/domains/my-profile"
private_key_path = "YOUR PRIVATE KEY PATH : https://cloud.oracle.com/identity/domains/my-profile/api-keys"
fingerprint = "THE FINGERPRINT FOR YOUR PRIVATE KEY : ^^"
region = "us-sanjose-1"
compartment_ocid = "YOUR COMPARTMENT OCID : https://cloud.oracle.com/identity/compartments # cloudnative.coop"
#+end_src

Log into ~oci-cli~, using the links provided in the hcl file above to get keys and other data.

#+begin_src shell
oci setup bootstrap
#+end_src

A kubeconfig must be at =~/.kube/config-fop= for using shared tfstate.

* Usage

You'll need .tfvar files, currently we create those via org files.
init

#+begin_src tmux :session ":tofu"
cd ~/infra
mkdir -p ./tmp/
touch ./tmp/cluster-kubeconfig
tofu init -var-file=./config.tfvars -var-file=./secrets.tfvars -var github_token="$(gh auth token)" --upgrade
#+begin_src shell
tofu init --var-file=./.tfvars
#+end_src


plan

#+begin_src tmux :session ":tofu"
tofu plan -var-file=./config.tfvars -var-file=./secrets.tfvars -var github_token="$(gh auth token)"
tofu plan --var-file=./.tfvars
#+end_src

apply

#+begin_src tmux :session ":tofu"
tofu apply -var-file=./config.tfvars -var-file=./secrets.tfvars -var github_token="$(gh auth token)"
#+end_src

get talosconfig

#+begin_src tmux :session ":talos"
mkdir -p ~/.talos
tofu output -raw talosconfig > ~/.talos/config
tofu apply --var-file=./.tfvars
#+end_src

get kubeconfig

#+begin_src tmux
mkdir -p ~/.kube
tofu output -raw kubeconfig > ~/.kube/config
#+end_src

* Flux usage

force a reconciliation

#+begin_src tmux
CLUSTER_NAME=cloudnative-coop
flux --kubeconfig ~/.kube/config-"$CLUSTER_NAME" reconcile source git flux-system
tofu output -raw cluster-sharingio-oke-kubeconfig > ~/.kube/config
#+end_src

* Force tear down

#+begin_src tmux :session ":tofu"
tofu state list | grep -E 'talos|flux|manifests|kubernetes_manifest' | xargs -I{} tofu state rm {}
tofu destroy -var-file=./config.tfvars -var-file=./secrets.tfvars -var github_token="$(gh auth token)"
tofu destroy -var-file=./.tfvars
#+end_src

* Notes

- Equinix Metal Cloud Provider 401 error regarding IP allocation and assigning
20 changes: 0 additions & 20 deletions authentik.tf
Original file line number Diff line number Diff line change
@@ -1,21 +1 @@
module "cluster-authentik-config" {
source = "./terraform/authentik-config"
github_oauth_app_id = var.authentik_github_oauth_app_id
github_oauth_app_secret = var.authentik_github_oauth_app_secret
authentik_coder_oidc_client_id = module.cluster-manifests.authentik_coder_oidc_client_id
authentik_coder_oidc_client_secret = module.cluster-manifests.authentik_coder_oidc_client_secret
authentik_bootstrap_token = module.cluster-manifests.authentik_bootstrap_token
domain = var.domain
# repo = var.github_repository
# # repo = "${var.github_org}/${var.github_repository}"
# domain = "${var.domain}"
# secret = module.cluster-manifests.flux_receiver_token

providers = {
authentik = authentik
flux = flux
kubernetes = kubernetes.cluster
}

depends_on = [module.cluster-manifests]
}
22 changes: 1 addition & 21 deletions coder_dns.tf
Original file line number Diff line number Diff line change
@@ -1,21 +1 @@
resource "powerdns_zone" "coder" {
name = "coder.${var.domain}."
kind = "Native"
nameservers = ["ns1.sharing.io.", "ns2.sharing.io."]
}
resource "powerdns_record" "coder-A" {
zone = "coder.${var.domain}."
name = "coder.${var.domain}."
type = "A"
ttl = 300
records = [module.cluster.cluster_ingress_ip]
depends_on = [powerdns_zone.coder]
}
resource "powerdns_record" "coder-WILDCARD" {
zone = "coder.${var.domain}."
name = "*.coder.${var.domain}."
type = "A"
ttl = 300
records = [module.cluster.cluster_ingress_ip]
depends_on = [powerdns_zone.coder]
}

97 changes: 8 additions & 89 deletions main.tf
Original file line number Diff line number Diff line change
@@ -1,94 +1,13 @@
module "cluster" {
source = "./terraform/equinix-metal-talos-cluster"

talos_version = var.talos_version
kubernetes_version = var.kubernetes_version
kubernetes_apiserver_fqdn = "k8s.${var.domain}"
controlplane_nodes = var.kubernetes_control_plane_nodes
cluster_name = var.github_org
domain = var.domain
equinix_metal_project_id = var.equinix_metal_project_id
equinix_metal_metro = var.equinix_metal_metro
equinix_metal_auth_token = var.equinix_metal_auth_token
equinix_metal_plan = var.equinix_metal_plan
talos_install_disk = var.talos_install_disk
longhorn_disk = var.longhorn_disk
module "cluster-sharingio-oke" {
source = "./terraform/oci-oke-cluster"

providers = {
talos = talos
helm = helm
equinix = equinix
dns = dns
http = http
oci = oci
}
}
resource "local_sensitive_file" "cluster-kubeconfig" {
content = module.cluster.kubeconfig.kubeconfig_raw
filename = "./tmp/cluster-kubeconfig"

lifecycle {
ignore_changes = all
}
}
module "cluster-manifests" {
source = "./terraform/manifests"

equinix_metal_project_id = var.equinix_metal_project_id
equinix_metal_metro = var.equinix_metal_metro
equinix_metal_auth_token = var.equinix_metal_auth_token
ingress_ip = module.cluster.cluster_ingress_ip
dns_ip = module.cluster.cluster_dns_ip
wg_ip = module.cluster.cluster_wireguard_ip
acme_email_address = var.acme_email_address
rfc2136_nameserver = var.rfc2136_nameserver
rfc2136_tsig_keyname = var.rfc2136_tsig_keyname
rfc2136_tsig_algorithm = var.rfc2136_tsig_algorithm
rfc2136_tsig_key = var.rfc2136_tsig_key
domain = var.domain
pdns_host = var.pdns_host
pdns_api_key = var.pdns_api_key
# for coder to directly authenticate via github
coder_version = var.coder_version
coder_oauth2_github_client_id = var.coder_oauth2_github_client_id
coder_oauth2_github_client_secret = var.coder_oauth2_github_client_secret
# for coder to create gh tokens for rw within workspaces
coder_gitauth_0_client_id = var.coder_gitauth_0_client_id
coder_gitauth_0_client_secret = var.coder_gitauth_0_client_secret
providers = {
kubernetes = kubernetes.cluster
random = random
}
authentik_version = var.authentik_version
depends_on = [local_sensitive_file.cluster-kubeconfig, module.cluster]
}

module "cluster-flux-bootstrap" {
source = "./terraform/flux-bootstrap"

github_org = var.github_org
github_repository = var.github_repository
kubeconfig = module.cluster.kubeconfig.kubeconfig_raw

providers = {
github = github
flux = flux.cluster
}
depends_on = [local_sensitive_file.cluster-kubeconfig, module.cluster-manifests]
compartment_ocid = var.compartment_ocid
tenancy_ocid = var.tenancy_ocid
user_ocid = var.user_ocid
private_key_path = var.private_key_path
fingerprint = var.fingerprint
}

# module "cluster-flux-github-webhook" {
# source = "./terraform/flux-github-webhook"

# repo = var.github_repository
# # repo = "${var.github_org}/${var.github_repository}"
# domain = var.domain
# secret = module.cluster-manifests.flux_receiver_token

# providers = {
# github = github
# kubernetes = kubernetes.cluster
# }

# depends_on = [local_sensitive_file.cluster-kubeconfig, module.cluster-manifests, module.cluster-flux-bootstrap]
# }

34 changes: 2 additions & 32 deletions outputs.tf
Original file line number Diff line number Diff line change
@@ -1,34 +1,4 @@
output "talosconfig" {
value = module.cluster.talosconfig
sensitive = true
}

output "kubeconfig" {
value = module.cluster.kubeconfig.kubeconfig_raw
sensitive = true
}

output "akadmin-password" {
value = module.cluster-manifests.authentik_bootstrap_password
sensitive = true
}

output "akadmin-token" {
value = module.cluster-manifests.authentik_bootstrap_token
sensitive = true
}

output "cluster-apiserver-ip" {
value = module.cluster.cluster_apiserver_ip
}

output "cluster-ingress-ip" {
value = module.cluster.cluster_ingress_ip
}
output "coder_admin_email" {
value = module.cluster-manifests.coder_admin_email
}
output "coder_admin_password" {
value = module.cluster-manifests.coder_admin_password
output "cluster-sharingio-oke-kubeconfig" {
value = module.cluster-sharingio-oke.kubeconfig
sensitive = true
}
Loading