cloud-platform-terraform-ingress-controller

Terraform module that deploys cloud-platform ingress controllers among another resources (like certificates)

This module is also responsilbe for our WAF. It is provided by modsec. Although we have a cluster wide set of fluent-bit containers which collect and ship our logs to open search/ elastic search. We can't rely on that to collect modsec audit logs which are written to file. We need to write these logs to file because when we push them directly to stdout we lose logs. This is due to the scale of traffic in our live cluster. We increase log reliability by writing to file.

This means we need to ship modsec audit logs separtely as the cluster level fluent-bit cannot access internal container files. So we introduced a fluent-bit side car which has the filesystem mounted and accessible. We have one further sidecar mounted to handle log rotation using logrotate, this prevents our logs filling up our master node file space and causing node issues.

Usage

See example dir

Requirements

Name	Version
terraform	>= 1.2.5
helm	>=2.6.0
kubectl	2.0.4
kubernetes	>=2.12.1

Providers

Name	Version
helm	>=2.6.0
kubectl	2.0.4
kubernetes	>=2.12.1

Modules

No modules.

Resources

Name	Type
helm_release.nginx_ingress	resource
kubectl_manifest.nginx_ingress_default_certificate	resource
kubectl_manifest.prometheus_rule_alert	resource
kubernetes_config_map.fluent-bit-config	resource
kubernetes_config_map.fluent_bit_lua_script	resource
kubernetes_config_map.logrotate_config	resource
kubernetes_config_map.modsecurity_nginx_config	resource
kubernetes_namespace.ingress_controllers	resource

Inputs

Name	Description	Type	Default	Required
backend_repo	repository for the default backend app	string	"ministryofjustice/cloud-platform-custom-error-pages"	no
backend_tag	tag of the default backend app	string	"1.1.5"	no
cluster	cluster name used for opensearch indicies	string	""	no
cluster_domain_name	The cluster domain used for externalDNS annotations and certmanager	any	n/a	yes
controller_name	Will be used as the ingress controller name and the class annotation	string	n/a	yes
default_cert	Useful if you want to use a default certificate for your ingress controller. Format: namespace/secretName	string	"ingress-controllers/default-certificate"	no
enable_cross_zone_lb	cross-zone load balancing distributes traffic across the registered targets in all enabled Availability Zones	bool	true	no
enable_external_dns_annotation	Add external dns annotation for service	bool	false	no
enable_latest_tls	Provide support to tlsv1.3 along with tlsv1.2	bool	false	no
enable_modsec	Enable https://github.com/SpiderLabs/ModSecurity-nginx	bool	false	no
enable_owasp	Use default ruleset from https://github.com/SpiderLabs/owasp-modsecurity-crs/	bool	false	no
fluent_bit_version	fluent bit container version used to exrtact modsec audit logs	string	"3.0.2-amd64"	no
is_live_cluster	For live clusters externalDNS annotation will have var.live_domain (default *.cloud-platform.service.justice.gov.uk)	bool	false	no
keepalive	the maximum number of idle keepalive connections to upstream servers that are preserved in the cache of each worker process. When this number is exceeded, the least recently used connections are closed. https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive	number	320	no
live1_cert_dns_name	This is to add the live-1 dns name for eks-live cluster default certificate	string	""	no
live_domain	The live domain used for externalDNS annotation	string	"cloud-platform.service.justice.gov.uk"	no
memory_limits	value for resources:limits memory value	string	"2Gi"	no
memory_requests	value for resources:requests memory value	string	"512Mi"	no
opensearch_modsec_audit_host	domain endpoint for the opensearch cluster	string	""	no
proxy_response_buffering	nginx receives a response from the proxied server as soon as possible, saving it into the buffers set by the proxy_buffer_size and proxy_buffers directives. If the whole response does not fit into memory, a part of it can be saved to a temporary file on the disk. https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering	string	"off"	no
replica_count	Number of replicas set in deployment	string	n/a	yes
upstream_keepalive_time	Limits the maximum time during which requests can be processed through one keepalive connection. After this time is reached, the connection is closed following the subsequent request processing.	string	"1h"	no

Outputs

Name	Description
helm_nginx_ingress_status	n/a