Merge pull request #3 from futurice/extract-aws-reverse-proxy-module

Extract aws_reverse_proxy module

aws_domain_redirect/README.md

@@ -1,25 +1,39 @@
 # aws_domain_redirect
 
-Creates the necessary resources on AWS to implement an HTTP redirect from a domain (e.g. `redir.example.com`) to a given URL (e.g. `https://www.futurice.com/careers/women-who-code-helsinki`). Useful for creating human-friendly shortcuts for deeper links into a site, or for dynamic links (e.g. `download.example.com` always pointing to your latest release).
+This module implements a domain that redirects clients to another URL. Useful for creating human-friendly shortcuts for deeper links into a site, or for dynamic links (e.g. `download.example.com` always pointing to your latest release).
 
-Implementing this on AWS actually requires quite a few resources:
+Main features:
 
-- DNS records on [Route 53](https://aws.amazon.com/route53/)
-- A [CloudFront](https://aws.amazon.com/cloudfront/) distribution for SSL termination
-- An SSL certificate for the distribution from [ACM](https://aws.amazon.com/certificate-manager/)
-- A [Lambda@Edge](https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.html) function that implements the redirect itself
+- DNS entries are created automatically
+- HTTPS enabled by default
+- HTTP Strict Transport Security supported
 
-Luckily, this module encapsulates this configuration quite neatly.
+Optional features:
 
-The Lambda function also adds [HSTS](https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security) headers to prevent [man-in-the-middle attacks](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security#An_example_scenario).
+- Plain HTTP instead of HTTPS
+- Sending a permanent redirect (`301 Moved Permanently`) instead of default (`302 Found`)
+
+Resources used:
+
+- Route53 for DNS entries
+- ACM for SSL certificates
+- CloudFront for proxying requests
+- Lambda@Edge for transforming requests
+- IAM for permissions
+
+## About CloudFront operations
+
+This module manages CloudFront distributions, and these operations are generally very slow. Your `terraform apply` may take anywhere **from 10 minutes up to 45 minutes** to complete. Be patient: if they start successfully, they almost always finish successfully, it just takes a while.
+
+Additionally, this module uses Lambda@Edge functions with CloudFront. Because Lambda@Edge functions are replicated, [they can't be deleted immediately](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-delete-replicas.html). This means a `terraform destroy` won't successfully remove all resources on its first run. It should complete successfully when running it again after a few hours, however.
 
 ## Example
 
 Assuming you have the [AWS provider](https://www.terraform.io/docs/providers/aws/index.html) set up, and a DNS zone for `example.com` configured on Route 53:
 
 ```tf
-# Several AWS services (such as ACM & Lambda@Edge) are presently only available in the US East region.
-# To be able to use them, we need a separate AWS provider for that region, which can be used with an alias.
+# Lambda@Edge and ACM, when used with CloudFront, need to be used in the US East region.
+# Thus, we need a separate AWS provider for that region, which can be used with an alias.
 # Make sure you customize this block to match your regular AWS provider configuration.
 # https://www.terraform.io/docs/configuration/providers.html#multiple-provider-instances
 provider "aws" {

aws_domain_redirect/main.tf

@@ -1,13 +1,14 @@
-module "aws_static_site" {
-  # Available inputs: https://github.com/futurice/terraform-utils/tree/master/aws_static_site#inputs
-  # Check for updates: https://github.com/futurice/terraform-utils/compare/v9.4...master
-  source = "git::ssh://[email protected]/futurice/terraform-utils.git//aws_static_site?ref=v9.4"
+module "aws_reverse_proxy" {
+  # Available inputs: https://github.com/futurice/terraform-utils/tree/master/aws_reverse_proxy#inputs
+  # Check for updates: https://github.com/futurice/terraform-utils/compare/v11.0...master
+  source = "git::ssh://[email protected]/futurice/terraform-utils.git//aws_reverse_proxy?ref=v11.0"
 
+  origin_url             = "http://example.com/"           # note that this is just a dummy value to satisfy CloudFront, it won't ever be used with the override_* variables in place
   site_domain            = "${var.redirect_domain}"
   name_prefix            = "${var.name_prefix}"
   comment_prefix         = "${var.comment_prefix}"
-  bucket_override_name   = "-"                             # providing this ensures an S3 bucket isn't unnecessarily created, even if this isn't a valid bucket name
-  price_class            = "${var.redirect_price_class}"
+  cloudfront_price_class = "${var.cloudfront_price_class}"
+  viewer_https_only      = "${var.viewer_https_only}"
   lambda_logging_enabled = "${var.lambda_logging_enabled}"
   tags                   = "${var.tags}"
 

aws_domain_redirect/variables.tf

@@ -16,11 +16,16 @@ variable "comment_prefix" {
   default     = "Domain redirect: "
 }
 
-variable "redirect_price_class" {
+variable "cloudfront_price_class" {
   description = "Price class to use (`100`, `200` or `\"All\"`, see https://aws.amazon.com/cloudfront/pricing/)"
   default     = 100
 }
 
+variable "viewer_https_only" {
+  description = "Set this to `false` if you need to support insecure HTTP access for clients, in addition to HTTPS"
+  default     = true
+}
+
 variable "redirect_permanently" {
   description = "Which HTTP status code to use for the redirect; if `true`, uses `301 Moved Permanently`, instead of `302 Found`"
   default     = false

aws_reverse_proxy/README.md

@@ -0,0 +1,108 @@
+# aws_reverse_proxy
+
+This module implements a website that proxies content from another server.
+
+Main features:
+
+- DNS entries are created automatically
+- HTTPS enabled by default
+- HTTP Strict Transport Security supported
+
+Optional features:
+
+- HTTP Basic Auth
+- Plain HTTP instead of HTTPS
+- Cache TTL overrides
+- Custom response headers sent to clients
+- Custom request headers sent to origin server
+- Static response status/body override
+
+Resources used:
+
+- Route53 for DNS entries
+- ACM for SSL certificates
+- CloudFront for proxying requests
+- Lambda@Edge for transforming requests
+- IAM for permissions
+
+## About CloudFront operations
+
+This module manages CloudFront distributions, and these operations are generally very slow. Your `terraform apply` may take anywhere **from 10 minutes up to 45 minutes** to complete. Be patient: if they start successfully, they almost always finish successfully, it just takes a while.
+
+Additionally, this module uses Lambda@Edge functions with CloudFront. Because Lambda@Edge functions are replicated, [they can't be deleted immediately](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-delete-replicas.html). This means a `terraform destroy` won't successfully remove all resources on its first run. It should complete successfully when running it again after a few hours, however.
+
+## Examples
+
+Some common use cases for this module are:
+
+- [Static website hosting with S3](../aws_static_site)
+- [Redirecting clients from a domain to another URL](../aws_domain_redirect)
+- SSL termination in front of a server/load balancer elsewhere on AWS
+
+## How CloudFront caching works
+
+It's important to understand that CloudFront, by default, **respects cache headers given by the origin**, that is, the server it's proxying requests to.
+
+### Default cache behaviour
+
+Consider an origin server that doesn't give any `Cache-Control` headers. Any changes you make to its responses **will be reflected immediately** on the CloudFront distribution. That's is because this module will **by default** not cache such objects at all. This is a sensible default, because the AWS default TTL for CloudFront is 24 hours, and for an origin that doesn't explicitly send `Cache-Control` headers, it's rarely the desired behaviour: your site will be serving stale content for up to 24 hours. Users will be sad, and engineers will be yelled at.
+
+Having immediate updates on CloudFront is convenient, but the downside is that every request for every file will be forwarded to your origin, to make sure the CloudFront cache still has the latest version. This can increase request latency for users, and infrastructure costs for you.
+
+### Specifying cache lifetimes on the origin
+
+Let's say we're serving static files from an S3 bucket. Using the official [AWS CLI](https://aws.amazon.com/cli/), you can specify cache lifetimes as your objects are uploaded:
+
+```bash
+aws s3 cp --cache-control=no-store,must-revalidate index.html "s3://my-bucket/"
+aws s3 cp --cache-control=max-age=31536000 static/image-v123.jpg "s3://my-bucket/"
+```
+
+This will upload `index.html` so that CloudFront will **never** serve its content to a user, without first checking that it's not been updated on S3. However, `image-v123.jpg` will be uploaded with cache headers that allow CloudFront to keep its copy for that object **forever** (well, technically 1 year, which is the maximum recommended value for `max-age`; in practice CloudFront will probably evict it before that for other reasons).
+
+The above is a good middle ground caching strategy, for when you want immediate updates for your HTML documents (e.g. `index.html`), but static assets (e.g. `image-v123.jpg`) can be cached for much longer. This means that for the HTML document itself, you won't get any boost from CloudFront, but as the browser starts downloading the various linked static assets, they can be served directly from the CloudFront edge location, which should be much closer to the user, geographically. When you need to update the linked image, instead of updating `image-v123.jpg`, you should instead upload `image-v124.jpg`, and update any links in `index.html` to point to the new version. This ensures that:
+
+1. Users will see the new document (including its updated images) immediately
+1. Users won't see an inconsistent version of the document, where the document content is updated, but it's still showing the old images
+
+### Overriding cache lifetimes on CloudFront
+
+If your origin server doesn't give out sensible cache control headers, or you're just feeling lazy, this module supports overriding cache behaviour on CloudFront, effectively ignoring anything your origin says about caching objects.
+
+That is, if you specify `cache_ttl_override = 0` for your site, every object will always be fetched from the origin, for every request. Importantly, though, this won't invalidate objects that *are already* in the CloudFront cache with a longer TTL. If you have an object that's "stuck" in your cache and you can't shake it, the CloudFront feature you're looking for is [file invalidation](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html).
+
+Conversely, if you specify `cache_ttl_override = 300`, every object will stay in CloudFront for 5 minutes, regardless of its cache headers. This can be a good performance boost for your site, since only 1 request per file per 5 minutes will need to go all the way to the origin, and all the others can be served immediately from the CloudFront edge location. Keep in mind the aforementioned warning about "inconsistent versions", however: each object has their own TTL counter, so `index.html` and `image.jpg` may update at different times in the cache, even if you update content at your origin at the same time.
+
+<!-- terraform-docs:begin -->
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|:----:|:-----:|:-----:|
+| add_response_headers | Map of HTTP headers (if any) to add to outgoing responses before sending them to clients | map | `<map>` | no |
+| basic_auth_body | When using HTTP Basic Auth, and authentication has failed, this will be displayed by the browser as the page content | string | `"Unauthorized"` | no |
+| basic_auth_password | When non-empty, require this password with HTTP Basic Auth | string | `""` | no |
+| basic_auth_realm | When using HTTP Basic Auth, this will be displayed by the browser in the auth prompt | string | `"Authentication Required"` | no |
+| basic_auth_username | When non-empty, require this username with HTTP Basic Auth | string | `""` | no |
+| bucket_override_name | When provided, assume a bucket with this name already exists for the site content, instead of creating the bucket automatically (e.g. `"my-bucket"`) | string | `""` | no |
+| cache_ttl_override | When >= 0, override the cache behaviour for ALL objects in S3, so that they stay in the CloudFront cache for this amount of seconds | string | `"-1"` | no |
+| comment_prefix | This will be included in comments for resources that are created | string | `"Static site: "` | no |
+| default_root_object | The object to return when the root URL is requested | string | `"index.html"` | no |
+| https_only | Set this to `false` if you want to support insecure HTTP access, in addition to HTTPS | string | `"true"` | no |
+| lambda_logging_enabled | When true, writes information about incoming requests to the Lambda function's CloudWatch group | string | `"false"` | no |
+| name_prefix | Name prefix to use for objects that need to be created (only lowercase alphanumeric characters and hyphens allowed, for S3 bucket name compatibility) | string | `"aws-static-site---"` | no |
+| override_response_body | Same as `override_response_status` | string | `""` | no |
+| override_response_status | When this and the other `override_response_*` variables are non-empty, skip sending the request to the origin altogether, and instead respond as instructed here | string | `""` | no |
+| override_response_status_description | Same as `override_response_status` | string | `""` | no |
+| price_class | CloudFront price class to use (`100`, `200` or `"All"`, see https://aws.amazon.com/cloudfront/pricing/) | string | `"100"` | no |
+| site_domain | Domain on which the static site will be made available (e.g. `"www.example.com"`) | string | n/a | yes |
+| tags | AWS Tags to add to all resources created (where possible); see https://aws.amazon.com/answers/account-management/aws-tagging-strategies/ | map | `<map>` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| bucket_domain_name | Full S3 domain name for the bucket used for hosting the content (e.g. `"aws-static-site---hello-example-com.s3-website.eu-central-1.amazonaws.com"`) |
+| bucket_name | The name of the S3 bucket that's used for hosting the content (either auto-generated or externally provided) |
+| cloudfront_id | The ID of the CloudFront distribution that's used for hosting the content |
+| site_domain | Domain on which the static site will be made available |
+<!-- terraform-docs:end -->