Skip to content

Commit

Permalink
Move most docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@Mosnar
Mosnar committed May 4, 2020
1 parent 46b486d commit d76822c
Show file tree
Hide file tree
Showing 9 changed files with 314 additions and 177 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## Unreleased

### Added
- Started new docs!
- Add `SLS_VALID_PATH_REGEX` setting to explicitly control what paths are allowed (Thanks @bs-thomas) (#52)
- Add support for deploying to custom region (#61)
- Added support for SSL & custom domain (Thanks @bs-thomas) (#47)
Expand Down
132 changes: 5 additions & 127 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
# Serverless Sharp Image Processor
A solution to dynamically optimize and transform images on the fly, utilizing [Sharp](https://sharp.pixelplumbing.com/en/stable/) and AWS Lambda.

[Documentation](https://venveo.github.io/serverless-sharp/index.html)

Brought to you by [Venveo](https://www.venveo.com)

## Who is this for?
This software is for people who want to optimize and run basic transformations (crop, scale, convert, etc) on images from an existing S3
bucket without running computationally expensive processes or servers or paying for expensive third-party services.

Serverless Sharp is written to be a drop-in replacement for most essential features of Imgix and costs magnitudes less for
Serverless Sharp is a drop-in replacement for most essential features of Imgix and costs magnitudes less for
most users.

## How does it work?
Expand All @@ -18,136 +22,10 @@ than monthly cost). The most important of which are:
Once deployed, a Cloudfront CDN distribution is generated that is directed to the generated API Gateway. This distribution
ensures the Lambda function does not get run multiple times for the same image request.

## Configuration & Environment Variables
Make a copy of `settings.example.yml` and populate accordingly.

- `SOURCE_BUCKET` An S3 bucket in your account where your images are stored - you can include a path here if you like.
For example: `mybucket/images`
- `SERVERLESS_PORT` For local development, this controls what port the Serverless service runs on
- `SECURITY_KEY` See security section
- `SLS_VALID_PATH_REGEX` Regular expression for path validation - see Security section
- `SLS_IGNORE` A comma-delineated string of paths that should be ignored (for example, `favicon.ico`)
- `CUSTOM_DOMAIN` See CUSTOM_DOMAIN section
- `DEFAULT_CACHE_CONTROL` Sets the Cache-Control header on objects that don't have one set

You can define multiple environments, each of which will inherit your default settings. This is useful if you have
different buckets for staging & production, for example. You may also wish to create an environment without a sign-key
for local development.

## API & Usage
We chose to base our API around the [Imgix service](https://docs.imgix.com/apis/url) to allow for backwards compatibility
with the already popular service. The idea is that all CMS plugins should be able to seamlessly use this service in-place of
an Imgix URL. We've only implemented a hand-full of the features Imgix offers; however, the one's we've
implemented should cover most use-cases.

The benefits of using this method over other methods (such as hashing the entire URL payload in base64) are:
- Much more intuitive
- Easier to develop & debug
- Provides clear prefix matching your original object's path with which you can create invalidations with wildcards

You may access images in your `SOURCE_BUCKET` via the Cloudfront URL that is generated for your distribution just like
normal images. Transforms can be appended to the filename as query parameters. The following query parameters are
supported:
- **fm** - output format - can be one of: `webp`, `png`, `jpeg`, `tiff`
- **w** - width - Scales image to supplied width while maintaining aspect ratio
- **h** - height - Scales image to supplied height while maintaining aspect ratio

*If both width and height are supplied, the aspect ratio will be preserved and scaled to minimum of either width/height*

- **q** - quality (75) - 1-100
- **ar** - aspect-ratio (1.0:1.0) - When fit=crop, an aspect ratio such as 16:9 can be supplied, optionally with a height or width. If neither height or width are defined, the original image size will be used
- **dpr** - device-pixel-ratio (1) - scales requested image dimensions by this multiplier.
- **fit** - resize fitting mode - can be one of: `fill`, `scale`, `crop`, `clip`, `min`, `max`
- **fill-color** - used when `fit` is set to `fill` can be a loosely formatted color such as "red" or "rgb(255,0,0)"
- **crop** - resize fitting mode - can be one of: `focalpoint`, `entropy`, any comma separated combination of `top`, `bottom`, `left` `right`
- **fp-x**, **fp-y** - focal point x & y - percentage, 0 to 1 for where to focus on the image when cropping with focalpoint mode
- **s** - security hash - See security section
- **auto** - can be a comma separated combination of: `compress`, `format`
- **blur** - gaussian blur between 0-2000

### `auto`: format
If `auto` includes format, the service will try to determine the ideal format to convert the image to. The rules are:
- If the browser supports it, everything except for gifs is returned as webp
- If a png is requested and that png has no alpha channel, it will be returned as a jpeg

### `auto`: compress
The `compress` parameter will try to run post-processed optimizations on the image prior to returning it.
- `png` images will run through `pngquant`

## Security
### Request hashing
To prevent abuse of your lambda function, you can set a security key. When the security key environment variable is set,
every request is required to have the `s` query parameter set. This parameter is a simple md5 hash of the following:

`SECURITY KEY + / + PATH + QUERY`

For example, if my security key is set to `asdf` and someone requests:

https://something.cloudfront.net/web/general-images/photo.jpg?auto=compress%2Cformat&crop=focalpoint&fit=crop&fp-x=0.5&fp-y=0.5&h=380&q=80&w=700

__NOTE:__ The parameters are URI encoded!

They would also need to pass a security key param, `s`,

`md5('asdf' + '/' + 'web/general-images/photo.jpg' + '?auto=compress%2Cformat&crop=focalpoint&fit=crop&fp-x=0.5&fp-y=0.5&h=380&q=80&w=700')`

or to be more exact...

`md5('asdf/web/general-images/photo.jpg?auto=compress%2Cformat&crop=focalpoint&fit=crop&fp-x=0.5&fp-y=0.5&h=380&q=80&w=700')`

which equals...

`a0144a80b5b67d7cb6da78494ef574db`

and on our URL...

`https://something.cloudfront.net/web/general-images/photo.jpg?auto=compress%2Cformat&crop=focalpoint&fit=crop&fp-x=0.5&fp-y=0.5&h=380&q=80&w=700&s=a0144a80b5b67d7cb6da78494ef574db`

### Bucket prefix
The environment variable `SOURCE_BUCKET` may be configured with an optional path prefix if your images are stored in
a "sub-directory". For example, you may use:
`my-bucket/images`

This will tell Serverless to create an IAM role with access to all objects in the my-bucket bucket with an object prefix
of `images`.

Serverless Sharp will then validate all requests to this prefix, if it doesn't start with that prefix, the system will
automatically prepend it to the request images. This means these requests are effectively equivalent:

`localhost/images/my-image.png`

`localhost/my-image.png`

### SLS_VALID_PATH_REGEX
The SLS_VALID_PATH_REGEX setting allows you to explicitly control what assets may be handled by the Lambda function. If the
request path fails this test, a 404 response will be served.

Example:
```yaml
# All requests must start with /images/
SLS_VALID_PATH_REGEX: ^\/images\/.*`
```
### CUSTOM_DOMAIN
Setting the `CUSTOM_DOMAIN` setting will configure your Cloudfront distribution with a custom domain as well as a
standard Amazon issued SSL certificate. The SSL certificate is verified via DNS, so you'll need to access your ACM
console during the initial deployment to finish configuring the DNS records.

## Should I run this in production?
Sure!

## Running Locally
This package uses Serverless to allow for local development by simulating API Gateway and Lambda.
1. `npm ci`
2. `cp settings.example.yml settings.yml`
3. Configure settings.yml file
4. Ensure you have AWS CLI configured on your machine with proper access to the S3 bucket you're using in your settings.
5. Run `serverless offline`

## Deploying to AWS
Ensure your `settings.yml` file is properly configured as shown in the previous steps.

Run: `serverless deploy [--stage=dev] [--settings=settings.yml]`

If you need to deploy using a specific AWS profile, you should run:
`AWS_SDK_LOAD_CONFIG=true serverless deploy [--stage=dev] [--settings=settings.yml] --aws-profile PROFILE_NAME `
136 changes: 88 additions & 48 deletions docs/configuration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,19 @@
Serverless Sharp stores its settings in YAML files. When deploying the application, you specify the YAML file and the
stage (your environment). For example:

```sh
```shell-session
serverless deploy --settings mysite.com.yml --stage prod
```

## Settings File Structure
We use YML files to make use of their inheritance. This allows us to define some defaults and then branch environment
-specific changes off of them. For example:
-specific changes off of them. You can define multiple environments, each of which will inherit your default settings.
This is useful if you have different buckets for staging & production. You may also wish to create an
environment without a sign-key for local development. For example:


```yml
```yaml
defaults: &defaults
serviceName: image-handler-test
serviceName: image-handler
offlinePort: 80
region: 'us-east-1'
environment: &defaults.environment
Expand All @@ -24,79 +25,98 @@ defaults: &defaults
MAX_IMAGE_WIDTH: 2000
MAX_IMAGE_HEIGHT: 1000
PNGQUANT_SPEED: 10
SOURCE_BUCKET: 'images.mydomain.com'
DEFAULT_CACHE_CONTROL: 'max-age=2592000'

stages:
dev:
<<: *defaults
environment:
<<: *defaults.environment
SOURCE_BUCKET: 'my-bucket/prefix/path'
SECURITY_KEY: ''
CUSTOM_DOMAIN: ''
prod:
<<: *defaults
environment:
<<: *defaults.environment
SOURCE_BUCKET: 'random-string-here'
SECURITY_KEY: ''
CUSTOM_DOMAIN: 'my.domain.com'

SECURITY_KEY: 'random-string-here'
CUSTOM_DOMAIN: 'img.mydomain.com'
```
## Environment Settings
The majority of the application and environment specific settings are configured as environment variables that are
passed into the Lambda service handler.
### `DEFAULT_QUALITY`

- Type: `integer`
- Default: `75`
|Setting|Default|
|-|-|
|[CUSTOM_DOMAIN](#custom_domain)|*empty*|
|[DEFAULT_CACHE_CONTROL](#default_cache_control)|*empty*|
|[DEFAULT_COMPRESS_QUALITY](#default_compress_quality)|75|
|[DEFAULT_QUALITY](#default_quality)|75|
|[MAX_IMAGE_HEIGHT](#max_image_height)|`2000`|
|[MAX_IMAGE_WIDTH](#max_image_width)|`2000`|
|[PNGQUANT_SPEED](#pngquant_speed)|`10`|
|[SECURITY_KEY](#security_key)|*empty*|
|[SOURCE_BUCKET](#source_bucket)|*empty*|
|[SLS_IGNORE](#sls_ignore)|*empty*|
|[SLS_VALID_PATH_REGEX](#sls_valid_path_regex)|`.*`|

A number between 1 and 100 representing the default image quality.

### `DEFAULT_COMPRESS_QUALITY`
### `CUSTOM_DOMAIN`

- Type: `integer`
- Default: `75`
- Type: `string`
- Default: *empty*

A number between 1 and 100 for the default quality when an image is requested to be compressed
If set, Serverless Sharp will generate an Amazon issued certificate for the domain and link it to your Cloudfront
distribution.

### `SLS_IGNORE`
<Note type="tip">

- Type: `string`
- Default: ``
This requires deployment in the `us-east-1` region!

A comma-delineated string of paths that should be ignored (for example, `favicon.ico`)
</Note>

### `SLS_VALID_PATH_REGEX`
<Note type="tip">

- Type: `Regular Expression`
- Default: `.*`
During deployment, you'll need to go into your Amazon ACM console and verify the domain.

A regular expression for path validation. Allows you to explicitly control what assets may be handled by the Lambda
function. If the request path fails this test, a 404 response will be served.
</Note>

Example:
```yaml
# All requests must start with /images/
SLS_VALID_PATH_REGEX: ^\/images\/.*`
```
### `DEFAULT_CACHE_CONTROL`

### `MAX_IMAGE_WIDTH`
- Type: `string`
- Default: *empty*

By default, Serverless Sharp will serve responses with the same Cache-Control header as the object being served. If
the Cache-Control header is not set, this value will be used.

### `DEFAULT_COMPRESS_QUALITY`

- Type: `integer`
- Default: `2000`
- Default: `75`

Images may not be requested with a width larger than this value
A number between 1 and 100 for the default quality when an image is requested to be compressed

### `DEFAULT_QUALITY`

- Type: `integer`
- Default: `75`


### `MAX_IMAGE_HEIGHT`

- Type: `integer`
- Default: `2000`

Images may not be requested with a height larger than this value

### `MAX_IMAGE_WIDTH`

- Type: `integer`
- Default: `2000`

Images may not be requested with a width larger than this value

### `PNGQUANT_SPEED`

- Type: `integer`
Expand All @@ -106,28 +126,48 @@ The speed value to pass to the `pngquant` optimization program. From the [pngqua

> Speed/quality trade-off from 1 (brute-force) to 10 (fastest). The default is 3. Speed 10 has 5% lower quality, but is 8 times faster than the default.

### `DEFAULT_CACHE_CONTROL`
### `SECURITY_KEY`

- Type: `string`
- Default: ``

By default, Serverless Sharp will serve responses with the same Cache-Control header as the object being served. If
the Cache-Control header is not set, this value will be used.
- Default: *empty*

If set, a hash must be provided as the `s` parameter in all requests. See [Security](/usage/security) for more.

### `SOURCE_BUCKET`

- Type: `string`
- Default: ``
- Default: *empty*

### `SECURITY_KEY`
The S3 bucket in your account where your images are stored - you can include a path here if you like. For example
: `mybucket/images`

- Type: `string`
- Default: ``
This will tell Serverless to create an IAM role with access to all objects in the `mybucket` bucket with an object prefix
of `images`.

### `CUSTOM_DOMAIN`
Serverless Sharp will then validate all requests to this prefix, if it doesn't start with that prefix, the system
will automatically prepend it to the request. This means these requests are effectively equivalent:

`localhost/images/my-image.png`

`localhost/my-image.png`

### `SLS_IGNORE`

- Type: `string`
- Default: ``
- Default: *empty*

A comma-delineated string of paths that should be ignored (for example, `favicon.ico`)

### `SLS_VALID_PATH_REGEX`

> **NOTE:** This requires deployment in the `us-east-1` region!
- Type: `Regular Expression`
- Default: `.*`

A regular expression for path validation. Allows you to explicitly control what assets may be handled by the Lambda
function. If the request path fails this test, a 404 response will be served.

Example:
```yaml
# All requests must start with /images/
SLS_VALID_PATH_REGEX: ^\/images\/.*`
```
8 changes: 8 additions & 0 deletions docs/deployment.md
View file
Original file line number Diff line number Diff line change
@@ -1 +1,9 @@
# Deployment

Ensure your settings yml file is properly configured as shown in the Quick Start and Configuration sections

Run: `serverless deploy [--stage=dev] [--settings=settings.yml]`

If you need to deploy using a specific AWS profile, you should run:

`AWS_SDK_LOAD_CONFIG=true serverless deploy [--stage=dev] [--settings=settings.yml] --aws-profile PROFILE_NAME `
Loading

0 comments on commit d76822c

Please sign in to comment.