Skip to content

Latest commit

 

History

History
367 lines (257 loc) · 11.6 KB

README.md

File metadata and controls

367 lines (257 loc) · 11.6 KB
Raw

Lambda Comments

This project which implements a minimal blog commenting service.

Blog posts:

Hacker News thread: https://news.ycombinator.com/item?id=11644042

It is completely "serverless", designed to use the following Amazon services:

The Lambda functions are written in ES6, with async/await, transpiled using Babel, and bundled using Webpack.

The AWS resources are provisioned using the CloudFormation service.

Additionally, we use Apex to simplify the uploading of the Lambda functions (without the shim).

Costs

It should cost very little to run. The following resources are provisioned:

  • DynamoDB - only provisioned for 1 read capacity unit, 1 write capacity unit (which limits it to 1 job per second). This is the most expensive resource, approximately $0.65 a month.
  • S3 - storage for comments and private data, plus requests and data transfer
  • CloudWatch Logs
  • Lambda functions - only pay for invocations, first million requests per month are free (hopefully your blog isn't that popular)
  • API gateway - only pay for API calls

Deployment Instructions

Prerequisites

  • You will need an AWS Account
  • You will need OS X, Linux, *BSD or another Unix-based OS (scripts will need some modifications for Windows)
  • Install the AWS CLI and ensure credentials are setup under ~/.aws/credentials (Instructions)
  • Install Node.js (tested with v4.2.6 and v5.7.0)
  • git clone https://github.com/jimpick/lambda-comments.git (https)
    or
    git clone [email protected]:jimpick/lambda-comments.git (git)
  • cd lambda-comments
  • npm run install-all (runs npm install in top directory, and then sets up sub-packages under packages)
  • Install Apex

Configuration

Copy .env.SAMPLE to .env and customize it.

cp .env.SAMPLE .env

The default .env.SAMPLE contains:

# The URL of your blog/website that will be hosting the comments
# Used to generate CORS headers and also for Akismet
BLOG=https://example.com/blog/

# A name for your CloudFormation stack
# Also prefixed to the API Gateway REST API name
CLOUDFORMATION=myBlogComments

# The AWS region to provision the resources in
REGION=us-west-2

# The name for the API Gateway stage
STAGE=prod

# The Akismet.com API key (optional, but recommended)
# Akismet is a service for combatting blog spam from Automattic (WordPress)
#AKISMET=0123456789ab

# A Slack webhook to send notifications to (optional)
#SLACK=https://hooks.slack.com/services/XXXXXXXXX/YYYYYYYYY/ZZZZZZZZZZZZZZZZZZZZZZZZ

We use dotenv so it is also possible to configure the project by setting environment variables.

Parameters

BLOG: The full base url of the blog/website

CLOUDFORMATION: The name of the CloudFormation stack

REGION: The AWS region

STAGE: The API Gateway stage to create

AKISMET: (Optional, but recommended) API key from akismet.com for spam filtering

SLACK: (Optional) Slack webhook - configure this if you want a notification in a Slack channel each time a comment is posted

Installation

For now, follow the step-by-step instructions below. In the future, we will develop a streamlined installation procedure.

Use CloudFormation to create the AWS resources

npm run create-cloudformation

The command returns immediately, but it will take a while to complete (typically 3-4 minutes). It's a good idea to watch the CloudFormation task in the AWS Web Console to ensure that it completes without errors.

Note: When working with the CloudFormation recipe, you can also use npm run update-cloudformation and npm run delete-cloudformation

Save the references to the provisioned CloudFormation resources

npm run save-cloudformation

This will create a file in deploy/state/cloudFormation.json

Temporary fix - update Lambda functions to use Node.js 4.3

CloudFormation has issues setting the runtime to 'nodejs4.3', see:

https://forums.aws.amazon.com/thread.jspa?threadID=229072

Until CloudFormation is updated, here's an extra step to update the Lambda functions to use Node.js 4.3 using a custom script.

npm run flip-lambdas-to-4.3

Generate an API key

npm run gen-api-key

This will create a file in deploy/state/apiKey.json containing an apiKey variable that will be baked into the client to sign requests.

The purpose of the API key is to try to minimize spam to the API, but as the API key is distributed publicly as part of the javascript, it's only there to stop non-sophisticated spammers who don't care enough to extract the API key and sign their requests.

Setup the Apex build directory

npm run setup-apex

This generates build/apex/project.json

Compile the Lambda scripts using babel

npm run compile-lambda

This will use webpack and babel to compile the source code in src/server/lambdaFunctions into build/apex/functions

The webpack configuration is in deploy/apex/webpack.config.es6.js

Deploy the lambda functions

npm run deploy-lambda

This will run apex deploy in the build/apex directory to upload the compiled lambda functions.

Alternatively, if you want to execute the compile and deploy steps in one command, you can run: npm run deploy-backend

Build the frontend javascript

npm run build-frontend

This builds the code in the packages/frontend directory.

Upload the frontend javascript script to S3

npm run upload-script

This will copy lambda-comments.js to the S3 bucket.

Alternatively, if you want to execute the compile and deploy steps in one command, you can run: npm run deploy-frontend.

If you want to deploy the backend and frontend all in one step, you can use: npm run deploy

Run the test suite

npm run test

This will run both the local tests, and remote test which test the deployed API and lambda functions.

The local tests can be run as npm run test-local, and the remote tests can be run as npm run test-remote.

Currently the test suite expects some data to pre-exist in the S3 bucket. Until the tests are properly mocked, they will fail unless the data is created.

View logs

You can tail the CloudWatch logs:

npm run logs

This just executes apex logs -f in build/apex

Embed the front-end JavaScript client

First, get the URL for the script:

npm run get-client-js-url

This will return a URL you will use below. eg.

//s3-us-west-2.amazonaws.com/myblogcomments-websites3-1ttpk69ph7gr7/lambda-comments.js

In the target web page (perhaps a blog generated by a static site generator such as Jekyll or Hugo), add the following HTML to insert the comments from the development server into the page:

<div id="lambda-comments"></div>
<script src="*** url from above ***"></script>

Providing that the webpage is located at the web address matching the 'BLOG' setting in the .env configuration file, the comments form should appear on the page. If not, check the developer tools console in the web browser to see if there are any errors (typically due to CORS).

Front-end development

The code for the "front-end" javascript that displays the comments and the comment form embedded in a web page lives in the packages/frontend directory.

To run a development server, change into the packages/frontend directory, copy .env.SAMPLE to .env, and run the development server.

cd packages/frontend
cp .env.SAMPLE .env
npm start

The development server is based on react-project with a heavily modified webpack configuration in webpack.config.js.

In the target web page (perhaps a blog generated by a static site generator such as Jekyll or Hugo), add the following HTML to insert the comments from the development server into the page:

<div id="lambda-comments"></div>
<script src="http://localhost:8081/lambda-comments.js"></script>

To Do List

  • Limit length of comments and metadata
  • Simplified installation
  • Check that permalink and blog match
  • Override for path location
  • Fetch source page to confirm script is installed on first post
  • Test on various browsers, polyfills
  • CORS override
  • Rearrange code: put lambda scripts under packages directory
  • Admin: auth
  • Admin: moderation
  • Admin: submit ham/spam to akismet
  • Admin: Turn comments on/off
  • Support for editing blog posts for a limited time
  • Detect DDoS style attacks and automatically throttle API Gateway to prevent unlimited charges
  • Mocks for AWS/API calls
  • Integration test
  • Selenium tests
  • Coverage
  • Emoji support
  • Handle DynamoDB ProvisionedThroughputExceededException
  • Investigate Swagger
  • Generate API docs
  • Webpack 2 tree-shaking support
  • Optimize download size
  • Plugins for server-side rendering on common static site generators
  • Optimized bundle for ES6-capable platforms
  • Library for bundling with existing client-side javascript builds
  • Investigate deep integration with React.js static site generators
  • Gatsby
  • Phenomic
  • Instructions for static-ish hosting platforms
  • GitHub Pages
  • Surge
  • Netlify
  • Aerobatic
  • Firebase
  • Zeit

Interesting links

Lots of related projects, many of which I haven't investigated yet.

Lambda / Serverless Frameworks

Lambda Libraries

Awesome Lists

Serverless Comment Systems

Open-source Comment Systems

Hosted Comment Platforms