[wip] Convert docker support from lambci to official AWS docker images and add support for node 14

[wilson208]

This is a WIP PR for some feedback, it is functional for HTTP events and has not been tested with other event types as of yet, I will do when I get the time but for now would appreciate some early feedback on the approach.

Description

Convert docker support from using lambci lambda runtime images to using official AWS docker images

Motivation and Context

Motivation for this is to add support for node 14, and since lambci is no longer keeping up with latest official AWS runtimes by not supporting node 14, it is a good time to switch to the official AWS Images which are not available

Relates to the following issue: #1262

How Has This Been Tested?

So far testing has been limited to basic HTTP events as such:

serverless.yml

service: http

plugins:
  - serverless-offline

provider:
  memorySize: 128
  name: aws
  region: us-east-1 # default
  runtime: nodejs14.x
  stage: dev
  versionFunctions: false

custom:
  serverless-offline:
    useDocker: true

functions:
  hello:
    events:
      - http:
          # integration: lambda
          method: get
          path: hello
    handler: handler.hello

handler.js

'use strict'

exports.hello = async function hello() {
  return {
    body: JSON.stringify({ foo: 'bar', IS_OFFLINE: process.env.IS_OFFLINE }),
    statusCode: 200,
  }
}

Output

[wilson208]

This PR will also add support for Python 3.9 with docker which was added in this recent PR #1267

[apancutt]

DOCKER_LAMBDA_STAY_OPEN is redundant since AWS images always stay open.

DOCKER_LAMBDA_WATCH is redundant since there is no watch capability on the AWS images. Instead, this would need to be implemented within DockerContainer or DockerRunner (whichever you feel makes most sense). You will need to execute a docker restart on change detection, but debounce it (say, 5 seconds). The watch path should be configurable, but use the code path by default. Having it configurable allows users to specify the build output directory (when using, for example, serverless-webpack), target only the files they care about (e.g. ignore changes to source maps or node_modules, etc.), and allows them to watch a mounted volume instead of a host directory (in cases where running Docker-in-Docker - e.g. Docker Compose).

What are you getting when you run docker port <id of lambda container>? I'm curious if Docker will map 8080 on the host to 8080 on the container. If so, you'll need to explicitly map the host port to something random (and less commonly used than 8080 - see how aws-sam-cli does it).

/ping should return a 200 status with body of Healthy Connection and a Content-Length: 18 header but no Content-Type header (to mirror what API Gateway true does). This behaviour should be added whether --useDocker is used or not (so not part of DockerContainer.

I've not crawled through the serverless-offline code, but you'll need to make sure it queues requests and only sends a single request to the Lambda container at a time, else it'll crash.

Again, I've not thoroughly checked the existing code, but a quick search suggests serverless-docker does not set the Lambda timeout. If that's the case, it will need adding via the AWS_LAMBDA_FUNCTION_TIMEOUT env var, along with a bunch of others:

• AWS_ACCESS_KEY_ID
• AWS_ACCOUNT_ID
• AWS_DEFAULT_REGION
• AWS_LAMBDA_FUNCTION_HANDLER
• AWS_LAMBDA_FUNCTION_MEMORY_SIZE
• AWS_LAMBDA_FUNCTION_NAME
• AWS_LAMBDA_FUNCTION_TIMEOUT
• AWS_LAMBDA_FUNCTION_VERSION
• AWS_LAMBDA_LOG_GROUP_NAME
• AWS_LAMBDA_LOG_STREAM_NAME
• AWS_REGION
• AWS_SECRET_ACCESS_KEY

Again, refer to aws-sam-cli for guidance.

A few additions I'd also recommend:

1. Add the --rm option to docker create so that containers are removed on shutdown to reduce clutter.
2. DockerRunner has a stop() method but I'm not sure if it's called on shutdown (interrupt signal). If not, it should be.
3. Add the ability to skip image pull to speed up startup time

[dherault]

If this PR is still relevant, please reopen it.

[EcksDy]

@dherault
Why is this PR closed?
I'm trying to use serverless-offline with nodejs14.x runtime and keep getting "nodejs14.x" runtime is not supported with docker. See https://github.com/lambci/docker-lambda/issues/329 error.

Was this solved in another PR?

@wilson208
Were you able to run serverless-offline with node 14?

[plezan]

@dherault Why is this PR closed ?
Is it beacause of the PR #1267 ?
AFAIK it was only a temporary fix (error message) until a PR such as this one is merged

Has anyone found a way to run serverless-offline with node 14?

[Markkos89]

Hi there, facing the same issue with node14 or 16 like my folks here...

any chance to re open this or add support???