Skip to content

Model server images

Thomas Bird edited this page Apr 27, 2020 · 3 revisions

In a production setting, we have found that there's often a requirement to deploy models in a bespoke ways for different needs. Therefore, the build and deploy features of catwalk are split into multiple commands for maximum flexibility.

Building docker images

A model can be built into a docker image with two build commands.

Generating the Dockerfile

The build-prep command generates a Dockerfile (and a .dockerignore file) that's ready to build.

We can use the sklearn tutorial as an example:

$ cd tutorials/sklearn
$ catwalk build-prep

This will generate a following Dockerfile somewhat resembling the following:

FROM localhost:5000/catwalk:0.1.0

COPY . model/
RUN if [ -f "model/requirements.txt" ]; then cd model && pip install -r requirements.txt && cd ..; fi

ENV MODEL_PATH=model
ENV SERVER_CONFIG=false
ENV SERVER_PORT=9090
ENV RUN_TESTS=true

CMD catwalk serve

Note that here we're using all the defaults, so the docker registry is set to a private local registry and the catwalk base image matches your installed version of catwalk.

The rest of the Dockerfile simply copies the model path (in this case ".") into a model/ folder, installs any dependencies, and sets up environment variables for the catwalk server.

There are several options available to build-prep:

$ catwalk build-prep --help
Usage: catwalk build-prep [OPTIONS]

Options:
  -m, --model-path TEXT       The path to the model directory we're working
                              with  [default: .]
  -p, --server-port INTEGER   Specifies the port Flask will listen on.
                              [default: 9090]
  -c, --server-config TEXT    Specifies the path to the server's
                              configuration.
  -r, --docker-registry TEXT  Specifies the Docker repo this image is tagged
                              against.  [default: localhost:5000]
  --help                      Show this message and exit.

Performing a build

This image can be simply built with the catwalk build command. This will build the image and push to the docker registry. The docker image tag is automatically defined by the model name and version number in the model.yml file.

Here are the options available for catwalk build:

$ catwalk build --help
Usage: catwalk build [OPTIONS]

Options:
  -m, --model-path TEXT       The path to the model directory we're working
                              with  [default: .]
  -r, --docker-registry TEXT  Specifies the Docker repo this image is tagged
                              against.  [default: localhost:5000]
  -C, --no-cache              If specified, docker will not use the build
                              cache.
  --help                      Show this message and exit.

Testing the built image

Once you have a model wrapped in a server image, you may wish to test it locally with the catwalk test-image command. This test will start the docker container and send it some requests.

Deploying the built image

At the time of writing, deployment to cloud services such as AWS of Azure is out-of-scope for catwalk.

However, we do provide a catwalk deploy-prep command that will generate configuration files for deploying into different environments. We will add to the list of supported deployments over time, but for now we only support a docker-compose deployment.

Docker-compose deployments

Calling catwalk deploy-prep will generate a docker-compose.yml file in the model directory.

Again, if we use the sklearn tutorial as an example, the generated file will look something like this:

version: '2.0'
services:
  model:
    image: localhost:5000/catwalk-sklearn-tutorial:0.1.0
    ports:
      - "9090:9090"
    environment:
      - MODEL_PATH=model
      - SERVER_CONFIG=false
      - SERVER_PORT=9090
      - RUN_TESTS=true

You can then start the server with docker-compose up.

Adding a configuration

Servers can be configured with a YAML configuratoin file (see Model servers, and this needs to be mouted as a volume in the docker container.

This can be done by specifying a ``--server-config` option:

$ catwalk deploy-prep --server-config /path/to/config.yml

This will result in a docker-compose.yml that looks something like this:

version: '2.0'
services:
  model:
    image: localhost:5000/catwalk-sklearn-tutorial:0.1.0
    ports:
      - "9090:9090"
    environment:
      - MODEL_PATH=model
      - SERVER_CONFIG=/config/config.yml
      - SERVER_PORT=9090
      - RUN_TESTS=true
    volumes:
      - /path/to:/config

Note that the config file can be specified as a relative path, and catwalk will create the absolute path for you.

Once this file has been created, the server can be started by simply calling docker-compose up.

Additional volumes

Multiple additional volumes can be specified with the --volumes option:

$ catwalk deploy-prep --volumes /path/to/vol1:/vol1 --volumes /path/to/vol2:vol2

Will result in a docker-compose file that looks something like this:

version: '2.0'
services:
  model:
    image: localhost:5000/catwalk-sklearn-tutorial:0.1.0
    ports:
      - "9090:9090"
    environment:
      - MODEL_PATH=model
      - SERVER_CONFIG=false
      - SERVER_PORT=9090
      - RUN_TESTS=true
    volumes:
      - /path/to/vol1:/vol1
      - /path/to/vol2:/vol2

Here are all the options available for catwalk deploy-prep:

$ catwalk deploy-prep --help
Usage: catwalk deploy-prep [OPTIONS]

Options:
  -m, --model-path TEXT        The path to the model directory we're working
                               with  [default: .]
  -p, --server-port INTEGER    Specifies the port Flask will listen on.
                               [default: 9090]
  -c, --server-config TEXT     Specifies the path to the server's
                               configuration.
  -r, --docker-registry TEXT   Specifies the Docker repo this image is tagged
                               against.  [default: localhost:5000]
  -d, --deploy-mode [compose]  The deploy mode controls where we are deploying
                               to. For now, the only option is `compose`.
                               [default: compose]
  -v, --volumes TEXT           Adds extra volume mounts to the deployed
                               container.
  --help                       Show this message and exit.