-
Notifications
You must be signed in to change notification settings - Fork 1
Model server images
In a production setting, we have found that there's often a requirement to deploy models in a bespoke ways for different needs. Therefore, the build and deploy features of catwalk are split into multiple commands for maximum flexibility.
A model can be built into a docker image with two build
commands.
The build-prep
command generates a Dockerfile
(and a .dockerignore
file) that's ready to build.
We can use the sklearn tutorial as an example:
$ cd tutorials/sklearn
$ catwalk build-prep
This will generate a following Dockerfile somewhat resembling the following:
FROM localhost:5000/catwalk:0.1.0
COPY . model/
RUN if [ -f "model/requirements.txt" ]; then cd model && pip install -r requirements.txt && cd ..; fi
ENV MODEL_PATH=model
ENV SERVER_CONFIG=false
ENV SERVER_PORT=9090
ENV RUN_TESTS=true
CMD catwalk serve
Note that here we're using all the defaults, so the docker registry is set to a private local registry and the catwalk base image matches your installed version of catwalk.
The rest of the Dockerfile simply copies the model path (in this case ".") into a model/
folder, installs any dependencies, and sets up environment variables for the catwalk server.
There are several options available to build-prep
:
$ catwalk build-prep --help
Usage: catwalk build-prep [OPTIONS]
Options:
-m, --model-path TEXT The path to the model directory we're working
with [default: .]
-p, --server-port INTEGER Specifies the port Flask will listen on.
[default: 9090]
-c, --server-config TEXT Specifies the path to the server's
configuration.
-r, --docker-registry TEXT Specifies the Docker repo this image is tagged
against. [default: localhost:5000]
--help Show this message and exit.
This image can be simply built with the catwalk build
command. This will build the image and push to the docker registry. The docker image tag is automatically defined by the model name and version number in the model.yml
file.
Here are the options available for catwalk build
:
$ catwalk build --help
Usage: catwalk build [OPTIONS]
Options:
-m, --model-path TEXT The path to the model directory we're working
with [default: .]
-r, --docker-registry TEXT Specifies the Docker repo this image is tagged
against. [default: localhost:5000]
-C, --no-cache If specified, docker will not use the build
cache.
--help Show this message and exit.
Once you have a model wrapped in a server image, you may wish to test it locally with the catwalk test-image
command. This test will start the docker container and send it some requests.
At the time of writing, deployment to cloud services such as AWS of Azure is out-of-scope for catwalk.
However, we do provide a catwalk deploy-prep
command that will generate configuration files for deploying into different environments. We will add to the list of supported deployments over time, but for now we only support a docker-compose
deployment.
Calling catwalk deploy-prep
will generate a docker-compose.yml
file in the model directory.
Again, if we use the sklearn tutorial as an example, the generated file will look something like this:
version: '2.0'
services:
model:
image: localhost:5000/catwalk-sklearn-tutorial:0.1.0
ports:
- "9090:9090"
environment:
- MODEL_PATH=model
- SERVER_CONFIG=false
- SERVER_PORT=9090
- RUN_TESTS=true
You can then start the server with docker-compose up
.
Servers can be configured with a YAML configuratoin file (see Model servers, and this needs to be mouted as a volume in the docker container.
This can be done by specifying a ``--server-config` option:
$ catwalk deploy-prep --server-config /path/to/config.yml
This will result in a docker-compose.yml that looks something like this:
version: '2.0'
services:
model:
image: localhost:5000/catwalk-sklearn-tutorial:0.1.0
ports:
- "9090:9090"
environment:
- MODEL_PATH=model
- SERVER_CONFIG=/config/config.yml
- SERVER_PORT=9090
- RUN_TESTS=true
volumes:
- /path/to:/config
Note that the config file can be specified as a relative path, and catwalk will create the absolute path for you.
Once this file has been created, the server can be started by simply calling docker-compose up
.
Multiple additional volumes can be specified with the --volumes
option:
$ catwalk deploy-prep --volumes /path/to/vol1:/vol1 --volumes /path/to/vol2:vol2
Will result in a docker-compose file that looks something like this:
version: '2.0'
services:
model:
image: localhost:5000/catwalk-sklearn-tutorial:0.1.0
ports:
- "9090:9090"
environment:
- MODEL_PATH=model
- SERVER_CONFIG=false
- SERVER_PORT=9090
- RUN_TESTS=true
volumes:
- /path/to/vol1:/vol1
- /path/to/vol2:/vol2
Here are all the options available for catwalk deploy-prep
:
$ catwalk deploy-prep --help
Usage: catwalk deploy-prep [OPTIONS]
Options:
-m, --model-path TEXT The path to the model directory we're working
with [default: .]
-p, --server-port INTEGER Specifies the port Flask will listen on.
[default: 9090]
-c, --server-config TEXT Specifies the path to the server's
configuration.
-r, --docker-registry TEXT Specifies the Docker repo this image is tagged
against. [default: localhost:5000]
-d, --deploy-mode [compose] The deploy mode controls where we are deploying
to. For now, the only option is `compose`.
[default: compose]
-v, --volumes TEXT Adds extra volume mounts to the deployed
container.
--help Show this message and exit.
Copyright 2020 Leap Beyond Emerging Technologies B.V. (CC BY 4.0 )