Merge pull request #4 from max-pfeiffer/feature/switch_to_new_base_image

Feature/switch to new base image

.github/workflows/pipeline.yml

@@ -23,13 +23,14 @@ jobs:
       - name: pylint
         run: |
           source .venv/bin/activate
-          pylint publish.py build tests
+          pylint build tests
       - name: black
         run: |
           source .venv/bin/activate
           black --check .
 
   run-tests:
+    needs: code-quality
     runs-on: ubuntu-20.04
     steps:
       - name: Checkout Repository
@@ -89,4 +90,4 @@ jobs:
           DOCKER_HUB_PASSWORD: ${{ secrets.DOCKER_HUB_PASSWORD }}
         run: |
           source .venv/bin/activate
-          python publish.py
+          python -m build.publish

.pre-commit-config.yaml

@@ -7,7 +7,7 @@ repos:
       - id: detect-private-key
       - id: check-added-large-files
   - repo: https://github.com/psf/black
-    rev: 22.10.0
+    rev: 23.1.0
     hooks:
       - id: black
   - repo: local
@@ -18,6 +18,5 @@ repos:
         language: system
         types: [python]
         args:
-          - publish.py
           - build
           - tests

README.md

@@ -9,7 +9,7 @@ It provides [Poetry](https://python-poetry.org/) for managing dependencies and s
 
 This image aims to follow the best practices for a production grade container image for hosting Python web applications based
 on micro frameworks like [FastAPI](https://fastapi.tiangolo.com/).
-Therefore source and documentation contain a lot of references to documentation of dependencies used in this project, so users
+Therefore, source and documentation contain a lot of references to documentation of dependencies used in this project, so users
 of this image can follow up on that.
 
 If you would like to run your Python application with Uvicorn on [Kubernetes](https://kubernetes.io/), please check out my other project which does not use
@@ -21,20 +21,13 @@ Any feedback is highly appreciated and will be considered.
 
 **GitHub Repository:** [https://github.com/max-pfeiffer/uvicorn-gunicorn-poetry](https://github.com/max-pfeiffer/uvicorn-gunicorn-poetry)
 
-**IMPORTANT:** Please be aware of a bug with Gunicorn: [worker reload with Uvicorn workers is currently broken.](https://github.com/benoitc/gunicorn/issues/2339)
-So the latest version of that image does not provide that functionality any more. Meanwhile [the bug became fixed in Gunicorn](https://github.com/bigsbug/gunicorn/pull/1),
-but the fix did not become merged yet.
-
 ## Docker Image Features
 1. Supported architectures:
    1. Python v3.9, Debian or Debian-slim
    2. Python v3.10, Debian or Debian-slim
 2. Poetry is available as Python package dependency management tool
 3. A virtual environment for the application and application server
-4. An [entrypoint for running the Python application with Gunicorn](https://github.com/max-pfeiffer/uvicorn-gunicorn-poetry/blob/main/build/scripts/start_gunicorn.sh)
-5. Additional entrypoints for [pytest](https://github.com/max-pfeiffer/uvicorn-gunicorn-poetry/blob/main/build/scripts/pytest_entrypoint.sh)
-   and [black](https://github.com/max-pfeiffer/uvicorn-gunicorn-poetry/blob/main/build/scripts/black_entrypoint.sh) which can be used in
-   multi stage builds for building docker executables
+4. The application is run with [Gunicorn](https://gunicorn.org/) and Uvicorn workers
 
 ## Usage
 It just provides a platform that you can use to build upon your own multistage builds. So it consequently does not contain an
@@ -44,7 +37,6 @@ Please check out the [example application for multistage builds](https://github.
 on how to use that image and build containers efficiently.
 
 There is also another [sample app demonstrating a very simple single stage build](https://github.com/max-pfeiffer/uvicorn-gunicorn-poetry/tree/main/examples/fast_api_singlestage_build).
-If you are not concerned about image size, go for that image.
 
 Please be aware that your application needs an application layout without src folder which is proposed in
 [fastapi-realworld-example-app](https://github.com/nsidnev/fastapi-realworld-example-app).
@@ -74,11 +66,11 @@ If your application uses FastAPI framework this needs to be added as well:
 * fastapi = "0.85.0"
 
 **IMPORTANT:** make sure you have a [.dockerignore file](https://github.com/max-pfeiffer/uvicorn-gunicorn-poetry/blob/master/examples/fast_api_multistage_build/.dockerignore)
-in your application root which excludes your local virtual environment in .venv! Otherwise you will have an issue activating that virtual
+in your application root which excludes your local virtual environment in .venv! Otherwise, you will have an issue activating that virtual
 environment when running the container.
 
 ## Configuration
-Configuration is done trough the following environment variables during docker build.
+Configuration is done through the following environment variables during docker build.
 For all the following configuration options please see always the
 [official Gunicorn documentation](https://docs.gunicorn.org/en/stable/settings.html)
 if you would like to do a deep dive. Following environment variables are supported:
@@ -102,7 +94,7 @@ if you would like to do a deep dive. Following environment variables are support
 **default:** `-`
 
 ### [Worker processes](https://docs.gunicorn.org/en/stable/settings.html#worker-processes)
-`WORKERS` : The number of worker processes for handling requests. By default this is set to one
+`WORKERS` : The number of worker processes for handling requests. By default, this is set to one
 worker as this image is meant to be used on a production grade Kubernetes environment. There you
 have usually monitoring data exported to Prometheus which will not work properly with multiple workers.   
 
@@ -122,11 +114,11 @@ have usually monitoring data exported to Prometheus which will not work properly
 
 ### [Server mechanics](https://docs.gunicorn.org/en/stable/settings.html?highlight=worker_tmp_dir#worker-tmp-dir)
 `WORKER_TMP_DIR` : A directory to use for the worker heartbeat temporary file.
-By default this is set to /dev/shm to speed up the startup of workers by using a in memory file system
+By default, this is set to /dev/shm to speed up the startup of workers by using a in memory file system
 
 **default:** `/dev/shm`
 
 ### [Server socket](https://docs.gunicorn.org/en/stable/settings.html?highlight=bind#bind)
 `BIND` : The socket to bind.
 
-**default:** `0.0.0.0:80`
+**default:** `0.0.0.0:8000`

build/Dockerfile

@@ -3,8 +3,8 @@
 # Base image: https://hub.docker.com/r/pfeiffermax/python-poetry
 # Dockerfile: https://github.com/max-pfeiffer/python-poetry/blob/main/build/Dockerfile
 ARG BASE_IMAGE
-ARG APPLICATION_SERVER_PORT
 FROM ${BASE_IMAGE}
+ARG APPLICATION_SERVER_PORT
 
 LABEL maintainer="Max Pfeiffer <[email protected]>"
 
@@ -15,26 +15,35 @@ ENV PYTHONUNBUFFERED=1 \
     PYTHONPATH=/application_root \
     # https://python-poetry.org/docs/configuration/#virtualenvsin-project
     POETRY_VIRTUALENVS_IN_PROJECT=true \
-    VIRTUAL_ENVIRONMENT_PATH="/application_root/.venv"
+    POETRY_CACHE_DIR="/application_root/.cache" \
+    VIRTUAL_ENVIRONMENT_PATH="/application_root/.venv" \
+    APPLICATION_SERVER_PORT=$APPLICATION_SERVER_PORT
 
-ENV PATH="$POETRY_HOME/bin:$VIRTUAL_ENVIRONMENT_PATH/bin:$PATH"
+# Adding the virtual environment to PATH in order to "activate" it.
+# https://docs.python.org/3/library/venv.html#how-venvs-work
+ENV PATH="$VIRTUAL_ENVIRONMENT_PATH/bin:$PATH"
 
-# Set the WORKDIR to the application root. This needed for the entrypoints to
-# work. Also Uvicorn --reload feature relies by default on the current work dir.
+# Principle of least privilege: create a new user for running the application
+RUN groupadd -g 1001 python_application && \
+    useradd -r -u 1001 -g python_application python_application
+
+# Set the WORKDIR to the application root.
 # https://www.uvicorn.org/settings/#development
 # https://docs.docker.com/engine/reference/builder/#workdir
 WORKDIR ${PYTHONPATH}
+RUN chown python_application:python_application ${PYTHONPATH}
 
-COPY gunicorn_configuration.py ./scripts/start_gunicorn.sh /application_server/
-RUN chmod +x /application_server/start_gunicorn.sh
+# Create cache directory and set permissions because user 1001 has no home
+# and poetry cache directory.
+# https://python-poetry.org/docs/configuration/#cache-directory
+RUN mkdir ${POETRY_CACHE_DIR} && chown python_application:python_application ${POETRY_CACHE_DIR}
 
-COPY ./scripts/pytest_entrypoint.sh ./scripts/black_entrypoint.sh /entrypoints/
-RUN chmod +x /entrypoints/pytest_entrypoint.sh
-RUN chmod +x /entrypoints/black_entrypoint.sh
+# Copy the application server configuration
+COPY --chown=python_application:python_application gunicorn_configuration.py ${PYTHONPATH}
 
-# Activate entrypoint for running the Gunicorn application server
-CMD ["/application_server/start_gunicorn.sh"]
-
-# Document the exposed port which was configured in start_uvicorn.sh
+# Document the exposed port
 # https://docs.docker.com/engine/reference/builder/#expose
 EXPOSE ${APPLICATION_SERVER_PORT}
+
+# Activate entrypoint for running the Gunicorn application server
+CMD exec gunicorn -k uvicorn.workers.UvicornWorker -c $PYTHONPATH/gunicorn_configuration.py app.main:app

build/constants.py

@@ -2,35 +2,32 @@
 FAST_API_SINGLESTAGE_IMAGE_NAME: str = "fast-api-singlestage-build"
 FAST_API_MULTISTAGE_IMAGE_NAME = "fast-api-multistage-build"
 TARGET_ARCHITECTURES: list[str] = [
-    "python3.9.14-bullseye",
-    "python3.9.14-slim-bullseye",
-    "python3.10.7-bullseye",
-    "python3.10.7-slim-bullseye",
+    "python3.9.16-bullseye",
+    "python3.9.16-slim-bullseye",
+    "python3.10.9-bullseye",
+    "python3.10.9-slim-bullseye",
 ]
 BASE_IMAGES: dict = {
     TARGET_ARCHITECTURES[
         0
-    ]: "pfeiffermax/python-poetry:1.0.0-poetry1.2.2-python3.9.14-bullseye",
+    ]: "pfeiffermax/python-poetry:1.1.0-poetry1.3.2-python3.9.16-bullseye@sha256:3795ff170e143a5dfa960a81356e4cb3406ed6a7a3ccea0156c14e5cf4a67053",
     TARGET_ARCHITECTURES[
         1
-    ]: "pfeiffermax/python-poetry:1.0.0-poetry1.2.2-python3.9.14-slim-bullseye",
+    ]: "pfeiffermax/python-poetry:1.1.0-poetry1.3.2-python3.9.16-slim-bullseye@sha256:c6f545f175e7369017ae8d39e54497cb423ff8291d1a492b086dcd1a7439f9b0",
     TARGET_ARCHITECTURES[
         2
-    ]: "pfeiffermax/python-poetry:1.0.0-poetry1.2.2-python3.10.7-bullseye",
+    ]: "pfeiffermax/python-poetry:1.1.0-poetry1.3.2-python3.10.9-bullseye@sha256:c269b0872f11fd198703e9dea301a4cb3dd1bbd7e054ab723d801dccc0b631cd",
     TARGET_ARCHITECTURES[
         3
-    ]: "pfeiffermax/python-poetry:1.0.0-poetry1.2.2-python3.10.7-slim-bullseye",
-}
-OFFICIAL_PYTHON_IMAGES: dict = {
-    TARGET_ARCHITECTURES[0]: "python:3.9.14-bullseye",
-    TARGET_ARCHITECTURES[1]: "python:3.9.14-slim-bullseye",
-    TARGET_ARCHITECTURES[2]: "python:3.10.7-bullseye",
-    TARGET_ARCHITECTURES[3]: "python:3.10.7-slim-bullseye",
+    ]: "pfeiffermax/python-poetry:1.1.0-poetry1.3.2-python3.10.9-slim-bullseye@sha256:ee99ee20733201523728147ab0c9117d22266994a1919ec0d64937133a51f07d",
 }
 PYTHON_VERSIONS: dict = {
-    TARGET_ARCHITECTURES[0]: "3.9.14",
-    TARGET_ARCHITECTURES[1]: "3.9.14",
-    TARGET_ARCHITECTURES[2]: "3.10.7",
-    TARGET_ARCHITECTURES[3]: "3.10.7",
+    TARGET_ARCHITECTURES[0]: "3.9.16",
+    TARGET_ARCHITECTURES[1]: "3.9.16",
+    TARGET_ARCHITECTURES[2]: "3.10.9",
+    TARGET_ARCHITECTURES[3]: "3.10.9",
 }
-APPLICATION_SERVER_PORT: str = "80"
+
+# As we are running the server with an unprivileged user, we need to use
+# a high port.
+APPLICATION_SERVER_PORT: str = "8000"

build/gunicorn_configuration.py

@@ -2,15 +2,14 @@
 import json
 
 DEFAULT_GUNICORN_CONFIG = {
-    "bind": "0.0.0.0:80",
+    "bind": "0.0.0.0:8000",
     "workers": 1,
     "timeout": 30,
     "graceful_timeout": 30,
     "keepalive": 2,
     "loglevel": "info",
     "accesslog": "-",
     "errorlog": "-",
-#   "reload": False,
     "worker_tmp_dir": "/dev/shm",
 }
 
@@ -20,14 +19,6 @@
 accesslog = os.getenv("ACCESS_LOG", DEFAULT_GUNICORN_CONFIG["accesslog"])
 errorlog = os.getenv("ERROR_LOG", DEFAULT_GUNICORN_CONFIG["errorlog"])
 
-# Debugging
-# https://docs.gunicorn.org/en/stable/settings.html#debugging
-
-# There is a bug with Gunicorn reload with uvicorn workers, so this features is
-# not available any more until this bug became fixed
-# see: https://github.com/benoitc/gunicorn/issues/2339
-# reload = bool(os.getenv("RELOAD", DEFAULT_GUNICORN_CONFIG["reload"]))
-
 # Worker processes
 # https://docs.gunicorn.org/en/stable/settings.html#worker-processes
 workers = int(os.getenv("WORKERS", DEFAULT_GUNICORN_CONFIG["workers"]))
@@ -57,7 +48,6 @@
     "loglevel": loglevel,
     "errorlog": errorlog,
     "accesslog": accesslog,
-#   "reload": reload,
     "worker_tmp_dir": worker_tmp_dir,
 }
 print(json.dumps(log_data))

build/images.py

@@ -9,7 +9,6 @@
     BASE_IMAGES,
     FAST_API_MULTISTAGE_IMAGE_NAME,
     APPLICATION_SERVER_PORT,
-    OFFICIAL_PYTHON_IMAGES,
     FAST_API_SINGLESTAGE_IMAGE_NAME,
 )
 
@@ -121,10 +120,6 @@ def build(
 
         buildargs: dict[str, str] = {
             "BASE_IMAGE_NAME_AND_TAG": base_image_tag,
-            "OFFICIAL_PYTHON_IMAGE": OFFICIAL_PYTHON_IMAGES[
-                target_architecture
-            ],
-            "APPLICATION_SERVER_PORT": APPLICATION_SERVER_PORT,
         }
         image: Image = self.docker_client.images.build(
             path=str(self.absolute_docker_image_directory_path),

examples/fast_api_multistage_build/.dockerignore

@@ -1,5 +1,7 @@
+.idea
+.pytest_cache
 .venv
 .coverage
+test_coverage_reports/
 Dockerfile
 README.md
-test_coverage_reports/