Setup

New version

Production: Docker

The docker project configuration for the production environment is located in compose.yaml, the app server is called "app", the database "nlpdb" and the web server "web".

first install:

• install docker: https://www.docker.com/products/docker-desktop/
• copy / rename app.env-example to app.env 🤌
• copy / rename db.env-example to db.env

# build the images, and spin them up
docker compose -f compose.yaml up -d --build
# create the database
docker compose -f compose.yaml exec app flask --app nlp4all init-db
docker compose -f compose.yaml exec app flask --app nlp4all spacy-download

to monitor output:

docker compose logs -f

To destroy and rebuild (including volumes/DB)

docker compose down -v
docker compose up -d --build
docker compose exec app flask --app nlp4all init-db
docker compose exec app flask --app nlp4all spacy-download

the site will now be available outside the container, on port 80, if you run this on your local machine then you can access it from: http://localhost/

Development: Docker

The docker project configuration for the production environment is located in compose.development.yaml, the app server is called "app-dev", the database "db-dev".

you'll want to install the conda environment and pip requirements, mainly for linting, pre-commit, etc:

conda create -n nlp4all python=3.11
conda activate nlp4all
pip install -r requirements-dev.txt
pre-commit install

if you want to run github actions 🤌 locally, you'll also need to install docker, but once you have docker just follow the install instructions here: https://github.com/nektos/act

Then you can run at any time

act

and it will run the github actions for the repo

for the docker setup then...

The process is similar to the production environment but instead of using gunicorn and nginx, it will serve the flask development server directly with debugging enabled. You should also always specify after docker compose the file i.e. docker compose -f compose.development.yaml ...

First time

Build and start the dev env:

# build the images, and spin them up
docker compose -f compose.development.yaml up -d --build
# create the database
docker compose -f compose.development.yaml exec app-dev flask --app nlp4all init-db
docker compose -f compose.development.yaml exec app-dev flask --app nlp4all spacy-download

Subsequent runs

At any time, to stop the containers you just have to run

docker compose -f compose.development.yaml stop

This leaves everything in place (including database contents)

then when you want to use it again just run

# note, for this one we don't use the '--build' parameter
# so no updates to requirements*.txt or changes to the docker
# build images will be implemented (they shouldn't change often)
# but this is extremely quick for normal development
docker compose -f compose.development.yaml up -d

Complete rebuild

If you want to perform a rebuild of the project because you'd like to start with a fresh database or have made changes to the requirements.txt or versions in docker need updating etc then you can do a full rebuild.

To destroy and rebuild (including volumes/DB)

docker compose -f compose.development.yaml down -v
docker compose -f compose.development.yaml up -d --build
docker compose -f compose.development.yaml exec app-dev flask --app nlp4all init-db
docker compose -f compose.development.yaml exec app-dev flask --app nlp4all spacy-download

You can now visit the site by going to http://localhost:5000/

Development: Local

Try to avoid using this because while it might be easier at first because it's outside of docker, it doesn't really mirror the actual setup, especially the database using sqlite instead of postgres.

initial setup:

conda create -n nlp4all python=3.11
conda activate nlp4all
pip install -r requirements-dev.txt
pre-commit install

setup database:

# from root of repository
flask init-db

get spacy models:

# from root of repository
flask spacy-download

run the app:

flask run

the site should now be available at http://localhost:5000/

Old developer instructions (pre docker)

conda create -n nlp4all python=3.9
conda activate nlp4all
pip install -r requirements-dev.txt
pre-commit install
python db_del_and_init.py
python ./run_dev.py

Of course you can do this without miniconda/conda but save yourself some headache!

see also: https://github.com/NLP4ALL/nlp4all/wiki/Development-Environment