Please leave a ⭐ if you like this repo. Also please provide your comments if you would like to see something added
- Windows 10 PC (could run on Mac but I have not tested it!)
- Docker Desktop up and running with minimum 6 vcores and 6GB RAM
- Docker Desktop Kubernetes engine up and running
If you want to quickly get the environment up without getting into the details, do the following:
- Clone this git repo
- cd to dockerfiles folder and execute
docker_build.bat
- cd to k8s folder and execute
start_platform.bat
- Various services are started at the the following ports
- MLFlow UI :
http://localhost:30005
- Minio UI :
http://localhost:30009
- Postgres :
http://localhost:32345
- Jupyter notebook :
http://localhost:30008
- MLFlow UI :
- Connect to the MLFlow UI @
http://localhost:30005
. You can see that experiments are getting logged. These are also getting registered and promoted toStaging
. Finally 2 different versions of Regression models are promoted and served in Canary deployment mode. - To check the prediction you can use the client under the
client
folder by installing the requirements.txt and then executingstreamlit run client.py
. From this client UI you can also disable or enable the canary deployment
- Clone this get repo
- Download Kaggle Credit card fraud dataset from https://www.kaggle.com/mlg-ulb/creditcardfraud and put it under
./dataset
- Install Docker Desktop and enable Kubernetes from settings
- Install pgAdmin
- pip install all the required python libraries including mlflow and miniconda
docker pull postgres
docker pull minio/minio
- Build the docker images for mlflow server and logreg serving using
docker_build.bat
.- The mlflow server is for the mlflow UI and logreg serving is to serve the model registered as "Staging"
- Note: If you are running this in pure Docker environment without using Kubernetes you may have to change the IP addresses of the postgres and minio server in the docker files before building them
- Set the following environment variables:
AWS_ACCESS_KEY_ID=minioadmin
AWS_SECRET_ACCESS_KEY=minioadmin
MLFLOW_S3_ENDPOINT_URL=http://localhost:9000
MLFLOW_TRACKING_URI=postgresql+psycopg2://postgres:password@localhost:5432
MLFLOW_ARTIFACT_STORE=s3://mlruns
docker run -d --rm -p 5432:5432 -e POSTGRES_PASSWORD=password --name postgres postgres
- Open pgAdmin and connect to the postgres server running in the container (host=localhost, port=5432)
docker run -d --rm -p 9000:9000 --name minio minio/minio server /data
The above will start the containers with ephemeral storage. To start them with persistent volume, do the following:- Create a local folder for it to be mounted as persistent volume in the containers and add this as "file sharing" resource under Docker desktop --> Settings --> Resources --> File Sharing
- use the -v option with docker run command:
docker run -d --rm -p 9000:9000 -v [local path]:/data--name minio minio/minio server /data
- Running the postgres container with persistent volume on windows is a bit more involved and needs a docker compose file. The container path that needs to be mounted is
/var/lib/postgresql/data
docker run -d --rm -p 5000:5000 --name mlflow_server mlflow_server
- Logon to minio UI (http://localhost:9000) and add a bucket named
mlruns
- Logon to MLFlow UI using http://localhost:5000
- Run the
practice_code/mlflow_sk.pynb
for sklearn LogisticRegression Model - Use the MLFlow UI to register one of the models as "logreg" and promote it to "Staging"
docker run -d --rm -p 1235:1235 --name mlflow_logreg_infer mlflow_logreg_infer
to start model serving- Change the port in
practice_code/mlflow_serving.ipynb
to1235
and run it. Check the returned inference values
- Set the following environment variables:
AWS_ACCESS_KEY_ID=minioadmin
AWS_SECRET_ACCESS_KEY=minioadmin
MLFLOW_S3_ENDPOINT_URL=http://localhost:30009
MLFLOW_TRACKING_URI=postgresql+psycopg2://postgres:password@localhost:32345
MLFLOW_ARTIFACT_STORE=s3://mlruns
- Run
kubectl apply -f .
- Logon to minio UI (http://localhost:30009) and add a bucket named
mlruns
- Logon to MLFlow UI using http://localhost:30005
- Open pgAdmin and connect to the postgres server running in kubernetes (host=localhost, port=32345)
- Run the
practice_code/mlflow_sk.pynb
for sklearn LogisticRegression Model - Use the MLFlow UI to register one of the models as "logreg" and promote it to "Staging"
- Change the port in
practice_code/mlflow_serving.ipynb
to31235
and run it. Check the returned inference values
- Install sqlite3 if it does not already exist (Normally this comes by default with Python installation)
- Create a folder for the sqlite db
- Set the following environment variables:
MLFLOW_TRACKING_URI=sqlite:///[Path/to/sqlite db file]
MLFLOW_ARTIFACT_STORE=file:/[Path/to/local/folder]
- Run
mlflow ui --backend-store-uri %MLFLOW_TRACKING_URI%
ORmlflow server --backend-store-uri %MLFLOW_TRACKING_URI% --default-artifact-root %MLFLOW_ARTIFACT_STORE%
to start the MLFlow server - Logon to MLFlow UI using http://localhost:5000
- Run the
practice_code/mlflow_sk.pynb
for sklearn LogisticRegression Model - Use the MLFlow UI to register one of the models as "logreg" and promote it to "Staging"
- Run
mlflow models serve --model-uri models:/logreg/Staging -p 1235
to start serving the model - Change the port in
practice_code/mlflow_serving.ipynb
to1235
and run it. Check the returned inference values
- Try to serve the Tensorflow model for MNIST dataset in mlflow_tf.ipynb
- Trivial setup
- Intuitive server UI
- Minimal code instrumentation for simple tasks like logging params, metrics, models and data
- Trivial to serve a new version of the model --> Just promote the new version from the UI or using the API
- Integration with Git
- Community edition does not have any user managment and security feature
- If the developement environment is seperate from the production environment, how to deploy the model artifacts and the metadata from developement to production?
👉The above issues make MLFlow practically limited to internal developement enviornment only. It cannot be used in any real production usecase.
- It is possible to create replicable training pipeline flows using MLFlow Projects but this requires one to learn the MLflow Project DSL and also a good knowledge of MLFlow APIs to write the code. For eg, refer https://github.com/mlflow/mlflow/tree/master/examples/multistep_workflow