OSCON demo

This repository contains the demo code presented during the talk End-to-end ML streaming with Kubeflow, Kafka, and Redis at scale at O'Reilly OSCON 2019.

The code basically shows how to:

1. train and save a simple model from the California Housing prices dataset from scikit-learn
2. serve that model with seldon core kubernetes
3. stream raw features to kafka
4. aggregate features (perform feature engineering) and stream engineered features to redis
5. invoke model directly from redis engineered features

To run this demo, follow the jupyter notebook

The slides from my talk are available here

The following great projects were used / referenced:

• Python, scikit-learn, MLFlow
• Kafka, Redis, Docker, Kubernetes, Kubeflow, Seldon Core
• Prometheus, Grafana

Troubleshooting:

• Make sure not to use some VPN software while running this demo as it may interfere with docker desktop and k8s routing
• If your network changes, you may need to restart kafka and the redis docker container
• Make sure kubectl proxy isn't running

The code samples here are licensed under the Apache 2.0 License