Skip to content

Latest commit

 

History

History
51 lines (34 loc) · 2.7 KB

README.md

File metadata and controls

51 lines (34 loc) · 2.7 KB
Raw

MLRun Image Classification

Demo project using Distributed Tensorflow on Horovod with Nuclio

This example is using TensorFlow, Horovod, and Nuclio demonstrating end to end solution for image classification, it consists of 4 MLRun and Nuclio functions and Kubeflow Pipelines Orchastration:

  1. Download: import an image archive from S3 to the cluster file system
  2. Label: Tag the images based on their name structure
  3. Traing: Distrubuted training using TF, Keras and Horovod
  4. Inference: Automated deployment of Nuclio model serving function (form Notebook and from Dockerfile)



Running the demo

The demo is written as an MLRun Project, allowing you to use the load project to run the project through the main workflow as specified in the project.yaml

The demo applications are tested on the Iguazio's Data Science PaaS, and use Iguazio's shared data fabric (v3io), and can be modified to work with any shared file storage by replacing the apply(v3io_mount()) calls with other KubeFlow volume modifiers (e.g. apply(mlrun.platforms.mount_pvc())) . You can request a free trial of Iguazio PaaS.

Pre-requisites:

  • A Kubernetes cluster with pre-installed KubeFlow, Nuclio.
  • MLRun Service and UI installed, see MLRun readme.

Deployment Steps

  • Clone this repo to your own Git.
  • in a client or notebook properly configured with MLRun and KubeFlow run:

mlrun project my-proj/ -u git://github.com/<your-fork>/demo-image-classification.git

  1. Run the local playground notebook to build, test, and run functions.

  2. Open the project notebook and follow the instructions to run an automated ML Pipeline and source control.

Note: alternatively you can run the main pipeline from the CLI and specify artifacts path using:

mlrun project my-proj/ -r main -p "v3io:///users/admin/kfp/{{workflow.uid}}/"

Notebooks & Code