Skip to content

Commit

Permalink
[DOC] welcome.ipynb/README.md: demos doc edits, mainly for new how-tos
Browse files Browse the repository at this point in the history
Doc review for PR #319
  • Loading branch information
Sharon-iguazio committed Jan 21, 2021
1 parent f6fe095 commit 8a224a9
Show file tree
Hide file tree
Showing 3 changed files with 67 additions and 42 deletions.
43 changes: 24 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ An initial introduction to the Iguazio Data Science Platform and the platform tu
- [Data Science Workflow](#data-science-workflow)
- [The Tutorial Notebooks](#the-tutorial-notebooks)
- [Getting-Started Tutorial](#getting-started-tutorial)
- [End-to-End Use-Case Application and How-To Demos](#end-to-end-use-case-applications)
- [End-to-End Use-Case Application and How-To Demos](#demos)
- [Installing and Updating the MLRun Python Package](#mlrun-python-pkg-install-n-update)
- [Data Ingestion and Preparation](#data-ingestion-and-preparation)
- [Additional Platform Resources](#platform-resources)
Expand Down Expand Up @@ -53,15 +53,16 @@ The home directory of the platform's running-user directory (**/User/<running
> - The **welcome.ipynb** notebook and main **README.md** file provide the same introduction in different formats.
<a id="getting-started-tutorial"></a>

## Getting-Started Tutorial

Start out by running the getting-started tutorial to familiarize yourself with the platform and experience firsthand some of its main capabilities.<br>
<br>
Start out by running the getting-started tutorial to familiarize yourself with the platform and experience firsthand some of its main capabilities.

<a href="demos/getting-started-tutorial/tutorial-1-MLRun-basics.ipynb"><img src="./assets/images/view-tutorial-button.png" alt="View tutorial"/></a>

You can also view the tutorial on [**GitHub**](https://github.com/mlrun/demos/blob/release/v0.6.x-latest/getting-started-tutorial/tutorial-1-MLRun-basics.ipynb)
You can also view the tutorial on [GitHub](https://github.com/mlrun/demos/blob/release/v0.6.x-latest/getting-started-tutorial/tutorial-1-MLRun-basics.ipynb).

<a id="end-to-end-use-case-applications"></a>
<a id="demos"></a>

## End-to-End Use-Case Application and How-To Demos

Expand All @@ -85,6 +86,8 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
!/User/update-demos.sh --help
```

<a id="end-to-end-use-case-applications"></a>

### End-to-End Use-Case Application Demos

<table align="left">
Expand All @@ -95,7 +98,7 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
<th>Description</th>
</tr>
<tr>
<td><b>scikit-learn Demo: Full AutoML Pipeline</b></td>
<td><b>scikit-learn Demo: Full AutoML pipeline</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/scikit-learn-pipeline/sklearn-project.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
Expand All @@ -107,7 +110,7 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
</td>
</tr>
<tr>
<td><b>Image Classification with Distributed Training Demo</b></td>
<td><b>Image-Classification Demo: Image classification with distributed training</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/image-classification-with-distributed-training/horovod-project.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
Expand All @@ -119,7 +122,7 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
</td>
</tr>
<tr>
<td><b>Faces Demo: Real-Time Image Recognition with Deep Learning</b></td>
<td><b>Faces Demo: Real-time image recognition with deep learning</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/realtime-face-recognition/notebooks/face-recognition.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
Expand All @@ -130,7 +133,7 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
</td>
</tr>
<tr>
<td><b>Churn Demo: Real-Time Customer-Churn Prediction</b></td>
<td><b>Churn Demo: Real-time customer-churn prediction</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/customer-churn-prediction/churn-project.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
Expand All @@ -153,7 +156,7 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
</td>
</tr>
<tr>
<td><b>NetOps Demo: Predictive Network Operations / Telemetry</b></td>
<td><b>NetOps Demo: Predictive network operations / telemetry</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/network-operations/project.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
Expand All @@ -164,9 +167,11 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
The demo implements both model training and inference, including model monitoring and concept-drift detection.
</td>
</tr>
</table>
</table>

<a id="howto-demos"></a>

### How-to Demos
### How-To Demos

<table align="left">
<tr align="left" style="border-bottom: 1pt solid black;">
Expand All @@ -176,7 +181,7 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
<th>Description</th>
</tr>
<tr>
<td><b>How-To: Converting Existing ML Code to an MLRun Project</b></td>
<td><b>How-To: Converting existing ML code to an MLRun project</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/howto/converting-to-mlrun/mlrun-code.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
Expand All @@ -188,36 +193,36 @@ For full usage instructions, run the script with the `-h` or `--help` flag:
</td>
</tr>
<tr>
<td><b>How-To: Run a Spark job to read CSV file</b></td>
<td><b>How-To: Running a Spark job for reading a CSV file</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/howto/spark/spark_mlrun_read_csv.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
<td align="center", style="min-width:45px; padding: 10px;">
<a target="_blank" href="https://github.com/mlrun/demos/blob/release/v0.6.x-latest/howto/spark/spark_mlrun_read_csv.ipynb"><img src="./assets/images/GitHub-Mark-32px.png"/><br>View on GitHub</a>
</td>
<td>Run a Spark job which reads a csv file and logs the dataset to MLRun database.
<td>Demonstrates how to run a Spark job that reads a CSV file and logs the data set to an MLRun database.
</td>
</tr>
<tr>
<td><b>How-To: Run a Spark Job to analyze data</b></td>
<td><b>How-To: Running a Spark job for analyzing data</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/howto/spark/spark_mlrun_describe.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
<td align="center", style="min-width:45px; padding: 10px;">
<a target="_blank" href="https://github.com/mlrun/demos/blob/release/v0.6.x-latest/howto/spark/spark_mlrun_describe.ipynb"><img src="./assets/images/GitHub-Mark-32px.png"/><br>View on GitHub</a>
</td>
<td>Create and run a Spark job which generates profile report from an Apache Spark DataFrame (based on pandas_profiling).
<td>Demonstrates how to create and run a Spark job that generates a profile report from an Apache Spark DataFrame based on pandas profiling.
</td>
</tr>
<tr>
<td><b>How-To: Spark Job with Spark Operator</b></td>
<td><b>How-To: Running a Spark Job with Spark Operator</b></td>
<td align="center", style="min-width:45px; padding: 10px;">
<a href="demos/howto/spark/spark_operator.ipynb"><img src="./assets/images/Jupyter-Logo-32px.png"/><br>Open locally</a>
</td>
<td align="center", style="min-width:45px; padding: 10px;">
<a target="_blank" href="https://github.com/mlrun/demos/blob/release/v0.6.x-latest/howto/spark/spark_operator.ipynb"><img src="./assets/images/GitHub-Mark-32px.png"/><br>View on GitHub</a>
</td>
<td>Demonstrates how to use <a target="_blank" href="https://github.com/GoogleCloudPlatform/spark-on-k8s-operator">spark operator</a> for running a Spark job over Kubernetes.
<td>Demonstrates how to use <a target="_blank" href="https://github.com/GoogleCloudPlatform/spark-on-k8s-operator">Spark Operator</a> to run a Spark job over Kubernetes with MLRun.
</td>
</tr>
</table>
Expand Down
6 changes: 3 additions & 3 deletions platform-overview.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@
"\n",
"You can develop and test data science models in the platform's Jupyter Notebook service or in your preferred external editor.\n",
"When your model is ready, you can train it in Jupyter Notebook or by using scalable cluster resources such as Nuclio functions, Dask, Spark ML, or Kubernetes jobs.\n",
"You can find model-training examples in the following platform demos; for more information and download instructions, see [**welcome.ipynb**](welcome.ipynb#end-to-end-use-case-applications) (notebook) or [**README.md**](README.md#end-to-end-use-case-applications) (Markdown):\n",
"You can find model-training examples in the following platform demos; for more information and download instructions, see [**welcome.ipynb**](welcome.ipynb#demos) (notebook) or [**README.md**](README.md#demos) (Markdown):\n",
"\n",
"- The NetOps demo demonstrates predictive infrastructure-monitoring using scikit-learn.\n",
"- The image-classification demo demonstrates image recognition using TensorFlow and Horovod with MLRun.\n",
Expand All @@ -118,7 +118,7 @@
"Data scientists need a simple way to track and view current and historical experiments along with the metadata that is associated with each experiment. This capability is critical for comparing different runs, and eventually helps to determine the best model and configuration for production deployment.\n",
"\n",
"The platform leverages the open-source [MLRun](https://github.com/mlrun/mlrun) library to help tackle these challenges. You can find examples of using MLRun in the [MLRun demos](https://github.com/mlrun/demos/).\n",
"For information about retrieving and updating local copies of the MLRun demos, see [**welcome.ipynb**](welcome.ipynb#end-to-end-use-case-applications) (notebook) or [**README.md**](README.md#end-to-end-use-case-applications) (Markdown)."
"For information about retrieving and updating local copies of the MLRun demos, see [**welcome.ipynb**](welcome.ipynb#demos) (notebook) or [**README.md**](README.md#demos) (Markdown)."
]
},
{
Expand Down Expand Up @@ -185,4 +185,4 @@
},
"nbformat": 4,
"nbformat_minor": 4
}
}
60 changes: 40 additions & 20 deletions welcome.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"- [Data Science Workflow](#data-science-workflow)\n",
"- [The Tutorial Notebooks](#the-tutorial-notebooks)\n",
"- [Getting-Started Tutorial](#getting-started-tutorial)\n",
"- [End-to-End Use-Case Application and How-To Demos](#end-to-end-use-case-applications)\n",
"- [End-to-End Use-Case Application and How-To Demos](#demos)\n",
"- [Installing and Updating the MLRun Python Package](#mlrun-python-pkg-install-n-update)\n",
"- [Data Ingestion and Preparation](#data-ingestion-and-preparation)\n",
"- [Additional Platform Resources](#platform-resources)\n",
Expand Down Expand Up @@ -98,21 +98,27 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id=\"getting-started-tutorial\"></a>\n",
"<a id=\"getting-started-tutorial\"></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Getting-Started Tutorial\n",
"\n",
"Start out by running the getting-started tutorial to familiarize yourself with the platform and experience firsthand some of its main capabilities.<br>\n",
"<br>\n",
"Start out by running the getting-started tutorial to familiarize yourself with the platform and experience firsthand some of its main capabilities.\n",
"\n",
"<a href=\"demos/getting-started-tutorial/tutorial-1-MLRun-basics.ipynb\"><img src=\"./assets/images/view-tutorial-button.png\" alt=\"View tutorial\"/></a>\n",
"\n",
"You can also view the tutorial on [**GitHub**](https://github.com/mlrun/demos/blob/release/v0.6.x-latest/getting-started-tutorial/tutorial-1-MLRun-basics.ipynb)"
"You can also view the tutorial on [GitHub](https://github.com/mlrun/demos/blob/release/v0.6.x-latest/getting-started-tutorial/tutorial-1-MLRun-basics.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id=\"end-to-end-use-case-applications\"></a>"
"<a id=\"demos\"></a>"
]
},
{
Expand Down Expand Up @@ -155,6 +161,13 @@
"!/User/update-demos.sh --help"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id=\"end-to-end-use-case-applications\"></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -174,7 +187,7 @@
" <th>Description</th>\n",
" </tr>\n",
" <tr>\n",
" <td><b>scikit-learn Demo: Full AutoML Pipeline</b></td>\n",
" <td><b>scikit-learn Demo: Full AutoML pipeline</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/scikit-learn-pipeline/sklearn-project.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
Expand All @@ -186,7 +199,7 @@
" </td>\n",
" </tr>\n",
" <tr>\n",
" <td><b>Image Classification with Distributed Training Demo</b></td>\n",
" <td><b>Image-Classification Demo: Image classification with distributed training</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/image-classification-with-distributed-training/horovod-project.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
Expand All @@ -198,7 +211,7 @@
" </td>\n",
" </tr>\n",
" <tr>\n",
" <td><b>Faces Demo: Real-Time Image Recognition with Deep Learning</b></td>\n",
" <td><b>Faces Demo: Real-time image recognition with deep learning</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/realtime-face-recognition/notebooks/face-recognition.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
Expand All @@ -209,7 +222,7 @@
" </td>\n",
" </tr>\n",
" <tr>\n",
" <td><b>Churn Demo: Real-Time Customer-Churn Prediction</b></td>\n",
" <td><b>Churn Demo: Real-time customer-churn prediction</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/customer-churn-prediction/churn-project.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
Expand All @@ -232,7 +245,7 @@
" </td>\n",
" </tr>\n",
" <tr>\n",
" <td><b>NetOps Demo: Predictive Network Operations / Telemetry</b></td>\n",
" <td><b>NetOps Demo: Predictive network operations / telemetry</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/network-operations/project.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
Expand All @@ -243,14 +256,21 @@
" The demo implements both model training and inference, including model monitoring and concept-drift detection.\n",
" </td>\n",
" </tr>\n",
" </table>"
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id=\"howto-demos\"></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### How-to Demos"
"### How-To Demos"
]
},
{
Expand All @@ -265,7 +285,7 @@
" <th>Description</th>\n",
" </tr>\n",
" <tr>\n",
" <td><b>How-To: Converting Existing ML Code to an MLRun Project</b></td>\n",
" <td><b>How-To: Converting existing ML code to an MLRun project</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/howto/converting-to-mlrun/mlrun-code.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
Expand All @@ -277,36 +297,36 @@
" </td>\n",
" </tr>\n",
" <tr>\n",
" <td><b>How-To: Run a Spark job to read CSV file</b></td>\n",
" <td><b>How-To: Running a Spark job for reading a CSV file</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/howto/spark/spark_mlrun_read_csv.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a target=\"_blank\" href=\"https://github.com/mlrun/demos/blob/release/v0.6.x-latest/howto/spark/spark_mlrun_read_csv.ipynb\"><img src=\"./assets/images/GitHub-Mark-32px.png\"/><br>View on GitHub</a>\n",
" </td>\n",
" <td>Run a Spark job which reads a csv file and logs the dataset to MLRun database.\n",
" <td>Demonstrates how to run a Spark job that reads a CSV file and logs the data set to an MLRun database.\n",
" </td>\n",
" </tr>\n",
" <tr>\n",
" <td><b>How-To: Run a Spark Job to analyze data</b></td>\n",
" <td><b>How-To: Running a Spark job for analyzing data</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/howto/spark/spark_mlrun_describe.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a target=\"_blank\" href=\"https://github.com/mlrun/demos/blob/release/v0.6.x-latest/howto/spark/spark_mlrun_describe.ipynb\"><img src=\"./assets/images/GitHub-Mark-32px.png\"/><br>View on GitHub</a>\n",
" </td>\n",
" <td>Create and run a Spark job which generates profile report from an Apache Spark DataFrame (based on pandas_profiling).\n",
" <td>Demonstrates how to create and run a Spark job that generates a profile report from an Apache Spark DataFrame based on pandas profiling.\n",
" </td>\n",
" </tr>\n",
" <tr>\n",
" <td><b>How-To: Spark Job with Spark Operator</b></td>\n",
" <td><b>How-To: Running a Spark Job with Spark Operator</b></td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a href=\"demos/howto/spark/spark_operator.ipynb\"><img src=\"./assets/images/Jupyter-Logo-32px.png\"/><br>Open locally</a>\n",
" </td>\n",
" <td align=\"center\", style=\"min-width:45px; padding: 10px;\">\n",
" <a target=\"_blank\" href=\"https://github.com/mlrun/demos/blob/release/v0.6.x-latest/howto/spark/spark_operator.ipynb\"><img src=\"./assets/images/GitHub-Mark-32px.png\"/><br>View on GitHub</a>\n",
" </td>\n",
" <td>Demonstrates how to use <a target=\"_blank\" href=\"https://github.com/GoogleCloudPlatform/spark-on-k8s-operator\">spark operator</a> for running a Spark job over Kubernetes.\n",
" <td>Demonstrates how to use <a target=\"_blank\" href=\"https://github.com/GoogleCloudPlatform/spark-on-k8s-operator\">Spark Operator</a> to run a Spark job over Kubernetes with MLRun.\n",
" </td>\n",
" </tr>\n",
"</table>"
Expand Down

0 comments on commit 8a224a9

Please sign in to comment.