getting_started_notebooks/basics/Dask_Hello_World.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Dask: Hello World!\n",
    "#### By Paul Hendricks\n",
    "-------\n",
    "\n",
    "While the world’s data doubles each year, CPU computing has hit a brick wall with the end of Moore’s law. For the same reasons, scientific computing and deep learning has turned to NVIDIA GPU acceleration, data analytics and machine learning where GPU acceleration is ideal. \n",
    "\n",
    "NVIDIA created RAPIDS – an open-source data analytics and machine learning acceleration platform that leverages GPUs to accelerate computations. RAPIDS is based on Python, has pandas-like and Scikit-Learn-like interfaces, is built on Apache Arrow in-memory data format, and can scale from 1 to multi-GPU to multi-nodes. RAPIDS integrates easily into the world’s most popular data science Python-based workflows. RAPIDS accelerates data science end-to-end – from data prep, to machine learning, to deep learning. And through Arrow, Spark users can easily move data into the RAPIDS platform for acceleration.\n",
    "\n",
    "In this notebook, we will show how to quickly setup Dask and run a \"Hello World\" example.\n",
    "\n",
    "**Table of Contents**\n",
    "\n",
    "* Setup\n",
    "* Load Libraries\n",
    "* Setup Dask\n",
    "* Hello World!\n",
    "* Sleeping in Parallel\n",
    "* Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "This notebook was tested using the `nvcr.io/nvidia/rapidsai/rapidsai:0.5-cuda10.0-runtime-ubuntu18.04-gcc7-py3.7` Docker container from [NVIDIA GPU Cloud](https://ngc.nvidia.com) and run on the NVIDIA Tesla V100 GPU. Please be aware that your system may be different and you may need to modify the code or install packages to run the below examples. \n",
    "\n",
    "If you think you have found a bug or an error, please file an issue here: https://github.com/rapidsai/notebooks/issues\n",
    "\n",
    "Before we begin, let's check out our hardware setup by running the `nvidia-smi` command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's see what CUDA version we have:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvcc --version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Lbraries\n",
    "\n",
    "Next, let's load some libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import dask; print('Dask Version:', dask.__version__)\n",
    "from dask.delayed import delayed\n",
    "from dask.distributed import Client\n",
    "import os\n",
    "import subprocess\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Dask\n",
    "\n",
    "Dask is a library the allows for parallelized computing. Written in Python, it allows one to schedule tasks dynamically as well handle large data structures - similar to those found in NumPy and Pandas. In the subsequent tutorials, we'll show how to use Dask with Pandas and cuDF and how we can use both to accelerate common ETL tasks as well as build ML models like XGBoost.\n",
    "\n",
    "To learn more about Dask, check out the documentation here: http://docs.dask.org/en/latest/\n",
    "\n",
    "Dask operates using a concept of a \"Client\" and \"workers\". The client tells the workers what tasks to perform and when to perform. Typically, we set the number of works to be equal to the number of computing resources we have available to us. For example, we might set `n_workers = 8` if we have 8 CPU cores on our machine that can each operate in parallel. This allows us to take advantage of all of our computing resources and enjoy the most benefits from parallelization.\n",
    "\n",
    "Dask is a first class citizen in the world of General Purpose GPU computing and the RAPIDS ecosystem makes it very easy to use Dask with cuDF and XGBoost. As we see below, we can inititate a Cluster and Client using only few lines of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dask_cuda import LocalCUDACluster\n",
    "\n",
    "cluster = LocalCUDACluster()\n",
    "client = Client(cluster)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's show our current Dask status. We should see the IP Address for our Scheduler as well the the number of workers in our Cluster. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# show current Dask status\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also see the status and more information at the Dashboard, found at `http://<scheduler_uri>/status`. You can ignore this for now, we'll dive into this in subsequent tutorials."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hello World\n",
    "\n",
    "Our Dask Client and Dask Workers have been setup. It's time to execute our first program in parallel. We'll define a function that takes some value `x` and adds 5 to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_x_to_5(x):\n",
    "    return x + 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll iterate through our `n_workers` and create an execution graph, where each worker is responsible for taking it's ID and passing it to the function `add_x_to_5`. For example, Dask Worker 2 will result in the value 7.\n",
    "\n",
    "An important thing to note is that the Dask Workers aren't actually executing these results - we're just defining the execution graph for our Dask Client to execute later. The `delayed` function wrapper ensures that this computation is in fact \"delayed\" and not executed on the spot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_workers = 4\n",
    "results_delayed = [delayed(add_x_to_5)(i) for i in range(n_workers)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_delayed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use the Dask Client to compute the results. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results = client.compute(results_delayed, optimize_graph=False, fifo_timeout=\"0ms\")\n",
    "time.sleep(1)  # this will give Dask time to execute each worker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the results are not the \"actual results\" of adding 5 to each of `[0, 1, 2, 3]` - we need to collect and print the results. We can do so by calling the `result()` method for each of our results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print([result.result() for result in results])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sleeping in Parallel\n",
    "\n",
    "To see that Dask is truly executing in parallel, we'll define a function that sleeps for 1 second and returns the string \"Success!\". In serial, this function will take 4 seconds to execute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sleep_1():\n",
    "    time.sleep(1)\n",
    "    return 'Success!'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "for _ in range(n_workers):\n",
    "    sleep_1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using Dask, we see that this whole process takes a little over second - each worker is executing in parallel!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "# define delayed execution graph\n",
    "results_delayed = [delayed(sleep_1)() for _ in range(n_workers)]\n",
    "\n",
    "# use client to perform computations using execution graph\n",
    "results = client.compute(results_delayed, optimize_graph=False, fifo_timeout=\"0ms\")\n",
    "\n",
    "# collect and print results\n",
    "print([result.result() for result in results])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "To learn more about RAPIDS, be sure to check out: \n",
    "\n",
    "* [Open Source Website](http://rapids.ai)\n",
    "* [GitHub](https://github.com/rapidsai/)\n",
    "* [Press Release](https://nvidianews.nvidia.com/news/nvidia-introduces-rapids-open-source-gpu-acceleration-platform-for-large-scale-data-analytics-and-machine-learning)\n",
    "* [NVIDIA Blog](https://blogs.nvidia.com/blog/2018/10/10/rapids-data-science-open-source-community/)\n",
    "* [Developer Blog](https://devblogs.nvidia.com/gpu-accelerated-analytics-rapids/)\n",
    "* [NVIDIA Data Science Webpage](https://www.nvidia.com/en-us/deep-learning-ai/solutions/data-science/)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}