docs/source/examples/Cluster API.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "80a38ff6-dfe6-4c7a-a378-d16a4fbb0786",
   "metadata": {},
   "source": [
    "# Cluster API\n",
    "\n",
    "IPython Parallel 7 adds a `Cluster` API for starting/stopping clusters.\n",
    "\n",
    "This is the new implementation of `ipcluster`,\n",
    "which can be more easily re-used in Python programs.\n",
    "The `ipcluster` script is\n",
    "\n",
    "Controllers and Engines are started with \"Launchers\",\n",
    "which are objects representing a running process.\n",
    "\n",
    "Each **Cluster** has:\n",
    "\n",
    "- a **cluster id**\n",
    "- a **profile directory**\n",
    "- one **controller**\n",
    "- zero or more **engine sets**\n",
    "  - each of which has one or more **engines**\n",
    "  \n",
    "The combination of `profile_dir` and `cluster_id` uniquely identifies a cluster.\n",
    "You can have many clusters in one profile, but each must have a distinct cluster id.\n",
    "\n",
    "To create a cluster, instantiate a Cluster object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ef0dff26-f6d2-4d79-901b-049debe0c0d1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Cluster(cluster_id='touchy-1624880089-y1md', profile='default')>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import ipyparallel as ipp\n",
    "\n",
    "cluster = ipp.Cluster()\n",
    "cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db2688db-d6e6-4bb2-8fec-cb38fe4d8c59",
   "metadata": {},
   "source": [
    "To start the cluster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "23e385fc-6bc2-44ad-b544-0f39182c1a76",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Cluster(cluster_id='touchy-1623757384-cpbt', profile='default', controller=<running>)>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "await cluster.start_controller()\n",
    "cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8f48261b-6787-424f-a9e7-b24fb04d076d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting 4 engines with <class 'ipyparallel.cluster.launcher.LocalEngineSetLauncher'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Cluster(cluster_id='touchy-1623757384-cpbt', profile='default', controller=<running>, engine_sets=['1623757384-b3pm'])>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "engine_set_id = await cluster.start_engines(n=4)\n",
    "cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af046a0c-281c-4dad-9dc7-582a8cda6b06",
   "metadata": {},
   "source": [
    "As you can see, all methods on the Cluster object are async by default.\n",
    "Every async method also has a `_sync` variant, if you don't want to / can't use asyncio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e239a78a-2907-43e4-80be-eaba13dfac0a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting 2 engines with <class 'ipyparallel.cluster.launcher.LocalEngineSetLauncher'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'1623757385-pe8h'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "engine_set_2 = cluster.start_engines_sync(n=2)\n",
    "engine_set_2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f289aaba-7417-40d3-84ea-88bf83715140",
   "metadata": {},
   "source": [
    "At this point, we have a cluster with a controller and six engines in two groups.\n",
    "\n",
    "There is also a `start_cluster` method that starts the controller and one engine set, for convenience:\n",
    "\n",
    "```python\n",
    "engine_set_id = await cluster.start_cluster(n=4)\n",
    "```\n",
    "\n",
    "We can get a client object connected to the cluster with `connect_client()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fe2dbd68-1dcb-4de9-a45f-757314261b8c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 2, 3, 4, 5]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rc = await cluster.connect_client()\n",
    "rc.wait_for_engines(6)\n",
    "rc.ids"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9da9293-3a44-4f16-8f81-7faaf80f20cd",
   "metadata": {},
   "source": [
    "And we can use our classic `apply_async(...).get_dict()` pattern to get a dict by engine id of hostname, pid for each engine:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a00c2116-269e-4016-a3d3-ec71ae1b093c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: {'host': 'touchy', 'pid': 24774},\n",
       " 1: {'host': 'touchy', 'pid': 24775},\n",
       " 2: {'host': 'touchy', 'pid': 24776},\n",
       " 3: {'host': 'touchy', 'pid': 24762},\n",
       " 4: {'host': 'touchy', 'pid': 24769},\n",
       " 5: {'host': 'touchy', 'pid': 24773}}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def identify():\n",
    "    import os\n",
    "    import socket\n",
    "\n",
    "    return {\"host\": socket.gethostname(), \"pid\": os.getpid()}\n",
    "\n",
    "\n",
    "rc[:].apply_async(identify).get_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e71632f-d2d2-4c1b-b258-54ee639a8409",
   "metadata": {},
   "source": [
    "We can send signals to engine sets by id\n",
    "\n",
    "*(sending signals to just one engine is still a work in progress)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6ff6f7da-38e0-4fea-83af-ff875eebcc76",
   "metadata": {},
   "outputs": [],
   "source": [
    "import signal\n",
    "import time\n",
    "\n",
    "ar = rc[:].apply_async(time.sleep, 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e898c30f-996b-44cd-bfad-84516d27a004",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sending signal 2 to engine(s) 1623757384-b3pm\n",
      "Sending signal 2 to engine(s) 1623757385-pe8h\n"
     ]
    },
    {
     "ename": "CompositeError",
     "evalue": "one or more exceptions from call to method: sleep\n[0:apply]: KeyboardInterrupt: \n[1:apply]: KeyboardInterrupt: \n[2:apply]: KeyboardInterrupt: \n[3:apply]: KeyboardInterrupt: \n.... 2 more exceptions ...",
     "output_type": "error",
     "traceback": [
      "[0:apply]: ",
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)\u001b[0;32m<string>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: ",
      "",
      "[1:apply]: ",
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)\u001b[0;32m<string>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: ",
      "",
      "[2:apply]: ",
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)\u001b[0;32m<string>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: ",
      "",
      "[3:apply]: ",
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)\u001b[0;32m<string>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: ",
      "",
      "... 2 more exceptions ..."
     ]
    }
   ],
   "source": [
    "# oops! I meant 1!\n",
    "\n",
    "await cluster.signal_engines(signal.SIGINT)\n",
    "ar.get()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba286e10-0e9d-41af-9016-ceb8d7943cbe",
   "metadata": {},
   "source": [
    "Now it's time to cleanup. Every `start_` method has a correspinding `stop_method`.\n",
    "\n",
    "We can stop one engine set  at a time with `stop_engines`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "70574035-e97b-4a3b-acfd-ca404bab7e04",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopping engine(s): 1623757385-pe8h\n"
     ]
    }
   ],
   "source": [
    "await cluster.stop_engines(engine_set_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89026a15-60a2-40b5-beff-8b281ae624b3",
   "metadata": {},
   "source": [
    "Or stop the whole cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "81275363-a1db-4d6d-9330-514d2355a219",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopping engine(s): 1623757384-b3pm\n",
      "Stopping controller\n",
      "Controller stopped: {'exit_code': 0, 'pid': 24758}\n"
     ]
    }
   ],
   "source": [
    "await cluster.stop_cluster()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7835fa97-59dc-4146-9bdf-43640bbb1801",
   "metadata": {},
   "source": [
    "## Cluster as a context manager\n",
    "\n",
    "Cluster can also be used as a Context manager,\n",
    "in which case:\n",
    "\n",
    "1. entering the context manager starts the cluster\n",
    "2. the `as` returns a connected client\n",
    "3. the context is only entered when all the engines are fully registered and available\n",
    "4. when the context exits, the cluster is torn down\n",
    "\n",
    "This makes it a lot easier to scope an IPython cluster for the duration of a computation\n",
    "and ensure that it is cleaned up when you are done."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "31bf1cfb-9349-46a1-b412-50678368a017",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting 4 engines with <class 'ipyparallel.cluster.launcher.LocalEngineSetLauncher'>\n",
      "Stopping engine(s): 1623757397-ng0s\n",
      "Stopping controller\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{0: 24989, 1: 24991, 2: 24990, 3: 24992}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "with Cluster(n=4) as rc:\n",
    "    engine_pids = rc[:].apply_async(os.getpid).get_dict()\n",
    "engine_pids"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d2d5f22-8116-45e8-8c82-c24e19f7e0de",
   "metadata": {},
   "source": [
    "It can also be async"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3c0cdb81-4288-4e3a-b25a-7b0b0ce6016a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting 2 engines with <class 'ipyparallel.cluster.launcher.LocalEngineSetLauncher'>\n",
      "Controller stopped: {'exit_code': 0, 'pid': 24988}\n",
      "Stopping engine(s): 1623757400-5fq1\n",
      "Stopping controller\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{0: 25058, 1: 25059}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "async with Cluster(n=2) as rc:\n",
    "    engine_pids = rc[:].apply_async(os.getpid).get_dict()\n",
    "engine_pids"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7537226-dc39-4859-bcfb-16c79a16b951",
   "metadata": {},
   "source": [
    "## Launcher classes\n",
    "\n",
    "IPython's mechanism for launching controllers and engines is called `Launchers`.\n",
    "These are in `ipyparallel.cluster.launcher`.\n",
    "\n",
    "There are two kinds of Launcher:\n",
    "\n",
    "- ControllerLauncher, which starts a controller\n",
    "- EngineSetLauncher, which starts `n` engines\n",
    "\n",
    "You can use abbreviations to access the launchers that ship with IPython parallel,\n",
    "such as 'MPI', 'Local', or 'SGE',\n",
    "or you can pass classes themselves (or their import strings, such as 'mymodule.MyEngineSetLauncher').\n",
    "\n",
    "I'm going to start a cluster with engines using MPI:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b85d2009-d60c-4642-a15c-84c80adcc361",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Controller stopped: {'exit_code': 0, 'pid': 25057}\n",
      "Starting 4 engines with <class 'ipyparallel.cluster.launcher.MPIEngineSetLauncher'>\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "os.environ[\"OMPI_MCA_rmaps_base_oversubscribe\"] = \"1\"\n",
    "\n",
    "cluster = Cluster(n=4, engines='MPI')\n",
    "await cluster.start_cluster()\n",
    "rc = await cluster.connect_client()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "91062c98-3ff2-494f-ae02-6cb96648b6ef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 2, 3]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rc.wait_for_engines(4)\n",
    "rc.ids"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9274020a-a556-4d4c-a67b-170b448d1a91",
   "metadata": {},
   "source": [
    "Now I'm going to run a test with another new feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9a078e90-bdf8-4f8e-aedc-bcabf2c9ba87",
   "metadata": {},
   "outputs": [
    {
     "ename": "TimeoutError",
     "evalue": "Result not ready.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTimeoutError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/ipykernel_24747/824703262.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     13\u001b[0m \u001b[0mar\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrc\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply_async\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0muhoh\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 14\u001b[0;31m \u001b[0mar\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/dev/ip/parallel/ipyparallel/client/asyncresult.py\u001b[0m in \u001b[0;36mget\u001b[0;34m(self, timeout)\u001b[0m\n\u001b[1;32m    227\u001b[0m                 \u001b[0;32mraise\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    228\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 229\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0merror\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTimeoutError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Result not ready.\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    230\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    231\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_check_ready\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTimeoutError\u001b[0m: Result not ready."
     ]
    }
   ],
   "source": [
    "def uhoh():\n",
    "    import time\n",
    "    from mpi4py import MPI\n",
    "\n",
    "    rank = MPI.COMM_WORLD.rank\n",
    "    if rank == 0:\n",
    "        print(\"rank 0: oh no.\")\n",
    "        1 / 0\n",
    "    print(f\"rank {rank}: barrier\")\n",
    "    MPI.COMM_WORLD.barrier()\n",
    "\n",
    "\n",
    "ar = rc[:].apply_async(uhoh)\n",
    "ar.get(timeout=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f69526d7-4c39-422e-8654-76fbac9ce14e",
   "metadata": {},
   "source": [
    "Uh oh! We are stuck in barrier because engine 0 failed.\n",
    "\n",
    "Let's try interrupting and getting the errors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebe3e6d1-10cf-49be-83b6-da8dbaf712b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import signal\n",
    "await cluster.signal_engines(signal.SIGINT)\n",
    "ar.get(timeout=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afd8854b-2138-458d-b2bb-69acdc009b7f",
   "metadata": {},
   "source": [
    "It didn't work! This is because MPI.barrier isn't actually interruptible 😢.\n",
    "\n",
    "We are going to have to resort to more drastic measures, and *restart* the engines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "2ae1cb50-965b-400e-b39f-4567df667da2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopping engine(s): 1623757404-oexv\n",
      "Starting 4 engines with <class 'ipyparallel.cluster.launcher.MPIEngineSetLauncher'>\n"
     ]
    }
   ],
   "source": [
    "await cluster.restart_engines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c676323b-1bf7-4983-8139-5b085c2fdf91",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 2, 3]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "engine set stopped 1623757404-oexv: {'exit_code': -9, 'pid': 25078}\n"
     ]
    }
   ],
   "source": [
    "rc.wait_for_engines(4)\n",
    "rc.ids"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c49fa83-087c-4237-acbe-7d6c8ca208c9",
   "metadata": {},
   "source": [
    "We are now back to having 4 responsive engines.\n",
    "Their IPP engine id may have changed, but I can get back to using them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "7e261d39-0793-4207-813a-24caa4ec9a92",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{4: 0, 5: 2, 6: 3, 7: 1}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_rank():\n",
    "    from mpi4py import MPI\n",
    "\n",
    "    return MPI.COMM_WORLD.rank\n",
    "\n",
    "rank_map = rc[:].apply_async(get_rank).get_dict()\n",
    "rank_map"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a67ca12-67f5-4539-939f-80a6e61b5692",
   "metadata": {},
   "source": [
    "Finally, clean everything up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "060108bb-229b-4829-bf32-0dc7d0db0345",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopping engine(s): 1623757404-oexv\n",
      "Stopping controller\n",
      "Controller stopped: {'exit_code': 0, 'pid': 25076}\n",
      "engine set stopped 1623757404-oexv: {'exit_code': -9, 'pid': 25154}\n"
     ]
    }
   ],
   "source": [
    "await cluster.stop_cluster()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4607e5e6-e585-413f-85aa-281e606f7f82",
   "metadata": {},
   "source": [
    "## Connecting to existing clusters\n",
    "\n",
    "a Cluster object writes its state to disk,\n",
    "in a file accessible as `cluster.cluster_file`.\n",
    "By default, this willb e `$PROFILE_DIR/security/ipcluster-$cluster-id.json`.\n",
    "\n",
    "Cluster objects can load state from a dictionary with `Cluster.from_dict(d)`\n",
    "or from a JSON file containing that information with `Cluster.from_file()`.\n",
    "\n",
    "The default arguments for `from_file` are to use the current IPython profile (default: 'default')\n",
    "and empty cluster id,\n",
    "so if you start a cluster with `ipcluster start`, you can connect to it immediately with\n",
    "\n",
    "```python\n",
    "cluster = ipp.Cluster.from_file()\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0da44ee0-2a3f-4b98-b31a-9c71621b9777",
   "metadata": {},
   "outputs": [],
   "source": [
    "import ipyparallel as ipp\n",
    "cluster = ipp.Cluster.from_file()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0df1b2b2-0d35-45a7-8ad8-c7f55eb52a5f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Cluster(cluster_id='', profile='default', controller=<running>, engine_sets=['1624884556-z9qr'])>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "947edd74-5b1c-42ca-a83a-1755a96c6469",
   "metadata": {},
   "source": [
    "`ipp.ClusterManager` provides an API for collecting/discovering/loading all the clusters on your system.\n",
    "\n",
    "By default, it finds loads clusters in all your IPython profiles,\n",
    "but can be confined to one profile or use explicit profile directories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c8b30824-b833-449c-b1eb-71b56ff47ba6",
   "metadata": {},
   "outputs": [],
   "source": [
    "clusters = ipp.ClusterManager().load_clusters()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6c5e2fe0-c45c-4528-8732-d6bd56580885",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'mpi:abc-123': <Cluster(cluster_id='abc-123', profile='mpi', controller=<running>, engine_sets=['1624884663-euj7'])>,\n",
       " 'default:': <Cluster(cluster_id='', profile='default', controller=<running>, engine_sets=['1624884556-z9qr'])>}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clusters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b75d78c5-5f49-436f-b313-c56085f70e43",
   "metadata": {},
   "source": [
    "This is the class that powers the new `ipcluster list`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "cbf3d3f3-70e1-4c92-b6d5-bf17ea913253",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROFILE          CLUSTER ID                       RUNNING ENGINES LAUNCHER\n",
      "default          ''                               True          4 Local\n",
      "mpi              abc-123                          True          4 MPI\n"
     ]
    }
   ],
   "source": [
    "!ipcluster list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "08a6c241-8d05-45fa-bfe1-ba89f4257ade",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-28 14:53:00.591 [IPClusterStop] Stopping engine(s): 1624884663-euj7\n",
      "2021-06-28 14:53:00.592 [IPClusterStop] Stopping controller\n"
     ]
    }
   ],
   "source": [
    "!ipcluster stop --profile mpi --cluster-id abc-123"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5ec2c3c2-153c-4fbf-b52d-567530533a3c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROFILE          CLUSTER ID                       RUNNING ENGINES LAUNCHER\n",
      "default          ''                               True          4 Local\n"
     ]
    }
   ],
   "source": [
    "!ipcluster list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b3e53cf-4b30-4496-8b84-993daf4fc527",
   "metadata": {},
   "source": [
    "The same operation can be done from the Python API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7be40506-7108-4fbc-8b8f-16e4d478ed84",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopping engine(s): 1624884556-z9qr\n",
      "Stopping controller\n"
     ]
    }
   ],
   "source": [
    "cluster = ipp.Cluster.from_file(profile=\"default\", cluster_id=\"\")\n",
    "await cluster.stop_cluster()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "572b2e45-4b20-4f8d-81b0-9334a42e69e7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROFILE          CLUSTER ID                       RUNNING ENGINES LAUNCHER\n"
     ]
    }
   ],
   "source": [
    "!ipcluster list"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}