vis-common-plots.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(vis-common-plots)=\n",
    "# Common Plots\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In this chapter, we'll look at some of the most common plots that you might want to make--and how to create them using the most popular data visualisations libraries, including [**matplotlib**](https://matplotlib.org/), [**plotnine**](https://plotnine.readthedocs.io/en/stable/), [**seaborn**](https://seaborn.pydata.org/), [**altair**](https://altair-viz.github.io/), and [**plotly**](https://plotly.com/python/). If you need an introduction to these libraries, see the previous chapter.\n",
    " \n",
    "This chapter has benefited from [**viztech**](https://github.com/cstorm125/viztech), a repository that aims to recreate the entire Financial Times Visual Vocabulary using **plotnine**, from the **plotnine** documentation, from the **matplotlib** documentation, from the **seaborn** documentation, from the **altair** documentation, from the **plotly** documentation, and from examples posted around the web on forums and in blog posts. It's also worth noting that I'm more of an expert in **matplotlib** than anything else--I would really welcome contributions in the form of plots with particular libraries that I have not been able to find or implement myself.\n",
    "\n",
    "Bear in mind that for many of the **matplotlib** examples, using the `df.plot.*` syntax can get the plot you want more quickly! To be more comprehensive, the solution for any kind of data is shown in the examples below.\n",
    "\n",
    "Throughout, we'll assume that the data are in a tidy format (one row per observation, one variable per column). Remember that all Altair plots can be made interactive by adding `.interactive()` at the end.\n",
    "\n",
    "First, though, let's import the libraries we'll need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from plotnine import *\n",
    "import altair as alt\n",
    "import plotly.express as px\n",
    "from vega_datasets import data\n",
    "import os\n",
    "from pathlib import Path\n",
    "import warnings\n",
    "\n",
    "# Set seed for reproducibility\n",
    "np.random.seed(10)\n",
    "# Turn off warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51a55374",
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "import matplotlib_inline.backend_inline\n",
    "\n",
    "# Plot settings\n",
    "plt.style.use(\n",
    "    \"https://github.com/aeturrell/coding-for-economists/raw/main/plot_style.txt\"\n",
    ")\n",
    "matplotlib_inline.backend_inline.set_matplotlib_formats(\"svg\")\n",
    "\n",
    "# Set max rows displayed for readability\n",
    "pd.set_option(\"display.max_rows\", 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scatter plot\n",
    "\n",
    "In this example, we will see a simple scatter plot with several categories using the \"cars\" data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cars = data.cars()\n",
    "cars.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "for origin in cars[\"Origin\"].unique():\n",
    "    cars_sub = cars[cars[\"Origin\"] == origin]\n",
    "    ax.scatter(cars_sub[\"Horsepower\"], cars_sub[\"Miles_per_Gallon\"], label=origin)\n",
    "ax.set_ylabel(\"Miles per Gallon\")\n",
    "ax.set_xlabel(\"Horsepower\")\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "In this first example, I'll also show how to tweak the labels by using the underlying matplolib `Axes` object, here called `ax`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "sns.scatterplot(data=cars, x=\"Horsepower\", y=\"Miles_per_Gallon\", hue=\"Origin\", ax=ax)\n",
    "ax.set_ylabel(\"Miles per Gallon\")\n",
    "ax.set_xlabel(\"Horsepower\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(cars, aes(x=\"Horsepower\", y=\"Miles_per_Gallon\", color=\"Origin\"))\n",
    "    + geom_point()\n",
    "    + ylab(\"Miles per Gallon\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n",
    "For this first example, we'll also show how to make the altair plot interactive with movable axes and more info on mouse-hover."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(cars).mark_circle(size=60).encode(\n",
    "    x=\"Horsepower\",\n",
    "    y=\"Miles_per_Gallon\",\n",
    "    color=\"Origin\",\n",
    "    tooltip=[\"Name\", \"Origin\", \"Horsepower\", \"Miles_per_Gallon\"],\n",
    ").interactive()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n",
    "Plotly is another declarative plotting library, at least sometimes (!), but one that is interactive by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(\n",
    "    cars,\n",
    "    x=\"Horsepower\",\n",
    "    y=\"Miles_per_Gallon\",\n",
    "    color=\"Origin\",\n",
    "    hover_data=[\"Name\", \"Origin\", \"Horsepower\", \"Miles_per_Gallon\"],\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Facets\n",
    "\n",
    "This applies to all plots, so in some sense is common! Facets, aka panels or small multiples, are ways of showing the same chart multiple times. Let's see how to achieve them in a few of the most popular plotting libraries.\n",
    "\n",
    "We'll use the \"tips\" dataset for this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = sns.load_dataset(\"tips\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "There are many ways to create facets using Matplotlib, and you can get facets in any shape or sizes you like. \n",
    "\n",
    "The easiest way, though, is to specify the number of rows and columns. This is achieved by specifying `nrows` and `ncols` when calling `plt.subplots`. It returns an array of shape `(nrows, ncols)` of `Axes` objects. For most purposes, you'll want to flatten these to a vector before iterating over them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(nrows=1, ncols=4, sharex=True, sharey=True)\n",
    "flat_axes = axes.flatten()  # Not needed with 1 row or 1 col, but good to be aware of\n",
    "\n",
    "facet_grp = list(df[\"day\"].unique())\n",
    "# This part just to get some colours from the default color cycle\n",
    "cycle = axes[0]._get_lines.prop_cycler\n",
    "for i, ax in enumerate(flat_axes):\n",
    "    sub_df = df.loc[df[\"day\"] == facet_grp[i]]\n",
    "    ax.scatter(\n",
    "        sub_df[\"tip\"],\n",
    "        sub_df[\"total_bill\"],\n",
    "        s=30,\n",
    "        edgecolor=\"k\",\n",
    "        color=next(cycle)[\"color\"],\n",
    "    )\n",
    "    ax.set_title(facet_grp[i])\n",
    "fig.text(0.5, 0.01, \"Tip\", ha=\"center\")\n",
    "fig.text(0.0, 0.5, \"Total bill\", va=\"center\", rotation=\"vertical\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different facet sizes are possible in numerous ways. In practice, it's often better to have evenly sized facets laid out in a grid--especially each facet is of the same x and y axes. But, just to show it's possible, here's an example that gives more space to the weekend than to weekdays using the tips dataset: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This part just to get some colours\n",
    "colormap = plt.cm.Dark2\n",
    "\n",
    "fig = plt.figure(constrained_layout=True)\n",
    "ax_dict = fig.subplot_mosaic([[\"Thur\", \"Fri\", \"Sat\", \"Sat\", \"Sun\", \"Sun\"]])\n",
    "facet_grp = list(ax_dict.keys())\n",
    "colorst = [colormap(i) for i in np.linspace(0, 0.9, len(facet_grp))]\n",
    "for i, grp in enumerate(facet_grp):\n",
    "    sub_df = df.loc[df[\"day\"] == facet_grp[i]]\n",
    "    ax_dict[grp].scatter(\n",
    "        sub_df[\"tip\"],\n",
    "        sub_df[\"total_bill\"],\n",
    "        s=30,\n",
    "        edgecolor=\"k\",\n",
    "        color=colorst[i],\n",
    "    )\n",
    "    ax_dict[grp].set_title(facet_grp[i])\n",
    "    if grp != \"Thurs\":\n",
    "        ax_dict[grp].set_yticklabels([])\n",
    "plt.tight_layout()\n",
    "fig.text(0.5, 0, \"Tip\", ha=\"center\")\n",
    "fig.text(0, 0.5, \"Total bill\", va=\"center\", rotation=\"vertical\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As well as using lists, you can also specify the layout using an array or using text, eg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "axd = plt.figure(constrained_layout=True).subplot_mosaic(\n",
    "    \"\"\"\n",
    "    ABD\n",
    "    CCD\n",
    "    CC.\n",
    "    \"\"\"\n",
    ")\n",
    "kw = dict(ha=\"center\", va=\"center\", fontsize=60, color=\"darkgrey\")\n",
    "for k, ax in axd.items():\n",
    "    ax.text(0.5, 0.5, k, transform=ax.transAxes, **kw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "Seaborn makes it easy to quickly create facet plots. Note the use of `col_wrap`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.relplot(\n",
    "    data=df,\n",
    "    y=\"total_bill\",\n",
    "    x=\"tip\",\n",
    "    col=\"day\",\n",
    "    hue=\"day\",\n",
    "    col_wrap=2,\n",
    "    kind=\"scatter\",\n",
    "    height=3,\n",
    "    aspect=1.25,\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A nice feature of seaborn that is much more fiddly in (base) matplotlib is the ability to specify rows and columns separately: (smoker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.relplot(\n",
    "    data=df,\n",
    "    y=\"total_bill\",\n",
    "    x=\"tip\",\n",
    "    col=\"day\",\n",
    "    row=\"smoker\",\n",
    "    hue=\"smoker\",\n",
    "    kind=\"scatter\",\n",
    "    height=2.5,\n",
    "    aspect=1,\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n",
    "Plotnine has several ways to wrap facets but perhaps the most delightful is to specify a formula for the variations to be encoded in the facets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(df, aes(x=\"tip\", y=\"total_bill\", color=\"smoker\"))\n",
    "    + geom_point()\n",
    "    + facet_wrap(\n",
    "        \"~ smoker + day\", nrow=2\n",
    "    )  # use ~ + to add additional faceting variables\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(df).mark_point().encode(\n",
    "    x=\"tip:Q\",\n",
    "    y=\"total_bill:Q\",\n",
    "    color=\"smoker:N\",\n",
    "    facet=alt.Facet(\"day:N\", columns=2),\n",
    ").properties(\n",
    "    width=200,\n",
    "    height=100,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(\n",
    "    df, x=\"tip\", y=\"total_bill\", color=\"smoker\", facet_row=\"smoker\", facet_col=\"day\"\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connected scatter plot\n",
    "\n",
    "A simple variation on the scatter plot designed to show an ordering, usually of time. We'll trace out a Beveridge curve based on US data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas_datareader.data as web\n",
    "import datetime\n",
    "\n",
    "start = datetime.datetime(2000, 1, 1)\n",
    "end = datetime.datetime(2021, 1, 1)\n",
    "code_dict = {\n",
    "    \"Vacancies\": \"LMJVTTUVUSA647N\",\n",
    "    \"Unemployment\": \"UNRATE\",\n",
    "    \"LabourForce\": \"CLF16OV\",\n",
    "}\n",
    "list_dfs = [\n",
    "    web.DataReader(value, \"fred\", start, end)\n",
    "    .rename(columns={value: key})\n",
    "    .groupby(pd.Grouper(freq=\"AS\"))\n",
    "    .mean()\n",
    "    for key, value in code_dict.items()\n",
    "]\n",
    "df = pd.concat(list_dfs, axis=1)\n",
    "df = df.assign(Vacancies=100 * df[\"Vacancies\"] / (df[\"LabourForce\"] * 1e3)).dropna()\n",
    "df[\"Year\"] = df.index.year\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.close(\"all\")\n",
    "fig, ax = plt.subplots()\n",
    "quivx = -df[\"Unemployment\"].diff(-1)\n",
    "quivy = -df[\"Vacancies\"].diff(-1)\n",
    "# This connects the points\n",
    "ax.quiver(\n",
    "    df[\"Unemployment\"],\n",
    "    df[\"Vacancies\"],\n",
    "    quivx,\n",
    "    quivy,\n",
    "    scale_units=\"xy\",\n",
    "    angles=\"xy\",\n",
    "    scale=1,\n",
    "    width=0.006,\n",
    "    alpha=0.3,\n",
    ")\n",
    "ax.scatter(\n",
    "    df[\"Unemployment\"],\n",
    "    df[\"Vacancies\"],\n",
    "    marker=\"o\",\n",
    "    s=35,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.2,\n",
    "    alpha=0.9,\n",
    ")\n",
    "for j in [0, -1]:\n",
    "    ax.annotate(\n",
    "        df[\"Year\"].iloc[j],\n",
    "        xy=(df[[\"Unemployment\", \"Vacancies\"]].iloc[j].tolist()),\n",
    "        xycoords=\"data\",\n",
    "        xytext=(-20, -40),\n",
    "        textcoords=\"offset points\",\n",
    "        arrowprops=dict(arrowstyle=\"->\", connectionstyle=\"angle3,angleA=0,angleB=-90\"),\n",
    "    )\n",
    "ax.set_xlabel(\"Unemployment rate, %\")\n",
    "ax.set_ylabel(\"Vacancy rate, %\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "sns.lineplot(data=df, x=\"Unemployment\", y=\"Vacancies\", sort=False, marker=\"o\", ax=ax)\n",
    "ax.set_ylabel(\"Vacancy rate, %\")\n",
    "ax.set_xlabel(\"Unemployment rate, %\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = (\n",
    "    ggplot(\n",
    "        df.reset_index()[[\"Vacancies\", \"Unemployment\", \"Year\"]],\n",
    "        aes(x=\"Unemployment\", y=\"Vacancies\"),\n",
    "    )\n",
    "    + geom_point(color=\"brown\")\n",
    "    + geom_path()\n",
    "    + geom_label(\n",
    "        aes(x=df.iloc[0, 1], y=df.iloc[0, 0]), label=df.iloc[0, -1], nudge_y=0.01\n",
    "    )\n",
    "    + geom_label(\n",
    "        aes(x=df.iloc[-1, 1], y=df.iloc[-1, 0]), label=df.iloc[-1, -1], nudge_y=0.01\n",
    "    )\n",
    ")\n",
    "g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bubble plot\n",
    "\n",
    "This is a scatter plot where the size of the point carries an extra dimension of information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "scat = ax.scatter(\n",
    "    cars[\"Horsepower\"], cars[\"Miles_per_Gallon\"], s=cars[\"Displacement\"], alpha=0.4\n",
    ")\n",
    "ax.set_ylabel(\"Miles per Gallon\")\n",
    "ax.set_xlabel(\"Horsepower\")\n",
    "ax.legend(\n",
    "    *scat.legend_elements(prop=\"sizes\", num=4),\n",
    "    loc=\"upper right\",\n",
    "    title=\"Displacement\",\n",
    "    frameon=False,\n",
    ")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.scatterplot(data=cars, x=\"Horsepower\", y=\"Miles_per_Gallon\", size=\"Displacement\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(cars, aes(x=\"Horsepower\", y=\"Miles_per_Gallon\", size=\"Displacement\"))\n",
    "    + geom_point()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(cars).mark_circle().encode(\n",
    "    x=\"Horsepower\", y=\"Miles_per_Gallon\", size=\"Displacement\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adding a new col is easiest way to get displacement into legend with plotly:\n",
    "cars[\"Displacement_Size\"] = pd.cut(cars[\"Displacement\"], bins=4)\n",
    "fig = px.scatter(\n",
    "    cars,\n",
    "    x=\"Horsepower\",\n",
    "    y=\"Miles_per_Gallon\",\n",
    "    size=\"Displacement\",\n",
    "    color=\"Displacement_Size\",\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Line plot\n",
    "\n",
    "First, let's get some data on GDP growth:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas_datareader.data as web\n",
    "\n",
    "ts_start_date = pd.to_datetime(\"1999-01-01\")\n",
    "\n",
    "df = pd.concat(\n",
    "    [\n",
    "        web.DataReader(\"ticker=RGDP\" + x, \"econdb\", start=ts_start_date)\n",
    "        for x in [\"US\", \"UK\"]\n",
    "    ],\n",
    "    axis=1,\n",
    ")\n",
    "df.columns = [\"US\", \"UK\"]\n",
    "df.index.name = \"Date\"\n",
    "df = 100 * df.pct_change(4)\n",
    "df = pd.melt(\n",
    "    df.reset_index(),\n",
    "    id_vars=[\"Date\"],\n",
    "    value_vars=df.columns,\n",
    "    value_name=\"Real GDP growth, %\",\n",
    "    var_name=\"Country\",\n",
    ")\n",
    "df = df.set_index(\"Date\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "Note that **Matplotlib** prefers data to be one variable per column, in which case we could have just run\n",
    "\n",
    "```python\n",
    "fig, ax = plt.subplots()\n",
    "df.plot(ax=ax)\n",
    "ax.set_title('Real GDP growth, %', loc='right')\n",
    "ax.yaxis.tick_right()\n",
    "```\n",
    "\n",
    "but we are working with tidy data here, so we'll do the plotting slightly differently."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "for i, country in enumerate(df[\"Country\"].unique()):\n",
    "    df_sub = df[df[\"Country\"] == country]\n",
    "    ax.plot(df_sub.index, df_sub[\"Real GDP growth, %\"], label=country, lw=2)\n",
    "ax.set_title(\"Real GDP growth, %\", loc=\"right\")\n",
    "ax.yaxis.tick_right()\n",
    "ax.spines[\"right\"].set_visible(True)\n",
    "ax.spines[\"left\"].set_visible(False)\n",
    "ax.legend(loc=\"lower left\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "Note that [only *some* **seaborn** commands currently support the use of named indexes](https://seaborn.pydata.org/tutorial/data_structure.html), so we use `df.reset_index()` to make the 'Date' index into a regular column in the snippet below (although in recent versions of **seaborn**, `lineplot` would actually work fine with `data=df`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "y_var = \"Real GDP growth, %\"\n",
    "sns.lineplot(x=\"Date\", y=y_var, hue=\"Country\", data=df.reset_index(), ax=ax)\n",
    "ax.yaxis.tick_right()\n",
    "ax.spines[\"right\"].set_visible(True)\n",
    "ax.spines[\"left\"].set_visible(False)\n",
    "ax.set_ylabel(\"\")\n",
    "ax.set_title(y_var)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(df.reset_index(), aes(x=\"Date\", y=\"Real GDP growth, %\", color=\"Country\"))\n",
    "    + geom_line()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(df.reset_index()).mark_line().encode(\n",
    "    x=\"Date:T\",\n",
    "    y=\"Real GDP growth, %\",\n",
    "    color=\"Country\",\n",
    "    strokeDash=\"Country\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.line(\n",
    "    df.reset_index(),\n",
    "    x=\"Date\",\n",
    "    y=\"Real GDP growth, %\",\n",
    "    color=\"Country\",\n",
    "    line_dash=\"Country\",\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bar chart\n",
    "\n",
    "Let's see a bar chart, using the 'barley' dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "barley = data.barley()\n",
    "barley = pd.DataFrame(barley.groupby([\"site\"])[\"yield\"].sum())\n",
    "barley.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "Just remove the 'h' in `ax.barh` to get a vertical plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.barh(barley[\"yield\"].index, barley[\"yield\"], 0.35)\n",
    "ax.set_xlabel(\"Yield\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "Just switch x and y variables to get a vertical plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.catplot(\n",
    "    data=barley.reset_index(),\n",
    "    kind=\"bar\",\n",
    "    y=\"site\",\n",
    "    x=\"yield\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n",
    "Just omit `coord_flip()` to get a vertical plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ggplot(barley.reset_index(), aes(x=\"site\", y=\"yield\")) + geom_col() + coord_flip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n",
    "Just switch x and y to get a vertical plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(barley.reset_index()).mark_bar().encode(y=\"site\", x=\"yield\",).properties(\n",
    "    width=alt.Step(40)  # controls width of bar.\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.bar(barley.reset_index(), y=\"site\", x=\"yield\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Grouped bar chart\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "barley = data.barley()\n",
    "barley = pd.DataFrame(barley.groupby([\"site\", \"year\"])[\"yield\"].sum()).reset_index()\n",
    "barley.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = barley[\"site\"].unique()\n",
    "y = np.arange(len(labels))  # the label locations\n",
    "width = 0.35  # the width of the bars\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "ax.barh(y - width / 2, barley.loc[barley[\"year\"] == 1931, \"yield\"], width, label=\"1931\")\n",
    "ax.barh(y + width / 2, barley.loc[barley[\"year\"] == 1932, \"yield\"], width, label=\"1932\")\n",
    "\n",
    "# Add some text for labels, title and custom x-axis tick labels, etc.\n",
    "ax.set_xlabel(\"Yield\")\n",
    "ax.set_yticks(y)\n",
    "ax.set_yticklabels(labels)\n",
    "ax.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.catplot(data=barley, kind=\"bar\", y=\"site\", x=\"yield\", hue=\"year\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(barley, aes(x=\"site\", y=\"yield\", fill=\"factor(year)\"))\n",
    "    + geom_col(position=\"dodge\")\n",
    "    + coord_flip()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(barley.reset_index()).mark_bar().encode(\n",
    "    y=\"year:O\", x=\"yield\", color=\"year:N\", row=\"site:N\"\n",
    ").properties(\n",
    "    width=alt.Step(40)  # controls width of bar.\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "px_barley = barley.reset_index()\n",
    "# This prevents plotly from using a continuous scale for year\n",
    "px_barley[\"year\"] = px_barley[\"year\"].astype(\"category\")\n",
    "fig = px.bar(px_barley, y=\"site\", x=\"yield\", barmode=\"group\", color=\"year\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacked bar chart\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = barley[\"site\"].unique()\n",
    "y = np.arange(len(labels))  # the label locations\n",
    "width = 0.35  # the width (or height) of the bars\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "ax.barh(y, barley.loc[barley[\"year\"] == 1931, \"yield\"], width, label=\"1931\")\n",
    "ax.barh(\n",
    "    y,\n",
    "    barley.loc[barley[\"year\"] == 1932, \"yield\"],\n",
    "    width,\n",
    "    label=\"1932\",\n",
    "    left=barley.loc[barley[\"year\"] == 1931, \"yield\"],\n",
    ")\n",
    "\n",
    "# Add some text for labels, title and custom x-axis tick labels, etc.\n",
    "ax.set_xlabel(\"Yield\")\n",
    "ax.set_yticks(y)\n",
    "ax.set_yticklabels(labels)\n",
    "ax.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "As far as I know, there's no easy way of doing this, though some [inspiration is available](https://seaborn.pydata.org/examples/part_whole_bars.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(barley, aes(x=\"site\", y=\"yield\", fill=\"factor(year)\"))\n",
    "    + geom_col()\n",
    "    + coord_flip()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(barley.reset_index()).mark_bar().encode(\n",
    "    y=\"site\",\n",
    "    x=\"yield\",\n",
    "    color=\"year:N\",\n",
    ").properties(\n",
    "    width=alt.Step(40)  # controls width of bar.\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.bar(px_barley, y=\"site\", x=\"yield\", barmode=\"relative\", color=\"year\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kernel density estimate\n",
    "\n",
    "We'll use the diamonds dataset to demonstrate this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds = sns.load_dataset(\"diamonds\").sample(1000)\n",
    "diamonds.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "Technically, there is a way to do this but it's pretty inelegant if you want a quick plot. That's because **matplotlib** doesn't do the density estimation itself. [Jake Vanderplas](https://jakevdp.github.io/PythonDataScienceHandbook/05.13-kernel-density-estimation.html) has a nice example but as it relies on a few extra libraries, I won't reproduce it here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.displot(diamonds, x=\"carat\", kind=\"kde\", hue=\"cut\", fill=True);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ggplot(diamonds, aes(x=\"carat\", fill=\"cut\", colour=\"cut\")) + geom_density(alpha=0.5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(diamonds).transform_density(\n",
    "    density=\"carat\", as_=[\"carat\", \"density\"], groupby=[\"cut\"]\n",
    ").mark_area(fillOpacity=0.5).encode(\n",
    "    x=\"carat:Q\",\n",
    "    y=\"density:Q\",\n",
    "    color=\"cut:N\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.figure_factory as ff\n",
    "\n",
    "px_di = diamonds.pivot(columns=\"cut\", values=\"carat\")\n",
    "ff.create_distplot(\n",
    "    [px_di[c].dropna() for c in px_di.columns],\n",
    "    group_labels=px_di.columns,\n",
    "    show_rug=False,\n",
    "    show_hist=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Histogram or probability density function\n",
    "\n",
    "For this, let's go back to the penguins dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "penguins = sns.load_dataset(\"penguins\")\n",
    "penguins.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "The `density=` keyword parameter decides whether to create counts or a probability density function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.hist(penguins[\"flipper_length_mm\"], bins=30, density=True, edgecolor=\"k\")\n",
    "ax.set_xlabel(\"Flipper length (mm)\")\n",
    "ax.set_ylabel(\"Probability density\")\n",
    "fig.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.histplot(data=penguins, x=\"flipper_length_mm\", bins=30, stat=\"density\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(penguins, aes(x=\"flipper_length_mm\", y=\"stat(density)\"))\n",
    "    + geom_histogram(bins=30)  # specify the binwidth\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(penguins).mark_bar().encode(\n",
    "    alt.X(\"flipper_length_mm:Q\", bin=True),\n",
    "    y=\"count()\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.histogram(penguins, x=\"flipper_length_mm\", nbins=30)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Marginal histograms\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Maplotlib\n",
    "\n",
    "[Jaker Vanderplas's excellent notes](https://jakevdp.github.io/PythonDataScienceHandbook/04.08-multiple-subplots.html) have a great example of this, but now there's an easier way to do it with Matplotlib's new `constrained_layout` options."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(constrained_layout=True)\n",
    "# Create a layout with 3 panels in the given ratios\n",
    "axes_dict = fig.subplot_mosaic(\n",
    "    [[\".\", \"histx\"], [\"histy\", \"scat\"]],\n",
    "    gridspec_kw={\"width_ratios\": [1, 7], \"height_ratios\": [2, 7]},\n",
    ")\n",
    "# Glue all the relevant axes together\n",
    "axes_dict[\"histy\"].invert_xaxis()\n",
    "axes_dict[\"histx\"].sharex(axes_dict[\"scat\"])\n",
    "axes_dict[\"histy\"].sharey(axes_dict[\"scat\"])\n",
    "# Plot the data\n",
    "axes_dict[\"scat\"].scatter(penguins[\"bill_length_mm\"], penguins[\"bill_depth_mm\"])\n",
    "axes_dict[\"histx\"].hist(penguins[\"bill_length_mm\"])\n",
    "axes_dict[\"histy\"].hist(penguins[\"bill_depth_mm\"], orientation=\"horizontal\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.jointplot(data=penguins, x=\"bill_length_mm\", y=\"bill_depth_mm\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n",
    "I couldn't find an easy way to do this in plotnine but you can make rug plots, which have some similarities in terms of information conveyed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(penguins, aes(x=\"bill_length_mm\", y=\"bill_depth_mm\"))\n",
    "    + geom_point()\n",
    "    + geom_rug()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n",
    "This is a bit fiddly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base = alt.Chart(penguins)\n",
    "\n",
    "xscale = alt.Scale(domain=(20, 60))\n",
    "yscale = alt.Scale(domain=(10, 30))\n",
    "\n",
    "area_args = {\"opacity\": 0.5, \"interpolate\": \"step\"}\n",
    "\n",
    "points = base.mark_circle().encode(\n",
    "    alt.X(\"bill_length_mm\", scale=xscale), alt.Y(\"bill_depth_mm\", scale=yscale)\n",
    ")\n",
    "\n",
    "top_hist = (\n",
    "    base.mark_area(**area_args)\n",
    "    .encode(\n",
    "        alt.X(\n",
    "            \"bill_length_mm:Q\",\n",
    "            # when using bins, the axis scale is set through\n",
    "            # the bin extent, so we do not specify the scale here\n",
    "            # (which would be ignored anyway)\n",
    "            bin=alt.Bin(maxbins=30, extent=xscale.domain),\n",
    "            stack=None,\n",
    "            title=\"\",\n",
    "        ),\n",
    "        alt.Y(\"count()\", stack=None, title=\"\"),\n",
    "    )\n",
    "    .properties(height=60)\n",
    ")\n",
    "\n",
    "right_hist = (\n",
    "    base.mark_area(**area_args)\n",
    "    .encode(\n",
    "        alt.Y(\n",
    "            \"bill_depth_mm:Q\",\n",
    "            bin=alt.Bin(maxbins=30, extent=yscale.domain),\n",
    "            stack=None,\n",
    "            title=\"\",\n",
    "        ),\n",
    "        alt.X(\"count()\", stack=None, title=\"\"),\n",
    "    )\n",
    "    .properties(width=60)\n",
    ")\n",
    "\n",
    "top_hist & (points | right_hist)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(\n",
    "    penguins,\n",
    "    x=\"bill_length_mm\",\n",
    "    y=\"bill_depth_mm\",\n",
    "    marginal_x=\"histogram\",\n",
    "    marginal_y=\"histogram\",\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Heatmap\n",
    "\n",
    "Heatmaps, or sometimes known as correlation maps, represent data in 3 dimensions by having two axes that forms a grid showing colour that corresponds to (usually) continuous values.\n",
    "\n",
    "We'll use the flights data to show the number of passengers by month-year:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights = sns.load_dataset(\"flights\")\n",
    "flights = flights.pivot(\"month\", \"year\", \"passengers\").T\n",
    "flights.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "im = ax.imshow(flights.values, cmap=\"inferno\")\n",
    "cbar = ax.figure.colorbar(im, ax=ax)\n",
    "ax.set_xticks(np.arange(len(flights.columns)))\n",
    "ax.set_yticks(np.arange(len(flights.index)))\n",
    "# Labels\n",
    "ax.set_xticklabels(flights.columns, rotation=90)\n",
    "ax.set_yticklabels(flights.index)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.heatmap(flights);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n",
    "Plotnine uses tidy data, rather than the wide data preferred by **matplotlib**, so we need to first get the original format of the flights data back:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights = sns.load_dataset(\"flights\")\n",
    "(\n",
    "    ggplot(flights, aes(\"month\", \"factor(year)\", fill=\"passengers\"))\n",
    "    + geom_tile()\n",
    "    + scale_y_reverse()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(flights).mark_rect().encode(\n",
    "    x=alt.X(\"month\", type=\"nominal\", sort=None), y=\"year:O\", color=\"passengers:Q\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.imshow(\n",
    "    flights.values,\n",
    "    labels=dict(x=\"Month\", y=\"Year\", color=\"Number\"),\n",
    "    x=flights.columns,\n",
    "    y=flights.index,\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boxplot\n",
    "\n",
    "Let's use the tips dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tips = sns.load_dataset(\"tips\")\n",
    "tips.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "There isn't a very direct way to create multiple box plots of different data in matplotlib in the case where the groups are unbalanced, so we create several different boxplot objects.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "colormap = plt.cm.Set1\n",
    "colorst = [colormap(i) for i in np.linspace(0, 0.9, len(tips[\"time\"].unique()))]\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "for i, grp in enumerate(tips[\"time\"].unique()):\n",
    "    bplot = ax.boxplot(\n",
    "        tips.loc[tips[\"time\"] == grp, \"tip\"],\n",
    "        positions=[i],\n",
    "        vert=True,  # vertical box alignment\n",
    "        patch_artist=True,  # fill with color\n",
    "        labels=[grp],\n",
    "    )  # X label\n",
    "    for patch in bplot[\"boxes\"]:\n",
    "        patch.set_facecolor(colorst[i])\n",
    "\n",
    "ax.set_ylabel(\"Tip\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.boxplot(data=tips, x=\"time\", y=\"tip\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ggplot(tips) + geom_boxplot(aes(y=\"tip\", x=\"time\", fill=\"time\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(tips).mark_boxplot(size=50).encode(\n",
    "    x=\"time:N\", y=\"tip:Q\", color=\"time:N\"\n",
    ").properties(width=300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.box(tips, x=\"time\", y=\"tip\", color=\"time\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Violin plot\n",
    "\n",
    "We'll use the same data as before, the tips dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "colormap = plt.cm.Set1\n",
    "colorst = [colormap(i) for i in np.linspace(0, 0.9, len(tips[\"time\"].unique()))]\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "for i, grp in enumerate(tips[\"time\"].unique()):\n",
    "    vplot = ax.violinplot(\n",
    "        tips.loc[tips[\"time\"] == grp, \"tip\"], positions=[i], vert=True\n",
    "    )\n",
    "labels = list(tips[\"time\"].unique())\n",
    "ax.set_xticks(np.arange(len(labels)))\n",
    "ax.set_xticklabels(labels)\n",
    "ax.set_ylabel(\"Tip\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.violinplot(data=tips, x=\"time\", y=\"tip\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ggplot(tips, aes(x=\"time\", y=\"tip\", fill=\"time\")) + geom_violin())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(tips).transform_density(\n",
    "    \"tip\", as_=[\"tip\", \"density\"], groupby=[\"time\"]\n",
    ").mark_area(orient=\"horizontal\").encode(\n",
    "    y=\"tip:Q\",\n",
    "    color=\"time:N\",\n",
    "    x=alt.X(\n",
    "        \"density:Q\",\n",
    "        stack=\"center\",\n",
    "        impute=None,\n",
    "        title=None,\n",
    "        axis=alt.Axis(labels=False, values=[0], grid=False, ticks=True),\n",
    "    ),\n",
    "    column=alt.Column(\n",
    "        \"time:N\",\n",
    "        header=alt.Header(\n",
    "            titleOrient=\"bottom\",\n",
    "            labelOrient=\"bottom\",\n",
    "            labelPadding=0,\n",
    "        ),\n",
    "    ),\n",
    ").properties(\n",
    "    width=100\n",
    ").configure_facet(\n",
    "    spacing=0\n",
    ").configure_view(\n",
    "    stroke=None\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.violin(\n",
    "    tips,\n",
    "    y=\"tip\",\n",
    "    x=\"time\",\n",
    "    color=\"time\",\n",
    "    box=True,\n",
    "    points=\"all\",\n",
    "    hover_data=tips.columns,\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lollipop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = sns.load_dataset(\"planets\").groupby(\"year\")[\"number\"].count()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.stem(df.index, df)\n",
    "ax.yaxis.tick_right()\n",
    "ax.spines[\"left\"].set_visible(False)\n",
    "ax.set_ylim(0, 200)\n",
    "ax.set_title(\"Number of exoplanets discovered per year\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(df.reset_index(), aes(x=\"year\", y=\"number\"))\n",
    "    + geom_point()\n",
    "    + geom_segment(aes(x=\"year\", xend=\"year\", y=0, yend=\"number\"))\n",
    "    + ggtitle(\"Number of exoplanets discovered per year\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "px_df = df.reset_index()\n",
    "\n",
    "fig1 = go.Figure()\n",
    "# Draw points\n",
    "fig1.add_trace(\n",
    "    go.Scatter(\n",
    "        x=px_df[\"year\"],\n",
    "        y=px_df[\"number\"],\n",
    "        mode=\"markers\",\n",
    "        marker_color=\"darkblue\",\n",
    "        marker_size=10,\n",
    "    )\n",
    ")\n",
    "# Draw lines\n",
    "for index, row in px_df.iterrows():\n",
    "    fig1.add_shape(type=\"line\", x0=row[\"year\"], y0=0, x1=row[\"year\"], y1=row[\"number\"])\n",
    "fig1.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overlapping Area plot\n",
    "\n",
    "For this, let's look at the dominance of the three most used methods for detecting exoplanets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = sns.load_dataset(\"planets\")\n",
    "most_pop_methods = (\n",
    "    df.groupby([\"method\"])[\"number\"].sum().sort_values(ascending=False).index[:3].values\n",
    ")\n",
    "df = df[df[\"method\"].isin(most_pop_methods)]\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "The easiest way to do this in matplotlib is to adjust the data a bit first and then use the built-in **pandas** plot function. (This is true in other cases too, but in this case it's much more complex otherwise)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    df.groupby([\"year\", \"method\"])[\"number\"]\n",
    "    .sum()\n",
    "    .unstack()\n",
    "    .plot.area(alpha=0.6, ylim=(0, None))\n",
    "    .set_title(\"Planets dicovered by top 3 methods\", loc=\"left\")\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "I couldn't find an option for this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(\n",
    "        df.groupby([\"year\", \"method\"])[\"number\"].sum().reset_index(),\n",
    "        aes(x=\"year\", y=\"number\", fill=\"method\", order=\"method\"),\n",
    "    )\n",
    "    + geom_area(alpha=0.5)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(\n",
    "    df.groupby([\"year\", \"method\"])[\"number\"]\n",
    "    .sum()\n",
    "    .reset_index()\n",
    "    .assign(\n",
    "        year=lambda x: pd.to_datetime(x[\"year\"], format=\"%Y\")\n",
    "        + pd.tseries.offsets.YearEnd()\n",
    "    )\n",
    ").mark_area().encode(x=\"year:T\", y=\"number:Q\", color=\"method:N\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slope chart\n",
    "\n",
    "A slope chart has two points connected by a line and is good for indicating how relationships between variables have changed over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    \"https://raw.githubusercontent.com/selva86/datasets/master/gdppercap.csv\"\n",
    ")\n",
    "df = pd.melt(\n",
    "    df,\n",
    "    id_vars=[\"continent\"],\n",
    "    value_vars=df.columns[1:],\n",
    "    value_name=\"GDP per capita\",\n",
    "    var_name=\"Year\",\n",
    ").rename(columns={\"continent\": \"Continent\"})\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "There isn't an off-the-shelf way to do this in matplotlib but the example below shows that, with matplotlib, where there's a will there's a way! It's where the 'build-what-you-want' comes into its own. Note that the functino that's defined returns an `Axes` object so that you can do further processing and tweaking as you like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import lines as mlines\n",
    "\n",
    "\n",
    "def slope_plot(data, x, y, group, before_txt=\"Before\", after_txt=\"After\"):\n",
    "    if len(data[x].unique()) != 2:\n",
    "        raise ValueError(\"Slope plot must have two unique periods.\")\n",
    "    wide_data = data[[x, y, group]].pivot(index=group, columns=x, values=y)\n",
    "    x_names = list(wide_data.columns)\n",
    "    klass = [\n",
    "        \"red\" if (y1 - y2) < 0 else \"green\"\n",
    "        for y1, y2 in zip(wide_data[x_names[0]], wide_data[x_names[1]])\n",
    "    ]\n",
    "    fig, ax = plt.subplots()\n",
    "\n",
    "    def newline(p1, p2, color=\"black\"):\n",
    "        ax = plt.gca()\n",
    "        l = mlines.Line2D(\n",
    "            [p1[0], p2[0]],\n",
    "            [p1[1], p2[1]],\n",
    "            color=\"red\" if p1[1] - p2[1] > 0 else \"green\",\n",
    "            marker=\"o\",\n",
    "            markersize=6,\n",
    "        )\n",
    "        ax.add_line(l)\n",
    "        return l\n",
    "\n",
    "    # Vertical Lines\n",
    "    y_min = data[y].min()\n",
    "    y_max = data[y].max()\n",
    "    ax.vlines(\n",
    "        x=1,\n",
    "        ymin=y_min,\n",
    "        ymax=y_max,\n",
    "        color=\"black\",\n",
    "        alpha=0.7,\n",
    "        linewidth=1,\n",
    "        linestyles=\"dotted\",\n",
    "    )\n",
    "    ax.vlines(\n",
    "        x=3,\n",
    "        ymin=y_min,\n",
    "        ymax=y_max,\n",
    "        color=\"black\",\n",
    "        alpha=0.7,\n",
    "        linewidth=1,\n",
    "        linestyles=\"dotted\",\n",
    "    )\n",
    "    # Points\n",
    "    ax.scatter(\n",
    "        y=wide_data[x_names[0]],\n",
    "        x=np.repeat(1, wide_data.shape[0]),\n",
    "        s=15,\n",
    "        color=\"black\",\n",
    "        alpha=0.7,\n",
    "    )\n",
    "    ax.scatter(\n",
    "        y=wide_data[x_names[1]],\n",
    "        x=np.repeat(3, wide_data.shape[0]),\n",
    "        s=15,\n",
    "        color=\"black\",\n",
    "        alpha=0.7,\n",
    "    )\n",
    "    # Line Segmentsand Annotation\n",
    "    for p1, p2, c in zip(wide_data[x_names[0]], wide_data[x_names[1]], wide_data.index):\n",
    "        newline([1, p1], [3, p2])\n",
    "        ax.text(\n",
    "            1 - 0.05,\n",
    "            p1,\n",
    "            c,\n",
    "            horizontalalignment=\"right\",\n",
    "            verticalalignment=\"center\",\n",
    "            fontdict={\"size\": 14},\n",
    "        )\n",
    "        ax.text(\n",
    "            3 + 0.05,\n",
    "            p2,\n",
    "            c,\n",
    "            horizontalalignment=\"left\",\n",
    "            verticalalignment=\"center\",\n",
    "            fontdict={\"size\": 14},\n",
    "        )\n",
    "    # 'Before' and 'After' Annotations\n",
    "    ax.text(\n",
    "        1 - 0.05,\n",
    "        y_max + abs(y_max) * 0.1,\n",
    "        before_txt,\n",
    "        horizontalalignment=\"right\",\n",
    "        verticalalignment=\"center\",\n",
    "        fontdict={\"size\": 16, \"weight\": 700},\n",
    "    )\n",
    "    ax.text(\n",
    "        3 + 0.05,\n",
    "        y_max + abs(y_max) * 0.1,\n",
    "        after_txt,\n",
    "        horizontalalignment=\"left\",\n",
    "        verticalalignment=\"center\",\n",
    "        fontdict={\"size\": 16, \"weight\": 700},\n",
    "    )\n",
    "    # Decoration\n",
    "    ax.set(\n",
    "        xlim=(0, 4), ylabel=y, ylim=(y_min - 0.1 * abs(y_min), y_max + abs(y_max) * 0.1)\n",
    "    )\n",
    "    ax.set_xticks([1, 3])\n",
    "    ax.set_xticklabels(x_names)\n",
    "    # Lighten borders\n",
    "    for ax_pos in [\"top\", \"bottom\", \"right\", \"left\"]:\n",
    "        ax.spines[ax_pos].set_visible(False)\n",
    "    return ax\n",
    "\n",
    "\n",
    "slope_plot(df, x=\"Year\", y=\"GDP per capita\", group=\"Continent\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.pointplot(x=\"Year\", y=\"GDP per capita\", hue=\"Continent\", data=df)\n",
    "plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.0);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(df, aes(x=\"Year\", y=\"GDP per capita\", group=\"Continent\"))\n",
    "    + geom_line(aes(color=\"Continent\", alpha=1), size=2)\n",
    "    + geom_point(aes(color=\"Continent\", alpha=1), size=4)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(df).mark_line().encode(x=\"Year:O\", y=\"GDP per capita\", color=\"Continent\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "yr_names = [int(x) for x in df[\"Year\"].unique()]\n",
    "px_df = (\n",
    "    df.pivot(index=\"Continent\", columns=\"Year\", values=\"GDP per capita\")\n",
    "    .reset_index()\n",
    "    .rename(columns=dict(zip(df[\"Year\"].unique(), range(len(df[\"Year\"].unique())))))\n",
    ")\n",
    "\n",
    "x_offset = 5\n",
    "\n",
    "fig1 = go.Figure()\n",
    "# Draw lines\n",
    "for index, row in px_df.iterrows():\n",
    "    fig1.add_shape(\n",
    "        type=\"line\",\n",
    "        x0=yr_names[0],\n",
    "        y0=row[0],\n",
    "        x1=yr_names[1],\n",
    "        y1=row[1],\n",
    "        name=row[\"Continent\"],\n",
    "        line=dict(color=px.colors.qualitative.Plotly[index]),\n",
    "    )\n",
    "    fig1.add_trace(\n",
    "        go.Scatter(\n",
    "            x=[yr_names[0]],\n",
    "            y=[row[0]],\n",
    "            text=row[\"Continent\"],\n",
    "            mode=\"text\",\n",
    "            name=None,\n",
    "        )\n",
    "    )\n",
    "\n",
    "\n",
    "fig1.update_xaxes(range=[yr_names[0] - x_offset, yr_names[1] + x_offset])\n",
    "fig1.update_yaxes(\n",
    "    range=[px_df[[0, 1]].min().min() * 0.8, px_df[[0, 1]].max().max() * 1.2]\n",
    ")\n",
    "fig1.update_layout(showlegend=False)\n",
    "fig1.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dumbbell Plot\n",
    "\n",
    "These are excellent for showing a change in time with a large number of categories, as we will do here with continents and mean GDP per capita."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    \"https://raw.githubusercontent.com/selva86/datasets/master/gdppercap.csv\"\n",
    ")\n",
    "df = pd.melt(\n",
    "    df,\n",
    "    id_vars=[\"continent\"],\n",
    "    value_vars=df.columns[1:],\n",
    "    value_name=\"GDP per capita\",\n",
    "    var_name=\"Year\",\n",
    ").rename(columns={\"continent\": \"Continent\"})\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "Again, no off-the-shelf method--but that's no problem when you can build it yourself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import lines as mlines\n",
    "\n",
    "\n",
    "def dumbbell_plot(data, x, y, change):\n",
    "    if len(data[x].unique()) != 2:\n",
    "        raise ValueError(\"Dumbbell plot must have two unique periods.\")\n",
    "    if type(data[y].iloc[0]) != str:\n",
    "        raise ValueError(\"Dumbbell plot y variable only works with category values.\")\n",
    "    wide_data = data[[x, y, change]].pivot(index=y, columns=x, values=change)\n",
    "    x_names = list(wide_data.columns)\n",
    "    y_names = list(wide_data.index)\n",
    "\n",
    "    def newline(p1, p2, color=\"black\"):\n",
    "        ax = plt.gca()\n",
    "        l = mlines.Line2D([p1[0], p2[0]], [p1[1], p2[1]], color=\"skyblue\", zorder=0)\n",
    "        ax.add_line(l)\n",
    "        return l\n",
    "\n",
    "    fig, ax = plt.subplots()\n",
    "    # Points\n",
    "    ax.scatter(\n",
    "        y=range(len(y_names)),\n",
    "        x=wide_data[x_names[1]],\n",
    "        s=50,\n",
    "        color=\"#0e668b\",\n",
    "        alpha=0.9,\n",
    "        zorder=2,\n",
    "        label=x_names[1],\n",
    "    )\n",
    "    ax.scatter(\n",
    "        y=range(len(y_names)),\n",
    "        x=wide_data[x_names[0]],\n",
    "        s=50,\n",
    "        color=\"#a3c4dc\",\n",
    "        alpha=0.9,\n",
    "        zorder=1,\n",
    "        label=x_names[0],\n",
    "    )\n",
    "    # Line segments\n",
    "    for i, p1, p2 in zip(\n",
    "        range(len(y_names)), wide_data[x_names[0]], wide_data[x_names[1]]\n",
    "    ):\n",
    "        newline([p1, i], [p2, i])\n",
    "    ax.set_yticks(range(len(y_names)))\n",
    "    ax.set_yticklabels(y_names)\n",
    "    # Decoration\n",
    "    # Lighten borders\n",
    "    for ax_pos in [\"top\", \"right\", \"left\"]:\n",
    "        ax.spines[ax_pos].set_visible(False)\n",
    "    ax.set_xlabel(change)\n",
    "    ax.legend(frameon=False, loc=\"lower right\")\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "dumbbell_plot(df, x=\"Year\", y=\"Continent\", change=\"GDP per capita\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "fig1 = go.Figure()\n",
    "\n",
    "yr_names = df[\"Year\"].unique()\n",
    "\n",
    "\n",
    "# Draw lines\n",
    "for i, cont in enumerate(df[\"Continent\"].unique()):\n",
    "    cdf = df[df[\"Continent\"] == cont]\n",
    "    fig1.add_shape(\n",
    "        type=\"line\",\n",
    "        x0=cdf.loc[cdf[\"Year\"] == yr_names[0], \"GDP per capita\"].values[0],\n",
    "        y0=cont,\n",
    "        x1=cdf.loc[cdf[\"Year\"] == yr_names[1], \"GDP per capita\"].values[0],\n",
    "        y1=cont,\n",
    "        line=dict(color=px.colors.qualitative.Plotly[0], width=2),\n",
    "    )\n",
    "# Draw points\n",
    "for i, year in enumerate(yr_names):\n",
    "    yrdf = df[df[\"Year\"] == year]\n",
    "    fig1.add_trace(\n",
    "        go.Scatter(\n",
    "            y=yrdf[\"Continent\"],\n",
    "            x=yrdf[\"GDP per capita\"],\n",
    "            mode=\"markers\",\n",
    "            name=year,\n",
    "            marker_color=px.colors.qualitative.Plotly[i],\n",
    "            marker_size=10,\n",
    "        ),\n",
    "    )\n",
    "\n",
    "fig1.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Polar\n",
    "\n",
    "I'm not sure I've ever seen a polar plots in economics, but you never know.\n",
    "\n",
    "Let's generate some polar data first:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = np.arange(0, 2, 0.01)\n",
    "theta = 2 * np.pi * r\n",
    "polar_data = pd.DataFrame({\"r\": r, \"theta\": theta})\n",
    "polar_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = plt.subplot(111, projection=\"polar\")\n",
    "ax.plot(polar_data[\"theta\"], polar_data[\"r\"])\n",
    "ax.set_rmax(2)\n",
    "ax.set_rticks([0.5, 1, 1.5, 2])  # Fewer radial ticks\n",
    "ax.set_rlabel_position(-22.5)  # Move radial labels away from plotted line\n",
    "ax.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = go.Figure(\n",
    "    data=go.Scatterpolar(\n",
    "        r=polar_data[\"r\"].values,\n",
    "        theta=polar_data[\"theta\"].values * 180 / (np.pi),\n",
    "        mode=\"lines\",\n",
    "    )\n",
    ")\n",
    "\n",
    "fig.update_layout(showlegend=False)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Radar (or spider) chart\n",
    "\n",
    "Let's generate some synthetic data for this one. Assumes that result to be shown is the sum of observations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(\n",
    "    dict(\n",
    "        zip(\n",
    "            [\"var\" + str(i) for i in range(1, 6)],\n",
    "            [np.random.randint(30, size=(4)) for i in range(1, 6)],\n",
    "        )\n",
    "    )\n",
    ")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import pi\n",
    "\n",
    "\n",
    "def radar_plot(data, variables):\n",
    "    n_vars = len(variables)\n",
    "    # Plot the first line of the data frame.\n",
    "    # Repeat the first value to close the circular graph:\n",
    "    values = data.loc[data.index[0], variables].values.flatten().tolist()\n",
    "    values += values[:1]\n",
    "    # What will be the angle of each axis in the plot? (we divide / number of variable)\n",
    "    angles = [n / float(n_vars) * 2 * pi for n in range(n_vars)]\n",
    "    angles += angles[:1]\n",
    "    # Initialise the spider plot\n",
    "    ax = plt.subplot(111, polar=True)\n",
    "    # Draw one axe per variable + add labels\n",
    "    plt.xticks(angles[:-1], variables)\n",
    "    # Draw ylabels\n",
    "    ax.set_rlabel_position(0)\n",
    "    # Plot data\n",
    "    ax.plot(angles, values, linewidth=1, linestyle=\"solid\")\n",
    "    # Fill area\n",
    "    ax.fill(angles, values, \"b\", alpha=0.1)\n",
    "    return ax\n",
    "\n",
    "\n",
    "radar_plot(df, df.columns);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = px.data.wind()\n",
    "print(df.head())\n",
    "fig = px.line_polar(\n",
    "    df,\n",
    "    r=\"frequency\",\n",
    "    theta=\"direction\",\n",
    "    color=\"strength\",\n",
    "    line_close=True,\n",
    "    color_discrete_sequence=px.colors.sequential.Plasma_r,\n",
    "    template=\"plotly_dark\",\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wordcloud\n",
    "\n",
    "These should be used sparingly. Let's grab part of a famous text from Project Gutenberg:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# To run this example, download smith_won.txt from\n",
    "# https://github.com/aeturrell/coding-for-economists/blob/main/data/smith_won.txt\n",
    "# and put it in a sub-folder called 'data\n",
    "\n",
    "book_text = open(Path(\"data\", \"smith_won.txt\"), \"r\").read()\n",
    "# Print some lines\n",
    "print(\"\\n\".join(book_text.split(os.linesep)[107:117]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from wordcloud import WordCloud\n",
    "\n",
    "wordcloud = WordCloud(width=700, height=400).generate(book_text)\n",
    "fig, ax = plt.subplots(facecolor=\"k\")\n",
    "ax.imshow(wordcloud, interpolation=\"bilinear\")\n",
    "plt.axis(\"off\")\n",
    "plt.tight_layout();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also create a 'mask' for the wordcloud to shape it how we like, here in the shape of a book."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To run this example, download book_mask.png from\n",
    "# https://github.com/aeturrell/coding-for-economists/raw/main/data/book_mask.png\n",
    "# and put it in a sub-folder called 'data\n",
    "from PIL import Image\n",
    "\n",
    "mask = np.array(Image.open(Path(\"data\", \"book_mask.png\")))\n",
    "wc = WordCloud(width=700, height=400, mask=mask, background_color=\"white\")\n",
    "wordcloud = wc.generate(book_text)\n",
    "fig, ax = plt.subplots(facecolor=\"white\")\n",
    "ax.imshow(wordcloud, interpolation=\"bilinear\")\n",
    "plt.axis(\"off\")\n",
    "plt.tight_layout();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Network diagrams\n",
    "\n",
    "### networkx\n",
    "\n",
    "The most well-established network visualisation package is [**networkx**](https://networkx.org/documentation/stable/index.html), which does a *lot* more than just visualisation. It has many different positioning options for rendering any given network, for instance in circular, spectral, spring, Fruchterman-Reingold, or other styles. In the below example, we use a **pandas** dataframe to specify the edges in two columns but there are various other ways to specify the network too, including ones that do not rely on **pandas**.\n",
    "\n",
    "The underlying plot is rendered with **matplotlib**, meaning that you can customise it further should you need to. You can pass an `Axes` object `ax` to `nx.draw` using `nx.draw(..., ax=ax)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "\n",
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"source\": [\"A\", \"B\", \"C\", \"A\", \"E\", \"F\", \"E\", \"G\", \"G\", \"D\", \"F\"],\n",
    "        \"target\": [\"D\", \"A\", \"E\", \"C\", \"A\", \"F\", \"G\", \"D\", \"B\", \"G\", \"C\"],\n",
    "    }\n",
    ")\n",
    "G = nx.from_pandas_edgelist(df)\n",
    "nx.draw(G, with_labels=True, node_size=500, node_color=\"skyblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n",
    "Thanks to [Juan Mateos Garcia](https://juan-mateos.github.io/), we have a way of doing this in **altair** too. (I highly recommend you [follow Juan](https://twitter.com/JMateosGarcia) on twitter for all things data science and innovation.)\n",
    "\n",
    "Now bear with me, because the code is going to be quite long as there's a bit of setting up to do. But once we have set up the functions they can be used again for arbitrary networks, ie the code is not specific to the network we're going to look at. Everything was written by Juan, and his original code can be found [here](https://gist.github.com/Juan-Mateos/e1e274652c6247bf2114a463b73bd2d7)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def node_layer(\n",
    "    node_df,\n",
    "    graph,\n",
    "    node_label_lookup,\n",
    "    node_label,\n",
    "    node_size,\n",
    "    node_color,\n",
    "    show_neighbours,\n",
    "):\n",
    "    \"\"\"Creates node_layer in the plot\"\"\"\n",
    "    node_chart = (\n",
    "        alt.Chart(node_df)\n",
    "        .mark_point(filled=True, stroke=\"black\", strokeWidth=0.5)\n",
    "        .encode(\n",
    "            x=alt.X(\"x\", axis=None), y=alt.Y(\"y\", axis=None), fillOpacity=alt.value(1)\n",
    "        )\n",
    "    )\n",
    "\n",
    "    if node_size in node_df.columns:\n",
    "        node_chart = node_chart.encode(\n",
    "            size=alt.Size(node_size, legend=alt.Legend(orient=\"bottom\"))\n",
    "        )\n",
    "\n",
    "    if node_color in node_df.columns:\n",
    "        node_chart = node_chart.encode(\n",
    "            color=alt.Color(\n",
    "                node_color,\n",
    "                legend=alt.Legend(columns=2),\n",
    "                scale=alt.Scale(scheme=\"Spectral\"),\n",
    "            )\n",
    "        )\n",
    "    if show_neighbours is True:\n",
    "        neighbors = {\n",
    "            node: \", \".join(\n",
    "                [str(node_label_lookup[n]) for n in nx.neighbors(graph, node)]\n",
    "            )\n",
    "            for node in graph.nodes()\n",
    "        }\n",
    "        node_df[\"neighbors\"] = node_df[\"node\"].map(neighbors)\n",
    "\n",
    "        node_chart = node_chart.encode(tooltip=[node_label, \"neighbors\"])\n",
    "    else:\n",
    "        node_chart = node_chart.encode(tooltip=[node_label])\n",
    "\n",
    "    return node_chart\n",
    "\n",
    "\n",
    "def calculate_edge_positions(graph, node_pos_lookup, edge_scale):\n",
    "    \"\"\"Calculates the positions of the edges\"\"\"\n",
    "    if len(nx.get_edge_attributes(graph, \"weight\")) > 0:\n",
    "        weighted = True\n",
    "        edges_df = pd.DataFrame(\n",
    "            [\n",
    "                {\"e1\": e[0], \"e2\": e[1], \"weight\": e[2][\"weight\"]}\n",
    "                for e in list(graph.edges(data=True))\n",
    "            ]\n",
    "        )\n",
    "    else:\n",
    "        weighted = False\n",
    "        edges_df = pd.DataFrame(\n",
    "            [\n",
    "                {\"e1\": e[0], \"e2\": e[1], \"weight\": 1}\n",
    "                for e in list(graph.edges(data=True))\n",
    "            ]\n",
    "        )\n",
    "\n",
    "    edge_pos_cont = []\n",
    "\n",
    "    for _id, r in edges_df.iterrows():\n",
    "        x1, y1 = [node_pos_lookup[r[\"e1\"]][n] for n in [0, 1]]\n",
    "        x2, y2 = [node_pos_lookup[r[\"e2\"]][n] for n in [0, 1]]\n",
    "        w = int(r[\"weight\"]) / edge_scale\n",
    "        edge_pos_cont.append(\n",
    "            pd.Series([x1, y1, x2, y2, w], index=[\"x1\", \"y1\", \"x2\", \"y2\", \"weight\"])\n",
    "        )\n",
    "\n",
    "    edges_pos_df = pd.DataFrame(edge_pos_cont)\n",
    "    return edges_pos_df, weighted\n",
    "\n",
    "\n",
    "def edge_layer(edges_pos_df, weighted, edge_opacity):\n",
    "    \"\"\"Creates edge layer in the plot\"\"\"\n",
    "    edge_chart = (\n",
    "        alt.Chart(edges_pos_df)\n",
    "        .mark_line()\n",
    "        .encode(x=\"x1\", x2=\"x2\", y=\"y1\", y2=\"y2\", strokeOpacity=alt.value(edge_opacity))\n",
    "    )\n",
    "    if weighted is True:\n",
    "        edge_chart = edge_chart.encode(\n",
    "            strokeWidth=alt.StrokeWidth(\"weight\", legend=alt.Legend(orient=\"bottom\"))\n",
    "        )\n",
    "    return edge_chart\n",
    "\n",
    "\n",
    "def plot_altair_network(\n",
    "    node_df,\n",
    "    graph,\n",
    "    node_label=None,\n",
    "    node_size=None,\n",
    "    node_color=None,\n",
    "    show_neighbours=True,\n",
    "    edge_scale=1,\n",
    "    edge_opacity=0.1,\n",
    "):\n",
    "    \"\"\"Plot a network graph with altair\n",
    "    Args:\n",
    "        node_df (df): dataframe where the rows are nodes and the\n",
    "        node_label (str): node label variable in node_df\n",
    "        node_size (str): node size variable in node_df\n",
    "        node_color (str): node color variable (this is a categorical variable)\n",
    "        columns are relevant variables including node position, node_label, node_size,node_color\n",
    "        graph (networkx graph): graph object generated with networkx that we use to extract edges & edgeWidths\n",
    "        show_neighbours (bool): if we want neighbours to be extracted and showed in a tooltip\n",
    "        edge_scale (float): scale for weight value\n",
    "        edge_opacity (float): weight opacity\n",
    "    \"\"\"\n",
    "\n",
    "    # Node chart\n",
    "    node_df_ = node_df.copy()\n",
    "\n",
    "    # Make node name - label lookup\n",
    "    node_label_lookup = node_df_.set_index(\"node\")[\"node_name\"].to_dict()\n",
    "    node_pos_lookup = {r[\"node\"]: (r[\"x\"], r[\"y\"]) for _id, r in node_df_.iterrows()}\n",
    "\n",
    "    # Plot nodes\n",
    "    node_plot = node_layer(\n",
    "        node_df_,\n",
    "        graph,\n",
    "        node_label_lookup,\n",
    "        node_label,\n",
    "        node_size,\n",
    "        node_color,\n",
    "        show_neighbours,\n",
    "    )\n",
    "    # Plot edges (after calculating their positions)\n",
    "    edge_positions, weighted = calculate_edge_positions(\n",
    "        graph, node_pos_lookup, edge_scale\n",
    "    )\n",
    "\n",
    "    edge_plot = edge_layer(edge_positions, weighted, edge_opacity)\n",
    "\n",
    "    # Combine plots\n",
    "\n",
    "    net_plot = (\n",
    "        (node_plot + edge_plot).configure_axis(grid=False).configure_view(strokeWidth=0)\n",
    "    )\n",
    "    return net_plot\n",
    "\n",
    "\n",
    "##########\n",
    "# Example\n",
    "##########\n",
    "\n",
    "# Read network\n",
    "m = nx.les_miserables_graph()\n",
    "\n",
    "# Extract node positions\n",
    "m_pos = nx.kamada_kawai_layout(m)\n",
    "\n",
    "# Create node df (inelegantly)\n",
    "node_df = (\n",
    "    pd.DataFrame(m_pos)\n",
    "    .T.rename(columns={0: \"x\", 1: \"y\"})\n",
    "    .reset_index(drop=False)\n",
    "    .rename(columns={\"index\": \"node\"})\n",
    "    .assign(node_name=lambda x: x[\"node\"])\n",
    ")\n",
    "\n",
    "# Plot\n",
    "plot_altair_network(\n",
    "    node_df,\n",
    "    node_label=\"node_name\",\n",
    "    node_size=None,\n",
    "    node_color=None,\n",
    "    graph=m,\n",
    "    show_neighbours=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ridge, or 'joy', plots\n",
    "\n",
    "These are famous from the front cover of “Unkown Pleasures” by Joy Division. Although **seaborn** [can be coaxed](https://seaborn.pydata.org/examples/kde_ridgeplot.html) into plotting ridge plots, by far the easiest way is via the [**joypy**](https://github.com/sbebo/joypy) package. These plots sometimes seem to prioritise form over function. In this case, let's look at an example where I think they do add value: showing the global increase in temperature.\n",
    "\n",
    "We'll use a summary of the daily land-surface average temperature anomaly produced by the Berkeley Earth averaging method. Temperatures are in Celsius and reported as anomalies relative to the Jan 1951-Dec 1980 average (the estimated Jan 1951-Dec 1980 land-average temperature is 8.63 +/- 0.06 C)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To run this example, download the pickle file from\n",
    "# https://github.com/aeturrell/coding-for-economists/blob/main/data/berkeley_data.pkl\n",
    "# and put it in a sub-folder called 'data'\n",
    "df = pd.read_pickle(Path(\"data/berkeley_data.pkl\"))\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import joypy\n",
    "from matplotlib import cm\n",
    "\n",
    "final_year = df[\"Year\"].max()\n",
    "first_year = df[\"Year\"].min()\n",
    "\n",
    "labels = [y if y % 10 == 0 else None for y in list(df.Year.unique())]\n",
    "fig, axes = joypy.joyplot(\n",
    "    df,\n",
    "    by=\"Year\",\n",
    "    column=\"Anomaly\",\n",
    "    labels=labels,\n",
    "    range_style=\"own\",\n",
    "    grid=\"y\",\n",
    "    linewidth=1,\n",
    "    legend=False,\n",
    "    title=f\"Global daily temperature anomaly {first_year}-{final_year} \\n(°C above 1951-80 average)\",\n",
    "    colormap=cm.inferno,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Waterfall chart\n",
    "\n",
    "Waterfall charts are good for showing how different contributions combine to net out at a certain value. There's a package dedicated to them called [**waterfallcharts**](https://github.com/chrispaulca/waterfall). It builds on **matplotlib**. First, let's create some data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [\"sales\", \"returns\", \"credit fees\", \"rebates\", \"late charges\", \"shipping\"]\n",
    "b = [10, -30, -7.5, -25, 95, -7]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot this data. Because the defaults of **waterfallcharts** don't play that nicely with the plot style used for this book, we'll temporarily switch back to the **matplotlib** default plot style using a *context* and `with` statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import waterfall_chart\n",
    "\n",
    "with plt.style.context(\"default\"):\n",
    "    plot = waterfall_chart.plot(a, b, sorted_value=True, rotation_value=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "px_b = b + [sum(b)]\n",
    "\n",
    "fig = go.Figure(\n",
    "    go.Waterfall(\n",
    "        name=\"20\",\n",
    "        orientation=\"v\",\n",
    "        measure=[\"relative\"] * len(a) + [\"total\"],\n",
    "        x=a + [\"net\"],\n",
    "        textposition=\"outside\",\n",
    "        text=[str(x) for x in b] + [\"net\"],\n",
    "        y=px_b,\n",
    "        connector={\"line\": {\"color\": \"rgb(63, 63, 63)\"}},\n",
    "    )\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Venn\n",
    "\n",
    "Venn diagrams show the overlap between groups. As with some of these other, more unsual chart types, there's a special package that produces these and which builds on **matplotlib**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib_venn import venn2\n",
    "\n",
    "venn2(subsets=(10, 5, 2), set_labels=(\"Group A\", \"Group B\"), alpha=0.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Priestly Timeline\n",
    "\n",
    "This displays a timeline of start and end events in time, and their overlap."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = (\n",
    "    pd.read_csv(\n",
    "        \"https://github.com/aeturrell/coding-for-economists/raw/main/data/priestley-timeline.csv\"\n",
    "    )\n",
    "    .melt(id_vars=\"Name\", value_name=\"Date\")\n",
    "    .assign(Date=lambda x: pd.to_datetime(x[\"Date\"]))\n",
    ")\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n",
    "This example is from [**viztech**](https://github.com/cstorm125/viztech). It relies on an extra package for axes scaling and formatting called [**mizani**](https://mizani.readthedocs.io/en/stable/index.html).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mizani.breaks import date_breaks\n",
    "from mizani.formatters import date_format\n",
    "\n",
    "g = (\n",
    "    ggplot(df, aes(x=\"Name\", y=\"Date\", group=\"Name\"))\n",
    "    + geom_line(size=10, color=\"green\")\n",
    "    + coord_flip()\n",
    "    + geom_text(\n",
    "        aes(x=\"Name\", y=\"Date\", label=\"Name\"),\n",
    "        data=df[df.variable == \"Born\"],\n",
    "        nudge_y=6500,\n",
    "        color=\"white\",\n",
    "    )\n",
    "    + scale_y_date(breaks=date_breaks(\"20 years\"), labels=date_format(\"%Y\"))\n",
    "    + theme(axis_text_y=element_blank())\n",
    ")\n",
    "g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Waffle, isotype, or pictogram charts\n",
    "\n",
    "These are great for showing easily-understandable magnitudes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "There is a package called [**pywaffle**](https://github.com/gyli/PyWaffle) that provides a convenient way of doing this. It expects a dictionary of values. Note that the icon can be changed and, because it builds on **matplotlib**, you can tweak to your heart's content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pywaffle import Waffle\n",
    "\n",
    "data = {\"Democratic\": 48, \"Republican\": 46, \"Libertarian\": 3}\n",
    "fig = plt.figure(\n",
    "    FigureClass=Waffle,\n",
    "    rows=5,\n",
    "    values=data,\n",
    "    colors=[\"#232066\", \"#983D3D\", \"#DCB732\"],\n",
    "    legend={\"loc\": \"upper left\", \"bbox_to_anchor\": (1, 1)},\n",
    "    icons=\"child\",\n",
    "    font_size=12,\n",
    "    icon_legend=True,\n",
    ")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine\n",
    "\n",
    "As ever, plotnine prefers tidy format data. The **plotnine** example comes from **viztech**. We'll create a mini dataset just to demonstrate its use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "\n",
    "df = pd.DataFrame(list(itertools.product(range(10), range(10))), columns=[\"x\", \"y\"])\n",
    "df[\"filled\"] = 0\n",
    "df.iloc[:32, 2] = 1\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = (\n",
    "    ggplot(df, aes(x=\"x\", y=\"y\", fill=\"factor(filled)\"))\n",
    "    + geom_tile(alpha=0.5, color=\"black\")\n",
    "    + guides(fill=False)\n",
    "    + scale_fill_manual([\"grey\", \"blue\"])\n",
    "    + theme_void()\n",
    "    + coord_flip()\n",
    "    + geom_text(x=5, y=5, label=f\"{int(100*df.filled.mean())}%\", size=30, color=\"white\")\n",
    "    + theme(axis_text=element_blank())\n",
    "    + xlab(\"\")\n",
    "    + ylab(\"\")\n",
    ")\n",
    "g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pyramid\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    \"https://raw.githubusercontent.com/selva86/datasets/master/email_campaign_funnel.csv\"\n",
    ")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib/Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "group_col = \"Gender\"\n",
    "order_of_bars = df.Stage.unique()[::-1]\n",
    "colors = [\n",
    "    plt.cm.Spectral(i / float(len(df[group_col].unique()) - 1))\n",
    "    for i in range(len(df[group_col].unique()))\n",
    "]\n",
    "\n",
    "for c, group in zip(colors, df[group_col].unique()):\n",
    "    sns.barplot(\n",
    "        x=\"Users\",\n",
    "        y=\"Stage\",\n",
    "        data=df.loc[df[group_col] == group, :],\n",
    "        order=order_of_bars,\n",
    "        color=c,\n",
    "        label=group,\n",
    "        ax=ax,\n",
    "        lw=0,\n",
    "    )\n",
    "\n",
    "divisor = 1e6\n",
    "ax.set_xticklabels([str(abs(x) / divisor) for x in ax.get_xticks()])\n",
    "plt.xlabel(\"Users (millions)\")\n",
    "plt.ylabel(\"Stage of Purchase\")\n",
    "plt.yticks(fontsize=12)\n",
    "plt.title(\"Population Pyramid of the Marketing Funnel\", fontsize=22)\n",
    "plt.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotnine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = (\n",
    "    ggplot(df, aes(x=\"Stage\", y=\"Users\", fill=\"Gender\"))\n",
    "    + geom_col(width=0.8)  # baseplot\n",
    "    + coord_flip()  # type of plot\n",
    "    + theme_minimal()  # flip coordinates\n",
    "    + scale_y_continuous(  # theme\n",
    "        labels=lambda x: [i / 1e6 if i >= 0 else -i / 1e6 for i in x]\n",
    "    )\n",
    "    + ylab(\"Users (millions)\")\n",
    ")\n",
    "g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.funnel(df, y=\"Stage\", x=\"Users\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sankey diagram\n",
    "\n",
    "Sankey diagrams show how a flow breaks into pieces."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "labels = [\"A1\", \"A2\", \"B1\", \"B2\", \"C1\", \"C2\"]\n",
    "\n",
    "fig = go.Figure(\n",
    "    data=[\n",
    "        go.Sankey(\n",
    "            node=dict(\n",
    "                pad=15,\n",
    "                thickness=20,\n",
    "                line=dict(color=\"black\", width=0.5),\n",
    "                label=labels,\n",
    "                color=px.colors.qualitative.Plotly[: len(labels)],\n",
    "            ),\n",
    "            # indices correspond to labels, eg A1, A2, A1, B1, ...\n",
    "            link=dict(\n",
    "                source=[0, 1, 0, 2, 3, 3, 2],\n",
    "                target=[2, 3, 3, 4, 4, 5, 5],\n",
    "                value=[7, 3, 2, 6, 4, 2, 1],\n",
    "            ),\n",
    "        )\n",
    "    ]\n",
    ")\n",
    "\n",
    "fig.update_layout(title_text=\"Basic Sankey Diagram\", font_size=10)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dendrogram or hierarchical clustering\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data\n",
    "df = (\n",
    "    pd.read_csv(\n",
    "        \"https://vincentarelbundock.github.io/Rdatasets/csv/datasets/mtcars.csv\"\n",
    "    )\n",
    "    .rename(columns={\"Unnamed: 0\": \"Model\"})\n",
    "    .set_index(\"Model\")\n",
    ")\n",
    "# Plot\n",
    "sns.clustermap(df, metric=\"correlation\", method=\"single\", standard_scale=1, cmap=\"vlag\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Treemap\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.express as px\n",
    "import numpy as np\n",
    "\n",
    "df = px.data.gapminder().query(\"year == 2007\")\n",
    "fig = px.treemap(\n",
    "    df,\n",
    "    path=[px.Constant(\"world\"), \"continent\", \"country\"],\n",
    "    values=\"pop\",\n",
    "    color=\"lifeExp\",\n",
    "    hover_data=[\"iso_alpha\"],\n",
    "    color_continuous_scale=\"RdBu\",\n",
    "    color_continuous_midpoint=np.average(df[\"lifeExp\"], weights=df[\"pop\"]),\n",
    ")\n",
    "fig.update_layout(margin=dict(t=50, l=25, r=25, b=25))\n",
    "fig.show()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3.8.12 ('codeforecon')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "c4570b151692b3082981c89d172815ada9960dee4eb0bedb37dc10c95601d3bd"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}