vis-common-plots.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(vis-common-plots)=\n",
    "# Common Plots\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In this chapter, we'll look at some of the most common plots that you might want to make—and how to create them using the most popular data visualisations libraries, including [**matplotlib**](https://matplotlib.org/), [**lets-plot**](https://lets-plot.org/), [**seaborn**](https://seaborn.pydata.org/), [**altair**](https://altair-viz.github.io/), and [**plotly**](https://plotly.com/python/). If you need an introduction to these libraries, check out the other data visualisation chapters.\n",
    "\n",
    "This chapter has benefited from the phenomenal **matplotlib** documentation, the **lets-plot** documentation, [**viztech**](https://github.com/cstorm125/viztech) (a repository that aimed to recreate the entire Financial Times Visual Vocabulary using **plotnine**), from the **seaborn** documentation, from the **altair** documentation, from the **plotly** documentation, and from examples posted around the web on forums and in blog posts. You may be wondering why **plotnine** isn't featured here: its functions have almost exactly the same names as those in **lets-plot**, and we have opted to include the latter as it is currently the more mature plotting package. However, most of the code below for **lets-plot** also works in **plotnine**, and you can read more about **plotnine** in {ref}`vis-plotnine`.\n",
    "\n",
    "Bear in mind that for many of the **matplotlib** examples, using the `df.plot.*` syntax can get the plot you want more quickly! To be more comprehensive, the solution for any kind of data is shown in the examples below.\n",
    "\n",
    "Throughout, we'll assume that the data are in a tidy format (one row per observation, one variable per column). Remember that all Altair plots can be made interactive by adding `.interactive()` at the end.\n",
    "\n",
    "First, though, let's import the libraries we'll need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "from itertools import cycle\n",
    "from pathlib import Path\n",
    "\n",
    "import altair as alt\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import plotly.express as px\n",
    "import seaborn as sns\n",
    "import seaborn.objects as so\n",
    "from lets_plot import *\n",
    "from lets_plot.mapping import as_discrete\n",
    "from vega_datasets import data\n",
    "\n",
    "# Set seed for reproducibility\n",
    "# Set seed for random numbers\n",
    "seed_for_prng = 78557\n",
    "prng = np.random.default_rng(\n",
    "    seed_for_prng\n",
    ")  # prng=probabilistic random number generator\n",
    "\n",
    "# Turn off warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "# Set up lets-plot charts\n",
    "LetsPlot.setup_html()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "import matplotlib_inline.backend_inline\n",
    "\n",
    "# Plot settings\n",
    "plt.style.use(\n",
    "    \"https://github.com/aeturrell/coding-for-economists/raw/main/plot_style.txt\"\n",
    ")\n",
    "matplotlib_inline.backend_inline.set_matplotlib_formats(\"svg\")\n",
    "# some faffing here to try and get seaborn not to change theme in object API\n",
    "# sns.set_theme(rc=plt.rcParams)\n",
    "# Set max rows displayed for readability\n",
    "pd.set_option(\"display.max_rows\", 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scatter plot\n",
    "\n",
    "In this example, we will see a simple scatter plot with several categories using the \"cars\" data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cars = data.cars()\n",
    "cars.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "for origin in cars[\"Origin\"].unique():\n",
    "    cars_sub = cars[cars[\"Origin\"] == origin]\n",
    "    ax.scatter(cars_sub[\"Horsepower\"], cars_sub[\"Miles_per_Gallon\"], label=origin)\n",
    "ax.set_ylabel(\"Miles per Gallon\")\n",
    "ax.set_xlabel(\"Horsepower\")\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "Note that this uses the seaborn objects API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(so.Plot(cars, x=\"Horsepower\", y=\"Miles_per_Gallon\", color=\"Origin\").add(so.Dot()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(cars, aes(x=\"Horsepower\", y=\"Miles_per_Gallon\", color=\"Origin\"))\n",
    "    + geom_point()\n",
    "    + ylab(\"Miles per Gallon\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n",
    "For this first example, we'll also show how to make the altair plot interactive with movable axes and a tooltip that reveals more info when you hover your mouse over points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(cars).mark_circle(size=60).encode(\n",
    "    x=\"Horsepower\",\n",
    "    y=\"Miles_per_Gallon\",\n",
    "    color=\"Origin\",\n",
    "    tooltip=[\"Name\", \"Origin\", \"Horsepower\", \"Miles_per_Gallon\"],\n",
    ").interactive()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n",
    "Plotly is another declarative plotting library, at least sometimes (!), but one that is interactive by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(\n",
    "    cars,\n",
    "    x=\"Horsepower\",\n",
    "    y=\"Miles_per_Gallon\",\n",
    "    color=\"Origin\",\n",
    "    hover_data=[\"Name\", \"Origin\", \"Horsepower\", \"Miles_per_Gallon\"],\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Facets\n",
    "\n",
    "This applies to all plots, so in some sense is common! Facets, aka panels or small multiples, are ways of showing the same chart multiple times. Let's see how to achieve them in a few of the most popular plotting libraries.\n",
    "\n",
    "We'll use the \"tips\" dataset for this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = sns.load_dataset(\"tips\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "There are many ways to create facets using Matplotlib, and you can get facets in any shape or sizes you like. \n",
    "\n",
    "The easiest way, though, is to specify the number of rows and columns. This is achieved by specifying `nrows` and `ncols` when calling `plt.subplots()`. It returns an array of shape `(nrows, ncols)` of `Axes` objects. For most purposes, you'll want to flatten these to a vector before iterating over them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(nrows=1, ncols=4, sharex=True, sharey=True)\n",
    "flat_axes = axes.flatten()  # Not needed with 1 row or 1 col, but good to be aware of\n",
    "\n",
    "facet_grp = list(df[\"day\"].unique())\n",
    "# This part just to get some colours from the default color cycle\n",
    "colour_list = plt.rcParams[\"axes.prop_cycle\"].by_key()[\"color\"]\n",
    "iter_cycle = cycle(colour_list)\n",
    "\n",
    "for i, ax in enumerate(flat_axes):\n",
    "    sub_df = df.loc[df[\"day\"] == facet_grp[i]]\n",
    "    ax.scatter(\n",
    "        sub_df[\"tip\"],\n",
    "        sub_df[\"total_bill\"],\n",
    "        s=30,\n",
    "        edgecolor=\"k\",\n",
    "        color=next(iter_cycle),\n",
    "    )\n",
    "    ax.set_title(facet_grp[i])\n",
    "fig.text(0.5, 0.01, \"Tip\", ha=\"center\")\n",
    "fig.text(0.0, 0.5, \"Total bill\", va=\"center\", rotation=\"vertical\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different facet sizes are possible in numerous ways. In practice, it's often better to have evenly sized facets laid out in a grid--especially each facet is of the same x and y axes. But, just to show it's possible, here's an example that gives more space to the weekend than to weekdays using the tips dataset: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This part just to get some colours\n",
    "colormap = plt.cm.Dark2\n",
    "\n",
    "fig = plt.figure(constrained_layout=True)\n",
    "ax_dict = fig.subplot_mosaic([[\"Thur\", \"Fri\", \"Sat\", \"Sat\", \"Sun\", \"Sun\"]])\n",
    "facet_grp = list(ax_dict.keys())\n",
    "colorst = [colormap(i) for i in np.linspace(0, 0.9, len(facet_grp))]\n",
    "for i, grp in enumerate(facet_grp):\n",
    "    sub_df = df.loc[df[\"day\"] == facet_grp[i]]\n",
    "    ax_dict[grp].scatter(\n",
    "        sub_df[\"tip\"],\n",
    "        sub_df[\"total_bill\"],\n",
    "        s=30,\n",
    "        edgecolor=\"k\",\n",
    "        color=colorst[i],\n",
    "    )\n",
    "    ax_dict[grp].set_title(facet_grp[i])\n",
    "    if grp != \"Thurs\":\n",
    "        ax_dict[grp].set_yticklabels([])\n",
    "plt.tight_layout()\n",
    "fig.text(0.5, 0, \"Tip\", ha=\"center\")\n",
    "fig.text(0, 0.5, \"Total bill\", va=\"center\", rotation=\"vertical\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As well as using lists, you can also specify the layout using an array or using text, eg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "axd = plt.figure(constrained_layout=True).subplot_mosaic(\n",
    "    \"\"\"\n",
    "    ABD\n",
    "    CCD\n",
    "    CC.\n",
    "    \"\"\"\n",
    ")\n",
    "kw = dict(ha=\"center\", va=\"center\", fontsize=60, color=\"darkgrey\")\n",
    "for k, ax in axd.items():\n",
    "    ax.text(0.5, 0.5, k, transform=ax.transAxes, **kw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "Seaborn makes it easy to quickly create facet plots. Note the use of `col_wrap`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(df, x=\"tip\", y=\"total_bill\", color=\"day\")\n",
    "    .facet(col=\"day\", wrap=2)\n",
    "    .add(so.Dot())\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A nice feature of seaborn that is much more fiddly in (base) matplotlib is the ability to specify rows and columns separately: (smoker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(df, x=\"tip\", y=\"total_bill\", color=\"day\")\n",
    "    .facet(col=\"day\", row=\"smoker\")\n",
    "    .add(so.Dot())\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(df, aes(x=\"tip\", y=\"total_bill\", color=\"smoker\"))\n",
    "    + geom_point(size=3)\n",
    "    + facet_wrap([\"smoker\", \"day\"])\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(df).mark_point().encode(\n",
    "    x=\"tip:Q\",\n",
    "    y=\"total_bill:Q\",\n",
    "    color=\"smoker:N\",\n",
    "    facet=alt.Facet(\"day:N\", columns=2),\n",
    ").properties(\n",
    "    width=200,\n",
    "    height=100,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(\n",
    "    df, x=\"tip\", y=\"total_bill\", color=\"smoker\", facet_row=\"smoker\", facet_col=\"day\"\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connected scatter plot\n",
    "\n",
    "A simple variation on the scatter plot designed to show an ordering, usually of time. We'll trace out a Beveridge curve based on US data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "\n",
    "import pandas_datareader.data as web\n",
    "\n",
    "start = datetime.datetime(2000, 1, 1)\n",
    "end = datetime.datetime(datetime.datetime.now().year, 1, 1)\n",
    "code_dict = {\n",
    "    \"Vacancies\": \"LMJVTTUVUSA647N\",\n",
    "    \"Unemployment\": \"UNRATE\",\n",
    "    \"LabourForce\": \"CLF16OV\",\n",
    "}\n",
    "list_dfs = [\n",
    "    web.DataReader(value, \"fred\", start, end)\n",
    "    .rename(columns={value: key})\n",
    "    .groupby(pd.Grouper(freq=\"AS\"))\n",
    "    .mean()\n",
    "    for key, value in code_dict.items()\n",
    "]\n",
    "df = pd.concat(list_dfs, axis=1)\n",
    "df = df.assign(Vacancies=100 * df[\"Vacancies\"] / (df[\"LabourForce\"] * 1e3)).dropna()\n",
    "df[\"Year\"] = df.index.year\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.close(\"all\")\n",
    "fig, ax = plt.subplots()\n",
    "quivx = -df[\"Unemployment\"].diff(-1)\n",
    "quivy = -df[\"Vacancies\"].diff(-1)\n",
    "# This connects the points\n",
    "ax.quiver(\n",
    "    df[\"Unemployment\"],\n",
    "    df[\"Vacancies\"],\n",
    "    quivx,\n",
    "    quivy,\n",
    "    scale_units=\"xy\",\n",
    "    angles=\"xy\",\n",
    "    scale=1,\n",
    "    width=0.006,\n",
    "    alpha=0.3,\n",
    ")\n",
    "ax.scatter(\n",
    "    df[\"Unemployment\"],\n",
    "    df[\"Vacancies\"],\n",
    "    marker=\"o\",\n",
    "    s=35,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.2,\n",
    "    alpha=0.9,\n",
    ")\n",
    "for j in [0, -1]:\n",
    "    ax.annotate(\n",
    "        df[\"Year\"].iloc[j],\n",
    "        xy=(df[[\"Unemployment\", \"Vacancies\"]].iloc[j].tolist()),\n",
    "        xycoords=\"data\",\n",
    "        xytext=(-20, -40),\n",
    "        textcoords=\"offset points\",\n",
    "        arrowprops=dict(arrowstyle=\"->\", connectionstyle=\"angle3,angleA=0,angleB=-90\"),\n",
    "    )\n",
    "ax.set_xlabel(\"Unemployment rate, %\")\n",
    "ax.set_ylabel(\"Vacancy rate, %\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(df, x=\"Unemployment\", y=\"Vacancies\")\n",
    "    .add(so.Dots())\n",
    "    .add(so.Path(marker=\"o\"))\n",
    "    .label(\n",
    "        x=\"Unemployment rate, %\",\n",
    "        y=\"Vacancy rate, %\",\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n",
    "You can also use `geom_curve()` in place of `geom_segment()` below to get curved lines instead of straight lines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is a convencience and creates a dataframe of the form\n",
    "# Vacancies_from\tUnemployment_from\tLabourForce_from\tYear_from\tVacancies_to\tUnemployment_to\tLabourForce_to\tYear_to\n",
    "# 0\t3.028239\t4.741667\t143768.916667\t2001\t2.387254\t5.783333\t144856.083333\t2002\n",
    "# 1\t 2.387254\t5.783333\t144856.083333\t2002\t2.212237\t5.991667\t146499.500000\t2003\n",
    "# so that we have both years (from and to) in each row\n",
    "path_df = (\n",
    "    df.iloc[:-1]\n",
    "    .reset_index(drop=True)\n",
    "    .join(df.iloc[1:].reset_index(drop=True), lsuffix=\"_from\", rsuffix=\"_to\")\n",
    ")\n",
    "\n",
    "min_yr = df[\"Year\"].min()\n",
    "max_yr = df[\"Year\"].max()\n",
    "\n",
    "(\n",
    "    ggplot(df, aes(\"Unemployment\", \"Vacancies\"))\n",
    "    + geom_segment(\n",
    "        aes(\n",
    "            x=\"Unemployment_from\",\n",
    "            y=\"Vacancies_from\",\n",
    "            xend=\"Unemployment_to\",\n",
    "            yend=\"Vacancies_to\",\n",
    "        ),\n",
    "        data=path_df,\n",
    "        size=1,\n",
    "        color=\"gray\",\n",
    "        arrow=arrow(type=\"closed\", length=15, angle=15),\n",
    "        spacer=5\n",
    "        + 1,  # Avoids arrowheads being sunk into points (+1 as circles are size 1)\n",
    "    )\n",
    "    + geom_point(shape=21, color=\"gray\", fill=\"#c28dc3\", size=5)\n",
    "    + geom_text(\n",
    "        aes(label=\"Year\"),\n",
    "        data=df[df[\"Year\"].isin([min_yr, max_yr])],\n",
    "        position=position_nudge(y=0.3),\n",
    "    )\n",
    "    + labs(x=\"Unemployment rate, %\", y=\"Vacancy rate, %\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bubble plot\n",
    "\n",
    "This is a scatter plot where the size of the point carries an extra dimension of information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "scat = ax.scatter(\n",
    "    cars[\"Horsepower\"], cars[\"Miles_per_Gallon\"], s=cars[\"Displacement\"], alpha=0.4\n",
    ")\n",
    "ax.set_ylabel(\"Miles per Gallon\")\n",
    "ax.set_xlabel(\"Horsepower\")\n",
    "ax.legend(\n",
    "    *scat.legend_elements(prop=\"sizes\", num=4),\n",
    "    loc=\"upper right\",\n",
    "    title=\"Displacement\",\n",
    "    frameon=False,\n",
    ")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(cars, x=\"Horsepower\", y=\"Miles_per_Gallon\", pointsize=\"Displacement\").add(\n",
    "        so.Dot()\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(cars, aes(x=\"Horsepower\", y=\"Miles_per_Gallon\", size=\"Displacement\"))\n",
    "    + geom_point()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(cars).mark_circle().encode(\n",
    "    x=\"Horsepower\", y=\"Miles_per_Gallon\", size=\"Displacement\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adding a new col is easiest way to get displacement into legend with plotly:\n",
    "cars[\"Displacement_Size\"] = pd.cut(cars[\"Displacement\"], bins=4)\n",
    "fig = px.scatter(\n",
    "    cars,\n",
    "    x=\"Horsepower\",\n",
    "    y=\"Miles_per_Gallon\",\n",
    "    size=\"Displacement\",\n",
    "    color=\"Displacement_Size\",\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Line plot\n",
    "\n",
    "First, let's get some data on GDP growth:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "todays_date = datetime.datetime.now().strftime(\"%Y-%m-%d\")\n",
    "fred_df = web.DataReader([\"GDPC1\", \"NGDPRSAXDCGBQ\"], \"fred\", \"1999-01-01\", \"2021-12-31\")\n",
    "fred_df.columns = [\"US\", \"UK\"]\n",
    "fred_df.index.name = \"Date\"\n",
    "fred_df = 100 * fred_df.pct_change(4)\n",
    "df = pd.melt(\n",
    "    fred_df.reset_index(),\n",
    "    id_vars=[\"Date\"],\n",
    "    value_vars=fred_df.columns,\n",
    "    value_name=\"Real GDP growth, %\",\n",
    "    var_name=\"Country\",\n",
    ")\n",
    "df = df.set_index(\"Date\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "Note that **Matplotlib** prefers data to be one variable per column, in which case we could have just run\n",
    "\n",
    "```python\n",
    "fig, ax = plt.subplots()\n",
    "df.plot(ax=ax)\n",
    "ax.set_title('Real GDP growth, %', loc='right')\n",
    "ax.yaxis.tick_right()\n",
    "```\n",
    "\n",
    "but we are working with tidy data here, so we'll do the plotting slightly differently."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "for i, country in enumerate(df[\"Country\"].unique()):\n",
    "    df_sub = df[df[\"Country\"] == country]\n",
    "    ax.plot(df_sub.index, df_sub[\"Real GDP growth, %\"], label=country, lw=2)\n",
    "ax.set_title(\"Real GDP growth per capita, %\", loc=\"right\")\n",
    "ax.yaxis.tick_right()\n",
    "ax.spines[\"right\"].set_visible(True)\n",
    "ax.spines[\"left\"].set_visible(False)\n",
    "ax.legend(loc=\"lower left\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "Note that [only *some* **seaborn** commands currently support the use of named indexes](https://seaborn.pydata.org/tutorial/data_structure.html), so we use `df.reset_index()` to make the 'Date' index into a regular column in the snippet below (although in recent versions of **seaborn**, `lineplot()` would actually work fine with `data=df`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "y_var = \"Real GDP growth, %\"\n",
    "sns.lineplot(x=\"Date\", y=y_var, hue=\"Country\", data=df.reset_index(), ax=ax)\n",
    "ax.yaxis.tick_right()\n",
    "ax.spines[\"right\"].set_visible(True)\n",
    "ax.spines[\"left\"].set_visible(False)\n",
    "ax.set_ylabel(\"\")\n",
    "ax.set_title(y_var)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(df.reset_index(), x=\"Date\", y=\"Real GDP growth, %\", color=\"Country\").add(\n",
    "        so.Line()\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(df.reset_index(), aes(x=\"Date\", y=\"Real GDP growth, %\", color=\"Country\"))\n",
    "    + geom_line(size=1)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(df.reset_index()).mark_line().encode(\n",
    "    x=\"Date:T\",\n",
    "    y=\"Real GDP growth, %\",\n",
    "    color=\"Country\",\n",
    "    strokeDash=\"Country\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.line(\n",
    "    df.reset_index(),\n",
    "    x=\"Date\",\n",
    "    y=\"Real GDP growth, %\",\n",
    "    color=\"Country\",\n",
    "    line_dash=\"Country\",\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bar chart\n",
    "\n",
    "Let's see a bar chart, using the 'barley' dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "barley = data.barley()\n",
    "barley = pd.DataFrame(barley.groupby([\"site\"])[\"yield\"].sum())\n",
    "barley.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "Just remove the 'h' in `ax.barh()` to get a vertical plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.barh(barley[\"yield\"].index, barley[\"yield\"], 0.35)\n",
    "ax.set_xlabel(\"Yield\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n",
    "Just switch x and y variables to get a vertical plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(barley.reset_index(), x=\"yield\", y=\"site\", color=\"site\").add(\n",
    "        so.Bar(), so.Agg()\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n",
    "Just omit `coord_flip()` to get a vertical plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(barley.reset_index(), aes(x=\"site\", y=\"yield\", fill=\"site\"))\n",
    "    + geom_bar(stat=\"identity\")\n",
    "    + coord_flip()\n",
    "    + theme(legend_position=\"none\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n",
    "Just switch x and y to get a vertical plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(barley.reset_index()).mark_bar().encode(\n",
    "    y=\"site\",\n",
    "    x=\"yield\",\n",
    ").properties(\n",
    "    width=alt.Step(40)  # controls width of bar.\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.bar(barley.reset_index(), y=\"site\", x=\"yield\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Grouped bar chart\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "barley = data.barley()\n",
    "barley = pd.DataFrame(barley.groupby([\"site\", \"year\"])[\"yield\"].sum()).reset_index()\n",
    "barley.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = barley[\"site\"].unique()\n",
    "y = np.arange(len(labels))  # the label locations\n",
    "width = 0.35  # the width of the bars\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "ax.barh(y - width / 2, barley.loc[barley[\"year\"] == 1931, \"yield\"], width, label=\"1931\")\n",
    "ax.barh(y + width / 2, barley.loc[barley[\"year\"] == 1932, \"yield\"], width, label=\"1932\")\n",
    "\n",
    "# Add some text for labels, title and custom x-axis tick labels, etc.\n",
    "ax.set_xlabel(\"Yield\")\n",
    "ax.set_yticks(y)\n",
    "ax.set_yticklabels(labels)\n",
    "ax.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "barley[\"year\"] = barley[\"year\"].astype(\"category\")  # to force category\n",
    "\n",
    "(\n",
    "    so.Plot(barley.reset_index(), x=\"yield\", y=\"site\", color=\"year\").add(\n",
    "        so.Bar(), so.Dodge()\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(barley, aes(x=\"site\", y=\"yield\", group=\"year\", fill=as_discrete(\"year\")))\n",
    "    + geom_bar(position=\"dodge\", stat=\"identity\")\n",
    "    + coord_flip()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(barley.reset_index()).mark_bar().encode(\n",
    "    y=\"year:O\", x=\"yield\", color=\"year:N\", row=\"site:N\"\n",
    ").properties(\n",
    "    width=alt.Step(40)  # controls width of bar.\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "px_barley = barley.reset_index()\n",
    "# This prevents plotly from using a continuous scale for year\n",
    "px_barley[\"year\"] = px_barley[\"year\"].astype(\"category\")\n",
    "fig = px.bar(px_barley, y=\"site\", x=\"yield\", barmode=\"group\", color=\"year\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacked bar chart"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = barley[\"site\"].unique()\n",
    "y = np.arange(len(labels))  # the label locations\n",
    "width = 0.35  # the width (or height) of the bars\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "ax.barh(y, barley.loc[barley[\"year\"] == 1931, \"yield\"], width, label=\"1931\")\n",
    "ax.barh(\n",
    "    y,\n",
    "    barley.loc[barley[\"year\"] == 1932, \"yield\"],\n",
    "    width,\n",
    "    label=\"1932\",\n",
    "    left=barley.loc[barley[\"year\"] == 1931, \"yield\"],\n",
    ")\n",
    "\n",
    "# Add some text for labels, title and custom x-axis tick labels, etc.\n",
    "ax.set_xlabel(\"Yield\")\n",
    "ax.set_yticks(y)\n",
    "ax.set_yticklabels(labels)\n",
    "ax.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "barley[\"year\"] = barley[\"year\"].astype(\"category\")  # to force category\n",
    "(\n",
    "    so.Plot(barley.reset_index(), x=\"yield\", y=\"site\", color=\"year\").add(\n",
    "        so.Bar(), so.Stack()\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(barley, aes(x=\"site\", y=\"yield\", fill=as_discrete(\"year\")))\n",
    "    + geom_bar(stat=\"identity\")\n",
    "    + coord_flip()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(barley.reset_index()).mark_bar().encode(\n",
    "    y=\"site\",\n",
    "    x=\"yield\",\n",
    "    color=\"year:N\",\n",
    ").properties(\n",
    "    width=alt.Step(40)  # controls width of bar.\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.bar(px_barley, y=\"site\", x=\"yield\", barmode=\"relative\", color=\"year\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Diverging stacked bar chart\n",
    "\n",
    "First, let's create some data to use in our examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "category_names = [\n",
    "    \"Strongly disagree\",\n",
    "    \"Disagree\",\n",
    "    \"Neither agree nor disagree\",\n",
    "    \"Agree\",\n",
    "    \"Strongly agree\",\n",
    "]\n",
    "results = [\n",
    "    [10, 15, 17, 32, 26],\n",
    "    [26, 22, 29, 10, 13],\n",
    "    [35, 37, 7, 2, 19],\n",
    "    [32, 11, 9, 15, 33],\n",
    "    [21, 29, 5, 5, 40],\n",
    "    [8, 19, 5, 30, 38],\n",
    "]\n",
    "\n",
    "likert_df = pd.DataFrame(\n",
    "    results, columns=category_names, index=[f\"Question {i}\" for i in range(1, 7)]\n",
    ")\n",
    "likert_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "middle_index = likert_df.shape[1] // 2\n",
    "offsets = (\n",
    "    likert_df.iloc[:, range(middle_index)].sum(axis=1)\n",
    "    + likert_df.iloc[:, middle_index] / 2\n",
    ")\n",
    "category_colors = plt.get_cmap(\"coolwarm_r\")(\n",
    "    np.linspace(0.15, 0.85, likert_df.shape[1])\n",
    ")\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 5))\n",
    "\n",
    "# Plot Bars\n",
    "for i, (colname, color) in enumerate(zip(likert_df.columns, category_colors)):\n",
    "    widths = likert_df.iloc[:, i]\n",
    "    starts = likert_df.cumsum(axis=1).iloc[:, i] - widths - offsets\n",
    "    rects = ax.barh(\n",
    "        likert_df.index, widths, left=starts, height=0.5, label=colname, color=color\n",
    "    )\n",
    "\n",
    "# Add Zero Reference Line\n",
    "ax.axvline(0, linestyle=\"--\", color=\"black\", alpha=1, zorder=0, lw=0.3)\n",
    "\n",
    "# X Axis\n",
    "ax.set_xlim(-90, 90)\n",
    "ax.set_xticks(np.arange(-90, 91, 10))\n",
    "ax.xaxis.set_major_formatter(lambda x, pos: str(abs(int(x))))\n",
    "\n",
    "# Y Axis\n",
    "ax.invert_yaxis()\n",
    "\n",
    "# Remove spines\n",
    "ax.spines[\"right\"].set_visible(False)\n",
    "ax.spines[\"top\"].set_visible(False)\n",
    "ax.spines[\"left\"].set_visible(False)\n",
    "\n",
    "# Legend\n",
    "ax.legend(\n",
    "    ncol=len(category_names),\n",
    "    bbox_to_anchor=(0, 1),\n",
    "    loc=\"lower left\",\n",
    "    fontsize=\"small\",\n",
    "    frameon=False,\n",
    ")\n",
    "\n",
    "# Set Background Color\n",
    "fig.set_facecolor(\"#FFFFFF\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kernel density estimate\n",
    "\n",
    "We'll use the diamonds dataset to demonstrate this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds = sns.load_dataset(\"diamonds\").sample(1000)\n",
    "diamonds.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "Technically, there is a way to do this but it's pretty inelegant if you want a quick plot. That's because **matplotlib** doesn't do the density estimation itself. [Jake Vanderplas](https://jakevdp.github.io/PythonDataScienceHandbook/05.13-kernel-density-estimation.html) has a nice example but as it relies on a few extra libraries, we won't reproduce it here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note that there isn't a clear way to do this in the seaborn objects API yet\n",
    "sns.displot(diamonds, x=\"carat\", kind=\"kde\", hue=\"cut\", fill=True);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ggplot(diamonds, aes(x=\"carat\", fill=\"cut\", colour=\"cut\")) + geom_density(alpha=0.5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(diamonds).transform_density(\n",
    "    density=\"carat\", as_=[\"carat\", \"density\"], groupby=[\"cut\"]\n",
    ").mark_area(fillOpacity=0.5).encode(\n",
    "    x=\"carat:Q\",\n",
    "    y=\"density:Q\",\n",
    "    color=\"cut:N\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.figure_factory as ff\n",
    "\n",
    "px_di = diamonds.pivot(columns=\"cut\", values=\"carat\")\n",
    "ff.create_distplot(\n",
    "    [px_di[c].dropna() for c in px_di.columns],\n",
    "    group_labels=px_di.columns,\n",
    "    show_rug=False,\n",
    "    show_hist=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Histogram or probability density function\n",
    "\n",
    "For this, let's go back to the penguins dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "penguins = sns.load_dataset(\"penguins\")\n",
    "penguins.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "The `density=` keyword parameter decides whether to create counts or a probability density function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.hist(penguins[\"flipper_length_mm\"], bins=30, density=True, edgecolor=\"k\")\n",
    "ax.set_xlabel(\"Flipper length (mm)\")\n",
    "ax.set_ylabel(\"Probability density\")\n",
    "fig.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(penguins, x=\"flipper_length_mm\").add(\n",
    "        so.Bars(), so.Hist(bins=30, stat=\"density\")\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(penguins, aes(x=\"flipper_length_mm\"))\n",
    "    + geom_histogram(bins=30)  # specify the binwidth\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(penguins).mark_bar().encode(\n",
    "    alt.X(\"flipper_length_mm:Q\", bin=True),\n",
    "    y=\"count()\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.histogram(penguins, x=\"flipper_length_mm\", nbins=30)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Marginal histograms\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Maplotlib\n",
    "\n",
    "[Jaker Vanderplas's excellent notes](https://jakevdp.github.io/PythonDataScienceHandbook/04.08-multiple-subplots.html) have a great example of this, but now there's an easier way to do it with Matplotlib's new `constrained_layout` options."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(constrained_layout=True)\n",
    "# Create a layout with 3 panels in the given ratios\n",
    "axes_dict = fig.subplot_mosaic(\n",
    "    [[\".\", \"histx\"], [\"histy\", \"scat\"]],\n",
    "    gridspec_kw={\"width_ratios\": [1, 7], \"height_ratios\": [2, 7]},\n",
    ")\n",
    "# Glue all the relevant axes together\n",
    "axes_dict[\"histy\"].invert_xaxis()\n",
    "axes_dict[\"histx\"].sharex(axes_dict[\"scat\"])\n",
    "axes_dict[\"histy\"].sharey(axes_dict[\"scat\"])\n",
    "# Plot the data\n",
    "axes_dict[\"scat\"].scatter(penguins[\"bill_length_mm\"], penguins[\"bill_depth_mm\"])\n",
    "axes_dict[\"histx\"].hist(penguins[\"bill_length_mm\"])\n",
    "axes_dict[\"histy\"].hist(penguins[\"bill_depth_mm\"], orientation=\"horizontal\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.jointplot(data=penguins, x=\"bill_length_mm\", y=\"bill_depth_mm\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lets_plot.bistro.joint import *\n",
    "\n",
    "(\n",
    "    joint_plot(penguins, x=\"bill_length_mm\", y=\"bill_depth_mm\", reg_line=False)\n",
    "    + labs(x=\"Bill length (mm)\", y=\"Bill depth (mm)\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n",
    "\n",
    "This is a bit fiddly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base = alt.Chart(penguins)\n",
    "\n",
    "xscale = alt.Scale(domain=(20, 60))\n",
    "yscale = alt.Scale(domain=(10, 30))\n",
    "\n",
    "area_args = {\"opacity\": 0.5, \"interpolate\": \"step\"}\n",
    "\n",
    "points = base.mark_circle().encode(\n",
    "    alt.X(\"bill_length_mm\", scale=xscale), alt.Y(\"bill_depth_mm\", scale=yscale)\n",
    ")\n",
    "\n",
    "top_hist = (\n",
    "    base.mark_area(**area_args)\n",
    "    .encode(\n",
    "        alt.X(\n",
    "            \"bill_length_mm:Q\",\n",
    "            # when using bins, the axis scale is set through\n",
    "            # the bin extent, so we do not specify the scale here\n",
    "            # (which would be ignored anyway)\n",
    "            bin=alt.Bin(maxbins=30, extent=xscale.domain),\n",
    "            stack=None,\n",
    "            title=\"\",\n",
    "        ),\n",
    "        alt.Y(\"count()\", stack=None, title=\"\"),\n",
    "    )\n",
    "    .properties(height=60)\n",
    ")\n",
    "\n",
    "right_hist = (\n",
    "    base.mark_area(**area_args)\n",
    "    .encode(\n",
    "        alt.Y(\n",
    "            \"bill_depth_mm:Q\",\n",
    "            bin=alt.Bin(maxbins=30, extent=yscale.domain),\n",
    "            stack=None,\n",
    "            title=\"\",\n",
    "        ),\n",
    "        alt.X(\"count()\", stack=None, title=\"\"),\n",
    "    )\n",
    "    .properties(width=60)\n",
    ")\n",
    "\n",
    "top_hist & (points | right_hist)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(\n",
    "    penguins,\n",
    "    x=\"bill_length_mm\",\n",
    "    y=\"bill_depth_mm\",\n",
    "    marginal_x=\"histogram\",\n",
    "    marginal_y=\"histogram\",\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Heatmap\n",
    "\n",
    "Heatmaps, or sometimes known as correlation maps, represent data in 3 dimensions by having two axes that forms a grid showing colour that corresponds to (usually) continuous values.\n",
    "\n",
    "We'll use the flights data to show the number of passengers by month-year:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights = sns.load_dataset(\"flights\")\n",
    "flights = flights.pivot(index=\"month\", columns=\"year\", values=\"passengers\").T\n",
    "flights.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "im = ax.imshow(flights.values, cmap=\"inferno\")\n",
    "cbar = ax.figure.colorbar(im, ax=ax)\n",
    "ax.set_xticks(np.arange(len(flights.columns)))\n",
    "ax.set_yticks(np.arange(len(flights.index)))\n",
    "# Labels\n",
    "ax.set_xticklabels(flights.columns, rotation=90)\n",
    "ax.set_yticklabels(flights.index)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.heatmap(flights);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n",
    "Lets-Plot uses tidy data, rather than the wide data preferred by **matplotlib**, so we need to first get the original format of the flights data back:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights = sns.load_dataset(\"flights\")\n",
    "(\n",
    "    ggplot(flights, aes(\"month\", as_discrete(\"year\"), fill=\"passengers\"))\n",
    "    + geom_tile()\n",
    "    + scale_y_reverse()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(flights).mark_rect().encode(\n",
    "    x=alt.X(\"month\", type=\"nominal\", sort=None), y=\"year:O\", color=\"passengers:Q\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boxplot\n",
    "\n",
    "Let's use the tips dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tips = sns.load_dataset(\"tips\")\n",
    "tips.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "There isn't a very direct way to create multiple box plots of different data in matplotlib in the case where the groups are unbalanced, so we create several different boxplot objects.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "colormap = plt.cm.Set1\n",
    "colorst = [colormap(i) for i in np.linspace(0, 0.9, len(tips[\"time\"].unique()))]\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "for i, grp in enumerate(tips[\"time\"].unique()):\n",
    "    bplot = ax.boxplot(\n",
    "        tips.loc[tips[\"time\"] == grp, \"tip\"],\n",
    "        positions=[i],\n",
    "        vert=True,  # vertical box alignment\n",
    "        patch_artist=True,  # fill with color\n",
    "        labels=[grp],\n",
    "    )  # X label\n",
    "    for patch in bplot[\"boxes\"]:\n",
    "        patch.set_facecolor(colorst[i])\n",
    "\n",
    "ax.set_ylabel(\"Tip\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.boxplot(data=tips, x=\"time\", y=\"tip\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ggplot(tips) + geom_boxplot(aes(y=\"tip\", x=\"time\", fill=\"time\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(tips).mark_boxplot(size=50).encode(\n",
    "    x=\"time:N\", y=\"tip:Q\", color=\"time:N\"\n",
    ").properties(width=300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.box(tips, x=\"time\", y=\"tip\", color=\"time\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Violin plot\n",
    "\n",
    "We'll use the same data as before, the tips dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "colormap = plt.cm.Set1\n",
    "colorst = [colormap(i) for i in np.linspace(0, 0.9, len(tips[\"time\"].unique()))]\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "for i, grp in enumerate(tips[\"time\"].unique()):\n",
    "    vplot = ax.violinplot(\n",
    "        tips.loc[tips[\"time\"] == grp, \"tip\"], positions=[i], vert=True\n",
    "    )\n",
    "labels = list(tips[\"time\"].unique())\n",
    "ax.set_xticks(np.arange(len(labels)))\n",
    "ax.set_xticklabels(labels)\n",
    "ax.set_ylabel(\"Tip\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.violinplot(data=tips, x=\"time\", y=\"tip\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ggplot(tips, aes(x=\"time\", y=\"tip\", fill=\"time\")) + geom_violin())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(tips).transform_density(\n",
    "    \"tip\", as_=[\"tip\", \"density\"], groupby=[\"time\"]\n",
    ").mark_area(orient=\"horizontal\").encode(\n",
    "    y=\"tip:Q\",\n",
    "    color=\"time:N\",\n",
    "    x=alt.X(\n",
    "        \"density:Q\",\n",
    "        stack=\"center\",\n",
    "        impute=None,\n",
    "        title=None,\n",
    "        axis=alt.Axis(labels=False, values=[0], grid=False, ticks=True),\n",
    "    ),\n",
    "    column=alt.Column(\n",
    "        \"time:N\",\n",
    "        header=alt.Header(\n",
    "            titleOrient=\"bottom\",\n",
    "            labelOrient=\"bottom\",\n",
    "            labelPadding=0,\n",
    "        ),\n",
    "    ),\n",
    ").properties(width=100).configure_facet(spacing=0).configure_view(stroke=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.violin(\n",
    "    tips,\n",
    "    y=\"tip\",\n",
    "    x=\"time\",\n",
    "    color=\"time\",\n",
    "    box=True,\n",
    "    points=\"all\",\n",
    "    hover_data=tips.columns,\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lollipop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "planets = sns.load_dataset(\"planets\").groupby(\"year\")[\"number\"].count()\n",
    "planets.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.stem(planets.index, planets)\n",
    "ax.yaxis.tick_right()\n",
    "ax.spines[\"left\"].set_visible(False)\n",
    "ax.set_ylim(0, 200)\n",
    "ax.set_title(\"Number of exoplanets discovered per year\")\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(planets.reset_index(), x=\"year\", y=\"number\")\n",
    "    .add(so.Dot(), so.Agg(\"sum\"))\n",
    "    .add(so.Bar(width=0.1), so.Agg(\"sum\"))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(planets.reset_index(), aes(x=\"year\", y=\"number\"))\n",
    "    + geom_lollipop()\n",
    "    + ggtitle(\"Number of exoplanets discovered per year\")\n",
    "    + scale_x_continuous(format=\"d\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "px_df = planets.reset_index()\n",
    "\n",
    "fig1 = go.Figure()\n",
    "# Draw points\n",
    "fig1.add_trace(\n",
    "    go.Scatter(\n",
    "        x=px_df[\"year\"],\n",
    "        y=px_df[\"number\"],\n",
    "        mode=\"markers\",\n",
    "        marker_color=\"darkblue\",\n",
    "        marker_size=10,\n",
    "    )\n",
    ")\n",
    "# Draw lines\n",
    "for index, row in px_df.iterrows():\n",
    "    fig1.add_shape(type=\"line\", x0=row[\"year\"], y0=0, x1=row[\"year\"], y1=row[\"number\"])\n",
    "fig1.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overlapping Area plot\n",
    "\n",
    "For this, let's look at the dominance of the three most used methods for detecting exoplanets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "planets = sns.load_dataset(\"planets\")\n",
    "most_pop_methods = (\n",
    "    planets.groupby([\"method\"])[\"number\"]\n",
    "    .sum()\n",
    "    .sort_values(ascending=False)\n",
    "    .index[:3]\n",
    "    .values\n",
    ")\n",
    "planets = planets[planets[\"method\"].isin(most_pop_methods)]\n",
    "planets.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "The easiest way to do this in matplotlib is to adjust the data a bit first and then use the built-in **pandas** plot function. (This is true in other cases too, but in this case it's much more complex otherwise)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    planets.groupby([\"year\", \"method\"])[\"number\"]\n",
    "    .sum()\n",
    "    .unstack()\n",
    "    .plot.area(alpha=0.6, ylim=(0, None))\n",
    "    .set_title(\"Planets dicovered by top 3 methods\", loc=\"left\")\n",
    ");"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(\n",
    "        planets.groupby([\"year\", \"method\"])[\"number\"].sum().reset_index(),\n",
    "        x=\"year\",\n",
    "        y=\"number\",\n",
    "        color=\"method\",\n",
    "    ).add(so.Area(alpha=0.3), so.Agg(), so.Stack())\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(\n",
    "        planets.groupby([\"year\", \"method\"])[\"number\"].sum().reset_index(),\n",
    "        aes(x=\"year\", y=\"number\", fill=\"method\", group=\"method\", color=\"method\"),\n",
    "    )\n",
    "    + geom_area(stat=\"identity\", alpha=0.5)\n",
    "    + scale_x_continuous(format=\"d\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(\n",
    "    planets.groupby([\"year\", \"method\"])[\"number\"]\n",
    "    .sum()\n",
    "    .reset_index()\n",
    "    .assign(\n",
    "        year=lambda x: pd.to_datetime(x[\"year\"], format=\"%Y\")\n",
    "        + pd.tseries.offsets.YearEnd()\n",
    "    )\n",
    ").mark_area().encode(x=\"year:T\", y=\"number:Q\", color=\"method:N\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slope chart\n",
    "\n",
    "A slope chart has two points connected by a line and is good for indicating how relationships between variables have changed over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    \"https://raw.githubusercontent.com/selva86/datasets/master/gdppercap.csv\"\n",
    ")\n",
    "df = pd.melt(\n",
    "    df,\n",
    "    id_vars=[\"continent\"],\n",
    "    value_vars=df.columns[1:],\n",
    "    value_name=\"GDP per capita\",\n",
    "    var_name=\"Year\",\n",
    ").rename(columns={\"continent\": \"Continent\"})\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "There isn't an off-the-shelf way to do this in matplotlib but the example below shows that, with matplotlib, where there's a will there's a way! It's where the 'build-what-you-want' comes into its own. Note that the functino that's defined returns an `Axes` object so that you can do further processing and tweaking as you like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import lines as mlines\n",
    "\n",
    "\n",
    "def slope_plot(data, x, y, group, before_txt=\"Before\", after_txt=\"After\"):\n",
    "    if len(data[x].unique()) != 2:\n",
    "        raise ValueError(\"Slope plot must have two unique periods.\")\n",
    "    wide_data = data[[x, y, group]].pivot(index=group, columns=x, values=y)\n",
    "    x_names = list(wide_data.columns)\n",
    "\n",
    "    fig, ax = plt.subplots()\n",
    "\n",
    "    def newline(p1, p2, color=\"black\"):\n",
    "        ax = plt.gca()\n",
    "        line = mlines.Line2D(\n",
    "            [p1[0], p2[0]],\n",
    "            [p1[1], p2[1]],\n",
    "            color=\"red\" if p1[1] - p2[1] > 0 else \"green\",\n",
    "            marker=\"o\",\n",
    "            markersize=6,\n",
    "        )\n",
    "        ax.add_line(line)\n",
    "        return line\n",
    "\n",
    "    # Vertical Lines\n",
    "    y_min = data[y].min()\n",
    "    y_max = data[y].max()\n",
    "    ax.vlines(\n",
    "        x=1,\n",
    "        ymin=y_min,\n",
    "        ymax=y_max,\n",
    "        color=\"black\",\n",
    "        alpha=0.7,\n",
    "        linewidth=1,\n",
    "        linestyles=\"dotted\",\n",
    "    )\n",
    "    ax.vlines(\n",
    "        x=3,\n",
    "        ymin=y_min,\n",
    "        ymax=y_max,\n",
    "        color=\"black\",\n",
    "        alpha=0.7,\n",
    "        linewidth=1,\n",
    "        linestyles=\"dotted\",\n",
    "    )\n",
    "    # Points\n",
    "    ax.scatter(\n",
    "        y=wide_data[x_names[0]],\n",
    "        x=np.repeat(1, wide_data.shape[0]),\n",
    "        s=15,\n",
    "        color=\"black\",\n",
    "        alpha=0.7,\n",
    "    )\n",
    "    ax.scatter(\n",
    "        y=wide_data[x_names[1]],\n",
    "        x=np.repeat(3, wide_data.shape[0]),\n",
    "        s=15,\n",
    "        color=\"black\",\n",
    "        alpha=0.7,\n",
    "    )\n",
    "    # Line Segmentsand Annotation\n",
    "    for p1, p2, c in zip(wide_data[x_names[0]], wide_data[x_names[1]], wide_data.index):\n",
    "        newline([1, p1], [3, p2])\n",
    "        ax.text(\n",
    "            1 - 0.05,\n",
    "            p1,\n",
    "            c,\n",
    "            horizontalalignment=\"right\",\n",
    "            verticalalignment=\"center\",\n",
    "            fontdict={\"size\": 14},\n",
    "        )\n",
    "        ax.text(\n",
    "            3 + 0.05,\n",
    "            p2,\n",
    "            c,\n",
    "            horizontalalignment=\"left\",\n",
    "            verticalalignment=\"center\",\n",
    "            fontdict={\"size\": 14},\n",
    "        )\n",
    "    # 'Before' and 'After' Annotations\n",
    "    ax.text(\n",
    "        1 - 0.05,\n",
    "        y_max + abs(y_max) * 0.1,\n",
    "        before_txt,\n",
    "        horizontalalignment=\"right\",\n",
    "        verticalalignment=\"center\",\n",
    "        fontdict={\"size\": 16, \"weight\": 700},\n",
    "    )\n",
    "    ax.text(\n",
    "        3 + 0.05,\n",
    "        y_max + abs(y_max) * 0.1,\n",
    "        after_txt,\n",
    "        horizontalalignment=\"left\",\n",
    "        verticalalignment=\"center\",\n",
    "        fontdict={\"size\": 16, \"weight\": 700},\n",
    "    )\n",
    "    # Decoration\n",
    "    ax.set(\n",
    "        xlim=(0, 4), ylabel=y, ylim=(y_min - 0.1 * abs(y_min), y_max + abs(y_max) * 0.1)\n",
    "    )\n",
    "    ax.set_xticks([1, 3])\n",
    "    ax.set_xticklabels(x_names)\n",
    "    # Lighten borders\n",
    "    for ax_pos in [\"top\", \"bottom\", \"right\", \"left\"]:\n",
    "        ax.spines[ax_pos].set_visible(False)\n",
    "    return ax\n",
    "\n",
    "\n",
    "slope_plot(df, x=\"Year\", y=\"GDP per capita\", group=\"Continent\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(df, x=\"Year\", y=\"GDP per capita\", color=\"Continent\")\n",
    "    .add(so.Line(marker=\"o\"), so.Agg())\n",
    "    .add(so.Range())\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(df, aes(x=\"Year\", y=\"GDP per capita\", group=\"Continent\"))\n",
    "    + geom_line(aes(color=\"Continent\"), size=1)\n",
    "    + geom_point(aes(color=\"Continent\"), size=4)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alt.Chart(df).mark_line().encode(x=\"Year:O\", y=\"GDP per capita\", color=\"Continent\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "yr_names = [int(x) for x in df[\"Year\"].unique()]\n",
    "px_df = (\n",
    "    df.pivot(index=\"Continent\", columns=\"Year\", values=\"GDP per capita\")\n",
    "    .reset_index()\n",
    "    .rename(columns=dict(zip(df[\"Year\"].unique(), range(len(df[\"Year\"].unique())))))\n",
    ")\n",
    "\n",
    "x_offset = 5\n",
    "\n",
    "fig1 = go.Figure()\n",
    "# Draw lines\n",
    "for index, row in px_df.iterrows():\n",
    "    fig1.add_shape(\n",
    "        type=\"line\",\n",
    "        x0=yr_names[0],\n",
    "        y0=row[0],\n",
    "        x1=yr_names[1],\n",
    "        y1=row[1],\n",
    "        name=row[\"Continent\"],\n",
    "        line=dict(color=px.colors.qualitative.Plotly[index]),\n",
    "    )\n",
    "    fig1.add_trace(\n",
    "        go.Scatter(\n",
    "            x=[yr_names[0]],\n",
    "            y=[row[0]],\n",
    "            text=row[\"Continent\"],\n",
    "            mode=\"text\",\n",
    "            name=None,\n",
    "        )\n",
    "    )\n",
    "\n",
    "\n",
    "fig1.update_xaxes(range=[yr_names[0] - x_offset, yr_names[1] + x_offset])\n",
    "fig1.update_yaxes(\n",
    "    range=[px_df[[0, 1]].min().min() * 0.8, px_df[[0, 1]].max().max() * 1.2]\n",
    ")\n",
    "fig1.update_layout(showlegend=False)\n",
    "fig1.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dumbbell Plot\n",
    "\n",
    "These are excellent for showing a change in time with a large number of categories, as we will do here with continents and mean GDP per capita."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    \"https://raw.githubusercontent.com/selva86/datasets/master/gdppercap.csv\"\n",
    ")\n",
    "df = pd.melt(\n",
    "    df,\n",
    "    id_vars=[\"continent\"],\n",
    "    value_vars=df.columns[1:],\n",
    "    value_name=\"GDP per capita\",\n",
    "    var_name=\"Year\",\n",
    ").rename(columns={\"continent\": \"Continent\"})\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "Again, no off-the-shelf method--but that's no problem when you can build it yourself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def dumbbell_plot(data, x, y, change):\n",
    "    if len(data[x].unique()) != 2:\n",
    "        raise ValueError(\"Dumbbell plot must have two unique periods.\")\n",
    "    if not isinstance(data[y].iloc[0], str):\n",
    "        raise ValueError(\"Dumbbell plot y variable only works with category values.\")\n",
    "    wide_data = data[[x, y, change]].pivot(index=y, columns=x, values=change)\n",
    "    x_names = list(wide_data.columns)\n",
    "    y_names = list(wide_data.index)\n",
    "\n",
    "    def newline(p1, p2, color=\"black\"):\n",
    "        ax = plt.gca()\n",
    "        line = mlines.Line2D([p1[0], p2[0]], [p1[1], p2[1]], color=\"skyblue\", zorder=0)\n",
    "        ax.add_line(line)\n",
    "        return line\n",
    "\n",
    "    fig, ax = plt.subplots()\n",
    "    # Points\n",
    "    ax.scatter(\n",
    "        y=range(len(y_names)),\n",
    "        x=wide_data[x_names[1]],\n",
    "        s=50,\n",
    "        color=\"#0e668b\",\n",
    "        alpha=0.9,\n",
    "        zorder=2,\n",
    "        label=x_names[1],\n",
    "    )\n",
    "    ax.scatter(\n",
    "        y=range(len(y_names)),\n",
    "        x=wide_data[x_names[0]],\n",
    "        s=50,\n",
    "        color=\"#a3c4dc\",\n",
    "        alpha=0.9,\n",
    "        zorder=1,\n",
    "        label=x_names[0],\n",
    "    )\n",
    "    # Line segments\n",
    "    for i, p1, p2 in zip(\n",
    "        range(len(y_names)), wide_data[x_names[0]], wide_data[x_names[1]]\n",
    "    ):\n",
    "        newline([p1, i], [p2, i])\n",
    "    ax.set_yticks(range(len(y_names)))\n",
    "    ax.set_yticklabels(y_names)\n",
    "    # Decoration\n",
    "    # Lighten borders\n",
    "    for ax_pos in [\"top\", \"right\", \"left\"]:\n",
    "        ax.spines[ax_pos].set_visible(False)\n",
    "    ax.set_xlabel(change)\n",
    "    ax.legend(frameon=False, loc=\"lower right\")\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "dumbbell_plot(df, x=\"Year\", y=\"Continent\", change=\"GDP per capita\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    so.Plot(df, y=\"Continent\", x=\"GDP per capita\", color=\"Year\").add(\n",
    "        so.Dots(pointsize=10, fillalpha=1)\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ggplot(df, aes(y=\"Continent\", x=\"GDP per capita\", group=\"Continent\"))\n",
    "    + geom_line(color=\"black\", size=2)\n",
    "    + geom_point(aes(color=\"Year\"), size=5)\n",
    "    + ggsize(400, 500)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "fig1 = go.Figure()\n",
    "\n",
    "yr_names = df[\"Year\"].unique()\n",
    "\n",
    "\n",
    "# Draw lines\n",
    "for i, cont in enumerate(df[\"Continent\"].unique()):\n",
    "    cdf = df[df[\"Continent\"] == cont]\n",
    "    fig1.add_shape(\n",
    "        type=\"line\",\n",
    "        x0=cdf.loc[cdf[\"Year\"] == yr_names[0], \"GDP per capita\"].values[0],\n",
    "        y0=cont,\n",
    "        x1=cdf.loc[cdf[\"Year\"] == yr_names[1], \"GDP per capita\"].values[0],\n",
    "        y1=cont,\n",
    "        line=dict(color=px.colors.qualitative.Plotly[0], width=2),\n",
    "    )\n",
    "# Draw points\n",
    "for i, year in enumerate(yr_names):\n",
    "    yrdf = df[df[\"Year\"] == year]\n",
    "    fig1.add_trace(\n",
    "        go.Scatter(\n",
    "            y=yrdf[\"Continent\"],\n",
    "            x=yrdf[\"GDP per capita\"],\n",
    "            mode=\"markers\",\n",
    "            name=year,\n",
    "            marker_color=px.colors.qualitative.Plotly[i],\n",
    "            marker_size=10,\n",
    "        ),\n",
    "    )\n",
    "\n",
    "fig1.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Polar\n",
    "\n",
    "I'm not sure I've ever seen a polar plots in economics, but you never know.\n",
    "\n",
    "Let's generate some polar data first:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = np.arange(0, 2, 0.01)\n",
    "theta = 2 * np.pi * r\n",
    "polar_data = pd.DataFrame({\"r\": r, \"theta\": theta})\n",
    "polar_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = plt.subplot(111, projection=\"polar\")\n",
    "ax.plot(polar_data[\"theta\"], polar_data[\"r\"])\n",
    "ax.set_rmax(2)\n",
    "ax.set_rticks([0.5, 1, 1.5, 2])  # Fewer radial ticks\n",
    "ax.set_rlabel_position(-22.5)  # Move radial labels away from plotted line\n",
    "ax.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = go.Figure(\n",
    "    data=go.Scatterpolar(\n",
    "        r=polar_data[\"r\"].values,\n",
    "        theta=polar_data[\"theta\"].values * 180 / (np.pi),\n",
    "        mode=\"lines\",\n",
    "    )\n",
    ")\n",
    "\n",
    "fig.update_layout(showlegend=False)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Radar (or spider) chart\n",
    "\n",
    "Let's generate some synthetic data for this one. Assumes that result to be shown is the sum of observations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(\n",
    "    dict(\n",
    "        zip(\n",
    "            [\"var\" + str(i) for i in range(1, 6)],\n",
    "            [np.random.randint(30, size=(4)) for i in range(1, 6)],\n",
    "        )\n",
    "    )\n",
    ")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import pi\n",
    "\n",
    "\n",
    "def radar_plot(data, variables):\n",
    "    n_vars = len(variables)\n",
    "    # Plot the first line of the data frame.\n",
    "    # Repeat the first value to close the circular graph:\n",
    "    values = data.loc[data.index[0], variables].values.flatten().tolist()\n",
    "    values += values[:1]\n",
    "    # What will be the angle of each axis in the plot? (we divide / number of variable)\n",
    "    angles = [n / float(n_vars) * 2 * pi for n in range(n_vars)]\n",
    "    angles += angles[:1]\n",
    "    # Initialise the spider plot\n",
    "    ax = plt.subplot(111, polar=True)\n",
    "    # Draw one axe per variable + add labels\n",
    "    plt.xticks(angles[:-1], variables)\n",
    "    # Draw ylabels\n",
    "    ax.set_rlabel_position(0)\n",
    "    # Plot data\n",
    "    ax.plot(angles, values, linewidth=1, linestyle=\"solid\")\n",
    "    # Fill area\n",
    "    ax.fill(angles, values, \"b\", alpha=0.1)\n",
    "    return ax\n",
    "\n",
    "\n",
    "radar_plot(df, df.columns);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = px.data.wind()\n",
    "print(df.head())\n",
    "fig = px.line_polar(\n",
    "    df,\n",
    "    r=\"frequency\",\n",
    "    theta=\"direction\",\n",
    "    color=\"strength\",\n",
    "    line_close=True,\n",
    "    color_discrete_sequence=px.colors.sequential.Plasma_r,\n",
    "    template=\"plotly_dark\",\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wordcloud\n",
    "\n",
    "These should be used sparingly. Let's grab part of a famous text from Project Gutenberg:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# To run this example, download smith_won.txt from\n",
    "# https://github.com/aeturrell/coding-for-economists/blob/main/data/smith_won.txt\n",
    "# and put it in a sub-folder called 'data\n",
    "\n",
    "book_text = open(Path(\"data\", \"smith_won.txt\"), \"r\", encoding=\"utf-8\").read()\n",
    "# Print some lines\n",
    "print(\"\\n\".join(book_text.split(\"\\n\")[107:117]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from wordcloud import WordCloud\n",
    "\n",
    "wordcloud = WordCloud(width=700, height=400).generate(book_text)\n",
    "fig, ax = plt.subplots(facecolor=\"k\")\n",
    "ax.imshow(wordcloud, interpolation=\"bilinear\")\n",
    "plt.axis(\"off\")\n",
    "plt.tight_layout();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also create a 'mask' for the wordcloud to shape it how we like, here in the shape of a book."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To run this example, download book_mask.png from\n",
    "# https://github.com/aeturrell/coding-for-economists/raw/main/data/book_mask.png\n",
    "# and put it in a sub-folder called 'data\n",
    "from PIL import Image\n",
    "\n",
    "mask = np.array(Image.open(Path(\"data\", \"book_mask.png\")))\n",
    "wc = WordCloud(width=700, height=400, mask=mask, background_color=\"white\")\n",
    "wordcloud = wc.generate(book_text)\n",
    "fig, ax = plt.subplots(facecolor=\"white\")\n",
    "ax.imshow(wordcloud, interpolation=\"bilinear\")\n",
    "plt.axis(\"off\")\n",
    "plt.tight_layout();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Network diagrams\n",
    "\n",
    "### networkx\n",
    "\n",
    "The most well-established network visualisation package is [**networkx**](https://networkx.org/documentation/stable/index.html), which does a *lot* more than just visualisation. It has many different positioning options for rendering any given network, for instance in circular, spectral, spring, Fruchterman-Reingold, or other styles. In the below example, we use a **pandas** dataframe to specify the edges in two columns but there are various other ways to specify the network too, including ones that do not rely on **pandas**.\n",
    "\n",
    "The underlying plot is rendered with **matplotlib**, meaning that you can customise it further should you need to. You can pass an `Axes` object `ax` to `nx.draw()` using `nx.draw(..., ax=ax)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "\n",
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"source\": [\"A\", \"B\", \"C\", \"A\", \"E\", \"F\", \"E\", \"G\", \"G\", \"D\", \"F\"],\n",
    "        \"target\": [\"D\", \"A\", \"E\", \"C\", \"A\", \"F\", \"G\", \"D\", \"B\", \"G\", \"C\"],\n",
    "    }\n",
    ")\n",
    "G = nx.from_pandas_edgelist(df)\n",
    "nx.draw(G, with_labels=True, node_size=500, node_color=\"skyblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ridge, or 'joy', plots\n",
    "\n",
    "These are famous from the front cover of “Unkown Pleasures” by Joy Division. Let's look at an example showing the global increase in temperature.\n",
    "\n",
    "We'll use a summary of the daily land-surface average temperature anomaly produced by the Berkeley Earth averaging method. Temperatures are in Celsius and reported as anomalies relative to the Jan 1951-Dec 1980 average (the estimated Jan 1951-Dec 1980 land-average temperature is 8.63 +/- 0.06 C)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To run this example, download the pickle file from\n",
    "# https://github.com/aeturrell/coding-for-economists/blob/main/data/berkeley_data.pkl\n",
    "# and put it in a sub-folder called 'data'\n",
    "df = pd.read_pickle(Path(\"data/berkeley_data.pkl\"))\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_year = df[\"Year\"].max()\n",
    "first_year = df[\"Year\"].min()\n",
    "\n",
    "breaks = [y for y in list(df.Year.unique()) if y % 10 == 0]\n",
    "(\n",
    "    ggplot(df, aes(\"Anomaly\", \"Year\", fill=\"Year\"))\n",
    "    + geom_area_ridges(scale=20, alpha=1, size=0.2, trim=True, show_legend=False)\n",
    "    + scale_y_continuous(breaks=breaks, trans=\"reverse\")\n",
    "    + scale_fill_viridis(option=\"inferno\")\n",
    "    + ggtitle(\n",
    "        \"Global daily temperature anomaly {0}-{1} \\n(°C above 1951-80 average)\".format(\n",
    "            first_year, final_year\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contour Plot\n",
    "\n",
    "Contour plots can help you show how a third variable, Z, varies with both X and Y (ie Z is a surface). The way that Z is depicted could be via the density of lines drawn in the X-Y plane (use `ax.contour()` for this) or via colour, as in the example below (using `ax.contourf()`).\n",
    "\n",
    "The heatmap (or contour plot) below, which has a colour bar legend and a title that's rendered with latex, uses a perceptually uniform distribution that makes equal changes look equal; **matplotlib** has a few of these. If you need more colours, check out the packages [**colorcet**](https://colorcet.holoviz.org/) and [**palettable**](https://jiffyclub.github.io/palettable/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that, in the below, Z is returned by a function that accepts a grid of X and Y values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x, y):\n",
    "    return np.sin(x) ** 10 + np.cos(10 + y * x) * np.cos(x)\n",
    "\n",
    "\n",
    "x = np.linspace(0, 5, 100)\n",
    "y = np.linspace(0, 5, 100)\n",
    "\n",
    "X, Y = np.meshgrid(x, y)\n",
    "Z = f(X, Y)\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "cf = ax.contourf(X, Y, Z, cmap=\"plasma\")\n",
    "ax.set_title(r\"$f(x,y) = \\sin^{10}(x) + \\cos(x)\\cos\\left(10 + y\\cdot x\\right)$\")\n",
    "cbar = fig.colorbar(cf);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "contour_data = {\"x\": X.flatten(), \"y\": Y.flatten(), \"z\": Z.flatten()}\n",
    "(\n",
    "    ggplot(contour_data)\n",
    "    + geom_contourf(aes(x=\"x\", y=\"y\", z=\"z\", fill=\"..level..\"))\n",
    "    + scale_fill_viridis(option=\"plasma\")\n",
    "    + ggtitle(\"Maths equations don't currently work\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "grid_fig = go.Figure(data=go.Contour(z=Z, x=x, y=y))\n",
    "\n",
    "grid_fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Waterfall chart\n",
    "\n",
    "Waterfall charts are good for showing how different contributions combine to net out at a certain value. There's a package dedicated to them called [**waterfallcharts**](https://github.com/chrispaulca/waterfall). It builds on **matplotlib**. First, let's create some data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [\"sales\", \"returns\", \"credit fees\", \"rebates\", \"late charges\", \"shipping\"]\n",
    "b = [10, -30, -7.5, -25, 95, -7]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot this data. Because the defaults of **waterfallcharts** don't play that nicely with the plot style used for this book, we'll temporarily switch back to the **matplotlib** default plot style using a *context* and `with` statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import waterfall_chart\n",
    "\n",
    "with plt.style.context(\"default\"):\n",
    "    plot = waterfall_chart.plot(a, b, sorted_value=True, rotation_value=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "px_b = b + [sum(b)]\n",
    "\n",
    "fig = go.Figure(\n",
    "    go.Waterfall(\n",
    "        name=\"20\",\n",
    "        orientation=\"v\",\n",
    "        measure=[\"relative\"] * len(a) + [\"total\"],\n",
    "        x=a + [\"net\"],\n",
    "        textposition=\"outside\",\n",
    "        text=[str(x) for x in b] + [\"net\"],\n",
    "        y=px_b,\n",
    "        connector={\"line\": {\"color\": \"rgb(63, 63, 63)\"}},\n",
    "    )\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Venn\n",
    "\n",
    "Venn diagrams show the overlap between groups. As with some of these other, more unsual chart types, there's a special package that produces these and which builds on **matplotlib**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib_venn import venn2\n",
    "\n",
    "venn2(subsets=(10, 5, 2), set_labels=(\"Group A\", \"Group B\"), alpha=0.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Priestley Timeline\n",
    "\n",
    "This displays a timeline of start and end events in time, and their overlap."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    \"https://github.com/aeturrell/coding-for-economists/raw/main/data/priestley-timeline.csv\",\n",
    "    parse_dates=[\"Born\", \"Died\"],\n",
    "    dayfirst=True,\n",
    ")\n",
    "df = df.sort_values(\"Born\")\n",
    "\n",
    "# Create the plot\n",
    "fig, ax = plt.subplots(figsize=(12, 6))\n",
    "\n",
    "for i, (index, row) in enumerate(df.iterrows()):\n",
    "    lifespan = (row[\"Died\"] - row[\"Born\"]).days\n",
    "    bar = ax.barh(len(df) - 1 - i, lifespan, left=row[\"Born\"], height=0.5)\n",
    "    text_x = row[\"Born\"] + pd.Timedelta(days=lifespan / 2)\n",
    "\n",
    "    # Add text inside the bar\n",
    "    ax.text(\n",
    "        text_x,\n",
    "        len(df) - 1 - i,\n",
    "        row[\"Name\"],\n",
    "        va=\"center\",\n",
    "        ha=\"center\",\n",
    "        color=\"k\",\n",
    "        fontweight=\"bold\",\n",
    "        fontsize=8,\n",
    "    )\n",
    "\n",
    "ax.set_yticks([])\n",
    "plt.xlabel(\"Year\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Waffle, isotype, or pictogram charts\n",
    "\n",
    "These are great for showing easily-understandable magnitudes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib\n",
    "\n",
    "There is a package called [**pywaffle**](https://github.com/gyli/PyWaffle) that provides a convenient way of doing this. It expects a dictionary of values. Note that the icon can be changed and, because it builds on **matplotlib**, you can tweak to your heart's content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pywaffle import Waffle\n",
    "\n",
    "data = {\"Democratic\": 48, \"Republican\": 46, \"Libertarian\": 3}\n",
    "fig = plt.figure(\n",
    "    FigureClass=Waffle,\n",
    "    rows=5,\n",
    "    values=data,\n",
    "    colors=[\"#232066\", \"#983D3D\", \"#DCB732\"],\n",
    "    legend={\"loc\": \"upper left\", \"bbox_to_anchor\": (1, 1)},\n",
    "    icons=\"child\",\n",
    "    font_size=12,\n",
    "    icon_legend=True,\n",
    ")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n",
    "As ever, **Lets-Plot** prefers tidy format data. We'll create a mini dataset just to demonstrate its use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "\n",
    "df = pd.DataFrame(list(itertools.product(range(10), range(10))), columns=[\"x\", \"y\"])\n",
    "df[\"filled\"] = 0\n",
    "df.iloc[:32, 2] = 1\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = (\n",
    "    ggplot(df, aes(x=\"x\", y=\"y\", fill=as_discrete(\"filled\")))\n",
    "    + geom_tile(alpha=0.5, color=\"black\")\n",
    "    + scale_fill_manual([\"green\", \"blue\"])\n",
    "    + coord_flip()\n",
    "    + geom_text(x=5, y=5, label=f\"{int(100*df.filled.mean())}%\", size=30, color=\"white\")\n",
    "    + theme(\n",
    "        axis=element_blank(),\n",
    "        panel_grid_major=element_blank(),\n",
    "        panel_grid_minor=element_blank(),\n",
    "    )\n",
    "    + xlab(\"\")\n",
    "    + ylab(\"\")\n",
    ")\n",
    "g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pyramid\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    \"https://raw.githubusercontent.com/selva86/datasets/master/email_campaign_funnel.csv\"\n",
    ")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib/Seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "group_col = \"Gender\"\n",
    "order_of_bars = df.Stage.unique()[::-1]\n",
    "colors = [\n",
    "    plt.cm.Spectral(i / float(len(df[group_col].unique()) - 1))\n",
    "    for i in range(len(df[group_col].unique()))\n",
    "]\n",
    "\n",
    "for c, group in zip(colors, df[group_col].unique()):\n",
    "    sns.barplot(\n",
    "        x=\"Users\",\n",
    "        y=\"Stage\",\n",
    "        data=df.loc[df[group_col] == group, :],\n",
    "        order=order_of_bars,\n",
    "        color=c,\n",
    "        label=group,\n",
    "        ax=ax,\n",
    "        lw=0,\n",
    "    )\n",
    "\n",
    "divisor = 1e6\n",
    "ax.set_xticklabels([str(abs(x) / divisor) for x in ax.get_xticks()])\n",
    "plt.xlabel(\"Users (millions)\")\n",
    "plt.ylabel(\"Stage of Purchase\")\n",
    "plt.yticks(fontsize=12)\n",
    "plt.title(\"Population Pyramid of the Marketing Funnel\", fontsize=22)\n",
    "plt.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets-Plot\n",
    "\n",
    "Unfortunately, the 20 character limit is hardcoded, so y labels are cut off. But the full text can be seen in the axial tooltip."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = (\n",
    "    ggplot(df, aes(x=\"Stage\", y=\"Users\", fill=\"Gender\", weight=\"Users\"))\n",
    "    + geom_bar(width=0.8)  # baseplot\n",
    "    + coord_flip()  # flip coordinates\n",
    "    + theme_minimal()\n",
    "    + ylab(\"Users (millions)\")\n",
    ")\n",
    "g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.funnel(df, y=\"Stage\", x=\"Users\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sankey diagram\n",
    "\n",
    "Sankey diagrams show how a flow breaks into pieces."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objects as go\n",
    "\n",
    "labels = [\"A1\", \"A2\", \"B1\", \"B2\", \"C1\", \"C2\"]\n",
    "\n",
    "fig = go.Figure(\n",
    "    data=[\n",
    "        go.Sankey(\n",
    "            node=dict(\n",
    "                pad=15,\n",
    "                thickness=20,\n",
    "                line=dict(color=\"black\", width=0.5),\n",
    "                label=labels,\n",
    "                color=px.colors.qualitative.Plotly[: len(labels)],\n",
    "            ),\n",
    "            # indices correspond to labels, eg A1, A2, A1, B1, ...\n",
    "            link=dict(\n",
    "                source=[0, 1, 0, 2, 3, 3, 2],\n",
    "                target=[2, 3, 3, 4, 4, 5, 5],\n",
    "                value=[7, 3, 2, 6, 4, 2, 1],\n",
    "            ),\n",
    "        )\n",
    "    ]\n",
    ")\n",
    "\n",
    "fig.update_layout(title_text=\"Basic Sankey Diagram\", font_size=10)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dendrogram or hierarchical clustering\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data\n",
    "df = (\n",
    "    pd.read_csv(\n",
    "        \"https://vincentarelbundock.github.io/Rdatasets/csv/datasets/mtcars.csv\"\n",
    "    )\n",
    "    .rename(columns={\"rownames\": \"Model\"})\n",
    "    .set_index(\"Model\")\n",
    ")\n",
    "# Plot\n",
    "sns.clustermap(\n",
    "    df, metric=\"correlation\", method=\"single\", standard_scale=1, cmap=\"vlag\"\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Treemap\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotly\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import plotly.express as px\n",
    "\n",
    "df = px.data.gapminder().query(\"year == 2007\")\n",
    "fig = px.treemap(\n",
    "    df,\n",
    "    path=[px.Constant(\"world\"), \"continent\", \"country\"],\n",
    "    values=\"pop\",\n",
    "    color=\"lifeExp\",\n",
    "    hover_data=[\"iso_alpha\"],\n",
    "    color_continuous_scale=\"RdBu\",\n",
    "    color_continuous_midpoint=np.average(df[\"lifeExp\"], weights=df[\"pop\"]),\n",
    ")\n",
    "fig.update_layout(margin=dict(t=50, l=25, r=25, b=25))\n",
    "fig.show()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "codeforecon",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}