Concatenation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DataFrame concatenation\n",
    "\n",
    "There are a lot of situations when you'll want to stick data with the same columns of data together. This often comes up for me when I want to take data from multiple files and put it all together into one file, or just analyze or visualize it together in a single DataFrame.\n",
    "\n",
    "We'll get to this functionality, but along the way we'll learn how to\n",
    "\n",
    "- get a random sample of DataFrame rows\n",
    "- drop (delete) a column\n",
    "- drop duplicate rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic concatenation\n",
    "\n",
    "To show the basic principle we're just going to put a DataFrame together with some random samples from itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Agriculture</th>\n",
       "      <th>Business</th>\n",
       "      <th>Engineering</th>\n",
       "      <th>Health</th>\n",
       "      <th>Psychology</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1970</td>\n",
       "      <td>4.229798</td>\n",
       "      <td>9.064439</td>\n",
       "      <td>0.8</td>\n",
       "      <td>77.1</td>\n",
       "      <td>44.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1971</td>\n",
       "      <td>5.452797</td>\n",
       "      <td>9.503187</td>\n",
       "      <td>1.0</td>\n",
       "      <td>75.5</td>\n",
       "      <td>46.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1972</td>\n",
       "      <td>7.420710</td>\n",
       "      <td>10.558962</td>\n",
       "      <td>1.2</td>\n",
       "      <td>76.9</td>\n",
       "      <td>47.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1973</td>\n",
       "      <td>9.653602</td>\n",
       "      <td>12.804602</td>\n",
       "      <td>1.6</td>\n",
       "      <td>77.4</td>\n",
       "      <td>50.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1974</td>\n",
       "      <td>14.074623</td>\n",
       "      <td>16.204850</td>\n",
       "      <td>2.2</td>\n",
       "      <td>77.9</td>\n",
       "      <td>52.6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year  Agriculture   Business  Engineering  Health  Psychology\n",
       "0  1970     4.229798   9.064439          0.8    77.1        44.4\n",
       "1  1971     5.452797   9.503187          1.0    75.5        46.2\n",
       "2  1972     7.420710  10.558962          1.2    76.9        47.6\n",
       "3  1973     9.653602  12.804602          1.6    77.4        50.4\n",
       "4  1974    14.074623  16.204850          2.2    77.9        52.6"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('data/women_percent_deg_usa_subset.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random sampling\n",
    "\n",
    "Sometimes data is really large, and for visualization you may want to only show a random subset of your data. We can use `df.sample()` and specify the number of samples, or the fraction of the original.\n",
    "\n",
    "*Notice that each time we run `.sample()` we get a different subset. Execute this line more than once and compare.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Agriculture</th>\n",
       "      <th>Business</th>\n",
       "      <th>Engineering</th>\n",
       "      <th>Health</th>\n",
       "      <th>Psychology</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1972</td>\n",
       "      <td>7.420710</td>\n",
       "      <td>10.558962</td>\n",
       "      <td>1.2</td>\n",
       "      <td>76.9</td>\n",
       "      <td>47.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20</td>\n",
       "      <td>1990</td>\n",
       "      <td>32.703444</td>\n",
       "      <td>47.200851</td>\n",
       "      <td>14.1</td>\n",
       "      <td>83.9</td>\n",
       "      <td>72.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26</td>\n",
       "      <td>1996</td>\n",
       "      <td>38.969775</td>\n",
       "      <td>48.647393</td>\n",
       "      <td>16.7</td>\n",
       "      <td>81.3</td>\n",
       "      <td>73.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>1994</td>\n",
       "      <td>36.032674</td>\n",
       "      <td>47.983924</td>\n",
       "      <td>15.7</td>\n",
       "      <td>81.8</td>\n",
       "      <td>72.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>1976</td>\n",
       "      <td>22.252760</td>\n",
       "      <td>23.430038</td>\n",
       "      <td>4.5</td>\n",
       "      <td>79.2</td>\n",
       "      <td>56.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Year  Agriculture   Business  Engineering  Health  Psychology\n",
       "2   1972     7.420710  10.558962          1.2    76.9        47.6\n",
       "20  1990    32.703444  47.200851         14.1    83.9        72.6\n",
       "26  1996    38.969775  48.647393         16.7    81.3        73.9\n",
       "24  1994    36.032674  47.983924         15.7    81.8        72.9\n",
       "6   1976    22.252760  23.430038          4.5    79.2        56.9"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sample(n=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Agriculture</th>\n",
       "      <th>Business</th>\n",
       "      <th>Engineering</th>\n",
       "      <th>Health</th>\n",
       "      <th>Psychology</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>1998</td>\n",
       "      <td>41.912403</td>\n",
       "      <td>49.258515</td>\n",
       "      <td>17.8</td>\n",
       "      <td>82.1</td>\n",
       "      <td>75.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27</td>\n",
       "      <td>1997</td>\n",
       "      <td>40.685685</td>\n",
       "      <td>48.561050</td>\n",
       "      <td>17.0</td>\n",
       "      <td>81.9</td>\n",
       "      <td>74.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1972</td>\n",
       "      <td>7.420710</td>\n",
       "      <td>10.558962</td>\n",
       "      <td>1.2</td>\n",
       "      <td>76.9</td>\n",
       "      <td>47.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>1995</td>\n",
       "      <td>36.844807</td>\n",
       "      <td>48.573181</td>\n",
       "      <td>16.2</td>\n",
       "      <td>81.5</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Year  Agriculture   Business  Engineering  Health  Psychology\n",
       "28  1998    41.912403  49.258515         17.8    82.1        75.1\n",
       "27  1997    40.685685  48.561050         17.0    81.9        74.4\n",
       "2   1972     7.420710  10.558962          1.2    76.9        47.6\n",
       "25  1995    36.844807  48.573181         16.2    81.5        73.0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sample(frac=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating duplicates with the sample\n",
    "\n",
    "Now, we'll create a new DataFrame that has duplicates built in by concatenating our original DataFrame with a sample of its own rows. \n",
    "\n",
    "*You give `pd.concat()` a list of all the DataFrames you want to concatenate.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Agriculture</th>\n",
       "      <th>Business</th>\n",
       "      <th>Engineering</th>\n",
       "      <th>Health</th>\n",
       "      <th>Psychology</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>2007</td>\n",
       "      <td>47.605026</td>\n",
       "      <td>49.000459</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.4</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>2008</td>\n",
       "      <td>47.570834</td>\n",
       "      <td>48.888027</td>\n",
       "      <td>16.5</td>\n",
       "      <td>85.2</td>\n",
       "      <td>77.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>2009</td>\n",
       "      <td>48.667224</td>\n",
       "      <td>48.840474</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.1</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>2010</td>\n",
       "      <td>48.730042</td>\n",
       "      <td>48.757988</td>\n",
       "      <td>17.2</td>\n",
       "      <td>85.0</td>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>2011</td>\n",
       "      <td>50.037182</td>\n",
       "      <td>48.180418</td>\n",
       "      <td>17.5</td>\n",
       "      <td>84.8</td>\n",
       "      <td>76.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>1980</td>\n",
       "      <td>30.759390</td>\n",
       "      <td>36.765725</td>\n",
       "      <td>10.3</td>\n",
       "      <td>83.5</td>\n",
       "      <td>65.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>1988</td>\n",
       "      <td>31.085087</td>\n",
       "      <td>46.764828</td>\n",
       "      <td>13.9</td>\n",
       "      <td>85.2</td>\n",
       "      <td>70.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35</td>\n",
       "      <td>2005</td>\n",
       "      <td>47.672754</td>\n",
       "      <td>49.791851</td>\n",
       "      <td>17.9</td>\n",
       "      <td>86.0</td>\n",
       "      <td>77.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19</td>\n",
       "      <td>1989</td>\n",
       "      <td>31.612403</td>\n",
       "      <td>46.781565</td>\n",
       "      <td>14.1</td>\n",
       "      <td>84.6</td>\n",
       "      <td>71.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>1981</td>\n",
       "      <td>31.318655</td>\n",
       "      <td>39.266230</td>\n",
       "      <td>11.6</td>\n",
       "      <td>84.1</td>\n",
       "      <td>66.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Year  Agriculture   Business  Engineering  Health  Psychology\n",
       "37  2007    47.605026  49.000459         16.8    85.4        77.1\n",
       "38  2008    47.570834  48.888027         16.5    85.2        77.2\n",
       "39  2009    48.667224  48.840474         16.8    85.1        77.1\n",
       "40  2010    48.730042  48.757988         17.2    85.0        77.0\n",
       "41  2011    50.037182  48.180418         17.5    84.8        76.7\n",
       "10  1980    30.759390  36.765725         10.3    83.5        65.1\n",
       "18  1988    31.085087  46.764828         13.9    85.2        70.9\n",
       "35  2005    47.672754  49.791851         17.9    86.0        77.5\n",
       "19  1989    31.612403  46.781565         14.1    84.6        71.6\n",
       "11  1981    31.318655  39.266230         11.6    84.1        66.9"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_w_dups = pd.concat([df, df.sample(n=5)])\n",
    "df_w_dups.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reset index\n",
    "\n",
    "In a minute we'll remove duplicate rows, but I want to show you that it's not detecting duplicates in the Index.\n",
    "\n",
    "First, it's a little weird to me that the Index doesn't have to be unique. (If you notice above, some index values are repeated after the concatenation.) That evidently affects the speed of lookup – unique Index is faster – but it's allowed. \n",
    "\n",
    "- Sometimes, though, after a sorting operation, or something like this, you'll want to reset the index to a new range of integers. You do that with `df.reset_index()`. \n",
    "- It's also a way to move the Index to a regular column, which comes in handy sometimes, too. \n",
    "- Here I'll do it \"inplace\". \n",
    "\n",
    "**Remember to be careful with \"inplace\" operations, because you'll be writing over your original data!** \n",
    "\n",
    "(You have the option of dropping the index with `drop=True`, but here I'll leave it to show you how to explicitly drop a column.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Year</th>\n",
       "      <th>Agriculture</th>\n",
       "      <th>Business</th>\n",
       "      <th>Engineering</th>\n",
       "      <th>Health</th>\n",
       "      <th>Psychology</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>2007</td>\n",
       "      <td>47.605026</td>\n",
       "      <td>49.000459</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.4</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>38</td>\n",
       "      <td>2008</td>\n",
       "      <td>47.570834</td>\n",
       "      <td>48.888027</td>\n",
       "      <td>16.5</td>\n",
       "      <td>85.2</td>\n",
       "      <td>77.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>2009</td>\n",
       "      <td>48.667224</td>\n",
       "      <td>48.840474</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.1</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>40</td>\n",
       "      <td>2010</td>\n",
       "      <td>48.730042</td>\n",
       "      <td>48.757988</td>\n",
       "      <td>17.2</td>\n",
       "      <td>85.0</td>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>41</td>\n",
       "      <td>2011</td>\n",
       "      <td>50.037182</td>\n",
       "      <td>48.180418</td>\n",
       "      <td>17.5</td>\n",
       "      <td>84.8</td>\n",
       "      <td>76.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>10</td>\n",
       "      <td>1980</td>\n",
       "      <td>30.759390</td>\n",
       "      <td>36.765725</td>\n",
       "      <td>10.3</td>\n",
       "      <td>83.5</td>\n",
       "      <td>65.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>18</td>\n",
       "      <td>1988</td>\n",
       "      <td>31.085087</td>\n",
       "      <td>46.764828</td>\n",
       "      <td>13.9</td>\n",
       "      <td>85.2</td>\n",
       "      <td>70.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44</td>\n",
       "      <td>35</td>\n",
       "      <td>2005</td>\n",
       "      <td>47.672754</td>\n",
       "      <td>49.791851</td>\n",
       "      <td>17.9</td>\n",
       "      <td>86.0</td>\n",
       "      <td>77.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>19</td>\n",
       "      <td>1989</td>\n",
       "      <td>31.612403</td>\n",
       "      <td>46.781565</td>\n",
       "      <td>14.1</td>\n",
       "      <td>84.6</td>\n",
       "      <td>71.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>11</td>\n",
       "      <td>1981</td>\n",
       "      <td>31.318655</td>\n",
       "      <td>39.266230</td>\n",
       "      <td>11.6</td>\n",
       "      <td>84.1</td>\n",
       "      <td>66.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    index  Year  Agriculture   Business  Engineering  Health  Psychology\n",
       "37     37  2007    47.605026  49.000459         16.8    85.4        77.1\n",
       "38     38  2008    47.570834  48.888027         16.5    85.2        77.2\n",
       "39     39  2009    48.667224  48.840474         16.8    85.1        77.1\n",
       "40     40  2010    48.730042  48.757988         17.2    85.0        77.0\n",
       "41     41  2011    50.037182  48.180418         17.5    84.8        76.7\n",
       "42     10  1980    30.759390  36.765725         10.3    83.5        65.1\n",
       "43     18  1988    31.085087  46.764828         13.9    85.2        70.9\n",
       "44     35  2005    47.672754  49.791851         17.9    86.0        77.5\n",
       "45     19  1989    31.612403  46.781565         14.1    84.6        71.6\n",
       "46     11  1981    31.318655  39.266230         11.6    84.1        66.9"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_w_dups.reset_index(inplace=True)\n",
    "df_w_dups.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rename a column\n",
    "\n",
    "For fun and instruction we'll rename the old \"index\" column before we delete it. We do that by specifying that we're renaming columns, and then feed `rename` a dictionary with the old name as the key, and the new name as the value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>old_index</th>\n",
       "      <th>Year</th>\n",
       "      <th>Agriculture</th>\n",
       "      <th>Business</th>\n",
       "      <th>Engineering</th>\n",
       "      <th>Health</th>\n",
       "      <th>Psychology</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>2007</td>\n",
       "      <td>47.605026</td>\n",
       "      <td>49.000459</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.4</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>38</td>\n",
       "      <td>2008</td>\n",
       "      <td>47.570834</td>\n",
       "      <td>48.888027</td>\n",
       "      <td>16.5</td>\n",
       "      <td>85.2</td>\n",
       "      <td>77.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>2009</td>\n",
       "      <td>48.667224</td>\n",
       "      <td>48.840474</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.1</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>40</td>\n",
       "      <td>2010</td>\n",
       "      <td>48.730042</td>\n",
       "      <td>48.757988</td>\n",
       "      <td>17.2</td>\n",
       "      <td>85.0</td>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>41</td>\n",
       "      <td>2011</td>\n",
       "      <td>50.037182</td>\n",
       "      <td>48.180418</td>\n",
       "      <td>17.5</td>\n",
       "      <td>84.8</td>\n",
       "      <td>76.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>10</td>\n",
       "      <td>1980</td>\n",
       "      <td>30.759390</td>\n",
       "      <td>36.765725</td>\n",
       "      <td>10.3</td>\n",
       "      <td>83.5</td>\n",
       "      <td>65.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>18</td>\n",
       "      <td>1988</td>\n",
       "      <td>31.085087</td>\n",
       "      <td>46.764828</td>\n",
       "      <td>13.9</td>\n",
       "      <td>85.2</td>\n",
       "      <td>70.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44</td>\n",
       "      <td>35</td>\n",
       "      <td>2005</td>\n",
       "      <td>47.672754</td>\n",
       "      <td>49.791851</td>\n",
       "      <td>17.9</td>\n",
       "      <td>86.0</td>\n",
       "      <td>77.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>19</td>\n",
       "      <td>1989</td>\n",
       "      <td>31.612403</td>\n",
       "      <td>46.781565</td>\n",
       "      <td>14.1</td>\n",
       "      <td>84.6</td>\n",
       "      <td>71.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>11</td>\n",
       "      <td>1981</td>\n",
       "      <td>31.318655</td>\n",
       "      <td>39.266230</td>\n",
       "      <td>11.6</td>\n",
       "      <td>84.1</td>\n",
       "      <td>66.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    old_index  Year  Agriculture   Business  Engineering  Health  Psychology\n",
       "37         37  2007    47.605026  49.000459         16.8    85.4        77.1\n",
       "38         38  2008    47.570834  48.888027         16.5    85.2        77.2\n",
       "39         39  2009    48.667224  48.840474         16.8    85.1        77.1\n",
       "40         40  2010    48.730042  48.757988         17.2    85.0        77.0\n",
       "41         41  2011    50.037182  48.180418         17.5    84.8        76.7\n",
       "42         10  1980    30.759390  36.765725         10.3    83.5        65.1\n",
       "43         18  1988    31.085087  46.764828         13.9    85.2        70.9\n",
       "44         35  2005    47.672754  49.791851         17.9    86.0        77.5\n",
       "45         19  1989    31.612403  46.781565         14.1    84.6        71.6\n",
       "46         11  1981    31.318655  39.266230         11.6    84.1        66.9"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_w_dups.rename(columns={'index':'old_index'}, inplace=True)\n",
    "df_w_dups.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Drop a column\n",
    "\n",
    "Now we don't really need the \"index\" column, and it's good to know how to delete columns you don't want.\n",
    "\n",
    "Here we'll use the `columns=` argument, but you can drop rows the same way with `rows=` and a list of rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Agriculture</th>\n",
       "      <th>Business</th>\n",
       "      <th>Engineering</th>\n",
       "      <th>Health</th>\n",
       "      <th>Psychology</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>2007</td>\n",
       "      <td>47.605026</td>\n",
       "      <td>49.000459</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.4</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>2008</td>\n",
       "      <td>47.570834</td>\n",
       "      <td>48.888027</td>\n",
       "      <td>16.5</td>\n",
       "      <td>85.2</td>\n",
       "      <td>77.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>2009</td>\n",
       "      <td>48.667224</td>\n",
       "      <td>48.840474</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.1</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>2010</td>\n",
       "      <td>48.730042</td>\n",
       "      <td>48.757988</td>\n",
       "      <td>17.2</td>\n",
       "      <td>85.0</td>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>2011</td>\n",
       "      <td>50.037182</td>\n",
       "      <td>48.180418</td>\n",
       "      <td>17.5</td>\n",
       "      <td>84.8</td>\n",
       "      <td>76.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>1980</td>\n",
       "      <td>30.759390</td>\n",
       "      <td>36.765725</td>\n",
       "      <td>10.3</td>\n",
       "      <td>83.5</td>\n",
       "      <td>65.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>1988</td>\n",
       "      <td>31.085087</td>\n",
       "      <td>46.764828</td>\n",
       "      <td>13.9</td>\n",
       "      <td>85.2</td>\n",
       "      <td>70.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44</td>\n",
       "      <td>2005</td>\n",
       "      <td>47.672754</td>\n",
       "      <td>49.791851</td>\n",
       "      <td>17.9</td>\n",
       "      <td>86.0</td>\n",
       "      <td>77.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>1989</td>\n",
       "      <td>31.612403</td>\n",
       "      <td>46.781565</td>\n",
       "      <td>14.1</td>\n",
       "      <td>84.6</td>\n",
       "      <td>71.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>1981</td>\n",
       "      <td>31.318655</td>\n",
       "      <td>39.266230</td>\n",
       "      <td>11.6</td>\n",
       "      <td>84.1</td>\n",
       "      <td>66.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Year  Agriculture   Business  Engineering  Health  Psychology\n",
       "37  2007    47.605026  49.000459         16.8    85.4        77.1\n",
       "38  2008    47.570834  48.888027         16.5    85.2        77.2\n",
       "39  2009    48.667224  48.840474         16.8    85.1        77.1\n",
       "40  2010    48.730042  48.757988         17.2    85.0        77.0\n",
       "41  2011    50.037182  48.180418         17.5    84.8        76.7\n",
       "42  1980    30.759390  36.765725         10.3    83.5        65.1\n",
       "43  1988    31.085087  46.764828         13.9    85.2        70.9\n",
       "44  2005    47.672754  49.791851         17.9    86.0        77.5\n",
       "45  1989    31.612403  46.781565         14.1    84.6        71.6\n",
       "46  1981    31.318655  39.266230         11.6    84.1        66.9"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_w_dups.drop(columns='old_index', inplace=True)\n",
    "df_w_dups.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Drop duplicate rows\n",
    "\n",
    "Now we'll drop (delete) any duplicate rows. By default it looks at all columns (but not the Index). You can specify a subset of rows to consider for duplicates with the `subset=[]` argument. Here we'll just stick with looking at all columns.\n",
    "\n",
    "*Notice that the index only goes to 41 now, and we had added 5 duplicate rows earlier.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Agriculture</th>\n",
       "      <th>Business</th>\n",
       "      <th>Engineering</th>\n",
       "      <th>Health</th>\n",
       "      <th>Psychology</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>32</td>\n",
       "      <td>2002</td>\n",
       "      <td>47.134658</td>\n",
       "      <td>50.552335</td>\n",
       "      <td>18.7</td>\n",
       "      <td>85.8</td>\n",
       "      <td>77.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33</td>\n",
       "      <td>2003</td>\n",
       "      <td>47.935187</td>\n",
       "      <td>50.345598</td>\n",
       "      <td>18.8</td>\n",
       "      <td>86.5</td>\n",
       "      <td>77.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34</td>\n",
       "      <td>2004</td>\n",
       "      <td>47.887140</td>\n",
       "      <td>49.950894</td>\n",
       "      <td>18.2</td>\n",
       "      <td>86.5</td>\n",
       "      <td>77.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35</td>\n",
       "      <td>2005</td>\n",
       "      <td>47.672754</td>\n",
       "      <td>49.791851</td>\n",
       "      <td>17.9</td>\n",
       "      <td>86.0</td>\n",
       "      <td>77.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36</td>\n",
       "      <td>2006</td>\n",
       "      <td>46.790300</td>\n",
       "      <td>49.210914</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.9</td>\n",
       "      <td>77.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>2007</td>\n",
       "      <td>47.605026</td>\n",
       "      <td>49.000459</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.4</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>2008</td>\n",
       "      <td>47.570834</td>\n",
       "      <td>48.888027</td>\n",
       "      <td>16.5</td>\n",
       "      <td>85.2</td>\n",
       "      <td>77.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>2009</td>\n",
       "      <td>48.667224</td>\n",
       "      <td>48.840474</td>\n",
       "      <td>16.8</td>\n",
       "      <td>85.1</td>\n",
       "      <td>77.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>2010</td>\n",
       "      <td>48.730042</td>\n",
       "      <td>48.757988</td>\n",
       "      <td>17.2</td>\n",
       "      <td>85.0</td>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>2011</td>\n",
       "      <td>50.037182</td>\n",
       "      <td>48.180418</td>\n",
       "      <td>17.5</td>\n",
       "      <td>84.8</td>\n",
       "      <td>76.7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Year  Agriculture   Business  Engineering  Health  Psychology\n",
       "32  2002    47.134658  50.552335         18.7    85.8        77.7\n",
       "33  2003    47.935187  50.345598         18.8    86.5        77.8\n",
       "34  2004    47.887140  49.950894         18.2    86.5        77.8\n",
       "35  2005    47.672754  49.791851         17.9    86.0        77.5\n",
       "36  2006    46.790300  49.210914         16.8    85.9        77.4\n",
       "37  2007    47.605026  49.000459         16.8    85.4        77.1\n",
       "38  2008    47.570834  48.888027         16.5    85.2        77.2\n",
       "39  2009    48.667224  48.840474         16.8    85.1        77.1\n",
       "40  2010    48.730042  48.757988         17.2    85.0        77.0\n",
       "41  2011    50.037182  48.180418         17.5    84.8        76.7"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_wo_dups = df_w_dups.drop_duplicates()\n",
    "df_wo_dups.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "# Concatenate multiple files\n",
    "\n",
    "Now we'll do the concatenation in a more typical situation – combining data from multiple files into a single DataFrame.\n",
    "\n",
    "Remember that what we feeded `pd.concat()` was a list of DataFrames, so if we're reading data from multiple files, all we have to do is\n",
    "\n",
    "- Get a list of the files we want to combine\n",
    "- Create an empty list to hold all the DataFrames\n",
    "- Loop through the files\n",
    "    - Read each file into a new DataFrame\n",
    "    - Add columns to specify where the data came from\n",
    "    - Add the new DataFrame to a list\n",
    "- Concatenate the list of DataFrames\n",
    "\n",
    "To get the list of files, we'll use the `glob` module, which, according to \n",
    "[the glob documentation page](https://docs.python.org/3/library/glob.html), \n",
    "\"finds all the pathnames matching a specified pattern according to the rules used by the Unix shell\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from glob import glob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['data/NC_baby_names_2016.csv',\n",
       " 'data/NC_baby_names_1916.csv',\n",
       " 'data/MN_baby_names_2016.csv',\n",
       " 'data/MN_baby_names_1916.csv']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glob('data/*baby_names*')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The whole code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sex</th>\n",
       "      <th>name</th>\n",
       "      <th>births</th>\n",
       "      <th>state</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>F</td>\n",
       "      <td>Ava</td>\n",
       "      <td>618</td>\n",
       "      <td>NC</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>Emma</td>\n",
       "      <td>607</td>\n",
       "      <td>NC</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>F</td>\n",
       "      <td>Olivia</td>\n",
       "      <td>557</td>\n",
       "      <td>NC</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>F</td>\n",
       "      <td>Charlotte</td>\n",
       "      <td>407</td>\n",
       "      <td>NC</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>F</td>\n",
       "      <td>Harper</td>\n",
       "      <td>404</td>\n",
       "      <td>NC</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75</td>\n",
       "      <td>M</td>\n",
       "      <td>Harold</td>\n",
       "      <td>428</td>\n",
       "      <td>MN</td>\n",
       "      <td>1916</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>76</td>\n",
       "      <td>M</td>\n",
       "      <td>James</td>\n",
       "      <td>385</td>\n",
       "      <td>MN</td>\n",
       "      <td>1916</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>77</td>\n",
       "      <td>M</td>\n",
       "      <td>Edward</td>\n",
       "      <td>346</td>\n",
       "      <td>MN</td>\n",
       "      <td>1916</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>78</td>\n",
       "      <td>M</td>\n",
       "      <td>Raymond</td>\n",
       "      <td>346</td>\n",
       "      <td>MN</td>\n",
       "      <td>1916</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>79</td>\n",
       "      <td>M</td>\n",
       "      <td>Joseph</td>\n",
       "      <td>323</td>\n",
       "      <td>MN</td>\n",
       "      <td>1916</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>80 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   sex       name  births state  year\n",
       "0    F        Ava     618    NC  2016\n",
       "1    F       Emma     607    NC  2016\n",
       "2    F     Olivia     557    NC  2016\n",
       "3    F  Charlotte     407    NC  2016\n",
       "4    F     Harper     404    NC  2016\n",
       "..  ..        ...     ...   ...   ...\n",
       "75   M     Harold     428    MN  1916\n",
       "76   M      James     385    MN  1916\n",
       "77   M     Edward     346    MN  1916\n",
       "78   M    Raymond     346    MN  1916\n",
       "79   M     Joseph     323    MN  1916\n",
       "\n",
       "[80 rows x 5 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_list = glob('data/*baby_names*')\n",
    "df_list = []\n",
    "\n",
    "for file in file_list:\n",
    "    df = pd.read_csv(file)\n",
    "    df['state'] = file[5:7]\n",
    "    df['year'] = file[19:23]\n",
    "    df_list.append(df)\n",
    "\n",
    "df_all = pd.concat(df_list)\n",
    "df_all.reset_index(drop=True, inplace=True)\n",
    "df_all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}