Vectorization-weighting-PS.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "source": [
    "Copyright 2021 Andrew M. Olney and made available under [CC BY-SA](https://creativecommons.org/licenses/by-sa/4.0) for text and [Apache-2.0](http://www.apache.org/licenses/LICENSE-2.0) for code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Vectorization and weighting: Problem solving\n",
    "\n",
    "\n",
    "For this session, we'll use some data on breakfast cereals, which consists mostly of nutrition information along with some other properties.\n",
    "\n",
    "Our goal is to predict `fiber` using the name of the cereal.\n",
    "\n",
    "| Variable | Type | Description |\n",
    "|:-------|:-------|:-------|\n",
    "| name     | Nominal | Name of cereal (an ID)                                                                                                                          |\n",
    "| mfr      | Nominal | Manufacturer of cereal: (A)merican Home Food Products; (G)eneral Mills; (K)elloggs; (N)abisco; (P)ost; (Q)uaker Oats; (R)alston Purina |\n",
    "| type     | Nominal | (H)ot or (C)old                                                                                                                        |\n",
    "| calories | Ratio   | calories per serving                                                                                                                   |\n",
    "| protein  | Ratio   | grams of protein                                                                                                                       |\n",
    "| fat      | Ratio   | grams of fat                                                                                                                           |\n",
    "| sodium   | Ratio   | milligrams of sodium                                                                                                                   |\n",
    "| fiber    | Ratio   | grams of dietary fiber                                                                                                                 |\n",
    "| carbo    | Ratio   | grams of complex carbohydrates                                                                                                         |\n",
    "| sugars   | Ratio   | grams of sugars                                                                                                                        |\n",
    "| potass   | Ratio   | milligrams of potassium                                                                                                                |\n",
    "| vitamins | Ordinal | vitamins and minerals - 0, 25, or 100, indicating the typical percentage of FDA recommended                                            |\n",
    "| shelf    | Ratio   | display shelf (1, 2, or 3, counting from the floor)                                                                                    |\n",
    "| weight   | Ratio   | weight in ounces of one serving                                                                                                        |\n",
    "| cups     | Ratio   | number of cups in one serving                                                                                                          |\n",
    "| rating   | Ratio   | a rating of the cereals (Possibly from Consumer Reports?)                                                                              |\n",
    "      \n",
    "<div style=\"text-align:center;font-size: smaller\">\n",
    "    <b>Source:</b> This dataset was taken from <a href=\"https://www.kaggle.com/crawford/80-cereals\">Kaggle</a>.\n",
    "</div>\n",
    "<br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the data\n",
    "\n",
    "Start by importing `pandas`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"A]5Vf/x,=8dk=KKk3OrJ\">pd</variable></variables><block type=\"importAs\" id=\"G*NhDz5Jo?CcaJf3rUv}\" x=\"150\" y=\"308\"><field name=\"libraryName\">pandas</field><field name=\"libraryAlias\" id=\"A]5Vf/x,=8dk=KKk3OrJ\">pd</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And load the dataframe with `datasets/cereal.csv`, displaying it to make sure it looks right."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>mfr</th>\n",
       "      <th>type</th>\n",
       "      <th>calories</th>\n",
       "      <th>protein</th>\n",
       "      <th>fat</th>\n",
       "      <th>sodium</th>\n",
       "      <th>fiber</th>\n",
       "      <th>carbo</th>\n",
       "      <th>sugars</th>\n",
       "      <th>potass</th>\n",
       "      <th>vitamins</th>\n",
       "      <th>shelf</th>\n",
       "      <th>weight</th>\n",
       "      <th>cups</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100% Bran</td>\n",
       "      <td>N</td>\n",
       "      <td>C</td>\n",
       "      <td>70</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>130</td>\n",
       "      <td>10.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.33</td>\n",
       "      <td>68.402973</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100% Natural Bran</td>\n",
       "      <td>Q</td>\n",
       "      <td>C</td>\n",
       "      <td>120</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "      <td>15</td>\n",
       "      <td>2.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>135.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>33.983679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>All-Bran</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>70</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>260</td>\n",
       "      <td>9.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>320.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.33</td>\n",
       "      <td>59.425505</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>All-Bran with Extra Fiber</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>50</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>140</td>\n",
       "      <td>14.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>330.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.50</td>\n",
       "      <td>93.704912</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Almond Delight</td>\n",
       "      <td>R</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>200</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>34.384843</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>Triples</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>250</td>\n",
       "      <td>0.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>39.106174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>Trix</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>140</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>25</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>27.753301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>Wheat Chex</td>\n",
       "      <td>R</td>\n",
       "      <td>C</td>\n",
       "      <td>100</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>230</td>\n",
       "      <td>3.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.67</td>\n",
       "      <td>49.787445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>Wheaties</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>100</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>200</td>\n",
       "      <td>3.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>51.592193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>Wheaties Honey Gold</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>200</td>\n",
       "      <td>1.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>36.187559</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>77 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/latex": [
       "None"
      ],
      "text/plain": [
       "                         name mfr type  ...  weight  cups     rating\n",
       "0                   100% Bran   N    C  ...     1.0  0.33  68.402973\n",
       "1           100% Natural Bran   Q    C  ...     1.0  1.00  33.983679\n",
       "2                    All-Bran   K    C  ...     1.0  0.33  59.425505\n",
       "3   All-Bran with Extra Fiber   K    C  ...     1.0  0.50  93.704912\n",
       "4              Almond Delight   R    C  ...     1.0  0.75  34.384843\n",
       "..                        ...  ..  ...  ...     ...   ...        ...\n",
       "72                    Triples   G    C  ...     1.0  0.75  39.106174\n",
       "73                       Trix   G    C  ...     1.0  1.00  27.753301\n",
       "74                 Wheat Chex   R    C  ...     1.0  0.67  49.787445\n",
       "75                   Wheaties   G    C  ...     1.0  1.00  51.592193\n",
       "76        Wheaties Honey Gold   G    C  ...     1.0  0.75  36.187559\n",
       "\n",
       "[77 rows x 16 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe = pd.read_csv('datasets/cereal.csv')\n",
    "\n",
    "dataframe\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</variable><variable id=\"A]5Vf/x,=8dk=KKk3OrJ\">pd</variable></variables><block type=\"variables_set\" id=\"FJGldw(qFp?V=E]uaN)4\" x=\"31\" y=\"222\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><value name=\"VALUE\"><block type=\"varDoMethod\" id=\"4s18v%0iqE%_l@{fLYCo\"><field name=\"VAR\" id=\"A]5Vf/x,=8dk=KKk3OrJ\">pd</field><field name=\"MEMBER\">read_csv</field><data>pd:read_csv</data><value name=\"INPUT\"><block type=\"text\" id=\"tN4Jmaffy/={ZFo;uu;W\"><field name=\"TEXT\">datasets/cereal.csv</field></block></value></block></value></block><block type=\"variables_get\" id=\"yN6MT1B[)Pk)cEECN_YZ\" x=\"31\" y=\"286\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Descriptives\n",
    "\n",
    "`describe` the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>calories</th>\n",
       "      <th>protein</th>\n",
       "      <th>fat</th>\n",
       "      <th>sodium</th>\n",
       "      <th>fiber</th>\n",
       "      <th>carbo</th>\n",
       "      <th>sugars</th>\n",
       "      <th>potass</th>\n",
       "      <th>vitamins</th>\n",
       "      <th>shelf</th>\n",
       "      <th>weight</th>\n",
       "      <th>cups</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>77.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>76.000000</td>\n",
       "      <td>76.000000</td>\n",
       "      <td>75.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>77.000000</td>\n",
       "      <td>77.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>106.883117</td>\n",
       "      <td>2.545455</td>\n",
       "      <td>1.012987</td>\n",
       "      <td>159.675325</td>\n",
       "      <td>2.151948</td>\n",
       "      <td>14.802632</td>\n",
       "      <td>7.026316</td>\n",
       "      <td>98.666667</td>\n",
       "      <td>28.246753</td>\n",
       "      <td>2.207792</td>\n",
       "      <td>1.029610</td>\n",
       "      <td>0.821039</td>\n",
       "      <td>42.665705</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>19.484119</td>\n",
       "      <td>1.094790</td>\n",
       "      <td>1.006473</td>\n",
       "      <td>83.832295</td>\n",
       "      <td>2.383364</td>\n",
       "      <td>3.907326</td>\n",
       "      <td>4.378656</td>\n",
       "      <td>70.410636</td>\n",
       "      <td>22.342523</td>\n",
       "      <td>0.832524</td>\n",
       "      <td>0.150477</td>\n",
       "      <td>0.232716</td>\n",
       "      <td>14.047289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>50.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>18.042851</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>100.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>130.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>42.500000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.670000</td>\n",
       "      <td>33.174094</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>110.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>180.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>14.500000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>90.000000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.750000</td>\n",
       "      <td>40.400208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>110.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>210.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>120.000000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>50.828392</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>160.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>320.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>330.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>93.704912</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/latex": [
       "None"
      ],
      "text/plain": [
       "         calories    protein        fat  ...     weight       cups     rating\n",
       "count   77.000000  77.000000  77.000000  ...  77.000000  77.000000  77.000000\n",
       "mean   106.883117   2.545455   1.012987  ...   1.029610   0.821039  42.665705\n",
       "std     19.484119   1.094790   1.006473  ...   0.150477   0.232716  14.047289\n",
       "min     50.000000   1.000000   0.000000  ...   0.500000   0.250000  18.042851\n",
       "25%    100.000000   2.000000   0.000000  ...   1.000000   0.670000  33.174094\n",
       "50%    110.000000   3.000000   1.000000  ...   1.000000   0.750000  40.400208\n",
       "75%    110.000000   3.000000   2.000000  ...   1.000000   1.000000  50.828392\n",
       "max    160.000000   6.000000   5.000000  ...   1.500000   1.500000  93.704912\n",
       "\n",
       "[8 rows x 13 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe.describe()\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</variable></variables><block type=\"varDoMethod\" id=\"Lg=BRVvroBMlk45X$C+B\" x=\"-124\" y=\"-134\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><field name=\"MEMBER\">describe</field><data>dataframe:describe</data></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------------------\n",
    "**QUESTION:**\n",
    "\n",
    "Does the range of fiber look good for regression?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ANSWER: (click here to edit)**\n",
    "\n",
    "*It ranges from 0 to 14, but the two inner quartiles range from 1 to 3.\n",
    "So high fiber is clearly an outlier.*\n",
    "\n",
    "----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vectorize to get features\n",
    "\n",
    "Import `sklearn.feature_extraction.text` as `text`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.feature_extraction.text as text\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"y*%FH]Xz:N5?J=p7So4;\">text</variable></variables><block type=\"importAs\" id=\"oTX6-0d~y$]Xv#M)+_z!\" x=\"-33\" y=\"-322\"><field name=\"libraryName\">sklearn.feature_extraction.text</field><field name=\"libraryAlias\" id=\"y*%FH]Xz:N5?J=p7So4;\">text</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a `CountVectorizer` using `text`, but pass in a list containing\n",
    "     - freestyle `stop_words='english'`\n",
    "     - freestyle `min_df=2` (only include words that occur at least twice)\n",
    "     \n",
    "*Filtering out hapax and stopwords will help us avoid overfit/fitting noise*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "vectorizer = text.CountVectorizer(stop_words='english', min_df=2)\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"Xu=hAdiWJ.`f(n4Tn*5t\">vectorizer</variable><variable id=\"y*%FH]Xz:N5?J=p7So4;\">text</variable></variables><block type=\"variables_set\" id=\"`#yMT01y=4DItX`0~kI.\" x=\"13\" y=\"-227\"><field name=\"VAR\" id=\"Xu=hAdiWJ.`f(n4Tn*5t\">vectorizer</field><value name=\"VALUE\"><block type=\"varCreateObject\" id=\"e+2wjiwOjX%pfxd5nKAj\"><field name=\"VAR\" id=\"y*%FH]Xz:N5?J=p7So4;\">text</field><field name=\"MEMBER\">CountVectorizer</field><data>text:CountVectorizer</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\")/|_zo`p.YN?DYaEPR]p\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"dummyOutputCodeBlock\" id=\"n9!jxI#!FgOzCFMIOh(L\"><field name=\"CODE\">stop_words='english'</field></block></value><value name=\"ADD1\"><block type=\"dummyOutputCodeBlock\" id=\"!QhL-02Lc;OBF[QiFRZ/\"><field name=\"CODE\">min_df=2</field></block></value></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Call `fit_transform` with the vectorizer using the `name` column of `dataframe`, and store in `matrix`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "matrix = vectorizer.fit_transform(dataframe['name'])\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"g#vU}+I%b#efZeGj-i8*\">matrix</variable><variable id=\"Xu=hAdiWJ.`f(n4Tn*5t\">vectorizer</variable><variable id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</variable></variables><block type=\"variables_set\" id=\"scYNV_rn?aY)~0.5mcLO\" x=\"-12\" y=\"-180\"><field name=\"VAR\" id=\"g#vU}+I%b#efZeGj-i8*\">matrix</field><value name=\"VALUE\"><block type=\"varDoMethod\" id=\"]#ME2Y;blQoh|J?@,oYL\"><field name=\"VAR\" id=\"Xu=hAdiWJ.`f(n4Tn*5t\">vectorizer</field><field name=\"MEMBER\">fit_transform</field><data>vectorizer:fit_transform</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"H]4d}7bP.$l`IjllnR~l\"><mutation items=\"1\"></mutation><value name=\"ADD0\"><block type=\"indexer\" id=\"Dxj+)aFUxS~s{NqB[21O\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><value name=\"INDEX\"><block type=\"text\" id=\"Wb3X:Ik2gnOo4mmiS}6^\"><field name=\"TEXT\">name</field></block></value></block></value></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we're going to put the features back into the `dataframe` as columns.\n",
    "\n",
    "First create a new dataframe, using a list containing\n",
    "    - with `matrix` do `todense` \n",
    "    - freestyle `columns = ` with `vectorizer` do `get_feature_names`\n",
    "    \n",
    "Store this in `matrix_df` and display it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>100</th>\n",
       "      <th>almond</th>\n",
       "      <th>apple</th>\n",
       "      <th>bran</th>\n",
       "      <th>cheerios</th>\n",
       "      <th>chex</th>\n",
       "      <th>cinnamon</th>\n",
       "      <th>corn</th>\n",
       "      <th>crisp</th>\n",
       "      <th>crispy</th>\n",
       "      <th>crunch</th>\n",
       "      <th>dates</th>\n",
       "      <th>flakes</th>\n",
       "      <th>frosted</th>\n",
       "      <th>fruit</th>\n",
       "      <th>golden</th>\n",
       "      <th>grain</th>\n",
       "      <th>grape</th>\n",
       "      <th>honey</th>\n",
       "      <th>just</th>\n",
       "      <th>muesli</th>\n",
       "      <th>nut</th>\n",
       "      <th>nutri</th>\n",
       "      <th>nuts</th>\n",
       "      <th>oat</th>\n",
       "      <th>oatmeal</th>\n",
       "      <th>puffed</th>\n",
       "      <th>quaker</th>\n",
       "      <th>raisin</th>\n",
       "      <th>raisins</th>\n",
       "      <th>rice</th>\n",
       "      <th>right</th>\n",
       "      <th>shredded</th>\n",
       "      <th>squares</th>\n",
       "      <th>total</th>\n",
       "      <th>wheat</th>\n",
       "      <th>wheaties</th>\n",
       "      <th>wheats</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>77 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/latex": [
       "None"
      ],
      "text/plain": [
       "    100  almond  apple  bran  cheerios  ...  squares  total  wheat  wheaties  wheats\n",
       "0     1       0      0     1         0  ...        0      0      0         0       0\n",
       "1     1       0      0     1         0  ...        0      0      0         0       0\n",
       "2     0       0      0     1         0  ...        0      0      0         0       0\n",
       "3     0       0      0     1         0  ...        0      0      0         0       0\n",
       "4     0       1      0     0         0  ...        0      0      0         0       0\n",
       "..  ...     ...    ...   ...       ...  ...      ...    ...    ...       ...     ...\n",
       "72    0       0      0     0         0  ...        0      0      0         0       0\n",
       "73    0       0      0     0         0  ...        0      0      0         0       0\n",
       "74    0       0      0     0         0  ...        0      0      1         0       0\n",
       "75    0       0      0     0         0  ...        0      0      0         1       0\n",
       "76    0       0      0     0         0  ...        0      0      0         1       0\n",
       "\n",
       "[77 rows x 38 columns]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "matrix_df = pd.DataFrame(matrix.todense(), columns= (vectorizer.get_feature_names()))\n",
    "\n",
    "matrix_df\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"W[q[,eoSbY~~V15N=5p{\">matrix_df</variable><variable id=\"A]5Vf/x,=8dk=KKk3OrJ\">pd</variable><variable id=\"g#vU}+I%b#efZeGj-i8*\">matrix</variable><variable id=\"Xu=hAdiWJ.`f(n4Tn*5t\">vectorizer</variable></variables><block type=\"variables_set\" id=\"AVm^$xGTvQWc9/B#F{G_\" x=\"-22\" y=\"-70\"><field name=\"VAR\" id=\"W[q[,eoSbY~~V15N=5p{\">matrix_df</field><value name=\"VALUE\"><block type=\"varCreateObject\" id=\"c*gvq*J);=ExtPVOl-m;\"><field name=\"VAR\" id=\"A]5Vf/x,=8dk=KKk3OrJ\">pd</field><field name=\"MEMBER\">DataFrame</field><data>pd:DataFrame</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"Y=(/%bNx8K/3Uyij;S8G\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"varDoMethod\" id=\"i+eh0Bi~@3Z,^7aLgNMz\"><field name=\"VAR\" id=\"g#vU}+I%b#efZeGj-i8*\">matrix</field><field name=\"MEMBER\">todense</field><data>matrix:todense</data></block></value><value name=\"ADD1\"><block type=\"valueOutputCodeBlock\" id=\"g]qN!:S$E}_SG?%8KD@$\"><field name=\"CODE\">columns=</field><value name=\"INPUT\"><block type=\"varDoMethod\" id=\"d11CQcX7D#woR(qS]GMU\"><field name=\"VAR\" id=\"Xu=hAdiWJ.`f(n4Tn*5t\">vectorizer</field><field name=\"MEMBER\">get_feature_names</field><data>vectorizer:get_feature_names</data></block></value></block></value></block></value></block></value></block><block type=\"variables_get\" id=\"v,jztGFO5TKlcoeDE_W!\" x=\"-17\" y=\"46\"><field name=\"VAR\" id=\"W[q[,eoSbY~~V15N=5p{\">matrix_df</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------------------\n",
    "**QUESTION:**\n",
    "\n",
    "Does it make sense to weight the term document matrix? Why or why not?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ANSWER: (click here to edit)**\n",
    "\n",
    "*In general, the name of the cereal will only have a single instance of a given word, so it doesn't make much sense to use term frequency.\n",
    "We might consider document frequency weighting for common words, but it's unclear what they would be right now.*\n",
    "\n",
    "----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have the vectorized text in a dataframe, but we still need to add it to `dataframe`.\n",
    "\n",
    "Set `dataframe` to with `pd` do `concat` using a list containing\n",
    "    - a list containing `dataframe` and `matrix_df`\n",
    "    - freestyle `axis=1` (this appends the columns of the dataframes rather than the rows)\n",
    "Display it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>mfr</th>\n",
       "      <th>type</th>\n",
       "      <th>calories</th>\n",
       "      <th>protein</th>\n",
       "      <th>fat</th>\n",
       "      <th>sodium</th>\n",
       "      <th>fiber</th>\n",
       "      <th>carbo</th>\n",
       "      <th>sugars</th>\n",
       "      <th>potass</th>\n",
       "      <th>vitamins</th>\n",
       "      <th>shelf</th>\n",
       "      <th>weight</th>\n",
       "      <th>cups</th>\n",
       "      <th>rating</th>\n",
       "      <th>100</th>\n",
       "      <th>almond</th>\n",
       "      <th>apple</th>\n",
       "      <th>bran</th>\n",
       "      <th>cheerios</th>\n",
       "      <th>chex</th>\n",
       "      <th>cinnamon</th>\n",
       "      <th>corn</th>\n",
       "      <th>crisp</th>\n",
       "      <th>crispy</th>\n",
       "      <th>crunch</th>\n",
       "      <th>dates</th>\n",
       "      <th>flakes</th>\n",
       "      <th>frosted</th>\n",
       "      <th>fruit</th>\n",
       "      <th>golden</th>\n",
       "      <th>grain</th>\n",
       "      <th>grape</th>\n",
       "      <th>honey</th>\n",
       "      <th>just</th>\n",
       "      <th>muesli</th>\n",
       "      <th>nut</th>\n",
       "      <th>nutri</th>\n",
       "      <th>nuts</th>\n",
       "      <th>oat</th>\n",
       "      <th>oatmeal</th>\n",
       "      <th>puffed</th>\n",
       "      <th>quaker</th>\n",
       "      <th>raisin</th>\n",
       "      <th>raisins</th>\n",
       "      <th>rice</th>\n",
       "      <th>right</th>\n",
       "      <th>shredded</th>\n",
       "      <th>squares</th>\n",
       "      <th>total</th>\n",
       "      <th>wheat</th>\n",
       "      <th>wheaties</th>\n",
       "      <th>wheats</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100% Bran</td>\n",
       "      <td>N</td>\n",
       "      <td>C</td>\n",
       "      <td>70</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>130</td>\n",
       "      <td>10.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.33</td>\n",
       "      <td>68.402973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100% Natural Bran</td>\n",
       "      <td>Q</td>\n",
       "      <td>C</td>\n",
       "      <td>120</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "      <td>15</td>\n",
       "      <td>2.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>135.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>33.983679</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>All-Bran</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>70</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>260</td>\n",
       "      <td>9.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>320.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.33</td>\n",
       "      <td>59.425505</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>All-Bran with Extra Fiber</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>50</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>140</td>\n",
       "      <td>14.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>330.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.50</td>\n",
       "      <td>93.704912</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Almond Delight</td>\n",
       "      <td>R</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>200</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>34.384843</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>Triples</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>250</td>\n",
       "      <td>0.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>39.106174</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>Trix</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>140</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>25</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>27.753301</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>Wheat Chex</td>\n",
       "      <td>R</td>\n",
       "      <td>C</td>\n",
       "      <td>100</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>230</td>\n",
       "      <td>3.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>115.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.67</td>\n",
       "      <td>49.787445</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>Wheaties</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>100</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>200</td>\n",
       "      <td>3.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>51.592193</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>Wheaties Honey Gold</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>200</td>\n",
       "      <td>1.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>36.187559</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>77 rows × 54 columns</p>\n",
       "</div>"
      ],
      "text/latex": [
       "None"
      ],
      "text/plain": [
       "                         name mfr type  ...  wheat  wheaties  wheats\n",
       "0                   100% Bran   N    C  ...      0         0       0\n",
       "1           100% Natural Bran   Q    C  ...      0         0       0\n",
       "2                    All-Bran   K    C  ...      0         0       0\n",
       "3   All-Bran with Extra Fiber   K    C  ...      0         0       0\n",
       "4              Almond Delight   R    C  ...      0         0       0\n",
       "..                        ...  ..  ...  ...    ...       ...     ...\n",
       "72                    Triples   G    C  ...      0         0       0\n",
       "73                       Trix   G    C  ...      0         0       0\n",
       "74                 Wheat Chex   R    C  ...      1         0       0\n",
       "75                   Wheaties   G    C  ...      0         1       0\n",
       "76        Wheaties Honey Gold   G    C  ...      0         1       0\n",
       "\n",
       "[77 rows x 54 columns]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe = pd.concat([dataframe, matrix_df], axis=1)\n",
    "\n",
    "dataframe\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</variable><variable id=\"A]5Vf/x,=8dk=KKk3OrJ\">pd</variable><variable id=\"W[q[,eoSbY~~V15N=5p{\">matrix_df</variable></variables><block type=\"variables_set\" id=\"@Kkmf29ozFtEn1.t2t):\" x=\"-21\" y=\"-132\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><value name=\"VALUE\"><block type=\"varDoMethod\" id=\"H+cy/0E_FGGJqDiq[1Qn\"><field name=\"VAR\" id=\"A]5Vf/x,=8dk=KKk3OrJ\">pd</field><field name=\"MEMBER\">concat</field><data>pd:concat</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"GC-,V%~PUk`9LjFfxe8.\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"lists_create_with\" id=\"S1+X8{!g42B!R0tyFUV)\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"variables_get\" id=\"?0E;LOMz6k}Bf8/4+:XI\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field></block></value><value name=\"ADD1\"><block type=\"variables_get\" id=\"6]!)L/nocX6_--Y,.=F2\"><field name=\"VAR\" id=\"W[q[,eoSbY~~V15N=5p{\">matrix_df</field></block></value></block></value><value name=\"ADD1\"><block type=\"dummyOutputCodeBlock\" id=\"-2}W(*#m~vO{^${QMJO:\"><field name=\"CODE\">axis=1</field></block></value></block></value></block></value></block><block type=\"variables_get\" id=\"ED@*AeRW1/:rr71,#3^S\" x=\"-17\" y=\"-22\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model\n",
    "\n",
    "Let's create a linear regression model to predict `fiber`.\n",
    "You may wish to [review multiple regression](https://github.com/memphis-iis/datawhys-content-notebooks/blob/master/Multiple-linear-regression.ipynb).\n",
    "\n",
    "Import `sklearn.linear_model` and `numpy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.linear_model as linear_model\n",
    "import numpy as np\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"!+Hi;Yx;ZB!EQYU8ItpO\">linear_model</variable><variable id=\"YynR+H75hTgW`vKfMxOx\">np</variable></variables><block type=\"importAs\" id=\"m;0Uju49an!8G3YKn4cP\" x=\"93\" y=\"288\"><field name=\"libraryName\">sklearn.linear_model</field><field name=\"libraryAlias\" id=\"!+Hi;Yx;ZB!EQYU8ItpO\">linear_model</field><next><block type=\"importAs\" id=\"^iL#`T{6G3.Uxfj*r`Cv\"><field name=\"libraryName\">numpy</field><field name=\"libraryAlias\" id=\"YynR+H75hTgW`vKfMxOx\">np</field></block></next></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a linear regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "lm = linear_model.LinearRegression()\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"F]q147x/*m|PMfPQU-lZ\">lm</variable><variable id=\"!+Hi;Yx;ZB!EQYU8ItpO\">linear_model</variable></variables><block type=\"variables_set\" id=\"!H`J#y,K:4I.h#,HPeK{\" x=\"127\" y=\"346\"><field name=\"VAR\" id=\"F]q147x/*m|PMfPQU-lZ\">lm</field><value name=\"VALUE\"><block type=\"varCreateObject\" id=\"h:O3ZfE(*c[Hz3sF=$Mm\"><field name=\"VAR\" id=\"!+Hi;Yx;ZB!EQYU8ItpO\">linear_model</field><field name=\"MEMBER\">LinearRegression</field><data>linear_model:LinearRegression</data></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a dataframe `X` based on columns `100` to `wheats`\n",
    "\n",
    "To define these columns, use this freestyle: `dataframe.loc[:,'100':'wheats']`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = dataframe.loc[:,'100':'wheats']\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"!)hBTrKL{[o_8MuO:[mm\">X</variable></variables><block type=\"variables_set\" id=\"?KPFp90zrlAS;}6TH^-#\" x=\"-24\" y=\"-125\"><field name=\"VAR\" id=\"!)hBTrKL{[o_8MuO:[mm\">X</field><value name=\"VALUE\"><block type=\"dummyOutputCodeBlock\" id=\"OOe|HL1zi2f#}/4I1koS\"><field name=\"CODE\">dataframe.loc[:,'100':'wheats']</field></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model to predict `fiber` using `X`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm.fit(X, dataframe['fiber'])\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"F]q147x/*m|PMfPQU-lZ\">lm</variable><variable id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</variable><variable id=\"!)hBTrKL{[o_8MuO:[mm\">X</variable></variables><block type=\"varDoMethod\" id=\"SH-.KDuf!yWP%},Vp-f#\" x=\"-111\" y=\"-134\"><field name=\"VAR\" id=\"F]q147x/*m|PMfPQU-lZ\">lm</field><field name=\"MEMBER\">fit</field><data>lm:</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"rvX$!5J5B.`S`737FEWR\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"variables_get\" id=\"ZBs|.MY}JohrThyj%x0t\"><field name=\"VAR\" id=\"!)hBTrKL{[o_8MuO:[mm\">X</field></block></value><value name=\"ADD1\"><block type=\"indexer\" id=\"Pyox~Ig5!cJ+EwOp3-0G\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><value name=\"INDEX\"><block type=\"text\" id=\"gXtWN[O*AGEl[tQaa--(\"><field name=\"TEXT\">fiber</field></block></value></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the $r^2$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6580210737680805"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm.score(X, dataframe['fiber'])\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"F]q147x/*m|PMfPQU-lZ\">lm</variable><variable id=\"!)hBTrKL{[o_8MuO:[mm\">X</variable><variable id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</variable></variables><block type=\"varDoMethod\" id=\"SH-.KDuf!yWP%},Vp-f#\" x=\"-111\" y=\"-134\"><field name=\"VAR\" id=\"F]q147x/*m|PMfPQU-lZ\">lm</field><field name=\"MEMBER\">score</field><data>lm:score</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"rvX$!5J5B.`S`737FEWR\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"variables_get\" id=\"ZBs|.MY}JohrThyj%x0t\"><field name=\"VAR\" id=\"!)hBTrKL{[o_8MuO:[mm\">X</field></block></value><value name=\"ADD1\"><block type=\"indexer\" id=\"Pyox~Ig5!cJ+EwOp3-0G\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><value name=\"INDEX\"><block type=\"text\" id=\"gXtWN[O*AGEl[tQaa--(\"><field name=\"TEXT\">fiber</field></block></value></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can explain almost 66% of the variance of `fiber` using the name of the cereal!\n",
    "\n",
    "Let's see what words are doing the work for us:\n",
    "\n",
    "- for each item `i` in list as sorted zip a list containing freestyle `lm.coef_` and from `X` get `columns`\n",
    "    - print `i`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(-202458672735294.22, 'just')\n",
      "(-23508344698801.395, 'nuts')\n",
      "(-2.528457892445477, 'oat')\n",
      "(-2.0027899034589813, 'raisin')\n",
      "(-1.717047306424311, 'raisins')\n",
      "(-1.3679816284048034, 'golden')\n",
      "(-1.0721135348111535, 'flakes')\n",
      "(-1.000557552485842, 'nut')\n",
      "(-0.8612322105316365, '100')\n",
      "(-0.7077912870236237, 'chex')\n",
      "(-0.6600999184466105, 'puffed')\n",
      "(-0.6218143357579544, 'rice')\n",
      "(-0.528651693397785, 'crunch')\n",
      "(-0.46529513196688804, 'cinnamon')\n",
      "(-0.35246773674370185, 'shredded')\n",
      "(-0.25053319951339253, 'honey')\n",
      "(-0.039141901232693406, 'total')\n",
      "(-0.008956658643282128, 'apple')\n",
      "(0.12459005337670034, 'quaker')\n",
      "(0.133703802240185, 'wheats')\n",
      "(0.15468490062446807, 'corn')\n",
      "(0.4863786212759184, 'crisp')\n",
      "(0.7096806223771343, 'wheat')\n",
      "(0.8338754831446739, 'cheerios')\n",
      "(0.8665749346086379, 'almond')\n",
      "(0.9794710833195207, 'grain')\n",
      "(1.0880246102906006, 'wheaties')\n",
      "(1.107992257379842, 'dates')\n",
      "(1.160425769438441, 'nutri')\n",
      "(1.4139691656399291, 'frosted')\n",
      "(1.7040460191039914, 'oatmeal')\n",
      "(1.95817897981296, 'crispy')\n",
      "(2.183006199405229, 'fruit')\n",
      "(3.0581076785166994, 'muesli')\n",
      "(3.1538108021025106, 'squares')\n",
      "(5.7789317676680545, 'bran')\n",
      "(23508344698803.82, 'grape')\n",
      "(202458672735293.97, 'right')\n"
     ]
    }
   ],
   "source": [
    "for i in sorted(zip(lm.coef_, X.columns)):\n",
    "  print(i)\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\":Kr5%b?Ez_%b!yoj+ua4\">i</variable><variable id=\"!)hBTrKL{[o_8MuO:[mm\">X</variable></variables><block type=\"controls_forEach\" id=\"#?O,@s%7zW|@gYPHVa;@\" x=\"-93\" y=\"73\"><field name=\"VAR\" id=\":Kr5%b?Ez_%b!yoj+ua4\">i</field><value name=\"LIST\"><block type=\"sortedBlock\" id=\"!Kw_pgqz!v-.60.+N#74\"><value name=\"x\"><block type=\"zipBlock\" id=\"Y};j[{F2%yZ0$0`kx%ww\"><value name=\"x\"><block type=\"lists_create_with\" id=\"xo?#mRg,l/LdhW?):9Ep\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"dummyOutputCodeBlock\" id=\"#;Nw/pe6|^LG`6j0cl7,\"><field name=\"CODE\">lm.coef_</field></block></value><value name=\"ADD1\"><block type=\"varGetProperty\" id=\"_C7CMp=u2`#A];}?EP0(\"><field name=\"VAR\" id=\"!)hBTrKL{[o_8MuO:[mm\">X</field><field name=\"MEMBER\">columns</field><data>X:columns</data></block></value></block></value></block></value></block></value><statement name=\"DO\"><block type=\"text_print\" id=\"n-#hKK{Ai[0S?W6jV9q2\"><value name=\"TEXT\"><shadow type=\"text\" id=\"DvmAgfF2mRc2bT#m4q6y\"><field name=\"TEXT\">abc</field></shadow><block type=\"variables_get\" id=\"jUL^N4!74tnnxxarX8aF\"><field name=\"VAR\" id=\":Kr5%b?Ez_%b!yoj+ua4\">i</field></block></value></block></statement></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some of these seem intuitive, like `bran` but others less so.\n",
    "\n",
    "Show all the rows of the dataframe where fiber > 3 (the 4th quartile)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>mfr</th>\n",
       "      <th>type</th>\n",
       "      <th>calories</th>\n",
       "      <th>protein</th>\n",
       "      <th>fat</th>\n",
       "      <th>sodium</th>\n",
       "      <th>fiber</th>\n",
       "      <th>carbo</th>\n",
       "      <th>sugars</th>\n",
       "      <th>potass</th>\n",
       "      <th>vitamins</th>\n",
       "      <th>shelf</th>\n",
       "      <th>weight</th>\n",
       "      <th>cups</th>\n",
       "      <th>rating</th>\n",
       "      <th>100</th>\n",
       "      <th>almond</th>\n",
       "      <th>apple</th>\n",
       "      <th>bran</th>\n",
       "      <th>cheerios</th>\n",
       "      <th>chex</th>\n",
       "      <th>cinnamon</th>\n",
       "      <th>corn</th>\n",
       "      <th>crisp</th>\n",
       "      <th>crispy</th>\n",
       "      <th>crunch</th>\n",
       "      <th>dates</th>\n",
       "      <th>flakes</th>\n",
       "      <th>frosted</th>\n",
       "      <th>fruit</th>\n",
       "      <th>golden</th>\n",
       "      <th>grain</th>\n",
       "      <th>grape</th>\n",
       "      <th>honey</th>\n",
       "      <th>just</th>\n",
       "      <th>muesli</th>\n",
       "      <th>nut</th>\n",
       "      <th>nutri</th>\n",
       "      <th>nuts</th>\n",
       "      <th>oat</th>\n",
       "      <th>oatmeal</th>\n",
       "      <th>puffed</th>\n",
       "      <th>quaker</th>\n",
       "      <th>raisin</th>\n",
       "      <th>raisins</th>\n",
       "      <th>rice</th>\n",
       "      <th>right</th>\n",
       "      <th>shredded</th>\n",
       "      <th>squares</th>\n",
       "      <th>total</th>\n",
       "      <th>wheat</th>\n",
       "      <th>wheaties</th>\n",
       "      <th>wheats</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100% Bran</td>\n",
       "      <td>N</td>\n",
       "      <td>C</td>\n",
       "      <td>70</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>130</td>\n",
       "      <td>10.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.33</td>\n",
       "      <td>68.402973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>All-Bran</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>70</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>260</td>\n",
       "      <td>9.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>320.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.33</td>\n",
       "      <td>59.425505</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>All-Bran with Extra Fiber</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>50</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>140</td>\n",
       "      <td>14.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>330.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.50</td>\n",
       "      <td>93.704912</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Bran Chex</td>\n",
       "      <td>R</td>\n",
       "      <td>C</td>\n",
       "      <td>90</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>200</td>\n",
       "      <td>4.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>125.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.67</td>\n",
       "      <td>49.120253</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Bran Flakes</td>\n",
       "      <td>P</td>\n",
       "      <td>C</td>\n",
       "      <td>90</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>210</td>\n",
       "      <td>5.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.67</td>\n",
       "      <td>53.313813</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Cracklin' Oat Bran</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>140</td>\n",
       "      <td>4.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>160.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.50</td>\n",
       "      <td>40.448772</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Fruit &amp; Fibre Dates; Walnuts; and Oats</td>\n",
       "      <td>P</td>\n",
       "      <td>C</td>\n",
       "      <td>120</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>160</td>\n",
       "      <td>5.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>200.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.25</td>\n",
       "      <td>0.67</td>\n",
       "      <td>40.917047</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Fruitful Bran</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>120</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>240</td>\n",
       "      <td>5.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.33</td>\n",
       "      <td>0.67</td>\n",
       "      <td>41.015492</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>Post Nat. Raisin Bran</td>\n",
       "      <td>P</td>\n",
       "      <td>C</td>\n",
       "      <td>120</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>200</td>\n",
       "      <td>6.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>260.0</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>1.33</td>\n",
       "      <td>0.67</td>\n",
       "      <td>37.840594</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>Raisin Bran</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>120</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>210</td>\n",
       "      <td>5.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>240.0</td>\n",
       "      <td>25</td>\n",
       "      <td>2</td>\n",
       "      <td>1.33</td>\n",
       "      <td>0.75</td>\n",
       "      <td>39.259197</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>Shredded Wheat 'n'Bran</td>\n",
       "      <td>N</td>\n",
       "      <td>C</td>\n",
       "      <td>90</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>140.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.67</td>\n",
       "      <td>74.472949</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>Total Raisin Bran</td>\n",
       "      <td>G</td>\n",
       "      <td>C</td>\n",
       "      <td>140</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>190</td>\n",
       "      <td>4.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>230.0</td>\n",
       "      <td>100</td>\n",
       "      <td>3</td>\n",
       "      <td>1.50</td>\n",
       "      <td>1.00</td>\n",
       "      <td>28.592785</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/latex": [
       "None"
      ],
      "text/plain": [
       "                                      name mfr type  ...  wheat  wheaties  wheats\n",
       "0                                100% Bran   N    C  ...      0         0       0\n",
       "2                                 All-Bran   K    C  ...      0         0       0\n",
       "3                All-Bran with Extra Fiber   K    C  ...      0         0       0\n",
       "8                                Bran Chex   R    C  ...      0         0       0\n",
       "9                              Bran Flakes   P    C  ...      0         0       0\n",
       "19                      Cracklin' Oat Bran   K    C  ...      0         0       0\n",
       "27  Fruit & Fibre Dates; Walnuts; and Oats   P    C  ...      0         0       0\n",
       "28                           Fruitful Bran   K    C  ...      0         0       0\n",
       "52                   Post Nat. Raisin Bran   P    C  ...      0         0       0\n",
       "58                             Raisin Bran   K    C  ...      0         0       0\n",
       "64                  Shredded Wheat 'n'Bran   N    C  ...      1         0       0\n",
       "70                       Total Raisin Bran   G    C  ...      0         0       0\n",
       "\n",
       "[12 rows x 54 columns]"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe[(dataframe['fiber'] > 3)]\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</variable></variables><block type=\"indexer\" id=\"(feI)IP?mEyrRC5;~PVN\" x=\"-124\" y=\"-132\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><value name=\"INDEX\"><block type=\"logic_compare\" id=\"Sxca#F6{U$UpZy]%9C8P\"><field name=\"OP\">GT</field><value name=\"A\"><block type=\"indexer\" id=\"#6%*]Thox/+k0kZJ5%M,\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><value name=\"INDEX\"><block type=\"text\" id=\"Y)C9:rQQ}ghSsP`5!/03\"><field name=\"TEXT\">fiber</field></block></value></block></value><value name=\"B\"><block type=\"math_number\" id=\"f~HIxit*U52Z?RG0a!|.\"><field name=\"NUM\">3</field></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the last element, `Total Raisin Bran`. \n",
    "We can see that `bran` is a strong positive coefficient and that `raisin` is a strong negative one.\n",
    "We can expect that a linear model will need to oppositely weight words in a name to get good overall fit.\n",
    "\n",
    "Let's look at the mystery of `right`.\n",
    "Find all rows where `name` contains \"Right\", i.e. freestyle `dataframe['name'].str.contains(\"Right\")`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>mfr</th>\n",
       "      <th>type</th>\n",
       "      <th>calories</th>\n",
       "      <th>protein</th>\n",
       "      <th>fat</th>\n",
       "      <th>sodium</th>\n",
       "      <th>fiber</th>\n",
       "      <th>carbo</th>\n",
       "      <th>sugars</th>\n",
       "      <th>potass</th>\n",
       "      <th>vitamins</th>\n",
       "      <th>shelf</th>\n",
       "      <th>weight</th>\n",
       "      <th>cups</th>\n",
       "      <th>rating</th>\n",
       "      <th>100</th>\n",
       "      <th>almond</th>\n",
       "      <th>apple</th>\n",
       "      <th>bran</th>\n",
       "      <th>cheerios</th>\n",
       "      <th>chex</th>\n",
       "      <th>cinnamon</th>\n",
       "      <th>corn</th>\n",
       "      <th>crisp</th>\n",
       "      <th>crispy</th>\n",
       "      <th>crunch</th>\n",
       "      <th>dates</th>\n",
       "      <th>flakes</th>\n",
       "      <th>frosted</th>\n",
       "      <th>fruit</th>\n",
       "      <th>golden</th>\n",
       "      <th>grain</th>\n",
       "      <th>grape</th>\n",
       "      <th>honey</th>\n",
       "      <th>just</th>\n",
       "      <th>muesli</th>\n",
       "      <th>nut</th>\n",
       "      <th>nutri</th>\n",
       "      <th>nuts</th>\n",
       "      <th>oat</th>\n",
       "      <th>oatmeal</th>\n",
       "      <th>puffed</th>\n",
       "      <th>quaker</th>\n",
       "      <th>raisin</th>\n",
       "      <th>raisins</th>\n",
       "      <th>rice</th>\n",
       "      <th>right</th>\n",
       "      <th>shredded</th>\n",
       "      <th>squares</th>\n",
       "      <th>total</th>\n",
       "      <th>wheat</th>\n",
       "      <th>wheaties</th>\n",
       "      <th>wheats</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Just Right Crunchy  Nuggets</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>110</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>170</td>\n",
       "      <td>1.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>100</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>36.523683</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>Just Right Fruit &amp; Nut</td>\n",
       "      <td>K</td>\n",
       "      <td>C</td>\n",
       "      <td>140</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>170</td>\n",
       "      <td>2.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>95.0</td>\n",
       "      <td>100</td>\n",
       "      <td>3</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.75</td>\n",
       "      <td>36.471512</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/latex": [
       "None"
      ],
      "text/plain": [
       "                           name mfr type  ...  wheat  wheaties  wheats\n",
       "38  Just Right Crunchy  Nuggets   K    C  ...      0         0       0\n",
       "39       Just Right Fruit & Nut   K    C  ...      0         0       0\n",
       "\n",
       "[2 rows x 54 columns]"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe[dataframe['name'].str.contains(\"Right\")]\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</variable></variables><block type=\"indexer\" id=\"YJfBaX!6tlDaI8cIwMi`\" x=\"-124\" y=\"257\"><field name=\"VAR\" id=\"Y+grzOaS*0=OHV6zO8m/\">dataframe</field><value name=\"INDEX\"><block type=\"dummyOutputCodeBlock\" id=\"i=hv?U%[ge!cecE5WjLp\"><field name=\"CODE\">dataframe['name'].str.contains(\"Right\")</field></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It seems these two cereals have the min/max weighted words - these extreme weights seem to be a colinearity issue because \"Just\" and \"Right\" only occur together in these names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "xpython",
   "language": "python",
   "name": "xpython"
  },
  "language_info": {
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}