code/03_Modeling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7b9770f7-b62b-4cb0-92ff-a391feded4ef",
   "metadata": {},
   "source": [
    "# 3.0 Modeling "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "7c80b224-811b-4c72-82ff-8d9a32e66e75",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV\n",
    "\n",
    "from sklearn.metrics import plot_confusion_matrix, confusion_matrix\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "2b52cb88-f0ee-4ae5-9350-a4536939f13d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>title</th>\n",
       "      <th>author</th>\n",
       "      <th>selftext</th>\n",
       "      <th>media_only</th>\n",
       "      <th>num_comments</th>\n",
       "      <th>subreddit</th>\n",
       "      <th>created_utc</th>\n",
       "      <th>title_length</th>\n",
       "      <th>title_word_count</th>\n",
       "      <th>...</th>\n",
       "      <th>lemma_selftext</th>\n",
       "      <th>stemma_selftext</th>\n",
       "      <th>t_neg</th>\n",
       "      <th>t_neu</th>\n",
       "      <th>t_pos</th>\n",
       "      <th>t_compound</th>\n",
       "      <th>s_neg</th>\n",
       "      <th>s_neu</th>\n",
       "      <th>s_pos</th>\n",
       "      <th>s_compound</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Is there anyway to Trade US options in UK</td>\n",
       "      <td>yoghin</td>\n",
       "      <td>I can't find any platforms that offer US optio...</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1645974032</td>\n",
       "      <td>41</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>I can not find any platform that offer US opti...</td>\n",
       "      <td>I can not find any platform that offer US opti...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.876</td>\n",
       "      <td>0.124</td>\n",
       "      <td>0.1779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>NVDA's short leg</td>\n",
       "      <td>listenless</td>\n",
       "      <td>I bought 240-200  Jan 2023 bear puts spread o...</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1645971541</td>\n",
       "      <td>16</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>I bought 240-200 Jan 2023 bear put spread on N...</td>\n",
       "      <td>I bought 240-200 Jan 2023 bear put spread on N...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.094</td>\n",
       "      <td>0.816</td>\n",
       "      <td>0.090</td>\n",
       "      <td>-0.1830</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Patrick Boyle breaks down tons of consequences...</td>\n",
       "      <td>professorfundamental</td>\n",
       "      <td>This video: \\n\\n[https://www.youtube.com/watch...</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1645955748</td>\n",
       "      <td>152</td>\n",
       "      <td>25</td>\n",
       "      <td>...</td>\n",
       "      <td>This video : [ http : //www.youtube.com/watch ...</td>\n",
       "      <td>This video : [ http : //www.youtube.com/watch ...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.923</td>\n",
       "      <td>0.077</td>\n",
       "      <td>0.2500</td>\n",
       "      <td>0.053</td>\n",
       "      <td>0.873</td>\n",
       "      <td>0.075</td>\n",
       "      <td>0.5122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Did a play paper trading on TOS. Made $900 now...</td>\n",
       "      <td>doyzoe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1645954258</td>\n",
       "      <td>125</td>\n",
       "      <td>26</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.783</td>\n",
       "      <td>0.217</td>\n",
       "      <td>0.6249</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Patrick Boyle breaks down tons of consequences...</td>\n",
       "      <td>professorfundamental</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1645951913</td>\n",
       "      <td>152</td>\n",
       "      <td>25</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.923</td>\n",
       "      <td>0.077</td>\n",
       "      <td>0.2500</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0                                              title  \\\n",
       "0           0          Is there anyway to Trade US options in UK   \n",
       "1           1                                   NVDA's short leg   \n",
       "2           2  Patrick Boyle breaks down tons of consequences...   \n",
       "3           3  Did a play paper trading on TOS. Made $900 now...   \n",
       "4           4  Patrick Boyle breaks down tons of consequences...   \n",
       "\n",
       "                 author                                           selftext  \\\n",
       "0                yoghin  I can't find any platforms that offer US optio...   \n",
       "1            listenless   I bought 240-200  Jan 2023 bear puts spread o...   \n",
       "2  professorfundamental  This video: \\n\\n[https://www.youtube.com/watch...   \n",
       "3                doyzoe                                                NaN   \n",
       "4  professorfundamental                                                NaN   \n",
       "\n",
       "   media_only  num_comments  subreddit  created_utc  title_length  \\\n",
       "0       False             0          0   1645974032            41   \n",
       "1       False             0          0   1645971541            16   \n",
       "2       False             0          0   1645955748           152   \n",
       "3       False             1          0   1645954258           125   \n",
       "4       False             1          0   1645951913           152   \n",
       "\n",
       "   title_word_count  ...                                     lemma_selftext  \\\n",
       "0                 9  ...  I can not find any platform that offer US opti...   \n",
       "1                 3  ...  I bought 240-200 Jan 2023 bear put spread on N...   \n",
       "2                25  ...  This video : [ http : //www.youtube.com/watch ...   \n",
       "3                26  ...                                                NaN   \n",
       "4                25  ...                                                NaN   \n",
       "\n",
       "                                     stemma_selftext t_neg  t_neu  t_pos  \\\n",
       "0  I can not find any platform that offer US opti...   0.0  1.000  0.000   \n",
       "1  I bought 240-200 Jan 2023 bear put spread on N...   0.0  1.000  0.000   \n",
       "2  This video : [ http : //www.youtube.com/watch ...   0.0  0.923  0.077   \n",
       "3                                                NaN   0.0  0.783  0.217   \n",
       "4                                                NaN   0.0  0.923  0.077   \n",
       "\n",
       "  t_compound  s_neg  s_neu  s_pos s_compound  \n",
       "0     0.0000  0.000  0.876  0.124     0.1779  \n",
       "1     0.0000  0.094  0.816  0.090    -0.1830  \n",
       "2     0.2500  0.053  0.873  0.075     0.5122  \n",
       "3     0.6249  0.000  0.000  0.000     0.0000  \n",
       "4     0.2500  0.000  0.000  0.000     0.0000  \n",
       "\n",
       "[5 rows x 28 columns]"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('../data/clean_data/reddit_content_clean.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd2fc1e8-ad47-4b40-af96-9eb240b184f4",
   "metadata": {},
   "source": [
    "**Find and Fill missing values**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "9a4d9098-ac8d-4c27-96d3-4945374d15e5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0               0\n",
       "title                    0\n",
       "author                   0\n",
       "selftext               298\n",
       "media_only               0\n",
       "num_comments             0\n",
       "subreddit                0\n",
       "created_utc              0\n",
       "title_length             0\n",
       "title_word_count         0\n",
       "selftext_length          0\n",
       "selftext_word_count      0\n",
       "no_cont_title            0\n",
       "no_cont_selftext       298\n",
       "tokenized_title          0\n",
       "tokenized_selftext       0\n",
       "lemma_title              0\n",
       "stemma_title             0\n",
       "lemma_selftext         298\n",
       "stemma_selftext        298\n",
       "t_neg                    0\n",
       "t_neu                    0\n",
       "t_pos                    0\n",
       "t_compound               0\n",
       "s_neg                    0\n",
       "s_neu                    0\n",
       "s_pos                    0\n",
       "s_compound               0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "80a3fffb-d7fd-40b3-862f-cb1fa58fee37",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "df.fillna({'selftext':'','no_cont_selftext':'','lemma_selftext':'','stemma_selftext':''}, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "aa66cdca-be1f-4dc6-babe-28532887d3b9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0             0\n",
       "title                  0\n",
       "author                 0\n",
       "selftext               0\n",
       "media_only             0\n",
       "num_comments           0\n",
       "subreddit              0\n",
       "created_utc            0\n",
       "title_length           0\n",
       "title_word_count       0\n",
       "selftext_length        0\n",
       "selftext_word_count    0\n",
       "no_cont_title          0\n",
       "no_cont_selftext       0\n",
       "tokenized_title        0\n",
       "tokenized_selftext     0\n",
       "lemma_title            0\n",
       "stemma_title           0\n",
       "lemma_selftext         0\n",
       "stemma_selftext        0\n",
       "t_neg                  0\n",
       "t_neu                  0\n",
       "t_pos                  0\n",
       "t_compound             0\n",
       "s_neg                  0\n",
       "s_neu                  0\n",
       "s_pos                  0\n",
       "s_compound             0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee0391d6-5fa8-4c14-82a9-8c25f6095348",
   "metadata": {},
   "source": [
    "**Model Data Setup**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "0554a5db-5b19-46bc-9766-c6e17995d4c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0             Is there anyway to Trade US option in UK\n",
       "1                                    NVDA 's short leg\n",
       "2    Patrick Boyle break down ton of consequence of...\n",
       "3    Did a play paper trading on TOS . Made $ 900 n...\n",
       "4    Patrick Boyle break down ton of consequence of...\n",
       "Name: lemma_title, dtype: object"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# X = df['title']\n",
    "X = df['lemma_title']\n",
    "# X = df['stemma_title']\n",
    "\n",
    "# X = df['selftext']\n",
    "# X = df['lemma_selftext']\n",
    "# X = df['stemma_selftext']\n",
    "\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "ebe6094d-6314-4d23-89c4-7a68289486d1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0\n",
       "1    0\n",
       "2    0\n",
       "3    0\n",
       "4    0\n",
       "Name: subreddit, dtype: int64"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['subreddit']\n",
    "y.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "e8e9b95a-72ea-4f51-906b-12e437cc38ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.50025\n",
       "1    0.49975\n",
       "Name: subreddit, dtype: float64"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.value_counts(normalize=True)   # baseline is 50.025% accurate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "c899c822-5b5e-4423-9f8e-3577b78c7118",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X,y,random_state=42, stratify=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1acb10a6-9418-4a67-a478-b529b8b64d57",
   "metadata": {},
   "source": [
    "**3.1 Logistic Regression and Count Vectorizer**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "b17749c9-8d7a-4d62-96e2-fee53b03d537",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 6 candidates, totalling 30 fits\n",
      "0.827877017250974\n",
      "{'lr__max_iter': 1000, 'vect__ngram_range': (1, 2), 'vect__stop_words': None}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAEGCAYAAAAg6I3HAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAi3UlEQVR4nO3de5xVdb3/8dd7BgS5CCKo3FJUzIMaqHjNTM0CrZNa9jtgeTlWVEKmeU5JN1N/nKOV2cmjlmlKqRmlJXmFCI/iURFQuSmBoYigCOIFlIGZ+Zw/1hrc4syePbDX7FnD+/l4rAd7f/d3fdd3zzCf/d3f9b0oIjAzs/KrqnQFzMzaKwdYM7OMOMCamWXEAdbMLCMOsGZmGelQ6Qq0JV133iF69d+x0tWwFnjj+W6VroK1wIaa19m4ab22pYwRx3WNNa/VlZR39tyaByJi5LZcb1s4wBbo1X9HvjHpiEpXw1rg/jEfqXQVrAVmPnXdNpex+rU6Hn9gQEl5O/Z9rvc2X3AbOMCaWc4EdVFf6UqUxAHWzHIlgHryMUHKAdbMcqcet2DNzMouCDa5i8DMrPwCqHMXgZlZNtwHa2aWgQDqcrIKoAOsmeVOPnpgHWDNLGeCcB+smVkWImBTPuKrA6yZ5Y2oY5uWM2g1DrBmlisB1LsFa2aWDbdgzcwykEw0cIA1Myu7ADZFPvYKcIA1s1wJRF1ONmNxgDWz3KkPdxGYmZVdnvpg89HONjPbTNRFVUlHSaVJ1ZKelHR3+ryXpKmSFqf/7lyQd7ykJZIWSRrRXNkOsGaWK8mOBlUlHSX6BvBMwfOLgGkRMRiYlj5H0hBgFLA/MBK4VlJ1sYIdYM0sVyLExqgu6WiOpAHAJ4EbCpJPBiamjycCpxSk3x4RNRGxFFgCHFasfAdYM8udelTSAfSWNKvgGLNFUT8DvsV7F+jaLSJWAqT/7pqm9wdeLMi3PE1rkm9ymVmuJDe5Sm4bro6I4Y29IOlTwKqImC3p2BLKauzOWtFJuw6wZpYzKvkGVjM+DHxa0klAZ2AnSbcAr0jqGxErJfUFVqX5lwMDC84fAKwodgF3EZhZrpTrJldEjI+IARGxJ8nNq79FxBeAycBZabazgLvSx5OBUZI6SRoEDAZmFruGW7Bmljt12U40uByYJOmLwDLgcwARsUDSJGAhUAuMjYi6YgU5wJpZrgRiU5Q3dEXEg8CD6eM1wMeayDcBmFBquQ6wZpYrLbzJVVEOsGaWK4Gy7iIoGwdYM8udFszSqigHWDPLlQjKNUwrcw6wZpYryU2u5qfBtgUOsGaWO77JZWaWgUBecNvMLCtuwZqZZSCAet/kMjPLgnKzZYwDrJnlSrJtt0cRmJmVXYTcRWBmlhVPNDAzy0CyHqz7YM3MMlC2HQ0y5wBrZrmSDNNyC9bMrOy8FoGZWYa8XKGZWQaS5Qrz0UWQj48BM7MC9aGSjmIkdZY0U9LTkhZIuiRN/6GklyQ9lR4nFZwzXtISSYskjWiunm7BmlmuJKtplaVtWAMcHxHrJHUEZki6L33tqoj4SWFmSUNItvfeH+gH/FXSvsV2lnWANbNcSabKbnuAjYgA1qVPO6ZHFDnlZOD2iKgBlkpaAhwGPNrUCQ6wOVdXAzPP6kr9Rog62P3jtewzroanL9yR9c8n/wk3vSU6dg+OumP95vPeWSke+XQ39j63hkH/urFS1d8uffPcRzjikJd4/Y3OjPnmpwH48hmzOGL4cjbVVrHy5e785JoPs/7tHQAYtMdavjHmUbp02UTUi3EXfZJNm/JxFz0b5ZsqK6kamA3sA1wTEY9LOhEYJ+lMYBZwYUSsBfoDjxWcvjxNa1KmAVbSAOAaYAhJf+/dwL9HRKN/0ZJ6AqdHxLXp837AzyPitCzrmWdVO8Chv15Phy5QvwlmntmV3h+pZeiV72zO8+yPO9Gh23s/mJ+9ojO9P1Lb2tU1YOr0fZh833586+uPbE6bM7cfN956MPX1VXzxC7MZ9Zl53HjLIVRV1fPt8x7mRz8/mn+80Ivu3TZQV5ePGzxZasFMrt6SZhU8vz4irm94kn69H5bGnj9JOgC4DriMpDV7GXAlcA40etFiLd7sbnJJEnAn8OeIGAzsC3QDJhQ5rSdwbsOTiFjh4FqcBB26JI+jFuprec9/gwh45f6O9D3p3WD6yrQOdBlQT7e961u3sgbAvGd24611nd6TNvvpftTXJ3+Oz/69D312eRuAQ4auYOkLO/OPF3oB8Na6zpvzba8aRhGUcgCrI2J4wXF942XG68CDwMiIeCUi6iKiHvgVSTcAJC3WgQWnDQBWFKtrlr+p44ENEXETbP6kuAA4R9K5ku6SdH96N+7i9JzLgb3TO3c/lrSnpPmw+Y7fTZLmSXpS0nFp+tmS7kzLWizpR2l6taSbJc1Pz7kgw/daUVEH//vZrkw/pju7HFlLzw+92+e+dnY1O+wSdN0jCaa1b8PSX+/A3ufWVKq61owRxy/hiTnJN88B/d4kQvzH96ZyzY/u5nMnz69w7dqG+qgq6ShGUp+05YqkHYETgGcl9S3IdirQ8EOfDIyS1EnSIGAwMLPYNbLsItifpG9js4h4U9Ky9LqHAQcAbwNPSLoHuAg4ICKGAUjas+D0sWkZB0raD5giad/0tWHAQSR3BRdJuhrYFegfEQekZfVsrJKSxgBjAHr27bxNb7hSVA1H3bGeTW/Ck9/owluLq+g+OAmoL9/bkb4nbdqc97lrOrHnGRs3t3qtbRn9mbnU1YlpDw8CoLo6OGC/VYy76CRqajpwxcVTWPyPXXhqXt9mSmq/yrgnV19gYtoPWwVMioi7Jf1W0jCSr//PA18BiIgFkiYBC4FaYGyxEQSQbYAVjfdPNKRPjYg1AJLuBI4G/lykvKOBqwEi4llJL5B0OwBMi4g30rIWAnsAC4C90mB7DzClsULTrwzXAww8oEfR/pS2ruNO0OvQWlbP6ED3wRupr4VX/tqBIye9e3Pr9XnVvDy1I4t+CrVvCQRVnYI9Tt9UpGRrDR//6HMcfshyvn3JJ2jo51m9pgtzF+7Gm28lH/5PPDmAwYPWbOcBFmrLM4pgLknDbMv0M4qcM4Hi3ZzvkWUXwQJgeGGCpJ1I+jDqeH/wbS64FfvIKvy+Wwd0SO/6DSXpVxkL3NB8lfNn42ti05vJ47oNsOaxDnQdlLRe1zxWTde96um8+7s/2sN/8zYfnbKOj05Zxx5f2MheX65xcG0Dhg97if93ynwuvuJ4aja+2+6Z9VQ/Bu2xlk471FJVVc+BQ17mheU9K1fRNqIcXQStIcsW7DTgcklnRsRv0mb4lcDNJN0CH5fUC3gHOIXkLt1bQPcmynsI+Dzwt7Rr4APAIuDgxjJL6g1sjIg7JD2XXrfdqXlVzPtuFyL9yNptxCZ2PTa5ofXyfR3pe6KDZ1sz/vyH+ND+r9Cj+wZu/eUf+e3vh/Ivp85nh451XP79qQA8s7gPP7/+CNat78SdfxnC1VfcAyFmzunPzDkDKvwOKqyEWVptRWYBNiJC0qnAtZK+T9Javhf4DjAamAH8lmT82W0RMQtA0iPpja37SIZ4NbgW+IWkeST9H2dHRE0yWKFR/YGbJDV8jI0v6xtsI7p/sJ6j/ri+0dcOnLCh6Ln7jPWNrkr4z58d8760+/82uMn80x7ei2kP75VllXLFC26nIuJF4J+3TE+D4qqIGNfIOadvkXRAmr4BOLuR/DdT0DqNiE8VvNxo69bM8m27b8GamWXBC243Y8tWp5lZqQJRm5PJFm7BmlnuuA/WzCwL4S4CM7NMuA/WzCxDDrBmZhkIRJ1vcpmZZcM3uczMMhC+yWVmlp1wgDUzy4IXezEzy4xbsGZmGYiAunoHWDOzTHgUgZlZBgJ3EZiZZSQ/N7nyMR3CzKxARGlHMZI6S5op6WlJCyRdkqb3kjRV0uL0350LzhkvaYmkRZJGNFdPB1gzy50IlXQ0owY4PiKGAsOAkZKOAC4i2al6MMneghcBSBoCjAL2B0aSbIdVXewCDrBmlivJKIKqko7i5URExLr0acf0COBkYGKaPpFkU1bS9NsjoiYilgJLgMOKXcMB1sxypwVdBL0lzSo4xhSWI6la0lPAKmBqRDwO7BYRK5PrxEpg1zR7f+DFgtOXp2lN8k0uM8udFowiWB0Rw5suJ+qAYZJ6An+SdECRshq7aNGeXrdgzSxXgtL6X1sylCsiXgceJOlbfUVSX4D031VptuXAwILTBgAripXrAGtmuRMlHsVI6pO2XJG0I3AC8CwwGTgrzXYWcFf6eDIwSlInSYOAwcDMYtdwF4GZ5UtAlGeqbF9gYjoSoAqYFBF3S3oUmCTpi8Ay4HMAEbFA0iRgIVALjE27GJrkAGtmuVOOmVwRMRc4qJH0NcDHmjhnAjCh1Gs4wJpZ7jQ3iaCtaDLASrqaIt0YEXFeJjUyMyuivaxFMKvVamFmVqoA8h5gI2Ji4XNJXSNiffZVMjMrLi9dBM0O05J0pKSFwDPp86GSrs28ZmZmjRJRX9pRaaWMg/0ZMAJYAxARTwPHZFgnM7PiyjEQthWUNIogIl6U3vNpUHTsl5lZZqJ93ORq8KKko4CQtANwHml3gZlZRbSB1mkpSuki+CowlmTVmJdI1k0cm2GdzMyaoRKPymq2BRsRq4HPt0JdzMxKU1/pCpSmlFEEe0n6i6RXJa2SdJekvVqjcmZm79MwDraUo8JK6SK4DZhEsjBCP+APwO+yrJSZWTHl2JOrNZQSYBURv42I2vS4hdx0MZtZu5T3YVqSeqUPp0u6CLidpMr/AtzTCnUzM2tcG/j6X4piN7lmkwTUhnfylYLXArgsq0qZmRWjNtA6LUWxtQgGtWZFzMxKEoI2MA22FCXN5Eo3AhsCdG5Ii4jfZFUpM7Oi8t6CbSDpYuBYkgB7L3AiMANwgDWzyshJgC1lFMFpJNsnvBwR/woMBTplWiszs2LyPoqgwDsRUS+pVtJOJFvYeqKBmVVGjhbcLqUFOyvd2vZXJCML5tDMVrVmZllSlHYULUMaKGm6pGckLZD0jTT9h5JekvRUepxUcM54SUskLZI0orl6lrIWwbnpw19Iuh/YKd2N0cysMsrz9b8WuDAi5kjqDsyWNDV97aqI+ElhZklDgFHA/iSzWv8qad9iW3cXm2hwcLHXImJOC96ImVnZlGMcbESsBFamj9+S9AzJqoFNORm4PSJqgKWSlgCHAY82dUKxFuyVxeoGHF/k9Vx6Y0E1DxywU6WrYS0wZcXE5jNZm3HYiDXlKaj0Ptjekgo3cL0+Iq7fMpOkPYGDgMeBDwPjJJ1JsvnrhRGxliT4PlZw2nKKB+SiEw2OK/UdmJm1mpaNEFgdEcOLZZDUDbgDOD8i3pR0HclM1YYZq1cC59D4ArNFa1LKTS4zs7alTMO0JHUkCa63RsSdABHxSkTURUQ9yc39w9Lsy4GBBacPAFYUK98B1sxyR/WlHUXLSDYavBF4JiJ+WpDetyDbqcD89PFkYJSkTpIGAYNpZkRVSVNlzczalPKMIvgwcAYwT9JTadp3gNGShqVXeZ50oauIWCBpErCQZATC2GIjCKC0qbIi2TJmr4i4VNIHgN0jwmNhzazVlTLGtRQRMYPG+1XvLXLOBGBCqdcopYvgWuBIYHT6/C3gmlIvYGZWdjnZMqaULoLDI+JgSU8CRMTadPtuM7PKaAPrDJSilAC7SVI16VuS1Ifc7OloZu1R7hfcLvBz4E/ArpImkKyu9b1Ma2Vm1pRofoRAW1HKWgS3SppNsmShgFMi4pnMa2Zm1pT20oJNRw28DfylMC0ilmVZMTOzJrWXAEuyg2zD5oedgUHAIpIVZczMWl276YONiAMLn6erbH2liexmZpZq8UyudO3EQ7OojJlZSdpLC1bSNwueVgEHA69mViMzs2La0ygCoHvB41qSPtk7sqmOmVkJ2kMLNp1g0C0i/r2V6mNmVpRoBze5JHWIiNpiW8eYmVVE3gMsyTqHBwNPSZoM/AFY3/Biw+K0ZmatqkyrabWGUvpgewFrSPbgahgPG4ADrJlVRju4ybVrOoJgPu8G1gY5+fwws/aoPbRgq4FubMVGX2ZmmcpJBCoWYFdGxKWtVhMzs1K0bFfZiioWYCu/HLiZWSPaQxfBx1qtFmZmLZGTANvknlwR8VprVsTMrFRl2rZ7oKTpkp6RtEDSN9L0XpKmSlqc/rtzwTnjJS2RtEjSiObqWcqmh2ZmbUe04CiuFrgwIv4JOAIYK2kIcBEwLSIGA9PS56SvjSJZqnUkcG0627VJDrBmlitqwVFMRKyMiDnp47eAZ4D+wMnAxDTbROCU9PHJwO0RURMRS4ElwGHFruEAa2b5U3oLtrekWQXHmMaKk7QncBDwOLBbRKyEJAgDu6bZ+gMvFpy2PE1rUovXgzUzq7QWjCJYHRHDi5YldSNZIfD8iHhTarLt2+I5AW7Bmln+lKcPFkkdSYLrrQXrq7wiqW/6el9gVZq+HBhYcPoAYEWx8h1gzSxfomyjCATcCDwTET8teGkycFb6+CzgroL0UZI6SRoEDCZZFKtJ7iIws/wpzzjYDwNnAPMkPZWmfQe4HJgk6YvAMuBzABGxQNIkYCHJCISxEVFX7AIOsGaWO+WYyRURM2h6sEGjE60iYgIwodRrOMCaWf7kZCaXA6yZ5U57WIvAzKztCdrFgttmZm1Ou9j00MyszXKANTPLhiIfEdYB1szypZ3saGBm1ia5D9bMLCPNTYNtKxxgzSx/3II1M8tAuIvAzCw7DrBmZuXniQZmZhlSfT4irAOsmeWLx8FapXTdqY4LfvIie+63gQj46TcH0rvvJs648GUGDq7hvJMGs3hul0pX04C6Ovj6yH3Zpe8mLvvNUib+aHcefaAHEvTsvYl/+9kydtm9lk0bxX99awCL53ZBVfC1S19i6FHrKl39isrLMK2KbBkj6buSFkiaK+kpSYdLOl/SVv3lS/qhpH8rdz3z6GuXvsSsB7vzpWP242sn7MuyxZ15/tnOXPqlPZn3WNdKV88K/PmGPgwcXLP5+WlfW8Uvpi3iur8u4vAT3uSWq3YH4L5bdwHgl39bxOW3P8f1l/SjPicBJjNl2pMra60eYCUdCXwKODgiPgScQLIV7vmAm1bboEu3Og48Yj3339YLgNpNVax/s5oXl3Rm+XOdK1w7K/Tqio7MnLYTJ56+ZnNa1+7vRs0N71TRsLnpsr934qCPJC3Wnr1r6dajjr8/vX3/qShKOyqtEi3YviRb6dYARMRq4DSgHzBd0nQASaMlzZM0X9IVDSdLGilpjqSnJU3bsnBJX5Z0n6QdJZ0naWHaUr69dd5e5ey+x0beWFPNhVe9yDVTFnH+T16k045FtwyyCvnFxf350vdWoC3+Am+6fHc+f8gQ/nbnzpz57ysB2Gv/DTz6QA/qauHlZTuweG4XXl3RsQK1biMCiCjtqLBKBNgpwEBJf5d0raSPRsTPSba/PS4ijpPUD7gCOB4YBhwq6RRJfYBfAZ+NiKGkm5E1kDQO+GfglIh4B7gIOChtKX+1scpIGiNplqRZm6hpLEtuVFcH+xz4Dnf/ZhfGfuKDbHi7in8Zt6r5E61VPTZ1J3r2rmXwh95532v/etHL3Dp7Icd/Zi2Tf90HgBGj1tC770bGjfwg1/2gP0OGr6e6uvLBo5LKsassgKRfS1olaX5B2g8lvZR2Xz4l6aSC18ZLWiJpkaQRzZXf6gE2ItYBhwBjgFeB30s6e4tshwIPRsSrEVEL3AocAxwBPBQRS9OyXis45wzgRJLg2xAp5wK3SvoCyS6QjdXn+ogYHhHDO9KpLO+xUlav7MirKzuy6Mmkr3XG3T3Y58D3/xFbZS18oiuPTdmJMw8bwn9+bQ+entGdK8Z94D15jjt1LTPu7QFAdQf46iUruO6vi7jk5qWse6Oa/nvluzGwLRrGwZapi+BmYGQj6VdFxLD0uBdA0hBgFLB/es61kqqLFV6Rm1wRURcRD0bExcA44LNbZGlqp0fRdNf1fGBPYEBB2ieBa0gC+mxJ7XrUxNpXO7J6xQ4M2HsDAMM+so5li9332tac852V3Dp7Ib+ZuZDx173A0KPf4tv/vYyX/rHD5jyPPdCDgfskQXTD22LD28mf6uz/6UZ1h2CPfbffAFty90AJXQQR8RDwWrMZEycDt0dETdrIWwIcVuyEVg84kj4I1EfE4jRpGPACSXDsDqwGHgf+S1JvYC0wGrgaeBS4RtKgiFgqqVdBK/ZJ4Dpgctp0fxkYGBHTJc0ATge6Aa9n/y4r55rv9efb/72MDh2Dl5ftwJUXDOSokW9w7v9/iR671HLZb5fy3ILOfPf0vStdVdvCjf/Rj+XPdaKqCnbtv5HzrlgOwOtrOvLd0XuhKthl90186+oXKlzTymuFG1jjJJ0JzAIujIi1QH/gsYI8y9O0JlWiRdcNuFpST5Kv7UtIugtGA/dJWpn2w44HppO0Wu+NiLsg6TMF7pRUBawCPt5QcETMSIdr3QN8ArhFUo+0jKsi4vVWeo8V848FO/L1E/d9T9r/3t+D/72/R4VqZMUMPWrd5jGtP7jh+Ubz7D5wIzfOeLYVa5UDpQfY3pJmFTy/PiKub+ac64DL0qtcBlwJnEPj36yL1qTVA2xEzAaOauSlq9OjId9twG2NnH8fcN8WaT8sePwA8ED69Ohtr7GZtTUtaMGujojhLSk7Il7ZfB3pV8Dd6dPlwMCCrANIbs43qSJ9sGZmWy2Auijt2AqS+hY8PZXk/g7AZGCUpE6SBgGDgZnFymrXN33MrH0qVx+spN8Bx5J0JSwHLgaOlTSMJJQ/D3wFICIWSJoELCTp3hwbEUUHmjvAmln+lGkSQUSMbiT5xiL5JwATSi3fAdbMcqctTIMthQOsmeVLG1nIpRQOsGaWKwK0lTewWpsDrJnljtrAQi6lcIA1s3xxF4GZWVbaxlKEpXCANbPc8SgCM7OsuAVrZpaB8CgCM7Ps5CO+OsCaWf54mJaZWVYcYM3MMhBACRsatgUOsGaWKyLcRWBmlpn6fDRhHWDNLF/cRWBmlh13EZiZZcUB1swsC17sxcwsGw27yuaAt+02s9xRRElHs+VIv5a0StL8grRekqZKWpz+u3PBa+MlLZG0SNKI5sp3gDWz/Iko7WjezcDILdIuAqZFxGBgWvocSUOAUcD+6TnXSqouVrgDrJnlSwD1UdrRXFERDwGvbZF8MjAxfTwROKUg/faIqImIpcAS4LBi5TvAmlnOlNh6TVqwvSXNKjjGlHCB3SJiJUD6765pen/gxYJ8y9O0Jvkml5nlT+mjCFZHxPAyXVWN1aTYCQ6wZpYvAdRlOpXrFUl9I2KlpL7AqjR9OTCwIN8AYEWxgtxFYGY5ExD1pR1bZzJwVvr4LOCugvRRkjpJGgQMBmYWK8gtWDPLnzJNNJD0O+BYkr7a5cDFwOXAJElfBJYBn0suGQskTQIWArXA2IioK1a+A6yZ5UvDKIJyFBUxuomXPtZE/gnAhFLLd4A1s/zxVFkzs4w4wJqZZSAC6op2fbYZDrBmlj9uwZqZZcQB1swsC6WtM9AWOMCaWb4ExNZPImhVDrBmlj/ZTpUtGwdYM8uXCG/bbWaWGd/kMjPLRrgFa2aWBe8qa2aWjTIu9pI1B1gzy5UAwlNlzcwyELEti2m3KgdYM8udcBeBmVlGctKCVeTkblxrkPQq8EKl65GB3sDqSlfCWqS9/s72iIg+21KApPtJfj6lWB0RI7fletvCAXY7IGlWGbcutlbg31n74F1lzcwy4gBrZpYRB9jtw/WVroC1mH9n7YD7YM3MMuIWrJlZRhxgzcwy4gDbhkkaIOkuSYslPSfpvyTtUCR/T0nnFjzvJ+mPrVNb25Kk70paIGmupKckHS7pfEldtrK8H0r6t3LX07LjANtGSRJwJ/DniBgM7At0AyYUOa0nsDnARsSKiDgty3pa4yQdCXwKODgiPgScALwInA9sVYC1/HGAbbuOBzZExE0AEVEHXACcI+nctGV7v6RFki5Oz7kc2DttLf1Y0p6S5gNI6izpJknzJD0p6bg0/WxJd6ZlLZb0ozS9WtLNkuan51zQ6j+BfOtLMouoBiAiVgOnAf2A6ZKmA0ganf5850u6ouFkSSMlzZH0tKRpWxYu6cuS7pO0o6TzJC1MW8q3t87bs1J4LYK2a39gdmFCRLwpaRnJ7+0w4ADgbeAJSfcAFwEHRMQwAEl7Fpw+Ni3jQEn7AVMk7Zu+Ngw4CKgBFkm6GtgV6B8RB6Rl9Sz/W2zXpgA/kPR34K/A7yPi55K+CRwXEasl9QOuAA4B1pL8Tk4BHgF+BRwTEUsl9SosWNI44BPAKRFRI+kiYFD6uGdrvUFrnluwbZdIlr5sKn1qRKyJiHdIuhKObqa8o4HfAkTEsyRrLjQE2GkR8UZEbAAWAnsA/wD2knS1pJHAm9v6hrYnEbGOJHCOAV4Ffi/p7C2yHQo8GBGvRkQtcCtwDHAE8FBELE3Leq3gnDOAE4HPNrSOgbnArZK+ANRm9JZsKzjAtl0LgPfMRZe0EzAQqOP9wbe5Ac0q8lpNweM6oENErAWGAg+StH5vaL7KVigi6iLiwYi4GBgHfHaLLE39Tpr6cAWYD+wJDChI+yRwDUlAny3J30zbCAfYtmsa0EXSmZD0iQJXAjeTdAt8XFIvSTsCp5B8rXwL6N5EeQ8Bn0/L2hf4ALCoqYtL6g1URcQdwPeBg7f9LW0/JH1Q0uCCpGEk3xoKf0ePAx+V1Dv9/Y4G/gd4NE0flJZV2EXwJPAVYHI6SqQKGBgR04Fvkdzo7JbZG7MW8SddGxURIelU4FpJ3yf5MLwX+A7JH+IMkq/8+wC3RcQsAEmPpDe27iNp1TS4FviFpHkkXyPPTvvsmqpCf+Cm9A8YYHxZ32D71w24Ou0TrQWWkHQXjAbuk7QyIo6TNB6YTtJqvTci7gKQNAa4M/35rwI+3lBwRMxIh2vdQ9IXe4ukHmkZV0XE6630Hq0ZniqbQ2lf3vCIGFfpuphZ09xFYGaWEbdgzcwy4hasmVlGHGDNzDLiAGtmlhEHWCuZpLp0nYP5kv6wtatCpWXdLOm09PENkoYUyXuspKO24hrPp+N5S0rfIs+6Fl7LK13Z+zjAWku8ExHD0vUJNgJfLXwxHSzfYhHxpYhYWCTLsUCLA6xZpTnA2tZ6GNgnbV1Ol3QbMC9dhevHkp5IV3f6CiTLL0r673TVp3tIFpMhfe1BScPTx+9ZRSpdsOarwAVp6/kjkvpIuiO9xhOSPpyeu4ukKUpWC/slxacHN1z7z5JmK1m3dcwWr12Z1mWapD5p2t5KVh6bLenhdOEcs0Z5Jpe1WDrX/UTg/jTpMJJVvJamQeqNiDhUUifgEUlTSFbr+iBwILAbyaIyv96i3D5ssYpURLwm6RfAuoj4SZrvNpIZSzMkfQB4APgn4GJgRkRcKumTJDOnmnNOeo0dSVYluyMi1gBdgTkRcaGkH6RljyPZjPCrEbFY0uEkM+SO34ofo20HHGCtJXaU9FT6+GHgRpKv7jMbVn4imbr5oYb+VaAHMJhklajfpevarpD0t0bKL7aKVKETgCEF03x3ktQ9vcZn0nPvkbS2hPd0XjolGZKFdAYDa4B64Pdp+i0k01a7pe/3DwXX7lTCNWw75QBrLfFOw1qzDdJAs74wCfh6RDywRb6TKG3Fr1JmvlQBR6ZLNW5Zl5Jnzkg6liRYHxkRb0t6EOjcRPZIr/v6lj8Ds6a4D9bK7QHga5I6QrJyl6SuJKt5jUr7aPsCxzVyblOrSG25StgUkq/rpPmGpQ8LVww7Edi5mbr2ANamwXU/khZ0gyqSHQgATifpengTWCrpc+k1JGloM9ew7ZgDrJXbDST9q3PSVb1+SfJN6U/AYmAecB3JsnzvERGvkvSb3inpad79iv4X4NSGm1zAecDw9CbaQt4dzXAJcIykOSRdFcuaqev9QAdJc4HLgMcKXlsP7C9pNkkf66Vp+ueBL6b1WwCcXMLPxLZTXovAzCwjbsGamWXEAdbMLCMOsGZmGXGANTPLiAOsmVlGHGDNzDLiAGtmlpH/A8m8rUGM7uLdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pipe = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('lr', LogisticRegression(random_state=42))\n",
    "])\n",
    "\n",
    "# Just trying to get a feel for the scores on train vs test data\n",
    "# pipe.fit(X_train,y_train)\n",
    "# pipe.score(X_train, y_train), pipe.score(X_test, y_test)\n",
    "params = { \n",
    "            'vect__stop_words':[None,'english'],\n",
    "            # 'lr__penalty':['l1', 'l2', 'elasticnet', 'none'],    #l2, Ridge, is default\n",
    "            'lr__max_iter': [1000],\n",
    "            'vect__ngram_range' : [(1,1),(1,2),(2,2)]\n",
    "}\n",
    "\n",
    "\n",
    "gs = GridSearchCV(    pipe,         # What is the model we want to fit?\n",
    "                      param_grid = params,  # What is the dictionary of hyperparameters?\n",
    "                      cv=5,           # What number of folds in CV will we use?\n",
    "                      verbose=1)\n",
    "\n",
    "gs.fit(X_train,y_train)\n",
    "print(gs.best_score_)   # <- Crossval score\n",
    "print(gs.best_params_)\n",
    "plot_confusion_matrix(gs, X_test, y_test, display_labels=['Options','Stocks']);\n",
    "\n",
    "# with help from https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html\n",
    "# and https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d01e637f-8910-49d6-b88e-f2e728bee421",
   "metadata": {},
   "source": [
    "**3.3 Random Forest and Count Vectorizer**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "b9d9c4ed-181c-4030-aaec-0ae8ed4be2ce",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 6 candidates, totalling 30 fits\n",
      "0.8368953811908737\n",
      "{'vect__ngram_range': (1, 2), 'vect__stop_words': None}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAEGCAYAAAAg6I3HAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAiQUlEQVR4nO3de7xVdZ3/8dcbREC5SaAhUIKCBl5QCVMbEy+J1oxa+gtq0kZLK8yxaZofOJV24fezizUjE5lZykwqUppa3kPIsfEGXrgZiqKIkFwVFLkdPvPH+h7c4Dn77CN7nX3W4f18PNaDvb97re/67nM4n/3dn7W+368iAjMzq752tW6AmVlb5QBrZpYTB1gzs5w4wJqZ5cQB1swsJ7vVugGtSee9Okb3PnvWuhnWDG++sketm2DNsOGtNWze9KZ2po5TRu4Zq1bXVbTvrNkb742IUTtzvp3hAFuie589+cyNJ9W6GdYMs8YdWesmWDPMenjiTtexcnUdj97br6J9O/R5vtdOn3AnOMCaWcEEdbG11o2oiAOsmRVKAFspxgApB1gzK5ytuAdrZlZ1QbDZKQIzs+oLoM4pAjOzfDgHa2aWgwDqCjILoAOsmRVOMTKwDrBmVjBBOAdrZpaHCNhcjPjqAGtmRSPq2KnpDFqMA6yZFUoAW92DNTPLh3uwZmY5yAYaOMCamVVdAJujGGsFOMCaWaEEoq4gi7E4wJpZ4WwNpwjMzKrOOVgzs9yIOudgzcyqL1vRwAHWzKzqIsSmaF/rZlSkGB8DZmYltqKKtkpIai/pSUl/SM97Srpf0nPp371K9h0vaaGkBZJOaapuB1gzK5TsIle7irYK/SPwTMnzccC0iBgETEvPkTQEGA0MBUYBkySV7Uo7wJpZwWQXuSrZmqxJ6gd8DLi2pPh0YHJ6PBk4o6R8SkRsjIhFwEJgRLn6nYM1s0Jp5kWuXpJmljy/JiKuKXn+b8C/AF1LyvaJiGUAEbFM0t6pvC/wSMl+S1JZoxxgzaxw6iofaLAyIoY39IKkjwPLI2KWpOMrqKuhk5ad18sB1swKJRCboyqh61jg7ySdBnQCukn6NfCqpD6p99oHWJ72XwL0Lzm+H7C03AmcgzWzQqnWRa6IGB8R/SJiP7KLVw9ExN8DdwDnpt3OBW5Pj+8ARkvqKGkAMAh4rNw53IM1s0IJ1JwUwbtxBTBV0vnAYuBsgIiYJ2kqMB/YAoyNiLpyFTnAmlnhVHskV0TMAGakx6uAExvZbwIwodJ6HWDNrFAi8FwEZmZ5yC5yFWOorAOsmRWOJ9w2M8tBIE+4bWaWF/dgzcxyEMBWX+QyM8uDvGSMmVkesmW7fReBmVnVRcgpAjOzvHiggZlZDrL5YJ2DNTPLgZftNjPLRXablnuwZmZV57kIzMxyVO3pCvPiAGtmhZJNV+gUgZlZLpyDNTPLQTabVjFSBMVopZlZkg2VbVfRVo6kTpIek/S0pHmSvp3KL5f0iqSn0nZayTHjJS2UtEDSKU211T3Ygtu6MXj+8xCbIOqg+4nw3i9lX59WTglW3gxqD90+DH0uEevnBku+lw4O2OdC6H5CMb5utRVfP+9BPjTsZV5b24nzv/FJAP7hE7M45vCXiBCvre3E9689jlWv7cmJRy/kU6fO2XbswH6rufDyM3h+8Xtq1fxWoGo92I3ACRHxhqQOwEOS7k6v/SQifrTdWaUhZKvPDgX2Bf4oaXC5hQ9zDbCS+gE/BYaQ9Zb/AHw9IjY1sn8P4NMRMSk93xe4KiLOyrOdRabdYeDPof0eIjYHC8+HrscGsRHWzoDBN0O73cWW1QFAp/1h0K9Bu4nNK4JnR0O34wLt5iDbUu59aBC3TRvCuC/8aVvZzXcdwnW3HgnAmSfN47OnP8W/TT6WaQ8fwLSHDwBgQL/VfPfiP+7iwTVTjZFcERHAG+lph7RFmUNOB6ZExEZgkaSFwAjg4cYOyC1FIEnArcBtETEIGAx0ofyKjD2AL9c/iYilDq7lSaL9Htl/ttiSbRKs+i30/ocsuALs1jP7t11nbQumsSnb11rW7Gf7sPbNjtuVrd+w+7bHnTpuafDP/ISjXuCBRwfm3bxWr/4ugkq2pkhqL+kpYDlwf0Q8ml66SNJsSb+StFcq6wu8XHL4klTWqDxzsCcAGyLiOoDUjf4qcJ6kL0u6XdI9KZdxWTrmCmD/lPf4oaT9JM2FbfmS6yTNkfSkpJGp/HOSbk11PSfpB6m8vaTrJc1Nx3w1x/daU1EXPDs6mH8SdD0K9jhEbHwJ3nwCnjsneP7zwfp5b//Frp8TLDgrePb/QN9Lce+1lTjvkzOZcuUUTjp6Idf97oh3vD7yqBd44BEHWMgm3K5kA3pJmlmyXVBaT0TURcQwoB8wQtLBwM+A/YFhwDLgyrR7Q38o5Xq8uaYIhgKztmtJxFpJi9N5RwAHA+uBxyXdCYwDDk5vGEn7lRw+NtVxiKSDgPskDU6vDQMOJ8upLJA0Edgb6BsRB6e6ejTUyPQDvwCga589duoN14rai8FToG5d8OLXYMPCIOqgbh0cMBnemgcv/V846PeBJPY4RBz4W9jwQvDyZVlKoV1HB9la+9Utw/nVLcMZ87GnOePEZ5h829tB9qCBy9mwcTdefKVnDVvYOjRzTa6VETG8yTojXpM0AxhVmnuV9Auy1CZkPdb+JYf1A5aWqzfPHqxoOLrXl98fEasi4i2yVMKHm6jvw8B/AUTEX4CXyNIOANMi4vWI2ADMB94PvAAMlDRR0ihgbUOVRsQ1ETE8Iobv0aNjQ7sURvuuosuRsO5/oMPe0P2ELIWwx8FC7aDute337zRQtOsMG56vSXOtEQ88MpDjhi/arszpgbcFsCXaVbSVI6l3fcdLUmfgJOAvkvqU7HYmMDc9vgMYLamjpAHAIOCxcufIM8DOA7b75JDUjewToI53Bt+yXW0a7p7X21jyuA7YLSLWAIcBM8h6v9c23eTi2bImqFuX/ei2bgjWPQod94PuI+GNx7N9Nr4UxGZo3wM2vRLElmz/TUuDjS/C7n0arNpaUN99Xt/2+JjDF7N4WY9tz6XgIx9cxHQH2G2akSIopw8wXdJs4HGyTt8fgB+ktOJsYCRZapOImAdMJevE3QOMLXcHAeSbIpgGXCHpnIj4T0ntyXIZ15OlBU6W1BN4CzgDOA9YB3RtpL4Hgc8AD6TUwPuABcA7k1WApF7Apoi4RdLz6bxtzuYV8PJlQF0QAT1Ohm7Hia2bgyWXw4KzA3WA/t/OerNvPhksvx60W0A76DsedtvL6YGW9I0vTuewg5bRvcsGbv7xTVx/2xEcdegS+r/3NbaGWL6qCz+5/tht+x964F9ZsWZPlq3oVsNWtyJRnWW7I2I2WWpxx/LPljlmAuUv1G8ntwAbESHpTGCSpG+S9ZbvAi4FxgAPkX3lPwC4MSJmAkj6c7qwdTfZLV71JgFXS5oDbAE+FxEb1fhl8L7AdZLqP8bGV/UNthKdB4vBN72zvF0H8b4G/hvs9XGx18fzb5c17ntXj3xH2d0PHtjo/k//pQ8Xfffv8mxSoXjC7SQiXgb+dsfyFBSXR8RFDRzz6R2KDk7lG4DPNbD/9ZT0TiOiNHw02Ls1s2LzXARmZjnwhNtN2LHXaWZWqUBs2VqMaVTcgzWzwnEO1swsD+EUgZlZLpyDNTPLkQOsmVkOAlHni1xmZvnwRS4zsxyEL3KZmeUnHGDNzPJQncleWoIDrJkVjnuwZmY5iIC6rQ6wZma58F0EZmY5CJwiMDPLiS9ymZnlJppawa+VKMZ4MzOzEhGqaCtHUidJj0l6WtI8Sd9O5T0l3S/pufTvXiXHjJe0UNICSac01U4HWDMrlOwugnYVbU3YCJwQEYcBw4BRkj4EjAOmRcQgssVbxwFIGgKMBoYCo8jWG2xf7gQOsGZWOBGVbeXriIiIN9LTDmkL4HRgciqfTLbqNal8SkRsjIhFwEJgRLlzOMCaWeE0I0XQS9LMku2C0noktZf0FLAcuD8iHgX2iYhl2XliGbB32r0v8HLJ4UtSWaN8kcvMCiVoOr9aYmVEDG+0rog6YJikHsDvJB1cpq6GTlq2n+werJkVTlS4VVxfxGvADLLc6quS+gCkf5en3ZYA/UsO6wcsLVevA6yZFUtAbFVFWzmSeqeeK5I6AycBfwHuAM5Nu50L3J4e3wGMltRR0gBgEPBYuXM4RWBmhVOlkVx9gMnpToB2wNSI+IOkh4Gpks4HFgNnZ+eMeZKmAvOBLcDYlGJolAOsmRVONQYaRMRs4PAGylcBJzZyzARgQqXnaDTASppImTRGRFxc6UnMzKqlrcxFMLPFWmFmVqkAih5gI2Jy6XNJe0bEm/k3ycysvDYzF4GkoyXNB55Jzw+TNCn3lpmZNaiyOwiauougJVRym9a/AacAqwAi4mnguBzbZGZWXrVvhM1JRXcRRMTL0nafBmVvTTAzy020jYtc9V6WdAwQknYHLialC8zMaqIV9E4rUUmK4IvAWLJJDV4hm9ZrbI5tMjNrgircaqvJHmxErAQ+0wJtMTOrzNZaN6AyldxFMFDS7yWtkLRc0u2SBrZE48zM3qH+PthKthqrJEVwIzCVbNzuvsBvgJvybJSZWTnVmHC7JVQSYBUR/xURW9L2awqTYjazNqnot2lJ6pkeTpc0DphC1uRPAXe2QNvMzBrWCr7+V6LcRa5ZZAG1/p1cWPJaAN/Nq1FmZuWoFfROK1FuLoIBLdkQM7OKhKAVDIOtREUjudI6NUOATvVlEfGfeTXKzKysovdg60m6DDieLMDeBZwKPAQ4wJpZbRQkwFZyF8FZZLN7/zUi/gE4DOiYa6vMzMop+l0EJd6KiK2StkjqRrbCogcamFltFGjC7Up6sDPTyou/ILuz4AmaWEnRzCxPisq2snVI/SVNl/SMpHmS/jGVXy7pFUlPpe20kmPGS1ooaYGkU5pqZyVzEXw5Pbxa0j1At7RYmJlZbVTn6/8W4GsR8YSkrsAsSfen134SET8q3VnSEGA0MJRsVOsfJQ0ut7JsuYEGR5R7LSKeaMYbMTOrmmrcBxsRy4Bl6fE6Sc+QzRrYmNOBKRGxEVgkaSEwAni4sQPK9WCvLNc24IQyrxfS+mdg9hGtIDNuFXtg6bW1boI1w4hTVlanospzsL0klS7gek1EXLPjTpL2I1vC+1HgWOAiSeeQLf76tYhYQxZ8Hyk5bAnlA3LZgQYjK30HZmYtpnl3CKyMiOHldpDUBbgFuCQi1kr6GdlI1foRq1cC59HwBLNlW1LJRS4zs9alSrdpSepAFlxviIhbASLi1Yioi4itZBf3R6TdlwD9Sw7vBywtV78DrJkVjrZWtpWtI1to8JfAMxHx45LyPiW7nQnMTY/vAEZL6ihpADCIJu6oqmiorJlZq1KdSyXHAp8F5kh6KpVdCoyRNCyd5UXSRFcRMU/SVGA+2R0IY8vdQQCVDZUV2ZIxAyPiO5LeB7w3InwvrJm1uEruca1ERDxEw3nVu8ocMwGYUOk5KkkRTAKOBsak5+uAn1Z6AjOzqivIkjGVpAiOiogjJD0JEBFr0vLdZma1UZC7KSsJsJsltSe9JUm9KcyajmbWFhV+wu0SVwG/A/aWNIFsdq1v5NoqM7PGRNN3CLQWlcxFcIOkWWRTFgo4IyKeyb1lZmaNaSs92HTXwHrg96VlEbE4z4aZmTWqrQRYshVk6xc/7AQMABaQzShjZtbi2kwONiIOKX2eZtm6sJHdzcwsafZIrjR34gfzaIyZWUXaSg9W0j+VPG0HHAGsyK1FZmbltKW7CICuJY+3kOVkb8mnOWZmFWgLPdg0wKBLRHy9hdpjZlaWaAMXuSTtFhFbyi0dY2ZWE0UPsGTzHB4BPCXpDuA3wJv1L9ZPTmtm1qKqNJtWS6gkB9sTWEW2Blf9/bABOMCaWW20gYtce6c7CObydmCtV5DPDzNri9pCD7Y90IV3sdCXmVmuChKBygXYZRHxnRZriZlZJZq3qmxNlQuwtZ8O3MysAUVJEZRbMubEFmuFmVlzVGHZbkn9JU2X9IykeZL+MZX3lHS/pOfSv3uVHDNe0kJJCySd0lQzGw2wEbG6yTdpZlYD1Vi2m2xk6tci4gPAh4CxkoYA44BpETEImJaek14bTTaT4ChgUhqM1ahKFj00M2s9Ku29NtGDjYhlEfFEerwOeAboC5wOTE67TQbOSI9PB6ZExMaIWAQsBEaUO4cDrJkVipqxAb0kzSzZLmiwTmk/4HDgUWCfiFgGWRAG9k679QVeLjlsSSprVLOnKzQzq7nKL3KtjIjh5XaQ1IVsAqtLImKt1Oj1/WbfsuoerJkVjqKyrcl6pA5kwfWGkuH/r0rqk17vAyxP5UuA/iWH9wOWlqvfAdbMiqc6dxEI+CXwTET8uOSlO4Bz0+NzgdtLykdL6ihpADCIbM6WRjlFYGbFUr0Jt48FPgvMkfRUKrsUuAKYKul8YDFwNkBEzJM0FZhPdgfC2IioK3cCB1gzK54qDDSIiIdofEBVg+MAImICMKHSczjAmlnhFGUklwOsmRWPA6yZWT7cgzUzy0PQJibcNjNrddrEoodmZq2WA6yZWT4UxYiwDrBmVixtZEUDM7NWyTlYM7OcVGmobO4cYM2seNyDNTPLQYVTEbYGDrBmVjwOsGZm1eeBBmZmOdLWYkRYB1gzK5YC3QfrJWPamMmPzufqaQuYdP8CJt797HavnfXF5dy79Gm69dxSo9ZZqbo6+PLJg/nmOQO2K//Nz3pzyr7DeH1V+21lL8zvxCV/O4gvHH8gF55wIJs2NLow3y5BWyvbaq0mPVhJ/wp8GqgjmxfnQuBo4JqIWP8u6rsceCMiflTNdhbVv5y9P2tXb/+r7b3vJg4/bh2vLulQo1bZjm67tjf9B21k/Rtv93OWv9KBJx/syt59N20rq9sCP/jK+/n6VS+x/9ANrF3dnvYdCtKFy0tB3n6L92AlHQ18HDgiIg4FTiJba/wSYI+Wbs+u4sLLl/LL7+1LQYZwt3krlnbgsWndOPXTq7Yr//nlfTn/G0spXTl61p+6MuADb7H/0A0AdOtZR/v27NKquKrsryQtlzS3pOxySa9Ieiptp5W8Nl7SQkkLJJ3SVP21SBH0IVurfCNARKwEzgL2BaZLmg4gaYykOZLmSvp+/cGSRkl6QtLTkqbtWLmkL0i6W1JnSRdLmi9ptqQpLfP2aizE/7vpBf7jnmc59TPZH++HPvo6K//agRfmd65x46ze1Zf15fPfWIpK/gIfvrcbvd67eVsgrbfkhU5IcOmYgYz96GCm/nTvFm5tKxNARGVb064HRjVQ/pOIGJa2uwAkDQFGA0PTMZMklf2oq0WK4D7gW5KeBf4I3BwRV0n6J2BkRKyUtC/wfeBIYA1wn6QzgD8DvwCOi4hFknqWVizpIuCjwBkRsVHSOGBAetyjocZIugC4AKBTG+hAf/X0A1j9age6v2czV0x5gZcXdmTMxcsZP2ZgrZtmySP3d6NHry0MOvQtnv6fLgBsWC9uumof/v9Nz79j/7otMPexPZl417N07LyVcZ86gEGHrufwv3mjpZvealQrvxoRD0rar8LdTwempM7hIkkLgRHAw40d0OIBNiLekHQk8DfASODmFAhLfRCYERErACTdABxHlrN9MCIWpbpWlxzzWWAJWXDdnMpmAzdIug24rZH2XANcA9BNPQv/BXr1q1mO9fVVHfjzPd059Og3ee/7NvGzPy4AoHefzfz03me5+LRBrFnhfGwtzH98Tx65rxuPTxvCpo1i/br2/ODi9/PXxbvzpZMOAmDFsg6MPeVArrrrWXr32cyhR79J9/dkK0R/8IS1LJzTeZcNsC10H+xFks4BZgJfi4g1QF/gkZJ9lqSyRtXkIldaS3wGMEPSHODcHXZp7BKpaDy9PRcYBvQDFqWyj5EF5r8DvilpaES02UvoHTvX0a4dvPVmezp2ruPIj6zjhh/vw6cOHbptn8mPzucrpw5+x0UwaznnXbqM8y5dBsDT/9OF317dm29d++J2+5wzYggT715A9/fUceTx6/jNpL3ZsF502D2Y/XAXPnHBihq0vJWo/Os/QC9JM0ueX5M6VeX8DPguWaz5LnAlcB4Nx6WyDWnxvzJJBwJbI+K5VDQMeAnYD+gKrAQeBf5dUi+yFMEYYCJZV/ynkgbUpwhKerFPkv1g7kjJ578C/SNiuqSHyO5a6AK8lv+7rI29em/hsl++CED73YLpv9uLmTO61bZRttO69qjjExeu4CunDUaCESes5aiT1ta6WTXVjB7syogY3py6I+LVbeeRfgH8IT1dAvQv2bUfsLRcXbXoxnQBJqac6BZgIVkOdAxwt6RlETFS0nhgOtmnxl0RcTtsy5neKqkdsBw4ub7iiHhI0j8Dd5LlYn8tqXuq4ycR8VoLvcea+Ovijnzp5APL7nPuUUNaqDVWicOOeYPDjnnnV/3/fGz+ds9P/OQaTvzkmpZqVuuXY4pAUp+IWJaenkn27RjgDuBGST8muyg/CHisXF21yMHOAo5p4KWJaavf70bgxgaOvxu4e4eyy0se3wvcm55+eOdbbGatTbVysJJuAo4nSyUsAS4Djpc0jCyMv0h2nz4RMU/SVGA+WedwbEp3NsqJODMrlgDqqhNhI2JMA8W/LLP/BGBCpfU7wJpZ4Xg2LTOzvBRkSKIDrJkVjnuwZmZ5KNB0hQ6wZlYoAlSli1x5c4A1s8KRc7BmZjlwisDMLC/NmougphxgzaxwfBeBmVle3IM1M8tB+C4CM7P8FCO+OsCaWfH4Ni0zs7w4wJqZ5SCAKi16mDcHWDMrFBFOEZiZ5WZrMbqwDrBmVixOEZiZ5acoKYJ2tW6AmVmzRVS2NUHSryQtlzS3pKynpPslPZf+3avktfGSFkpaIOmUpup3gDWzgqkwuFbWy70eGLVD2ThgWkQMAqal50gaAowGhqZjJklqX65yB1gzK5b6VWUr2ZqqKuJBYPUOxacDk9PjycAZJeVTImJjRCwCFgIjytXvHKyZFU4zcrC9JM0seX5NRFzTxDH7RMQygIhYJmnvVN4XeKRkvyWprFEOsGZWPJUH2JURMbxKZ1VDLSl3gFMEZlYsAWyNyrZ351VJfQDSv8tT+RKgf8l+/YCl5SpygDWzgqnqRa6G3AGcmx6fC9xeUj5aUkdJA4BBwGPlKnKKwMyKp0r3wUq6CTieLFe7BLgMuAKYKul8YDFwdnbKmCdpKjAf2AKMjYi6cvU7wJpZsQRQV52hXBExppGXTmxk/wnAhErrd4A1s4IJiGKMlXWANbPiKchQWQdYMyuW+rsICsAB1syKxz1YM7OcOMCameUgAurK3h3VajjAmlnxuAdrZpYTB1gzszzs1DwDLcoB1syKJSA80MDMLCdVGiqbNwdYMyuWCC/bbWaWG1/kMjPLR7gHa2aWh52aTLtFOcCaWbF4shczs3wEEB4qa2aWg/CE22ZmuQmnCMzMclKQHqyiIFfjWoKkFcBLtW5HDnoBK2vdCGuWtvo7e39E9N6ZCiTdQ/bzqcTKiBi1M+fbGQ6wuwBJMyNieK3bYZXz76xtaFfrBpiZtVUOsGZmOXGA3TVcU+sGWLP5d9YGOAdrZpYT92DNzHLiAGtmlhMH2FZMUj9Jt0t6TtLzkv5d0u5l9u8h6cslz/eV9NuWaa3tSNK/SponabakpyQdJekSSXu8y/oul/TP1W6n5ccBtpWSJOBW4LaIGAQMBroAE8oc1gPYFmAjYmlEnJVnO61hko4GPg4cERGHAicBLwOXAO8qwFrxOMC2XicAGyLiOoCIqAO+Cpwn6cupZ3uPpAWSLkvHXAHsn3pLP5S0n6S5AJI6SbpO0hxJT0oamco/J+nWVNdzkn6QyttLul7S3HTMV1v8J1BsfchGEW0EiIiVwFnAvsB0SdMBJI1JP9+5kr5ff7CkUZKekPS0pGk7Vi7pC5LultRZ0sWS5qee8pSWeXtWCc9F0HoNBWaVFkTEWkmLyX5vI4CDgfXA45LuBMYBB0fEMABJ+5UcPjbVcYikg4D7JA1Orw0DDgc2AgskTQT2BvpGxMGprh7Vf4tt2n3AtyQ9C/wRuDkirpL0T8DIiFgpaV/g+8CRwBqy38kZwJ+BXwDHRcQiST1LK5Z0EfBR4IyI2ChpHDAgPe7RUm/QmuYebOslsqkvGyu/PyJWRcRbZKmEDzdR34eB/wKIiL+QzblQH2CnRcTrEbEBmA+8H3gBGChpoqRRwNqdfUO7koh4gyxwXgCsAG6W9LkddvsgMCMiVkTEFuAG4DjgQ8CDEbEo1bW65JjPAqcCn6zvHQOzgRsk/T2wJae3ZO+CA2zrNQ/Ybiy6pG5Af6COdwbfpm5oVpnXNpY8rgN2i4g1wGHADLLe77VNN9lKRURdRMyIiMuAi4BP7rBLY7+Txj5cAeYC+wH9Sso+BvyULKDPkuRvpq2EA2zrNQ3YQ9I5kOVEgSuB68nSAidL6impM3AG2dfKdUDXRup7EPhMqmsw8D5gQWMnl9QLaBcRtwDfBI7Y+be065B0oKRBJUXDyL41lP6OHgU+IqlX+v2OAf4EPJzKB6S6SlMETwIXAneku0TaAf0jYjrwL2QXOrvk9sasWfxJ10pFREg6E5gk6ZtkH4Z3AZeS/SE+RPaV/wDgxoiYCSDpz+nC1t1kvZp6k4CrJc0h+xr5uZSza6wJfYHr0h8wwPiqvsG2rwswMeVEtwALydIFY4C7JS2LiJGSxgPTyXqtd0XE7QCSLgBuTT//5cDJ9RVHxEPpdq07yXKxv5bUPdXxk4h4rYXeozXBQ2ULKOXyhkfERbVui5k1zikCM7OcuAdrZpYT92DNzHLiAGtmlhMHWDOznDjAWsUk1aV5DuZK+s27nRUq1XW9pLPS42slDSmz7/GSjnkX53gx3c9bUfkO+7zRzHN5pit7BwdYa463ImJYmp9gE/DF0hfTzfLNFhGfj4j5ZXY5Hmh2gDWrNQdYe7f+Gzgg9S6nS7oRmJNm4fqhpMfT7E4XQjb9oqT/SLM+3Uk2mQzptRmShqfH280ilSas+SLw1dR7/htJvSXdks7xuKRj07HvkXSfstnCfk754cH1575N0ixl87ZesMNrV6a2TJPUO5Xtr2zmsVmS/jtNnGPWII/ksmZLY91PBe5JRSPIZvFalILU6xHxQUkdgT9Luo9stq4DgUOAfcgmlfnVDvX2ZodZpCJitaSrgTci4kdpvxvJRiw9JOl9wL3AB4DLgIci4juSPkY2cqop56VzdCableyWiFgF7Ak8ERFfk/StVPdFZIsRfjEinpN0FNkIuRPexY/RdgEOsNYcnSU9lR7/N/BLsq/uj9XP/EQ2dPPQ+vwq0B0YRDZL1E1pXtulkh5ooP5ys0iVOgkYUjLMt5ukrukcn0jH3ilpTQXv6eI0JBmyiXQGAauArcDNqfzXZMNWu6T3+5uSc3es4By2i3KAteZ4q36u2Xop0LxZWgR8JSLu3WG/06hsxq9KRr60A45OUzXu2JaKR85IOp4sWB8dEeslzQA6NbJ7pPO+tuPPwKwxzsFatd0LfElSB8hm7pK0J9lsXqNTjrYPMLKBYxubRWrHWcLuI/u6TtpvWHpYOmPYqcBeTbS1O7AmBdeDyHrQ9dqRrUAA8Gmy1MNaYJGks9M5JOmwJs5huzAHWKu2a8nyq0+kWb1+TvZN6XfAc8Ac4Gdk0/JtJyJWkOVNb5X0NG9/Rf89cGb9RS7gYmB4uog2n7fvZvg2cJykJ8hSFYubaOs9wG6SZgPfBR4pee1NYKikWWQ51u+k8s8A56f2zQNOr+BnYrsoz0VgZpYT92DNzHLiAGtmlhMHWDOznDjAmpnlxAHWzCwnDrBmZjlxgDUzy8n/Agz/RLuxiTpoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pipe2 = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('rf', RandomForestClassifier(random_state=42))\n",
    "])\n",
    "params2 = { \n",
    "            'vect__stop_words':[None,'english'],\n",
    "            'vect__ngram_range' : [(1,1),(1,2),(2,2)]\n",
    "}\n",
    "gs2 = GridSearchCV(    pipe2,         # What is the model we want to fit?\n",
    "                      param_grid = params2,  # What is the dictionary of hyperparameters?\n",
    "                      cv=5,           # What number of folds in CV will we use?\n",
    "                      verbose=1)\n",
    "\n",
    "gs2.fit(X_train,y_train)\n",
    "print(gs2.best_score_)   # <- Crossval score\n",
    "print(gs2.best_params_)\n",
    "plot_confusion_matrix(gs2, X_test, y_test, display_labels=['Options','Stocks']);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5b7c438-b321-4599-aff3-cf1539e0f85c",
   "metadata": {},
   "source": [
    "**3.4 K Neighbors and Count Vectorizer**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "id": "dec293e4-8fc0-4c90-b7b8-b74185fa3247",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 6 candidates, totalling 30 fits\n",
      "0.7374785754034503\n",
      "{'vect__ngram_range': (1, 1), 'vect__stop_words': 'english'}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAEGCAYAAAAg6I3HAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhS0lEQVR4nO3de5gU1Z3/8fdnhuGOIiLITUFFjKCiIUZW44qXSC77qFk3wWRNsmajbmTdGJP9aRJj4i67uWhc43qJuaibaJREE413JbiR/IwXEBFEIgQVhMgdRHCY6f7uH1WjDc709MjU9NTweT1PPXZXV506PY3fPn3qnO9RRGBmZu2vptoVMDPrqhxgzcwy4gBrZpYRB1gzs4w4wJqZZaRbtSvQmQwcUBsjR9RVuxrWBn+a17vaVbA2eJM32Bb12pkyTp7UJ9auK1R07Ox59Q9GxOSdud7OcIAtMXJEHU8+OKLa1bA2OHno+GpXwdrgiZix02WsWVfgiQeHV3Rs3ZAlA3f6gjvBAdbMciYoRLHalaiIA6yZ5UoARfIxQcoB1sxyp4hbsGZm7S4IGtxFYGbW/gIouIvAzCwb7oM1M8tAAIWcZAF0gDWz3MlHD6wDrJnlTBDugzUzy0IENOQjvjrAmlneiAI7lc6gwzjAmlmuBFB0C9bMLBtuwZqZZSCZaOAAa2bW7gJoiHysFeAAa2a5EohCThZjcYA1s9wphrsIzMzanftgzcwyIwrugzUza3/JigYOsGZm7S5CbIvaalejIg6wZpY7RffBmpm1v+Qml7sIzMwy4JtcZmaZ8E0uM7MMFTzRwMys/QWiIfIRuvLRzjYzSzXd5KpkK0dST0lPSnpW0gJJ30r3f1PSq5LmptuHS865WNJiSYskndxaXfPxNWBmlgrUXl0E9cDxEbFZUh0wS9L96WtXRsTlpQdLOhiYAowFhgKPSDowIgotXcAtWDPLnSI1FW3lRGJz+rQu3cqtlXAKcFtE1EfEUmAxcGS5azjAmlmuREAhairaWiOpVtJcYBXwcEQ8kb40VdI8ST+VtEe6bxiwrOT05em+FjnAmlmuJDe5aivagIGSni7Zzt6urIhCRIwHhgNHShoHXAfsD4wHVgJXpIc31y9RdnUw98GaWe60YSbXmoiY0NpBEbFB0qPA5NK+V0k/Au5Jny4HRpScNhxYUa5ct2DNLFcCUYzKtnIk7SWpf/q4F3Ai8IKkISWHnQbMTx/fDUyR1EPSKGA08GS5a7gFa2a50065CIYAN0uqJWlsTo+IeyT9TNJ4kp//LwHnAETEAknTgeeBRuC8ciMIwAHWzHImgGI75CKIiHnA4c3sP7PMOdOAaZVewwHWzHJGXjLGzCwLybLdTrhtZtbuItQuXQQdwQHWzHLH+WDNzDKQ5IN1H6yZWQa8ooGZWSaSYVpuwZqZtbumXAR54ABrZrnjNbnMzDKQpCt0F4GZWSbcB2tmloEkm5a7CMzM2l0yVdYB1jrAtjfFhR87gIZtNRQa4QMf2cinv/IXlizoydUXjWDrGzUMHr6N/3fNy/TpV2T2//blp/8xlMYG0a0u+PwlKxh/zObWL2Tt5kvff4X3n/g6G9Z045zjxwDQr38jX73+ZQYP38Zry7sz7Zx92byxG7XdggsuX8YBh2yltlvwyC/34Pb/Hlzld1Bt+WnBZlpLScMl3SXpRUlLJF0lqXuZ4/tL+kLJ86GSfpVlHfOurkfw3V8u4fpHFnHdw4t4+tF+LJzdm//68j6c9dUV/PB3izj6Qxv51XWDANh9QIHLbv4zP/zdIr5y1St89/x9qvwOdj0P3T6Ar31q1Hb7Pj51Fc/M6stZx7yHZ2b15RNTVwFw7N9soK5HcO4JY5g6+UA+fOZaBg/fVo1qdypFVNFWbZkFWEkC7gR+ExGjgQOBvpTPpdgfeCvARsSKiDg9qzp2BRL06lMEoLFBFBqEBMuX9OCQo94A4PBjX2fWvf0BOOCQrey5dyMA+455k231NWyrr/4/xF3J/Cf68vr67X88Tjx5E49MHwDAI9MHMHHyJiC5Y96zd5Ga2qB7zyKN28SWzflovWWlaRRBJVu1ZflJHQ+8GRE3QrK4GHABcJakL6Qt2wckLZJ0aXrOt4H9Jc2V9D1JIyXNB5DUU9KNkp6T9IykSen+z0q6My3rRUnfTffXSrpJ0vz0nAsyfK9VVSjAP504hk8cOo7Dj32dg47Ywr5j3uTxB3cD4LF7+rN6Rd07zpt17+7sP3Yr3XuUXbfNOsAeAxtYtyr5jNatqqP/nsmX4GP39OfNLTX8Yu4Cfv7UQn51/SBe3+CevWLUVLRVW5af1FhgdumOiNgk6ZX0ukcC44AtwFOS7gUuAsalqzwiaWTJ6eelZRwi6SDgIUkHpq+NJ8lMXg8sknQ1MAgYFhHj0rL6N1fJdJXJswH2GZbPf7i1tXDdI4vYvLGWb31uJC+90JMvff8VrrtkGLdcuTcTP7iRbt23D6IvLerJT6YN5T9+saRKtbZKjDl8C8UCfPLwsfTdvZErfrOEZx7ry19e6VHtqlVN05pceZBliBfNL2nbtP/hiFgbEVtJuhKOaaW8Y4CfAUTEC8DLJN0OADMiYmNEvEmyXs6+wJ+B/SRdLWkysKm5QiPihoiYEBET9tozH9PvWtJ39wKHTdzMUzP7sc/oev7ztj9zzYN/4rhTNzBk3/q3jlu9oo7LPjeSr1z1CkNHuj+vM1i/po4BgxoAGDCogQ1rky/7Saet5+mZ/Sg0io1r63j+qd4ceNjWala16gJojJqKtmrLsgYLgO2Wy5W0G8mytwXeGXxb+51a7iurvuRxAegWEeuBw4BHSVq/P269yvmzYW0tmzcmXwz1W8Wcx/ox4oB6NqxJ/gctFuHWqwbz0TPXArB5Yy2XfHo//uHilYw98o2q1du298eHduPEj68D4MSPr3ure2f1q93TUR5Bj14FDjpiC8sW77qt1ybuIoAZwLclfToi/iddufEK4CaSboGTJA0AtgKnAmcBrwP9Wijv98CngN+lXQP7AIuAI5o7WNJAYFtE3CFpSXrdLmfda3Vc/i/7UCyKYjG563zUSZv49Y8H8tubBgJw9Ic28sEpyf+8d984kBVLu3PrlXtz65V7A/Cfty2h/8DGqr2HXc1F177MoRM3s/uARn7+9PP87IrB3P7fg/ja9S8zeco6Vr2aDNMCuPvGPbnwymXcMHMRKBmBsHRhryq/gyqrYEnuzkIR2d3gkDQCuBY4iKS1fB/wZeAM4MNAH+AA4NaI+FZ6zq3AocD9wDXAPRExTlJP4HrgvSRL5n4pImZK+iwwISKmpuffA1wOrAdu5O1W+sURcX+5+k44rGc8+eCIdnr31hFOHjq+2lWwNngiZrAp1u1UdNzjoEFx/E8rG1x059HXzY6ICa0fmY1M7+pExDLgb3bcn4zgYlVTUNzhnE/usGtcuv9N4LPNHH8TJa3TiPhoycvNtm7NLN/y0oLN521zM9tlOeF2K3ZsdZqZVSoQjcXq38CqhFuwZpY7nWEabCUcYM0sX8JdBGZmmXAfrJlZhhxgzcwyEIhCTm5y5aOWZmYl2iMfbJqh70lJz0paIKlpstMASQ+n2fkelrRHyTkXS1qcZgE8ubV6OsCaWa5EepOrkq0V9cDxEXEYSUa+yZKOIsnqNyPNYz0jfY6kg4EpJJkCJwPXpikAWuQAa2a5E6GKtvJlRERE03pJdekWwCnAzen+m0lypZDuvy0i6iNiKbCYJO1qixxgzSxnKmu9pi3YgZKeLtnO3q6kJDH/XGAVSQrVJ4DBEbESIP3voPTwYcCyktOXp/ta5JtcZpY7rbVOS6wpl+wlXWllfJqQ/9eSxpUpq7mLls2W5QBrZrkSAYVi+w7TiogNkh4l6Vt9TdKQiFgpaQhJ6xaSFmtpur3hwIpy5bqLwMxyp51GEezVtJSUpF7AicALwN3AZ9LDPgPclT6+G5giqYekUcBo4Mly13AL1sxyJWhTF0E5Q4Cb05EANcD0iLhH0uPAdEmfA14B/g4gIhZImk6yLFUjcF7axdAiB1gzy5n2WdEgIuaRLJa64/61wAktnDMNmFbpNRxgzSx3MlyIpV05wJpZ7rRTF0HmHGDNLFeSUQT5uD/vAGtmueMuAjOzjLiLwMwsA0HreQY6CwdYM8udnPQQOMCaWc4ERDtPlc2KA6yZ5Y67CMzMMpL7UQSSrqZMV0dEnJ9JjczMymjHXASZK9eCfbrDamFmVqkA8h5gI+Lm0ueS+kTEG9lXycysvLx0EbQ630zSREnPAwvT54dJujbzmpmZNUtEsbKt2iqZ0PtfwMnAWoCIeBY4NsM6mZmVFxVuVVbRKIKIWCZt921QNsmsmVlmomvc5GqyTNJfASGpO3A+aXeBmVlVdILWaSUq6SI4FziPZHnaV4Hx6XMzsypRhVt1tdqCjYg1wKc6oC5mZpUpVrsClalkFMF+kn4rabWkVZLukrRfR1TOzOwdmsbBVrJVWSVdBLcC00lWYBwK/BL4RZaVMjMrJ6KyrdoqCbCKiJ9FRGO6/ZzcdDGbWZeU92FakgakD2dKugi4jaTKnwDu7YC6mZk1rxP8/K9EuZtcs0kCatM7OafktQD+LatKmZmVo07QOq1EuVwEozqyImZmFQlBJ5gGW4mKZnJJGgccDPRs2hcR/5NVpczMysp7C7aJpEuB40gC7H3Ah4BZgAOsmVVHTgJsJaMITgdOAP4SEf8AHAb0yLRWZmbl5H0UQYmtEVGU1ChpN2AV4IkGZlYdOUq4XUkL9mlJ/YEfkYwsmAM8mWWlzMzKUVS2lS1DGiFppqSFkhZI+pd0/zclvSppbrp9uOSciyUtlrRI0smt1bOSXARfSB9eL+kBYLeImNfaeWZmmWmfn/+NwIURMUdSP2C2pIfT166MiMtLD5Z0MDAFGEsyq/URSQdGRIvpW8tNNDii3GsRMacNb8TMrN20xzjYiFgJrEwfvy5pIUnWwJacAtwWEfXAUkmLgSOBx1s6oVwL9opydQOOL/N6Li18dS/e97V/qnY1rA36PLSy2lWwNogvdG+ngirugx0oqXQB1xsi4oYdD5I0EjgceAI4Gpgq6dMki79eGBHrSYLvH0tOW075gFx2osGkSt+BmVmHadsIgTURMaHcAZL6AncAX4yITZKuI5mp2jRj9QrgLJpPMFu2JpXc5DIz61zaaZiWpDqS4HpLRNwJEBGvRUQhIookN/ePTA9fDowoOX04sKJc+Q6wZpY7Kla2lS0jWWjwJ8DCiPh+yf4hJYedBsxPH98NTJHUQ9IoYDStjKiqaKqsmVmn0j6jCI4GzgSekzQ33fdV4AxJ49OrvESa6CoiFkiaDjxPMgLhvHIjCKCyqbIiWTJmv4i4TNI+wN4R4bGwZtbhKhnjWomImEXz/ar3lTlnGjCt0mtU0kVwLTAROCN9/jpwTaUXMDNrdzlZMqaSLoL3R8QRkp4BiIj16fLdZmbV0QnyDFSikgDbIKmW9C1J2ovcrOloZl1R7hNul/gB8GtgkKRpJNm1vp5prczMWhKtjxDoLCrJRXCLpNkkKQsFnBoRCzOvmZlZS7pKCzYdNbAF+G3pvoh4JcuKmZm1qKsEWJIVZJsWP+wJjAIWkWSUMTPrcF2mDzYiDil9nmbZOqeFw83MLNXmmVxp7sT3ZVEZM7OKdJUWrKQvlTytAY4AVmdWIzOzcrrSKAKgX8njRpI+2TuyqY6ZWQW6Qgs2nWDQNyK+0kH1MTMrS3SBm1ySukVEY7mlY8zMqiLvAZYkz+ERwFxJdwO/BN5oerEpOa2ZWYdqp2xaHaGSPtgBwFqSNbiaxsMG4ABrZtXRBW5yDUpHEMzn7cDaJCffH2bWFXWFFmwt0Jd3sdCXmVmmchKBygXYlRFxWYfVxMysEm1bVbaqygXY6qcDNzNrRlfoIjihw2phZtYWeQ+wEbGuIytiZlaprjRV1sys8+gifbBmZp2OyM8NIgdYM8sft2DNzLLRFUYRmJl1Tg6wZmYZ6GIJt83MOhe3YM3MspGXPtiaalfAzKzNosKtDEkjJM2UtFDSAkn/ku4fIOlhSS+m/92j5JyLJS2WtEjSya1V0wHWzHJHUdnWikbgwoh4D3AUcJ6kg4GLgBkRMRqYkT4nfW0KMBaYDFybLqvVIgdYM8uXIEm4XclWrpiIlRExJ338OrAQGAacAtycHnYzcGr6+BTgtoioj4ilwGLgyHLXcIA1s1xpWvSwwhbsQElPl2xnN1umNBI4HHgCGBwRKyEJwsCg9LBhwLKS05an+1rkm1xmlj+V3+RaExETyh0gqS9wB/DFiNgktTgRt82LD7gFa2a5o4iKtlbLkepIgustJQu5viZpSPr6EGBVun85MKLk9OHAinLlO8CaWb5UOoKg9VEEAn4CLIyI75e8dDfwmfTxZ4C7SvZPkdRD0ihgNMnq2y1yF4GZ5U47jYM9GjgTeE7S3HTfV4FvA9MlfQ54Bfg7gIhYIGk68DzJCITzIqJQ7gIOsGaWO+0xVTYiZtFy5sNmV3SJiGnAtEqv4QBrZvmTk5lcDrBmli+VTSLoFBxgzSx/HGDNzNpf00SDPHCANbPcUTEfEdYB1szyxavKWke55GMzOWbMy6x/oxdTfvAJAEbvvYaLTnmM3t0bWLmhH5dMP4E36rsz+bA/ceYHnn3r3AMGr+XMa0/nTysHVqv6u6ZtReou/As0BBSg+IHeFD69B2wqUDdtNbzWCIO70fD1vaBfLXqhnm7/teat0wt/35/iMX2q+AaqzysalCHpa8AngQJJzptzgInADRGx5V2U901gc0Rc3p71zIN75oxh+h/H8a3Tf/fWvq+f9r9cdf9E5rw0lL957wuc+YG5XP/IkTzw7IE88OyBAOw/eC1X/P0DDq7VUCcavrs39KqBxqDugpUU39eLmllbKB7ek8KU/tTetoHa2zdS+McBxMg6Gq4ZCrWCtY10P3cF2yb2Tp7vqnLSgu3wqbKSJgIfBY6IiEOBE0ky1HwR6N3R9cm7Z14ayqYtPbbbt8/ADcx5aQgATy4ezqSxS99x3smHLubBeQd0SB1tB1ISXAEak1YsiJrHt1A4qS8AhZP6UvP/07ZGz5q3g+m2aHlo/C6knfLBZq4auQiGkGS4qQeIiDXA6cBQYKakmQCSzpD0nKT5kr7TdLKkyZLmSHpW0owdC5f0eUn3S+ol6XxJz0uaJ+m2jnl71ffn1wZw7HteAuCEcUsYvPvmdxxz0iFLeOjZ0R1cM3tLIag791W6f3wZxSN6Eu/pgdYXYM/0R+We3dCGt38Ha2E9dZ9/le7nrKDx/D3deo2obKuyanQRPAR8Q9KfgEeA2yPiB5K+BEyKiDWShgLfAd4LrAceknQq8AfgR8CxEbFU0oDSgiVNBT4InBoR9ZIuAkalj/s3V5k0P+TZAN377NHcIblz2Z3H8eWP/oF/nDSb378wkobC9t+jY4e/xpsN3ViyakALJVjmakXD9cNgc4G6b62muHRb2cPjPT1o+NEw9Mo2un1vDcUje0H3XTdXk/tgWxARmyW9F/gAMAm4PQ2Epd4HPBoRqwEk3QIcS/Jj6vdpNnEiYl3JOWeSpBM7NSIa0n3zgFsk/Qb4TQv1uQG4AaDPwBHV/8prBy+v2YN/vumjAOyz5waOGfPydq9/0N0DnUffWoqH9qTm6a3EHrWwtjFpxa5tJPq/M4DGPt2hZw16qYE4sEczBXZ9eRoHW5WvwIgoRMSjEXEpMBX42x0Oaen3j2i5e3s+MJIkR2OTjwDXkLSEZ0vaJUZN7NFnKwBScNakOdzx5Ni3XpOCE8b9mYcdYKtnQwE2p0mY6ovUPLOVGFFH8aje1D6cdOfUPryZ4sT0lsTKBiik/+xfa0TLGojBu8Q/5eZV2j2wK3YRSBoDFCPixXTXeOBlkuDYD1hDsmzDVZIGknQRnAFcDTwOXCNpVFMXQUkr9hngOuDudLXHvwAjImKmpFkkoxb6Ahuyf5cd598//gjv3W8F/Xu/yT3/+jNumDGB3t0bOP2oBQA8umAUv5095q3jDx+5glUb+/Dq+t2qVeVdntYV6Pa9NVAMKELxr/tQPKo3HNyDun9fTc0Dy2FQOkwLqFlQT+03VkEtUCMa/3kA7F52rb0uLy8t2Gp8DfYFrk77RBtJFg47mySI3i9pZURMknQxMJOk1XpfRNwFb/WZ3imphiTT+ElNBUfELElfBu4l6Yv9uaTd0zKujIgNHfQeO8zXp5/Y7P7bHj+02f1zlg7jrB9+LMsqWStiv+40XDf0nS/sVpsM39pB8cS+FE/s2wE1yxEH2OZFxGzgr5p56ep0azruVuDWZs6/H7h/h33fLHn8IPBg+vSYna+xmXU2bsGamWUheLtPupNzgDWz3HEL1swsK51ghEAlHGDNLHfcgjUzy4LTFZqZZUOAfJPLzCwbch+smVkG3EVgZpaVzpFnoBIOsGaWOx5FYGaWFbdgzcwyEB5FYGaWnXzE1+ok3DYz2xmKqGhrtRzpp5JWSZpfsu+bkl6VNDfdPlzy2sWSFktalOadLssB1szyp/1WNLgJmNzM/isjYny63Qcg6WBgCjA2PedaSWUznzvAmlm+BFCscGutqIjfA+taPTBxCnBbRNSn6wIuBo4sd4IDrJnliqise2AnZ3tNlTQv7UJoWm56GLCs5Jjl6b4WOcCaWf4Ui5VtMFDS0yXb2RWUfh2wP8l6gSuBK9L9zS3GWjaKexSBmeVLUxdBZdZExIQ2FR/xWtNjST8C7kmfLgdGlBw6HFhRriy3YM0sd7LsIpA0pOTpaUDTCIO7gSmSekgaBYwGnixXlluwZpY/7TSTS9IvgONIuhKWA5cCx0kaT9JWfgk4J7lkLJA0HXieZEXs8yKiUK58B1gzy5n2S/YSEWc0s/snZY6fBkyrtHwHWDPLF68qa2aWHSfcNjPLigOsmVkGAig6wJqZZcArGpiZZccB1swsAwEUKp/KVU0OsGaWMwHhAGtmlg13EZiZZcCjCMzMMuQWrJlZRhxgzcwyEAGFskmsOg0HWDPLH7dgzcwy4gBrZpaF8CgCM7NMBIQnGpiZZcRTZc3MMhDRtCR3p+cAa2b545tcZmbZCLdgzcyy4ITbZmbZcLIXM7NsBBCeKmtmloFwwm0zs8yEuwjMzDKSkxasIid34zqCpNXAy9WuRwYGAmuqXQlrk676me0bEXvtTAGSHiD5+1RiTURM3pnr7QwH2F2ApKcjYkK162GV82fWNdRUuwJmZl2VA6yZWUYcYHcNN1S7AtZm/sy6APfBmpllxC1YM7OMOMCamWXEAbYTkzRc0l2SXpS0RNJVkrqXOb6/pC+UPB8q6VcdU1vbkaSvSVogaZ6kuZLeL+mLknq/y/K+KenL7V1Py44DbCclScCdwG8iYjRwINAXmFbmtP7AWwE2IlZExOlZ1tOaJ2ki8FHgiIg4FDgRWAZ8EXhXAdbyxwG28zoeeDMibgSIiAJwAXCWpC+kLdsHJC2SdGl6zreB/dPW0vckjZQ0H0BST0k3SnpO0jOSJqX7PyvpzrSsFyV9N91fK+kmSfPTcy7o8L9Avg0hmUVUDxARa4DTgaHATEkzASSdkf5950v6TtPJkiZLmiPpWUkzdixc0ucl3S+pl6TzJT2ftpRv65i3Z5VwLoLOaywwu3RHRGyS9ArJ53YkMA7YAjwl6V7gImBcRIwHkDSy5PTz0jIOkXQQ8JCkA9PXxgOHA/XAIklXA4OAYRExLi2rf/u/xS7tIeAbkv4EPALcHhE/kPQlYFJErJE0FPgO8F5gPclncirwB+BHwLERsVTSgNKCJU0FPgicGhH1ki4CRqWP+3fUG7TWuQXbeYkk9WVL+x+OiLURsZWkK+GYVso7BvgZQES8QJJzoSnAzoiIjRHxJvA8sC/wZ2A/SVdLmgxs2tk3tCuJiM0kgfNsYDVwu6TP7nDY+4BHI2J1RDQCtwDHAkcBv4+IpWlZ60rOORP4EPC3Ta1jYB5wi6S/Bxozekv2LjjAdl4LgO3mokvaDRgBFHhn8G1tQLPKvFZf8rgAdIuI9cBhwKMkrd8ft15lKxURhYh4NCIuBaYCf7vDIS19Ji19uQLMB0YCw0v2fQS4hiSgz5bkX6adhANs5zUD6C3p05D0iQJXADeRdAucJGmApF7AqSQ/K18H+rVQ3u+BT6VlHQjsAyxq6eKSBgI1EXEHcAlwxM6/pV2HpDGSRpfsGk/yq6H0M3oC+GtJA9PP9wzgf4HH0/2j0rJKuwieAc4B7k5HidQAIyJiJvCvJDc6+2b2xqxN/E3XSUVESDoNuFbSJSRfhvcBXyX5H3EWyU/+A4BbI+JpAEl/SG9s3U/SqmlyLXC9pOdIfkZ+Nu2za6kKw4Ab0/+BAS5u1zfY9fUFrk77RBuBxSTdBWcA90taGRGTJF0MzCRptd4XEXcBSDobuDP9+68CTmoqOCJmpcO17iXpi/25pN3TMq6MiA0d9B6tFZ4qm0NpX96EiJha7bqYWcvcRWBmlhG3YM3MMuIWrJlZRhxgzcwy4gBrZpYRB1irmKRCmudgvqRfvtusUGlZN0k6PX38Y0kHlzn2OEl/9S6u8VI6nrei/Tscs7mN13KmK3sHB1hri60RMT7NT7ANOLf0xXSwfJtFxD9GxPNlDjkOaHOANas2B1h7tx4DDkhblzMl3Qo8l2bh+p6kp9LsTudAkn5R0n+nWZ/uJUkmQ/rao5ImpI+3yyKVJqw5F7ggbT1/QNJeku5Ir/GUpKPTc/eU9JCSbGE/pPz04KZr/0bSbCV5W8/e4bUr0rrMkLRXum9/JZnHZkt6LE2cY9Ysz+SyNkvnun8IeCDddSRJFq+laZDaGBHvk9QD+IOkh0iydY0BDgEGkySV+ekO5e7FDlmkImKdpOuBzRFxeXrcrSQzlmZJ2gd4EHgPcCkwKyIuk/QRkplTrTkrvUYvkqxkd0TEWqAPMCciLpT0jbTsqSSLEZ4bES9Kej/JDLnj38Wf0XYBDrDWFr0kzU0fPwb8hOSn+5NNmZ9Ipm4e2tS/CuwOjCbJEvWLNK/tCkm/a6b8clmkSp0IHFwyzXc3Sf3Sa3wsPfdeSesreE/np1OSIUmkMxpYCxSB29P9PyeZtto3fb+/LLl2jwquYbsoB1hri61NuWabpIHmjdJdwD9HxIM7HPdhKsv4VcnMlxpgYpqqcce6VDxzRtJxJMF6YkRskfQo0LOFwyO97oYd/wZmLXEfrLW3B4F/klQHSeYuSX1IsnlNSftohwCTmjm3pSxSO2YJe4jk5zrpcePTh6UZwz4E7NFKXXcH1qfB9SCSFnSTGpIVCAA+SdL1sAlYKunv0mtI0mGtXMN2YQ6w1t5+TNK/OifN6vVDkl9KvwZeBJ4DriNJy7ediFhN0m96p6Rnefsn+m+B05pucgHnAxPSm2jP8/Zohm8Bx0qaQ9JV8UordX0A6CZpHvBvwB9LXnsDGCtpNkkf62Xp/k8Bn0vrtwA4pYK/ie2inIvAzCwjbsGamWXEAdbMLCMOsGZmGXGANTPLiAOsmVlGHGDNzDLiAGtmlpH/AzIOALDjVApPAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pipe3 = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('knn', KNeighborsClassifier())\n",
    "])\n",
    "\n",
    "params3 = {\n",
    "     'vect__stop_words':[None,'english'],\n",
    "     'vect__ngram_range' : [(1,1),(1,2),(2,2)]\n",
    "}\n",
    "\n",
    "gs3 = GridSearchCV(    pipe3,         # What is the model we want to fit?\n",
    "                      param_grid = params3,  # What is the dictionary of hyperparameters?\n",
    "                      cv=5,           # What number of folds in CV will we use?\n",
    "                      verbose=1)\n",
    "\n",
    "gs3.fit(X_train,y_train)\n",
    "print(gs3.best_score_)   # <- Crossval score\n",
    "print(gs3.best_params_)\n",
    "plot_confusion_matrix(gs3, X_test, y_test, display_labels=['Options','Stocks']);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "80f2e70c-5381-49b8-9c7d-a3be371504b9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogReg Score: 0.828\n",
      "Random Forest Score: 0.837\n",
      "KNeighbors Score: 0.737\n",
      "\n",
      "LogReg Best Params: {'lr__max_iter': 1000, 'vect__ngram_range': (1, 2), 'vect__stop_words': None}\n",
      "Random Forest Best Params: {'vect__ngram_range': (1, 2), 'vect__stop_words': None}\n",
      "KNeighbors Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': 'english'}\n"
     ]
    }
   ],
   "source": [
    "print(f\"LogReg Score: {round(gs.best_score_,3)}\")  \n",
    "print(f\"Random Forest Score: {round(gs2.best_score_,3)}\") \n",
    "print(f\"KNeighbors Score: {round(gs3.best_score_,3)}\") \n",
    "print(f\"\\nLogReg Best Params: {gs.best_params_}\")\n",
    "print(f\"Random Forest Best Params: {gs2.best_params_}\")\n",
    "print(f\"KNeighbors Best Params: {gs3.best_params_}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c908906-eab5-4deb-89fb-8279d682f511",
   "metadata": {},
   "source": [
    "# 3.1 Model Selection "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57b6b0bb-a68e-4fc4-923b-6ac9fbef5299",
   "metadata": {},
   "source": [
    "After running the models against the original titles, the lemmatized titles, and stemmed titles, it looks like the Random Forest model with the lemmatized titles gives us the best cross_val_score. The next step is to attempt to optimize the model by adjusting hyperparameters. At the bottom of this notebook, we have saved all of the tracking data from each of the different combinations of model/title/subtext."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65b59012-00f6-40c3-9bf2-eefe104b6634",
   "metadata": {},
   "source": [
    "**Scores based on Lemma Title**\n",
    "<br> Random Forest Score: 0.835\n",
    "<br> Random Forest Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}\n",
    "<br>\n",
    "<br>\n",
    "<br>You'll notice below that we also ran the selftext (post) content through all of the models.  The scores on the selftext data are all lower but 15% of the selftext data was null.  We believe that given more time, we could optimize around the selftext data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3fb52b5-55d1-435c-8018-5f4b228887f6",
   "metadata": {},
   "source": [
    "# 3.2 Model Tuning "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "bb838f62-3488-47d1-8372-70cfcfb91a02",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df['lemma_title']\n",
    "y = df['subreddit']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,random_state=42, stratify=y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "e69cbd1a-0e77-4d3e-8890-69262a0d30e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 10 candidates, totalling 50 fits\n",
      "0.8232136894824709\n",
      "{'vect__stop_words': None, 'vect__ngram_range': (1, 2), 'rf__n_estimators': 800, 'rf__min_samples_split': 5, 'rf__min_samples_leaf': 4, 'rf__max_depth': 60, 'rf__bootstrap': False}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAEGCAYAAAAg6I3HAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAka0lEQVR4nO3de5xWVb3H8c+Xi1xFRNBGwGtooSV5kLx0Ot5KLEstPUFmmBZamCdPHY94LC3jpJ3Mk5YamYqmKR01ybuiZJZXvAJKYt4QFEEUVBiZmd/5Y6/RB5x55hl89jyzx++b137NftZee+31zDC/Wc/aa6+liMDMzKqvW60rYGbWVTnAmpnlxAHWzCwnDrBmZjlxgDUzy0mPWlegM+k9sHf0r+tf62pYO6x+sXetq2DtUP/mctbUv6H3UsZ+e/WLZa80VpR39qP1N0fE2PdyvffCAbZE/7r+fG7a52pdDWuHJ38ystZVsHZ4eNYv3nMZS19p5N6bh1WUt2fdU4PbyiOpO/AA8EJEHCDpVOAbwMspy0kRcUPKOxk4CmgEjouIm8uV7QBrZgUTNEZTNQv8N+BxYEBJ2lkR8bPSTJJGAuOAHYDNgdskbRcRrTan3QdrZoUSQBNR0dYWScOAzwIXVHDpA4ErIqI+Ip4GFgBjyp3gAGtmhdNU4b8K/C9wArwr87GSHpV0oaSNU9pQ4PmSPAtTWqscYM2sUIJgTTRVtAGDJT1Qsk1sLkfSAcCSiJi9ziXOA7YFRgGLgTObT2mxOmW4D9bMCiWAxgo+/idLI2J0K8f2AD4v6TNAb2CApN9FxFeaM0j6DXBderkQGF5y/jBgUbmLuwVrZoVTjT7YiJgcEcMiYiuym1e3R8RXJNWVZDsYmJP2ZwDjJPWStDUwAriv3DXcgjWzQgmgMd9ZAH8qaVS61DPA0QARMVfSdGAe0ABMKjeCABxgzayAqjpIC4iIWcCstH94mXxTgCmVlusAa2aFEkR7+mBrygHWzAolAtYUI746wJpZ0YjGFkdMdT4OsGZWKAE0uQVrZpYPt2DNzHKQPWjgAGtmVnUBrIliPCPlAGtmhRKIxoI8hOoAa2aF0xTuIjAzqzr3wZqZ5UY0ug/WzKz6shUNHGDNzKouQrwV3WtdjYo4wJpZ4TS5D9bMrPqym1zuIjAzy4FvcpmZ5cI3uczMctRYkAcNivFnwMwsCcSa6FHRVglJ3SU9JOm69HqQpFslPZm+blySd7KkBZLmS9qvrbIdYM2sUJpvclWyVejfgMdLXp8IzIyIEcDM9BpJI8lWn90BGAucK6nseDEHWDMrlEA0RmVbWyQNAz4LXFCSfCAwLe1PAw4qSb8iIuoj4mlgATCmXPnugzWzwmnHTa7Bkh4oeT01IqaWvP5f4ARgw5K0zSJiMUBELJa0aUofCtxTkm9hSmuVA6yZFUoE7RmmtTQiRrd0QNIBwJKImC1pzwrKaqlJXHbxGgdYMyuU7CZXVR6V3QP4vKTPAL2BAZJ+B7wkqS61XuuAJSn/QmB4yfnDgEXlLuA+WDMrnGrc5IqIyRExLCK2Irt5dXtEfAWYAUxI2SYA16b9GcA4Sb0kbQ2MAO4rdw23YM2sUALlPeH26cB0SUcBzwGHAkTEXEnTgXlAAzApIhrLFeQAa2aFU+25CCJiFjAr7S8D9mkl3xRgSqXlOsCaWaEE0OS5CMzM8iAvGWNmlods2W5PuG1mVnURcheBmVlePB+smVkOsvlg3QdrZpYDr2hgZpaLbJiWW7BmZlVXxbkIcucAa2aF4zW5zMxykE1X6C4CM7NcuA/WzCwH2Wxa7iIwM6u67FFZB1jrAFEfLPnmKngLohH67N2djb7Ri9d+U88bMxroPjD7KDXgmxvQZ/cevHnTGlZetubt89csaGLTaX3YYLti3JXtCv7z8D+z+0eeY/nKPhxx2iFrHRu376N864v38rnvHc5rb/Tmw1su4XuH/QUACS66bmf+8sjWtah2J+IWLPD2io2/AkaSrZ5wHfAfEfFWK/kHAl+OiHPT682BsyPikJbyG7ABDPllH7r1FdEQLJm4it67ZXMAbziuJxsetsFa2fuO7UnfsT0BWLOgkaUnrHZw7WA33b0d18zagZOOmLVW+qYbv87oDy/kxWX93077x6JBTDz9YBqburHJgDe58OSr+NtjW9LYVIwAk5eiPMmV209JkoCrgT+m9cW3A/pTfrLagcC3ml9ExCIH1/Ik0a1v9p8tGsjmWa/Qm7c20PdT/hDT0R5ZUMeKN3q9K/3YQ+7hvKs/vtYqevVrerwdTDfo2UAU5OZOnppHEVRj2e685flncG9gdURcBJCWVjgeOFLStyRdK+kmSfMlnZLOOR3YVtLDkv5H0laS5gBI6i3pIkmPSXpI0l4p/QhJV6eynpT005TeXdLFkuakc47P8b3WVDQGLx3+Jov3f4NeY7rTa8esRfr6H9bw0mFv8sqPV9O04t2LX755WwN9P+0A2xns8dFnWfpqX556YZN3HfvwVkuY9v0/cNHJV3Hm5Xu871uvkE24XclWToop90l6RNJcST9M6adKeiHFoYfToojN50yWtCDFrf3aqmeev107ALNLEyJihaTn0nXHADsCbwL3S7oeOBHYMSJGAUjaquT0SamMj0j6EHCLpO3SsVHAx4B6YL6kc4BNgaERsWMqa2BLlZQ0EZgI0O8D/d7TG64VdRebXdqXppXBsv9czZqnGun/hZ4MOHIDEKz49Vu8enY9g07u/fY59XMaUW/Rc1t3D9Rar54NHD72Ib579mdaPP74M5sy4bRD2fIDyzlpwp+5d+5w3mp4//5hrOKaXPXA3hHxuqSewF2SbkzHzoqIn5VmljSSbHHEHYDNgdskbVduXa48/xSKltcMb06/NSKWRcQqsq6ET7RR3ieASwEi4gngWbJuB4CZEfFaRKwmW5BsS+AfwDaSzpE0FljRUqERMTUiRkfE6N4De7eUpTC6bSh67dyd1fc00n2Tbqi7UDfR78CevDWvaa28q25z90BnMXTICuoGr+TCk6/iyh//niED3+CCk65m0IA318r37Isbs6q+B1tvvrxGNe0cAmiIbhVtZcvJvJ5e9kxbSzGr2YHAFRFRHxFPAwvIGoqtyjPAzgVGlyZIGkC2rngj734j5d4YULZXu75kvxHoERHLgZ3IFjKbBFzQdpWLp3F50LQy+9bF6mD1/Q302LIbjUvfCair/txAz23e+VFHU7BqpgNsZ/GPRYM48ITD+dLJ4/nSyeN5+dV+fP2/v8ArK/pSt8kKunfLfpabDVrJFpu9xovLNqxxjWuvHV0EgyU9ULJNLC0ndSU+DCwha/Tdmw4dK+lRSRdK2jilDQWeLzl9YUprVZ6/YTOB0yV9NSIukdQdOBO4mKxb4FOSBgGrgIOAI4GVQGv/e+4EDgNuT10DWwDzgZ1byixpMPBWRFwl6al03S6ncWkTy0+rh8as87/vPj3o84kevHLqat56MrvX2r1ObHziOzdV6h9qpPumosdQ9+XVwg+OvJ2PbbeIjfqv5v/++3Iuum5nrv/bh1rM+5FtX+Kw/W6mobEbEeLnV+zBa28U+5PWexbt6iJYGhGjWzuYPt6PSl2I10jaETgPOI2s0XcaWdw6kpYbeWUbhrkF2IgISQcD50r6Pllr+QbgJGA8cBfZR/4PApdHxAMAkv6abmzdSDbEq9m5wPmSHiO7V35ERNRngxVaNBS4SFJzFJlc1TfYSWwwojubXdL3XemDTm39l7D3P/Wg92/deq2VH124d9njXzp5/Nv7t9w3glvuG5F3lQoljwm3I+JVSbOAsaV9r5J+Qza8FLIW6/CS04YBi8qVm+tvWUQ8D3xu3fQUFJdExLEtnPPldZJ2TOmrgSNayH8xJa3TiDig5HCLrVszK7Zq3OSSNARYk4JrH2Bf4AxJdRGxOGU7GJiT9mcAl0v6OdlNrhHAfeWu4WaMmRVKFSfcrgOmpe7LbsD0iLhO0qWSRqVLPQMcDRARcyVNJ7uR3gBMKjeCAGoUYNdtdZqZVSoQDVUYCxwRj5IN71w3/fAy50yh/MNSa3EL1swKpyiPyjrAmlmxhOeDNTPLhRc9NDPLkQOsmVkOAhVmwhsHWDMrHN/kMjPLQfgml5lZfooy8bgDrJkVTNXmg82dA6yZFY5bsGZmOYiAxiYHWDOzXHgUgZlZDgJ3EZiZ5cQ3uczMchNtreDXSTjAmlnhuIvAzCwH2SiCYsxFUIxampmViKhsK0dSb0n3SXpE0lxJP0zpgyTdKunJ9HXjknMmS1ogab6k/dqqpwOsmRVOhCra2lAP7B0ROwGjgLGSdgVOBGZGxAhgZnqNpJHAOGAHYCzZitndy13AAdbMCiWoLLi2FWAj83p62TNtARwITEvp04CD0v6BwBURUR8RTwMLgDHlruEAa2aFExVuwGBJD5RsE0vLkdRd0sPAEuDWiLgX2Kx52e70ddOUfSjwfMnpC1Naq3yTy8yKJSAqf1R2aUSMbrWobNntUZIGAtdI2rFMWS1dtGxPr1uwZlY4VeqDLSkvXgVmkfWtviSpDiB9XZKyLQSGl5w2DFhUrlwHWDMrnCqNIhiSWq5I6gPsCzwBzAAmpGwTgGvT/gxgnKRekrYGRgD3lbtGq10Eks6hTPM3Io4rX30zs+qr4lwEdcC0NBKgGzA9Iq6TdDcwXdJRwHPAoQARMVfSdGAe0ABMSl0MrSrXB/tANd6BmVlVBVCFABsRjwIfayF9GbBPK+dMAaZUeo1WA2xETCt9LalfRLxRacFmZnkpylwEbfbBStpN0jzg8fR6J0nn5l4zM7MWiWiqbKu1Sm5y/S+wH7AMICIeAT6ZY53MzMprx0DYWqpoHGxEPC+t9degbMeumVluomvNpvW8pN2BkLQBcBypu8DMrCY6Qeu0EpV0ERwDTCJ7JOwFskkRJuVYJzOzNqjCrbbabMFGxFLgsA6oi5lZZZpqXYHKVDKKYBtJf5L0sqQlkq6VtE1HVM7M7F2ax8FWstVYJV0ElwPTyZ562Bz4A/D7PCtlZlZONR6V7QiVBFhFxKUR0ZC231GYLmYz65KKPkxL0qC0e4ekE4EryKr8JeD6DqibmVnLOsHH/0qUu8k1myygNr+To0uOBXBaXpUyMytHnaB1WolycxFs3ZEVMTOrSAg6wWOwlajoSa40y/dIoHdzWkRcklelzMzKKnoLtpmkU4A9yQLsDcD+wF2AA6yZ1UZBAmwlowgOIZsb8cWI+BqwE9Ar11qZmZVT9FEEJVZFRJOkBkkDyNan8YMGZlYbVZpwuyNU0oJ9IK1b8xuykQUP0sY6NGZmeVJUtpUtQxou6Q5Jj0uaK+nfUvqpkl6Q9HDaPlNyzmRJCyTNl7RfW/WsZC6Cb6Xd8yXdBAxISy2YmdVGdT7+NwDfjYgHJW0IzJZ0azp2VkT8rDSzpJHAOGAHsqdab5O0Xbl1uco9aLBzuWMR8WA73oiZWdVUYxxsRCwGFqf9lZIeJ5s1sDUHAldERD3wtKQFwBjg7tZOKNeCPbNc3YC9yxwvpLeeaGLhrq/XuhrWDncumlrrKlg7jNlvaXUKqrwPdrCk0gVcp0bEu/7TSNqKbAHEe4E9gGMlfZVs8dfvRsRysuB7T8lpCykfkMs+aLBXpe/AzKzDtG+EwNKIGF0ug6T+wFXAdyJihaTzyJ5UbX5i9UzgSFqeYLZsTSq5yWVm1rlUaZiWpJ5kwfWyiLgaICJeiojGiGgiu7k/JmVfCAwvOX0YsKhc+Q6wZlY4aqpsK1tGttDgb4HHI+LnJel1JdkOBuak/RnAOEm9JG0NjKCNEVUVPSprZtapVGcUwR7A4cBjkh5OaScB4yWNSld5hjTRVUTMlTQdmEc2AmFSuREEUNmjsiJbMmabiPiRpC2AD0SEx8KaWYerZIxrJSLiLlruV72hzDlTgCmVXqOSLoJzgd2A8en1SuBXlV7AzKzqCrJkTCVdBB+PiJ0lPQQQEcvT8t1mZrXRCeYZqEQlAXaNpO6ktyRpCIVZ09HMuqLCT7hd4mzgGmBTSVPIZtc6OddamZm1JtoeIdBZVDIXwWWSZpNNWSjgoIh4PPeamZm1pqu0YNOogTeBP5WmRcRzeVbMzKxVXSXAkq0g27z4YW9ga2A+2YwyZmYdrsv0wUbER0pfp1m2jm4lu5mZJe1+kivNnbhLHpUxM6tIV2nBSvr3kpfdgJ2Bl3OrkZlZOV1pFAGwYcl+A1mf7FX5VMfMrAJdoQWbHjDoHxH/0UH1MTMrS3SBm1ySekREQ7mlY8zMaqLoAZZsnsOdgYclzQD+ALzRfLB5clozsw5Vpdm0OkIlfbCDgGVka3A1j4cNwAHWzGqjC9zk2jSNIJjDO4G1WUH+fphZV9QVWrDdgf6sx0JfZma5KkgEKhdgF0fEjzqsJmZmlWjfqrKtkjQcuAT4AFmnw9SI+IWkQcCVwFZkS8b8a1q2G0mTgaOARuC4iLi53DXKrWhQ++nAzcxa0LxsTFtbGxqA70bEh4FdgUmSRgInAjMjYgQwM70mHRtHNg/LWODcNJS1VeUC7D4VvE8zs45XhWW7I2JxRDyY9lcCjwNDgQOBaSnbNOCgtH8gcEVE1EfE08AC3lnSu0WtdhFExCvlq2dmVhvteFR2sKQHSl5PjYip7ypP2gr4GHAvsFlELIYsCEvaNGUbCtxTctrClNYqL9ttZsXSvj7YpRExulwGSf3JHv//TkSsyBbSbjlrK7VpVSWrypqZdRpqx9ZmWVJPsuB6WcnDUy9JqkvH64AlKX0hMLzk9GHAonLlO8CaWfFUoQ9WWVP1t8DjEfHzkkMzgAlpfwJwbUn6OEm9JG0NjCB74rVV7iIws8Kp0oMGewCHA49JejilnQScDkyXdBTwHHAoQETMlTQdmEc2AmFSRDSWu4ADrJkVTxUCbETcRes9CS2OooqIKcCUSq/hAGtmxdLFJtw2M+tcusCjsmZmnVJXmOzFzKxzcoA1M8uHW7BmZnkIusSE22ZmnU6XWPTQzKzTcoA1M8uHohgR1gHWzIqlSisadAQHWDMrHPfBmpnlxI/KmpnlxS1YM7McVLagYafgAGtmxeMAa2ZWfX7QwMwsR2oqRoT1mlxmViyVrsdVQQyWdKGkJZLmlKSdKukFSQ+n7TMlxyZLWiBpvqT92irfLdguqFu34Jyb/s6yxT35wYRt+Pr3F7Hrp1aw5i2x+NkNOPP4LXhjRfdaV/N9r7ERvj12OzapW8NplzzNpT/7ADdePoiNBmXLPH1t8iLG7LOShjVw1ve2YMFjfWhsEPse+grjvr2kjdK7tioO07oY+CVwyTrpZ0XEz9a6pjQSGAfsAGwO3CZpu3LrctWkBSvpvyTNlfRo+gvxcUnfkdR3Pcs7VdL3ql3Pojro60t5/sneb79+8M4NmbjX9nxz3+154R+9GPftl2pYO2v2xwuGMHxE/VppB3/jZc67bT7n3TafMfusBODOPw1kTb349e3z+eVN87nh0sG8+PwGtahy51GlFmxE3Am8UuFVDwSuiIj6iHgaWACMKXdChwdYSbsBBwA7R8RHgX2B54HvAOsVYO0dg+veYsw+K7jx8kFvpz345w1paszWdnt8dj8G162pVfUseXlRT+6bOYD9v7yszbwSrH6zG40N8NbqbvTYoIm+/csuZtrlKSrbgMGSHijZJlZ4iWNTA/BCSRuntKFksarZwpTWqlq0YOuApRFRDxARS4FDyJrcd0i6A0DSeEmPSZoj6YzmkyWNlfSgpEckzVy3cEnfkHSjpD6SjpM0L32jruiYt1dbx/xwERf8uI5oanmxzP3Gv8L9tw/o4FrZus4/ZShfP3kRWuc38E8XDeGYfbbnzOOHs/LVrBvnnw94ld59mxg/ake+sstIDjnmZQZs/D4OsAFEVLZlsWZ0yTa1giucB2wLjAIWA2em9JZ+qcq2k2sRYG8Bhkv6u6RzJf1LRJwNLAL2ioi9JG0OnAHsTfYmd5F0kKQhwG+AL0bETqT1yptJOhb4HHBQRKwCTgQ+llrKx7RUGUkTm/+6raG+pSyF8fF9V/Dq0h4seKzlDwLjj3uJxga4/eqBHVsxW8s9tw5g4OAGRnx01VrpB0xYykV3z+PcW+czaLM1TP3h5gDMf6gf3boHlz80h0vufZyrzh/C4mff310EaqpsWx8R8VJENEZEE1m8ae4GWAgML8k6jCxutarDA2xEvA78EzAReBm4UtIR62TbBZgVES9HRANwGfBJYFfgztT/QUSU9p0cDuxPFnybI+WjwGWSvgI0tFKfqc1/3XrSqyrvsVZG7vIGu356BdPuncfk855lp0+8zgnnPAvAvoe+wph9V3DGsVvS+lLw1hHm3d+Pe24ZwFfHjOQn39ySR+7akDOO3YKNhzTQvTt06wb7H/YK8x/O/lDecc1ARu+1kh49YeDgBkbu8gZ/f+T925vWPA62wi6C9pcv1ZW8PBhoHmEwAxgnqZekrYERwH3lyqrJKIJ0120WMEvSY8CEdbK0FgFE603yOWSt3WHA0ynts2SB+fPA9yXtkAJ2l3TRT+q46CfZ/42P7vY6hxyzhJ9+e0tG77mCf520hP/4wgepX+WRebV25EmLOfKkxQA88rf+/N/5Q/jPXz7Hspd6sMlm2X/Pv924EVttvxqAIUPX8PBd/dnni8upX9WNJx7sx8HfeLlm9a+5dz7+v2eSfg/sSdZXuxA4BdhT0iiyWPMMcHR22ZgraTowj6zBNqncCAKoQYCVtD3QFBFPpqRRwLPAVsCGwFLgXuAXkgYDy4HxwDnA3cCvJG0dEU9LGlTSin2IrO9kRhqf9iIwPCLukHQX8GWgP/Bq/u+yc5k05QV69gp+cuVTADwxux9nnzisxrWydf32x5vz1Nw+SLDZsLc47qfZ/ZTPf20pZx6/BRP32h5CfPpLy9hm5Ooa17a2qvUkV0SMbyH5t2XyTwGmVFp+LVqw/YFzJA0k+yuwgKy7YDxwo6TFqR92MnAHWav1hoi4FrI+U+BqSd2AJcCnmguOiLvScK3rgU8Dv5O0USrjrIh4tYPeY809end/Hr27PwBf2+PDNa6NtWan3V9np91fB+CEc55rMU+ffk2cPPWZDqxVARTjQa6OD7ARMRvYvYVD56StOd/lwOUtnH8jcOM6aaeW7N8M3JxefuK919jMOhvPRWBmlocAGosRYR1gzaxw3II1M8uLV5U1M8uHW7BmZnnwst1mZvkQIN/kMjPLh9wHa2aWA3cRmJnlpXpzEeTNAdbMCsejCMzM8uIWrJlZDsKjCMzM8lOM+OoAa2bF42FaZmZ5KUiA9fohZlYsATRVuLUhLcu9RNKckrRBkm6V9GT6unHJscmSFkian1ZOKcsB1swKRQSKyrYKXAyMXSftRGBmRIwAZqbXSBoJjAN2SOecK6l7ucIdYM2seJqaKtvaEBF3Aq+sk3wgMC3tTwMOKkm/IiLq08rWC3hnSe8WOcCaWbG0r4tgsKQHSraJFVxhs4hYDJC+bprShwLPl+RbmNJa5ZtcZlY47RhFsDQiRlfrsi2kla2IW7BmVjwRlW3r5yVJdQDp65KUvhAYXpJvGLCoXEEOsGZWMBUG1/UPsDOACWl/AnBtSfo4Sb0kbQ2MAO4rV5C7CMysWKq4qqyk3wN7kvXVLgROAU4Hpks6CngOOBQgIuZKmg7MAxqASRHRWK58B1gzK5xqPckVEeNbObRPK/mnAFMqLd8B1syKpyBPcjnAmlmxBNDkAGtmlgOvaGBmlh8HWDOzHATQWMFMLp2AA6yZFUxAOMCameXDXQRmZjnwKAIzsxy5BWtmlhMHWDOzHERAY9kpADoNB1gzKx63YM3McuIAa2aWh/AoAjOzXASEHzQwM8uJH5U1M8tBREVLcncGDrBmVjxVuskl6RlgJdAINETEaEmDgCuBrYBngH+NiOXrU74XPTSzwommpoq2Cu0VEaNKlvc+EZgZESOAmen1enGANbOCyX1V2QOBaWl/GnDQ+hbkAGtmxdI82UslW2Wl3SJptqSJKW2ziFgMkL5uur5VdR+smRVKAFH5o7KDJT1Q8npqREwteb1HRCyStClwq6QnqlVPcIA1s6KJdk24vbSkb7WFomJR+rpE0jXAGOAlSXURsVhSHbBkfavqLgIzK5xoioq2ciT1k7Rh8z7waWAOMAOYkLJNAK5d33q6BWtmxVOdJ7k2A66RBFksvDwibpJ0PzBd0lHAc8Ch63sBRUEmTegIkl4Gnq11PXIwGFha60pYu3TVn9mWETHkvRQg6Say708llkbE2PdyvffCAfZ9QNID5fqhrPPxz6xrcB+smVlOHGDNzHLiAPv+MLXtLNbJ+GfWBbgP1swsJ27BmpnlxAHWzCwnDrCdmKRhkq6V9KSkpyT9QtIGZfIPlPStktebS/q/jqmtrUvSf0maK+lRSQ9L+rik70jqu57lnSrpe9Wup+XHAbaTUvZ4ydXAH9O8lNsB/YEpZU4bCLwdYCNiUUQckmc9rWWSdgMOAHaOiI8C+wLPA98B1ivAWvE4wHZeewOrI+IigIhoBI4HjpT0rdSyvUnSfEmnpHNOB7ZNraX/kbSVpDkAknpLukjSY5IekrRXSj9C0tWprCcl/TSld5d0saQ56ZzjO/w7UGx1ZE8R1QNExFLgEGBz4A5JdwBIGp++v3MkndF8sqSxkh6U9IikmesWLukbkm6U1EfScZLmpZbyFR3z9qwSnoug89oBmF2aEBErJD1H9nMbA+wIvAncL+l6spnXd4yIUQCStio5fVIq4yOSPkQ2B+Z26dgo4GNAPTBf0jlkc2AOjYgdU1kDq/8Wu7RbgB9I+jtwG3BlRJwt6d/JZtBfKmlz4Azgn4DlZD+Tg4C/Ar8BPhkRT6clTN4m6ViyiUkOioh6SScCW6f9gR31Bq1tbsF2XiKb+rK19FsjYllErCLrSvhEG+V9ArgUICKeIJtzoTnAzoyI1yJiNTAP2BL4B7CNpHMkjQVWvNc39H4SEa+TBc6JwMvAlZKOWCfbLsCsiHg5IhqAy4BPArsCd0bE06msV0rOORzYH/hic+sYeBS4TNJXgIac3pKtBwfYzmsusNaz6JIGAMPJFmhbN/i2NaBZZY7Vl+w3Aj3SIm87AbPIWr8XtF1lKxURjRExKyJOAY4FvrhOltZ+Jq39cYVsOr2tgGElaZ8FfkUW0GdL8ifTTsIBtvOaCfSV9FXI+kSBM4GLyboFPiVpkKQ+ZGsG/ZVsdcwNWynvTuCwVNZ2wBbA/NYuLmkw0C0irgK+D+z83t/S+4ek7SWNKEkaRfapofRndC/wL5IGp5/veODPwN0pfetUVmkXwUPA0cCMNEqkGzA8Iu4ATiC70dk/tzdm7eK/dJ1URISkg4FzJX2f7I/hDcBJZL+Id5F95P8g2TyWDwBI+mu6sXUjWaum2bnA+ZIeI/sYeUTqs2utCkOBi9IvMMDkqr7Brq8/cE7qE20AFpB1F4wHbpS0OCL2kjQZuIOs1XpDRFwLoGx9qKvT938J8KnmgiPirjRc63qyvtjfSdoolXFWRLzaQe/R2uBHZQso9eWNjohja10XM2uduwjMzHLiFqyZWU7cgjUzy4kDrJlZThxgzcxy4gBrFZPUmOY5mCPpD+s7K1Qq62JJh6T9CySNLJN3T0m7r8c1nknjeStKXyfP6+28lme6sndxgLX2WBURo9L8BG8Bx5QeTIPl2y0ivh4R88pk2RNod4A1qzUHWFtffwE+mFqXd0i6HHgszcL1P5LuT7M7HQ3Z9IuSfplmfbqebDIZ0rFZkkan/bVmkUoT1hwDHJ9az/8saYikq9I17pe0Rzp3E0m3KJst7NeUfzy4+dp/lDRb2bytE9c5dmaqy0xJQ1LatspmHpst6S9p4hyzFvlJLmu39Kz7/sBNKWkM2SxeT6cg9VpE7CKpF/BXSbeQzda1PfARYDOySWUuXKfcIawzi1REvCLpfOD1iPhZync52RNLd0naArgZ+DBwCnBXRPxI0mfJnpxqy5HpGn3IZiW7KiKWAf2AByPiu5J+kMo+lmwxwmMi4klJHyd7Qm7v9fg22vuAA6y1Rx9JD6f9vwC/Jfvofl/zzE9kj25+tLl/FdgIGEE2S9Tv07y2iyTd3kL55WaRKrUvMLLkMd8BkjZM1/hCOvd6ScsreE/HpUeSIZtIZwSwDGgCrkzpvyN7bLV/er9/KLl2rwquYe9TDrDWHqua55ptlgLNG6VJwLcj4uZ18n2Gymb8quTJl27AbmmqxnXrUvGTM5L2JAvWu0XEm5JmAb1byR7puq+u+z0wa437YK3abga+KaknZDN3SepHNpvXuNRHWwfs1cK5rc0ite4sYbeQfVwn5RuVdktnDNsf2LiNum4ELE/B9UNkLehm3chWIAD4MlnXwwrgaUmHpmtI0k5tXMPexxxgrdouIOtffTDN6vVrsk9K1wBPAo8B55FNy7eWiHiZrN/0akmP8M5H9D8BBzff5AKOA0anm2jzeGc0ww+BT0p6kKyr4rk26noT0EPSo8BpwD0lx94AdpA0m6yP9Ucp/TDgqFS/ucCBFXxP7H3KcxGYmeXELVgzs5w4wJqZ5cQB1swsJw6wZmY5cYA1M8uJA6yZWU4cYM3McvL/rVPkz01g0O4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pipe4 = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('rf', RandomForestClassifier(random_state=42))\n",
    "])\n",
    "params4 = { \n",
    "            'vect__stop_words':[None,'english'],\n",
    "            'vect__ngram_range' : [(1,1),(1,2),(2,2)],\n",
    "            'rf__bootstrap': [True, False],\n",
    "            'rf__min_samples_leaf': [1, 2, 4],\n",
    "            'rf__min_samples_split': [2, 5, 10]   ,\n",
    "            'rf__n_estimators':[200, 400, 600, 800, 1000, 1200, 1400, 1600, 1800, 2000]    ,\n",
    "            'rf__max_depth': [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, None]   \n",
    "    \n",
    "}\n",
    "gs4 = RandomizedSearchCV(    pipe4,         # What is the model we want to fit?\n",
    "                      param_distributions = params4,  # What is the dictionary of hyperparameters?\n",
    "                      cv=5,           # What number of folds in CV will we use?\n",
    "                      verbose=1)\n",
    "# When trying GridSearchCV, this was attempging more than 59,000 fits.  Decided to use RandomizedSearch instead.\n",
    "gs4.fit(X_train,y_train)\n",
    "print(gs4.best_score_)   # <- Crossval score\n",
    "print(gs4.best_params_)\n",
    "plot_confusion_matrix(gs4, X_test, y_test, display_labels=['Options','Stocks']);\n",
    "\n",
    "#Tuning was done with a lot help from https://towardsdatascience.com/hyperparameter-tuning-the-random-forest-in-python-using-scikit-learn-28d2aa77dd74"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37d5a9cf-df0f-4a36-b000-f9321b2054b0",
   "metadata": {},
   "source": [
    "# 3.3 Model Evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "id": "616f8d66-774b-4962-91b5-ca76528c48f0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy:  0.815\n",
      "Misclassification Rate:  0.185\n",
      "Sensitivity:  0.916\n",
      "Specificity:  0.714\n",
      "Precision:  0.762063227953411\n"
     ]
    }
   ],
   "source": [
    "preds = gs4.predict(X_test)\n",
    "tn, fp, fn, tp = confusion_matrix(y_test, preds).ravel()\n",
    "print(\"Accuracy: \", (tn + tp) / (tn + fp + fn + tp))\n",
    "print('Misclassification Rate: ', (fp+fn)/(tp+fp+tn+fn))\n",
    "\n",
    "print(\"Sensitivity: \", tp / (tp + fn))\n",
    "print(\"Specificity: \", tn / (tn + fp))\n",
    "print(\"Precision: \", tp / (tp + fp))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "5c1f8680-28fb-4159-85cd-8b5cdfb80817",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "357 143 42 458\n"
     ]
    }
   ],
   "source": [
    "print(tn, fp, fn, tp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "id": "e001e245-68d4-4f94-bd57-bba9e387a551",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Text(0, 0.5, 'Options'), Text(0, 1.5, 'Stocks')]"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAExCAYAAAByP2k/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABIjElEQVR4nO3dd3wUxRvH8c83hQ7SkSZgQaQooCgKKnZU/IkdBMWKBRS7InZF7A0rKmJFUUERbIgioiJNLCAqCiiC9F4CSZ7fH7vBI6TcQUIul+fNa1+5m92dnb07npubnZ2RmeGccy5xJRV1AZxzzhUuD/TOOZfgPNA751yC80DvnHMJzgO9c84lOA/0zjmX4DzQFwBJZSV9IGmVpLd3IJ9ukj4tyLIVBUkfSeqxnfveI2mppH8LulzbUZa5ko4uivwldZA0P+L5DEkdCqsssZA0RNI94eNDJf26nfk8K+nWgi2dy0mJCvSSzpY0RdJaSQvDgNS+ALI+HagFVDOzM7Y3EzN73cyOLYDybCUMGiZpeLb0/cL0cVHmc4ek1/LbzsyON7OXt6Oc9YFrgaZmtmus++eS58mSpktaHX6BjJXUsCDy3pnMrJmZjYPo3ofwS2RD+FlfJOklSRUKoVxfmdne+W0n6TxJE7Lte6mZ3V3QZXLbKjGBXtI1wGPAvQRBeTfgaeDkAsi+AfCbmaUXQF6FZQlwiKRqEWk9gN8K6gAK7MhnqgGwzMwWb8exU3JI2xN4heDLYxegEcF7nrkDZYz6+HHgJDOrALQG2gC3ZN8gTsvtCpqZJfxC8J98LXBGHtuUJvgiWBAujwGlw3UdgPkEAWMxsBA4P1x3J7AJ2Bwe40LgDuC1iLwbAgakhM/PA/4E1gBzgG4R6RMi9jsEmAysCv8eErFuHHA38HWYz6dA9VzOLav8zwK9wrTkMO02YFzEto8DfwOrganAoWF6x2zn+UNEOfqH5dgA7BmmXRSufwZ4JyL/+4GxgLKV8ehw/8ww/yFh+v+AGcDKMN99IvaZC9wI/AikZb2+EetPB6bn8Z4PAe7J/jply78vMBNYAbwElMn2mt4I/Au8SlBxugn4A1gGDAOqRuR3DjAvXNcvzP/ocF3ZsDwrwuNdn0NZjs7tfcjh3LbkHT5/EBgVPjagF/A7MCdM6wRMD1/nb4B9I/ZtBUwj+Jy9BbyZ9brl8JrVB4YTVCyWAU8C+wAbgYywzCtzef0vBmYDy4GRQJ2IdQZcGpZ5BfAU4WeI4DP3JcH/k6XAW0Udc+JtKfIC7JSTDP5zpGcPBNm2uQuYCNQEaoQf9rvDdR3C/e8CUoETgPVAlXD9HWwd2LM/bxh+UFOA8gRBdO9wXW2gWfj4PMJAD1QNP9DnhPt1DZ9XC9ePIwgojQmCxDjgvlzOrQNBUDoE+C5MOwH4BLiIrQN9d6BaeMxrCYJYmZzOK6IcfwHNwn1S2TrQlyP41XAecGj4H7FeXuWMeN4YWAccE+Z7QxgISoXr5xIEp/pA2Rzy250gwDwKHAFUyLZ+CPkH+p/D/KsSfJlFBrh0gi+u0uF7cBXBZ6hemPYcMDTcvilBkDssXPdIuH9WoL8P+Co8Tv3wuNsE+tzehxzOPXL7+gRfllmfZwPGhMcqS1DjXwwcRFAB6BHuXxooRfDldHX4HpxO8CWzTaAP9/0hfL3LA2WA9tk/2zm9/sCRBJ+N1uFxBwLjI7Y1YBRQmeDX+BKgY7huKMEXZ1LkMX35bykpTTfVgKWWd9NKN+AuM1tsZksIaurnRKzfHK7fbGYfEvynzbdtMheZQHNJZc1soZnNyGGbE4HfzexVM0s3s6HALOCkiG1eMrPfzGwDQe2xZV4HNbNvgKqS9gbOJWjWyL7Na2a2LDzmwwT/6fI7zyFmNiPcZ3O2/NYTfHk8ArwGXGFm83PKJAdnAaPNbEyY70MEgemQiG2eMLO/w9cg+7n8SRCI6hK8PkvDC4mxtFU/Gea/nOCXS9eIdZnA7WaWFh7/EqCfmc03szSCgHx62DxyOkGNeny47la2bkI6E+hvZsvN7G/giRjKmJv3JK0EJhDUeO+NWDcgPNYGgpr0c2b2nZllWHB9JQ1oGy6pwGPhZ/8dgl+XOTkQqANcb2brzGyjmU3IZdvsugGDzWxa+Pr0BQ7Odj3lPjNbaWZ/AV/w3+d9M0GzX50Yj1lilJRAvwyonk97ZB2CmkuWeWHaljyyfVGsB2K+uGVm6wgC2KXAQkmjJTWJojxZZaob8TyyZ0q05XkV6E1Qwx2RfaWkayX9EvYgWknQ7FU9nzz/zmulmU0iaKoSQcCN1lavgZllhseKfA3yO/ZEMzvTzGoQ/KI4jKD2F63I/LN/JpaY2caI5w2AEZJWhq/dLwTNFbXC/bbkFX4OlkXsu9V6tn3vt0dnM6tsZg3M7PJsX4aRx2oAXJtV7rDs9cMy1QH+MbPI0Q9zK1t9YF4+FarcZH+v1xK8PtF83m8g+GxNCnsnXbAdx09oJSXQf0vwE75zHtssIPjAZ9ktTNse6wiaLLJs1YPEzD4xs2MImm1mAc9HUZ6sMv2znWXK8ipwOfBhWNveQtKhBG3OZxI0S1UmaPdUVtFzyTPPIVAl9SL4ZbCA4D9ltLZ6DSSJIJhEvgZRD79qZpMJ2o+bh0l5vk+h+hGPs38msh/7b+D4MLhmLWXM7B+C6zpb8pJUjuCXZpat1ofHyvVU8lgXrcg8/ib4NRFZ7nLhr8iFQN3wtc+vbH8Du+VSocqvzNnf6/IEr0++n3cz+9fMLjazOgS/qp4OL8S7UIkI9Ga2iuCi41OSOksqJylV0vGSHgg3GwrcIqmGpOrh9vl2JczFdOAwSbtJ2oXgZygAkmpJ+l/4QU4jaALKyCGPD4HGYZfQFElnEbTzjtrOMgFgZnOAw8m5VluRoN14CZAi6TagUsT6RUDDWHrWSGoM3EPQfHMOcIOkllHuPgw4UdJRklIJrhmkEVw/iebY7SVdLKlm+LwJwcXdieEm04ETJFWVtCtBG3t2vSTVk1QVuJngYmRungX6S2oQHq+GpKxeXe8AncIylSK43hP5Og4D+kqqIqkecEUex4n5fcjH88Clkg4Ke06Vl3SipIoElaR04Mrwc3gqQRNNTiYRfDHcF+ZRRlK7iDLXC889J28A50tqKak0QTPTd2Y2N7/CSzojfM0guI5l5Px/qsQqEYEewMweAa4h6GK2hKD20Rt4L9zkHmAKQQ+Onwh6GdyznccaQxAQfiTouRIZnJMIAtYCgt4FhxPUsLPnsYygJ8S1BD9hbwA6mdnS7SlTtrwnmFlOv1Y+AT4iuHg6j+BXUORP/KybwZZJmpbfccKa3WvA/Wb2g5n9ThAsXw3/M+dXzl8JviAGElyoO4mgy+Cm/PYNrSQI7D9JWgt8TNBclfXl/irBxcO5BL2Wcgrib4Tr/gyXvD4TjxP0FvlU0hqCL5SDwnOZQdDT5Q2CYLiC4AJ5ljsJXvM54fFezeM4Mb0P+TGzKQTt9E+G5ZpNcPGU8LU+NXy+gqDZcXgu+WQQvEd7Elygnx9uD/A5wQXhfyVt8xk2s7EE1y3eJXh99gC6RHkKbYDvwvd4JNAnrNC4UFb3JOeccwmqxNTonXOupPJA75xzCc4DvXPOJTgP9M45l+A80DvnXILzQO+ccwnOA71zziU4D/TOOZfgPNA751yC80DvnHMJzgO9c84lOA/0zjmX4DzQO+dcgvNA75xzCc4DvXPOJTgP9M45l+A80DvnXILzQO+ccwnOA71zziU4D/TOOZfgUoq6AAWl0VWjfZZzt42hV7Qr6iK4ONR2j8rakf3LtuoddbzZ8P2TUR1LUjIwBfjHzDpJugO4GFgSbnKzmX0YbtsXuBDIAK40s0/yyjthAr1zzu00ScmFkWsf4BegUkTao2b2UORGkpoCXYBmQB3gM0mNzSwjt4y96cY552KlpOiXaLKT6gEnAi9EsfnJwJtmlmZmc4DZwIF57eCB3jnnYiVFv0TnMeAGIDNbem9JP0oaLKlKmFYX+Dtim/lhWq480DvnXKxiqNFL6ilpSsTSc6uspE7AYjObmu0ozwB7AC2BhcDDWbvkUKI8rxl4G71zzsUq+po6ZjYIGJTHJu2A/0k6ASgDVJL0mpl1/+9weh4YFT6dD9SP2L8esCCvMniN3jnnYlWAbfRm1tfM6plZQ4KLrJ+bWXdJtSM2OwX4OXw8EugiqbSkRsBewKS8juE1eueci1Xh9LrJ7gFJLQmaZeYClwCY2QxJw4CZQDrQK68eN+CB3jnnYhdD000szGwcMC58fE4e2/UH+kebrwd655yLVZTdJuOFB3rnnItVIdXoC4sHeueci5XX6J1zLsF5oHfOuQSXvFN63RQYD/TOORcrb6N3zrkE5003zjmX4LxG75xzCc5r9M45l+B2zhAIBcYDvXPOxcqbbpxzLsF5041zziU4r9E751yC8xq9c84lOA/0zjmX4LzXjXPOJThvo3fOuQRXzJpuildpnXMuHkjRL1FnqWRJ30saFT6vKmmMpN/Dv1Uitu0rabakXyUdl1/eHuidcy5GkqJeYtAH+CXi+U3AWDPbCxgbPkdSU6AL0AzoCDwtKc+LBh7onXMuRkpS1EtU+Un1gBOBFyKSTwZeDh+/DHSOSH/TzNLMbA4wGzgwr/w90DvnXIxiqdFL6ilpSsTSM4csHwNuADIj0mqZ2UKA8G/NML0u8HfEdvPDtFz5xVjnnItRLE0yZjYIGJRHXp2AxWY2VVKHaA6f02Hy2sEDvXPOxSjGtvf8tAP+J+kEoAxQSdJrwCJJtc1soaTawOJw+/lA/Yj96wEL8jqAN90451yMCvJirJn1NbN6ZtaQ4CLr52bWHRgJ9Ag36wG8Hz4eCXSRVFpSI2AvYFJex/AavXPOxWrn3C91HzBM0oXAX8AZAGY2Q9IwYCaQDvQys4y8MvJA75xzMUpKKpzGEDMbB4wLHy8Djsplu/5A/2jz9UDvnHMxKuA2+kLngd4552LkgX47SeoDvASsIbhpoBVwk5l9WqQFK0SVy6Xyeq+DAKhRsTQZmbB8XRoAnR/5ms0ZefaYisrQ3m0pVyqZkx/5GoAW9Xfh5pP3oeuTE3c4b1c4zut0MPUb7rHl+ZW3PkCNWnVy3LbnqR0YNHzcDh3v+UfuYtZP0yhXvgJSEudefj177tNih/JMeMUrzsdPoAcuMLPHw3EbagDnEwT+hA30K9dv5sQHJwDQp+NerE/L4Pkv/tyyPjlJZGTueLCvVrE0h+9Tgy9/WbLDebnCV6pUae5+8rWdeswuF15Bm/ZH8dO0ibw08D76P/36Tj1+ceM1+u2X9cqdALxkZj+ouL2aBeDBs/dl5frNNKu7CzPmr2JtWvpWXwAf33gYFz4/mX+Wb6Dz/nU577CGpKaI6fNWcuvbP5PT98Lzn/9J72P23CbQJwluPKkJB+1ZjdIpSbwyYR5Dv/kLCe48rTkH7VGVv5evJ0ni7e/+5qMf/t0ZL4HLZuOG9Tx+1/WsW7uGjPR0Tjv3EloffPhW26xcvpSnBvRj44Z1ZGRk0KPXDezdvBU/TZvIiNeeJ33zZmrWrstFV99KmbLlcj3W3s1bsXhhcNPlx8PfYPyYDwA4/Lj/cVznrqRt3MBTA25m+dLFZGZmcnKXCzjo8GMK7+TjVHELTfEU6KdK+hRoBPSVVJGtbwcuMRrVKE/3pyeSaUFNPyd71KpAp1a1Of3xb0jPNO46vTmdD6jL8Mn/bLPttLkrOLZFLdruWY11aelb0s9qW581G9Lp/MjXlEpO4u0+B/PVrCW0qL8L9aqWpeMD46lWoTSf9T2ct7/7e5t8XeHYtCmNW3t3B6B6rTr0vvlerrz1fsqWq8CaVSu565oLadX2sK2CzbfjPqHF/m35X5fzyczIIC1tI2tWrWTk0Je48d4nKV2mLKPffoWPR7xB57MvyvXY07+bQL0GezLn91/4aswobn90MGbGnVdfQJMWrVm88B8qV63BNXc+CsD6dWsL98WIU9GOYRMv4inQXwi0BP40s/WSqhE035Q4H05fmGPNPFK7varRvP4uvH9tOwDKpCazbG1arts/OWY2vY/dk/s/mLUl7dC9a9CkTkWOb7krABXLpNKoRnkO2L0qH05fiBksXZPGt78v2/GTclHL3nSTnp7O20Oe4defp5OUJFYsW8KqFcupXLXalm0a7dWUFx+7h4z0dFoffDgN9mjM9O8msODvOdxz3cVBPps359r2/uaLAxn55ktUrFSFC6/qx8zpk9n/kMMpXaYsAAcc0oFff57OvvsfzJsvPsFbg5+k5YHt2Lt5q0J8JeKX1+i3k5llSloENJUUN+UqChs2/XfvQ0aGEVl5KJ0S9N+VxLuT5/PgqF+jyvPb35dxzfGNadWg8n+JgjuGz2D8rKVbbXtE05q4+PHtFx+zZtVK7nziZVJSUrj2vM5s3rz1l3qTFq24+YFn+WHy1wx66A6OP7075StUpFmrA7n8xnvyPUZWG32WGdNzvtFy13q7cecTL/PD5G94e8jTNG99UJ6/EBJVcQv0cTMEgqT7ga+BW4Drw+W6Ii1UHJi/fAPN6u0CQLN6lahfLWhf/fq3pRy/X22qVSgFwC7lUqlbpWyeeT01ZjY9j/qvN8dXs5bQrV0DUsJvkkY1ylO2VDJT/lxOx/12RYLqFUrRds+qhXFqLkob1q2lUuUqpKSk8MsPU1i6eOE22yxdtJBKlavQoWNnDjvuJObNnsUeTZrz+8wfWbQgaHZL27iRf+f/FdUx927eiqnfjidt40bSNm5g6rdfsnfzlqxYtoRSpcvQ7sjjOf7U7sybHV1FI9EU0nj0hSaeas6dgb3NLPf2hxLoox8Xcmqbuoy+vj0//rWKOUvWATB70Voe/vBXXrnsQJIkNmcYt73zM/+s2JBrXuN+WcLytZu2PH9z4t/UrVqOD65rjySWr93EJS9O4aMf/+WQxtX55MbDmLNkHdP/WsmaDem55usK18FHdOTRO6/l9it7sNvujaldv+E228z6aRofvvsayckplClblp7X3kGlXapw8dW38cz9t7J582YATjv3Enatt1u+x2y4ZxMOPfpE7rw6aD09/Lj/0WCPvflp6kTefHEgSUkiOTmFHr1uLNBzLS7iJYBHS2Y73n2vIEj6CDjDzLbr6k6jq0bHx4kkiHKlklm/KYPK5VJ575p2nP74tyxdU/y+g4de0a6oi+DiUNs9Ku9QpK572Yio480/z5xS5N8K8VSjXw9MlzQW2BJRzOzKoitSyfVizzZUKptCanISAz+dXSyDvHOFpbjV6OMp0I8MFxcH/M5Z53LngX47mdnLkkoBjcOkX81sc1GWyTnnclS84nz8BPpwCq2XgbkEL2N9ST3MbHwRFivulEpJYtgVB1MqJYnkJPHRDwt57OPf6dNxL7q03W3LWDkPjvqVcb8s4eT969DzyN237N+kdiU6PTyBX/5ZXVSn4ArBC4/ezfRJX1OpchXufWboVus+fPc13npxIE8O/YSKu1Tmj19nMGTgAADMjM7dLuaAQzoUQamLL6/Rb7+HgWPN7FcASY2BocD+RVqqOLMpPZOzn5rI+k0ZpCSJt/sczLhwaIPBX87ZaqwcgPenLuD9qcEsY3vXrsigCw/wIJ+A2h/diaNPOoNBD9+5VfqyJYuY8f0kqtXYdUtavQZ7cMfjQ0hOTmHl8qXc0qs7rQ5qT3JyPIWD+FbcAn3c9KMHUrOCPICZ/QakFmF54tb68IaqlGSREsMECCe1rsMH0/KcWtIVU01atKJ8xUrbpL8x6FHOuqD3VoGpdJkyW4L65k2bKGYxKy4kJSVFvcSD+ChFYIqkFyV1CJfngalFXah4lCQYfX17ptxzDBN+W8r0eSsBOPfQBnx0w6Hc33VfKpXdtnbWqVVtRk7bdiwcl5imTRxPlWo12G33xtus+2PWz/S9tAv9Lj+bHr1v8tp8rBTDkl9WUhlJkyT9IGmGpDvD9Dsk/SNpericELFPX0mzJf0ajvibp3gK9JcBM4ArgT4E8yFemtcOknpKmiJpypqfPt4JRYwPmQYnPjiBg+8Yy367VabxrhV4fcI8Dr/7C0548CuWrEqjX+emW+3TskFlNmzK4Ld/S+YgVCVN2saNfPDmEE4955Ic1+/RpDkDnn2TOx57iVHDXmbTJu8+G4sCvjM2DTjSzPYjGO+ro6S24bpHzaxluHwYHrspwSTizYCOwNOSkvM6QNwEejNLM7NHzOxUMzvFzB7N7y5ZMxtkZgeY2QEVW3TcWUWNG2s2pDNx9jIO36cmS9duItPADIZO/Iv9dqu81badWtX2ZpsSZPHC+SxZtIBbe3Xn2vM6s3zpYm678lxWLt96gLo6uzWidJky/DP3z1xycjkpyEBvgawaWGq45HVD1snAm2HMnAPMBg7M6xhFHujD2cyR9JOkH7MvRV2+eFO1fCkqhs0ypVOTaN+4On8sWkuNSqW3bHNci135beGaLc8lOKFlbT743gN9SVG/0Z48OfRjHh7yHg8PeY+q1Wty1xOvULlqNZb8u4CMjGBIi6WLFvLv/L+oXqt2EZe4eJFiWf5reQiXntvmp2RJ04HFwBgz+y5c1TuMhYMlVQnT6gKR44bPD9NyFQ8Nc33Cv52KtBTFRM1KpXmo234kJwW1hdHTF/D5zMU80m0/9qkbXIybv3wDNw/7acs+B+5RlX9XbuTvZbmPg+OKt6fvv4VZP05j7eqVXHVOJ07p3pPDj/tfjtv+NmM6o95+hZSUlHDqwBuouEvlnVvgYi6WXjdmNggYlM82GUBLSZWBEZKaA88AdxPU7u8m6Jl4ATm3/Oc5JEM8jXVzv5ndmF9abnysG5cTH+vG5WRHx7rZ+8ZPoo43v95/XEzHknQ7sM7MHopIawiMMrPmkvoCmNmAcN0nwB1m9m1ueRZ5002EnOYjO36nl8I55/IRS9NN/nmpRliTR1JZ4GhglqTI9rRTgJ/DxyOBLpJKS2oE7AXkPIFAqMibbiRdBlwO7J6tTb4iwfj0zjkXV5IKdirB2sDLYc+ZJGCYmY2S9KqklgTNMnOBSwDMbEZ4bXMmkA70Cpt+clXkgR54A/gIGADcFJG+xsyWF02Rdr5YhzaItHvN8gzs8d+UbvWrlePRj37jpS/nsku5VJ7s0Yq6Vcvxz/L19BoyjdUb0tm/URXuOaM5aemZ9Hnle+YtXU/Fsik82aM1PZ7Ns3LgdrJYhjeItGlTGvfecCnpmzeRkZFBm/ZHcmr34Drg2jWreHrALSxdvIDqNevQq29/ylesxG8zfuDlpx4gNTWVy268m1p16rNu7Rqevq8f1939eLG7I7SwFOTLYGY/AtvMyWhm5+SxT3+gf7THKPJAb2argFVAV0mtgfYE32BfAyUm0Mc6tEGkPxev48QHJwDBzVQT7zyKT39cBMBlR+3B178t49mxk7j0qD247Ohg3tiLOuzOZYOnUq9qObq3a0D/93/hymP34ukxswv/ZF1MYhneIFJqailuGvAUZcqWIz09nf7X9WTfAw5mzyYtGD3sFZq2PIBOZ/Zg1LCXGfX2K5x1QW8+HvEGV/QbwNJFC/l89HC6XtyHkUMHc9JZ53mQj1DcXou4aaOXdCvBoGbVgOrAS5JuKdpS7VzbO7RBpHaNqzNv6fotM00d06IW706eD8C7k+dzbItaAKRnZlImNZmypZLZnJHJbtXKUWuXMnz3R4n5bi02YhneIJIkypQNpp7MSE8nIyMdhR02pk0cT/ujTwSg/dEnMu3bLwFITk5h06Y0NqVtJDklhUUL57Ni2WKatGhdGKdWbCUlKeolHhR5jT7C2UArM9sIIOk+YBqQ/8zGCSJJ8MF17WlQvTyvTpjH9HkrOXyfGpx7aANObVOXH/9eRf/3ZrI6j2n9OmUbz6Z6xdIsWR00+yxZnUa1CkF/+6c/+4N7z2rBxs2ZXPPadG4+eR8e+bBkzv9ZHOU1vEGkzIwMbu/Tg0UL5nNUp9PZo0lzAFavXE7lqtUBqFy1OqtXrQCg05k9GPLEAFJLl+aS6+7gzReeyPXu2pKsuNXo4ynQzwXKABvD56WBP4qsNEUga2iDimVTeO6CA7YMbTDwk98x4Nrj96Zf56bcODTn+8hSk8XRzWrx4Aez8j3WL/+s5tTHvgHgwN2rsmj1RiQxsEcr0jOM/u/NZGnE/LIufmQNb3B9/yfy3TYpOZm7n3yNdWvX8MQ9NzB/7h/Ua7hHrts32KMxtz06GIBZP31P5ao1wOCpAf1ITkmh60VXskuVagV2LsVVMYvz+TfdSDpDUsXw8S2Shodt6QUtDZghaYikIQRdidZKekJS/p/oBBLL0AaROuxTkxnzV20VoJeuSdty12yNSqVZtnbbUSV6H7snAz/5nSuP24tHP/qNEVP+4bzDGhX4ebmCEe3wBpHKV6hIkxb78+PUoKt1pcpVWbl8KQArly+l0i5VttrezBj55mBO7noB773xAqd0v5hDjujImJHDCu/EipECHuum0EXTEHyrma2R1B44jqAd/ZlCKMsnwH3AZGAi0I+gN85USsAoltsztEF2J7Wuw8hs49l89vMiTmtTD4DT2tRjzE+Ltlp/2oH1+HzmYlZvSKdsqWTMgv/kZUrFzeUbl01ewxtEWr1qBevWBp+XTWkbmTl9EnXqNQSgVdtDmfDZaAAmfDaa1m0P22rfCZ+NpmWbdpSvWIm0tI0kKQklibS0jbiC7Ue/M0TTdJPVP/NE4Bkze1/SHQVVAEkpwL0Et/bOI/jyqQ+8BNxcUqYTjHVog5qVSnNfl325YNBkAMqkJtF+7+r0ixj6AOCZz/7gyfNac2bb+ixYsYFeQ6ZtWVcmNYnT2tTj3GeCYTVeHPcnT5/fms0ZRp9Xvt8Zp+2iEMvwBiuWLWHw4/259q7HWLl8Kc8/fBeZmZmYZXLgoUfR8qD2AHQ6owdPDbiZ8Z+OpFqNXel1871b8kjbuJEJY0dz/T0DAeh4SlcG9r+J5NQULr+hxFwyy1O81NSjle8QCJJGAf8Q3K21P7ABmBQOqbnjBZAeJbg56mozWxOmVQIeAtab2VXR5ONDILic+BAILic7OgRCm/7joo43k/t1KPJvhWh+n59J0KzS0cxWAlWB6wuwDJ2Ai7OCPICZrSYYn/7EAjyOc84ViOLWdBNNoH/OzIab2e8AZrYQyPWOre1glsPPivCWXq+lO+fiTiJejG0W+SQcj6EgJ+yeKenc7ImSugP59xN0zrmdrLjV6HO9GBsOhXkzUFbS6qxkYBP5jK0co17AcEkXEPSuMaANUJZgxDbnnIsr8VJTj1augT4c63iApAFm1rewCmBm/wAHSTqS4NeDgI/MbGxhHdM553ZEMYvzedbom5jZLODtnG6QMrNpOey23czsc+DzgszTOecKQ7yMYROtvPrRXwtcTDB9VXYGHFkoJXLOuTiXSE03F4d/j9h5xXHOufiXMIFe0ql57Whmwwu+OM45F/+KWZzPs+nmpPBvTeAQ/ms/PwIYB3igd86VSAVZo5dUBhhPMGJvCvCOmd0uqSrwFtCQYHTfM81sRbhPX+BCgiFqrjSzT/I6Rl5NN+eHGY4CmoY3ShFOWPvUDp2Zc84VYwV8MTYNONLM1kpKBSZI+gg4FRhrZvdJuolgqtUbJTUFuhD0UqwDfCapcV7zxkZzw1TDrCAfWgTkPduBc84lsIK8YcoCa8OnqeFiwMkEowUT/u0cPj4ZeNPM0sxsDjAbODCvY0QzeuU4SZ8AQ8ODdwG+iGI/55xLSEkxNN1I6gn0jEgaZGaDsm2TTHDD6J7AU2b2naRaWZVsM1soqWa4eV2CodyzzA/TcpVvoDez3uGF2UMjCjkiv/2ccy5RxdJEHwb1PEcTCJtdWkqqDIyQ1Dyvw+eURV75RzWVYNjDxi++Oucchde90sxWShoHdAQWSaod1uZrA4vDzeYTzNmRpR6w9YxD2eTaRi9pjaTVuS07djrOOVd8JSn6JT+SaoQ1eSSVJZj7YxYwEugRbtYDeD98PBLoIqm0pEbAXsCkvI6RV6+brHli7wL+BV4l+MnQjWCiEOecK5EKuNdNbeDlsJ0+CRhmZqMkfQsMk3Qh8BdwBoCZzZA0DJgJpAO98upxA9E13RxnZgdFPH9G0nfAA7Gfj3POFX/KsZl8+5jZj0CrHNKXAUflsk9/oH+0x4ime2WGpG6SkiUlSerGf/PIOudciVOQTTc7QzSB/myC6QQXhcsZYZpzzpVIxW2GqWi6V84l6KDvnHOO4jfWTb41ekmNJY2V9HP4fF9JtxR+0ZxzLj4lJynqJR5E03TzPNAX2AxbLhx0KcxCOedcPEu4phugnJlNylbg9EIqj3POxb04id9RiybQL5W0B+EttpJOBxbmvYtzziWuWMa6iQfRBPpeBOM0NJH0DzCH4KYp55wrkYpXmM8n0Id3al1mZkdLKg8kmdmanVM055yLT/HS9h6tPAO9mWVI2j98vG7nFMk55+JbvPSmiVY0TTffSxoJvA1sCfY+Z6xzrqQqZhX6qAJ9VWAZcGREmuHDFjvnSqiEarqB/+aOdc45FyhmLTdR3Rm7u6QPJC2RtFjS++EYyM45VyIVtxumorkz9g1gGMGYyXUI2urfLMxCOedcPFMMSzyIJtDLzF41s/RweY185id0zrlEVtzGusm1jV5S1fDhF5JuIqjFG3AWMHonlM055+JSvDTJRCuvi7FTCQJ71hldErHOgLsLq1DOORfPCjLOS6oPvALsCmQCg8zscUl3ABcDS8JNbzazD8N9+gIXEkwCdaWZfZLXMfKaM9YvuDrnXA4KeKybdOBaM5smqSIwVdKYcN2jZvZQ5MaSmhKMINyM4LrpZ5Ia5zVvbL7dKyWdm1O6mb0S5Uk451xCKcg4b2YLCQeKNLM1kn4B6uaxy8nAm2aWBsyRNBs4EPg2tx2iuWGqTcTjMgST1U4j+KkRN3556MSiLoKLQ1Xa9C7qIrg4tOH7J3do/+QYIr2knkDPiKRBZjYol20bEkwU/h3QDugdVranENT6VxB8CUyM2G0+eX8xRHXD1BXZCrIL8Gp++znnXKKK5WJsGNRzDOzZ8qwAvAtcZWarJT1DcC0065row8AF5NxrM8+ekNHU6LNbD+y1Hfs551xCKOhek5JSCYL861njiJnZooj1zwOjwqfzgfoRu9cDFuSVfzRt9B/w37dFMrAPwQ1UzjlXIhVkoFfw8+BF4BczeyQivXbYfg9wCvBz+Hgk8IakRwguxu4FTMrrGNHU6COv+KYD88xsfnSn4JxziaeA+9G3A84BfpI0PUy7GegqqSVBRXsuYRd3M5shaRgwkyAm98qrxw1E10b/JYCkasBhwEaCnw7OOVciFWSN3swmkHO7+4d57NMf6B/tMXIdAkHSKEnNw8e1CX42XAC8KumqaA/gnHOJprgNgZDXWDeNzCyrTeh8YIyZnQQcRBDwnXOuREqKYYkHeZVjc8Tjowh/RoRzxmYWZqGccy6eSdEv8SCvNvq/JV1B0B7fGvgYQFJZIHUnlM055+JSAQ+BUOjyqtFfSDCWwnnAWWa2MkxvC7xUuMVyzrn4lTA1ejNbDFyaQ/oXwBeFWSjnnItncXKNNWrbc2esc86VaPHSmyZaHuidcy5GxSzOe6B3zrlYKW5mg41OXlMJDiSPEdHM7MpCKZFzzsW5RKrRT9lppXDOuWIkYQK9mb28MwvinHPFRcJdjJVUA7gRaEowwxQAZnZkIZbLOefiVrz0j49WNEMxvA78AjQC7iQYLnNyIZbJOefiWpIU9RIPogn01czsRWCzmX1pZhcQ3B3rnHMlUpKiX+JBNN0rswY3WyjpRIIpq+oVXpGccy6+xUlFPWrRBPp7wgnBrwUGApWAqwu1VM45F8eSilk/+nybbsxslJmtMrOfzewIM9vfzEbujMI551w8Sk6KfsmPpPqSvpD0i6QZkvqE6VUljZH0e/i3SsQ+fSXNlvSrpOPyO0Y0vW5eIocbp8K2euecK3EK+CJrOnCtmU2TVBGYKmkMwcjBY83sPkk3ATcBN0pqCnQhGF24DvCZpMZ5zRsbTdPNqIjHZQhmI1+wXafjnHMJoCDjvJktBBaGj9dI+gWoC5wMdAg3exkYR9DV/WTgTTNLA+ZImg0cCHyb2zGimRz83cjnkoYCn8V4Ls45lzAKq9ukpIZAK+A7oFb4JYCZLZRUM9ysLjAxYrf5YVqutmdKw72A3bZjP+ecSwixTDwiqaekKRFLz5zzVAXgXeAqM1ud1+FzSMt1XDKIro1+TbZM/iX4+eCccyVSLDVkMxsEDMprG0mpBEH+dTMbHiYvklQ7rM3XBhaH6fOB+hG71yOf5vRoet1UNLNKEUvj7M05zjlXkhTknbGSBLwI/GJmj0SsGgn0CB/3AN6PSO8iqbSkRgStLJPyLG8UhRgbTZpzzpUUBTwEQjvgHOBISdPD5QTgPuAYSb8Dx4TPMbMZwDBgJvAx0CuvHjeQ93j0ZYByQPWw/2ZWiSsRdOlxzrkSqSAvxZrZhDyyPCqXffoD/aM9Rl5t9JcAVxEE9akRBVkNPBXtAZxzLtEkzBAIZvY48LikK8xs4E4sk3POxTUVs0gfzcXjTEmVs55IqiLp8sIrknPOxbdkKeolHkQT6C82s5VZT8xsBXBxoZXIOefinGJY4kE0QyAkSZKZGYCkZKBU4RbLOefiV3Fruokm0H8CDJP0LMGNU5cSdOlxzrkSaXuGFChK0QT6G4GewGUEv0Q+BZ4vzEI551w8S7gavZllAs+GC5LaE0xA0qtwi5bYWrXYh732arzl+aMDn6Ju3Zwn7mp7QCsmTvl+h45368038e23X/PhJ2MpVaoUK1Ys5+wzT+ejMZ/vUL6ucFTdpTwfPncFALWqVSIzM5MlK9YCcGj3B9mcnuf9MVH55Pk+7Fq9Ehs3bWbd+jQuueN1fp+3OP8dXdy0vUcrmho9kloCXYGzgDnA8Dx3cPkqXboMw4a/n/+GBSg5KZn3hr/DmV3O3qnHdbFbvmodbbvcB0C/S05g3fo0Hnv1vxvSk5OTyMjI3OHjnN/vZabN/IsLTm3HvVefwhlXPbfDeZYE8dKbJlp53RnbmGBw+67AMuAtQGZ2RGEXSlISUCGfEdwSyvp16+hzxeWsXr2a9PR0el/ZhyOOPHqrbZYsWcwN117NurVrSc/I4Jbb7qD1/gfwzdcTeOapgWzatIn69etz1z0DKFe+/DbH6HZOD1595WVOPf3MbdYNGfwCn378EZs2b+LIo47h8t5XAvDcM0/x4egP2HXX2lSuXIWmzZrR4/wLC+dFcHkadGd3Vqxez35712P6rL9Zsy5tqy+AKW/fzKlXPstfC5fT5YQ29Op6OKmpKUz+aS59BrxFZmbuAxxOmDab3t06AHDvVZ05tl1TzOD+Fz7mnU+nsWv1Srx6/wVULF+GlOQk+tz7Fl9//8fOOO24VMzifJ41+lnAV8BJZjYbQFKhzRUr6Q2CC70ZBHfi7iLpETN7sLCOWZTS0jZy5qknA1CnXj0eeuRxHn3iKSpUqMCKFcs5p+tZdDjiqK3aAj8cPYpD2rXn4ksuIyMjg40bN7BixXKef+4ZnnvhJcqVK8fgFwbxyssvcenlvbc5Zu3atWnVujWjPnifwzv89339zdcT+GvePF5/6x3MjCt7X8bUKZMpU6YMY8d8ylvvvEdGRjpdTj+Vps2aFf6L43K15241OeHSgWRmGv0uOSHHbfZuVIvTj23NEec/Qnp6Jo/1PZMuJ7ThjVG5j3t14mHNmfH7Ajof1ZJ9967HgWcNoHrlCkx47XomTJvNWccfwJhvfuGBFz8hKUmUK1OyO96pmDXe5BXoTyOo0X8h6WPgTQq3aaqpma2W1A34kOAi8FQgIQN99qabzZs388RjjzBt6mSSlMTixYtYtnQp1WvU2LJN8+YtuP2Wm0lPT+eII4+myT77MGXyF/z5x2zO6951Sz77tmyZ63EvuvhS+vS+jEMP67Al7dtvvubbb77mrNM6A7B+/XrmzZvL+nXr6HDkUZQpUwaAwzoU+o85l4/hn32fZ80c4IgD96Z1092Y8NoNAJQtncqS5Wtz3Pal/j3YkLaZvxYs45r73+bK7kcy7OMpZGYai5ev4aups9m/WQOmzJjHc7d3JzUlmQ+++IEff/unwM+tOEmYGr2ZjQBGSCoPdAauBmpJegYYYWafFnBZUsMxmTsDT5rZ5uJ2ZXtHfDjqA1asWM7QYcNJTU3l+GOOJG1T2lbb7H9AGwa/8hpfffkl/frewHnnX0jFSpVoe3A77n/okVxy3tpuDRqwd5N9+PTjj7akmRkXXNyTM87sstW2r748ZIfPyxWs9Rv++0ykZ2SQlPTf/5EypVKBoEfIax98x20DR+abX1YbfZbc/s99Pe0PjrnoMTq2b8aL9/Tg0Vc+y/MXQqJLKmY1+mjGo19nZq+bWSeCAe6nE0xSW9CeA+YC5YHxkhoAqwrhOHFp7do1VK1ajdTUVCZ9N5EFC7atMS1Y8A9Vq1bjtDPO5JRTT+OXmTPYd7+WTP9+Gn/NmwfAhg0bmDt3Tp7HuuiSS3llyOAtzw9p1573hr/L+nXrAFi0aBHLli2jVevWfDnuC9LS0li/bh1fjR9XcCfsdti8BctpuU8w/0TLJvVoWLcaAF9M+pVTjm5JjSoVAKhSqRy71a4SVZ4Tps3m9GP3JylJVK9Sgfb778mUn+eyW+0qLF6+hpdGfMPL731Dqyb1888sgSUlRb/Eg6h63WQxs+UEAbkwLs0/Z2ZPZD2R9BdB7b5EOKHTSVzZ6zK6nnkqezfZh0a7777NNlMmTWLISy+SkpJCuXLluGfA/VStWpW7+g/gpuuvYdPmTQD0vuIqGjZslOux9txzL5o0bcqsmTOBINDP+fMPzukW1OjLlSvHvfc9SPMW+9LhiCM549T/UbtOXZo1a06FChUL4ezd9nhv7HS6dTqQiW/exNQZ87Z0jZz157/c+dQoPnimN0kSm9MzuPq+Yfy1cEW+eb7/+Q8ctG8jJr3VFzPo99h7LFq2hm4nHcTV5x7F5vQM1q1P48JbXy3s04trxa2NXuHIBkVO0mjgZDNLD5/XBkaZ2f7R7L8xPe85E932Wb9uHeXKl2fDhg1c0KMbt91xN/s0LT4XZKu02faitHMbvn9yhyL12FlLo443RzWpXuTfCjHV6AvZe8A7kk4jmA9xJHBdkZbIcdcdt/HnH7NJ25TG/04+pVgFeecKS3Gr0cdNoDez5yWVIgj4DYFLzOybIi2U474HHy7qIjgXdwqyn4ikwUAnYLGZNQ/T7iAYJXhJuNnNZvZhuK4vcCFBV/QrzeyT/I5R5IFe0jWRTwlq89OBtpLaZpss1+UgIyODrmeeRs1atXjy6ed45KH7+XLcF6SmplKv/m7cdc8AKlWqVNTFdDtBUpL4+vUbWLB4Faf1eZZ+l5zABacesmX4hNufHMknE2aSkpLEM7d1o2WT+qQkJ/H66Ek8NLigO9IlrgKu0Q8BngReyZb+qJk9tNVxpaYE3d6bEcz+95mkxvnNGRsP14QrRiwVgBHA7Ig0l4/XX32F3XffY8vztge34933RvHOiA9o0KAhLz7vt7WXFL3PPoJf5yzaKm3ga1/Qtst9tO1yH59MCC7An3Z0a0qXSqHNmfdySLf7uei0duxWu2pRFLlYKsiJR8xsPLA8ykOfDLxpZmlmNocgVh6Y305FHujN7M68lqIuX7xb9O+/fDV+HKecdvqWtEPatSclJfixtu9+LVm86N+iKp7bierWrEzH9s14aUT+LZ6GUa5MKZKTkyhbuhSbNmewZt3GnVDKxCBFv+yA3pJ+lDRYUlb/2LrA3xHbzA/T8lTkgT6LpDE5TFmYb9tTSffAffdy9bXXk5RLh933hr9Lu0MP28mlckXhwetPo9/j721z5+ylXQ5j0lt9efb2blSuWBYI7rBdv3ETc8b057eP7uKxV8ayYvX6oih2sRTLDFOSekqaErH0jOIQzwB7AC2BhUDWxbKcvjry7QEUN4EeqJHDlIU1i6448e/LcV9QtWpVmjZrnuP65597huSUZE7s9L+dXDK3sx1/aHMWL1/D97/8vVX6829/RdOT7uCgLvfx79LV3HfNqQC0adaQjIxMdj+2H/uceDt9zjlyyw1XLn9JUtSLmQ0yswMilkH55W9mi8wsIxwm/nn+a56ZT3AdM0s9YEG+5d2ekywkGZJ2y3oS3hmb5zdV5Dfli8/n+9olnOnfT2PcuM85/pgjufG6a5j83UT63hj0SB353gjGfzmOAfc/VOwmSXCxO7jl7nQ6vAWzRt/JK/edT4c2jRl8z7ksXr6GzEzDzBg8/GsOaN4AgDOPP4BPv5lJenowzv230/9k/6a75XMUl6Ww54wN7yPKcgrwc/h4JNBFUmlJjYC9gHzHoijyXjcR+gETJH0ZPj+MYGarXIXfjIOgZN4w1efqa+lz9bUATJ70HS8PGcyA+x/i66/G89KLz/Piy69RtmzZIi6l2xluGzhyy9g2h+6/F1edexQX3PIKu1avxL9Lg9G+Tz5yP2b+sRCA+f8up0ObvRk6ejLlypTiwH0b8uQbXxRZ+Yudgu1eORToAFSXNB+4HegQzgNiBEPDXAJgZjMkDQNmAulAr/x63EAcBXoz+1hSa6BtmHS1mS0tyjIVVwP6382mzZu49KLzAWix337cevtdRVwqVxT69+nMvnvXw8yYt3A5V9wzFIBn3xrPoDu7M/Wdfkjw6vsT+fn3fFsAXCipAH8lm1nXHJJfzGP7/kD/WI4RN0MgAEj6H0FNHmCcmY2Kdt+SWKN3+fMhEFxOdnQIhMl/roo63rTZfZcibzuNmzZ6SfcBfQh+kswE+kgaULSlcs65HBR2I30Bi5umG+AEoGV4lRlJLwPfA32LtFTOOZeNj3WzYyrz3x1iuxRhOZxzLlfFrSNb3DTdAAOA7yUNCWvzU8O0EisjI4MzT+tM78sv2Wbd6FEjOf2Ukzj9lJM4t1sXfp01a8u6r78az/9OPI5OHY8hstvpow8/yOmnnES/vjdsSftg5Hu8/urLhXsiboclJYlvh97Iu49fCkC/S07gj0/uYeKbNzHxzZs4rn3THPe7otsRTH2nH1PevpmXB5xH6VJB3a5KpXKMeqY3P71/G6Oe6b3lRqqD99udSW/1ZcJr17N7/eoA7FKhLCOf6rUTzrL42El3xhaYuAn0ZjaUoMfN8HA5OEwrsbKPYROpbt16DB7yGu+M+ICel17GXXfcCgRfDvf2v4unn32BESNH8/GHo/hj9mzWrFnDD9O/550RH5CZkcHvv/3Kxo0bGfneCM7scvbOPC23HaIdwyZSnRq7cHnXw2nX7QEOOONekpOSOOO4YHqH684/hnGTfqXFyXcxbtKvXHf+sQD0OedIul7/ArcN/ICeZxwKQN+eHXlgsN+kHkkx/IsHcRPoJY01s4VmNtLM3jezfyWNLepyFZWcxrCJ1LJVayrtErRu7btvSxaF49n8/NOP1K/fgHr165NaqhQdTziRcV+MJSlJbN68GTNjY1oaKSkpDBn8Amd3P4fU1NSddl4udrGMYZNdSnIyZUunBmPalCnFwiXB7JydOuzLax98B8BrH3zHSUfsC8Dm9AzKlk6lXNlUNqdn0KhederUrMyEqbML7oQSgNfoYySpjKSqBDcLVJFUNVwaEgzDWSLlN4ZNpBHD36F9OJ7N4kWL2LX2rlvW1axVi0WLFlG+fAWOPuZYzjqtM3Xr1qNCxYrM+Plnjjjy6EI7B1cwYhnDJtKCJat47JWx/PbR3cwZ05/VazcwdmLQxFezWsUtN1L9u3Q1NaoGA8U+OPhTnrqlK73PPoJn3xzPnb1P4s6no+7lXGIUs043RR/oCe74mgo0Cf9OCZf3gaeKsFxFJr8xbCJN+m4iI4a/w1XXBEMfWA63E2QNgXD+hRczbPj7XHfDTTw18HEuv+JKhr/zNtdf04dBzz5dsCfhCkSsY9hEqlyxLJ06tGCfTrez+7H9KF+2FF1OaJPn8X787R8O7/EwHXs+QcN61Vi4ZBVCvHrf+Qy+51xqVvWRw4FiF+njIdB/AxwCXGdmuwN3Eozr8CXwRlEWrKjkNYZNpN9+ncWdt9/CYwOfpnLlYBTTWrV25d+F/w1LvHjRImrW3HpsuF9+CdpzGzRoyAcj3+PBRx5n9uzfmTdvbuGdlNsusY5hE+nIg5owd8Eylq5YS3p6Ju99/gNt9wsmjV+8bA27Vg8mo9m1eiWWLF+zzf43XdSRAYM+ot8lx3P3sx8y9MPJXN61Q6Geb3HhbfSxew5IM7OBkg4j6GnzMrCKcBybkqbP1dcy5vPxfDTmc+5/6BHaHNSWAfdvNdEMCxcs4Jo+V9B/wAM0bNhoS3qz5i3466+5zJ//N5s3beLjD0dz+BFHbrXvUwMf5/LeV5Kenk5mRjBMRpKS2LjBxyOPN7cNHMmeHW+lyYm3c+5NLzFu8m9bxrDJEjmGTaS//13OgS0aUbZMcA3miAP33nJBd/SXP9H9pIMA6H7SQYwa9+NW+3Y/6SA+/moGK9dsoFyZUmRmGpmZRrkyfj0HIEnRL/EgHvrRJ5tZVt/5s4BBZvYu8K6k6UVXrPgz7K2gE9KZZ3XluWefYuWqldx7950AJKckM3TYcFJSUujb7zYu63kRmZkZdD7lNPbcc68teXw+9jOaN29BzZq1ANi3ZStO63wSjRs3Zu8mTXb+SbntktsYNrVr7MLTt53NKVc8w+Sf5zHis+/59o0bSc/I5IdZ83nx3a8BeOilMbx2/wX06Hwwfy9cQbcb/htapWyZVLqfdBCdLn8SgCde+5yhD13Eps3p9Og7ZKefa1yKkwAerSIf60bSzwR3xKZLmgX0DKfWQtLPWZPl5sfHunE58bFuXE52dKybGf+sizreNKtbvsi/FuKhRj8U+FLSUmAD8BWApD0Jmm+ccy6uxEu3yWgVeaA3s/5hf/nawKf230+MJOCKoiuZc87lrJjF+aIP9ABmNjGHtN+KoizOOZevYhbp4yLQO+dccVKQE4/sDPHQvdI554qVgrxfStJgSYvDjilZaVUljZH0e/i3SsS6vpJmS/pV0nHRlNcDvXPOxapg74wdAnTMlnYTMNbM9gLGhs+R1BToAjQL93laUnJ+B/BA75xzMSrIO2PD7uTLsyWfTHDjKOHfzhHpb5pZmpnNAWYDB+Z3DA/0zjkXo50wemUtM1sIEP7NGsekLhA58NH8MC1PHuidcy5GsQR6ST0lTYlYeu7IoXNIy/fmLe9145xzMYplsDIzG0Ts43YtklTbzBZKqg0sDtPnA/UjtqsHLMgvM6/RO+dcjHZC081IoEf4uAfBsO1Z6V0klZbUCNgLmJRfZl6jd865GBVkL3pJQ4EOBJMvzQduB+4Dhkm6EPgLOAPAzGZIGgbMBNKBXmaWkd8xPNA751ysCjDSm1nXXFYdlcv2/YH+sRzDA71zzsUoXiYUiZYHeueci1G8TCgSLQ/0zjkXo2I21I0Heueci13xivQe6J1zLkZeo3fOuQRXzOK8B3rnnIuV1+idcy7BqZhFeg/0zjkXo+IV5j3QO+dczIpZhd4DvXPOxcrvjHXOuURXvOK8B3rnnIuVD4HgnHMJzptunHMuwRW3i7E+w5RzziU4r9E751yMiluN3gO9c87FqKDb6CXNBdYAGUC6mR0gqSrwFtAQmAucaWYrtid/b7pxzrkYJSn6JQZHmFlLMzsgfH4TMNbM9gLGhs+3r7zbu6NzzpVYimHZficDL4ePXwY6b29GHuidcy5GiuFflAz4VNJUST3DtFpmthAg/Ftze8vrbfTOORejWC7GhoG7Z0TSIDMblG2zdma2QFJNYIykWTteyv94oHfOuRjF0iITBvXsgT37NgvCv4sljQAOBBZJqm1mCyXVBhZvb3m96cY552JVgG30kspLqpj1GDgW+BkYCfQIN+sBvL+9xfUavXPOxSipYDvS1wJGhJOZpABvmNnHkiYDwyRdCPwFnLG9B5CZFUhJXfyQ1DOHNkBXwvnnouTyppvE1DP/TVwJ5J+LEsoDvXPOJTgP9M45l+A80Ccmb4d1OfHPRQnlF2Odcy7BeY3eOecSnAd655xLcB7o44ikepLel/S7pD8kPS6pVB7bV5Z0ecTzOpLe2TmldUVBUj9JMyT9KGm6pIMkXSWp3Hbmd4ek6wq6nC6+eKCPEwpuixsOvBeOP90YqAD0z2O3ysCWQG9mC8zs9MIspys6kg4GOgGtzWxf4Gjgb+AqYLsCvSsZPNDHjyOBjWb2EoCZZQBXAxdIujys6X8s6VdJt4f73AfsEdbsHpTUUNLPAJLKSHpJ0k+Svpd0RJh+nqThYV6/S3ogTE+WNETSz+E+V+/0V8Dlpzaw1MzSAMxsKXA6UAf4QtIXAJK6hu/hz5Luz9pZUkdJ0yT9IGls9swlXSzpI0llJV0paWb4y+HNnXN6rrD4WDfxoxkwNTLBzFZL+ovgfToQaA6sByZLGk0w40xzM2sJIKlhxO69wjxaSGpCMNZ143BdS6AVkAb8KmkgwVjXdc2seZhX5YI/RbeDPgVuk/Qb8Bnwlpk9IekagtmJlkqqA9wP7A+sIHjfOwNfA88Dh5nZnHCaui0k9SYYTKuzmaVJugloFD6uvLNO0BUOr9HHDxFMPpBb+hgzW2ZmGwiaeNrnk1974FUAM5sFzCNoDoJgerJVZrYRmAk0AP4Edpc0UFJHYPWOnpArWGa2liCA9wSWAG9JOi/bZm2AcWa2xMzSgdeBw4C2wHgzmxPmtTxin3OA44HTsn4tAD8Cr0vqDqQX0im5ncQDffyYARwQmSCpElCfYMLg7F8C+d0AkdfwemkRjzOAlHDS4f2AcQS/Bl7Iv8huZzOzDDMbZ2a3A72B07Jtktv7nltFAoIhcRsC9SLSTgSeIvhimSrJf/0XYx7o48dYoJykcyFoMwceBoYQNNccI6mqpLIEc0d+TTBrfMVc8hsPdAvzagzsBvya28ElVQeSzOxd4Fag9Y6fkitIkvaWtFdEUkuCX2qRn4PvgMMlVQ8/Q12BL4Fvw/RGYV6RTTffA5cAI8OeW0lAfTP7AriB4KJ/hUI7MVfo/Fs6TpiZSToFeFrSrQRfwh8CNxP8Z51A0BSzJ8F41VMAJH0dXoD9iKAGluVp4FlJPxH89D4vbG/NrQh1gZfC/+QAfQv0BF1BqAAMDNvM04HZBM04XYGPJC00syMk9QW+IKjFf2hm78OWKe2Gh+/xYuCYrIzNbELYzXI0QVv9a5J2CfN41MxW7qRzdIXAh0AoBsJ22APMrHdRl8U5V/x4041zziU4r9E751yC8xq9c84lOA/0zjmX4DzQO+dcgvNA75xzCc4DvXPOJTgP9M45l+A80DvnXILzQO+ccwnOA71zziU4D/TOOZfgPNA751yC80DvnHMJzgO9c84lOA/0zjmX4DzQu21IypA0XdLPkt6WVG4H8hoi6fTw8QuSmuaxbQdJh2zHMeaGUyFmT79A0k+SfgzP5eR88jlP0pOxHj+P/O4IZ23Knt4wnBUMSQdIeiJ8vF3n71x+fCpBl5MNZtYSQNLrwKXAI1krJSWbWUasmZrZRfls0gFYC3wTa97ZSaoH9ANam9kqSRWAGgWQrwjmccjc0bwAwikhp4RPO1BA5+9cJK/Ru/x8BewZ1ja/kPQG8JOkZEkPSpoc1pgvgSAQSnpS0kxJo4GaWRlJGifpgPBxR0nTJP0gaaykhgRfKFeHvyYOlVRD0rvhMSZLahfuW03Sp5K+l/Qcwbym2dUkmDR7LYCZrTWzOTmUo7qkuRH71Zf0saRfJd0ebtNQ0i+SngamhdtcH3Hud0acY79w38+AvSPS9w/P9VugV0R6B0mjcjn/M8JfIj9IGh/zO+dcyGv0LleSUoDjgY/DpAOB5mY2J5xoepWZtZFUGvha0qdAK4IA1wKoBcwEBmfLtwbwPHBYmFdVM1su6VlgrZk9FG73BsHE1BMk7QZ8AuwD3A5MMLO7JJ1IMEF2dj8Ai4A5ksYCw83sgyhO+0CgObAemBx+WS0Nz+l8M7tc0rHAXuG2AkZKOgxYB3QJX4MUgi+FqWG+LwFXmNmXkh7MflAzm5vD+f8EHGdm/yiYENy57eKB3uWkrKTp4eOvgBeBQ4BJWbVi4Fhg36z2d2AXguB3GDA0bNpZIOnzHPJvC4zPysvMludSjqOBpkFrCQCVJFUMj3FquO9oSSuy72hmGZI6Am2Ao4BHJe1vZnfkc+5jzGwZgKThQHvgPWCemU2MOPdjge/D5xXCc68IjDCz9eH+I8O/uwCVzezLcPtXCb5A8/M1METSMGB4FNs7lyMP9C4nW9ros4TBdl1kEkEN9ZNs250A5DcRsaLYBoKmxYPNbEMOZcl3fwsmRJ4ETJI0hqBWfQeQzn/NlmWy75bL8+znPsDMnstWrqtyKVe057v1gc0ulXQQcCIwXVLLrC8h52LhbfRue30CXCYpFUBSY0nlgfFAl7ANvzZwRA77fgscLqlRuG/VMH0NQa04y6dA76wnklqGD8cD3cK044Eq2Q8gqY6k1hFJLYF54eO5wP7h49PZ2jGSqkoqC3QmqFXndO4XhBd4kVRXUs2wXKdIKhv+8jgJwMxWAqsktQ/375ZDnpDt/CXtYWbfmdltBM1H9XPZz7k8eY3eba8XgIbANAVV7CUEgXEEcCTwE/Ab8GX2Hc1sSdjGP1xSErAYOAb4AHhHQTfIK4Argack/UjwWR1PcMHyTmCopGlh/n/lUL5U4CFJdYCNYfkuDdc9BAyTdA6QvWlpAkHTyp7AG2Y2JbxQGln+TyXtA3wb/rpYC3Q3s2mS3gKmE3ypfBWx2/nAYEnrCb4ocpL9/K+WtBfBL4KxBNcdnIuZgl+3zjnnEpU33TjnXILzQO+ccwnOA71zziU4D/TOOZfgPNA751yC80DvnHMJzgO9c84lOA/0zjmX4P4PYjVSMtNXjv0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "group_names = ['True Neg','False Pos','False Neg','True Pos']\n",
    "\n",
    "group_counts = [\"{0:0.0f}\".format(value) for value in\n",
    "                cf_matrix.flatten()]\n",
    "\n",
    "group_percentages = [\"{0:.2%}\".format(value) for value in\n",
    "                     cf_matrix.flatten()/np.sum(cf_matrix)]\n",
    "\n",
    "labels = [f\"{v1}\\n{v2}\\n{v3}\" for v1, v2, v3 in\n",
    "          zip(group_names,group_counts,group_percentages)]\n",
    "\n",
    "labels = np.asarray(labels).reshape(2,2)\n",
    "\n",
    "ax = sns.heatmap(cf_matrix, annot=labels, fmt='', cmap='Blues')\n",
    "ax.set_title('Confusion Matrix for Subreddit Predictions\\n');\n",
    "ax.set_xlabel('\\nPredicted Subreddits')\n",
    "ax.set_ylabel('Actual Subreddits\\n');\n",
    "ax.xaxis.set_ticklabels(['Options','Stocks'])\n",
    "ax.yaxis.set_ticklabels(['Options','Stocks'])\n",
    "\n",
    "# with help from https://www.stackvidhya.com/plot-confusion-matrix-in-python-and-why/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "55fbe8bf-b4ac-4ab3-92f6-73e70932c947",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "748                            Price data at 3:45 v 4pm ?\n",
       "929                    Trading during such uncertain time\n",
       "1248                   Mildly Educational Memes ( Day 1 )\n",
       "489                                Podcast recommendation\n",
       "860                     Fed Rate increase v Impact to Rho\n",
       "                              ...                        \n",
       "154           Do you recommend sticking to a few ticker ?\n",
       "640                      What just happened with AFRM ? ?\n",
       "50                                Free money at no cost !\n",
       "821     The last time Americans liked their president ...\n",
       "2707                       Quick Tax / wash sale question\n",
       "Name: lemma_title, Length: 185, dtype: object"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#which were misclassified\n",
    "X_test[y_test != preds]\n",
    "\n",
    "#this wasn't too helpful.  Going to build a new dataframe and do some more analytics on the misclassified titles if there is enough time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "fa16c2c7-81e7-48ff-838f-8d6489d37b51",
   "metadata": {},
   "outputs": [],
   "source": [
    "# New dataframe to hopefully understand some of the misclassifications\n",
    "predictions = pd.DataFrame({\n",
    "    'Title' : X_test, \n",
    "    'Actual_Reddit' : y_test, \n",
    "    'Predicted_Reddit' : gs.predict(X_test),\n",
    "    'Prob_Options' : [i[0] for i in gs.predict_proba(X_test)],\n",
    "    'Prob_Stocks' : [i[1] for i in gs.predict_proba(X_test)]})\n",
    "\n",
    "predictions['Corr_Incorr'] = np.where(predictions['Actual_Reddit'] == predictions['Predicted_Reddit'], 'Correct', 'Incorrect')\n",
    "predictions['Difference'] = np.abs(predictions['Prob_Options'] - predictions['Prob_Stocks'])\n",
    "\n",
    "# with help from https://stackoverflow.com/questions/19913659/pandas-conditional-creation-of-a-series-dataframe-column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "eb87747b-a41f-42b9-9b13-04ca4f2ac990",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>Actual_Reddit</th>\n",
       "      <th>Predicted_Reddit</th>\n",
       "      <th>Prob_Options</th>\n",
       "      <th>Prob_Stocks</th>\n",
       "      <th>Corr_Incorr</th>\n",
       "      <th>Difference</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>748</th>\n",
       "      <td>Price data at 3:45 v 4pm ?</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.547205</td>\n",
       "      <td>0.452795</td>\n",
       "      <td>Correct</td>\n",
       "      <td>0.094410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>252</th>\n",
       "      <td>I need help deciding my exit strategy for an S...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.981710</td>\n",
       "      <td>0.018290</td>\n",
       "      <td>Correct</td>\n",
       "      <td>0.963420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1069</th>\n",
       "      <td>Goodmorning . What is the better option beginn...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.972364</td>\n",
       "      <td>0.027636</td>\n",
       "      <td>Correct</td>\n",
       "      <td>0.944729</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2886</th>\n",
       "      <td>What is the best ETF ?</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.174399</td>\n",
       "      <td>0.825601</td>\n",
       "      <td>Correct</td>\n",
       "      <td>0.651202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>902</th>\n",
       "      <td>$ BITO put are being dumped on the bid and cal...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.924253</td>\n",
       "      <td>0.075747</td>\n",
       "      <td>Correct</td>\n",
       "      <td>0.848506</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  Title  Actual_Reddit  \\\n",
       "748                          Price data at 3:45 v 4pm ?              0   \n",
       "252   I need help deciding my exit strategy for an S...              0   \n",
       "1069  Goodmorning . What is the better option beginn...              0   \n",
       "2886                             What is the best ETF ?              1   \n",
       "902   $ BITO put are being dumped on the bid and cal...              0   \n",
       "\n",
       "      Predicted_Reddit  Prob_Options  Prob_Stocks Corr_Incorr  Difference  \n",
       "748                  0      0.547205     0.452795     Correct    0.094410  \n",
       "252                  0      0.981710     0.018290     Correct    0.963420  \n",
       "1069                 0      0.972364     0.027636     Correct    0.944729  \n",
       "2886                 1      0.174399     0.825601     Correct    0.651202  \n",
       "902                  0      0.924253     0.075747     Correct    0.848506  "
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c0fa630-1939-47ed-ba07-dffb3e18822b",
   "metadata": {},
   "source": [
    "**Previous Model Scores**\n",
    "<br>Below we tracked each of the testing score as we tried different models and data including the selftext (post) content through all of the models.  The scores on the selftext data are all lower but 15% of the selftext data was null.  We believe that given more time, we could optimize around the selftext data.  It should be noted that these scores changed each time the model ran until we finally set a RandomState for each model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "5ecc484f-6506-4218-9646-42da94b27c07",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scores based on Original Title\n",
    "# LogReg Score: 0.82\n",
    "# Random Forest Score: 0.829\n",
    "# KNeighbors Score: 0.751\n",
    "\n",
    "# LogReg Best Params: {'lr__max_iter': 1000, 'vect__ngram_range': (1, 2), 'vect__stop_words': None}\n",
    "# Random Forest Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}\n",
    "# KNeighbors Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': 'english'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "028b034d-9d06-4e2d-aab8-f2594236fb84",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scores based on Lemma Title\n",
    "# LogReg Score: 0.828\n",
    "# Random Forest Score: 0.835\n",
    "# KNeighbors Score: 0.737\n",
    "\n",
    "# LogReg Best Params: {'lr__max_iter': 1000, 'vect__ngram_range': (1, 2), 'vect__stop_words': None}\n",
    "# Random Forest Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}\n",
    "# KNeighbors Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': 'english'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "id": "0d55065a-fe13-4515-b88a-6d560dcc87b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scores based on Stemma Title\n",
    "# LogReg Score: 0.828\n",
    "# Random Forest Score: 0.833\n",
    "# KNeighbors Score: 0.737\n",
    "\n",
    "# LogReg Best Params: {'lr__max_iter': 1000, 'vect__ngram_range': (1, 2), 'vect__stop_words': None}\n",
    "# Random Forest Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}\n",
    "# KNeighbors Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': 'english'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "0e543ab9-8fb4-4934-a685-1fe7697f240d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scores based on Original Selftext \n",
    "# LogReg Score: 0.769\n",
    "# Random Forest Score: 0.771\n",
    "# KNeighbors Score: 0.648\n",
    "\n",
    "# LogReg Best Params: {'lr__max_iter': 1000, 'vect__ngram_range': (1, 1), 'vect__stop_words': None}\n",
    "# Random Forest Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': 'english'}\n",
    "# KNeighbors Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "id": "5ad9b51f-8f20-4e88-8f35-bfeb3ab449b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scores based on Lemma Selftext \n",
    "# LogReg Score: 0.769\n",
    "# Random Forest Score: 0.77\n",
    "# KNeighbors Score: 0.662\n",
    "\n",
    "# LogReg Best Params: {'lr__max_iter': 1000, 'vect__ngram_range': (1, 1), 'vect__stop_words': 'english'}\n",
    "# Random Forest Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}\n",
    "# KNeighbors Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "id": "069ae269-e4bf-4b2f-823f-cd6cd4afa987",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scores based on Stemma Selftext \n",
    "# LogReg Score: 0.769\n",
    "# Random Forest Score: 0.772\n",
    "# KNeighbors Score: 0.662\n",
    "\n",
    "# LogReg Best Params: {'lr__max_iter': 1000, 'vect__ngram_range': (1, 1), 'vect__stop_words': 'english'}\n",
    "# Random Forest Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}\n",
    "# KNeighbors Best Params: {'vect__ngram_range': (1, 1), 'vect__stop_words': None}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f619b60-f3b8-473e-9a84-a505f64e7026",
   "metadata": {},
   "source": [
    "**Recommendations**\n",
    "<br>The first recommendation to improve the model would be to gather data from discrete periods of time, rather by the number of posts.   As time goes on, the social, political, and economic conditions that affect the market and topics being discussed on social media will change. The topics that affect stocks and options will overlap but not always as the prices are controlled by different factors. And since the number of posts per day is drastically different for r/Stocks vs r/Options, if we pulled data from the last week, rather than the 500 newest posts, we could probably do a better job of predicting which subreddit the post came from.  This article from Merril does a pretty good job explaining how options are priced https://www.merrilledge.com/investment-products/options/options-pricing-valuation \n",
    "<br>\n",
    "<br>\n",
    "For all of the reasons mentioned above, I would also try to include the actual date of the post as a feature.   Similarly, I would like to examine the text and dates of the comments as features in the model.  Almost certainly some particular equities are more popular and being discessed with greater frequency at any given point in time.  Given that the titles are short, there may be more clues in the comments as to which subreddit a particular post came from. \n",
    "<br>\n",
    "<br>\n",
    "Given more time, a few other things I would like to try include identifying the ticker symbols being discussed using RegEx, more detailed analysis on sentiment and stop words, and identifying correlations between authors, commentors, and the subreddit of origin.\n",
    "<br>\n",
    "<br> \n",
    "Lastly, I would use GridSearchCV, rather than RandomSearchCV, to find the best combination of hyperparameters.  Originally, GridSearchCV was attempting more than 59,000 different combinations.  For the sake of time, this plan was abandoned."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57b17187-7f89-491b-80bd-494e9b1e356b",
   "metadata": {},
   "source": [
    "\n",
    "**Conclusions**\n",
    "<br> While there are a number of posts in foreign languages, many with no selftext, and a few different tactics we would like to employ in order to improve accuracy, the model is still accurate about 82% of the time in predicting the origin subreddit based on the lemmatized title of the post. This represents an improvement above the baseline of 32%."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db04b994-8521-43f8-8c57-127f9fbec566",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5748a463-fe5f-427b-80fd-a11d4af7dee4",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}