Session #6 Homework.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-JrenrSGmAeg"
   },
   "source": [
    "**HOMEWORK**  \n",
    "\n",
    "The goal of this homework is to create a regression model for predicting housing prices (column 'median_house_value').  \n",
    "\n",
    "In this homework we'll again use the California Housing Prices dataset - the same one we used in homework 2 and 3.\n",
    "\n",
    "You can take it from [Kaggle](https://www.kaggle.com/datasets/camnugent/california-housing-prices) or download using wget link mentioned below:\n",
    "\n",
    "```\n",
    "wget https://raw.githubusercontent.com/alexeygrigorev/datasets/master/housing.csv\n",
    "\n",
    "```\n",
    "\n",
    "> Note: sometimes your answer doesn't match one of the options exactly. That's fine. \n",
    "Select the option that's closest to your solution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting xgboost\n",
      "  Downloading xgboost-1.6.2-py3-none-win_amd64.whl (125.4 MB)\n",
      "Requirement already satisfied: numpy in c:\\users\\darkcorp\\anaconda3\\lib\\site-packages (from xgboost) (1.21.5)\n",
      "Requirement already satisfied: scipy in c:\\users\\darkcorp\\anaconda3\\lib\\site-packages (from xgboost) (1.7.3)\n",
      "Installing collected packages: xgboost\n",
      "Successfully installed xgboost-1.6.2\n"
     ]
    }
   ],
   "source": [
    "!pip install xgboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "nLosiCNJUjDM"
   },
   "outputs": [],
   "source": [
    "#@ IMPORTING LIBRARIES AND DEPENDENCIES:\n",
    "import re\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import xgboost as xgb\n",
    "import matplotlib.pyplot as plt\n",
    "from tqdm.auto import tqdm\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn.tree import export_text\n",
    "from sklearn.feature_extraction import DictVectorizer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the data\n",
    "Use only the following columns:  \n",
    "* `'latitude'`,\n",
    "* `'longitude'`,\n",
    "* `'housing_median_age'`,\n",
    "* `'total_rooms'`,\n",
    "* `'total_bedrooms'`,\n",
    "* `'population'`,\n",
    "* `'households'`,\n",
    "* `'median_income'`,\n",
    "* `'median_house_value'`,\n",
    "* `'ocean_proximity'`  \n",
    "\n",
    "\n",
    "* Fill NAs with 0.\n",
    "* Apply the log tranform to `median_house_value`.\n",
    "* Do train/validation/test split with 60%/20%/20% distribution. \n",
    "* Use the `train_test_split` function and set the `random_state` parameter to 1.\n",
    "* Use `DictVectorizer` to turn the dataframe into matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "id": "4Psk224nnyIF"
   },
   "outputs": [],
   "source": [
    "#@ DOWNLOADING THE DATASET: UNCOMMENT BELOW:\n",
    "#!wget https://raw.githubusercontent.com/alexeygrigorev/datasets/master/housing.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "492Hn6-3n08G",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "      <th>ocean_proximity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-122.23</td>\n",
       "      <td>37.88</td>\n",
       "      <td>41.0</td>\n",
       "      <td>880.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>322.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>8.3252</td>\n",
       "      <td>452600.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-122.22</td>\n",
       "      <td>37.86</td>\n",
       "      <td>21.0</td>\n",
       "      <td>7099.0</td>\n",
       "      <td>1106.0</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>1138.0</td>\n",
       "      <td>8.3014</td>\n",
       "      <td>358500.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-122.24</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1467.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>496.0</td>\n",
       "      <td>177.0</td>\n",
       "      <td>7.2574</td>\n",
       "      <td>352100.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1274.0</td>\n",
       "      <td>235.0</td>\n",
       "      <td>558.0</td>\n",
       "      <td>219.0</td>\n",
       "      <td>5.6431</td>\n",
       "      <td>341300.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1627.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>259.0</td>\n",
       "      <td>3.8462</td>\n",
       "      <td>342200.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20635</th>\n",
       "      <td>-121.09</td>\n",
       "      <td>39.48</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1665.0</td>\n",
       "      <td>374.0</td>\n",
       "      <td>845.0</td>\n",
       "      <td>330.0</td>\n",
       "      <td>1.5603</td>\n",
       "      <td>78100.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20636</th>\n",
       "      <td>-121.21</td>\n",
       "      <td>39.49</td>\n",
       "      <td>18.0</td>\n",
       "      <td>697.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>356.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>2.5568</td>\n",
       "      <td>77100.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20637</th>\n",
       "      <td>-121.22</td>\n",
       "      <td>39.43</td>\n",
       "      <td>17.0</td>\n",
       "      <td>2254.0</td>\n",
       "      <td>485.0</td>\n",
       "      <td>1007.0</td>\n",
       "      <td>433.0</td>\n",
       "      <td>1.7000</td>\n",
       "      <td>92300.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20638</th>\n",
       "      <td>-121.32</td>\n",
       "      <td>39.43</td>\n",
       "      <td>18.0</td>\n",
       "      <td>1860.0</td>\n",
       "      <td>409.0</td>\n",
       "      <td>741.0</td>\n",
       "      <td>349.0</td>\n",
       "      <td>1.8672</td>\n",
       "      <td>84700.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20639</th>\n",
       "      <td>-121.24</td>\n",
       "      <td>39.37</td>\n",
       "      <td>16.0</td>\n",
       "      <td>2785.0</td>\n",
       "      <td>616.0</td>\n",
       "      <td>1387.0</td>\n",
       "      <td>530.0</td>\n",
       "      <td>2.3886</td>\n",
       "      <td>89400.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20640 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       longitude  latitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0        -122.23     37.88                41.0        880.0           129.0   \n",
       "1        -122.22     37.86                21.0       7099.0          1106.0   \n",
       "2        -122.24     37.85                52.0       1467.0           190.0   \n",
       "3        -122.25     37.85                52.0       1274.0           235.0   \n",
       "4        -122.25     37.85                52.0       1627.0           280.0   \n",
       "...          ...       ...                 ...          ...             ...   \n",
       "20635    -121.09     39.48                25.0       1665.0           374.0   \n",
       "20636    -121.21     39.49                18.0        697.0           150.0   \n",
       "20637    -121.22     39.43                17.0       2254.0           485.0   \n",
       "20638    -121.32     39.43                18.0       1860.0           409.0   \n",
       "20639    -121.24     39.37                16.0       2785.0           616.0   \n",
       "\n",
       "       population  households  median_income  median_house_value  \\\n",
       "0           322.0       126.0         8.3252            452600.0   \n",
       "1          2401.0      1138.0         8.3014            358500.0   \n",
       "2           496.0       177.0         7.2574            352100.0   \n",
       "3           558.0       219.0         5.6431            341300.0   \n",
       "4           565.0       259.0         3.8462            342200.0   \n",
       "...           ...         ...            ...                 ...   \n",
       "20635       845.0       330.0         1.5603             78100.0   \n",
       "20636       356.0       114.0         2.5568             77100.0   \n",
       "20637      1007.0       433.0         1.7000             92300.0   \n",
       "20638       741.0       349.0         1.8672             84700.0   \n",
       "20639      1387.0       530.0         2.3886             89400.0   \n",
       "\n",
       "      ocean_proximity  \n",
       "0            NEAR BAY  \n",
       "1            NEAR BAY  \n",
       "2            NEAR BAY  \n",
       "3            NEAR BAY  \n",
       "4            NEAR BAY  \n",
       "...               ...  \n",
       "20635          INLAND  \n",
       "20636          INLAND  \n",
       "20637          INLAND  \n",
       "20638          INLAND  \n",
       "20639          INLAND  \n",
       "\n",
       "[20640 rows x 10 columns]"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#@ READING DATASET:\n",
    "PATH = 'https://raw.githubusercontent.com/alexeygrigorev/datasets/master/housing.csv'\n",
    "select_cols = [\"longitude\", \"latitude\", \"housing_median_age\", \"total_rooms\", \"total_bedrooms\", \"population\", \"households\", \n",
    "               \"median_income\", \"median_house_value\", \"ocean_proximity\"]\n",
    "df = pd.read_csv(PATH, usecols=select_cols)\n",
    "df.total_bedrooms = df.total_bedrooms.fillna(0)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "longitude             0\n",
       "latitude              0\n",
       "housing_median_age    0\n",
       "total_rooms           0\n",
       "total_bedrooms        0\n",
       "population            0\n",
       "households            0\n",
       "median_income         0\n",
       "median_house_value    0\n",
       "ocean_proximity       0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OqXppJ_Ro0H0"
   },
   "source": [
    "- Apply the log transform to `median_house_value`. \n",
    "- Do train/validation/test split with 60%/20%/20% distribution.\n",
    "- Use the `train_test_split` function and set the `random_state parameter` to 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "      <th>ocean_proximity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-122.23</td>\n",
       "      <td>37.88</td>\n",
       "      <td>41.0</td>\n",
       "      <td>880.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>322.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>8.3252</td>\n",
       "      <td>13.022766</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-122.22</td>\n",
       "      <td>37.86</td>\n",
       "      <td>21.0</td>\n",
       "      <td>7099.0</td>\n",
       "      <td>1106.0</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>1138.0</td>\n",
       "      <td>8.3014</td>\n",
       "      <td>12.789687</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-122.24</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1467.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>496.0</td>\n",
       "      <td>177.0</td>\n",
       "      <td>7.2574</td>\n",
       "      <td>12.771673</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1274.0</td>\n",
       "      <td>235.0</td>\n",
       "      <td>558.0</td>\n",
       "      <td>219.0</td>\n",
       "      <td>5.6431</td>\n",
       "      <td>12.740520</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1627.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>259.0</td>\n",
       "      <td>3.8462</td>\n",
       "      <td>12.743154</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   longitude  latitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0    -122.23     37.88                41.0        880.0           129.0   \n",
       "1    -122.22     37.86                21.0       7099.0          1106.0   \n",
       "2    -122.24     37.85                52.0       1467.0           190.0   \n",
       "3    -122.25     37.85                52.0       1274.0           235.0   \n",
       "4    -122.25     37.85                52.0       1627.0           280.0   \n",
       "\n",
       "   population  households  median_income  median_house_value ocean_proximity  \n",
       "0       322.0       126.0         8.3252           13.022766        NEAR BAY  \n",
       "1      2401.0      1138.0         8.3014           12.789687        NEAR BAY  \n",
       "2       496.0       177.0         7.2574           12.771673        NEAR BAY  \n",
       "3       558.0       219.0         5.6431           12.740520        NEAR BAY  \n",
       "4       565.0       259.0         3.8462           12.743154        NEAR BAY  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.median_house_value = np.log1p(df.median_house_value)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "id": "W1MmdcRDpHS1"
   },
   "outputs": [],
   "source": [
    "#@ SPLITTING THE DATASET FOR TRAINING AND TEST:\n",
    "df_full_train, df_test = train_test_split(df, test_size=0.2, random_state=1)\n",
    "df_train, df_val = train_test_split(df_full_train, test_size=0.25, random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = df_train.reset_index(drop=True)\n",
    "df_val = df_val.reset_index(drop=True)\n",
    "df_test = df_test.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = df_train.median_house_value.values\n",
    "y_val = df_val.median_house_value.values\n",
    "y_test = df_test.median_house_value.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "del df_train['median_house_value']\n",
    "del df_val['median_house_value']\n",
    "del df_test['median_house_value']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>ocean_proximity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-118.18</td>\n",
       "      <td>34.63</td>\n",
       "      <td>19.0</td>\n",
       "      <td>3562.0</td>\n",
       "      <td>606.0</td>\n",
       "      <td>1677.0</td>\n",
       "      <td>578.0</td>\n",
       "      <td>4.1573</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-116.90</td>\n",
       "      <td>34.69</td>\n",
       "      <td>10.0</td>\n",
       "      <td>337.0</td>\n",
       "      <td>102.0</td>\n",
       "      <td>108.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>0.4999</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-118.26</td>\n",
       "      <td>33.78</td>\n",
       "      <td>27.0</td>\n",
       "      <td>1672.0</td>\n",
       "      <td>491.0</td>\n",
       "      <td>1723.0</td>\n",
       "      <td>462.0</td>\n",
       "      <td>2.0458</td>\n",
       "      <td>NEAR OCEAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-117.33</td>\n",
       "      <td>34.17</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4718.0</td>\n",
       "      <td>1140.0</td>\n",
       "      <td>2564.0</td>\n",
       "      <td>1056.0</td>\n",
       "      <td>2.9877</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-119.02</td>\n",
       "      <td>35.35</td>\n",
       "      <td>42.0</td>\n",
       "      <td>1239.0</td>\n",
       "      <td>251.0</td>\n",
       "      <td>776.0</td>\n",
       "      <td>272.0</td>\n",
       "      <td>1.9830</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12379</th>\n",
       "      <td>-118.19</td>\n",
       "      <td>34.06</td>\n",
       "      <td>32.0</td>\n",
       "      <td>555.0</td>\n",
       "      <td>159.0</td>\n",
       "      <td>748.0</td>\n",
       "      <td>163.0</td>\n",
       "      <td>1.9762</td>\n",
       "      <td>&lt;1H OCEAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12380</th>\n",
       "      <td>-122.30</td>\n",
       "      <td>37.90</td>\n",
       "      <td>37.0</td>\n",
       "      <td>2125.0</td>\n",
       "      <td>489.0</td>\n",
       "      <td>912.0</td>\n",
       "      <td>462.0</td>\n",
       "      <td>2.9219</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12381</th>\n",
       "      <td>-118.36</td>\n",
       "      <td>33.90</td>\n",
       "      <td>18.0</td>\n",
       "      <td>3380.0</td>\n",
       "      <td>922.0</td>\n",
       "      <td>2276.0</td>\n",
       "      <td>854.0</td>\n",
       "      <td>4.0727</td>\n",
       "      <td>&lt;1H OCEAN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12382</th>\n",
       "      <td>-119.79</td>\n",
       "      <td>36.78</td>\n",
       "      <td>41.0</td>\n",
       "      <td>2227.0</td>\n",
       "      <td>462.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>415.0</td>\n",
       "      <td>2.3190</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12383</th>\n",
       "      <td>-117.62</td>\n",
       "      <td>34.02</td>\n",
       "      <td>9.0</td>\n",
       "      <td>4265.0</td>\n",
       "      <td>587.0</td>\n",
       "      <td>2280.0</td>\n",
       "      <td>589.0</td>\n",
       "      <td>5.5632</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>12384 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       longitude  latitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0        -118.18     34.63                19.0       3562.0           606.0   \n",
       "1        -116.90     34.69                10.0        337.0           102.0   \n",
       "2        -118.26     33.78                27.0       1672.0           491.0   \n",
       "3        -117.33     34.17                 5.0       4718.0          1140.0   \n",
       "4        -119.02     35.35                42.0       1239.0           251.0   \n",
       "...          ...       ...                 ...          ...             ...   \n",
       "12379    -118.19     34.06                32.0        555.0           159.0   \n",
       "12380    -122.30     37.90                37.0       2125.0           489.0   \n",
       "12381    -118.36     33.90                18.0       3380.0           922.0   \n",
       "12382    -119.79     36.78                41.0       2227.0           462.0   \n",
       "12383    -117.62     34.02                 9.0       4265.0           587.0   \n",
       "\n",
       "       population  households  median_income ocean_proximity  \n",
       "0          1677.0       578.0         4.1573          INLAND  \n",
       "1           108.0        50.0         0.4999          INLAND  \n",
       "2          1723.0       462.0         2.0458      NEAR OCEAN  \n",
       "3          2564.0      1056.0         2.9877          INLAND  \n",
       "4           776.0       272.0         1.9830          INLAND  \n",
       "...           ...         ...            ...             ...  \n",
       "12379       748.0       163.0         1.9762       <1H OCEAN  \n",
       "12380       912.0       462.0         2.9219        NEAR BAY  \n",
       "12381      2276.0       854.0         4.0727       <1H OCEAN  \n",
       "12382      1129.0       415.0         2.3190          INLAND  \n",
       "12383      2280.0       589.0         5.5632          INLAND  \n",
       "\n",
       "[12384 rows x 9 columns]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5VKq8FX7qIVM"
   },
   "source": [
    "- We will use `DictVectorizer` to turn train and validation into matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "id": "WnfFrWe9qtuc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 578.  ,   19.  ,   34.63, ..., 1677.  ,  606.  , 3562.  ],\n",
       "       [  50.  ,   10.  ,   34.69, ...,  108.  ,  102.  ,  337.  ],\n",
       "       [ 462.  ,   27.  ,   33.78, ..., 1723.  ,  491.  , 1672.  ],\n",
       "       ...,\n",
       "       [ 854.  ,   18.  ,   33.9 , ..., 2276.  ,  922.  , 3380.  ],\n",
       "       [ 415.  ,   41.  ,   36.78, ..., 1129.  ,  462.  , 2227.  ],\n",
       "       [ 589.  ,    9.  ,   34.02, ..., 2280.  ,  587.  , 4265.  ]])"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#@ IMPLEMENTATION OF DICTVECTORIZER:\n",
    "train_dicts = df_train.to_dict(orient='records')\n",
    "dv = DictVectorizer(sparse=False)\n",
    "X_train = dv.fit_transform(train_dicts)\n",
    "X_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qvxmhaD5hmAK"
   },
   "source": [
    "## Question 1\n",
    "\n",
    "Let's train a decision tree regressor to predict the `median_house_value` variable.\n",
    "\n",
    "Train a model with `max_depth=1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "id": "cqBcSbpIq7kJ"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(max_depth=1)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#@ TRAINING THE REGRESSION MODEL:\n",
    "dt = DecisionTreeRegressor(max_depth=1)\n",
    "dt.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "val_dicts = df_val.to_dict(orient='records')\n",
    "X_val = dv.transform(val_dicts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "id": "sLAnqRwuikqQ"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|--- ocean_proximity=INLAND <= 0.50\n",
      "|   |--- value: [12.31]\n",
      "|--- ocean_proximity=INLAND >  0.50\n",
      "|   |--- value: [11.61]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#@ INSPECTION:\n",
    "print(export_text(dt, feature_names=dv.get_feature_names()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JNBm-qeRp6aZ"
   },
   "source": [
    "Which feature is used for splitting the data?\n",
    "\n",
    "* **`ocean_proximity=INLAND`**\n",
    "* `total_rooms`\n",
    "* `latitude`\n",
    "* `population`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Xhnbdy-CqQoB"
   },
   "source": [
    "## Question 2\n",
    "\n",
    "Train a random forest model with these parameters:\n",
    "\n",
    "* `n_estimators=10`\n",
    "* `random_state=1`\n",
    "* `n_jobs=-1`  (optional - to make training faster)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "id": "2w-YSGYMisFt"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(n_estimators=10, n_jobs=-1, random_state=1)"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#@ TRAINING RANDOM FOREST MODEL:\n",
    "rf = RandomForestRegressor(n_estimators=10,\n",
    "                           random_state=1,\n",
    "                           n_jobs=-1)\n",
    "rf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "id": "jCxUAO2Rq1Ps"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train: 0.11\n",
      "val: 0.25\n"
     ]
    }
   ],
   "source": [
    "#@ CALCULATING MEAN SQUARED ERROR:\n",
    "y_pred = rf.predict(X_train)\n",
    "rmse = np.sqrt(mean_squared_error(y_train, y_pred))\n",
    "print('train:', rmse.round(2))\n",
    "\n",
    "y_pred = rf.predict(X_val)\n",
    "rmse = np.sqrt(mean_squared_error(y_val, y_pred))\n",
    "print('val:', rmse.round(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jfcDllHxrTqu"
   },
   "source": [
    "What's the RMSE of this model on validation?\n",
    "\n",
    "* 0.05\n",
    "* **0.25**\n",
    "* 0.55\n",
    "* 0.85"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y7K69bFSrg_V"
   },
   "source": [
    "## Question 3\n",
    "\n",
    "Now let's experiment with the `n_estimators` parameter\n",
    "\n",
    "* Try different values of this parameter from 10 to 200 with step 10.\n",
    "* Set `random_state` to `1`.\n",
    "* Evaluate the model on the validation dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "id": "-oUFCQwOrQNN"
   },
   "outputs": [],
   "source": [
    "#@ TRAINING THE RANDOM FOREST MODEL:\n",
    "scores = []\n",
    "n_estimators = [*range(10, 201, 10)]\n",
    "\n",
    "for n in n_estimators:\n",
    "    rf = RandomForestRegressor(n_estimators=n,\n",
    "                               random_state=1,\n",
    "                               n_jobs=-1)\n",
    "    rf.fit(X_train, y_train)\n",
    "\n",
    "    y_pred = rf.predict(X_val)\n",
    "    rmse = np.sqrt(mean_squared_error(y_val, y_pred))\n",
    "    \n",
    "    scores.append((n, rmse))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "id": "3J-wFLh4s8Ky"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x23cf87b5be0>]"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD6CAYAAACoCZCsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAl8klEQVR4nO3de3hc9X3n8fd3ZiRZsiTLlmVLtuQLGGxsY4MRhpakxCQkckNw2iY8UEPZBMrjdGlJs20gm6dss2zb3DZNnxbCuoQmLUmI05KEpRAgBZMlLhcZX7DxTdjGlm0sWb5Jvkga6bt/zJEZxiNrZF3OSPN5Pc88mvM7v3P0nSP7fObczd0REZHcEwm7ABERCYcCQEQkRykARERylAJARCRHKQBERHKUAkBEJEdlFABmVmdm28yswczuSzN+uZltDF5rzGxhyvioma0zs6dS2v84mO9mM/v6wD6KiIj0R6yvDmYWBR4ErgcagdfN7El3fyup2y7gWnc/YmZLgZXAVUnj7wG2AKVJ810CLAMWuHu7mU3qq5aJEyf6jBkz+v5UIiJyxtq1aw+5e0Vqe58BACwGGtx9J4CZPU5ixX0mANx9TVL/V4DqngEzqwY+DvwV8IWkfp8Dvuru7cE8mvoqZMaMGdTX12dQsoiI9DCzd9K1Z7ILaCqwN2m4MWjrzR3AM0nD3wa+CHSn9LsY+KCZvWpmL5nZlelmZmZ3mVm9mdU3NzdnUK6IiGQikwCwNG1p7x8R7Na5A7g3GL4BaHL3tWm6x4DxwNXAnwOrzOys3+XuK9291t1rKyrO2oIREZHzlEkANAI1ScPVwP7UTma2AHgEWObuLUHzNcCNZrYbeBy4zsweS5rvE57wGokthInn9SlERKTfMgmA14GLzGymmeUDNwNPJncws2nAE8Bt7r69p93dv+Tu1e4+I5juBXe/NRj9M+C6YPqLgXzg0MA+joiIZKrPg8DuHjezu4FngSjwqLtvNrMVwfiHgfuBcuChYC9O3N1r+5j1o8CjZrYJ6ABud92aVERk2NhIWufW1ta6zgISEekfM1ub7ku5rgQWEclROREAL25t4jur3w67DBGRrJITAfDrhkN8+5fb6eoeObu7RESGWk4EwJyqUtrj3exuORF2KSIiWSM3AqCyBICtB1pDrkREJHvkRADMmlRMxGDbu8fDLkVEJGvkRACMyYtyQUUxW97VFoCISI+cCABI7Abaqi0AEZEzcioA9h4+RVt7POxSRESyQg4FQOJZNNu0G0hEBMilAKgKzgTSbiARESCHAmBqWSElBTGdCioiEsiZADAzZutAsIjIGTkTAJDYDbT13VZG0h1QRUSGSm4FQGUprafj7D92OuxSRERCl2MB0HNLCO0GEhHJqQC4uCcAdCqoiEhuBUDpmDyqxxcqAEREyLEAgMRxAO0CEhHJyQAoYeehE7THu8IuRUQkVBkFgJnVmdk2M2sws/vSjF9uZhuD1xozW5gyPmpm68zsqTTT/pmZuZlNPP+Pkbk5VSV0dTsNTW3D8etERLJWnwFgZlHgQWApMBe4xczmpnTbBVzr7guAB4CVKePvAbakmXcNcD2wp/+ln5+eewLpimARyXWZbAEsBhrcfae7dwCPA8uSO7j7Gnc/Egy+AlT3jDOzauDjwCNp5v23wBeBYbsya0Z5EQWxiK4IFpGcl0kATAX2Jg03Bm29uQN4Jmn42yRW8t3JnczsRmCfu2841y83s7vMrN7M6pubmzMo99xi0QgXTS7WmUAikvMyCQBL05b2G7uZLSERAPcGwzcATe6+NqVfEfBl4P6+frm7r3T3WnevraioyKDcvs2pLFUAiEjOyyQAGoGapOFqYH9qJzNbQGI3zzJ3bwmarwFuNLPdJHYdXWdmjwEXAjOBDcG4auANM6s8z8/RL3MqS2hubedQW/tw/DoRkayUSQC8DlxkZjPNLB+4GXgyuYOZTQOeAG5z9+097e7+JXevdvcZwXQvuPut7v6mu09y9xnBuEZgkbu/Ozgf69wuqdLDYURE+gwAd48DdwPPkjiTZ5W7bzazFWa2Iuh2P1AOPGRm682sfsgqHgSzg1tCbNEFYSKSw2KZdHL3p4GnU9oeTnp/J3BnH/NYDazuZdyMTOoYLBOLC5hYXKAtABHJaTl3JXCPS4JnA4iI5KqcDYA5lSVsP9hKvKu7784iIqNQzgbA7MpS2uPd7G45GXYpIiKhyNkA6Hk4jI4DiEiuytkAmDWpmGjEdEsIEclZORsAY/KizJw4li26KZyI5KicDQBI7AbSFoCI5KqcDoBLqkppPHKK1tOdYZciIjLscjoAeg4Ebz+o3UAikntyOgDeuyWEAkBEck9OB8DUskJKCmI6DiAiOSmnA8DMmFNVomsBRCQn5XQAQPBwmAOtuA/bUylFRLJCzgfA7MoSWtvj7Dt6KuxSRESGVc4HwCVVuiWEiOSmnA+AiycnAkC3hhaRXJPzAVAyJo+aCYV6OpiI5JycDwCA2ZNLtQUgIjlHAUDiOMCuQyc43dkVdikiIsMmowAwszoz22ZmDWZ2X5rxy81sY/BaY2YLU8ZHzWydmT2V1PYNM9saTPNTMysb8Kc5T3MqS+nqdhqa2sIqQURk2PUZAGYWBR4ElgJzgVvMbG5Kt13Ate6+AHgAWJky/h5gS0rb88D8YJrtwJf6X/7gmFOlA8Eiknsy2QJYDDS4+0537wAeB5Yld3D3Ne5+JBh8BajuGWdm1cDHgUdSpnnO3ePpphluM8rHUhCLsFUHgkUkh2QSAFOBvUnDjUFbb+4Ankka/jbwReBcT1//bMo0Z5jZXWZWb2b1zc3NGZTbf9GIcfHkErbprqAikkMyCQBL05b2vglmtoREANwbDN8ANLn72l5nbvZlIA78IN14d1/p7rXuXltRUZFBuednTmWJ7goqIjklkwBoBGqShquB/amdzGwBid08y9y9JWi+BrjRzHaT2HV0nZk9ljTN7cANwHIP+WY8sytLONTWTnNre5hliIgMm0wC4HXgIjObaWb5wM3Ak8kdzGwa8ARwm7tv72l39y+5e7W7zwime8Hdbw2mqSOxpXCju58clE8zAJdUlQK6JYSI5I4+AyA4UHs38CyJM3lWuftmM1thZiuCbvcD5cBDZrbezOoz+N3/AJQAzwfTPHx+H2Fw9DwdTM8GEJFcEcukk7s/DTyd0vZw0vs7gTv7mMdqYHXS8Kx+1DnkyosLqCgp0KmgIpIzdCVwkjmVJdoCEJGcoQBIMqeyhO0H24h3neuMVRGR0UEBkGROZSkd8W52t4R+TFpEZMgpAJK8d0sI7QYSkdFPAZBk1qRiohFjqy4IE5EcoABIUhCLcsHEsdoCEJGcoABIMadKD4cRkdygAEgxp7KExiOnOH66M+xSRESGlAIgRc8Vwdu1FSAio5wCIMWc4J5AWxQAIjLKKQBSTBk3hpIxMbbpQLCIjHIKgBRmxiWVpToVVERGPQVAGrMrS9j6bishP6JARGRIKQDSmFNVQlt7nMYjp8IuRURkyCgA0phTqYfDiMjopwBIY7YeDiMiOUABkEZxQYyaCYU6FVRERjUFQC/mVJZqF5CIjGoKgF5cUlnCzuY2Tnd2hV2KiMiQyCgAzKzOzLaZWYOZ3Zdm/HIz2xi81pjZwpTxUTNbZ2ZPJbVNMLPnzWxH8HP8wD/O4JldWUq3Q0NTW9iliIgMiT4DwMyiwIPAUmAucIuZzU3ptgu41t0XAA8AK1PG3wNsSWm7D/gPd78I+I9gOGv0PBxmywEdCBaR0SmTLYDFQIO773T3DuBxYFlyB3df4+5HgsFXgOqecWZWDXwceCRlvsuA7wfvvw98st/VD6EZ5WMpiEV0HEBERq1MAmAqsDdpuDFo680dwDNJw98GvgikPml9srsfAAh+TsqglmETjdiZK4JFREajTALA0rSlvUeCmS0hEQD3BsM3AE3uvvZ8CzSzu8ys3szqm5ubz3c252X25BJdCyAio1YmAdAI1CQNVwP7UzuZ2QISu3mWuXtL0HwNcKOZ7Sax6+g6M3ssGHfQzKqCaauApnS/3N1Xunutu9dWVFRkUO7gmVNVyqG2Dppb24f194qIDIdMAuB14CIzm2lm+cDNwJPJHcxsGvAEcJu7b+9pd/cvuXu1u88IpnvB3W8NRj8J3B68vx34+YA+yRC4JLgiWMcBRGQ06jMA3D0O3A08S+JMnlXuvtnMVpjZiqDb/UA58JCZrTez+gx+91eB681sB3B9MJxVdEsIERnNYpl0cvengadT2h5Oen8ncGcf81gNrE4abgE+nHmpw6+8uICKkgK26NkAIjIK6UrgPsyp1IFgERmdFAB9uKSqlB1NbcS7Us9iFREZ2RQAfZhTWUJHvJvdLSfCLkVEZFApAPrQcyBYxwFEZLRRAPRh1qRiohHTcQARGXUUAH0oiEW5sGKsrgUQkVFHAZCBOZWlvLX/OO5p74AhIjIiKQAycOWM8ew/dlrPBhCRUUUBkIGPzqsE4JlN74ZciYjI4FEAZGBy6RiumD6eXygARGQUUQBkqG5eJW8dOM6elpNhlyIiMigUABmqm5/YDfSLzQdCrkREZHAoADJUM6GI+VNLdRxAREYNBUA/1M2rZN2eo7x77HTYpYiIDJgCoB/q5lcB8OxmbQWIyMinAOiHWZOKmTWpWGcDiciooADop6XzK3l1VwstbXpOsIiMbAqAfvrYvEq6HX655WDYpYiIDIgCoJ/mTSmlZkKhzgYSkRFPAdBPZkbdvEp+3XCI46c7wy5HROS8ZRQAZlZnZtvMrMHM7kszfrmZbQxea8xsYdA+xsxeM7MNZrbZzL6SNM1lZvaKma03s3ozWzx4H2to1c2vorPLeWFLU9iliIictz4DwMyiwIPAUmAucIuZzU3ptgu41t0XAA8AK4P2duA6d18IXAbUmdnVwbivA19x98uA+4PhEeHymjImlxbobCARGdEy2QJYDDS4+0537wAeB5Yld3D3Ne5+JBh8BagO2t3de+6hnBe8em6q70Bp8H4csP+8P8Uwi0SMj82rZPX2Jk52xMMuR0TkvGQSAFOBvUnDjUFbb+4AnukZMLOoma0HmoDn3f3VYNTngW+Y2V7gm8CX0s3MzO4KdhHVNzc3Z1Du8KibV8npzm5e2pY9NYmI9EcmAWBp2tI+GsvMlpAIgHvPdHTvCnbzVAOLzWx+MOpzwJ+6ew3wp8B3083T3Ve6e62711ZUVGRQ7vBYPHMC44vy+IWuChaRESqTAGgEapKGq0mzu8bMFgCPAMvcvSV1vLsfBVYDdUHT7cATwfufkNjVNGLEohGunzuZF7Y00R7vCrscEZF+yyQAXgcuMrOZZpYP3Aw8mdzBzKaRWJnf5u7bk9orzKwseF8IfATYGozeD1wbvL8O2DGAzxGKpfOraG2Ps6bhrLwTEcl6sb46uHvczO4GngWiwKPuvtnMVgTjHyZxFk858JCZAcTdvRaoAr4fnEkUAVa5+1PBrP8Q+DsziwGngbsG96MNvd+cVU5JQYxnNh1gyZxJYZcjItIvfQYAgLs/DTyd0vZw0vs7gTvTTLcRuLyXeb4MXNGfYrNNQSzKdZdM4vm3DhLv6iYW1XV1IjJyaI01QHXzKjlyspPXdh0OuxQRkX5RAAzQtbMrGJMX0dlAIjLiKAAGqCg/xrUXV/CLTe/S3Z327FgRkaykABgES+dX0dTazrq9R8MuRUQkYwqAQbBkziTyosYvNh0IuxQRkYwpAAbBuMI8rpk1kV9sfhd37QYSkZFBATBIls6vZO/hU2zefzzsUkREMqIAGCQfuWQyEYNndTaQiIwQCoBBUl5cwFUzy/WoSBEZMRQAg6hufiUNTW00NLWGXYqISJ8UAIPoY/MqAfSkMBEZERQAg6hy3Bgun1amq4JFZERQAAyypfMr2bTvOHsPnwy7FBGRc1IADLK6eVWAzgYSkeynABhk08qLmFtVqrOBRCTrKQCGQN38Sta+c4Sm46fDLkVEpFcKgCGwdH7ibCDtBhKRbKYAGAKzJhVzQcVYnQ0kIllNATAEzIyl8yt5ZedhjpzoCLscEZG0MgoAM6szs21m1mBm96UZv9zMNgavNWa2MGgfY2avmdkGM9tsZl9Jme6Pg/luNrOvD85Hyg5186ro6nae33Iw7FJERNLqMwDMLAo8CCwF5gK3mNnclG67gGvdfQHwALAyaG8HrnP3hcBlQJ2ZXR3MdwmwDFjg7vOAbw7842SP+VNLmVpWqKuCRSRrZbIFsBhocPed7t4BPE5ixX2Gu69x9yPB4CtAddDu7t4WtOcFr54b5n8O+Kq7twd9mwb0SbKMmVE3v5KXdxyi9XRn2OWIiJwlkwCYCuxNGm4M2npzB/BMz4CZRc1sPdAEPO/urwajLgY+aGavmtlLZnZlupmZ2V1mVm9m9c3NzRmUmz2Wzq+ko6ubF7aOqmwTkVEikwCwNG1pH3sV7Na5A7j3TEf3Lne/jMRWwWIzmx+MigHjgauBPwdWmdlZv8vdV7p7rbvXVlRUZFBu9lg0bTwVJQXaDSQiWSmTAGgEapKGq4H9qZ3MbAHwCLDM3VtSx7v7UWA1UJc03yeC3USvAd3AxP4Un+0iEeNj8yazelszpzq6wi5HROR9MgmA14GLzGymmeUDNwNPJncws2nAE8Bt7r49qb3CzMqC94XAR4CtweifAdcF4y4G8oFDA/kw2ahuXhWnOrt4afvI2n0lIqNfnwHg7nHgbuBZYAuwyt03m9kKM1sRdLsfKAceMrP1ZlYftFcBL5rZRhJB8ry7PxWMexS4wMw2kTiwfLuPwieqX3XBBMqK8nRVsIhkHRtJ69za2lqvr6/vu2OW+eK/buDJDft57I6rqJ0xIexyRCTHmNlad69NbdeVwMPgv310NlPGFfIHj77GqzvPOjwiIhIKBcAwmFw6hsfvupopZYXc/k+vsaZh1B3qEJERSAEwTCYFITB9wlg+873X+ZUOCotIyBQAw2hicQE/uutqLqwo5s7v1/OiLhATkRApAIbZhLH5/PAPr2J2ZQl3/Us9z+nsIBEJiQIgBGVF+Tx251XMmzKOP/rBGzzz5oGwSxKRHKQACMm4wjz+5Y7FLKwp4+4freP/bjjr4moRkSGlAAhRyZg8/vmzi7li+njueXwdP13XGHZJIpJDFAAhG1sQ43ufuZKrLyjnC6s2sKp+b98TiYgMAgVAFijKj/Hof7mSD8yayBf/dSM/fHVP2CWJSA5QAGSJMXlR/vEPalkyu4L//tM3+ef/3B12SSIyyikAssiYvCgP33YF18+dzP0/38x3X94VdkkiMoopALJMQSzKg7+/iKXzK3ngqbf4Py+9HXZJIjJKKQCyUH4swt/fcjmfWDiFv3lmK//wwo6wSxKRUSgWdgGSXiwa4W9vWkgsYnzzue3Eu53Pf+TisMsSkVFEAZDFYtEI3/z0QqIR49u/3MEFFcXcuHBK2GWJyCihXUBZLhox/uZ3L6V2+nju+7eN7DjYGnZJIjJKKABGgLxohAeXL6IoP8qKx9bS1h4PuyQRGQUUACPE5NIx/P0ti9h16AT3/ttGRtKjPEUkO2UUAGZWZ2bbzKzBzO5LM365mW0MXmvMbGHQPsbMXjOzDWa22cy+kmbaPzMzN7OJA/84o9tvXFjOn39sDv++8QD/9OvdYZcjIiNcnwFgZlHgQWApMBe4xczmpnTbBVzr7guAB4CVQXs7cJ27LwQuA+rM7OqkedcA1wO690GGVlx7AdfPncxfP72Fte8cDrscERnBMtkCWAw0uPtOd+8AHgeWJXdw9zXufiQYfAWoDtrd3duC9rzglbzv4m+BL6a0yTmYGd/89EKmji/kj37wBofa2sMuSURGqEwCYCqQfIvKxqCtN3cAz/QMmFnUzNYDTcDz7v5q0H4jsM/dN/S36Fw3rjCP7yy/gqMnO/njH64j3tUddkkiMgJlEgCWpi3tN3YzW0IiAO4909G9y90vI7FVsNjM5ptZEfBl4P4+f7nZXWZWb2b1zc16kHqPuVNK+avfuZT/3NnCt57fHnY5IjICZRIAjUBN0nA1cNbjq8xsAfAIsMzdW1LHu/tRYDVQB1wIzAQ2mNnuYJ5vmFllmulWunutu9dWVFRkUG7u+NQV1dyyeBoPrX6b5986GHY5IjLCZBIArwMXmdlMM8sHbgaeTO5gZtOAJ4Db3H17UnuFmZUF7wuBjwBb3f1Nd5/k7jPcfQaJkFnk7npCej/9j0/M5dKp4/jCqvW803Ii7HJEZATpMwDcPQ7cDTwLbAFWuftmM1thZiuCbvcD5cBDZrbezOqD9irgRTPbSCJInnf3pwb9U+SwMXlRHlq+iIgZn3vsDU53doVdkoiMEDaSLiiqra31+vr6vjvmoBe3NvGZ773OTbXVfP1TC8MuR0SyiJmtdffa1HZdCTxKLJkziT+5bhar6hv58eu6rEJE+qYAGEXu+cjFfPCiifzFzzezad+xsMsRkSynABhFohHj726+nIlj81nx2FqOnuwIuyQRyWIKgFFmwth8Hly+iIPHT/OFVRvo7h45x3hEZHgpAEahy6eN5/4b5vLC1iYeWt0QdjkikqUUAKPUrVdP55OXTeFbz2/n5R2Hwi5HRLKQAmCUMjP++ncvZdakYv7k8XUcOHYq7JJEJMsoAEaxovwY37n1Cjri3dz23df4hxd2sKbhECf0RDERQQ+FH/UurCjm73//cv7637fwzecSd+mIGMyuLOWK6WUsmjaeRdPGM728CLN09/0TkdFKVwLnkGMnO1m39whv7DnKuj1HWLfn6JnnC5ePzefyaeNZFITCwuoyCvOjIVcsIoOhtyuBtQWQQ8YV5fGh2ZP40OxJAHR1OzuaWnnjnaOsfecI6/Yc4ZdbEncVjUaMuVWlLJpWxqLp41k8cwJV4wrDLF9EBpm2AOR9Dp/oYN2eI7yx5whvvHOU9XuPcqqzCzP4wKyJLL9qGh++ZDJ5UR0+EhkpetsCUADIOcW7utn6biu/3HKQH7++lwPHTlNRUsBNtdXcfOU0aiYUhV2iiPRBASAD1tXtrN7WxI9e28MLW5twtFUgMhIoAGRQ7T96ilX1e7VVIDICKABkSAz2VsHx0500Hj7F3iMn2Xv4JI1HTtF45CSnO7v5jQvL+dDsCuZWleqUVZF+UADIkMtkq+BkR5x9R3pW8ImV+95ghd945BTHTnW+b55j86PUTEhco7DlwHEAJpcWcO3FFSyZPYlrLppI6Zi8Yf+sIiOJAkCGTbyrm5e2N/PDV/fw4rbEVsHFk0poOdHOobb336K6IBahenwh1eOLqJlQSM34ove9LyvKO/Ntv6n1NC9ta2b1tmZ+taOZ1tNxYhHjiunjWTJnEh+aXcHsySXaOhBJoQCQUPRsFazbc5SqcWOoHl9IzYRgJT++kInFBUQi/V9hx7u6eWPPUVZva+LFbc1ntg6qxo3hQ7Mr+NDsSVwzayLFBbrURUQBIKPau8dO89L2Jl7c2szLDYdoa4+TFzWunDGBJbMn8VsXVzBrUjHR8wgbkZFuQAFgZnXA3wFR4BF3/2rK+OXAvcFgG/A5d99gZmOAXwEFJK46/ld3/x/BNN8APgF0AG8Dn3H3o+eqQwEgmeiId7P2nSOs3t7E6q3NbDvYCkBRfpT5U8ZxafU4FlSP49Kp45hRPva8tkBERpLzDgAziwLbgeuBRuB14BZ3fyupz28CW9z9iJktBf7S3a+yxM7Yse7eZmZ5wMvAPe7+ipl9FHjB3eNm9jUAd7+Xc1AAyPnYf/QUa95uYdO+Y2xsPMrm/cdpj3cDUFIQY97UUhZUl3Hp1EQwTJugG+PJ6DKQewEtBhrcfWcwo8eBZcCZAHD3NUn9XwGqg3YnsUUAkBe8PBj3XMo0n8r0w4j0x5SyQj51RTWfuqIaSBw/2NHUxpv7jvFm4zE27jvG9369m46uRCiUjoklAiHYSrh06jiqxxcqFGTUySQApgJ7k4YbgavO0f8O4JmegWALYi0wC3jQ3V9NM81ngR+nm5mZ3QXcBTBt2rQMyhU5t1g0wiVVpVxSVcpNtTVAYrfR9oOtiVAIguGR/7eTzq7EFnJFSQE3LpzCTbU1zK4sCbN8kUGTSQCk+9qTdr+RmS0hEQAfONPRvQu4zMzKgJ+a2Xx335Q0zZeBOPCDdPN095XASkjsAsqgXpF+y49FmD91HPOnjuOWoK093sW2d1vZ2HiMl3cc4p//czfffXkXC6vH8anaGm5cOIVxhboGYbTq6nYOHDvFOy0n2X/0FBEz8mMRCmIR8oNXQSxCQSyaGI5GzhqfH41k9ZZjJgHQCNQkDVcD+1M7mdkC4BFgqbu3pI5396NmthqoAzYF09wO3AB82EfS6UiSEwpiURZUl7Gguoxbr55OS1s7P1u/n5/U7+UvfraJ//XUW3xsXiU31dbwmxeW62DyCNTZ1U3jkVPsbjnBnpaT7G45wTvBz8bDp87sFhyI/GiEgrwI5WPzmVhckHiVJL0vLqAiaXjsMJ66nMlB4BiJg8AfBvaROAj8++6+OanPNOAF4A+SjweYWQXQGaz8C4HngK+5+1PBmUXfAq519+ZMitVBYMkG7s6mfcf5ydq9/GzdPo6fjjO1rJDfu6KaT19RHcq9kDq7ujl8ooPm1nYOtbXT0tbBobb24NXBsVOdGInnPMSiRsSMWMSIRiJEIxCNRILhxCv5fTSS6N8R76Y93kV7vJv2zsT7jq6e92ePa493B9Mkfp7FzjlI8hfniBlF+VHGFsQYmx+juCDG2IIoRQUxivNjjC2IUVwQjA/GvdcvRl40wr6jp3in5cSZlfw7LSfZd/QUXd3vrQPH5keZVj6WGeVFTA9+TisvorqsCMfPfJ6ez9TRFfwMPn9yW3K/051dtJzo4FBr+5m/y5GTnaRTmBc9OyCK8/nk5VO5oKL4vP59DPQ00N8Gvk3iNNBH3f2vzGwFgLs/bGaPAL8HvBNMEnf32mCr4PvBdBFglbv/z2CeDSROD+3ZWnjF3Vecqw4FgGSb051dPPfWQX5Sv5eXGw7hDr9xQTk3XVlN3byqAT1VrT3exaG29680DrWlX8n3tjIZkxdhYnEBZUV5uCd2a5x5uRPveu99V7cT7+p+/3C307OKiBiMyYu+b7dHQSzx7bYg1tOeflxeNPK+FXrqasdT9yqnDHZ1Oyc6ujjZEedEe5y29jgn2rs40R7nREfwviN+1nzTKSvKY/qE91bw08vHMj34ObE4f9h22aSG9qGev2fr2X/rwyc7eOyOq7hm1sTz+l26EExkiO07eoon1jbyk7WN7Dl8kpKCGDcsnMKna6u5vKYMM+N0Z1fa//DNPSv41kRbc1s7rafjaX9PcUGMicVn704oD74pvtdewNj86IBXaN3dTrc7sSy/3Xd3t3OqsycUupKCIk5HvJspZYVMLy+irCg/7FL7LR7sijrfv4ECQGSYdHc7r+0+zKr6vTzz5ruc6uyioqSAUx1dZ57BnKp0TIyJJT2b+wXvreBLenYD5Af7igsYk6dnNUv/6JnAIsMkEjGuvqCcqy8o5ys3dvLvGw/w6q7DjCvMo6KkIOXbewHlY/O1UpdQKABEhlDJmDxuXjyNmxfrGhbJPtm9U09ERIaMAkBEJEcpAEREcpQCQEQkRykARERylAJARCRHKQBERHKUAkBEJEeNqFtBmFkz791wLttMBA6FXcQ5qL6BUX0Do/oGbiA1Tnf3itTGERUA2czM6tPdayNbqL6BUX0Do/oGbihq1C4gEZEcpQAQEclRCoDBszLsAvqg+gZG9Q2M6hu4Qa9RxwBERHKUtgBERHKUAkBEJEcpAM6DmdWY2YtmtsXMNpvZPUH7X5rZPjNbH7x+O8Qad5vZm0Ed9UHbBDN73sx2BD/Hh1Tb7KRltN7MjpvZ58Ncfmb2qJk1mdmmpLZel5eZfcnMGsxsm5l9LKT6vmFmW81so5n91MzKgvYZZnYqaTk+HFJ9vf49s2T5/Tiptt1mtj5oD2P59bZOGdp/g+6uVz9fQBWwKHhfAmwH5gJ/CfxZ2PUFde0GJqa0fR24L3h/H/C1LKgzCrwLTA9z+QG/BSwCNvW1vIK/9QagAJgJvA1EQ6jvo0AseP+1pPpmJPcLcfml/Xtmy/JLGf+/gftDXH69rVOG9N+gtgDOg7sfcPc3gvetwBZgarhVZWQZ8P3g/feBT4ZXyhkfBt5291Cv8Hb3XwGHU5p7W17LgMfdvd3ddwENwOLhrs/dn3P3nqfMvwJUD2UN59LL8utNViy/HmZmwE3Aj4ayhnM5xzplSP8NKgAGyMxmAJcDrwZNdweb5I+GtYsl4MBzZrbWzO4K2ia7+wFI/IMDJoVW3Xtu5v3/8bJl+UHvy2sqsDepXyPhfwH4LPBM0vBMM1tnZi+Z2QfDKor0f89sW34fBA66+46kttCWX8o6ZUj/DSoABsDMioF/Az7v7seB7wAXApcBB0hsVoblGndfBCwF/quZ/VaItaRlZvnAjcBPgqZsWn7nYmnaQjuf2sy+DMSBHwRNB4Bp7n458AXgh2ZWGkJpvf09s2r5Abfw/i8hoS2/NOuUXrumaev3MlQAnCczyyPxh/qBuz8B4O4H3b3L3buBf2SIN2vPxd33Bz+bgJ8GtRw0syqA4GdTWPUFlgJvuPtByK7lF+hteTUCNUn9qoH9w1wbAGZ2O3ADsNyDncPBboGW4P1aEvuHLx7u2s7x98ym5RcDfhf4cU9bWMsv3TqFIf43qAA4D8E+w+8CW9z9W0ntVUndfgfYlDrtcDCzsWZW0vOexMHCTcCTwO1Bt9uBn4dRX5L3ffPKluWXpLfl9SRws5kVmNlM4CLgteEuzszqgHuBG939ZFJ7hZlFg/cXBPXtDKG+3v6eWbH8Ah8Btrp7Y09DGMuvt3UKQ/1vcDiPdI+WF/ABEptbG4H1weu3gX8B3gzanwSqQqrvAhJnCGwANgNfDtrLgf8AdgQ/J4S4DIuAFmBcUltoy49EEB0AOkl8u7rjXMsL+DKJb4bbgKUh1ddAYj9wz7/Bh4O+vxf83TcAbwCfCKm+Xv+e2bD8gvbvAStS+oax/Hpbpwzpv0HdCkJEJEdpF5CISI5SAIiI5CgFgIhIjlIAiIjkKAWAiEiOUgCIiOQoBYCISI76/67ninaRVzRqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#@ INSPECTING THE RMSE SCORES:\n",
    "df_scores = pd.DataFrame(scores, columns=['n_estimators', 'rmse'])\n",
    "\n",
    "plt.plot(df_scores.n_estimators, df_scores.rmse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NAXZcW3AuC-d"
   },
   "source": [
    "After which value of `n_estimators` does RMSE stop improving?\n",
    "\n",
    "- 10\n",
    "- 50\n",
    "- 70\n",
    "- **150**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CYhRv3kEvWjK"
   },
   "source": [
    "## Question 4\n",
    "\n",
    "Let's select the best `max_depth`:\n",
    "\n",
    "* Try different values of `max_depth`: `[10, 15, 20, 25]`\n",
    "* For each of these values, try different values of `n_estimators` from 10 till 200 (with step 10)\n",
    "* Fix the random seed: `random_state=1`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "id": "fO9TackKvKcS"
   },
   "outputs": [],
   "source": [
    "#@ TRAINING THE MODEL WITH DEPTH:\n",
    "\n",
    "scores = []\n",
    "max_depth = [10, 15, 20, 25]\n",
    "n_estimators = [*range(10, 201, 10)]\n",
    "\n",
    "for d in max_depth:\n",
    "    for n in n_estimators:\n",
    "        rf = RandomForestRegressor(n_estimators=n,\n",
    "                                   max_depth=d,\n",
    "                                   random_state=1,\n",
    "                                   n_jobs=-1)\n",
    "        rf.fit(X_train, y_train)\n",
    "\n",
    "        y_pred = rf.predict(X_val)\n",
    "        rmse = np.sqrt(mean_squared_error(y_val, y_pred))\n",
    "\n",
    "        scores.append((d, n, rmse))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns = ['max_depth', 'n_estimators', 'rmse']\n",
    "df_scores = pd.DataFrame(scores, columns=columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x23cf87924f0>"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA/zUlEQVR4nO3deXhU1fnA8e+ZfZLJnhBCAgSURXYBcQFXpFVrpVXbalERd6vW7VfFtm6trWjdqnVtq1K11VZFrdVWVBAVUfZ9hwiBELKRfTLb+f1xbyaTkJCQhUky7+d57nPPPffMnXfuJPedu52rtNYIIYSIPZZoByCEECI6JAEIIUSMkgQghBAxShKAEELEKEkAQggRo2zRDuBwpKen69zc3GiHIYQQPcry5cuLtdYZTet7VALIzc1l2bJl0Q5DCCF6FKXUt83VyyEgIYSIUZIAhBAiRkkCEEKIGCUJQAghYpQkACGEiFFtSgBKqbOUUpuVUtuUUrObmT9DKbXGHBYrpcZGzMtTSq1VSq1SSi2LqL9PKbXHrF+llDqncz6SEEKItmj1MlCllBV4GpgG5ANLlVLvaa03RDTbCZyqtS5TSp0NvAAcHzH/dK11cTOLf1xr/Uj7wxdCCNFebdkDmARs01rv0Fr7gNeB6ZENtNaLtdZl5uQSIKdzw+yYlbvKeO6z7dEOQwghupW2JIBsYHfEdL5Z15IrgQ8jpjXwkVJquVLqmiZtbzQPG72olEppU8TtMG/lHuZ8uIk3l+d31VsIIUSP05YEoJqpa/YpMkqp0zESwJ0R1ZO11uOBs4EblFKnmPXPAkcB44AC4NEWlnmNUmqZUmpZUVFRG8I92N3njmDy0Wnc9fYavtpe0q5lCCFEb9OWBJAP9I+YzgH2Nm2klBoD/AWYrrUOb2W11nvN8X5gHsYhJbTWhVrroNY6BPy5vr4prfULWuuJWuuJGRkHdWXRJnarhWdmTGBgWjzXvbqcHUVV7VqOEEL0Jm1JAEuBIUqpQUopB3AR8F5kA6XUAOBt4FKt9ZaI+nilVEJ9GfgOsM6czopYxA/r67tKktvOS5cfh82iuOLlpZRW+7ry7YQQottrNQForQPAjcD/gI3AP7XW65VS1ymlrjOb3QOkAc80udwzE/hCKbUa+Ab4j9b6v+a8h83LQ9cApwO3dt7Hal7/1DheuGwie8u9XPfKcuoCwa5+SyGE6LZUT3oo/MSJE3Vn9Ab679V7uekfK/nhsdk89uOxKNXcaQ4hhOgdlFLLtdYTm9b3qO6gO8v3x/Yjr7iaR+dvYVB6PD+fOiTaIQkhxBEXkwkA4MYzjmZnSTWPzd/CwLQ4po871JWtQgjR+8RsX0BKKR48fzSTBqXyi3+tYVleabRDEkKIIypmEwCA02bl+UsmkJ3i5ppXlrOrpCbaIQkhxBET0wkAICXewYuXH0dIa2a9/A3lNf5ohySEEEdEzCcAgEHp8Tx/yQR2ldZw/WvL8QdD0Q5JCCG6nCQA0/GD05hz/hgWby/h1/PW0ZMujxVCiPaI2auAmnPBhBy+LanmyU+3MSgjnutOPSraIQkhRJeRBNDErdOGsrOkhjkfbmJgahxnj85q/UVCCNEDySGgJpRS/OHCMYwfkMwtb6xi1e4D0Q5JCCG6hCSAZrjsVv582UT6JDq5au4y9hyojXZIQgjR6SQBtCDN4+Sly4+jLhDkipeWUumVy0OFEL2LJIBDOLpPAs9dMoHtRVVc/TfZExBC9C6SAFox+eh0Hr5wDKt2H2Dqowt58pOteP3SjbQQoueTBNAG54/P4ZPbT+OM4X14bP4Wpj3+GfM3FMq9AkKIHk0SQBtlJ7t5ZsYEXrvqeFw2K1f/bRmXv7RUHi8phOixJAEcpslHp/PBzSdz97kjWPFtGd99YhEPfriRqrpAtEMTQojDIgmgHexWC1dOGcSn/3ca08dl8/xnO5j66ELeWblHDgsJIXoMSQAdkJHg5JEfjeXtn51EnwQXt7yxip88v4QNeyuiHZoQQrRKEkAnGD8ghXdumMyD549mW1EV5z71OXe/s44DNb5ohyaEEC2SBNBJrBbFxZMGsOD207j0hIG89vW3nP7IQl77+luCITksJITofiQBdLKkODv3Tx/F+zedzJA+Cfxq3jqmP/0Fi7cXE5JEIIToRlRPOmk5ceJEvWzZsmiH0WZaa95bvZfff7CRwoo60j1Opg7vw5kjMplydDpuhzXaIQohYoBSarnWeuJB9ZIAul51XYCPNxYyf0Mhn20uorIugNNmYcrR6Zw5IpOpw/vQJ9EV7TCFEL1USwlAngdwBMQ7bUwfl830cdn4AiG+2VkaTgifbNoPwNj+yZxp7h0M75uAUirKUQshejvZA4girTWbCyv5eEMh8zfuZ7X57IHsZDfTRmRy5jGZTBqUisMmp2qEEO0nh4B6gP0VXj7dtJ+PNxby+dZi6gIhEpw2ThmWwdmj+vLdkX2xWyUZCCEOjySAHqbWF+TLbcV8vLGQjzfup7iqjqwkF7Mm53LRpAEkuuzRDlEI0UNIAujBQiHNwi37eWHRDpbsKMXjtHHRcf2ZNWUQ2cnuaIcnhOjmJAH0Emvzy/nz5zv4z9oCAL43OourTx7M6JykKEcmhOiuJAH0MnsO1PLSFzt5feluquoCnDA4lWtOGcxpQ/tgscgVREKIBpIAeqkKr5/Xv9nFS1/mUVDu5aiMeK4+eTA/ODYbl11uNBNCSALo9fzBEB+sLeCFRTtYv7eCdI+Dy07M5ZITBpIa74h2eEKIKOpQAlBKnQX8EbACf9Faz2kyfwZwpzlZBVyvtV5tzssDKoEgEKgPQimVCrwB5AJ5wI+11mWHikMSQOu01ny1vYQ/f76DBZuLcNktXDA+h58c1580jxOPw0a804pNLicVIma0OwEopazAFmAakA8sBS7WWm+IaHMSsFFrXaaUOhu4T2t9vDkvD5iotS5ustyHgVKt9Ryl1GwgRWt9J4cgCeDwbC2s5C+f72Teyj34gqFG8xw2Cx6nkQziHTaz3DAd72yo8zitJLjsZCQ4yUhwku5xkuy2y7kGIXqIjiSAEzE26N81p+8C0Fo/2EL7FGCd1jrbnM6j+QSwGThNa12glMoCFmqthx0qFkkA7bO/0ss3O0uprgtQVRekui5glo1xta9pXZBqnzHdUgemNosi3eMMJ4UMj5P0BAcZHicZCa6G+gQn8Q6rdG0hRBR1pC+gbGB3xHQ+cPwh2l8JfBgxrYGPlFIaeF5r/YJZn6m1LgAwk0CfFgK/BrgGYMCAAW0IVzTVJ8HFuWP6HfbrtNZ4/SGq6gJUeP0UV9ZRVFVHUWXEUFXH/kov6/eWU1zla/bZBy67hawkN0f38TA008PQzASGZiYwOCMep01OVAsRLW1JAM39dGv2d6FS6nSMBDAlonqy1nqvuYGfr5TapLVe1NYAzYTxAhh7AG19neg4pRRuhxW3w0pGgpOjMjyHbB8KacpqfI2SRLFZ3nOgli2FVXy6aX84SVgtity0uHBCGJqZwLC+HgamxUuXF0IcAW1JAPlA/4jpHGBv00ZKqTHAX4CztdYl9fVa673meL9Sah4wCVgEFCqlsiIOAe1v/8cQ3YHFokjzOEnzOBnet/k2dYEgO4ur2VJYxZZ9lWwprGRjQQX/Xb+P+qORdqticLqHoX0TGNrHw5DMBIZmeuifGieJQYhO1JYEsBQYopQaBOwBLgJ+GtlAKTUAeBu4VGu9JaI+HrBorSvN8neA35iz3wNmAnPM8bsd/CyiB3DarAzvm8jwvokwtqHe6w+ybX8VWworjeRQWMnKXWX8e3XDbw2rRZGT4mZgWjyD0uLITY83hrR4clLckhyEOEytJgCtdUApdSPwP4zLQF/UWq9XSl1nzn8OuAdIA54xT/bVX+6ZCcwz62zA37XW/zUXPQf4p1LqSmAX8KNO/WSiR3HZrYzKTmJUduMuLarrAmzdX8W2/VXkFVezs6SavOJqlueVUu0LhttZLYr+9ckhPZ7c+gRhJge57FWIg8mNYKJH0lpTXOUjz0wIxrgmPB2ZHGwWRb9kN2keB2nxTtLiHaR6HKTFO0jzOEitrzMHuYNa9DbyRDDRqyilwpeZHpeb2mie1pqiqjq+LalhZ7GREHaX1VJaXUd+WQ1r8g9QWu0j0MI1rh6nLZwM6pNEusdJTkoc/VPd9E+Jo1+yu8c8qKcuEKS81k+V17jMt8oboLJ+7PVTVdcwHTm/0hugqs54XY0viFJgUQqFOVbGeR+LUliU8Z1YzDbh+fV1FoXLZlxQ4LZHjM2yK1y24HbYIuZZcNmNe1MyE43Li6295P6TUEgT1JpgSBOqH4c4qK6+3CfB1enPEZcEIHodpRR9Elz0SXAdlBzqaa2pqA1QUl1HabWPkmqfMa6qC5dLq33sLfeybm85JVWNE4ZFQd9EFzmpcQxIjaN/fXIwy30SnJ16o1wgGKK81k95rZ8D5riifrrG3zCvJqK+1kd5rR+vP9Tq8h02CwlOGwkuGx6XcRNgdrKbBFcCHqeNOHPDE9IarSGk68s6XA5pzGkdMd8YB0KaOn+QWn+QGl+AkmofXn+QWp9RV+sLHnSzYnNsFkVmoou+SS6ywoPbGCcb43RP+5NEKKSp9hmJsNJbPxhJstYXxOsP4vWHjJjNuI06c9ofwmt+pvq6+tcEQqFGG/jD9fKs4zhtWLNXy7ebJAARk5RSJMXZSYqzMzij9fbBkGZfhZfdpTXGUFZLfmkNu0pr+HxrEYUVdY3aO2wWcpLd5KTG0T/FTWaii0AwhDcQos7cINQFWh/X+UN4A0H8wUNvMOIcVpLddhLddpLcdnLT40hyJ5HktpMc5yDRZSPBZccTsZFPcNrxuIy7v7vD/Rj166d+o1obkSAqvQEKK7wUlNdSUO6l4ICXdXvKmb+hkLpA48RRnySykoxE0S/ZTbrHgdcfCm/MK7xN9oDM6SpfgLYeFXdYLbjslvDejMvesCeTkeDEbbfitFvC82wWhcWisKqGsdVCRNnYc7Jamsw364b3Tez0dS7nAIToBF5/kD0Hag9KDrvLathdWkt5rR8Ap804pFE/dtktOG0Hj51Npt12K0luG0lxdpLdjvCGPjnOTqLL3mMOR3U2rTVlNX4jMRyISBDljcs+M0m47BYSXPZGezv1iTDBTJKN5kUkzchDVy67tUcdipJzAEJ0IZfdylEZnhZvlvMFQtitSrrE6GRKqfD5mpH9mn8oktaayroAbrtVLhVuQhKAEEdArP5C7w6UUvIM7RbIX6UQQsQoSQBCCBGjJAEIIUSMkgQghBAxShKAEELEKEkAQggRoyQBCCFEjJIEIIQQMUoSgBBCxChJAEIIEaMkAQghRIySBCCEEDFKOoMTQoT5/X7y8/Pxer3RDkW0g8vlIicnB7u9bZ3fSQIQQoTl5+eTkJBAbm6udF3dw2itKSkpIT8/n0GDBrXpNXIISAgR5vV6SUtLk41/D6SUIi0t7bD23iQBCCEakY1/z3W4311MJICQDrGvel+0wxBCiG4lJhLAvYvvZcZ/ZtCTnn8shBBdLSYSwNSPS7jp+b18W/FttEMRQnRTubm5FBcXt+u1L7/8Mnv37u2UZV1xxRX06dOHUaNGNaovLS1l2rRpDBkyhGnTplFWVtau5UeKiauActIHk7lnASvWf0zuSVdFOxwheoT7/72eDXsrOnWZI/olcu/3R3bqMruDl19+mVGjRtGvX78OL+vyyy/nxhtv5LLLLmtUP2fOHKZOncrs2bOZM2cOc+bM4aGHHurQe8XEHkD2qWcDUPjFJ1GORAjRmry8PIYPH85VV13FqFGjmDFjBh9//DGTJ09myJAhfPPNN3zzzTecdNJJHHvssZx00kls3rwZgMcee4wrrrgCgLVr1zJq1ChqamqafZ+SkhK+853vcOyxx3Lttdc2OkT86quvMmnSJMaNG8e1115LMBgEwOPxcPvttzN+/HimTp1KUVERb775JsuWLWPGjBmMGzeO2tpaAJ566inGjx/P6NGj2bRpU5s//ymnnEJqaupB9e+++y4zZ84EYObMmbzzzjttXmaLtNY9ZpgwYYJuj1AgoFceO1r/+aJxOhQKtWsZQsSCDRs2RDsEvXPnTm21WvWaNWt0MBjU48eP17NmzdKhUEi/8847evr06bq8vFz7/X6ttdbz58/X559/vtZa62AwqE8++WT99ttv6wkTJugvvviixfe56aab9P3336+11vr999/XgC4qKtIbNmzQ5557rvb5fFprra+//no9d+5crbXWgH711Ve11lrff//9+oYbbtBaa33qqafqpUuXhpc9cOBA/eSTT2qttX766af1lVdeqbXW+tNPP9Vjx449aDjxxBMPWgcjR45sVJeUlNRoOjk5udnP1dx3CCzTzWxTY+IQkLJa8Y0bytB169latpWhqUOjHZIQ4hAGDRrE6NGjARg5ciRTp05FKcXo0aPJy8ujvLycmTNnsnXrVpRS+P1+ACwWCy+//DJjxozh2muvZfLkyS2+x6JFi3j77bcB+N73vkdKSgoAn3zyCcuXL+e4444DoLa2lj59+oSX/5Of/ASASy65hPPPP7/F5dfPmzBhQvh9Tj/9dFatWtXe1dLpYiIBAGScPBXv4vWsXvU/hp4hCUCI7szpdIbLFoslPG2xWAgEAtx9992cfvrpzJs3j7y8PE477bRw+61bt+LxeBqdlG1Jc9fNa62ZOXMmDz74YLte3/QzWK1WAoEAAAsWLODWW289qG1cXByLFy8+5HtlZmZSUFBAVlYWBQUF4aTUETFxDgCg36nfBaD4iwVRjkQI0VHl5eVkZ2cDxgnYyPqbb76ZRYsWUVJSwptvvtniMk455RRee+01AD788MPwVTVTp07lzTffZP/+/YBx9c233xpXEIZCofAy//73vzNlyhQAEhISqKysbDXu+j2ApkNrG3+A8847j7lz5wIwd+5cpk+f3uprWhMzCcAxaBA1yW7iVm8jGApGOxwhRAfccccd3HXXXUyePDl8ghbg1ltv5Wc/+xlDhw7lr3/9K7Nnzw5vyJu69957WbRoEePHj+ejjz5iwIABAIwYMYIHHniA73znO4wZM4Zp06ZRUFAAQHx8POvXr2fChAl8+umn3HPPPYBx5c51113X6CRwe1188cWceOKJbN68mZycHP76178CMHv2bObPn8+QIUOYP38+s2fP7tD7ACjdg26Omjhxol62bFm7X//N9TMIfb2ChP/9k5EZozsxMiF6h40bN3LMMcdEO4xuy+PxUFVVFe0wDqm571AptVxrPbFp2zbtASilzlJKbVZKbVNKHZR2lFIzlFJrzGGxUmpsk/lWpdRKpdT7EXX3KaX2KKVWmcM5bf6E7ZR16ndJqoF1X3/Y1W8lhBDdXqsJQCllBZ4GzgZGABcrpUY0abYTOFVrPQb4LfBCk/k3AxubWfzjWutx5vDBYUd/mPqeciYAB75a1NVvJYToJl566SXGjRvXaLjhhhvatazu/uv/cLXlKqBJwDat9Q4ApdTrwHRgQ30DrXXkGYwlQE79hFIqB/ge8Dvgtk6I+fBpDVX7sffrR1VmAolrvsUf8mO3tO2hCUKInmvWrFnMmjUr2mF0S205BJQN7I6YzjfrWnIlEHmM5QngDiDUTNsbzcNGLyqlUppbmFLqGqXUMqXUsqKiojaE24z3b4E/nwGhEEwYzdBvA6zft6Z9yxJCiF6iLQmguQtdmz1zrJQ6HSMB3GlOnwvs11ovb6b5s8BRwDigAHi0uWVqrV/QWk/UWk/MyMhoQ7jNGDgZKvJh99dkn3o2cT7YtPj91l8nhBC9WFsSQD7QP2I6BzjoDgul1BjgL8B0rXWJWT0ZOE8plQe8DpyhlHoVQGtdqLUOaq1DwJ8xDjV1jWHngM0N696kz8lnAFD5VevX3QohRG/WlgSwFBiilBqklHIAFwHvRTZQSg0A3gYu1Vpvqa/XWt+ltc7RWuear/tUa32J+ZqsiEX8EFjXoU9yKE4PDDsL1r+DLSmR8gEppKzPxxf0ddlbCiF6lu7eHfR9991HdnZ2+ET2Bx90/LqZVhOA1joA3Aj8D+NKnn9qrdcrpa5TSl1nNrsHSAOeMS/pbMvF+g8rpdYqpdYApwMH3x/dmUZdCDXFsHMhtuPGM2R3iNX5S7v0LYUQsaFpAuiIyy+/nP/+97/Nzrv11lvDdw+fc07Hr5xvU19A5iWaHzSpey6ifBVwyI72tdYLgYUR05ceRpwdN2QaOJNg7VsMOP08St76hPWfv89xA1vuLEqImPbhbNi3tnOX2Xc0nD3nkE3y8vI466yzmDJlCkuWLGHs2LHMmjWLe++9l/3794e7b7jllluora3F7Xbz0ksvMWzYMB577DHWrVvHiy++yNq1a7n44ov55ptviIuLO+h9SkpKuPjiiykqKmLSpEkHdQf95JNP4vP5OP7443nmmWewWq14PB6uvfZaFixYQEpKCq+//jqfffZZuDtot9vNV199BRjdQf/73//G7/fzr3/9i+HDh7dpFZ1yyink5eW1cYV2TMx0BYHNCcd8Hza9T9qE4whZoHbJ19GOSgjRjG3btnHzzTezZs0aNm3axN///ne++OILHnnkEX7/+98zfPhwFi1axMqVK/nNb37DL3/5S8BICtu2bWPevHnMmjWL559/vtmNP8D999/PlClTWLlyJeeddx67du0CjDtp33jjDb788ktWrVqF1WoNJ53q6mrGjx/PihUrOPXUU7n//vu58MILmThxIq+99hqrVq3C7XYDkJ6ezooVK7j++ut55JFHAKMzuKb3JIwbN46TTjqpTevlT3/6E2PGjOGKK66QJ4IdttEXwKpXsRYspvyoPmRs2EdtoBa3zR3tyIToflr5pd6VpDvog11//fXcfffdKKW4++67uf3223nxxRfbtax6sZUAck+B+AxY9yau4ydx1Gvvs3LHl5w09MxoRyaEiCDdQR8sMzMzXL766qs599xzW42vNbFzCAjAaoORP4Qt/yN38hlYNOz8TO4HEKKnicXuoOt7JAWYN2/eQVcJtUdsJQAwrgYKeEn2lOC3K3xLV0Q7IiHEYYrF7qDvuOMORo8ezZgxY1iwYAGPP/54h94HYqw7aMDoF+iJMZAxjC/fKaeycA+TP/maBEdC5wQpRA8m3UEfWkx2B92rKAWjzocdC4ifeCwDizQrNy2MdlRCCHHExV4CABh9IYQCDBoQD8Cuz7q8J2ohRJRId9Ati62rgOpljoL0YSTWLmGny4petjraEQkhuoh0B92y2NwDUApGX4ja/RWVx2STs7mMA94D0Y5KCCGOqNhMAACjLgA0SUcl0/cArFzzUbQjEkKIIyp2E0DaUdDvWAa7jWfd7FkozwkWQsSW2E0AAKMuJN63nhqPDcuK9dGORgghjqgYTwDno5Si5qhEcrdWUlTTzkdOCiF6vO7wPIDdu3dz+umnc8wxxzBy5Ej++Mc/hueVlpYybdo0hgwZwrRp0zqlM7jYTgCJ/WDgZFLTqkiphtVL/xPtiIQQPVBnPQ/AZrPx6KOPsnHjRpYsWcLTTz/Nhg0bAJgzZw5Tp05l69atTJ06lTlzOt5ZX2xeBhpp9AUMXv9/5JHJvkXz4dTLox2REN3CQ988xKbSTZ26zOGpw7lz0p2HbBPLzwPIysoiK8t4WGJCQgLHHHMMe/bsYcSIEbz77rssXLgQgJkzZ3Laaafx0EMPtWm9tyS29wAARvwAd6KiItmKY2Xn/rELIdpHngdgJMKVK1dy/PHHA1BYWBhODllZWS32cXQ4ZA8gLhWOOoO6fusZtKOGgoo9ZCVmRzsqIaKutV/qXSnWnwdQVVXFBRdcwBNPPEFiYmKr7dtLEgDAqAvps2AxekMiaxa/S9ZZP4t2RELEtFh+HoDf7+eCCy5gxowZjRJMZmYmBQUFZGVlUVBQEE5KHSGHgACGn8PgTKNL2eLPF0Q5GCFEa3rr8wC01lx55ZUcc8wx3HbbbY1ee9555zF37lwA5s6dy/Tp01t9v9ZIAgBwJuAY+13KUsG9eis9qYtsIWJRb30ewJdffskrr7zCp59+Gj4/8MEHRmeVs2fPZv78+QwZMoT58+cze/bsdr9Pvdh7HkBLNv6bz399MwmbnaQufJ8B6Ud1zfsI0Y3J8wAOTZ4H0FsdPY2+WeAMwLpF86IdjRBCdDlJAPXsLgaddAYhBQe+XBTtaIQQnUSeB9AyuQoogm3SxZRkfE7Cmh1orQ95hl8I0TPI8wBaJnsAkQadispR5O4JsqNgQ7SjEUKILiUJIJLVRv/x47CFYOPHr0c7GiGE6FKSAJoYcN71BKxQ9cXCaIcihBBdShJAE5ajT6YkC5I3lxDSoWiHI4QQXUYSQFMWC7ZjcuhfqNm8Se4KFiJWdPfnAdx3331kZ2cfdINYR0gCaEbuWT/BAmx77/lohyKE6AGOxPMAwLjTub77iHPOOafj79fhJfRC2dMuY5XjUWpXbYx2KEJEzb7f/566jZ3bRbrzmOH0Nbtubok8D6D55wF0hTbtASilzlJKbVZKbVNKHdQBhVJqhlJqjTksVkqNbTLfqpRaqZR6P6IuVSk1Xym11RyndPzjdA7lcFCWG0/6twECB3ZHOxwhYo48D+Dg5wEA/OlPf2LMmDFcccUVnfJISLTWhxwAK7AdGAw4gNXAiCZtTgJSzPLZwNdN5t8G/B14P6LuYWC2WZ4NPNRaLBMmTNBHyhe/u15vGDZcr3nr/47YewoRbRs2bIh2CHrnzp366KOPDk9feuml+tVXX9Vaa719+3Y9duxYvWvXLv2DH/xAjxw5Uo8aNUoPGzYs3H779u06Pj5e33bbbYd8n7Fjx+rt27eHp1NSUnRRUZF+6qmndFZWlh47dqweO3asHjp0qL733nu11lpbLBbt9/sbxaK11qeeeqpeunRpeFkDBw7U+fn5WmutlyxZoqdOnXpY66CyslKPHz9ev/XWW+G6ffv26UAgoIPBoP7lL3+pZ82a1exrm/sOgWW6mW1qWw4BTQK2aa13ACilXgemA+EDU1rrxRHtlwA59RNKqRzge8DvzERQbzpwmlmeCywEovcEiiaOOucyyv62gJ2ff8zolp/5IIToAvI8gOafB1Dv6quv5txzz201vta05RBQNhB5HCTfrGvJlcCHEdNPAHcATa+pzNRaFwCY42afbqCUukYptUwptayoqKgN4XaOzDGTqHYr/Hk1ULL9iL2vEKJ1sfg8gPouqQHmzZvHqFGjWn2/1rQlATSX4prtQ1opdTpGArjTnD4X2K+1Xt7eALXWL2itJ2qtJ2ZkZLR3MYdNWSwcGNGPzHwLvtVyV7AQ3UksPg/gjjvuYPTo0YwZM4YFCxbw+OOPt/t96rX6PACl1InAfVrr75rTdwForR9s0m4MMA84W2u9xax7ELgUCAAuIBF4W2t9iVJqM3Ca1rpAKZUFLNRaDztULF36PIBmfPXMfSQ/+Qbey1wce9cKkM7hRC8nzwM4tFh8HsBSYIhSapBSygFcBLzXZOEDgLeBS+s3/gBa67u01jla61zzdZ9qrS8xZ78HzDTLM4F32/LhjqShUy8AYNfOA7BvbXSDEUKITtbqSWCtdUApdSPwP4wrgl7UWq9XSl1nzn8OuAdIA54xT4oEmss2TcwB/qmUuhLYBfyo/R+ja6QOHcXWJBuhfQ5Y9yZkjYl2SEKIw/TSSy81uqMWYPLkyTz99NOHvazu/uv/cMkjIVsx/+rzSFq2ldE/DeG+4SuISz2i7y/EkbRx40aGDx8uz8LoobTWbNq0SR4J2VkST5xMQi2sK66GN6+AYCDaIQnRZVwuFyUlJfSkH4bCoLWmpKQEl8vV5tdIVxCtGD7tR+x9+GX26DEct2MBzL8Hzvp9tMMSokvk5OSQn5/PkbzkWnQel8tFTk5O6w1NkgBakdR/MGsznFi2lsDV18KSp6HvKBj302iHJkSns9vtDBo0KNphiCNEDgG1Qd2EYxi4tYJl2afBoFPg3zfD7qXRDksIITpEEkAbHHfrAygUW35/L77z/wyJ/eCNGVDR8e5fhRAiWiQBtEHiwKMIXHwux66u4l//+RNc9A/wVcPrM8Df/rv+hBAimiQBtNG4W++jOsVFyrNvs91mh/NfgL0rjMNBcsWEEKIHkgTQRpa4OLLuvItBhZp3n/g5oWFnw+m/gjVvwOKnoh2eEEIcNkkAh6Hf9B9RMzKXye/n8e7Kv8Mpv4AR0+Hje2Hrx9EOTwghDoskgMOglGLEbx/F44Xdf3yEYm8J/OBZ6DPSuEmseGu0QxRCiDaTBHCY3CNGYP/B2ZzxTR3Pv3M3OOLh4r+D1Qb/uBi85dEOUQgh2kQSQDsMvuNuQnFOjpr7GZ/vXgTJA+DHr0DZTnjzSggFW1+IEEJEmSSAdrClpND35zczNk/zzsu/osZfA7mT4eyHYdt8+OT+aIcohBCtkgTQThk/vYRQbjbn/qeY55eaVwEddyVMvAK+/COs+Wd0AxRCiFZIAmgnZbeTe89v6HsADsx9hY0lG40ZZz0EAyfDezfBnhVRjVEIIQ5FEkAHxJ90Eq4zTuUHX4V47L+/JhgKgs0BP/4bxPcx7hSu3BftMIUQolmSADoo+5e/xqmtHPf2Rv6x6R9GZXy6cWWQ9wC8cSkE6qIaoxBCNEcSQAc5cnJIv+JKTl6v+eC9xymoKjBm9B1t3COQ/w28f5t0FyGE6HYkAXSC9GuvQfVJ55L/efn9V79reJrSyB/AKXfAqlfhqz9JEhBCdCuSADqBJS6OrF/cyaCCEOrDBXy8K6JbiNPuguHnwke/hr+dB3tXRS1OIYSIJAmgkySe+z1c44/l0s8UTyz8HZW+SmOGxQIXvmRcHbRvHbxwKrx1NRzYFd2AhRAxTxJAJ1FK0fdXvyK+JsTpnxTxxxV/bJhpc8AJ18HNq2DKrbDxPXhqInx0N9SWRS1mIURskwTQidwjR5Jy4QWcvVzz5ZdvsGr/qsYNXElw5n1w4zIYdb7RjfSTx8JXT8uVQkKII04SQCfLuOUWrHHxXLXAyv2L78Mf9B/cKLk//PA5uHYRZI2F//0S/nQcrHtLThQLIY4YSQCdzJaWRp8bb2TEtjqSlm3l5fUvt9w4awxc+g5c8hY4PEaX0n8+A/K+PFLhCiFimCSALpDy05/iOOoorvvMxV+WP8uuikOc8FUKjj4Trvscpj8NlQXw8jlG19JFW45c0EKImCMJoAsou53Mu+4iqaiG7y3T/GbJbxruDWiJxQrHXgI3rYAzfg07P4dnToD3b4Wq/UcmcCFETJEE0EU8UybjOeMMzv8yxJYtS/j3jn+37YWOOONRkz9fafQsuuJvxoniz/4gJ4qFEJ1KEkAXypx9J7ag5oYlSTyw5AGeXfWs8eyAtvBkwPcegZ99DYNPgwUPwLOTjT0DIYToBJIAupBjwABSL7+cMctKOb9uJM+sfoZz3j6Hf27+J/5QM1cHNSf9aLjoNZjxFgR9MPdceOcGqCnt2uCFEL2eJIAulnbttdgyMvjxvBJeHfUwAxMH8tslv+X8d8/nk12ftH5uoN6QM+FnS2DyLbD6H/CnibD6dblsVAjRbpIAupjVE0/WA7/FX1CA8/I7+cOmcTx1wh9QSnHLglu47MPLDr5hrCWOOJh2v3H/QOpgmHct/G06lGzv0s8ghOidVJt/gXYDEydO1MuWLYt2GO0SKCpi/yOPUP7ue9j6ZZEx+04+HljB06ufobi2mKkDpnLz+JsZlDSobQsMhWD5i/Dx/cbJ4VN+AZNvNrqdEEKICEqp5VrriU3r27QHoJQ6Sym1WSm1TSk1u5n5M5RSa8xhsVJqrFnvUkp9o5RarZRar5S6P+I19yml9iilVpnDOR35gN2dLSODfg89xMBX/obVk0DBz29h0qPzeWfCc9ww7ga+2vsVP3z3hzyw5AGKa4tbX6DFAsddBTcuhWFnGyeJn5sC3y7u+g8jhOgVWt0DUEpZgS3ANCAfWApcrLXeENHmJGCj1rpMKXU2cJ/W+nillALitdZVSik78AVws9Z6iVLqPqBKa/1IW4PtyXsAkXQgQNlrr1H05FNon4+0q69CXXIBz295mTe3vIndamfWyFnMHDmTOHtc2xa65X/wn/+D8l0w/jI4836IS+3aDyKE6BE6sgcwCdimtd6htfYBrwPTIxtorRdrreu7tVwC5Jj1WmtdZdbbzaHnHHPqIspmI3XmTAZ/+AEJZ51F8TPPcuBHM/l5zYnMmz6PKdlTDv+KoaHfhRuWwEk3wcrX4OlJsOZfcpJYCNGitiSAbGB3xHS+WdeSK4EP6yeUUlal1CpgPzBfa/11RNsbzcNGLyqlUppbmFLqGqXUMqXUsqKiojaE23PY+/Qh+w8PM+Bvc7HEucn/2Q1Y75jDnMG38Oo5rza6Ymjh7oWtL9ARD995AK5ZCMkD4O2r4NXzoXRHF38SIURP1JZDQD8Cvqu1vsqcvhSYpLW+qZm2pwPPAFO01iVN5iUD84CbtNbrlFKZQDHGHsFvgSyt9RWHiqW3HAJqjvb7KX31NYqfegodCJB29dWkXnUli4qW8PiKx9lZvpOLh1/MLyb+ArvV3voCQ0FY+lf45DcQ8sPEKyGhL9jdYI+LGLuaTLvBZtbZnEZfRUKIHq2lQ0BtSQAnYhzT/645fReA1vrBJu3GYGzgz9ZaN9uLmVLqXqC66XF/pVQu8L7WetShYunNCaCev3A/+x9+mIr//Ad7Tg6Zv/olrlOn8MTyJ/jbhr8xJmMMj576KH3j+7ZtgRV74cM7jYfQHDbVkBTi0407ko+eBrmTjTohRI/QkQRgwzgJPBXYg3ES+Kda6/URbQYAnwKXaa0XR9RnAH6t9QGllBv4CHhIa/2+UipLa11gtrsVOF5rfdGhYomFBFCvesnX7Pvtb/Ft347n9NPJ/NUvWRjYwD2L78FhcTDnlDmc1O+kti8wGIBALfhrwV8Dfq85NutanFcDAS+U5UHeF0bZ5oLcKUYyGDIN0o7qsvUghOi4dicA88XnAE8AVuBFrfXvlFLXAWitn1NK/QW4APjWfElAaz3R3CuYa77OAvxTa/0bc5mvAOMwDgHlAdfWJ4SWxFICANA+H6WvvErR008DkP3II5RMGMStC29l+4Ht/Gzcz7hmzDVY1BG6n89fC99+CVs/hm3zoWSbUZ8yyOjSesg0IzE44o9MPEKINulQAuguYi0B1PMXFJB/08/xrl9PnzvuwDXjRzzw9QO8v+N9pmRP4cEpD5LsSj7ygZXuhG0fG8PORcbegtUJA08yksHRZ0L6UDmPIESUSQLo4UK1tey9czaVH31E8o9/TOavf8W/dszjoaUPkeHO4LHTHmNk+sjoBej3wq7FsO0T2Dofijcb9UkDjH6MBk4GT6Zxb4I71RjbnNGLV4gYIgmgF9ChEEV/fJKS558n7oQTyPnjE2z07+a2hbdRXFvM7Emz+dHQH6G6wy/usm/NvYNPYOdn4Ks6uI093kwIKY0TQ3ic0lCOT4eEftLVhRDtIAmgFznwzjsU3H0Pjuxs+j//HDWZScz+YjZf7vmS7w/+PnefeDduWze6SifgM/YIakqhtjRiXNZk2hzXHqD5+wUVePpAYjYk5TQMidmQ1B+SsiG+j9FNhhAiTBJAL1OzbBn5N94EWpP91JO4j5vI82ue59lVz3JU8lE8ftrj5CblRjvM9gmFwHsAassakkJ1EZTvgfLdULEHyvONaX9149da7JDYr0lyqE8Q/SG5PzgTovKxhIgWSQC9kG/XLnZfdz2+3bvJuu8+ki84ny/3fMnsz2fjD/l5YPIDnDnwzGiH2XW0NpJEOCGYQ2SCqNgDOtj4de4UMxkMaBgn928ou1PkxLXoVSQB9FLBigr23HIL1Yu/Iu3qq8i49Vb21RRy+2e3s7Z4LTNHzOTmCTdjt7Th7uHeKBSEqkIjIRzYZQzlu+HA7oZx070Ie3yTpNAfUnIhc7TxHAY5xCR6GEkAvZj2+9n3wO848MYbJEw7k34PPUTAaeMPS//A65tfZ3yf8Tx8ysOkulIJ6ADBUJCgDhIIBQjpULgc1MHwvPpyfXuXzcWQlCG9L5FobRxmKt/VOCkc2NVQ5z3Q0N4eD31HQd8xkDXGGPc5Rq5oEt2aJIBeTmtN2SuvUDjnIVzDh5Pz7DPYMzP5z47/cP9X91MbqO3we7htbsZmjGVC5gQmZE5gdPpoXDZXJ0TfzXkrjA719q2FfWvM8dqGK5ssNsgYHpEURhuDK6nrYwv4jDh8VVBXBb5q8FVGlKugrtK4g1tZwWozzpNY7UbcVnvEtDWibD+4rc1p3AVudZhlp3HfR2/vMyroN/4G6ioa1mddlbme68v19ZXNt/HXGuvK7jYGR3xDuWlfXPa45uv6HQvxae36CJIAYkTlwoXsve12LB4POc8+g3vkSHaU72B+3nwsyoLVYsWqrNgsNmPaLFuVFavFik3Zwu1syhZuX15Xzor9K1heuJytZVvRaGwWG6PTRzO+z3gmZE5gXJ9xJDhi5ARrKARlO6FgdUNiKFgD1fsb2qTkGkmh7xhIHwI6ZGxMgnXGU9zqy0G/OR1ZbtIu4I3YoFc1bPSDvqitgkasjpaTg83ZMD9cjmzjMMeuiHKT5VmdZkKKHKxNpluqsxkx1lWaFxccAG95C2VzOrLc3CXMzbHHg9MDDo9xoYEzoaFsdxvfo7+6ofuVyO5W6ut81Qefs6o34y3jnpp2kAQQQ7ybN7P7+usJlh0g+w8Pk3Bm554ILq8rZ9X+VSzfv5zlhcvZULyBgA5gURaGpQxjQuYExmeOZ3yf8aS52/eLpceqLDSTweqGvYW2dMetLOZGzmFuBB0RGz+7Ma9+4+LwNF8+qC4eHAlG2eZqSEAhvzkONEyHgi3PCwaMcaDOSDgBr1GuT1rhcv08cxw5L9zW1zAOtzHL0XxUiMMDrmRwJxtjV1JD2W1OR27QG5XNdW6xdk4sAV/zySF9iBFLO0gCiDGBoiJ233gj3jVr6XP7baReeWWX3SBW469hbfFaVhQaewiri1bjDXoByE3MZULmBEakjcBj9+CyuXBZXca4mbLT6sTaWf9I3YW3wuhMz2pv2LCHN/TmRt9qi3aU0aW1kXTCiaTOTBC+hrpQoMkQjEhUEdNN54cCRvJzJkRs4JMbb+x7+fqXBBCDQl4vBb/8JRUffIjnzKnETZyIo39/7Dn9ceRkY4nvmk7b/EE/G0o3hBPCiv0rqPRVtvn1DosDp82J2+rGZXPhtDnx2D1ke7LJScghx5MTHmfEZRy5zvCE6KEkAcQoHQpR/OyzlL48l1Bl442wNTUVe/8cHDn9jXH//tizc3D0z8HWty/K2jm/xEM6RGF1IbXBWuoCdXiDXmoDDWVvwHvwOOClLlhHbaAWb8BLha+CvVV72Vezj5AOhZftsDjo5+nXODFElOPt0jOpEJIAYpzWmlB5Ob7d+fjzdxvj3bvx78k3ynv3QjDi5JPdjr1fFo7sHOz9++MYlEv88cfjHDYMFcXr4P1BPwXVBeRX5pNfld9ovLtyN1X+xifsUl2p5HhyyPJkkeZKI8WVQqorlVRXKimuFFJcKaS50khwJMiehOi1JAGIQ9KBAP59+/Dv3o1v9278+XsaJYrggQMAWFNSiDv+eOJPOIH4E0/APmBA9+h8DiPJVfgqjGRQtZv8ynz2VO0hvzKfguoCSmtLqfQ3fyjKqqwkO5NJdaeS6kwNJ4r6cbo7nYGJA+mf0B+HVTqkEz2LJADRIf7CQqq/+oqar5ZQvWQJgcJCAGz9sog/4UTiTzyR+BOOx5aREeVID80f9FNWV0aptzQ8lHnLKPOWNZquHzdNGBZlIduTTW5iLrlJueQm5jIoaRC5ibmku9O7TTIUIpIkANFptNb4duZRvcRMCF9/TaiiAgDnkKOJO+FE4k88gbjjjsOa0LPvC/AFfZR5y9hfs5+8ijxjKDfG31Z8S12wLtzWY/cwMHFgODHkJuUyKHEQAxIHHLHeWbXW+EI+fEEfdcE6fEGjHCJEujudBHuCJKkYJAlAdBkdDOLdsDGcEGpWrEB7vWC14h41irgTTyDu2GOxJiVh8XiMIT4eS1xcp51ojoaQDrGveh955XnsrNgZTgx5FXnsq97XqG26Ox27xR6++S48tjSZbmZssViwYAlv2Os37v6Qv9FGvr7uUNw2N5lxmfSJ60OfuD7hcngcn0maK633XYrbBiEd4kDdAYpriymuLaaktiQ8Lqsrw26x47a5G12+7La5m62LvLy5fjqa61QSgDhiQj4ftStXhRNC7dq1jU8wR7DExTUkBY8HqyceS7wnIlHEYfV4sCQmYs/qhz27H/Z+/bC4uncXFDX+GnZV7gonh33V+xr1vRTSIaMcCjaqa3YcChEihNPqxGFx4LA2DE3rnFanUbZElM1zFsW1xRTWFLK/Zj+F1cZ4f+1+AqFAo9itykqaO42+cX3DiSLVlUpIh/CH/PiCPmNsJiR/yI8/6D9oXmRdUAdRqIY70M07zMODxUh29Xef15cj70qP/HxOqxOnzWmMI+tamLYqK6XeUoq9xga9fuMeubEv8ZYQbOYuXJfVRbIrmWAoiDfgpTZYe9A6a4t4ezxJjiSSnEkkOhPD5SRnEkmOhrpEZ2K4LsmZ1CndrUgCEFETrKqibstWQtVVhKqqCFZVEaqqJlRlTlc3ng5VVzeqIxQ6aJnWtDTs/fo1HrIbytbExCh80p4npEOUekuNZGAmhnCSMMf7a/aHr66yKRt2qx27xRjqE0z9tN1qx2FxhOfV19kt9nDCC+hAuBzueLCZjghDOmS0DRnj+r2c+qE9G+F6VmUlzZVGmjuNdHd6eGg6ne5OJ84Wd9BhM3/IH75cuT4phKfNy5wjp2v8NVT4KqjwVVBeV24MPmNcUVdBQLf8WZxWJ0mOJH5/8u85Puv4dn3elhJA7779TXQLVo+HuPHHtuu1Wmt0bS3BsjL8BQX49+41hj178O/ZS93mzVQtXIiuq2v0OovH0yg52LL6Yk1IxJLgwZqQgMWTgDXBg8UsW+IP/iePBRZlCW/oRqSNaLGdP+gP/xrvLoKhYPgQWP3YG/QelCjqk0WKKyX8WZOdyR267NdusWN32Dul7yutNTWBmoMSQ3ldeThhVPgqSHend/i9mpIEILo1pRQqLg5LXBz27Oxm22itCZaURCSHvQ3lvXupWbbM2JM4FIvFPARlJAVrQoKRHBI8WD0JWFNScB41GMfRR+PMzUU5es6loKG6OoIlJSiHA+VyY3E5UbbD+9e3W7tfN+BWi5U4Sxxx9rgOL0sHgwQPHCBYWkqgrIxgaRnBsohyeXnDnmj4qEnD0ZNGR1LCsyPqlMLiiceakIg1KdH4G0tMwpqYgCUhEVtSIhkJCfRNGozFeeS6FpcEIHo8pRS29HRs6em4x4xptk2outo49FRZSbCy0jjUVFlJsLKKUJVZFy4b8/z79hHaUmG8rqKi4R/aasUxcCDOo482hiHG2JGbi7If+Q1lsKoa/949DXtGEcnPv2cvweLig19kt2NxubC4XCi32xiHp11YXO6GstOFcjjQfj/a50P7fWifj5DPh/aZdT4fuq4uPP+geYGAsYdlsxkn/m1WlMVqls06qwVltTXUWSxGO7NO2e0oh90Y2+1gbyg3HhwH1WFRhMrLCZSWmRv5UoJlxgY/WFpKMPL7bcKSkIA1KanxBQv1e4uRe43NleurgiHjb7CiwrhA4hCUw4ElMRFrYqLxQ8Qsp111Ja5jjmnlr+HwSAIQMcESH2/0fZSZ2a7Xh+rq8O3cSd3WbdRtMwbv5k1Uzp/fsOGw2XDkDsR59JBGycExYECriUEHg+hAAO0PoP0+CASMDW4ggA4ECFXX4C/Ye9DG3b93L6Hy8kbLUnZ7+JyI87RTjUNg6enGsmq9hLy1aG8dIa8X7a0lVOtF13kJmfNCxSX467xmWy+6tpaQ34+y27HY7caeRNPB6cAS50YlJTU/32aDUAgdCkEwgA4E0cEABEPoYLChLhSEQLChLhhCBwNor9dcP/5DDi1dbBBmtWJNScGWkoI1NRXnsGHYUlOwpqQa9alGvTUlxZhOTu70vb2Qz2f8+CivIFRZQbCikmBFeTN1FYQqKggeOIBv17eEamo6NQ6Qk8BCdEjI68W3Y4eRFCKSgz8/vyExmN1qENLmBt0PPn/DBi0QaPZEd0uMw2H9sJnnNxzZ2Q3nOsyNfTS764imhkRqJgWfmRhCQeMy5MTEmDzXIyeBhegCFpcL14gRuEY0PoEaqq2lbvsOfNvNhLBnD1jMwxg2m3lowhhTP22zN8x3NLTDZsPicmHr2xdHdjaWpKSY3Ii1hbKah5WO4HH0nkwSgBBdwOJ24x41EveokdEORYgWxeZ+ohBCCEkAQggRqyQBCCFEjJIEIIQQMUoSgBBCxChJAEIIEaMkAQghRIySBCCEEDGqR3UFoZQqAr6NdhwtSAea6XWr25D4Okbi6xiJr+M6EuNArfVBD+zuUQmgO1NKLWuur43uQuLrGImvYyS+juuKGOUQkBBCxChJAEIIEaMkAXSeF6IdQCskvo6R+DpG4uu4To9RzgEIIUSMkj0AIYSIUZIAhBAiRkkCaAelVH+l1AKl1Eal1Hql1M1m/X1KqT1KqVXmcE4UY8xTSq0141hm1qUqpeYrpbaa45QoxTYsYh2tUkpVKKVuieb6U0q9qJTar5RaF1HX4vpSSt2llNqmlNqslPpulOL7g1Jqk1JqjVJqnlIq2azPVUrVRqzH56IUX4vfZzdZf29ExJanlFpl1kdj/bW0Tenav0GttQyHOQBZwHiznABsAUYA9wH/F+34zLjygPQmdQ8Ds83ybOChbhCnFdgHDIzm+gNOAcYD61pbX+Z3vRpwAoOA7YA1CvF9B7CZ5Yci4suNbBfF9dfs99ld1l+T+Y8C90Rx/bW0TenSv0HZA2gHrXWB1nqFWa4ENgLZ0Y2qTaYDc83yXOAH0QslbCqwXWsd1Tu8tdaLgNIm1S2tr+nA61rrOq31TmAbMOlIx6e1/khrHTAnlwA5XRnDobSw/lrSLdZfPWU8YPnHwD+6MoZDOcQ2pUv/BiUBdJBSKhc4FvjarLrR3CV/MVqHWEwa+EgptVwpdY1Zl6m1LgDjDw7oE7XoGlxE43+87rL+oOX1lQ3sjmiXT/R/AFwBfBgxPUgptVIp9ZlS6uRoBUXz32d3W38nA4Va660RdVFbf022KV36NygJoAOUUh7gLeAWrXUF8CxwFDAOKMDYrYyWyVrr8cDZwA1KqVOiGEuzlFIO4DzgX2ZVd1p/h6KaqYva9dRKqV8BAeA1s6oAGKC1Pha4Dfi7UioxCqG19H12q/UHXEzjHyFRW3/NbFNabNpM3WGvQ0kA7aSUsmN8Ua9prd8G0FoXaq2DWusQ8Ge6eLf2ULTWe83xfmCeGUuhUioLwBzvj1Z8prOBFVrrQuhe68/U0vrKB/pHtMsB9h7h2ABQSs0EzgVmaPPgsHlYoMQsL8c4Pjz0SMd2iO+zO60/G3A+8EZ9XbTWX3PbFLr4b1ASQDuYxwz/CmzUWj8WUZ8V0eyHwLqmrz0SlFLxSqmE+jLGycJ1wHvATLPZTODdaMQXodEvr+6y/iK0tL7eAy5SSjmVUoOAIcA3Rzo4pdRZwJ3AeVrrmoj6DKWU1SwPNuPbEYX4Wvo+u8X6M50JbNJa59dXRGP9tbRNoav/Bo/kme7eMgBTMHa31gCrzOEc4BVgrVn/HpAVpfgGY1whsBpYD/zKrE8DPgG2muPUKK7DOKAESIqoi9r6w0hEBYAf49fVlYdaX8CvMH4ZbgbOjlJ82zCOA9f/DT5ntr3A/N5XAyuA70cpvha/z+6w/sz6l4HrmrSNxvpraZvSpX+D0hWEEELEKDkEJIQQMUoSgBBCxChJAEIIEaMkAQghRIySBCCEEDFKEoAQQsQoSQBCCBGj/h9aFzFPigSfYQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "for d in max_depth:\n",
    "    df_subset = df_scores[df_scores.max_depth == d]\n",
    "    \n",
    "    plt.plot(df_subset.n_estimators, df_subset.rmse,\n",
    "             label='max_depth=%d' % d)\n",
    "\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eCRCaKlSwp7D"
   },
   "source": [
    "What's the best `max_depth`:\n",
    "\n",
    "* 10\n",
    "* 15\n",
    "* 20\n",
    "* **25**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BDLbx3N5xAoY"
   },
   "source": [
    "## Question 5\n",
    "\n",
    "We can extract feature importance information from tree-based models. \n",
    "\n",
    "At each step of the decision tree learning algorith, it finds the best split. \n",
    "When doint it, we can calculate \"gain\" - the reduction in impurity before and after the split. \n",
    "This gain is quite useful in understanding what are the imporatant features \n",
    "for tree-based models.\n",
    "\n",
    "In Scikit-Learn, tree-based models contain this information in the\n",
    "[`feature_importances_`](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html#sklearn.ensemble.RandomForestRegressor.feature_importances_)\n",
    "field. \n",
    "\n",
    "For this homework question, we'll find the most important feature:\n",
    "\n",
    "* Train the model with these parametes:\n",
    "    * `n_estimators=10`,\n",
    "    * `max_depth=20`,\n",
    "    * `random_state=1`,\n",
    "    * `n_jobs=-1` (optional)\n",
    "* Get the feature importance information from this model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "id": "UICqST2Qwhbj",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(max_depth=20, n_estimators=10, n_jobs=-1, random_state=1)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#@ TRAINING THE RANDOM FOREST MODEL:\n",
    "rf = RandomForestRegressor(n_estimators=10,\n",
    "                           max_depth=20,\n",
    "                           random_state=1,\n",
    "                           n_jobs=-1)\n",
    "rf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "importances = rf.feature_importances_\n",
    "std = np.std([tree.feature_importances_ for tree in rf.estimators_], axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEfCAYAAAAUfVINAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABD3klEQVR4nO2dZ7gkVbWG34+RnBFEBXEAESSqgJJUDBhQBK9kRAFFMQFmxEBSMYDiNV2CZIULqAgKoggDAoIMOGS5IoiAgSBJQCR898fePadOT/c5PXNqV52w3ufpp7uqumvt7nO6V+211/qWbBMEQRAE44352h5AEARBEPQiHFQQBEEwLgkHFQRBEIxLwkEFQRAE45JwUEEQBMG4JBxUEARBMC4JBxUEQRCMS0Z1UJJmSvqgpKWbGFAQBEEQwGAzqB2B5wJXSjpV0hskqfC4giAIgimOBlWSkDQf8Bbge8DTwLHAN23/s9zwgiAIgqnKQGtQktYFDge+BvwI2BZ4CLig3NCCIAiCqcwzRnuCpKuAB4DvA/vZfjwfukLSpgXHFgRBEExhRg3xSVrF9q1d+1a2fVvRkQVBEARTmkFCfGcMuC8IxjWS9pd0TNvjmErEZx6Mhb4hPklrAGsBS0r6r8qhJYCFSg8sGF9I+jOwPPBUZfcLbf91jOd8j+3zxza6wbD9pSbsDIKkA4EX2H5H22MpScnPXJKBu4EVbD+Z9z0D+CuwnG3lfTOAjYAnAAN/BE4HvtFZspgqf4+JxkgzqNVJWXtLAVtVbi8F9iw+smA8spXtxSq3eXZOdZB/jCYcE3Xc45QHgDdVtrcE7u/xvA/ZXhx4DvAxUvnMOVEyM86xPeIN2Hi058Rt8t+APwOv67F/SVICzd+Au4AvANPysVVJmZ73AfcCPwCWysdOIpUrPAb8C/gksDlwZz+7wIGk8PLJpCzS94xkv8dYDwROzo+nk66mdwfuIP2o7QVsCFxL+uH7duW1uwGXAt8CHgT+ALy2cvy5wFnAP4FbgD277FbH/SHgP6Qr+n8B1+Tn7Q7cBDwM3Aq8r3KOzYE7ST+ud+f3u3vl+MKkTNvb8/guARbOxzYCLsvv6Rpg8673dWu2eRuwS5/P7njgC93jqWx/Kn/+DwM3dz6bPp/5u4C/kP4nPtP1Hk7If4ubSP8Td/YaT36+gc8Cp1f2nQF8BnBl3wzSTL362pWAR4G3dI8zbuPnNlKI75O2vwrsLGmn7uO29+732mBKcQLwD+AFwKLAz0g/+EcCAg4FLiaFhn9E+iHY1/aukl5BJcQnafMB7G0NbAe8E1gQOGUE+4PwcmA14JUkB/ML4HXA/MDvJZ1u+6LKc88AlgX+C/hxThj6Zx7HDSRHtQbwK0m32v51n3Evy5whpbtJUYtb83jOlXSl7avz8WeTHPIKwBbAGZLOtH0/cBgpJL8J8Pc81qclrQD8HNg1v7fXAj/KIfxHgf8GNrR9s6TnAMsM+LnNRtLqJKe7oe2/SpoOTBvhJZuRIjQvBH4n6ce2bwIOIDmxVUh/y3MGMH8m8GFJS+XtV5D+x74w0ots/0XSzPz8nw1gJ2iBkUJ8N+X7mcBVPW7B1ONMSQ/k25mSlieFV/a1/Yjtu4FvkMIn2L7F9q9sP277HuDrwKvGOIbf2j7T9tMkp9fX/oAcYvvftn8JPAKcYvtu23cBvwFeUnnu3cARtp+w/b+kmcKbJT2P9KP7qXyuWcAxJKcwx7htP9ZrILZ/bvtPTlwE/JL0A9rhCeDgbP8c0uxr9VxEvwewj+27bD9l+zKn9ZV3AOfYPifb/hXpO71lPufTwNqSFrb9N9s3zMVn1+EpktNdU9L8tv9s+08jPP8g24/ZvoY0o1sv798e+JLt+23fSXKeo/Fv4GxgB9Lf/ay8bxD+yjw45KA5+s6gbJ8taRqwtu1PNDimYPyyjSsJDZJeRppp/K0Syp+PNINB0rNIPzKvABbPx3qtD8wNd1QeP38k+wPyj8rjx3psL1bZvst2tS7jdtKM6bnAP20/3HVsgz7j7omkN5FmES8kvY9FgOsqT7nPORkg82ge37KkxKVeTuH5wHaStqrsmx+40PYjknYAPg58X9KlwMds/2G0sVaxfYukfUkzl7UknQd81P3XKP/e4z1A+hyrn9Ogf8cTSTN1kUKNg7ICKfQZjFNGTDO3/RSwfkNjCSYedwCPA8vaXirflrC9Vj5+KGmdYF3bS5Cu5quL0t1FeI+QfpQByBdIy3U9p/qa0ezXzQpdi+orka7C/wosI2nxrmN39Rn3HNuSFiSFQA8Dlre9FCnENcgi/r2kWcOqPY7dAZxU+XyWsr2o7S8D2D7P9hak5IE/AEf3sTHsb0MKNw69GfuHtjcjOUQDXxlg3N38DVixsv28AV/3G9L4lyetvY1KnvWun18bjFMGqYP6vaSzJO0q6b86t+IjC8Y9tv9GCkMdLmkJSfNJWlVSJ4y3OCkM9UBeC+meif+DtN7Q4f+AhSS9WdL8pAXwBcdgv26eBewtaX5J2wEvIoXP7iBdiR8qaaEsDfZuUlJIP/4BTM/hOYAFSO/1HuDJPJt6/SCDyuHOY4GvS3qupGmSNs5O72RgqyzyPC2Pb3NJK0paXtJbJS1KcvT/YngZQZVZwJaSlpH0bGDfzgFJq0t6Tbb3b9LMs995RuI04NOSls7/Lx8a8P2blGH81q4Z7hxIWiT/f/wU+B2DrXMFLTGIg1qGlIX1GoZSzd9SclDBhOKdpB/XG0nhuzNIV7MAB5HKEh4kLdT/uOu1hwKfzWtaH7f9IPAB0vrNXaSr9jvHYL9uriAlVNwLfBHY1vZ9+dhOpAX+vwI/AQ7I6z39OD3f3yfp6hwe3Jv0I30/sDNpPWVQPk4KB15JyiT8CjBfdp5bA/uTnN8dpAuF+fLtY3nM/yStD36gz/lPIq0X/Zl0UfC/lWMLAl8mfS5/Jzny/edi7B0OJv29bwPOJ/0tHx/xFRnbN4yyfvZtSQ+TLgyOIM1W35idezBOGVjNPAimMpJ2I2Ucbtb2WKYKkt4P7Gi71Iw4GOcMIhZ7HHPGz7G9R5ERBUEwJclp7qsAvyXNVD8GfLvVQQWtMkhFe7VGYCHgbaSQQBAEQZ0sQKpfW5lUVHwq8N02BxS0y1yH+PKi7vm2X1NmSEEQBEEwYMPCLlYjpdAGQRAEQTEGWYN6mLQGpXz/d+auGK5Wll12WU+fPr0t80EQBEHNXHXVVffa7q55HN1BOSkAjxumT5/OzJkz2x5GEARBUBOSbu+1fyDZ/1yYuxlpBvUb22fWN7QgCIIgmJNR16AkfZfUhuA64HpgL0nfGeTkkt4o6WZJt0jab4TnbSjpKUnbDjrwIAiCYHIzyAzqVSTBWANIOoHhApY9yTpq3yG1BbgTuFLSWbZv7PG8rwDnzeXYgynI5ptvDsCMGTNaHUcQBOUZJIvvZoZn7T2P1NBtNF4G3GL7Vtv/IdU0bN3jeR8myY7cPcA5gyAIginCIA7qmcBNkmZImkHSPFsuC8iOpBW2AsPl8u/M+2aTBSHfBvzPSAOQ9F5JMyXNvOeeewYYchAEQTDRGSTE9/l5PHevNgHdVcFHkJq8PTW8i0HXi+yjgKMANthggxAPDIIgmAIMkmZ+EYCkJarPz22uR+JOhvdzWZE5JZI2AE7NzmlZkpz/k5ElGARBEAxSqPte4BBSj5enGSrYXWWk15Fk/1eTtDKpdcKOpBYCs7G9csXO8cDPwjkFQRAEMFiI7xPAWrbvnZsT235S0odI2XnTgGNt3yBpr3x8xHWnIAiCYGoziIP6E/DovJzc9jl0dazs55hs7zYvNoIgCILJySAO6tPAZZKuoNLd0vbexUYVBEEQTHkGcVBHAheQinOjPXIQBEHQCIM4qCdtf7T4SIKgwvT9ft5z/99vvW/E4wB//vKbi4wpCIJmGaRQ98JcKPscSct0bsVHFgRTmM0333y2rFMQTFUGmUF1UsM/Xdk3SJp5EARBEMwzgxTqrjzac4IgCIKgbvo6KEmvsX1B7gU1B7Z/XG5YQRAEwVRnpBnUq0jZe1v1OGYgHFQQBEFQjL4OyvYB+X735oYTBEEQBIlBsviCIAiCoHHCQQVBEATjkkHSzINg3PDsnb/c9hCCIGiIgRyUpE2A6QzvB3VioTEFwZRhXhUzQi0jmAoM0g/qJGBVYBbwVN5tIBxUEARBUIxBZlAbAGvajlbrQRAEQWMMkiRxPfDs0gMJgiAIgiqDzKCWBW6U9DuG94N6a7FRBUEQBFOeQRzUgaUHEQRBEATdDCIWe1ETAwmCIAiCKiOJxV5iezNJD5Oy9mYfAmx7ieKjC4IgCKYsI2nxbZbvF29uOEEQQBQkBwGE1FEQBEEwTgkHFQRBEIxLwkEFQRAE45KBHJSk50t6XX68sKRYlwqCIAiKMqqDkrQncAZwZN61InBmwTEFQRAEwUAzqA8CmwIPAdj+I/CskoMKgiAIgkEc1OO2/9PZkPQMhtdFBUEQBEHtDOKgLpK0P7CwpC2A04Gzyw4rCIIgmOoM4qD2A+4BrgPeB5wDfLbkoIIgCIJgVAdl+2nbR9veDngvcMWgvaEkvVHSzZJukbRfj+O7SLo23y6TtN7cv4UgCIJgMjJIFt8MSUtIWobUVfc4SV8f4HXTgO8AbwLWBHaStGbX024DXmV7XeAQ4Ki5HH8QBEEwSRkkxLek7YeA/wKOs70+8LoBXvcy4Bbbt+Yki1OBratPsH2Z7fvz5uWkFPYgCIIgGMhBPUPSc4DtgZ/NxblXAO6obN+Z9/Xj3cC5vQ5Ieq+kmZJm3nPPPXMxhCAIgmCiMoiDOhg4jzQbulLSKsAfB3ideuzruXYl6dUkB/WpXsdtH2V7A9sbLLfccgOYDoIgCCY6gzQsPJ2UWt7ZvhV4+wDnvhN4XmV7ReCv3U+StC5wDPAm2/cNcN4gCIJgCjCqg5K0EGl2sxawUGe/7T1GeemVwGqSVgbuAnYEdu4690rAj4Fdbf/f3A09CIIgmMwMEuI7CXg28AbgItJM6OHRXmT7SeBDpPDgTcBptm+QtJekvfLTPg88E/iupFmSZs7DewiCIAgmIaPOoIAX2N5O0ta2T5D0Q5LTGRXb55AKe6v7/qfy+D3Ae+ZmwEEQBMHUYJAZ1BP5/gFJawNLAtOLjSgIgiAIGGwGdZSkpYHPAWcBi5FCc0EQBEFQjEGy+I7JDy8CVik7nCAIgiBIDCJ1tLyk70s6N2+vKend5YcWBEEQTGUGWYM6npQU8dy8/X/AvoXGEwRBEATAYA5qWdunAU/D7PTxp4qOKgiCIJjyDOKgHpH0TLJMkaSNgAeLjioIgiCY8gySxfdRUvbeqpIuBZYDti06qiAIgmDKM6KDyj2dXpVvq5MEYG+2/cRIrwuCIAiCsTJiiM/2U8DWtp+0fYPt68M5BUEQBE0wSIjvUknfBv4XeKSz0/bVxUYVBEEQTHkGcVCb5PuDK/sMvKb+4QRBEARBYhAliVc3MZBg7Gy++eYAzJgxo9VxBEEQ1MEg/aC+BHzV9gN5e2ngY7Y/W3hsQR+m7/fznvv/fut9Ix7/85ffXPtYwikGQVCKQUJ8b7K9f2fD9v2StgTCQY0znr3zl4udezw5xSAIpgaDOKhpkha0/TiApIWBBcsOK5golHSKQRBMbQZxUCcDv5Z0HCk5Yg/ghKKjCoIgCKY8gyRJfFXStcDrSIW6h9geqKNuEARBEMwrg8ygAG4CnrR9vqRFJC1u++GSAwuCIAimNoP0g9oTOAM4Mu9aATiz4JiCIAiCYCA18w8CmwIPAdj+I/CskoMKgiAIgkEc1OO2/9PZkPQMcuuNIAiCICjFIA7qIkn7AwtL2gI4HTi77LCCIAiCqc4gDmo/4B7gOuB9wDlEkW4QBEFQmEHSzJ8Gjs63IAiCIGiEvg5K0nWMsNZke90iIwqCIAgCRp5BvSXffzDfn5TvdwEeLTaiCUZfjbof7gf0lwIKjbogmBqEoPK809dB2b4dQNKmtjetHNpP0qUM7w8VdBEadUEwteh3scpGnxjxeFys9meQJIlFJW3W2ZC0CbBouSEFQRAEE4nNN9989kyxTgaROno3cKykJUlrUg+SBGNHRdIbgW8C04BjbH+567jy8S1JYcPdxtJKvu8VDBFyC4JgcjHS791ozOvvXdOzxEGy+K4C1pO0BCDbDw5yYknTgO8AWwB3AldKOsv2jZWnvQlYLd9eDnwv39dOhNyCIAgmFoOE+ACw/dCgzinzMuAW27dmJYpTga27nrM1cKITlwNLSXrOXNgIgiAYmFKhqKAMg6qZzwsrAHdUtu9kztlRr+esAPyt4LiCoCfzGjKZSOHhNsLg85rpWsJmJCxMLGSXkdWTtB3wBtvvydu7Ai+z/eHKc34OHGr7krz9a+CTOaxYPdd7gfcCrLTSSuvffvvtRcYcBEEQNI+kq2xv0L1/oBlUztybXn2+7RNHedmdwPMq2ysCf52H52D7KOAogA022CCEaoMgCKYAozooSScBqwKzgKfybgOjOagrgdUkrQzcBewI7Nz1nLOAD0k6lRT+e9B2hPeCIAiCgWZQGwBrei5jgbaflPQh4DxSmvmxtm+QtFc+/j8k4dktgVtIaea7z42NIAiCYPIy6hqUpNOBvcfLzEbSPcC8LkItC9xb43DGm72pYnMqvMepYnMqvMepYnMs9p5ve7nunYPMoJYFbpT0O+Dxzk7bb53HgYyJXm9iUCTN7LUQV4qm7U0Vm1PhPU4Vm1PhPU4VmyXsDeKgDqzTYBAEQRAMwiBKEhc1MZAgCIIgqDKqkoSkjSRdKelfkv4j6SlJDzUxuAIcNcntTRWbU+E9ThWbU+E9ThWbtdsbJEliJilF/HRSRt87gdVs71/3YIIgCIKgw0CFurZvkTTN9lPAcZIuKzyuIAiCYIoziIN6VNICwCxJXyXp5EU/qCAIgqAog6iZ75qf9yHgEZI00dtLDioIgsmHpHMkTW97HMHEYSCxWEkLAyvZvrn8kOpD0qrAnbYfl7Q5sC6pvccDDdiekJ/ZeCd3d17N9nGSlgMWs31bzTZuI8l5zd5V2bbtVeu0N1WQtD3wBeAE4Ku2n2jA5itHOm774sL2pwHLM1zH9C8lbU4mBkmS2Ao4DFjA9sqSXgwc3Fah7twgaRYpsWM6SXLpLGB121sWttvoZybphaRmj8vbXlvSusBbbX+hhL1scxHgYyQnvKek1Uif7c8K2jyA9Pdc3fYLJT0XON32pjXbeWbXrvmA7YGPA1fbLhJBkPQuYB9g9bzrJuC/BxBmHovN1UmdAtao2Dy61IWVpEWBzwNvBE4Cnu4cs/31AvbO7rHbwHrAiran1W2zYvvDwAHAPxh6n7a9biF72wG/sP2wpM8CLwW+MJYu5QPY/CrpouMx4Bekz3Vf2yfXcf5BQnwHkpoPPgBgexbpB38i8LTtJ4G3AUfY/gjQREPEA2n2Mzsa+DTwRLZ3LSnzsiTHkZRFNs7bd5L+UUvyNuCtpFAztv8KLF63Edv32b4PuB94C3Ah6X2+uaBzeiewL8npP5fUF+2TwD75WAmbGwMzgIdJKcJHkz7bCyVtVMIm6X/0EWBB0t+ueqsd21tVb8BXgPlJa+nblLBZYR/SxdRattfJtyLOKfO57Jw2A95Amql+r6A9gNfbfoj0PbkTeCHwibpOPkiSxJO2H5RUl80meULSTsC7gK3yvvkbsNv0Z7aI7d912XuysM1Vbe+QP19sP6byb/g/ti3JMPtqvHYkzQ/sAXwEuATY2vafStiq8AHgbbb/XNl3gaS3k7pRl5hFfR7YyfaMyr4zJV1AuvJ/U53GJL0R+DopkvFS24/Wef5RbL8W+Bxp9vQl279qwOwdwNx0IR8rnW4Tbwa+Z/unkg4sbLPze7olcIrtf9b5MzCIg7pe0s7AtBzG2RuYKGnmuwN7AV+0fVtu/VHL1HMUmv7M7s3rbZ0f7m0p35X4P3mdrWNzVSpajYU4TdKRwFKS9iQ5kaML2LmN5OCPAP4CrCdpvc5B2z8uYHOJLufUsfVnSUsUsAfpImNGD5sXSSpR5PkZYDvbN1R3SnoesKPtr9VtUNKbs90Hgc/YvrRuGyNwKzAjN2at6pjWHsrM3JW/H68DviJpQQaLko2FsyX9gRTi+0BeF/53XScfZA1qEdIf+PWkxeLzgENs1zaIyUbTn5mkVUghmk1IYanbgHf0+sGr0eYWwGeBNYFfApsCu/X6wStgd/bnWuJKWNLxDE+SqGLbexSweZXt9ef2WEGbV9t+ad02K+dfFtgO2InUqPTHtj9ewM7TpNDTNfT4m5ZcS89rpnNg+6BC9hYhre1dZ/uPkp4DrGP7lyXsVewuDTxk+6k8hiVs/72Wc89lm6cJgaTr6P8DQ+E4cGvkkNd8th9uyN4zgY1IzuJy241I++cZRTUr6p9N2M22l7f9jwLnfZTUF22OQ8AqtmsPZ0q6mxQ+7GVze9vL12xvcdI64s6ktYqfADvYXrFOO102XzXS8Sa0RvP7tu1/NWBraVIpUPX7UTJJYhoppDi9y2Yts8S+IT5JZ430wnGexfeWfP/BfH9Svt+F1BixKDlzqNtBPgjMBI6sayYl6aN99gPFsqK6r6o7ocSVJK1U+MvwPuBgUjjhaYbSv1cpZTPbXZJU+7cz8CJSAkPdvKjAOUdjpMXsmQXs3Q38jjTzviSvJ76tgJ3Z9HNAnbAiUMxBSVqb9NuzTN6+F3hnd4izRnuHALsBf6JSFgG8poS9zNmkkN51VDIy66LvDEqpMeAdwCnAFaQfg9lMBJVzSZd2pyD32lfA7jeB5UifHcAOwN+BhUnT311rstMJIawObEhafIaUEHKx7ffUYafL5oX54UKklO9rSP8b6wJX2N6sbpsV238ENm5ippbX195KckovJWWZbUP6XGv/Ik4FJH2E5BQWBX4I/C/wK9tFLzAq9qthxRWAn5QIK1bsXUZa97owb29OStDYpJC9m0khvf+UOH8fm9eWjEiNlCTxbGAL0h9zZ+DnpCyNIt6/EItK2sz2JQCSNqEZmaaX2K4WCJ4t6WLbr5RU2+fXiWVL+iUpK+rhvH0gSdy3dmy/Ots4FXiv7evy9tqkOqGS/IlmZsA/AF5JWlv7NnABcEvJ9TW1UBws6ThGXmt7d532bH8D+EZeM90JOBN4rqRPkZzF/9VpD/qGFVcpGVassGjHOQHYnlEq8zRzPbAUaabaFOdKen2pda6+DspJGPYXwC9yNshOpIyUg21/q8RgCvBu4NgcooFUl1T7AncPlsvhrr8ASFqJ1JkYoMTVzUpd5/0P5WvV1ug4JwDb1ysVJJfk08Blkq5geFbU3jXbWZuUbHIT8Ie8+Ft6sba7E2m1OPj3hWz2KqpeiVSPVayA1fatwBeBL0pah+Q8zgVKKHQ0HlascKukzzG0xPAOUgJTKQ4Ffi/peprrfn458BNJ85Fq3JRMupbM0xGTJLJjejPJOU0nhZCOtX1XHcabIi+qy3YjNQmStgT+h3TFL2BlUp3LDGBP20fUbO8zpB+zn5CuiN8GnGb7S3Xa6bJ5Cqng8uRs8x0k2aGdCtr8HakuaVi82/YJBWytQfrh3IH0I7cGKXxSS3bSCHbnI+lffgKYRQoJ3VjSZra7CrA/aeb4DeD7TYSKctjtPhfK1mozrJgTFg4CNiP9DlwMHGj7/kL2bgCOZM7vR8l1tltJoe/rSvwNR1qDOoF0JXkucKrt6+s2Xop+yQMdCtYhVMewIOlHTaSr8KJp+Tl54RV582Lbpa66O/YWAt5P+kGD9OX7Xsn3KemyUvH7UexuQLpI246k7Vj7GDRncfChLl8cjKQXkUoiXgJ8DTjZSX2lhK2NgC8D/wQOIc0sliXNFt9p+xcl7GbbnbDijsBqpELkImHFtpB0ke0RsxYL2DwPeFOpddmRHNTTZEkZesTG65rClaBf/UGHUnUIXWNYm1QjtFDFbhFNtRxCnANPMlFKSV8EbidlDlVDGI2kmSulR76yxBWppDsZXhw8DBcoDpbUaUJ6GHAaQ0oEHZu1fq5KzU/3B5Yk1e29yfblebZ6iu2X1GlvhHGsQ3JWOxRa2zvC9r59snmLhdwkfZ30vTiL4d+Pkpm1x5OyaM+lQDHypKyDapvsIDcnOahzSJIxl9jetpC9at3XwqSQ4s221yphL9vsXtQHoGToJNvsYbJem5JOs719fvwV25+qHPul7dfXaS+f93iaLw7+c8Vm576TrVvic51l+8X58U22X1Q59vsSDkrSGrb/kB8vaPvxyrGNbf+2gM31bV/VrwarVMitkmHbZc7F0sz7TQbqmgQM1FF3oiJpReBbJJUDk0In+9i+s7DpbUmqvr+3vbuk5YFjShmzvU51O4f73lfKXqa6qL8QKfy1TEmDtlcuef4Kq1UebwF8qrK9XAmDtnfrdyz//5SwOb3EeUegGgZ6rOtYqSvlH5LKBAB+W3kM8J2u7VqwfVV++GLb36wek7QPhWqvOhm2TVLJJC5SjFxap6ltjiNNdzvq0GfnfaV5LMdkn8wJGndTuJi0Sp7Sb1jYxn2V21058aNkQSCS5pe0t6Qz8u1Dee2mbkb6sWwk5CBpSUl7SDofKBai6WF3VUmfyZlgdbOepIckPQysmx93ttcZ7cXziPo87rVdN+/qsW+3Usby/8zXJc3Mt8MrGcylbK4t6fekFPcbJF0lqbbIzaSeQQHL2a46pOMl7duA3ZmSliIJmV4F/IuU6lqErqSQ+UhXhfeUspdtVq885yPNqIq0TKjwPZJ68nfz9q55X90FyYtIegnpfS2cHyvfFq7Z1mw0QnFwKZvZ7nNI2Yo7kwquDyWt0dSKC/ZeGslsn8e9tmtBSeF/Z2BlDVfkWRy4r4TNzLEkR7F93t6VdEH+XwVtHgV81MOLkY8m6YKOmUm9BpWvPo9nSNFhJ2B3269tcAzTSeoR11b2reUaC5674sBPAn8GflQ4o64a736SVN9xuAt2EJZ0je31RttXg50ZjKzlWHsoRcOLg09lqDi4WFhTSRG+I9Z6Wr79tMFQanE0pDcokhPuaA8W0RvMNp9PWgc+FNivcuhh4NqCWZKz1/hG2lezzaLfyck+g9qDpATwDdIPzmU0U6g7G/dWFD+JemPfN9oephyh1F2ziJpE5t254LJqs/QP21OSVu2kX+fU4adGec1cY3vzus85AG0UB3+HtC6zs+2ZAA3YbJqq3mC3vmAJvUFs307KNt14tOfWzGMarpyzKXOu9dVN0WLkST2DGq/UnbGkHq0Reu2rkz42i7SFqJz/taSQxa2kK+Dnk2bEvbKXxmLnHaTvxkld+/cEHrH9wzrtVc7faHGwhmvTLU+aQe1m+3kl7LVBrtdb3PY9XfufRWoRUTLKsBEpSetFwAIkdY5HSpXoKCm5nEBK4xep3uxd1ehNAZtFi5EntYNSap61J3NKwTc6i+qmLuch6U2kTpbbkyrkOywBrGn7ZWO10cPmGsBawFcZfnW6BPCJkqnt2f6CJHHcTgF07U0S86LvK93VtiQnvFxY0glXbBUvDu6ytyKpiHUnYBFSEev+JW1WbE8jNSz8QYFzHwX8oruOTNIuwGa231+3zYqNmaTPtFNv9k7gBbY/U8pmtrsEgFMr9kbINp+OLL6546ekq4nzSWK3ndtk4a+kMMW/SckYndtZwBsK2Vyd1M5kKZJqeuf2UtLFQDEkfRBY2Pa1tq8hJTN8oICpad3OCWZ/4UtkDc6B7Zm2P8aQxFhpe3faPiw7322osStqB0lLSPq0pG9Ler0SHybNiLcf7fXzyGbdzgkgO8NX9nh+rdi+hfT/9FRO2CqWCt7J4iOtX17QUBbfOvmC7jqGsvjWru38k3wGVXSBcF6RdLntjWo83zNKLbyOYLNIkeMoNnstAtde4CnpJmAD24907V8cuNL2GnXaG2A8f7HdUy1kItmU9FPSOttvgdcCS5NCX/vYnlWnrYrNYQXBgx6ryfbFpPbrx5Da7fyNFEKtNamnYu9HpCy+jjblrsB6totl8alwS5HJniTxM0lb2j6nacOSViCtkVRDixfn+1qck4YUD37fa3HbBfq0SPqk7a8CO+d02m6bdSuLV5lPkpyvqnJoaIECdr4PnCHp/Z0kl5yN+Z18rGlK1+s0ZXOVTlG5pGOAe4GVes1Wa+RuSS+zPazMQ9KGFC7FIDmIacCHSBqLzyM1vizFqrar5z9I0qyC9qBwS5FJ6aCUCv9M+pLtL+k/pBYUjegISvoKaaH7RoayzEz99Sz75Pu3jPiserkp3xfJgBqF84DTJP0P6fPci9QSplZsHybpX8BFkhbLu/8FfNn29+q2N8iQJonNJ2afPGUp3lbYOUFaJz1NSUqqo/DQWQ/asaThnM0HKZOuuP4nkcUXDIJSZ8t1Syzg97E3TC+u376JjlIriveRwkMi1Qwd49S7rJTNxUjfk6I/pOojLEp6n6+xXXujO/VX/RcpbFOrdJWkpxgSoO4UPT9K4QtHJamoD5BS+QFuAL5tu0hjPw3XxpyDEpGNbHc94ETSujukcGqTWXyQLsIPiiy+AZAkYBdgZduHSHoe8Jzu6X4Bu+cC29Wd0TKCvV4p30VbMff5QX2QNLM6smT6bhPkhd5PkDIWTZoNH+ZKk8aa7Y3YJsFlFNRbV/1vipxu/gLS3/JPhdPLnz/S8crMqk6b00gz/E80lcWXbZ5n+3XFbExyB/U9kkDla2y/KHv7X9ouqlOXFyvXA35Nwc6vkt5PujJchdQcscPiwKW231GnvS7b3yQJp3ZUOnYgLQQvTFLO2LWAzU2BAxla2+tcedetur01qQXFoSSHK2B9Ukffj9v+aZ32pjJ5vWIbUrHwmwuc/xnAl4DdSW1M5iMpZxxHmiU+McLLJxSSLnBB5fI+Ns8CdnWhZrCT3UFdbful1UwvFZDG6WG3l0hk7Z1fcwrp0vSQVHHhHkmSLrb9yl77JN1Qoh5K0h9Ii81XUVGQsF2rvpmka4Ct3aUCkhMlflri/0dJOmqkdhu1y3MpFR7PsP3HHG34PmkR/3ZSaKhI00tJC5Dq93YG3gj8CPix7bML2PoG6YLtI50wbZ5hHEYSdd5npNeP0XZnLRxSMs/8lC3UPZykxH86Q6HUIr3EKjZPAzYCftVls5aL8UmZJFHhiTwN7WR9Lcdwyf8i1O2IRrDzICmsthPMro5fCFhM0mIu27BwOUkrdWwoNU1cNh8r1Sr8QdvnFjp3lfm7nRMk2SqVUU8H+HiPfRsBnySpSpRgH5JWJaT/ofVIs/GXAP/NUIfmWpC0RbbzBuBC0sL6y2zvXqedLt4CvLCT+Qkp9JWjD39gKNGodmwPE0+WtA1Qe/F8hWVIYrTVWZSBYg6KwrWlk91B/TfwE+BZSt1YtwU+W9qopNVIs5rujrpFWm5I2gr4OqmtyN2kENhNpPWTUnwMuETSn0ghsJWBD+SQTSkHfaGkr5G+cCU7hj5Rdb4d8tpCkXozD/UQ6qxHfQ5YENiroFN+shLiegtwYp6Nni/pqwXsnQf8hlQ8exvMDhWXxFXnVNnZhNZht80zJe03+jPn+fwlHX0/myN+1yX9qCv1fa6Y1A7K9g8kXcVQ1tc2tm8a5WV1cBxwAEmk9tWk+HfJWpYvkK62z7f9EkmvpkC7hCq2z8mOeA2GZIc6C89HFDL78nxfbZZo6u9DdQDpR/pLpHCiSf219mN488JakfQGkmP6N/BF16wx2IOnlVpt3E/6jnyxcqxEW5H1Sand50u6laQsXroFx42S3mn7xOpOJb3FP5Q0LKlaINtpSVO7U5T0rZHOW7g2cTTGdFE+2degViXpmD2eK5zXJV0lPlDY7lW215d0XaUw8Te2aw2ZVOzNtL1BXjt5ie2nJf3OBbT4uuxuwpw6hyf2fcEEIqfsfow0CxWpQv9wJ4mlEvauJCWdfI2ktDCMArNEJL0FOJLkJM62vWfe/yrgkyWSFiq2NyVdRL0dmEXS/juqgJ0VSDPuxxh+sbEw8Dbbd9Vts2K72ouu0wbn6LrT2ytr3puSojYdXc7tgKtsf6ROe3NDrwzjuXr9JHdQs0hXLdNJBZ1nA6vb3rKw3UtJ8fszSLpYd5FSQFcvZO98UibUoaR1oLuBDV1QYFTSScCqpB+X2cXIJa7WJL3D9sn96nZsf71um02jkXtQuVR2Vs5yW7xat5LDtGqiTCLXtm0B7OCCIs6SXsPQxcYNtn9dylZb5ESb13fCtnm99JduoRV8ZUxjclCTOsRHUtd9Mk+1v2n7W0rChqXZl6QIvTdwCCkE1TOzrya2JoWFPkKq+1oSOLigPUiOf81e8f0CdIpUS3fsbQ230IOqGoJKSXxzUOvieudCIz/e1Paltp8Gzsvh4mLYvoB0sdgYSv3KvkkKv5s0M/6Iu/qo1chzSd+RTgbvYnlfm4xpaWOyz6CuIK2HfAbYyvZtkq63XZva7lRF0unA3rb/1vZYOkj6tO1D2x7HvNC1XjEHJVKFu0JQPUzWO6OpXk13X1mP9Up7PCLpcpJ+Y6dWcEfgw7Zf3v9VY7K3O6lOsLN2+SpSb6ZGsor7jOn1tn85z6+f5A5qTZJe229tn6LU8XUH218uZO8I2/uqj2yN7bfWbK9aZzHsEIU1B3M44cXA7xieUVfre5zLMU3YH7mmnUUbdNUjzn7ca3syIOmKbmekmjsZ9LD5bIaSia5wuWaX/eScOr89tajYTGoH1TSS1rd9lfrI1riAXE1bjMf3WNeP3DjPiiqCpNfYvqBzX8jGuJlBqWCTxIqNLwMPkLIVTVJbWZA0q8I1F9PnYutdSKrxB+faxGe7gLSbGpJzmtQOStJt9J7JFKlHmmooiXB2ZKN+V3d20jyMp65OxdX1woNIaeezKR0yacJZ9LDZUV0p5igkPQrcQrrKXjU/Jm+v4jKCuEsAHwRWIDV+/BWp/cXHgVm2t67bZsX2SKrervt3SC1Ju5VksidJVOtlFiKlXdaq0FxlhGkvUE7FuA0kbU9KiZ5B+oH5lqRP2D6jzWHVcZKqA5K0bwsx/MNIHYo7901Ssl6vWHPAETiJoSaJ7yEJAC9AkrKaVdKw7ZVLnr8HL88XGb/P9u9XkpUqhqSNgG+R/rYLkEoWapNzmtQOynNqtB0h6RLg84VMdvoyfTDfd3qk7EJqKzCZ+Awplf1umC0jdT4ptb4tTi9wzjZDDG00KixGXWGfuaSNJolke/MD72eotfwMktJ/KYHaNqTdvk1K/jidoT5bL6jr5JPaQUmqXn12KrmLpSp3voA5hXbTyqH9cm1U6dTvJpmvK6R3H+kzLkb+wu3JnMXBe+T7L5W0H4yNHiF3VbZte9UCZttoktjheySB2O/m7V3zvvcUsteRdlteDUq72b5F0jSnvmzHKbWBr4VJ7aCAwyuPO5Xc2zdgd1EN72y5CUO1PJOFX0g6j+HtNs4pbPOnJC2386momddNV3bkIpI6fXUa6cg8idmga3s+0vfx40Cp+sT1uv5+C+ftJv6WG3q48v0FWe2lCB4u7QbNSLs9msOIs5T0G/9Gjb91k9pBtVhB/W7gWKV2GCYpjk/4NOEqTo3R3k6SVxFwlO2fFDa7iBvoEuwuFeopQkc1otjsohNyz+oRu5LWg2YBb7Z9YyGbpbX+RuIpSava/hPMLtwtdmGVWYS0DmTK6Cl2syvpQuNDJKGA5wEj1vTNDZM9i29JUgZWJwZ8EXCwCzXX6mF/CdJn3Ii9yY6kLwCX2S49UxtpDIu5sASQhvpqzdFzayKT12T2IP2QXQIc2vnxbngcRZskVuy8liQc3VGOmA7s7kIiwJI+T0oE+xHponEb4HTbXyhhL9vcx/Y3R9s3z+ef5A7qRySRz04W1q7AerZr8/B97C5P6uL5XNtvygXDG9v+fkm7TdBycfDDpPDB46S1hcZDbpL+Ynulpuw1gaTDgGNLzWIqdu4khdqPIHW3HYbLNtZrrElixeZCJMHhTsjtV8A3XKjdvKSbSGLR/87bCwNX2y6WPdmrLKHOoutJHeIDVvXwXiQHKQnIluZ4ckvpvP1/JIXhCe+g2gx/NWVbfURpSQ5xscK2G3EWXfwBOFpJOPY44JRCs/7zSRc36+VblSKN9dROk8QOJwIPkfQ4yeM4iTTLKcGfSeU0HQe4IFBkhippJ5KzX1mp7XuHJUgJU7Uw2R3UY13JCpuSZPdLs6zt0yR9GsBJsLZ07HlKkIsPV2N4I8iLazbzJVKNV6/mhEUzFWnOWczG9jHAMZJWJ/UuuzZnnR5dZzjK9m51nWsuaKNJYofVu5IkLiyRJFFRPnkcuEHSr/L2FqRQagkuIyVELMvwZLSHgWvrMjLZHdT7gRPyWhSkgr2SquIdHpH0TIbqETYiJUoEY0DSe0gtulckLa5vRCrArLsVxdXAma50ue0aQzGachbd5PqZNfLtXuAa4KOS3md7xwL2mlLLaKNJYoffS9rI9uUAkl4OXFrAzsx8fxUpzbzDjAK2gNklNbcDG3cpytxku7au05N9DWpBUi3AqsBSJCdh20XrkXL91beAtUlrYMsB29qu7cpiKpKVOjYELrf9YklrAAfZ3qFmO6sD99m+t8ex5W3/o057PWxMIxV9707KijoN2IxUoV/CWXwd2IrUjuL7rmi3SbrZBfqYNSGt1MNmI00SK/ZuAlZnaL1tJeAmUvGsm1aW0Rjbr/c553YkxZMZpBD4K4DaFGUmu4P6BUms8Woq6Z22D+/3mhptP4P0zyng5oLV41MGSVfa3jCvI77cqVPyLNsvbnlotdGSs9gDONX2HGonkpYsEWKsOKjaFtTnwnZTTRIbEVQdlBKfdQ5ZbuEuRZmu0OY8M9lDfCvafmPTRvMV8JYMKR68XtKk6PzaMndKWgo4E/iVpPuBv5YwlNe6Pmb7s5V9OwG3266tUr4H1wOf7eUsgJcVsrmL7WOrOyT92vZrJ3qJhNptktiGtNNIlJiNFFWUKb3g2zaXSVqnBbtnA7sBzyRJK3VuwRiw/TbbD9g+EPgcKStym0K27ge2kFTVFfs8KSOzJLt0OydJv85jqtVZSFpI0jLAspKWlrRMvk2n/U6sdVHNyPxW17FJVTzfEudKOk/SbpJ2A35OjYoyk3IGpSFV8WcAu+fF0ceh3mZaI7Bi0/HlyYykJWw/lH9MO1yX7xdjqMV13Xyf9CO2v6TNgRt6rUvVQa6ZWYTsLBgSil2Ccs7ifcC++fxXV/Y/RO5ZNAlQn8e9tic7Jd6vgSNJa6QCjiIlL9XCpHRQDKmKt8W5GmOr42AYPyT9Ta8ifSGqXzQDpfp7nQLMlPQZ0oz4mEJ2oAVnkav9vynpw7a7ZxelKS6tlHGfx722JzslZMK2cJIfm13DJumgumxN6iSJtpD0NuBkUgi1FcWDoB6UWrGfR5LMWtOFvzBNOotKqndPZZWSyg5NoRaaJDaNGmq/3mXz/cAHSBeH1WLgxYFLbb+jDjuTdQbVNocDGwPXlf5BmwpoeNuUObB99UjHx8gxpDXFb5f8W1bqge7q5TAKOYtXkbIFt+pxrIiyAzSultFGk8SmaSNi9EPgXOBQYL/K/oddYyv7mEEVQKkNxZtytlAwRiR1ClQXIrVsuIZ0dbgucIXtzQrb/yZwuO059ONqtHGQ7QPyjK0bl0yHbppc7Lw76QK5EbWMYGISDqoAko4nTX3PJSVnAESa+RiRdCrwRdvX5e21gY+3JKEzacip++9kzkaQexe221HL2ImksFC7WobaaZLYCircfr0NIsRXhtvybYF8C+phjY5zArB9vaQXlzTYoCRPx95SNO8szgEuJ2VGNjLrb1BaqY0miW1RtP16G8QMqgUkfcv2h9sex0RD0inAI6QEFAPvABazvVNBm41K8ii1y57DWdg+oe+Lxm6zMbmhbK8NtYzuJolfamgNrDEkzbS9gaRrO4kRki6zvUnbY5tXYgbVDpu2PYAJyu4kAeB98vbFwPcast1UzcxCtvu1+yjFSZL2BH7G8JB0qfqyxtQyNGeTxK3dQpPEhijafr0NYgbVAk1fsU4mlJqwrWT75obsNaoZJ+kjpBqhppwFkj4IfJGkW1ldnylSX9aRURptX022WmuS2DRZ++8fpGWFjwBLAt+ZyA45ZlDBhEHSW0l9mhYgNUp7MXCw7be2OrB6+Q/pPX6GirOgXDEyJDmgF5RSyejQklpG400SW2SbXHz9b+AgAEn7AE31v6qdcFDtMNUkVuriAFIIaAaA7VlZN24y0Yiz6OIGoFe4rW7aUMvYrcR5xynvYk5ntFuPfROGcFDtMGH/YVrmSdsPSo3696YkeTo05SyqPEVat7iQ4WHFWjMHW5ZWajwjsynUUPv1NggHVQBJZzOn9MiDpM6XR9o+vvFBTQ6ul7QzMC23Stib1Hq6GLZfWb1vgEacRRdn5ltRWlLLqHIY8NLK/WShkfbrbRAOqgy3krronpK3dyAtXr4QOJqU7hrMPR8mrc08TpJaOQ84pJSxhiV5OpxJA86iSskU9i5akVbqwaQKsbuh9uttEFl8BZB0cfcVd2efpBtsr9XW2CYykjYgOajpDF1cFWufMtkleSSdZnv7fmKjk61lTNMZmU2jwu3X2yBmUGVYTtJKHe02SSuRpt+QsrSCeeMHJAWA62lA8cD2McAxFUmeayWVkuRpw1l06skaFRttS1ppCvBZYEN3tV8HwkEFw/gYcImkP5GuZFYGPiBpUaCpcMpk5B7bZzdpsEFJnsadhe2/5YeLdocxc4PGUi3LG5dWmiIUbb/eBhHiK4SkBUk/agL+YPvfLQ9pwiPptSRh0V8zPIGgVFuINiR51uzlLGzPqNtW5fzXAycBXyUpxn8V2MD2xoXstVKoXgmzzxGCnwxk9Yj1GL72fa1TQ8EJSTioQkjahDlDGCe2NqBJgKSTSU7/BoauvIu1opC0B3BqL0keSUuWWI9q2llkm4sCXwHWJzWc+wHwlVLtYtpQy5gKSPoKcAVD7dcvBjaayA4qQnwFkHQSqXvnLFLaMKR1hXBQY2M92+s0aG8X28dWd3QkeQomS7yc5CwuY8hZlNZufAJ4DFiY5BRvK+WcMo2rZbSUkdk0Rduvt0E4qDJsQAPtwacgl/cKgdVNS5I8HZp2FgBXAj8lpSc/EzhS0ra2ty1krw21jD8AR0uadBmZ1fbrkqp1T4uT+mxNWCLEVwBJpwN7VxahgxqQdBNpZnobKTQkCqSZZ/2yfUnO6K+VQw+RMvi+Xae9LtvXkJzFIWRnATxR0FkgaQPbM7v27Wr7pEL2zgJ27KNmXpQmmiQ2jaQlgaUp3H69DcJBFSCrALwY+B3DY+yTSdS0cbJa8xzkQsUS9hqX5GnaWeTzz09qY9JJHJhBUjx5opC9nwBrAU2qZXQyMt9CclDPA04jrdc8UnNGZlAT4aAKIOlVvfbbvqjpsQRzT0WzbQ45HigrydO0s8g2jwHmZ6gEYleS7uGehey9q9f+kooWbWRkBmMnHFQQdCHpINsHSDqux+FiWYPZdqPOItu8xvZ6o+2byLSRkRmMnXBQNSLpEtubSXqY4WoAnbWSJVoaWjBBaMNZSLoa2M65sZ2kVYAz6q5ValNaqckmiUF9RBZfjdjeLN8v3vZYgrHTkiTPU5JW7XIWT43ymrHyceBCSbeSLqaeT1qnqZvG1TJazsgMxkg4qAJIWhW40/bjWTJmXeBE2w+0Oa5grmlDkqcpZwHMThxYD1gNWJ0h5ZPHR3zhPNCStFLjTRKD+ogQXwEkzSLVQk0ntYQ4C1jd9pYtDiuYS5qW5MnOYm/guxR2Fl12L7T96pI2uuy1oZbRSpPEYGyEgypARdb/E8C/bX9rskr8T2bakORp2llkm18ElgT+F3iks9/21X1fNDZ7jUkrtZmRGYydCPGV4QmlNszvYqg52/wtjieYNxqX5AEuk/RtGnIWmU3y/cGVfQZeU8hek2oZ46VJYjAPxAyqAJLWBPYCfmv7FEkrAzvY/nLLQwvmgtwu5eVNSvLkIu9ubLuUs2icNtQygolJOKgg6EObkjxNIumZwAEkVQUDlwAH276vkL021DKWIpokTjgixFcASbfRu86jZGgoqJ+ngFl5VtOIJE/TziJzKqk1w9vz9i6kEOPrCtm7RtLedKllFLLVIZokTkBiBlWA/CPTYSFgO2AZ259vaUjBPNCSJM+vSM7i5LxrF2Bz26WcBZKusr1+176ZtjcoZK8NtYxWmiQGYyMcVEN0VCbaHkcwvmnaWeTzHwbMJImnAmwLrGX7gEL22lDLiCaJE5AI8RVAUvVKbT5STVSoS0wQ2pTkIRXp7shwZ/HzgvYgFbN+lFSbBDANeETSRykj0dWGWkYbGZnBGIkZVAG6MrGeBP4MHGb75nZGFMwNkp5j+29Nt/fIth8GFmXoB3saQ+nmreg5SlrL9g01nu81wPHAMLWMkn2Z2sjIDMZOzKAK0HShZVAvLUnydGyPONOu21kMyElALes3TUordXEDMKmzMScjMYMqQO5weQBDWUoXkTKxQtJ/AtGGJM8AY2p8sb9uFZSW1DJaaZIYjI2YQZXhWOB6YPu8vStwHNBTbiUYt7ycJMlzGUOSPJu2OqIhNe4mqfsqtg21jDPzLZhAhIMqw6q2317ZPigLyAYTiyYleQZlMoQ8mpZWKloaEJQjHFQZHpO0me1LACRtSvqhCyYWV5IkeTYkS/JI2nYKSvL8p86TNayc3mZGZjBGYg2qAJJeTCpCXDLvuh94l+1rWxtUMNe0IckzwJgut71RgfOuQMqmq8oAXVy3nWyrMbWMNjMyg7ETDqoAkhYk1a+sCiwFPEhKET54pNcF4wtJ8wPvp0uSx/YThe025iyyva8AOwA3MpTebttvLWSvDbWMNXtlZNqeUcpmMHbCQRVA0i+AB0gdPGcXINo+vK0xBXNPS5I8jTqLbPNmYN0GUr079tpQyxh3GZnB6MQaVBlWtP3GtgcRjJkNu+R3LsitIkqyDan7ciPOInMryRE3ZbMNtYzxmJEZjEI4qDJcJmkd29e1PZBgTLQhydO0s4BUwDpL0q9ppkaoaWklGJ8ZmcEohIOqkUqm0DOA3SXdSvrCi/TFi4yhicXHSVf7wyR5Ctts2lkAnJVvjdCSWkZkZE5AYg2qRvplCnWIjKGJQ5bk2Rv4Lg1K8rTR4mO8UUItYzxmZAajEw4qCPrQhiRPG0haDTgUWJMU/gLaa7BZt7RSPmcrGZnB2IgQXxD0p3FJnpacxXGkuqRvAK8mhTHbkFTqUOKq+Xuktb3v5u1d8+NiGZnB2IkZVBD0oattSgfbLibJI+kShpzFVmRnUap5YLZ5le31JV1ne5287ze2X1HK5ijjKRHia7xJYjB2YgYVBH1oKby3sO1fS1JeszxQ0m9ITqsU/5Y0H/BHSR8C7gKeVdDeaNQqrZRpIyMzGCPhoIKgD01K8lRow1nsCyxCSgo5hBTm65msURcjqWWUkHKinYzMYIxEiC8I+tCSJM+GwE0kiaxDgCWAr9m+vJTNiu1FbT8y+jPHbKdpaaVWMjKDsRMOKgj60IYkT8VOI84i29oY+D6wmO2VJK0HvM/2BwrZa1RaKducEhmZk4352h5AEIxjLpS0o6T58m17CkvySNpY0o2kWRSS1pP03VFeNlaOAN4A3Adg+xqG0rFL0FHLaJLLJH1b0iskvbRza3gMwVwSM6gg6IOkh4FFGQpDTWMo3byIJI+kK0jadGd1aoEkXW977bptVW3afnm1/qhkhpukHwHrAY2pZbSRkRmMnUiSCII+tCTJg+07pGFlSKWzze6QtAlgSQuQ1mtuKmivUWklaC0jMxgj4aCCYN45Cag7TNS0swDYC/gmsAJwJ/BL4IOljLUh29RSRmYwRiLEFwTzSCFJnmVJzuJ1pGyzXwL7TKYf0jbUMtrIyAzGTiRJBMG8U/vVne17be9ie3nbz7L9jtLOSdIJkpaqbC8t6diCJo8jSQ89Saq5OpGh1hulWMb2IbZvy7cvkFL5g3FMOKggGEe04CwgpXw/0NmwfT9Q68ywi4Vt/5oUwbnd9oFA6WSFxjMyg7ETa1BBMO+UkOSZw1lIKuksAOaTtHR2TEhahrK/DW2oZbTRJDEYI+GggmAEWpDkadpZABwO/FbS6Xl7O+CLBe3tS8PSSm1lZAZjI5IkgqAPTUvyZJvvBPYHhjmL0o31cubgBsDTwFW2f1vSXrbZmFrGaJRQUA/GTjioIOhDG5I82W6jzkLSPsB7gB+TMge3AY62/a1C9hqVVhpwTLVnZAZjJ5IkgqA/jUvyZGdxJPBM0rrMkZI+XNjsu4GNbB9g+/PAxpRt5HcEzUorDUJcqY9DYg0qCPrzKDBLUmOSPAw5i0dgdpjxt0CR2UxGDFereIrCHXVbUMsIJiDhoIKgP41L8tCCsyDVJV0h6Sd5extSCK4UbahljEaJjMxgjMQaVBCMI3La87uAqrM43vYRhe2+lCQDJOBi278vaKsVtYyRMjKD8Uk4qCDoQxuSPNluY85iqtBGRmYwdsJBBUEfJF1CEhj9BrAVqUW4bB/Q6sAmOJJOIM2YHsjbSwOH296joM1WMjKDsRFZfEHQnzYkeaYCTUsrQTtNEoMxEkkSQdCfNiR5pgJtqGW0kZEZjJFwUEHQn31pWJJninA4qQX7GXm7tLQStJORGYyRWIMKglEYT5I8kwVJazIULr3A9o1tjicYn8QaVBD0QdLGkm4k1+hIWk/Sd1se1mRhfobqu4qvDUlaTdIZkm6UdGvnVtpuMDbCQQVBf45g/EnyTHiynNMPgGVJa3onNyDn1EaTxGCMRIgvCPog6QrbL68KiUq6xvZ6bY9tIiPpWmDjipzTosBvba9b0OZVtteXdJ3tdfK+39h+RSmbwdiJJIkg6M94lOSZDLQh5xQZmROQcFBB0J+9SJI8KwB3kiR5PtjqiCYHTWv/QWRkTkgixBcEQeO0JecUGZkTi0iSCII+SDpB0lKV7aUlHdvikCYNtq+2/d+2v9mEc4qMzIlJOKgg6E8bkjxBGY4gMjInHOGggqA/82UhU6AxSZ6gELbv6NoVTRLHOfFlC4L+tCHJE5QhMjInIJEkEQQjEJI8k4O2miQGYyNmUEEwMh1JHhPtGiYstu8Fdml7HMHcEWtQQdCHliR5ggJERubEJEJ8QdCHNiR5gjJU5apG2heML2IGFQT9aUOSJyhDZGROQOIPFAT9aUOSJyhDZGROQCLEFwQj0JYkT1A/kZE58YgZVBCMgO2rgavbHkdQC5GROcGINaggCCY9kZE5MYkQXxAEk57IyJyYxAwqCIKpQGRkTkBiDSoIgqlAZGROQCLEFwTBlCAyMice4aCCIAiCcUmsQQVBEATjknBQQRAEwbgkHFQQBEEwLgkHFQRBEIxL/h/JTArbBj1QuwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "forest_importances = pd.Series(importances, index=dv.feature_names_)\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "forest_importances.plot.bar(yerr=std, ax=ax)\n",
    "ax.set_title(\"Feature importances using MDI\")\n",
    "ax.set_ylabel(\"Mean decrease in impurity\")\n",
    "fig.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kSCqlx9jye1s"
   },
   "source": [
    "What's the most important feature? \n",
    "\n",
    "* `total_rooms`\n",
    "* **`median_income`**\n",
    "* `total_bedrooms`\n",
    "* `longitude`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "K5KEdiTMzPaI"
   },
   "source": [
    "## Question 6\n",
    "\n",
    "Now let's train an XGBoost model! For this question, we'll tune the `eta` parameter:\n",
    "\n",
    "* Install XGBoost\n",
    "* Create DMatrix for train and validation\n",
    "* Create a watchlist\n",
    "* Train a model with these parameters for 100 rounds:\n",
    "\n",
    "```\n",
    "xgb_params = {\n",
    "    'eta': 0.3, \n",
    "    'max_depth': 6,\n",
    "    'min_child_weight': 1,\n",
    "    \n",
    "    'objective': 'reg:squarederror',\n",
    "    'nthread': 8,\n",
    "    \n",
    "    'seed': 1,\n",
    "    'verbosity': 1,\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "id": "TUP0uoL5yZTc"
   },
   "outputs": [],
   "source": [
    "#@ CREATING THE DMARTIX:\n",
    "features = dv.feature_names_\n",
    "\n",
    "regex = re.compile(r\"<\", re.IGNORECASE)\n",
    "features = [regex.sub(\"_\", col) if any(x in str(col) for x in set(('[', ']', '<'))) else col for col in features]\n",
    "\n",
    "dtrain = xgb.DMatrix(X_train, label=y_train, feature_names=features)\n",
    "dval = xgb.DMatrix(X_val, label=y_val, feature_names=features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "id": "lRRzLznP-Z-Q"
   },
   "outputs": [],
   "source": [
    "xgb_params = {\n",
    "    'eta': 0.3, \n",
    "    'max_depth': 6,\n",
    "    'min_child_weight': 1,\n",
    "\n",
    "    'objective': 'reg:squarederror',\n",
    "    'nthread': 8,\n",
    "\n",
    "    'seed': 1,\n",
    "    'verbosity': 1,\n",
    "}\n",
    "model = xgb.train(xgb_params, dtrain, num_boost_round=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3RmcQ1BQ64X-"
   },
   "source": [
    "- Now, change eta first to 0.1 and then to 0.01."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "eta=0.3 rmse: 0.226\n"
     ]
    }
   ],
   "source": [
    "y_pred = model.predict(dval)\n",
    "\n",
    "rmse = np.sqrt(mean_squared_error(y_val, y_pred))\n",
    "print('eta=0.3 rmse:', rmse.round(3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_params = {\n",
    "    'eta': 0.1, \n",
    "    'max_depth': 6,\n",
    "    'min_child_weight': 1,\n",
    "\n",
    "    'objective': 'reg:squarederror',\n",
    "    'nthread': 8,\n",
    "\n",
    "    'seed': 1,\n",
    "    'verbosity': 1,\n",
    "}\n",
    "model = xgb.train(xgb_params, dtrain, num_boost_round=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "eta=0.1 rmse: 0.231\n"
     ]
    }
   ],
   "source": [
    "y_pred = model.predict(dval)\n",
    "\n",
    "rmse = np.sqrt(mean_squared_error(y_val, y_pred))\n",
    "print('eta=0.1 rmse:', rmse.round(3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_params = {\n",
    "    'eta': 0.01, \n",
    "    'max_depth': 6,\n",
    "    'min_child_weight': 1,\n",
    "\n",
    "    'objective': 'reg:squarederror',\n",
    "    'nthread': 8,\n",
    "\n",
    "    'seed': 1,\n",
    "    'verbosity': 1,\n",
    "}\n",
    "model = xgb.train(xgb_params, dtrain, num_boost_round=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "eta=0.01 rmse: 4.265\n"
     ]
    }
   ],
   "source": [
    "y_pred = model.predict(dval)\n",
    "\n",
    "rmse = np.sqrt(mean_squared_error(y_val, y_pred))\n",
    "print('eta=0.01 rmse:', rmse.round(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Yl6Xn6Zb76iH"
   },
   "source": [
    "Now change `eta` from `0.3` to `0.1`.\n",
    "\n",
    "Which eta leads to the best RMSE score on the validation dataset?\n",
    "\n",
    "* **0.3**\n",
    "* 0.1\n",
    "* Both gives same"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submit the results\n",
    "\n",
    "* Submit your results here: https://forms.gle/3yMSuQ4BeNuZFHTU8\n",
    "* You can submit your solution multiple times. In this case, only the last submission will be used \n",
    "* If your answer doesn't match options exactly, select the closest one\n",
    "\n",
    "\n",
    "## Deadline\n",
    "\n",
    "The deadline for submitting is 17 October (Monday), 23:00 CEST.\n",
    "\n",
    "After that, the form will be closed."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}