lstm_autoencoder_classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LSTM Autoencoder for Rare Event Binary Classification\n",
    "\n",
    "This is a continuation of the regular autoencoder for rare event classification presented in\n",
    "https://towardsdatascience.com/extreme-rare-event-classification-using-autoencoders-in-keras-a565b386f098\n",
    "and code present in\n",
    "https://github.com/cran2367/autoencoder_classifier/blob/master/autoencoder_classifier.ipynb\n",
    "Here we will show an implementation of building a binary classifier using LSTM Autoencoders. \n",
    "Similar to the previous post, the purpose is to show the implementation steps. The Autoencoder tuning for performance improvement can be done.\n",
    "\n",
    "LSTM requires closer attention to preparing the data. Here we have all the steps, and few tests to validate the data preparation.\n",
    "\n",
    "The dataset used here is taken from here,\n",
    "\n",
    "**Dataset: Rare Event Classification in Multivariate Time Series** https://arxiv.org/abs/1809.10717 (please cite this article, if using the dataset)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from pylab import rcParams\n",
    "\n",
    "import tensorflow as tf\n",
    "from keras import optimizers, Sequential\n",
    "from keras.models import Model\n",
    "from keras.utils import plot_model\n",
    "from keras.layers import Dense, LSTM, RepeatVector, TimeDistributed\n",
    "from keras.callbacks import ModelCheckpoint, TensorBoard\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix, precision_recall_curve\n",
    "from sklearn.metrics import recall_score, classification_report, auc, roc_curve\n",
    "from sklearn.metrics import precision_recall_fscore_support, f1_score\n",
    "\n",
    "from numpy.random import seed\n",
    "seed(1)\n",
    "from tensorflow import set_random_seed\n",
    "set_random_seed(123)\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "SEED = 123 #used to help randomly select the data points\n",
    "DATA_SPLIT_PCT = 0.2\n",
    "\n",
    "rcParams['figure.figsize'] = 8, 6\n",
    "LABELS = [\"Normal\",\"Break\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading and preparing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data is taken from https://arxiv.org/abs/1809.10717. Please use this source for any citation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>y</th>\n",
       "      <th>x1</th>\n",
       "      <th>x2</th>\n",
       "      <th>x3</th>\n",
       "      <th>x4</th>\n",
       "      <th>x5</th>\n",
       "      <th>x6</th>\n",
       "      <th>x7</th>\n",
       "      <th>x8</th>\n",
       "      <th>...</th>\n",
       "      <th>x52</th>\n",
       "      <th>x53</th>\n",
       "      <th>x54</th>\n",
       "      <th>x55</th>\n",
       "      <th>x56</th>\n",
       "      <th>x57</th>\n",
       "      <th>x58</th>\n",
       "      <th>x59</th>\n",
       "      <th>x60</th>\n",
       "      <th>x61</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5/1/99 0:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0.376665</td>\n",
       "      <td>-4.596435</td>\n",
       "      <td>-4.095756</td>\n",
       "      <td>13.497687</td>\n",
       "      <td>-0.118830</td>\n",
       "      <td>-20.669883</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>...</td>\n",
       "      <td>10.091721</td>\n",
       "      <td>0.053279</td>\n",
       "      <td>-4.936434</td>\n",
       "      <td>-24.590146</td>\n",
       "      <td>18.515436</td>\n",
       "      <td>3.473400</td>\n",
       "      <td>0.033444</td>\n",
       "      <td>0.953219</td>\n",
       "      <td>0.006076</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5/1/99 0:02</td>\n",
       "      <td>0</td>\n",
       "      <td>0.475720</td>\n",
       "      <td>-4.542502</td>\n",
       "      <td>-4.018359</td>\n",
       "      <td>16.230659</td>\n",
       "      <td>-0.128733</td>\n",
       "      <td>-18.758079</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>...</td>\n",
       "      <td>10.095871</td>\n",
       "      <td>0.062801</td>\n",
       "      <td>-4.937179</td>\n",
       "      <td>-32.413266</td>\n",
       "      <td>22.760065</td>\n",
       "      <td>2.682933</td>\n",
       "      <td>0.033536</td>\n",
       "      <td>1.090502</td>\n",
       "      <td>0.006083</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5/1/99 0:04</td>\n",
       "      <td>0</td>\n",
       "      <td>0.363848</td>\n",
       "      <td>-4.681394</td>\n",
       "      <td>-4.353147</td>\n",
       "      <td>14.127998</td>\n",
       "      <td>-0.138636</td>\n",
       "      <td>-17.836632</td>\n",
       "      <td>0.010803</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>...</td>\n",
       "      <td>10.100265</td>\n",
       "      <td>0.072322</td>\n",
       "      <td>-4.937924</td>\n",
       "      <td>-34.183774</td>\n",
       "      <td>27.004663</td>\n",
       "      <td>3.537487</td>\n",
       "      <td>0.033629</td>\n",
       "      <td>1.840540</td>\n",
       "      <td>0.006090</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/1/99 0:06</td>\n",
       "      <td>0</td>\n",
       "      <td>0.301590</td>\n",
       "      <td>-4.758934</td>\n",
       "      <td>-4.023612</td>\n",
       "      <td>13.161567</td>\n",
       "      <td>-0.148142</td>\n",
       "      <td>-18.517601</td>\n",
       "      <td>0.002075</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>...</td>\n",
       "      <td>10.104660</td>\n",
       "      <td>0.081600</td>\n",
       "      <td>-4.938669</td>\n",
       "      <td>-35.954281</td>\n",
       "      <td>21.672449</td>\n",
       "      <td>3.986095</td>\n",
       "      <td>0.033721</td>\n",
       "      <td>2.554880</td>\n",
       "      <td>0.006097</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5/1/99 0:08</td>\n",
       "      <td>0</td>\n",
       "      <td>0.265578</td>\n",
       "      <td>-4.749928</td>\n",
       "      <td>-4.333150</td>\n",
       "      <td>15.267340</td>\n",
       "      <td>-0.155314</td>\n",
       "      <td>-17.505913</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>...</td>\n",
       "      <td>10.109054</td>\n",
       "      <td>0.091121</td>\n",
       "      <td>-4.939414</td>\n",
       "      <td>-37.724789</td>\n",
       "      <td>21.907251</td>\n",
       "      <td>3.601573</td>\n",
       "      <td>0.033777</td>\n",
       "      <td>1.410494</td>\n",
       "      <td>0.006105</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 63 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          time  y        x1        x2        x3         x4        x5  \\\n",
       "0  5/1/99 0:00  0  0.376665 -4.596435 -4.095756  13.497687 -0.118830   \n",
       "1  5/1/99 0:02  0  0.475720 -4.542502 -4.018359  16.230659 -0.128733   \n",
       "2  5/1/99 0:04  0  0.363848 -4.681394 -4.353147  14.127998 -0.138636   \n",
       "3  5/1/99 0:06  0  0.301590 -4.758934 -4.023612  13.161567 -0.148142   \n",
       "4  5/1/99 0:08  0  0.265578 -4.749928 -4.333150  15.267340 -0.155314   \n",
       "\n",
       "          x6        x7        x8 ...         x52       x53       x54  \\\n",
       "0 -20.669883  0.000732 -0.061114 ...   10.091721  0.053279 -4.936434   \n",
       "1 -18.758079  0.000732 -0.061114 ...   10.095871  0.062801 -4.937179   \n",
       "2 -17.836632  0.010803 -0.061114 ...   10.100265  0.072322 -4.937924   \n",
       "3 -18.517601  0.002075 -0.061114 ...   10.104660  0.081600 -4.938669   \n",
       "4 -17.505913  0.000732 -0.061114 ...   10.109054  0.091121 -4.939414   \n",
       "\n",
       "         x55        x56       x57       x58       x59       x60  x61  \n",
       "0 -24.590146  18.515436  3.473400  0.033444  0.953219  0.006076    0  \n",
       "1 -32.413266  22.760065  2.682933  0.033536  1.090502  0.006083    0  \n",
       "2 -34.183774  27.004663  3.537487  0.033629  1.840540  0.006090    0  \n",
       "3 -35.954281  21.672449  3.986095  0.033721  2.554880  0.006097    0  \n",
       "4 -37.724789  21.907251  3.601573  0.033777  1.410494  0.006105    0  \n",
       "\n",
       "[5 rows x 63 columns]"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'''\n",
    "Download data here:\n",
    "https://docs.google.com/forms/d/e/1FAIpQLSdyUk3lfDl7I5KYK_pw285LCApc-_RcoC0Tf9cnDnZ_TWzPAw/viewform\n",
    "'''\n",
    "df = pd.read_csv(\"data/processminer-rare-event-mts - data.csv\") \n",
    "df.head(n=5)  # visualize the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shift the data\n",
    "\n",
    "This is a timeseries data in which we have to predict the event (y = 1) ahead in time. In this data, consecutive rows are 2 minutes apart. We will shift the labels in column `y` by 2 rows to do a 4 minute ahead prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sign = lambda x: (1, -1)[x < 0]\n",
    "\n",
    "def curve_shift(df, shift_by):\n",
    "    '''\n",
    "    This function will shift the binary labels in a dataframe.\n",
    "    The curve shift will be with respect to the 1s. \n",
    "    For example, if shift is -2, the following process\n",
    "    will happen: if row n is labeled as 1, then\n",
    "    - Make row (n+shift_by):(n+shift_by-1) = 1.\n",
    "    - Remove row n.\n",
    "    i.e. the labels will be shifted up to 2 rows up.\n",
    "    \n",
    "    Inputs:\n",
    "    df       A pandas dataframe with a binary labeled column. \n",
    "             This labeled column should be named as 'y'.\n",
    "    shift_by An integer denoting the number of rows to shift.\n",
    "    \n",
    "    Output\n",
    "    df       A dataframe with the binary labels shifted by shift.\n",
    "    '''\n",
    "\n",
    "    vector = df['y'].copy()\n",
    "    for s in range(abs(shift_by)):\n",
    "        tmp = vector.shift(sign(shift_by))\n",
    "        tmp = tmp.fillna(0)\n",
    "        vector += tmp\n",
    "    labelcol = 'y'\n",
    "    # Add vector to the df\n",
    "    df.insert(loc=0, column=labelcol+'tmp', value=vector)\n",
    "    # Remove the rows with labelcol == 1.\n",
    "    df = df.drop(df[df[labelcol] == 1].index)\n",
    "    # Drop labelcol and rename the tmp col as labelcol\n",
    "    df = df.drop(labelcol, axis=1)\n",
    "    df = df.rename(columns={labelcol+'tmp': labelcol})\n",
    "    # Make the labelcol binary\n",
    "    df.loc[df[labelcol] > 0, labelcol] = 1\n",
    "\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before shifting\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>y</th>\n",
       "      <th>x1</th>\n",
       "      <th>x2</th>\n",
       "      <th>x3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>256</th>\n",
       "      <td>5/1/99 8:32</td>\n",
       "      <td>0</td>\n",
       "      <td>1.016235</td>\n",
       "      <td>-4.058394</td>\n",
       "      <td>-1.097158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>257</th>\n",
       "      <td>5/1/99 8:34</td>\n",
       "      <td>0</td>\n",
       "      <td>1.005602</td>\n",
       "      <td>-3.876199</td>\n",
       "      <td>-1.074373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>258</th>\n",
       "      <td>5/1/99 8:36</td>\n",
       "      <td>0</td>\n",
       "      <td>0.933933</td>\n",
       "      <td>-3.868467</td>\n",
       "      <td>-1.249954</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>259</th>\n",
       "      <td>5/1/99 8:38</td>\n",
       "      <td>1</td>\n",
       "      <td>0.892311</td>\n",
       "      <td>-13.332664</td>\n",
       "      <td>-10.006578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>260</th>\n",
       "      <td>5/1/99 10:50</td>\n",
       "      <td>0</td>\n",
       "      <td>0.020062</td>\n",
       "      <td>-3.987897</td>\n",
       "      <td>-1.248529</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             time  y        x1         x2         x3\n",
       "256   5/1/99 8:32  0  1.016235  -4.058394  -1.097158\n",
       "257   5/1/99 8:34  0  1.005602  -3.876199  -1.074373\n",
       "258   5/1/99 8:36  0  0.933933  -3.868467  -1.249954\n",
       "259   5/1/99 8:38  1  0.892311 -13.332664 -10.006578\n",
       "260  5/1/99 10:50  0  0.020062  -3.987897  -1.248529"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "After shifting\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "      <th>time</th>\n",
       "      <th>x1</th>\n",
       "      <th>x2</th>\n",
       "      <th>x3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>255</th>\n",
       "      <td>0.0</td>\n",
       "      <td>5/1/99 8:30</td>\n",
       "      <td>0.997107</td>\n",
       "      <td>-3.865720</td>\n",
       "      <td>-1.133779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>256</th>\n",
       "      <td>0.0</td>\n",
       "      <td>5/1/99 8:32</td>\n",
       "      <td>1.016235</td>\n",
       "      <td>-4.058394</td>\n",
       "      <td>-1.097158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>257</th>\n",
       "      <td>1.0</td>\n",
       "      <td>5/1/99 8:34</td>\n",
       "      <td>1.005602</td>\n",
       "      <td>-3.876199</td>\n",
       "      <td>-1.074373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>258</th>\n",
       "      <td>1.0</td>\n",
       "      <td>5/1/99 8:36</td>\n",
       "      <td>0.933933</td>\n",
       "      <td>-3.868467</td>\n",
       "      <td>-1.249954</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>260</th>\n",
       "      <td>0.0</td>\n",
       "      <td>5/1/99 10:50</td>\n",
       "      <td>0.020062</td>\n",
       "      <td>-3.987897</td>\n",
       "      <td>-1.248529</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       y          time        x1        x2        x3\n",
       "255  0.0   5/1/99 8:30  0.997107 -3.865720 -1.133779\n",
       "256  0.0   5/1/99 8:32  1.016235 -4.058394 -1.097158\n",
       "257  1.0   5/1/99 8:34  1.005602 -3.876199 -1.074373\n",
       "258  1.0   5/1/99 8:36  0.933933 -3.868467 -1.249954\n",
       "260  0.0  5/1/99 10:50  0.020062 -3.987897 -1.248529"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "'''\n",
    "Shift the data by 2 units, equal to 4 minutes.\n",
    "\n",
    "Test: Testing whether the shift happened correctly.\n",
    "'''\n",
    "print('Before shifting')  # Positive labeled rows before shifting.\n",
    "one_indexes = df.index[df['y'] == 1]\n",
    "display(df.iloc[(one_indexes[0]-3):(one_indexes[0]+2), 0:5].head(n=5))\n",
    "\n",
    "# Shift the response column y by 2 rows to do a 4-min ahead prediction.\n",
    "df = curve_shift(df, shift_by = -2)\n",
    "\n",
    "print('After shifting')  # Validating if the shift happened correctly.\n",
    "display(df.iloc[(one_indexes[0]-4):(one_indexes[0]+1), 0:5].head(n=5))  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we note here, we moved the positive label at 5/1/99 8:38 to t-1 and t-2 timestamps, and dropped row t. There is a time difference of more than 2 minutes between a break row and the next row because in the data consecutive break rows are deleted. This was done to prevent a classification model learn predicting a break after the break has happened. Refer https://arxiv.org/abs/1809.10717 for details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Remove time column, and the categorical columns\n",
    "df = df.drop(['time', 'x28', 'x61'], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prepare data for LSTM models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LSTM is a bit more demanding than other models. Significant amount of time and attention goes in preparing the data that fits an LSTM.\n",
    "\n",
    "First, we will create the 3-dimensional arrays of shape: (samples x timesteps x features). Samples mean the number of data points. Timesteps is the number of time steps we look back at any time t to make a prediction. This is also referred to as lookback period. The features is the number of features the data has, in other words, the number of predictors in a multivariate data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "input_X = df.loc[:, df.columns != 'y'].values  # converts the df to a numpy array\n",
    "input_y = df['y'].values\n",
    "\n",
    "n_features = input_X.shape[1]  # number of features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def temporalize(X, y, lookback):\n",
    "    X = []\n",
    "    y = []\n",
    "    for i in range(len(input_X)-lookback-1):\n",
    "        t = []\n",
    "        for j in range(1,lookback+1):\n",
    "            # Gather past records upto the lookback period\n",
    "            t.append(input_X[[(i+j+1)], :])\n",
    "        X.append(t)\n",
    "        y.append(input_y[i+lookback+1])\n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In LSTM, to make prediction at any time t, we will look at data from (t-lookback):t. In the following, we have an example to show how the input data are transformed with the `temporalize` function with `lookback=5`. For the modeling, we may use a longer lookback."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First instance of y = 1 in the original data\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "      <th>x1</th>\n",
       "      <th>x2</th>\n",
       "      <th>x3</th>\n",
       "      <th>x4</th>\n",
       "      <th>x5</th>\n",
       "      <th>x6</th>\n",
       "      <th>x7</th>\n",
       "      <th>x8</th>\n",
       "      <th>x9</th>\n",
       "      <th>...</th>\n",
       "      <th>x51</th>\n",
       "      <th>x52</th>\n",
       "      <th>x53</th>\n",
       "      <th>x54</th>\n",
       "      <th>x55</th>\n",
       "      <th>x56</th>\n",
       "      <th>x57</th>\n",
       "      <th>x58</th>\n",
       "      <th>x59</th>\n",
       "      <th>x60</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>252</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.987078</td>\n",
       "      <td>-4.025989</td>\n",
       "      <td>-1.210205</td>\n",
       "      <td>0.899603</td>\n",
       "      <td>0.450338</td>\n",
       "      <td>14.098854</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.051043</td>\n",
       "      <td>-0.059966</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.248703</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.014893</td>\n",
       "      <td>-67.454037</td>\n",
       "      <td>66.232568</td>\n",
       "      <td>4.114269</td>\n",
       "      <td>0.033726</td>\n",
       "      <td>4.845087</td>\n",
       "      <td>0.007776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>253</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.921726</td>\n",
       "      <td>-3.728572</td>\n",
       "      <td>-1.230373</td>\n",
       "      <td>-1.598718</td>\n",
       "      <td>0.227178</td>\n",
       "      <td>14.594612</td>\n",
       "      <td>0.000061</td>\n",
       "      <td>-0.051043</td>\n",
       "      <td>-0.040129</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.253342</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.014987</td>\n",
       "      <td>-58.029477</td>\n",
       "      <td>66.310022</td>\n",
       "      <td>3.537487</td>\n",
       "      <td>0.032518</td>\n",
       "      <td>4.969500</td>\n",
       "      <td>0.007760</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>254</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.975947</td>\n",
       "      <td>-3.913736</td>\n",
       "      <td>-1.304682</td>\n",
       "      <td>0.561987</td>\n",
       "      <td>0.004034</td>\n",
       "      <td>14.630532</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.051043</td>\n",
       "      <td>-0.040129</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.257736</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.015081</td>\n",
       "      <td>-61.783749</td>\n",
       "      <td>71.917352</td>\n",
       "      <td>3.473400</td>\n",
       "      <td>0.031310</td>\n",
       "      <td>2.981432</td>\n",
       "      <td>0.007743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>255</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.997107</td>\n",
       "      <td>-3.865720</td>\n",
       "      <td>-1.133779</td>\n",
       "      <td>0.377295</td>\n",
       "      <td>-0.219126</td>\n",
       "      <td>14.666420</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>-0.040129</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.262375</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.015176</td>\n",
       "      <td>-70.151791</td>\n",
       "      <td>73.876977</td>\n",
       "      <td>3.473400</td>\n",
       "      <td>0.030776</td>\n",
       "      <td>2.563593</td>\n",
       "      <td>0.007727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>256</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.016235</td>\n",
       "      <td>-4.058394</td>\n",
       "      <td>-1.097158</td>\n",
       "      <td>2.327307</td>\n",
       "      <td>-0.442286</td>\n",
       "      <td>14.702309</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>-0.040129</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.267013</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.015270</td>\n",
       "      <td>-60.884701</td>\n",
       "      <td>72.188928</td>\n",
       "      <td>4.114269</td>\n",
       "      <td>0.031186</td>\n",
       "      <td>2.982454</td>\n",
       "      <td>0.007711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>257</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.005602</td>\n",
       "      <td>-3.876199</td>\n",
       "      <td>-1.074373</td>\n",
       "      <td>0.844397</td>\n",
       "      <td>-0.553050</td>\n",
       "      <td>14.738228</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>-0.030057</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.271652</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.015364</td>\n",
       "      <td>-69.553891</td>\n",
       "      <td>70.500879</td>\n",
       "      <td>4.050182</td>\n",
       "      <td>0.031596</td>\n",
       "      <td>3.746714</td>\n",
       "      <td>0.007695</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>6 rows × 60 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       y        x1        x2        x3        x4        x5         x6  \\\n",
       "252  0.0  0.987078 -4.025989 -1.210205  0.899603  0.450338  14.098854   \n",
       "253  0.0  0.921726 -3.728572 -1.230373 -1.598718  0.227178  14.594612   \n",
       "254  0.0  0.975947 -3.913736 -1.304682  0.561987  0.004034  14.630532   \n",
       "255  0.0  0.997107 -3.865720 -1.133779  0.377295 -0.219126  14.666420   \n",
       "256  0.0  1.016235 -4.058394 -1.097158  2.327307 -0.442286  14.702309   \n",
       "257  1.0  1.005602 -3.876199 -1.074373  0.844397 -0.553050  14.738228   \n",
       "\n",
       "           x7        x8        x9    ...           x51        x52       x53  \\\n",
       "252  0.000732 -0.051043 -0.059966    ...     29.984624  11.248703 -0.752385   \n",
       "253  0.000061 -0.051043 -0.040129    ...     29.984624  11.253342 -0.752385   \n",
       "254  0.000732 -0.051043 -0.040129    ...     29.984624  11.257736 -0.752385   \n",
       "255  0.000732 -0.061114 -0.040129    ...     29.984624  11.262375 -0.752385   \n",
       "256  0.000732 -0.061114 -0.040129    ...     29.984624  11.267013 -0.752385   \n",
       "257  0.000732 -0.061114 -0.030057    ...     29.984624  11.271652 -0.752385   \n",
       "\n",
       "          x54        x55        x56       x57       x58       x59       x60  \n",
       "252 -5.014893 -67.454037  66.232568  4.114269  0.033726  4.845087  0.007776  \n",
       "253 -5.014987 -58.029477  66.310022  3.537487  0.032518  4.969500  0.007760  \n",
       "254 -5.015081 -61.783749  71.917352  3.473400  0.031310  2.981432  0.007743  \n",
       "255 -5.015176 -70.151791  73.876977  3.473400  0.030776  2.563593  0.007727  \n",
       "256 -5.015270 -60.884701  72.188928  4.114269  0.031186  2.982454  0.007711  \n",
       "257 -5.015364 -69.553891  70.500879  4.050182  0.031596  3.746714  0.007695  \n",
       "\n",
       "[6 rows x 60 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For the same instance of y = 1, we are keeping past 3 samples in the 3D predictor array, X.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>49</th>\n",
       "      <th>50</th>\n",
       "      <th>51</th>\n",
       "      <th>52</th>\n",
       "      <th>53</th>\n",
       "      <th>54</th>\n",
       "      <th>55</th>\n",
       "      <th>56</th>\n",
       "      <th>57</th>\n",
       "      <th>58</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.921726</td>\n",
       "      <td>-3.728572</td>\n",
       "      <td>-1.230373</td>\n",
       "      <td>-1.598718</td>\n",
       "      <td>0.227178</td>\n",
       "      <td>14.594612</td>\n",
       "      <td>0.000061</td>\n",
       "      <td>-0.051043</td>\n",
       "      <td>-0.040129</td>\n",
       "      <td>0.001791</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.253342</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.014987</td>\n",
       "      <td>-58.029477</td>\n",
       "      <td>66.310022</td>\n",
       "      <td>3.537487</td>\n",
       "      <td>0.032518</td>\n",
       "      <td>4.969500</td>\n",
       "      <td>0.007760</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.975947</td>\n",
       "      <td>-3.913736</td>\n",
       "      <td>-1.304682</td>\n",
       "      <td>0.561987</td>\n",
       "      <td>0.004034</td>\n",
       "      <td>14.630532</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.051043</td>\n",
       "      <td>-0.040129</td>\n",
       "      <td>0.001791</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.257736</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.015081</td>\n",
       "      <td>-61.783749</td>\n",
       "      <td>71.917352</td>\n",
       "      <td>3.473400</td>\n",
       "      <td>0.031310</td>\n",
       "      <td>2.981432</td>\n",
       "      <td>0.007743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.997107</td>\n",
       "      <td>-3.865720</td>\n",
       "      <td>-1.133779</td>\n",
       "      <td>0.377295</td>\n",
       "      <td>-0.219126</td>\n",
       "      <td>14.666420</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>-0.040129</td>\n",
       "      <td>0.001791</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.262375</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.015176</td>\n",
       "      <td>-70.151791</td>\n",
       "      <td>73.876977</td>\n",
       "      <td>3.473400</td>\n",
       "      <td>0.030776</td>\n",
       "      <td>2.563593</td>\n",
       "      <td>0.007727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.016235</td>\n",
       "      <td>-4.058394</td>\n",
       "      <td>-1.097158</td>\n",
       "      <td>2.327307</td>\n",
       "      <td>-0.442286</td>\n",
       "      <td>14.702309</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>-0.040129</td>\n",
       "      <td>0.001791</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.267013</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.015270</td>\n",
       "      <td>-60.884701</td>\n",
       "      <td>72.188928</td>\n",
       "      <td>4.114269</td>\n",
       "      <td>0.031186</td>\n",
       "      <td>2.982454</td>\n",
       "      <td>0.007711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.005602</td>\n",
       "      <td>-3.876199</td>\n",
       "      <td>-1.074373</td>\n",
       "      <td>0.844397</td>\n",
       "      <td>-0.553050</td>\n",
       "      <td>14.738228</td>\n",
       "      <td>0.000732</td>\n",
       "      <td>-0.061114</td>\n",
       "      <td>-0.030057</td>\n",
       "      <td>0.001791</td>\n",
       "      <td>...</td>\n",
       "      <td>29.984624</td>\n",
       "      <td>11.271652</td>\n",
       "      <td>-0.752385</td>\n",
       "      <td>-5.015364</td>\n",
       "      <td>-69.553891</td>\n",
       "      <td>70.500879</td>\n",
       "      <td>4.050182</td>\n",
       "      <td>0.031596</td>\n",
       "      <td>3.746714</td>\n",
       "      <td>0.007695</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 59 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         0         1         2         3         4          5         6   \\\n",
       "0  0.921726 -3.728572 -1.230373 -1.598718  0.227178  14.594612  0.000061   \n",
       "1  0.975947 -3.913736 -1.304682  0.561987  0.004034  14.630532  0.000732   \n",
       "2  0.997107 -3.865720 -1.133779  0.377295 -0.219126  14.666420  0.000732   \n",
       "3  1.016235 -4.058394 -1.097158  2.327307 -0.442286  14.702309  0.000732   \n",
       "4  1.005602 -3.876199 -1.074373  0.844397 -0.553050  14.738228  0.000732   \n",
       "\n",
       "         7         8         9     ...            49         50        51  \\\n",
       "0 -0.051043 -0.040129  0.001791    ...     29.984624  11.253342 -0.752385   \n",
       "1 -0.051043 -0.040129  0.001791    ...     29.984624  11.257736 -0.752385   \n",
       "2 -0.061114 -0.040129  0.001791    ...     29.984624  11.262375 -0.752385   \n",
       "3 -0.061114 -0.040129  0.001791    ...     29.984624  11.267013 -0.752385   \n",
       "4 -0.061114 -0.030057  0.001791    ...     29.984624  11.271652 -0.752385   \n",
       "\n",
       "         52         53         54        55        56        57        58  \n",
       "0 -5.014987 -58.029477  66.310022  3.537487  0.032518  4.969500  0.007760  \n",
       "1 -5.015081 -61.783749  71.917352  3.473400  0.031310  2.981432  0.007743  \n",
       "2 -5.015176 -70.151791  73.876977  3.473400  0.030776  2.563593  0.007727  \n",
       "3 -5.015270 -60.884701  72.188928  4.114269  0.031186  2.982454  0.007711  \n",
       "4 -5.015364 -69.553891  70.500879  4.050182  0.031596  3.746714  0.007695  \n",
       "\n",
       "[5 rows x 59 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "'''\n",
    "Test: The 3D tensors (arrays) for LSTM are forming correctly.\n",
    "'''\n",
    "print('First instance of y = 1 in the original data')\n",
    "display(df.iloc[(np.where(np.array(input_y) == 1)[0][0]-5):(np.where(np.array(input_y) == 1)[0][0]+1), ])\n",
    "\n",
    "lookback = 5  # Equivalent to 10 min of past data.\n",
    "# Temporalize the data\n",
    "X, y = temporalize(X = input_X, y = input_y, lookback = lookback)\n",
    "\n",
    "print('For the same instance of y = 1, we are keeping past 5 samples in the 3D predictor array, X.')\n",
    "display(pd.DataFrame(np.concatenate(X[np.where(np.array(y) == 1)[0][0]], axis=0 ))) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The two tables are the same. This testifies that we are correctly taking 3 samples (= lookback), X(t):X(t-3) to predict y(t)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Divide the data into train, valid, and test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(np.array(X), np.array(y), test_size=DATA_SPLIT_PCT, random_state=SEED)\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(X_train, y_train, test_size=DATA_SPLIT_PCT, random_state=SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11691, 5, 1, 59)"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train_y0 = X_train[y_train==0]\n",
    "X_train_y1 = X_train[y_train==1]\n",
    "\n",
    "X_valid_y0 = X_valid[y_valid==0]\n",
    "X_valid_y1 = X_valid[y_valid==1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11536, 5, 1, 59)"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_y0.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Reshaping the data\n",
    "The tensors we have here are 4-dimensional. We will reshape them into the desired 3-dimensions corresponding to sample x lookback x features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train = X_train.reshape(X_train.shape[0], lookback, n_features)\n",
    "X_train_y0 = X_train_y0.reshape(X_train_y0.shape[0], lookback, n_features)\n",
    "X_train_y1 = X_train_y1.reshape(X_train_y1.shape[0], lookback, n_features)\n",
    "\n",
    "X_test = X_test.reshape(X_test.shape[0], lookback, n_features)\n",
    "\n",
    "X_valid = X_valid.reshape(X_valid.shape[0], lookback, n_features)\n",
    "X_valid_y0 = X_valid_y0.reshape(X_valid_y0.shape[0], lookback, n_features)\n",
    "X_valid_y1 = X_valid_y1.reshape(X_valid_y1.shape[0], lookback, n_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Standardize the data\n",
    "It is usually better to use a standardized data (transformed to Gaussian, mean 0 and sd 1) for autoencoders.\n",
    "\n",
    "One common mistake is: we normalize the entire data and then split into train-test. This is not correct. Test data should be completely unseen to anything during the modeling. We should normalize the test data using the feature summary statistics computed from the training data. For normalization, these statistics are the mean and variance for each feature. \n",
    "\n",
    "The same logic should be used for the validation set. This makes the model more stable for a test data.\n",
    "\n",
    "To do this, we will require two UDFs.\n",
    "\n",
    "- `flatten`: This function will re-create the original 2D array from which the 3D arrays were created. This function is the inverse of `temporalize`, meaning `X = flatten(temporalize(X))`.\n",
    "- `scale`: This function will scale a 3D array that we created as inputs to the LSTM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def flatten(X):\n",
    "    '''\n",
    "    Flatten a 3D array.\n",
    "    \n",
    "    Input\n",
    "    X            A 3D array for lstm, where the array is sample x timesteps x features.\n",
    "    \n",
    "    Output\n",
    "    flattened_X  A 2D array, sample x features.\n",
    "    '''\n",
    "    flattened_X = np.empty((X.shape[0], X.shape[2]))  # sample x features array.\n",
    "    for i in range(X.shape[0]):\n",
    "        flattened_X[i] = X[i, (X.shape[1]-1), :]\n",
    "    return(flattened_X)\n",
    "\n",
    "def scale(X, scaler):\n",
    "    '''\n",
    "    Scale 3D array.\n",
    "\n",
    "    Inputs\n",
    "    X            A 3D array for lstm, where the array is sample x timesteps x features.\n",
    "    scaler       A scaler object, e.g., sklearn.preprocessing.StandardScaler, sklearn.preprocessing.normalize\n",
    "    \n",
    "    Output\n",
    "    X            Scaled 3D array.\n",
    "    '''\n",
    "    for i in range(X.shape[0]):\n",
    "        X[i, :, :] = scaler.transform(X[i, :, :])\n",
    "        \n",
    "    return X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Initialize a scaler using the training data.\n",
    "scaler = StandardScaler().fit(flatten(X_train_y0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train_y0_scaled = scale(X_train_y0, scaler)\n",
    "X_train_y1_scaled = scale(X_train_y1, scaler)\n",
    "X_train_scaled = scale(X_train, scaler)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "colwise mean [ 0. -0.  0.  0.  0. -0.  0.  0.  0.  0. -0. -0. -0. -0. -0.  0. -0. -0.\n",
      " -0. -0.  0.  0. -0. -0.  0.  0. -0.  0. -0.  0.  0.  0. -0. -0. -0.  0.\n",
      "  0.  0. -0.  0.  0.  0. -0. -0.  0.  0.  0.  0.  0.  0.  0. -0. -0.  0.\n",
      "  0. -0. -0.  0.  0.]\n",
      "colwise variance [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n",
      " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n",
      " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "'''\n",
    "Test: Check if the scaling is correct.\n",
    "\n",
    "The test succeeds if all the column means \n",
    "and variances are 0 and 1, respectively, after\n",
    "flattening.\n",
    "'''\n",
    "a = flatten(X_train_y0_scaled)\n",
    "print('colwise mean', np.mean(a, axis=0).round(6))\n",
    "print('colwise variance', np.var(a, axis=0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The test succeeded. Now we will _scale_ the validation and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_valid_scaled = scale(X_valid, scaler)\n",
    "X_valid_y0_scaled = scale(X_valid_y0, scaler)\n",
    "\n",
    "X_test_scaled = scale(X_test, scaler)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LSTM Autoencoder training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we will initialize the Autoencoder architecture. We are building a simple autoencoder. More complex architectures and other configurations should be explored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "timesteps =  X_train_y0_scaled.shape[1] # equal to the lookback\n",
    "n_features =  X_train_y0_scaled.shape[2] # 59\n",
    "\n",
    "epochs = 1000\n",
    "batch = 64\n",
    "lr = 0.0001"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "lstm_16 (LSTM)               (None, 5, 5)              1300      \n",
      "_________________________________________________________________\n",
      "lstm_17 (LSTM)               (None, 5, 16)             1408      \n",
      "_________________________________________________________________\n",
      "lstm_18 (LSTM)               (None, 1)                 72        \n",
      "_________________________________________________________________\n",
      "repeat_vector_4 (RepeatVecto (None, 5, 1)              0         \n",
      "_________________________________________________________________\n",
      "lstm_19 (LSTM)               (None, 5, 5)              140       \n",
      "_________________________________________________________________\n",
      "lstm_20 (LSTM)               (None, 5, 16)             1408      \n",
      "_________________________________________________________________\n",
      "time_distributed_4 (TimeDist (None, 5, 59)             1003      \n",
      "=================================================================\n",
      "Total params: 5,331\n",
      "Trainable params: 5,331\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "lstm_autoencoder = Sequential()\n",
    "# Encoder\n",
    "lstm_autoencoder.add(LSTM(timesteps, activation='relu', input_shape=(timesteps, n_features), return_sequences=True))\n",
    "lstm_autoencoder.add(LSTM(16, activation='relu', return_sequences=True))\n",
    "lstm_autoencoder.add(LSTM(1, activation='relu'))\n",
    "lstm_autoencoder.add(RepeatVector(timesteps))\n",
    "# Decoder\n",
    "lstm_autoencoder.add(LSTM(timesteps, activation='relu', return_sequences=True))\n",
    "lstm_autoencoder.add(LSTM(16, activation='relu', return_sequences=True))\n",
    "lstm_autoencoder.add(TimeDistributed(Dense(n_features)))\n",
    "\n",
    "lstm_autoencoder.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a rule-of-thumb, look at the number of parameters. If not using any regularization, keep this less than the number of samples. If using regularization, depending on the degree of regularization you can let more parameters in the model that is greater than the sample size. For example, if using dropout with 0.5, you can have up to double the sample size (loosely speaking)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 11536 samples, validate on 2880 samples\n",
      "Epoch 1/1000\n",
      " - 7s - loss: 0.9958 - val_loss: 1.0278\n",
      "Epoch 2/1000\n",
      " - 2s - loss: 0.9955 - val_loss: 1.0265\n",
      "Epoch 3/1000\n",
      " - 2s - loss: 0.9717 - val_loss: 0.9727\n",
      "Epoch 4/1000\n",
      " - 2s - loss: 0.9199 - val_loss: 0.9383\n",
      "Epoch 5/1000\n",
      " - 2s - loss: 0.8918 - val_loss: 0.9146\n",
      "Epoch 6/1000\n",
      " - 2s - loss: 0.8699 - val_loss: 0.8955\n",
      "Epoch 7/1000\n",
      " - 2s - loss: 0.8526 - val_loss: 0.8809\n",
      "Epoch 8/1000\n",
      " - 2s - loss: 0.8399 - val_loss: 0.8703\n",
      "Epoch 9/1000\n",
      " - 2s - loss: 0.8300 - val_loss: 0.8611\n",
      "Epoch 10/1000\n",
      " - 2s - loss: 0.8209 - val_loss: 0.8522\n",
      "Epoch 11/1000\n",
      " - 2s - loss: 0.8119 - val_loss: 0.8430\n",
      "Epoch 12/1000\n",
      " - 2s - loss: 0.8027 - val_loss: 0.8340\n",
      "Epoch 13/1000\n",
      " - 2s - loss: 0.7937 - val_loss: 0.8250\n",
      "Epoch 14/1000\n",
      " - 2s - loss: 0.7852 - val_loss: 0.8169\n",
      "Epoch 15/1000\n",
      " - 2s - loss: 0.7773 - val_loss: 0.8095\n",
      "Epoch 16/1000\n",
      " - 2s - loss: 0.7704 - val_loss: 0.8029\n",
      "Epoch 17/1000\n",
      " - 2s - loss: 0.7643 - val_loss: 0.7972\n",
      "Epoch 18/1000\n",
      " - 2s - loss: 0.7590 - val_loss: 0.7921\n",
      "Epoch 19/1000\n",
      " - 2s - loss: 0.7544 - val_loss: 0.7877\n",
      "Epoch 20/1000\n",
      " - 2s - loss: 0.7501 - val_loss: 0.7837\n",
      "Epoch 21/1000\n",
      " - 2s - loss: 0.7460 - val_loss: 0.7797\n",
      "Epoch 22/1000\n",
      " - 2s - loss: 0.7420 - val_loss: 0.7759\n",
      "Epoch 23/1000\n",
      " - 2s - loss: 0.7380 - val_loss: 0.7719\n",
      "Epoch 24/1000\n",
      " - 2s - loss: 0.7337 - val_loss: 0.7679\n",
      "Epoch 25/1000\n",
      " - 2s - loss: 0.7294 - val_loss: 0.7639\n",
      "Epoch 26/1000\n",
      " - 2s - loss: 0.7252 - val_loss: 0.7601\n",
      "Epoch 27/1000\n",
      " - 2s - loss: 0.7211 - val_loss: 0.7563\n",
      "Epoch 28/1000\n",
      " - 2s - loss: 0.7173 - val_loss: 0.7528\n",
      "Epoch 29/1000\n",
      " - 2s - loss: 0.7136 - val_loss: 0.7493\n",
      "Epoch 30/1000\n",
      " - 2s - loss: 0.7100 - val_loss: 0.7458\n",
      "Epoch 31/1000\n",
      " - 2s - loss: 0.7064 - val_loss: 0.7423\n",
      "Epoch 32/1000\n",
      " - 2s - loss: 0.7029 - val_loss: 0.7387\n",
      "Epoch 33/1000\n",
      " - 2s - loss: 0.6992 - val_loss: 0.7349\n",
      "Epoch 34/1000\n",
      " - 2s - loss: 0.6955 - val_loss: 0.7308\n",
      "Epoch 35/1000\n",
      " - 2s - loss: 0.6914 - val_loss: 0.7259\n",
      "Epoch 36/1000\n",
      " - 2s - loss: 0.6870 - val_loss: 0.7206\n",
      "Epoch 37/1000\n",
      " - 2s - loss: 0.6820 - val_loss: 0.7149\n",
      "Epoch 38/1000\n",
      " - 2s - loss: 0.6768 - val_loss: 0.7089\n",
      "Epoch 39/1000\n",
      " - 2s - loss: 0.6715 - val_loss: 0.7029\n",
      "Epoch 40/1000\n",
      " - 2s - loss: 0.6664 - val_loss: 0.6971\n",
      "Epoch 41/1000\n",
      " - 2s - loss: 0.6613 - val_loss: 0.6914\n",
      "Epoch 42/1000\n",
      " - 2s - loss: 0.6561 - val_loss: 0.6854\n",
      "Epoch 43/1000\n",
      " - 2s - loss: 0.6510 - val_loss: 0.6793\n",
      "Epoch 44/1000\n",
      " - 2s - loss: 0.6459 - val_loss: 0.6738\n",
      "Epoch 45/1000\n",
      " - 2s - loss: 0.6408 - val_loss: 0.6672\n",
      "Epoch 46/1000\n",
      " - 2s - loss: 0.6359 - val_loss: 0.6614\n",
      "Epoch 47/1000\n",
      " - 2s - loss: 0.6311 - val_loss: 0.6563\n",
      "Epoch 48/1000\n",
      " - 2s - loss: 0.6266 - val_loss: 0.6516\n",
      "Epoch 49/1000\n",
      " - 2s - loss: 0.6221 - val_loss: 0.6459\n",
      "Epoch 50/1000\n",
      " - 2s - loss: 0.6183 - val_loss: 0.6422\n",
      "Epoch 51/1000\n",
      " - 2s - loss: 0.6144 - val_loss: 0.6381\n",
      "Epoch 52/1000\n",
      " - 2s - loss: 0.6106 - val_loss: 0.6332\n",
      "Epoch 53/1000\n",
      " - 2s - loss: 0.6071 - val_loss: 0.6297\n",
      "Epoch 54/1000\n",
      " - 2s - loss: 0.6037 - val_loss: 0.6257\n",
      "Epoch 55/1000\n",
      " - 2s - loss: 0.6007 - val_loss: 0.6219\n",
      "Epoch 56/1000\n",
      " - 2s - loss: 0.5979 - val_loss: 0.6186\n",
      "Epoch 57/1000\n",
      " - 2s - loss: 0.5948 - val_loss: 0.6155\n",
      "Epoch 58/1000\n",
      " - 2s - loss: 0.5920 - val_loss: 0.6121\n",
      "Epoch 59/1000\n",
      " - 2s - loss: 0.5891 - val_loss: 0.6086\n",
      "Epoch 60/1000\n",
      " - 2s - loss: 0.5866 - val_loss: 0.6056\n",
      "Epoch 61/1000\n",
      " - 2s - loss: 0.5840 - val_loss: 0.6023\n",
      "Epoch 62/1000\n",
      " - 2s - loss: 0.5816 - val_loss: 0.5999\n",
      "Epoch 63/1000\n",
      " - 2s - loss: 0.5793 - val_loss: 0.5972\n",
      "Epoch 64/1000\n",
      " - 2s - loss: 0.5773 - val_loss: 0.5963\n",
      "Epoch 65/1000\n",
      " - 2s - loss: 0.5755 - val_loss: 0.5936\n",
      "Epoch 66/1000\n",
      " - 2s - loss: 0.5736 - val_loss: 0.5919\n",
      "Epoch 67/1000\n",
      " - 2s - loss: 0.5720 - val_loss: 0.5898\n",
      "Epoch 68/1000\n",
      " - 2s - loss: 0.5701 - val_loss: 0.5885\n",
      "Epoch 69/1000\n",
      " - 2s - loss: 0.5686 - val_loss: 0.5866\n",
      "Epoch 70/1000\n",
      " - 2s - loss: 0.5669 - val_loss: 0.5843\n",
      "Epoch 71/1000\n",
      " - 2s - loss: 0.5654 - val_loss: 0.5830\n",
      "Epoch 72/1000\n",
      " - 2s - loss: 0.5639 - val_loss: 0.5805\n",
      "Epoch 73/1000\n",
      " - 2s - loss: 0.5624 - val_loss: 0.5785\n",
      "Epoch 74/1000\n",
      " - 2s - loss: 0.5607 - val_loss: 0.5764\n",
      "Epoch 75/1000\n",
      " - 2s - loss: 0.5592 - val_loss: 0.5749\n",
      "Epoch 76/1000\n",
      " - 2s - loss: 0.5576 - val_loss: 0.5735\n",
      "Epoch 77/1000\n",
      " - 2s - loss: 0.5562 - val_loss: 0.5724\n",
      "Epoch 78/1000\n",
      " - 2s - loss: 0.5550 - val_loss: 0.5702\n",
      "Epoch 79/1000\n",
      " - 2s - loss: 0.5536 - val_loss: 0.5689\n",
      "Epoch 80/1000\n",
      " - 2s - loss: 0.5520 - val_loss: 0.5684\n",
      "Epoch 81/1000\n",
      " - 2s - loss: 0.5508 - val_loss: 0.5662\n",
      "Epoch 82/1000\n",
      " - 2s - loss: 0.5493 - val_loss: 0.5643\n",
      "Epoch 83/1000\n",
      " - 2s - loss: 0.5476 - val_loss: 0.5631\n",
      "Epoch 84/1000\n",
      " - 2s - loss: 0.5462 - val_loss: 0.5614\n",
      "Epoch 85/1000\n",
      " - 2s - loss: 0.5447 - val_loss: 0.5596\n",
      "Epoch 86/1000\n",
      " - 2s - loss: 0.5434 - val_loss: 0.5585\n",
      "Epoch 87/1000\n",
      " - 2s - loss: 0.5417 - val_loss: 0.5571\n",
      "Epoch 88/1000\n",
      " - 2s - loss: 0.5403 - val_loss: 0.5548\n",
      "Epoch 89/1000\n",
      " - 2s - loss: 0.5388 - val_loss: 0.5534\n",
      "Epoch 90/1000\n",
      " - 2s - loss: 0.5376 - val_loss: 0.5524\n",
      "Epoch 91/1000\n",
      " - 2s - loss: 0.5357 - val_loss: 0.5499\n",
      "Epoch 92/1000\n",
      " - 2s - loss: 0.5341 - val_loss: 0.5479\n",
      "Epoch 93/1000\n",
      " - 2s - loss: 0.5321 - val_loss: 0.5458\n",
      "Epoch 94/1000\n",
      " - 2s - loss: 0.5307 - val_loss: 0.5451\n",
      "Epoch 95/1000\n",
      " - 2s - loss: 0.5292 - val_loss: 0.5433\n",
      "Epoch 96/1000\n",
      " - 2s - loss: 0.5274 - val_loss: 0.5409\n",
      "Epoch 97/1000\n",
      " - 2s - loss: 0.5257 - val_loss: 0.5391\n",
      "Epoch 98/1000\n",
      " - 2s - loss: 0.5240 - val_loss: 0.5373\n",
      "Epoch 99/1000\n",
      " - 2s - loss: 0.5224 - val_loss: 0.5364\n",
      "Epoch 100/1000\n",
      " - 2s - loss: 0.5208 - val_loss: 0.5342\n",
      "Epoch 101/1000\n",
      " - 2s - loss: 0.5190 - val_loss: 0.5323\n",
      "Epoch 102/1000\n",
      " - 2s - loss: 0.5176 - val_loss: 0.5300\n",
      "Epoch 103/1000\n",
      " - 2s - loss: 0.5160 - val_loss: 0.5296\n",
      "Epoch 104/1000\n",
      " - 2s - loss: 0.5145 - val_loss: 0.5280\n",
      "Epoch 105/1000\n",
      " - 2s - loss: 0.5127 - val_loss: 0.5252\n",
      "Epoch 106/1000\n",
      " - 2s - loss: 0.5111 - val_loss: 0.5243\n",
      "Epoch 107/1000\n",
      " - 2s - loss: 0.5094 - val_loss: 0.5222\n",
      "Epoch 108/1000\n",
      " - 2s - loss: 0.5078 - val_loss: 0.5209\n",
      "Epoch 109/1000\n",
      " - 2s - loss: 0.5068 - val_loss: 0.5184\n",
      "Epoch 110/1000\n",
      " - 2s - loss: 0.5049 - val_loss: 0.5171\n",
      "Epoch 111/1000\n",
      " - 2s - loss: 0.5034 - val_loss: 0.5153\n",
      "Epoch 112/1000\n",
      " - 2s - loss: 0.5017 - val_loss: 0.5132\n",
      "Epoch 113/1000\n",
      " - 2s - loss: 0.5005 - val_loss: 0.5136\n",
      "Epoch 114/1000\n",
      " - 2s - loss: 0.4994 - val_loss: 0.5109\n",
      "Epoch 115/1000\n",
      " - 2s - loss: 0.4978 - val_loss: 0.5091\n",
      "Epoch 116/1000\n",
      " - 2s - loss: 0.4963 - val_loss: 0.5082\n",
      "Epoch 117/1000\n",
      " - 2s - loss: 0.4949 - val_loss: 0.5070\n",
      "Epoch 118/1000\n",
      " - 2s - loss: 0.4933 - val_loss: 0.5048\n",
      "Epoch 119/1000\n",
      " - 2s - loss: 0.4919 - val_loss: 0.5039\n",
      "Epoch 120/1000\n",
      " - 2s - loss: 0.4907 - val_loss: 0.5017\n",
      "Epoch 121/1000\n",
      " - 2s - loss: 0.4896 - val_loss: 0.5017\n",
      "Epoch 122/1000\n",
      " - 2s - loss: 0.4885 - val_loss: 0.4998\n",
      "Epoch 123/1000\n",
      " - 2s - loss: 0.4872 - val_loss: 0.4991\n",
      "Epoch 124/1000\n",
      " - 2s - loss: 0.4858 - val_loss: 0.4984\n",
      "Epoch 125/1000\n",
      " - 2s - loss: 0.4849 - val_loss: 0.4959\n",
      "Epoch 126/1000\n",
      " - 2s - loss: 0.4836 - val_loss: 0.4942\n",
      "Epoch 127/1000\n",
      " - 2s - loss: 0.4827 - val_loss: 0.4943\n",
      "Epoch 128/1000\n",
      " - 2s - loss: 0.4817 - val_loss: 0.4931\n",
      "Epoch 129/1000\n",
      " - 2s - loss: 0.4804 - val_loss: 0.4916\n",
      "Epoch 130/1000\n",
      " - 2s - loss: 0.4798 - val_loss: 0.4909\n",
      "Epoch 131/1000\n",
      " - 2s - loss: 0.4791 - val_loss: 0.4901\n",
      "Epoch 132/1000\n",
      " - 2s - loss: 0.4778 - val_loss: 0.4885\n",
      "Epoch 133/1000\n",
      " - 2s - loss: 0.4767 - val_loss: 0.4868\n",
      "Epoch 134/1000\n",
      " - 2s - loss: 0.4758 - val_loss: 0.4858\n",
      "Epoch 135/1000\n",
      " - 2s - loss: 0.4751 - val_loss: 0.4850\n",
      "Epoch 136/1000\n",
      " - 2s - loss: 0.4741 - val_loss: 0.4845\n",
      "Epoch 137/1000\n",
      " - 2s - loss: 0.4732 - val_loss: 0.4841\n",
      "Epoch 138/1000\n",
      " - 2s - loss: 0.4724 - val_loss: 0.4833\n",
      "Epoch 139/1000\n",
      " - 2s - loss: 0.4713 - val_loss: 0.4825\n",
      "Epoch 140/1000\n",
      " - 2s - loss: 0.4705 - val_loss: 0.4814\n",
      "Epoch 141/1000\n",
      " - 2s - loss: 0.4699 - val_loss: 0.4803\n",
      "Epoch 142/1000\n",
      " - 2s - loss: 0.4688 - val_loss: 0.4800\n",
      "Epoch 143/1000\n",
      " - 2s - loss: 0.4681 - val_loss: 0.4793\n",
      "Epoch 144/1000\n",
      " - 2s - loss: 0.4674 - val_loss: 0.4788\n",
      "Epoch 145/1000\n",
      " - 2s - loss: 0.4668 - val_loss: 0.4775\n",
      "Epoch 146/1000\n",
      " - 2s - loss: 0.4663 - val_loss: 0.4770\n",
      "Epoch 147/1000\n",
      " - 2s - loss: 0.4656 - val_loss: 0.4766\n",
      "Epoch 148/1000\n",
      " - 2s - loss: 0.4652 - val_loss: 0.4759\n",
      "Epoch 149/1000\n",
      " - 2s - loss: 0.4644 - val_loss: 0.4753\n",
      "Epoch 150/1000\n",
      " - 2s - loss: 0.4648 - val_loss: 0.4745\n",
      "Epoch 151/1000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " - 2s - loss: 0.4638 - val_loss: 0.4747\n",
      "Epoch 152/1000\n",
      " - 2s - loss: 0.4628 - val_loss: 0.4734\n",
      "Epoch 153/1000\n",
      " - 2s - loss: 0.4623 - val_loss: 0.4726\n",
      "Epoch 154/1000\n",
      " - 2s - loss: 0.4618 - val_loss: 0.4723\n",
      "Epoch 155/1000\n",
      " - 2s - loss: 0.4612 - val_loss: 0.4718\n",
      "Epoch 156/1000\n",
      " - 2s - loss: 0.4608 - val_loss: 0.4717\n",
      "Epoch 157/1000\n",
      " - 2s - loss: 0.4605 - val_loss: 0.4711\n",
      "Epoch 158/1000\n",
      " - 2s - loss: 0.4600 - val_loss: 0.4707\n",
      "Epoch 159/1000\n",
      " - 2s - loss: 0.4594 - val_loss: 0.4693\n",
      "Epoch 160/1000\n",
      " - 2s - loss: 0.4588 - val_loss: 0.4687\n",
      "Epoch 161/1000\n",
      " - 2s - loss: 0.4583 - val_loss: 0.4689\n",
      "Epoch 162/1000\n",
      " - 2s - loss: 0.4579 - val_loss: 0.4676\n",
      "Epoch 163/1000\n",
      " - 2s - loss: 0.4575 - val_loss: 0.4671\n",
      "Epoch 164/1000\n",
      " - 2s - loss: 0.4569 - val_loss: 0.4669\n",
      "Epoch 165/1000\n",
      " - 2s - loss: 0.4564 - val_loss: 0.4665\n",
      "Epoch 166/1000\n",
      " - 2s - loss: 0.4558 - val_loss: 0.4663\n",
      "Epoch 167/1000\n",
      " - 2s - loss: 0.4558 - val_loss: 0.4662\n",
      "Epoch 168/1000\n",
      " - 2s - loss: 0.4553 - val_loss: 0.4653\n",
      "Epoch 169/1000\n",
      " - 2s - loss: 0.4549 - val_loss: 0.4652\n",
      "Epoch 170/1000\n",
      " - 2s - loss: 0.4547 - val_loss: 0.4649\n",
      "Epoch 171/1000\n",
      " - 2s - loss: 0.4541 - val_loss: 0.4645\n",
      "Epoch 172/1000\n",
      " - 2s - loss: 0.4538 - val_loss: 0.4641\n",
      "Epoch 173/1000\n",
      " - 2s - loss: 0.4529 - val_loss: 0.4630\n",
      "Epoch 174/1000\n",
      " - 2s - loss: 0.4526 - val_loss: 0.4633\n",
      "Epoch 175/1000\n",
      " - 2s - loss: 0.4524 - val_loss: 0.4622\n",
      "Epoch 176/1000\n",
      " - 2s - loss: 0.4521 - val_loss: 0.4622\n",
      "Epoch 177/1000\n",
      " - 2s - loss: 0.4515 - val_loss: 0.4620\n",
      "Epoch 178/1000\n",
      " - 2s - loss: 0.4514 - val_loss: 0.4611\n",
      "Epoch 179/1000\n",
      " - 2s - loss: 0.4509 - val_loss: 0.4606\n",
      "Epoch 180/1000\n",
      " - 2s - loss: 0.4502 - val_loss: 0.4605\n",
      "Epoch 181/1000\n",
      " - 2s - loss: 0.4500 - val_loss: 0.4602\n",
      "Epoch 182/1000\n",
      " - 2s - loss: 0.4494 - val_loss: 0.4598\n",
      "Epoch 183/1000\n",
      " - 2s - loss: 0.4490 - val_loss: 0.4595\n",
      "Epoch 184/1000\n",
      " - 2s - loss: 0.4488 - val_loss: 0.4597\n",
      "Epoch 185/1000\n",
      " - 2s - loss: 0.4485 - val_loss: 0.4585\n",
      "Epoch 186/1000\n",
      " - 2s - loss: 0.4481 - val_loss: 0.4580\n",
      "Epoch 187/1000\n",
      " - 2s - loss: 0.4477 - val_loss: 0.4576\n",
      "Epoch 188/1000\n",
      " - 2s - loss: 0.4475 - val_loss: 0.4573\n",
      "Epoch 189/1000\n",
      " - 2s - loss: 0.4471 - val_loss: 0.4575\n",
      "Epoch 190/1000\n",
      " - 2s - loss: 0.4468 - val_loss: 0.4566\n",
      "Epoch 191/1000\n",
      " - 2s - loss: 0.4462 - val_loss: 0.4560\n",
      "Epoch 192/1000\n",
      " - 2s - loss: 0.4461 - val_loss: 0.4558\n",
      "Epoch 193/1000\n",
      " - 2s - loss: 0.4458 - val_loss: 0.4553\n",
      "Epoch 194/1000\n",
      " - 2s - loss: 0.4454 - val_loss: 0.4551\n",
      "Epoch 195/1000\n",
      " - 2s - loss: 0.4451 - val_loss: 0.4549\n",
      "Epoch 196/1000\n",
      " - 2s - loss: 0.4449 - val_loss: 0.4549\n",
      "Epoch 197/1000\n",
      " - 2s - loss: 0.4446 - val_loss: 0.4545\n",
      "Epoch 198/1000\n",
      " - 2s - loss: 0.4444 - val_loss: 0.4537\n",
      "Epoch 199/1000\n",
      " - 2s - loss: 0.4439 - val_loss: 0.4535\n",
      "Epoch 200/1000\n",
      " - 2s - loss: 0.4438 - val_loss: 0.4539\n",
      "Epoch 201/1000\n",
      " - 2s - loss: 0.4432 - val_loss: 0.4528\n",
      "Epoch 202/1000\n",
      " - 2s - loss: 0.4431 - val_loss: 0.4527\n",
      "Epoch 203/1000\n",
      " - 2s - loss: 0.4428 - val_loss: 0.4523\n",
      "Epoch 204/1000\n",
      " - 2s - loss: 0.4426 - val_loss: 0.4525\n",
      "Epoch 205/1000\n",
      " - 2s - loss: 0.4435 - val_loss: 0.4518\n",
      "Epoch 206/1000\n",
      " - 2s - loss: 0.4424 - val_loss: 0.4513\n",
      "Epoch 207/1000\n",
      " - 2s - loss: 0.4421 - val_loss: 0.4514\n",
      "Epoch 208/1000\n",
      " - 2s - loss: 0.4420 - val_loss: 0.4506\n",
      "Epoch 209/1000\n",
      " - 2s - loss: 0.4414 - val_loss: 0.4503\n",
      "Epoch 210/1000\n",
      " - 2s - loss: 0.4412 - val_loss: 0.4517\n",
      "Epoch 211/1000\n",
      " - 2s - loss: 0.4411 - val_loss: 0.4502\n",
      "Epoch 212/1000\n",
      " - 2s - loss: 0.4407 - val_loss: 0.4499\n",
      "Epoch 213/1000\n",
      " - 2s - loss: 0.4404 - val_loss: 0.4498\n",
      "Epoch 214/1000\n",
      " - 2s - loss: 0.4403 - val_loss: 0.4492\n",
      "Epoch 215/1000\n",
      " - 2s - loss: 0.4398 - val_loss: 0.4495\n",
      "Epoch 216/1000\n",
      " - 2s - loss: 0.4397 - val_loss: 0.4487\n",
      "Epoch 217/1000\n",
      " - 2s - loss: 0.4398 - val_loss: 0.4488\n",
      "Epoch 218/1000\n",
      " - 2s - loss: 0.4395 - val_loss: 0.4489\n",
      "Epoch 219/1000\n",
      " - 2s - loss: 0.4393 - val_loss: 0.4497\n",
      "Epoch 220/1000\n",
      " - 2s - loss: 0.4389 - val_loss: 0.4484\n",
      "Epoch 221/1000\n",
      " - 2s - loss: 0.4385 - val_loss: 0.4482\n",
      "Epoch 222/1000\n",
      " - 2s - loss: 0.4385 - val_loss: 0.4479\n",
      "Epoch 223/1000\n",
      " - 2s - loss: 0.4381 - val_loss: 0.4478\n",
      "Epoch 224/1000\n",
      " - 2s - loss: 0.4380 - val_loss: 0.4473\n",
      "Epoch 225/1000\n",
      " - 2s - loss: 0.4382 - val_loss: 0.4473\n",
      "Epoch 226/1000\n",
      " - 2s - loss: 0.4378 - val_loss: 0.4475\n",
      "Epoch 227/1000\n",
      " - 2s - loss: 0.4377 - val_loss: 0.4467\n",
      "Epoch 228/1000\n",
      " - 2s - loss: 0.4372 - val_loss: 0.4469\n",
      "Epoch 229/1000\n",
      " - 2s - loss: 0.4370 - val_loss: 0.4477\n",
      "Epoch 230/1000\n",
      " - 2s - loss: 0.4374 - val_loss: 0.4471\n",
      "Epoch 231/1000\n",
      " - 2s - loss: 0.4366 - val_loss: 0.4467\n",
      "Epoch 232/1000\n",
      " - 2s - loss: 0.4367 - val_loss: 0.4465\n",
      "Epoch 233/1000\n",
      " - 2s - loss: 0.4365 - val_loss: 0.4460\n",
      "Epoch 234/1000\n",
      " - 2s - loss: 0.4361 - val_loss: 0.4463\n",
      "Epoch 235/1000\n",
      " - 2s - loss: 0.4363 - val_loss: 0.4458\n",
      "Epoch 236/1000\n",
      " - 2s - loss: 0.4359 - val_loss: 0.4461\n",
      "Epoch 237/1000\n",
      " - 2s - loss: 0.4358 - val_loss: 0.4448\n",
      "Epoch 238/1000\n",
      " - 2s - loss: 0.4359 - val_loss: 0.4451\n",
      "Epoch 239/1000\n",
      " - 2s - loss: 0.4356 - val_loss: 0.4449\n",
      "Epoch 240/1000\n",
      " - 2s - loss: 0.4357 - val_loss: 0.4444\n",
      "Epoch 241/1000\n",
      " - 2s - loss: 0.4351 - val_loss: 0.4463\n",
      "Epoch 242/1000\n",
      " - 2s - loss: 0.4350 - val_loss: 0.4453\n",
      "Epoch 243/1000\n",
      " - 2s - loss: 0.4352 - val_loss: 0.4442\n",
      "Epoch 244/1000\n",
      " - 2s - loss: 0.4346 - val_loss: 0.4444\n",
      "Epoch 245/1000\n",
      " - 2s - loss: 0.4342 - val_loss: 0.4443\n",
      "Epoch 246/1000\n",
      " - 2s - loss: 0.4342 - val_loss: 0.4442\n",
      "Epoch 247/1000\n",
      " - 2s - loss: 0.4341 - val_loss: 0.4441\n",
      "Epoch 248/1000\n",
      " - 2s - loss: 0.4338 - val_loss: 0.4440\n",
      "Epoch 249/1000\n",
      " - 2s - loss: 0.4341 - val_loss: 0.4440\n",
      "Epoch 250/1000\n",
      " - 2s - loss: 0.4339 - val_loss: 0.4437\n",
      "Epoch 251/1000\n",
      " - 2s - loss: 0.4335 - val_loss: 0.4430\n",
      "Epoch 252/1000\n",
      " - 2s - loss: 0.4335 - val_loss: 0.4428\n",
      "Epoch 253/1000\n",
      " - 2s - loss: 0.4330 - val_loss: 0.4435\n",
      "Epoch 254/1000\n",
      " - 2s - loss: 0.4330 - val_loss: 0.4434\n",
      "Epoch 255/1000\n",
      " - 2s - loss: 0.4331 - val_loss: 0.4441\n",
      "Epoch 256/1000\n",
      " - 2s - loss: 0.4333 - val_loss: 0.4429\n",
      "Epoch 257/1000\n",
      " - 2s - loss: 0.4326 - val_loss: 0.4428\n",
      "Epoch 258/1000\n",
      " - 2s - loss: 0.4324 - val_loss: 0.4426\n",
      "Epoch 259/1000\n",
      " - 2s - loss: 0.4325 - val_loss: 0.4425\n",
      "Epoch 260/1000\n",
      " - 2s - loss: 0.4324 - val_loss: 0.4418\n",
      "Epoch 261/1000\n",
      " - 2s - loss: 0.4323 - val_loss: 0.4427\n",
      "Epoch 262/1000\n",
      " - 2s - loss: 0.4322 - val_loss: 0.4418\n",
      "Epoch 263/1000\n",
      " - 2s - loss: 0.4321 - val_loss: 0.4420\n",
      "Epoch 264/1000\n",
      " - 2s - loss: 0.4317 - val_loss: 0.4416\n",
      "Epoch 265/1000\n",
      " - 2s - loss: 0.4317 - val_loss: 0.4413\n",
      "Epoch 266/1000\n",
      " - 2s - loss: 0.4316 - val_loss: 0.4411\n",
      "Epoch 267/1000\n",
      " - 2s - loss: 0.4314 - val_loss: 0.4416\n",
      "Epoch 268/1000\n",
      " - 2s - loss: 0.4313 - val_loss: 0.4413\n",
      "Epoch 269/1000\n",
      " - 2s - loss: 0.4310 - val_loss: 0.4413\n",
      "Epoch 270/1000\n",
      " - 2s - loss: 0.4310 - val_loss: 0.4402\n",
      "Epoch 271/1000\n",
      " - 2s - loss: 0.4311 - val_loss: 0.4408\n",
      "Epoch 272/1000\n",
      " - 2s - loss: 0.4307 - val_loss: 0.4412\n",
      "Epoch 273/1000\n",
      " - 2s - loss: 0.4306 - val_loss: 0.4404\n",
      "Epoch 274/1000\n",
      " - 2s - loss: 0.4306 - val_loss: 0.4409\n",
      "Epoch 275/1000\n",
      " - 2s - loss: 0.4309 - val_loss: 0.4397\n",
      "Epoch 276/1000\n",
      " - 2s - loss: 0.4314 - val_loss: 0.4394\n",
      "Epoch 277/1000\n",
      " - 2s - loss: 0.4307 - val_loss: 0.4396\n",
      "Epoch 278/1000\n",
      " - 2s - loss: 0.4307 - val_loss: 0.4396\n",
      "Epoch 279/1000\n",
      " - 2s - loss: 0.4303 - val_loss: 0.4394\n",
      "Epoch 280/1000\n",
      " - 2s - loss: 0.4295 - val_loss: 0.4388\n",
      "Epoch 281/1000\n",
      " - 2s - loss: 0.4296 - val_loss: 0.4386\n",
      "Epoch 282/1000\n",
      " - 2s - loss: 0.4296 - val_loss: 0.4385\n",
      "Epoch 283/1000\n",
      " - 2s - loss: 0.4292 - val_loss: 0.4387\n",
      "Epoch 284/1000\n",
      " - 2s - loss: 0.4291 - val_loss: 0.4387\n",
      "Epoch 285/1000\n",
      " - 2s - loss: 0.4291 - val_loss: 0.4387\n",
      "Epoch 286/1000\n",
      " - 2s - loss: 0.4293 - val_loss: 0.4391\n",
      "Epoch 287/1000\n",
      " - 2s - loss: 0.4289 - val_loss: 0.4385\n",
      "Epoch 288/1000\n",
      " - 2s - loss: 0.4287 - val_loss: 0.4388\n",
      "Epoch 289/1000\n",
      " - 2s - loss: 0.4285 - val_loss: 0.4384\n",
      "Epoch 290/1000\n",
      " - 2s - loss: 0.4285 - val_loss: 0.4394\n",
      "Epoch 291/1000\n",
      " - 2s - loss: 0.4287 - val_loss: 0.4391\n",
      "Epoch 292/1000\n",
      " - 2s - loss: 0.4283 - val_loss: 0.4386\n",
      "Epoch 293/1000\n",
      " - 2s - loss: 0.4284 - val_loss: 0.4386\n",
      "Epoch 294/1000\n",
      " - 2s - loss: 0.4280 - val_loss: 0.4391\n",
      "Epoch 295/1000\n",
      " - 2s - loss: 0.4287 - val_loss: 0.4367\n",
      "Epoch 296/1000\n",
      " - 2s - loss: 0.4282 - val_loss: 0.4380\n",
      "Epoch 297/1000\n",
      " - 2s - loss: 0.4276 - val_loss: 0.4383\n",
      "Epoch 298/1000\n",
      " - 2s - loss: 0.4279 - val_loss: 0.4373\n",
      "Epoch 299/1000\n",
      " - 2s - loss: 0.4278 - val_loss: 0.4375\n",
      "Epoch 300/1000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " - 2s - loss: 0.4273 - val_loss: 0.4373\n",
      "Epoch 301/1000\n",
      " - 2s - loss: 0.4274 - val_loss: 0.4376\n",
      "Epoch 302/1000\n",
      " - 2s - loss: 0.4269 - val_loss: 0.4369\n",
      "Epoch 303/1000\n",
      " - 2s - loss: 0.4267 - val_loss: 0.4376\n",
      "Epoch 304/1000\n",
      " - 2s - loss: 0.4272 - val_loss: 0.4386\n",
      "Epoch 305/1000\n",
      " - 2s - loss: 0.4269 - val_loss: 0.4371\n",
      "Epoch 306/1000\n",
      " - 2s - loss: 0.4267 - val_loss: 0.4371\n",
      "Epoch 307/1000\n",
      " - 2s - loss: 0.4265 - val_loss: 0.4369\n",
      "Epoch 308/1000\n",
      " - 2s - loss: 0.4267 - val_loss: 0.4374\n",
      "Epoch 309/1000\n",
      " - 2s - loss: 0.4262 - val_loss: 0.4368\n",
      "Epoch 310/1000\n",
      " - 2s - loss: 0.4263 - val_loss: 0.4366\n",
      "Epoch 311/1000\n",
      " - 2s - loss: 0.4260 - val_loss: 0.4369\n",
      "Epoch 312/1000\n",
      " - 2s - loss: 0.4259 - val_loss: 0.4366\n",
      "Epoch 313/1000\n",
      " - 2s - loss: 0.4259 - val_loss: 0.4358\n",
      "Epoch 314/1000\n",
      " - 2s - loss: 0.4259 - val_loss: 0.4370\n",
      "Epoch 315/1000\n",
      " - 2s - loss: 0.4255 - val_loss: 0.4371\n",
      "Epoch 316/1000\n",
      " - 2s - loss: 0.4259 - val_loss: 0.4363\n",
      "Epoch 317/1000\n",
      " - 2s - loss: 0.4253 - val_loss: 0.4360\n",
      "Epoch 318/1000\n",
      " - 2s - loss: 0.4255 - val_loss: 0.4357\n",
      "Epoch 319/1000\n",
      " - 2s - loss: 0.4251 - val_loss: 0.4354\n",
      "Epoch 320/1000\n",
      " - 2s - loss: 0.4251 - val_loss: 0.4363\n",
      "Epoch 321/1000\n",
      " - 2s - loss: 0.4248 - val_loss: 0.4354\n",
      "Epoch 322/1000\n",
      " - 2s - loss: 0.4247 - val_loss: 0.4357\n",
      "Epoch 323/1000\n",
      " - 2s - loss: 0.4246 - val_loss: 0.4358\n",
      "Epoch 324/1000\n",
      " - 2s - loss: 0.4254 - val_loss: 0.4358\n",
      "Epoch 325/1000\n",
      " - 2s - loss: 0.4248 - val_loss: 0.4344\n",
      "Epoch 326/1000\n",
      " - 2s - loss: 0.4247 - val_loss: 0.4350\n",
      "Epoch 327/1000\n",
      " - 2s - loss: 0.4243 - val_loss: 0.4349\n",
      "Epoch 328/1000\n",
      " - 2s - loss: 0.4244 - val_loss: 0.4347\n",
      "Epoch 329/1000\n",
      " - 2s - loss: 0.4246 - val_loss: 0.4336\n",
      "Epoch 330/1000\n",
      " - 2s - loss: 0.4241 - val_loss: 0.4341\n",
      "Epoch 331/1000\n",
      " - 2s - loss: 0.4241 - val_loss: 0.4341\n",
      "Epoch 332/1000\n",
      " - 2s - loss: 0.4235 - val_loss: 0.4344\n",
      "Epoch 333/1000\n",
      " - 2s - loss: 0.4239 - val_loss: 0.4341\n",
      "Epoch 334/1000\n",
      " - 2s - loss: 0.4247 - val_loss: 0.4344\n",
      "Epoch 335/1000\n",
      " - 2s - loss: 0.4235 - val_loss: 0.4352\n",
      "Epoch 336/1000\n",
      " - 2s - loss: 0.4230 - val_loss: 0.4336\n",
      "Epoch 337/1000\n",
      " - 2s - loss: 0.4233 - val_loss: 0.4334\n",
      "Epoch 338/1000\n",
      " - 2s - loss: 0.4228 - val_loss: 0.4333\n",
      "Epoch 339/1000\n",
      " - 2s - loss: 0.4231 - val_loss: 0.4341\n",
      "Epoch 340/1000\n",
      " - 2s - loss: 0.4226 - val_loss: 0.4329\n",
      "Epoch 341/1000\n",
      " - 2s - loss: 0.4223 - val_loss: 0.4331\n",
      "Epoch 342/1000\n",
      " - 2s - loss: 0.4223 - val_loss: 0.4331\n",
      "Epoch 343/1000\n",
      " - 2s - loss: 0.4224 - val_loss: 0.4329\n",
      "Epoch 344/1000\n",
      " - 2s - loss: 0.4221 - val_loss: 0.4328\n",
      "Epoch 345/1000\n",
      " - 2s - loss: 0.4220 - val_loss: 0.4334\n",
      "Epoch 346/1000\n",
      " - 2s - loss: 0.4222 - val_loss: 0.4319\n",
      "Epoch 347/1000\n",
      " - 2s - loss: 0.4219 - val_loss: 0.4317\n",
      "Epoch 348/1000\n",
      " - 2s - loss: 0.4216 - val_loss: 0.4318\n",
      "Epoch 349/1000\n",
      " - 2s - loss: 0.4231 - val_loss: 0.4307\n",
      "Epoch 350/1000\n",
      " - 2s - loss: 0.4223 - val_loss: 0.4324\n",
      "Epoch 351/1000\n",
      " - 2s - loss: 0.4219 - val_loss: 0.4316\n",
      "Epoch 352/1000\n",
      " - 2s - loss: 0.4210 - val_loss: 0.4319\n",
      "Epoch 353/1000\n",
      " - 2s - loss: 0.4215 - val_loss: 0.4304\n",
      "Epoch 354/1000\n",
      " - 2s - loss: 0.4209 - val_loss: 0.4304\n",
      "Epoch 355/1000\n",
      " - 2s - loss: 0.4207 - val_loss: 0.4313\n",
      "Epoch 356/1000\n",
      " - 2s - loss: 0.4208 - val_loss: 0.4308\n",
      "Epoch 357/1000\n",
      " - 2s - loss: 0.4204 - val_loss: 0.4314\n",
      "Epoch 358/1000\n",
      " - 2s - loss: 0.4205 - val_loss: 0.4302\n",
      "Epoch 359/1000\n",
      " - 2s - loss: 0.4209 - val_loss: 0.4308\n",
      "Epoch 360/1000\n",
      " - 2s - loss: 0.4207 - val_loss: 0.4320\n",
      "Epoch 361/1000\n",
      " - 2s - loss: 0.4208 - val_loss: 0.4303\n",
      "Epoch 362/1000\n",
      " - 2s - loss: 0.4205 - val_loss: 0.4307\n",
      "Epoch 363/1000\n",
      " - 2s - loss: 0.4200 - val_loss: 0.4309\n",
      "Epoch 364/1000\n",
      " - 2s - loss: 0.4198 - val_loss: 0.4310\n",
      "Epoch 365/1000\n",
      " - 2s - loss: 0.4197 - val_loss: 0.4308\n",
      "Epoch 366/1000\n",
      " - 2s - loss: 0.4207 - val_loss: 0.4307\n",
      "Epoch 367/1000\n",
      " - 2s - loss: 0.4197 - val_loss: 0.4299\n",
      "Epoch 368/1000\n",
      " - 2s - loss: 0.4193 - val_loss: 0.4294\n",
      "Epoch 369/1000\n",
      " - 2s - loss: 0.4190 - val_loss: 0.4291\n",
      "Epoch 370/1000\n",
      " - 2s - loss: 0.4194 - val_loss: 0.4303\n",
      "Epoch 371/1000\n",
      " - 2s - loss: 0.4193 - val_loss: 0.4292\n",
      "Epoch 372/1000\n",
      " - 2s - loss: 0.4195 - val_loss: 0.4293\n",
      "Epoch 373/1000\n",
      " - 2s - loss: 0.4188 - val_loss: 0.4297\n",
      "Epoch 374/1000\n",
      " - 2s - loss: 0.4193 - val_loss: 0.4291\n",
      "Epoch 375/1000\n",
      " - 2s - loss: 0.4190 - val_loss: 0.4296\n",
      "Epoch 376/1000\n",
      " - 2s - loss: 0.4189 - val_loss: 0.4299\n",
      "Epoch 377/1000\n",
      " - 2s - loss: 0.4190 - val_loss: 0.4268\n",
      "Epoch 378/1000\n",
      " - 2s - loss: 0.4186 - val_loss: 0.4282\n",
      "Epoch 379/1000\n",
      " - 2s - loss: 0.4185 - val_loss: 0.4307\n",
      "Epoch 380/1000\n",
      " - 2s - loss: 0.4183 - val_loss: 0.4274\n",
      "Epoch 381/1000\n",
      " - 2s - loss: 0.4178 - val_loss: 0.4280\n",
      "Epoch 382/1000\n",
      " - 2s - loss: 0.4179 - val_loss: 0.4286\n",
      "Epoch 383/1000\n",
      " - 2s - loss: 0.4178 - val_loss: 0.4287\n",
      "Epoch 384/1000\n",
      " - 2s - loss: 0.4176 - val_loss: 0.4281\n",
      "Epoch 385/1000\n",
      " - 2s - loss: 0.4176 - val_loss: 0.4281\n",
      "Epoch 386/1000\n",
      " - 2s - loss: 0.4185 - val_loss: 0.4288\n",
      "Epoch 387/1000\n",
      " - 2s - loss: 0.4176 - val_loss: 0.4289\n",
      "Epoch 388/1000\n",
      " - 2s - loss: 0.4172 - val_loss: 0.4285\n",
      "Epoch 389/1000\n",
      " - 2s - loss: 0.4169 - val_loss: 0.4269\n",
      "Epoch 390/1000\n",
      " - 2s - loss: 0.4167 - val_loss: 0.4276\n",
      "Epoch 391/1000\n",
      " - 2s - loss: 0.4168 - val_loss: 0.4280\n",
      "Epoch 392/1000\n",
      " - 2s - loss: 0.4168 - val_loss: 0.4269\n",
      "Epoch 393/1000\n",
      " - 2s - loss: 0.4167 - val_loss: 0.4272\n",
      "Epoch 394/1000\n",
      " - 2s - loss: 0.4168 - val_loss: 0.4272\n",
      "Epoch 395/1000\n",
      " - 2s - loss: 0.4166 - val_loss: 0.4267\n",
      "Epoch 396/1000\n",
      " - 2s - loss: 0.4162 - val_loss: 0.4272\n",
      "Epoch 397/1000\n",
      " - 2s - loss: 0.4161 - val_loss: 0.4262\n",
      "Epoch 398/1000\n",
      " - 2s - loss: 0.4160 - val_loss: 0.4262\n",
      "Epoch 399/1000\n",
      " - 2s - loss: 0.4161 - val_loss: 0.4272\n",
      "Epoch 400/1000\n",
      " - 2s - loss: 0.4159 - val_loss: 0.4272\n",
      "Epoch 401/1000\n",
      " - 2s - loss: 0.4158 - val_loss: 0.4264\n",
      "Epoch 402/1000\n",
      " - 2s - loss: 0.4157 - val_loss: 0.4269\n",
      "Epoch 403/1000\n",
      " - 2s - loss: 0.4154 - val_loss: 0.4261\n",
      "Epoch 404/1000\n",
      " - 2s - loss: 0.4155 - val_loss: 0.4265\n",
      "Epoch 405/1000\n",
      " - 2s - loss: 0.4151 - val_loss: 0.4266\n",
      "Epoch 406/1000\n",
      " - 2s - loss: 0.4155 - val_loss: 0.4257\n",
      "Epoch 407/1000\n",
      " - 2s - loss: 0.4151 - val_loss: 0.4261\n",
      "Epoch 408/1000\n",
      " - 2s - loss: 0.4156 - val_loss: 0.4267\n",
      "Epoch 409/1000\n",
      " - 2s - loss: 0.4154 - val_loss: 0.4261\n",
      "Epoch 410/1000\n",
      " - 2s - loss: 0.4151 - val_loss: 0.4255\n",
      "Epoch 411/1000\n",
      " - 2s - loss: 0.4151 - val_loss: 0.4260\n",
      "Epoch 412/1000\n",
      " - 2s - loss: 0.4153 - val_loss: 0.4272\n",
      "Epoch 413/1000\n",
      " - 2s - loss: 0.4150 - val_loss: 0.4258\n",
      "Epoch 414/1000\n",
      " - 2s - loss: 0.4147 - val_loss: 0.4237\n",
      "Epoch 415/1000\n",
      " - 2s - loss: 0.4145 - val_loss: 0.4256\n",
      "Epoch 416/1000\n",
      " - 2s - loss: 0.4141 - val_loss: 0.4244\n",
      "Epoch 417/1000\n",
      " - 2s - loss: 0.4141 - val_loss: 0.4246\n",
      "Epoch 418/1000\n",
      " - 2s - loss: 0.4137 - val_loss: 0.4245\n",
      "Epoch 419/1000\n",
      " - 2s - loss: 0.4145 - val_loss: 0.4248\n",
      "Epoch 420/1000\n",
      " - 2s - loss: 0.4136 - val_loss: 0.4236\n",
      "Epoch 421/1000\n",
      " - 2s - loss: 0.4138 - val_loss: 0.4247\n",
      "Epoch 422/1000\n",
      " - 2s - loss: 0.4135 - val_loss: 0.4266\n",
      "Epoch 423/1000\n",
      " - 2s - loss: 0.4132 - val_loss: 0.4253\n",
      "Epoch 424/1000\n",
      " - 2s - loss: 0.4129 - val_loss: 0.4250\n",
      "Epoch 425/1000\n",
      " - 2s - loss: 0.4130 - val_loss: 0.4244\n",
      "Epoch 426/1000\n",
      " - 2s - loss: 0.4129 - val_loss: 0.4254\n",
      "Epoch 427/1000\n",
      " - 2s - loss: 0.4129 - val_loss: 0.4234\n",
      "Epoch 428/1000\n",
      " - 2s - loss: 0.4135 - val_loss: 0.4244\n",
      "Epoch 429/1000\n",
      " - 2s - loss: 0.4128 - val_loss: 0.4241\n",
      "Epoch 430/1000\n",
      " - 2s - loss: 0.4127 - val_loss: 0.4245\n",
      "Epoch 431/1000\n",
      " - 2s - loss: 0.4124 - val_loss: 0.4233\n",
      "Epoch 432/1000\n",
      " - 2s - loss: 0.4125 - val_loss: 0.4244\n",
      "Epoch 433/1000\n",
      " - 2s - loss: 0.4121 - val_loss: 0.4238\n",
      "Epoch 434/1000\n",
      " - 2s - loss: 0.4122 - val_loss: 0.4229\n",
      "Epoch 435/1000\n",
      " - 2s - loss: 0.4123 - val_loss: 0.4231\n",
      "Epoch 436/1000\n",
      " - 2s - loss: 0.4126 - val_loss: 0.4227\n",
      "Epoch 437/1000\n",
      " - 2s - loss: 0.4120 - val_loss: 0.4223\n",
      "Epoch 438/1000\n",
      " - 2s - loss: 0.4124 - val_loss: 0.4225\n",
      "Epoch 439/1000\n",
      " - 2s - loss: 0.4115 - val_loss: 0.4227\n",
      "Epoch 440/1000\n",
      " - 2s - loss: 0.4117 - val_loss: 0.4226\n",
      "Epoch 441/1000\n",
      " - 2s - loss: 0.4112 - val_loss: 0.4225\n",
      "Epoch 442/1000\n",
      " - 2s - loss: 0.4114 - val_loss: 0.4217\n",
      "Epoch 443/1000\n",
      " - 2s - loss: 0.4111 - val_loss: 0.4216\n",
      "Epoch 444/1000\n",
      " - 2s - loss: 0.4120 - val_loss: 0.4224\n",
      "Epoch 445/1000\n",
      " - 2s - loss: 0.4109 - val_loss: 0.4223\n",
      "Epoch 446/1000\n",
      " - 2s - loss: 0.4112 - val_loss: 0.4224\n",
      "Epoch 447/1000\n",
      " - 2s - loss: 0.4110 - val_loss: 0.4214\n",
      "Epoch 448/1000\n",
      " - 2s - loss: 0.4105 - val_loss: 0.4215\n",
      "Epoch 449/1000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " - 2s - loss: 0.4104 - val_loss: 0.4212\n",
      "Epoch 450/1000\n",
      " - 2s - loss: 0.4107 - val_loss: 0.4218\n",
      "Epoch 451/1000\n",
      " - 2s - loss: 0.4106 - val_loss: 0.4217\n",
      "Epoch 452/1000\n",
      " - 2s - loss: 0.4102 - val_loss: 0.4211\n",
      "Epoch 453/1000\n",
      " - 2s - loss: 0.4100 - val_loss: 0.4217\n",
      "Epoch 454/1000\n",
      " - 2s - loss: 0.4100 - val_loss: 0.4213\n",
      "Epoch 455/1000\n",
      " - 2s - loss: 0.4096 - val_loss: 0.4213\n",
      "Epoch 456/1000\n",
      " - 2s - loss: 0.4096 - val_loss: 0.4226\n",
      "Epoch 457/1000\n",
      " - 2s - loss: 0.4096 - val_loss: 0.4203\n",
      "Epoch 458/1000\n",
      " - 2s - loss: 0.4106 - val_loss: 0.4211\n",
      "Epoch 459/1000\n",
      " - 2s - loss: 0.4096 - val_loss: 0.4218\n",
      "Epoch 460/1000\n",
      " - 2s - loss: 0.4094 - val_loss: 0.4222\n",
      "Epoch 461/1000\n",
      " - 2s - loss: 0.4095 - val_loss: 0.4214\n",
      "Epoch 462/1000\n",
      " - 2s - loss: 0.4091 - val_loss: 0.4203\n",
      "Epoch 463/1000\n",
      " - 2s - loss: 0.4088 - val_loss: 0.4214\n",
      "Epoch 464/1000\n",
      " - 2s - loss: 0.4090 - val_loss: 0.4207\n",
      "Epoch 465/1000\n",
      " - 2s - loss: 0.4087 - val_loss: 0.4199\n",
      "Epoch 466/1000\n",
      " - 2s - loss: 0.4089 - val_loss: 0.4212\n",
      "Epoch 467/1000\n",
      " - 2s - loss: 0.4091 - val_loss: 0.4223\n",
      "Epoch 468/1000\n",
      " - 2s - loss: 0.4088 - val_loss: 0.4216\n",
      "Epoch 469/1000\n",
      " - 2s - loss: 0.4102 - val_loss: 0.4206\n",
      "Epoch 470/1000\n",
      " - 2s - loss: 0.4106 - val_loss: 0.4203\n",
      "Epoch 471/1000\n",
      " - 2s - loss: 0.4089 - val_loss: 0.4187\n",
      "Epoch 472/1000\n",
      " - 2s - loss: 0.4081 - val_loss: 0.4187\n",
      "Epoch 473/1000\n",
      " - 2s - loss: 0.4077 - val_loss: 0.4203\n",
      "Epoch 474/1000\n",
      " - 2s - loss: 0.4085 - val_loss: 0.4192\n",
      "Epoch 475/1000\n",
      " - 2s - loss: 0.4081 - val_loss: 0.4189\n",
      "Epoch 476/1000\n",
      " - 2s - loss: 0.4086 - val_loss: 0.4191\n",
      "Epoch 477/1000\n",
      " - 2s - loss: 0.4077 - val_loss: 0.4182\n",
      "Epoch 478/1000\n",
      " - 2s - loss: 0.4080 - val_loss: 0.4192\n",
      "Epoch 479/1000\n",
      " - 2s - loss: 0.4080 - val_loss: 0.4189\n",
      "Epoch 480/1000\n",
      " - 2s - loss: 0.4074 - val_loss: 0.4181\n",
      "Epoch 481/1000\n",
      " - 2s - loss: 0.4074 - val_loss: 0.4188\n",
      "Epoch 482/1000\n",
      " - 2s - loss: 0.4070 - val_loss: 0.4192\n",
      "Epoch 483/1000\n",
      " - 2s - loss: 0.4071 - val_loss: 0.4182\n",
      "Epoch 484/1000\n",
      " - 2s - loss: 0.4068 - val_loss: 0.4175\n",
      "Epoch 485/1000\n",
      " - 2s - loss: 0.4066 - val_loss: 0.4177\n",
      "Epoch 486/1000\n",
      " - 2s - loss: 0.4065 - val_loss: 0.4181\n",
      "Epoch 487/1000\n",
      " - 2s - loss: 0.4063 - val_loss: 0.4172\n",
      "Epoch 488/1000\n",
      " - 2s - loss: 0.4062 - val_loss: 0.4177\n",
      "Epoch 489/1000\n",
      " - 2s - loss: 0.4062 - val_loss: 0.4176\n",
      "Epoch 490/1000\n",
      " - 2s - loss: 0.4058 - val_loss: 0.4179\n",
      "Epoch 491/1000\n",
      " - 2s - loss: 0.4058 - val_loss: 0.4163\n",
      "Epoch 492/1000\n",
      " - 2s - loss: 0.4056 - val_loss: 0.4189\n",
      "Epoch 493/1000\n",
      " - 2s - loss: 0.4065 - val_loss: 0.4168\n",
      "Epoch 494/1000\n",
      " - 2s - loss: 0.4056 - val_loss: 0.4134\n",
      "Epoch 495/1000\n",
      " - 2s - loss: 0.4058 - val_loss: 0.4154\n",
      "Epoch 496/1000\n",
      " - 2s - loss: 0.4060 - val_loss: 0.4163\n",
      "Epoch 497/1000\n",
      " - 2s - loss: 0.4051 - val_loss: 0.4164\n",
      "Epoch 498/1000\n",
      " - 2s - loss: 0.4054 - val_loss: 0.4164\n",
      "Epoch 499/1000\n",
      " - 2s - loss: 0.4051 - val_loss: 0.4157\n",
      "Epoch 500/1000\n",
      " - 2s - loss: 0.4047 - val_loss: 0.4174\n",
      "Epoch 501/1000\n",
      " - 2s - loss: 0.4056 - val_loss: 0.4165\n",
      "Epoch 502/1000\n",
      " - 2s - loss: 0.4048 - val_loss: 0.4151\n",
      "Epoch 503/1000\n",
      " - 2s - loss: 0.4044 - val_loss: 0.4157\n",
      "Epoch 504/1000\n",
      " - 2s - loss: 0.4042 - val_loss: 0.4156\n",
      "Epoch 505/1000\n",
      " - 2s - loss: 0.4045 - val_loss: 0.4164\n",
      "Epoch 506/1000\n",
      " - 2s - loss: 0.4056 - val_loss: 0.4171\n",
      "Epoch 507/1000\n",
      " - 2s - loss: 0.4048 - val_loss: 0.4158\n",
      "Epoch 508/1000\n",
      " - 2s - loss: 0.4040 - val_loss: 0.4157\n",
      "Epoch 509/1000\n",
      " - 2s - loss: 0.4042 - val_loss: 0.4154\n",
      "Epoch 510/1000\n",
      " - 2s - loss: 0.4038 - val_loss: 0.4145\n",
      "Epoch 511/1000\n",
      " - 2s - loss: 0.4037 - val_loss: 0.4155\n",
      "Epoch 512/1000\n",
      " - 2s - loss: 0.4035 - val_loss: 0.4150\n",
      "Epoch 513/1000\n",
      " - 2s - loss: 0.4034 - val_loss: 0.4160\n",
      "Epoch 514/1000\n",
      " - 2s - loss: 0.4031 - val_loss: 0.4158\n",
      "Epoch 515/1000\n",
      " - 2s - loss: 0.4031 - val_loss: 0.4162\n",
      "Epoch 516/1000\n",
      " - 2s - loss: 0.4032 - val_loss: 0.4158\n",
      "Epoch 517/1000\n",
      " - 2s - loss: 0.4032 - val_loss: 0.4154\n",
      "Epoch 518/1000\n",
      " - 2s - loss: 0.4026 - val_loss: 0.4161\n",
      "Epoch 519/1000\n",
      " - 2s - loss: 0.4026 - val_loss: 0.4157\n",
      "Epoch 520/1000\n",
      " - 2s - loss: 0.4026 - val_loss: 0.4147\n",
      "Epoch 521/1000\n",
      " - 2s - loss: 0.4028 - val_loss: 0.4145\n",
      "Epoch 522/1000\n",
      " - 2s - loss: 0.4024 - val_loss: 0.4147\n",
      "Epoch 523/1000\n",
      " - 2s - loss: 0.4025 - val_loss: 0.4155\n",
      "Epoch 524/1000\n",
      " - 2s - loss: 0.4024 - val_loss: 0.4146\n",
      "Epoch 525/1000\n",
      " - 2s - loss: 0.4021 - val_loss: 0.4134\n",
      "Epoch 526/1000\n",
      " - 2s - loss: 0.4025 - val_loss: 0.4155\n",
      "Epoch 527/1000\n",
      " - 2s - loss: 0.4027 - val_loss: 0.4141\n",
      "Epoch 528/1000\n",
      " - 2s - loss: 0.4023 - val_loss: 0.4124\n",
      "Epoch 529/1000\n",
      " - 2s - loss: 0.4018 - val_loss: 0.4143\n",
      "Epoch 530/1000\n",
      " - 2s - loss: 0.4018 - val_loss: 0.4129\n",
      "Epoch 531/1000\n",
      " - 2s - loss: 0.4016 - val_loss: 0.4125\n",
      "Epoch 532/1000\n",
      " - 2s - loss: 0.4013 - val_loss: 0.4117\n",
      "Epoch 533/1000\n",
      " - 2s - loss: 0.4009 - val_loss: 0.4125\n",
      "Epoch 534/1000\n",
      " - 2s - loss: 0.4021 - val_loss: 0.4130\n",
      "Epoch 535/1000\n",
      " - 2s - loss: 0.4010 - val_loss: 0.4123\n",
      "Epoch 536/1000\n",
      " - 2s - loss: 0.4010 - val_loss: 0.4120\n",
      "Epoch 537/1000\n",
      " - 2s - loss: 0.4006 - val_loss: 0.4118\n",
      "Epoch 538/1000\n",
      " - 2s - loss: 0.4005 - val_loss: 0.4129\n",
      "Epoch 539/1000\n",
      " - 2s - loss: 0.4009 - val_loss: 0.4121\n",
      "Epoch 540/1000\n",
      " - 2s - loss: 0.4003 - val_loss: 0.4120\n",
      "Epoch 541/1000\n",
      " - 2s - loss: 0.4004 - val_loss: 0.4132\n",
      "Epoch 542/1000\n",
      " - 2s - loss: 0.4006 - val_loss: 0.4121\n",
      "Epoch 543/1000\n",
      " - 2s - loss: 0.4002 - val_loss: 0.4129\n",
      "Epoch 544/1000\n",
      " - 2s - loss: 0.3999 - val_loss: 0.4138\n",
      "Epoch 545/1000\n",
      " - 2s - loss: 0.4001 - val_loss: 0.4114\n",
      "Epoch 546/1000\n",
      " - 2s - loss: 0.3997 - val_loss: 0.4124\n",
      "Epoch 547/1000\n",
      " - 2s - loss: 0.3994 - val_loss: 0.4119\n",
      "Epoch 548/1000\n",
      " - 2s - loss: 0.3992 - val_loss: 0.4119\n",
      "Epoch 549/1000\n",
      " - 2s - loss: 0.3992 - val_loss: 0.4112\n",
      "Epoch 550/1000\n",
      " - 2s - loss: 0.4002 - val_loss: 0.4125\n",
      "Epoch 551/1000\n",
      " - 2s - loss: 0.3995 - val_loss: 0.4121\n",
      "Epoch 552/1000\n",
      " - 2s - loss: 0.3993 - val_loss: 0.4132\n",
      "Epoch 553/1000\n",
      " - 2s - loss: 0.3991 - val_loss: 0.4120\n",
      "Epoch 554/1000\n",
      " - 2s - loss: 0.3989 - val_loss: 0.4119\n",
      "Epoch 555/1000\n",
      " - 2s - loss: 0.3991 - val_loss: 0.4122\n",
      "Epoch 556/1000\n",
      " - 2s - loss: 0.3990 - val_loss: 0.4119\n",
      "Epoch 557/1000\n",
      " - 2s - loss: 0.3987 - val_loss: 0.4102\n",
      "Epoch 558/1000\n",
      " - 2s - loss: 0.3982 - val_loss: 0.4104\n",
      "Epoch 559/1000\n",
      " - 2s - loss: 0.3982 - val_loss: 0.4100\n",
      "Epoch 560/1000\n",
      " - 2s - loss: 0.3982 - val_loss: 0.4105\n",
      "Epoch 561/1000\n",
      " - 2s - loss: 0.3983 - val_loss: 0.4102\n",
      "Epoch 562/1000\n",
      " - 2s - loss: 0.3979 - val_loss: 0.4105\n",
      "Epoch 563/1000\n",
      " - 2s - loss: 0.3982 - val_loss: 0.4095\n",
      "Epoch 564/1000\n",
      " - 2s - loss: 0.3979 - val_loss: 0.4106\n",
      "Epoch 565/1000\n",
      " - 2s - loss: 0.3977 - val_loss: 0.4099\n",
      "Epoch 566/1000\n",
      " - 2s - loss: 0.3973 - val_loss: 0.4096\n",
      "Epoch 567/1000\n",
      " - 2s - loss: 0.3982 - val_loss: 0.4114\n",
      "Epoch 568/1000\n",
      " - 2s - loss: 0.3976 - val_loss: 0.4096\n",
      "Epoch 569/1000\n",
      " - 2s - loss: 0.3970 - val_loss: 0.4093\n",
      "Epoch 570/1000\n",
      " - 2s - loss: 0.3972 - val_loss: 0.4095\n",
      "Epoch 571/1000\n",
      " - 2s - loss: 0.3968 - val_loss: 0.4089\n",
      "Epoch 572/1000\n",
      " - 2s - loss: 0.3966 - val_loss: 0.4093\n",
      "Epoch 573/1000\n",
      " - 2s - loss: 0.3967 - val_loss: 0.4092\n",
      "Epoch 574/1000\n",
      " - 2s - loss: 0.3973 - val_loss: 0.4088\n",
      "Epoch 575/1000\n",
      " - 2s - loss: 0.3965 - val_loss: 0.4085\n",
      "Epoch 576/1000\n",
      " - 2s - loss: 0.3964 - val_loss: 0.4087\n",
      "Epoch 577/1000\n",
      " - 2s - loss: 0.3962 - val_loss: 0.4084\n",
      "Epoch 578/1000\n",
      " - 2s - loss: 0.3962 - val_loss: 0.4087\n",
      "Epoch 579/1000\n",
      " - 2s - loss: 0.3961 - val_loss: 0.4084\n",
      "Epoch 580/1000\n",
      " - 2s - loss: 0.3961 - val_loss: 0.4083\n",
      "Epoch 581/1000\n",
      " - 2s - loss: 0.3963 - val_loss: 0.4093\n",
      "Epoch 582/1000\n",
      " - 2s - loss: 0.3975 - val_loss: 0.4085\n",
      "Epoch 583/1000\n",
      " - 2s - loss: 0.3966 - val_loss: 0.4080\n",
      "Epoch 584/1000\n",
      " - 2s - loss: 0.3962 - val_loss: 0.4073\n",
      "Epoch 585/1000\n",
      " - 2s - loss: 0.3956 - val_loss: 0.4072\n",
      "Epoch 586/1000\n",
      " - 2s - loss: 0.3958 - val_loss: 0.4074\n",
      "Epoch 587/1000\n",
      " - 2s - loss: 0.3953 - val_loss: 0.4081\n",
      "Epoch 588/1000\n",
      " - 2s - loss: 0.3955 - val_loss: 0.4062\n",
      "Epoch 589/1000\n",
      " - 2s - loss: 0.3951 - val_loss: 0.4078\n",
      "Epoch 590/1000\n",
      " - 2s - loss: 0.3948 - val_loss: 0.4072\n",
      "Epoch 591/1000\n",
      " - 2s - loss: 0.3948 - val_loss: 0.4061\n",
      "Epoch 592/1000\n",
      " - 2s - loss: 0.3949 - val_loss: 0.4079\n",
      "Epoch 593/1000\n",
      " - 2s - loss: 0.3947 - val_loss: 0.4076\n",
      "Epoch 594/1000\n",
      " - 2s - loss: 0.3948 - val_loss: 0.4078\n",
      "Epoch 595/1000\n",
      " - 2s - loss: 0.3947 - val_loss: 0.4070\n",
      "Epoch 596/1000\n",
      " - 2s - loss: 0.3944 - val_loss: 0.4070\n",
      "Epoch 597/1000\n",
      " - 2s - loss: 0.3947 - val_loss: 0.4066\n",
      "Epoch 598/1000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " - 2s - loss: 0.3944 - val_loss: 0.4069\n",
      "Epoch 599/1000\n",
      " - 2s - loss: 0.3946 - val_loss: 0.4082\n",
      "Epoch 600/1000\n",
      " - 2s - loss: 0.3942 - val_loss: 0.4073\n",
      "Epoch 601/1000\n",
      " - 2s - loss: 0.3944 - val_loss: 0.4069\n",
      "Epoch 602/1000\n",
      " - 2s - loss: 0.3940 - val_loss: 0.4070\n",
      "Epoch 603/1000\n",
      " - 2s - loss: 0.3940 - val_loss: 0.4068\n",
      "Epoch 604/1000\n",
      " - 2s - loss: 0.3938 - val_loss: 0.4068\n",
      "Epoch 605/1000\n",
      " - 2s - loss: 0.3936 - val_loss: 0.4062\n",
      "Epoch 606/1000\n",
      " - 2s - loss: 0.3942 - val_loss: 0.4078\n",
      "Epoch 607/1000\n",
      " - 2s - loss: 0.3943 - val_loss: 0.4066\n",
      "Epoch 608/1000\n",
      " - 2s - loss: 0.3938 - val_loss: 0.4069\n",
      "Epoch 609/1000\n",
      " - 2s - loss: 0.3935 - val_loss: 0.4076\n",
      "Epoch 610/1000\n",
      " - 2s - loss: 0.3946 - val_loss: 0.4045\n",
      "Epoch 611/1000\n",
      " - 2s - loss: 0.3933 - val_loss: 0.4053\n",
      "Epoch 612/1000\n",
      " - 2s - loss: 0.3932 - val_loss: 0.4056\n",
      "Epoch 613/1000\n",
      " - 2s - loss: 0.3930 - val_loss: 0.4055\n",
      "Epoch 614/1000\n",
      " - 2s - loss: 0.3930 - val_loss: 0.4052\n",
      "Epoch 615/1000\n",
      " - 2s - loss: 0.3931 - val_loss: 0.4052\n",
      "Epoch 616/1000\n",
      " - 2s - loss: 0.3938 - val_loss: 0.4038\n",
      "Epoch 617/1000\n",
      " - 2s - loss: 0.3933 - val_loss: 0.4059\n",
      "Epoch 618/1000\n",
      " - 2s - loss: 0.3929 - val_loss: 0.4057\n",
      "Epoch 619/1000\n",
      " - 2s - loss: 0.3926 - val_loss: 0.4054\n",
      "Epoch 620/1000\n",
      " - 2s - loss: 0.3927 - val_loss: 0.4057\n",
      "Epoch 621/1000\n",
      " - 2s - loss: 0.3925 - val_loss: 0.4051\n",
      "Epoch 622/1000\n",
      " - 2s - loss: 0.3926 - val_loss: 0.4057\n",
      "Epoch 623/1000\n",
      " - 2s - loss: 0.3925 - val_loss: 0.4061\n",
      "Epoch 624/1000\n",
      " - 2s - loss: 0.3923 - val_loss: 0.4051\n",
      "Epoch 625/1000\n",
      " - 2s - loss: 0.3921 - val_loss: 0.4047\n",
      "Epoch 626/1000\n",
      " - 2s - loss: 0.3928 - val_loss: 0.4064\n",
      "Epoch 627/1000\n",
      " - 2s - loss: 0.3921 - val_loss: 0.4050\n",
      "Epoch 628/1000\n",
      " - 2s - loss: 0.3919 - val_loss: 0.4058\n",
      "Epoch 629/1000\n",
      " - 2s - loss: 0.3919 - val_loss: 0.4050\n",
      "Epoch 630/1000\n",
      " - 2s - loss: 0.3920 - val_loss: 0.4040\n",
      "Epoch 631/1000\n",
      " - 2s - loss: 0.3922 - val_loss: 0.4048\n",
      "Epoch 632/1000\n",
      " - 2s - loss: 0.3922 - val_loss: 0.4042\n",
      "Epoch 633/1000\n",
      " - 2s - loss: 0.3915 - val_loss: 0.4050\n",
      "Epoch 634/1000\n",
      " - 2s - loss: 0.3917 - val_loss: 0.4052\n",
      "Epoch 635/1000\n",
      " - 2s - loss: 0.3917 - val_loss: 0.4046\n",
      "Epoch 636/1000\n",
      " - 2s - loss: 0.3917 - val_loss: 0.4047\n",
      "Epoch 637/1000\n",
      " - 2s - loss: 0.3916 - val_loss: 0.4048\n",
      "Epoch 638/1000\n",
      " - 2s - loss: 0.3924 - val_loss: 0.4053\n",
      "Epoch 639/1000\n",
      " - 2s - loss: 0.3915 - val_loss: 0.4051\n",
      "Epoch 640/1000\n",
      " - 2s - loss: 0.3912 - val_loss: 0.4046\n",
      "Epoch 641/1000\n",
      " - 2s - loss: 0.3912 - val_loss: 0.4043\n",
      "Epoch 642/1000\n",
      " - 2s - loss: 0.3919 - val_loss: 0.4053\n",
      "Epoch 643/1000\n",
      " - 2s - loss: 0.3911 - val_loss: 0.4044\n",
      "Epoch 644/1000\n",
      " - 2s - loss: 0.3911 - val_loss: 0.4048\n",
      "Epoch 645/1000\n",
      " - 2s - loss: 0.3910 - val_loss: 0.4039\n",
      "Epoch 646/1000\n",
      " - 2s - loss: 0.3909 - val_loss: 0.4048\n",
      "Epoch 647/1000\n",
      " - 2s - loss: 0.3907 - val_loss: 0.4043\n",
      "Epoch 648/1000\n",
      " - 2s - loss: 0.3909 - val_loss: 0.4042\n",
      "Epoch 649/1000\n",
      " - 2s - loss: 0.3907 - val_loss: 0.4061\n",
      "Epoch 650/1000\n",
      " - 2s - loss: 0.3909 - val_loss: 0.4034\n",
      "Epoch 651/1000\n",
      " - 2s - loss: 0.3910 - val_loss: 0.4035\n",
      "Epoch 652/1000\n",
      " - 2s - loss: 0.3912 - val_loss: 0.4030\n",
      "Epoch 653/1000\n",
      " - 2s - loss: 0.3909 - val_loss: 0.4024\n",
      "Epoch 654/1000\n",
      " - 2s - loss: 0.3905 - val_loss: 0.4030\n",
      "Epoch 655/1000\n",
      " - 2s - loss: 0.3902 - val_loss: 0.4034\n",
      "Epoch 656/1000\n",
      " - 2s - loss: 0.3899 - val_loss: 0.4035\n",
      "Epoch 657/1000\n",
      " - 2s - loss: 0.3901 - val_loss: 0.4047\n",
      "Epoch 658/1000\n",
      " - 2s - loss: 0.3901 - val_loss: 0.4040\n",
      "Epoch 659/1000\n",
      " - 2s - loss: 0.3919 - val_loss: 0.4041\n",
      "Epoch 660/1000\n",
      " - 2s - loss: 0.3902 - val_loss: 0.4057\n",
      "Epoch 661/1000\n",
      " - 2s - loss: 0.3903 - val_loss: 0.4044\n",
      "Epoch 662/1000\n",
      " - 2s - loss: 0.3898 - val_loss: 0.4042\n",
      "Epoch 663/1000\n",
      " - 2s - loss: 0.3898 - val_loss: 0.4045\n",
      "Epoch 664/1000\n",
      " - 2s - loss: 0.3897 - val_loss: 0.4046\n",
      "Epoch 665/1000\n",
      " - 2s - loss: 0.3896 - val_loss: 0.4035\n",
      "Epoch 666/1000\n",
      " - 2s - loss: 0.3896 - val_loss: 0.4048\n",
      "Epoch 667/1000\n",
      " - 2s - loss: 0.3896 - val_loss: 0.4037\n",
      "Epoch 668/1000\n",
      " - 2s - loss: 0.3897 - val_loss: 0.4034\n",
      "Epoch 669/1000\n",
      " - 2s - loss: 0.3895 - val_loss: 0.4040\n",
      "Epoch 670/1000\n",
      " - 2s - loss: 0.3896 - val_loss: 0.4035\n",
      "Epoch 671/1000\n",
      " - 2s - loss: 0.3893 - val_loss: 0.4028\n",
      "Epoch 672/1000\n",
      " - 2s - loss: 0.3898 - val_loss: 0.4033\n",
      "Epoch 673/1000\n",
      " - 2s - loss: 0.3894 - val_loss: 0.4040\n",
      "Epoch 674/1000\n",
      " - 2s - loss: 0.3892 - val_loss: 0.4033\n",
      "Epoch 675/1000\n",
      " - 2s - loss: 0.3893 - val_loss: 0.4032\n",
      "Epoch 676/1000\n",
      " - 2s - loss: 0.3892 - val_loss: 0.4030\n",
      "Epoch 677/1000\n",
      " - 2s - loss: 0.3894 - val_loss: 0.4032\n",
      "Epoch 678/1000\n",
      " - 2s - loss: 0.3888 - val_loss: 0.4039\n",
      "Epoch 679/1000\n",
      " - 2s - loss: 0.3890 - val_loss: 0.4043\n",
      "Epoch 680/1000\n",
      " - 2s - loss: 0.3890 - val_loss: 0.4037\n",
      "Epoch 681/1000\n",
      " - 2s - loss: 0.3889 - val_loss: 0.4033\n",
      "Epoch 682/1000\n",
      " - 2s - loss: 0.3893 - val_loss: 0.4033\n",
      "Epoch 683/1000\n",
      " - 2s - loss: 0.3886 - val_loss: 0.4029\n",
      "Epoch 684/1000\n",
      " - 2s - loss: 0.3886 - val_loss: 0.4025\n",
      "Epoch 685/1000\n",
      " - 2s - loss: 0.3887 - val_loss: 0.4040\n",
      "Epoch 686/1000\n",
      " - 2s - loss: 0.3886 - val_loss: 0.4026\n",
      "Epoch 687/1000\n",
      " - 2s - loss: 0.3889 - val_loss: 0.4029\n",
      "Epoch 688/1000\n",
      " - 2s - loss: 0.3887 - val_loss: 0.4042\n",
      "Epoch 689/1000\n",
      " - 2s - loss: 0.3884 - val_loss: 0.4037\n",
      "Epoch 690/1000\n",
      " - 2s - loss: 0.3884 - val_loss: 0.4041\n",
      "Epoch 691/1000\n",
      " - 2s - loss: 0.3891 - val_loss: 0.4011\n",
      "Epoch 692/1000\n",
      " - 2s - loss: 0.3887 - val_loss: 0.4023\n",
      "Epoch 693/1000\n",
      " - 2s - loss: 0.3889 - val_loss: 0.4027\n",
      "Epoch 694/1000\n",
      " - 2s - loss: 0.3889 - val_loss: 0.4011\n",
      "Epoch 695/1000\n",
      " - 2s - loss: 0.3885 - val_loss: 0.4022\n",
      "Epoch 696/1000\n",
      " - 2s - loss: 0.3881 - val_loss: 0.4025\n",
      "Epoch 697/1000\n",
      " - 2s - loss: 0.3883 - val_loss: 0.4023\n",
      "Epoch 698/1000\n",
      " - 2s - loss: 0.3878 - val_loss: 0.4028\n",
      "Epoch 699/1000\n",
      " - 2s - loss: 0.3877 - val_loss: 0.4028\n",
      "Epoch 700/1000\n",
      " - 2s - loss: 0.3880 - val_loss: 0.4025\n",
      "Epoch 701/1000\n",
      " - 2s - loss: 0.3875 - val_loss: 0.4013\n",
      "Epoch 702/1000\n",
      " - 2s - loss: 0.3881 - val_loss: 0.4025\n",
      "Epoch 703/1000\n",
      " - 2s - loss: 0.3875 - val_loss: 0.4022\n",
      "Epoch 704/1000\n",
      " - 2s - loss: 0.3875 - val_loss: 0.4024\n",
      "Epoch 705/1000\n",
      " - 2s - loss: 0.3877 - val_loss: 0.4012\n",
      "Epoch 706/1000\n",
      " - 2s - loss: 0.3876 - val_loss: 0.4021\n",
      "Epoch 707/1000\n",
      " - 2s - loss: 0.3874 - val_loss: 0.4027\n",
      "Epoch 708/1000\n",
      " - 2s - loss: 0.3876 - val_loss: 0.4025\n",
      "Epoch 709/1000\n",
      " - 2s - loss: 0.3873 - val_loss: 0.4018\n",
      "Epoch 710/1000\n",
      " - 2s - loss: 0.3877 - val_loss: 0.4028\n",
      "Epoch 711/1000\n",
      " - 2s - loss: 0.3876 - val_loss: 0.4027\n",
      "Epoch 712/1000\n",
      " - 2s - loss: 0.3875 - val_loss: 0.4021\n",
      "Epoch 713/1000\n",
      " - 2s - loss: 0.3870 - val_loss: 0.4017\n",
      "Epoch 714/1000\n",
      " - 2s - loss: 0.3871 - val_loss: 0.4031\n",
      "Epoch 715/1000\n",
      " - 2s - loss: 0.3874 - val_loss: 0.4021\n",
      "Epoch 716/1000\n",
      " - 2s - loss: 0.3871 - val_loss: 0.4019\n",
      "Epoch 717/1000\n",
      " - 2s - loss: 0.3878 - val_loss: 0.4028\n",
      "Epoch 718/1000\n",
      " - 2s - loss: 0.3875 - val_loss: 0.4019\n",
      "Epoch 719/1000\n",
      " - 2s - loss: 0.3868 - val_loss: 0.4029\n",
      "Epoch 720/1000\n",
      " - 2s - loss: 0.3870 - val_loss: 0.4025\n",
      "Epoch 721/1000\n",
      " - 2s - loss: 0.3871 - val_loss: 0.4025\n",
      "Epoch 722/1000\n",
      " - 2s - loss: 0.3869 - val_loss: 0.4019\n",
      "Epoch 723/1000\n",
      " - 2s - loss: 0.3866 - val_loss: 0.4026\n",
      "Epoch 724/1000\n",
      " - 2s - loss: 0.3866 - val_loss: 0.4021\n",
      "Epoch 725/1000\n",
      " - 2s - loss: 0.3868 - val_loss: 0.4011\n",
      "Epoch 726/1000\n",
      " - 2s - loss: 0.3867 - val_loss: 0.4009\n",
      "Epoch 727/1000\n",
      " - 2s - loss: 0.3864 - val_loss: 0.4008\n",
      "Epoch 728/1000\n",
      " - 2s - loss: 0.3865 - val_loss: 0.4013\n",
      "Epoch 729/1000\n",
      " - 2s - loss: 0.3863 - val_loss: 0.4014\n",
      "Epoch 730/1000\n",
      " - 2s - loss: 0.3861 - val_loss: 0.4012\n",
      "Epoch 731/1000\n",
      " - 2s - loss: 0.3865 - val_loss: 0.4007\n",
      "Epoch 732/1000\n",
      " - 2s - loss: 0.3870 - val_loss: 0.4008\n",
      "Epoch 733/1000\n",
      " - 2s - loss: 0.3863 - val_loss: 0.4012\n",
      "Epoch 734/1000\n",
      " - 2s - loss: 0.3863 - val_loss: 0.4019\n",
      "Epoch 735/1000\n",
      " - 2s - loss: 0.3862 - val_loss: 0.4027\n",
      "Epoch 736/1000\n",
      " - 2s - loss: 0.3861 - val_loss: 0.4017\n",
      "Epoch 737/1000\n",
      " - 2s - loss: 0.3865 - val_loss: 0.4000\n",
      "Epoch 738/1000\n",
      " - 2s - loss: 0.3873 - val_loss: 0.3999\n",
      "Epoch 739/1000\n",
      " - 2s - loss: 0.3865 - val_loss: 0.3992\n",
      "Epoch 740/1000\n",
      " - 2s - loss: 0.3867 - val_loss: 0.3985\n",
      "Epoch 741/1000\n",
      " - 2s - loss: 0.3864 - val_loss: 0.3988\n",
      "Epoch 742/1000\n",
      " - 2s - loss: 0.3862 - val_loss: 0.3986\n",
      "Epoch 743/1000\n",
      " - 2s - loss: 0.3863 - val_loss: 0.3994\n",
      "Epoch 744/1000\n",
      " - 2s - loss: 0.3858 - val_loss: 0.3990\n",
      "Epoch 745/1000\n",
      " - 2s - loss: 0.3857 - val_loss: 0.3998\n",
      "Epoch 746/1000\n",
      " - 2s - loss: 0.3857 - val_loss: 0.3992\n",
      "Epoch 747/1000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " - 2s - loss: 0.3857 - val_loss: 0.3996\n",
      "Epoch 748/1000\n",
      " - 2s - loss: 0.3855 - val_loss: 0.3997\n",
      "Epoch 749/1000\n",
      " - 2s - loss: 0.3854 - val_loss: 0.4007\n",
      "Epoch 750/1000\n",
      " - 2s - loss: 0.3855 - val_loss: 0.3999\n",
      "Epoch 751/1000\n",
      " - 2s - loss: 0.3857 - val_loss: 0.4007\n",
      "Epoch 752/1000\n",
      " - 2s - loss: 0.3857 - val_loss: 0.3996\n",
      "Epoch 753/1000\n",
      " - 2s - loss: 0.3855 - val_loss: 0.4008\n",
      "Epoch 754/1000\n",
      " - 2s - loss: 0.3858 - val_loss: 0.3980\n",
      "Epoch 755/1000\n",
      " - 2s - loss: 0.3864 - val_loss: 0.3975\n",
      "Epoch 756/1000\n",
      " - 2s - loss: 0.3860 - val_loss: 0.3991\n",
      "Epoch 757/1000\n",
      " - 2s - loss: 0.3860 - val_loss: 0.3966\n",
      "Epoch 758/1000\n",
      " - 2s - loss: 0.3853 - val_loss: 0.3976\n",
      "Epoch 759/1000\n",
      " - 2s - loss: 0.3850 - val_loss: 0.3979\n",
      "Epoch 760/1000\n",
      " - 2s - loss: 0.3853 - val_loss: 0.3971\n",
      "Epoch 761/1000\n",
      " - 2s - loss: 0.3851 - val_loss: 0.3986\n",
      "Epoch 762/1000\n",
      " - 2s - loss: 0.3850 - val_loss: 0.3992\n",
      "Epoch 763/1000\n",
      " - 2s - loss: 0.3848 - val_loss: 0.3995\n",
      "Epoch 764/1000\n",
      " - 2s - loss: 0.3845 - val_loss: 0.3977\n",
      "Epoch 765/1000\n",
      " - 2s - loss: 0.3848 - val_loss: 0.3996\n",
      "Epoch 766/1000\n",
      " - 2s - loss: 0.3850 - val_loss: 0.3986\n",
      "Epoch 767/1000\n",
      " - 2s - loss: 0.3850 - val_loss: 0.3996\n",
      "Epoch 768/1000\n",
      " - 2s - loss: 0.3849 - val_loss: 0.3997\n",
      "Epoch 769/1000\n",
      " - 2s - loss: 0.3852 - val_loss: 0.3996\n",
      "Epoch 770/1000\n",
      " - 2s - loss: 0.3850 - val_loss: 0.3997\n",
      "Epoch 771/1000\n",
      " - 2s - loss: 0.3845 - val_loss: 0.3998\n",
      "Epoch 772/1000\n",
      " - 2s - loss: 0.3843 - val_loss: 0.3990\n",
      "Epoch 773/1000\n",
      " - 2s - loss: 0.3854 - val_loss: 0.3964\n",
      "Epoch 774/1000\n",
      " - 2s - loss: 0.3852 - val_loss: 0.3986\n",
      "Epoch 775/1000\n",
      " - 2s - loss: 0.3844 - val_loss: 0.3998\n",
      "Epoch 776/1000\n",
      " - 2s - loss: 0.3843 - val_loss: 0.3997\n",
      "Epoch 777/1000\n",
      " - 2s - loss: 0.3842 - val_loss: 0.3994\n",
      "Epoch 778/1000\n",
      " - 2s - loss: 0.3844 - val_loss: 0.3990\n",
      "Epoch 779/1000\n",
      " - 2s - loss: 0.3843 - val_loss: 0.3992\n",
      "Epoch 780/1000\n",
      " - 2s - loss: 0.3841 - val_loss: 0.3985\n",
      "Epoch 781/1000\n",
      " - 2s - loss: 0.3840 - val_loss: 0.3990\n",
      "Epoch 782/1000\n",
      " - 2s - loss: 0.3841 - val_loss: 0.3993\n",
      "Epoch 783/1000\n",
      " - 2s - loss: 0.3844 - val_loss: 0.3986\n",
      "Epoch 784/1000\n",
      " - 2s - loss: 0.3839 - val_loss: 0.3982\n",
      "Epoch 785/1000\n",
      " - 2s - loss: 0.3839 - val_loss: 0.3982\n",
      "Epoch 786/1000\n",
      " - 2s - loss: 0.3839 - val_loss: 0.3990\n",
      "Epoch 787/1000\n",
      " - 2s - loss: 0.3838 - val_loss: 0.3981\n",
      "Epoch 788/1000\n",
      " - 2s - loss: 0.3840 - val_loss: 0.3980\n",
      "Epoch 789/1000\n",
      " - 2s - loss: 0.3836 - val_loss: 0.3980\n",
      "Epoch 790/1000\n",
      " - 2s - loss: 0.3838 - val_loss: 0.3989\n",
      "Epoch 791/1000\n",
      " - 2s - loss: 0.3837 - val_loss: 0.3995\n",
      "Epoch 792/1000\n",
      " - 2s - loss: 0.3843 - val_loss: 0.3932\n",
      "Epoch 793/1000\n",
      " - 2s - loss: 0.3844 - val_loss: 0.3964\n",
      "Epoch 794/1000\n",
      " - 2s - loss: 0.3841 - val_loss: 0.3961\n",
      "Epoch 795/1000\n",
      " - 2s - loss: 0.3837 - val_loss: 0.3968\n",
      "Epoch 796/1000\n",
      " - 2s - loss: 0.3836 - val_loss: 0.3978\n",
      "Epoch 797/1000\n",
      " - 2s - loss: 0.3836 - val_loss: 0.3965\n",
      "Epoch 798/1000\n",
      " - 2s - loss: 0.3837 - val_loss: 0.3973\n",
      "Epoch 799/1000\n",
      " - 2s - loss: 0.3834 - val_loss: 0.3957\n",
      "Epoch 800/1000\n",
      " - 2s - loss: 0.3840 - val_loss: 0.3964\n",
      "Epoch 801/1000\n",
      " - 2s - loss: 0.3832 - val_loss: 0.3968\n",
      "Epoch 802/1000\n",
      " - 2s - loss: 0.3834 - val_loss: 0.3976\n",
      "Epoch 803/1000\n",
      " - 2s - loss: 0.3834 - val_loss: 0.3973\n",
      "Epoch 804/1000\n",
      " - 2s - loss: 0.3833 - val_loss: 0.3963\n",
      "Epoch 805/1000\n",
      " - 2s - loss: 0.3831 - val_loss: 0.3973\n",
      "Epoch 806/1000\n",
      " - 2s - loss: 0.3828 - val_loss: 0.3956\n",
      "Epoch 807/1000\n",
      " - 2s - loss: 0.3831 - val_loss: 0.3959\n",
      "Epoch 808/1000\n",
      " - 2s - loss: 0.3828 - val_loss: 0.3966\n",
      "Epoch 809/1000\n",
      " - 2s - loss: 0.3832 - val_loss: 0.3969\n",
      "Epoch 810/1000\n",
      " - 2s - loss: 0.3830 - val_loss: 0.3964\n",
      "Epoch 811/1000\n",
      " - 2s - loss: 0.3829 - val_loss: 0.3960\n",
      "Epoch 812/1000\n",
      " - 2s - loss: 0.3826 - val_loss: 0.3956\n",
      "Epoch 813/1000\n",
      " - 2s - loss: 0.3830 - val_loss: 0.3962\n",
      "Epoch 814/1000\n",
      " - 2s - loss: 0.3828 - val_loss: 0.3975\n",
      "Epoch 815/1000\n",
      " - 2s - loss: 0.3827 - val_loss: 0.3968\n",
      "Epoch 816/1000\n",
      " - 2s - loss: 0.3828 - val_loss: 0.3959\n",
      "Epoch 817/1000\n",
      " - 2s - loss: 0.3827 - val_loss: 0.3971\n",
      "Epoch 818/1000\n",
      " - 2s - loss: 0.3827 - val_loss: 0.3974\n",
      "Epoch 819/1000\n",
      " - 2s - loss: 0.3833 - val_loss: 0.3980\n",
      "Epoch 820/1000\n",
      " - 2s - loss: 0.3830 - val_loss: 0.3955\n",
      "Epoch 821/1000\n",
      " - 2s - loss: 0.3825 - val_loss: 0.3967\n",
      "Epoch 822/1000\n",
      " - 2s - loss: 0.3826 - val_loss: 0.3954\n",
      "Epoch 823/1000\n",
      " - 2s - loss: 0.3827 - val_loss: 0.3950\n",
      "Epoch 824/1000\n",
      " - 2s - loss: 0.3830 - val_loss: 0.3958\n",
      "Epoch 825/1000\n",
      " - 2s - loss: 0.3825 - val_loss: 0.3967\n",
      "Epoch 826/1000\n",
      " - 2s - loss: 0.3824 - val_loss: 0.3966\n",
      "Epoch 827/1000\n",
      " - 2s - loss: 0.3824 - val_loss: 0.3968\n",
      "Epoch 828/1000\n",
      " - 2s - loss: 0.3822 - val_loss: 0.3975\n",
      "Epoch 829/1000\n",
      " - 2s - loss: 0.3832 - val_loss: 0.3953\n",
      "Epoch 830/1000\n",
      " - 2s - loss: 0.3825 - val_loss: 0.3957\n",
      "Epoch 831/1000\n",
      " - 2s - loss: 0.3825 - val_loss: 0.3953\n",
      "Epoch 832/1000\n",
      " - 2s - loss: 0.3822 - val_loss: 0.3961\n",
      "Epoch 833/1000\n",
      " - 2s - loss: 0.3824 - val_loss: 0.3958\n",
      "Epoch 834/1000\n",
      " - 2s - loss: 0.3821 - val_loss: 0.3968\n",
      "Epoch 835/1000\n",
      " - 2s - loss: 0.3821 - val_loss: 0.3961\n",
      "Epoch 836/1000\n",
      " - 2s - loss: 0.3821 - val_loss: 0.3968\n",
      "Epoch 837/1000\n",
      " - 2s - loss: 0.3819 - val_loss: 0.3988\n",
      "Epoch 838/1000\n",
      " - 2s - loss: 0.3824 - val_loss: 0.3979\n",
      "Epoch 839/1000\n",
      " - 2s - loss: 0.3819 - val_loss: 0.3959\n",
      "Epoch 840/1000\n",
      " - 2s - loss: 0.3816 - val_loss: 0.3956\n",
      "Epoch 841/1000\n",
      " - 2s - loss: 0.3820 - val_loss: 0.3940\n",
      "Epoch 842/1000\n",
      " - 2s - loss: 0.3820 - val_loss: 0.3951\n",
      "Epoch 843/1000\n",
      " - 2s - loss: 0.3823 - val_loss: 0.3945\n",
      "Epoch 844/1000\n",
      " - 2s - loss: 0.3816 - val_loss: 0.3954\n",
      "Epoch 845/1000\n",
      " - 2s - loss: 0.3815 - val_loss: 0.3948\n",
      "Epoch 846/1000\n",
      " - 2s - loss: 0.3818 - val_loss: 0.3953\n",
      "Epoch 847/1000\n",
      " - 2s - loss: 0.3816 - val_loss: 0.3952\n",
      "Epoch 848/1000\n",
      " - 2s - loss: 0.3812 - val_loss: 0.3959\n",
      "Epoch 849/1000\n",
      " - 2s - loss: 0.3818 - val_loss: 0.3947\n",
      "Epoch 850/1000\n",
      " - 2s - loss: 0.3815 - val_loss: 0.3965\n",
      "Epoch 851/1000\n",
      " - 2s - loss: 0.3813 - val_loss: 0.3957\n",
      "Epoch 852/1000\n",
      " - 2s - loss: 0.3815 - val_loss: 0.3955\n",
      "Epoch 853/1000\n",
      " - 2s - loss: 0.3813 - val_loss: 0.3955\n",
      "Epoch 854/1000\n",
      " - 2s - loss: 0.3814 - val_loss: 0.3953\n",
      "Epoch 855/1000\n",
      " - 2s - loss: 0.3810 - val_loss: 0.3962\n",
      "Epoch 856/1000\n",
      " - 2s - loss: 0.3816 - val_loss: 0.3954\n",
      "Epoch 857/1000\n",
      " - 2s - loss: 0.3811 - val_loss: 0.3948\n",
      "Epoch 858/1000\n",
      " - 2s - loss: 0.3815 - val_loss: 0.3923\n",
      "Epoch 859/1000\n",
      " - 2s - loss: 0.3811 - val_loss: 0.3953\n",
      "Epoch 860/1000\n",
      " - 2s - loss: 0.3810 - val_loss: 0.3938\n",
      "Epoch 861/1000\n",
      " - 2s - loss: 0.3815 - val_loss: 0.3944\n",
      "Epoch 862/1000\n",
      " - 2s - loss: 0.3811 - val_loss: 0.3954\n",
      "Epoch 863/1000\n",
      " - 2s - loss: 0.3810 - val_loss: 0.3947\n",
      "Epoch 864/1000\n",
      " - 2s - loss: 0.3805 - val_loss: 0.3945\n",
      "Epoch 865/1000\n",
      " - 2s - loss: 0.3813 - val_loss: 0.3943\n",
      "Epoch 866/1000\n",
      " - 2s - loss: 0.3809 - val_loss: 0.3946\n",
      "Epoch 867/1000\n",
      " - 2s - loss: 0.3810 - val_loss: 0.3951\n",
      "Epoch 868/1000\n",
      " - 2s - loss: 0.3807 - val_loss: 0.3944\n",
      "Epoch 869/1000\n",
      " - 2s - loss: 0.3807 - val_loss: 0.3946\n",
      "Epoch 870/1000\n",
      " - 2s - loss: 0.3805 - val_loss: 0.3945\n",
      "Epoch 871/1000\n",
      " - 2s - loss: 0.3805 - val_loss: 0.3953\n",
      "Epoch 872/1000\n",
      " - 2s - loss: 0.3809 - val_loss: 0.3944\n",
      "Epoch 873/1000\n",
      " - 2s - loss: 0.3808 - val_loss: 0.3943\n",
      "Epoch 874/1000\n",
      " - 2s - loss: 0.3804 - val_loss: 0.3943\n",
      "Epoch 875/1000\n",
      " - 2s - loss: 0.3803 - val_loss: 0.3940\n",
      "Epoch 876/1000\n",
      " - 2s - loss: 0.3807 - val_loss: 0.3940\n",
      "Epoch 877/1000\n",
      " - 2s - loss: 0.3806 - val_loss: 0.3941\n",
      "Epoch 878/1000\n",
      " - 2s - loss: 0.3804 - val_loss: 0.3946\n",
      "Epoch 879/1000\n",
      " - 2s - loss: 0.3803 - val_loss: 0.3944\n",
      "Epoch 880/1000\n",
      " - 2s - loss: 0.3802 - val_loss: 0.3943\n",
      "Epoch 881/1000\n",
      " - 2s - loss: 0.3799 - val_loss: 0.3941\n",
      "Epoch 882/1000\n",
      " - 2s - loss: 0.3816 - val_loss: 0.3945\n",
      "Epoch 883/1000\n",
      " - 2s - loss: 0.3805 - val_loss: 0.3950\n",
      "Epoch 884/1000\n",
      " - 2s - loss: 0.3803 - val_loss: 0.3943\n",
      "Epoch 885/1000\n",
      " - 2s - loss: 0.3800 - val_loss: 0.3941\n",
      "Epoch 886/1000\n",
      " - 2s - loss: 0.3800 - val_loss: 0.3951\n",
      "Epoch 887/1000\n",
      " - 2s - loss: 0.3802 - val_loss: 0.3947\n",
      "Epoch 888/1000\n",
      " - 2s - loss: 0.3800 - val_loss: 0.3941\n",
      "Epoch 889/1000\n",
      " - 2s - loss: 0.3801 - val_loss: 0.3955\n",
      "Epoch 890/1000\n",
      " - 2s - loss: 0.3808 - val_loss: 0.3961\n",
      "Epoch 891/1000\n",
      " - 2s - loss: 0.3802 - val_loss: 0.3949\n",
      "Epoch 892/1000\n",
      " - 2s - loss: 0.3796 - val_loss: 0.3948\n",
      "Epoch 893/1000\n",
      " - 2s - loss: 0.3797 - val_loss: 0.3948\n",
      "Epoch 894/1000\n",
      " - 2s - loss: 0.3803 - val_loss: 0.3937\n",
      "Epoch 895/1000\n",
      " - 2s - loss: 0.3796 - val_loss: 0.3940\n",
      "Epoch 896/1000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " - 2s - loss: 0.3798 - val_loss: 0.3940\n",
      "Epoch 897/1000\n",
      " - 2s - loss: 0.3795 - val_loss: 0.3939\n",
      "Epoch 898/1000\n",
      " - 2s - loss: 0.3794 - val_loss: 0.3944\n",
      "Epoch 899/1000\n",
      " - 2s - loss: 0.3799 - val_loss: 0.3935\n",
      "Epoch 900/1000\n",
      " - 2s - loss: 0.3794 - val_loss: 0.3936\n",
      "Epoch 901/1000\n",
      " - 2s - loss: 0.3794 - val_loss: 0.3934\n",
      "Epoch 902/1000\n",
      " - 2s - loss: 0.3792 - val_loss: 0.3936\n",
      "Epoch 903/1000\n",
      " - 2s - loss: 0.3791 - val_loss: 0.3931\n",
      "Epoch 904/1000\n",
      " - 2s - loss: 0.3794 - val_loss: 0.3936\n",
      "Epoch 905/1000\n",
      " - 2s - loss: 0.3793 - val_loss: 0.3939\n",
      "Epoch 906/1000\n",
      " - 2s - loss: 0.3800 - val_loss: 0.3923\n",
      "Epoch 907/1000\n",
      " - 2s - loss: 0.3793 - val_loss: 0.3920\n",
      "Epoch 908/1000\n",
      " - 2s - loss: 0.3792 - val_loss: 0.3926\n",
      "Epoch 909/1000\n",
      " - 2s - loss: 0.3791 - val_loss: 0.3919\n",
      "Epoch 910/1000\n",
      " - 2s - loss: 0.3788 - val_loss: 0.3926\n",
      "Epoch 911/1000\n",
      " - 2s - loss: 0.3789 - val_loss: 0.3927\n",
      "Epoch 912/1000\n",
      " - 2s - loss: 0.3789 - val_loss: 0.3932\n",
      "Epoch 913/1000\n",
      " - 2s - loss: 0.3788 - val_loss: 0.3932\n",
      "Epoch 914/1000\n",
      " - 2s - loss: 0.3787 - val_loss: 0.3932\n",
      "Epoch 915/1000\n",
      " - 2s - loss: 0.3787 - val_loss: 0.3928\n",
      "Epoch 916/1000\n",
      " - 2s - loss: 0.3794 - val_loss: 0.3928\n",
      "Epoch 917/1000\n",
      " - 2s - loss: 0.3787 - val_loss: 0.3931\n",
      "Epoch 918/1000\n",
      " - 2s - loss: 0.3788 - val_loss: 0.3934\n",
      "Epoch 919/1000\n",
      " - 2s - loss: 0.3790 - val_loss: 0.3900\n",
      "Epoch 920/1000\n",
      " - 2s - loss: 0.3789 - val_loss: 0.3923\n",
      "Epoch 921/1000\n",
      " - 2s - loss: 0.3786 - val_loss: 0.3929\n",
      "Epoch 922/1000\n",
      " - 2s - loss: 0.3785 - val_loss: 0.3919\n",
      "Epoch 923/1000\n",
      " - 2s - loss: 0.3786 - val_loss: 0.3921\n",
      "Epoch 924/1000\n",
      " - 2s - loss: 0.3784 - val_loss: 0.3922\n",
      "Epoch 925/1000\n",
      " - 2s - loss: 0.3785 - val_loss: 0.3919\n",
      "Epoch 926/1000\n",
      " - 2s - loss: 0.3786 - val_loss: 0.3929\n",
      "Epoch 927/1000\n",
      " - 2s - loss: 0.3783 - val_loss: 0.3921\n",
      "Epoch 928/1000\n",
      " - 2s - loss: 0.3784 - val_loss: 0.3919\n",
      "Epoch 929/1000\n",
      " - 2s - loss: 0.3782 - val_loss: 0.3924\n",
      "Epoch 930/1000\n",
      " - 2s - loss: 0.3789 - val_loss: 0.3923\n",
      "Epoch 931/1000\n",
      " - 2s - loss: 0.3782 - val_loss: 0.3931\n",
      "Epoch 932/1000\n",
      " - 2s - loss: 0.3783 - val_loss: 0.3926\n",
      "Epoch 933/1000\n",
      " - 2s - loss: 0.3781 - val_loss: 0.3915\n",
      "Epoch 934/1000\n",
      " - 2s - loss: 0.3779 - val_loss: 0.3933\n",
      "Epoch 935/1000\n",
      " - 2s - loss: 0.3781 - val_loss: 0.3931\n",
      "Epoch 936/1000\n",
      " - 2s - loss: 0.3780 - val_loss: 0.3931\n",
      "Epoch 937/1000\n",
      " - 2s - loss: 0.3780 - val_loss: 0.3928\n",
      "Epoch 938/1000\n",
      " - 2s - loss: 0.3783 - val_loss: 0.3932\n",
      "Epoch 939/1000\n",
      " - 2s - loss: 0.3780 - val_loss: 0.3926\n",
      "Epoch 940/1000\n",
      " - 2s - loss: 0.3778 - val_loss: 0.3923\n",
      "Epoch 941/1000\n",
      " - 2s - loss: 0.3777 - val_loss: 0.3925\n",
      "Epoch 942/1000\n",
      " - 2s - loss: 0.3778 - val_loss: 0.3923\n",
      "Epoch 943/1000\n",
      " - 2s - loss: 0.3777 - val_loss: 0.3914\n",
      "Epoch 944/1000\n",
      " - 2s - loss: 0.3778 - val_loss: 0.3933\n",
      "Epoch 945/1000\n",
      " - 2s - loss: 0.3781 - val_loss: 0.3923\n",
      "Epoch 946/1000\n",
      " - 2s - loss: 0.3777 - val_loss: 0.3941\n",
      "Epoch 947/1000\n",
      " - 2s - loss: 0.3777 - val_loss: 0.3925\n",
      "Epoch 948/1000\n",
      " - 2s - loss: 0.3778 - val_loss: 0.3932\n",
      "Epoch 949/1000\n",
      " - 2s - loss: 0.3784 - val_loss: 0.3925\n",
      "Epoch 950/1000\n",
      " - 2s - loss: 0.3776 - val_loss: 0.3933\n",
      "Epoch 951/1000\n",
      " - 2s - loss: 0.3776 - val_loss: 0.3923\n",
      "Epoch 952/1000\n",
      " - 2s - loss: 0.3777 - val_loss: 0.3917\n",
      "Epoch 953/1000\n",
      " - 2s - loss: 0.3773 - val_loss: 0.3921\n",
      "Epoch 954/1000\n",
      " - 2s - loss: 0.3771 - val_loss: 0.3927\n",
      "Epoch 955/1000\n",
      " - 2s - loss: 0.3774 - val_loss: 0.3929\n",
      "Epoch 956/1000\n",
      " - 2s - loss: 0.3776 - val_loss: 0.3910\n",
      "Epoch 957/1000\n",
      " - 2s - loss: 0.3774 - val_loss: 0.3909\n",
      "Epoch 958/1000\n",
      " - 2s - loss: 0.3770 - val_loss: 0.3915\n",
      "Epoch 959/1000\n",
      " - 2s - loss: 0.3772 - val_loss: 0.3911\n",
      "Epoch 960/1000\n",
      " - 2s - loss: 0.3769 - val_loss: 0.3913\n",
      "Epoch 961/1000\n",
      " - 2s - loss: 0.3772 - val_loss: 0.3929\n",
      "Epoch 962/1000\n",
      " - 2s - loss: 0.3773 - val_loss: 0.3918\n",
      "Epoch 963/1000\n",
      " - 2s - loss: 0.3775 - val_loss: 0.3931\n",
      "Epoch 964/1000\n",
      " - 2s - loss: 0.3774 - val_loss: 0.3910\n",
      "Epoch 965/1000\n",
      " - 2s - loss: 0.3771 - val_loss: 0.3911\n",
      "Epoch 966/1000\n",
      " - 2s - loss: 0.3771 - val_loss: 0.3927\n",
      "Epoch 967/1000\n",
      " - 2s - loss: 0.3770 - val_loss: 0.3918\n",
      "Epoch 968/1000\n",
      " - 2s - loss: 0.3768 - val_loss: 0.3925\n",
      "Epoch 969/1000\n",
      " - 2s - loss: 0.3770 - val_loss: 0.3932\n",
      "Epoch 970/1000\n",
      " - 2s - loss: 0.3770 - val_loss: 0.3924\n",
      "Epoch 971/1000\n",
      " - 2s - loss: 0.3768 - val_loss: 0.3923\n",
      "Epoch 972/1000\n",
      " - 2s - loss: 0.3769 - val_loss: 0.3927\n",
      "Epoch 973/1000\n",
      " - 2s - loss: 0.3769 - val_loss: 0.3925\n",
      "Epoch 974/1000\n",
      " - 2s - loss: 0.3768 - val_loss: 0.3922\n",
      "Epoch 975/1000\n",
      " - 2s - loss: 0.3770 - val_loss: 0.3920\n",
      "Epoch 976/1000\n",
      " - 2s - loss: 0.3770 - val_loss: 0.3914\n",
      "Epoch 977/1000\n",
      " - 2s - loss: 0.3767 - val_loss: 0.3916\n",
      "Epoch 978/1000\n",
      " - 2s - loss: 0.3767 - val_loss: 0.3916\n",
      "Epoch 979/1000\n",
      " - 2s - loss: 0.3766 - val_loss: 0.3914\n",
      "Epoch 980/1000\n",
      " - 2s - loss: 0.3767 - val_loss: 0.3925\n",
      "Epoch 981/1000\n",
      " - 2s - loss: 0.3763 - val_loss: 0.3923\n",
      "Epoch 982/1000\n",
      " - 2s - loss: 0.3764 - val_loss: 0.3926\n",
      "Epoch 983/1000\n",
      " - 2s - loss: 0.3767 - val_loss: 0.3924\n",
      "Epoch 984/1000\n",
      " - 2s - loss: 0.3768 - val_loss: 0.3930\n",
      "Epoch 985/1000\n",
      " - 2s - loss: 0.3763 - val_loss: 0.3924\n",
      "Epoch 986/1000\n",
      " - 2s - loss: 0.3762 - val_loss: 0.3919\n",
      "Epoch 987/1000\n",
      " - 2s - loss: 0.3762 - val_loss: 0.3920\n",
      "Epoch 988/1000\n",
      " - 2s - loss: 0.3765 - val_loss: 0.3926\n",
      "Epoch 989/1000\n",
      " - 2s - loss: 0.3765 - val_loss: 0.3920\n",
      "Epoch 990/1000\n",
      " - 2s - loss: 0.3764 - val_loss: 0.3917\n",
      "Epoch 991/1000\n",
      " - 2s - loss: 0.3762 - val_loss: 0.3918\n",
      "Epoch 992/1000\n",
      " - 2s - loss: 0.3763 - val_loss: 0.3918\n",
      "Epoch 993/1000\n",
      " - 2s - loss: 0.3763 - val_loss: 0.3914\n",
      "Epoch 994/1000\n",
      " - 2s - loss: 0.3759 - val_loss: 0.3911\n",
      "Epoch 995/1000\n",
      " - 2s - loss: 0.3759 - val_loss: 0.3915\n",
      "Epoch 996/1000\n",
      " - 2s - loss: 0.3760 - val_loss: 0.3917\n",
      "Epoch 997/1000\n",
      " - 2s - loss: 0.3764 - val_loss: 0.3921\n",
      "Epoch 998/1000\n",
      " - 2s - loss: 0.3759 - val_loss: 0.3912\n",
      "Epoch 999/1000\n",
      " - 2s - loss: 0.3762 - val_loss: 0.3910\n",
      "Epoch 1000/1000\n",
      " - 2s - loss: 0.3762 - val_loss: 0.3914\n"
     ]
    }
   ],
   "source": [
    "adam = optimizers.Adam(lr)\n",
    "lstm_autoencoder.compile(loss='mse', optimizer=adam)\n",
    "\n",
    "cp = ModelCheckpoint(filepath=\"lstm_autoencoder_classifier.h5\",\n",
    "                               save_best_only=True,\n",
    "                               verbose=0)\n",
    "\n",
    "tb = TensorBoard(log_dir='./logs',\n",
    "                histogram_freq=0,\n",
    "                write_graph=True,\n",
    "                write_images=True)\n",
    "\n",
    "lstm_autoencoder_history = lstm_autoencoder.fit(X_train_y0_scaled, X_train_y0_scaled, \n",
    "                                                epochs=epochs, \n",
    "                                                batch_size=batch, \n",
    "                                                validation_data=(X_valid_y0_scaled, X_valid_y0_scaled),\n",
    "                                                verbose=2).history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAGDCAYAAAAyM4nNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XmcnWV9///X5yyz71u2STLZFyCEJAQQBEREdlyokoqKVqm2aqu1Lbb+3L610tpFaq2WWnCpQqk7loqiIKJACFvISvZkMjOZfZ856/X74zpDhskkmWTm5MyZeT8fj3nMnOu+z30+5zwC73Nd93VftznnEBERkewXyHQBIiIiMjEU6iIiIlOEQl1ERGSKUKiLiIhMEQp1ERGRKUKhLiIiMkUo1EUEM6szM2dmoTHse5uZPTHe44jIxFOoi2QZM9tvZlEzqxrR/kIqUOsyU5mIZJpCXSQ77QM2DD0ws3OA/MyVIyKTgUJdJDt9G3jXsMfvBr41fAczKzWzb5lZi5kdMLNPmlkgtS1oZv9gZq1mthe4bpTn/qeZNZrZYTP7GzMLnmqRZjbbzH5iZu1mttvM3j9s23oz22Rm3WZ2xMz+KdWeZ2b/ZWZtZtZpZs+Y2YxTfW2R6UihLpKdngJKzGxFKmzfDvzXiH2+DJQCC4HL8F8C3pPa9n7geuA8YB1w84jnfhOIA4tT+1wFvO806rwPqAdmp17jb83s9altdwF3OedKgEXAA6n2d6fqngtUAh8ABk7jtUWmHYW6SPYa6q2/AdgBHB7aMCzoP+Gc63HO7Qf+EXhnape3AV9yzh1yzrUDXxj23BnANcCfOuf6nHPNwD8Dt5xKcWY2F7gE+Evn3KBz7gXg68NqiAGLzazKOdfrnHtqWHslsNg5l3DOPeuc6z6V1xaZrhTqItnr28DvA7cxYugdqAJygAPD2g4Ac1J/zwYOjdg2ZD4QBhpTw9+dwL8DNadY32yg3TnXc5wa/gBYCuxIDbFfP+x9PQzcb2YNZvb3ZhY+xdcWmZYU6iJZyjl3AD9h7lrgByM2t+J7vPOHtc3jaG++ET+8PXzbkENABKhyzpWlfkqcc2edYokNQIWZFY9Wg3Nul3NuA/7Lwt8B3zOzQudczDn3WefcSuA1+NME70JETkqhLpLd/gC4wjnXN7zROZfAn6P+vJkVm9l84GMcPe/+APARM6s1s3LgjmHPbQR+DvyjmZWYWcDMFpnZZadSmHPuEPA74AupyW+rUvV+B8DMbjWzaudcEuhMPS1hZq8zs3NSpxC68V9OEqfy2iLTlUJdJIs55/Y45zYdZ/OHgT5gL/AE8F3gntS2/8APcb8IPMexPf134YfvtwEdwPeAWadR4gagDt9r/yHwaefcL1Lbrga2mlkvftLcLc65QWBm6vW6ge3Arzl2EqCIjMKcc5muQURERCaAeuoiIiJThEJdRERkilCoi4iITBEKdRERkSlCoS4iIjJFZN09j6uqqlxdXV2myxARETkjnn322VbnXPVY9s26UK+rq2PTpuNdlisiIjK1mNmBk+/lafhdRERkilCoi4iITBEKdRERkSki686pi4jI9BCLxaivr2dwcDDTpZwReXl51NbWEg6f/p2GFeoiIjIp1dfXU1xcTF1dHWaW6XLSyjlHW1sb9fX1LFiw4LSPo+F3ERGZlAYHB6msrJzygQ5gZlRWVo57VEKhLiIik9Z0CPQhE/FeFeoiIiKjaGtrY/Xq1axevZqZM2cyZ86cVx5Ho9ExHeM973kPO3fuTHOlR+mcuoiIyCgqKyt54YUXAPjMZz5DUVERH//4x1+1j3MO5xyBwOh95HvvvTftdQ6nnrqIiMgp2L17N2effTYf+MAHWLNmDY2Njdx+++2sW7eOs846i8997nOv7HvJJZfwwgsvEI/HKSsr44477uDcc8/loosuorm5ecJrU09dREQmvbo7/jctx91/53Wn9bxt27Zx77338rWvfQ2AO++8k4qKCuLxOK973eu4+eabWbly5aue09XVxWWXXcadd97Jxz72Me655x7uuOOOcb+H4aZ3T71pC9y3ASI9ma5ERESyyKJFizj//PNfeXzfffexZs0a1qxZw/bt29m2bdsxz8nPz+eaa64BYO3atezfv3/C65q+PXXn4ME/gcOb4Fefh2vuzHRFIiJyHKfbo06XwsLCV/7etWsXd911Fxs3bqSsrIxbb7111EvTcnJyXvk7GAwSj8cnvK7p21M3g0v/3P9d/0xmaxERkazV3d1NcXExJSUlNDY28vDDD2eslunbUwcoqvG/kxP/bUlERKaHNWvWsHLlSs4++2wWLlzIxRdfnLFazDmXsRc/HevWrXMTdj/1xs3w76+FGefAB5+YmGOKiMiE2L59OytWrMh0GWfUaO/ZzJ51zq0by/On7/A7QCA1UKGeuoiITAEKdYBkLLN1iIiITIDpHepB9dRFRGTqmN6h/kpPPZHZOkRERCaAQh3UUxcRkSlBoQ6Q0Dl1ERHJfmkLdTO7x8yazWzLcbabmf2Lme02s81mtiZdtRyXeuoiInIcl19++TELyXzpS1/ij/7oj477nKKiIgAaGhq4+eabj3vcCbs0e4R09tS/AVx9gu3XAEtSP7cDX01jLaPTOXURETmODRs2cP/997+q7f7772fDhg0nfe7s2bP53ve+l67Sjittoe6cexxoP8EuNwHfct5TQJmZzUpXPaNST11ERI7j5ptv5qc//SmRSASA/fv309DQwOrVq3n961/PmjVrOOecc/jxj398zHP379/P2WefDcDAwAC33HILq1at4u1vfzsDAwNpqzmTy8TOAQ4Ne1yfamscuaOZ3Y7vzTNv3rwJK6AzkqQMdJ26iMhk95nSNB2367ibKisrWb9+PT/72c+46aabuP/++3n7299Ofn4+P/zhDykpKaG1tZULL7yQG2+8ETMb9Thf/epXKSgoYPPmzWzevJk1a9J3tjmTE+VGe/ejrlnrnLvbObfOObeuurp6wgr49ycO+j+ScX/XNhERkWGGD8EPDb075/irv/orVq1axZVXXsnhw4c5cuTIcY/x+OOPc+uttwKwatUqVq1albZ6M9lTrwfmDntcCzScyQJ+t6edhDOC5sAlwYJn8uVFRGSsTtCjTqc3velNfOxjH+O5555jYGCANWvW8I1vfIOWlhaeffZZwuEwdXV1o95qdbjj9eInWiZ76j8B3pWaBX8h0OWcO2boPZ3ufOsq4qSCXOfVRURkhKKiIi6//HLe+973vjJBrquri5qaGsLhMI8++igHDhw44TEuvfRSvvOd7wCwZcsWNm/enLZ609ZTN7P7gMuBKjOrBz4NhAGcc18DHgKuBXYD/cB70lXL8eSHgyQIAnF/rXoo90yXICIik9yGDRt4y1ve8sow/Dve8Q5uuOEG1q1bx+rVq1m+fPkJn//BD36Q97znPaxatYrVq1ezfv36tNWatlB3zp1wzr/z93z943S9/ljkhYPEhwYr1FMXEZFRvPnNb2b4bcqrqqp48sknR923t7cXgLq6OrZs8cu05OfnH3NpXLpM6xXl8sPBYcPvulZdRESy27QO9dxwIDX8jnrqIiKS9aZ3qIcCr/TUE/FohqsREREZn2kd6mZGMhXqkahCXURksnHTaA2RiXiv0zrUARKpa9MHFeoiIpNKXl4ebW1t0yLYnXO0tbWRl5c3ruNkcvGZSSFpQXC8sraviIhMDrW1tdTX19PS0pLpUs6IvLw8amtrx3UMhbqFwEFUoS4iMqmEw2EWLFiQ6TKyyrQffo9b2P+OKdRFRCS7TftQT5gfrEjGFeoiIpLdpn2oxy0HABfTRDkREclu0z7Uh3rqLqFQFxGR7DbtQ33onHoyduLb5omIiEx20z7Uk4HU8LvOqYuISJab9qGeSPXUNfwuIiLZTqEe8KGOeuoiIpLlFOqp4Xd0QxcREcly0z7UkwENv4uIyNSgUB/qqSc0/C4iItlNoa7hdxERmSKmfai7oYlyGn4XEZEsN+1DPRn0PXVTqIuISJab9qHuAgp1ERGZGqZ9qCdDuQAENFFORESynEI9mA9AID6Q4UpERETGZ9qHejzkQz2YUKiLiEh2m/ah7lKhHlKoi4hIlpv2oZ4MFQAQTOjWqyIikt0U6mHfUw+rpy4iIllu2oe6ht9FRGSqmPahTrjQ/0pq+F1ERLLbtA91F/bn1MNJ9dRFRCS7TftQD+YOhbp66iIikt2mfajn5OaTcEbIxSERy3Q5IiIip23ah3peToh+8vyDWH9mixERERkHhXo4yCCpe6rHdF5dRESyl0I9FKDf+Zu6EO3LbDEiIiLjkNZQN7OrzWynme02sztG2T7fzH5pZpvN7DEzq01nPaPJCwfpJxXqGn4XEZEslrZQN7Mg8BXgGmAlsMHMVo7Y7R+AbznnVgGfA76QrnqOxw+/D4W6ht9FRCR7pbOnvh7Y7Zzb65yLAvcDN43YZyXwy9Tfj46yPe3ywhp+FxGRqSGdoT4HODTscX2qbbgXgbem/n4zUGxmlWms6Rj5Gn4XEZEpIp2hbqO0uRGPPw5cZmbPA5cBh4H4MQcyu93MNpnZppaWlgktMlez30VEZIpIZ6jXA3OHPa4FGobv4JxrcM69xTl3HvDXqbaukQdyzt3tnFvnnFtXXV09oUX64ffUderR3gk9toiIyJmUzlB/BlhiZgvMLAe4BfjJ8B3MrMrMhmr4BHBPGusZVU4wQDf+pi7JgWO+T4iIiGSNtIW6cy4OfAh4GNgOPOCc22pmnzOzG1O7XQ7sNLOXgRnA59NVz/GYGb2BYgDifW1n+uVFREQmTCidB3fOPQQ8NKLtU8P+/h7wvXTWMBb9wWJwkOxrz3QpIiIip23arygH0B8sBcD1K9RFRCR7KdSBwVCJ/2OgI7OFiIiIjINCHYiEfU+dgc7MFiIiIjIOCnUgmuN76sGIQl1ERLKXQh2IhcsACA5q+F1ERLKXQh0gp5CYCxJMDEBsMNPViIiInBaFOpCXE6KTIv9gUEPwIiKSnRTq+NuvdrpUqGsGvIiIZCmFOpCfE6Qdv6ocvc2ZLUZEROQ0KdSB4rwQzc5PlqP3SGaLEREROU0KdaAkL8wRV+4f9DRlthgREZHTpFBnRE9doS4iIllKoc7InnpjZosRERE5TQp1Uj11UqGuc+oiIpKlFOpAcV542PC7euoiIpKdFOr4nvqrJso5l9mCREREToNCHSjJD9NLPv3kQawfIj2ZLklEROSUKdTxPXUwWjQDXkREsphCHSjKCWEGTTqvLiIiWUyhDgQCRlHOsPPqmgEvIiJZSKGeUpKvGfAiIpLdFOopx8yAFxERyTIK9ZTyghytKiciIllNoZ5SU5JLC0PD7zqnLiIi2UehnlJdlKueuoiIZDWFekpNSa5WlRMRkaymUE+pLs6lj3wilgfxAYh0Z7okERGRU6JQT6kpzgOgLVDpGzQDXkREsoxCPaWmOBeAZp1XFxGRLKVQT6lOhXpDosQ3aAa8iIhkGYV6Sml+mJxggMMJrSonIiLZSaGeYmZUF+dqVTkREclaCvVhqotzj55T7z6c2WJEREROkUJ9mJriXOpdlX/QeTCzxYiIiJwihfow1cW5HHI1/kHngcwWIyIicooU6sPUFOfRQikxy4WBDhjUAjQiIpI9FOrD1JTkAkZ7zgzfoCF4ERHJImkNdTO72sx2mtluM7tjlO3zzOxRM3vezDab2bXprOdkqov8tepNNhTqGoIXEZHskbZQN7Mg8BXgGmAlsMHMVo7Y7ZPAA86584BbgH9LVz1j4XvqcMhV+4aO/ZkrRkRE5BSls6e+HtjtnNvrnIsC9wM3jdjHAakl3CgFGtJYz0nNKPHrv++IpXrqrS9nsBoREZFTk85QnwMcGva4PtU23GeAW82sHngI+PBoBzKz281sk5ltamlpSUetAFQW5hAweH5wpm9o2Zm21xIREZlo6Qx1G6Vt5E3KNwDfcM7VAtcC3zazY2pyzt3tnFvnnFtXXV2dhlK9UDBAdXEuu5K1vqF5u+6rLiIiWSOdoV4PzB32uJZjh9f/AHgAwDn3JJAHVKWxppOaWZJHM2XEc4phsBN6mzNZjoiIyJilM9SfAZaY2QIzy8FPhPvJiH0OAq8HMLMV+FBP3/j6GPjz6kZ30WLf0Lw1k+WIiIiMWdpC3TkXBz4EPAxsx89y32pmnzOzG1O7/RnwfjN7EbgPuM25zI53zyz1k+UaC5f7hsPPZbAaERGRsQul8+DOuYfwE+CGt31q2N/bgIvTWcOpGpoBvzu8jLMADj+b0XpERETGSivKjTAzFeovutTwe/0zmiwnIiJZQaE+wtDw+5aBSsivgL4WLRcrIiJZQaE+wtDw+5GeCNSu842HN2WwIhERkbFRqI8w1FNv6hrEzUmF+qGNGaxIRERkbBTqIxTlhijODRGJJ+muSYX6gd9ltigREZExUKiPYm5FAQD78lZAIAxHtuje6iIiMukp1EdRV+VD/UC3g9mrwSU1BC8iIpOeQn0U8yoKAdjf2g/zLvKNBzUELyIik5tCfRTzK1M99fY+mP8a33jgyQxWJCIicnIK9VEMhfrBtn6Ye4FvPLwJYoMZrEpEROTEFOqjWFDlh9/3tvbh8suhZiUkotCgdeBFRGTyUqiPYmZJHiV5Idr7orT0Ro6eV9elbSIiMokp1EdhZiyfWQLAjsaeo+fVD+q8uoiITF4K9eNYPqsYgB1N3Ud76oc2QjKRwapERESOT6F+HEM99W0N3VA6B8rmQaTbL0QjIiIyCSnUj+Os2T7UNx/u8g3zdGmbiIhMbgr141gxq4ScUIC9LX109ceGnVfXZDkREZmcFOrHkRMKvNJbf7G+89WL0DiXwcpERERGp1A/gdVzywB44VAnVC6Gwmroa4a2PRmuTERE5FgK9RMYCvUXD3WCGcy70G/QELyIiExCCvUTOG9uOeB76s45TZYTEZFJTaF+AnMr8qkozKGtL0p9x8DRnvqhpzJbmIiIyCgU6idgZpxbWwrA84c6YeYqCBdC+17oOZLh6kRERF5NoX4Sq1ND8C8e6oRgCGrX+Q3qrYuIyCSjUD+J1fOGzYCHo0vGHnw6QxWJiIiMTqF+EqtrfahvOdxFLJGEean7q9dvzGBVIiIix1Kon0RpQZiFVYVE4kl/x7YZ5/gNzTu0CI2IiEwqCvUxOLoITQcUVUNBFUR7oKs+w5WJiIgcpVAfg/NS59WfO5g6r16zwv9u2ZGhikRERI6lUB+DNfP9DPhNB9p9Q/Vy/7t5e4YqEhEROdaYQt3MFplZburvy83sI2ZWlt7SJo/lM0soyg1xqH2AI92D6qmLiMikNNae+veBhJktBv4TWAB8N21VTTLBgL0yBL9pf8fRUG/elsGqREREXm2soZ50zsWBNwNfcs59FJiVvrImn3XzKwB4Zn/70eH3lp2QTGawKhERkaPGGuoxM9sAvBv4aaotnJ6SJqd1df68+jP726Ggwt+GNdYPPY0ZrkxERMQba6i/B7gI+Lxzbp+ZLQD+K31lTT7nzSsjFDC2N3bTPRjz91cHaNuV2cJERERSxhTqzrltzrmPOOfuM7NyoNg5d2eaa5tUCnJCnFNbStLBswc6hoX67swWJiIikjLW2e+PmVmJmVUALwL3mtk/jeF5V5vZTjPbbWZ3jLL9n83shdTPy2bWeepv4cxZvyB1Xn1f+9FQb1Woi4jI5DDW4fdS51w38BbgXufcWuDKEz3BzILAV4BrgJXABjNbOXwf59xHnXOrnXOrgS8DPzjVN3AmXZAK9Y372qFqiW9UT11ERCaJsYZ6yMxmAW/j6ES5k1kP7HbO7XXORYH7gZtOsP8G4L4xHjsj1s6vwAxerO8kUrrAN+qcuoiITBJjDfXPAQ8De5xzz5jZQuBkaTYHODTscX2q7RhmNh9/7fuvxlhPRpTmh1kxs4RYwvF8TxlYADoPQjyS6dJERETGPFHuf5xzq5xzH0w93uuce+tJnmajHeo4+94CfM85lxj1QGa3m9kmM9vU0tIylpLTZui8+tMH+6BsPrgktO/LaE0iIiIw9olytWb2QzNrNrMjZvZ9M6s9ydPqgbnDHtcCDcfZ9xZOMPTunLvbObfOObeuurp6LCWnzVCob9zfphnwIiIyqYx1+P1e4CfAbPwQ+oOpthN5BlhiZgvMLAcf3D8ZuZOZLQPKgSfHWnQmnV/nQ/3ZAx0kKnStuoiITB5jDfVq59y9zrl46ucbwAm7zKllZT+EPxe/HXjAObfVzD5nZjcO23UDcL9z7nhD85NKdXEuC6sKGYwlaQylpgiopy4iIpNAaIz7tZrZrRwdIt8AtJ3sSc65h4CHRrR9asTjz4yxhknjnNpS9rb28XJ8JrUAbXsyXZKIiMiYe+rvxV/O1gQ0Ajfjl46dllbOKgHg2d5K39Cq4XcREcm8sc5+P+icu9E5V+2cq3HOvQm/EM20dNbsUgCebs2BcAH0t8JAR4arEhGR6W6sPfXRfGzCqsgyZ832PfVtTb24ykW+UUPwIiKSYeMJ9dGuQ58WygtzmFGSS380QX9RnW/UZDkREcmw8YR6VsxWT5dlM31vvSmcuhRf59VFRCTDThjqZtZjZt2j/PTgr1mftpbPLAZgd3Kmb1BPXUREMuyEl7Q554rPVCHZZtkM/9G80F/NG0GhLiIiGTee4fdpbVmqp/7bDj8TnrY9kExmsCIREZnuFOqnaXFNEQGDre2GK6yB+AB0H850WSIiMo0p1E9TXjhIXVUhiaSjv7jON2oIXkREMkihPg5Dk+VaclIz4BXqIiKSQQr1cVg2w1/WttfN8g26rE1ERDJIoT4OQ5PlXhic4RtatmewGhERme4U6uMwNPz+WGcq1Ju2QHbcQVZERKYghfo4zKsoID8cZHNPIcm8chhoh+6GTJclIiLTlEJ9HAIBY+mMIsDoKVvuG49syWhNIiIyfSnUx2lpamW5w7mpu7U1bc5gNSIiMp0p1MdpaLLc1uR839CknrqIiGSGQn2clqfu1vZkX+r+Nk0vZbAaERGZzhTq4zTUU3+0rRwXCEP7Xoj0ZrgqERGZjhTq41RdnEtlYQ4dEYhVLAUcHNma6bJERGQaUqhPgKHeekvJSt9weFMGqxERkelKoT4BhkJ9RzgV6oeezmA1IiIyXSnUJ8DQynJPRlOXtR3aqJXlRETkjFOoT4BlqRnwv2krg/xy6GmEzoMZrkpERKYbhfoE8KvKwZ7WPpK1633joY0ZrEhERKYjhfoEKMgJsbCqkHjS0Vx6rm/UeXURETnDFOoT5Ny5ZQC8GEitAa9QFxGRM0yhPkHOrS0F4Nc9tWBBf2MXLUIjIiJnkEJ9ggz11Dc1DMKsVeCScPjZDFclIiLTiUJ9gqyYVUI4aOxq7iU6+3zfqCF4ERE5gxTqEyQvHGT5zBKcg335Z/nGA7/NbFEiIjKtKNQn0Nr55QA8Hl0JGBz4nc6ri4jIGaNQn0Dn11UA8HiDg9p1kIjCvl9nuCoREZkuFOoT6PwFvqf+7IEOEouv8o0vP5zBikREZDpRqE+gmuI8FlQV0h9NsKf0Nb5x1y+0DryIiJwRCvUJdn6d760/1j0TimZCTwM0vZThqkREZDpIa6ib2dVmttPMdpvZHcfZ521mts3MtprZd9NZz5lw8eIqAH69qxWWvME37vp5BisSEZHpIm2hbmZB4CvANcBKYIOZrRyxzxLgE8DFzrmzgD9NVz1nyqVLqgkYbNzXzsCCK32jQl1ERM6AdPbU1wO7nXN7nXNR4H7gphH7vB/4inOuA8A515zGes6I8sIczptXTizheCJxFgTCUP8M9LdnujQREZni0hnqc4BDwx7Xp9qGWwosNbPfmtlTZnb1aAcys9vNbJOZbWppaUlTuRPniuU1APxiTz/UXeyXjN3+YIarEhGRqS6doW6jtI2cBh4ClgCXAxuAr5tZ2TFPcu5u59w659y66urqCS90og2F+q92tJBcdr1vfPTzkIhlsCoREZnq0hnq9cDcYY9rgYZR9vmxcy7mnNsH7MSHfFZbPrOYOWX5tPZGeHHGm6B8AfQegb1aiEZERNInnaH+DLDEzBaYWQ5wC/CTEfv8CHgdgJlV4Yfj96axpjPCzLhyhe+tP7KzDda80294/Iu6Zl1ERNImbaHunIsDHwIeBrYDDzjntprZ58zsxtRuDwNtZrYNeBT4c+dcW7pqOpOuXDkDgJ9vPQLrb4f8Cjj0FOz+ZYYrExGRqSqt16k75x5yzi11zi1yzn0+1fYp59xPUn8759zHnHMrnXPnOOfuT2c9Z9IFCyopzQ+zq7mX7e0OLkldrffo36i3LiIiaaEV5dIkJxTg+lWzAPjh84fh/PdDQSU0PA/1mzJcnYiITEUK9TR6y5paAH70/GESoXw471a/4ZHPZK4oERGZshTqabRmXhl1lQU090T47e5WeO2fQU4RHHgCmndkujwREZliFOppZGa8+TzfW/+vpw5AXimc83t+4zNfz2BlIiIyFSnU02zDBXPJCQX4xfYj7G7u9TPhAZ77FnQ3ZrY4ERGZUhTqaVZTnMfNa2txDr762B6YsRJW3ACJCDz0cc2EFxGRCaNQPwP+8NKFhALGD56vZ1tDN1z1ecgthR0/hT26bl1ERCaGQv0MmF9ZyDsvmo9z8Df/uw1XNg8u/rDf+PNP6Q5uIiIyIRTqZ8ifvH4JpflhfrenjUe2N8P574OSOdC8FX71/zJdnoiITAEK9TOkrCCHP73S36vmbx/aTjRcCrd8x2984T7omxKr44qISAYp1M+gWy+cz8LqQva19vHtpw7A7PNg8RsgPuCXjxURERkHhfoZFA4G+OtrVwBw1yMv09EXhav+HwRCsOleLUgjIiLjolA/w65YXsPFiyvpHoxz1y93Qc0KWPNuwMGDH4FEPNMliohIllKon2FmxievW0nA4NtPHWBHUzdc8UkongWHnoatP8h0iSIikqUU6hmwYlYJv3/BPBJJx5898CKx3DK49M/9xse/CNG+zBYoIiJZSaGeIXdcs4La8ny2NnRz9+N74dwNULUMWl+GJ76U6fJERCQLKdQzpCg3xJ1vWQXAv/xyF4f7Da7/J7/x2W9APJq54kR6+WQaAAAgAElEQVREJCsp1DPokiVVXL9qFpF4kjv/bwfMvxiqV0BfM+x4MNPliYhIllGoZ9gnrl1BbijAgy828MyBDjj/D/yGhz8Jg12ZLU5ERLKKQj3D5pTl84eXLQLgsw9uJXnuO6DmLOhpgOe/k+HqREQkmyjUJ4EPXLaQWaV5bDnczfc2t8Hr/spv2Pjvum5dRETGTKE+CRTkhLjjmuUA/P3DO+mZfyVULISO/bD5vzNbnIiIZA2F+iRx47mzWTu/nNbeCP/6631w6V/4DT/7BPQ2Z7Y4ERHJCgr1ScLM+PQNKwG454l97J99HdS9FiJd8Ow3M1ydiIhkA4X6JLKqtoyb19YSSzg+/7OX4dKP+w3P3qtz6yIiclIK9UnmL964jMKcIL/YdoQn4mdB5RLoPgw/+8tMlyYiIpOcQn2SqSnJ44+vWAzAF362g+SVn/Ubnvk6tO7KYGUiIjLZKdQnofdevIAZJblsbejmodh5sPodfsOmezJbmIiITGoK9UkoLxzkI69fAsA//vxl4qtvBQye/hq078tscSIiMmkp1Cept62bS11lAfta+/if5lo49xZwSXjkM5kuTUREJimF+iQVDgb46BuWAvClR15m4LWfgEAItj8I3Q0Zrk5ERCYjhfokdsOq2Zwzp5Qj3RG+8uwgLL8OXAJ++blMlyYiIpOQQn0SCwSMz9zoF6S5+/G9NJz/V2ABePE+ePrfM1ydiIhMNgr1SW7t/AretHo20USSL24chBU3+g2/+BTEBjNbnIiITCoK9SzwZ1ctIxw0fvTCYXasSw29xwdhy/cyW5iIiEwqCvUsMLeigFsvnI9z8KmfN5C44ct+w0N/rgVpRETkFWkNdTO72sx2mtluM7tjlO23mVmLmb2Q+nlfOuvJZh++YglVRbls3N/O/bHXwllvhlg/PPq3mS5NREQmibSFupkFga8A1wArgQ1mtnKUXf/bObc69fP1dNWT7SoKc16ZNPflX+0lctn/BxaErT+ATfdmuDoREZkM0tlTXw/sds7tdc5FgfuBm9L4elPetWfPYsWsEpq6B7nruRhc8lG/4ad/Ch37M1qbiIhkXjpDfQ5waNjj+lTbSG81s81m9j0zm5vGerJeIODvuR4w+LfH9rBx1juObvzO70FfW+aKExGRjEtnqNsobW7E4weBOufcKuAR4JujHsjsdjPbZGabWlpaJrjM7HLhwko+fIVfF/5TP68n/idboGoptL4MT/1bhqsTEZFMSmeo1wPDe961wKvWN3XOtTnnIqmH/wGsHe1Azrm7nXPrnHPrqqur01JsNvng5YuYW5HPjqYevrM9ATf8i9/wm3+AHQ9ltjgREcmYdIb6M8ASM1tgZjnALcBPhu9gZrOGPbwR2J7GeqaMvHCQT17nJ8198eGdHCpaBbPX+I3ffx8062MUEZmO0hbqzrk48CHgYXxYP+Cc22pmnzOz1LJofMTMtprZi8BHgNvSVc9Uc9XKGbzxrBn0RuJ89IEXib37f2HRFRDrg2+9CfpaM12iiIicYebcyNPck9u6devcpk2bMl3GpNDeF+Waux7nSHeED16+iL+8Yh588wY4vAnW/yFc+/eZLlFERMbJzJ51zq0by75aUS6LVRTm8OUNawgYfPWxPfzs5W648V8Ag413w8b/gCz70iYiIqdPoZ7l1i+o4I5rlgPwZw+8wMvMgzd8FnDw0Mfhv94C0f7MFikiImeEQn0KeP9rF3LDubPpiya47Z6NtKz6ALzl61BQCXt+BffdomvYRUSmAYX6FGBmfPHmVZxbW0pD1yDXf/k3NMy7Ht7zM8ivgH2/hn9dB/XPZrpUERFJI4X6FJEXDnL3u9axem4ZR7ojfPi+5+kvXQi3/RTmXwwD7fDtN8NLul2riMhUpVCfQmaU5PGf717HzJI8nj3QwW33PENv2TJ4549g2bUQ6YLv/wF8YS40bcl0uSIiMsEU6lNMZVEu333/BcwsyWPj/nY23P0U3XGDW74LN9zld4p0wzeug0c+C52HTnxAERHJGgr1KWhhdREP/OFFzKso4KXDXbzjP56muTcCa2+DOw7BvNfAYCc88U/+XPsL39WlbyIiU4BCfYqaV1nAd953AXPK8nnpcBdv+9qTbDncBXkl8J6H4L0/h9rzIT4IP/ogPPgROLQRkolMly4iIqdJK8pNcW29Ed75nxvZ1thNOGisnlvGJ65dwZp55TDYBT/8AOwcdhOY8jq49C/gnJshlJuxukVExDuVFeUU6tPAYCzBFx7azjefPABAKGD849vO5abVqdvb738CNv83bP2xn0w35MrPwvrbIacgA1WLiAgo1OU4ntzTxofve57WXn+328uWVvOR1y9m7fwKv8NAJ2z9ATz2d9Db5NvKF8DFH4H5l0DlIggEM1S9iMj0pFCX43LO8Y3f7ecff/4yvZE4ANetmsVfX7uC2WX5fqe+VvjFp2HvY9Bdf/TJS66Cq/4GqpaC2ZkvXkRkGlKoy0m19Ua457f7+M8n9jEYS5IbCvDOC+ez4YJ5LKou8jtF++GlB2D7g7D7kaNPnrMWrvl7/1vhLiKSVgp1GbP6jn6+8NAO/velRsBn9GVLq/n4Vcs4e07p0R2btsCv7/QBP6SwBhZe5nvw5/yeAl5EJA0U6nLKNu1v51tPHuBnW5uIxpMAvHZJFTevreW6c2YRCqaufuxrhcf/ATbfDwMdRw+QUwwrb4IlV/oe/rkbIKArJkVExkuhLqettTfC3/3fDr7/XD3J1D+NJTVFfPQNS7lyxQxyQqmgdg4OPgWPfBoOPX3sgRa/AZZdDZEefw5++XVn7k2IiEwhCnUZt/a+KPc/c5DvPn2Q+o4BACoKc7hh1SzeuraWVbVlR3eODfo7wT33Ldjx09EPWLse5qyB3iNQuRgu+AAUVvltQ/8GNXwvInIMhbpMmEg8wXefPsj9Gw+x80jPK+3LZxZzw7mzeeNZM1hUXYQNBXJsANp2w+YHYO+j0PTS2F5ozjq46V+hYhGEcnybcwp6EZn2FOoy4ZxzbG3o5vvP1XPfxoMMxpKvbFtYVcjM0jxmluTxsauWUls+YrGagQ44+DTsehhad0FPow/+0VgASudCfhm074Oz3gwzzoZZ5/plbXWeXkSmGYW6pNVgLMFvd7fy0EtN/GrHETr6Y69sM4M188q5YnkNly+rZlF1EXnhURasSSZg6w/9pXIvP+zv914yB7obgOP8mwzmQkElxPohpwgWvQ6u/SKE89PzRkVEJgGFupwx0XiSX24/woObG3jopSZyggGiiaO9+NxQgCtXzmDtvHLmVhSwfkEFpfnh4x8wHoGXfwZd9T74m7fBi/eduIjSeX5SXn8bNG6GvhaoWuKDf/GVcO4t/sY1Ox6C1Rt8u4b1RSRLKNQlY3ojcZ7Y1cqvdhzhiV2tNHQNHrNPYU6QK1bMYNmMIlbMKqG6OJdz5pQePS8/GufgyFboPgxbvg/bfuyD+nQUz4Jk3If/ee/0N7EpqoGy+TDvQt3IRkQmFYW6TBr1Hf08vPUIWw93sbe1jxcOdY66X24owGsWVXL2nFKqi3O5fGkNJfkhckNB8nOOs958Ig6xPn9DmtZd4JLQ0wS5xRDthae/dnpFF1T5BXWObPHHXP0OWHYNFM2ARATyy0/vuCIip0GhLpNW10CM5w52sLOphy2Hu+gejPPCwQ66B+Oj7l+YE2TN/HJmleaxclYJC6uLWFBVSEVhDh39UaLxJAuHlrUdqWkLlMyGvFLoOuTDuH2fP29/eBO07PRtz3977G8gmAM1K/y5/XXv9b3+3iN+UZ5zN0AwrKF9EZlQCnXJKvFEkkMdAzz+cgsH2vqp7+jnyT1t9ERGD3rwuTn0T3dVbSlXrZzB8pklFOaGWFdXTihgqf1OIWCbd0DTZtjzKx/a8UE/a//IGC/LAwjl+aH8mpV+Fr8Z7Pw/WPpGuPrvIBEFl/CjCSIiY6BQlykhmXRsb+qmuTvC/rY+tjV0s7ull/2tfa+acT+agEHSQW15PuvrKlhb53v78ysLmVOWP/qM/BNxDgY74amv+TXwxyun2A/pz10PM8/xEwNnnA1lc/35/mQCDj8Hi19/tOefiPmRABGZVhTqMuX1RuIEDAZjSb715H7qOwY42N5Pa2+EvS19J33+wupCSvLCVBfnsmxGMQurC2nqHuScOaUsqCpkdmk+Lb0Raopzj+3tDy2K077PX4bXst33vItnQ8Pz8OjnYf9v/NB8T+P43+zN9/oV+174Lsy/GNbeBgsuhZxCTeoTmQYU6jKtReIJjnRFONTRT1tflD3NvTy5p43eSJxDHf30HOf8/WhWzirhiuU1vGZRJfMqC5hdms+PXzzMylmlLJs5hiH0330Zdv3CT7xb+kbA/Dn8p74Ks1ZB/TOn/0YBll8Pq97uh/1nnu1vppNXCkXVfnukx8/yL50LFvSL90R6dFmfSBZRqIucQM9gjIPt/RzuGGBbYzdtvVF2NvWw+XAnlYW5tPVFXrVi3vGcW1vKm86bgwGxhOPsOaX0RuKcX1dOWUHO2AuK9EJukV95r7vRz7g/sgV+/kkfyKcjr8yfLhipdB50HYS618L1/wy5JVA84/ReQ0TOCIW6yDgkk47dLb00dg3y292tNHQOsOtILw1dA2Pq5eeGAswpy6ejP8qFCyuZXZbPhQsrWTe/nM2Hu1g2o5iZpXknL2SwG6J9UDILWndDxUI/ee/Q035W//4noPFFaHjO98IbXziNd2uA86cKimf5of35r/HHf+5bMPs8f+2+iGSMQl0kDZxzdPbHiCWSbGvs5pHtR0gkHT2DcfLDQZ7c2/bKHe1OpCg3xJUragAoL8xh9dwyaorzWFBVSGVRDs5BKGAEAqc4PN7dCIXVfrZ+6y7/haCvxc++X34tbPmBX6Fv2XX+d8e+sR332n+A//tLmHsBrP59v4b/uvf68/sHn/QjC4Gw3x4IaX1+kQmmUBfJAOcc8aTj+YOdRONJDrb3A9DQOcBDLzXS0hM54WV6w+WFA5Tl53DBwgpqinMJBgLceO5sls0sZjCWIC8cJHiqoT9c627Y9kO/zH7TZmh92X8J6Dp0+scM5UM89aVm9nlw3T/52+0OcQ46D/g1/EtmHW3TuX2RE1Koi0xSzjk213exvbEbM2jqirC5vpPWvijbGrqIJU7832NeOEAknmRGcR7r6srJCQWo7xhgfV0F6xdU0DMY54rlNcdfhe/kBfqFdAa74F/XHm0vng09DX7xnUTU303PpeYd5JX6/UeTWwLFM/0Evu76o+3n/B689D/+uKt+D85/H5TN89sSMehv17l+kRSFukgWcs5xpDtCRWEOA9EEjd0D/HZ3G/dvPMiu5l5qy/PHNLwPMKcsn7qqAlbPLWNHYw+VRTm866I6FlUXsaelly898jKrasv4yOuXHP8gHfvhv98Jl3wUzn4LxAZ8LzsQgLY9foh/waWQUwC9zf5uey//DHb89PQ+gDlr4fCzRx9f+VlYeLlfwS+U6ycSWhDySk7v+CJZSqEuMkW190Xpi8Rp7omwv7WPIz2DPLazhfa+KLube0/5eHnhAFVFuaydX85rl1Rz3rwyFlYVntpKfCPFBmHPL33ot77sz/PvfdRfVhcI+cl9LumH4pdeDXsfO/HNeQJh/8VhsMvP6l97mz9lUDbPD/kXVvmJfYc2+hv0DHb6hXvK5voRgcBpjlqITBIKdZFprLU3wp7mXuo7BjjQ3s9vdvnQDwWMA239mHHCYf6i3BC15fnk5wRp6BzgrNmlzKsooDA3yLXnzOK3u1u5ftVsZpdN0H3suxv80rzdjX6RnWifvxtfIup75v2tp3/ssvmw6m3QssPP7m/d5b9gAKz/Q/8FoaveLwsc7fGvXVjtvwj87stw4R/D3PP96YNdP/e39z33Fn+L4IKKCXn7IiczaULdzK4G7gKCwNedc6Our2lmNwP/A5zvnDthYivURU5fIukYml/3y+3NPHuwg1g8SW8kzu7mXva19tHWFx3TsV67pIr1dRXk5wSZW1HA0hnFFOWGXpnIV108Qavd9RyBSLcP+bY98PgXfU99SNFM6G069u+JcslH4Xf/CskRSxMXVEHpHCis8cv9bvm+v43vYJe/KiCY608pXP0FKK2F5u1+RKGgwo8w9DTCQLu/2yD4Owvml/s5CIU1EAz5duf8l5ySOZpUOE1NilA3syDwMvAGoB54BtjgnNs2Yr9i4H+BHOBDCnWRzHHO0TUQY1tjNzubemjqHqS1J8ru5h5erD/OZLhRhALG4poidjT1cNHCSt6yZg554SBLZxSzdEYRZkYkniA3dJpD4wMdEI/64fzCyldv62v1XwC+cZ2/sc7BJ30Pe/Ya3wNv3+sfDwkXpmbjG7TtOr16JlrlYlh0hf97aHRh7W1ww11H9xno8DccckmYvdp/YZApabKE+kXAZ5xzb0w9/gSAc+4LI/b7EvAI8HHg4wp1kclrIJogNxTgUEc/LxzqZHN9Fx19UXY09dAXjdPeFz3pAj0Bg5L8ML2Dca4+eyZzKwpYPrOYxTVFFOWGmFmaR9dAjJriMSzQMxaDXdDbAlWLR7yZTh/yw++Yl0z4dpf01/JvvBvadvvZ+Qsu873o3iNw8Ck/pN+x3/ee88v94kDJBDz9tdFXAjzRVQJjVbUMwvm+puiIORQ3fQXmrINHPu0nLIKfdxAI+9MKZfP8iAEOas/3X4gSUT//4MYv+3sJnGwkIJn0+4y2X8Pz/nTJrFXje49yjMkS6jcDVzvn3pd6/E7gAufch4btcx7wSefcW83sMY4T6mZ2O3A7wLx589YeOHAgLTWLyPgkk85f+t49yEv1Xfzo+cPEk0n2tPQRMGjuiYx57f3z68qZV1FINJHk2rNnUt8xwCPbj/Anr1/CRYsqxzeZ70xIxOHZe/0lerXnQ+063+Munu2XAV54ub9EsPeIP6cfCPkvGF2H4MX7/ReA3BJ/6iHSA899M43FplYWBL/GQMPzfinh+a/xcwy2/8SHf7QXqpfDVZ+HHQ/6+RCXf8J/+XnxPn+p47sfhLpL/LG6G/xIw9wL/NyFkV+sTmbo1ENp7YS+22wzWUL994A3jgj19c65D6ceB4BfAbc55/afKNSHU09dJHs552jqHuT5g53EEkna+6I0dg3y9L52egZiNHYNMhBLjOlY6+sqmFOeT24oQFFuiKK8EAuriyjOC9HRF+VwxwDXnDOTeRWFdA/GqCrK8jvaNW6G397lw75sLqy8Cea9Bn7zD/Drvzt2/+XX+6sB7nu7f3zV531Yd9X7nv54JiCOxfC1DEaa9xqoWuJPlUR7oHKJ/zLR0+RHT5pe8qMgDc/5/eeshbPe7OcazFoF5Qt82JcvOP0VDAc6/Reomeec3vPPoMkS6iccfjezUmAPMDSGNBNoB248UbAr1EWmtoNt/XQORNne2M0z+zs42N5PUW6Itt4IWxu6CZgRTZz8hjtDSvPDdA/GOL+ugrnlBTgcOcEA6xdU0N4XZUFVIW19UeIJx6raUuZXFlCcd+x967cc7mJxTRF54Ul4iVz7Ptj2I5hxju8lh4edunjuW37S3rlvP/Z5P/+kn+UPfpg+GTu6MBD4pYBbX/anE2atfvX9BaqWQevOo49nnOO/LMTHtpbCxDA/OdEl/N8u6YN6+fV+hCO/ApLxo2snFM/y+/c2Q/ueo4epPR8qFsGsc/38iv+5zX8eRTMgEfFfOKqXw7yL/PGbXoKWnVCz3N/xcMGlULXUf9HILQGcf85EvctJEuoh/ES51wOH8RPlft85t/U4+z+GeuoichLOOeo7BnjpcBeDsQQd/TGaewbp6o9R3zFAU/cgeeEAWw53n/ZrVBTmEAoYkXiS2WX5lOaHeGpvO+Gg8fc3r6KsIIeAGbF4koqiHCoKcphRkvfKqeah4HfOTe7TBG174GuXwMV/Cpf/5ej7jFzKd6DD96hrVvibDiViRycrdh6C+3/fn1a48IO+112z0ofrQ38G237s91vzLr9PIOxPP+QW+8WNXNLPGRjo8Of7k3E46y1+LYO2XX6NAwumQnwSswB8dOuETV6cFKGeKuRa4Ev4S9rucc593sw+B2xyzv1kxL6PoVAXkQninKOlJ0LnQIyy/DCbDnTQPRBjV3MvG/e1M6csn1giyeHOAfa19hGJj733fyI5oQAzS/Loi8Rp74+SFwqybGYx580roz+SYE55Pouqi/x8M/xNfYIBY19rH5csrqKiMOdVowGxRJJwMEBT1yAVhTnkhCb4hjlDGZDuLx+RHnjhPli94dWTE09HIg67H4FHPgMzVvo1BwDuueroPvMv9l8yllwJ9c/4iYxrb/NfDA5thJ0Pwfrb/cJH7ftg/29Gf62l1/hed++RowslVS3xx+tv919wcP70yNAoRdl8ePO/w/yLxvc+UyZNqKeDQl1E0sU5R/dAnFgySTzhSDrHlsNdPLO/nba+KCV5YVp6I3QPxIjEknQPxujsjzEQS9A1EDv5C4xRKGDUlvvFfQ51DDCvooB9rX0ArJtfztlzSplbUUBeOEBrT5RNB9qpLS/g8mXVLJ1RTGVRDo2dg0TjSRyO/miCCxZUTO5Rg4nQvMOH87r3nvpKgs750wwFlf650T6/nsCpnrOPDfpljSfws1aoi4icYdFUT/9AWx8J5wiYEU84ygrC/G5PG1sbutjR2EMskaS6OBfnIJ5Msr2xB/BLAJtBJJ4kkZz4/y9XFuZQU5JHwPCTC/PCBAyOdEdwzvGaRVUMxOIEA8bSGcXMrSggmfT1B8wozguxr7WfisIcckMBFlQVsqOph7kV+RN3+aGMSqEuIpKlovEk0USS+o5+jnRHqCzMIZ50VBbmEIkn2dnUw96WXtr6ogxEE5Snzv8f6R7k5eZeGjsHaO+LUluez0AswZHuSNprLsoNMas0j/5ogsLcIL2DcZbPKiEcNCoKcyjMCdHYNUhNSS7lBTkEzM87qKsspKo4l2g8SWl+mIKcIIfa+/ntnlY+cNkiCnNC7GruZXFNEcD4bjecxRTqIiLTWCLpXgnArv4YoaDR3BOhLxKno9/fFCgUCNDSG2FBVSHReJLN9Z3saOohGDDyw0H2t/XRF0nQH41jZsQTST/6kPSrDnYPxjiT8REOGmvnl5MbCjKjJJf9bf3khgJcuLCSmuJcmroG6YsmWFhdSG15fupLQoidTd0srilibkUBsYQjFDDywkGGsq+zP0Z5Yc6ZeyOnQaEuIiJpNXS+3jCaewZp7okwGE2QSE1Q7OiP0TsYJ5FMMhhPsqOph7J8P+Tf0DVIY9cA8ysKaegcIOkc3YNxAma09qZ3ZCEUMKqLc2ntjRAOBuiPJphVmsfSGcVE4glmluQRMCMcDJAbDlCQE6IgJ0gylZWzSvMozA3RF4mzoKqISNzf62BWaR7R1H0UivPCzC7LO/1lkEc4lVAPTcgriojItDJ8Fn5teQG15QUTctyu/hj5OUGiCT+3wAx2NvWQTDoOdw7QF4kTDgZo64syGEsQiSdp7h5kX2sfsYSjrqqAI91+VKI/muBgez+hgB9hCAf978Yuf6vfWMJfGtfYNfhK20T56Ycv4ew5pRN6zLFQqIuIyKRRWuAX/hn+peH8uvHd5tY5RzSRJDcUpDcSp603QlVRLh39UYIBo7FrkP2tfeSEAtR3DFCcFyJoxmAsQX8sQX8kQcCgL5qgoz9Kc3eEotwQe1t7Kc0PE0s4GrsGONIdobwgTElqfkAmKNRFRGRKM7NXhsKLckMU5froK0z9nlWaz5p55RmrbyJN8CoGIiIikikKdRERkSlCoS4iIjJFKNRFRESmCIW6iIjIFKFQFxERmSIU6iIiIlOEQl1ERGSKUKiLiIhMEQp1ERGRKUKhLiIiMkUo1EVERKYIhbqIiMgUYS514/dsYWYtwIEJPGQV0DqBx5uu9DmOnz7D8dNnOH76DCfGRH6O851z1WPZMetCfaKZ2Sbn3LpM15Ht9DmOnz7D8dNnOH76DCdGpj5HDb+LiIhMEQp1ERGRKUKhDndnuoApQp/j+OkzHD99huOnz3BiZORznPbn1EVERKYK9dRFRESmiGkd6mZ2tZntNLPdZnZHpuuZrMxsrpk9ambbzWyrmf1Jqr3CzH5hZrtSv8tT7WZm/5L6XDeb2ZrMvoPJw8yCZva8mf009XiBmT2d+gz/28xyUu25qce7U9vrMln3ZGFmZWb2PTPbkfr3eJH+HZ66/7+9uw2xozzDOP6/SDRGxfqGYhPbNLhordQXRIMtpUSxrZamUEFFMGigIIJailbxgxT8IpTaSkVofakWsdDUVukHrazSIrbxjdimKho1aGo0EZNYRTTq1Q9zbzrdnGx6trt7DmeuHww7zzMPy3Nu7t37zDNzzkj6Xv0tr5N0j6R9kotTk3S7pM2S1rX6+s49SStr/IuSVs70PDtb1CXNA24GvgEcC5wv6djBzmpofQR83/bngWXApRWrq4Fx22PAeLWhielYbd8Fbpn7KQ+ty4HnWu0bgBsrhluBVdW/Cthq+yjgxhoX8FPgAdvHAMfTxDJ52AdJi4DLgJNtHwfMA84jubgnvwS+Pqmvr9yTdDBwHXAqcApw3cQbgZnS2aJOE9D1tl+2/SHwa2DFgOc0lGxvsv107f+L5h/pIpp43VnD7gS+XfsrgLvc+CtwoKQj5njaQ0fSYuBs4NZqC1gOrK4hk2M4EdvVwOk1vrMkHQB8BbgNwPaHtreRPJyO+cBCSfOBfYFNJBenZPvPwNuTuvvNva8BD9l+2/ZW4CF2faPwf+lyUV8EvNZqb6y+mEItvZ0IrAEOt70JmsIPHFbDEtvefgJcBXxS7UOAbbY/qnY7TjtjWMe31/guWwpsAe6oSxi3StqP5GFfbP8T+BHwKk0x3w48RXJxOvrNvVnPyS4X9V7vNPNRgClI2h/4LXCF7XemGtqjr9OxlfRNYLPtp5p7nOMAAANTSURBVNrdPYb6fzjWVfOBk4BbbJ8IvMd/ljt7SQx7qOXeFcDngE8D+9EsF0+WXJy+3cVs1mPZ5aK+ETiy1V4MvD6guQw9SXvRFPS7bd9b3W9OLGfWz83Vn9ju6kvAtyRtoLnUs5zmzP3AWgKF/47TzhjW8U+x69Jf12wENtpeU+3VNEU+edifM4BXbG+xvQO4FziN5OJ09Jt7s56TXS7qTwBjdcfn3jQ3itw/4DkNpbp+dhvwnO0ftw7dD0zcvbkSuK/Vf2HdAboM2D6xRNVVtq+xvdj2Eppce9j2BcAjwDk1bHIMJ2J7To3v9NmR7TeA1yQdXV2nA8+SPOzXq8AySfvW3/ZEHJOL/es39x4EzpR0UK2YnFl9M8d2ZzfgLOAF4CXg2kHPZ1g34Ms0S0R/A9bWdhbNdbVx4MX6eXCNF80nC14C/k5zl+3AX8ewbMBXgT/U/lLgcWA98BtgQfXvU+31dXzpoOc9DBtwAvBk5eLvgYOSh9OK4w+B54F1wK+ABcnFPcbsHpp7EHbQnHGvmk7uARdXLNcDF830PPONchERESOiy8vvERERIyVFPSIiYkSkqEdERIyIFPWIiIgRkaIeERExIlLUIzpG0seS1ra2GXtCoaQl7adYRcTcmr/nIRExYt63fcKgJxERMy9n6hEBgKQNkm6Q9HhtR1X/ZyWN13OhxyV9pvoPl/Q7Sc/Udlr9qnmSflHP6/6jpIUDe1ERHZOiHtE9Cyctv5/bOvaO7VOAn9F8Nz21f5ftLwJ3AzdV/03An2wfT/Md7P+o/jHgZttfALYB35nl1xMRJd8oF9Exkt61vX+P/g3Actsv1wN83rB9iKS3gCNs76j+TbYPlbQFWGz7g9bvWELzvOixav8A2Mv29bP/yiIiZ+oR0ebd7O9uTC8ftPY/JvfuRMyZFPWIaDu39fMvtf8YzZPlAC4AHq39ceASAEnzJB0wV5OMiN7yDjqiexZKWttqP2B74mNtCyStoXnDf371XQbcLulKYAtwUfVfDvxc0iqaM/JLaJ5iFREDkmvqEQHsvKZ+su23Bj2XiJieLL9HRESMiJypR0REjIicqUdERIyIFPWIiIgRkaIeERExIlLUIyIiRkSKekRExIhIUY+IiBgR/wa6wuK49Y0fTgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(lstm_autoencoder_history['loss'], linewidth=2, label='Train')\n",
    "plt.plot(lstm_autoencoder_history['val_loss'], linewidth=2, label='Valid')\n",
    "plt.legend(loc='upper right')\n",
    "plt.title('Model loss')\n",
    "plt.ylabel('Loss')\n",
    "plt.xlabel('Epoch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAGDCAYAAAAyM4nNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XmcXFWZ//HP03vSS0LSWTsh+0pWCBAEBGQxoCSAIDDqiCKOqIijzgjjhgz+REUdHXEblQgiiKgQMMoSWWRPgCRk35fO0km6s/S+nt8ft5JU1b3dXd3pWrrq+369+tVV596q+9Tt6nrqnHsWc84hIiIivV9WsgMQERGRnqGkLiIikiaU1EVERNKEkrqIiEiaUFIXERFJE0rqIiIiaUJJXU6Yma02s/M72edkM6sxs+wEhdUtZrbQzO5KgTicmY2PJSYzu8vMDpjZ3jjF8ryZfSJ0+0Nm9nTYtrPNbGPob3uFmQ0xsxfNrNrMvh+PeHpa+OuL83FuMLOXuvnYO8zsdx1s32ZmF3U/unaf99j7UHqHnGQHIPFjZtuAIUArUAssBm5xztX05HGcc6fEsM8OoKgnjytgZiOBLwKjnHP74n0859yDwINhRXcCP3HO/SgUz9eAA0CJS/AkGGY2GtgK5DrnWtrZ5w5gvHPuw4mLTCRxVFNPf5c754qAU4HTga9G72AevRe6yMxS4UvxKKCyOwm9h+IfBayOur+mOwk9Rc5nh3pDjJLZ9EGeIZxzu4C/AdPgWJPjt8zsZaAOGGtm/czs12a2x8x2hZp1jzWXm9lNZrY21LS6xsxODZUfa/ozszPMbJmZHTGzCjP7Qah8dKgpLyd0f7iZLTKzKjPbZGY3hR3nDjN7xMzuDx1rtZnNae+1mdmPzGxn6Jhvmtm5sT6Xmc02s7dC2/4AFHRwnBvM7GUz+6GZVQF3hMo/HjovB83sKTMbFfaYU8zsmdDrrDCz/wo7T6+a2aHQ+f6JmeXF8KcMj+ci4BlgeKj5e2GofH7odR4K/Z2nhD1mm5l92cxWArVBScrMLjazdWZ22Mx+AljUOXgpdHszMBZ4InT8h4CPAv8Zun+RmWWZ2W1mttnMKkN/iwGhxx99T9xoZjuAf4TK55rZK6H4V1jYpZ3Q6/nv0N+h2syeNrPS0OYXQ78PhY5/VtTrmgf8F3BtaPuKsM2jgp6zmzHeYGZbQs+11cw+FBXHPaH3ylYzuzSsvN3/iYC/0UfMbHvonH6lg/3mmtlei/w/vjL09+/S+9CiLlNY1OUEM5sc9l5fb2YfDNt2mXmfGdXmfbZ8qb2Y5QQ55/STpj/ANuCi0O2ReDWq/w7dfx7YAZyCdxkmF3gM+AVQCAwG3gD+LbT/NcAuvNq+AePxmnyjj/Mq8JHQ7SJgbuj2aMABOaH7LwA/xUuis4D9wIWhbXcADcBlQDbwbeC1Dl7nh4GBodfxRWAvUNDZcwF5wHbg30Ov/2qgGbirnePcALQAt4SO1Qe4AtgETAmVfRV4JbR/MbAnFFNB6P6ZoW2nAXNDjxkNrAU+H3Ysh9dMDLCwg5jOB8rD7k/Eu9Ryceg1/Wcovrywv9Xy0PuhT8DzlQJHQuciN3RuWoBPhJ2Dl4LeY0GxAp8HXgNGAPl476+Hot4T9+O95/oAZUBl6O+VFXodlcCgsPft5tDr7BO6f3fQe6yd83UH8LuoslieM6YYQ/scASaFHj8MOCXs3DUDN+G9F28GdgMW4//E70K3pwI1wLtD5/QHob/RRe285s3AxWH3/wjc1o334fOE3gfR74XQ694JfCz0XKfiXYY5+tr3AOeGbp8EnJrsz8d0/Ul6APqJ4x/X+8CtAQ7hJa+fEvogD/2D3hm27xCgkbAPeuB64LnQ7aeAWzs4ztGk/iLwTaA0ap+jH445eAmlFSgO2/5tYGHo9h3As2HbpgL1XXjdB4GZnT1X6EPx2IdqqOwVOk7qO6LK/gbcGHY/C6/lY1To/L0dY8yfB/4Sdr+7Sf1rwCNR8ewCzg/7W328gzj+lbAvUHhf4MrpflJfSygxhe4Pw0tsR5OIA8aGbf8y8EBUTE8BHw173341bNungb9Hv8c6eH13EJzUO3vOmGLES26HgA8Q9aUpdO42hd3vG3ruocT2P3E0qX8deDhsv0KgifaT+l3Ab0K3i/G+9I3qxvvwedpP6tcC/4x6rl8A3wjd3gH8G15fi6R/Nqbzj5rf098Vzrn+zrlRzrlPO+fqw7btDLs9Cq9mtifUFHcI759ycGj7SLxv/J25Ea/Gs87MlprZ+wP2GQ5UOeeqw8q249WAjgrvyV0HFAQ1FQOY2RdDzd+HQ3H3w6txdvZcw4FdLvSpExZHR3ZG3R8F/CjsnFXhJcIyOjhnZjbRzJ4MNY0eAf5fVMzdNZyw1+CcawvFHH5uo19D9OOPbQ+dm47278wo4C9h52ctXvIa0k48o4Brju4fesw5eF8Gjor+e/ZEB8zOnjOmGJ1ztXgJ7lN4/0t/NbPJQcdxztWFbhYR2//EUdF/o1q8loL2/B64yszygauAt5xz26FH34ejgDOjzsmH8L6wgPcl5zJgu5m9EH1pRHqOknpmC09mO/Fq6qWhLwH9nXMl7njP9p3AuE6f0LmNzrnr8b4MfAd41MwKo3bbDQwws+KwspPxapRdYt718y8DHwROcs71Bw4Tdh24A3uAMjML3/fkTh7jou7vxLtE0T/sp49z7hU6Pmc/A9YBE5xzJXjXemOJuTO78T5gAa8TJN6Xi/BzG/0awu0J7R/9+O7aCVwadX4KnNfHIyienXi14PD9C51zd8dwrI5eV1f26exxHcbonHvKOXcx3heRdcD/xfD8XfmfiP4b9cW7/BQcuHNr8L4gXAr8C16SP6or78NavNaFo4aG3d4JvBB1ToqcczeHYljqnFuA97nwGPBIe/HKiVFSFwCcc3uAp4Hvm1mJeR2cxpnZeaFdfgV8ycxOM894C+sQdpSZfdjMBoVqiIdCxa1Rx9qJ18z9bTMrMLMZeDX8B+m6YrzrifuBHDP7OlAS42NfDT32c2aWY2ZXAWd08fg/B243s1MAzOtseE1o25PAUDP7vJnlm1mxmZ0ZFvcRoCZUk7u5i8dtzyPA+8zsQjPLxbue34h3vmPxV+AUM7sq1JrxOSI/vLvq58C3jr5XzGyQmS3oYP/fAZeb2XvNLDv0/jjfzEbEcKz9QBte5732VACj7cRGe7Qbo3nj9OeHvsg24l3+au346br8P/Eo8H4zOyfUqe1OOv8s/z3e3/LdeNfUj+rK+3A5Xo2/r3lj128M2/YkMNG8Dny5oZ/TzWyKmeWZN79BP+dcc+h4nZ4T6R4ldQn3r3idx9bgXZd+lFCzp3Puj8C38D4cqvG+bQ8IeI55wGozqwF+BFznnGsI2O96vOuVu4G/4F17e6YbMT+Fd117A15tpIEYm4udc014zZE34L3ea4E/d+Xgzrm/4LVIPBxqvlyFVyMi1JR6MXA5XrPrRuCC0EO/hFdrqsaryf2hK8ftIJ71eB0H/xevo9LleMMam2J8/AG8TpF34zXpTgBePoGQfgQsAp42s2q8TnNntrdzKLktwKsx7sf7W/4HMXxWhZqzvwW8HGoCnhuw29GEVmlmb3XlhcQYYxbeF6ndeJdizsO7Rh+LmP4nnHOrgc/g/S/uwXvvlnfy3A/h9b/4R+hvfFRX3oc/xLt2XwH8lrAvHKH3+iXAdaH49+L9X+SHdvkIsC30P/IpvPeoxMHRXpciIiLSy6mmLiIikiaU1EVERNKEkrqIiEiaUFIXERFJE0rqIiIiaaLXrThUWlrqRo8enewwREREEuLNN9884JwbFMu+vS6pjx49mmXLliU7DBERkYQws86mrz5Gze8iIiJpQkldREQkTSipi4iIpAkldRERkTShpC4iIpImlNRFRETShJK6iIhImlBSFxERSRNK6iIiImkibjPKmdlvgPcD+5xz0wK2G/Aj4DKgDrjBOfdWvOIREZHeoamljTbnkh3GCTOD/JzshB4zntPELgR+AtzfzvZLgQmhnzOBn4V+i4hIBtpZVcdtf17Ja1uqaG3r/Un9kqlD+OW/zknoMePW/O6cexGo6mCXBcD9zvMa0N/MhsUrHhERSV3OOW556G1e3lSZFgk9WZJ5Tb0M2Bl2vzxU5mNmnzSzZWa2bP/+/QkJTkREEmf5zkMs33ko2WH0eslM6hZQFvj1zDn3S+fcHOfcnEGDYlp9TkREepHHl+9OdghpIZlLr5YDI8PujwD0VxURyTAtrW08udL/8Z+XnYUFVf96idycxNebk5nUFwGfNbOH8TrIHXbO7UliPCIikgSvbK7kQE1TRFleThbLvnoRJQW5SYqqd4rnkLaHgPOBUjMrB74B5AI4534OLMYbzrYJb0jbx+IVi09rMxza4d3OKYCS4fTqr4MiIr3YY8t3+creM2mwEno3xC2pO+eu72S7Az4Tr+N36HA5/O+px+8PGAvXPwyDJiUlHBGRTNXQ3MpTq/b6yhfMGp6EaHo/zSgHULUFFt2S7ChERDLOkrX7qG1qjSgrzs/hgsmDkxRR76akflT5UmhrS3YUIiIZJajpfd60oRTkJnYmtnShpH6UawPX2vl+IiLSIw7XNfP8+n2+8itmB05ZIjHIzKSe1U5XgraWxMYhIpLBFq/aQ3Nr5PQkg4rzmTt2YJIi6v0yM6n3Hwl5Rf5yJXURkYR5PKDp/fIZw8nO0mik7srMpA6QFXC9prU58XGIiGSgPYfreX2rf3mQK2ar1/uJyOCkHtAE36Zr6iIiifDEit1Er646prSQ6WX9khNQmlBSD9fa5C8TEZEeFzTX+/yZwzFNBHZCMjep11T4y1oaEh+HiEiG2bSvmtW7j/jKNeHMicvcpC4iIkkRVEufMaIfYwcFdGCWLsncpF460V+mjnIiInHlnAtM6gtmaWx6T8jcpJ4VsFCAhrSJiMTV2zsPsaOqLqLMDC6fMSxJEaWXDE7qAUPalNRFROJqUUAt/V3jBjK4pCAJ0aSfDE7qQUPalNRFROKlpbWNJ1eq6T2elNTD1R5IfBwiIhni5c2VHKiJHDqcl5PFvGlDkxRR+sncpF7+hr8srzDxcYiIZIigaWEvnDyYkoKAPk7SLZmb1EVEJGHqm1p5atVeX7nGpveszE3qo8/1lzmtpy4iEg9L1lVQ2xQ5FXdxQQ7nTxqcpIjSU+YmdQt46UrqIiJxETQ2/dJpQynIDRiJJN2mpB5OSV1EpMcdqmvi+fX7fOXq9d7zlNTDRS8ZJCIiJ+xvq/bS3Br5+Tq4OJ+5YwcmKaL0paQeTjV1EZEeF9Tr/fKZw8nO0opsPU1JPZySuohIj9pzuJ7Xt1b5ytXrPT6U1MMpqYuI9KgnVuz2XdkcW1rI9LJ+yQkozSmph1NSFxHpUY+97e/1Pn/WcMzU9B4PGZzUA95QSuoiIj1mY0U1a/Yc8ZWr13v8ZHBSV01dRCSeFq3w19JnjujHmFJNyR0vSurhmmoSH4eISBpyzgVOODNftfS4UlIPV7E68XGIiKSht3ceYkdVXURZlsHlM4YlKaLMkLlJffM//GUnjU54GCIi6ejxt/1j0981rpTBJQVJiCZzZG5SH32Ov6ytJfFxiIikmZbWNp5cucdXPl9j0+Muc5N6UK1cSV1E5IS9vLmSytqmiLK8nCzmTRuapIgyR+Ym9awcf5mSuojICQtqer9w8mBKCnKTEE1mUVIP19bqLxMRkZjVN7Xy1Oq9vnJNC5sYSurh9qxIfBwiImlkyboKapsiK0jFBTmcP2lwkiLKLJmb1IOGtB3anvg4RETSSNC0sJdOG0pBbnYSosk8mZvUKzf6y4ZMS3wcIiJp4lBdEy9s2Ocrv0ITziRM5ib1Mef5y9RRTkSk2xa/s5fm1sgl2QYX53Pm2IFJiijzZG5Sz+vrL1NSFxHptseX+3u9Xz5zONlZWpEtUTI3qWtIm4hIj9l9qJ43tlX5ytX0nlhK6uE0pE1EpFueWLEbF9nyztjSQqaVlSQnoAylpB5u/eLExyEikgaCV2Qbjpma3hMpc5N69FdKgNyA6+wiItKhjRXVrNlzxFe+QE3vCZe5ST0rYMzkoEmJj0NEpJcLqqXPHNGPMaWFSYgms2VuUi8KmN1IHeVERLrEOcfjK/y93lVLT47MTerqKCcicsLe2nGInVX1EWVZBu+fMSxJEWU2JfVwrc2Jj0NEpBdbFDA2/V3jShlcUpCEaERJPVyjv6OHiIgEa25t48mVe3zlWpEteTI4qQd0lKupSHwcIiK91MubDlBZ2xRRlpeTxXunDU1SRJK5ST1HTUMiIidiUUCv9wsnD6akIDcJ0QjEOamb2TwzW29mm8zstoDtJ5vZc2b2tpmtNLPL4hlPhKIhweVB49dFRCRCfVMrT63e6ytXr/fkiltSN7Ns4F7gUmAqcL2ZTY3a7avAI8652cB1wE/jFU9AgMHl6gEvItKpZ9dWUNsU+XlZXJDD+ZMGJSkigfjW1M8ANjnntjjnmoCHgQVR+zjg6MTA/QB/W048BXWWc20JDUFEpDcKmnDmsmnDKMgN6K8kCROQ1XpMGbAz7H45cGbUPncAT5vZLUAhcFEc4/GzbCBqwhkldRGRDh2qa+KFDft85er1nnzxrKkHtW9HX7C+HljonBsBXAY8YGa+mMzsk2a2zMyW7d+/vwcjDHj5SuoiIh1a/M5emlsjP84HF+dz5tiBSYpIjopnUi8HRobdH4G/ef1G4BEA59yrQAFQGv1EzrlfOufmOOfmDBrUg9drlNRFRLrssYAJZ+bPHE52llZkS7Z4JvWlwAQzG2NmeXgd4RZF7bMDuBDAzKbgJfUerIp3QkldRKRLdh+q542tVb5y9XpPDXFL6s65FuCzwFPAWrxe7qvN7E4zmx/a7YvATWa2AngIuMG5BI4pU1IXEemSJ1b4O8iNHVTItLKSgL0l0eLZUQ7n3GJgcVTZ18NurwHOjmcMHQoa1qakLiLSrscCer0vmFmGtTdMWBIqc2eUg3Zq6pp8RkQkyIaKatbu8a+RoV7vqUNJPZpq6iIigR4P6CA3c2R/RpcWJiEaCaKkHs1pRjkRkWjOucAJZxbMVC09lSipR6v2LyMoIpLp3tpxiPKD9RFlWQbvnzksSRFJkMxO6kFLre7fkPg4RERSXFDT+9njSxlcrBUvU0lmJ3XfBHcE195FRDJYc2sbf13pb8Wcr6b3lJPZGeyUK/1lbc2Jj0NEJIW9vOkAlbVNEWV5OVnMmzY0SRFJezI7qecG9Nhsa/GXiYhksKAOchdNGUxxQW4SopGOZHZSzwpYIlBJXUTkmPqmVp5avddXPn+mpoVNRZmd1LMDvmXWHkh8HCIiKeqZtRXUNUUO9S0uyOGCyT24uJb0mMxO6lkBs+S+eE/i4xARSVGLAnq9XzZtGPk5AS2dknSZndQb/NMdMnhK4uMQEUlBB2ubeH69f+FMTQubujI7qZed6i/Lzkt8HCIiKWjxqj20tEUO/R1Sks+ZYwcmKSLpTGYn9WGz/GWa+11EBAju9X75jOFkZ2lFtlSV2Uldc7+LiATadaieN7ZW+cqvmK1e76ksw5O61lMXEQnyxAp/LX3soEJOGV6ShGgkVpmd1IPGqSupi4i0syJbGRZUGZKUkdlJPbD5PWA+eBGRDLKhopq1e/yjg9TrPfUpqUdTTV1EMlzQimwzR/ZndGnA1NqSUpTUoympi0gGc84FNr1foVp6r6CkHq1Nvd9FJHO9teMg5QfrI8qyDN43Y1iSIpKuyPCkro5yIiLhgmrpZ48vZXBxQRKika7K8KSuIW0iIkc1t7bx5Mo9vvIFszQ2vbfI8KQe8PIPbk18HCIiKeClTQeoqm2KKMvLyeK9pwxJUkTSVUrqQVqagstFRNLYooCm94umDKa4IGCZaklJmZ3UC0uDy4+UJzYOEZEkq2tq4anVe33lanrvXTI7qee1M+ayTdfVRSSzPLt2H3VNkaN/SgpyOH/SoCRFJN2R2UkdYOB4f1lbS+LjEBFJosff9k84c9n0YeTnBIwSkpSlpJ6d7y9TUheRDHKwtokXNuz3lc/XhDO9jpJ60KIuSuoikkEWr9pDS1vkuhdDSvI5c8zAJEUk3aWknpXjL6tYnfg4RESS5PG3/b3e588cTnaWVmTrbZTUg5L68gcTH4eISBLsOlTPG9uqfOXq9d47KalX+2dPIr848XGIiCTBEyv8tfSxgwo5ZXhJEqKRE6WkPutD/jJdUxeRDPFYQK/3K2aVYUHTaEvKU1IfeYa/TCu1iUgGWL+3mnV7q33l82eq13tvpaQedE1dNXURyQCPL/fX0meN7M/o0nYm5pKUp6QemNRVUxeR9OacC1xmdYHGpvdqSupBSX3HK4mPQ0Qkgd7acZBdh+ojyrIM3jdjWJIikp6gpB40+YyISJp7LGBs+tnjSxlcXJCEaKSnKKkXDfaX5Wsoh4ikr+bWNv76jn84r8am935K6n1O8pc5rdImIunrpY0HqKptiijLz8nivacMSVJE0lOU1C3gFCipi0gaC+r1ftGUIRQX5CYhGulJSupK6iKSQeqaWnh6TYWvXCuypQcldSV1Eckgz6ypoK4pcthuSUEO508alKSIpCcpqSupi0gGWRQwNv2y6cPIz9FIoHSgpK6kLiIZ4mBtEy9s2O8rV9N7+lBSV1IXkQzx13f20NLmIsqGlhRw5piBSYpIepqSensrETkXXC4i0ksFNb1fPnMY2VlakS1dBMyRepyZVQNB2c0A55xLj1laLMtfO3dtYLrGJCLpofxgHW9sq/KVa8KZ9NJhUnfOFScqkKRqL6mjpC4i6eGJFf4Z5MYNKuSU4elRNxNPZzX1AR1td875v/b1RrquLiJpLmjCmQWzyrD2LkFKr9TZNfU3gWWh39E/yzp7cjObZ2brzWyTmd3Wzj4fNLM1ZrbazH7ftfB7SFBSX7Mo8XGIiMTB+r3VrNtb7SvXMqvpp7Pm9zHdfWIzywbuBS4GyoGlZrbIObcmbJ8JwO3A2c65g2YWsLpKArQ2+8vWLoIZ1yQ+FhGRHhZUS581sj+jBhYmIRqJpw6TejgzOwmYABxbl88592IHDzkD2OSc2xJ6/MPAAmBN2D43Afc65w6Gnm9f7KH3INfqL2uuS3wcIiI9rK3N8XhAr3fV0tNTTEPazOwTwIvAU8A3Q7/v6ORhZcDOsPvlobJwE4GJZvaymb1mZvPaOf4nzWyZmS3bv98/ccIJm3e3v6ytpeePIyKSYG/tOMiuQ/URZVkG75+hpJ6OYh2nfitwOrDdOXcBMBvoLLsG9b6IHh6Xg1f7Px+4HviVmfX3Pci5Xzrn5jjn5gwaFIf5iQdP9Ze1BdTeRUR6maBa+tnjSxlUnJ+EaCTeYk3qDc65BgAzy3fOrQMmdfKYcmBk2P0RQPS7qxx43DnX7JzbCqzHS/KJlRVwFUJJXUR6uebWNv76jn8o2xUam562Yk3q5aEa9GPAM2b2OP4EHW0pMMHMxphZHnAdEN2l/DHgAgAzK8Vrjt8Sa/A9JitgPHpzbcLDEBHpSS9tPEBVbVNEWX5OFpecMiRJEUm8xdRRzjl3ZejmHWb2HNAP+Hsnj2kxs8/iXX/PBn7jnFttZncCy5xzi0LbLjGzNUAr8B/OucpuvpbuC6qp71mR8DBERHpSUK/3i6YMobggNwnRSCLElNTNbC6w2jlX7Zx7wcyK8a6rv97R45xzi4HFUWVfD7vtgC+EfkREpIfUNbXw9JoKX7l6vae3WJvffwbUhN2vDZWlh+Jh/rLCOHTIExFJkGfWVFDXFNk3qKQgh/Mm6bMtncWa1C1UqwbAOddGF8a4p7ycgF6g6ignIr1YUK/3980YRn6O1rRIZ7Em9S1m9jkzyw393EoyOrTFS1BHOSV1EemlqmqbeHGDf9Tx/Jnq9Z7uYk3qnwLeBezCG4Z2JvDJeAWVcEFLrAbNMici0gssfmcPLW2R04IMLSngzDEdrtElaSDW3u/78IakpaegmnpTjb9MRKQXCOr1Pn/WcLKytCJbuot1mtiJZrbEzFaF7s8ws6/GN7QEChrSJiLSC5UfrGPptoO+8vkz1es9E8Ta/P5/eKupNQM451aSTjX39pJ6TXLWlxER6a5FK/wd5MYNKuSU4SVJiEYSLdYqal/n3BtmEU036bPiSVDzO0BzfXC5iEiKWhTQ6/2KWWVEfX7HRXNzM+Xl5TQ0NMT9WOmooKCAESNGkJvb/cmBYk3qB8xsHKEFWczsasA/oXBvllvonxrWtSUnFhGRbli39wjr9lb7yucnaMKZ8vJyiouLGT16dEK+RKQT5xyVlZWUl5czZsyYbj9PrM3vnwF+AUw2s13A54Gbu33UVFQ02F+mpC4ivUjQ2PTZJ/dn1MDChBy/oaGBgQMHKqF3g5kxcODAE27liLX3+xbgIjMrBLKcc/6vgr1dUBO8krqI9BJtbS6w6X1BgjvIKaF3X0+cu05r6maWHVpBDedcLdBoZjeZ2doTPnoqsYBToaQuIr3EWzsOsutQZD+g7CzjfTPU6z2TdJjUzew6oApYaWYvmNkFeDPJXQZ8KAHxJU5QUtesciLSSzwWMDb97PGlDCoOmAY7jWVnZzNr1iymTZvGNddcQ11d3Qk/57Jly/jc5z7X7vbdu3dz9dVXn/BxekJnNfWvAqc554YD/4633OotzrkrnXNvxT26RAqcVU41dRFJfc2tbfx1pb/vcqKb3lNBnz59WL58OatWrSIvL4+f//znEdudc7S1de2zfc6cOfz4xz9ud/vw4cN59NFHuxVvT+vsmnqTc24TgHPuLTPb6pz7SwLiSjw1v4tIL/XPjfs5WNccUZafk8UlpwxJSjyjb/tr3I+x7e73dbrPueeey8qVK9m2bRuXXnopF1xwAa+++iqPPfYY69ev5xvf+AaNjY2MGzeO++67j6KiIpYuXcqtt95KbW0t+fn5LFmyhDfffJN77rmHJ598khdeeIFbb70V8K6Bv/jii1RWVvL+97+fVatW0dDQwM0338yyZcvIycnhBz/4ARdccAELFy5k0aJF1NXVsXnzZq688kq++93v9vh56SypDzaz8LXOi8LvO+d+0OMRJUtQBwXN/y7GbdVXAAAgAElEQVQivUBQr/eLpg6huKD74517u5aWFv72t78xb948ANavX899993HT3/6Uw4cOMBdd93Fs88+S2FhId/5znf4wQ9+wG233ca1117LH/7wB04//XSOHDlCnz59Ip73nnvu4d577+Xss8+mpqaGgoKCiO333nsvAO+88w7r1q3jkksuYcOGDQAsX76ct99+m/z8fCZNmsQtt9zCyJEje/R1d5bU/w8o7uB++lDvdxHpheqaWnh6dYWvPBOb3gHq6+uZNWsW4NXUb7zxRnbv3s2oUaOYO3cuAK+99hpr1qzh7LPPBqCpqYmzzjqL9evXM2zYME4//XQASkr8s/CdffbZfOELX+BDH/oQV111FSNGjIjY/tJLL3HLLbcAMHnyZEaNGnUsqV944YX069cPgKlTp7J9+/aEJ/UNwNPOucoePWoqCmx+d/4yEZEU8syaCuqbI1sV+/XJ5fxJAXNvZICj19SjFRYeH6vvnOPiiy/moYceithn5cqVnQ4ru+2223jf+97H4sWLmTt3Ls8++2xEbd11kDfy8493WszOzqalpecnZu2so9wo4I9m9k8zu8PMzrR0HYQY2Ps9fWbCFZH0FNT0ftn0oeTlxDq3WOaZO3cuL7/8Mps2bQKgrq6ODRs2MHnyZHbv3s3SpUsBqK6u9iXezZs3M336dL785S8zZ84c1q1bF7H93e9+Nw8++CAAGzZsYMeOHUyaNCkBr8rTYU3dOXc3cLeZFQMXAR8Hfh4ao/534CnnnL/dpzcKSuo7X4eT5yY+FhGRGFTVNvHihv2+8vkzy5IQzXGxdGJLpkGDBrFw4UKuv/56GhsbAbjrrruYOHEif/jDH7jllluor6+nT58+PPvssxGP/Z//+R+ee+45srOzmTp1Kpdeeil79hwfefDpT3+aT33qU0yfPp2cnBwWLlwYUUOPN+uoqaDdB5lNBS4FLnHOvbfHo+rAnDlz3LJly3r+ib8zBuqrIssu+iac8/meP5aISA944LXtfO2xVRFlQ0sKeOW29yRl7fS1a9cyZcqUhB83nQSdQzN70zk3J5bHx7yQuJmV4TXHH33MUufc92N9fMoLWn61vdXbRERSwKKACWfmzxqelIQuqSGmpG5m3wGuBdYAR3tkOODFOMWVeDOvg1eiJhfQNXURSVHlB+tYuu2gr3x+hvZ6F0+sNfUrgEnOucZ4BpNUQTV1JXURSVGvb6nylY0fXMQpw/3DsCRzxNo9cguQ3rMYBCZ1TT4jIqmpotq/ROe5E0q1SlqGi7WmXgcsN7MlwLHaunOu/Rnue5vsgO8s9f6mLRGRVFBV0+Qry7TFW8Qv1qS+KPSTvoI6xW161l8mIpICqmr9SX1gYV4SIpFUElNSd8791szygImhovXOueaOHtPrVG7xl415d+LjEBGJQWVAUh9QqJp6dnY206dPp6WlhTFjxvDAAw/Qv3//Hnv+hQsXsmzZMn7yk59wxx13UFRUxJe+9KUee/4TFdM1dTM7H9gI3Av8FNhgZumV8U4a5S/T3O8ikqKCauoDVFOPWHp1wIABxxZYyRSxNr9/H2+imfUAZjYReAg4LV6BJVzxUH9Zq3q/i0hqqqzxD0ZKmeb3O/ol4BiHO93lrLPOYuXKlcfuf+973+ORRx6hsbGRK6+8km9+85sA3H///dxzzz2YGTNmzOCBBx7giSee4K677qKpqYmBAwfy4IMPMmRIcpay7YpYk3ru0YQO4JzbYGbp1RteQ9pEpJdwzgU3vxelSFJPAa2trSxZsoQbb7wRgKeffpqNGzfyxhtv4Jxj/vz5vPjiiwwcOJBvfetbvPzyy5SWllJV5Q0VPOecc3jttdcwM371q1/x3e9+l+9/P/XnW4s1qS8zs18DD4Tufwh4Mz4hJYmSuoj0EnVNrTS2RF4ezMvOojg/5klC09bRpVe3bdvGaaedxsUXXwx4Sf3pp59m9uzZANTU1LBx40ZWrFjB1VdfTWlpKQADBgwAoLy8nGuvvZY9e/bQ1NTEmDFjkvOCuijWceo3A6uBzwG34s0s96l4BZUUQUl970p/mYhIkrV3PV1j1I9fU9++fTtNTU3Hrqk757j99ttZvnw5y5cvZ9OmTdx444045wLP2y233MJnP/tZ3nnnHX7xi1/Q0OCfFyAVxZTUnXONzrkfOOeucs5d6Zz7YdrNLheU1A9sSHwcIiKdCO75rqb3cP369ePHP/4x99xzD83Nzbz3ve/lN7/5DTU1NQDs2rWLffv2ceGFF/LII49QWVkJcKz5/fDhw5SVeavd/fa3v03Oi+iGDttqzOwR59wHzewdvLneIzjnZsQtskRrPOIvG3lm4uMQEelEVW1AJ7lUup4eQye2RJg9ezYzZ87k4Ycf5iMf+Qhr167lrLPOAqCoqIjf/e53nHLKKXzlK1/hvPPOIzs7m9mzZ7Nw4ULuuOMOrrnmGsrKypg7dy5bt25N8quJTYdLr5rZMOfcHjMLGO8FzrntcYusHXFbenXnG/DriyPLyubATUt6/lgiIifgkWU7+c9HIy8PLpg1nB9dNztJEXm09OqJO9GlVztsfnfOHV35/QCwM5TE84GZwO6uh5vCgqaJbUuv+XVEJD1ojLq0J9aOci8CBaE11ZcAHwMWxiuopNCCLiLSSwQl9dIizSYnsSd1c87VAVcB/+ucuxKYGr+wkiAoqbeqpi4iqacyYDGXVKmpd3RJVzrWE+cu5qRuZmfhjU//a6gsvQZEZgU0vx9Y7y8TEUmyoI5yqZDUCwoKqKysVGLvBucclZWVFBQUnNDzxJqYPw/cDvzFObfazMYCz53QkVNN0CptAG1tkBXrdx8RkfhL1RXaRowYQXl5Ofv37092KL1SQUEBI0aMOKHniHWVtheAF8Lub8GbiCZ9lAwPLm+uhfzixMYiItKBAyna/J6bm9trZl5LV52NU/8f59znzewJgsepz49bZImW004nE00VKyIpJrimro5y0nlN/ehc7/fEO5CU0Hcg1FVGlqkHvIikkPqmVuqbIz+XcrKMkj7p1c1JuqfDd4Fz7uiiLcuAeue8BcbNLBtvvHp60aIuIpLiKtvpJKd53wVi7/2+BOgbdr8P8GzPh5NkSuoikuI08Yx0JNakXuCcqzl6J3S7bwf7905BPeDrqhIfh4hIO4IWc0mped8lqWJN6rVmdurRO2Z2GlAfn5CSyAJOx/aXEx+HiEg7gieeSb+rodI9XRmn/kczOzrf+zDg2viElEQHt/nLghK9iEiSBK7QpuZ3CYl1nPpSM5sMTAIMWOecS785VE8+C3a8GlmmmZFEJIUENr8rqUtITNVQM+sLfBm41Tn3DjDazN4fw+Pmmdl6M9tkZrd1sN/VZubMLKal5eJm2MyAQiV1EUkdVUHN77qmLiGxti3fBzQBZ4XulwN3dfSA0LC3e4FL8RZ/ud7MfIvAmFkx3ux0r8cYSxwFDAnxRvGJiKSEVJ0iVlJDrEl9nHPuu0AzgHOunsAMGOEMYJNzbotzrgl4GFgQsN9/A98FGmKMJX6Crp+r+V1EUkhQ87s6yslRsSb1JjPrQ6gt2szGAf7eGpHKgJ1h98tDZceY2WxgpHPuyY6eyMw+aWbLzGxZXBcKCJq8QTV1EUkhGqcuHYk1qX8D+Dsw0swexJuM5j87eUxQTf5YtdfMsoAfAl/s7ODOuV865+Y45+YMGjQoxpB7imrqIpI6KmvU+13a12nvd/PmHlwHXAXMxUvWtzrnDnTy0HJgZNj9EcDusPvFwDTg+dD0hkOBRWY23zm3LOZX0JPU/C4iKayhuZXapsh537OzjH59cpMUkaSaTpO6c86Z2WPOudOAv3bhuZcCE8xsDLALuA74l7DnPQyUHr1vZs8DX0paQveC8Jep+V1EUkRQ0/tJffPIytK87+KJtfn9NTM7vStP7JxrAT4LPAWsBR5xzq02szvNLEWXbO3wioGISFKp57t0JtYZ5S4APmVm24BavOznnHMzOnqQc24xsDiq7Ovt7Ht+jLHEj5rfRSSFBfd8V1KX42JN6pfGNYpUEdj8rqQuIqkhaIpYTTwj4TpM6mZWAHwKGA+8A/w61KyengLneVdSF5HUELSYi5rfJVxn19R/C8zBS+iXAt+Pe0RJpY5yIpK61Pwunems+X2qc246gJn9Gngj/iElkZrfRSSFBc37PrBIs8nJcZ3V1I+txJbWze5HqfldRFKYVmiTznRWU59pZkdCtw3oE7p/tPd7SVyjSzg1v4tI6grsKKekLmE6TOrOuexEBZIS1PwuIilM49SlM7FOPpMZNKOciKSwoN7vqqlLOCX1CJpRTkRSU2NLK9WNkV2bzKB/XyV1OU5JPZxmlBORFHWwttlXNqBvHtma913CKKmHU/O7iKSoSnWSkxgoqUdQ87uIpKagTnJK6hIt1rnfM4Oa30UkyZxzVBxpZH1FNRv2Vnu/K6rZWFHj23eg5n2XKErq4TSkTUQSqKq2ifV7q9m4r5r1e73kvX5vNUcaYpvrSzV1iaakHi6opt7qb/ISEemKmsYWNkTVvNfvreFAjf86eVecPKBvD0Uo6UJJPUJATX3p/8H77kl8KCLS6zQ0t7J5f82xpH205r3rUH2PH6t/31zeN2N4jz+v9G5K6uEaDvvLyuYkPg4RSWktrW1sq6w7lrQ3VHg18G0HammL4xW7koIcJg0tZsaI/nx47ijK+veJ38GkV1JSD1c0yF+WX5z4OEQkJbS1OXYdqj+WtL3m8xo276uhqTV+w10LcrOYOKSYiUOKmTSkmIlDvd9DSvKxoL4/IiFK6uEGTvCXtaX/4nQimc45x/5qr8f58Zp3DRsrqqlrao3bcXOzjbGlRaGkXeQl8aHFjDypL1maVEa6QUk9XFbA6WiL3z+0iCTeobomNlTU+IaMHarzz9jWU8xg9MBCJg4piqh5jy4tJDdb04VIz1FSDxeY1FVTF+mN6ppa2BiQvCuOnFiP884M71dwLGkfrXmPG1REn7zMWvRSkkNJPVxQUq/ek/g4RCRmTS1tbDlQEzbO2+t1vqOqLq7HHVCYx6RQ0vaSdxEThhRTUpAb1+OKdERJPVxWwDfpwzsTH4eI+LS2ObZX1h5P3Pu8GvjWA7W0xLHLeVF+jtdsPjSy41ppUX7cjinSXUrq4YKa2vufnPg4RDKYc47dhxuON5mHfm/aV0NjS/x6nOflZDFhcOQ174lDixner0A9zqXXUFIPV9DPX5alpjSReDlQ0xg1y1o1GypqqGmMX1+W7CxjbGmh77r3yQP6ahlT6fWU1MMFLuii3u8iJ+pIQzMbo2ZZ21BRTWXAymM96eQBfY9d7z6avMeUFpKfo05rkp6U1MMFXVNv03rqIrFqaG5l076aiFnWNuytZvfhhrged0hJvm+ilvGDiyjM10ecZBa948NZQFJXTV3Ep7m1ja0HaiNWFttQUc32qrq4LmzYr08uk4ZGJu+JQ4ro31erlYmAknqkoJr6kV2Jj0MkRbS1OXYerIuYZW3D3mq2HKihuTV+2btvXjYThkTOsjZpSDGDijVNqkhHlNTDBY1TB29NdX2QSBpzzlFxpNE3UcvGihrqm+PXWpWXncW4wUVe8g7ruFbWv4+mSRXpBiX1cO0t3lJTAcVDExuLSJxU1TZ5a3uHz3O+t5ojDfHrcZ5lMLq0MKK3+cQhxYwe2JccTZMq0mOU1MPltrOMoaaKlV6oprHFS95hNe8NFTXsr47vNKll/ftEzLI2cYg3TWpBrnqci8Sbknq0wsFQuy+yzKkHvKSuhuZWNu+viZgidf3eanYdqo/rcUuL8o8PFQt1XJswuIhiTZMqkjRK6tFyC/xlSuqSAlpa29hWWRfZbF5RzbYDtcRxllSKC3Kiept7Pc4HappUkZSjpB4tcAIaJXVJnLY2x65D9RHjvNdX1LB5Xw1NrfF7LxbkZoUS9vGa98QhRQwt0TSpIr2Fknq0oKSuCWgkDpxz7K8O9TgPDRVbX1HNxopqapvi1+M8J8sYN+hob/PjQ8ZGnKRpUkV6OyX1aKqpSxxV1Tbx21e28eqWSjZUVHOorjluxzKDUcemST3e63z0wELyctTjXCQdKalHC5xVTkldTtzK8kN8fOFSDtT0/Hznw/oVRCbv0DSpffLU41wkkyipR1NNXeJgydoKPvv7t094IpeT+vqnSZ0wpJh+fdTjXESU1P20Upv0sAdf387XHlvVpR7qRfk5TBxSFFHznjCkmNKiPHVaE5F2KalHU01dekhbm+N7T6/nZ89vbnefvJwsJgwuihwyNrSY4f3U41xEuk5JPVqWkrqcuMaWVv7z0ZU8vnx34PYpw0r44bUzGT+oSNOkikiPUVKPFlRTr9oCw2cnPhbplQ7XNfPJB5bx+taqwO3nTijlpx86VTOviUiPUxUhWlBSX/5Q4uOQXqn8YB1X//yVdhP6NaeN4Dc3nK6ELiJxoZp6tIPb/WX5RYmPQ3qdVbsO87GFS9tdMOXfL5rI5y4cr2vlIhI3qqlHm3Gtv6xNvd+lY8+t38cHf/FqYELPyTLuuWYmt140QQldROJKST3a6LP9ZUrq0oGH39jBJ367jLqAqV2L8nO472Onc/VpI5IQmYhkGjW/R8sKOCVaT10COOf44TMb+PE/NgVuH1pSwH0fO50pw0oSHJmIZCol9WhZAdNqtsVvfm7pnZpa2rjtTyv589u7ArdPGlLMwo+fzrB+fRIcmYhkMiX1aEE19c3/SHwckrKONDRz8+/e5OVNlYHbzx4/kJ99+DRK1MNdRBIsrtfUzWyema03s01mdlvA9i+Y2RozW2lmS8xsVDzjiUlQUgctvyoA7D5UzzU/e7XdhH7VqWXcd8MZSugikhRxS+pmlg3cC1wKTAWuN7OpUbu9Dcxxzs0AHgW+G694YjZwQnB5a8+vrCW9y7q9R7jqp6+wvqI6cPvn3jOe718zU8uaikjSxPPT5wxgk3Nui3OuCXgYWBC+g3PuOedcXejua0DyuwgXDwku13X1jNbY0srH7lvK3iMNvm3ZWcbdV03nC5dM0pA1EUmqeCb1MmBn2P3yUFl7bgT+Fsd4Ypcf0FtZPeAz2utbqthz2J/QC/Oy+fVH53DdGScnISoRkUjx7CgXVGUJXHzSzD4MzAHOa2f7J4FPApx8cgI+PLMDrodqrHpGW7HzkK9sUHE+991wOtPK+iUhIhERv3jW1MuBkWH3RwC+JavM7CLgK8B851zg/JrOuV865+Y45+YMGjQoLsFGCOos11znL5OMsaLcn9Rvec94JXQRSSnxTOpLgQlmNsbM8oDrgEXhO5jZbOAXeAl9Xxxj6ZqgpP7YpxMfh6QE5xzLdx72lc8Y0T8J0YiItC9uSd051wJ8FngKWAs84pxbbWZ3mtn80G7fA4qAP5rZcjNb1M7TJVZQ8/u2f0JzfeJjkaTbc7iBAzWRjUi52caUYcVJikhEJFhcJ59xzi0GFkeVfT3s9kXxPH63DZkGB7f5yxuOQK5mCMs0KwOa3qcMKyE/J2D2QRGRJNKA2iAXfCW43KmzXCYKanqfqaZ3EUlBSupBhkyFPgP85eoBn5GCauozRqiDnIikHiX19uQV+ctUU884bW2Od8r9NfVZI1VTF5HUowVd2pMV8H1HNfWMUt/Uyp1Prqa6MXLiocK8bMYOCvjSJyKSZKqpt8cCTs2qPyU+DkmKNbuPcPlPXuKhN3b6tk0f0Y/sLE0HKyKpR0m9PUG18k3PJj4OSSjnHAtf3soVP32ZTftqAve5cHI76wOIiCSZmt/bE5TUs/MSH4ckTGVNI//x6Er+sa79eZDeN30YH33X6MQFJSLSBaqpt+eiO/xlWtQlbf1z437m/eif7Sb0/Jws7rpiGj/5l9laWlVEUpZq6u3pH7BwjJJ62mlqaeP7T6/nFy9uaXefyUOL+fH1s5k4RDPIiUhqU1JvT9D8770sqe870sDzG/bz/Pp97D7UwNjSQm6/bAqDivOTHVrSOed4dUsld/9tHSsDhqwd9dGzRnH7ZVMoyNXscSKS+pTU25MV8CF+xLfIXEppbXMs33mI59fv47n1+1i160jE9uU7D7H3SAO/v2lukiJMvrY2x9Nr9vKzF7YELqd61El9c/ne1TO5aKo6xYlI76Gk3p6gmnpNReLjiNH9r27jR89upLK2qcP9Xtlcyb4jDQwuKWh3H+ccZuk1ZKuxpZXH3t7FL17cwpb9tR3ue/b4gfzgg7MY0sE5EhFJRUrq7clp5wO9Zj8UJWBN9y5YsraCrz++Oub9d1TVBSb1leWH+MIjK9hRWce8aUP57yum0a9PwIp1vUhNYwu/f307v35pKxVHGjvcNyfL+OIlk/i3d48lS+PQRaQXUlJvz4CxweVN1UDqJPW2Nsf3nlrfpcfsOlTPnIDn+bcH3mTP4QYAFq3YzbB+Bdx+2ZQeijSxDtQ0svDlbdz/6jaONHTeF2LcoEK+/8FZmv5VRHo1JfX2BE0TC9DWltg4OvH0mgrW7a1ud3vfvGzqmiLH3O865F8XftP+mmMJ/ahfvLil1yX1qtomfvjMBh5ZtpPGls7/VtPKSrj5vPHMmzZUs8SJSK+npN6RgeOhclNkmUuNpL5pXw2/eXkrf36r3LdtbGkhN547hvMnDeavK3fz/xavi9i+OyCpH2znWvyBmkZKi3pHb/m6phbm/+Qlyg/6X1+0s8cP5ObzxnP2+IFp139ARDKXknpHLKAHfJJXamtrc9zxxGruf3V7u/t8Y/4pnDfRu0RQ1r+vb/vuQw2+svas2nWY8ycN7nqgSfCXt3d1mNDN4NJpQ/nUeeOYofXQRSQNKal3JGhY254VMDh5TdJPrNzdYUKfNbI/755Qeuz+8P7+DnG7AhLfxnbmOe9NSX3J2uDZ4PKys/jAaWXcdO5Yra4mImlNSb0jQTX1v/wbTP9g+9fc4+xPb+1qd1tRfg7/vWBaRHNy2Ul9fPsFNb//5e3g540e656qGppbeWXzAV/5De8azc3nj9PwNBHJCErqHcnzN10DULkRBk1KbCxAc2sbr22pDNx207ljuPGcsQztF5m8SgvzycvOoqn1eF+A6sYWDtc3U36wjqdXV/Dom+WBnecA3tnV/mxrqeTVzZU0NEf2dxhQmMfX3j9VHeBEJGMoqXdkzHmw83V/eVPHk5fEy5/fKqcpoEf3yjsuoaQgeDx5VpYxvH8B2yrrIspnfvPpmI6561A9B2ubOKkwdVeoa2tz/Oz5zb7y8ycOUkIXkYyi5aY68u7/CC5PQg/4/12ykS//6R1f+bkTSttN6EcN7+9vgu+KVbtTu7b++zd28Ma2Kl/5e6b0jr4AIiI9RTX1juTkwcgz/bX1BCzs0tjSyhtbq3hr+yH++ObOdnt1Ty/r1+lzlZ1gUn9n12HOnZA6E+6E23u4gbv/ts5XfvKAvlysedtFJMMoqXcmqLNc9R44HNCxrKAf5J947+q1e45ww31vdDqtaUFuFh84bUSnzxdrTX3qsBL2HK7nYF1zRPmqFL6u/sBr26hp9H/J+vZV08nP0cpqIpJZlNQ7EzSs7Y83tLNvDky9Aq74KeR0b8KW1jbH5x56u9OEDvDQTXMZF8MQrdGlwR3+srOMM8cM4OKpQ7h46hBGnNSXpduquObnr0bsl8o94JduO+gr++CcEZw9vjRgbxGR9Kak3pmg1dra09YCqx6F4bPhXZ/t1uGeXLm73THj4ZZ+5aKY10V/z+QhFOXnRNRozxlfyt0fmM6IkyIT/tRhJZiBc8fLdlTVcbiumX59U29xlze2+q+lX3/GyUmIREQk+ZTUO1NQ0vXH7Hi1W0m9ubWNHz27sd3tfXKzede4gfz7xRNjTugA/frk8ujNZ/HjJRupbmjhytllXDm7LHB61ML8HMaWFrI5annSVbsPJ73229rmqG1qoa6xlZrGFp5cGby+fZ88NbuLSGZSUu/MxHmw5vGuPaal86bzIL97bTtbDviHy/3Heydx3sRBTB5aTE529wYsTB5awk8/dFpM+04v6+dP6ru6ntRb2xw1jS3UNbVQ29hCTWMrtY3e7dqm4/frwrbVNHn3a0OJu7bJu13b2EJ9c+dT9JrBsJIT6xgoItJbKal3Zub1UH8QVjwEtQETv7Q0QH1UE3Bbs3+/Tjjn+NU/t/rK3zVuIJ+5YHyXn+9ETCvrx2PLI2vB3/7bOvrmZVPbFEq+x5Lz8URd09galsBbfJPBJMJ1p49MycsEIiKJoKTeGTM46zPeT5CtL8JvL48sa+36kLcDNU2Bs7rddunkLj/XiZrWzjC5rz2+OsGRdM1Vp5Zx54JpyQ5DRCRplNRPVFBHuoZDUP5ml56mtrKGmRa5zGtpUT4z2Az+1VW7Livbm9o2t/Om6VOGd6MfQZJ94pwx/NdlU8jSDHIiksGU1E9UVkBTb8Uq+NV7uvQ0o4HHo/u+NQO/6mZcQfKK4AO/gkmXdrhbcUEuowf29U0tmwr65mVTmJ9DUX4OffOyKS3K58rZZSyYNVzrootIxlNSP1HZvegUNtXAE7fCuAu92fI6MGNE/x5J6oWhJOz9ZFOYF0rI+TkUhe4f2xZK1oFl+Tn0zc1WTVxEpAO9KCOlqD4Dkh1B19RUwN53YETHPeHPmziIRSv8Q8ZuOncMffOOJ9rCsMRcFJaIlYRFRBJPSf1EnTQKyk6DXV27hp5U5Us7TeoLZg3nyZW7eW79fgA+cOoI7v7AdHK7OaRORETiT0m9J3zoUXj+btj9FrR1PpY6XHVjC5v3B88gN6OsPydc0a07AId2RJaVLwU+1eHDcrKzuO9jZ7B2zxHyc7IYU1qoa9YiIilOSb0n9B0Al323Sw9xznHz797i71v2Bm6//dLJzDpv3InHtvWf8Nv3R5aVL4354VOG9b6e8CIimUpJPQn2Vzfymd+/FThvOXhDyv6tJxI6ePPQW1bkGvCHtsPKR9of3lbQD0acAbkFPRODiIgkhJJ6giSCob0AAAtuSURBVK3be4TL//clmltdu/t8c/4pPXfA/CIYfApUvBNZ/uebOn5c4WD4+N9hYA99uRARkbhTr6cE2bSvmo/8+nXm/c8/O0zoP75+NnNG93CP+hFzuv6Y2n3wp09ELtcmIiIpTTX1OFu9+zAPvLqdh5fu7HC/mSP789BNZ9I3Lw5/kpFnwpv3df1xu9+CdU/ClMs731dERJJOSb0HOOfYUFFDQ3Mrk4YWU5CbzVs7DvJ/L27hb6uCO8KFu/GcMXwlnlOcTp0PS+6E6uClSjv0j7tg0mXeNLMiIpLSlNSj7Kyq49E3yykuyOEDp47gpML2Z1470tDMI0t38tAbO44tVZqXncXkYcWsLD/c6bFysowvXDKRT58f51XY8grhE8/C6z+DKv9KcMc018PmJZFl+9fByj/ArH+Jb4wiInLCzPWya6Zz5sxxy5Yt6/HnLT9Yxznfec5XvvbOefTJi6ylVjc0872n1nP/q9u7fbyrTi3j8xdO5OSBfbv9HD3OObjvMtjxSmR5v5PhlmWQEz05vYiIxJuZvemci6lzlJI6UNfUwtSvP9Xu9sumD+W2eVMY2q+A7/59Hfe/tp2mlq6vFd43L5t/OeNkPvnusQwuSdHhYjteg9+8119+6r/CkOmJj0dEMk9WFpSM8Ebf9B/V6VoV6U5JvQucc4y5fXGPPV+QvnnZ/PtFE/nw3FG+Wn9KevCDsLH9LzkiIgljWdD/ZBgwzkvyx36P9RJ+b1pUq5u6ktTT/2x0Ip4JvbQon389axQ3nD2akoKAJVpT1YVfU1IXkdTg2uDgNu8nus9PVo6X2KOT/cBx0G9kRnbwzdik/tLGA3z416/H7fk/PPdkbr90CoX5vfAUD50O0z4Aq/6U7EhERNrX1gJVm72faFm5cNLosIQ/9njiLxnhNfGnoV6YcU5Ma5vjv/78Dn9Y1vG48VicNXYgX798Kgdrm7j9L++w53AD544v5T/nTWbS0OIeiDaJLr4Ttr/avWFwIiLJ1tYMlRu9n2jZ+TBgjD/ZDxgHxcN6dcLPuKSenWU0t3W9k1u0hR87nfMnDT52//kvnU9rmyMnXZYm7TcCPvEMrP4LHOx+L38RkS5rroeDW6FyM9R0PtdHl7U2esN196/zb8vpE2rCH+u/jl80BFJ8tcqM7Ch3qK6JWXc+E7jt9f+6EOdg7reXBG6/5T3j+ex7xpOfk3nXakREEq6xBqq2eE3slZu925WhJvfa/YmNJa8orIY/LvJ3YWncEn7K9H43s3nAj4Bs4FfOubujtucD9wOnAZXAtc65bR09Z0/1fv/O39fxs+cjr8O8evt7GNbPv3JZW5ujsraJPnnZFPXGa+QiIumo4XBYkg9L9pWboT54Fcy4yS853kkvPNmPmHPCyT4ler+bWTZwL3AxUA4sNbNFzrk1YbvdCBx0zo03s+uA7wDXxiumcF+eN5n91Y2MH1zER88a3eFQs6wsY1CxJl4REUkpBf285aWHz/Zvqz8IleE1/LDfDZ3P+NlljUdgz3Lv56g+A+DLHcziGQfxrHaeAWxyzm0BMLOHgQVAeFJfANwRuv0o8BMzM5egawL3XDMzEYcREZFE63MSjDjN+wnnHNRVBSf7yi3QVN1zMSRh6ep4JvUyILyLeTlwZnv7OOdazOwwMBA4EL6TmX0S+CTAySefHK94RUQk3ZlB4UDvZ+QZkduc867TByX7qi3QXNu1Yw1Ir6QedBEhugYeyz44534J/BK8a+onHpqIiEgUMyga7P2MOitym3NQvTeghh9K+C0N/udLs5p6OTAy7P4IIHrQ89F9ys0sB+gHJLh3g4iISCfMoGSY9zP6nMhtbW3enB7Hknyodj/81ISHGc+kvhSYYGZjgF3AdUD0+p2LgI8CrwJXA/9I1PV0ERGRHpGV5c3t0W8EjD0vqaHELamHrpF/FngKb0jbb5xzq83sTmCZc24R8GvgATPbhFdDvy5e8YiIiKS7uA66ds4tBhZHlX097HYDcE08YxAREckUaTKnqYiIiCipi4iIpAkldRERkTShpC4iIpImlNRFRETShJK6iIhImlBSFxERSRNK6iIiImlCSV1ERCRNWG+bat3M9gPbe+CpSola4lUSSuc/uXT+k0vnP7l62/kf5ZwbFMuOvS6p9xQzW+acm5PsODKVzn9y6fwnl85/cqXz+Vfzu4iISJpQUhcREUkTmZzUf5nsADKczn9y6fwnl85/cqXt+c/Ya+oiIiLpJpNr6iIiImklI5O6mc0zs/VmtsnMbkt2PJnGzLaZ2TtmttzMliU7nnRnZr8xs31mtiqsbICZPWNmG0O/T0pmjOmsnfN/h5ntCv0PLDezy5IZYzozs5Fm9pyZrTWz1WZ2a6g8Lf8HMi6pm1k2cC9wKTAVuN7MpiY3qox0gXNuVroOK0kxC4F5UWW3AUuccxOAJaH7Eh8L8Z9/gB+G/gdmOecWJzimTNICfNE5NwWYC3wm9Jmflv8DGZfUgTOATc65Lc65JuBhYEGSYxKJG+fci0BVVPEC4Leh278FrkhoUBmknfMvCeKc2+Oceyt0uxpYC5SRpv8DmZjUy4CdYffLQ2WSOA542szeNLNPJjuYDDXEObcH/n97dxdiRR3Gcfz7y7WSLIIoCCo2TYoi0CgitnIh66K7CKmbMK8qFNuLrrqpLoq6UAqMBCvowqIXe5EikC6UiDJ72Vwp8aYI0dYIJAgrcp8u5pmYTrsL0Zmd+M/vA8uZl//Mec7AnGf/M2f+T/WlB1zQcTx9tFHSgbw8X8Sl3/87SaPAKmAfhZ4DfUzqmmWZHwFYWGMRcQ3VLZANkm7uOiCzBfYcsBxYCRwDNncbTvkkLQV2AhMR8XPX8bSlj0n9CHBxY/4i4GhHsfRSRBzN1+PAW1S3RGxhTUu6ECBfj3ccT69ExHREnIqIGWA7PgdaJWkxVULfERFv5uIiz4E+JvX9wApJl0o6Hbgb2NVxTL0h6SxJZ9fTwG3Awfm3shbsAtbl9DrgnQ5j6Z06maQ78DnQGkkCXgC+iYgtjVVFngO9HHwmHx95GlgEvBgRj3ccUm9IWkbVOwcYAV728W+XpFeAcarKVNPAI8DbwGvAJcD3wNqI8I+5WjDH8R+nuvQewHfAffX9XRsuSTcCHwJTwEwufpjqvnpx50Avk7qZmVmJ+nj53czMrEhO6mZmZoVwUjczMyuEk7qZmVkhnNTNzMwK4aRuVgBJ5zUqfv3QqAB2QtLXLbzfuKR3/+U2eyT9o4CPpHslbR1edGb95aRuVoCI+Kmu+AVsIyuAUT0LPTP/1iBppO0Yzax9Tupm5VskaXvWkt4taQn81XN+QtJe4EFJ50vaKWl//o1lu9WNqwBf1iMCAkslvSHpkKQdOXIXkm7JdlNZrOSMwYAkrZd0ON97bIGOg1nxnNTNyrcCeDYirgJOAHc21p0bEasjYjPwDFUP/7ps83y2eQjYkD3/m4CTuXwVMAFcCSwDxiSdSVU//K6IuJpq1MAHmsHkEKmPUSXzW3N7MxsCJ3Wz8n0bEZM5/Tkw2lj3amN6DbBV0iTVuNjnZK/8I2CLpE1U/wT8ke0/jYgjWZRkMvd7eb7f4WzzEjBYhe96YE9E/BgRvw/EYGb/ge+jmZXvt8b0KWBJY/6XxvRpwA0RcZK/e1LSe8DtwCeS1syx3xFmL208G49PbdYC99TNrLYb2FjPSFqZr8sjYioingI+A66YZx+HgFFJl+X8PcDegTb7gPH8xf5iYO2wPoBZ3zmpm1ltE3CtpAP5GNz9uXxC0kFJX1HdT39/rh1ExK/AeuB1SXVVrG0DbY4BjwIfAx8AXwz7g5j1lau0mZmZFcI9dTMzs0I4qZuZmRXCSd3MzKwQTupmZmaFcFI3MzMrhJO6mZlZIZzUzczMCuGkbmZmVog/ATY80U7dTawvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "valid_x_predictions = lstm_autoencoder.predict(X_valid_scaled)\n",
    "mse = np.mean(np.power(flatten(X_valid_scaled) - flatten(valid_x_predictions), 2), axis=1)\n",
    "\n",
    "error_df = pd.DataFrame({'Reconstruction_error': mse,\n",
    "                        'True_class': y_valid.tolist()})\n",
    "\n",
    "precision_rt, recall_rt, threshold_rt = precision_recall_curve(error_df.True_class, error_df.Reconstruction_error)\n",
    "plt.plot(threshold_rt, precision_rt[1:], label=\"Precision\",linewidth=5)\n",
    "plt.plot(threshold_rt, recall_rt[1:], label=\"Recall\",linewidth=5)\n",
    "plt.title('Precision and recall for different threshold values')\n",
    "plt.xlabel('Threshold')\n",
    "plt.ylabel('Precision/Recall')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAGDCAYAAAA2xlnwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xt8nGWd///XZyZpEkp6SsuhhdIStpSWagvl1KIUsFBcvwsipwUEBEFFRSlWxV1X1pWfLmJRVlYEOakoJ/EsZQsISFqQchAoLaWhhbb0RHpISpM2M3P9/rjvmd4zmZlMJjOZzOT9fDzyyMw9M/f9uY+f+7ru675uc84hIiIilSNU6gBERESksJTcRUREKoySu4iISIVRchcREakwSu4iIiIVRsldRESkwii5ixSYmS01s1mljqNYzOxzZrbRzHaYWUMRxn+3mX3Hf/0hM3sj8NmhZvaSmbWZ2VVmVmdmfzSz7Wb2YKFjKSYzW21mHyl1HFKZqkodgJQfM1sN7AtEgR3AAuALzrkdpYwrHTO7DjjEOXdhkcZ/N7DWOffv8WHOucnFmFZ/YGbVwHzgWOfcP4o9Pefc34BDA4O+CjzpnJvmx/NJvG2xwTkXKXY8qczMAf/knFvZ19MWyUYld8nX/3PO7Q1MBaYB15Y4nryYp+L2AzPrcuKeblhPx4GXSGuBpXnEVIhlfVDKtA8CVuST2Hu6PETKScUd1KRvOec2AI/iJXkAzKzGzG40s3f86ttbzawu8PnpZvaymbWaWbOZzfGHjzazP5jZFjNbaWaXB35znZk9YGY/96tkl5rZ9MDnXzOzdf5nb5jZyf54vwGc61ch/8P/7pNmdr2ZNQE7gYNTq0j96f0y8P54M1tkZtvMbI2ZXWJmVwAXAF/1x/9H/7uJcfnL4odm9q7/90Mzq/E/m2Vma83sGjPbZGbrzexTmZa1mQ01szv8760zs++YWdj/7BIzazKzm8xsC3BdhmEhM/t3M3vbn+bPzWyoP45xZubM7DIzewd4ImX6E4B4Ffk2M3vCHz7DzJ73q8afN7MZgd90WdZp5muamb3or7v78U4e4p/NMrO1/usngBOBH/vL+9fAfwTW72X+9y41s2VmttXMHjWzgwLjc2b2eTN7E3jTHzbRzBb6290bZnZO4Pt3m9ktZvZnP77nzKzR/+xp/2v/8Kd/bob1drkfT5uZvW5mR6T5ztFmttjfvtab2Y/NbJD/mfnrcJO/jF8xs8P9zz7qj7PN3ya+Ehjnx8zbz7b52+4HAp912V/SxS5lzDmnP/316A9YDXzEf30A8Crwo8DnPwT+AIwA6oE/At/1Pzsa2A7Mxju5HANM9D97CvhfvIP7VGAzcLL/2XVAB/BRIAx8F3jW/+xQYA0w2n8/DmgM/O6XKfE/CbwDTMa7NFUdnKfU3wFjgTbgX/3vNgBT/c/uBr6TZfl8G3gW2AcYBSwC/sv/bBYQ8b9T7c/bTmB4huX+O+CnwGB/fH8HPuN/dok/ri/681SXYdilwEq8JLs38DDwi8Byc8DP/WnUpYkh/p0q//0IYCvwSX8a/+q/b8i0rFPGNwh4G7jaXwZnAZ3xZeovo7Up6+7T6daT//4Mf/4O86f378CiwOcOWOjHXefP5xrgU/73jwDeAyYH1u8WvO22CrgXuC9lfIdk2VfOBtYBRwEGHAIclGY7ORI41p/GOGAZ8GX/s1OBF4Bh/jgOA/b3P1sPfMh/PRw4wn99BLAJOAZvf7nYn14NWfYX/VXOX8kD0F/5/fkHiR14Cc8BjwPD/M8MeD94sACOA1b5r38K3JRmnAfiXcOvDwz7LnC3//o64LHAZ5OAdv/1If6B7CN0TR5JB39/2JPAt9PMU6bkfi3w2wzL4m6yJ/dm4KOBz04FVvuvZwHt+InSH7YJ73p26nT2BXYRSLh4ifSv/utLgHdSfpNu2OPAlYH3h+Il03hSccDBWdZ9/Dvx5P5J4O8p31kMXJJpWad898PAu4AFhi0i/+T+CHBZ4H0I74TpIP+9A04KfH4u8LeUmH4KfCuwfn8W+OyjwPLA++6S+6PAl7LsRx/J8NmX49sccBKwAi/5h1K+9w7wGWBIyvCf4J9EBoa9AZxAlv1Ff5Xzp2p5ydcZzrl6vIPvRGCkP3wUsBfwgl8duA2vwd0o//MD8RJeqtHAFudcW2DY23gl+7gNgdc7gVozq3JeY6Yv4x3oN5nZfWY2upv413TzeVCmmHMxGm8+4t72h8W1uOTrxTvxStSpDsIr2a4PLNef4pXg49LNU+qwdPFU4Z08ZBtPJqnji48zuN6yjW80sM752Sfw+3wdBPwosIy24J1wZornIOCY+Pf931wA7Bf4Tup2l279ZJLTtmNmE8zsT2a2wcxagf8Pf59yzj0B/Bi4BdhoZreZ2RD/p5/AO+F428yeMrPjAvN1Tcp8HYhXWs9nf5Eyo+QuveKcewqvdHOjP+g9vNLoZOfcMP9vqPMa34F3YG1MM6p3gRFmVh8YNhavSjOXOH7lnDse76DmgP+Of5TpJynv38c7KYkLHtwzxZxt/HHv+jHFjfWH9dQavJL7yMByHeKSW+aniyV1WLp4IsDGbsaTSer44uMMrrds41sPjDEzS/l9vtbgXaoYFvirc84tyhDPGuCplO/v7Zz7XC9iSI0n07YT9BNgOV7L+yF4bUUSy8Q5d7Nz7ki8yxsTgHn+8Oedc6fjneT9DnggMN3rU+ZrL+fcr/3fZdpfpEIouUsh/BCYbWZTnXMx4HbgJjPbB8DMxpjZqf537wA+ZV6Dt5D/2UTn3Bq86tjvmlmt3/jnMrxrnFmZd+/zSeY1VOvAO7mI+h9vBMZZ9620XwbOM7Nq8xrqnRX47F7gI2Z2jplVmVmDmcUbEG4kTSOxgF8D/25mo8xsJF4DsF9m+X5azrn1wP8BPzCzIf6yazSzE3o4ql8DV5vZeDPbG6+EeL/L/zayvwATzOx8f9mci3fJ5E85/n4x3snFVf7vz8S7vp2vW4FrzWwyJBohnp3l+3/y4/+kv+6rzewoMzssx+l1t/5/BnzFzI70G8YdYoEGfgH1QCuww8wmAomTCz+eY8y7DfF9vG08amaDzOwCMxvqnOv0fx/f7m8HPuv/zsxssJn9s5nVd7O/SIVQcpdec85txmuE9U1/0NfwGjU961cxPoZ/r7Jz7u94jZduwmtY9xR7Sn7/indN913gt3jXPRfmEEIN8D28WoMNeKWYb/ifxTs2aTGzF7OM45t4JaytwH8CvwrM3zt4VZ/X4FXzvgx80P/4DmCSX/X5uzTj/Q6wBHgFr+Hhi/6wfFyE1wDtdT/Oh4D9eziOO4FfAE8Dq/AO7l/MMx6ccy3Ax/CWTQvefegfc869l+PvdwNn4rUP2Ip3DfzhXsTzW7xS6H3+tvcacFqW77cBpwDn4W13G/zf1+Q4yeuAe/z1f07qh865B4Hr8banNrzS9Yg04/kKcL7/nduB+wOfDfGHbcW7ZNHCnpqyTwKr/Xn9LHChP90lwOV41flb8fbHS/zfZNtfpEJY8qUuERERKXcquYuIiFQYJXcREZEKo+QuIiJSYZTcRUREKoySu4iISIUpi6cijRw50o0bN67UYYiIiPSJF1544T3n3Kjuv5leWST3cePGsWTJklKHISIi0ifMrDfdMKtaXkREpNIouYuIiFQYJXcREZEKUxbX3EVEpH/o7Oxk7dq1dHR0lDqUilBbW8sBBxxAdXV1Qcer5C4iIjlbu3Yt9fX1jBs3juQn9UpPOedoaWlh7dq1jB8/vqDjVrW8iIjkrKOjg4aGBiX2AjAzGhoailILouQuIiI9osReOMValkruIiJSVsyMa665JvH+xhtv5LrrruvTGC655BIeeuihPp1mTyi5i4hIWampqeHhhx/mvffey+v3kUikwBH1P2pQJyIiRROJxpi/cAWLmluY0djA3NkTqAr3rlxZVVXFFVdcwU033cT111+f9Nnbb7/NpZdeyubNmxk1ahR33XUXY8eO5ZJLLmHEiBG89NJLHHHEEdTX17Nq1SrWr1/PihUrmD9/Ps8++yyPPPIIY8aM4Y9//CPV1dV8+9vf5o9//CPt7e3MmDGDn/70p2VxWUIld+mRSDTGDQuWc8YtTdywYDmRaKzUIYlIPzZ/4QrubFrFy2u2cWfTKm5auKIg4/385z/Pvffey/bt25OGf+ELX+Ciiy7ilVde4YILLuCqq65KfLZixQoee+wxfvCDHwDQ3NzMn//8Z37/+99z4YUXcuKJJ/Lqq69SV1fHn//858T4nn/+eV577TXa29v505/+VJD4i03JXXqkWDuqiFSmRc0tdHR6hYCOzhhNzS0FGe+QIUO46KKLuPnmm5OGL168mPPPPx+AT37ykzzzzDOJz84++2zC4XDi/WmnnUZ1dTVTpkwhGo0yZ84cAKZMmcLq1asB+Otf/8oxxxzDlClTeOKJJ1i6dGlB4i82JXfpkWLtqCJSmWY0NlBb7aWa2uoQMxsbCjbuL3/5y9xxxx28//77Gb8TrEIfPHhw0mc1NTUAhEIhqqurE98NhUJEIhE6Ojq48soreeihh3j11Ve5/PLLy6bzHiV36ZFi7qgiUnnmzp7AZTPHM/XAYVw2czxXz55QsHGPGDGCc845hzvuuCMxbMaMGdx3330A3HvvvRx//PF5jz+eyEeOHMmOHTv6dev4VGpQJz0yd/YEDGhqbmFmY0NBd1QRqTxV4RDz5kxkXpHGf8011/DjH/848f7mm2/m0ksv5fvf/36iQV2+hg0bxuWXX86UKVMYN24cRx11VCFC7hPmnCt1DN2aPn260/PcRURKb9myZRx22GGlDqOipFumZvaCc256vuNUtbyIiEiFUXIXERGpMEruIiIiFUbJXUREpMIouYuIiFQYJXcREZEKo+QuIiJlJRwOM3XqVD74wQ9yxBFHsGjRooKNe9y4cXk/ba4/USc2IiJSVurq6nj55ZcBePTRR7n22mt56qmnkr4TjUaT+pEfaFRyFxGR4olG4LH/hNtP9v5HC/ss9dbWVoYPHw7Ak08+yYknnsj555/PlClTAPjlL3/J0UcfzdSpU/nMZz5DNBoF4HOf+xzTp09n8uTJfOtb3+oy3vb2dubMmcPtt99e0Hj7ikruIiJSPH+9Hp77CXS2w8algMFH/qNXo2xvb2fq1Kl0dHSwfv16nnjiicRnf//733nttdcYP348y5Yt4/7776epqYnq6mquvPJK7r33Xi666CKuv/56RowYQTQa5eSTT+aVV17hAx/4AAA7duzgvPPO46KLLuKiiy7qVaylouQuIiLFs+ppL7EDRNph1VPZv5+DYLX84sWLueiii3jttdcAOProoxk/fjwAjz/+OC+88EKiT/j29nb22WcfAB544AFuu+02IpEI69ev5/XXX08k99NPP52vfvWrXHDBBb2OtVSU3EVEpHjGf9grsUfaoaoOxp9Q0NEfd9xxvPfee2zevBlIfqyrc46LL76Y7373u0m/WbVqFTfeeCPPP/88w4cP55JLLkl6lOvMmTN55JFHOP/885MeGVtOdM1dRESK58R/g2OvhDHTvf8nfqOgo1++fDnRaJSGhq6Pnz755JN56KGH2LRpEwBbtmzh7bffprW1lcGDBzN06FA2btzII488kvS7b3/72zQ0NHDllVcWNNa+pJK7iIgUT7iq19fYU8WvuYNXOr/nnnvStoyfNGkS3/nOdzjllFOIxWJUV1dzyy23cOyxxzJt2jQmT57MwQcfzMyZM7v89oc//CGXXnopX/3qV7nhhhsKGn9f0CNfRUQkZ3rka+Hpka8iIiLSLSV3ERGRCqPkLiIiUmGU3EVERCqMkruIiEiFUXIXERGpMEruFSQSjXHDguWccUsTNyxYTiQaK3VIIiIF1dLSwtSpU5k6dSr77bcfY8aMYerUqQwbNoxJkyYVfHpPPvkkH/vYx3r0m1mzZpHu9u27776bL3zhC4UKLSt1YlNB5i9cwZ1Nq+jojLF8QysGzJszsdRhiYgUTENDQ6Jf+euuu469996br3zlK6xevTqnJByJRKiqqvzUp5J7BVnU3EJHp1da7+iM0dTcUuKIRET6TjQa5fLLL2fy5MmccsoptLd7D6yZNWsW3/jGNzjhhBP40Y9+xObNm/nEJz7BUUcdxVFHHUVTUxMATz31VKJWYNq0abS1tQHeU+LOOussJk6cyAUXXEC887fHH3+cadOmMWXKFC699FJ27drVJaa77rqLCRMmcMIJJySm0xcq//RlAJnR2MDyDa10dMaorQ4xs7FrX8siIgU1a1Zhx/fkk3n/9M033+TXv/41t99+O+eccw6/+c1vuPDCCwHYtm0bTz3lPZHu/PPP5+qrr+b444/nnXfe4dRTT2XZsmXceOON3HLLLcycOZMdO3ZQW1sLwEsvvcTSpUsZPXo0M2fOpKmpienTp3PJJZfw+OOPM2HCBC666CJ+8pOf8OUvfzkRz/r16/nWt77FCy+8wNChQznxxBOZNm1a/sumB5TcK8jc2RMwoKm5hZmNDVw9e0KpQxIR6TPjx49P9Dl/5JFHsnr16sRn5557buL1Y489xuuvv55439raSltbGzNnzmTu3LlccMEFnHnmmRxwwAGA9xjZ+OupU6eyevVq6uvrGT9+PBMmeMfZiy++mFtuuSUpuT/33HPMmjWLUaNGJWJYsWJFcWY+hZJ7BakKh5g3ZyLzSh2IiAwcvShpF1pNTU3idTgcTlTLQ/KjYGOxGIsXL6auri7p91//+tf553/+Z/7yl79w7LHH8thjj6UdbyQSIdfnspTqkbG65i4iIgPKKaecwo9//OPE+3gDvebmZqZMmcLXvvY1pk+fzvLlyzOOY+LEiaxevZqVK1cC8Itf/IITTkh+Vv0xxxzDk08+SUtLC52dnTz44INFmJv0lNxFRGRAufnmm1myZAkf+MAHmDRpErfeeivgPeb18MMP54Mf/CB1dXWcdtppGcdRW1vLXXfdxdlnn82UKVMIhUJ89rOfTfrO/vvvz3XXXcdxxx3HRz7yEY444oiizleQHvkqIiI50yNfC6+sHvlqZgea2V/NbJmZLTWzL/nDR5jZQjN70/8/vFgxiIiIDETFrJaPANc45w4DjgU+b2aTgK8Djzvn/gl43H8vIiIiBVK05O6cW++ce9F/3QYsA8YApwP3+F+7BzijWDGIiIgMRH3SoM7MxgHTgOeAfZ1z68E7AQD26YsYRESkMMqhrVa5KNayLHpyN7O9gd8AX3bOtfbgd1eY2RIzW7J58+biBSgiIjmrra2lpaVFCb4AnHO0tLQkesIrpKJ2YmNm1XiJ/V7n3MP+4I1mtr9zbr2Z7Q9sSvdb59xtwG3gtZYvZpwiIpKbAw44gLVr16JCV2HU1tYmer8rpKIld/O65bkDWOacmx/46A/AxcD3/P+/L1YMIiLiiURjzF+4gkXNLcxobGDu7AlUhXteeVtdXc348eOLEKEUUjFL7jOBTwKvmtnL/rBv4CX1B8zsMuAd4OwixiAiIuiR0ANN0ZK7c+4ZIFOnuicXa7oiItJVukdC6zkUlUvdz4qIDAAzGhuorfYO+XokdOXTU+FERAYAPRJ6YFFyFxEZAPRI6IFF1fIiIiIVRsldRESkwii5i4iIVBgldxERkQqj5C4iIlJhlNxFREQqjJK7iIhIhVFyFxERqTBK7iIiIhVGyV1ERKTCKLmLiIhUGCV3ERGRCqPkLiIiUmGU3EVERCqMkruIiEiFUXIXERGpMEruIiIiFUbJXUREpMIouYuIiFQYJXcREZEKo+QuIiJSYZTcRUREKoySu4iISIVRchcREakwSu4iIiIVRsldRESkwii5i4iIVBgldxERkQpTVeoARCRZJBpj/sIVLGpuYUZjA3NnT6AqrPNwEcmdkrtIPzN/4QrubFpFR2eM5RtaMWDenImlDktEyoiKAyL9zKLmFjo6YwB0dMZoam4pcUQiUm6U3EX6mRmNDdRWe7tmbXWImY0NJY5IRMqNquVF+pm5sydgQFNzCzMbG7h69oRShyQiZUbJXaSfqQqHmDdnIvNKHYiIlC1Vy4uIiFQYJXcREZEKo2p5kSLR/eoiUipK7iJFovvVRaRUVIwQKRLdry4ipaLkLlIkul9dREpF1fJS8Up17Vv3q4tIqSi5S8Ur1bVv3a8uIqWianmpeLr2LSIDjZK7VDxd+xaRgUbV8lLxdO1bRAYaJXepeLr2LSIDjZK7iEgfUs+F0heU3EVE+pB6LpS+oNNFEZE+pLs3pC8ouYuI9CHdvSF9QdXyIlJ0us68h+7ekL6g5C4iRafrzHvo7g3pCwPz1FlE+pSuM4v0razJ3czCZvbLfEZsZnea2SYzey0w7DozW2dmL/t/H81n3CJSXnSdWaRvZa2Wd85FzWyUmQ1yzu3u4bjvBn4M/Dxl+E3OuRt7OC4RKWO6zizSt3K55r4aaDKzPwDvxwc65+Zn+5Fz7mkzG9eb4ESkMug6s0jfyuWa+7vAn/zv1gf+8vUFM3vFr7Yf3ovxiIiISBrdltydc/8JYGb13lu3oxfT+wnwX4Dz//8AuDTdF83sCuAKgLFjx/ZikiIiIgNLtyV3MzvczF4CXgOWmtkLZjY5n4k55zY656LOuRhwO3B0lu/e5pyb7pybPmrUqHwmJyIiMiDlUi1/GzDXOXeQc+4g4Bq8xNxjZrZ/4O3H8U4YREREpIByaVA32Dn31/gb59yTZja4ux+Z2a+BWcBIM1sLfAuYZWZT8arlVwOfySfocqAeuUREpFRySe5vmdk3gV/47y8EVnX3I+fcv6YZfEcPYitr6pFLRERKJZei5KXAKOBh/28k8KliBlUJ1COXiIiUStaSu5mFgW84567qo3gqxozGBpZvaKWjM6YeuUREpE/l0kPdkX0VTCVRj1wiIlIquVxzf8nvne5Bknuoe7hoUVUA9cglIiKlkktyHwG0ACcFhjm86+8iIiLSz+Ryzf0V59xNfRSPiIiI9FLW1vLOuSjwL30Ui4iIiBRALtXyi8zsx8D9JF9zf7FoUYmIiEjecknuM/z/3w4McyRfgxcREZF+Ipenwp3YF4GIiIhIYeTyVLh9zewOM3vEfz/JzC4rfmgiIiKSj1y6n70beBQY7b9fAXy5WAGJiIhI7+SS3Ec65x4AYgDOuQgQLWpUIiIikrdckvv7ZtaA14gOMzsW2F7UqERERCRvubSWnwv8AWg0sya8J8SdVdSoREREJG+5tJZ/0cxOAA4FDHjDOddZ9MhEREQkL7mU3OPX2ZcWORYREREpgFyuuYuIiEgZUXIXERGpMDlVy5vZGOCg4Pedc08XKygRERHJX7fJ3cz+GzgXeJ0997c7QMldRESkH8ql5H4GcKhzblexgxEREZHey+Wa+1tAdbEDERERkcLIpeS+E3jZzB4HEqV359xVRYtKRERE8pZLcv+D/yciIiJlIJce6u4xs0HABH+QeqgTERHpx3JpLT8LuAdYjdf97IFmdrFuhRMREemfcqmW/wFwinPuDQAzmwD8GjiymIGJiIhIfnJpLV8dT+wAzrkVqPW8iIhIv5VLyX2Jmd0B/MJ/fwHwQvFCEhERkd7IJbl/Dvg8cBXeNfengf8tZlAiIiKSv1xay+8C5vt/IiIi0s9lTO5m9oBz7hwzexWvL/kkzrkPFDUyERERyUu2kvuX/P8f64tAREQGukg0xvyFK1jU3MKMxgbmzp5AVVhP5paey5jcnXPr/ZdXOue+FvzMf1Lc17r+SkRE8jV/4QrubFpFR2eM5RtaMWDenImlDkvKUC6nhLPTDDut0IGIiAx0i5pb6OiMAdDRGaOpuaXEEUm5ypjczexz/vX2iWb2SuBvFfBq34UoIjIwzGhsoLbaOyzXVoeY2dhQ4oikXGW75v4r4BHgu8DXA8PbnHNbihqViMgANHf2BAxoam5hZmMDV8+e0O1vRNIx57o0hE/+gtmxwFLnXJv/vh6Y5Jx7rg/iA2D69OluyZIlfTU5ERGRkjKzF5xz0/P9fS7X3H8C7Ai8f98fJiIiIv1QLj3UmQsU751zMTPL5XcywOm2HhGR0sglSb9lZlexp7R+JfBW8UKSSqHbekRESiOXYtRngRnAOmAtcAxwRTGDksqg23pEREojl77lNwHn9UEsUmFmNDawfEMrHZ0x3dYjItKHuk3uZnYX6fuWv7QoEUnF0G09IiKlkcs19z8FXtcCHwfeLU44UkmqwiHmzZnIvFIHIiIywORSLf+b4Hsz+zXwWNEiEhERkV7J576kfwLGFjoQERERKYxcrrm3kXzNfQN6IpyIiEi/lTW5m5kBk51z7/RRPCIiItJLWavl/Z7pfttHsYiIiEgB5HLN/VkzO6rokYiIiEhB5HIr3InAZ8zsbbyHxhheof4DRY1MRERE8pJLcj+t6FGIiIhIweRSLf8d59zbwT/gO8UOTERERPKTS3KfHHxjZmHgyOKEIyIiIr2VMbmb2bX+Pe4fMLNW/68N2AT8vrsRm9mdZrbJzF4LDBthZgvN7E3///CCzIWIiIgkZEzuzrnvOufqge8754b4f/XOuQbn3LU5jPtuYE7KsK8Djzvn/gl43H8vIiIiBZRLtfyfzGwwgJldaGbzzeyg7n7knHsa2JIy+HTgHv/1PcAZPQlWREREupdLcv8JsNPMPgh8FXgb+Hme09vXObcewP+/T57jERERkQxySe4Rv6e604EfOed+BNQXNywwsyvMbImZLdm8eXOxJyciIlIxcknubWZ2LXAh8Ge/tXx1ntPbaGb7A/j/N2X6onPuNufcdOfc9FGjRuU5ORERkYEnl+R+LrALuMw5twEYA3w/z+n9AbjYf30xObS6FxERkZ7ptoc6P6HPD7x/hxyuuZvZr4FZwEgzWwt8C/ge8ICZXQa8A5ydX9giIiKSSS7Pcz8T+G+8xm/Gnr7lh2T7nXPuXzN8dHJPgxQREZHc5dK3/A3A/3POLSt2MCIiItJ7uST3jUrsIiIDRyQaY/7CFSxqbmFGYwNzZ0+gKpxLEy3pL3JJ7kvM7H7gd3gN6wBwzj1ctKhERKRk5i9cwZ1Nq+jojLF8QysGzJszsdRhSQ/kktyHADuBUwIbC3F/AAAgAElEQVTDHKDkLiJSgRY1t9DRGQOgozNGU3ML80ock/RMLq3lP9UXgYiISP8wo7GB5Rta6eiMUVsdYmZjQ6lDkh7KpbX8AcD/ADPxSuzPAF9yzq0tcmwiIlICc2dPwICm5hZmNjZw9ewJpQ5JeiiXavm7gF+x5570C/1hs4sVlIiIlE5VOMS8ORNVFV/Gcmn+OMo5d5dzLuL/3Q2oP1gREZF+Kpfk/p7/qNew/3ch0FLswERERCQ/uST3S4FzgA3AeuAsf5iIiIj0Q7m0ln8H+Jc+iEVEREQKoNuSu5ndY2bDAu+Hm9mdxQ1LRERE8pVLtfwHnHPb4m+cc1uBacULSURERHojl+QeMrPh8TdmNoLcbqETERGREsglSf8AWGRmD+F1YnMOcH1RoxIREZG85dKg7udmtgQ4Ce9Z7mc6514vemQiIiKSl1yf4TcCeN859z/AZjMbX8SYREREpBdyaS3/LeBrwLX+oGrgl8UMSkRERPKXS8n943j3ub8P4Jx7F6gvZlAiIiKSv1yS+27nnMNrTIeZDS5uSCIiItIbuST3B8zsp8AwM7sceAz4WXHDEhERkXzl0lr+RjObDbQChwL/4ZxbWPTIpKxEojHmL1zBouYWZjQ2MHf2BKrCubbXFBGRQsqpMxo/mS8E8J8Md4Fz7t6iRiZlZf7CFdzZtIqOzhjLN7RiwLw5E0sdlojIgJSxaGVmQ8zsWjP7sZmdYp4vAG/hdWQjkrCouYWOzhgAHZ0xmpr1VGARkVLJVm/6C7xq+FeBTwP/B5wNnO6cO70PYpMyMqOxgdpqb3OqrQ4xs7GhxBGJiAxc2arlD3bOTQEws58B7wFjnXNtfRKZlJW5sydgQFNzCzMbG7h69oRShyQiMmBlS+6d8RfOuaiZrVJil0yqwiHmzZnIvFIHIiIiWZP7B82s1X9tQJ3/3gDnnBtS9OhERESkxzImd+dcuC8DERER6S3dluvRc9lFRKRi6LZcz8A7nRERkYql23I9Su4iBRCJxrhhwXLOuKWJGxYsJxKNlTokkQFJt+V6VC0vUgCqChTpH3RbrkfJXaQA0lUF6rZAkb6n23I9qpYXKQBVBYpIf6KSu0gBqCpQRPoTJXeRAlBVoIj0J6qWFxERqTBK7iIiIhVGyV1ERKTCKLmLiIhUGCV3ERGRCqPW8mVCTzoSkXKn41jfUXIvE+reVETKnY5jfUenTGVCTzoSkXJX6uPYQHrAk0ruZWJGYwPLN7TS0RlT96YiUpZKfRwbSDUHSu5lQt2biki5K/VxbCA94EnJvUxk6t5UDVREpFyUupvmUtcc9CUl9zI3kKqZRER6o9Q1B31Jyb3MDaRqJhGR3ih1zUFfUv1tmdNzxEVEJJVK7mVuIFUziYhIbpTcy9xAqmYSESmEgdAQWcldREQGlIHQELmyTlVERES6Ueqe8vqCkruIiAwoA6Ehckmq5c1sNdAGRIGIc256KeIQEZGBZyA0RC7lNfcTnXPvlXD6IiIyAA2EhsiqlhcREakwpUruDvg/M3vBzK4oUQwiIiIVqVTV8jOdc++a2T7AQjNb7px7OvgFP+lfATB27NhSxCgiIlKWSlJyd8696//fBPwWODrNd25zzk13zk0fNWpUX4coIiJStvo8uZvZYDOrj78GTgFe6+s4REREKlUpquX3BX5rZvHp/8o5t6AEcYiIiFSkPk/uzrm3gA/29XRFREQGCt0KJyIiUmGU3EVERCqMkruIiEiFUXIXERGpMEruIiIiFaaUD44RkTITicaYv3AFi5pbmNHYwNzZE6gKq4wg0t8ouYsMEIVIzPMXruDOplV0dMZYvqEVA+bNmVicgEUkb0ruIgNEIRLzouYWOjpjAHR0xmhqbqnox2aKlCvVp4kMEOkSc0/NaGygtto7bNRWh5jZ2FDQGEWkMFRyF0DXUgeCGY0NLN/QSkdnLO/EPHf2BAxoam5hZmMDV8+eUPhARaTXlNwF0LXUbCrlxKcQibkqHGLenImqihfp55TcBdC11Gwq5cRHiVlk4Ci/4ocUha6lZlaIa9UiIn1JJXcBdC01m0JcqxYR6UtK7gKoyjYbnfiISLlRcpeCqpTGZ0E68RGRcqPkXkH6Q2KtlMZnIiLlTMm9gvSHxKpW9yIipVfe9aWSpD+06lar+74Xica4YcFyzriliRsWLCcSjZU6JBEpMZXcK0h/aNWtxmd9rz/U2IhI/6LkXkH6Q2JV47O+p0shIpJKyb2CKLEOTP2hxmag6Q+NV0WyUXIXKXP9ocZmoNGlEOnvlNxLoBzO+sshRvGoxqbvDbRLIToelB8l9xIoh7P+TDFqJ5f+opTbYikvhZRivtMdD66ePUHHgn5Myb0EyuGsP1OM5XBiIgNDKbfFUl4KKcV8pzseOHQs6M90mlUC5XAveKYY+8O99CJQ2m0xfinkd5+fybw5E/u0xFqK+U53PNCxoH9Tyb0ESt0AKpdqvUwxqmW29BcDdVssxXynOx44VgzI5V8uzDlX6hi6NX36dLdkyZJSh9G/RSPw1+th1dMw/sNw4r9BOP252w0Llieq02qrQ1w2c3zO1WmRaIybFq5I2sl1nU1KYaBui/1lvvtLHJXKzF5wzk3P+/dK7hXisf+E534Cne1QVQfHXgkf+Y+0Xz3jliZeXrMt8X7qgcP43edn9lWkItIH1Pi1vPU2uatavlKsetpL7ACRdlj1VNLHwR09hHfdTNVp+dFBU8pBuTZ+1f5VGEruBZDPxhj/TdPK9wibEXWOmYeMZK5/bbvHG/f4D8PGpV5ir6qD8SckfRzc0WuqjMmjhxJzlLTTk1LuxL2Zdl8eNHWgq1zFXrepDd7uWrQaB72aTqaYCzkv+e5fwRiOHT8CgGdXbRmw+42SewHMX7iCO555i10Rxz/WbOPZ5hYe+OxxGTemSDTGObcu5qW12wheFXljYxsGOOj5PaUn/htgXol9/Alw4jeSphnc0XdFHDFH2qr41J30qpMO4eYnVmacbkmSZA/aFxR82vTtrYzBOF9du42HXljLWUceUPSDVS7rtVJOPHKdj0LPb7FPEoMN7wB27o5yZ9OqXk0nU8w3PvoGt//tLaIOXl27DRdzfO2jh+U1jXT719U5LPvUfcXMiMRcWdVaFJKSewEsam5hV8TL0g54ac02blq4ImOnL/MXruClNdtIbe0QvJ2kx/eUhqsyXmOH3FvYpu68zza38Lr/u3TTLUmS/Ov1e9oXbFwKWNZ5L+i0KW5r5dTtpWnle4k4ow42te3K+wDdk+SUbb3Gx/PQC2vZ8v7unA+gxToZyHe8mebDxRwWsi7jK3QyLvZJYryF+12LVrNzdzQxnd6U4DPF/PBL64j6B7Sog9+8tC7v5J5u/8pl2QdjizqIl5z6a18ixabkXgAzGhv4RyBZO8ja6cui5pYuiR323D/qoMvG3dTLA0Gut9+l7rzLN7ZlnW5JkmQ37QuKOm2Keytj6vYyab8hifYRcbks50wnlbkmp2zrNTiensRUrJJqvuPNNB+/eWkdrR2dXcZX6GRc7Fva4vfiB2sCoXcl+L64DS/d/nXWrYu7XfbB2MJGouQ+UNsVKbkXwNzZE3i2uSVRGu+u05cZjQ0sW789Udrfd0gN+w2p5fhDRiYSRSHvKU090Ge7ZSV15524b32i5J5uuiVJkt20LyjqtCluX+6p20vUOS6bOZ4HA6XLXJZzppPKXJNTtvUaHE/Qu9vauWHB8owlwmKVVPMdb7r5iHfUkm586ZZJb2oj+qq/i0wl+HyWf2rMXzzpEG5YsBznXOKSYlXI+MQRY/KKNdOxKpfjzNzZE3Axx29eWodzjgOG70XMOY4/ZGQiznK/hNQTSu4FUBUO8cBnj+tyzyekPyBcnWanTt3QUpNHbw4EPSnZpNt5/+eJlRmnm2lnz2UnyjtJdtO+IBf99WErXbaX8UM5/p3/5V/cYl4bOpVf7nUhxzbuQzTmOOOWpozLONNJZa4nYtm2t9RruXHdXTIoVqkv3/EmlfRC0DC4hrOPPIBozHH34tVdxpdumfSmNqKvtsF0Jfh8l39qzME+M4LLMJfjU09ql3I5/lWFQ1jIErUubbsiiT48gnEOlGvwus+9yPpDRw+53NdeqOuhvekgpxjxFGt8xZpu6vYy8+1bmPbufdTZbtrdIF4ecz5/G/u5RANOA6YdOCypAWeiwWagJumymeP54kmHcP7PnmP5hjYm7lfPrz59DLWDen5+H48xWBIMytRvQrH2he7Gm62Fd7rf9STOcuozohjLvzfzn+5Y0dTc0qvlmRrPPvU1jB5Wx7vb2tnUtivv8ZaC7nPv51LPdCPRWJ9XD+VSsinU9dBCVb0W+vpsqe757el0U7eXN77zHHW2G4A6282ITYtZ1HlOxgac8WkuXb8dh3d5Z9L+QxKlzNfXe9vB6+tb+Z8nVua1DDJdy4Xsz0ooVkm1u/FmWgeZfteTOMupC9zeLv90J0nB+a+pMkKQtUYpqLe1S+mkXnff8v5uNrXtImze5YKBdA1eyb2PlSLJ5FKlVaikXKiDXaGvz5bqSXzZpputVB//bFR0EmPdqkTJfcs+xzFjbOYGnPFpBpN/zHkH9kIvg/h29bc3N7OxdRetHREO26+ez51wcJ+fwGZblsVc94W6bl4OtxWmO3YF5z8EWe+sSZXrJcueCMYTLK1HHeyz9yBGD6srad8efUnJvY/19kCTz0EglzP2QiXlQh3sClkiikRjhCDR4KcQ48t1HRw7fgSvrt1G1EHY4Di/cw3IfqIX/6yz80x2VsU4oep12sbM5KhPfZ+jQlUZG3BC5mVX6FJmuhL86xtauejO53t0kC+EbMuyt/OdbX0XqjaiHHqTS3vsmrNn/s+4palHx7Z0x4reLs/g71Or/c8+8oB+t0yLScm9j/X2QFOsTk0KlZQLdbArZEviTNXUvRlffB0sW7+dZ5tbiEHGRG9m4Jz33/YMz3ait+ezMN+PnMvC/Yfxuyv2XCPM1IATMi+7dMMzJa6O3RHOv/05lm9sY+K+9fzq8szX53t6+2QxZFuWvd2W0iXeTJ1K5VsCL1btQiFrBLIdu/I5gS7GZYLgvJX66ZulpuReZOl6fOvNBpfaUUNPOzVJt0NAcne3uTa0KWZVYiGvz6ZWU6/d2s5Zty7OO+bU3v5e9BvwpCtxLX6rhUjMm3Yk5ljUvOeZ19kOlt2dBGZaPtlue0z3m0ytiM+//bnEfL24Zhvn/+w5Hr4yfQOk1Ouu9TVVtO+Opj3IF2ubyba80s13T+JITbzPrHyPxc0tiR4mg8stUwk8W3fTVeGQV8OzbjvRmCNkYGS+G6JQnRH1VLZkWegT6Fx0N2/99Y6YvqLkXmCpO14scFtNcAPMd4NLdxtS8Ey/ux0/dYdwMcdzq7akPVB1pz9WJXbX6Ae8E6JNbbvSxpxL/9QzGhsSVe1B6UpcYbOk74RDe95nO1jmW+ro6ToJ9oAXT1zzgOUb25K+t3xDW5pfd401BFkP8sXaZnq6vHpS+5J64hA2S+phMrjeM5XA03WYE+9ues/8e2OMOXh1XWvGnv8K1RlRT1UR45rwfZy/8//4v+cm8KEl/8rpR4zlK6cemrGdR6pCntyVqh1NuVByL7DUHW9IbXW3G2BPNvirTjqEZ5tbeGXdNqKxrlVg3e34qTvEb15ax+a2XWkPVKlSSx/LNmSvfi1FI6FsjX4efGFt0u0w6WLOpX/qubMn8FDKuCB9VWQ05VbTaGzP+2wli56UOoLL+d1t7Tkd8OK/WZGSxOMnHxP3rU+U3AEm7lefcfrBWM+4pSnpIL9my05ufPSNxAlSupMJt2B5xhJtrnpaSkutfcl2cnvVSYfw7FstiVsII1HXpYfJeCvxdE9cjERjPPTC2qTEHp//+Pp5dtUWooGP47U96dZhTzsjineYZX6ckWgsv/3wr9cTXfwTxsQ6OM+9yfu7Y/zwmfMIh6zbKvv49tnTRnfZlNOdCqWg5F5gqTvekFqoqbKsO1dPzsRvfmIlr29oJRIjbacR3T10IfXgAyQdqAxy6nc+VbqdK5f5iu/4z67cxLxBD3GMvUZo/Al5PQwm0/zPmxPi6tkTePCFtUnfTZ3X1INwpv6pq8IhzjrygJw675h5yEje8K9B11aHOP6QkT2ep+4El3O6W36ydRbS2dnJvKoHmRFayqLYZB6LfhqAX11+TJd74nORWqvx3o7d/OyZVYkTpGB3uvFScPcl2uzyOYkMJgYjsZrp6Oza9/rNT6xMuoVw0n5DkvbpfeprEgkr3RMXb3z0jS4ngpC8zwSr5TN9J13s3fWUl+g90z95eX1Da9Jtkz1ajquepjrWAXi3Zc4Mv8aNux1NzS089NnjslbZx9dx/Jp8mChfdPfxkSVvQNU/57y/p9asfWrGOBa/tWVAXlPvjpJ7gaXueJ+YNiap2nvp+u2cc+vipCrAnpyJJ11zj8HoYXWJa3o3LFjOu9vaCZlXLRY/mbjx0TcSlwZSDz7RmOOuRav2dIgydlhO/c6nSneNLXW+HnxhbZfr+fEd/4vuV0wNLyBku2Hj6+TzMBjIfDY/f+EKWnYkH2BT5zX1O4ZXkk13b2ywq0uAM6eNSZq3YC3HpP2HEI25pO6FCym1HUbqLT/ZuqKdV/UgnwovYC/bzURbQ93WKm5YsC9zZ0/IeI09neD81lSHE53bOJJLoZFYjEn7DUk01IvEXMYSbS5PAgOSTlRO2/hTNr26ktHTTvV7MqTLEwQjhIjFHENqqxlSC2OG1SbVQqX2vZ5a2xCJxfj08QcnEtkzK99jo5+8d0Ucyze0ccmMcYnt4WF/G4mrChmTRw9Jsz107VAsBHzuhIOThqXWJHzxpEOy3ssfdS7p5CV+6SXdcsz6dMvxH6bz3Vepdrtod4Noih5OVciY2diQteYkuH3G53BufLuL7MY9+w6W4/6eWrPWsHcNZ/kn1an7XrZ2Rf3xVsNCU3IvsEwPPYjvXOmqALurXkqt1kqt9oP0pWoH/GPddtZs3ZlUBRmNOX7/heOJRGPc+OgbDK0bBMAnjhjDNaccCtDlPmUgqTVsqnTX2GY0NvDKmm3EI9rUtouzb13Eg5+d0eUe5BmDliY6ayHSzrsvLeDKN07udkfM1GDxGb+aN1jtG7xGXhUyjh43Imlci5pbkr4zcu9BnHXkAWlLBqldXd69eDXhkCVOtIKP9O1NT32ZZNsmUm/5ydZZyAxbyl6BTnKmRf7BWU819/gujEy1B6kP8KgKhXh9QyudnZ2cuvGnnDhoOX8dNJEbdp9FlDCwp7Saa41W8ETlk7aAvd7fTWfTSn734joOGFbLsZsewCJ7niA4P3JO0gntmKG1fGrGOO5Z/HbavtdT202kJjK3YHmidga6fzDLiMGD+P0Xjk8allotH7ezM8Yn7/w7xzWOTGrHk9oZUaYCQiQaY8P2jqRxBtt9pC7HbJ0jceK/EXLw7suPsqD9UO6tPo/Ljxyb8YQ1vo2+u62dcMiIxhw1VcawukHM2LVnu7MePPwplwbFNz76RqK26NV1271+7wM1RP2lfVCxDczuZ2fNKty4cvDOlp28u6097Wd711QxefRQ1mzdSWt7J0Pqqjlw+F4EjyfvbNnJhu0dxPyNdHBNGBxJ33113Xbe3xXJKZ69a6s4fPTQpPGGzNhvaC1jR+yVdjiQNA+hkBHzS2Tx7xw4fK+k+ThgeB1LVm8llrKNjR5Wx9gRe3k1Ge9uZ8euCAfaJvazLYRxxAixgRG8ExuVFJdzdFlOa7amjzU+DOBA28RQ28l2N5g1btSeeQiMO3U5p34WFI9jQ2tHYhkE12d9bTXrtyev78E1VQytq2azX8IbVV/DAcPrWLu1vct6D85nfW01bR2d7OyMsld1mEmjhxAyy2mbyLj9DAp7Dd4M9olsYGSshRAxohgbXENiGaUug3TLPz6d19Z56zGuOhyipirEkLpqgMRvtrd38n6a9b2REbzjRuGc99tBVSF2R2J0BjLe3jVVHD5maJdYwEuoh9kq6tmz3Nuow4C9A8NcTT1LY+OSYo1vk8HtJjjvqftWfF0Gt/O1W9u7bA/xfTu+jYO30Een2a6C6yiVdwelJeIKh6zLchlSV51xX0499gyuqWKKvxxTY8j1u7lImieD6lCIUfU1bG/vZMTud5PWf2joGBh+UM/GGRA8jq7f3kEwr5kZVWmW2eF5zldGTz5Z0NGp+9kyFjJjSF01ZniJCu9gtYadSQfO1vbOxMbsnANHlw0z/bl4evHtPjjemHOJA2W64bsiKcUK5x0QMyXaeAkoHDJiKc3K49NZs3Un7/vfW+v2YVBViCHspCVaxzuxUUnTD54IAOzYFaG13Ut6wVhTd/xgEqkzrwQTT17BeQZvHcTji89TOukOIHE7dkXY2dm1v/XdkVjSgXP99g4v/t1e/Dt2RdjctotR9TXAniQTTEI7dkV4fX0rh48e2mWbcI7EyUN8PPFtKD4fm9t2EY0lj3N3eBQ1g8JUd25nayz55Cd1+QTXbzDeA4fvxZC66sS8hMwSw9ds3cn29k4MEv9DZgyxnYT9OqAQMfbmfZw/7c5oclIPijnH2q3tbG7bRSTmEqWywTVhduwezF50EMYRxWh1g8Ff7/FhOxjMkLrqLsm9tb2TyaO9fWp7eye7I16Jt9WPOXVdBrdz56CtozNpe4jv20mJHdi7JsyBw7ueqB4wvC4x7fbd0ZTkZUnbeBjDzJKmF/x9fFm/s8Vb9qmG+idcqTGkK+e1d0Z5Z8vOtCeewROb+HSdP/4Dh++VtI3ioKYq5A/fzlq3D2AMtfeJ1gxl6LCxiXEHx2V4J24OGDwozGH7DwFIWv/xZZ1pv3TO0Rl1iWUW/36lG5jJvQBnWLk04ol/566m1bSnOeDvW1/DvkNqmdHYwHOrWnh57Xac8xrgTd5/KDHg6HHD+d3L69jU5lVh1VaFuOz48Uz0r6XGWxm/vr6VjtQEHBDfWUIGU0bXs2LT+0nXweJVxxNnT+A/0jx0JLWlOcDnPnwwFrKk1vPB+Zx64DCOHjec2/62KjEsHDI+++GDOXzORK5KechD2Ohye1kIqPEb/rVnuN6fzW8HfZMxIS+ph4FNsdGct/u/AG85T9pvCGu27mTL+504oG5QmAuPGctX50zEMtxbvGJjW9oHpqTG3V20ew0KdxlPsBo7029e+dYpSesonaqQcdnMcTy/eivLN7ZRX1PFdv8SQqp0yx1gUBiG1lbz3vudYFBXFe5y4hLfHq+ePYFfP/pGUhsEIFH9HRznsLpBfGrXz7kktIA6200Hg7in8zRujJ7bJYa66hAdkVhivxhaW82mHbu7fG/K6Hrea23ngvZ7mRl+jabo4dwUPQuAq8MPMTP8GouikxhUZXx072bWxo7ha1v/hShhwiHjig+N5/DTDmMscOYtTUl3CqRbtpnWD3jr6FMzxtEZjSVt+95vYcTgGkYPreW1d73b3UIGHzxgKA7j2PEjWNz8Hv9Y15r0u0RDNINPHz+e51dvTXvZJ9hvQbwKfGNgv923voan5s2CQVV8P/Dd7uzl7xfPr96SOE6FDQZVheiMuqTlEY/x76u28NLa7YlhV3zoYP4SMn72t2aCm33YoGHvQTgHW3d2Zl220w4cynGNI71jTsiIRGOEzVizdae3naYRJsrcqgc5oWoZy+qmcv/gi+gklNddGeVkYFbLF0DqThRPxsFE/72/LOP2v72V9sAZlO7gGj9LTrXXoBAv/vtsbn5iZaLxSyHsVR3CYQyprWJb++6kBnYPfOY4bnz0DW59+q2k39RVGc4sY+v5Tx03jmdXtfDSmu2B+MNcdOxBfOXUQ5m/cEWXHT2oKkTidr+g+M4ab+E9P3J24nptqnlV93FpeEGib/Y7Il4S2WtQmAn7DOblta1dfmPAZ/0Tl3ir3Gffei/tdzPpLgEYMPXAoUkNuXKx16Awh+67N/9Ys73bk4dMSTtXXoPM7pd1/Alb3/vLssS1zpBBbXXXk5dEbEQTSfc5pvD93Z9Iuw73qg6xM4flU1cd6vbkb17VfYnGg8FtIRyCK44/mGtOPTTtdp6PK2c18uCSNWxOcyKSTS7rbOoBQ732NGv37Ff71New6Osncdati5NOmNP/fgjHHjySnz/7drcnqUGZ2tvkap/6QbR1RJMKAD3Zl8E7JlSF97QvOWzf+sQJRCbB9d5pNdy2ew7f908k96kfxNPzTszr6YjF1ttqeSX3PKU+WjCoNgwdue8zPfb5WY08s/K9pJ27WEJ4Z9VAzgeqQRbl2tqHmRFeymMdE9PusF6tRQ0rN+3IePDeq7prSREyH6TTCSaReGkuHktvD1a9VVcdYuJ+Q1i7dSfv7diduEWoJwe7YstlWe9TX8OZ08b0OFnE9dU8/3bQN5kWak683+CGsd41sCg2mV/UXMAZRx7EnX97ky+F+8/yzySXmqFy0JN9GbzkHqygzGU5pK73l2KNfNyvvQPvBPLFb87udwle19xL5OhxwzMm92ImdoBbnmzu/ksFEiP3pB4/SH8i/DQNsVaqXYxDwm9yjC3j3M7/SDpIbmzblVRdmE66xA4wI5Tcwntm+DWvBJYhSdwYPTftASNdYi9Wokk33vZOr0XyoFCaW4T8W9Ocs6wHu2LLtKyDNrXt6lVpt6/meVFssne7n+2m04VooI39Qtu8aXYYP3zmPK4O96/ln0klJHbIbfsKSr3ymMtyCK73+C18QTs7Y5x/+3M83M+f795TlXmxoQ88v3pLwcYVJsq8qvv47aBvMq/qPsJkrsrM5XulEj9I72fbqDZvtwub44jQSq4OP1Sw6SyKTabdebUJwZ01Pv1poWYuDS/Ia5qFGEdPx7s7cIRKd7ArpUzLOlVvts2+muf5kbO5I3IaL8UaaXFDqLZo0jQjMdcvln8h9vNs4yjWcSSf8ea6ffVGcL3fETkt0RYj6JV1xa8F7VpuBHMAAA5aSURBVGsquecpW1/buQiW5GIYh9k73ZYWcinhFLLk2dNxBQ+MQSFzac/I8411fuRsnLMuDad6Wgrobh7yHUdvxttdKQP6tuo+07JO1ZvSdy7zXAjBWpx5VfdxqS3oMs18Ygmuj8WxwwDjuNDrea+bQtRkZBtHoWpKUrdDw3FJ+NEejTd1+7o5+nHmVd1XsGUJZK29q2QlSe5mNgf4EV4D5p85575Xijh6I5+W20HBHSzqjLB5FbPZDvy5JIhCVnH2dFzBA2PU+Z3eWOYz8nxjzbSzFiJJFCvR5DreXJJpX1bd53pg7M1JUaZ5Tk0eP4qcyZeqHi7ISU1wmouikwhZjN8O+iaLY4dxZ+RUZoRfz3oyExRcH4fbKsCotmje66bYJ6mFOoFN3Q63u716PN7U7St4Db4QyxK87eiaqvs5M/wMOHg49iF+EDknaduJxBwduyP97rp7b/T5nJhZGLgFmA2sBZ43sz84517v61hKKbiDhc0Rc0bIXNYDfy4JopAlz56OK/WACWQ9SBa6lJxrKbPY4+jNeHNJpsWqXeiN3pwUZZrn1ORxjC1jUujtgpzUpJbiLwlM547IaUkNrroTXB/xy1GQ/7opxAlmDMO5PXfdRN2eK7CFOoFN3Q63sxftblCvxlvoZQnednR5+C/e+Awutz8Tc6Eu48r2WONyVIrTlKOBlc65twDM7D7gdGBAJffkHayapbFxhC2W9cCfS4IoZMmzp+NKe5DOctmt0KXkQlS/FasKr5Dj7atq7J4oxklRavI4LPROSS+ZZJLaUA+8xJTvuinEsgwRS3SCZQZh27MjFmpdpW6Hv4l8CEeoV+Mt9LIEb/0GTxSqLZZ2Hff2Umt/U4rkPgZYE3i/FsjtkVP9yAf235tX1u/I+/fpdrDuqhhzSRCFPMgWqxTbV+OvVP1xuRXjpCg1eSyLjWVy6O2SXTLJpKc1Vt0pxLJcFDucibY2ME9TCjp+yHwMK1TtWyGWJXjr93BblUjwnS6Udh1ne6xxOerz+9zN7GzgVOfcp/33nwSOds59MeV7VwBXAIwdO/bIt99+u0/j7E7H7gjn3bY4545NwgZVBruKeA9Lqe/b7mvVBp39YIZrwjCktprNGXrIKoYQMKjK6ChQJ0b9UWofBTdHP85V4d8m3v8wehaRAt2mmKkvhHJVifOUq9R738NEmRt+gE+E/wbAb6IfYn40+Zr7Bw8Ywv1XHNevrrmXXSc2ZnYccJ1z7lT//bUAzrnvZvpNf+zEJpNINMYPAt1wxp+0lksXh5FojJsWrqCpuYXjxo8AI+mJZFXhUNJ3gsNzHd+i5pZEt41V4RDRmGPGwQ3dTitTPIVedqnzBvQ4hvh4nvG7qMw2j90tq5DBum1e97U9WZe5zl+2+OPf++JJh/A/T6zM+LtM4+3JtpJt2cUfTdpdrMVabh27I12eLR8/CGf7LJdl+kygG9P4/pBufvPd74Lb8Q8efYOHXlzL+7uj7F1TxcenjQbg4RfXsWNXBDNj75qqrMsrWxzd7T+F3G/TTSsSjXH+z55j2fpWhtRVs299LR/6p8zbTk+Wa2+Oq+WqHJN7FbACOBlYBzwPnO+cW5rpN+WU3EVERHqr7Hqoc85FzOwLwKN4t8LdmS2xi4iISM+U5AKDc+4vwF9KMW0REZFKV7kXLERERAYoJXcREZEKo+QuIiJSYZTcRUREKoySu4iISIVRchcREakwSu4iIiIVRsldRESkwii5i4iIVJg+71s+H2a2Gehfj4XbYyTwXqmDKIBKmQ+onHmplPmAypmXSpkPqJx5qZT5gOR5Ocg5NyrfEZVFcu/PzGxJbzr37y8qZT6gcualUuYDKmdeKmU+oHLmpVLmAwo7L6qWFxERqTBK7iIiIhVGyb33bit1AAVSKfMBlTMvlTIfUDnzUinzAZUzL5UyH1DAedE1dxERkQqjkruIiEiFUXLPk5nNMbM3zGylmX291PHkwsxWm9mrZvaymS3xh40ws4Vm9qb/f7g/3MzsZn/+XjGzI0oY951mtsnMXgsM63HcZnax//03zezifjQv15nZOn+9vGxmHw18dq0/L2+Y2amB4SXd/szsQDP7q5ktM7OlZvYlf3hZrZcs81GO66TWzP5uZv/w5+U//eHjzew5f/neb2aD/OE1/vuV/ufjupvHEs/H3Wa2KrBOpvrD++W2lTJPYTN7ycz+5L8v/jpxzumvh39AGGgGDgYGAf8AJpU6rhziXg2MTBl2A/B1//XXgf/2X38UeAQw4FjguRLG/WHgCOC1fOMGRgBv+f+H+6+H95N5uQ74SprvTvK3rRpgvL/NhfvD9gfsDxzhv64HVvjxltV6yTIf5bhODNjbf10NPOcv6weA8/zhtwKf819fCdzqvz4PuD/bPPaD+bgbOCvN9/vltpUS41zgV8Cf/PdFXycquefnaGClc+4t59xu4D7g9BLHlK/TgXv81/cAZwSG/9x5ngWGmdn+pQjQOfc0sCVlcE/jPhVY6Jzb4pzbCiwE5hQ/+mQZ5iWT04H7nHO7nHOrgJV4217Jtz/n3Hrn3Iv+6zZgGTCGMlsvWeYjk/68Tpxzbof/ttr/c8BJwEP+8NR1El9XDwEnm5mReR77RJb5yKRfbltxZvb/t3eusXJVVRz//WOhFEqKIiXECtdWKBAs5aERC8QANoKGYEIChUBFHhHhA+WlBCLwoQmE8AgRSiiCREwx9REJX3hYkYreqFW8LVCxVRIampYvlIeA0C4/7DW9k+nM3EfaO2dO/79kMufsvfY9a2WdM+vsx91rBvAN4KE8FxPgEwf38fEZ4PWm8w10/0GoCgE8LWmVpMuy7MCI2Ajlhw6YnuVVt3GselfdnitzSPHhxlA2fWJLDh0eQ+lh9a1fWuyAPvRJDv++CGymBLP1wFsR8XEbvbbrnPVbgP2pgC2tdkREwyeL0yd3S5qcZZX2CXAPcD2wLc/3ZwJ84uA+PtSmrB/+7WBeRBwLnA5cIenkLrL9amMnvatszxJgFjAX2AjcmeWVt0XSVOCXwFUR8XY30TZllbGljR196ZOI2BoRc4EZlJ7dEe3E8ruytrTaIeko4AbgcOCLlKH276d4Ze2Q9E1gc0Ssai5uI7rTfeLgPj42AJ9tOp8BvNEjXUZNRLyR35uBX1Me/k2N4fb83pziVbdxrHpX1p6I2JQ/ZtuApQwPt1XaFkl7UALizyLiV1ncd35pZ0e/+qRBRLwFPEeZg95P0qQ2em3XOeunUaaMKmNLkx1fzymUiIgPgUfoD5/MA86U9BplquYUSk9+l/vEwX18/AU4NFc87klZ+PBEj3XqiqR9JO3bOAbmA2soejdWkS4EfpPHTwAX5krULwNbGsOtFWGsej8FzJf0yRxinZ9lPadlLcO3KH6BYsu5uYL2c8ChwJ+pwP2X84A/Bl6JiLuaqvrKL53s6FOfHCBpvzyeApxGWUPwO+DsFGv1ScNXZwMroqze6mTjhNDBjrVNL42izFE3+6Ry9xZARNwQETMiYoByT6yIiPOZCJ90W23nT9fVj2dQVtauB27stT6j0HcmZbXlP4CXGjpT5nN+C/wrvz+V5QLuS/tWA8f3UPdllKHRjyhvsBePR2/gO5SFKOuAiypky09T16F8iA9qkr8xbfkncHpV7j/gRMqw4BDwYn7O6De/dLGjH30yB/h76rwG+GGWz6QEgnXAcmBylu+V5+uyfuZINvbYjhXpkzXAYwyvqK/kvdXGrq8yvFp+l/vEO9QZY4wxNcPD8sYYY0zNcHA3xhhjaoaDuzHGGFMzHNyNMcaYmuHgbowxxtQMB3djeoikrSoZrl5SyYJ1taSuz6WkAUnnTYBuD0k6cgSZszrJSPqupAvHeM3nJB0/ljbGmB2ZNLKIMWYX8n6UbTaRNJ2SOWoacHOXNgPAeSm7y4iIS0YhdhbwJPBym/YP7HSljDGjwj13YypClG2BL6MkLFH20FdK+lt+vpKitwEnZY9/URe57aTMWkmPZuKNX0jaO+tOVck1vVolScrkLN/ei5b0rqTFObowKOnAvM6ZwB2py6yWa94i6dqmv3W7Sp7uVyWdlOVTJD2eOv0cmNLUfr6kP6VNyyVNlTRNJZ/17JRZJunSneoIY2qAg7sxFSIi/k15LqdT9mX/WpRkP+cA96bYD4CVETE3Iu7uItfKbODBiJgDvA18T9JelDzZ50TEFyijeZe3absPMBgRRwPPA5dGxB8pu7ddl7qsH8G8SRHxJeAqhkcmLgf+mzotBo4DkPRp4CbgtLTrr8DVEbEFuBL4iaRzKfm5l45wXWN2OxzcjakejQxQewBLJa2mbEnZaf57tHKvR8QLefwYZevV2cB/IuLVLH8UaJct8H+U4XeAVZSpgbHSSC7T3P7k1IWIGKJsOQol4cmRwAsqqT8XAoek3DOUbUbvA0YzdWDMbofn3I2pEJJmAlspvfGbgU3A0ZQX8Q86NFs0SrnWvaY7pZJsx0cxvFf1Vsb32/Fhh/bt9sAWJY/3gh0qyoLDI4D3Kak/N4xDF2NqjXvuxlQESQcADwA/ykA6DdgYJe3oBcAnUvQdYN+mpp3kWjlY0gl5vAD4A7AWGJD0+Sy/APj9GNRu1WWsPA+cD6CSs3tOlg8C8xp6Sdpb0mFZt4iS7WwB8LBKylZjTBMO7sb0limNf4UDngWeBm7NuvuBhZIGgcOA97J8CPg4F7ct6iLXyispN0Tp8S6JiA+Ai4DlOay/jfKCMVoeB67LBXmzRpTekSXA1NTpejKNZUS8CXwbWJZ1g8DhGeAvAa6JiJWUl4ObxnFdY2qNs8IZsxsgaYCSbvKoHqtijJkA3HM3xhhjaoZ77sYYY0zNcM/dGGOMqRkO7sYYY0zNcHA3xhhjaoaDuzHGGFMzHNyNMcaYmuHgbowxxtSM/wMInBjNmBoYvAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_x_predictions = lstm_autoencoder.predict(X_test_scaled)\n",
    "mse = np.mean(np.power(flatten(X_test_scaled) - flatten(test_x_predictions), 2), axis=1)\n",
    "\n",
    "error_df = pd.DataFrame({'Reconstruction_error': mse,\n",
    "                        'True_class': y_test.tolist()})\n",
    "\n",
    "threshold_fixed = 0.8\n",
    "groups = error_df.groupby('True_class')\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "for name, group in groups:\n",
    "    ax.plot(group.index, group.Reconstruction_error, marker='o', ms=3.5, linestyle='',\n",
    "            label= \"Break\" if name == 1 else \"Normal\")\n",
    "ax.hlines(threshold_fixed, ax.get_xlim()[0], ax.get_xlim()[1], colors=\"r\", zorder=100, label='Threshold')\n",
    "ax.legend()\n",
    "plt.title(\"Reconstruction error for different classes\")\n",
    "plt.ylabel(\"Reconstruction error\")\n",
    "plt.xlabel(\"Data point index\")\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pred_y = [1 if e > threshold_fixed else 0 for e in error_df.Reconstruction_error.values]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAGDCAYAAAAlPdtBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XecXVXZ9vHflUYSAoQYIJDQifRiKCKPIoKGUCTAixRpUoxSfFSKiEE6ooK0hxoIJSBVQAMiECOIUQIJgdB7DRAihJZCyMzc7x97TziE6Wf2nDNrri+f/ck5a5e1TjLcs8691l5bEYGZmaWrW6UbYGZmxXKgNzNLnAO9mVniHOjNzBLnQG9mljgHejOzxDnQW9kk9ZF0h6QPJd1SxnX2lXRve7atUiR9Q9JzlW6HGYA8j77rkPR94ChgHeBj4DHgjIiYVOZ19wd+AmwVETVlN7TKSQpgaES8WOm2mLWEe/RdhKSjgPOA3wArAKsAFwMj2+HyqwLPd4Ug3xKSelS6DWalHOi7AEnLAKcCR0TEbRExNyIWRsQdEXFsfswSks6T9Fa+nSdpiXzfNpJmSDpa0ixJb0s6KN93CnAisJekOZIOkXSypOtK6l9NUtQHQEk/kPSypI8lvSJp35LySSXnbSVpSp4SmiJpq5J990s6TdK/8+vcK2lgI5+/vv2/KGn/rpJ2lPS8pNmSflVy/BaSHpT0QX7shZJ65fseyA+bnn/evUquf5ykmcBV9WX5OWvmdQzL368k6V1J25T1D2vWQg70XcPXgN7A7U0cMxrYEtgE2BjYAjihZP8gYBlgMHAIcJGkZSPiJLJvCTdFRL+IGNtUQyQtCVwA7BARSwFbkaWQFj9uAPDX/NgvAecAf5X0pZLDvg8cBCwP9AKOaaLqQWR/B4PJfjFdDuwHbAp8AzhR0hr5sbXAz4GBZH932wGHA0TE1vkxG+ef96aS6w8g+3YzqrTiiHgJOA74o6S+wFXA1RFxfxPtNWs3DvRdw5eAd5tJrewLnBoRsyLiv8ApwP4l+xfm+xdGxF3AHGDtNranDthAUp+IeDsinmrgmJ2AFyLi2oioiYgbgGeB75Ycc1VEPB8R84GbyX5JNWYh2XjEQuBGsiB+fkR8nNf/FLARQEQ8EhGT83pfBS4DvtmCz3RSRCzI2/M5EXE58ALwELAi2S9Wsw7hQN81vAcMbCZ3vBLwWsn71/KyRddY7BfFPKBfaxsSEXOBvYAfA29L+qukdVrQnvo2DS55P7MV7XkvImrz1/WB+J2S/fPrz5f0ZUl3Spop6SOybywNpoVK/DciPmnmmMuBDYD/i4gFzRxr1m4c6LuGB4FPgF2bOOYtsrRDvVXysraYC/QteT+odGdE3BMR3yHr2T5LFgCba099m95sY5ta4xKydg2NiKWBXwFq5pwmp69J6kc2GD4WODlPTZl1CAf6LiAiPiTLS1+UD0L2ldRT0g6Sfp8fdgNwgqTl8kHNE4HrGrtmMx4Dtpa0Sj4QfHz9DkkrSNolz9UvIEsB1TZwjbuAL0v6vqQekvYC1gPubGObWmMp4CNgTv5t47DF9r8DrPGFs5p2PvBIRBxKNvZwadmtNGshB/ouIiLOIZtDfwLwX+AN4Ejgz/khpwNTgceBJ4BpeVlb6poA3JRf6xE+H5y7AUeT9dhnk+W+D2/gGu8BO+fHvgf8Atg5It5tS5ta6Riygd6Pyb5t3LTY/pOBa/JZOXs2dzFJI4ERZOkqyP4dhtXPNjIrmm+YMjNLnHv0ZmaJc6A3M0ucA72ZWeIc6M3MEudAb2aWuKpdZW/huy97OpB9wSpr7VzpJlgVevuDp5u7oa1Z5cacngPXKLsNRanaQG9m1qHqGrpvLw1O3ZiZJc49ejMzgKirdAsK40BvZgZQ50BvZpa0SLhH7xy9mVni3KM3MwOnbszMkpdw6saB3swMkp5H70BvZgZJ9+g9GGtmljj36M3MwIOxZmapS3kevQO9mRm4R29mlryEe/QejDUzS5x79GZm4Hn0ZmbJSzh140BvZgZJD8Y6R29mljj36M3MwKkbM7PkJZy6caA3MwMiPOvGzCxtCaduPBhrZpY49+jNzMA5ejOz5CWcunGgNzMDL4FgZpa8hHv0How1M0uce/RmZuDBWDOz5Dl1Y2aWuLq68rZmSOot6WFJ0yU9JemUvHx1SQ9JekHSTZJ65eVL5O9fzPevVnKt4/Py5yRt31zdDvRmZh1jAbBtRGwMbAKMkLQl8Dvg3IgYCrwPHJIffwjwfkSsBZybH4ek9YC9gfWBEcDFkro3VbEDvZkZFN6jj8yc/G3PfAtgW+BPefk1wK7565H5e/L920lSXn5jRCyIiFeAF4Etmqrbgd7MjGxRs3K2lpDUXdJjwCxgAvAS8EFE1OSHzAAG568HA29kbYsa4EPgS6XlDZzTIA/GmplB2bNuJI0CRpUUjYmIMaXHRPYbYRNJ/YHbgXUbuFTUX7KRfY2VN8qB3swMyp51kwf1Mc0emB37gaT7gS2B/pJ65L32IcBb+WEzgJWBGZJ6AMsAs0vK65We0yCnbszMOoCk5fKePJL6AN8GngHuA/bIDzsQ+Ev+enz+nnz/PyIi8vK981k5qwNDgYebqts9ejMz6IgbplYErslnyHQDbo6IOyU9Ddwo6XTgUWBsfvxY4FpJL5L15PcGiIinJN0MPA3UAEdEM4MEDvRmZlD4DVMR8TjwlQbKX6aBWTMR8QnwvUaudQZwRkvrdqA3MwMvgWBmljwvgWBmZp2Ve/RmZuDUjZlZ8hzozcwS5xy9mZl1Vu7Rm5mBUzdmZslLOHXjQG9mBu7Rm5klL+EevQdjzcwS5x69mRk4dWNmljwHejOzxEWTT+Pr1Bzozcwg6R69B2PNzBLnHr2ZGSTdo3egNzODpOfRO9CbmUHSPXrn6M3MEucevZkZeHqlmVnyEk7dONCbmYEDvZlZ8hKedePBWDOzxLlHb2YGRJ0HY83M0uYcvZlZ4hLO0TvQm5kBJJy68WCsmVni3KM3MwPn6M3MkudAb2aWuITXunGO3swsce7RV8CCBZ9y4BHH8unChdTW1PKdb32dIw/dn1+feS5PPfsCEcFqKw/mjNFH07dvH96eOYtfnf4HPp4zh9q6On7+44PYeqstFl3v7Zmz2GW/H3H4wfty0Pf3qOAns/ay0uBBXHDpmSy//EDq6oLrrrmZKy69jl+fegzDR2zDpwsX8torb/CzI0bz0Ycfs+yyy3D5uPPY5CsbctP1tzP6F2dU+iN0PgmnbhRV+nVl4bsvV2fD2kFEMH/+J/Tt24eFNTUccNgx/PKnP2LN1Veh35JLAvD7C8YwYNn+HLr/npz8u/NZ58trsvduO/PSK69x2DEncu+t1yy63s9+dTrduokN11s7+UC/ylo7V7oJHWL5FQaywqDleGL6MyzZry/33P8nDt73J6y40gpMeuAhamtrGX3yUQCccfI59Onbhw03Wpd11h3K2uuu1eUC/dsfPK1yrzHv7EPLijl9j7mi7DYUxambCpBE3759AKipqaGmpgZJi4J8RPDJggVInx0/d+48AD6eO4/lBn5p0bUmPvAfhqw0iDVXX7VjP4QVatY77/LE9GcAmDtnHi88/zKDVlyef973H2prawGYNnU6K600CID58+bz8ORpfLJgQcXa3OlFXXlbFSskdSNp96b2R8RtRdTbmdTW1rLnwf/L62++xT6778xG668DwAlnnMMDD05hzdVW4dif/BCAww/ej1E/H831fxrP/E8WcPl5vwFg3vxPuPK6W7j8vN9w1Q23VuyzWLGGrLISG264LtMeefxz5Xvvtzvjb7u7Qq1KkG+YarXvNrE1+t1b0ihJUyVNvWLcDQU1rTp0796dW6+5iIm3X8sTTz/PCy+/CsDpo4/ivr9cxxqrrczdEx8A4K6/38/IHb/NxD9fx8Vnn8rxp51FXV0dF429lv332m3RtwNLT98l+zJ23Pmc+KszmfPx3EXlPz36R9TW1HLrzXdUsHXWWRTSo4+Ig9p43hhgDKSdoy+19FL92HzYRkyaPJWha6wGZL8ERmy3NVddfyu77TSc2+64h0vPOR2ATTZYl08/Xcj7H37EE089x4T7JnHOxWP5eM5cJLFEr158f49dKviJrL306NGDsePO47Zb7uSuO/6+qPx7+4zk29t/kz1HHlzB1qUnEh6MLXzWjaSdgPWB3vVlEXFq0fVWs9nvf0CPHj1Yeql+fLJgAZOnPMrB++7B6zPeYpUhKxER3P/vh1h91SEArDhoeR6a+hi77vQdXnr1dRYs+JQB/Zdh3CVnL7rmRWOvo2+f3g7yCTnnwtN44fmXueyizwbev7Xd1znyp4ey+04HMH/+JxVsXYISTt0UGuglXQr0Bb4FXAHsATxcZJ2dwX/fe5/Rp59NbV0dURdsv+032HqrLTjg8GOZO3ceEcHaa63Or489EoBjjzyUk353AeNuvh0hTh99FFLVDvBbO9hiy2F8b++RPP3Uc0z4Vzakdeap53H6735Fr149ufHPYwGYNmU6xx11CgAPPz6Bfkv1o1fPnozYaTv22f2HPP/cSxX7DJ1OlQ+olqPQ6ZWSHo+IjUr+7AfcFhHDmzu3q6RurHW6yvRKa532mF459/T9yoo5S55wXdX2vopO3czP/5wnaSXgPWD1gus0M2s9p27a7E5J/YGzgGlAkKVwzMyqiwdj2yYiTstf3irpTqB3RHxYZJ1mZm3iHn3bSOoO7ASsVl+XJCLinCLrNTNrtYQHY4tO3dwBfAI8AaT7t2hmVsWKDvRDImKjguswMytfwakbSSsD44BBZB3fMRFxfsn+Y8jGM5eLiHeVzaE+H9gRmAf8ICKm5cceCJyQn3p6RHx2s0UDig70f5M0PCLuLbgeM7OydMCdsTXA0RExTdJSwCOSJkTE0/kvge8Ar5ccvwMwNN++ClwCfFXSAOAkYDOyCS6PSBofEe83VnHRq1dOBm6XNF/SR5I+lvRRwXWambVeXZS3NSMi3q7vkUfEx8AzwOB897nAL8gCd72RwLjITAb6S1oR2B6YEBGz8+A+ARjRVN1F9+j/AHwNeCKqdeF7MzPo0Fk3klYDvgI8JGkX4M2ImL7YHe+DgTdK3s/Iyxorb1TRgf4F4EkHeTNLnaRRwKiSojH5Qo2LH9cPuBX4GVk6ZzTQ0GoBDd1pG02UN6roQP82cL+kvwGLnojg6ZVmVnXKnF5ZuvpuYyT1JAvyf4yI2yRtSLZaQH1vfggwTdIWZD31lUtOHwK8lZdvs1j5/U3VW3SO/hVgItALWKpkMzOrLgXn6PNZNGOBZ+o7uxHxREQsHxGrRcRqZEF8WETMBMYDByizJfBhRLwN3AMMl7SspGXJvg3c01TdhfXo85ul+kXEsUXVYWbWXqL4HP3/APsDT0h6LC/7VUTc1cjxd5FNrXyRbHrlQQARMVvSacCU/LhTI2J2UxUXFugjolbSsKKub2bWmUTEJBrOr5ces1rJ6wCOaOS4K4ErW1p30Tn6xySNB24BFj0Hzc+MNbOq47Vu2mwA2dLE25aUBeBAb2bVxatXtk1bnx1rZtbhEu7RFzrrRtIQSbdLmiXpHUm3ShpSZJ1mZm1S8KybSip6euVVZFOEViK7c+uOvMzMzDpI0YF+uYi4KiJq8u1qYLmC6zQza7WIKGurZkUH+ncl7Sepe77tRzY4a2ZWXZy6abODgT2BmWTLIeyRl5mZVZeEA33Rs25eB3Ypsg4zs/bQAXfGVkwhgV7SiU3sjpKHhpuZWcGK6tHPbaBsSeAQ4EuAA72ZVRf36FsnIv5Q/zp/ZNZPyRbkuZHsYSRmZtUl3RtjC129cgBwFLAvcA3Z0puNPtPQzKySnKNvJUlnAbuTLcK/YUTMKaIeMzNrXlE9+qPJnih1AjC65DmIIhuMXbqges3M2sY9+taJiKLn55uZtS/n6M3M0uYcvZlZ6hLu0TvFYmaWOPfozcxw6sbMLH0Jp24c6M3MgHCgNzNLXMKB3oOxZmaJc4/ezAynbszM0udAb2aWtpR79M7Rm5klzj16MzPS7tE70JuZ4UBvZpa+UPPHdFIO9GZmpN2j92CsmVni3KM3MwOizqkbM7OkpZy6caA3MwPCg7FmZmlLuUfvwVgzs8S5R29mhgdjF5G0DDA4Ip4uqD1mZhUR6T4ytvlAL2kisBvQHZgOzJY0ISKOLbpxZmYdJeUefUty9AMi4iNgd+CaiNgE2L7YZpmZWXtpSeqmh6TlgO8BJxbcHjOziujqPfozgH8Cr0fEw5LWAF4ptllmZh0rorytmjXbo4+IG4EbS96/DIwsslFmZh2tS/foJZ0paWlJPSTdI+kdSd/viMaZmXWUCJW1VbOWpG52yAdjdwZmAesDxxXaKjMzazctGozN/9wRuCEi3pVU5RkpM7PWSXkJhJYE+r9JehKoBY6QNBBYUGyzzMw6Vl2Vp1/K0WzqJr8xaltg04hYCHxCNqfezCwZRefoJV0paVbeca4v20TSZEmPSZoqaYu8XJIukPSipMclDSs550BJL+TbgS35bC1dAmEA8HVJvUvKrm/huWZmVa8DZt1cDVwIjCsp+z1wSkT8TdKO+fttgB2Aofn2VeAS4KuSBgAnAZsBATwiaXxEvN9UxS2ZdXMCMAa4NK/8PGCPVnw4M7MuLyIeAGYvXgwsnb9eBngrfz0SGBeZyUB/SSuSrUowISJm58F9AjCiubpb0qPfC9gEmBYR++eVXdaC88zMOo0K3fT0M+AeSWeTdby3yssHA2+UHDcjL2usvEktmV45PyJqgRpJSwEzgTVacJ6ZWacRdSprkzQqz7PXb6NaUO1hwM8jYmXg58DYvLyhPFI0Ud6klvToH5XUH7gSmAp8BExrwXlmZp1GubNuImIMWZq7NQ4Efpq/vgW4In89A1i55LghZGmdGWQ5/NLy+5urpCWzbn4UER9ExEXATsCPIuKA5s4zM7NmvQV8M3+9LfBC/no8cEA++2ZL4MOIeBu4BxguaVlJywLD87ImNdqjl7RRI7tqJG0UEY+38IOYmVW9opcxkHQDWW98oKQZZLNnfgicL6kH2dT1+nTPXWQ3qb4IzAMOytoYsyWdBkzJjzs1IhYf4P2CplI3FzWxL4Ctm7u4mVlnUfRgbETs08iuTRs4NoAjGrnOlWSp9BZrNNBHxDdacyEzs86sS98ZK+nH+WBs/ftlWziabGbWaXT11St/HBEf1L/JJ+kfVlyTzMysPbVkemX30jeSugE9i2mOmVllVPtTosrRkkA/IR8tvpRsEPYw4O+FtgpYcrDHeu2L6lL+v9EqKuUcfUsC/bHkd2+R3ZV1L14CwcwSU+159nK05JmxtWQrrl1YfHPMzCoj5R59SwZjzcysE2vpevRmZklLefSnxYFe0hIR4UcImlmSunTqRtIWkp4gX2xH0saS/q/wlpmZdaCufsPUBcDOwHsAETEd+FaRjTIzs/bTktRNt4h4Tfrcb6zagtpjZlYRdZVuQIFaEujfyJ9MHpK6Az8Bni+2WWZmHSsafHhTGloS6A8jS9+sArxDdles17oxs6TUJTztpiU3TM0C9u6AtpiZVUxdV+7RS7qcBqaYRoSXKjYz6wRakropXcCsN7Ab8EYxzTEzq4wunaOPiJtK30u6FphQWIvMzCqgq8+6WdzqwKrt3RAzs0rq0j16Se/zWY6+GzAb+GWRjTIzs/bTZKBXdpfUxsCbeVFd/nRyM7OkpJy6aXIJhDyo3x4RtfnmIG9mSaorc6tmLVnr5mFJwwpviZlZBQUqa6tmjaZuJPWIiBrg68APJb0EzCV7nGBEhIO/mSWjrrpjdVmaytE/DAwDdu2gtpiZWQGaCvQCiIiXOqgtZmYV01WXQFhO0lGN7YyIcwpoj5lZRaQ806SpQN8d6AcJ/5ozM8tV+8yZcjQV6N+OiFM7rCVmZhVUp3T7tE1Nr0z3U5uZdSFN9ei367BWmJlVWJfM0UfE7I5siJlZJXXVHL2ZWZeR8g1TLVkCwczMOjH36M3M6Lo3TJmZdRldcjDWzKwrSTlH70BvZkbas248GGtmljj36M3McI7ezCx5ztGbmSUu5Ry9A72ZGWkHeg/Gmpklzj16MzMgnKM3M0tbyqkbB3ozM9IO9M7Rm5klzoHezIzshqlytuZIulLSLElPlpSdJelZSY9Lul1S/5J9x0t6UdJzkrYvKR+Rl70o6Zct+WwO9GZmZDdMlbO1wNXAiMXKJgAbRMRGwPPA8QCS1gP2BtbPz7lYUndJ3YGLgB2A9YB98mOb5EBvZkaWoy9na05EPADMXqzs3oioyd9OBobkr0cCN0bEgoh4BXgR2CLfXoyIlyPiU+DG/NgmOdCbmVF8oG+Bg4G/5a8HA2+U7JuRlzVW3iQHejOzdiBplKSpJduoVpw7GqgB/lhf1MBh0UR5kzy90syM8levjIgxwJjWnifpQGBnYLuIqG/GDGDlksOGAG/lrxsrb5R79GZmdMhg7BdIGgEcB+wSEfNKdo0H9pa0hKTVgaHAw8AUYKik1SX1IhuwHd9cPe7Rm5lR/A1Tkm4AtgEGSpoBnEQ2y2YJYIIkgMkR8eOIeErSzcDTZCmdIyKiNr/OkcA9QHfgyoh4qtm6P/umUF16LTGkOhtmFVVXpT+vVlk1n75Z9ko1Z666X1k/XMe/dl3Vrpbj1I2ZWeKcujEzA+oSfpigA72ZGWkvauZAb2ZG2g8Hd47ezCxx7tGbmeHUjZlZ8tp601Nn4EBvZoZn3ZiZJS/dMO/BWDOz5LlHb2aGB2PNzJLnHL2ZWeLSDfMO9GZmQNqpGw/Gmpklzj16MzOcozczS166Yd6B3swMcI7ezMw6MffozcyASDh540BvZkbaqRsHejMzPOvGzCx56YZ5D8aamSXPgb7KLLHEEvx70p1MnXIvjz06kRN/fTQA22yzFQ9N/huPTvs7Y684l+7du1e4pdaRLh/zB96aMZ3HHp24qGzZZftz91038MxTk7j7rhvo33+ZCraw86sjytqqmQN9lVmwYAHDt9+TzTYfzmabb8/w4duw5ZabMvaK89hv/8P5yrBv8/rrb7L//t+rdFOtA40bdzM77bzv58qO+8UR/OO+Say7/tf5x32TOO4XR1SodWmoK3OrZg70VWju3HkA9OzZg549e1BbW8eCTz/lhRdeAeDvEx9gt912rGQTrYP9a9JDzH7/g8+Vffe72zPu2lsAGHftLeyyy4hKNC0ZUeZ/1azQQC/pkAbKfltknSno1q0bUx6+hzdnTGfixH8xZcqj9OzRg2HDNgJg9913YuUhK1W4lVZpKyw/kJkzZwEwc+Ysll/uSxVuUefmHn3b7SFp0fdNSRcDyzV2sKRRkqZKmlpXO7fgplWvuro6Nt9ie1ZfY3M222wT1l9vbfbb/3DOPusk/j3pTuZ8PIeamppKN9PMOomip1fuDoyXVAfsAMyOiMMbOzgixgBjAHotMaS6vwt1gA8//IgHHniQ4dtvw7nnXsa22/0/AL797a0ZOnSNCrfOKu2dWe8yaNDyzJw5i0GDlmfWf9+rdJM6tWpPv5SjkB69pAGSBgB9gEOBXwAfAafm5daIgQMHsMwySwPQu3dvtt326zz33Issl38t79WrF8ccczhjLr+2ks20KnDnHfdyQD4of8D+3+OOO+6pcIs6t5RTN0X16B8hu/9AJX/ulG8BuDvaiBUHrcDYsdn0yW7dxJ/+dCd33TWRM888gZ123I5u3bpx2Zhx3H//fyrdVOtA1117Ed/c+msMHDiAV1+eyimnns3vzrqIG6+/lIN+sA9vvPEme+3zo0o3s1Ori3R79Ioq/XBO3VhDUv6f0dqu5tM3Ve419l9197J+uK597bay21CUwpdAkLQBsB7Qu74sIsYVXa+ZWWuk3IUoNNBLOgnYhizQ30U2IDsJcKA3s6pS7Xe3lqPw6ZXAdsDMiDgI2BhYouA6zcxaLeUbpopO3cyPiDpJNZKWBmbhgVgzq0LVPnOmHEUH+qmS+gOXk83EmQM8XHCdZmZWotBAX3Jz1KWS7gaWjojHi6zTzKwtnKNvI2X2k3RiRLwKfCBpiyLrNDNri5Rz9EUPxl4MfA3YJ3//MXBRwXWambWa74xtu69GxDBJjwJExPuSehVcp5lZq1XrzaPtoege/UJJ3cnvRZC0HNX/y8/MLClF9+gvAG4Hlpd0Btm8+hMKrtPMrNVSHowtetbNHyU9QnbTlIBdI+KZIus0M2uLlFMNhQV6Sd2AxyNiA+DZouoxM2sP1T5zphyF5egjog6YLmmVouowM7PmFZ2jXxF4StLDwKJnA0bELgXXa2bWKs7Rt90pBV/fzKxdpDy9sujB2H/Wv5Y0EHgvUv7bNLNOqyMGY/O1v64ANiCbdn4w8BxwE7Aa8CqwZ37PkYDzgR2BecAPImJaW+ot6pmxW0q6X9Jtkr4i6UngSeAdSSOKqNPMrBwdtATC+cDdEbEO2bLtzwC/BCZGxFBgYv4esud3DM23UcAlbf1sRQ3GXgj8BrgB+AdwaEQMArYGziyoTjOzqpUv1b41MBYgIj6NiA+AkcA1+WHXALvmr0cC4yIzGegvacW21F1UoO8REfdGxC1kDx2ZDBARnmZpZlWpjihra4E1gP8CV0l6VNIVkpYEVoiItwHyP5fPjx8MvFFy/oy8rNWKCvSl6a75i+1zjt7Mqk5ElLVJGiVpask2arEqegDDgEsi4itkMxF/+YWGfKahh423KX4WNRi7saSPyBraJ39N/r5346eZmVVGudMrI2IMMKaJQ2YAMyLiofz9n8gC/TuSVoyIt/PUzKyS41cuOX8I8FZb2lZIjz4iukfE0hGxVET0yF/Xv+9ZRJ1mZtUsImYCb0haOy/aDngaGA8cmJcdCPwlfz0eOCB/rseWwIf1KZ7WKnoevZlZp9BBSyD8BPhjvlz7y8BBZB3umyUdArwOfC8/9i6yqZUvkk2vPKitlTrQm5kBdR1wi09EPAZs1sCu7Ro4NoAj2qNeB3ozM9KeJeJAb2ZG2mvdFP2EKTMzqzD36M3MSLtH70BvZoZXrzQzS5579GZmifOjBM3MrNNyj97MDOfozcyS5xy9mVniUu7RO0dvZpY49+jNzHC7sh4WAAAHWklEQVTqxswseSlPr3SgNzOjY5YprhQHejMz0u7RezDWzCxx7tGbmeHUjZlZ8lJO3TjQm5nhHr2ZWfJS7tF7MNbMLHHu0ZuZ4dSNmVnyUk7dONCbmQERdZVuQmGcozczS5x79GZmePVKM7PkpfzgEQd6MzPcozczS17KPXoPxpqZJc49ejMzfMOUmVnyfMOUmVniUs7RO9CbmZH2rBsPxpqZJc49ejMznLoxM0ueZ92YmSUu5R69c/RmZolzj97MjLRn3TjQm5mRdurGgd7MDA/GmpklL+UlEDwYa2aWOPfozcxw6sbMLHkejDUzS1zKOXoHejMz0u7RezDWzCxx7tGbmZF2j96B3swMEs7Qg1L+LZYKSaMiYkyl22HVxT8X1lLO0XcOoyrdAKtK/rmwFnGgNzNLnAO9mVniHOg7B+dhrSH+ubAW8WCsmVni3KM3M0ucA33BJIWkP5S8P0bSyR3chqsl7dGRdVrrSKqV9Jik6ZKmSdqqHa/9qqSB7XU963wc6Iu3ANi9rf+jSfJNbV3D/IjYJCI2Bo4Hzlz8AEndO75ZlgIH+uLVkA2a/XzxHZJWlTRR0uP5n6vk5VdLOkfSfcDvJJ0s6RpJ9+a9s90l/V7SE5LultQzP+9ESVMkPSlpjCR16Ce19rI08D6ApG0k3SfpeuCJvGw/SQ/n3wAuq/8FIOkSSVMlPSXplMUvKqlP/vPyw478MFZ5DvQd4yJgX0nLLFZ+ITAuIjYC/ghcULLvy8C3I+Lo/P2awE7ASOA64L6I2BCYn5cDXBgRm0fEBkAfYOdCPo0VoU8euJ8FrgBOK9m3BTA6ItaTtC6wF/A/EbEJUAvsmx83OiI2AzYCvilpo5Jr9APuAK6PiMuL/jBWXRzoO0BEfASMA/53sV1fA67PX18LfL1k3y0RUVvy/m8RsZCsV9cduDsvfwJYLX/9LUkPSXoC2BZYv90+hBWtPnWzDjACGFfyjezhiHglf70dsCkwRdJj+fs18n17SpoGPEr2b79eyfX/AlwVEeOK/iBWfZz/7TjnAdOAq5o4pnSu69zF9i0AiIg6SQvjs3mxdUAPSb2Bi4HNIuKNfMC3d7u03DpURDyYj+kslxeV/iwIuCYiji89R9LqwDHA5hHxvqSr+fy//7+BHSRdH55T3eW4R99BImI2cDNwSEnxf4C989f7ApPKqKL+f+p3JfUDPMumk5K0Dtm3tvca2D0R2EPS8vmxAyStSpbXnwt8KGkFYIfFzjsxv97FhTXcqpYDfcf6A1A6++Z/gYMkPQ7sD/y0rReOiA+Ay8lSOX8GppTRTut49Tn6x4CbgAMXS90BEBFPAycA9+Y/NxOAFSNiOlnK5ingSrIe/OJ+BvSW9PuiPoRVJ98Za2aWOPfozcwS50BvZpY4B3ozs8Q50JuZJc6B3swscQ701qiSFRWflHSLpL5lXGsbSXfmr3eR9Msmju0v6fA21HGypGNacfyc1tZh1hk50FtT6m/L3wD4FPhx6U5lWv0zFBHjI+K3TRzSH2h1oDezhjnQW0v9C1hL0mqSnpF0MdmSDitLGi7pwXwd9VvyO3ORNELSs5ImAbvXX0jSDyRdmL9eQdLt+Trs0/N12H8LrJl/mzgrP+7YfGXOx0tXZpQ0WtJzkv4OrN1Qwxupo3R/v3z10Gn5iqAj8/IlJf01P+dJSXvl5b+V9HTelrPb7W/YrCBe68aapWxN/B34bCG1tYGDIuLwfE2WE8hW2pwr6TjgqPzuy8vJFld7kexuz4ZcAPwzInbLl9vtB/wS2CBfnRFJw4GhZKs4ChgvaWuyW/73Br5C9rM8DXikhXWU+gTYLSI+yj/PZEnjyRYXeysidsrbsYykAcBuwDoREZL6t+xv0axyHOitKX3yW/Ih69GPBVYCXouIyXn5lmSrJP47X2yxF/AgsA7wSkS8ACDpOmBUA3VsCxwAkN/y/6GkZRc7Zni+PZq/70cW+JcCbo+IeXkd4xv5HF+oY7H9An6T//KoAwYDK5AtJ3G2pN8Bd0bEv/Jfep8AV0j6K3BnI3WaVQ0HemvK/Ppedb08mC++muKEiNhnseM24fOrcZZDwJkRcdlidfysnerYl2ylyE0jYqGkV4HeEfG8pE2BHYEzJd0bEadK2oJseeC9gSPJfpGYVS3n6K1ck4H/kbQWgKS+kr4MPAusLmnN/Lh9Gjl/InBYfm53SUsDH5P11uvdAxxckvsfnK/e+ACwm7InJy0FfLcVdZRaBpiVB/lvAavmx64EzIuI64CzgWF5G5aJiLvIFgnbBLMq5x69lSUi/ivpB8ANkpbIi0/Ie8OjgL9KepdsCeYNGrjET4Exkg4he1rSYfl67P+W9CTZA1eOVfZkpQfzbxRzgP0iYpqkm4DHgNfI0ksN+UIdZOmlen8E7pA0Nb/Ws3n5hsBZkuqAhfl5SwF/Ubb+v2jgEZFm1carV5qZJc6pGzOzxDnQm5klzoHezCxxDvRmZolzoDczS5wDvZlZ4hzozcwS50BvZpa4/w93axnd9w3tqgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "conf_matrix = confusion_matrix(error_df.True_class, pred_y)\n",
    "\n",
    "plt.figure(figsize=(6, 6))\n",
    "sns.heatmap(conf_matrix, xticklabels=LABELS, yticklabels=LABELS, annot=True, fmt=\"d\");\n",
    "plt.title(\"Confusion matrix\")\n",
    "plt.ylabel('True class')\n",
    "plt.xlabel('Predicted class')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAGDCAYAAAAoD2lDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XecVOX1x/HP2UoHKSptBREUMIpKsAsWFBtdgyWJiGKJGgvGGltsUSP2glhijOBPimJFwd7BgooiRUAWkN7LsuX5/XFnzbDcuztb5k7Z7/v14sXOPGfmnpmdnTP3znPuY845REREJP1kJDoBERERiQ8VeRERkTSlIi8iIpKmVORFRETSlIq8iIhImlKRFxERSVMq8pIwZnaGmb2V6DwSzczyzGyjmWWGuM12ZubMLCusbcaTmc00s15VuF3avQbN7GMz2y/kbV5iZneGuU2JjYq8AGBmC8xsS6TY/Gpmz5hZg3hu0zn3X+fcsfHcRjKKPNfHlF52zv3inGvgnCtOZF6JEvmwsUd17sM519U5914F29nhg026vQbN7GRgg3Pu68jlm8ysMPJ3vdbMPjGzg8vcpomZPRr5u99sZt+Z2VCf+z7dzKZH7mupmb1hZodFhkcBZ5rZznF/kFIpKvIS7WTnXAOgG7AfcE2C86mSRO6dpsuecWXo+a66OOR/PvCfMte9EPm7bg68C7wYtf0cYAqwG3Aw0Bi4ErjTzC6PirscuA+4HdgFyAMeAfoBOOe2Am8Af6rhxyPV5ZzTP/0DWAAcE3X5LuC1qMu5wD3AL8Ay4DGgbtR4P+AbYD0wD+gTub4x8CSwFFgM3ApkRsbOAj6K/PwYcE+ZnF4GLo/83AoYD6wA5gOXRMXdBIwDnots/xyfx9cYeDZy+4XA9UBGVB4fAw8C64BZwNFlblveY/gYGAmsjox1AN4BVgErgf8CTSLx/wFKgC3ARuBvQDvAAVmRmPeAf0TudwPwFtA8Kp8/RR7DKuDvZX93ZR53XeBfkfh1wEeR60q3+efI73QlcF3U7XoAnwJrI4/7ISAnatwBfwHmAPMj190PLIr8Dr4EDo+KzwSujbw2NkTG2wIfRO5rU+T5+EMk/iS819Na4BNgnzKv1auAb4ECICv6OYjkPj2SxzLg3sj1v0S2tTHy72CiXoORmK7A25Hf5TLg2ko+r72A/KC/LXZ8rd4QeS00jYrfL/L7yI5cPhv4EVgDTAZ2C8gpJ3Jfbcr8bTwXdblL5DloEbk8DFgO1C9zX3+IPEeN8F7/G4FTKngPOQN4N9HvZfpX5veS6AT0Lzn+lXkjagN8B9wfNX4fMAloCjQEXgHuiIz1iLzR9cY7OtQa2Csy9hLwOFAf2Bn4AjgvMvbbGyxwBF6BsMjlnSJvWK0i9/ll5A0xB9gd+Bk4LhJ7E1AI9I/E1vV5fM/ifWhoiFfgZgPDovIoAi4DsiNvcOtK33hjeAxFwMV4xaYusEfkucgFWuAVsvv8nuvI5XbsWOTnAZ0i9/cecGdkrEvkDfewyHNxT+SxBxX5hyO3b41XaA+J5FW6zSci29gXr2B2jtzuAOCgyGNqh1dkLo26X4dXDJuWPt/AmUCzyG2uAH4F6kTGrsR7Te0JWGR7zaLua4+o+94fr/AcGMn5z5HnLDfq+fsG70NC3bLPKd6Hkz9Gfm4AHOT3PPu8BhvifaC5AqgTuXxgJZ/XXlRc5Ld7reJ9IDw3Kv5u4LHIz/2BuUDnyPN6PfBJQE5dgU1lrruJSJHHe73cifcBovS1Nhb4t899ZeG9ro8D+kR+zvLbbpnf2+pEv5fpX5nfS6IT0L/k+Bd5I9qIt5flgKn8b+/T8Pa0OkTFH8z/9uAeB0b63OcueIUjeo//NCKf9su8wRrentYRkcvnAu9Efj4Q+KXMfV8DPB35+Sbgg3IeW2Ykjy5R150HvBeVxxIiHzAi130B/DHGx/BL0LYjMf2Br8s81xUV+eujxi8E3oz8fAMwJmqsHrANnyKPV0S2APv6jJVus02Zxzwk4DFcCkyMuuyAoyp43GtKtw38BPQLiCtb5B8F/lEm5iegZ9Tzd7bP67e0kH4A3EzU0Q+/59nnNXha9O+pnMdV3vPai4qL/Adlxs/hf691w/uwW/p38AaRD6NR296Mz948cCjwa5nrboq8PtYCxXhHf3pFjU8h8gHS5/5+xds7P6Ps/QbEdwSKK4rTv3D/6Tt5idbfOdcQ741qL7zv8MDbG60HfBmZvLMWeDNyPXh7VPN87m83vD3jpVG3exxvb3g7znuXGIv3RgtwOt5h7tL7aVV6H5H7uRavAJdaVM7jao63F7Mw6rqFeHthpRZHcogebxXjY9hu22a2s5mNNbPFZrYe79Bscyrn16ifN+PtkRLJ6bftOec2471x+2mOt0fq97spdztm1snMXo1MxlqP911s2cdQ9nFfYWY/mtm6yPPUOOo2Qa8RP7sBV5T5fbfFe+y+2y5jGN5RkFlmNs3MTopxu7HmGMvzWp6yuY8DDjazVnhHtBzwYWRsN+D+qOdhNd4HgdbsaA3e0Yey/s851wTv7+V7vKM0pVYCLcveIDJXoHlkfBXQPIb5Aw3xjoBJElGRlx04594HnsE7FAzeH/oWoKtzrknkX2PnTeYB702rg89dLcLbC24edbtGzrmuAZseAww2s93w9t7HR93P/Kj7aOKca+icOyE67XIe0kq8Q6S7RV2Xh/f9eqnWZmZlxpfE+BjKbvuOyHX7OOca4R3GtnLiK2Mp3tcpAJhZXbxD5H5WAlvx/91U5FG8uQkdI4/hWrZ/DBD1OMzscLzvyU8FdooUlXVRtwl6jfhZBNxW5vddzzk3xm/bZTnn5jjnTsP7IPZPYJyZ1S/vNpXMsbzndRPeB2IAIm2RLcrEbJeHc24t3ryLU/E+3I6J+sC5CO+roejnoq5z7hOfbc/xNml+HwBwzq3EO4J1k5mVFvYpwPGR5yfaILzX/Wd4X39sxTsiVZ7OwIwKYiRkKvIS5D6gt5l1c86V4H13O7K0RcbMWpvZcZHYJ4GhZna0mWVExvZyzi3Fe/P6l5k1iox1MLOefht0XtvPCmA0MDny5gfeYeT1ZnaVmdU1s0wz29vMfh/LA3Fea9r/AbeZWcPIh4jL8fawS+0MXGJm2WZ2Ct4b1uuVfQwRDfG++lgbecO9ssz4Mrx5BVUxDjjZzA6JzIy+mR2LLwCR39tTwL1m1iryvB1sZrkxbKch3sSwjWa2F3BBDPFFeL+/LDO7AW/SVqnRwD/MrKN59jGz0g8nZZ+PJ4DzzezASGx9MzvRzPz2UndgZmeaWYvI4y99DRVHcish+Ll/FdjVzC41s9zIa+XAskEVPK+zgTqRfLPxvkOP5fl+Hm9C5aDIz6UeA64xs66Rx9Y48vrcgXOuEK9oB742nXOz8Cbv/S1y1X+AfOBF81oMsyN/1w8ANznn1jnn1uF9TfSwmfU3s3qRuOPN7K6ou++J9/WCJBEVefHlnFuBN1nt75GrrsKbAPRZ5PDtFLxJVDjnvgCG4s0wXwe8z//2mv+Ed6j8B7zDiePwOTwYZQxwDFFvdJEifTJea998vD2p0XiHg2N1Md5e1s94M6Gfx3ujLvU53neKK4HbgMHOudLD4JV9DDfjTUJaB7wGTCgzfgdwfeQQ7IhKPAacczMjj2Us3l79BrxJagUBNxmBN+FtGt6h3n8S29/9CLy9yg14RfeFCuIn473Bz8b7qmMr2x+Wvhfvg9ZbeB8ensSbdAbe98b/jjwfpzrnpuPNyXgI7/mei/fdeaz6ADPNbCPejP8hzrmtka82bgM+jmzroOgbOec24E2YPBnva4w5wJEB2/B9XiMF8UK81+divNdcfgw5T8J7/S1zzv22N+ycmxi577GRv7vvgePLuZ/H8eaSlOduYLiZ7eycK8D7e1uE9zewHu93dZ1z7u6oPO7F+2B8Pd6HpUXARXiTUjGzOsAJwL9jeKwSotKZzCK1lpmdhdd2d1hFscnGvBMWrcU7rD4/0flI4pnZR8DFkSNjYW3zYqCtc+5vFQZLqFL6RBIitZF5ZzWbineY/h68PcoFicxJkkciPqw65x4Me5sSGx2uF0k9/fAmBS7BO8Q7xOmQnIj40OF6ERGRNKU9eRERkTQVtyJvZk+Z2XIz+z5g3MzsATOba2bfmtn+8cpFRESkNornxLtn8Fpgng0YPx7v+8SOeCc+eTTyf7maN2/u2rVrVzMZioiIJC0HG37ly9lLVjrnyp5UKSZxK/LOuQ/MrF05If2AZyMThj4zb03jlpGTjwRq164d06dPr8FMRUQkXa3cWMCIF2fwybxVbCsqSXQ6MWtjK7g/+yEOyNiI3bzdKbkrJZEtdK3Z/mQZ+ZHryi3yIiIisRrx4gze+2lFotOolOMzPuef2U/QyDZX+74SWeT9TsXpO9XfzIYDwwHy8vLimZOIiKSRT+YFrd+UfOpQwA1Z/+H0rHdq7D4TObs+H2/Vp1Jt8Pp+d+CcG+Wc6+6c696iRZW+lhARkVooVQ7Rd7JFvJzz9xot8JDYIj8J+FNklv1BwLqKvo8XERFJL47TM6cyKed69syIZZmDyonb4XozG4O3LnlzM8sHbsRblxvn3GPA63gLGszFW8d6aLxyERERKbXgzhMTnYJnyxqYdAn8OCk4Zq+T2H5hwsqJ5+z60yoYd8Bf4rV9ERFJXqk6673G/PIZjD8H1i3yH8+qA8fdDt3PhtOSsMiLiIgEScVZ7zWipBg+uhfevQNcsX9Mi71g8NOwS5dqb05FXkREQpeoWe85WQmcirZ+KUw4FxZ8GBxzwFlw3B2QU69GNqkiLyIioUvUIfpDOjRLyHaZPRleugA2B3y4yW0Mfe+HrgNqdLMq8iIikvZysjI4pEMz7jll33A3XFQAU26Czx4JjmnTAwaNhp12q/HNq8iLiEhSSJpZ7zVl5VwYNxR+/TYgwODwy6HXNZCZHZcUVORFRERq2jdj4LUroHCT/3iDXWDgKNi9V1zTUJEXEZFQrdxYkOgU4qdgg1fcv30hOGaP3tD/UWgQ/zO4qsiLiEioRrw4I9EpxMeSr2Hc2bD6Z//xjGw45iY46ELICGeWv4q8iIiEyq99LqGtbdVVUuJNrJtyE5QU+sc03R0GPwWt9gs1NRV5EREJlV/7XMJa26pr4wp4+UKY81ZwzD5/gBP/BbkNw8srQkVeREQSLvTWtprw83swYThsXOY/nl3fK+7dyj3Le1ypyIuISMI1b5Cb6BRiV1wI794OH40EnH/Mrvt4p6ZtvkeoqZWlIi8iIhKrNQth/DDInxYcc9CF3gS7rMR/cFGRFxGRKql1K8nNnAiT/goF6/zH6zXzWuM6HRduXuVQkRcRkSqpNSvJbdsMk6+BL58Jjml3OAx8Ahq1DC2tWKjIi4hIldTUSnJJ3T637Afv1LQrZvmPWyYceQ0cdjlkZIabWwxU5EVEpEpq6hB9UrbPOQfTn4LJ10LRVv+Yxm1h0JOQd2C4uVWCiryIiCREwlaGq8iWNTDpEvhxUnBM577Q9wGou1N4eVWBiryIiNSYlF9J7pfPYPw5sG6R/3hWHehzBxwwFMzCza0KVORFRKTS0m6RmZJi+PBeeO8OcMX+MS328nrfd+kSbm7VoCIvIiKVllaLzKxf4p25bsGHwTEHDIXjboeceuHlVQNU5EVEpNLSZpGZn96Ely6ALav9x3Mbe9+9d+0fbl41REVeREQqLeUXmSkqgLdvhM8fDY5p0wMGjYaddgsvrxqmIi8iIjUi6WbJB1k51+t9//XbgACDwy+HXtdAZnaoqdU0FXkREakRKbHIzDdj4LUroHCT/3iDXWHgKNi9Z7h5xYmKvIiIpL+CDV5x//aF4JiOx3rnnq/fPLy84kxFXkREgDRecGbxVzDubFgz3388Ixt63+ytHpcCve+VoSIvIiJAGi44U1ICnz0MU26GkkL/mKa7w+CnoNV+4eYWEhV5EREBqrfgTNK1z21cAS+dD3OnBMfsMwROvAdyG4aXV8hU5EVEBKjegjNJ1T7383veyW02LvMfz64PJ90L+w4JNa1EUJEXEZEqS6pFZooL4d3b4aORgPOPabmvd2raZh1CTS1RVORFRCRQyiw4s2YhjB8G+dOCYw76CxxzI2SlQKtfDVGRFxGphLSdgZ7KZk6ESX+FgnX+4/WaQf/HoNOx4eaVBFTkRUQqIe1moKeybZvhzavhq38Hx7Q/AgaMgkYtw8sriajIi4hUQnVmoKeapJsxH23ZTK/3fcUs/3HLhCOvhcMug4zMcHNLIiryIiKVUJsO0SfVjPlSzsH0J2HydVC01T+mcVsY9CTkHRhubklIRV5ERLaTVDPmo21eDZMuhlmvBsd07ustDVt3p/DySmIq8iIi1ZQyM9BT2cJPYfw5sD7ffzyrDvS5Ew44K+1OTVsdKvIiIpK8Sorhw3/Be3eAC/iqpEVn79S0u3QJN7cUoCIvIuJDrXJJYP0S78x1Cz4MjjlgKBx3O+TUCy+vFKIiLyLiQ61yCfbTm/DSBbBltf94ncbQ90Ho0i/cvFKMiryIiI9YW+WSus0sFRUVwNs3wOePBce0PRAGjYYmeeHllaJU5EVEfMR6iD4p28xS1co5MG4o/PpdQIDB4VdAr2sgU+UrFnqWRESqIGnbzFKRc/DN8/D6lVC4yT+mwa4wcBTs3jPc3FKciryISIzUKhcHW9fDa5fDdy8Gx3Q8Dvo/AvWbh5dXmlCRFxGRxFj8lXdq2jXz/cczsqH3LXDQBep9ryIVeRFJGWprSxMlJfDZwzDlZigp9I9p2sHrfW/VLdzc0oyKvIikDLW1pYGNK+Cl82HulOCYfU+DE+6G3Ibh5ZWmVORFJGUkcgU4tcrVgHnvwsTzYOMy//GcBnDivbDvH8LNK42pyItIykjkIXq1ylVDcSG8ext8dB/g/GNadvMOzzfrEGpq6U5FXkSkHGqVq6Y1C7yFZfKnBcccfBEcfSNk5YSWVm2hIi8iKU1tbUns+wnwyl+hYL3/eL1m0P8x6HRsuHnVIiryIpKUNJM+hW3bDG9eBV89GxzT/ggYMAoatQwvr1pIRV5EkpJm0qeoZTPhxaGw8if/ccuEo66DQy+FjMxwc6uFVORFJCnFMpNeM96TiHMwbTRMvg6KC/xjGud5C8vkHRhubrWYiryIJKVYDtFrxnuS2LwaJl0Ms14NjunSD05+AOo2CS8vIa4fg82sj5n9ZGZzzexqn/E8M3vXzL42s2/N7IR45iMi6SEnK4Nee7bQjPdksPBTeOzw4AKfVQdOug9O+bcKfALEbU/ezDKBh4HeQD4wzcwmOed+iAq7Hvg/59yjZtYFeB1oF6+cRCS1aSZ9Eikphg/ugffvBBdw1GXnLl7v+86dw81NfhPPw/U9gLnOuZ8BzGws0A+ILvIOaBT5uTGwJI75iIhITVi3GCYMh4UfBcd0PxuOux2y64aXl+wgnkW+NbAo6nI+UHa2xU3AW2Z2MVAfOMbvjsxsODAcIC8vr8YTFZFwqT0uhc16HV6+ELas8R+v0xj6Puh9By8JF8/v5P3WBSx7PsPTgGecc22AE4D/mNkOOTnnRjnnujvnurdo0SIOqYpImErb41TgU0jhVnj9bzD2tOAC3/YgOP8jFfgkEs89+XygbdTlNux4OH4Y0AfAOfepmdUBmgPL45iXiCRYVRaaUbtcAq2cA+OGwq/fBQQYHDECel4NmWraSibx/KuZBnQ0s/ZmlgMMASaVifkFOBrAzDoDdQCd/UIkzVVlD17tcgngHHz9X3i8Z3CBb9gS/jwJjrpeBT4Jxe034pwrMrOLgMlAJvCUc26mmd0CTHfOTQKuAJ4ws8vwDuWf5ZwLWKJIRGojLRCTIFvXw2uXw3cvBsd06gP9HoH6+gCWrOL6scs59zpeW1z0dTdE/fwDcGg8cxCR1KD2uCSy+EsYNwzWzPcfz8yB3rfAgeeD+U2/kmShYysiUmWaJZ9mSkrg04dg6s1QUuQf07SD1/veqlu4uUmVqMiLSJVpEZk0snE5TDwf5k0Njtn3dDjhbshtEF5eUi0q8iJSZVWZJe9HM+cTbN47MOE82BTQ2JTTAE68F/b9Q7h5SbWpyItIldXUIXrNnE+Q4kJ451b4+L7gmJbdvMPzzTqEl5fUGBV5EUkYzZxPoDULvMl1i6cHxxx8ERx9I2TlhJaW1CwVeRGpUZolnwK+Hw+vXAoF6/3H6zWHAY9Bx97h5iU1TkVeRKS22LYJ3rgKvv5PcEz7njBwFDTcNby8JG5U5EUkZmqZS2G/fu+dmnblbP9xy4SjroNDL4MMTYRMFyryIhIztcylIOdg2miYfB0UF/jHNM6DwU9C2x7h5iZxpyIvIjGrqGVOrXBJZvNqmHQxzHo1OKZLfzj5fqjbJLy8JDQq8iISs4oO0asVLoks/ATGnwPrF/uPZ9WF4++E/f+sU9OmMRV5Eak2tcIlkZJi+OBueP+f4AI+lO3cBQY/DTvvFW5uEjoVeRGpFrXMJZF1i2HCcFj4UXBM92Fw3G2QXTe8vCRhVORFRNLBrNfh5Qthyxr/8TqNoe9D0KVvuHlJQqnIi9RCaoVLI4Vb4e0b4IvHg2PaHgSDRkOTtuHlJUlBRV6kFlIrXJpYMRvGnQ3LvgsIMDjiSuh5FWTq7b420m9dpBbS6nEpzjn45r/w+pVQuNk/pmFLGPgEtD883NwkqajIi9RCWj0uhW1dD69eBt+PC47p1Af6PQL19fup7VTkRaTS1DKXIPlfwvizvRXk/GTmQO9/wIHnqfddABV5EYlQK1wSKymBTx+EqbdASZF/TLM9vHXfW+qDl/yPirxImtHM+TSzcTlMPB/mTQ2O6XYGHH8X5DYILy9JCSryImlGM+fTyNypXoHftNx/PKchnHQv7HNquHlJylCRF0kzVZk5r1nySaa4EN75B3x8f3BMq/28w/NNdw8vL0k5KvIiaaYqh+g1Sz6JrJ4P44fB4i+DYw6+CI6+EbJywstLUpKKvEgtplnySea7cV57XMF6//F6zWHAY9Cxd7h5ScpSkRepBTRzPslt2wRv/A2+fi44ZvdeMOBxaLhrWFlJGlCRFxFJpF+/805Nu3K2/7hlwlHXw6GXQobmTkjlqMiLpDi1zKUo5+CLJ+Ct66G4wD+mSR4Megra/j7c3CRtqMiLpDi1zKWgzavh5Yvgp9eCY7oOgJPug7pNwstL0o6KvEiKq6hlTu1xSWbBxzDhXFi/2H88qy4c/0/Y/086Na1Um4q8SIqr6BC92uOSREkxfHA3vP9PcAG/s527er3vO+8Vbm6StlTkRdKU2uOSyLp8mDAcFn4cHPP7c+DYWyG7bnh5SdpTkRdJQ2qZSyKzXoOX/wJb1viP12kC/R6CzieHm5fUCiryIikoeka9JKnCrfD23+GLUcExeQfDwCegSdvw8pJaRUVeJAVpRn2SWzHb631f9l1AgEHPv8ERf4NMvQ1L/OjVJZKCytuD12z6BHLOO2vdG3+Dws3+MQ1bwcBR0P7wcHOTWklFXiQFlTejXrPpE2TrOu+889+PD47pdDz0exjq63ck4VCRF0kTmk2fQPlfwrihsHah/3hmjjdzvsdw9b5LqFTkRdLE7FuPT3QKtU9JCXzygLf2e0mRf0yzjl7ve8t9ws1NBBV5EZGq2bAMXjof5r0THNPtDDj+LshtEF5eIlFU5EVSzMqNAYuZSHjmToWJ58GmgA6HnIZw0kjY55Rw8xIpQ0VeJMWMeHFGolOovYq2eYfmP3kgOKbVft7h+aa7h5eXSAAVeZEU49c+p7a5EKyeD+OHweIvg2MOuRiOugGycsLLS6QcKvIiKcavfU5tc3H23Th45VLYtsF/vH4L6P8YdDwm3LxEKqAiL5IG1DYXJ9s2eSe2+fq54Jjde8GAUdBwl7CyEomZirxIGmjeIDfRKaSfpd96p6ZdNcd/PCMLjroeDvkrZOjrEklOMRV5M8sB8pxzc+Ocj4iw/QI0Fa0XLzXMOfjiCXjrOije5h/TJA8GPw1tuoebm0glVfjx08xOBL4D3o5c7mZmE+OdmEhtVroAjQp8yDavhrGnwxtXBhf4rgPh/I9U4CUlxLInfwtwIPAugHPuGzPbI65ZidRylVlCVjPra8iCj2D8ubBhif94Vl044S7Y7486Na2kjFiKfKFzbq1t/6J2ccpHRCh/AZqyNLO+moqL4IO74YO7wAU877vs7fW+t9gz3NxEqimWIv+jmZ0KZJhZe+CvwGfxTUtEKqIFaWrAunxv7/2XT4Jjfn+ut7hMdp3w8hKpIbEU+YuAG4ASYAIwGbgmnkmJyI4W3HliolNIL7Neg5cuhK1r/cfrNPGWhe18Urh5idSgWIr8cc65q4CrSq8ws4F4BV9EJLUUboW3rodpTwTH5B0Cg56Axm3Cy0skDmIp8tezY0G/zuc6EakCtcuFaMVPXu/7su/9xy0DjvgbHHElZOo0IpL6Al/FZnYc0AdobWb3Rg01wjt0LyI1oLRdTuLIOfj6P/DGVVC42T+mYStv773dYeHmJhJH5X1UXQ58D2wFZkZdvwG4Op5JidQmsbTLqU2uGrau8847P7Ocg4+djof+j0C9puHlJRKCwCLvnPsa+NrM/uuc21qVOzezPsD9QCYw2jl3p0/MqcBNeG15M5xzp1dlWyKpKpZD9GqTq6L86d7h+bUL/cczc7yZ8z2Gq/dd0lIsXzq1NrPbgC7Abz0kzrlO5d3IzDKBh4HeQD4wzcwmOed+iIrpiDdT/1Dn3Boz27kKj0EkbalNropKSuCT++GdW6GkyD+mWUev973lPuHmJhKiWIr8M8CtwD3A8cBQYvtOvgcw1zn3M4CZjQX6AT9ExZwLPOycWwPgnFsec+YiaUztctWwYRlMPA9+fjc4ptuZ3tnrcuqHl5dIAsTyRV8959xkAOfcPOfc9cCRMdyuNbAo6nJ+5LponYBOZvaxmX0WOby/AzMbbmbTzWz6ihWaoCQiAeZOgccODS7wOQ1h0JPQ/2EVeKkVYtmTLzDvnLbzzOx8YDEQy2F1vy+4yp4ONwvoCPQC2gAfmtnezrntzk7hnBsFjALo3r27TqkrItsr2gbv/AM+eSA4ptX+MPhJaLqY1i7uAAAgAElEQVR7eHmJJFgsRf4yoAFwCXAb0Bg4O4bb5QNtoy63Acqu/JAPfOacKwTmm9lPeEV/Wgz3LyICq3+GccNgyVfBMYdcAkf9HbJywstLJAlUWOSdc59HftwA/BHAzGI5DdQ0oGPkfPeLgSFA2ZnzLwGnAc+YWXO8w/c/x5a6iNR6343z2uO2bfAfr98CBjwGexwTbl4iSaLcIm9mv8f7Hv0j59xKM+uKd3rbo/D2zAM554rM7CK8c91nAk8552aa2S3AdOfcpMjYsWb2A1AMXOmci32NTRGpnQo2eie2+ea54Jjdj4QBj0PDXcLLSyTJlHfGuzuAQcAM4Hozm4i3At0/gfNjuXPn3OvA62WuuyHqZwdcHvknIlKxpd96ve+r5viPZ2R5h+YPuQQydBIhqd3K25PvB+zrnNtiZk3xvk/f1zn3UzipiYhEcQ6+GOUtLlO8zT+myW5e73ub7uHmJpKkyivyW51zWwCcc6vNbJYKvIg/LTITZ5tXw8t/gZ9eD47pOhBOvg/qNA4vL5EkV16R393MSk/2bEC7qMs45wbGNTORFKJFZuJowUcw/lzYULY5JyK7Hhx/F+x3pk5NK1JGeUV+UJnLD8UzEZFUFssiM7HSYjQRxUXwwV3wwd3gAo6O7LK3d3i+xZ7h5iaSIspboGZqmImIpLKaPESvxWiAdfne3vsvnwTH9BgOvf8B2XWCY0RquVhOhiMiIdBiNBE/vup9/751rf943Z2g38Owl87vL1IRFXmRONEiM5VUuMWbOT9tdHBM3iEw6AloHMv5uEQk5iJvZrnOuYJ4JiMitdSKn+DFobB8pv+4ZUDPq+DwEZCpfRORWFX412JmPYAn8c5Zn2dm+wLnOOcujndyIlWllrYU4Rx89ax39rqiLf4xDVvBoNHQ7tBwcxNJA7F8JH4AOAnvPPM452aYWSxLzYokjFraUsDWdd5552dOCI7Z8wTv+/d6TcPLSySNxFLkM5xzC237/tPiOOUjUiNqsqWtKtQGV4FF02D82bD2F//xzFw49lboca5630WqIZYivyhyyN6ZWSZwMTA7vmmJVE+iD9GrDS5ASQl8cj+8cyuUFPnHNOsIpzwNu/4u3NxE0lAsRf4CvEP2ecAyYErkOhEpQ21w5diwDCYOh5/fC47Z70zv7HU59UNLSySdxVLki5xzQ+KeiUicqaUtgeZMgYnnweaV/uM5Db3zzv9ucLh5iaS5WIr8NDP7CXgBmOCc2xDnnEQCadZ8iinaBu/cAp88GBzT+gAY9CQ0bR9eXiK1RIWzg5xzHYBbgQOA78zsJTPTnr0kROmseRX4FLBqHjx1bPkF/tC/wtA3VeBF4iSmKcDOuU+cc5cA+wPrgf/GNSuRAFWdNa/Z7iH79kV4vCcs+dp/vH4LOHMC9L4FsnLCzU2kFonlZDgNgH7AEKAz8DJwSJzzEvFV1T14zXYPScFGeONv8E05+wEdjoIBj0ODncPLS6SWiuU7+e+BV4C7nHMfxjkfkRql2e4hWvotjBsKq+b6j2dkwdE3wMEXQ4aOrIiEIZYiv7tzQYs5iySeZs0nmHPw+ePw9t+heJt/zE7tYNBT0OaAUFMTqe0Ci7yZ/cs5dwUw3sxc2XHn3MC4ZiYiyW/TKm9Z2NlvBMfsPRhOuhfqNA4vLxEByt+TfyHy/0NhJCJSltrlktz8D2HCubBhqf94dj044W7odoZOTSuSIIFF3jn3ReTHzs657Qq9mV0ETI1nYiJaZCZJFRfB+/+ED+4GdjjI59llbxj8NLToFGpqIrK9WGa/nO1z3bCaTkSkrFja5dQaF7K1i+DfJ8EHdxFY4HucB+dMVYEXSQLlfSf/B7y2ufZmFr0WZENgbbwTE4nlEL1a40L04yvw8kWwNeDPv+5O3rKwe2kipEiyKO87+S+AVUAb4OGo6zcAAWe4EAmHWuNCVLgFJl8H058MjtntUBj4BDRuHV5eIlKh8r6Tnw/Mx1t1TiQpqF0uZMtnwbizYflM/3HLgJ5XwxEjICMz3NxEpELlHa5/3znX08zWsP2XbwY451zTuGcnIonhHHz1LLxxFRRt8Y9p1Nrbe293aLi5iUjMyjtcf2Tk/+ZhJCISbeXGgkSnUHttWQuvXgozJwbH7Hki9HsI6umzvkgyK+9wfemsp7bAEufcNjM7DNgHeA5voRqRuBjx4oxEp1A7LZoG48+Gtb/4j2fmwnG3we/PUe+7SAqIpf/oJcCZWQfgWbxFap6Pa1ZS6/m1z6ldLo5KSuDDe+Gp44ILfPNOcO5U6HGuCrxIiojl3PUlzrlCMxsI3Oece8DMNLte4sqvfU7tcnGy4VeYMBzmvx8cs98f4fh/Qk798PISkWqLpcgXmdkpwB+B/pHrsuOXkog/tcvFwZwpMPE82LzSfzy3EZx8H+w9KNy8RKRGxFLkzwYuxFtq9mczaw+MiW9aIjtq3iA30Smkj6JtMPVm+LScpSlad4dBo6Fp+/DyEpEaVWGRd859b2aXAHuY2V7AXOfcbfFPTVKdFphJUqvmwfhhsKScb90OvRSOuh4yddBOJJVVWOTN7HDgP8BivB75Xc3sj865j+OdnKQ2LTCThL79P3j1Mti20X+8/s4w8HHocFS4eYlIXMRyuH4kcIJz7gcAM+uMV/S7xzMxSX2xLDATK82sr6aCjfD6lTCjnMaYDkfDgMegwc7h5SUicRVLkc8pLfAAzrkfzSwnjjlJmqjJQ/SaWV8NS2d4p6ZdNdd/PCMLjr4RDr4IMvRhSiSdxFLkvzKzx/H23gHOQAvUSEi0EE01OAefPwZv3wDF2/xjdmoHg56CNgeEmpqIhCOWIn8+cAnwN7zv5D8AHoxnUpK+tMBMSDatgpcvhNlvBsfsPRhOGgl1GoWXl4iEqtwib2a/AzoAE51zd4WTkohUy/wPYcK5sGGp/3h2PTjhHuh2us5cJ5LmyluF7lpgGPAV8Hszu8U591RomUlKUbtcEigugvfvhA/uYfuFI6Ps8jsY/BS06BRqaiKSGOXtyZ8B7OOc22RmLYDXARV58aV2uQRbuwjGnwOLPguOOfB8OOZmyK4TXl4iklDlFfkC59wmAOfcCjPTtFsJFEu7nNrg4uSHSTDpIti6zn+87k7Q7xHY64Rw8xKRhCuvyO9uZhMiPxvQIeoyzrmBcc1MUkosh+jVBlfDCrfA5Otg+pPBMbsdBgNHQePW4eUlIkmjvCJfdkWKck5yLRJMbXBxsPxHr/d9+Q/+45YBPa+GI0ZARma4uYlI0ggs8s65qWEmIulH7XJx4Bx8+Qy8eQ0UbfGPadTaW1hmt0NCTU1Ekk8sffJSy2imfJLashZe+Sv88FJwzF4nQd8HoV7T8PISkaSlIi870Ez5JLToCxg3DNb94j+emQvH3Qa/P0e97yLym5iLvJnlOucK4pmMJIeaWFhGM+lrSEkJfDwS3rkNXLF/TPM9vd73XfcONzcRSXoVvhObWQ8z+w6YE7m8r5nptLZprCYO0WsmfQ3Y8Cv8pz9MvSW4wO//Jxj+rgq8iPiKZU/+AeAk4CUA59wMMzsyrllJytJM+hoy522YeD5sXuk/ntsITr4P9i7bBCMi8j+xFPkM59xC2/57voDdCklXmikfkqJtMPVm+LScjtXW3WHwk94KciIi5YilyC8ysx6AM7NM4GJgdnzTEqmFVs3zet+XfhMQYHDYpXDkdZCZHWpqIpKaYinyF+Adss8DlgFTItdJGlq5UXMrE2LGC/Da5bBto/94/Z1h4OPQ4ahw8xKRlFZhkXfOLQeGVOXOzawPcD+QCYx2zt0ZEDcYeBH4vXNuelW2JTVjxIszEp1C7VKwEV4fATPGBMd0OBoGPAYNdg4vLxFJCxUWeTN7Ap91K51zwyu4XSbwMNAbyAemmdkk59wPZeIaApcAn1cib4kTv/Y5tcPFyZJvvMPzq+f5j2dkwTE3wUF/gQz9DkSk8mI5XD8l6uc6wABgUQy36wHMdc79DGBmY4F+QNmTbf8DuAsYEcN9Spz5tc+pHa6GOQefPQpTboTibf4xO7X3Jte1PiDc3EQkrcRyuP6F6Mtm9h/g7RjuuzXbfxjIBw4sc1/7AW2dc6+amYp8klI7XA3atBJeuhDmTA6O+d0pcOK9UKdReHmJSFqqymlt2wO7xRDnd27N3w77R9anHwmcVeEdmQ0HhgPk5eXFlKTUnOYNchOdQnqY/wGMPxc2/uo/nl0PTrgHup2uU9OKSI2I5Tv5NfyvOGcAq4GrY7jvfKBt1OU2wJKoyw2BvYH3Ij34uwKTzKxv2cl3zrlRwCiA7t277zA/QCSpFRfB+3fCB/fgM73Fs+vvYPDT0LxjqKmJSHort8ibV333BRZHripxzsVaZKcBHc2sfeT2Q4DTSwedc+uA5lHbeg8Yodn1NUsryiXY2l9g/DmwqJx5pQeeD71vgSwdMRGRmlVukXfOOTOb6Jyr9Owf51yRmV0ETMZroXvKOTfTzG4BpjvnJlUtZakMrSiXQD+8DJMuhq3r/MfrNoX+j8Cex4ebl4jUGrF8J/+Fme3vnPuqsnfunHsdeL3MdTcExPaq7P1Lxaq7opza56qgcAu8eQ18+XRwzG6HwaAnoFGr8PISkVonsMibWZZzrgg4DDjXzOYBm/Am1Dnn3P4h5SjVUN1D9Gqfq6TlP3q978vLdopGWAb0ugYOvwIyMsPNTURqnfL25L8A9gf6h5SLJBGtJldJzsGXz3h78EVb/GMatYFBo2G3g0NNTURqr/KKvAE45wJOxyWpSivK1bAta+GVS7zv4IPsdRL0fRDqNQ0vLxGp9cor8i3M7PKgQefcvXHIR2pI6ax6ibNFX8C4YbDuF//xzFzoczt0H6bedxEJXXlFPhNogP9JbSTJaVZ9nJUUw0cj4d3bwRX7xzTfE055GnbpGm5uIiIR5RX5pc65W0LLRGpU0Kx6zZavAeuXwsTh3hnsguz/Z+hzJ+TUCy8vEZEyKvxOXlJT0Kx6zZavptlvwUvnw+aA1sTcRnDy/bD3wHDzEhHxUV6RPzq0LCQUvfZsodnyVVVUAFNuhs8eDo5p83tv9vxO7UJLS0SkPIFF3jm3OsxEJP6eGdoj0SmkplXzYNxQWBo0kdHgsMvgyGshMzvU1EREylOVVehEao8ZY+G1K2DbRv/xBrvAgMehw5Hh5iUiEgMV+TS0cmNBolNIfQUb4LUR8O3Y4Jg9ekP/R6FBi/DyEhGpBBX5NKT++Gpa8o13atrVAeeBysiGY26Eg/4CGepWEJHkpSKfhvza59Q6FwPn4LNH4e0boKTQP2an9jD4KWitpRtEJPmpyKchv/Y5tc5VYNNKeOkCmPNWcMzvToUT/wV1GoWXl4hINajI1xJqnSvHz+/DhOGw8Vf/8ez6cOI9sO9pOjWtiKQUFflaonmD3ESnkHyKi+C92+HDewHnH7PrPt7h+eYdQ01NRKQmqMinkNJFZz6Zt6ra68TXemt/gfHnwKLPg2MOvAB63wxZ+oAkIqlJRT6FaNGZGvLDyzDpYti6zn+8blOvNW7PPuHmJSJSw1TkU0jQojMV0cz6iMIt8OY18OXTwTHtDoeBo6BRq/DyEhGJExX5FFLVQ/SaWQ8s/xFeHAorfvQftwzodS0cfjlkZIabm4hInKjIp7GcrAwO6dCsds+sd87bc3/zGija6h/TuK23sEzeQeHmJiISZyryKW7BnScmOoXktWUNTLoEfpwUHNP5ZOj7INTdKby8RERCoiIv6emXz2H8MFi3yH88Mxf63AHdz1bvu4ikLRX5JKV2uSoqKYaP7oV37wBX7B/TYi+v932XruHmJiISMhX5JKV2uSpYvxQmnAsLPgyOOeAsOO4OyKkXWloiIomiIp+kYmmXU2tclNmTvXPPbw543nIbQ9/7oeuAcPMSEUkgFfkkFcsherXGAUUFMOUm+OyR4Jg2PbzZ8zvtFlpaIiLJQEU+Bak1LmLVPBg3FJbOCAgwOOwyOPJayMwONTURkWSgIp9C1C4XZcZYeO0K2LbRf7zBLt6Z63bvFWZWIiJJRUVeUkvBBq+4f/tCcMwevb1zzzdoEV5eIiJJSEU+yZS2zomPJV/DuLNh9c/+4xnZcMxNcNCFkKFJiSIiKvJJRq1zPkpKvIl1U26CkkL/mKa7w6AnofX+oaYmIpLMVOSTTFDrXK1tl9u00muNm/NWcMw+f4AT/wW5DcPLS0QkBajIJ5mg1rla2S738/swYThs/NV/PLu+V9y7nRZuXiIiKUJFPgX02rNF7WqXKy6E9+6AD+8FnH/MrvvA4Keh+R6hpiYikkpU5FPAM0N7JDqF8KxZCOPPgfwvgmMOutCbYJeVG1ZWIiIpSUU+gbQITRkzX/KWhi1Y5z9erxn0ewT27BNuXiIiKUpFPoE0kz5i22aYfA18+UxwTLvDYeAT0KhlaGmJiKQ6FfkE0iI0wLIfvN73FT/6j1smHHkNHHY5ZGSGm5uISIpTkU+gWr0IjXMw/SmYfC0UbfWPadzWW1gm76BwcxMRSRMq8kkqrReh2bLG++79x0nBMZ37Qt8HoO5O4eUlIpJmVOSTTNovQvPLZ97s+XWL/Mez6kCfO+CAoWAWbm4iImlGRV7CUVLs9b2/dwe4Yv+YFnt5ve+7dAk3NxGRNKUiH6Ja2zK3filMOBcWfBgcc8BZcNwdkFMvtLRERNKdinyIamXL3E9veuee37Lafzy3MfS9H7oOCDcvEZFaQEU+RBW1zKVVu1xRAbx9I3z+aHBMmx7e7PmddgsvLxGRWkRFPkQVHaJPm3a5lXNh3FD49duAAIPDL4de10BmdqipiYjUJirySSCt2uW+GQOvXQGFm/zHG+wKA0fB7j3DzUtEpBZSkU+wtGmZK9jgFfdvXwiO6Xgs9H8U6jcPLy8RkVpMRb4Kau0s+SBLvvZOTbv6Z//xjGzofTMceAFkpNG8AxGRJKciXwW1cpa8n5IS+OwRmHITlBT6xzTdHQY/Ba32CzU1ERFRka+SWBaWiUVKz6bfuMJrjZv7dnDMPkPgxHsgt2F4eYmIyG9U5Kugpg7Rp+xs+p/fgwnDYeMy//Hs+nDSvbDvkFDTEhGR7anIJ0DKzqYvLoR3b4ePRgLOP6blvt6paZt1CDU1ERHZkYp8DUmbWfJB1iyE8cMgf1pwzEF/gWNuhKzc8PISEZFAKvJSsZkTYdJfoWCd/3i9ZtD/Meh0bLh5iYhIueJa5M2sD3A/kAmMds7dWWb8cuAcoAhYAZztnFsYz5yqota2zG3bDG9eDV/9Ozim3eEw8Alo1DK8vEREJCZxK/Jmlgk8DPQG8oFpZjbJOfdDVNjXQHfn3GYzuwC4C/hDvHKqqlrZMrdsptf7vmKW/7hlwpHXwmGXQUZmuLmJiEhM4rkn3wOY65z7GcDMxgL9gN+KvHPu3aj4z4Az45hPldWqhWWcg+lPwuTroGirf0zjtjDoScg7MNzcRESkUuJZ5FsDi6Iu5wPlVYVhwBt+A2Y2HBgOkJeXV1P5xazWLCyzeTW8cgn8+EpwTOe+0PcBqLtTeHmJiEiVxLPIm891vn1XZnYm0B3wXbXEOTcKGAXQvXv3gN6t8KVsK5yfhZ/C+HNgfb7/eFYd6HMHHDAUzO9XKyIiySaeRT4faBt1uQ2wpGyQmR0DXAf0dM4VxDGfGpU2LXMlxfDhvfDe7eACjli06OydmnaXLuHmJiIi1RLPIj8N6Ghm7YHFwBDg9OgAM9sPeBzo45xbHsdcqqR0Vn3aWr/EO3Pdgg+DYw4YCsfdDjn1wstLRERqRNyKvHOuyMwuAibjtdA95ZybaWa3ANOdc5OAu4EGwIvmHQL+xTnXN145VVZaz6r/6U3v3PNbVvuP12kMfR+ELv3CzUtERGpMXPvknXOvA6+Xue6GqJ+Pief2qytoVn1Kz6YvKoC3b4TPHw2OaXsgDBoNTcKf5CgiIjVHZ7wrR9Cs+pSdTb9yLowbCr9+GxBgcPgV0OsayNRLQ0Qk1emdvJJ67dki9WbTOwczxsBrI6Bwk39Mg11h4CjY3bfBQUREUpCKfCU9M7RHolOonK3r4bUr4Lv/C47peCz0fxTqNw8vLxERiTsV+XS2+Cvv1LRr5vuPZ2RD71vgoAvU+y4ikoZU5NNRSQl89jBMuRlKCv1jmnbwet9bdQs3NxERCY2KfLrZuAJeOh/mTgmO2fc0OOFuyG0YXl4iIhI6Ffl0Mu9dmHgebFzmP57TAE68F/ZNuoX+REQkDlTk00FxIbx7G3x0HwHLA0DLfWHw09CsQ6ipiYhI4qjIp7o1C7yFZfKnBcccfBEcfQNk5YaWloiIJJ6KfCr7fgK88lcoWO8/Xq8Z9H8MOh0bbl4iIpIUVORT0bbN8OZV8NWzwTHtj4ABo6BRy/DyEhGRpKIiX0bpynNB561PuGUz4cWhsPIn/3HLhCOvhcMug4zMcHMTEZGkoiJfRtKuPOccTH8S3rwWigv8YxrneQvL5B0Ybm4iIpKUVOTLKG8PPmGrz21eDZMuhlmvBsd06QcnPwB1m4SXl4iIJDUV+TKCVp6DBK0+t/BTb/b8+nz/8aw60OdOOOAsnZpWRES2oyIfg5ysDA7p0Czc1edKiuHDf8F7d4AL+ODRojOc8jTs3Dm8vEREJGWoyMdg9q3Hh7vB9UtgwnBY8GFwTPez4bjbIbtueHmJiEhKUZGPsnJjwIS2MP30Brx0IWxZ7T9epzH0fdD7Dl5ERKQcKvJRRrw4I3EbL9wKU26Ezx8Ljml7oDd7vkleeHmJiEjKUpGP4jezPpQZ9SvnwLih8Ot3AQEGR4yAnldDpn5lIiISG1WMKH4z6+M6o945+OZ5eP1KKNzkH9OwJQwc5Z3BTkREpBJU5CsQtxn1W9fDa5fDdy8Gx3TqA/0egfoJaN0TEZGUpyJfgeYN4rBy2+IvYdwwWDPffzwzB3rfAgeer953ERGpMhX5MJWUwKcPwdSboaTIP6ZpBxj8FLTqFm5uIiKSdlTkw7JxObx0AcydEhyz7+lwwl2Q2zC8vEREJG2pyIdh3jsw4TzYtNx/PKcBnHgv7PuHcPMSEZG0piIfT8WF8M6t8PH9gPOPadnNOzzfrEOoqYmISPpTkY+XNQu8yXWLpwfHHHwRHH0jZOWElpaIiNQeKvLx8P14eOVSKFjvP16vOQx4DDr2DjcvERGpVVTka9K2TfDm1fDVs8Ex7Xt6J7dpuGt4eYmISK1Uq4v8yo0FjHhxBp/MW1XuOvIx+fV779S0K2f7j1smHHUdHHoZZIRwqlwREan1anWRH/HiDN77aUX17sQ5mDYaJl8HxQGr2DXOg8FPQtse1duWiIhIJdTqIu+3IE20Chen2bwaJl0Ms14NjunSH06+H+o2qUKGIiIiVVeri3xFh+jLXZxm4Scw/lxYn+8/nlUXjr8T9v+zTk0rIiIJUauLfJCcrAwO6dDMf3GakmL44B54/05wAR8Sdu4Cg5+GnfeKb6IiIiLlUJEvY8GdJwYPrlsME4bDwo+CY7oPg+Nug+y6NZ+ciIhIJajIx2rW6/DyhbBljf94ncbQ9yHo0jfcvEREqqGwsJD8/Hy2bt2a6FRqvTp16tCmTRuys7Nr7D5rRZGvVqtc4VZ4+wb44vHgmLYHwaDR0KRt9RIVEQlZfn4+DRs2pF27dpjmDyWMc45Vq1aRn59P+/bta+x+a0WRr3Kr3Mo58OJQWPZdQIDBEVdCz6sgs1Y8lSKSZrZu3aoCnwTMjGbNmrFiRTXbusuoFZWpola5Ur+1zDkH3/wXXr8SCjf7Bzds6Z25rv0RNZSliEhiqMAnh3j8HmrFqddiPUR/SIdmsHU9jD8HXv5LcIHv1AfO/1gFXkSkhkycOBEzY9asWb9d995773HSSSdtF3fWWWcxbtw4wJtPcPXVV9OxY0f23ntvevTowRtvvFHtXO644w722GMP9txzTyZPnuwb45zjuuuuo1OnTnTu3JkHHnhgu7y7detG165d6dmz52/Xjxw5kq5du7L33ntz2mmnhTIPolbsyVektGXuvsNK4PHDvRXk/GTmQO9/wIHnqfddRKQGjRkzhsMOO4yxY8dy0003xXSbv//97yxdupTvv/+e3Nxcli1bxvvvv1+tPH744QfGjh3LzJkzWbJkCccccwyzZ88mMzNzu7hnnnmGRYsWMWvWLDIyMli+fDkAa9eu5cILL+TNN98kLy/vt+sXL17MAw88wA8//EDdunU59dRTGTt2LGeddVa18q1IrS3y27XKlZTApw/CmFugpMj/Bk07wClPQ0uf3nkRkRTX7urX4r6NoBbljRs38vHHH/Puu+/St2/fmIr85s2beeKJJ5g/fz65ubkA7LLLLpx66qnVyvHll19myJAh5Obm0r59e/bYYw+++OILDj744O3iHn30UZ5//nkyImuR7LzzzgA8//zzDBw4kLy8vO2uBygqKmLLli1kZ2ezefNmWrVqVa1cY1ErDteXa+Ny+O9gbwZ9UIHf93Q47wMVeBGROHjppZfo06cPnTp1omnTpnz11VcV3mbu3Lnk5eXRqFGjCmMvu+wyunXrtsO/O++8c4fYxYsX07bt/zql2rRpw+LFi3eImzdvHi+88ALdu3fn+OOPZ86cOQDMnj2bNWvW0KtXLw444ACefdZblbR169aMGDGCvLw8WrZsSePGjTn22GMrzL260nJPPuaWuXnvwITzYNNy//GcBnDSSNinep8MRUQk2JgxY7j00ksBGDJkCGPGjGH//fcPnIhW2QlqI0eOjDnWORfT9goKCqhTpw7Tp09nwoQJnH322Xz44YcUFRXx5ZdfMnXqVLZs2cLBBx/MQQcdRIsWLXj55ZeZP38+TZo04ZRTTuG5557jzDPPrNRjqay0LPIVtcxlUeTtuX98f/CdtNoPBj0JzTrEIUMREQFYtWoV77zzDt9//z1mRnFxMWbGXXfdRbNmzVizZvsTkK1evZrmzZuzx7ZIa2AAAAxWSURBVB578Msvv7BhwwYaNmxY7jYuu+wy3n333R2uHzJkCFdfffV217Vp04ZFixb9djk/P9/3sHqbNm0YNGgQAAMGDGDo0KG/Xd+8eXPq169P/fr1OeKII5gxYwYA7du3p0WLFgAMHDiQTz75JO5FPi0P15fXMtfWljE+9+byC/zBF8HZb6nAi4jE2bhx4/jTn/7EwoULWbBgAYsWLaJ9+/Z89NFHdOzYkSVLlvDjjz8CsHDhQmbMmEG3bt2oV68ew4YN45JLLmHbtm0ALF26lOeee26HbYwcOZJvvvlmh39lCzxA3759GTt2LAUFBcyfP585c+bQo8eOy4T379+fd955B4D333+fTp06AdCvX7/f9ug3b97M559/TufOncnLy+Ozzz5j8+bNOOeYOnUqnTt3rrHnMUha7skHHaI/KeNTbs8eTSPb4n/Des1hwGPQsXccsxMRST7lrtsRR2PGjNmh2A4aNIjnn3+eww8/nOeee46hQ4eydetWsrOzGT16NI0bNwbg1ltv5frrr6dLly7UqVOH+vXrc8stt1Qrn65du3LqqafSpUsXsrKyePjhh3+bWX/CCScwevRoWrVqxdVXX80ZZ5zByJEjadCgAaNHjwagc+fO9OnTh3322YeMjAzOOecc9t57bwAGDx7M/vvvT1ZWFvvttx/Dhw+vVq6xML/vH5JZ9+7d3fTp08uNKTtLtC5buTHrWYZkvRd8o917wYDHoeGu1c5RRCRV/Pjjj6HsUUps/H4fZvalc657Ve4vLffko+1lv/BQ9gPskbHEP8Ay4ajr4dBLISMtv70QEZFaKq2KfOmseo/jj5lvc33Wf8m1Qv8bNMmDQU9B29+HlqOIiEhY0qrIl86qb8xG7s5+nGMzvwwO7joATvr/9u4+Ro66juP4+0O5ckWwQJsa4JCWAIUDoZaCVRIRiwQwUCWVtuGpykMoICkIRgNRFP7gQaJFiqViU2p4qCVFGixBguUhtUdphBZoCpRS8QxKrbUqtFLar3/M78Ky7N3NHrd7t3OfVzLJ7sxvZr73zWa/N7+Z/f1+BkP2ql+AZmZmdVSoIv/H1zdxrNYyc/Ad7Kd/Vm606xA49WYYe56HpjUzI/ttuCep6Xu1eEauOEV+5w6mx0KuGLyIQeokUSOOgElzYcRh9Y3NzKyfam5uZtOmTQwbNsyFvg91zCff3Nzcq8ctRpHf8ldYdBFXNi3rvM2xF8LJN0LTkPrFZWbWz7W0tNDe3t7r85hb9Zqbm2lpaenVY9a0yEs6BZgJDALujoibyrbvBswHjgE2AZMjYkNVJ1m7BB6+FLZurry9eShMnAWHn151/GZmRdfU1MSoUaP6OgyrkZr9ZkzSIGAWcCrQCkyV1FrW7AJgc0QcDPwUuDn3CbZvgyXXwANTOy3wK3aOzuZ9d4E3M7MBqJY/DD8OWBcR6yPiPeABYGJZm4nAPen1g8AE5bkptPFV1txwHKyYU3HzzhAz3z+Tqe9dB3sdULGNmZlZ0dWyu35/4C8l79uBz3XWJiLel7QFGAb8o8sjPzKD1l3+XHHT32JvZmy/jLad5Z0GZmZmA0sti3ylK/Lyx97ztEHSxUDHIL//1bd4pWTzcD70T8G/ge9+sG/+GwDWubIcWy9zfmvPOa4t57e2Rvd0x1oW+XagtK+8BSgfW7ajTbukXYGhwEd+4B4Rc4CKffOSVvZ0TF/LxzmuLee39pzj2nJ+a0tS1xO2dKGW9+SfAw6RNErSYGAKsLiszWLg/PR6EvCHaLQZc8zMzPqpml3Jp3vslwOPkf2Ebm5EvCzpx8DKiFgM/Ar4taR1ZFfwU2oVj5mZ2UBT09/JR8QSYEnZuh+UvN4GfONjnqbyI/bWm5zj2nJ+a885ri3nt7Z6nN+Gm0/ezMzM8vEE6mZmZgXVMEVe0imSXpG0TtL3KmzfTdKCtP1ZSSPrH2Vjy5HjqyStkbRa0hOSDuyLOBtVd/ktaTdJUkjy08pVyJNfSWelz/DLku6rd4yNLsd3xKclLZX0fPqeOK0v4mxUkuZKelvSS51sl6TbU/5XSxrb7UEjot8vZA/uvQ4cBAwGVgGtZW0uBWan11OABX0ddyMtOXN8IrB7ej3dOe7d/KZ2ewJPA23AuL6Ou1GWnJ/fQ4Dngb3T+xF9HXcjLTlzPAeYnl63Ahv6Ou5GWoAvAmOBlzrZfhrwKNkYM+OBZ7s7ZqNcydduiFzr0G2OI2JpRLyb3raRjX1g+eT5DAPcANwCbKtncAWQJ78XAbMiYjNARLxd5xgbXZ4cB/DJ9HooHx0bxboQEU9TYayYEhOB+ZFpA/aStG9Xx2yUIl9piNz9O2sTEe8DHUPkWj55clzqArL/KC2fbvMr6bPAARHxSD0DK4g8n99DgUMlLZPUlmbJtPzy5Ph64BxJ7WS/rPp2fUIbMKr9nm6Y+eR7bYhc61Tu/Ek6BxgHnFDTiIqly/xK2oVsJsZp9QqoYPJ8fncl67L/Elkv1DOSjoyIf9U4tqLIk+OpwLyIuE3S58nGQTkyInbWPrwBoeo61yhX8tUMkUtXQ+Rap/LkGEknAdcCZ0TE/+oUWxF0l989gSOBJyVtILvfttgP3+WW9zvi4YjYHhFvAK+QFX3LJ0+OLwB+AxARy4FmsnHtrXfk+p4u1ShF3kPk1l63OU7dyXeRFXjfz6xOl/mNiC0RMTwiRkbESLJnHs6IiB6PWT3A5PmO+C3Zw6NIGk7Wfb++rlE2tjw5fhOYACDpcLIiv7GuURbbYuC89JT9eGBLRLzV1Q4N0V0fHiK35nLm+FZgD2BheqbxzYg4o8+CbiA582s9lDO/jwEnS1oD7ACuiYhNfRd1Y8mZ4+8Av5R0JVk38jRfbOUn6X6y20nD03MNPwSaACJiNtlzDqcB64B3gW92e0zn38zMrJgapbvezMzMquQib2ZmVlAu8mZmZgXlIm9mZlZQLvJmZmYF5SJvVmeSdkh6oWQZ2UXbkZ3NSFXlOZ9Ms4etSsO6ju7BMS6RdF56PU3SfiXb7pbU2stxPidpTI59Zkja/eOe26yIXOTN6m9rRIwpWTbU6bxnR8TRZBM53VrtzhExOyLmp7fTgP1Ktl0YEWt6JcoP4ryTfHHOAFzkzSpwkTfrB9IV+zOS/pSWL1Roc4SkFenqf7WkQ9L6c0rW3yVpUDenexo4OO07Ic39/WKay3q3tP6mNO/6akk/Seuul3S1pElkcxfcm845JF2Bj5M0XdItJTFPk/TzHsa5nJLJNyT9QtJKZXPB/yitu4Lsn42lkpamdSdLWp7yuFDSHt2cx6ywXOTN6m9ISVf9Q2nd28BXImIsMBm4vcJ+lwAzI2IMWZFtT0OHTgaOT+t3AGd3c/7TgRclNQPzgMkR8RmyETCnS9oH+DpwREQcBdxYunNEPAisJLviHhMRW0s2PwicWfJ+MrCgh3GeQjYUbYdrI2IccBRwgqSjIuJ2srG7T4yIE9NwtdcBJ6VcrgSu6uY8ZoXVEMPamhXM1lToSjUBd6R70DvIxlUvtxy4VlILsCgiXpM0ATgGeC4NNTyE7B+GSu6VtBXYQDYF6GjgjYh4NW2/B7gMuINsPvu7Jf0OyD31bURslLQ+jav9WjrHsnTcauL8BNnQqWNL1p8l6WKy7619gVZgddm+49P6Zek8g8nyZjYgucib9Q9XAn8HjibrYdtW3iAi7pP0LPBV4DFJF5JNPXlPRHw/xznOLp3wRtKwSo3SGOXHkU00MgW4HPhyFX/LAuAsYC3wUESEsoqbO05gFXATMAs4U9Io4Grg2IjYLGke2eQn5QQ8HhFTq4jXrLDcXW/WPwwF3krzbp9LdhX7IZIOAtanLurFZN3WTwCTJI1IbfaRdGDOc64FRko6OL0/F3gq3cMeGhFLyB5qq/SE+3/IpsetZBHwNbK5xRekdVXFGRHbybrdx6eu/k8C7wBbJH0KOLWTWNqA4zv+Jkm7S6rUK2I2ILjIm/UPdwLnS2oj66p/p0KbycBLkl4ADgPmpyfarwN+L2k18DhZV3a3ImIb2SxWCyW9COwEZpMVzEfS8Z4i62UoNw+Y3fHgXdlxNwNrgAMjYkVaV3Wc6V7/bcDVEbEKeB54GZhLdgugwxzgUUlLI2Ij2ZP/96fztJHlymxA8ix0ZmZmBeUreTMzs4JykTczMysoF3kzM7OCcpE3MzMrKBd5MzOzgnKRNzMzKygXeTMzs4JykTczMyuo/wM5D9cCvE+ziQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "false_pos_rate, true_pos_rate, thresholds = roc_curve(error_df.True_class, error_df.Reconstruction_error)\n",
    "roc_auc = auc(false_pos_rate, true_pos_rate,)\n",
    "\n",
    "plt.plot(false_pos_rate, true_pos_rate, linewidth=5, label='AUC = %0.3f'% roc_auc)\n",
    "plt.plot([0,1],[0,1], linewidth=5)\n",
    "\n",
    "plt.xlim([-0.01, 1])\n",
    "plt.ylim([0, 1.01])\n",
    "plt.legend(loc='lower right')\n",
    "plt.title('Receiver operating characteristic curve (ROC)')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}