Indoor_User_Movement_Prediction.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "timeseries.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "metadata": {
        "id": "rFFI9ECVkWp6",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# ‘Indoor User Movement Prediction‘"
      ]
    },
    {
      "metadata": {
        "id": "Hf2ZIjqVkWp8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Importing the necessary libraries."
      ]
    },
    {
      "metadata": {
        "id": "21s69y9PkWp9",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt\n",
        "from os import listdir"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ChDiTIdokWqB",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from keras.preprocessing import sequence\n",
        "import tensorflow as tf\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Dense\n",
        "from keras.layers import LSTM\n",
        "\n",
        "from keras.optimizers import Adam\n",
        "from keras.models import load_model\n",
        "from keras.callbacks import ModelCheckpoint"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "QMcHY30-kWqL",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##  About and Understanding the Data\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "ICnL1QgwkWqM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Dataset comprises of 316 files:\n",
        "\n",
        "314 MovementAAL csv files containing the readings from motion sensors placed in the environment.\n"
      ]
    },
    {
      "metadata": {
        "id": "fIYT19SukWqN",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "A Target csv file that contains the target variable for each MovementAAL file"
      ]
    },
    {
      "metadata": {
        "id": "G6Tws3_0kWqO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Group Data csv file to identify which MovementAAL file belongs to which setup group"
      ]
    },
    {
      "metadata": {
        "id": "IX173sjtkWqP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The Path csv file that contains the path which the object took"
      ]
    },
    {
      "metadata": {
        "id": "W_NCZf97kWqP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Loading the Data"
      ]
    },
    {
      "metadata": {
        "id": "bYAv1hSh5Djv",
        "colab_type": "code",
        "outputId": "ab5dab29-b71b-435f-c829-5719c8cb75a4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 139
        }
      },
      "cell_type": "code",
      "source": [
        "!gdown https://drive.google.com/uc?id=1XW3q_tluk3XiXJ-ABSnCmOUgStnOn40l\n",
        "!unzip MovementAAL.zip"
      ],
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1XW3q_tluk3XiXJ-ABSnCmOUgStnOn40l\n",
            "To: /content/MovementAAL.zip\n",
            "\r  0% 0.00/333k [00:00<?, ?B/s]\r100% 333k/333k [00:00<00:00, 47.5MB/s]\n",
            "Archive:  MovementAAL.zip\n",
            "replace MovementAAL/MovementAAL_RSS_184.csv? [y]es, [n]o, [A]ll, [N]one, [r]ename: "
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "ZYcUytKukWqR",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "df1 = pd.read_csv(\"MovementAAL/MovementAAL_RSS_1.csv\")\n",
        "df2 = pd.read_csv(\"MovementAAL/MovementAAL_RSS_2.csv\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "collapsed": true,
        "id": "E_UMWgBukWqU",
        "colab_type": "code",
        "outputId": "3f0d5451-55ae-4633-8523-d9e41b8ad956",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "df1.head()"
      ],
      "execution_count": 69,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>#RSS_anchor1</th>\n",
              "      <th>RSS_anchor2</th>\n",
              "      <th>RSS_anchor3</th>\n",
              "      <th>RSS_anchor4</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-0.90476</td>\n",
              "      <td>-0.48</td>\n",
              "      <td>0.28571</td>\n",
              "      <td>0.30</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-0.57143</td>\n",
              "      <td>-0.32</td>\n",
              "      <td>0.14286</td>\n",
              "      <td>0.30</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-0.38095</td>\n",
              "      <td>-0.28</td>\n",
              "      <td>-0.14286</td>\n",
              "      <td>0.35</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-0.28571</td>\n",
              "      <td>-0.20</td>\n",
              "      <td>-0.47619</td>\n",
              "      <td>0.35</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>-0.14286</td>\n",
              "      <td>-0.20</td>\n",
              "      <td>0.14286</td>\n",
              "      <td>-0.20</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   #RSS_anchor1   RSS_anchor2   RSS_anchor3   RSS_anchor4\n",
              "0      -0.90476         -0.48       0.28571          0.30\n",
              "1      -0.57143         -0.32       0.14286          0.30\n",
              "2      -0.38095         -0.28      -0.14286          0.35\n",
              "3      -0.28571         -0.20      -0.47619          0.35\n",
              "4      -0.14286         -0.20       0.14286         -0.20"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 69
        }
      ]
    },
    {
      "metadata": {
        "collapsed": true,
        "id": "IlzVQk1bkWqY",
        "colab_type": "code",
        "outputId": "1bf4decf-0fd0-4e39-ffad-f64b4e5cf0ff",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "df2.head()"
      ],
      "execution_count": 70,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>#RSS_anchor1</th>\n",
              "      <th>RSS_anchor2</th>\n",
              "      <th>RSS_anchor3</th>\n",
              "      <th>RSS_anchor4</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-0.57143</td>\n",
              "      <td>-0.20</td>\n",
              "      <td>0.71429</td>\n",
              "      <td>0.50</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-0.76190</td>\n",
              "      <td>-0.48</td>\n",
              "      <td>0.76190</td>\n",
              "      <td>-0.25</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-0.85714</td>\n",
              "      <td>-0.60</td>\n",
              "      <td>0.85714</td>\n",
              "      <td>0.55</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-0.76190</td>\n",
              "      <td>-0.40</td>\n",
              "      <td>0.71429</td>\n",
              "      <td>0.60</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>-0.76190</td>\n",
              "      <td>-0.84</td>\n",
              "      <td>0.85714</td>\n",
              "      <td>0.45</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   #RSS_anchor1   RSS_anchor2   RSS_anchor3   RSS_anchor4\n",
              "0      -0.57143         -0.20       0.71429          0.50\n",
              "1      -0.76190         -0.48       0.76190         -0.25\n",
              "2      -0.85714         -0.60       0.85714          0.55\n",
              "3      -0.76190         -0.40       0.71429          0.60\n",
              "4      -0.76190         -0.84       0.85714          0.45"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 70
        }
      ]
    },
    {
      "metadata": {
        "id": "NJxydSuekWqc",
        "colab_type": "code",
        "outputId": "c391137a-1d72-47d9-856e-00e5e42c1b68",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "df1.shape, df2.shape\n"
      ],
      "execution_count": 71,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((27, 4), (26, 4))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 71
        }
      ]
    },
    {
      "metadata": {
        "id": "y0cUgBhHkWqh",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The files contain normalized data from the four sensors – A1, A2, A3, A4.\n",
        "The first reading was for a duration of 27 seconds (so 27 rows), while another reading was for 26 seconds (so 26 rows)."
      ]
    },
    {
      "metadata": {
        "id": "bpIy7M7AkWqi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Read and store the values from the sensors "
      ]
    },
    {
      "metadata": {
        "collapsed": true,
        "id": "8kFAKWU9kWqj",
        "colab_type": "code",
        "outputId": "4d380394-140e-4f82-82f3-1aef0b7ee1d2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 5355
        }
      },
      "cell_type": "code",
      "source": [
        "path = 'MovementAAL/MovementAAL_RSS_'\n",
        "sequences = list()\n",
        "for i in range(1,315):\n",
        "    file_path = path + str(i) + '.csv'\n",
        "    print(file_path)\n",
        "    df = pd.read_csv(file_path, header=0)\n",
        "    values = df.values\n",
        "    sequences.append(values)"
      ],
      "execution_count": 72,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "MovementAAL/MovementAAL_RSS_1.csv\n",
            "MovementAAL/MovementAAL_RSS_2.csv\n",
            "MovementAAL/MovementAAL_RSS_3.csv\n",
            "MovementAAL/MovementAAL_RSS_4.csv\n",
            "MovementAAL/MovementAAL_RSS_5.csv\n",
            "MovementAAL/MovementAAL_RSS_6.csv\n",
            "MovementAAL/MovementAAL_RSS_7.csv\n",
            "MovementAAL/MovementAAL_RSS_8.csv\n",
            "MovementAAL/MovementAAL_RSS_9.csv\n",
            "MovementAAL/MovementAAL_RSS_10.csv\n",
            "MovementAAL/MovementAAL_RSS_11.csv\n",
            "MovementAAL/MovementAAL_RSS_12.csv\n",
            "MovementAAL/MovementAAL_RSS_13.csv\n",
            "MovementAAL/MovementAAL_RSS_14.csv\n",
            "MovementAAL/MovementAAL_RSS_15.csv\n",
            "MovementAAL/MovementAAL_RSS_16.csv\n",
            "MovementAAL/MovementAAL_RSS_17.csv\n",
            "MovementAAL/MovementAAL_RSS_18.csv\n",
            "MovementAAL/MovementAAL_RSS_19.csv\n",
            "MovementAAL/MovementAAL_RSS_20.csv\n",
            "MovementAAL/MovementAAL_RSS_21.csv\n",
            "MovementAAL/MovementAAL_RSS_22.csv\n",
            "MovementAAL/MovementAAL_RSS_23.csv\n",
            "MovementAAL/MovementAAL_RSS_24.csv\n",
            "MovementAAL/MovementAAL_RSS_25.csv\n",
            "MovementAAL/MovementAAL_RSS_26.csv\n",
            "MovementAAL/MovementAAL_RSS_27.csv\n",
            "MovementAAL/MovementAAL_RSS_28.csv\n",
            "MovementAAL/MovementAAL_RSS_29.csv\n",
            "MovementAAL/MovementAAL_RSS_30.csv\n",
            "MovementAAL/MovementAAL_RSS_31.csv\n",
            "MovementAAL/MovementAAL_RSS_32.csv\n",
            "MovementAAL/MovementAAL_RSS_33.csv\n",
            "MovementAAL/MovementAAL_RSS_34.csv\n",
            "MovementAAL/MovementAAL_RSS_35.csv\n",
            "MovementAAL/MovementAAL_RSS_36.csv\n",
            "MovementAAL/MovementAAL_RSS_37.csv\n",
            "MovementAAL/MovementAAL_RSS_38.csv\n",
            "MovementAAL/MovementAAL_RSS_39.csv\n",
            "MovementAAL/MovementAAL_RSS_40.csv\n",
            "MovementAAL/MovementAAL_RSS_41.csv\n",
            "MovementAAL/MovementAAL_RSS_42.csv\n",
            "MovementAAL/MovementAAL_RSS_43.csv\n",
            "MovementAAL/MovementAAL_RSS_44.csv\n",
            "MovementAAL/MovementAAL_RSS_45.csv\n",
            "MovementAAL/MovementAAL_RSS_46.csv\n",
            "MovementAAL/MovementAAL_RSS_47.csv\n",
            "MovementAAL/MovementAAL_RSS_48.csv\n",
            "MovementAAL/MovementAAL_RSS_49.csv\n",
            "MovementAAL/MovementAAL_RSS_50.csv\n",
            "MovementAAL/MovementAAL_RSS_51.csv\n",
            "MovementAAL/MovementAAL_RSS_52.csv\n",
            "MovementAAL/MovementAAL_RSS_53.csv\n",
            "MovementAAL/MovementAAL_RSS_54.csv\n",
            "MovementAAL/MovementAAL_RSS_55.csv\n",
            "MovementAAL/MovementAAL_RSS_56.csv\n",
            "MovementAAL/MovementAAL_RSS_57.csv\n",
            "MovementAAL/MovementAAL_RSS_58.csv\n",
            "MovementAAL/MovementAAL_RSS_59.csv\n",
            "MovementAAL/MovementAAL_RSS_60.csv\n",
            "MovementAAL/MovementAAL_RSS_61.csv\n",
            "MovementAAL/MovementAAL_RSS_62.csv\n",
            "MovementAAL/MovementAAL_RSS_63.csv\n",
            "MovementAAL/MovementAAL_RSS_64.csv\n",
            "MovementAAL/MovementAAL_RSS_65.csv\n",
            "MovementAAL/MovementAAL_RSS_66.csv\n",
            "MovementAAL/MovementAAL_RSS_67.csv\n",
            "MovementAAL/MovementAAL_RSS_68.csv\n",
            "MovementAAL/MovementAAL_RSS_69.csv\n",
            "MovementAAL/MovementAAL_RSS_70.csv\n",
            "MovementAAL/MovementAAL_RSS_71.csv\n",
            "MovementAAL/MovementAAL_RSS_72.csv\n",
            "MovementAAL/MovementAAL_RSS_73.csv\n",
            "MovementAAL/MovementAAL_RSS_74.csv\n",
            "MovementAAL/MovementAAL_RSS_75.csv\n",
            "MovementAAL/MovementAAL_RSS_76.csv\n",
            "MovementAAL/MovementAAL_RSS_77.csv\n",
            "MovementAAL/MovementAAL_RSS_78.csv\n",
            "MovementAAL/MovementAAL_RSS_79.csv\n",
            "MovementAAL/MovementAAL_RSS_80.csv\n",
            "MovementAAL/MovementAAL_RSS_81.csv\n",
            "MovementAAL/MovementAAL_RSS_82.csv\n",
            "MovementAAL/MovementAAL_RSS_83.csv\n",
            "MovementAAL/MovementAAL_RSS_84.csv\n",
            "MovementAAL/MovementAAL_RSS_85.csv\n",
            "MovementAAL/MovementAAL_RSS_86.csv\n",
            "MovementAAL/MovementAAL_RSS_87.csv\n",
            "MovementAAL/MovementAAL_RSS_88.csv\n",
            "MovementAAL/MovementAAL_RSS_89.csv\n",
            "MovementAAL/MovementAAL_RSS_90.csv\n",
            "MovementAAL/MovementAAL_RSS_91.csv\n",
            "MovementAAL/MovementAAL_RSS_92.csv\n",
            "MovementAAL/MovementAAL_RSS_93.csv\n",
            "MovementAAL/MovementAAL_RSS_94.csv\n",
            "MovementAAL/MovementAAL_RSS_95.csv\n",
            "MovementAAL/MovementAAL_RSS_96.csv\n",
            "MovementAAL/MovementAAL_RSS_97.csv\n",
            "MovementAAL/MovementAAL_RSS_98.csv\n",
            "MovementAAL/MovementAAL_RSS_99.csv\n",
            "MovementAAL/MovementAAL_RSS_100.csv\n",
            "MovementAAL/MovementAAL_RSS_101.csv\n",
            "MovementAAL/MovementAAL_RSS_102.csv\n",
            "MovementAAL/MovementAAL_RSS_103.csv\n",
            "MovementAAL/MovementAAL_RSS_104.csv\n",
            "MovementAAL/MovementAAL_RSS_105.csv\n",
            "MovementAAL/MovementAAL_RSS_106.csv\n",
            "MovementAAL/MovementAAL_RSS_107.csv\n",
            "MovementAAL/MovementAAL_RSS_108.csv\n",
            "MovementAAL/MovementAAL_RSS_109.csv\n",
            "MovementAAL/MovementAAL_RSS_110.csv\n",
            "MovementAAL/MovementAAL_RSS_111.csv\n",
            "MovementAAL/MovementAAL_RSS_112.csv\n",
            "MovementAAL/MovementAAL_RSS_113.csv\n",
            "MovementAAL/MovementAAL_RSS_114.csv\n",
            "MovementAAL/MovementAAL_RSS_115.csv\n",
            "MovementAAL/MovementAAL_RSS_116.csv\n",
            "MovementAAL/MovementAAL_RSS_117.csv\n",
            "MovementAAL/MovementAAL_RSS_118.csv\n",
            "MovementAAL/MovementAAL_RSS_119.csv\n",
            "MovementAAL/MovementAAL_RSS_120.csv\n",
            "MovementAAL/MovementAAL_RSS_121.csv\n",
            "MovementAAL/MovementAAL_RSS_122.csv\n",
            "MovementAAL/MovementAAL_RSS_123.csv\n",
            "MovementAAL/MovementAAL_RSS_124.csv\n",
            "MovementAAL/MovementAAL_RSS_125.csv\n",
            "MovementAAL/MovementAAL_RSS_126.csv\n",
            "MovementAAL/MovementAAL_RSS_127.csv\n",
            "MovementAAL/MovementAAL_RSS_128.csv\n",
            "MovementAAL/MovementAAL_RSS_129.csv\n",
            "MovementAAL/MovementAAL_RSS_130.csv\n",
            "MovementAAL/MovementAAL_RSS_131.csv\n",
            "MovementAAL/MovementAAL_RSS_132.csv\n",
            "MovementAAL/MovementAAL_RSS_133.csv\n",
            "MovementAAL/MovementAAL_RSS_134.csv\n",
            "MovementAAL/MovementAAL_RSS_135.csv\n",
            "MovementAAL/MovementAAL_RSS_136.csv\n",
            "MovementAAL/MovementAAL_RSS_137.csv\n",
            "MovementAAL/MovementAAL_RSS_138.csv\n",
            "MovementAAL/MovementAAL_RSS_139.csv\n",
            "MovementAAL/MovementAAL_RSS_140.csv\n",
            "MovementAAL/MovementAAL_RSS_141.csv\n",
            "MovementAAL/MovementAAL_RSS_142.csv\n",
            "MovementAAL/MovementAAL_RSS_143.csv\n",
            "MovementAAL/MovementAAL_RSS_144.csv\n",
            "MovementAAL/MovementAAL_RSS_145.csv\n",
            "MovementAAL/MovementAAL_RSS_146.csv\n",
            "MovementAAL/MovementAAL_RSS_147.csv\n",
            "MovementAAL/MovementAAL_RSS_148.csv\n",
            "MovementAAL/MovementAAL_RSS_149.csv\n",
            "MovementAAL/MovementAAL_RSS_150.csv\n",
            "MovementAAL/MovementAAL_RSS_151.csv\n",
            "MovementAAL/MovementAAL_RSS_152.csv\n",
            "MovementAAL/MovementAAL_RSS_153.csv\n",
            "MovementAAL/MovementAAL_RSS_154.csv\n",
            "MovementAAL/MovementAAL_RSS_155.csv\n",
            "MovementAAL/MovementAAL_RSS_156.csv\n",
            "MovementAAL/MovementAAL_RSS_157.csv\n",
            "MovementAAL/MovementAAL_RSS_158.csv\n",
            "MovementAAL/MovementAAL_RSS_159.csv\n",
            "MovementAAL/MovementAAL_RSS_160.csv\n",
            "MovementAAL/MovementAAL_RSS_161.csv\n",
            "MovementAAL/MovementAAL_RSS_162.csv\n",
            "MovementAAL/MovementAAL_RSS_163.csv\n",
            "MovementAAL/MovementAAL_RSS_164.csv\n",
            "MovementAAL/MovementAAL_RSS_165.csv\n",
            "MovementAAL/MovementAAL_RSS_166.csv\n",
            "MovementAAL/MovementAAL_RSS_167.csv\n",
            "MovementAAL/MovementAAL_RSS_168.csv\n",
            "MovementAAL/MovementAAL_RSS_169.csv\n",
            "MovementAAL/MovementAAL_RSS_170.csv\n",
            "MovementAAL/MovementAAL_RSS_171.csv\n",
            "MovementAAL/MovementAAL_RSS_172.csv\n",
            "MovementAAL/MovementAAL_RSS_173.csv\n",
            "MovementAAL/MovementAAL_RSS_174.csv\n",
            "MovementAAL/MovementAAL_RSS_175.csv\n",
            "MovementAAL/MovementAAL_RSS_176.csv\n",
            "MovementAAL/MovementAAL_RSS_177.csv\n",
            "MovementAAL/MovementAAL_RSS_178.csv\n",
            "MovementAAL/MovementAAL_RSS_179.csv\n",
            "MovementAAL/MovementAAL_RSS_180.csv\n",
            "MovementAAL/MovementAAL_RSS_181.csv\n",
            "MovementAAL/MovementAAL_RSS_182.csv\n",
            "MovementAAL/MovementAAL_RSS_183.csv\n",
            "MovementAAL/MovementAAL_RSS_184.csv\n",
            "MovementAAL/MovementAAL_RSS_185.csv\n",
            "MovementAAL/MovementAAL_RSS_186.csv\n",
            "MovementAAL/MovementAAL_RSS_187.csv\n",
            "MovementAAL/MovementAAL_RSS_188.csv\n",
            "MovementAAL/MovementAAL_RSS_189.csv\n",
            "MovementAAL/MovementAAL_RSS_190.csv\n",
            "MovementAAL/MovementAAL_RSS_191.csv\n",
            "MovementAAL/MovementAAL_RSS_192.csv\n",
            "MovementAAL/MovementAAL_RSS_193.csv\n",
            "MovementAAL/MovementAAL_RSS_194.csv\n",
            "MovementAAL/MovementAAL_RSS_195.csv\n",
            "MovementAAL/MovementAAL_RSS_196.csv\n",
            "MovementAAL/MovementAAL_RSS_197.csv\n",
            "MovementAAL/MovementAAL_RSS_198.csv\n",
            "MovementAAL/MovementAAL_RSS_199.csv\n",
            "MovementAAL/MovementAAL_RSS_200.csv\n",
            "MovementAAL/MovementAAL_RSS_201.csv\n",
            "MovementAAL/MovementAAL_RSS_202.csv\n",
            "MovementAAL/MovementAAL_RSS_203.csv\n",
            "MovementAAL/MovementAAL_RSS_204.csv\n",
            "MovementAAL/MovementAAL_RSS_205.csv\n",
            "MovementAAL/MovementAAL_RSS_206.csv\n",
            "MovementAAL/MovementAAL_RSS_207.csv\n",
            "MovementAAL/MovementAAL_RSS_208.csv\n",
            "MovementAAL/MovementAAL_RSS_209.csv\n",
            "MovementAAL/MovementAAL_RSS_210.csv\n",
            "MovementAAL/MovementAAL_RSS_211.csv\n",
            "MovementAAL/MovementAAL_RSS_212.csv\n",
            "MovementAAL/MovementAAL_RSS_213.csv\n",
            "MovementAAL/MovementAAL_RSS_214.csv\n",
            "MovementAAL/MovementAAL_RSS_215.csv\n",
            "MovementAAL/MovementAAL_RSS_216.csv\n",
            "MovementAAL/MovementAAL_RSS_217.csv\n",
            "MovementAAL/MovementAAL_RSS_218.csv\n",
            "MovementAAL/MovementAAL_RSS_219.csv\n",
            "MovementAAL/MovementAAL_RSS_220.csv\n",
            "MovementAAL/MovementAAL_RSS_221.csv\n",
            "MovementAAL/MovementAAL_RSS_222.csv\n",
            "MovementAAL/MovementAAL_RSS_223.csv\n",
            "MovementAAL/MovementAAL_RSS_224.csv\n",
            "MovementAAL/MovementAAL_RSS_225.csv\n",
            "MovementAAL/MovementAAL_RSS_226.csv\n",
            "MovementAAL/MovementAAL_RSS_227.csv\n",
            "MovementAAL/MovementAAL_RSS_228.csv\n",
            "MovementAAL/MovementAAL_RSS_229.csv\n",
            "MovementAAL/MovementAAL_RSS_230.csv\n",
            "MovementAAL/MovementAAL_RSS_231.csv\n",
            "MovementAAL/MovementAAL_RSS_232.csv\n",
            "MovementAAL/MovementAAL_RSS_233.csv\n",
            "MovementAAL/MovementAAL_RSS_234.csv\n",
            "MovementAAL/MovementAAL_RSS_235.csv\n",
            "MovementAAL/MovementAAL_RSS_236.csv\n",
            "MovementAAL/MovementAAL_RSS_237.csv\n",
            "MovementAAL/MovementAAL_RSS_238.csv\n",
            "MovementAAL/MovementAAL_RSS_239.csv\n",
            "MovementAAL/MovementAAL_RSS_240.csv\n",
            "MovementAAL/MovementAAL_RSS_241.csv\n",
            "MovementAAL/MovementAAL_RSS_242.csv\n",
            "MovementAAL/MovementAAL_RSS_243.csv\n",
            "MovementAAL/MovementAAL_RSS_244.csv\n",
            "MovementAAL/MovementAAL_RSS_245.csv\n",
            "MovementAAL/MovementAAL_RSS_246.csv\n",
            "MovementAAL/MovementAAL_RSS_247.csv\n",
            "MovementAAL/MovementAAL_RSS_248.csv\n",
            "MovementAAL/MovementAAL_RSS_249.csv\n",
            "MovementAAL/MovementAAL_RSS_250.csv\n",
            "MovementAAL/MovementAAL_RSS_251.csv\n",
            "MovementAAL/MovementAAL_RSS_252.csv\n",
            "MovementAAL/MovementAAL_RSS_253.csv\n",
            "MovementAAL/MovementAAL_RSS_254.csv\n",
            "MovementAAL/MovementAAL_RSS_255.csv\n",
            "MovementAAL/MovementAAL_RSS_256.csv\n",
            "MovementAAL/MovementAAL_RSS_257.csv\n",
            "MovementAAL/MovementAAL_RSS_258.csv\n",
            "MovementAAL/MovementAAL_RSS_259.csv\n",
            "MovementAAL/MovementAAL_RSS_260.csv\n",
            "MovementAAL/MovementAAL_RSS_261.csv\n",
            "MovementAAL/MovementAAL_RSS_262.csv\n",
            "MovementAAL/MovementAAL_RSS_263.csv\n",
            "MovementAAL/MovementAAL_RSS_264.csv\n",
            "MovementAAL/MovementAAL_RSS_265.csv\n",
            "MovementAAL/MovementAAL_RSS_266.csv\n",
            "MovementAAL/MovementAAL_RSS_267.csv\n",
            "MovementAAL/MovementAAL_RSS_268.csv\n",
            "MovementAAL/MovementAAL_RSS_269.csv\n",
            "MovementAAL/MovementAAL_RSS_270.csv\n",
            "MovementAAL/MovementAAL_RSS_271.csv\n",
            "MovementAAL/MovementAAL_RSS_272.csv\n",
            "MovementAAL/MovementAAL_RSS_273.csv\n",
            "MovementAAL/MovementAAL_RSS_274.csv\n",
            "MovementAAL/MovementAAL_RSS_275.csv\n",
            "MovementAAL/MovementAAL_RSS_276.csv\n",
            "MovementAAL/MovementAAL_RSS_277.csv\n",
            "MovementAAL/MovementAAL_RSS_278.csv\n",
            "MovementAAL/MovementAAL_RSS_279.csv\n",
            "MovementAAL/MovementAAL_RSS_280.csv\n",
            "MovementAAL/MovementAAL_RSS_281.csv\n",
            "MovementAAL/MovementAAL_RSS_282.csv\n",
            "MovementAAL/MovementAAL_RSS_283.csv\n",
            "MovementAAL/MovementAAL_RSS_284.csv\n",
            "MovementAAL/MovementAAL_RSS_285.csv\n",
            "MovementAAL/MovementAAL_RSS_286.csv\n",
            "MovementAAL/MovementAAL_RSS_287.csv\n",
            "MovementAAL/MovementAAL_RSS_288.csv\n",
            "MovementAAL/MovementAAL_RSS_289.csv\n",
            "MovementAAL/MovementAAL_RSS_290.csv\n",
            "MovementAAL/MovementAAL_RSS_291.csv\n",
            "MovementAAL/MovementAAL_RSS_292.csv\n",
            "MovementAAL/MovementAAL_RSS_293.csv\n",
            "MovementAAL/MovementAAL_RSS_294.csv\n",
            "MovementAAL/MovementAAL_RSS_295.csv\n",
            "MovementAAL/MovementAAL_RSS_296.csv\n",
            "MovementAAL/MovementAAL_RSS_297.csv\n",
            "MovementAAL/MovementAAL_RSS_298.csv\n",
            "MovementAAL/MovementAAL_RSS_299.csv\n",
            "MovementAAL/MovementAAL_RSS_300.csv\n",
            "MovementAAL/MovementAAL_RSS_301.csv\n",
            "MovementAAL/MovementAAL_RSS_302.csv\n",
            "MovementAAL/MovementAAL_RSS_303.csv\n",
            "MovementAAL/MovementAAL_RSS_304.csv\n",
            "MovementAAL/MovementAAL_RSS_305.csv\n",
            "MovementAAL/MovementAAL_RSS_306.csv\n",
            "MovementAAL/MovementAAL_RSS_307.csv\n",
            "MovementAAL/MovementAAL_RSS_308.csv\n",
            "MovementAAL/MovementAAL_RSS_309.csv\n",
            "MovementAAL/MovementAAL_RSS_310.csv\n",
            "MovementAAL/MovementAAL_RSS_311.csv\n",
            "MovementAAL/MovementAAL_RSS_312.csv\n",
            "MovementAAL/MovementAAL_RSS_313.csv\n",
            "MovementAAL/MovementAAL_RSS_314.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "vjOHDg1VkWqo",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "targets = pd.read_csv('MovementAAL/MovementAAL_target.csv')\n",
        "targets = targets.values[:,1]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4rd2XvTSkWqr",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This is a list ‘sequences’ that contains the data from the motion sensors and ‘targets’ which holds the labels for the csv files. When we print sequences[0], we get the values of sensors from the first csv file:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "collapsed": true,
        "id": "lwD8B5FukWqs",
        "colab_type": "code",
        "outputId": "c312138d-7816-4234-d4a5-7d4fc068f976",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 476
        }
      },
      "cell_type": "code",
      "source": [
        "sequences[0]"
      ],
      "execution_count": 74,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.90476 , -0.48    ,  0.28571 ,  0.3     ],\n",
              "       [-0.57143 , -0.32    ,  0.14286 ,  0.3     ],\n",
              "       [-0.38095 , -0.28    , -0.14286 ,  0.35    ],\n",
              "       [-0.28571 , -0.2     , -0.47619 ,  0.35    ],\n",
              "       [-0.14286 , -0.2     ,  0.14286 , -0.2     ],\n",
              "       [-0.14286 , -0.2     ,  0.047619,  0.      ],\n",
              "       [-0.14286 , -0.16    , -0.38095 ,  0.2     ],\n",
              "       [-0.14286 , -0.04    , -0.61905 , -0.2     ],\n",
              "       [-0.095238, -0.08    ,  0.14286 , -0.55    ],\n",
              "       [-0.047619,  0.04    , -0.095238,  0.05    ],\n",
              "       [-0.19048 , -0.04    ,  0.095238,  0.4     ],\n",
              "       [-0.095238, -0.04    , -0.14286 ,  0.35    ],\n",
              "       [-0.33333 , -0.08    , -0.28571 , -0.2     ],\n",
              "       [-0.2381  ,  0.04    ,  0.14286 ,  0.35    ],\n",
              "       [ 0.      ,  0.08    ,  0.14286 ,  0.05    ],\n",
              "       [-0.095238,  0.04    ,  0.095238,  0.1     ],\n",
              "       [-0.14286 , -0.2     ,  0.14286 ,  0.5     ],\n",
              "       [-0.19048 ,  0.04    , -0.42857 ,  0.3     ],\n",
              "       [-0.14286 , -0.08    , -0.2381  ,  0.15    ],\n",
              "       [-0.33333 ,  0.16    , -0.14286 , -0.8     ],\n",
              "       [-0.42857 ,  0.16    , -0.28571 , -0.1     ],\n",
              "       [-0.71429 ,  0.16    , -0.28571 ,  0.2     ],\n",
              "       [-0.095238, -0.08    ,  0.095238,  0.35    ],\n",
              "       [-0.28571 ,  0.04    ,  0.14286 ,  0.2     ],\n",
              "       [ 0.      ,  0.04    ,  0.14286 ,  0.1     ],\n",
              "       [ 0.      ,  0.04    , -0.047619, -0.05    ],\n",
              "       [-0.14286 , -0.6     , -0.28571 , -0.1     ]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 74
        }
      ]
    },
    {
      "metadata": {
        "id": "usTsdgTdkWqx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Loading the DatasetGroup csv file now"
      ]
    },
    {
      "metadata": {
        "id": "4PGw9jbGkWqy",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The dataset was collected in three different pairs of rooms – hence three groups. This information can be used to divide the dataset into train, test and validation sets"
      ]
    },
    {
      "metadata": {
        "id": "0xaa0t5ukWq0",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "groups = pd.read_csv('MovementAAL/MovementAAL_DatasetGroup.csv', header=0)\n",
        "groups = groups.values[:,1]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "vsYBxSwfkWq3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Preprocessing Steps"
      ]
    },
    {
      "metadata": {
        "id": "_AY-cFZwkWq5",
        "colab_type": "code",
        "outputId": "bb0719d3-1e21-4572-ab51-7894b2c3574c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "cell_type": "code",
      "source": [
        "len_sequences = []\n",
        "for one_seq in sequences:\n",
        "    len_sequences.append(len(one_seq))\n",
        "pd.Series(len_sequences).describe()"
      ],
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count    314.000000\n",
              "mean      42.028662\n",
              "std       16.185303\n",
              "min       19.000000\n",
              "25%       26.000000\n",
              "50%       41.000000\n",
              "75%       56.000000\n",
              "max      129.000000\n",
              "dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 76
        }
      ]
    },
    {
      "metadata": {
        "id": "d4S_ud6mkWrA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "#Padding the sequence with the values in last row to max length\n",
        "to_pad = 129\n",
        "new_seq = []\n",
        "for one_seq in sequences:\n",
        "    len_one_seq = len(one_seq)\n",
        "    last_val = one_seq[-1]\n",
        "    n = to_pad - len_one_seq\n",
        "   \n",
        "    to_concat = np.repeat(one_seq[-1], n).reshape(4, n).transpose()\n",
        "    new_one_seq = np.concatenate([one_seq, to_concat])\n",
        "    new_seq.append(new_one_seq)\n",
        "final_seq = np.stack(new_seq)\n",
        "\n",
        "#truncate the sequence to length 60\n",
        "from keras.preprocessing import sequence\n",
        "seq_len = 60\n",
        "final_seq=sequence.pad_sequences(final_seq, maxlen=seq_len, padding='post', dtype='float', truncating='post')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "0_x4UVQ-kWrH",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Preparing the train, validation and test sets:"
      ]
    },
    {
      "metadata": {
        "id": "UiMsIVuOkWrJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "train = [final_seq[i] for i in range(len(groups)) if (groups[i]==2)]\n",
        "validation = [final_seq[i] for i in range(len(groups)) if groups[i]==1]\n",
        "test = [final_seq[i] for i in range(len(groups)) if groups[i]==3]\n",
        "\n",
        "train_target = [targets[i] for i in range(len(groups)) if (groups[i]==2)]\n",
        "validation_target = [targets[i] for i in range(len(groups)) if groups[i]==1]\n",
        "test_target = [targets[i] for i in range(len(groups)) if groups[i]==3]\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1VxR7wZxkWrN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "train = np.array(train)\n",
        "validation = np.array(validation)\n",
        "test = np.array(test)\n",
        "\n",
        "train_target = np.array(train_target)\n",
        "train_target = (train_target+1)/2\n",
        "\n",
        "validation_target = np.array(validation_target)\n",
        "validation_target = (validation_target+1)/2\n",
        "\n",
        "test_target = np.array(test_target)\n",
        "test_target = (test_target+1)/2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "puNWuVhAkWrQ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Building a Time Series Classification model"
      ]
    },
    {
      "metadata": {
        "id": "8ZI8VMx8kWrR",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model = Sequential()\n",
        "model.add(LSTM(256, input_shape=(seq_len, 4)))\n",
        "model.add(Dense(1, activation='sigmoid'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1X6mEImxkWrU",
        "colab_type": "code",
        "outputId": "79b63d13-be9e-4ec8-f3ea-54f256307aa7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "model.summary()"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "lstm_3 (LSTM)                (None, 256)               267264    \n",
            "_________________________________________________________________\n",
            "dense_3 (Dense)              (None, 1)                 257       \n",
            "=================================================================\n",
            "Total params: 267,521\n",
            "Trainable params: 267,521\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "SWHLGSqTkWrZ",
        "colab_type": "code",
        "outputId": "d1f5bfd3-ac4f-4767-fdde-e3a033027b4d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34071
        }
      },
      "cell_type": "code",
      "source": [
        "adam = Adam(lr=0.0001)\n",
        "chk = ModelCheckpoint('best_model.pkl', monitor='val_acc', save_best_only=True, mode='max', verbose=1)\n",
        "model.compile(loss='binary_crossentropy', optimizer=adam, metrics=['accuracy'])\n",
        "model.fit(train, train_target, epochs=500, batch_size=128, callbacks=[chk], validation_data=(validation,validation_target))\n"
      ],
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 106 samples, validate on 104 samples\n",
            "Epoch 1/500\n",
            "106/106 [==============================] - 4s 33ms/step - loss: 0.4255 - acc: 0.7170 - val_loss: 0.8834 - val_acc: 0.7212\n",
            "\n",
            "Epoch 00001: val_acc improved from -inf to 0.72115, saving model to best_model.pkl\n",
            "Epoch 2/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.4173 - acc: 0.7264 - val_loss: 0.9253 - val_acc: 0.7019\n",
            "\n",
            "Epoch 00002: val_acc did not improve from 0.72115\n",
            "Epoch 3/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.4110 - acc: 0.7264 - val_loss: 0.9574 - val_acc: 0.6923\n",
            "\n",
            "Epoch 00003: val_acc did not improve from 0.72115\n",
            "Epoch 4/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.4144 - acc: 0.7170 - val_loss: 0.9807 - val_acc: 0.6923\n",
            "\n",
            "Epoch 00004: val_acc did not improve from 0.72115\n",
            "Epoch 5/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.4080 - acc: 0.7264 - val_loss: 0.9990 - val_acc: 0.6538\n",
            "\n",
            "Epoch 00005: val_acc did not improve from 0.72115\n",
            "Epoch 6/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.4050 - acc: 0.7264 - val_loss: 1.0016 - val_acc: 0.6250\n",
            "\n",
            "Epoch 00006: val_acc did not improve from 0.72115\n",
            "Epoch 7/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.4032 - acc: 0.7264 - val_loss: 1.0112 - val_acc: 0.6154\n",
            "\n",
            "Epoch 00007: val_acc did not improve from 0.72115\n",
            "Epoch 8/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.4003 - acc: 0.7358 - val_loss: 1.0249 - val_acc: 0.5865\n",
            "\n",
            "Epoch 00008: val_acc did not improve from 0.72115\n",
            "Epoch 9/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3968 - acc: 0.7358 - val_loss: 1.0496 - val_acc: 0.5865\n",
            "\n",
            "Epoch 00009: val_acc did not improve from 0.72115\n",
            "Epoch 10/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3911 - acc: 0.7736 - val_loss: 1.0838 - val_acc: 0.5769\n",
            "\n",
            "Epoch 00010: val_acc did not improve from 0.72115\n",
            "Epoch 11/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3855 - acc: 0.8019 - val_loss: 1.1056 - val_acc: 0.5865\n",
            "\n",
            "Epoch 00011: val_acc did not improve from 0.72115\n",
            "Epoch 12/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3759 - acc: 0.8113 - val_loss: 1.1323 - val_acc: 0.5769\n",
            "\n",
            "Epoch 00012: val_acc did not improve from 0.72115\n",
            "Epoch 13/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3810 - acc: 0.8019 - val_loss: 1.1583 - val_acc: 0.5865\n",
            "\n",
            "Epoch 00013: val_acc did not improve from 0.72115\n",
            "Epoch 14/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3752 - acc: 0.8113 - val_loss: 1.1768 - val_acc: 0.5769\n",
            "\n",
            "Epoch 00014: val_acc did not improve from 0.72115\n",
            "Epoch 15/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3678 - acc: 0.8585 - val_loss: 1.1981 - val_acc: 0.5769\n",
            "\n",
            "Epoch 00015: val_acc did not improve from 0.72115\n",
            "Epoch 16/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3691 - acc: 0.8396 - val_loss: 1.2045 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00016: val_acc did not improve from 0.72115\n",
            "Epoch 17/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3507 - acc: 0.8585 - val_loss: 1.2100 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00017: val_acc did not improve from 0.72115\n",
            "Epoch 18/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3490 - acc: 0.8585 - val_loss: 1.2147 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00018: val_acc did not improve from 0.72115\n",
            "Epoch 19/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3491 - acc: 0.8585 - val_loss: 1.2188 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00019: val_acc did not improve from 0.72115\n",
            "Epoch 20/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3415 - acc: 0.8679 - val_loss: 1.2218 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00020: val_acc did not improve from 0.72115\n",
            "Epoch 21/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3405 - acc: 0.8679 - val_loss: 1.2233 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00021: val_acc did not improve from 0.72115\n",
            "Epoch 22/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3394 - acc: 0.8679 - val_loss: 1.2264 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00022: val_acc did not improve from 0.72115\n",
            "Epoch 23/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3389 - acc: 0.8679 - val_loss: 1.2266 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00023: val_acc did not improve from 0.72115\n",
            "Epoch 24/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3387 - acc: 0.8679 - val_loss: 1.2251 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00024: val_acc did not improve from 0.72115\n",
            "Epoch 25/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3382 - acc: 0.8679 - val_loss: 1.2227 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00025: val_acc did not improve from 0.72115\n",
            "Epoch 26/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3372 - acc: 0.8679 - val_loss: 1.2197 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00026: val_acc did not improve from 0.72115\n",
            "Epoch 27/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3360 - acc: 0.8679 - val_loss: 1.2162 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00027: val_acc did not improve from 0.72115\n",
            "Epoch 28/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3360 - acc: 0.8585 - val_loss: 1.2130 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00028: val_acc did not improve from 0.72115\n",
            "Epoch 29/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3357 - acc: 0.8585 - val_loss: 1.2097 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00029: val_acc did not improve from 0.72115\n",
            "Epoch 30/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3345 - acc: 0.8585 - val_loss: 1.2063 - val_acc: 0.5769\n",
            "\n",
            "Epoch 00030: val_acc did not improve from 0.72115\n",
            "Epoch 31/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3520 - acc: 0.8585 - val_loss: 1.2000 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00031: val_acc did not improve from 0.72115\n",
            "Epoch 32/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3338 - acc: 0.8679 - val_loss: 1.1942 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00032: val_acc did not improve from 0.72115\n",
            "Epoch 33/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3313 - acc: 0.8679 - val_loss: 1.1885 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00033: val_acc did not improve from 0.72115\n",
            "Epoch 34/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3310 - acc: 0.8585 - val_loss: 1.1832 - val_acc: 0.5673\n",
            "\n",
            "Epoch 00034: val_acc did not improve from 0.72115\n",
            "Epoch 35/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3285 - acc: 0.8679 - val_loss: 1.1786 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00035: val_acc did not improve from 0.72115\n",
            "Epoch 36/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3240 - acc: 0.8679 - val_loss: 1.1747 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00036: val_acc did not improve from 0.72115\n",
            "Epoch 37/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3229 - acc: 0.8679 - val_loss: 1.1720 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00037: val_acc did not improve from 0.72115\n",
            "Epoch 38/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3222 - acc: 0.8679 - val_loss: 1.1714 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00038: val_acc did not improve from 0.72115\n",
            "Epoch 39/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3218 - acc: 0.8679 - val_loss: 1.1727 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00039: val_acc did not improve from 0.72115\n",
            "Epoch 40/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3211 - acc: 0.8679 - val_loss: 1.1733 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00040: val_acc did not improve from 0.72115\n",
            "Epoch 41/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3203 - acc: 0.8679 - val_loss: 1.1751 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00041: val_acc did not improve from 0.72115\n",
            "Epoch 42/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3203 - acc: 0.8679 - val_loss: 1.1755 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00042: val_acc did not improve from 0.72115\n",
            "Epoch 43/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3208 - acc: 0.8585 - val_loss: 1.1758 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00043: val_acc did not improve from 0.72115\n",
            "Epoch 44/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3182 - acc: 0.8679 - val_loss: 1.1766 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00044: val_acc did not improve from 0.72115\n",
            "Epoch 45/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3163 - acc: 0.8774 - val_loss: 1.1777 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00045: val_acc did not improve from 0.72115\n",
            "Epoch 46/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3135 - acc: 0.8774 - val_loss: 1.1795 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00046: val_acc did not improve from 0.72115\n",
            "Epoch 47/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3121 - acc: 0.8774 - val_loss: 1.1829 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00047: val_acc did not improve from 0.72115\n",
            "Epoch 48/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3107 - acc: 0.8774 - val_loss: 1.1857 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00048: val_acc did not improve from 0.72115\n",
            "Epoch 49/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3092 - acc: 0.8774 - val_loss: 1.1895 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00049: val_acc did not improve from 0.72115\n",
            "Epoch 50/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3049 - acc: 0.8868 - val_loss: 1.1922 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00050: val_acc did not improve from 0.72115\n",
            "Epoch 51/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3048 - acc: 0.8868 - val_loss: 1.1946 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00051: val_acc did not improve from 0.72115\n",
            "Epoch 52/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3055 - acc: 0.8774 - val_loss: 1.1968 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00052: val_acc did not improve from 0.72115\n",
            "Epoch 53/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3049 - acc: 0.8774 - val_loss: 1.1988 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00053: val_acc did not improve from 0.72115\n",
            "Epoch 54/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3040 - acc: 0.8774 - val_loss: 1.2006 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00054: val_acc did not improve from 0.72115\n",
            "Epoch 55/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3031 - acc: 0.8774 - val_loss: 1.2021 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00055: val_acc did not improve from 0.72115\n",
            "Epoch 56/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3021 - acc: 0.8774 - val_loss: 1.2033 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00056: val_acc did not improve from 0.72115\n",
            "Epoch 57/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.3010 - acc: 0.8774 - val_loss: 1.2043 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00057: val_acc did not improve from 0.72115\n",
            "Epoch 58/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2995 - acc: 0.8774 - val_loss: 1.2050 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00058: val_acc did not improve from 0.72115\n",
            "Epoch 59/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2978 - acc: 0.8774 - val_loss: 1.2050 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00059: val_acc did not improve from 0.72115\n",
            "Epoch 60/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2972 - acc: 0.8774 - val_loss: 1.2049 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00060: val_acc did not improve from 0.72115\n",
            "Epoch 61/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2969 - acc: 0.8868 - val_loss: 1.2048 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00061: val_acc did not improve from 0.72115\n",
            "Epoch 62/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2966 - acc: 0.8868 - val_loss: 1.2048 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00062: val_acc did not improve from 0.72115\n",
            "Epoch 63/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2962 - acc: 0.8868 - val_loss: 1.2048 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00063: val_acc did not improve from 0.72115\n",
            "Epoch 64/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2956 - acc: 0.8868 - val_loss: 1.2047 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00064: val_acc did not improve from 0.72115\n",
            "Epoch 65/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2950 - acc: 0.8868 - val_loss: 1.2046 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00065: val_acc did not improve from 0.72115\n",
            "Epoch 66/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2943 - acc: 0.8868 - val_loss: 1.2044 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00066: val_acc did not improve from 0.72115\n",
            "Epoch 67/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2936 - acc: 0.8868 - val_loss: 1.2041 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00067: val_acc did not improve from 0.72115\n",
            "Epoch 68/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2928 - acc: 0.8868 - val_loss: 1.2034 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00068: val_acc did not improve from 0.72115\n",
            "Epoch 69/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2919 - acc: 0.8868 - val_loss: 1.2027 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00069: val_acc did not improve from 0.72115\n",
            "Epoch 70/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2910 - acc: 0.8774 - val_loss: 1.2039 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00070: val_acc did not improve from 0.72115\n",
            "Epoch 71/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2894 - acc: 0.8774 - val_loss: 1.2053 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00071: val_acc did not improve from 0.72115\n",
            "Epoch 72/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2887 - acc: 0.8774 - val_loss: 1.2051 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00072: val_acc did not improve from 0.72115\n",
            "Epoch 73/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2880 - acc: 0.8774 - val_loss: 1.2043 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00073: val_acc did not improve from 0.72115\n",
            "Epoch 74/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2871 - acc: 0.8774 - val_loss: 1.2034 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00074: val_acc did not improve from 0.72115\n",
            "Epoch 75/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2862 - acc: 0.8868 - val_loss: 1.2021 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00075: val_acc did not improve from 0.72115\n",
            "Epoch 76/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2857 - acc: 0.8868 - val_loss: 1.2005 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00076: val_acc did not improve from 0.72115\n",
            "Epoch 77/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2849 - acc: 0.8774 - val_loss: 1.1986 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00077: val_acc did not improve from 0.72115\n",
            "Epoch 78/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2846 - acc: 0.8774 - val_loss: 1.1976 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00078: val_acc did not improve from 0.72115\n",
            "Epoch 79/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2840 - acc: 0.8868 - val_loss: 1.1970 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00079: val_acc did not improve from 0.72115\n",
            "Epoch 80/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2839 - acc: 0.8868 - val_loss: 1.1965 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00080: val_acc did not improve from 0.72115\n",
            "Epoch 81/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2831 - acc: 0.8868 - val_loss: 1.1962 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00081: val_acc did not improve from 0.72115\n",
            "Epoch 82/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2827 - acc: 0.8774 - val_loss: 1.1964 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00082: val_acc did not improve from 0.72115\n",
            "Epoch 83/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2819 - acc: 0.8868 - val_loss: 1.1968 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00083: val_acc did not improve from 0.72115\n",
            "Epoch 84/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2813 - acc: 0.8868 - val_loss: 1.1974 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00084: val_acc did not improve from 0.72115\n",
            "Epoch 85/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2807 - acc: 0.8868 - val_loss: 1.1978 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00085: val_acc did not improve from 0.72115\n",
            "Epoch 86/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2801 - acc: 0.8868 - val_loss: 1.1983 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00086: val_acc did not improve from 0.72115\n",
            "Epoch 87/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2797 - acc: 0.8774 - val_loss: 1.1990 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00087: val_acc did not improve from 0.72115\n",
            "Epoch 88/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2791 - acc: 0.8868 - val_loss: 1.1998 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00088: val_acc did not improve from 0.72115\n",
            "Epoch 89/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2787 - acc: 0.8868 - val_loss: 1.2003 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00089: val_acc did not improve from 0.72115\n",
            "Epoch 90/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2783 - acc: 0.8868 - val_loss: 1.2009 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00090: val_acc did not improve from 0.72115\n",
            "Epoch 91/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2779 - acc: 0.8962 - val_loss: 1.2017 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00091: val_acc did not improve from 0.72115\n",
            "Epoch 92/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2776 - acc: 0.8962 - val_loss: 1.2028 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00092: val_acc did not improve from 0.72115\n",
            "Epoch 93/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2773 - acc: 0.8962 - val_loss: 1.2040 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00093: val_acc did not improve from 0.72115\n",
            "Epoch 94/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2769 - acc: 0.8962 - val_loss: 1.2053 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00094: val_acc did not improve from 0.72115\n",
            "Epoch 95/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2765 - acc: 0.8962 - val_loss: 1.2066 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00095: val_acc did not improve from 0.72115\n",
            "Epoch 96/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2761 - acc: 0.8962 - val_loss: 1.2080 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00096: val_acc did not improve from 0.72115\n",
            "Epoch 97/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2757 - acc: 0.8962 - val_loss: 1.2093 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00097: val_acc did not improve from 0.72115\n",
            "Epoch 98/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2753 - acc: 0.8962 - val_loss: 1.2105 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00098: val_acc did not improve from 0.72115\n",
            "Epoch 99/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2749 - acc: 0.8962 - val_loss: 1.2117 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00099: val_acc did not improve from 0.72115\n",
            "Epoch 100/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2746 - acc: 0.8962 - val_loss: 1.2128 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00100: val_acc did not improve from 0.72115\n",
            "Epoch 101/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2742 - acc: 0.8962 - val_loss: 1.2140 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00101: val_acc did not improve from 0.72115\n",
            "Epoch 102/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2739 - acc: 0.8962 - val_loss: 1.2153 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00102: val_acc did not improve from 0.72115\n",
            "Epoch 103/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2735 - acc: 0.8962 - val_loss: 1.2164 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00103: val_acc did not improve from 0.72115\n",
            "Epoch 104/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2732 - acc: 0.8962 - val_loss: 1.2176 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00104: val_acc did not improve from 0.72115\n",
            "Epoch 105/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2729 - acc: 0.8962 - val_loss: 1.2187 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00105: val_acc did not improve from 0.72115\n",
            "Epoch 106/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2725 - acc: 0.8962 - val_loss: 1.2198 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00106: val_acc did not improve from 0.72115\n",
            "Epoch 107/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2722 - acc: 0.8962 - val_loss: 1.2209 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00107: val_acc did not improve from 0.72115\n",
            "Epoch 108/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2718 - acc: 0.8962 - val_loss: 1.2221 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00108: val_acc did not improve from 0.72115\n",
            "Epoch 109/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2715 - acc: 0.8962 - val_loss: 1.2235 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00109: val_acc did not improve from 0.72115\n",
            "Epoch 110/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2711 - acc: 0.8962 - val_loss: 1.2251 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00110: val_acc did not improve from 0.72115\n",
            "Epoch 111/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2708 - acc: 0.8962 - val_loss: 1.2271 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00111: val_acc did not improve from 0.72115\n",
            "Epoch 112/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2704 - acc: 0.8962 - val_loss: 1.2291 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00112: val_acc did not improve from 0.72115\n",
            "Epoch 113/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2701 - acc: 0.8962 - val_loss: 1.2308 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00113: val_acc did not improve from 0.72115\n",
            "Epoch 114/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2698 - acc: 0.8962 - val_loss: 1.2322 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00114: val_acc did not improve from 0.72115\n",
            "Epoch 115/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2694 - acc: 0.8962 - val_loss: 1.2335 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00115: val_acc did not improve from 0.72115\n",
            "Epoch 116/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2691 - acc: 0.8962 - val_loss: 1.2349 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00116: val_acc did not improve from 0.72115\n",
            "Epoch 117/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2688 - acc: 0.8962 - val_loss: 1.2364 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00117: val_acc did not improve from 0.72115\n",
            "Epoch 118/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2684 - acc: 0.8962 - val_loss: 1.2379 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00118: val_acc did not improve from 0.72115\n",
            "Epoch 119/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2681 - acc: 0.8962 - val_loss: 1.2394 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00119: val_acc did not improve from 0.72115\n",
            "Epoch 120/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2677 - acc: 0.8962 - val_loss: 1.2408 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00120: val_acc did not improve from 0.72115\n",
            "Epoch 121/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2674 - acc: 0.8962 - val_loss: 1.2422 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00121: val_acc did not improve from 0.72115\n",
            "Epoch 122/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2670 - acc: 0.8962 - val_loss: 1.2436 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00122: val_acc did not improve from 0.72115\n",
            "Epoch 123/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2666 - acc: 0.8962 - val_loss: 1.2450 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00123: val_acc did not improve from 0.72115\n",
            "Epoch 124/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2663 - acc: 0.8962 - val_loss: 1.2464 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00124: val_acc did not improve from 0.72115\n",
            "Epoch 125/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2659 - acc: 0.8962 - val_loss: 1.2478 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00125: val_acc did not improve from 0.72115\n",
            "Epoch 126/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2655 - acc: 0.8962 - val_loss: 1.2491 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00126: val_acc did not improve from 0.72115\n",
            "Epoch 127/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2651 - acc: 0.8962 - val_loss: 1.2503 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00127: val_acc did not improve from 0.72115\n",
            "Epoch 128/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2647 - acc: 0.8962 - val_loss: 1.2514 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00128: val_acc did not improve from 0.72115\n",
            "Epoch 129/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2643 - acc: 0.8962 - val_loss: 1.2525 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00129: val_acc did not improve from 0.72115\n",
            "Epoch 130/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2639 - acc: 0.8962 - val_loss: 1.2535 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00130: val_acc did not improve from 0.72115\n",
            "Epoch 131/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2635 - acc: 0.8962 - val_loss: 1.2545 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00131: val_acc did not improve from 0.72115\n",
            "Epoch 132/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2631 - acc: 0.8962 - val_loss: 1.2555 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00132: val_acc did not improve from 0.72115\n",
            "Epoch 133/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2627 - acc: 0.8962 - val_loss: 1.2566 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00133: val_acc did not improve from 0.72115\n",
            "Epoch 134/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2623 - acc: 0.8962 - val_loss: 1.2577 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00134: val_acc did not improve from 0.72115\n",
            "Epoch 135/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2620 - acc: 0.8962 - val_loss: 1.2589 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00135: val_acc did not improve from 0.72115\n",
            "Epoch 136/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2616 - acc: 0.8962 - val_loss: 1.2601 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00136: val_acc did not improve from 0.72115\n",
            "Epoch 137/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2612 - acc: 0.8962 - val_loss: 1.2613 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00137: val_acc did not improve from 0.72115\n",
            "Epoch 138/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2609 - acc: 0.8962 - val_loss: 1.2624 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00138: val_acc did not improve from 0.72115\n",
            "Epoch 139/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2606 - acc: 0.8962 - val_loss: 1.2635 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00139: val_acc did not improve from 0.72115\n",
            "Epoch 140/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2602 - acc: 0.8962 - val_loss: 1.2647 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00140: val_acc did not improve from 0.72115\n",
            "Epoch 141/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2599 - acc: 0.8962 - val_loss: 1.2658 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00141: val_acc did not improve from 0.72115\n",
            "Epoch 142/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2596 - acc: 0.8962 - val_loss: 1.2669 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00142: val_acc did not improve from 0.72115\n",
            "Epoch 143/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2593 - acc: 0.8962 - val_loss: 1.2680 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00143: val_acc did not improve from 0.72115\n",
            "Epoch 144/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2589 - acc: 0.8962 - val_loss: 1.2691 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00144: val_acc did not improve from 0.72115\n",
            "Epoch 145/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2586 - acc: 0.8962 - val_loss: 1.2703 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00145: val_acc did not improve from 0.72115\n",
            "Epoch 146/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2583 - acc: 0.8962 - val_loss: 1.2714 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00146: val_acc did not improve from 0.72115\n",
            "Epoch 147/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2580 - acc: 0.8962 - val_loss: 1.2726 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00147: val_acc did not improve from 0.72115\n",
            "Epoch 148/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2577 - acc: 0.8962 - val_loss: 1.2738 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00148: val_acc did not improve from 0.72115\n",
            "Epoch 149/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2575 - acc: 0.8962 - val_loss: 1.2751 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00149: val_acc did not improve from 0.72115\n",
            "Epoch 150/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2572 - acc: 0.8962 - val_loss: 1.2764 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00150: val_acc did not improve from 0.72115\n",
            "Epoch 151/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2569 - acc: 0.8962 - val_loss: 1.2778 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00151: val_acc did not improve from 0.72115\n",
            "Epoch 152/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2566 - acc: 0.8962 - val_loss: 1.2791 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00152: val_acc did not improve from 0.72115\n",
            "Epoch 153/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2563 - acc: 0.8962 - val_loss: 1.2804 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00153: val_acc did not improve from 0.72115\n",
            "Epoch 154/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2561 - acc: 0.8962 - val_loss: 1.2816 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00154: val_acc did not improve from 0.72115\n",
            "Epoch 155/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2558 - acc: 0.8962 - val_loss: 1.2829 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00155: val_acc did not improve from 0.72115\n",
            "Epoch 156/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2555 - acc: 0.8962 - val_loss: 1.2842 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00156: val_acc did not improve from 0.72115\n",
            "Epoch 157/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2552 - acc: 0.8962 - val_loss: 1.2856 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00157: val_acc did not improve from 0.72115\n",
            "Epoch 158/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2550 - acc: 0.8962 - val_loss: 1.2869 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00158: val_acc did not improve from 0.72115\n",
            "Epoch 159/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2547 - acc: 0.8962 - val_loss: 1.2883 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00159: val_acc did not improve from 0.72115\n",
            "Epoch 160/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2544 - acc: 0.8962 - val_loss: 1.2897 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00160: val_acc did not improve from 0.72115\n",
            "Epoch 161/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2542 - acc: 0.8962 - val_loss: 1.2911 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00161: val_acc did not improve from 0.72115\n",
            "Epoch 162/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2539 - acc: 0.8962 - val_loss: 1.2925 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00162: val_acc did not improve from 0.72115\n",
            "Epoch 163/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2536 - acc: 0.8962 - val_loss: 1.2939 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00163: val_acc did not improve from 0.72115\n",
            "Epoch 164/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2533 - acc: 0.8962 - val_loss: 1.2952 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00164: val_acc did not improve from 0.72115\n",
            "Epoch 165/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2530 - acc: 0.8962 - val_loss: 1.2966 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00165: val_acc did not improve from 0.72115\n",
            "Epoch 166/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2527 - acc: 0.8962 - val_loss: 1.2979 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00166: val_acc did not improve from 0.72115\n",
            "Epoch 167/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2524 - acc: 0.8962 - val_loss: 1.2993 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00167: val_acc did not improve from 0.72115\n",
            "Epoch 168/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2521 - acc: 0.8962 - val_loss: 1.3006 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00168: val_acc did not improve from 0.72115\n",
            "Epoch 169/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2518 - acc: 0.8962 - val_loss: 1.3017 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00169: val_acc did not improve from 0.72115\n",
            "Epoch 170/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2516 - acc: 0.8962 - val_loss: 1.3025 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00170: val_acc did not improve from 0.72115\n",
            "Epoch 171/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2513 - acc: 0.8962 - val_loss: 1.3031 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00171: val_acc did not improve from 0.72115\n",
            "Epoch 172/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2511 - acc: 0.9057 - val_loss: 1.3036 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00172: val_acc did not improve from 0.72115\n",
            "Epoch 173/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2508 - acc: 0.9057 - val_loss: 1.3041 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00173: val_acc did not improve from 0.72115\n",
            "Epoch 174/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2505 - acc: 0.9057 - val_loss: 1.3045 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00174: val_acc did not improve from 0.72115\n",
            "Epoch 175/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2502 - acc: 0.8962 - val_loss: 1.3051 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00175: val_acc did not improve from 0.72115\n",
            "Epoch 176/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2499 - acc: 0.8962 - val_loss: 1.3058 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00176: val_acc did not improve from 0.72115\n",
            "Epoch 177/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2496 - acc: 0.8962 - val_loss: 1.3064 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00177: val_acc did not improve from 0.72115\n",
            "Epoch 178/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2493 - acc: 0.8962 - val_loss: 1.3067 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00178: val_acc did not improve from 0.72115\n",
            "Epoch 179/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2490 - acc: 0.8962 - val_loss: 1.3068 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00179: val_acc did not improve from 0.72115\n",
            "Epoch 180/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2487 - acc: 0.8962 - val_loss: 1.3068 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00180: val_acc did not improve from 0.72115\n",
            "Epoch 181/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2484 - acc: 0.8962 - val_loss: 1.3069 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00181: val_acc did not improve from 0.72115\n",
            "Epoch 182/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2481 - acc: 0.8962 - val_loss: 1.3074 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00182: val_acc did not improve from 0.72115\n",
            "Epoch 183/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2478 - acc: 0.8962 - val_loss: 1.3081 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00183: val_acc did not improve from 0.72115\n",
            "Epoch 184/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2475 - acc: 0.8962 - val_loss: 1.3089 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00184: val_acc did not improve from 0.72115\n",
            "Epoch 185/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2472 - acc: 0.8962 - val_loss: 1.3097 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00185: val_acc did not improve from 0.72115\n",
            "Epoch 186/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2469 - acc: 0.8962 - val_loss: 1.3104 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00186: val_acc did not improve from 0.72115\n",
            "Epoch 187/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2466 - acc: 0.9057 - val_loss: 1.3110 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00187: val_acc did not improve from 0.72115\n",
            "Epoch 188/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2463 - acc: 0.9057 - val_loss: 1.3117 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00188: val_acc did not improve from 0.72115\n",
            "Epoch 189/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2460 - acc: 0.9057 - val_loss: 1.3124 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00189: val_acc did not improve from 0.72115\n",
            "Epoch 190/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2458 - acc: 0.9057 - val_loss: 1.3132 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00190: val_acc did not improve from 0.72115\n",
            "Epoch 191/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2455 - acc: 0.9057 - val_loss: 1.3140 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00191: val_acc did not improve from 0.72115\n",
            "Epoch 192/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2452 - acc: 0.9057 - val_loss: 1.3149 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00192: val_acc did not improve from 0.72115\n",
            "Epoch 193/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2450 - acc: 0.9057 - val_loss: 1.3158 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00193: val_acc did not improve from 0.72115\n",
            "Epoch 194/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2447 - acc: 0.9057 - val_loss: 1.3168 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00194: val_acc did not improve from 0.72115\n",
            "Epoch 195/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2444 - acc: 0.9057 - val_loss: 1.3177 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00195: val_acc did not improve from 0.72115\n",
            "Epoch 196/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2442 - acc: 0.9057 - val_loss: 1.3186 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00196: val_acc did not improve from 0.72115\n",
            "Epoch 197/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2438 - acc: 0.9057 - val_loss: 1.3196 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00197: val_acc did not improve from 0.72115\n",
            "Epoch 198/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2433 - acc: 0.9057 - val_loss: 1.3205 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00198: val_acc did not improve from 0.72115\n",
            "Epoch 199/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2423 - acc: 0.9057 - val_loss: 1.3214 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00199: val_acc did not improve from 0.72115\n",
            "Epoch 200/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2415 - acc: 0.9057 - val_loss: 1.3222 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00200: val_acc did not improve from 0.72115\n",
            "Epoch 201/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2415 - acc: 0.9057 - val_loss: 1.3230 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00201: val_acc did not improve from 0.72115\n",
            "Epoch 202/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2411 - acc: 0.9057 - val_loss: 1.3239 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00202: val_acc did not improve from 0.72115\n",
            "Epoch 203/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2405 - acc: 0.8962 - val_loss: 1.3250 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00203: val_acc did not improve from 0.72115\n",
            "Epoch 204/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2401 - acc: 0.8962 - val_loss: 1.3260 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00204: val_acc did not improve from 0.72115\n",
            "Epoch 205/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2395 - acc: 0.9057 - val_loss: 1.3270 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00205: val_acc did not improve from 0.72115\n",
            "Epoch 206/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2390 - acc: 0.9057 - val_loss: 1.3282 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00206: val_acc did not improve from 0.72115\n",
            "Epoch 207/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2387 - acc: 0.9057 - val_loss: 1.3294 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00207: val_acc did not improve from 0.72115\n",
            "Epoch 208/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2386 - acc: 0.9057 - val_loss: 1.3306 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00208: val_acc did not improve from 0.72115\n",
            "Epoch 209/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2382 - acc: 0.9057 - val_loss: 1.3318 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00209: val_acc did not improve from 0.72115\n",
            "Epoch 210/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2377 - acc: 0.9057 - val_loss: 1.3331 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00210: val_acc did not improve from 0.72115\n",
            "Epoch 211/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2372 - acc: 0.9057 - val_loss: 1.3345 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00211: val_acc did not improve from 0.72115\n",
            "Epoch 212/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2369 - acc: 0.9057 - val_loss: 1.3360 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00212: val_acc did not improve from 0.72115\n",
            "Epoch 213/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2367 - acc: 0.9057 - val_loss: 1.3375 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00213: val_acc did not improve from 0.72115\n",
            "Epoch 214/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2364 - acc: 0.9057 - val_loss: 1.3391 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00214: val_acc did not improve from 0.72115\n",
            "Epoch 215/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2360 - acc: 0.9057 - val_loss: 1.3407 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00215: val_acc did not improve from 0.72115\n",
            "Epoch 216/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2357 - acc: 0.9057 - val_loss: 1.3425 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00216: val_acc did not improve from 0.72115\n",
            "Epoch 217/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2354 - acc: 0.9057 - val_loss: 1.3443 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00217: val_acc did not improve from 0.72115\n",
            "Epoch 218/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2350 - acc: 0.9057 - val_loss: 1.3462 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00218: val_acc did not improve from 0.72115\n",
            "Epoch 219/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2347 - acc: 0.9057 - val_loss: 1.3480 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00219: val_acc did not improve from 0.72115\n",
            "Epoch 220/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2343 - acc: 0.9057 - val_loss: 1.3500 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00220: val_acc did not improve from 0.72115\n",
            "Epoch 221/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2340 - acc: 0.9057 - val_loss: 1.3521 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00221: val_acc did not improve from 0.72115\n",
            "Epoch 222/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2337 - acc: 0.9057 - val_loss: 1.3543 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00222: val_acc did not improve from 0.72115\n",
            "Epoch 223/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2333 - acc: 0.9057 - val_loss: 1.3566 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00223: val_acc did not improve from 0.72115\n",
            "Epoch 224/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2330 - acc: 0.9057 - val_loss: 1.3587 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00224: val_acc did not improve from 0.72115\n",
            "Epoch 225/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2326 - acc: 0.9057 - val_loss: 1.3607 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00225: val_acc did not improve from 0.72115\n",
            "Epoch 226/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2322 - acc: 0.9057 - val_loss: 1.3627 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00226: val_acc did not improve from 0.72115\n",
            "Epoch 227/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2318 - acc: 0.9057 - val_loss: 1.3647 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00227: val_acc did not improve from 0.72115\n",
            "Epoch 228/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2314 - acc: 0.9057 - val_loss: 1.3668 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00228: val_acc did not improve from 0.72115\n",
            "Epoch 229/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2310 - acc: 0.9057 - val_loss: 1.3691 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00229: val_acc did not improve from 0.72115\n",
            "Epoch 230/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2306 - acc: 0.9057 - val_loss: 1.3716 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00230: val_acc did not improve from 0.72115\n",
            "Epoch 231/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2302 - acc: 0.9057 - val_loss: 1.3745 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00231: val_acc did not improve from 0.72115\n",
            "Epoch 232/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2297 - acc: 0.9057 - val_loss: 1.3776 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00232: val_acc did not improve from 0.72115\n",
            "Epoch 233/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2293 - acc: 0.9057 - val_loss: 1.3809 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00233: val_acc did not improve from 0.72115\n",
            "Epoch 234/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2288 - acc: 0.9057 - val_loss: 1.3840 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00234: val_acc did not improve from 0.72115\n",
            "Epoch 235/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2283 - acc: 0.9057 - val_loss: 1.3870 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00235: val_acc did not improve from 0.72115\n",
            "Epoch 236/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2278 - acc: 0.9057 - val_loss: 1.3899 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00236: val_acc did not improve from 0.72115\n",
            "Epoch 237/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2272 - acc: 0.9057 - val_loss: 1.3927 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00237: val_acc did not improve from 0.72115\n",
            "Epoch 238/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2267 - acc: 0.9057 - val_loss: 1.3955 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00238: val_acc did not improve from 0.72115\n",
            "Epoch 239/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2262 - acc: 0.9057 - val_loss: 1.3982 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00239: val_acc did not improve from 0.72115\n",
            "Epoch 240/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2258 - acc: 0.9057 - val_loss: 1.4009 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00240: val_acc did not improve from 0.72115\n",
            "Epoch 241/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2254 - acc: 0.9057 - val_loss: 1.4035 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00241: val_acc did not improve from 0.72115\n",
            "Epoch 242/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2250 - acc: 0.9057 - val_loss: 1.4062 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00242: val_acc did not improve from 0.72115\n",
            "Epoch 243/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2245 - acc: 0.9057 - val_loss: 1.4089 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00243: val_acc did not improve from 0.72115\n",
            "Epoch 244/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2241 - acc: 0.9057 - val_loss: 1.4118 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00244: val_acc did not improve from 0.72115\n",
            "Epoch 245/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2237 - acc: 0.9057 - val_loss: 1.4146 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00245: val_acc did not improve from 0.72115\n",
            "Epoch 246/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2233 - acc: 0.9057 - val_loss: 1.4173 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00246: val_acc did not improve from 0.72115\n",
            "Epoch 247/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2229 - acc: 0.9057 - val_loss: 1.4197 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00247: val_acc did not improve from 0.72115\n",
            "Epoch 248/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2224 - acc: 0.9057 - val_loss: 1.4220 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00248: val_acc did not improve from 0.72115\n",
            "Epoch 249/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2220 - acc: 0.9057 - val_loss: 1.4242 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00249: val_acc did not improve from 0.72115\n",
            "Epoch 250/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2215 - acc: 0.9057 - val_loss: 1.4263 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00250: val_acc did not improve from 0.72115\n",
            "Epoch 251/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2210 - acc: 0.9057 - val_loss: 1.4285 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00251: val_acc did not improve from 0.72115\n",
            "Epoch 252/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2205 - acc: 0.9057 - val_loss: 1.4308 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00252: val_acc did not improve from 0.72115\n",
            "Epoch 253/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2199 - acc: 0.9057 - val_loss: 1.4330 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00253: val_acc did not improve from 0.72115\n",
            "Epoch 254/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2193 - acc: 0.9057 - val_loss: 1.4352 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00254: val_acc did not improve from 0.72115\n",
            "Epoch 255/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2187 - acc: 0.9057 - val_loss: 1.4373 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00255: val_acc did not improve from 0.72115\n",
            "Epoch 256/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2180 - acc: 0.9057 - val_loss: 1.4393 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00256: val_acc did not improve from 0.72115\n",
            "Epoch 257/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2174 - acc: 0.9057 - val_loss: 1.4414 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00257: val_acc did not improve from 0.72115\n",
            "Epoch 258/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2168 - acc: 0.9057 - val_loss: 1.4435 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00258: val_acc did not improve from 0.72115\n",
            "Epoch 259/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2161 - acc: 0.9057 - val_loss: 1.4455 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00259: val_acc did not improve from 0.72115\n",
            "Epoch 260/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2155 - acc: 0.9057 - val_loss: 1.4477 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00260: val_acc did not improve from 0.72115\n",
            "Epoch 261/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2149 - acc: 0.9057 - val_loss: 1.4498 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00261: val_acc did not improve from 0.72115\n",
            "Epoch 262/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2142 - acc: 0.9057 - val_loss: 1.4519 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00262: val_acc did not improve from 0.72115\n",
            "Epoch 263/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2136 - acc: 0.9057 - val_loss: 1.4541 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00263: val_acc did not improve from 0.72115\n",
            "Epoch 264/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2131 - acc: 0.9057 - val_loss: 1.4562 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00264: val_acc did not improve from 0.72115\n",
            "Epoch 265/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2125 - acc: 0.9057 - val_loss: 1.4583 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00265: val_acc did not improve from 0.72115\n",
            "Epoch 266/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2119 - acc: 0.9057 - val_loss: 1.4604 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00266: val_acc did not improve from 0.72115\n",
            "Epoch 267/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2113 - acc: 0.9057 - val_loss: 1.4624 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00267: val_acc did not improve from 0.72115\n",
            "Epoch 268/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2107 - acc: 0.9057 - val_loss: 1.4644 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00268: val_acc did not improve from 0.72115\n",
            "Epoch 269/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2101 - acc: 0.9057 - val_loss: 1.4664 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00269: val_acc did not improve from 0.72115\n",
            "Epoch 270/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2095 - acc: 0.9057 - val_loss: 1.4685 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00270: val_acc did not improve from 0.72115\n",
            "Epoch 271/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2089 - acc: 0.9057 - val_loss: 1.4705 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00271: val_acc did not improve from 0.72115\n",
            "Epoch 272/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2083 - acc: 0.9057 - val_loss: 1.4726 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00272: val_acc did not improve from 0.72115\n",
            "Epoch 273/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2076 - acc: 0.9057 - val_loss: 1.4746 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00273: val_acc did not improve from 0.72115\n",
            "Epoch 274/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2069 - acc: 0.9057 - val_loss: 1.4766 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00274: val_acc did not improve from 0.72115\n",
            "Epoch 275/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2061 - acc: 0.9057 - val_loss: 1.4786 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00275: val_acc did not improve from 0.72115\n",
            "Epoch 276/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2053 - acc: 0.9057 - val_loss: 1.4806 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00276: val_acc did not improve from 0.72115\n",
            "Epoch 277/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2045 - acc: 0.9057 - val_loss: 1.4825 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00277: val_acc did not improve from 0.72115\n",
            "Epoch 278/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2036 - acc: 0.9057 - val_loss: 1.4843 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00278: val_acc did not improve from 0.72115\n",
            "Epoch 279/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2026 - acc: 0.9057 - val_loss: 1.4860 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00279: val_acc did not improve from 0.72115\n",
            "Epoch 280/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.2015 - acc: 0.9057 - val_loss: 1.4878 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00280: val_acc did not improve from 0.72115\n",
            "Epoch 281/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1985 - acc: 0.9057 - val_loss: 1.4890 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00281: val_acc did not improve from 0.72115\n",
            "Epoch 282/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1968 - acc: 0.9151 - val_loss: 1.4897 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00282: val_acc did not improve from 0.72115\n",
            "Epoch 283/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1936 - acc: 0.9245 - val_loss: 1.4908 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00283: val_acc did not improve from 0.72115\n",
            "Epoch 284/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1922 - acc: 0.9245 - val_loss: 1.4900 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00284: val_acc did not improve from 0.72115\n",
            "Epoch 285/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1916 - acc: 0.9245 - val_loss: 1.4931 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00285: val_acc did not improve from 0.72115\n",
            "Epoch 286/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1889 - acc: 0.9245 - val_loss: 1.4951 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00286: val_acc did not improve from 0.72115\n",
            "Epoch 287/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1870 - acc: 0.9245 - val_loss: 1.4955 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00287: val_acc did not improve from 0.72115\n",
            "Epoch 288/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1857 - acc: 0.9245 - val_loss: 1.4969 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00288: val_acc did not improve from 0.72115\n",
            "Epoch 289/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1844 - acc: 0.9245 - val_loss: 1.4990 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00289: val_acc did not improve from 0.72115\n",
            "Epoch 290/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1827 - acc: 0.9245 - val_loss: 1.4997 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00290: val_acc did not improve from 0.72115\n",
            "Epoch 291/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1809 - acc: 0.9245 - val_loss: 1.4981 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00291: val_acc did not improve from 0.72115\n",
            "Epoch 292/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1789 - acc: 0.9245 - val_loss: 1.4988 - val_acc: 0.5577\n",
            "\n",
            "Epoch 00292: val_acc did not improve from 0.72115\n",
            "Epoch 293/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1768 - acc: 0.9245 - val_loss: 1.5006 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00293: val_acc did not improve from 0.72115\n",
            "Epoch 294/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1755 - acc: 0.9245 - val_loss: 1.5010 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00294: val_acc did not improve from 0.72115\n",
            "Epoch 295/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1737 - acc: 0.9245 - val_loss: 1.4987 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00295: val_acc did not improve from 0.72115\n",
            "Epoch 296/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1725 - acc: 0.9245 - val_loss: 1.4997 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00296: val_acc did not improve from 0.72115\n",
            "Epoch 297/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1706 - acc: 0.9245 - val_loss: 1.5046 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00297: val_acc did not improve from 0.72115\n",
            "Epoch 298/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1680 - acc: 0.9245 - val_loss: 1.5079 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00298: val_acc did not improve from 0.72115\n",
            "Epoch 299/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1667 - acc: 0.9245 - val_loss: 1.5104 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00299: val_acc did not improve from 0.72115\n",
            "Epoch 300/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1662 - acc: 0.9245 - val_loss: 1.5128 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00300: val_acc did not improve from 0.72115\n",
            "Epoch 301/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1647 - acc: 0.9340 - val_loss: 1.5157 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00301: val_acc did not improve from 0.72115\n",
            "Epoch 302/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1627 - acc: 0.9340 - val_loss: 1.5188 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00302: val_acc did not improve from 0.72115\n",
            "Epoch 303/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1611 - acc: 0.9340 - val_loss: 1.5184 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00303: val_acc did not improve from 0.72115\n",
            "Epoch 304/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1591 - acc: 0.9340 - val_loss: 1.5186 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00304: val_acc did not improve from 0.72115\n",
            "Epoch 305/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1581 - acc: 0.9340 - val_loss: 1.5216 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00305: val_acc did not improve from 0.72115\n",
            "Epoch 306/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1567 - acc: 0.9340 - val_loss: 1.5248 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00306: val_acc did not improve from 0.72115\n",
            "Epoch 307/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1558 - acc: 0.9340 - val_loss: 1.5268 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00307: val_acc did not improve from 0.72115\n",
            "Epoch 308/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1539 - acc: 0.9340 - val_loss: 1.5286 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00308: val_acc did not improve from 0.72115\n",
            "Epoch 309/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1533 - acc: 0.9434 - val_loss: 1.5320 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00309: val_acc did not improve from 0.72115\n",
            "Epoch 310/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1521 - acc: 0.9340 - val_loss: 1.5338 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00310: val_acc did not improve from 0.72115\n",
            "Epoch 311/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1511 - acc: 0.9434 - val_loss: 1.5341 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00311: val_acc did not improve from 0.72115\n",
            "Epoch 312/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1481 - acc: 0.9434 - val_loss: 1.5338 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00312: val_acc did not improve from 0.72115\n",
            "Epoch 313/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1502 - acc: 0.9434 - val_loss: 1.5386 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00313: val_acc did not improve from 0.72115\n",
            "Epoch 314/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1452 - acc: 0.9434 - val_loss: 1.5416 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00314: val_acc did not improve from 0.72115\n",
            "Epoch 315/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1455 - acc: 0.9528 - val_loss: 1.5443 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00315: val_acc did not improve from 0.72115\n",
            "Epoch 316/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1434 - acc: 0.9528 - val_loss: 1.5469 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00316: val_acc did not improve from 0.72115\n",
            "Epoch 317/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1425 - acc: 0.9528 - val_loss: 1.5495 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00317: val_acc did not improve from 0.72115\n",
            "Epoch 318/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1421 - acc: 0.9528 - val_loss: 1.5523 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00318: val_acc did not improve from 0.72115\n",
            "Epoch 319/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1406 - acc: 0.9528 - val_loss: 1.5550 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00319: val_acc did not improve from 0.72115\n",
            "Epoch 320/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1385 - acc: 0.9623 - val_loss: 1.5576 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00320: val_acc did not improve from 0.72115\n",
            "Epoch 321/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1375 - acc: 0.9623 - val_loss: 1.5599 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00321: val_acc did not improve from 0.72115\n",
            "Epoch 322/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1341 - acc: 0.9717 - val_loss: 1.5623 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00322: val_acc did not improve from 0.72115\n",
            "Epoch 323/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1318 - acc: 0.9717 - val_loss: 1.5643 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00323: val_acc did not improve from 0.72115\n",
            "Epoch 324/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1324 - acc: 0.9717 - val_loss: 1.5664 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00324: val_acc did not improve from 0.72115\n",
            "Epoch 325/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1321 - acc: 0.9717 - val_loss: 1.5684 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00325: val_acc did not improve from 0.72115\n",
            "Epoch 326/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1309 - acc: 0.9811 - val_loss: 1.5696 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00326: val_acc did not improve from 0.72115\n",
            "Epoch 327/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1295 - acc: 0.9811 - val_loss: 1.5717 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00327: val_acc did not improve from 0.72115\n",
            "Epoch 328/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1271 - acc: 0.9811 - val_loss: 1.5736 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00328: val_acc did not improve from 0.72115\n",
            "Epoch 329/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1256 - acc: 0.9811 - val_loss: 1.5754 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00329: val_acc did not improve from 0.72115\n",
            "Epoch 330/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1240 - acc: 0.9811 - val_loss: 1.5767 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00330: val_acc did not improve from 0.72115\n",
            "Epoch 331/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1223 - acc: 0.9811 - val_loss: 1.5777 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00331: val_acc did not improve from 0.72115\n",
            "Epoch 332/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1209 - acc: 0.9811 - val_loss: 1.5815 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00332: val_acc did not improve from 0.72115\n",
            "Epoch 333/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1196 - acc: 0.9811 - val_loss: 1.5872 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00333: val_acc did not improve from 0.72115\n",
            "Epoch 334/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1177 - acc: 0.9811 - val_loss: 1.5914 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00334: val_acc did not improve from 0.72115\n",
            "Epoch 335/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1165 - acc: 0.9811 - val_loss: 1.5942 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00335: val_acc did not improve from 0.72115\n",
            "Epoch 336/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1156 - acc: 0.9811 - val_loss: 1.5961 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00336: val_acc did not improve from 0.72115\n",
            "Epoch 337/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1149 - acc: 0.9811 - val_loss: 1.5975 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00337: val_acc did not improve from 0.72115\n",
            "Epoch 338/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1141 - acc: 0.9811 - val_loss: 1.5986 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00338: val_acc did not improve from 0.72115\n",
            "Epoch 339/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1130 - acc: 0.9811 - val_loss: 1.5995 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00339: val_acc did not improve from 0.72115\n",
            "Epoch 340/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1119 - acc: 0.9811 - val_loss: 1.6004 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00340: val_acc did not improve from 0.72115\n",
            "Epoch 341/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1107 - acc: 0.9811 - val_loss: 1.6016 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00341: val_acc did not improve from 0.72115\n",
            "Epoch 342/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1096 - acc: 0.9811 - val_loss: 1.6035 - val_acc: 0.5481\n",
            "\n",
            "Epoch 00342: val_acc did not improve from 0.72115\n",
            "Epoch 343/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1085 - acc: 0.9811 - val_loss: 1.6059 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00343: val_acc did not improve from 0.72115\n",
            "Epoch 344/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1071 - acc: 0.9811 - val_loss: 1.6083 - val_acc: 0.5385\n",
            "\n",
            "Epoch 00344: val_acc did not improve from 0.72115\n",
            "Epoch 345/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1060 - acc: 0.9811 - val_loss: 1.6105 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00345: val_acc did not improve from 0.72115\n",
            "Epoch 346/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1052 - acc: 0.9811 - val_loss: 1.6127 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00346: val_acc did not improve from 0.72115\n",
            "Epoch 347/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1046 - acc: 0.9811 - val_loss: 1.6149 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00347: val_acc did not improve from 0.72115\n",
            "Epoch 348/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1039 - acc: 0.9811 - val_loss: 1.6171 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00348: val_acc did not improve from 0.72115\n",
            "Epoch 349/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1033 - acc: 0.9811 - val_loss: 1.6191 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00349: val_acc did not improve from 0.72115\n",
            "Epoch 350/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1028 - acc: 0.9811 - val_loss: 1.6211 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00350: val_acc did not improve from 0.72115\n",
            "Epoch 351/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1022 - acc: 0.9811 - val_loss: 1.6230 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00351: val_acc did not improve from 0.72115\n",
            "Epoch 352/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1016 - acc: 0.9811 - val_loss: 1.6252 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00352: val_acc did not improve from 0.72115\n",
            "Epoch 353/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1012 - acc: 0.9811 - val_loss: 1.6274 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00353: val_acc did not improve from 0.72115\n",
            "Epoch 354/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.1006 - acc: 0.9811 - val_loss: 1.6299 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00354: val_acc did not improve from 0.72115\n",
            "Epoch 355/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0999 - acc: 0.9811 - val_loss: 1.6324 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00355: val_acc did not improve from 0.72115\n",
            "Epoch 356/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0993 - acc: 0.9811 - val_loss: 1.6351 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00356: val_acc did not improve from 0.72115\n",
            "Epoch 357/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0987 - acc: 0.9811 - val_loss: 1.6377 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00357: val_acc did not improve from 0.72115\n",
            "Epoch 358/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0981 - acc: 0.9811 - val_loss: 1.6404 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00358: val_acc did not improve from 0.72115\n",
            "Epoch 359/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0976 - acc: 0.9811 - val_loss: 1.6430 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00359: val_acc did not improve from 0.72115\n",
            "Epoch 360/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0970 - acc: 0.9811 - val_loss: 1.6455 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00360: val_acc did not improve from 0.72115\n",
            "Epoch 361/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0965 - acc: 0.9811 - val_loss: 1.6480 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00361: val_acc did not improve from 0.72115\n",
            "Epoch 362/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0959 - acc: 0.9811 - val_loss: 1.6505 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00362: val_acc did not improve from 0.72115\n",
            "Epoch 363/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0953 - acc: 0.9811 - val_loss: 1.6529 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00363: val_acc did not improve from 0.72115\n",
            "Epoch 364/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0946 - acc: 0.9811 - val_loss: 1.6552 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00364: val_acc did not improve from 0.72115\n",
            "Epoch 365/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0939 - acc: 0.9811 - val_loss: 1.6577 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00365: val_acc did not improve from 0.72115\n",
            "Epoch 366/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0934 - acc: 0.9811 - val_loss: 1.6604 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00366: val_acc did not improve from 0.72115\n",
            "Epoch 367/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0928 - acc: 0.9811 - val_loss: 1.6631 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00367: val_acc did not improve from 0.72115\n",
            "Epoch 368/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0924 - acc: 0.9811 - val_loss: 1.6661 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00368: val_acc did not improve from 0.72115\n",
            "Epoch 369/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0918 - acc: 0.9811 - val_loss: 1.6691 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00369: val_acc did not improve from 0.72115\n",
            "Epoch 370/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0913 - acc: 0.9811 - val_loss: 1.6723 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00370: val_acc did not improve from 0.72115\n",
            "Epoch 371/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0908 - acc: 0.9811 - val_loss: 1.6754 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00371: val_acc did not improve from 0.72115\n",
            "Epoch 372/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0903 - acc: 0.9811 - val_loss: 1.6785 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00372: val_acc did not improve from 0.72115\n",
            "Epoch 373/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0899 - acc: 0.9811 - val_loss: 1.6814 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00373: val_acc did not improve from 0.72115\n",
            "Epoch 374/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0895 - acc: 0.9811 - val_loss: 1.6841 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00374: val_acc did not improve from 0.72115\n",
            "Epoch 375/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0891 - acc: 0.9811 - val_loss: 1.6867 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00375: val_acc did not improve from 0.72115\n",
            "Epoch 376/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0886 - acc: 0.9811 - val_loss: 1.6892 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00376: val_acc did not improve from 0.72115\n",
            "Epoch 377/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0882 - acc: 0.9811 - val_loss: 1.6914 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00377: val_acc did not improve from 0.72115\n",
            "Epoch 378/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0878 - acc: 0.9811 - val_loss: 1.6936 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00378: val_acc did not improve from 0.72115\n",
            "Epoch 379/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0874 - acc: 0.9811 - val_loss: 1.6960 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00379: val_acc did not improve from 0.72115\n",
            "Epoch 380/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0871 - acc: 0.9811 - val_loss: 1.6988 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00380: val_acc did not improve from 0.72115\n",
            "Epoch 381/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0867 - acc: 0.9811 - val_loss: 1.7017 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00381: val_acc did not improve from 0.72115\n",
            "Epoch 382/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0863 - acc: 0.9811 - val_loss: 1.7046 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00382: val_acc did not improve from 0.72115\n",
            "Epoch 383/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0859 - acc: 0.9811 - val_loss: 1.7074 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00383: val_acc did not improve from 0.72115\n",
            "Epoch 384/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0855 - acc: 0.9811 - val_loss: 1.7100 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00384: val_acc did not improve from 0.72115\n",
            "Epoch 385/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0852 - acc: 0.9811 - val_loss: 1.7125 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00385: val_acc did not improve from 0.72115\n",
            "Epoch 386/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0848 - acc: 0.9811 - val_loss: 1.7150 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00386: val_acc did not improve from 0.72115\n",
            "Epoch 387/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0845 - acc: 0.9811 - val_loss: 1.7173 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00387: val_acc did not improve from 0.72115\n",
            "Epoch 388/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0841 - acc: 0.9811 - val_loss: 1.7196 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00388: val_acc did not improve from 0.72115\n",
            "Epoch 389/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0838 - acc: 0.9811 - val_loss: 1.7217 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00389: val_acc did not improve from 0.72115\n",
            "Epoch 390/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0834 - acc: 0.9811 - val_loss: 1.7237 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00390: val_acc did not improve from 0.72115\n",
            "Epoch 391/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0831 - acc: 0.9811 - val_loss: 1.7259 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00391: val_acc did not improve from 0.72115\n",
            "Epoch 392/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0827 - acc: 0.9811 - val_loss: 1.7284 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00392: val_acc did not improve from 0.72115\n",
            "Epoch 393/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0824 - acc: 0.9811 - val_loss: 1.7312 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00393: val_acc did not improve from 0.72115\n",
            "Epoch 394/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0821 - acc: 0.9811 - val_loss: 1.7343 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00394: val_acc did not improve from 0.72115\n",
            "Epoch 395/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0818 - acc: 0.9811 - val_loss: 1.7373 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00395: val_acc did not improve from 0.72115\n",
            "Epoch 396/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0814 - acc: 0.9811 - val_loss: 1.7403 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00396: val_acc did not improve from 0.72115\n",
            "Epoch 397/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0811 - acc: 0.9811 - val_loss: 1.7432 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00397: val_acc did not improve from 0.72115\n",
            "Epoch 398/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0808 - acc: 0.9811 - val_loss: 1.7460 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00398: val_acc did not improve from 0.72115\n",
            "Epoch 399/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0805 - acc: 0.9811 - val_loss: 1.7489 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00399: val_acc did not improve from 0.72115\n",
            "Epoch 400/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0802 - acc: 0.9811 - val_loss: 1.7518 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00400: val_acc did not improve from 0.72115\n",
            "Epoch 401/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0799 - acc: 0.9811 - val_loss: 1.7546 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00401: val_acc did not improve from 0.72115\n",
            "Epoch 402/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0795 - acc: 0.9811 - val_loss: 1.7575 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00402: val_acc did not improve from 0.72115\n",
            "Epoch 403/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0792 - acc: 0.9811 - val_loss: 1.7603 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00403: val_acc did not improve from 0.72115\n",
            "Epoch 404/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0788 - acc: 0.9811 - val_loss: 1.7633 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00404: val_acc did not improve from 0.72115\n",
            "Epoch 405/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0784 - acc: 0.9811 - val_loss: 1.7664 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00405: val_acc did not improve from 0.72115\n",
            "Epoch 406/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0780 - acc: 0.9811 - val_loss: 1.7694 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00406: val_acc did not improve from 0.72115\n",
            "Epoch 407/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0775 - acc: 0.9811 - val_loss: 1.7724 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00407: val_acc did not improve from 0.72115\n",
            "Epoch 408/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0771 - acc: 0.9811 - val_loss: 1.7758 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00408: val_acc did not improve from 0.72115\n",
            "Epoch 409/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0767 - acc: 0.9811 - val_loss: 1.7798 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00409: val_acc did not improve from 0.72115\n",
            "Epoch 410/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0764 - acc: 0.9811 - val_loss: 1.7838 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00410: val_acc did not improve from 0.72115\n",
            "Epoch 411/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0760 - acc: 0.9811 - val_loss: 1.7873 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00411: val_acc did not improve from 0.72115\n",
            "Epoch 412/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0757 - acc: 0.9811 - val_loss: 1.7903 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00412: val_acc did not improve from 0.72115\n",
            "Epoch 413/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0754 - acc: 0.9811 - val_loss: 1.7930 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00413: val_acc did not improve from 0.72115\n",
            "Epoch 414/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0750 - acc: 0.9811 - val_loss: 1.7950 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00414: val_acc did not improve from 0.72115\n",
            "Epoch 415/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0746 - acc: 0.9811 - val_loss: 1.7963 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00415: val_acc did not improve from 0.72115\n",
            "Epoch 416/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0742 - acc: 0.9811 - val_loss: 1.7973 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00416: val_acc did not improve from 0.72115\n",
            "Epoch 417/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0739 - acc: 0.9811 - val_loss: 1.7988 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00417: val_acc did not improve from 0.72115\n",
            "Epoch 418/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0737 - acc: 0.9811 - val_loss: 1.8015 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00418: val_acc did not improve from 0.72115\n",
            "Epoch 419/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0734 - acc: 0.9811 - val_loss: 1.8052 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00419: val_acc did not improve from 0.72115\n",
            "Epoch 420/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0731 - acc: 0.9811 - val_loss: 1.8090 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00420: val_acc did not improve from 0.72115\n",
            "Epoch 421/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0728 - acc: 0.9811 - val_loss: 1.8128 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00421: val_acc did not improve from 0.72115\n",
            "Epoch 422/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0724 - acc: 0.9811 - val_loss: 1.8164 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00422: val_acc did not improve from 0.72115\n",
            "Epoch 423/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0720 - acc: 0.9811 - val_loss: 1.8201 - val_acc: 0.5288\n",
            "\n",
            "Epoch 00423: val_acc did not improve from 0.72115\n",
            "Epoch 424/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0716 - acc: 0.9811 - val_loss: 1.8249 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00424: val_acc did not improve from 0.72115\n",
            "Epoch 425/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0712 - acc: 0.9811 - val_loss: 1.8314 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00425: val_acc did not improve from 0.72115\n",
            "Epoch 426/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0708 - acc: 0.9811 - val_loss: 1.8365 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00426: val_acc did not improve from 0.72115\n",
            "Epoch 427/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0705 - acc: 0.9811 - val_loss: 1.8405 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00427: val_acc did not improve from 0.72115\n",
            "Epoch 428/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0702 - acc: 0.9811 - val_loss: 1.8439 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00428: val_acc did not improve from 0.72115\n",
            "Epoch 429/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0699 - acc: 0.9811 - val_loss: 1.8468 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00429: val_acc did not improve from 0.72115\n",
            "Epoch 430/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0696 - acc: 0.9811 - val_loss: 1.8495 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00430: val_acc did not improve from 0.72115\n",
            "Epoch 431/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0693 - acc: 0.9811 - val_loss: 1.8518 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00431: val_acc did not improve from 0.72115\n",
            "Epoch 432/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0690 - acc: 0.9811 - val_loss: 1.8537 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00432: val_acc did not improve from 0.72115\n",
            "Epoch 433/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0687 - acc: 0.9811 - val_loss: 1.8550 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00433: val_acc did not improve from 0.72115\n",
            "Epoch 434/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0684 - acc: 0.9811 - val_loss: 1.8563 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00434: val_acc did not improve from 0.72115\n",
            "Epoch 435/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0682 - acc: 0.9811 - val_loss: 1.8581 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00435: val_acc did not improve from 0.72115\n",
            "Epoch 436/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0679 - acc: 0.9811 - val_loss: 1.8608 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00436: val_acc did not improve from 0.72115\n",
            "Epoch 437/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0677 - acc: 0.9811 - val_loss: 1.8642 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00437: val_acc did not improve from 0.72115\n",
            "Epoch 438/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0675 - acc: 0.9811 - val_loss: 1.8679 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00438: val_acc did not improve from 0.72115\n",
            "Epoch 439/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0672 - acc: 0.9811 - val_loss: 1.8718 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00439: val_acc did not improve from 0.72115\n",
            "Epoch 440/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0670 - acc: 0.9811 - val_loss: 1.8756 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00440: val_acc did not improve from 0.72115\n",
            "Epoch 441/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0668 - acc: 0.9811 - val_loss: 1.8796 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00441: val_acc did not improve from 0.72115\n",
            "Epoch 442/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0666 - acc: 0.9811 - val_loss: 1.8838 - val_acc: 0.5192\n",
            "\n",
            "Epoch 00442: val_acc did not improve from 0.72115\n",
            "Epoch 443/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0663 - acc: 0.9811 - val_loss: 1.8883 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00443: val_acc did not improve from 0.72115\n",
            "Epoch 444/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0661 - acc: 0.9811 - val_loss: 1.8930 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00444: val_acc did not improve from 0.72115\n",
            "Epoch 445/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0659 - acc: 0.9811 - val_loss: 1.8976 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00445: val_acc did not improve from 0.72115\n",
            "Epoch 446/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0657 - acc: 0.9811 - val_loss: 1.9020 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00446: val_acc did not improve from 0.72115\n",
            "Epoch 447/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0655 - acc: 0.9811 - val_loss: 1.9061 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00447: val_acc did not improve from 0.72115\n",
            "Epoch 448/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0653 - acc: 0.9811 - val_loss: 1.9097 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00448: val_acc did not improve from 0.72115\n",
            "Epoch 449/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0651 - acc: 0.9811 - val_loss: 1.9131 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00449: val_acc did not improve from 0.72115\n",
            "Epoch 450/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0649 - acc: 0.9811 - val_loss: 1.9162 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00450: val_acc did not improve from 0.72115\n",
            "Epoch 451/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0647 - acc: 0.9811 - val_loss: 1.9192 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00451: val_acc did not improve from 0.72115\n",
            "Epoch 452/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0645 - acc: 0.9811 - val_loss: 1.9223 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00452: val_acc did not improve from 0.72115\n",
            "Epoch 453/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0643 - acc: 0.9811 - val_loss: 1.9253 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00453: val_acc did not improve from 0.72115\n",
            "Epoch 454/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0641 - acc: 0.9811 - val_loss: 1.9284 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00454: val_acc did not improve from 0.72115\n",
            "Epoch 455/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0639 - acc: 0.9811 - val_loss: 1.9314 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00455: val_acc did not improve from 0.72115\n",
            "Epoch 456/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0637 - acc: 0.9811 - val_loss: 1.9344 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00456: val_acc did not improve from 0.72115\n",
            "Epoch 457/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0635 - acc: 0.9811 - val_loss: 1.9374 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00457: val_acc did not improve from 0.72115\n",
            "Epoch 458/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0633 - acc: 0.9811 - val_loss: 1.9405 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00458: val_acc did not improve from 0.72115\n",
            "Epoch 459/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0631 - acc: 0.9811 - val_loss: 1.9436 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00459: val_acc did not improve from 0.72115\n",
            "Epoch 460/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0629 - acc: 0.9811 - val_loss: 1.9468 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00460: val_acc did not improve from 0.72115\n",
            "Epoch 461/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0627 - acc: 0.9811 - val_loss: 1.9501 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00461: val_acc did not improve from 0.72115\n",
            "Epoch 462/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0626 - acc: 0.9811 - val_loss: 1.9534 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00462: val_acc did not improve from 0.72115\n",
            "Epoch 463/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0624 - acc: 0.9811 - val_loss: 1.9567 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00463: val_acc did not improve from 0.72115\n",
            "Epoch 464/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0622 - acc: 0.9811 - val_loss: 1.9601 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00464: val_acc did not improve from 0.72115\n",
            "Epoch 465/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0620 - acc: 0.9811 - val_loss: 1.9633 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00465: val_acc did not improve from 0.72115\n",
            "Epoch 466/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0618 - acc: 0.9811 - val_loss: 1.9665 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00466: val_acc did not improve from 0.72115\n",
            "Epoch 467/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0617 - acc: 0.9811 - val_loss: 1.9696 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00467: val_acc did not improve from 0.72115\n",
            "Epoch 468/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0615 - acc: 0.9811 - val_loss: 1.9727 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00468: val_acc did not improve from 0.72115\n",
            "Epoch 469/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0613 - acc: 0.9906 - val_loss: 1.9757 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00469: val_acc did not improve from 0.72115\n",
            "Epoch 470/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0611 - acc: 0.9906 - val_loss: 1.9786 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00470: val_acc did not improve from 0.72115\n",
            "Epoch 471/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0609 - acc: 0.9906 - val_loss: 1.9815 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00471: val_acc did not improve from 0.72115\n",
            "Epoch 472/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0608 - acc: 0.9906 - val_loss: 1.9843 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00472: val_acc did not improve from 0.72115\n",
            "Epoch 473/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0606 - acc: 0.9906 - val_loss: 1.9870 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00473: val_acc did not improve from 0.72115\n",
            "Epoch 474/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0604 - acc: 0.9906 - val_loss: 1.9897 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00474: val_acc did not improve from 0.72115\n",
            "Epoch 475/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0602 - acc: 0.9906 - val_loss: 1.9922 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00475: val_acc did not improve from 0.72115\n",
            "Epoch 476/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0600 - acc: 0.9906 - val_loss: 1.9947 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00476: val_acc did not improve from 0.72115\n",
            "Epoch 477/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0599 - acc: 0.9906 - val_loss: 1.9971 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00477: val_acc did not improve from 0.72115\n",
            "Epoch 478/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0597 - acc: 0.9906 - val_loss: 1.9995 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00478: val_acc did not improve from 0.72115\n",
            "Epoch 479/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0595 - acc: 0.9906 - val_loss: 2.0019 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00479: val_acc did not improve from 0.72115\n",
            "Epoch 480/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0594 - acc: 0.9906 - val_loss: 2.0044 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00480: val_acc did not improve from 0.72115\n",
            "Epoch 481/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0592 - acc: 0.9906 - val_loss: 2.0070 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00481: val_acc did not improve from 0.72115\n",
            "Epoch 482/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0590 - acc: 0.9906 - val_loss: 2.0098 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00482: val_acc did not improve from 0.72115\n",
            "Epoch 483/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0589 - acc: 0.9906 - val_loss: 2.0129 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00483: val_acc did not improve from 0.72115\n",
            "Epoch 484/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0587 - acc: 0.9906 - val_loss: 2.0161 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00484: val_acc did not improve from 0.72115\n",
            "Epoch 485/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0586 - acc: 0.9906 - val_loss: 2.0195 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00485: val_acc did not improve from 0.72115\n",
            "Epoch 486/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0584 - acc: 0.9906 - val_loss: 2.0229 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00486: val_acc did not improve from 0.72115\n",
            "Epoch 487/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0583 - acc: 0.9906 - val_loss: 2.0263 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00487: val_acc did not improve from 0.72115\n",
            "Epoch 488/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0581 - acc: 0.9906 - val_loss: 2.0297 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00488: val_acc did not improve from 0.72115\n",
            "Epoch 489/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0580 - acc: 0.9906 - val_loss: 2.0330 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00489: val_acc did not improve from 0.72115\n",
            "Epoch 490/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0578 - acc: 0.9906 - val_loss: 2.0361 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00490: val_acc did not improve from 0.72115\n",
            "Epoch 491/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0577 - acc: 0.9906 - val_loss: 2.0391 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00491: val_acc did not improve from 0.72115\n",
            "Epoch 492/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0575 - acc: 0.9906 - val_loss: 2.0421 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00492: val_acc did not improve from 0.72115\n",
            "Epoch 493/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0574 - acc: 0.9906 - val_loss: 2.0450 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00493: val_acc did not improve from 0.72115\n",
            "Epoch 494/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0572 - acc: 0.9906 - val_loss: 2.0479 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00494: val_acc did not improve from 0.72115\n",
            "Epoch 495/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0571 - acc: 0.9906 - val_loss: 2.0507 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00495: val_acc did not improve from 0.72115\n",
            "Epoch 496/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0569 - acc: 0.9906 - val_loss: 2.0535 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00496: val_acc did not improve from 0.72115\n",
            "Epoch 497/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0568 - acc: 0.9906 - val_loss: 2.0563 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00497: val_acc did not improve from 0.72115\n",
            "Epoch 498/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0567 - acc: 0.9906 - val_loss: 2.0590 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00498: val_acc did not improve from 0.72115\n",
            "Epoch 499/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0565 - acc: 0.9906 - val_loss: 2.0617 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00499: val_acc did not improve from 0.72115\n",
            "Epoch 500/500\n",
            "106/106 [==============================] - 0s 2ms/step - loss: 0.0564 - acc: 0.9906 - val_loss: 2.0645 - val_acc: 0.5096\n",
            "\n",
            "Epoch 00500: val_acc did not improve from 0.72115\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f7122cd7ac8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 88
        }
      ]
    },
    {
      "metadata": {
        "id": "vS9WnnqgkWrf",
        "colab_type": "code",
        "outputId": "48a512f6-b4d0-425d-f108-7a8da39e319a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "#loading the model and checking accuracy on the test data\n",
        "model = load_model('best_model.pkl')\n",
        "\n",
        "from sklearn.metrics import accuracy_score\n",
        "test_preds = model.predict_classes(test)\n",
        "accuracy_score(test_target, test_preds)"
      ],
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.7115384615384616"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 87
        }
      ]
    },
    {
      "metadata": {
        "id": "ljN-ohqQP6N_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "I got an accuracy score of **0.7115384615384616**. It’s quite a promising start but we can definitely improve the performance of the LSTM model by playing around with the hyperparameters, changing the **learning rate**, and/or the **number of epochs** as well.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "Vpd6Z2IyRFMd",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}