understanding_and_advanced_usage_of_recurrent_neural_networks.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "understanding and advanced usage of recurrent neural networks.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyMi//mxQ1R8TxYxlhsqUdiD",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/JSY0717/Deep-Learning-with-Python/blob/main/understanding_and_advanced_usage_of_recurrent_neural_networks.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vvYoHlTIA0HM"
      },
      "source": [
        "# Recurrent neural networks (순환신경망)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N3Uz8rIBFDY9"
      },
      "source": [
        "# 1) Simple RNN의 기본 구조"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AHZaBPQ4B-G7",
        "outputId": "a6a6157a-5900-4503-8bb4-3e20658c3eab"
      },
      "source": [
        "import numpy as np\r\n",
        "\r\n",
        "timesteps = 3\r\n",
        "input_features = 1\r\n",
        "output_features = 1\r\n",
        "i = 0\r\n",
        "j = 0\r\n",
        "\r\n",
        "# 3x1 크기의 난수 생성\r\n",
        "inputs = np.random.random((timesteps, input_features))\r\n",
        "print(inputs)\r\n",
        "inputs.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.23598762]\n",
            " [0.19008022]\n",
            " [0.02493997]]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(3, 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KI-wN9PlCaBa",
        "outputId": "f23682cf-888a-4aac-e3d1-6422c8c8bb2f"
      },
      "source": [
        "state_t = np.zeros((output_features))\r\n",
        "print(state_t)\r\n",
        "state_t.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0.]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1,)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Iu0ahIGrCkM-",
        "outputId": "cd9b886c-e401-4164-b649-5d586e89eac9"
      },
      "source": [
        "# RNN의 가중치값 설정\r\n",
        "W = np.random.random((output_features,input_features))\r\n",
        "U = np.random.random((output_features,output_features))\r\n",
        "\r\n",
        "# 바이어스 값 생성\r\n",
        "b = np.random.random((output_features))\r\n",
        "\r\n",
        "print(W)\r\n",
        "print(U)\r\n",
        "print(b)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.12388372]]\n",
            "[[0.8357658]]\n",
            "[0.93110354]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Rq8H9C0sC6oN",
        "outputId": "e2270aa8-4469-4256-d1d4-7f86b5e35875"
      },
      "source": [
        "\r\n",
        "successive_outputs = []\r\n",
        "for input_t in inputs:\r\n",
        "  print('input_t : ',input_t)\r\n",
        "  output_t = np.tanh(np.dot(W,input_t) + np.dot(U,state_t) + b)\r\n",
        "  print('output_t : ')\r\n",
        "  print(output_t)\r\n",
        "  print('----------')\r\n",
        "  successive_outputs.append(output_t)\r\n",
        "  state_t = output_t\r\n",
        "\r\n",
        "final_output_sequence = np.stack(successive_outputs, axis=0)\r\n",
        "print(final_output_sequence)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "input_t :  [0.23598762]\n",
            "output_t : \n",
            "[0.94049041]\n",
            "----------\n",
            "input_t :  [0.19008022]\n",
            "output_t : \n",
            "[0.94030558]\n",
            "----------\n",
            "input_t :  [0.02493997]\n",
            "output_t : \n",
            "[0.93787127]\n",
            "----------\n",
            "[[0.94049041]\n",
            " [0.94030558]\n",
            " [0.93787127]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sl9nBpv7Bp7A"
      },
      "source": [
        "## 2) kera의 순환층 사용"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "3Tf-DxTiAu5d",
        "outputId": "e991331f-c50b-4dc0-d939-82fc736993a2"
      },
      "source": [
        "import keras\r\n",
        "keras.__version__"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.4.3'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DLaZIBnkBAzK"
      },
      "source": [
        "from keras.layers import SimpleRNN"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YC3AoU3KEO_y",
        "outputId": "225a4a04-ea50-4af9-c0db-d2c7b6e1d9da"
      },
      "source": [
        "from keras.models import Sequential\r\n",
        "from keras.layers import Embedding, SimpleRNN\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "\r\n",
        "# Embedding(input_dim,output_dim)\r\n",
        "# output_dim = size of the voca, output_dim = dimension of the dense embedding\r\n",
        "model.add(Embedding(10000,32))  # 10000개까지의 어휘를 넣었을 때, 32차원의 벡터로 매핑\r\n",
        "\r\n",
        "model.add(SimpleRNN(32))  #return_sequences=False\r\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "embedding (Embedding)        (None, None, 32)          320000    \n",
            "_________________________________________________________________\n",
            "simple_rnn (SimpleRNN)       (None, 32)                2080      \n",
            "=================================================================\n",
            "Total params: 322,080\n",
            "Trainable params: 322,080\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3-unLvJVH7Gn",
        "outputId": "fbc688af-589d-480a-c784-3412b4be209a"
      },
      "source": [
        "model = Sequential()\r\n",
        "model.add(Embedding(10000, 32))\r\n",
        "model.add(SimpleRNN(32, return_sequences=True))  # 크기가 (batch_size, timesteps, output_features) 인 3D 텐서 반환\r\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
            "_________________________________________________________________\n",
            "simple_rnn_1 (SimpleRNN)     (None, None, 32)          2080      \n",
            "=================================================================\n",
            "Total params: 322,080\n",
            "Trainable params: 322,080\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eTmM3v7VIqCe"
      },
      "source": [
        "네트워크의 표현력 증가를 위해, 여러 개의 SimpleRNN을 쌓는 것이 도움이 되기도 한다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1L3oii0HIRj1",
        "outputId": "d50f7fe3-3f06-471f-ae6a-82e3e5af4ad4"
      },
      "source": [
        "model = Sequential()\r\n",
        "model.add(Embedding(10000, 32))\r\n",
        "model.add(SimpleRNN(32, return_sequences=True))  # 크기가 (batch_size, timesteps, output_features) 인 3D 텐서 반환\r\n",
        "model.add(SimpleRNN(32, return_sequences=True))\r\n",
        "model.add(SimpleRNN(32, return_sequences=True))\r\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "embedding_2 (Embedding)      (None, None, 32)          320000    \n",
            "_________________________________________________________________\n",
            "simple_rnn_2 (SimpleRNN)     (None, None, 32)          2080      \n",
            "_________________________________________________________________\n",
            "simple_rnn_3 (SimpleRNN)     (None, None, 32)          2080      \n",
            "_________________________________________________________________\n",
            "simple_rnn_4 (SimpleRNN)     (None, None, 32)          2080      \n",
            "=================================================================\n",
            "Total params: 326,240\n",
            "Trainable params: 326,240\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Qp9wuVZ9On2C"
      },
      "source": [
        "## 2-1) IMDB 영화 리뷰 분류 문제에 적용"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NNq_faywI0mg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c9c0bc8a-b62e-45da-e0f2-cf9b0267302f"
      },
      "source": [
        "from keras.datasets import imdb\r\n",
        "from keras.preprocessing import sequence\r\n",
        "\r\n",
        "max_features = 10000  # 특성으로 사용할 단어의 수 (가장 빈번한 몇개의 단어를 사용할 것인지)\r\n",
        "maxlen = 500  # 사용할 텍스트의 길이\r\n",
        "batch_size = 32  # 한번에 네트워크에 넘겨주는 데이터의 수 \r\n",
        "\r\n",
        "print('...데이터 로딩...')\r\n",
        "(input_train, ytrain), (input_test, y_test) = imdb.load_data(num_words=max_features)\r\n",
        "print(len(input_train), '훈련 시퀀스')\r\n",
        "print(len(input_test), '테스트 시퀀스')\r\n",
        "\r\n",
        "print('시퀀스 패딩 (sample x time)')\r\n",
        "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)  # 500개보다 적으면 빈자리에 0을 채우거나, 500개가 넘으면 잘라내는 등의 방식으로, 일정한 길이로 맞춘다.\r\n",
        "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\r\n",
        "print('input_train의 크기 : ', input_train.shape)\r\n",
        "print('input_test의 크기 : ', input_test.shape)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "...데이터 로딩...\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "<string>:6: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/datasets/imdb.py:159: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_train, y_train = np.array(xs[:idx]), np.array(labels[:idx])\n",
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/datasets/imdb.py:160: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_test, y_test = np.array(xs[idx:]), np.array(labels[idx:])\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "25000 훈련 시퀀스\n",
            "25000 테스트 시퀀스\n",
            "시퀀스 패딩 (sample x time)\n",
            "input_train의 크기 :  (25000, 500)\n",
            "input_test의 크기 :  (25000, 500)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1fOMODj9QWWi",
        "outputId": "a42a054f-ac3f-4c29-9ad2-7350ecff1178"
      },
      "source": [
        "# Embedding 층과 SimpleRNN 층을 사용해 훈련\r\n",
        "from keras.layers import Dense\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "model.add(Embedding(max_features, 32))\r\n",
        "model.add(SimpleRNN(32))\r\n",
        "model.add(Dense(1, activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\r\n",
        "history = model.fit(input_train, ytrain,\r\n",
        "                    epochs=10,\r\n",
        "                    batch_size=128,\r\n",
        "                    validation_split=0.2)"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "157/157 [==============================] - 24s 146ms/step - loss: 0.6726 - acc: 0.5567 - val_loss: 0.4371 - val_acc: 0.8198\n",
            "Epoch 2/10\n",
            "157/157 [==============================] - 22s 140ms/step - loss: 0.3949 - acc: 0.8421 - val_loss: 0.5073 - val_acc: 0.7518\n",
            "Epoch 3/10\n",
            "157/157 [==============================] - 22s 141ms/step - loss: 0.3097 - acc: 0.8766 - val_loss: 0.3730 - val_acc: 0.8568\n",
            "Epoch 4/10\n",
            "157/157 [==============================] - 23s 146ms/step - loss: 0.2119 - acc: 0.9218 - val_loss: 0.3919 - val_acc: 0.8310\n",
            "Epoch 5/10\n",
            "157/157 [==============================] - 22s 141ms/step - loss: 0.1659 - acc: 0.9409 - val_loss: 0.3909 - val_acc: 0.8390\n",
            "Epoch 6/10\n",
            "157/157 [==============================] - 23s 144ms/step - loss: 0.1212 - acc: 0.9574 - val_loss: 0.4336 - val_acc: 0.8464\n",
            "Epoch 7/10\n",
            "157/157 [==============================] - 22s 142ms/step - loss: 0.0852 - acc: 0.9724 - val_loss: 0.5626 - val_acc: 0.8432\n",
            "Epoch 8/10\n",
            "157/157 [==============================] - 22s 143ms/step - loss: 0.0681 - acc: 0.9777 - val_loss: 0.4950 - val_acc: 0.8328\n",
            "Epoch 9/10\n",
            "157/157 [==============================] - 22s 143ms/step - loss: 0.0367 - acc: 0.9896 - val_loss: 0.5003 - val_acc: 0.8506\n",
            "Epoch 10/10\n",
            "157/157 [==============================] - 22s 142ms/step - loss: 0.0236 - acc: 0.9933 - val_loss: 0.6812 - val_acc: 0.7974\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 563
        },
        "id": "Vj7ww9FMRCH_",
        "outputId": "434b3404-7675-4253-83b5-9cef04c90bd6"
      },
      "source": [
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "acc = history.history['acc']\r\n",
        "val_acc = history.history['val_acc']\r\n",
        "loss = history.history['loss']\r\n",
        "val_loss = history.history['val_loss']\r\n",
        "\r\n",
        "epochs = range(1, len(acc) + 1)\r\n",
        "\r\n",
        "plt.plot(epochs, acc, 'bo', label='Training acc')\r\n",
        "plt.plot(epochs, val_acc, 'b', label='Validation acc')\r\n",
        "plt.title('Training and validation accuracy')\r\n",
        "plt.legend()\r\n",
        "\r\n",
        "plt.figure()\r\n",
        "\r\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\r\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\r\n",
        "plt.title('Training and validation loss')\r\n",
        "plt.legend()"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f254981a5c0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5b3H8c+PqGAAEQSXEiDQgopFtshmrbiDUiguLTG1UHpFXKuvWxWrLVwsVlraeu3VtrRVrKJovS3FhVo3rrYuJVhcQHZBFhcE2Xf43T+ekzAJWSYwmZmcfN+v17xm5qy/OZP85jnPec7zmLsjIiLx1SDTAYiISO1SohcRiTklehGRmFOiFxGJOSV6EZGYU6IXEYk5Jfp6yMxmmtnwVC+bSWa23MzOrYXtupl9KXr9GzP7YTLLHsR+iszs7wcbp0hVTO3o6wYz25LwNhfYCeyN3l/l7lPTH1X2MLPlwH+4+wsp3q4DHd19SaqWNbN84APgcHffk4o4RapyWKYDkOS4e5OS11UlNTM7TMlDsoX+HrODqm7qODPrb2arzOxWM/sYeNDMmpvZ02a21sw+j17nJawzy8z+I3o9wsz+YWaTomU/MLOBB7lsezN7xcw2m9kLZnafmT1SSdzJxHinmf0z2t7fzaxlwvwrzGyFma0zs9urOD69zexjM8tJmDbUzN6JXvcys9fNbIOZfWRm/2NmR1SyrSlm9uOE9zdH66wxs5Hllr3IzP5tZpvMbKWZjUuY/Ur0vMHMtphZ35Jjm7B+PzObbWYbo+d+yR6bGh7nFmb2YPQZPjez6QnzhpjZ3OgzLDWzAdH0MtVkZjau5Hs2s/yoCuu7ZvYh8FI0/U/R97Ax+hs5JWH9I83s59H3uTH6GzvSzJ4xs+vLfZ53zGxoRZ9VKqdEHw/HAy2AdsAowvf6YPS+LbAd+J8q1u8NLARaAj8F/mBmdhDLPgr8CzgGGAdcUcU+k4nxcuA7wLHAEcD3AcysM/DraPtfiPaXRwXc/U1gK3B2ue0+Gr3eC9wUfZ6+wDnANVXETRTDgCie84COQPnrA1uBbwNHAxcBV5vZ16N5X42ej3b3Ju7+erlttwCeAe6NPtsvgGfM7Jhyn+GAY1OB6o7zw4SqwFOibf0yiqEX8Efg5ugzfBVYXtnxqMCZwMnABdH7mYTjdCzwFpBY1TgJ6An0I/wd3wLsAx4CvlWykJl1BVoTjo3UhLvrUccehH+4c6PX/YFdQKMqlu8GfJ7wfhah6gdgBLAkYV4u4MDxNVmWkET2ALkJ8x8BHknyM1UU4x0J768B/ha9/hEwLWFe4+gYnFvJtn8MPBC9bkpIwu0qWfZG4C8J7x34UvR6CvDj6PUDwN0Jy3VKXLaC7d4D/DJ6nR8te1jC/BHAP6LXVwD/Krf+68CI6o5NTY4zcAIhoTavYLnflsRb1d9f9H5cyfec8Nk6VBHD0dEyzQg/RNuBrhUs1wj4nHDdA8IPwv3p/n+Lw0Ml+nhY6+47St6YWa6Z/TY6Fd5EqCo4OrH6opyPS164+7boZZMaLvsFYH3CNICVlQWcZIwfJ7zelhDTFxK37e5bgXWV7YtQer/YzBoCFwNvufuKKI5OUXXGx1EcdxFK99UpEwOwotzn621mL0dVJhuB0Ulut2TbK8pNW0EozZao7NiUUc1xbkP4zj6vYNU2wNIk461I6bExsxwzuzuq/tnE/jODltGjUUX7iv6mHwe+ZWYNgELCGYjUkBJ9PJRvOvWfwIlAb3c/iv1VBZVVx6TCR0ALM8tNmNamiuUPJcaPErcd7fOYyhZ29/mERDmQstU2EKqAFhBKjUcBPziYGAhnNIkeBWYAbdy9GfCbhO1W19RtDaGqJVFbYHUScZVX1XFeSfjOjq5gvZXAFyvZ5lbC2VyJ4ytYJvEzXg4MIVRvNSOU+kti+AzYUcW+HgKKCFVq27xcNZckR4k+npoSToc3RPW9Y2t7h1EJuRgYZ2ZHmFlf4Gu1FOOTwCAz+0p04XQ81f8tPwp8j5Do/lQujk3AFjM7Cbg6yRieAEaYWefoh6Z8/E0JpeUdUX335Qnz1hKqTDpUsu1ngU5mdrmZHWZm3wQ6A08nGVv5OCo8zu7+EaHu/P7oou3hZlbyQ/AH4Dtmdo6ZNTCz1tHxAZgLDIuWLwAuTSKGnYSzrlzCWVNJDPsI1WC/MLMvRKX/vtHZF1Fi3wf8HJXmD5oSfTzdAxxJKC29AfwtTfstIlzQXEeoF3+c8A9ekYOO0d3nAdcSkvdHhHrcVdWs9hjhAuFL7v5ZwvTvE5LwZuB3UczJxDAz+gwvAUui50TXAOPNbDPhmsITCetuAyYA/7TQ2qdPuW2vAwYRSuPrCBcnB5WLO1nVHecrgN2Es5pPCdcocPd/ES72/hLYCPwf+88yfkgogX8O/Bdlz5Aq8kfCGdVqYH4UR6LvA+8Cs4H1wETK5qY/Al0I13zkIOiGKak1ZvY4sMDda/2MQuLLzL4NjHL3r2Q6lrpKJXpJGTM7zcy+GJ3qDyDUy06vbj2RykTVYtcAkzMdS12mRC+pdDyh6d8WQhvwq9393xmNSOosM7uAcD3jE6qvHpIqqOpGRCTmVKIXEYm5rOvUrGXLlp6fn5/pMERE6pQ5c+Z85u6tKpqXdYk+Pz+f4uLiTIchIlKnmFn5u6lLqepGRCTmqk30ZvaAmX1qZu9VMt/M7F4zWxJ1IdojYd5wM1scPbJ+lCIRkThKpkQ/BRhQxfyBhO5HOxK6yP01lHa1OpbQrW0vYKyZNT+UYEVEpOaqraN391csDH1WmSHAHz2003zDzI42sxMI3ec+7+7rAczsecIPxmM1DXL37t2sWrWKHTt2VL+wZESjRo3Iy8vj8MMPz3QoIlJOKi7GtqZsd62rommVTT+AmY0inA3Qtm35TgBh1apVNG3alPz8fCofD0Myxd1Zt24dq1aton379pkOR0TKyYqLse4+2d0L3L2gVasDWwft2LGDY445Rkk+S5kZxxxzjM64RA7S1KmQnw8NGoTnqVOrW6NmUpHoV1O2X+68aFpl0w+Kknx20/cjcnCmToVRo2DFCnAPz6NGpTbZpyLRzwC+HbW+6QNsjPq5fg44P+rnujlwfjRNREQit98O27aVnbZtW5ieKsk0r3yMMF7liWa2ysLo7qPNbHS0yLPAMkKf3L8jGlg5ugh7J6GP6dnA+JILs3XNunXr6NatG926deP444+ndevWpe937dpV5brFxcXccMMN1e6jX79+qQpXRJJU21Umyfjww5pNPyiZHrS2/KNnz55e3vz58w+YVpVHHnFv187dLDw/8kiNVq/S2LFj/Wc/+1mZabt3707dDuqwmn5PIpn0yCPuubnuocIkPHJzU5svktGuXdkYSh7t2tVsO0Cx15fBwdNR3wUwYsQIRo8eTe/evbnlllv417/+Rd++fenevTv9+vVj4cKFAMyaNYtBgwYBMG7cOEaOHEn//v3p0KED9957b+n2mjRpUrp8//79ufTSSznppJMoKirCox5Gn332WU466SR69uzJDTfcULrdRMuXL+eMM86gR48e9OjRg9dee6103sSJE+nSpQtdu3ZlzJgxACxZsoRzzz2Xrl270qNHD5YuPZTxoEXqjnRUmSRjwgTIzS07LTc3TE+Zyn4BMvU41BJ9qn4dK1NSoh8+fLhfdNFFvmfPHnd337hxY2nJ/vnnn/eLL77Y3d1ffvllv+iii0rX7du3r+/YscPXrl3rLVq08F27drm7e+PGjUuXP+qoo3zlypW+d+9e79Onj7/66qu+fft2z8vL82XLlrm7+7Bhw0q3m2jr1q2+fft2d3dftGiRlxzPZ5991vv27etbt251d/d169a5u3uvXr38z3/+s7u7b9++vXT+wVCJXuoSs4pzhVn6Y0lFLQRVlOizrlOzQ5WW+q7IZZddRk5ODgAbN25k+PDhLF68GDNj9+7dFa5z0UUX0bBhQxo2bMixxx7LJ598Ql5eXpllevXqVTqtW7duLF++nCZNmtChQ4fSduqFhYVMnnzgoDu7d+/muuuuY+7cueTk5LBo0SIAXnjhBb7zne+QGxUdWrRowebNm1m9ejVDhw4Fwk1PIvVF27bhjL+i6elWVBQetSV2VTeVfUm18eU1bty49PUPf/hDzjrrLN577z2eeuqpStuUN2zYsPR1Tk4Oe/bsOahlKvPLX/6S4447jrfffpvi4uJqLxaLZEI2XARNS5VJlohdos/Ul7dx40Zatw43/k6ZMiXl2z/xxBNZtmwZy5cvB+Dxxx+vNI4TTjiBBg0a8PDDD7N3714AzjvvPB588EG2RZWS69evp2nTpuTl5TF9ehjWdefOnaXzRWpLuq6jVaeoCCZPhnbtwCw8T55cuyXrTIldos/Ul3fLLbdw22230b179xqVwJN15JFHcv/99zNgwAB69uxJ06ZNadas2QHLXXPNNTz00EN07dqVBQsWlJ51DBgwgMGDB1NQUEC3bt2YNGkSAA8//DD33nsvp556Kv369ePjjz9OeewiibLlIiiEvLB8OezbF57jmOQhC8eMLSgo8PIDj7z//vucfPLJGYooe2zZsoUmTZrg7lx77bV07NiRm266KdNhldL3JMlo0CCU5MszCwlXDo6ZzXH3gormxa5EH2e/+93v6NatG6eccgobN27kqquuynRIUgdlun48ndfRJIhdq5s4u+mmm7KqBC91T0n9eEnVSUn9OKSv2mLChLIxQHwvgmYLlehF6pFsqB+vTxdBs4VK9CL1SDrvM6lKbbcbl7JUohepR1Q/Xj8p0YvUI/XpJiHZT4k+CWeddRbPPVe2K/177rmHq6++utJ1+vfvT0kz0QsvvJANGzYcsMy4ceNK27NXZvr06cyfP7/0/Y9+9CNeeOGFmoQvUkr14/WTEn0SCgsLmTZtWplp06ZNo7CwMKn1n332WY4++uiD2nf5RD9+/HjOPffcg9qWCNSfm4RkPyX6JFx66aU888wzpf3GLF++nDVr1nDGGWdw9dVXU1BQwCmnnMLYsWMrXD8/P5/PPvsMgAkTJtCpUye+8pWvlHZlDKGN/GmnnUbXrl255JJL2LZtG6+99hozZszg5ptvplu3bixdupQRI0bw5JNPAvDiiy/SvXt3unTpwsiRI9m5c2fp/saOHUuPHj3o0qULCxYsOCAmdWecfpluvy71V51rdXPjjTB3bmq32a0b3HNP5fNbtGhBr169mDlzJkOGDGHatGl84xvfwMyYMGECLVq0YO/evZxzzjm88847nHrqqRVuZ86cOUybNo25c+eyZ88eevToQc+ePQG4+OKLufLKKwG44447+MMf/sD111/P4MGDGTRoEJdeemmZbe3YsYMRI0bw4osv0qlTJ7797W/z61//mhtvvBGAli1b8tZbb3H//fczadIkfv/735dZ/9hjj+X555+nUaNGLF68mMLCQoqLi5k5cyZ//etfefPNN8nNzWX9+jAoWFFREWPGjGHo0KHs2LGDfbqFsUayof261F8q0ScpsfomsdrmiSeeoEePHnTv3p158+aVqWYp79VXX2Xo0KHk5uZy1FFHMXjw4NJ57733HmeccQZdunRh6tSpzJs3r8p4Fi5cSPv27enUqRMAw4cP55VXXimdf/HFFwPQs2fP0o7QEu3evZsrr7ySLl26cNlll5XGnWx3xrnlr+hJlbKh/brUX3WuRF9Vybs2DRkyhJtuuom33nqLbdu20bNnTz744AMmTZrE7Nmzad68OSNGjKi0e+LqjBgxgunTp9O1a1emTJnCrFmzDinekq6OK+vmOLE743379qkv+lqWLe3XpX5KqkRvZgPMbKGZLTGzMRXMb2dmL5rZO2Y2y8zyEubtNbO50WNGKoNPpyZNmnDWWWcxcuTI0tL8pk2baNy4Mc2aNeOTTz5h5syZVW7jq1/9KtOnT2f79u1s3ryZp556qnTe5s2bOeGEE9i9ezdTEypvmzZtyubNmw/Y1oknnsjy5ctZsmQJEHqhPPPMM5P+POrOOL3Ufl0yqdpEb2Y5wH3AQKAzUGhmncstNgn4o7ufCowHfpIwb7u7d4seg6nDCgsLefvtt0sTfdeuXenevTsnnXQSl19+OaeffnqV6/fo0YNvfvObdO3alYEDB3LaaaeVzrvzzjvp3bs3p59+OieddFLp9GHDhvGzn/2M7t27l7kA2qhRIx588EEuu+wyunTpQoMGDRg9enTSn0XdGaeX2q9LJlXbTbGZ9QXGufsF0fvbANz9JwnLzAMGuPtKMzNgo7sfFc3b4u5Nkg1I3RTXXfqeqjZ1aqiT//DDUJKfMEEXYiV1DrWb4tbAyoT3q6Jpid4GLo5eDwWamtkx0ftGZlZsZm+Y2dcrCXBUtEzx2rVrkwhJpO5R+3XJlFS1uvk+cKaZ/Rs4E1gN7I3mtYt+ZS4H7jGzL5Zf2d0nu3uBuxe0atUqRSGJiAgk1+pmNdAm4X1eNK2Uu68hKtGbWRPgEnffEM1bHT0vM7NZQHegxnfbuDuhVkiyUbaNVCYi+yVTop8NdDSz9mZ2BDAMKNN6xsxamlnJtm4DHoimNzezhiXLAKcDlTc0r0SjRo1Yt26dkkmWcnfWrVuXtU00dUeq1HfVlujdfY+ZXQc8B+QAD7j7PDMbDxS7+wygP/ATM3PgFeDaaPWTgd+a2T7Cj8rd7l7jRJ+Xl8eqVatQ/X32atSoEXl5edUvmGa6I1WkjgwOLnKw8vNDci+vXbtwQVQkLjQ4uNRbuiNVRIleYk53pIoo0UvM6Y5UESV6iTmNqCRSB3uvFKmpoiIldqnfVKIXEYk5JXoRkZhTohcRiTklehGRmFOil1qjPmZEsoNa3UitUB8zItlDJXqpFbffvj/Jl9i2LUwXkfRSopdaoT5mRLKHEr3UCvUxI5I9lOilVqiPGZHsoUQvtUJ9zIhkD7W6kVqjPmZEsoNK9CIiMadELyISc0klejMbYGYLzWyJmY2pYH47M3vRzN4xs1lmlpcwb7iZLY4ew1MZvIiIVK/aRG9mOcB9wECgM1BoZp3LLTYJ+KO7nwqMB34SrdsCGAv0BnoBY82seerCFxGR6iRTou8FLHH3Ze6+C5gGDCm3TGfgpej1ywnzLwCed/f17v458Dww4NDDFhGRZCWT6FsDKxPer4qmJXobuDh6PRRoambHJLkuZjbKzIrNrHjt2rXJxi4iIklI1cXY7wNnmtm/gTOB1cDeZFd298nuXuDuBa1atUpRSCIiAsm1o18NtEl4nxdNK+Xua4hK9GbWBLjE3TeY2Wqgf7l1Zx1CvCIiUkPJlOhnAx3NrL2ZHQEMA2YkLmBmLc2sZFu3AQ9Er58Dzjez5tFF2POjaSIikibVJnp33wNcR0jQ7wNPuPs8MxtvZoOjxfoDC81sEXAcMCFadz1wJ+HHYjYwPpomIiJpYu6e6RjKKCgo8OLi4kyHISJSp5jZHHcvqGie7oyNIQ3hJyKJ1KlZzGgIPxEpTyX6mNEQfiJSnhJ9zGgIPxEpT4k+ZjSEn4iUp0QfMxrCT0TKU6KPGQ3hJyLlqdVNDGkIPxFJpBK9iEjMKdGLiMScEr2ISMwp0YuIxJwSvYhIzCnRi4jEnBK9iEjMKdGLiMScEr2ISMwp0cfUpk3wzDOwc2emIxGRTFOij6lrr4VBg0KvlWPHwkcfZToiEcmUpBK9mQ0ws4VmtsTMxlQwv62ZvWxm/zazd8zswmh6vpltN7O50eM3qf4AcqA33oBHHoHLL4feveHOO0PCLyqCN9/MdHQikm7VJnozywHuAwYCnYFCM+tcbrE7gCfcvTswDLg/Yd5Sd+8WPUanKO6slA1jtbrDjTfC8cfDb34DM2bA4sVw/fXw9NPQpw/06hV+CFStI1I/JFOi7wUscfdl7r4LmAYMKbeMA0dFr5sBa1IXYt1QMlbrihUh2ZaM1ZruZP/oo6HUftdd0LRpmPbFL8IvfgGrV8N994X6+yuuCF0YjxsHH3+c3hhFssHy5eH/YeBAuOACWLky0xHVHnP3qhcwuxQY4O7/Eb2/Aujt7tclLHMC8HegOdAYONfd55hZPjAPWARsAu5w91cr2McoYBRA27Zte65YseLQP1ma5eeH5F5eu3bhDyodtm6FE0+E446D2bPDmUVF9u2DF16Ae+8NF2wPPxy+8Q343vfgtNPSE6tIuu3dG6o1n346PN57L0zv2BE++QSaNIGZM+HUUzMb58EysznuXlDRvFT1R18ITHH3n5tZX+BhM/sy8BHQ1t3XmVlPYLqZneLumxJXdvfJwGSAgoKCqn95slQ2jNU6aVIotT/2WOVJHsK8888Pj8WLQ6nmgQfC2UefPnDDDXDJJXDEEemLPa527w4lxRUrwg/+hx+G49+0KRx1VHhU9LppU8jJyXT0dd+GDfDccyGxP/ssrF8Phx0GX/1qOMu96CLo1AnefTeU7M84A/7yFzj77ExHnlrJlOj7AuPc/YLo/W0A7v6ThGXmEUr9K6P3y4A+7v5puW3NAr7v7sWV7a+goMCLiyudnbUyXaJfuTKU5r/2NXj88Zqvv3kzPPQQ/OpXsGgRnHACjB4NV10VzhCkYjt3huS9fHl4lCT0kuc1a8IZ1MHIza38h6CqH4jy0xo3rvqHP07cYeHC/aX2f/wjlORbtoQLLwwt0c4/H5o1O3DdlStDsl+0CKZMCY0Z6pKqSvTJJPrDCFUv5wCrgdnA5e4+L2GZmcDj7j7FzE4GXgRaAy2B9e6+18w6AK8CXdx9fWX7q6uJvqSOftu2/dNyc9M3jN+3vgVPPgkLFoQfnYO1bx/8/e+hWmfmzFCq/+Y3Qym/oMI/oXjbtu3A5J34XL7Zak4O5OWF7yA/P/zQJz7n5YWku3lzuFZS8lzT15s3w8aN4YyhOmahWiIx+R99dCjJfvnL+x8VJb+6YNcueOWV/cl96dIw/dRTQ2IfNCg0QEjmDGnDBvj61+H//g8mToSbbw7Hry44pEQfbeBC4B4gB3jA3SeY2Xig2N1nRK1wfgc0IVyYvcXd/25mlwDjgd3APmCsuz9V1b7qaqKHkOxvvz2U8Nq2DQNypyPJv/EG9O0LP/hBagcBX7QI/ud/4MEHYcuWsI+Sap3DD0/dfjJp8+bKk/jy5bB2bdnlDz88fLcVJfF27aB161A1kC47d9b8B2LTplCFsWBBeF+iTZuQ8Lt02Z/8Tz4ZGjVK3+dJ1iefhKqYZ54JBZPNm0Oc55wTEvuFF4bv6WDs3AnDh4cz4+uug3vuqRvVaIec6NOpLif6TNi3D/r1C4lp8eJQcku1TZvCqeyvfgVLlsAXvgBXXx2qdVq1Sv3+UmnLFvjgA1i2bH/1SmIi//zzsss3bFh5Es/PD81W68I/fTLcQ6HkvfdCHfV774XH+++HUjKEs4+OHQ/8AfjSl9J7HNxh7tz9pfZ//StMb916f6n97LPDWXQq7NsHt9wCP/85DB0aCnFHHpmabdcWJfoYmzo1VNs8+CCMGFG7+9q3D/72t1Ct89xzISkWFoY2+j161O6+q4ppzZqQyMs/li6FTz8tu3zjxpUn8Xbt4Nhj6099dmV27w4/6OV/AJYsCQkXQun55JMP/AHIy0tdVce2bfDiiyGxP/NMaGhgFqphSpJ71661W7Xy3/8NN90UzmZnzIBjjqm9fR0qJfqYKmlOefzxoYSTzgS1YEGo1pkyJcRx+umhWmfo0NRX6ySWyssn8uXLy9741aBBOGXv0OHAR35+uChXV+pcs822baG0X/4HYPXq/cs0a3Zg8u/SBVq0SG4fH34YkvrTT8NLL8GOHeEs9YILQmIfODD9jQOefDIUpvLzw3Wr9u3Tu/9kKdHH1Lhx8F//Ba++Cl/5SmZi2LgxnE386lch+bZuDddcA1demXy1TmWl8qVLw3P5UvlRR4WbwBKTeMn7tm3jc/2grli/HubNK5v83303XNgsccIJB/4AdO4czgzefHN/lcy774blv/jF0IJs0KDQ5DHTTX1ffRWGDAlxPPts5s5gq6JEH0OH2pwy1fbuDaWde++F558P1TqXXx5K+d26VVwqL0nk1ZXKyyf15s1VKs927uHHu3zpf968UEqH8B02bhz+NnJyQkIvqZLp1Cn7vuP334cBA8IP25NPhrOMbKJEH0NFRfC//xvaDLdrl+loypo/P1TrPPRQON1v0SL8cyRq1uzABF7yUKk8vvbuDT/uJT8AH38MZ54ZkubRR2c6uuqtWRNa9MybB7/7Xe1fF6sJJfqYef310NLm9tvhxz/OdDSV27AhVOssWBDqNVUqlzjYtAkuvTScud55Z/g/zIa/ZSX6GNm3L7QAWLkytHOvjeaUIlK1XbvCdag//jHcKHnffem9f6Ii6ejrRtLk0UdDC5spU5TkRTLliCPC/2BeXugpds0amDYtXHPIRvW8xXDdsnUrjBkTuiK44opMRyNSv5mFO9F//evQEufssw9sIZYtlOjrkJ/+NLRZvuce3dQjki1Gjw49Xr77brh2tmRJpiM6UGzSxc6doV+Kkg6N4ubDD0Oi/+Y3w81JIpI9Bg8ON3ht2BCSfUkXDdkiNom+pB/2gQPhs88yHU3qjYlG6p04MbNxiEjF+vSB114L18769w83gGWL2CT6Dh3gqadCa5Svfa1sd8F13WuvhR+x738/+9rMi8h+nTqF5s+nnBLupJ08OdMRBbFJ9BBOmaZODbdUFxWFmzPqun37QqdKJ5wAt96a6WhEpDrHHQcvvxzuor3qKvjRj/Z3BpcpsUr0ABdfHHqcmz49jIGa6QN8qKZODfV9d9+t5pQidUWTJvDXv8J3vxtuqho5MrlBYmpLLNvRX399uHg5aVKo6rj55kxHdHBKmlOedlroPU9E6o7DDgvdJLRpEzog/Ogj+NOfwihfaY8l/btMj4kTQ339LbeEmxoKCzMdUc399KfhRow//UnNKUXqIjMYOzYk+1GjwkXaZ54JXYunU2zTR4MGoVOtM88Mw4K9/HKmI6qZkuaUw4aFaw8iUneNHBkaiyxcGLowWbgwvfuPbaKH0FXuX/4ShkIbOjT0mFdXqDmlSLwMHAizZoUWgf36wT//mb59J5XozWyAmS00syVmNqaC+W3N7GUz+7eZvaPP6GUAAA2SSURBVBMNJl4y77ZovYVmlvYenJs3D/2k5+aGA504Gk62KmlOefPNBz/AsYhkn4KC0PzymGPg3HNDQTQdqk30ZpYD3AcMBDoDhWbWudxidwBPuHt3YBhwf7Ru5+j9KcAA4P5oe2nVtm3oi2LjxtCX9MaN6Y4gefv2wY03hgG4b7kl09GISKp16BAKc927wyWXhLEbalsyJfpewBJ3X+buu4BpwJByyzhwVPS6GbAmej0EmObuO939A2BJtL2069YtDNQxf344uCWj3GebqVNh9mw1pxSJs5Yt4YUXwk1V118f7pHZt6/29pdMom8NrEx4vyqalmgc8C0zWwU8C1xfg3Uxs1FmVmxmxWvXrk0y9Jo77zz4wx/CyPLf/W72tbHfsmV/c8qiokxHIyK1KTc3DEl4zTWh4cUVV9ReATRVzSsLgSnu/nMz6ws8bGZfTnZld58MTIYw8EiKYqrQt78dml3ecUeo0pkwoTb3VjNqTilSv+TkhKqbNm3gtttCN8d/+1uYnkrJJPrVQJuE93nRtETfJdTB4+6vm1kjoGWS66bdD34Qmi/edVc4wKNHZzoiWLECfvaz0N5fzSlF6g+zcCaflxfGVk51kofkEv1soKOZtSck6WHA5eWW+RA4B5hiZicDjYC1wAzgUTP7BfAFoCOQ8Q48zcLQX6tXw7XXhgufgwdnNqYxY0Jcd9+d2ThEJDNq8+73aisI3H0PcB3wHPA+oXXNPDMbb2Yl6fE/gSvN7G3gMWCEB/OAJ4D5wN+Aa909K7oaO+wwePxx6NEj3JT05puZi+W118IwZGpOKSK1od4PDv7pp+FOtU2bQvvWL30pbbsGwpX23r1D3fyiRdk75qSIZLeqBgev95f8jj02XPxwD92K1mKjnwo98ggUF4cqGyV5EakN9T7RQ+gi4emnQ539oEHpG7Rky5Zwpb1XLzWnFJHao0Qf6dMn1JMXF4c6+z17an+fEyeGKhsN9i0itUnpJcGQIXDvvaGXuRtuqN0bqlasCP3lX355uEYgIlJbYtsf/cG69tpwQ9XEiaEFzJgDunBLjVtvVXNKEUkPJfoK3HVXSPa33RZuYkh1+9Z//jM07fzRj8INWyIitUmJvgINGsADD4Shv0aODANzn3NOarZd0jtl69bqnVJE0kN19JVo2BD+/Gc48cQw4Pg776Rmuw8/rOaUIpJeSvRVOPro0I9906ahH/uVK6tfpyolzSl79w4XYUVE0kGJvhpt2oQRqjZvDiNUbdhw8NuaODFUB6k5pYikk9JNErp0CUN+LVoUxp7dubPm20hsTtmnT+pjFBGpjBJ9ks4+Gx58MAzu+53v1Hw0GDWnFJFMUaubGigq2t/ssm3b5JP2P/4RmlOOHavmlCKSfkr0NXTrrWHQkokTQ9K+9tqqly9pTpmXp+aUIpIZSvQ1ZAa/+lXoAO3660N7+K9/vfLlH34Y5swJvVTm5qYvThGREqqjPwg5OfDYY6HXycLC0I99RRKbUxYWpjdGEZESSvQHKTc3dH6Wlwdf+1pokVPe3XerOaWIZJ7SzyFo1Sq0sTcLbew//XT/vOXLQ3PKoiI1pxSRzFKiP0Rf+lIYtOSjj8KgJVu3hum33hpK8WpOKSKZllSiN7MBZrbQzJaY2QEd95rZL81sbvRYZGYbEubtTZg3I5XBZ4vevUPzyTlzwqAls2bBE0+EZJ+Xl+noRKS+q3ZwcDPLARYB5wGrgNlAobvPr2T564Hu7j4yer/F3ZskG1C6BwdPpd/+FkaPhkaNoGVLWLhQLW1EJD0OdXDwXsASd1/m7ruAacCQKpYvBB6reZh131VXwQ9+ADt2wE9/qiQvItkhmUTfGkjst3FVNO0AZtYOaA+8lDC5kZkVm9kbZlZFi/N4+PGPYelSNacUkeyR6humhgFPuvvehGnt3H21mXUAXjKzd919aeJKZjYKGAXQtm3bFIeUXmbQoUOmoxAR2S+ZEv1qILGHlrxoWkWGUa7axt1XR8/LgFlA9/Iruftkdy9w94JWrVolEZKIiCQrmUQ/G+hoZu3N7AhCMj+g9YyZnQQ0B15PmNbczBpGr1sCpwMVXsQVEZHaUW3VjbvvMbPrgOeAHOABd59nZuOBYncvSfrDgGlethnPycBvzWwf4Ufl7spa64iISO2otnllutXl5pUiIplyqM0rRUSkDlOiFxGJOSV6EZGYU6IXEYk5JXoRkZhTohcRiTklehGRmFOiFxGJOSV6EZGYU6IXEYk5JXoRkZhTohcRiTklehGRmFOiFxGJOSV6EZGYU6IXEYk5JXoRkZhTohcRiTklehGRmFOiFxGJuaQSvZkNMLOFZrbEzMZUMP+XZjY3eiwysw0J84ab2eLoMTyVwYuISPUOq24BM8sB7gPOA1YBs81shrvPL1nG3W9KWP56oHv0ugUwFigAHJgTrft5Sj+FiIhUKpkSfS9gibsvc/ddwDRgSBXLFwKPRa8vAJ539/VRcn8eGHAoAYuISM0kk+hbAysT3q+Kph3AzNoB7YGXarKumY0ys2IzK167dm0ycYuISJJSfTF2GPCku++tyUruPtndC9y9oFWrVikOSUSkfksm0a8G2iS8z4umVWQY+6ttarquiIjUgmQS/Wygo5m1N7MjCMl8RvmFzOwkoDnwesLk54Dzzay5mTUHzo+miYhImlTb6sbd95jZdYQEnQM84O7zzGw8UOzuJUl/GDDN3T1h3fVmdifhxwJgvLuvT+1HEBGRqlhCXs4KBQUFXlxcnOkwRETqFDOb4+4FFc3TnbEiIjGnRC8iEnNK9CIiMadELyISc0r0IiIxp0QvIhJzSvQiIjGnRC8iEnNK9CIiMadELyISc0r0IiIxp0QvIhJzSvQiIjGnRC8iEnNK9CIiMadELyISc0r0IiIxp0QvIhJzSvQiIjGXVKI3swFmttDMlpjZmEqW+YaZzTezeWb2aML0vWY2N3rMqGhdERGpPYdVt4CZ5QD3AecBq4DZZjbD3ecnLNMRuA043d0/N7NjEzax3d27pThuERFJUjIl+l7AEndf5u67gGnAkHLLXAnc5+6fA7j7p6kNU0REDlYyib41sDLh/apoWqJOQCcz+6eZvWFmAxLmNTKz4mj61yvagZmNipYpXrt2bY0+gIiIVK3aqpsabKcj0B/IA14xsy7uvgFo5+6rzawD8JKZvevuSxNXdvfJwGSAgoICT1FMIiJCciX61UCbhPd50bREq4AZ7r7b3T8AFhESP+6+OnpeBswCuh9izBWaOhXy86FBg/A8dWpt7EVEpO5JJtHPBjqaWXszOwIYBpRvPTOdUJrHzFoSqnKWmVlzM2uYMP10YD4pNnUqjBoFK1aAe3geNUrJXkQEkkj07r4HuA54DngfeMLd55nZeDMbHC32HLDOzOYDLwM3u/s64GSg2MzejqbfndhaJ1Vuvx22bSs7bdu2MF1EpL4z9+yqEi8oKPDi4uIardOgQSjJl2cG+/alKDARkSxmZnPcvaCiebG4M7Zt25pNFxGpT2KR6CdMgNzcstNyc8N0EZH6LhaJvqgIJk+Gdu1CdU27duF9UVGmIxMRybxUtaPPuKIiJXYRkYrEokQvIiKVU6IXEYk5JXoRkZhTohcRiTklehGRmMu6O2PNbC2wItNxHKKWwGeZDiKL6HiUpeOxn45FWYdyPNq5e6uKZmRdoo8DMyuu7Fbk+kjHoywdj/10LMqqreOhqhsRkZhTohcRiTkl+toxOdMBZBkdj7J0PPbTsSirVo6H6uhFRGJOJXoRkZhTohcRiTkl+hQyszZm9rKZzTezeWb2vUzHlGlmlmNm/zazpzMdS6aZ2dFm9qSZLTCz982sb6ZjyiQzuyn6P3nPzB4zs0aZjimdzOwBM/vUzN5LmNbCzJ43s8XRc/NU7EuJPrX2AP/p7p2BPsC1ZtY5wzFl2vcIYw0L/DfwN3c/CehKPT4uZtYauAEocPcvAznAsMxGlXZTgAHlpo0BXnT3jsCL0ftDpkSfQu7+kbu/Fb3eTPhHbp3ZqDLHzPKAi4DfZzqWTDOzZsBXgT8AuPsud9+Q2agy7jDgSDM7DMgF1mQ4nrRy91eA9eUmDwEeil4/BHw9FftSoq8lZpYPdAfezGwkGXUPcAugIdqhPbAWeDCqyvq9mTXOdFCZ4u6rgUnAh8BHwEZ3/3tmo8oKx7n7R9Hrj4HjUrFRJfpaYGZNgP8FbnT3TZmOJxPMbBDwqbvPyXQsWeIwoAfwa3fvDmwlRafldVFU9zyE8AP4BaCxmX0rs1FlFw9t31PS/l2JPsXM7HBCkp/q7n/OdDwZdDow2MyWA9OAs83skcyGlFGrgFXuXnKG9yQh8ddX5wIfuPtad98N/Bnol+GYssEnZnYCQPT8aSo2qkSfQmZmhDrY9939F5mOJ5Pc/TZ3z3P3fMJFtpfcvd6W2Nz9Y2ClmZ0YTToHmJ/BkDLtQ6CPmeVG/zfnUI8vTieYAQyPXg8H/pqKjSrRp9bpwBWE0uvc6HFhpoOSrHE9MNXM3gG6AXdlOJ6Mic5sngTeAt4l5KJ61R2CmT0GvA6caGarzOy7wN3AeWa2mHDWc3dK9qUuEERE4k0lehGRmFOiFxGJOSV6EZGYU6IXEYk5JXoRkZhTohcRiTklehGRmPt/2NwHb3Sf8xoAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1f3/8deHRRBwqSxaQQgqgqAIZXGhUq1aQS244IK0glRR1CpWRSwufFH6tUr9ohWxqRapjYLbj6JisS4gbpWAiIKgiIBRqsiOiBD4/P44E5iELJNkMncyeT8fDx4zc+fOvZ9MyGfOnHPu55i7IyIi1V+tqAMQEZHkUEIXEckQSugiIhlCCV1EJEMooYuIZAgldBGRDKGELsUys5fMbGCy942SmS03s1Or4LhuZofH7j9sZrclsm8FzjPAzF6uaJylHPckM8tL9nEl9epEHYAkj5ltjnvYAPgB2BF7fIW75yR6LHfvXRX7Zjp3vzIZxzGzLOBzoK6758eOnQMk/DuUmkcJPYO4e6OC+2a2HLjM3V8pup+Z1SlIEiKSOdTlUgMUfKU2s5vN7L/ARDP7kZm9YGarzWxd7H6LuNfMNLPLYvcHmdmbZjY2tu/nZta7gvu2NrM3zGyTmb1iZuPN7B8lxJ1IjHea2Vux471sZk3inv+1ma0wszVmNrKU9+dYM/uvmdWO23aOmS2I3e9uZu+Y2XozW2VmD5rZXiUc6zEzuyvu8U2x13xlZoOL7Hummb1vZhvN7AszGxX39Bux2/VmttnMji94b+Nef4KZzTGzDbHbExJ9b0pjZkfGXr/ezBaaWZ+4584ws0WxY35pZjfGtjeJ/X7Wm9laM5ttZsovKaY3vOY4CDgAaAUMIfzuJ8YetwS+Bx4s5fXHAkuAJsA9wKNmZhXY9wngPaAxMAr4dSnnTCTGi4FLgWbAXkBBgmkPTIgd/+DY+VpQDHf/D/Ad8PMix30idn8HcH3s5zkeOAW4qpS4icXQKxbPaUAboGj//XfAJcD+wJnAUDM7O/Zcz9jt/u7eyN3fKXLsA4AXgQdiP9t9wItm1rjIz7DHe1NGzHWB54GXY6/7LZBjZm1juzxK6L7bBzgKeC22/QYgD2gKHAj8HlBdkRRTQq85dgJ3uPsP7v69u69x92fdfYu7bwLGAD8r5fUr3P2v7r4DmAT8mPCHm/C+ZtYS6Abc7u7b3P1NYFpJJ0wwxonu/om7fw88BXSKbe8HvODub7j7D8BtsfegJE8C/QHMbB/gjNg23H2uu7/r7vnuvhz4SzFxFOeCWHwfuft3hA+w+J9vprt/6O473X1B7HyJHBfCB8Cn7v54LK4ngcXAL+P2Kem9Kc1xQCPg7tjv6DXgBWLvDbAdaG9m+7r7OnefF7f9x0Ard9/u7rNdhaJSTgm95ljt7lsLHphZAzP7S6xLYiPhK/7+8d0ORfy34I67b4ndbVTOfQ8G1sZtA/iipIATjPG/cfe3xMV0cPyxYwl1TUnnIrTGzzWzesC5wDx3XxGL44hYd8J/Y3H8gdBaL0uhGIAVRX6+Y83s9ViX0gbgygSPW3DsFUW2rQCaxz0u6b0pM2Z3j//wiz/ueYQPuxVmNsvMjo9tvxdYCrxsZsvMbERiP4YkkxJ6zVG0tXQD0BY41t33ZfdX/JK6UZJhFXCAmTWI23ZIKftXJsZV8ceOnbNxSTu7+yJC4upN4e4WCF03i4E2sTh+X5EYCN1G8Z4gfEM5xN33Ax6OO25ZrduvCF1R8VoCXyYQV1nHPaRI//eu47r7HHfvS+iOmUpo+ePum9z9Bnc/FOgD/M7MTqlkLFJOSug11z6EPun1sf7YO6r6hLEWby4wysz2irXuflnKSyoT4zPAWWb209gA5mjK/v/+BHAd4YPj6SJxbAQ2m1k7YGiCMTwFDDKz9rEPlKLx70P4xrLVzLoTPkgKrCZ0ER1awrGnA0eY2cVmVsfMLgTaE7pHKuM/hNb8cDOra2YnEX5Hk2O/swFmtp+7bye8JzsBzOwsMzs8NlaygTDuUFoXl1QBJfSaaxywN/At8C7wrxSddwBhYHENcBcwhTBfvjgVjtHdFwJXE5L0KmAdYdCuNAV92K+5+7dx228kJNtNwF9jMScSw0uxn+E1QnfEa0V2uQoYbWabgNuJtXZjr91CGDN4KzZz5Lgix14DnEX4FrMGGA6cVSTucnP3bYQE3pvwvj8EXOLui2O7/BpYHut6upLw+4Qw6PsKsBl4B3jI3V+vTCxSfqZxC4mSmU0BFrt7lX9DEMl0aqFLSplZNzM7zMxqxab19SX0xYpIJelKUUm1g4DnCAOUecBQd38/2pBEMoO6XEREMoS6XEREMkRkXS5NmjTxrKysqE4vIlItzZ0791t3b1rccwkl9Njg1f1AbeARd7+7yPP/B5wce9gAaObu+5d2zKysLHJzcxM5vYiIxJhZ0SuEdykzoccusx5PKDCUB8wxs2mxK+sAcPfr4/b/LdC5UhGLiEi5JdKH3h1Y6u7LYhcdTCZMNStJf2JFjUREJHUSSejNKVxgKI/CBYB2MbNWQGv2vCJORESqWLJnuVwEPBMrm7oHMxtiZrlmlrt69eokn1pEpGZLJKF/SeGKcS0ouaLbRZTS3eLu2e7e1d27Nm1a7CCtiIhUUCIJfQ7QxsLSYXsRkvYeixLEqtD9iFCYR0REUqzMhB5bTPgaYAbwMfCUuy80s9Hxaw0SEv1krVIiIhKNhOahu/t0Qv3l+G23F3k8KnlhiYhknp07YfhwGDIEjjgi+cfXpf8iIikycSL86U/w1ltVc3wldBGRFFi9OrTOe/aEQYOq5hxK6CIiKXDTTbBpE0yYAFZFK/cqoYuIVLGZM2HSpJDU27evuvMooYuIVKFt22DoUGjdGkaOrNpzacUiEZEqdO+9sHgxTJ8ODRpU7bnUQhcRqSKffQZ33QXnnw+9e1f9+ZTQRUSqgDtcfTXUrQvjxqXmnOpyERGpAk8/DTNmwAMPwMEHp+acaqGLiCTZhg0wbBh06QJXXZW686qFLiKSZLfeCl9/DdOmQe3aqTuvWugiIkmUmwvjx4f+865dU3tuJXQRkSTJz4crroCDDoI770z9+dXlIiKSJA89BPPmwVNPwX77pf78aqGLiCTBl1+GvvNevaBfv2hiqFYJPScHsrKgVq1wm5MTdUQiIsGwYbB9e+g/r6riW2WpNl0uOTmhKPyWLeHxihXhMcCAAdHFJSIyfTo88wyMGQOHHhpdHBbVinFdu3b13NzchPfPygpJvKhWrWD58qSFJSJSLlu2QIcOsPfeMH8+7LVX1Z7PzOa6e7HzZ6pNC33lyvJtFxFJhbvuCo3KWbOqPpmXpdr0obdsWb7tIiJVbeHCUE1x0KCwElHUqk1CHzNmz9KTDRqE7SIiqbZzZ6hzvu++Iamng2qT0AcMgOzs0GduFm6zszUgKiLRmDQJZs8OybxJk6ijCRIaFDWzXsD9QG3gEXe/u5h9LgBGAQ584O4Xl3bM8g6Kioiki2+/hXbt4MgjQ995rRQ2jSs1KGpmtYHxwGlAHjDHzKa5+6K4fdoAtwA93H2dmTVLTugiIuln+PBQUfHhh1ObzMuSSCjdgaXuvszdtwGTgb5F9rkcGO/u6wDc/Zvkhikikh7eeAMmToQbbwzTFdNJIgm9OfBF3OO82LZ4RwBHmNlbZvZurItmD2Y2xMxyzSx39erVFYtYRCQi27bBlVeG62Juuy3qaPaUrHnodYA2wElAC+ANMzva3dfH7+Tu2UA2hD70JJ1bRCQl/vQn+PhjePHFql/wuSISaaF/CRwS97hFbFu8PGCau29398+BTwgJXkQkIyxbBqNHw3nnwRlnRB1N8RJJ6HOANmbW2sz2Ai4CphXZZyqhdY6ZNSF0wSxLYpwiIpEpWPC5Tp3ULfhcEWV2ubh7vpldA8wgTFv8m7svNLPRQK67T4s99wszWwTsAG5y9zVVGbiISKo8+yz8618hmbdoEXU0Jas2xblERKKwcWOYb37ggfDee6GVHqWMKM4lIhKF226DVatg6tTok3lZ0mhKvIhIepk7Fx58EK66Crp1izqasimhi4gUY8eOsOBzs2bVpwhgmn+BEBGJxoQJoYU+eXI0Cz5XhFroIhnq3/+G66+Hr7+OOpLq56uv4Pe/h1/8Ai64IOpoEqeELpKB5syBvn3DNLu2bUM/cH5+1FFVH9dfHy7zf+ih6BZ8rggldJEM8/nncNZZcNBBoZBUt27w29+G23feiTq69Pevf8FTT8Gtt8Jhh0UdTfkooYtkkHXr4MwzQ+ty+nQ48UR4+eWQoFavhhNOgN/8JtyXPX3/fZjR0rYt3HRT1NGUnxK6SIb44Qc491xYujTMmW7XLmw3g/PPh8WLQ5L6+99Dwnr44TCTQ3YbMyZ8w3n4YahXL+poyk8JXSQDuMNll8HMmaFW989+tuc+jRrBPffA/PlwzDFhPczjjgv97RKqKN5zD1xyCZx0UtTRVIwSukgGGDUK/vEPuPPOstfZ7dABXnsNcnIgLw+OPTbU+F5Tg6svuYf3oFEjGDs26mgqTgldpJp77LFQ1vXSS2HkyMReYwYXXwxLlsCwYfDII6Eb5pFHwmr2Nc2kSWEA+Z57oGnTqKOpOCV0kWrs1Vfh8svh1FPhL38p/xS7ffeF++6DefNCAarLLw8Dp/PmVU286WjNmrCcXI8eMHhw1NFUjhK6SDX10UdhELRdO3jmGahbt+LH6tgxtFAnTQqDgt26wTXXhFkzme7mm8OCzxMmpNeCzxVRzcMXqZlWrQrTExs2DMuhJePSdLMwILhkSZi6N2FC6IZ57LHM7YaZPRsefRR+9zs4+uioo6k8JXSRambz5nDh0Jo18MIL0LJlco+///7w5z9Dbm64sObSS6FnT1iwILnnidq2bWGmT6tWcPvtUUeTHEroItXIjh3Qv3+YejhlCvzkJ1V3rs6d4a23Qgt2yZJwrmHDQvdEJrjvPli4MJRFaNgw6miSQwldpJpwh+uuC63yP/85dLlUtVq1wkDhkiVhwPSBB0KffU5OiKe6+vzzMDPonHPCt51MoYQuUk2MGwfjx4cZGVddldpzH3BA6FP/z3/Cmpq/+hWcfHJo4VY37mHAt3bt8AGVSZTQRaqBZ5+FG26A886DP/4xuji6dYN33w1TJBcsgE6dwgfMpk3RxVRezz0X6tyMHp3eCz5XhBaJFklz774bWsOdOoUrPPfeO+qIgm+/hREjQh/7wQeHPukLLkjvcrObNoUuo2bNQsmDdF8jtDilLRKdUAvdzHqZ2RIzW2pmI4p5fpCZrTaz+bF/l1U26HT26adw+unhAozq1DKR6uezz6BPH2jeHKZNS59kDtCkSbiy9J134MAD4aKL4LTTQhGwdHX77WHK58MPV89kXpYyE7qZ1QbGA72B9kB/M2tfzK5T3L1T7N8jSY4zLWzbBnfdFearvvsuvPde6EvM1Dm6Eq21a+GMM8LMlunT0/eS9IICXw8+GKY6duwIt9wC330XdWSFzZsX+syvvDLUr8lEiXxGdQeWuvsyADObDPQFFlVlYOnmzTdhyJBQke2CC8IA1TPPwLXXhqWq7r476gglk/zwA5x9NixfHi7vP+KIqCMqXe3acPXVoUzv8OHh7yEnJ/ydnHNO5bphdu6ELVvC/Pv4f999t+e20v59/nn4UPzDH5L3c6ebRBJ6c+CLuMd5QHGfb+eZWU/gE+B6d/+i6A5mNgQYAtAy2VdDVJF168J/0EceCRcgvPhiaDVBGClftCgMUh15JAwcGG2skhl27gwX88yeDU8+CT/9adQRJa5Zs3Bl6WWXhQR/3nmhe/KKK2Dr1uITbVmJuTwt/Vq1QsXEov+aNYNDDw0rN+2/f5X9+JFLVi/S88CT7v6DmV0BTAJ+XnQnd88GsiEMiibp3FXCPaz2PWxYuCLvppvgjjsKX4BgFr7CffJJaL0ffngo8CNSGbffHhL5//5v6Jeujn76U5g7N0yzvO02mDFjz33q1Ss++TZpUvhxw4bF71fcv/r103tQtqqVOcvFzI4HRrn76bHHtwC4+/+WsH9tYK27l1pdIp1nuSxbFub5zpgRpmllZ4cZBiVZuzb0yW3YEPrVs7JSFqpkmEcfDa3byy+vWPXEdLRmTfib2mefwkm6MsXEarLKznKZA7Qxs9ZmthdwETCtyAl+HPewD/BxRYON0vbtoe+vQwd4++1wNd4775SezCFcdPH882HQtE8fzXyRinn55dA1cfrpoWWbCckcoHHj0DBq1y7M+95/fyXzqlJmQnf3fOAaYAYhUT/l7gvNbLSZ9Yntdq2ZLTSzD4BrgUFVFXBVeeedUKvilltCH/nHH+++miwR7dqFhXgXLQorxmitRimPDz+Efv1CY+Kpp5TwpGJq/IVF69eHWSoPPxzm+o4fH1rZFfXgg2HgZfjwaK/ok+rjq69Cl93OnbsvrRcpSWldLhk4tT4x7runHX7zTSh6NHp06OerjKuvDvUt7rkH2rfXzBcp3aZNocjW+vVhVouSuVRGjUzoy5eHxDt9euhmeeEF6NIlOcfWzBdJVH4+XHhh6G554YWyx2pEylKjinPl54cVvTt0gFmz4P/+L3zFTVYyL1C3Ljz9dFh44JxzwgeISDz30DX30kvw0EPQq1fUEUkmqDEJfc6cMNJ+001wyilh8HLYsKqr53DAAaHVtW0b/PKXmvkihY0dG8ZtRowI3+REkiHjE/rGjaGf/NhjQ1/5s8/CP/+Z/GW7itO2bWipf/wxXHyxZr5I8PTTYdD8wgthzJioo5FMktEJ/f/9vzAw+eCDoc980aKwSnoq5/eedhrcf39ord9yS+rOK+np7bfh178O4yqPPVb9V5mX9JKRg6JffBH6J//5z1D57dlno62uVjDz5d57wwfMoEHRxSLRWbo0TIlt2RKmTg2XqYskU0a1D3bsCK3h9u3DVXf33BPKeaZDqcz77w9990OGhMqNUrN8+y307h3uT58e6pWIJFvGJPR580LiHjYMTjwxdK/cdFP6XHFXt264AjArSzNfapqtW0Mp3C++CItUHH541BFJpqr2CX3z5rDWYrdukJcHU6aEErfpWCCroOZLfr5mvtQUO3eGLra33oLHHw+rXIlUlWqd0J9/PnSv3Hdf6MpYvDj91zRs2za01DXzpWYYOTI0Mu65Jyz+IFKVqmVC/+qrUMioTx/Yd9/Q+pkwofoUrtfMl5ohOztU7xw6FG68MepopCaodrNcnnwylBjdvj0sJXXDDbDXXlFHVX4F0yg18yUzvfRSqKl/xhmhFEQ6f2uUzFHtEnqTJmFR2gkT4LDDoo6mcsaNgyVLdtd8qU5LjUnx1q4NjY4RI8KU2SlTMnN1eUlP1bJ8rnvmtHjWrQuzc9atC+UJ0nEwV0q3Y0dYyPlvfwvzy3/4IQzST50KBx8cdXSSaSq7YlHayZRkDvCjH2nmS3X12WdhvcysrLDK0L//Hb5tzZsXliJUMpdU05fBNFBQ86VXrzDzZerUxFdKktT67rtQR3/ixFCxs1atkMzvuy8M0terF3WEUpNVyxZ6Jjr11DB49sILof9V0od7qMFy2WVw0EFhAPvLL8Og/MqV4crP889XMpfoqYWeRq66KtR8GTs2zHy59NKoI6rZvvoqXAw0cWIYvG7YMFznMHhwKK6VSV1/khmU0NPMuHFhtaMrrggzX048MeqIapZt28KYxsSJYerhzp3hd3DzzaEV3qhR1BGKlEwJPc0U1Hw57rhQ6ve996B166ijynwLFoRZKjk5oZDWwQeHrq9Bg6BNm6ijE0lMQn3oZtbLzJaY2VIzK7GH18zOMzM3s2Kn1Ehi4me+9OkTFumQ5Fu7FsaPD0sQHnNMWAru5JNDn/jKlWHxCSVzqU7KTOhmVhsYD/QG2gP9zax9MfvtA1wH/CfZQdZERxyh1Y6qwo4dobTyRReFVvg114RulQcegFWrwrej3r01y0iqp0Ra6N2Bpe6+zN23AZOBvsXsdyfwR2BrEuOr0Qpmvrz4oma+VFZpc8bffz8siNK4cdRRilROIn3ozYEv4h7nAYWWjDCznwCHuPuLZnZTSQcysyHAEICWqVjUMwNcdVWo+aKZL+VX3JzxX/xCc8Ylc1V6UNTMagH3AYPK2tfds4FsCJf+V/bcNUVBzZd0nfniDl9/Dd9/H5Jm7drF35a0LZnrarrDO++EAc4pU0K9/MMPD/3hl1wCLVok71wi6SaRhP4lcEjc4xaxbQX2AY4CZlqYmHsQMM3M+rh7xYq1SCF16qTHzBd3+OabMFf+o4/CbcG/9esrd+xEEn8i2zZtCgOaBXPGL700FD3TnHGpCcoszmVmdYBPgFMIiXwOcLG7Lyxh/5nAjWUl88oU54paTk5YuGDlyrDg75gxMGBA1Z/3k09CIa/mzcOVi/vuW3Xn+vbbwgm7IIGvWbN7nwMOgA4d4KijQnfQPvuEQcedO3ffxt9PxbZatcKgpuaMS6YqrThXmS10d883s2uAGUBt4G/uvtDMRgO57j4tueGmt5ycMJi2ZUt4vGJFeAxVn9SPOCL0CZ9+epj58s9/Vn42xrp1hRN3QfL+5pvd++y3X0jc5567O4F36AAHHqiWr0g6qZblc6OUlRWSeFGtWqVu4ecJE8Jg6Q03hMHSRGzcGAZXi3aXfPXV7n0aNQot7YKEXfCveXMlbpF0UakWuhS2cmX5tleFoUNDMv7Tn0ICHjx493Pffbc7cce3uL+Im6e0997hdaeeWrjFfcghyR2gFJHUUkIvp5Yti2+hp3oWZsHMlyuvhA8+gGXLQvL+/PPd+9SrB+3aQc+ehVvcrVsrcYtkIiX0chozpnAfOkCDBmF7KhXMfDn55NAF07YtdO8eZnUUtLgPPVTLn4nUJPpzL6eCgc8oZrkU9aMfhasc8/NDUS8RqdmU0CtgwIBoEnhxzJTMRSRQT6qISIZQQhcRyRBK6CIiGUIJXUQkQyihi4hkCCV0EZEMoYQuIpIhlNBFRDKEErqISIZQQhcRyRBK6CIiGUIJXUQkQyihi4hkCCV0EZEMoYQuIpIhlNBFRDJEQgndzHqZ2RIzW2pmI4p5/koz+9DM5pvZm2bWPvmhiohIacpM6GZWGxgP9AbaA/2LSdhPuPvR7t4JuAe4L+mRiohIqRJpoXcHlrr7MnffBkwG+sbv4O4b4x42BDx5IYqISCISSejNgS/iHufFthViZleb2WeEFvq1xR3IzIaYWa6Z5a5evboi8UqcnBzIyoJatcJtTk7UEYlIlJI2KOru4939MOBm4NYS9sl2967u3rVp06bJOnWNlJMDQ4bAihXgHm6HDFFSF6nJEknoXwKHxD1uEdtWksnA2ZUJSso2ciRs2VJ425YtYbuI1EyJJPQ5QBsza21mewEXAdPidzCzNnEPzwQ+TV6IUpyVK8u3XUQyX52ydnD3fDO7BpgB1Ab+5u4LzWw0kOvu04BrzOxUYDuwDhhYlUELtGwZulmK2y4iNVOZCR3A3acD04tsuz3u/nVJjkvKMGZM6DOP73Zp0CBsF5GaSVeKVlMDBkB2NrRqBWbhNjs7bBeRmimhFrqkpwEDlMBFZDe10EVEMoQSuohIhlBCFxHJEEroIiIZQgldRCRDKKGLiGQIJXQRkQyhhC4ikiGU0EVEMoQSuohIhlBCFxHJEEroIiIZQgldRCRDKKGLiGQIJXSptJwcyMqCWrXCrRaqFomG6qFLpeTkFF45acWK8BhUq10k1dRCl0oZObLwMngQHo8cGU08IjWZErpUysqV5dsuIlVHCV0qpWXL8m0XkaqTUEI3s15mtsTMlprZiGKe/52ZLTKzBWb2qpm1Sn6oko7GjIEGDQpva9AgbBeR1CozoZtZbWA80BtoD/Q3s/ZFdnsf6OruHYFngHuSHaikpwEDIDsbWrUCs3Cbna0BUZEoJDLLpTuw1N2XAZjZZKAvsKhgB3d/PW7/d4FfJTNISW8DBiiBi6SDRLpcmgNfxD3Oi20ryW+AlyoTlIiIlF9S56Gb2a+ArsDPSnh+CDAEoKVGzUREkiqRFvqXwCFxj1vEthViZqcCI4E+7v5DcQdy92x37+ruXZs2bVqReEVEpASJJPQ5QBsza21mewEXAdPidzCzzsBfCMn8m+SHKSIiZSkzobt7PnANMAP4GHjK3Rea2Wgz6xPb7V6gEfC0mc03s2klHE5ERKpIQn3o7j4dmF5k2+1x909NclwiIlJOulJURCRDKKFLxlAZX6npVD5XMoLK+IqohS4ZQmV8RZTQJUOojK+IErpkCJXxFVFClwyhMr4iSuiSIVTGV0SzXCSDqIyv1HRqoYuIZAgldBGRDKGELiKSIZTQRUQyhBK6iEiGUEIXEckQSugiSaSKjxIlzUMXSRJVfJSoqYUukiSq+ChRU0IXSRJVfJSopVWXy/bt28nLy2Pr1q1RhyJlqF+/Pi1atKBu3bpRh5I2WrYM3SzFbRdJhbRK6Hl5eeyzzz5kZWVhZlGHIyVwd9asWUNeXh6tW7eOOpy0MWZM4T50UMVHSa206nLZunUrjRs3VjJPc2ZG48aN9U2qCFV8lKgllNDNrJeZLTGzpWY2opjne5rZPDPLN7N+lQlIybx60O+peAMGwPLlsHNnuFUyl1QqM6GbWW1gPNAbaA/0N7P2RXZbCQwCnkh2gCIikphEWujdgaXuvszdtwGTgb7xO7j7cndfAOysghhLlOyLONasWUOnTp3o1KkTBx10EM2bN9/1eNu2baW+Njc3l2uvvbbMc5xwwgmVCzJm5syZnHXWWUk5lohkhkQGRZsDX8Q9zgOOrcjJzGwIMASgZSWH/qviIo7GjRszf/58AEaNGkWjRo248cYbdz2fn59PnTrFv2Vdu3ala9euZZ7j7bffrlhwIuWQkxPmv69cGWbZjBmj7p+aIKWDou6e7e5d3b1r06ZNK3WsVF3EMWjQIK688kqOPfZYhg8fznvvvcfxxx9P586dOeGEE1iyZAlQuMU8atQoBg8ezEknncShhx7KAw88sOt4jRo12rX/SSedRL9+/WjXrh0DBgzA3QGYPn067dq1o0uXLlx77bVltsTXrl3L2WefTceOHTnuuONYsGABAEJFfPMAAAxzSURBVLNmzdr1DaNz585s2rSJVatW0bNnTzp16sRRRx3F7Nmzk/uGSeQKGjsrVoD77saOyhBkvkRa6F8Ch8Q9bhHbFqlUXsSRl5fH22+/Te3atdm4cSOzZ8+mTp06vPLKK/z+97/n2Wef3eM1ixcv5vXXX2fTpk20bduWoUOH7jFn+/3332fhwoUcfPDB9OjRg7feeouuXbtyxRVX8MYbb9C6dWv69+9fZnx33HEHnTt3ZurUqbz22mtccsklzJ8/n7FjxzJ+/Hh69OjB5s2bqV+/PtnZ2Zx++umMHDmSHTt2sKXop6JUe6U1dtRKz2yJJPQ5QBsza01I5BcBF1dpVAlI5UUc559/PrVr1wZgw4YNDBw4kE8//RQzY/v27cW+5swzz6RevXrUq1ePZs2a8fXXX9OiRYtC+3Tv3n3Xtk6dOrF8+XIaNWrEoYceumt+d//+/cnOzi41vjfffHPXh8rPf/5z1qxZw8aNG+nRowe/+93vGDBgAOeeey4tWrSgW7duDB48mO3bt3P22WfTqVOnSr03kn50xWrNVWaXi7vnA9cAM4CPgafcfaGZjTazPgBm1s3M8oDzgb+Y2cKqDBpCn2CDBoW3VdVFHA0bNtx1/7bbbuPkk0/mo48+4vnnny9xLna9evV23a9duzb5+fkV2qcyRowYwSOPPML3339Pjx49WLx4MT179uSNN96gefPmDBo0iL///e9JPadEr6RGja5YzXwJ9aG7+3R3P8LdD3P3MbFtt7v7tNj9Oe7ewt0buntjd+9QlUFDdBdxbNiwgebNmwPw2GOPJf34bdu2ZdmyZSxfvhyAKVOmlPmaE088kZxYB+nMmTNp0qQJ++67L5999hlHH300N998M926dWPx4sWsWLGCAw88kMsvv5zLLruMefPmJf1nkGilsrEj6SWtLv0vrwEDUt8nOHz4cAYOHMhdd93FmWeemfTj77333jz00EP06tWLhg0b0q1btzJfUzAI27FjRxo0aMCkSZMAGDduHK+//jq1atWiQ4cO9O7dm8mTJ3PvvfdSt25dGjVqpBZ6Bir4m9Asl5rHCmZWpFrXrl09Nze30LaPP/6YI488MpJ40snmzZtp1KgR7s7VV19NmzZtuP7666MOaw/6fUlZNH0y+cxsrrsXO0c6rWq5SPDXv/6VTp060aFDBzZs2MAVV1wRdUgi5abpk6mnFrpUmH5fUpqsrOJnorVqFercSMWohS4iKafpk6mnhC4iVULTJ1NPCV1EqoSmT6aeErqIVAkt+JF6SuhxTj75ZGbMmFFo27hx4xg6dGiJrznppJMoGNw944wzWL9+/R77jBo1irFjx5Z67qlTp7Jo0aJdj2+//XZeeeWV8oRfLJXZlShpwY/UUkKP079/fyZPnlxo2+TJkxMqkAWhSuL+++9foXMXTeijR4/m1FNPrdCxRKSwZK+dkK7S9krRYcMgVpo8aTp1gnHjSn6+X79+3HrrrWzbto299tqL5cuX89VXX3HiiScydOhQ5syZw/fff0+/fv34n//5nz1en5WVRW5uLk2aNGHMmDFMmjSJZs2accghh9ClSxcgzDHPzs5m27ZtHH744Tz++OPMnz+fadOmMWvWLO666y6effZZ7rzzTs466yz69evHq6++yo033kh+fj7dunVjwoQJ1KtXj6ysLAYOHMjzzz/P9u3befrpp2nXrl2JP9/atWsZPHgwy5Yto0GDBmRnZ9OxY0dmzZrFddddB4Sl5d544w02b97MhRdeyMaNG8nPz2fChAmceOKJlfsFiESgKtZOSFdqocc54IAD6N69Oy+99BIQWucXXHABZsaYMWPIzc1lwYIFzJo1a1fN8eLMnTuXyZMnM3/+fKZPn86cOXN2PXfuuecyZ84cPvjgA4488kgeffRRTjjhBPr06cO9997L/PnzOeyww3btv3XrVgYNGsSUKVP48MMPdyXXAk2aNGHevHkMHTq0zG6dgjK7CxYs4A9/+AOXXHIJwK4yu/Pnz2f27NnsvffePPHEE5x++unMnz+fDz74QFUZpdpK1doJ6SBtW+iltaSrUkG3S9++fZk8eTKPPvooAE899RTZ2dnk5+ezatUqFi1aRMeOHYs9xuzZsznnnHNoEBvi79Onz67nPvroI2699VbWr1/P5s2bOf3000uNZ8mSJbRu3ZojjjgCgIEDBzJ+/HiGDRsGhA8IgC5duvDcc8+VeiyV2ZWaqCbNh1cLvYi+ffvy6quvMm/ePLZs2UKXLl34/PPPGTt2LK+++ioLFizgzDPPLLFsblkGDRrEgw8+yIcffsgdd9xR4eMUKCjBW5nyuyqzK5ksnebDV3VfvhJ6EY0aNeLkk09m8ODBuwZDN27cSMOGDdlvv/34+uuvd3XJlKRnz55MnTqV77//nk2bNvH888/vem7Tpk38+Mc/Zvv27btK3gLss88+bNq0aY9jtW3bluXLl7N06VIAHn/8cX72s59V6GdTmV2pidJlPnwqatsooRejf//+fPDBB7sS+jHHHEPnzp1p164dF198MT169Cj19T/5yU+48MILOeaYY+jdu3ehErh33nknxx57LD169Cg0gHnRRRdx77330rlzZz777LNd2+vXr8/EiRM5//zzOfroo6lVqxZXXnllhX6uUaNGMXfuXDp27MiIESMKldk96qij6NixI3Xr1qV3797MnDlz1889ZcqUXYOmItVNusyHT0VfvopzSYXp9yWSuFq1Qsu8KLMwTz9RKs4lIhKxVPTlK6GLiKRAKvry0y6hR9UFJOWj35NI+aSiLz+t5qHXr1+fNWvW0LhxY8ws6nCkBO7OmjVrqF+/ftShiFQrVb0OckIJ3cx6AfcDtYFH3P3uIs/XA/4OdAHWABe6+/LyBtOiRQvy8vJYvXp1eV8qKVa/fn1atGgRdRgiEqfMhG5mtYHxwGlAHjDHzKa5+6K43X4DrHP3w83sIuCPwIXlDaZu3bq0bt26vC8TERES60PvDix192Xuvg2YDPQtsk9fYFLs/jPAKaY+ExGRlEokoTcHvoh7nBfbVuw+7p4PbAAaFz2QmQ0xs1wzy1W3iohIcqV0lou7Z7t7V3fv2rRp01SeWkQk4yUyKPolcEjc4xaxbcXtk2dmdYD9CIOjJZo7d+63ZraiHLGmoybAt1EHkUb0fuym96IwvR+FVeb9aFXSE4kk9DlAGzNrTUjcFwEXF9lnGjAQeAfoB7zmZUxUdvdq30Q3s9ySLsGtifR+7Kb3ojC9H4VV1ftRZkJ393wzuwaYQZi2+Dd3X2hmo4Fcd58GPAo8bmZLgbWEpC8iIimU0Dx0d58OTC+y7fa4+1uB85MbmoiIlEfaXfpfzWRHHUCa0fuxm96LwvR+FFYl70dk5XNFRCS51EIXEckQSugiIhlCCb0CzOwQM3vdzBaZ2UIzq/Hrs5lZbTN738xeiDqWqJnZ/mb2jJktNrOPzez4qGOKkpldH/s7+cjMnjSzGlOm08z+ZmbfmNlHcdsOMLN/m9mnsdsfJet8SugVkw/c4O7tgeOAq82sfcQxRe064OOog0gT9wP/cvd2wDHU4PfFzJoD1wJd3f0owtTnmjSt+TGgV5FtI4BX3b0N8GrscVIooVeAu69y93mx+5sIf7BF69vUGGbWAjgTeCTqWKJmZvsBPQnXZuDu29x9fbRRRa4OsHfsKvIGwFcRx5My7v4G4dqcePHFDCcBZyfrfErolWRmWUBn4D/RRhKpccBwoBxL3Was1sBqYGKsC+oRM2sYdVBRcfcvgbHASmAVsMHdX442qsgd6O6rYvf/CxyYrAMroVeCmTUCngWGufvGqOOJgpmdBXzj7nOjjiVN1AF+Akxw987AdyTxK3V1E+sf7kv4oDsYaGhmv4o2qvQRK5GStLnjSugVZGZ1Cck8x92fizqeCPUA+pjZckKt/J+b2T+iDSlSeUCeuxd8Y3uGkOBrqlOBz919tbtvB54DTog4pqh9bWY/BojdfpOsAyuhV0Bs8Y5HgY/d/b6o44mSu9/i7i3cPYsw2PWau9fYFpi7/xf4wszaxjadAiwq5SWZbiVwnJk1iP3dnEINHiSOKShmSOz2n8k6sBJ6xfQAfk1ojc6P/Tsj6qAkbfwWyDGzBUAn4A8RxxOZ2DeVZ4B5wIeEnFNjygCY2ZOEKrRtzSzPzH4D3A2cZmafEr7B3F3aMcp1Pl36LyKSGdRCFxHJEEroIiIZQgldRCRDKKGLiGQIJXQRkQyhhC4ikiGU0EVEMsT/B03G+j57FOwqAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TZwbBo8MSesy"
      },
      "source": [
        "이전에 순환넽워크를 사용하지 않은 모델보다, val_acc가 낮은 것을 알 수 있다. 단어의 개수를 500개로 제한해서, 적은 정보를 활용했기 때문이다.  \r\n",
        "더 성능이 좋은 고급 순환층을 살펴보자."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VZJ5ZEP5S8iQ"
      },
      "source": [
        "# 2) keras의 LSTM"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sutOcknGSZ2e",
        "outputId": "c784174e-1c1d-498d-a993-357271f60c22"
      },
      "source": [
        "from keras.layers import LSTM\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "model.add(Embedding(max_features, 32))\r\n",
        "model.add(LSTM(32))  # Long Short-Term Memory 층 추가\r\n",
        "model.add(Dense(1, activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop',\r\n",
        "              loss='binary_crossentropy',\r\n",
        "              metrics=['acc'])\r\n",
        "history = model.fit(input_train, ytrain,\r\n",
        "                    epochs=10,\r\n",
        "                    batch_size=128,\r\n",
        "                    validation_split=0.2)"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "157/157 [==============================] - 53s 329ms/step - loss: 0.6428 - acc: 0.6427 - val_loss: 0.4121 - val_acc: 0.8234\n",
            "Epoch 2/10\n",
            "157/157 [==============================] - 54s 346ms/step - loss: 0.3606 - acc: 0.8607 - val_loss: 0.3290 - val_acc: 0.8610\n",
            "Epoch 3/10\n",
            "157/157 [==============================] - 54s 345ms/step - loss: 0.2545 - acc: 0.9037 - val_loss: 0.3553 - val_acc: 0.8484\n",
            "Epoch 4/10\n",
            "157/157 [==============================] - 54s 344ms/step - loss: 0.2083 - acc: 0.9228 - val_loss: 0.3215 - val_acc: 0.8728\n",
            "Epoch 5/10\n",
            "157/157 [==============================] - 55s 348ms/step - loss: 0.1765 - acc: 0.9366 - val_loss: 0.3205 - val_acc: 0.8590\n",
            "Epoch 6/10\n",
            "157/157 [==============================] - 54s 345ms/step - loss: 0.1597 - acc: 0.9411 - val_loss: 0.3354 - val_acc: 0.8886\n",
            "Epoch 7/10\n",
            "157/157 [==============================] - 54s 347ms/step - loss: 0.1424 - acc: 0.9509 - val_loss: 0.3059 - val_acc: 0.8712\n",
            "Epoch 8/10\n",
            "157/157 [==============================] - 54s 346ms/step - loss: 0.1216 - acc: 0.9584 - val_loss: 0.4030 - val_acc: 0.8662\n",
            "Epoch 9/10\n",
            "157/157 [==============================] - 57s 361ms/step - loss: 0.1208 - acc: 0.9592 - val_loss: 0.3926 - val_acc: 0.8712\n",
            "Epoch 10/10\n",
            "157/157 [==============================] - 58s 367ms/step - loss: 0.1063 - acc: 0.9645 - val_loss: 0.3302 - val_acc: 0.8710\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 563
        },
        "id": "V3D4_9QUTNGa",
        "outputId": "1a1cefb2-87b8-4f21-c355-c4f6d8078bb0"
      },
      "source": [
        "acc = history.history['acc']\r\n",
        "val_acc = history.history['val_acc']\r\n",
        "loss = history.history['loss']\r\n",
        "val_loss = history.history['val_loss']\r\n",
        "\r\n",
        "epochs = range(1, len(acc) + 1)\r\n",
        "\r\n",
        "plt.plot(epochs, acc, 'bo', label='Training acc')\r\n",
        "plt.plot(epochs, val_acc, 'b', label='Validation acc')\r\n",
        "plt.title('Training and validation accuracy')\r\n",
        "plt.legend()\r\n",
        "\r\n",
        "plt.figure()\r\n",
        "\r\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\r\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\r\n",
        "plt.title('Training and validation loss')\r\n",
        "plt.legend()"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f254d7b6978>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwV1f3/8dcHVDCAyKYiEYIKIopsEQVqxa1FoVIRWzBaqK3UtUq1VqtWi+VX/Uorta1WrLiBRUstRYW6oLjVViLiAooCBllEEZUdZPn8/jiTcBMScoGbzM3k/Xw87iNzZ/3cSfK5Z86cc8bcHRERSa46cQcgIiJVS4leRCThlOhFRBJOiV5EJOGU6EVEEk6JXkQk4ZToayEzm2ZmQzO9bpzMrMjMTq2C/bqZHR5N/8XMbkxn3d04ToGZPbO7cYrsjKkdfc1gZmtT3uYAm4Ct0fufuPuE6o8qe5hZEfBjd38uw/t1oJ27z8/UumaWB3wE7O3uWzIRp8jO7BV3AJIed29YPL2zpGZmeyl5SLbQ32N2UNVNDWdmfcxsiZn9wsyWA/ebWRMze9LMVpjZl9F0bso2M8zsx9H0MDN7xcxGR+t+ZGan7+a6bc3sJTNbY2bPmdmfzWx8BXGnE+MtZvZqtL9nzKx5yvLzzWyRma00s+t3cn6OM7PlZlY3Zd5ZZvZ2NN3DzF4zs6/M7BMz+5OZ7VPBvh4ws9+kvP95tM0yM7ugzLr9zOxNM1ttZovN7OaUxS9FP78ys7Vm1rP43KZs38vMZprZquhnr3TPzS6e56Zmdn/0Gb40s8kpywaY2ezoMywws77R/FLVZGZ2c/Hv2czyoiqsH5nZx8Dz0fy/R7+HVdHfyFEp2+9rZr+Lfp+ror+xfc3sKTO7vMznedvMzirvs0rFlOiT4SCgKdAGGE74vd4fvW8NbAD+tJPtjwPmAc2B/wPuMzPbjXUfAV4HmgE3A+fv5JjpxHgu8EPgAGAf4GoAM+sI3B3t/+DoeLmUw93/B6wDTi6z30ei6a3AiOjz9AROAS7ZSdxEMfSN4jkNaAeUvT+wDvgBsD/QD7jYzL4bLftm9HN/d2/o7q+V2XdT4Cngzuiz/R54ysyalfkMO5ybclR2nh8mVAUeFe3rjiiGHsBDwM+jz/BNoKii81GOE4EjgW9H76cRztMBwCwgtapxNNAd6EX4O74G2AY8CJxXvJKZdQZaEc6N7Ap316uGvQj/cKdG032Ar4H6O1m/C/BlyvsZhKofgGHA/JRlOYADB+3KuoQksgXISVk+Hhif5mcqL8YbUt5fAvw7mv4VMDFlWYPoHJxawb5/A4yLphsRknCbCta9EvhnynsHDo+mHwB+E02PA25NWa996rrl7HcMcEc0nRetu1fK8mHAK9H0+cDrZbZ/DRhW2bnZlfMMtCQk1CblrHdPcbw7+/uL3t9c/HtO+WyH7iSG/aN1GhO+iDYAnctZrz7wJeG+B4QvhLuq+/8tCS+V6JNhhbtvLH5jZjlmdk90KbyaUFWwf2r1RRnLiyfcfX002XAX1z0Y+CJlHsDiigJOM8blKdPrU2I6OHXf7r4OWFnRsQil94FmVg8YCMxy90VRHO2j6ozlURz/j1C6r0ypGIBFZT7fcWb2QlRlsgq4KM39Fu97UZl5iwil2WIVnZtSKjnPhxB+Z1+Ws+khwII04y1Pybkxs7pmdmtU/bOa7VcGzaNX/fKOFf1NPwqcZ2Z1gCGEKxDZRUr0yVC26dRVwBHAce6+H9urCiqqjsmET4CmZpaTMu+Qnay/JzF+krrv6JjNKlrZ3ecSEuXplK62gVAF9D6h1Lgf8MvdiYFwRZPqEWAKcIi7Nwb+krLfypq6LSNUtaRqDSxNI66ydnaeFxN+Z/uXs91i4LAK9rmOcDVX7KBy1kn9jOcCAwjVW40Jpf7iGD4HNu7kWA8CBYQqtfVepppL0qNEn0yNCJfDX0X1vTdV9QGjEnIhcLOZ7WNmPYHvVFGMk4D+ZvaN6MbpSCr/W34EuIKQ6P5eJo7VwFoz6wBcnGYMjwHDzKxj9EVTNv5GhNLyxqi++9yUZSsIVSaHVrDvqUB7MzvXzPYys+8DHYEn04ytbBzlnmd3/4RQd35XdNN2bzMr/iK4D/ihmZ1iZnXMrFV0fgBmA4Oj9fOBQWnEsIlw1ZVDuGoqjmEboRrs92Z2cFT67xldfREl9m3A71Bpfrcp0SfTGGBfQmnpv8C/q+m4BYQbmisJ9eKPEv7By7PbMbr7HOBSQvL+hFCPu6SSzf5GuEH4vLt/njL/akISXgPcG8WcTgzTos/wPDA/+pnqEmCkma0h3FN4LGXb9cAo4FULrX2OL7PvlUB/Qml8JeHmZP8ycaersvN8PrCZcFXzGeEeBe7+OuFm7x3AKuBFtl9l3EgogX8J/JrSV0jleYhwRbUUmBvFkepq4B1gJvAFcBulc9NDQCfCPR/ZDeowJVXGzB4F3nf3Kr+ikOQysx8Aw939G3HHUlOpRC8ZY2bHmtlh0aV+X0K97OTKthOpSFQtdgkwNu5YajIlesmkgwhN/9YS2oBf7O5vxhqR1Fhm9m3C/YxPqbx6SHZCVTciIgmnEr2ISMJl3aBmzZs397y8vLjDEBGpUd54443P3b1FecuyLtHn5eVRWFgYdxgiIjWKmZXtTV1CVTciIgmnRC8iknBK9CIiCZd1dfTl2bx5M0uWLGHjxo2VryyxqF+/Prm5uey9995xhyIiZdSIRL9kyRIaNWpEXl4eFT8PQ+Li7qxcuZIlS5bQtm3buMMRkTJqRNXNxo0badasmZJ8ljIzmjVrpisukd00YQLk5UGdOuHnhAmVbbFrakSJHlCSz3L6/YjsngkTYPhwWB89smfRovAeoKAgM8eoESV6EZGkuv767Um+2Pr1YX6mKNGnYeXKlXTp0oUuXbpw0EEH0apVq5L3X3/99U63LSws5Kc//Wmlx+jVq1emwhWRGuTjj3dt/u5IZKLPdH1Xs2bNmD17NrNnz+aiiy5ixIgRJe/32WcftmzZUuG2+fn53HnnnZUe4z//+c+eBSkiNVLrsg+hrGT+7khcoi+u71q0CNy313dl+ubGsGHDuOiiizjuuOO45ppreP311+nZsyddu3alV69ezJs3D4AZM2bQv39/AG6++WYuuOAC+vTpw6GHHlrqC6Bhw4Yl6/fp04dBgwbRoUMHCgoKKB5hdOrUqXTo0IHu3bvz05/+tGS/qYqKijjhhBPo1q0b3bp1K/UFctttt9GpUyc6d+7MtddeC8D8+fM59dRT6dy5M926dWPBgj15HrRIzVLVN0HTMWoU5OSUnpeTE+ZnjLtn1at79+5e1ty5c3eYV5E2bdxDii/9atMm7V3s1E033eS33367Dx061Pv16+dbtmxxd/dVq1b55s2b3d392Wef9YEDB7q7+wsvvOD9+vUr2bZnz56+ceNGX7FihTdt2tS//vprd3dv0KBByfr77befL1682Ldu3erHH3+8v/zyy75hwwbPzc31hQsXurv74MGDS/abat26db5hwwZ3d//ggw+8+HxOnTrVe/bs6evWrXN395UrV7q7e48ePfzxxx93d/cNGzaULN8du/J7Eonb+PHuOTml80ROTpgfRyxt2ribhZ+7EwNQ6BXk1RrT6iZd1VHfVeycc86hbt26AKxatYqhQ4fy4YcfYmZs3ry53G369etHvXr1qFevHgcccACffvopubm5pdbp0aNHybwuXbpQVFREw4YNOfTQQ0vaqQ8ZMoSxY3d86M7mzZu57LLLmD17NnXr1uWDDz4A4LnnnuOHP/whOVHRoWnTpqxZs4alS5dy1llnAaHTk0htsbOboJlq7ZKugoKqPWbiqm6qo76rWIMGDUqmb7zxRk466STeffddnnjiiQrblNerV69kum7duuXW76ezTkXuuOMODjzwQN566y0KCwsrvVksEodsqDKpzkJh3NJK9GbW18zmmdl8M7u2nOVtzGy6mb1tZjPMLDdl2VYzmx29pmQy+PJUS31XOVatWkWrVq0AeOCBBzK+/yOOOIKFCxdSVFQEwKOPPlphHC1btqROnTo8/PDDbN26FYDTTjuN+++/n/VREeaLL76gUaNG5ObmMnlyeKzrpk2bSpaLVJXquo9WmeosFMat0kRvZnWBPwOnAx2BIWbWscxqo4GH3P0YYCTw25RlG9y9S/Q6M0NxV6igAMaOhTZtwCz8HDu26i/FrrnmGq677jq6du26SyXwdO27777cdddd9O3bl+7du9OoUSMaN268w3qXXHIJDz74IJ07d+b9998vuero27cvZ555Jvn5+XTp0oXRo0cD8PDDD3PnnXdyzDHH0KtXL5YvX57x2EVSVUe78XTEVSiMQ6XPjDWznsDN7v7t6P11AO7+25R15gB93X2xhS6Sq9x9v2jZWndvmG5A+fn5XvbBI++99x5HHnlkurtIrLVr19KwYUPcnUsvvZR27doxYsSIuMMqod+TpKNOnVCSL8sMtm2r3lgmTAhfMB9/HEryo0ZVf/18ppjZG+6eX96ydKpuWgGLU94viealegsYGE2fBTQys2bR+/pmVmhm/zWz71YQ4PBoncIVK1akEVLtdO+999KlSxeOOuooVq1axU9+8pO4Q5IaKO768WyqMikogKKi8AVTVFRzk3xlMnUz9mrgRDN7EzgRWApsjZa1ib5lzgXGmNlhZTd297Hunu/u+S1alPvIQ4GSjlpz585lwoQJJS1oRNKVDfXjtanKJFukk+iXAoekvM+N5pVw92XuPtDduwLXR/O+in4ujX4uBGYAXfc8bBHZHdlQPx7XfbTaLJ1EPxNoZ2ZtzWwfYDBQqvWMmTU3s+J9XQeMi+Y3MbN6xesAvYG5mQpeRHZNtjQprC1VJtmi0kTv7luAy4CngfeAx9x9jpmNNLPiVjR9gHlm9gFwIFB8EXYkUGhmbwEvALe6uxK9SEyyqX5cqk9aPWPdfSowtcy8X6VMTwImlbPdf4BOexijiGTIqFGlxz4H1Y/XBonrGVsVTjrpJJ5++ulS88aMGcPFF19c4TZ9+vShuJnoGWecwVdffbXDOjfffHNJe/aKTJ48mblzt18E/epXv+K5557blfAlS8Td2gVUP15bKdGnYciQIUycOLHUvIkTJzJkyJC0tp86dSr777//bh27bKIfOXIkp5566m7tS+KTDa1diql+vPZRok/DoEGDeOqpp0rGjSkqKmLZsmWccMIJXHzxxeTn53PUUUdx0003lbt9Xl4en3/+OQCjRo2iffv2fOMb3ygZyhhCG/ljjz2Wzp07c/bZZ7N+/Xr+85//MGXKFH7+85/TpUsXFixYwLBhw5g0KdSSTZ8+na5du9KpUycuuOACNm3aVHK8m266iW7dutGpUyfef//9HWLScMbVKxtau0jtVeNGr7zySpg9O7P77NIFxoypeHnTpk3p0aMH06ZNY8CAAUycOJHvfe97mBmjRo2iadOmbN26lVNOOYW3336bY445ptz9vPHGG0ycOJHZs2ezZcsWunXrRvfu3QEYOHAgF154IQA33HAD9913H5dffjlnnnkm/fv3Z9CgQaX2tXHjRoYNG8b06dNp3749P/jBD7j77ru58sorAWjevDmzZs3irrvuYvTo0fz1r38ttf0BBxzAs88+S/369fnwww8ZMmQIhYWFTJs2jX/961/873//Iycnhy+++AKAgoICrr32Ws466yw2btzIturuwljDZUtrF6mdVKJPU2r1TWq1zWOPPUa3bt3o2rUrc+bMKVXNUtbLL7/MWWedRU5ODvvttx9nnrl96J93332XE044gU6dOjFhwgTmzJmz03jmzZtH27Ztad++PQBDhw7lpZdeKlk+cGDoqNy9e/eSgdBSbd68mQsvvJBOnTpxzjnnlMSd7nDG6qy1a9TaReJU40r0Oyt5V6UBAwYwYsQIZs2axfr16+nevTsfffQRo0ePZubMmTRp0oRhw4ZVODxxZYYNG8bkyZPp3LkzDzzwADNmzNijeIuHOq5omOPU4Yy3bdumseirmFq7SJxUok9Tw4YNOemkk7jgggtKSvOrV6+mQYMGNG7cmE8//ZRp06btdB/f/OY3mTx5Mhs2bGDNmjU88cQTJcvWrFlDy5Yt2bx5MxNS7tA1atSINWvW7LCvI444gqKiIubPnw+EUShPPPHEtD+PhjOuXmrtInFSot8FQ4YM4a233ipJ9J07d6Zr16506NCBc889l969e+90+27duvH973+fzp07c/rpp3PssceWLLvllls47rjj6N27Nx06dCiZP3jwYG6//Xa6du1a6gZo/fr1uf/++znnnHPo1KkTderU4aKLLkr7s2g44+qn1i4Sl0qHKa5uGqa45tLvSSQ+ezpMsUiNlg0dlUTiVONuxorsiuKOSsW3FIo7KoGqTqT2qDEl+myrYpLSsvX3o45KIjUk0devX5+VK1dmbTKp7dydlStXZmUTTXVUEqkhVTe5ubksWbIEPWYwe9WvX5/c3Ny4w9hB69ahuqa8+SK1RY1I9HvvvTdt27aNOwypgdRRSaSGVN2I7C51VBKpISV6kT1RUKDELrWbSvQiIgmnRC9VRh2VRLKDqm6kSqijkkj2UIleqoQ6KolkDyV6qRLqqCSSPZTopUroiUoi2UOJXqrEqFGhY1IqdVQSiYcSvVQJdVQSyR5qdSNVRh2VRLKDSvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmnRJ9AGh5YRFKpw1TCaHhgESkrrRK9mfU1s3lmNt/Mri1neRszm25mb5vZDDPLTVk21Mw+jF5DMxm87EjDA4tIWZUmejOrC/wZOB3oCAwxs45lVhsNPOTuxwAjgd9G2zYFbgKOA3oAN5lZk8yFL2VpeGARKSudEn0PYL67L3T3r4GJwIAy63QEno+mX0hZ/m3gWXf/wt2/BJ4F+u552FIRDQ8sImWlk+hbAYtT3i+J5qV6CxgYTZ8FNDKzZmlui5kNN7NCMytcsWJFurFLOTQ8sIiUlalWN1cDJ5rZm8CJwFJga7obu/tYd8939/wWLVpkKKTaScMDi0hZ6bS6WQockvI+N5pXwt2XEZXozawhcLa7f2VmS4E+ZbadsQfxSho0PLCIpEqnRD8TaGdmbc1sH2AwMCV1BTNrbmbF+7oOGBdNPw18y8yaRDdhvxXNExGRalJponf3LcBlhAT9HvCYu88xs5Fmdma0Wh9gnpl9ABwIjIq2/QK4hfBlMRMYGc0TEZFqYu4edwyl5Ofne2FhYdxhiIjUKGb2hrvnl7dMQyCIiCScEr2ISMIp0YuIJJwSvUg1cYcnn4RZs+KORGobJXqRavDVV3DWWfCd70D37nDSSfDUU7BtW9yRSW2gRC9SxWbPhvz8kNhHjw6v+fOhf384+mi47z7YuDHuKCXJlOhFqtC4cdCzZ0jkL74IV10VXgsXwvjxUK8e/PjH4QExv/kNrFwZd8SSREr0IlVgwwb40Y/Cq3fvUC/fq9f25XvvHYapmDULpk+Hbt3gxhvhkEPgsstgwYL4YpfkUaIXybAFC0JSHzcObrgBnn4aDjig/HXN4OSTYepUePddGDIE7r0X2rWDs8+G116r3tglmZToRTJo8uRws3XRolAnf8stULduetsedVSory8qguuugxdeCF8YvXvD44/D1rTHgxUpTYleJAO2bIFf/CK0rGnXLlTJnHHG7u2rZcvw/IDFi+GPf4RPPgml+yOOgLvu2vFRkSKVUaIX2UPLl8Opp8L//R9cdBG88kq4ubqnGjQI9fUffgh//zs0bw6XXhqeFvarX8Gnn+75MaR2UKIX2QMvvghdu8LMmfDww3D33aElTSbVrQuDBoX6+ldegRNOCC102rSBCy+E997L7PEkeZToRXaDeyjBn3IKNG4M//sfnHde1R7TLNTX//OfMG8eXHBBaKLZsWPoiPXiiyEukbKU6EV2UXEv11/8AgYODKX5o4+u3hjatQv19R9/DL/+dfii6dMHjj0WJk4M9wxEiinRi+yC1F6uY8bAo49Co0bxxdOiRaivX7QoPBt47drQRPOww+COO2DNmvhik+yhRC+SprK9XK+4IlSnZIN99w319XPnwpQp4Wbwz34WOmD94hewdGmlu5AEU6KXKvXkk2Fsl48+ijuS3VdZL9dsUqfO9vr611+Hvn3D+c/Lgx/8AN56K+4IJQ56lKBUiSVL4PLLQweiYr17h27/3/seNGsWX2y7YsGC0OJl9uzQy/Xmm9PvAJUtiorgD38IPW7XrYPTTgul/V69YL/94o4uXu7w+efw2Wfh6swsfFnu7vSebr8ndvYoQSV6yaht20ITw+uuCzcEf/3r0NnnscdCC5E5c8I4L6efHlqp9O8fqh2y0eTJMGxY+EccP373O0Bliy+/DPX4f/hD6IQF0LQpHHootG27/VX8vk0b2GefeGPOhLVrwxVl6mvhwu3T69bFHeF2xx+/+8NeKNFXkwkT4PrrQ0uI1q1D78aCgrijqj7vvgvDh4c/1NNOCwn/sMO2L3eHt98OSfORR2DZslCiPPvskPRPPDE7SstbtsAvfwm33x5uvP7975npAJUtvv4apk0LTTRTE9+iRWFZMTNo1ariL4KWLcOXYNw2bw69iFOTd2oyX7Gi9PoNG+74WQ48MHzebdvC36l7PNOtWoX/od2hRF8NJkwIv6DU7uk5OaEElfRkv3Fj6MBz222w//6htUdBwc4vRbduhRkzQtL/xz9C65BWreDcc0PSP+aYagu/lE8+gcGD4aWXQi/XMWMy3wEqW23bFr58y5Z4i6eXLSvdTr9evVDqr+iLoEmTzMTlHnoBl42r+P3ixaUf4LLXXiGusvEUv5o3z56b6JmkRF8N8vJCiaisNm1CHWlSzZgRvuA+/DDc7Pvd78I/0q7YsAGeeCIk/WnTQon66KNDwj/33NBypDq8+GJI8qtXwz33VH0HqJpm06bwN15RyfnLL0uv37hxxV8CeXlQv/72dVevrjiRFxWFv5FUBx1Uet+p061ahWRf2yjRV4M6dcrvlVh8OZg0X3wBP/95aHJ46KEhMZ566p7v9/PPt9fnv/ZaOH8nnhiuEAYNClcMmeYeqml++Us4/HCYNKn6O0AlwapVFV8NFBXt+BStli1DP4AlS8LfU6r99tv5l0S23teJkxJ9NagtJXr30EnoiivC05Cuvjp02MnJyfyxFiwIdfnjx8MHH4Sqgv79Q0n79NMzU6Xy1Vfhhuu//gXnnBOGCY6zA1RSbdsWql/Kltg/+yxcsZVX7ZPE6pWqtLNEj7tn1at79+5eE40f756TU3xrJbxycsL8pPjoI/fTTw+f7dhj3WfPrp7jbtvm/vrr7ldc4X7AAeH4TZq4Dx/u/tJL7lu37t5+33zT/bDD3Pfay33MmHAckZoKKPQK8mrsib3sq6YmeveQ1Nu0cTcLP5OS5Ddvdv/978MXV4MGISlu2RJfLNOmuRcUbP9ibdPG/Ze/dJ8zJ/393Hefe/367q1aub/6apWFK1JtdpboVXWTYe7h8jQvLzuaCu6pN98MXevfeAP69QsDabVuHXdUwdq1ocpl/Hh45plQPdC1a6jaGTIk1AGXtWFDGON93Lgw8uQjj1T8mD+RmmRnVTdZ0Ao2OdzDuCKHHx7u/F9yCTz/fM0cSXD9erjmmjAa4pIloV7+iSeyJ8lDaA9dUBBa6ixdGppC1q0LV10FubnwrW/BQw9tH9hrV57lKpIkKtFniDuMGBF6HZ57buh48tRToQTZvDl897uh1cjJJ4eeodnsmWdCG/KPPgql+dtuy1yb6Oowb17o1zB+fPgM++4berU+91xyermKlKWbsVVs61b3iy8O9cVXXrn9pt7ate6TJrkPHuzesOH2m4jDhrk/+aT7xo3xxl3WZ5+5n3deiLN9e/cZM+KOaM9s2xbq3y++2L1ZM/eePcMNZZEkQnX0VWfbNvjJT+Cvfw1VHbfeWn6zsI0bQ0l50qQwjOyqVaGt8He+E0r63/52fG2D3UMVx89+Fqo5rr02tClP7dBS07mruZ4km9rRV5GtW8Pj3B56KNT5jhyZXjLZtAmmTw9d/ydPDp1FGjQIbcQHDQptxBs0qPr4AebPD9U006eH+uuxY+Goo6rn2CKSOboZWwW2bIHzzw9JfuRIuOWW9EuM9eqFOuL77oPly0NJv6Ag3Lg955zQW/Dss+Fvf6u6JwRt3hyuPjp1Co/Cu/tuePllJXmRJFKJfjds3hxuuE6aBL/9bajqyIQtW+CVV8J+//GP8CVQr16o1hk0KFTzZGIIgNdfDzdZ3347PPP0zjtDKyERqbn2uERvZn3NbJ6ZzTezHdKambU2sxfM7E0ze9vMzojm55nZBjObHb3+smcfJX6bNoVS96RJYQCvTCV5CAMx9ekDf/pTaC748suhWmXWrDBg2AEHhCuBcePC8AO7as2aMHTB8ceH7f/5z/CFoiQvknAV3aUtfgF1gQXAocA+wFtAxzLrjAUujqY7AkXRdB7wbmXHSH1lc6ubDRvc+/ULrVL++MfqO+7Wre7//a/71Ve75+WF49et637aae733OP+6aeV7+OJJ9wPOST02r30UvdVq6o+bhGpPuyk1U06JfoewHx3X+juXwMTgQFlvy+A4oeSNQaW7e4XT7basAEGDAht4//yl9C7srrUqQPHHRdGWFy4EAoLQwufjz4KLX5atoSTToI//zmMGZ7qk0/Co/u+853QyufVV8MVQ21/hJxIbVJpHb2ZDQL6uvuPo/fnA8e5+2Up67QEngGaAA2AU939DTPLA+YAHwCrgRvc/eVyjjEcGA7QunXr7ovKGwYyRuvWwZlnwgsvhGaUF1wQd0SBO7zzTqhGmjQJ3nsv3BDu3TvczN1nn9BMcuNGuPHGMKxwEh4NJyI72qPmlWkm+p9F+/qdmfUE7gOOBvYGGrr7SjPrDkwGjnL31RUdL9tuxq5ZE8Z4efVVePDB7H4Yxdy5oc590qRwoxVCnf8990D79rGGJiJVbGeJPp3nsCwFUp/xkxvNS/UjoC+Au79mZvWB5u7+GbApmv+GmS0A2gPZk8l3YtWq0Kb99ddDl/rBg+OOaLMwQgkAAAnvSURBVOc6dgyvG28M47cvWRKqdNRRSKR2S6eOfibQzszamtk+wGBgSpl1PgZOATCzI4H6wAoza2FmdaP5hwLtgIWZCr4qffllGBRr5swwoFe2J/my2rcP4+ooyYtIpSV6d99iZpcBTxNa4Ixz9zlmNpJwl3cKcBVwr5mNINyYHebubmbfBEaa2WZgG3CRu39RwaGyxsqVIcm/806oCjnzzLgjEhHZfeowVcaKFeHZp/PmweOPa5RDEakZ9rSOvtZYvjw8jGLhwjD2+mmnxR2RiMieU6KPLFsW6rQXL4apU8NNTBGRJFCiJyT3k08OJfp//xtOOCHuiEREMqfWJ/qiopDkV64Mo0j27Bl3RCIimVWrE/2CBSHJr14dHjN37LFxRyQiknm1NtF/8EFI8hs2hHHgu3aNOyIRkapRKxP9e++FJL91axi/5phj4o5IRKTq1LonTL3zDpx4YhgQbMYMJXkRSb5alehnzw7NJvfeG158MYwLIyKSdLUm0RcWhuqanJyQ5I84Iu6IRESqR61I9P/9b+jx2rhxSPKHHx53RCIi1Sfxif6VV8IAZS1ahCTftm3cEYmIVK9EJ/oZM6BvXzj44JDkW7eOOyIRkeqX2ET/3HNh5Mk2bULCb9Uq7ohEROKRyET/739D//7Qrl1I8gcdFHdEIiLxSVyif+IJGDAgNJ18/vlQNy8iUpslKtE//jgMHAidO8P06dCsWdwRiYjELzGJ/v334XvfCwOTPfssNGkSd0QiItkhMYm+Qwe47z54+unQXl5ERIJEDWo2dGjcEYiIZJ/ElOhFRKR8SvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmnRC8iknBpJXoz62tm88xsvpldW87y1mb2gpm9aWZvm9kZKcuui7abZ2bfzmTwIiJSuUofPGJmdYE/A6cBS4CZZjbF3eemrHYD8Ji7321mHYGpQF40PRg4CjgYeM7M2rv71kx/EBERKV86JfoewHx3X+juXwMTgQFl1nFgv2i6MbAsmh4ATHT3Te7+ETA/2p+IiFSTdBJ9K2Bxyvsl0bxUNwPnmdkSQmn+8l3YFjMbbmaFZla4YsWKNEMXEZF0ZOpm7BDgAXfPBc4AHjaztPft7mPdPd/d81u0aJGhkEREBNJ7OPhS4JCU97nRvFQ/AvoCuPtrZlYfaJ7mtiIiUoXSKXXPBNqZWVsz24dwc3VKmXU+Bk4BMLMjgfrAimi9wWZWz8zaAu2A1zMVvIiIVK7SEr27bzGzy4CngbrAOHefY2YjgUJ3nwJcBdxrZiMIN2aHubsDc8zsMWAusAW4VC1uRESql4V8nD3y8/O9sLAw7jBERGoUM3vD3fPLW6aesSIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJp0QvIpJwSvQiIgmXVqI3s75mNs/M5pvZteUsv8PMZkevD8zsq5RlW1OWTclk8CIiUrm9KlvBzOoCfwZOA5YAM81sirvPLV7H3UekrH850DVlFxvcvUvmQhYRkV2RTom+BzDf3Re6+9fARGDATtYfAvwtE8GJiMieSyfRtwIWp7xfEs3bgZm1AdoCz6fMrm9mhWb2XzP7bgXbDY/WKVyxYkWaoYuISDoyfTN2MDDJ3bemzGvj7vnAucAYMzus7EbuPtbd8909v0WLFhkOSUSkdksn0S8FDkl5nxvNK89gylTbuPvS6OdCYAal6+9FRKSKpZPoZwLtzKytme1DSOY7tJ4xsw5AE+C1lHlNzKxeNN0c6A3MLbutiIhUnUpb3bj7FjO7DHgaqAuMc/c5ZjYSKHT34qQ/GJjo7p6y+ZHAPWa2jfClcmtqax0REal6Vjovxy8/P98LCwvjDkNEpEYxszei+6E7UM9YEZGEU6IXEUk4JXoRkYRTohcRSbjEJPoJEyAvD+rUCT8nTIg7IhGR7FBp88qaYMIEGD4c1q8P7xctCu8BCgrii0tEJBskokR//fXbk3yx9evDfBGR2i4Rif7jj3dtvohIbZKIRN+69a7NFxGpTRKR6EeNgpyc0vNycsJ8EZHaLhGJvqAAxo6FNm3ALPwcO1Y3YkVEICGtbiAkdSV2EZEdJaJELyIiFVOiFxFJOCV6EZGEU6IXEUk4JXoRkYTLuidMmdkKYFHcceyh5sDncQeRRXQ+StP52E7norQ9OR9t3L1FeQuyLtEngZkVVvRIr9pI56M0nY/tdC5Kq6rzoaobEZGEU6IXEUk4JfqqMTbuALKMzkdpOh/b6VyUViXnQ3X0IiIJpxK9iEjCKdGLiCScEn0GmdkhZvaCmc01szlmdkXcMcXNzOqa2Ztm9mTcscTNzPY3s0lm9r6ZvWdmPeOOKU5mNiL6P3nXzP5mZvXjjqk6mdk4M/vMzN5NmdfUzJ41sw+jn00ycSwl+szaAlzl7h2B44FLzaxjzDHF7QrgvbiDyBJ/AP7t7h2AztTi82JmrYCfAvnufjRQFxgcb1TV7gGgb5l51wLT3b0dMD16v8eU6DPI3T9x91nR9BrCP3KreKOKj5nlAv2Av8YdS9zMrDHwTeA+AHf/2t2/ijeq2O0F7GtmewE5wLKY46lW7v4S8EWZ2QOAB6PpB4HvZuJYSvRVxMzygK7A/+KNJFZjgGuAbXEHkgXaAiuA+6OqrL+aWYO4g4qLuy8FRgMfA58Aq9z9mXijygoHuvsn0fRy4MBM7FSJvgqYWUPgH8CV7r467njiYGb9gc/c/Y24Y8kSewHdgLvdvSuwjgxdltdEUd3zAMIX4MFAAzM7L96osouHtu8Zaf+uRJ9hZrY3IclPcPfH444nRr2BM82sCJgInGxm4+MNKVZLgCXuXnyFN4mQ+GurU4GP3H2Fu28GHgd6xRxTNvjUzFoCRD8/y8ROlegzyMyMUAf7nrv/Pu544uTu17l7rrvnEW6yPe/utbbE5u7LgcVmdkQ06xRgbowhxe1j4Hgzy4n+b06hFt+cTjEFGBpNDwX+lYmdKtFnVm/gfELpdXb0OiPuoCRrXA5MMLO3gS7A/4s5nthEVzaTgFnAO4RcVKuGQzCzvwGvAUeY2RIz+xFwK3CamX1IuOq5NSPH0hAIIiLJphK9iEjCKdGLiCScEr2ISMIp0YuIJJwSvYhIwinRi4gknBK9iEjC/X92nwbdNMuTHwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5fn/8fdN2ERAZXEjbFqEFlkCAVQUUVBBLChSK8Ul0mqhrmBLqVTh0vptv5Vvf2irbXHBDQsuleJWrQvFrUpYioBYkYJGUQFlExEI9++PZwJJyEYymZM5+byuKxczZ86cc8+EfOaZ5zznOebuiIhI+qsTdQEiIpIcCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqUyMyeM7NLk71ulMxsjZkNrIbtupl9K3H7T2Z2Y0XWrcR+RpnZC5Wts4zt9jezvGRvV1KvbtQFSPKY2bZCdxsB3wD5ifs/dveZFd2Wuw+ujnXjzt3HJGM7ZtYO+C9Qz913J7Y9E6jw71BqHwV6jLh744LbZrYG+JG7v1h8PTOrWxASIhIf6nKpBQq+UpvZz83sU2CGmR1mZk+b2Xoz+zJxO7PQc+aZ2Y8St3PM7DUzm5pY979mNriS67Y3s/lmttXMXjSzO83s4VLqrkiNt5jZ64ntvWBmLQo9frGZrTWzjWY2qYz3p4+ZfWpmGYWWnWdmSxO3e5vZm2a2yczWmdkfzKx+Kdu638x+Vej+zxLP+cTMRhdbd4iZLTazLWb2kZlNKfTw/MS/m8xsm5mdWPDeFnr+SWa2wMw2J/49qaLvTVnM7NuJ528ys+VmNrTQY2eb2YrENj82s58mlrdI/H42mdkXZvaqmSlfUkxveO1xJNAMaAtcQfjdz0jcbwN8DfyhjOf3Ad4DWgC/Be41M6vEuo8AbwPNgSnAxWXssyI1/gC4DDgcqA8UBMx3gD8mtn90Yn+ZlMDd3wK+Ak4vtt1HErfzgXGJ13MiMAD4SRl1k6hhUKKeM4AOQPH++6+AS4BDgSHAWDM7N/FYv8S/h7p7Y3d/s9i2mwHPAHckXtvvgGfMrHmx17Dfe1NOzfWAp4AXEs+7GphpZh0Tq9xL6L5rAhwPvJxYfj2QB7QEjgBuADSvSIop0GuPPcBkd//G3b92943u/oS7b3f3rcCtwKllPH+tu9/t7vnAA8BRhD/cCq9rZm2AXsBN7r7T3V8D5pa2wwrWOMPd/+PuXwOPAt0Ty0cAT7v7fHf/Brgx8R6U5i/ASAAzawKcnViGuy9093+5+253XwP8uYQ6SnJBor5l7v4V4QOs8Oub5+7vuPsed1+a2F9FtgvhA+B9d38oUddfgJXAdwutU9p7U5YTgMbAbxK/o5eBp0m8N8Au4Dtm1tTdv3T3RYWWHwW0dfdd7v6qa6KolFOg1x7r3X1HwR0za2Rmf050SWwhfMU/tHC3QzGfFtxw9+2Jm40PcN2jgS8KLQP4qLSCK1jjp4Vuby9U09GFt50I1I2l7YvQGh9uZg2A4cAid1+bqOO4RHfCp4k6/ofQWi9PkRqAtcVeXx8zeyXRpbQZGFPB7RZse22xZWuBVoXul/belFuzuxf+8Cu83fMJH3ZrzeyfZnZiYvltwCrgBTNbbWYTK/YyJJkU6LVH8dbS9UBHoI+7N2XfV/zSulGSYR3QzMwaFVrWuoz1q1LjusLbTuyzeWkru/sKQnANpmh3C4Sum5VAh0QdN1SmBkK3UWGPEL6htHb3Q4A/Fdpuea3bTwhdUYW1AT6uQF3lbbd1sf7vvdt19wXuPozQHTOH0PLH3be6+/XufgwwFBhvZgOqWIscIAV67dWE0Ce9KdEfO7m6d5ho8eYCU8ysfqJ1990ynlKVGh8HzjGzkxMHMG+m/P/vjwDXEj44HitWxxZgm5l1AsZWsIZHgRwz+07iA6V4/U0I31h2mFlvwgdJgfWELqJjStn2s8BxZvYDM6trZt8HvkPoHqmKtwit+QlmVs/M+hN+R7MSv7NRZnaIu+8ivCd7AMzsHDP7VuJYyWbCcYeyurikGijQa69pwEHABuBfwN9TtN9RhAOLG4FfAbMJ4+VLUuka3X05cCUhpNcBXxIO2pWloA/7ZXffUGj5TwlhuxW4O1FzRWp4LvEaXiZ0R7xcbJWfADeb2VbgJhKt3cRztxOOGbyeGDlyQrFtbwTOIXyL2QhMAM4pVvcBc/edhAAfTHjf7wIucfeViVUuBtYkup7GEH6fEA76vghsA94E7nL3V6pSixw403ELiZKZzQZWunu1f0MQiTu10CWlzKyXmR1rZnUSw/qGEfpiRaSKdKaopNqRwF8JByjzgLHuvjjakkTiQV0uIiIxoS4XEZGYiKzLpUWLFt6uXbuodi8ikpYWLly4wd1blvRYZIHerl07cnNzo9q9iEhaMrPiZwjvpS4XEZGYUKCLiMSEAl1EJCY0Dl2kFtm1axd5eXns2LGj/JUlUg0bNiQzM5N69epV+DkKdJFaJC8vjyZNmtCuXTtKvz6JRM3d2bhxI3l5ebRv377Cz0urLpeZM6FdO6hTJ/w7U5fLFTkgO3bsoHnz5grzGs7MaN68+QF/k0qbFvrMmXDFFbA9cWmEtWvDfYBRo0p/nogUpTBPD5X5PaVNC33SpH1hXmD79rBcRETSKNA//PDAlotIzbNx40a6d+9O9+7dOfLII2nVqtXe+zt37izzubm5uVxzzTXl7uOkk05KSq3z5s3jnHPOScq2UiVtAr1N8Yt3lbNcRKou2cetmjdvzpIlS1iyZAljxoxh3Lhxe+/Xr1+f3bt3l/rc7Oxs7rjjjnL38cYbb1StyDSWNoF+663QqFHRZY0aheUiknwFx63WrgX3fcetkj0YIScnhzFjxtCnTx8mTJjA22+/zYknnkhWVhYnnXQS7733HlC0xTxlyhRGjx5N//79OeaYY4oEfePGjfeu379/f0aMGEGnTp0YNWoUBbPLPvvss3Tq1ImePXtyzTXXlNsS/+KLLzj33HPp2rUrJ5xwAkuXLgXgn//8595vGFlZWWzdupV169bRr18/unfvzvHHH8+rr76a3DesDGlzULTgwOekSaGbpU2bEOY6ICpSPco6bpXsv7u8vDzeeOMNMjIy2LJlC6+++ip169blxRdf5IYbbuCJJ57Y7zkrV67klVdeYevWrXTs2JGxY8fuN2Z78eLFLF++nKOPPpq+ffvy+uuvk52dzY9//GPmz59P+/btGTlyZLn1TZ48maysLObMmcPLL7/MJZdcwpIlS5g6dSp33nknffv2Zdu2bTRs2JDp06dz1llnMWnSJPLz89le/E2sRmkT6BD+EynARVIjlcetvve975GRkQHA5s2bufTSS3n//fcxM3bt2lXic4YMGUKDBg1o0KABhx9+OJ999hmZmZlF1undu/feZd27d2fNmjU0btyYY445Zu/47pEjRzJ9+vQy63vttdf2fqicfvrpbNy4kS1bttC3b1/Gjx/PqFGjGD58OJmZmfTq1YvRo0eza9cuzj33XLp3716l9+ZApE2Xi4ikViqPWx188MF7b994442cdtppLFu2jKeeeqrUsdgNGjTYezsjI6PE/veKrFMVEydO5J577uHrr7+mb9++rFy5kn79+jF//nxatWpFTk4ODz74YFL3WRYFuoiUKKrjVps3b6ZVq1YA3H///UnffseOHVm9ejVr1qwBYPbs2eU+55RTTmFm4uDBvHnzaNGiBU2bNuWDDz6gS5cu/PznP6dXr16sXLmStWvXcsQRR3D55Zfzox/9iEWLFiX9NZRGgS4iJRo1CqZPh7ZtwSz8O3169Xd7TpgwgV/84hdkZWUlvUUNcNBBB3HXXXcxaNAgevbsSZMmTTjkkEPKfM6UKVNYuHAhXbt2ZeLEiTzwwAMATJs2jeOPP56uXbtSr149Bg8ezLx58+jWrRtZWVnMnj2ba6+9NumvoTSRXVM0OzvbdYELkdR69913+fa3vx11GZHbtm0bjRs3xt258sor6dChA+PGjYu6rP2U9Psys4Xunl3S+mqhi0itc/fdd9O9e3c6d+7M5s2b+fGPfxx1SUmRVqNcRESSYdy4cTWyRV5VaqGLiMSEAl1EJCYU6CIiMaFAFxGJCQW6iKTMaaedxvPPP19k2bRp0xg7dmypz+nfvz8FQ5zPPvtsNm3atN86U6ZMYerUqWXue86cOaxYsWLv/ZtuuokXX3zxQMovUU2aZleBLiIpM3LkSGbNmlVk2axZsyo0QRaEWRIPPfTQSu27eKDffPPNDBw4sFLbqqkU6CKSMiNGjOCZZ57ZezGLNWvW8Mknn3DKKacwduxYsrOz6dy5M5MnTy7x+e3atWPDhg0A3HrrrRx33HGcfPLJe6fYhTDGvFevXnTr1o3zzz+f7du388YbbzB37lx+9rOf0b17dz744ANycnJ4/PHHAXjppZfIysqiS5cujB49mm+++Wbv/iZPnkyPHj3o0qULK1euLPP1RT3Nrsahi9RS110HS5Ykd5vdu8O0aaU/3qxZM3r37s1zzz3HsGHDmDVrFhdccAFmxq233kqzZs3Iz89nwIABLF26lK5du5a4nYULFzJr1iyWLFnC7t276dGjBz179gRg+PDhXH755QD88pe/5N577+Xqq69m6NChnHPOOYwYMaLItnbs2EFOTg4vvfQSxx13HJdccgl//OMfue666wBo0aIFixYt4q677mLq1Kncc889pb6+qKfZVQtdRFKqcLdL4e6WRx99lB49epCVlcXy5cuLdI8U9+qrr3LeeefRqFEjmjZtytChQ/c+tmzZMk455RS6dOnCzJkzWb58eZn1vPfee7Rv357jjjsOgEsvvZT58+fvfXz48OEA9OzZc++EXqV57bXXuPjii4GSp9m944472LRpE3Xr1qVXr17MmDGDKVOm8M4779CkSZMyt10RaqGL1FJltaSr07Bhwxg3bhyLFi1i+/bt9OzZk//+979MnTqVBQsWcNhhh5GTk1PqtLnlycnJYc6cOXTr1o3777+fefPmVanegil4qzL97sSJExkyZAjPPvssffv25fnnn987ze4zzzxDTk4O48eP55JLLqlSrWqhi0hKNW7cmNNOO43Ro0fvbZ1v2bKFgw8+mEMOOYTPPvuM5557rsxt9OvXjzlz5vD111+zdetWnnrqqb2Pbd26laOOOopdu3btnfIWoEmTJmzdunW/bXXs2JE1a9awatUqAB566CFOPfXUSr22qKfZVQtdRFJu5MiRnHfeeXu7Xgqmm+3UqROtW7emb9++ZT6/R48efP/736dbt24cfvjh9OrVa+9jt9xyC3369KFly5b06dNnb4hfeOGFXH755dxxxx17D4YCNGzYkBkzZvC9732P3bt306tXL8aMGVOp11VwrdOuXbvSqFGjItPsvvLKK9SpU4fOnTszePBgZs2axW233Ua9evVo3LhxUi6EoelzRWoRTZ+bXjR9rohILaVAFxGJCQW6SC0TVTerHJjK/J4U6CK1SMOGDdm4caNCvYZzdzZu3EjDhg0P6HkVGuViZoOA24EM4B53/02xx3OA24CPE4v+4O6ln04lIpHIzMwkLy+P9evXR12KlKNhw4ZkZmYe0HPKDXQzywDuBM4A8oAFZjbX3YufxjXb3a86oL2LSErVq1eP9u3bR12GVJOKdLn0Bla5+2p33wnMAoZVb1kiInKgKhLorYCPCt3PSywr7nwzW2pmj5tZ65I2ZGZXmFmumeXqK5+ISHIl66DoU0A7d+8K/AN4oKSV3H26u2e7e3bLli2TtGsREYGKBfrHQOEWdyb7Dn4C4O4b3f2bxN17gJ7JKU9ERCqqIoG+AOhgZu3NrD5wITC38ApmdlShu0OBd5NXooiIVES5o1zcfbeZXQU8Txi2eJ+7Lzezm4Fcd58LXGNmQ4HdwBdATjXWLCIiJdDkXCIiaUSTc4mI1AIKdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGJCgS4iEhMKdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITKRdoG/eDI88EnUVIiI1T9oF+tSpMGoUTJ8edSUiIjVL3agLOFA33giLFsGYMdC0KVx4YdQViYjUDGnXQq9fHx57DE45BS6+GJ55JuqKRERqhgoFupkNMrP3zGyVmU0sY73zzczNLDt5Je6vUSN46ino1g1GjIB586pzbyJSFatXw6OPQm4ufPFF1NXEW7ldLmaWAdwJnAHkAQvMbK67ryi2XhPgWuCt6ii0uKZN4e9/h1NPhe9+F15+GXr1SsWeRaSiVq+G7Gz48st9yw49FI45Jvwce2zRf1u3hrpp1xFcc1TkresNrHL31QBmNgsYBqwott4twP8CP0tqhWVo0QJeeCF0vwwaBPPnQ+fOqdq7iJRl2zY499xw+6WXYMuWEPCrV8MHH8DSpfC3v8GuXfueU7cutG1beuA3bRrNa0kXFQn0VsBHhe7nAX0Kr2BmPYDW7v6MmZUa6GZ2BXAFQJs2bQ682pKKawUvvggnnwxnnAGvvRZ+8SISHXe47DJYvhyeew5OP73k9fLz4eOP94V84cB//HHYuLHo+s2b7x/yBbdbtYI6aXdUMLmq/OXGzOoAvwNyylvX3acD0wGys7O9qvsucMwx8I9/QL9+MHAgvPpq+OWKSDT+939DIN92G5x5ZunrZWRAmzbhp3///R/fvLloyBfcfvvtMDgiP3/fuvXrQ/v2Jbfu27eHgw9O+suscSoS6B8DrQvdz0wsK9AEOB6YZ2YARwJzzWyou+cmq9DydO4c+tRPPz38B/rnP0OXjIik1nPPwQ03wMiRcP31VdvWIYdAVlb4KW73bvjww5ID/403wodBYUccAT/8Idx8c/ggiSNzL7uhbGZ1gf8AAwhBvgD4gbsvL2X9ecBPywvz7Oxsz81Nft7Pmxf607t0Cf126nMTSZ333w+DE9q3h9dfDyPSouAeDsQWb9XPmRO6ZmfNgmbNoqmtqsxsobuXOJKw3Ba6u+82s6uA54EM4D53X25mNwO57j43ueVWTf/+4aveeeeF0S9//zscdFDUVYnE39atMGxYOLD55JPRhTmAWQjsZs2Kjn675x648sow8ubJJ8PQ5zgpt4VeXaqrhV7gL38JUwQMHhx+cfXrV9uuRGq9PXvg/PPD+SEvvFD6QdCa4K23YPjw0IK/7770O9u8rBZ6bI8JjxwJf/oTPPssXHJJ0YMnIpJcv/pV6M74v/+r2WEO0KcPLFwIPXqEnPjZz0J/fBzENtABrrgCfvtbmD0bxo4N/Woiklxz58LkyaHhdM01UVdTMUceGU5G/MlPwoR/gwfvP0QyHcU60CF8+t5wA9x9N0yYoFAXSaaVK+Gii0Kf9J/+FPqu00X9+nDnnXDvveGkxOxsWLIk6qqqJvaBDuHr4JVXhk/iX/866mpE4mHz5nAQ9KCD4K9/Td/BB6NHh3NXdu2Ck05K7+st1IpAN4M77ggtiUmT4A9/iLoikfS2Z08YdLB6dRhV1rp1+c+pyXr3Dv3q2dnhdV1/fXr2q9eKQIdwSvCMGaFFcfXV8OCDUVckkr4mTw5TV99+e5hLKQ6OOCJMI3LVVfC738FZZ8GGDVFXdWBqTaBDGB87axYMGBC+Zj35ZNQViaSfJ54I3Zg//GEYbBAn9evD738fGn+vvx5a7IsXR11VxdWqQAdo2DAMr8rODuNPX3wx6opE0seyZXDppXDCCeGAYjodBD0QOTmhXz0/P/Srz5wZdUUVU+sCHaBx4zA+vWPHML3nm29GXZFIzffll+HvpUmT0Epv0CDqiqpXr17hohy9e4fjb+PH1/x+9VoZ6BBOCX7hBTjqKDj7bPj3v6OuSKTmys8PJ+F8+GEY0XL00VFXlBoF/epXXw3/7/+Fif/Wr4+6qtLV2kCHcHLBiy+GFvuZZ4aJhURkf5MmwfPPw113wYknRl1NatWrF0bJ3X9/mMUxOztcqL4mqtWBDuHqKP/4RzjhaOBA+Oij8p8jUpvMnh3mNx8zBn70o6iric6ll4YL6LhD377w0ENRV7S/Wh/oAJ06hdbHpk0h1D//POqKypefD6+8Er4KTp4Mn3wSdUUSR//+d7jyUN++YYhibZedHfrV+/QJUx1cd13RS+hFTYGekJUVxtV+9FEYf7ppU9QV7S8/P1y448orwxWZTj89nLZ8yy3hm8YPfhBmkhNJhg0bwkHQZs3CyUOasTQ4/PDwrf7aa8OH3Jln1pxGoAK9kJNPDmPTly+HIUPgq6+iriickffaa2HSo9atw3zvM2aEy+099lj4o1u1KrTUn3kmDCc74YQwffDOnVFXL+lq9274/vdh3brwN3HkkVFXVLPUqwfTpoUTFP/1r9ByX7gw6qoAd4/kp2fPnl5TPfaYe5067mee6b5jR+r3n5/v/vrr7tde63700e7g3rCh+/nnu8+e7b5tW8nP27LF/fe/dz/uuPCco45yv+UW988/T239kv7Gjw//h2bMiLqSmm/hQvc2bdwbNHB/4IHq3x/hwkIl5qoCvRT33RfenfPPd9+1q/r3t2eP+7/+5T5unHtmZth3gwbu557r/sgj7lu3Vnxb+fnuzz7rftZZ+7aTk+O+eHH11S/x8dBD4f/NNddEXUn6+Pxz99NOC+/b1Ve779xZfftSoFfStGnhHcrJCSFZ4OGH3du2dTcL/z78cOW2v2eP+9tvu//0p+ETHtzr13cfOjRsc/Pmqr+Gd991/8lP3Bs1Ctvv18/9iSdS8yEl6Sc3N3wbPPXU6g2lONq1KzTICv7OPvusevajQK+CKVP2tVb27AlBWxCOBT+NGlU81PfsCX80Eya4t2sXnl+vnvuQIe4PPui+aVP1vI4vv3SfOnXfPtu2df/tb92/+KJ69ifp57PP3Fu3Do0LddNV3sMPhw/FzEz3BQuSv30FehXs2eN+3XXhnZo8OQRh4TAv+GnbtuxtLF7s/otfuB97bFi/bl33wYNDH2UqQ3X3bve//tW9f/99H0ZjxrgvX566GqTm2bkztCobNgx9wlI1ixaFTGjQIPnHIRToVbRnj/vo0SUHecGP2f7P+fe/3SdNcu/QIayTkREOtN57r/vGjdG8lsKWLAmvq0GDUN8ZZ7g//XTR7iU5cPn54QB1OrnqqvB/oLLdh7K/9evdTz89vK9XXZW8LiwFehLs3u0+YkTpgV7QQl+2zP2mm9w7dQrL69RxHzjQffr08AuuiT7/3P3WW/eNqPnWt9xvvz39QikK33wTvn3dd184GHbyye5NmoQP+AEDwgHGr76KusqyFQwAuP76qCuJn1279o0YOuUU908/rfo2ywp0C4+nXnZ2tufm5kay78rauTPMwLZ0adHlDRuGCb5WroQVK8LFNE49NYzjPe+8cCJCOti1K8yid/vtYWxtkyZh3virr4Zjj426uuht2xbOnFy8eN/P8uX7xvs3agTduoWT1A45JJwLsGYNNG0a/i9cdlk4R6AmTTn71lvhnIZ+/eC558I1AyT5HnkkTJvQrFmY3Kx378pvy8wWunt2iY8p0A/M9u3hD/Y//wn369ULQWgW/iguuACGD0//EzHefjtMSPToo+EkkyFDwplxAwbUrECqLhs2FA3uRYvC5G0Ffy7Nm4f/B4V/OnSAjIx929izJ1x8eMaMcKbl9u1hyuacHLj44nC2b5Q+/RR69gzT4C5YEF6TVJ8lS8KZt+vWhXlgLrigcttRoCfZ5s1w2mnhF3TyyeEXc/75YSreuFm3Dv74x3BF9/XroXPncNbqRReFFmm6cw9TwhYO78WLIS9v3zpt2uwf3pmZB/bBtnVrOLN3xoxw5m+dOuGU8csug6FDw7e8VNq5c9//4TffhK5dU7v/2mrDhnClp5tvDt/mKkOBXg2+/hq2bAnzJdcGO3aEWfduvz0E3mGHweWXh3ll2rSJurqKyc+H997bP7y//DI8XqdOaEEXDu7u3ZPfcl21KkzF+sAD4YPjsMPCXOOXXRZazKn4BjRmDPz5z+F3WtmWokRDgS5J4x5amLffHub4MAvHCa65JrQ4MjJCP2xGRviJqntmxw54552iwb10afgghtDN0KVL0fDu2jW13zry8+Hll0Or/cknQ83HHx+6ZC66qPoaC3/+cwj0iRPh17+unn1I9VGgS7X48MNwXcm7797Xyi3OrGjAF75d3v3KrJufHw5UvvtuuA3hAGX37kXDu1OncPyjpti0KbSWZ8wIByozMsKB9ssuC8cvkjXT4euvh66WAQPg6aeL9vlLelCgS7X66qtw5P7zz8MB1Pz8fT+pvu++f7dJ+/bpdSB3xYrQHfPgg+HAZYsWMGpUCPfK9rsCfPxxmBWwceNw0Puww5JXs6SOAl0kDe3eHS68cv/98Le/hdFUWVmhS+YHPwhBX1E7doShtCtWhCGpnTtXV9VS3coKdM2HLlJD1a0bulseeyyMNrrjjvBN49prw0WaR4wI3SblXYnePRy8fvvt0OpXmMeXAl0kDTRvHk7wWrgwnNx05ZXh6lXf/W648MmECaH1XZK77oL77oMbbwwHsCW+1OUikqZ27oRnnw0HUp95JhxD6N079LVfeCEcemgI/YEDYdCg0G1TR024tKc+dJGY++wzmDkzhPuyZWFY5nnnwUsvhdPN33orjPaR9FflPnQzG2Rm75nZKjObWMLjY8zsHTNbYmavmdl3qlq0iFTcEUfA+PFhrP2CBWHekOefD634OXMU5rVFuS10M8sA/gOcAeQBC4CR7r6i0DpN3X1L4vZQ4CfuPqis7aqFLlK9vvkmDClt1izqSiSZqtpC7w2scvfV7r4TmAUMK7xCQZgnHAxE048jIns1aKAwr20qMllmK+CjQvfzgD7FVzKzK4HxQH3g9JI2ZGZXAFcAtEmXCUBERNJE0o55u/ud7n4s8HPgl6WsM93ds909u2XLlsnatYiIULFA/xhoXeh+ZmJZaWYB51alKBEROXAVCfQFQAcza29m9WGop4sAAAggSURBVIELgbmFVzCzDoXuDgHeT16JIiJSEeX2obv7bjO7CngeyADuc/flZnYz4dp2c4GrzGwgsAv4Eri0OosWEZH9VegKgu7+LPBssWU3Fbp9bZLrEhGRA6QTgUVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGJCgS4iEhMK9DQ2cya0axcu/NuuXbgvIrVXheZykZpn5ky44grYvj3cX7s23AcYNSq6ukQkOmqhp6lJk/aFeYHt28NyEamdFOhp6sMPD2y5iMSfAj1NlXZJVl2qVaT2UqCnqVtvhUaNii5r1CgsF5HaSYGepkaNgunToW1bMAv/Tp+uA6IitZlGuaSxUaMU4CKyj1roIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJCQW6VJmunCRSM2guF6kSXTlJpOZQC12qRFdOEqk5FOhSJbpykkjNoUCXKtGVk0RqDgW6VImunCRSc1Qo0M1skJm9Z2arzGxiCY+PN7MVZrbUzF4ys7bJL1VqIl05SaTmMHcvewWzDOA/wBlAHrAAGOnuKwqtcxrwlrtvN7OxQH93/35Z283Ozvbc3Nyq1i8iUquY2UJ3zy7psYq00HsDq9x9tbvvBGYBwwqv4O6vuHvBWId/AZlVKVhERA5cRQK9FfBRoft5iWWl+SHwXEkPmNkVZpZrZrnr16+veJUiIlKupB4UNbOLgGzgtpIed/fp7p7t7tktW7ZM5q5FRGq9ipwp+jHQutD9zMSyIsxsIDAJONXdv0lOeSIiUlEVaaEvADqYWXszqw9cCMwtvIKZZQF/Boa6++fJL1OkfJpTRmq7clvo7r7bzK4CngcygPvcfbmZ3QzkuvtcQhdLY+AxMwP40N2HVmPdIkVoThmRCgxbrC4atijJ1K5dCPHi2raFNWtSXY1I9anqsEWRGk9zyogo0CUmNKeMiAJdYkJzyogo0CUmNKeMiK5YJDEyapQCXGo3tdBFRGJCgS4iEhMKdJEk0tmqEiX1oYskic5WlaiphS6SJJMm7QvzAtu3h+UiqaBAF0kSna0qUVOgiySJzlaVqCnQRZJEZ6tK1BToIkmis1Ulagp0kSQaNSpM17tnT/g3qjDX8MnaScMWRWJGwydrL7XQRWJGwydrLwW6SMxo+GTtpUAXiZmaNHxSffmppUAXiZmaMnyyoC9/7Vpw39eXr1CvPgp0kZipKcMn1Zefeubukew4Ozvbc3NzI9m3iFS/OnVCy7w4szCsUyrHzBa6e3ZJj6mFLiLVQn35qadAF5Fqob781FOgi0i1UF9+6qkPXURiLW59+epDF5Faqyb15Vc3BbqIxFpN6ctPBQW6iMRaTenLh+ofbaPZFkUk9kaNin6myVTMgqkWuohICqRitI0CXUQkBVIxC6YCXUQkBVIx2kaBLiKSAqkYbVOhQDezQWb2npmtMrOJJTzez8wWmdluMxuRvPJEROIhFaNtyh3lYmYZwJ3AGUAesMDM5rr7ikKrfQjkAD9NXmkiIvFS3aNtKjJssTewyt1XA5jZLGAYsDfQ3X1N4rE0PJFWRCQeKtLl0gr4qND9vMQyERGpQVJ6UNTMrjCzXDPLXb9+fSp3LSISexUJ9I+B1oXuZyaWHTB3n+7u2e6e3bJly8psQkRESlGRQF8AdDCz9mZWH7gQmFu9ZYmIyIGq0HzoZnY2MA3IAO5z91vN7GYg193nmlkv4EngMGAH8Km7dy5nm+uBtVV9ARFrAWyIuogaRO/HPnovitL7UVRV3o+27l5iF0dkF7iIAzPLLW2i+dpI78c+ei+K0vtRVHW9HzpTVEQkJhToIiIxoUCvmulRF1DD6P3YR+9FUXo/iqqW90N96CIiMaEWuohITCjQRURiQoFeCWbW2sxeMbMVZrbczK6NuqaomVmGmS02s6ejriVqZnaomT1uZivN7F0zOzHqmqJkZuMSfyfLzOwvZtYw6ppSxczuM7PPzWxZoWXNzOwfZvZ+4t/DkrU/BXrl7Aaud/fvACcAV5rZdyKuKWrXAu9GXUQNcTvwd3fvBHSjFr8vZtYKuAbIdvfjCScnXhhtVSl1PzCo2LKJwEvu3gF4KXE/KRToleDu69x9UeL2VsIfbK2dgdLMMoEhwD1R1xI1MzsE6AfcC+DuO919U7RVRa4ucJCZ1QUaAZ9EXE/KuPt84Itii4cBDyRuPwCcm6z9KdCryMzaAVnAW9FWEqlpwARA8+FDe2A9MCPRBXWPmR0cdVFRcfePgamEi+CsAza7+wvRVhW5I9x9XeL2p8ARydqwAr0KzKwx8ARwnbtvibqeKJjZOcDn7r4w6lpqiLpAD+CP7p4FfEUSv1Knm0T/8DDCB93RwMFmdlG0VdUcHsaNJ23suAK9ksysHiHMZ7r7X6OuJ0J9gaFmtgaYBZxuZg9HW1Kk8oA8dy/4xvY4IeBrq4HAf919vbvvAv4KnBRxTVH7zMyOAkj8+3myNqxArwQzM0If6bvu/ruo64mSu//C3TPdvR3hYNfL7l5rW2Du/inwkZl1TCwaQKHLNdZCHwInmFmjxN/NAGrxQeKEucCliduXAn9L1oYV6JXTF7iY0Bpdkvg5O+qipMa4GphpZkuB7sD/RFxPZBLfVB4HFgHvEDKn1kwDYGZ/Ad4EOppZnpn9EPgNcIaZvU/4BvObpO1Pp/6LiMSDWugiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxMT/B2b8uFf5TVrrAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jFioB12BTVx9"
      },
      "source": [
        "val_acc이 약 87%로, 조금 증가했다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ngZtzHlc8vJj"
      },
      "source": [
        "# 3) 기온 예측 예제"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eTw2Hj8zVZSQ",
        "outputId": "247a0d9f-f7ef-4c40-dcb3-94816c7ec2c1"
      },
      "source": [
        "import zipfile\r\n",
        "!unzip /content/sample_data/jena_climate_2009_2016.csv.zip"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Archive:  /content/sample_data/jena_climate_2009_2016.csv.zip\n",
            "  inflating: jena_climate_2009_2016.csv  \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SELamR4u84Z9",
        "outputId": "534a8557-ee03-46d2-b336-f7e6fe7b1b5e"
      },
      "source": [
        "import os\r\n",
        "\r\n",
        "#data_dir = './datasets/jena_climate/'\r\n",
        "#fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')\r\n",
        "\r\n",
        "f = open('/content/jena_climate_2009_2016.csv')\r\n",
        "data = f.read()\r\n",
        "f.close()\r\n",
        "\r\n",
        "lines = data.split('\\n')\r\n",
        "header = lines[0].split(',') # column 이름 추출\r\n",
        "lines = lines[1:]\r\n",
        "\r\n",
        "print(header) \r\n",
        "print(len(header))  # column의 개수\r\n",
        "print(len(lines))  # 전체 데이터 개수"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['\"Date Time\"', '\"p (mbar)\"', '\"T (degC)\"', '\"Tpot (K)\"', '\"Tdew (degC)\"', '\"rh (%)\"', '\"VPmax (mbar)\"', '\"VPact (mbar)\"', '\"VPdef (mbar)\"', '\"sh (g/kg)\"', '\"H2OC (mmol/mol)\"', '\"rho (g/m**3)\"', '\"wv (m/s)\"', '\"max. wv (m/s)\"', '\"wd (deg)\"']\n",
            "15\n",
            "420551\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SLwnKSYRO5s0",
        "outputId": "4fd0a4de-35b9-49c0-c998-41263b58bce7"
      },
      "source": [
        "# numpy 베열로 변환\r\n",
        "import numpy as np\r\n",
        "\r\n",
        "float_data = np.zeros((len(lines),len(header)-1))\r\n",
        "print(float_data)\r\n",
        "for i, line in enumerate(lines):\r\n",
        "  values = [float(x) for x in line.split(',')[1:]]\r\n",
        "  float_data[i,:] = values\r\n",
        "\r\n",
        "print(float_data[0,:])  # 0번째 column의 모든 행 추출"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " ...\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]]\n",
            "[ 9.96520e+02 -8.02000e+00  2.65400e+02 -8.90000e+00  9.33000e+01\n",
            "  3.33000e+00  3.11000e+00  2.20000e-01  1.94000e+00  3.12000e+00\n",
            "  1.30775e+03  1.03000e+00  1.75000e+00  1.52300e+02]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 286
        },
        "id": "VAn0d79DVjva",
        "outputId": "fb76023c-e063-4500-c408-775d045b9b9d"
      },
      "source": [
        "# 시간에 따른 기온 그래프\r\n",
        "from matplotlib import pyplot as plt\r\n",
        "\r\n",
        "temp = float_data[:, 1]  # 온도(섭씨)\r\n",
        "plt.plot(range(len(temp)), temp)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f25341d7a90>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd3gVVfrHv28aPYQSasAAAqFIjTRRFERRsKw/XNFd1+66a9tFV7Gsaxfd4lrX1cW2igtWkCZdRREIIJ1AgFADCZ0ACYSc3x93bjK5mdtmzsycmbyf58mTe+fOzHnvuTPvnPOet5AQAgzDMIw/SXBbAIZhGMY+WMkzDMP4GFbyDMMwPoaVPMMwjI9hJc8wDONjWMkzDMP4GGlKnogSiWglEU3T3rcjoiVElEdEk4goRVZbDMMwTGzIHMnfD2CD7v2LAF4WQpwN4BCA2yS2xTAMw8QAyQiGIqIMAB8AeA7AWABXACgC0EIIUUZEAwE8KYS4NNJ5mjZtKjIzMy3LwzAMU5NYvnz5fiFEutFnSZLa+CeAhwA00N43AXBYCFGmvd8FoLXRgUR0J4A7AaBt27bIycmRJBLDMEzNgIi2h/vMsrmGiEYBKBRCLDdzvBDibSFEthAiOz3d8EHEMAzDmETGSP48AFcS0eUAagNIBfAKgDQiStJG8xkAdktoi2EYhokDyyN5IcQjQogMIUQmgDEA5gshfgVgAYDR2m43AZhitS2GYRgmPuz0k38YwFgiykPARj/BxrYYhmEYA2QtvAIAhBALASzUXm8F0E/m+RmGYZj44IhXhmEYH8NKnmEYxsewkmcYRml2HDiB7zcXuS2GZ5Fqk2cYhpHNBX9dAADIHz/SZUm8CY/kGYbxBCWnz7gtgidhJc8wjKd4d9E27D580m0xPAMreYZhPMO+oyV4etp63PLeUrdF8Qys5Jkaxe8/Xo4PF+e7LQZjkjPlgay5x0rKouzJBGElz9QoZqzZiyemrHNbDADAsZLTeHNhHsrLraf7rilwT8UPK3mGcYnnZ2zES7NyMXv9XrdFYXwMK3kPUVxahnsmrsCB4lK3RWEkcOJUwORQcrrcZUkYP8NK3kNMWrYT01YX4LX5eW6LwkgggQgAUC6hOltNg9wWwEOwkvcQQdttUgJf4l5kQ8FRdHh0RoX7n6bjMXbyKhel8g5Zf56FpdsOuC0GAODrVXsw9O8LDddT7vgwBxOX7HBBKmNYyXuIMu2C2rb/OGTU5mWcZeKSHThTLjBvwz4AAPF4NG7mrN/ntggAgAc+XYWtRcdx6kx1U9uc9fvw6JdrXJDKGFbyHqJMu6DmbSzEO99vdVkaxiqkgI7ffuA4ej89GzsPnqjYtuvQCRw5edpFqcKjH9t89NN2PDFlrStynCrzzjoKK3kPMX1NQcXrVbuOuCgJIwMVrG6Tlu3EoROnMeXnyuqcg19cgBH//M5FqcITfDAePHEKj3+1Fh8uDlu/mtFgJR8j324qqgjEcIuNe4+52n48rN19BKP/9SPnG4lAggJD+eAVTSGyFBwpcV6YGDh0PDDDYI+k2GElHwMLcwtx07tL8eYCdbxa3FcPkXliylrkbD+EdXvcn3FMWLQNV7y2yG0xKgiaHEIVqxus3R34fbYWHXdZkthYvFWNhdcgXlgaYyUfA4XHAn7p23V2S7c5eUrtEfKKHYcBQAnb7jPT1mPN7iMoLlUrFF4BHY+uLVMBAO3T67ksCWMXrOQ9yryNhW6LEBMqmZiWbz/ktggAKpW7AjpeidkEYy+s5ONApanZFT1buS1CTLjlJiiEwJai4hBZ1CB4Halgk1dABEP2F5di0z51BgjhECHZdE4buFS6DSt5xpd8tnwXhv392ypl49x+RocqVJXMf6px0d8W4pKX1fTw0RM68GMl73HcHPWoePHEglt9FlxQ3FJYHGVP9/huU+UDyK18RPu19aafth7AjDUFuGfiClfkCMUPqYQzx03HAgXMqqzkPcKhE6fcFsEUbuj4E6fKkKOI/V1P4dGAQl2/52i1z+ZucCeSc8WOQD99v3k/fv/xCkxbXRDlCEZPtNnhLe8vc0SOSFhW8kRUm4iWEtEqIlpHRE9p29sR0RIiyiOiSUSUYl1cd3HTJl/uzYG87SP5T5buwCdLq+YJeWDyKqzTFKn+J3M7FcSC3MCoblLOzmqfubUAur/Ym4MHN3hy6jrMWls1LfTzMzZg9L9+rEhVoSIyRvKlAIYKIXoC6AVgBBENAPAigJeFEGcDOATgNgltucJPW9z3zZ23sepFpOh6meM88sUaPPJF1Twh6wxGyoDavuCJLil5FVxcvcL7P+bjro+WV9k2cckO5Gw/hNs+yHFJquhYVvIiQNDwmaz9CQBDAXymbf8AwNVW23KLL1YGQr4/X7ELL83a6IoMZWeqjkLdXkSMFTe8a/SKS9+6fl3jQHEpCo44Wwy6LELEdAIbTn3BCQXjV6RcWkSUSEQ/AygEMAfAFgCHhRDB1ZNdAFqHOfZOIsohopyioiKjXZTizYVbHG0vJ/8gZq0tqGb2cHNBp+xMeczBWE4NUPUJo/RKXq9W9Tdg32fnYuAL850QrYJIaTFUcKcM5VYF7MleY9XOw26LUA0pSl4IcUYI0QtABoB+ALLiOPZtIUS2ECI7PT1dhji+YvRbi3HXR9U9HoLRm9NW78HSbQcdlenuiSvQ5YlZjrYZjU6Pz0TRseoeKvuOeqOKVqKWrezVeZuxfLuzv2c45ivgGeI1VHxYS50kCiEOA1gAYCCANCJK0j7KALA77IFMVMIVn75n4kr88t+LHZXlm3WRF5nydG6LTi4oBotx6HnrW2dnXmYJ2uT/MWcT/u9fzv6eKuO5BHfq6Xgp3jXpRJSmva4DYDiADQgo+9HabjcBmGK1LcYbvKhbt1Dpmn9l3ma3RQgLpxcwpu8zc9wWIT4UXCyTMZJvCWABEa0GsAzAHCHENAAPAxhLRHkAmgCYIKEtRjEmLduBzHHTK4pSA1Wr9zw9bT3WOJT7/plp6yGEQNP6tRxpTyZH2cvFkOMKLmRGIjTNgQrI8K5ZLYToLYToIYToLoR4Wtu+VQjRTwhxthDiWiGEN4yjTMxMWrajYiF63obw9tsrXncmze/y7YdQeKwUdVK856pSOyXRbREM2VBg7I6qEiplZFWxpKP37gYHWZBbiMxx090WQ1ke/rzSP/3eT1YCAOZvdDco5KVZuUolkouV+z5ZiSWK5UoHgMte+d7R9g4dPxV30NqxEjVmQXmFxWrZJzVYyYew+/BJZI6bjpU7DuGW99R2IctVII2v3ptACIFb33c3KOTzFbtcbT9WjhooJqNIWLvYe6REuQFMwZGT6P3MnLjdlFV5pt/+wTJ8+GO+22JUo8Yr+fJygdfnb6646RZpWQtDQ+XdINqIZvRbPzokSXi27a+MIg0WCnEbL6xhLtla3U1ydhSvJZn8fXauY23FSrDkoH5NJxZUmbnlHziBBbnqxfrUeCU/b2Mh/jZ7E56aur7KdhX8q6OVlA2NgnUf1eRRF6Mi3k5WrurZJs2xtuymXBUtryg1XskHIyVPng7cYMHr5dtN7j+Ro128J5XzIfbAEFoR3A6aaVA7KfpOHoFVfGRqvJIP5b8/bXdbhAoihcGriBfMJKpwzOV6s6VlHk1rakC5x+4Tp6nxSn734arVeVSqSeq1WeiBCGlrC4+WYK9mc7Ub9cxY1blP80Zyiwnfb3O1fSOCYwTVf72uiqX0iEaNVvKFx0rw/IyqWSVjGT2v3uXMAqPXbI13fBjes6bf8/Mw4IV5jshRYOJhMnXVHjw/Y4N0WS7+x7d457ut0s9rlVwF66dWRP3Ged07fZuomGkyEjVayR8+Yc6/drJDrm5nPKbkvcx9n6zE25KU8f7i0opyfnmFxXjOhoeH09z2/jJMWKTe6B9QM8pUJWq0ktcTz9O54LAzZgfhH7NpjSL72bno++xct8WQyryNhXhm2vroO7oAj4UiU6OVvH6dMCc/9pqg8xxKwaqauWbs5J/R/3l/KS8mPmatLcCfPl1ly7mD+Y/iverVukvUo0YreRXJHDcdT04NpBVWTcl/sWK35fiBlTvUK7DNxM5dH63Ap8vtiSq+4Z0lAOIfmat2n6gGK3kNldz/3lcwNFoWy7erpeR/yNsv9Xz7jjpjyvMzx+N0L2UdHxlW8hrHSsrwXRwBUKd85GfsJKr5Z//qP0uknk9/XcgoWFJ0rBQ3TliCQ8fDu6f6ja374yu4fuOEJUoXJO/33FxHo5lDqdFKPnT0/pt3l8Z87O8/rl6Sz032F8tPwzBxyQ70fGp23FkBmQDjZ1a655oNbPvPoq34fvN+fLLM/VxKdtHp8Zl46mvjymexUHCkBP9dnC9NnlDW7DpiqXhJ4bFSrN3tTE0FI2qckj984hRKy4KeNOZtNHM3uJtSN5TPl++SXuru0S/XKD1CUpFwz0OzPvjB9AeqPWdlppQ+VVaO937It3SOv83eJEcYA654fREOWJxJuTnzr3FKvtfTczxRhf5oyem4LtwXZm6sMnJUFRVnBVuLiqPvFCPhfLanry4wdb6Eivgga/120ESe9ki4nVLaa9RKck/V1jglDwA/5KlXnCGUHk/ONpXu2I4c4QrqZanILIQerq/2mlyQDVYaspKeZWtRMfo8MwcfmFzQ/1Hy4jTjLDVKyS/dVpnDW1+T1CyZ46YjJ796XnC/cay0DMu3y/meKhasPl4qL0xd9vMwwVykfxXyDwQWMs1mVr0hzOL095vdz9TqFdwcJ9UoJa8fsT302WopK96fLLWW4uBUWbnybnc9n5qN//uXnNGuFZPB8dIyPPzZaunrBDLD4sN9v+REkw83Co7k1ZtO3ThhKUqUS3fNhFKjlLyeaasLMHbyz5bPs26PtVXzhz5bhf7Pz0Np2RkscCiS1qt8uHg7JuXsxL/iLA8XjZLT9i+KnTaZGVNGZsZgyg473PhUfPgwVamxSh4AthbF549rRJeWqZaOn62VOis7I/D6gjzL8gCoSI6lIlbMNV5QKPLNNdbtNUHPlWX5h/D1qj1Y5lMTY7cnZilR7Ec1arSSl4HMCj8N6yRLOY+ZVLtOIcPDo9IFVj5WC1DsPyb3ARu0yVsRS++jf+8nK3HtW/IWmq98/Qes2qlGbd/jp87gH3PkuFKePlMutRiJm4VNWMn7EJUHvFZkCwaUWPWpjoTVojGyq3kFxxCbLOR/X7/nqCRpqpNXWIxnp6uZndIKHR+bids+kOdqfdrLSp6I2hDRAiJaT0TriOh+bXtjIppDRJu1/42si+s/gvZSgZpRIdXKxOdoifqBWbJv5aB5K2jWi5dJy3bg1Bm1UknokV4tTOIIZ0GuPNOPm/e2jJF8GYAHhBBdAQwAcDcRdQUwDsA8IURHAPO090wNR+VZBmDd00b297NqDXz48zVyBLEJ6YGJCrroAh53oRRCFAghVmivjwHYAKA1gKsAfKDt9gGAq622xcSGypVyrEjmxAPCahuy+17mmo9dkIVx6qET9iReE0IolUTQzUhvqTZ5IsoE0BvAEgDNhRDBWO69AJqHOeZOIsohopyiopq7Mi7zIlB9tGwWv36vSKiv4tUiuAj80je56PT4TGX8+MfP3Iihf1/oStvSlDwR1QfwOYA/CCGqrPSIgAYzvEWFEG8LIbKFENnp6emyxHEMWSO3kxIvxqve+AGnFbbDmkXlGUoQ2Q8iKyP5z2wq7uEFgilBTpooum3HqHvj3mNSXLbNIEXJE1EyAgr+YyHEF9rmfUTUUvu8JQBfRvqs2SUnhegDk+WWVFNlBCMT/b3356/WuidIBGSrByvusA/aVKZPJnbPVMz8HkUKx5mYQYZ3DQGYAGCDEOIfuo+mArhJe30TgClW21KRzYVyMhh+v3k/9kj0NFB1zGsl46P+O/33p+0oOHLSukCSkT0KfPeHbVLPZwsWNLXMa17PGS3C2NTvoerNYxIZI/nzANwIYCgR/az9XQ5gPIDhRLQZwMXaeyYCGwrk+TOrar/+6uc9po/VJ5gDgIEvzLcqTjWsL7zKQba/va0oKOoxCykc6tdOkiiJ+1j+NkKIRQj/LB9m9fyMOfRK4lRZOZITSckMkKph2e4vSeF1eHQG/vWrPnJOZjNLFUuToB+95+49hkFn1zJ9Ljujq52CI159Sh+tXNnB46fQ6fGZeOf7rS5LVMnN7y3Fk1PNl3urKcxYu9dtEWyj7Ey59HWoIPoasX+bnWvpXBOXeL/sIit5iyQoPjgORhR+sWK3y5JUsjC3CO+bLGChOjI9gL5eZd60pTprdh/B5yvs8f7Rz2JX7LCWV8cPXmqs5C2ierDKlysDN5KqNnq/Ue59nVDj8du9wkreIior+eXbD+Kd7wPeGbkWElzVJPx2gzPWeH6G+nWTo8FK3iJWdPyew/a6AOrL2rVrWs/WtsywIFe90AmrOp6fEdH5MW9/RUlCO/BC3QEn8ZevkAtYGcnPs7kSlF40N6vFh+ON+Xm4qHMzt8WQiqdcH10iXM1YWUxfXRB9pwjsOHhCkiRqoN6d7zH0C6/TVu/Bs9Niz63tZiEBFejYvL7bIlTDTPBMka5QyJ+n2BeJm/3sXNvObQcLcwuxepfzBUWsRnvn77dvlvGXKWtNpVqwAit5iyTotPw9E1fiP4tij1CUXas0lBsnLDV13ILcwooFWyYyizbvx7nPzcXsdQF3xyLJlaH07PdYuP3N7y3Dla//4LYYcT+47Rx6fbB4Oz5cnG9jC9VhJW8RK8uue486V6bv4PHYU7re8t4y/HGS/XlP/GA6XaWNVFcqUgKPqc7iLQfi2t/u6/KMwxc+K3mLqFx1R0+hjSNMs0yxkOLALszefn54YPmFoEdZkHjvUb8t3LKSt0jJaW8oeRWRmV5ZFkb39/ebi8JGPgYXt72QBrmmEkznIYSIyV5vt5J3+hnCSt4l4jGfMO5y44SlePRL4zJ6Vqoi2YUQwvG0EXmF6sZhBJfNJi3biaw/z8J2G903VYSVvAus3X2kIrcMExtP2Oi1YsTPOw9j7e44agUoNJCfu6HQ8bQRF//jO+w6pKbrYdAmP+6LwIN6axTvGZ9Za1jJW6Vlw9pxH7OJo0/j5sPF2x1qKXCHX/3GDxj12qKoe1eaa9Thf0vdSao1+MUFrrQbjTdDvNgSo8S22G16Kz19xtFaCKzkLbL3aAnKy0UVN633vVDogTHN8zM2YKoiycOOlpzG3RNX4LBWEPtYyWnsOqReMRW3+cecTRWvE6NkFbQ7/9Cr8/Mw8IX5OFZy2t6GNFjJW6RVwzpo/+gMPDNtQ8W2STnsY+5n3v5uK+77ZKXbYmDvkRI8N20Dpq8uwMtzNqHsTDnOeXK263mKii0U7LCLV+dtrngdTck7ZbOf7JCeYCVvkd1a/hl9mbZoS3F+s/lZ4cSpMtwzcQUKHYwZiES8v03wt7aj+HMkhBAY8MI8TMrZCSAQZKOCO++6PUfQ/S/fKDPTMSKo5D9cnI+sP89EcWkZvtGC2dbsOoJX5+c5IodT1wwreRtQODFlFfYcPun6YtnUn/dg2uoCy8UdZBHvbedWZoqHP19dbZsKWTLW7QmUsPxuU5HLkoQnmG/qiSnrUHK6HA99tgq//e9y5BUW44rXo6/DyGL34ZPIHDcda3bFscBvAt8r+SMnTzs+yvKKkh80fn61xbIdB5xV+sFfRkVXxFh4fX7ADLB8+yFTx//vzgGmjjOa6quUHE3l2WqouSaYkOzEKWfNTPO1BIUTbV4o97WS33P4JHo+Ndvx0ndeVFjbDxzHvqMljmfgCyqD4IPxn3M3xee6aJM8sXJcSza176i5iOIB7ZuYOs4ImYOZBiaLWXvhyj90/FQVv3637lenHoS+VvJBL4M56/c52q5XRvJ6hvx1Ifo/P8/0zW2WYNRrpZLfXM118ce8/Y7KZAanZ4tGyBzI/3V0D3knU4ynp61HwZHKNaDjDo/ggzg1oPK1kneL1Tbb2GSgV0pPfe1eUe1nKlIzh38yPmRgf7aLRZuLbE01aycyzTVDs5pbOl7lNA/b9h+v0ldbi6z/3nde0N70sZ8s3YGuT8yyLEM4WMnXUN77Id/wtZuzkHCjYScHya/Oz8OFf1voXINxEmnGIHM2kZKUgLaN68Z93P7igL/+t7nqLrwSAWVnqveVle5LSbSmSk/YmGO+Rij5g8dPVeT7VgEVxjifr1DLl59I7cU6IHLKWqdE37g3vA+87HXXuimJcR8T1HUHFM7NlJRAKDOIeJq3wVmzrlNIUfJE9C4RFRLRWt22xkQ0h4g2a/8byWjLDFuKjuPO/y53q/lqvLHAGT9cPZd2qzr9Drq6hZIbQYnYSQKFV5SqpH69/p2fwn7mlIinI/jCy+qnP13aGQDwn5uy4z62WYPIaT6cropkBBHh203V13mOnHQmAtVpZI3k3wcwImTbOADzhBAdAczT3nuKBrXsWYTcprDN96etB11pl0BhzQ2RFJsqOGWDPnoy/CKhLCV/90VnAwAyGsVvrgln7isuLcN1/16MLjbanmOFAKzcUd3llSzYKlVeg5Ci5IUQ3wEI1Q5XAfhAe/0BgKtltOUE1/RujfzxI1HHxHTV68i4WNMb1Ir7mOLSsrAtB+28KuPUSP6nrRFMRurqGcxdvw9LtrkzgDDiVJn6AwdZ2GmTby6ECJZN3wvAcLmeiO4kohwiyikqUmOx5pfntgFQGRlnBjs9NDqk14v7mJh9gSUoii9+NyjuY7YUFSutpKJhRvQfxw2N+5jXI5j67A6GymrRIOo+dta41dOnbZrpY0vLytHUYCCSnOhB3+cYcGThVQTm4YZXoBDibSFEthAiOz09XUp7ZWfKUVxaZtpTJFlbPTono6FpGez00AhOp+1AxpTfTL8TkS1T3vuGdZR+Tlm0SqtT5f3TV3WzdD6na4ca8ez0DdF3ksCHt/XHFT1bmT6+YZ3kattCywb6BTuV/D4iagkA2v9CG9uqwgOfrkL3v3xj+vjmqYGn/AWd5Dx0ZDM0q1ncx8SqeOMdDI7q0dKgrfi1/Kqdh/Hmgi3Rd4yTscM7ST+nDAZ1qB7p2qtNGn47xLy/dbnNI/k+Z5n3nZBdtL5+rSS0axr/jDaIFXfT67LbmD7WDexU8lMB3KS9vgnAFBvbqkKwQLTZ37FFasBDYMy5bfDHizth5Z+HyxLNMr3apCGtbgryx4/E1/cMln7+Qyfis383qB0YET00onPFNrOT3ld06WC9RrzXWrgZUyy+6eFMIr+esCQ+IeKkcd0UU8ct3nIA73wnL7VIb81UY8W4UifFvFPFw5dlYe7YIdCnwGlksm+cQJYL5ScAFgPoTES7iOg2AOMBDCeizQAu1t57iuTEBNx/cUc0qqfGD9isQS18dfd5Fe9rJcf+88VaNPv7zeZSCOjXL6ysZXiV0jiLkuvt5+e0rjQL3tCvbdRj3w1TlMZs/pxYMWsOWhxhsThe8sePxJe/D9wDPduYN6d+bSEVcuN6KTi7Wf2K9+OvOQc3D8o0fT67keVdc70QoqUQIlkIkSGEmCCEOCCEGCaE6CiEuFgI4fjS+sHj5i56K65UetbuPiLV9zZUrE7NG+D+GG3OC3OLUFpmh49y9Ru/eWot3HJepqWzfqrlSfcKx+IslNGknrEHUizX3r8WyjdrxYLZ9RqC9fX831/YAe/8pqrffhsTLp5W6d+uccXr4G81um8GkixGvNqJupJJwGzlFaPKMSO6tYj7PKNeW4Rf/Sd8AE28pNWpPqO4ceBZMR9vh9tYRRZJ3TYiwsMjsiyd90+fBfLVuF1laH9x1YGC1dQB53dsCgBoUr/6b+n2uunzvzgH2164POzn6fXjd42VxUMjsjC8a1UHvUwLNnmzZDZxvk2r+FrJF5fEryCMVt0BoFE94+3RWLvbOLLUDEaDvHgUgx1mlIu0ReBebdLw39v64e/X9pR6/sNxrhEEyTZYJHzt+t549fre+Oi2/jGf59KXv6vy3qqb4ojugcGC/jSqWLdG982IOJO4undrB6WpZOb95xtuT3Zh9Pzo5V2qbTO6IlQy3zibV9ZhjpoolBveV9b9O9FohhGP26FsZZI/fiQAYPWTlyC1dtWHoKy2jBJJxcJ/b+uPwycDD4iv7j4PqbWT0D49YEc9ciL26yI0B4s+Ra0Zgg9a/Yzgn9f1wlvfbkG3VqmWzm2FBQ9eiJQke5Sm1WvB7RmOnvq6VNyRvtaTV3bD+z/m2y5PLPhqJF9cWoZ3F1UuSkVK5hQv4Ub4ThCc4t9xvoF7ncsjeQDVFDwAJEpqq8zkyLlOSiJaNgz4ofdqk1ah4AFrUb2rdh2ueG2mdGKn5oGAoj5tK2ca7dPr46XRPV2161pxR4zGvqMlKPNAaopY0F/Vw7oEZrHh7qvacThG2ImvRvJPf71OQgV04x+sTeOAwrimT2t8sWK3xTbio1ZSIL2CUVbAVBcfPpGQobAWbd6Ppg3U8GwK8tiXFTn4sHlfcdzH9z2rEX4YNxStGkZO5KUaL19n3gy3/cAJHDVhOg0SvPdU45UxvbG/uLTaDHvybwcCAHpkpGGpAqkc1HjUSOJwHNPwcIQbgAbTAtSyaUobmfAjz9rJsefXUcX2Gysb9x7FMQvKwQ703lLxepsE3V9bp9WR5sFlN0H9dWXP1qbNJlYW/PPHj6yIxVCN2smJhknc+mkeOAN0njixUHDkpBS5QvGVkpfBNX2MF5eC96RV+2C83hnT7h2sq4NqTTF4rfbsweOnsDDXsUDpuIn3WujVxny+Fbd469d90S+zsZYK2tzFb2e6hQUPXogFD15o2/lDSTBYF5PFtFUF0Xcyga/MNTIGR0M6GqcyCK7kJ1lMYjRhkXEgSzi6t25YcWt5S0Vb582FW3B1L/P5ScIRNH9ZRaH1QEtESpNxSbcWuMSE+7Aes4vnsWDnWoLT7JOc+iGIr0byZhda9ekBBp3d1HCfq3q1wm+HtMdDFvy/p68usJTAyepDzCMWgiqctiEfS52URFzY2XpeolBzzUFJ1ZBWPXGJlPPEij7i1g7MLp57njhvuNCYDFn4SslvP2Cu+nnQB751WvgFnuTEBDxyWRdDT5JonNCqwd89cYUp+fpqPtycTVkAABk5SURBVN+tG6mzADWwffUEW3Zgl1dG8ygVjGIh1ArR55k5ls8JAA3rOmeDnn7f4NgzdZrU1WcMSu35kW/+cAGe0WUS7dS8foS9q/PVz+ZTLUTCN0q+JM7cIUHeuKGPZEmqY3VB+HdDOmDBgxciq4V7ftRmeP4X5+DTuwZaOsc361Suu+n9EWq3Vg0N4y+M0H/bBrVjt/SajRLt2tJb13vnFg1w48DMivejerQKG8jlJL5R8manhG0b1417Ae2vo3uYaitegsExCQkkzfa4PkxtVzu4oX9bnJvZGF0UvFnHXmI9BbHXrRBmc+03a1ArLrNlzvbqpfZiQZ8EzKuocO37RsmbxYzHwLXZbfD4yOrhzaoz9ec9uPzV7zFrrfVV/F+E8UIyQsWlgOap1s01ZmePqmAl1/4VBnUEwmF2rcLjz9AqNDNRElMWNV7JA4ECBAAwIA478+1G0adhEDCnEMwslD57dfewn23cGxjFbymyVpowf/xI/DKOwglGBTL8wNjJq2Led+ljw2yUxBnStLWCP13aGWk25U8/q0ml37mswuSyuL6fuWIhuc+OwKKH4y/1KAtfuVCaQQigUb0UzHtgCDJsWtgcP3OjqfzVsfq1r3/6Uizffgjna+6fj3+11nC/oHnhm3V745bFCuMuy8JvBmbigr8ucLRdOxiW1QzzNsbnu39ddhs0k7DQ6za1khIr8hXZhT4dRqdm0WvKOskL15gz0wZddpMTCadtdCcNR41X8kE6pNtn/7NSoCAW6qYkVSj4SORqLqardx2xVZ5QkhIT0LaJfbm/W6fVwe7DgWjBxy7vglM25kkxk/nQjtq1vkU3rrlnqH21jN0gq0Uq1ux29t4D2FxTo26/bfutmWlURb9Ad8cF7WMudL7o4Ysw548XxNVWWozujV/fMxi/HhC9yhNjTPZZjWL2+vEKbsWp+EbJ++tyCGClvJkRsZYA9BpmH9QZjeqiY/P4TAKx2onPyWiIW89rByCQ94WJjeFdAoVBjArEu0l7Cd5tbuUr8o255vgptRJZWWXs8E64a0gHqeeUFZGpGk4O+OJxm2yfXt92G7bfGHdZFv44vJNLiQDDU7eW9VQYGwqcc1/Wo1ZPWmBNHHbmtU9divbpaue86NYq1bYiDn7jxgGBEoh2h+cDwGfLraayZiJBRKidnKhclk4Zjj4jz3FndlIjtUj9WklooLlNWq3ZaReKiqUkdbR0y0FXWIaRjYyCO3deELvbtUx8o+RZKTJMzeWPF1uPYI5E5xZquXPGg2+UvN9woxK9V8nSQsdvHdzOZUkYt0hOste8c/k51tItAz72riGiEUSUS0R5RDTO7vZiRjGbn57lj1/si7wdTtG4Xgryx4/E8K7NTR0/477zMaST9dTDdtHSY6UC3cDumbzXCu7osVXJE1EigDcAXAagK4DriairHW3FGwKdpLAPbiObQsZlUM+gzqzX6doqFR/c2s9tMQy5qlcrFByxp5iEVVRaAyn3QLY4twqc2D2S7wcgTwixVQhxCsD/AFxlc5sx8cqYXrhtcDv0zJBXkq2D4h47Zsl77jK3RaixpCQmKJv7R6Vhku3mTQlfVlZFsnixW8m3BrBT936Xtq0CIrqTiHKIKKeoqMh0Q/uL4/MBz2hUF38e1dVSzcbQpFPT7pWTO1o1S1JSYkJFFr1uDrgpMpW1hru0TJV+Pdw1pAM+uWOAtPO9fF1Paecyy6geLXGV5FKRDesk4wLNjKfYLRkXri+8CiHeFkJkCyGy09PN20Xf/SG+2qkyCE06VUeSKUM1H2EAGKiNJs1m4gOAczMbyRLH99w7tCNap9WxJfJz3GVZFb+nFYKX6dAsc2shoTx5hXlLLhFVVFCThdk1HtWwW8nvBqDXChnaNunkFRbbcVpP0rttpQkqq0UDZQK/YklP26SeuusRTtKuaT38MG4omqXWVnbRb/JdA3HXkA5IjaNKVCRuPk+Od9T1/eTlDArG0ag48IoVu5X8MgAdiagdEaUAGANgqs1t1ni+/P15Fa9fuOYcpJjInBiKjEs8FssYR/lWR1X9ktUiFeMuy1JOAcoKcKQwr72GrXeUEKIMwD0AvgGwAcBkIcQ6O9tkqtLAROHxSFi5f2KJGrQ7qMUsXqs3GkpyopfVVGwEv2G4h07rtPjqRbRtbF96bCexfdgkhJghhOgkhOgghHjO7vaYUISUkGwZo7VY5OjfvrHlduzg898Ncq3tfplq9ome2xUKRJPhHf3ezefidxfKTRDoFjw3lsTNgzLdFiEsCRJ+5aCCtqLrYzk2SYJpyQ5kLaqb4e6Lzsb7t5xr6tjXb+gNwP5gnsdHWQt/US1b50VZzZCUmICXRvfAjQPOUtaNNRbUvKPiZOUOc9XgZfLkld3cFqEKr14fuLlbpdWRYpN/bGQX3DjgLIw8x7ybWidd7najBdaOzerHPaWuCSQkEC7s3MzUsRd38YeHiFu0bFgHz1zdXdnBRyx4V3Ide+OICGzT2F4lcm3fDNPHdpSYyuDKnq2QP34k6qYk4bUb+pg+T9CW27heCp65urulhdFgSmAAqJ1cfWRsZrT04CVq2vDtYN1Tl8a032vaAx6oOnt69+ZsvPVr89eCXXx0W385J5K0APyeyVmTqqgTl2yBaL/trwe0xUc/7QAAzLo/vnJv8fLi//XAorz9cYWijx3eCfVrJWFMvzY4Xiq/epOV0bHMAtSNdKP3nm0aVtRlDRLLaKlbq1S8dn1vDP37twCAe4Z2lCaf6tQzSCOw9NFh6Pf8vCrbrujZCu3T66G0rBzJmq3u3qFnS/Nnl8kHt/bD4I5NpZwruFAanDE2qZeCAyYK5VxkctYUC/VrJaG41NkCRz5R8pG1/KAOTSuUvNGNIpOEBMJdQzrgL1NjdyK6b1iloqqbotZPMv/BIVLPl5KUgFNl5YaLsOlaVG0k+rRthPY2Fl33Gs1SjR/C3VpVRiarZu8GgMQEwplyIbWew5BO6fjq7vPQM6MhBnZogib1UtD32bkVn798XS/88t+LpbVnBjd8nHxhronmtSHDuyQevlm319H27ER2vo3gL2FUpPmCjtEjni/pZv9o9NXre3vClr300WHRd1KU884OjN5lpxXr1SYNRIROzRugSf3KQcMrY3qhXzv3vZRG9XS+OpQvlHwkFd4itbbjVd9/M/Cs6DvVcBJDHrz540eia6vIvuj540fi/BgeBFa5smcrjOhuPX+43YQbxbvFY5d3iXnfzs0DszG7I5zvOD/g2jmoQ2wmoc/uGminOHjmqu62nt8ItWwDJikrLw/7mYCIyQxglo9v74/mITdbkgyfRQXQp0eQxee/G4SvV+2J2y456w9Vk78F1zHsoiYED8km1mLX9Wsl4aERWRjWpTl6SMwCa8RjI7visZGxu3dm2xyT4IaXji+U/MdLdoT97Ew50DOjIW45LxO3DJIfsBGcduppICmXh5tc0Cm9wsdaJt1bN0T31g0xZ/2+iL9bKFktqo7y9esYqtE81b5BhRepnZyAktOVA7GZ95+P5MQEDGjvvO95wzrJOHLytOPtuokvhpwlp8N7pAghQET4yxXd0LaJM2HK/V24eGVz15D2SJWcEkHP8K7N8RcLWQdVZsGDF9reRjedaWuwwUDDDYwCrj6+vT/mjq1cvK+Xkog2LqYLOBNSXOQXvVuH2dM/+ELJR4rmU79ejKI40HG3hMk6qMKNZyWNgxMeUtPuHVzx+qPbJfmZW8Soy847uykyGqmTAyZ0vezabPNxLV7B+3YFRPaTj7csYE0jq0UDbNx7rNp2p3otdCoPGHveOE3rNLUWNUMJ9xDq2cZeG3ckruzZCt/mFmFWBO+yBy/t7KBE1RnVoxXeXLil4v2gDk2RP34kTpWVQ/h0SOiPkXwEnZDRiMPkI9G5RQPD7U49G2fcdz7+OrpHlW3uq3ig71nuu9uZwc2+q1crCW/d2LfivdG957ZpqWurVGx74fJq21OSElwrzxdEVl7+UHyh5CP5wXdp4e0UsW7h1KimfXp9XJtdtdrUaAupIazQqTkHWcngkcuyAASCk0KRGUFtFtXy3+c8frGt5/elkp95f6W7nWK/p2dw08rVv30TfH3P4Og7SmbK3YOxPM4b7m3dyNUJorm1qnC919UydhrJ0rCufYv5XiUpIZjh1Z4fzxdKPrRvuugKPKhaOk0Vwilzt62TbiirOimJVaIkY6FLSDERmUnmjPj49v5Y9PBFtrZhFw1sTiniFf7t8MDAF73udNoCPxFOmcvMKeJnQt0BZReTDqVuSpJy+Y1i4ZM7BiCzqTpeNm5yaTdno6m9d7UYwDpePm7reJUe3BPviN1F0Y0AHz3q9FpVBnq46IbdNKyTjNsHt8M1fexZi/KHuSbSZ6pe9YoQbsTutjuZCm6UQeIJCjvbZnNNNFRbVGSM+eYPlSnPiQiPj+oaNXeTWXyh5CON+viaj0x4c42jYlRDxd+troslAL2I29eQyoRzXbYDXyj5eRsL3RZBecac28Zwezhd6vYNqpKODy1uwkRBxSd0DcYXSp6JTrj77pzWDattuy67DYZ0tj+lbyRU0hMHtepCCokUFhVkTKsTMG/F66nE2IMvFl7bNq6LHQdPhPlUhcteBYz7oa6BW9uLIRGobqCCbZkoMKMJjdKMfL25S1sXk38FGdWjJUrLynFlT/NF3xl5+GIkb5T7+5bzMgGoNSI0wqkoy3D9oGr3qCBXcy06Mynk+pp+X/hALbfMXLcPDiR7c9LWGw4iwui+GZaKvtcEVj1xCVY9cYnt7Vj6FYjoWiJaR0TlRJQd8tkjRJRHRLlEFFuZeZMYeWJ0ULwOaDCI5vp+bR1pL5yziqoPwdQ66kRGur0+EQ8quZ4ykWlYN9mRCGCrj9q1AK4B8J1+IxF1BTAGQDcAIwC8SUS2uSZ46cI+N9PeYJlwhIv8VTUiuKmC9lwVTEjhCKZJV1hExiUsKXkhxAYhRK7BR1cB+J8QolQIsQ1AHoB+VtqKhNG0MDj4Uu2ad0upfr16j+H2llpK3Zu4Lm3MRBrY21lqMhJuxzUw6mKX0aw1gJ2697u0bdUgojuJKIeIcoqKikw19sYNfapv1ObYqo5snE4bcPhE9ZJn1/bNwEWdm2Hi7f3xlyu6OSqPFwi9doImLyGAV8b0wp9CcqMPy2qGFg3dybI4vGtzABxZylQnqncNEc0FYJRs4TEhxBSrAggh3gbwNgBkZ2eb0nxulhOLG4UeOsH6tIMUKR+nKsGLcvJdA/HVyj1IrZ2Eq3pVH7O4WbsgWPyCYUKJquSFEGaSHe8GoI++ydC2MQrBVbPiI6tFKsZdVj30vEGtJBwrLcPt57d3QSqGiYxd5pqpAMYQUS0iagegI4ClNrVVhdZp2mhKm2urtiirkjSs4yNTcKQEQORC8QBQKzlwG9VO5rQHjHpYCoYiol8AeA1AOoDpRPSzEOJSIcQ6IpoMYD2AMgB3CyEi3ymSCIagX9s3A7l7j+KB4e7WlIyGm4qWdXxsHCg+hQ7uBgAzjGksKXkhxJcAvgzz2XMAnrNyfivUTk7Es1ef41bznoBzxsdG0/opbovAMKbxXUiaCmHdsfBLra5p0CvCDVjHRya4kJqcGNttwm6MjIr4IneNHtXTwQaXCLJaNnDdG4IXXiOT1aIBdh06WWFzD0/Qt9J2kRgmbnyn5FWOSnST1ml1qqXMZZ0UmX+O6Y01u46gWQN3fN8ZRga+M9d4RsU7rGGNnn2DOHAmIvVrJXFwEeN5fKfkVcettAZGrqScJVAOPHlkVIbvcofZfuA4AKC0rNzRdo2yUJazvUYKwdgMlerSMkwQH9rk3ZYgMnu0AJul+QdxUVYzx9o1GsmzC6UcJtyUjcVbD3AlJEZJfDeSV70aTYf0egCct4cbRWOyjpdDk/q1MKqH2tcdU5Vnru7uWtpvp/Gdkm+V5l6SqFhonhrw1HA63cIdFwQqB7VrWq9iGyt5pqZy44Cz8Oldg9wWwxF8p+RVN9fcppVpy3K4TFujuoGozYxGddCmceBBGBq807geR3YyjN/wn01eMSfKr+8ZjCteX1TxfliX5q4GQRFRRR+FjuTnjh2CQydOuSAVwzB2wSN5m+neOhUPDO/kthgVCCEqi1+EfNa4XorytXEZhokP/yl5twUIgYhw77CObotRJRK4Qe1A8WDV+ophGPn4z1zDmsuQ5qkB974eGQ3x6wFnYerPe3BWE28kc2MYxjy+U/J9z2rstghKktUiFdPvG4ysFqlITCD8dkgHt0ViGMYBfKfko2cMrLl0a9XQbREYhnEY1ogMwzA+xndKnk3yDMMwlfhOyQc9RxiGYRgfKnlVqKd4hSqGYWoGvlt4VYXVT16KU7p0wm/9ug8W5ha5KBHDMDURVvI2kZhAqKMbzY/o3hIjurd0USKGYWoibK5hGIbxMazkGYZhfIwlJU9EfyWijUS0moi+JKI03WePEFEeEeUS0aXWRWUYhmHixepIfg6A7kKIHgA2AXgEAIioK4AxALoBGAHgTSJidxOGYRiHsaTkhRCzhRBl2tufAGRor68C8D8hRKkQYhuAPAD9rLTF1Fyacu1UhjGNTJv8rQBmaq9bA9ip+2yXtq0aRHQnEeUQUU5REbsYMtW5Njsj+k4MwxgS1YWSiOYCaGHw0WNCiCnaPo8BKAPwcbwCCCHeBvA2AGRnZ3PVUaYaHFjGMOaJquSFEBdH+pyIbgYwCsAwISoKyu0G0Ea3W4a2jWHipqycn/0MYxar3jUjADwE4EohxAndR1MBjCGiWkTUDkBHAEuttMXUPF4a3QMA0DMjLcqeDMOEw2rE6+sAagGYo5WX+0kIcZcQYh0RTQawHgEzzt1CiDMW22JqGL/MboOLOjdDegNeeGUYs1hS8kKIsyN89hyA56ycn2FYwTOMNTjilWEYxsewkmcYhvExrOQZhmF8DCt5hmEYH+ObfPIf3toPR0tOuy0GwzCMUvhGyV/QKd1tERiGYZSDzTUMwzA+hpU8wzCMj2ElzzAM42NYyTMMw/gYVvIMwzA+hpU8wzCMj2ElzzAM42NYyTMMw/gYqizm5D5EVARgu8nDmwLYL1EcP8J9FBnun+hwH0XGrf45SwhhGBGqlJK3AhHlCCGy3ZZDZbiPIsP9Ex3uo8io2D9srmEYhvExrOQZhmF8jJ+U/NtuC+ABuI8iw/0THe6jyCjXP76xyTMMwzDV8dNInmEYhgmBlTzDMIyP8YWSJ6IRRJRLRHlENM5teWRDRO8SUSERrdVta0xEc4hos/a/kbadiOhVrS9WE1Ef3TE3aftvJqKbdNv7EtEa7ZhXiYgitaEaRNSGiBYQ0XoiWkdE92vbuY80iKg2ES0lolVaHz2lbW9HREu07zWJiFK07bW093na55m6cz2ibc8lokt12w3vw3BtqAgRJRLRSiKapr33fv8IITz9ByARwBYA7QGkAFgFoKvbckn+jhcA6ANgrW7bSwDGaa/HAXhRe305gJkACMAAAEu07Y0BbNX+N9JeN9I+W6rtS9qxl0VqQ7U/AC0B9NFeNwCwCUBX7qMqfUQA6muvkwEs0b7PZABjtO1vAfid9vr3AN7SXo8BMEl73VW7x2oBaKfde4mR7sNwbaj4B2AsgIkApkWS3Uv943qnSvhRBgL4Rvf+EQCPuC2XDd8zE1WVfC6AltrrlgBytdf/BnB96H4Argfwb932f2vbWgLYqNtesV+4NlT/AzAFwHDuo7D9UxfACgD9EYjOTNK2V9xLAL4BMFB7naTtR6H3V3C/cPehdoxhG6r9AcgAMA/AUADTIsnupf7xg7mmNYCduve7tG1+p7kQokB7vRdAc+11uP6ItH2XwfZIbSiLNm3ujcBIlftIh2aK+BlAIYA5CIwsDwshyrRd9N+roi+0z48AaIL4+65JhDZU458AHgJQrr2PJLtn+scPSr7GIwJDAFt9YZ1owypEVB/A5wD+IIQ4qv+M+wgQQpwRQvRCYMTaD0CWyyIpAxGNAlAohFjutiyy8YOS3w2gje59hrbN7+wjopYAoP0v1LaH649I2zMMtkdqQzmIKBkBBf+xEOILbTP3kQFCiMMAFiBgGkgjoiTtI/33qugL7fOGAA4g/r47EKENlTgPwJVElA/gfwiYbF6BD/rHD0p+GYCO2gp1CgKLIFNdlskJpgIIen/chIAdOrj9N5oHyQAARzRzwjcALiGiRpoHyCUI2P4KABwlogGax8hvQs5l1IZSaHJPALBBCPEP3UfcRxpElE5EadrrOgisWWxAQNmP1nYL7aPg9xoNYL42U5kKYIzmXdIOQEcEFqUN70PtmHBtKIMQ4hEhRIYQIhMB2ecLIX4FP/SP24sdkhZMLkfAo2ILgMfclseG7/cJgAIApxGw2d2GgC1vHoDNAOYCaKztSwDe0PpiDYBs3XluBZCn/d2i254NYK12zOuojIQ2bEO1PwCDETCTrAbws/Z3OfdRlT7qAWCl1kdrATyhbW+PgBLKA/ApgFra9tra+zzt8/a6cz2m9UMuNC8jbbvhfRiuDVX/AFyISu8az/cPpzVgGIbxMX4w1zAMwzBhYCXPMAzjY1jJMwzD+BhW8gzDMD6GlTzDMIyPYSXPMAzjY1jJMwzD+Jj/B6h3yioL//teAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "id": "un6ZPyc8V7R1",
        "outputId": "37ce370e-50b9-4fbe-987a-feb72bef6b76"
      },
      "source": [
        "# 처음 10일간의 온도 데이터\r\n",
        "# 10분마다 한개의 데이터가 생성되므로, 10일간은 1440개의 데이터가 존재한다.\r\n",
        "plt.plot(range(1440), temp[:1440])"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f2533d270f0>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5gb5bX/v6/6Sqvt1etd79pe27h3G2NTTQ2hJSQQLiWQSyA9ub9wIb65l5qQQJKbkAIkEHKpIbQQmgEbMM02Nu69rb1rr7c3Sav+/v6YGe1Iq64ZtT2f5/GDNCNpDrPSmTPnPed7GOccBEEQRH6iybQBBEEQhHqQkycIgshjyMkTBEHkMeTkCYIg8hhy8gRBEHmMLtMGyKmoqOCNjY2ZNoMgCCKn2Lx5czfnvDLcvqxy8o2Njdi0aVOmzSAIgsgpGGNHI+1TPV3DGLuAMbaPMXaQMXa72scjCIIgRlDVyTPGtAD+AOBCANMBXM0Ym67mMQmCIIgR1I7kFwM4yDk/zDl3A3gOwKUqH5MgCIIQUdvJ1wFolT1vE7cFYIzdzBjbxBjb1NXVpbI5BEEQY4uMl1Byzh/lnC/knC+srAy7OEwQBEEkidpO/jiAetnz8eI2giAIIg2o7eQ/A9DMGGtijBkAXAXgVZWPSRAEQYioWifPOfcyxr4DYDUALYDHOee71DwmkVusP9wDi0GHWeOLM20KQeQlqjdDcc7fAPCG2schso/DXTac/asPsHRiGZ67+dRR+w90DOGqR9cDAFru/0K6zSOIMUHGF16JzPPJoW5855nP4fcrN0DG4fbiq6IDX3+4F/LhNGv2dOAvHx7Gub9ZF9jm9voVOzZBECNklawBkRm+9ucNAID7Lp+F4gK9Ip/5ny/uQNeQK/B8YNiDErMBfXY3bvrbaOmK3e2DmFtfosixCYIYgSJ5IoDD7VXkc1xeH9bs6UBtsQl3XzoDAHC4247nN7Xigbf3Bb3211+ZAw0D1u7pUOTYBEEEQ5E8EcDuUsbJbzzSC4fbh4eunge9VogjfvLSDuw9ORR4zcWza3HLGZMws64YL285jpe3HscPz50CxpgiNhAEIUCRPBHA7vIp8jlr93bCqNNg2aQK1BSbAAB7Tw6husgIALhs7jj8/mvzMbNOqKi5fF4dWnuHselonyLHJwhiBIrkxziftfQGHicSyX94oAsmvRaLGssC2zjneHXbCfzfp0dxenMFCgxaNFcVQq9l8Pg47r9iNpZNLodOExxbnDu9GgBw5cOf4ulvLMFpkysACNU3VUUmfPvpz2E16bB0Yjm+srAeBQYtAGDn8QFMrLTAbKCvMUFEgn4dY5wrH/408NgWp5N3eny49rGNAIJLH1/echw/en4bAOC6UxsBAIwxPHb9IuxuH8TpUyqh1YxOx1hNesxrKMGWY/245i8bcMsZk2A16fDA6uD8/Zs7T+J/Xt2FJ29ajD6HB997dgvOnFqJJ76+OKH/Z4IYS5CTH8M4PcHpGYc7vnTN+/s6R23z+Tnu+tduAMCj1y7AWdOqAvtOn1KJ06dE1yV69t+XYu3eTtzz2m48/MGhwPYFE0rRXFUIj4+jqECHv37cErjACLZ04fNjfZjfUArOOfwcYS8kBDFWISc/hul3eIKexxvJf3qoJ/DY6fFBp2F4ZN1hDAx7sPKUapw3oyZhW0x6LS6aVYuzp1Xh4oc+wsFOG566aQmWN1cEve6WMybhzR3tWL2rAz+56BTc+LfPcP1jG/Hvp0/E3z9rhc/P8ep3T0OV1ZSwDQSRj5CTH8P0D7uDnseTk+ec461dJwPPh5xevLXrZCC18vuvzUvJJpNei3d/dEbE/dVFJtxwWhNuOK0JAPAf507B7S/twK/f2R94zXt7O/HVRQ0p2UEQ+QI5+TFMaCRvjyNd8+6eTnQMutBYbkZLjwM2lxe7jg8AAK5dOgEmvVYVWyNx1eIGTKoqRJnFgKZyC+bc/Ta2tw3gq4vSagZBZC1UQpkleH1+bD7ahz67O/aLFWKUk48jkt8sljn++PxpAICzHnwfz33WiiVNZbjnspnKGxkHixrLMKmyEBoNw+zxxdghXnQIgiAnnxXsPD6AyavexJf+9Am+8+znaTvultbguvR9smalSBzrtWNihQUVhYag7csnV0R4R3qZVVeCPe2DcHmVqfkniGTZ3xH795QOyMlnARc/9FHg8b6TtrQd95EPDgc9/+hgN3wxRMqO9TpQX2aG1RSscXPxnHGK25cMc8YXw+PjgTsOgsgEL29pw3m/WYf3wlSipRty8lmGz+8PUmxMFw1lZgBA55Az4mv8fo4jXXY0VVgwriS4eqWx3KyqffFy5tQqlJj1eHEzDSAjMsfGI0KQ0drryLAl5OQzTmjE2efwpC0KNelH/vzXLp0AILq0QUuPHXa3D9Nri1BiNuC17y4P7MsWzZkCgxbTa4twuDt9d0QEEYpUjlyQ5kKEcJCTzzD/2nYi8LjSaoSGAesOdKfl2PIvYLWoMRNp8dXn5/jxC9sBIKA5M7OuGP/zxem4YVmjuoYmSEOZOSsiKGLsIv2O4m0wVBMqocww29r6Mae+BFcvqseipjJ8/7ktWH+oB3wlVz06ln8BpYXUcE7e4/Pjxic+w+ajfWgoM+OUWmtg39fFevVsor7MjG6bG0NOz6i1A4JIB34x5br35GCGLaFIPqN0Djqx5Vg/ZtUVCfXelYU4e2oVNrb04gu/+wgen3rTkvx+DpfXjy/MqsW9l81EoVG43oerlf/0UA8+PNCNhRNK8eKty7ImNROJGeOKAAAPrT2YYUuIsYo06Wx/R+bThuTkM8hTG44BAL4wa6Qy5TtnN+PG05qwu30Q972+R7VjD4u6NbPGF+Pflk6AUSekbsKN4ZNKKx++dgEqrUbVbFIKScXy0XWHqZSSyAgu8XeUDWlDcvIZwufn+N2aAwCAmXVFge0GnQY/uWgalk0qxxOftOCTg+rk5yUnbxZlew064avg9o12ipuP9qG+rAAVhdnv4AFAr9XgD1+bDwD4v0+OZtgaYiwiif91Drni1oRSC3LyGWLXiZGuzNC8sU6rwZ+vWwiLQYvHPjoCzjm8CqduhsW0jCRDYBSdvMsTfBzOOTYd7cOiCWXIJU6bXA4AuP+tvUGzZgkiHcgVXh+RqapmAnLyGeIjMULf9F8rw+63GHX4/spmrNnbiXn3vIPJq97EB/u7FDt+5Eg+2Mk//nELum0uzGvIrSHbJWYDXv7WMvj8HIvuexftA8OZNokYQ7i8flw0S1BjDZcCTSfk5DPE3vYh1JVET4HctHwilk4sC2jM/OSlHfDH6EiNFymSLwiJ5EO/kJ8fE2r2zz6lWpHjppN5DaW46xJhkPhT6yltQ6QPp8eP4gIDigv0o+Y2pBty8hliT/tgUCliOLQahme+sRQbV52Dey+bieP9w/jj+8pUjEiRvOTkpUjeFeLki0x6VBQaUVdSoMhx0831yxqxqLEUb+w4mfEfG5F+Bp0eVavUIuHy+mDUaWDSa+D0UCQ/5nB6fDjcbce0mqKYr9VoGKqsJlyzpAFTqgvx/j5lUjaBSF5K12ilnHywIxwc9qC4ILfbKa5f1ogj3XZ88aGPMOT0xH4DkTfMvvNt/NtfNqT9uC6PHya9Fia9Fs4MV3iRk88ABztt8Pk5TqmN7eQlGGM4dWI59rQPKpKyCUTyopNnjKHIpMPAsOAEW3sdaLz9dby+oz3nG4ounj0Od35xOg502rBuf3q6iYnsYcOR3tgvUpBBpwdunx8mvQYmnTYQUGUKcvJp5vGPjuD//UMYdj0tRromlBnjimF3+9DSY0/ZjtCcPABUWI3otrnh8vpw7WMj0Y/k+HOZry2ZAJ2GBVU1EfmNUutXibLyVx8AAIw6rZCuoYXXsUP7wDDufm039p4cwqy6YjSVWxJ6/3Sxk3PnidRbpR0hkTwAVBQa0W1z4a2dJ9HSM9LEcV+GhoEoiUGnQVOFBQc6M9+BOFbw+PzosWWufDV0fSlddIolu3otg0GngZvSNbmJzeWN6wtsd3kD+hUbDgu3jdefOgGP3bAQGk1i8gBTqq0oMevxtmzGarI4w0XyhQZ021xYs6cT5RYD3vrBCuy663wsy5KBIKkyqbIQ7+zuoAXYNLHq5R1YcO+7GSshzMTfWT6Pwc85jDotlVDmKl/64ydYcO+7gedHe+z46Ss7R43vu/Xpz3HB/36IPrsbz2w4hoYyM/7nizNQZTWFfmRMDDoNljSVKTLezhHGyVcWGnGoy443d7bjzKlVmFZTBIsxtxdd5SyYUAoA+ON7pGmTDl7eImj6t/TYsamlN+ZAGqXJxIKnvLvVoNUIkXwGqnvkkJNPkn3iaK8Lf/sh+h1unPHA+3hy/VGc+eD7uP7xjeizu+H3c6wTG5jm3fMONrb04polDQlH8HIqCo042uPAmj0dKdk/7PHBoNVApx35CtSLg0M8Po5L5mbHpCcluXF5EwqNOry9O7VzR8SHpIf0989a8eWHP8VfPz6S1uPvaEv/+svHMhkSi1EHg1ZDkXwusvnoyGr9nvZBzL37ncDzgWEPPtjfhZe2HMf6wz2j3nvDaY0pHfvqxQ0AgAdW70vpc5weX9DQEACYIFsjkCZF5RNaDcMPz52CvSeHcKQ79cVrIjrFBUJV1mMfCc79aE/6xLo45/jn1pFZDYe60rMW862nhRnNJr0GF88eJ+bkycnnHGv3jp7bWGjUYdt/n4dyi6DLfrDTFpjw9MntZ+OOC6dh113nB6KbZJlZV4y7L52BvSeHUtLEcLi9QYuuQPAIvzKLIfQtecFFs2qgYcCLm9sybUreU1QQXHpbaEpf6m/t3k68vqM98PwcseIlXfzlukUoMGjJyecqm4/2Yfb4Yqz+wemBbY9cuwDFZj3e//GZOKW2CM9uPIZfvbMfAFBTZMI3z5ikWH778nl1AIC/b2pN+jOGPX6YDcH21Mui96I0/iDTSW1xAZY0lWfFgOV8Z9gdrL6o16bP3fSGrI2lG6N4l5zXOXnG2J2MseOMsa3iv4vUOlY6cHp88Ps5Xt7ShvWHezGrrhhTa6x48dZl+OYZE7GkSVBptJr0+LpsHF5ThSWlHHw4rCY9vnXmJBzrcSTdsj3s9gUUKCVMei2+vGA8HrxyTtYPBkmFxU1l2N0+iM7ByEPLidQJHX1nc6ZPctcYZrZqOqptpAbH+Q3CIr9Bq8lYKaeE2uHabzjnD6p8DNVxeX0468H30T4w4hSkmvUFE0oDVRsSK6dXAy8Kjx+8crYqNk2uKoTXz3G0x4HJVYUJv3/Y4w0oUMp58Mo5SpiX1Zw/owa/XXMA7+/rwlcW1WfanLzF4fYFpSs6Bp3w+PxpieglwT054QIbpSmz6LFwQim0YmBnpHRNbvCjv28LcvAAcMmcyNUnZRZDQNCruTqxrtZ4GSd+fkeS0eiw25cVk+QzwdQaKwr0WuwVJ14RysM5h8PtRYVsbef1He245cnNaTl+uPvQdKRNnB5/IFUDjKRrOM9M9y2gfiT/HcbYdQA2AfgPznlf6AsYYzcDuBkAGhoaVDYncdr6HIEFnCvm1aGh3IwblzfF1HN57ual0GkZilTSfZEWeHuSzD063D6UWXJj0pPSaDUMU2us2Hxs1NeRUAiX1w8/B8oLjTghC5DWhClaUAOPb7RTDR2IowYury9QVQQI6RrOAa+fQ6/NTAo0pUieMfYuY2xnmH+XAvgTgEkA5gJoB/CrcJ/BOX+Uc76Qc76wsrIyFXMUZU/7IO58dReW/+I9AMAX54zDA1fOwQ9WTonLcdeXmVFbrJ48r1T98syGo0k1mTg9vrDpmrHCWVOrsK21H4OkSqkKkjZSeWFmqrTCrVWlY96vy+MPShUZIsxpSCcpRfKc8/BjjUJgjP0ZwGupHCudPPLBIfz8zb2B59JiZDZRYhZ+POsP9+KJT1pw0/KmhN4/7Bm76RoAmCtOutp5fADLJuWHbEM2IWkjqRnoREPuVLUaBp+fp2UB1OX1B+X95U4+UzfOalbX1MqeXg5gp1rHUhq5gweyczFSK6vYaUmiscfh9o2qkx9LzKorBgBsz0BX5FjAIbb3L51Yhv/6win46w2LAABz69MzRlKef39cPHY6Inmnxxc+ks9gGaWaOflfMsbmAuAAWgB8U8VjKcZhsTOu3GLAv767HH2OzNbbRuOqRfV47rNW+JNY1HF6xraTL7MYMK3Gile2HMc3T5+Y1yWjmUAqn7QYdLh0hdDXsaSpDOlafpSna4wRpp6pgcsbkq7R5ni6Jhqc82vV+mw18Pr8GBj24PlNQifkf144DeNKCgJVLNnI/V+ajS3H+nEwQflcj88Pj4+P6XQNAHxp/njc98Ye9NjdUWftEokjOXmzceQ7ZtJr0Z+moClzTj64TFNqgAztGUgn+dnWmAS3v7QDL4it7uNLC3DZ3LoMWxQfp04qxzMbj8Hv53E3XYXOdx2rzB4vpGzW7umkenmFGfYI6Rp5V7VJn77GIClyrikyBaRE1K6u4ZwLJZSySL5QdPKZHDtJdfIiL8i0TP524+JALi3baa4uhNvrx8kE6uV7bUI0VZqn+jTxsripDFajDtuP92falLwjEMnLUoJGnTZtGu9usYTyo/88SzakXt1jSyJolUUjMuJWk+Tk09ftG0pueDKVCW1UmFSZeAdppmgUlSOX3b8WD605ENd7usRhJ5XWsZ2iYEyol99HTVGKE25egUmvgTMNteoA0DXkhFbDoNNq0pauOTkg/K6myhogpX6aIRc5+YwyKF5lv3dOM478PLckdibIlCN/9c5+/OG9gzFvDbvF8WSVlIfGgsZSfH6sP2254rHCcIRIPh0VLjaXF89ubA30j0gdqGo7ebsoyGaRrUOMRPKUrsko0jSnxnJzzlVZhNYhP7B6H/76cUvU90iRfIV1bKdrAOCMKZXw+Tm2USmlIqzZ04Ebn/gsMCFJrryarkg+dDqblJNXu8LFITl52ToEpWuyhF5H7uaotWEWW8MNK5HTPeSChgHlY1TWQM4pNYLQ3H5K2SjCTX/bhLV7O3G8fxiMBQuFmfRCJK+2jsvAcHDUbExTTt7uGn33UqDXQqthaVXgDIWcPEau/GXm3HPy4Yg1gafL5kKZxRD2AjHWKLUYUGk1BsY5EspwtMcOs14bdGds1Gng5+F1ZZRkMJKTV/kuQorkzbK7F8YYSgr0SWtMKQE5eQCHu4SO0dqSxIdrZwMrmoW2/JtPn4hrljTgxMBw1CqGriGqC5cztdqK/eTkFeXjgz2jJLCl+nG1B2y7xBr53141F4DgaA069cs3pUg+tDS5rrQAbX3pG30YCjl5COmNiRUWVFlz08k/cu0CfHjbWfjJRadgXkMpOMcoaWQ5fQ43SvPkrkUJZtYVY3vbALa1UillKvhDhPLkM4OB9EXUHtGZyy8yRq1G9XSNw+0NpGfk1JeZ0dY3rOqxozHmnTznHFta+0cN/sglzAZdYHRfiShzGnrLKqff4UapRR0J5FzkS/OFxreH1h7MsCW5TWiEHjonWJrWpHatvJQOMsiGkxjT0Ihld/uCKmsk6kvNaOtzJKUWqwRj1slLiz9tfcPotbsxJ03CSWojDU+OJqHb7/AEVCwJYbDLosZSdA7ROMBUGA5p3V8+OVjdU0rXqO1sJUkD+QQqo06rfnWNyztqbjIA1JcVwOPjSQ/4SZUx6eQ55zj3N+tw6s/XYIt4iz5nfH44eWlgQb8jvJPnnKN/2INSM0XychrLLRn7EeYLwyER+srp1UHPpXSN2pG8pPio18mdvPqRfJ8j/O+qvlS4y27tzUxefkw6+Y5BFw522tA+4MQ/NrXCoNNgao06Y/rSjbR4fLw/fA5w0OmFz89RUkCRvJxKqxHdNndGx7TlOrGct1RaqLZY10gkP5IbN+g0cKl8cemxu1AepqBBSqVmanF/TDr5Y7Ir6ocHurF0YnnOaNXEosikR5nFgKM94TXmB8QIv4Qi+SCKC/Tw+TnsGVQLzHWG3YJzPWtqJT694+xR+yUZjS6x41otpIXX4Jy8VvVIvsfmDozllDOhzIzqIiPWH+lV9fiRyA/PliAnQqLcmeOKMmSJOjSUmSPWykv6+FRdE0xxHAvW+QjnHPtODimyKCila25aPjHsRKhqsXpN7bSYtPAanJNXt7qGcy44+TCRvEbD0FhuQSfl5NPHVjEP/w1xZF5DmTnay3OOxvI4nDxV1wQhLUT32MaWhs1tL2zH+f+7Dq9tP5HyZwUkrCMMoykx62HQaVR38u6wC68aVRdeh1xeuH1+VESYaVtTbEpIKVZJ8srJ3/WvXXjsoyNRX7Pz+ACe+KQFAHDbBdNw/xWz8KUF49NgXfqoLzPjxMBw2GHG/YF0DUXyciZXCTXdB7vGTlOU0+PDP0SJ7UNdiY+QDGVY7PiMNKeAMYbqIqP6Tt47Oiev9sKrJPoXaXB5TZEJHYOujKz55JWT/+vHLbjntd1RXyOfomTQaXDV4oagK34+UFtcAM7D5z4ltUWpnp4QmFBugUGrwd4xpGHzz63HFf08SYRLEuUKR7VV/YjW4/NDr2Uhkgrq5uQl2YJIelDVRSa4vX70Rah6U5O88W7x5tukTtCnblqipjkZpbZYyH0+K06MkiN9yYrJyQeh12ows64IGw5nZnEsE3QOjgQBv1tzAPYUNc8l5clCYxQnX2wKOq4aCE4+2LWpnZOPVdBQI/4mT0bpRFeLvHDyNpcX7+3tCjyPdkvU2udAiVmP5c0VEV+T60hfqIfWHsTbu08G7etzuFFk0kGXZ3cvSrCiuRLb2vqjNpLlE8991hr0fGNLahc4KZIvjBHJp2PhNdTJW4w6DA6rpwQpdftGSlVVF6Vn0TkcefFLP9hpwy1PbQ48j6RZPeT04JkNx9BUYQm7P1+QInkA2Hy0L2hft82FijE+ESoS8ycIuj87x4i2vNRL8fQ3hLtaT4rpDJvLC5NeEzX9WVNshN3tU3WIhjtMJD+upAADw56U71YiIXX7miI4eSnwIiefJEUhkcOQK/wXSFqUXdxYprpNmUSeimkJqbLptpECZSRm1wmDvXe3D2bYkvTxhVm1geqy/hTLR4ecXhQao6cBRyJa9VI2Hq8fBm2wSFhdqVDSGalJMFWkRrBITr7KakShUYdtbekXwcsPJx+SX440hUVadP3Byimq25RJGGO4dukEAKPrvrttLhr7F4FSiwGFRl1GFQPTRa+4UDijrmhE7yhFJ29zeaMuugIjTv5Yb+rVPJHw+PxBkgYAMF508rFmLSSLlD2IVD6q12owsdISVR1WLfLCyYd+sSJNYTnQYcM506oi/iHyiXsum4mVp1QH5tcCQI/NhcNd9ohlXoSQ6mofyH8nf9WjnwIQRtVZjTowBjzxSUtK/+9DTk/URVcAmFtfApNeg3X7u5M+TizC5eSn1Vih1TCs298V4V2pIfUImKJ0zlsMOjhc6e+ozgsnL81wlLCFybt5fX4c7rahuTo/NGriocSsR69duC12e/1YcO+7AIRbRyI840oKcKI/v4XK/H6O/R3CXe35M2qg0TBwLiiy3v7ijqQ/1+aMHcmb9FosbirH27tORn1dKoTLyZsNOvj8HE+uP4qWbuXvIhxuHww6TdSCBotRGxj2nU7ywskDwOM3LMR/XzwdAHDnq7tGKb4d7XXA4+NoDplWk8+MLy1Ax6ALT60/iv7hkU7OfxNTOcRoaooy15mYLqRegF98aVZgQVDC609+8dXm8saM5AFgcWMpTgw4VVOj9PhG5+SBkbUqNfLyDrc3aLZrOMwGneribOHIGyd/9rRqnCtKmx7otOHbz3wetF8a8TdpDDn5ry6qBwDsOjEYyLf+7up51O0ahQqrAb1296j+gnxCUkNcMGGkAOHFW5cBiFyZFg9DTm/U8kmJcSVCfjxUQ0opwtXJAyNVRL0qzFu1u3ywhNGSl2MxasNmGdQmb5w8MKJyBwDb2wbw23cPBJ6vFm8Pw6nE5Su1xQU4pbYIXUPOwAR7aoKKTkWhET4/D2j85CPdNiGFJ/+9LJhQigtn1qS0+Drk9MAaRyRfV6JupYvHOzonD4ykKVOtIgpH3JE8OfnUMOm1+OrC+sBEmt+8ux+AIEj2gqjRYYnjS5hPVFmN6Bh0wSYu+BSGGU9GjCCVl3bnsVBZ15ALBq1mVOlxcYE+EAwkitPjw6DTG1d5bqCcUaUqJneY6hoAsJqEAEeNGn272wdzDN9iMerg8Ag9Asme52TIKycPAL/48mw8edNiLBRntrb2OoL+qOFmMOYzVVYjOoecMvGosXWRSxTJSfXY1G29zyRdNhcqrcYgbRdAKEVOtttXavIJzfGHo7rIBA1TMZKPkJM36TXQaVjEEutUcLi8sMSI5C0GLTgHznrwfcy5623FbYhE3jl5QKgTX/WFUwAAe9oHg0oqQytx8p3qIhO6hkYi+Vi3lGOdSquQzuvKYycvNMSNTlsWF+jh9PiT0ng51CVU60ysjN1NrtdqUGI2qJYSc3vD5+QZY7CadOpF8jFy8lKkL90lesOoxKpBXjp5YGSB9ZF1h1XJweUKVUVG+PnIfEly8tEZK+macGkVKX1z0W8/TPgzpSafupL4ZjMIzlad/HSkhVdAEOh7av0xxYXCHG5vzCxBWUjBQ7pSNnnr5IvE/Nvmo3043GWL8er8pUqcxvPbNcIi9FhoBEuF4oL0DLbIFN02F/a0DwaNwJQoECPRQ132hHXPJYddVBBfOlBNJx8pkpezZm+HYsfjnMPmjL3wGtqEmC7Z4bx18nL+/KGgWfObr87JsCXpp6ooOGKLdUs51mGMob60YFSfRb4gSXusaK4ctc8oW6xMVHt9yOmBVsMiqjCGUlygD8w2UJKBYQ9ODjpjdnUrWSHbPuBEj92N5qrojZahKTI1/v/DkddO/oMfnxn0/PJ5+TUBKh4krRAJrWb0ghQRTEOZOWykmw90ioNkrlpcP2qffJh9omqNNqfQCBW6mBuJupICVTSCum0u+DkwI8Lc5h+fPxVA4v9/0ZCmrY0rib7oXBYyUKQ/FyJ5xtiVjLFdjDE/Y2xhyL47GGMHGWP7GGPnp2ZmckwotwRqgSP90fMdEiNLnIYyMyROVo8AACAASURBVI71ODIyqk1tpGHS0lBtOXInH29nps/PMej0YCgOSQM5tcUF6BxyhR1RmQq2GNOpbj1jEgAo2m0rSRXEKs8OncaWrl6MVCP5nQCuALBOvpExNh3AVQBmALgAwB8ZYxlJBs8S5WO/fdbkTBw+4xiiCCYR4akvM2PI5U1bpJVOOodcMOo0YXPn8nRNvBorT37agtl3vo0dxwcCdejxIM006FO4+1TqKI3UfarRMBi0mpQ6eyMdM1YqVKNheOlby7Dux2cByJGFV875Hs75vjC7LgXwHOfcxTk/AuAggMWpHCtZbjljEhrKzGM2kgeAt394eqZNyCkkffV8TNk8s+EYGEPYtEqQk49TLXHN3k4AgpSILoFUYIXYea50FVNgBGGUuwqTXqNoJC8pS8bTgzO/oTTQDPbi58rO2I2EWqtwdQDWy563idtGwRi7GcDNANDQ0KC4IYubyrDutrMU/9xcYkq1FbddMBUT83willI0lAtOvqXHjjn1JRm2RjlsLm9U7RStRp6uiS+Sl1eUaBJw8uVS05ld2X4EKV0TTSjNpNcqOu9Vkhk2x9loKK2L7WkfxJFuu+qT6mJaxRh7F0BNmF2rOOf/TNUAzvmjAB4FgIULF+ZfEjRL+NaZYzNdlQyTKgth0Gmwo20Al84NG5vkJJLE7ooI842bqwoxrtiEEwPOuCP5PocH8xpKML+hFJfPi/9cSdUvPWpF8lGcfIFBq6gapHRXYNQnnhjZ3zGUeSfPOV+ZxOceByBfvh8vbiOIrEev1WDmuKKMjGpTE6ks9PYLp4XdbzHq8OzNS3HGA+/HHcl321yYVmPFT0WZ73ipsEhNZwpH8q7Yi6AlBXpFa9QDo/+S6KZPR6muWqtyrwK4ijFmZIw1AWgGsFGlYxGE4sxrKMW2toFAXXk+IJUsji+N3JUqLR7a44x0uyN0z8aiqECYRvXMhmPwKVi0bnN5odeyoPWFUMoshsAwHSWQegoSieSldbK2vmG8uLkNS3+2RjV561RLKC9njLUBOBXA64yx1QDAOd8F4HkAuwG8BeDbnPP0q+UTRJLctLwJRp0Gv197IPaLc4S2PgeKTLqoctPS4uHnR/tw0xOfRRVqc3njV54MhTFhGtXhbjve3Nme8PsjYXfFrtcvNRsUrZxyenxgDFEvLKFMqbZiWo0Vrb0O/Mc/tuHkoBM2laZGpVpd8zLnfDzn3Mg5r+acny/bdx/nfBLnfCrn/M3UTSWI9DGupACnTizH9raBTJuiGN02d5CGfDiklMPLW45jzd5OnPPrDyK+Vhq+kYyTB4AJ4gL3hwrOe7U5vTHr1S3isHalhoc4PT4YdZq4G8EkxpUUBA32Vnp9QoKKqAkiAjPrinG4265od2QmGXJ5Y9ayh1bIRIt4u4ckJ5/cIJ7Xv7cCS5rK8OnhnqTeH46hOEYQSheBU3++RpFjOj1+mOKUc5BTXWQMdCADwI1PfKaIPaGQkyeICEiyuUrUy/9z63EsuOcd1eaaxoPN6UmoKxUAptdG7i+RFk0rkhwMX2jUYU59CU4OOhXrLrbH4eSlwTmJ6vNEwunxJbXoajXpYXONXESnVkfXvkkWcvIEEYEJZYKTP9qTupP//nNb0WN3Y584RDsTxDto+/4rZgUea6J4CElzv8KSvHRGZaERbq9fsRy5zRV7zmyiaZVYOL1+mJIonzQbtEGdt3dfOkNJswKQkyeICEhNUcd67Yp95s1PblLssxJFEhGLxRXzx+OXX5qNC2fWRKwn55zjthe2AxCGnyeL1Gy24Uhv0p8hJ57/x1KFB9k7Pb6k0jVyO69dOgFVRbGnaiUDOXmCiEBxgR6FRh1O9CunLd8xmLmJU0Ou2IuSgKB39JVF9bCadIGW/VCOdI9c+FKRr549vhiMQbE7nHjuVq5cKKjR1ijkVJ0eH4xJOHn5eUukMidRyMkTRBRqi01oH1BnFmk64ZzD5kpMKdJs0EVsilq3vwsAcNnccSnZZdJrUVKgR5dNmQtpPE5er9Xg6sX18Cm0DuDy+GFKwklLGjYAoHAGKQhy8gQRhdqQMrdcxeH2gfPo7f6hFBi0AV0WCY/PD7fXj/ZBJ/Rahl9/ZW7KtpUXGtE1lPodjs/P4XD74rpbMem1cCokbeD0JpeumTt+RBepQMVhPuTkCSIKtUUmRZy8XC9Grc7GaMSjzhiKxaCFx8fRY3MFKmnO/tX7mHXnanQMOFFlNSUkShYJh8uL1bs68Od1h1P6HEkeOZ67FbNBiyGXVxG5X5cnuYXXYrMen61aiW+ePhHfWNGUsh2RICdPEFGoLTGh2+aCO8VyO78sNeBUUAExXobiUGcMRYouF9z7Lhbe+y4AoLV3GC6vHx2DLtQUK5PTvuvSmQCA/313f0qllJICZTyRvDSmcM5db6c8zzfZSB4AKq1G3HHRKYGZ1GpATp4golBbbALnSNkReH0jzktJBcR4kSL5RHLyoQ5X/rxj0KnYwuW506txx4XTYHf7Ujo39jgUKCXkTnmtqImfLMnWyacLcvIEEYXaYmFxLNWUjVyE65Utx9M+WvCVLYIIbKEx/ohxeYgk8aBzZBH2cLd91PzgVJDSSNH07mMxlEBKqkCmg//+vs6U/h7OJNM16SJ7LSOILEAazpxqhY1X5uTvfX0PXtmaPuXtTS29eOKTFgBCK328TKsJ7nYdDMlf1xQrNz9Yir6HnMk7+XgGhkjUlYxUtqze1YHVu04mdUy3149BpyfpdE06ICdPEFGoESP5VGvlvX4/5GuUSnTRxkuL7FjSnUm8WGQR76Az2MkrGclLOekhZ/ILof3iRSgeJ3/a5Ar8cOWUwPO9Sdbp7zs5BM6BZpUkCZSAnDxBRKHQqEO5xYAj3anpynt9HEUyiV+9Nn0/Pbl2eqKD3V/73gp8eYHQPNRnD3bA40sTu2BEQ4l0zXpR6KzMErujVa/V4PsrmwPPTyaZjpPsHafQIrQakJMniBhMH1eEXScGU/oMn58HLXoOp3HxtVd0zjcsa0z4vU0VlsD72vqEOwKpMmVqTWTxskSRzk0q6Rq3148iky6pO4zj/cml44Y9gr3yHH+2QU6eIGIwfVwRDnTYUiqj9Po5KmW663aVBkSEw+nxobhAjzsvSU4ASxoyIqU07rt8JnbceV5C5ZixkD7LloKTd3p8CWvbf21JAwDgwwPdeCuJ4SVSNVAq0g5qQ06eIGIwc1wx3D4/9p5MPpp3eXyYWFmIX355NoD0RvIury8lbRQpzSQt3paY9TF16ROlRBQN63UkPzgjGaGwn10+Cz+5SJh5e8tTnyd8zBEnT5E8QeQsixrLAAgj8ZLF5fXDqNPgKwvr0VRhiXuGqhK4vP6Ec/FyQscFFhcoq+IICJG82aBNOjcOJF/KeN2pjYHHLd2JKY5KF2tK1xBEDlNdZESRSYcDKQz1Fpy84AjMBi0caZw2JV1gUuGrC+sDj6PNiE2FGeOK8Nr2dnh8yaXFBoaTK2U06bV48qbFAIAfPb81ofdSJE8QeQBjDFOqrTjQkYqT98EoRpkWgy6tXa8uz8gFJlnGyerKS8zqOPkr5o9Ht82VVDS/5VgfdhwfwOYk77Zm1wliYZ8f60/ofcPi2gp1vBJEjtNcbcX+zqGkOiN9fg6PjweiabNRG1HCVw1cXl9K6RpA0H2XUCuSlxqUTiYhIfHhAWEYeLIj/YplF65E/sYOtw8Feq0iQm1qQU6eIOJgXn0J+h2epEoppaocKZouMulTWmBM5vippmvOnFoZeKxWjX9tsdRdnLiTdykg+vbTi6cDiD68PBSHx5fVqRqAnDxBxMXK6dXQahje2pl4+7sUtReI6ZoJ5Wa09g4nnVpIFJfXn9TkIjmMMWxcdQ5evPVUhawajaRqeSKJmnUpXSK/GCWKdJE5kYCExbDbl9WLrgA5eYKIizKLAUuayvBmErXUIwqQQkrgygXCIubavR3KGRgFl9cPgwLRd5XVhAUTyhSwKDxWkx7VRUbsT0JiQJIG+uM181M4vlDrnsh6icPthSWLa+QBcvIEETcXzarFoS47dp0YSOh9Uhen5EQays2oKTKhM03zXt2yRd9sZ3ptEXa3J5ES8/mgYak1JUmdvIk5eYrkCSJvOG9GNQDg8Y9aElqcCwzskMkalFkM6LWnJy/v8vphTKNWTipMH1eEg522hHPs7hR7AYCRWvf1h3vi/vsOuyknTxB5Q5XVhJtPn4gXP2/Dltb4S+0C6RqZlnuhSZeSGFciCDn53PipT68thtfPEy5X9fh4yikpKZL/0/uH8M+tJ+J6j4OcPEHkF985ezIMOg0e+/BI3NGeJJ8rFyizGtPn5N3e1Ovk08X0cYLo2Y7jiaXEhK7e1P4f5ameH/x9a1x/32GPT9Uh3EpATp4gEqDIpMfXT2vE6zva466OCTdEO52RvNOTep18uphQZkaJWY83E6xiUqJMtLwwWK5hw5HemO9xuL0wZ/HAEICcPEEkzC2nTwIwol8ei9CFV0DQaklFcTFevD4/XF5/1qcUJDQahnOmVWNPgouvbp8fem1qDUmh9f+Hu2Lr2NDCK0HkIaUWA6bVWLFO7LKMxZDTC4NWE5QySVckLwmhKSkLrDZTqgvRNeTCwHD8TUluBbp6gZHhH4zF13lLC68EkafUFpuw8UgvPtjfFfO1Npdn1HBpq1EHl9efkkZ9PNhd8c89zRYmVxUCAA4mIAjn8XFFnPw7PzoDW356LopMevxuzQHc/a/dEV/r9vrh9XNy8gSRj3z3HGF03BvbYzdHfbC/K1C5IREYkqFyNC85eUsOOfmpNcK81Dd2xN945lao4cti1KHUYgjcRTz+8RGc9eD78IZRxhyRGc7uc0tOniCSYH5DKVY0V2B7jCqQ1l4HWnuHR42Xk4Zk9KmsYWPLwUh+fKkZVy+ux18/PoL2OCUGlKiTj8SRbjva+kbb4RBH/1EkTxB5yrz6Euw7ORiIlsOxVayn/+HKKUHbpTF13UPqdr3acjCSB4CbT58EPwdWx1ll4/L5FRVO++1Vc4Oe7w0jtWB3Zb+WPJCik2eMXckY28UY8zPGFsq2NzLGhhljW8V/D6duKkFkF/MaSuHn0Wu6v/vsFgDAN1Y0BW2vsAqRfLdN3Uh+JF2T3Y4olMZyoZRyf5x5eY8CJZRyLp1bh42rzsF/feEUAMAtT22G3x9cNx9I1+R5CeVOAFcAWBdm3yHO+Vzx3y0pHocgso5ptULuONLEKKdHcAJ1JQWjImkpkr//rT0qWgjYXLlXXQMIqpeTKwvjXnx1+5RP11RZTfjGiomB5+/t68RfPjwMAOh3uPHF338EILuHeAMpOnnO+R7O+T6ljCGIXKKmyASzQYvDXeEdUZeYirn1zEmj9pWKOfnW3sRldROh1y7YUGpRfi6r2jSUm7HxSC+2HIvddOb0+FSbzvTzK2YBAG762ybc+/oeONxeHOt1BPaP5Tr5JsbYFsbYB4yxFZFexBi7mTG2iTG2qasrdjkaQWQLjDE0VVhGdb46PT5MWfUmVvzyPQDhx+VpNQwrmisAYFQaQEk6B10o0GthzbFIHhg5L19/4rOYr+13eIKmOynJnPElQc8PdtqCNO+zPScf8y/PGHsXQE2YXas45/+M8LZ2AA2c8x7G2AIArzDGZnDOR7Wxcc4fBfAoACxcuFC9bztBqEC/w4Pj/cNo6bajscICQIjg3bKSu1AnIbGiuQIfHujGsMen2sJox5ALVUVGMJa94+kiodUIMagvxkXQ7fXD5vKizKzO3Uqo3MElv/846Hm2O/mYkTznfCXnfGaYf5EcPDjnLs55j/h4M4BDAKZEej1B5Cq3XTAVAHDd4xsD24Y9IzK56358FurLzGHfK9VX21Wc99o56ES11aTa56vJt84S0lwzRNGySAyKAnBFKs2eLY1x8RiT6RrGWCVjTCs+ngigGcBhNY5FEJnk0rl1MOo0ONbrCHSvDsk0aRrKwzt4AAFhq+EEhlQkSueQC5VFRtU+X00mVRbivOnV6LNHlzeQzp9ad0OxFnSzfVE71RLKyxljbQBOBfA6Y2y1uOt0ANsZY1sBvADgFs55bEk3gshBHrxyDgBgf4dQSy3Vpn9xzrio75Nu8xOZRJQouRzJA0BVkRFdtui9BNKdkJppk68taUCVNfzFMtura1KyjnP+MoCXw2x/EcCLqXw2QeQKUjqmY9CJmXXFaOkW1AtvO39q1PcVBJy8Oukap8cHu9s3KqecSxSZ9Oi1u9HW58D40vB3RY6AvIB6Tv5nl8/ClQvG4/I/fhLY9tzNS1FXUqDaMZWCOl4JIkWkCK9TLJl8f18nSs36mA5ASi+oFcnnojhZKNKa672vRe4ncIi9AGoP1JZH7K99dzmWTiyPuN6STZCTJ4gUkRb8pAlQLT0OLJtUAY0mekWL1CmZ6Ki7eJEuHtle/RGN7549GUD0/wdHGtI1QHBn68y6YlWPpSTk5AkiRaQFVJvLh9ZeB4502wNj7KIhDRG5+7XIcrap4FB5QTIdWIw6NFVY8NKW43h710n02Fx4dN2hoN4CqZpJdSefoxdLcvIEkSIaDYPZoIXD5cU/tx4HY8Bl8+pivq9Bdqsf77zYREjHgmQ6qBUHefzloyO4+cnN+Nkbe3FI7DJ+duMxfP+5rQDUXwCVLso/WNms6nGUhpw8QSiAxaiD3e3FnvYhNJZb4lqQkzcouVQYHhLIVedwJA8AD109DwDQXFWItj5BTqB9wAmnx4f/emVn4HVqR9omvRYH77sQ3z+HnDxBjDkKjTrYXD60DwxjXEn8JYv3XDYTADCYwKi7eMmXSL680IiFE0qxtbUfOrEL9rrHN+K6xzcGNUql4/9Tp9XkXPcwOXmCUACLUUjXDHv8CaUNisVFW6lrU0mkBUm1q07SwfwJpaPUPjce6UV10cgFVUk9+XyCzgpBKIDZIAzmdnp8MCWgLy7lm3e3jx5KkSqBoRY5piUfjslVhXB7/aMmbMUoYCJATp4gFKHQqEPHoFOUvI3/Z7WgoRTVRUa8vSu+CUiJkE+RfLM43DsUp0dYy/j49rPTaU5OQU6eIBRgSrUVLT0OtA84E1oA1GgYZowrjns4RiJIkXy2Ty6Kh+Zqa9jtH+zvQnNVYU50nmYKcvIEoQCnTS4PPE4kXQMIqYjD3faYkrqJ4nB7YTZoYzZl5QKFRh02rjon7L5Ik7kIAXLyBKEAi5vKkn7v5Eoh39wqmzakBHa3L+cra+RUWU048vOLRm2fHCGVQwiQkycIBTDqtNj23+fhkjnjcMOyxoTeO0l0UkqnbOwub852aUYiXPniy99algFLcgdy8gShEMVmPX539TyMSzA/LEWihyLMik2W433DqC3Kv1z12dOqAAA3LGvE8988FVaTOsNC8oXcX3YniBynuECPErMerX3Kpmva+oaxXJwjm0/88Zr56LW7E76YjlUokieILKC+1IzW3uHYL0yAQacHJSqNxMskJr2WHHwCkJMniCxgfGlBQJdFCTw+Pxxun2pzT4ncgZw8QWQBgpMfVkyNUpozKyknEmMXcvIEkQWMLzXD5fXjZ2/sCdJKT5Z8mApFKAM5eYLIAurLhBzznz88gnf2dKT8eZICJTl5gpw8QWQB8iHVwwrMfB0RJyMnP9YhJ08QWYBce0UJeQMpXWPJs2YoInHIyRNEFiCf3qSEkw8oUFIkP+YhJ08QWcbRXntK73d7/TjR7wSQHzLDRGqQkyeILOHFW08FAOw8PpjS59z61Gbc/dpuAPkxMIRIDXLyBJElLJhQhi8vGI9dJwZSqpdfs7cz8Jiqawhy8gSRRTRXFaLb5oZNXDhNFWMCU6qI/IS+AQSRRVQVGQEAHYPOpD/DIDp2q1EXVpqXGFuQkyeILGJcsVBKuVaWckmUaTXCqLwhhe4GiNyGnDxBZBHShKk+hyfpz6iymgAAeTD1j1AAcvIEkUUwxlBpNaLX5k76M/ziou3r31uhlFlEDkNOniCyjHKLAT325J28x+fH/IYSnFJbpKBVRK5CTp4gsowDnTa8u6cj6cHeLq8fei39tAkB+iYQRJYhyRrsOD6Q1Ps9Pn+gwoYg6JtAEFnG898UOl+TrZX3+PwwUCRPiKT0TWCMPcAY28sY284Ye5kxViLbdwdj7CBjbB9j7PzUTSWIsUFzVSEAwOZM0sl7OaVriACpfhPeATCTcz4bwH4AdwAAY2w6gKsAzABwAYA/MsZIRIMg4qBQHNmXSiSvp3QNIZLSN4Fz/jbnXPomrgcwXnx8KYDnOOcuzvkRAAcBLE7lWAQxVtBrNTDpNRhyJlcr7/b5oddSkTwhoOTl/kYAb4qP6wC0yva1idtGwRi7mTG2iTG2qaurS0FzCCJ3sZr0lJMnFCHmN4Ex9i5jbGeYf5fKXrMKgBfA04kawDl/lHO+kHO+sLKyMtG3E0ReYjXqMCTm5L0+f0LDvd1UQknIiKlDyjlfGW0/Y+wGABcDOIeP6KMeB1Ave9l4cRtBEHFQaBpx8pNXvYm59SV45dunxfVej48WXokRUq2uuQDAbQAu4ZzLOzdeBXAVY8zIGGsC0AxgYyrHIoixhNWkg83lRa/Y+bq1tT/u97qpTp6QkepEgd8DMAJ4R5Q0Xc85v4Vzvosx9jyA3RDSON/mnKc+gp4gxgiFRh0+PtiB+fe8k9D7OOdiTp4WXgmBlJw853xylH33Abgvlc8niLGK1aRP6n0+PwfnoHQNEYC+CQSRhZSak3PyHp+wLEZ18oQEfRMIIguZUG4ZtW3n8YGouXmPz48Vv1wLgCJ5YgT6JhBEFvLF2eNGbbv4oY9w2R8+jviebpsL3aIOPeXkCQly8gSRhRQnka7plWnQUyRPSNA3gSCylL/dmJgSyO/XHgw8phJKQoK+CQSRpZwxpRL1ZQVxv77HRpE8MRr6JhBEFvPkjUtGReUenz/o+aqXd+DRdYfQZXMFtpGTJyRSbYYiCEJFGiss+NG5U3D/m3sD24Y9voAT9/j8eHrDMQBCA5WEkdI1hAh9EwgiyynQB49icLpHmsftMqVKuWqlUU8/bUKAvgkEkeV8dVE9vnnGRPz04ukAAIfbh+c2HoPT44soRxx6YSDGLpSuIYgsx6TX4o4LT8EbO9oBAK/vaMcDq/dhW1s/Fkwoi/geggDIyRNEziBF5+0DwwCAZze24tmNrWFfS06ekKB0DUHkCJLjtrtiC7qaKCdPiFAkTxA5QoFBcPIvb4k8f2dFcwXaB5yotprSZRaR5ZCTJ4gcIZ7F1FvPnIRlkyrSYA2RK9A9HUHkCPE4eaqPJ0KhbwRB5AgmQ/ifa1PFiCyxUUcLrkQwlK4hiByhpMAQePyna+bjwlm14FwYEtJ0xxsAgOm1RRmxjcheyMkTRI4g17CpKhIWVsXZytj4k3Pg54BGQzryRDDk5AkiB6kpDq6ekZw+QYRCOXmCyEGqrcZMm0DkCBTJE0QO8dRNS3Cifxg6khIm4oScPEHkEMubqQaeSAwKBwiCIPIYcvIEQRB5DDl5giCIPIacPEEQRB5DTp4gCCKPISdPEASRx5CTJwiCyGPIyRMEQeQxTFKxywYYY10AjqbwERUAuhUyR03ITmXJFTuB3LGV7FQeNW2dwDmvDLcjq5x8qjDGNnHOF2bajliQncqSK3YCuWMr2ak8mbKV0jUEQRB5DDl5giCIPCbfnPyjmTYgTshOZckVO4HcsZXsVJ6M2JpXOXmCIAgimHyL5AmCIAgZ5OQJgiDymLxw8oyxCxhj+xhjBxljt2fYlnrG2HuMsd2MsV2Mse+L28sYY+8wxg6I/y0VtzPG2O9E27czxuan2V4tY2wLY+w18XkTY2yDaM/fGWMGcbtRfH5Q3N+YZjtLGGMvMMb2Msb2MMZOzcZzyhj7ofh338kYe5YxZsqWc8oYe5wx1skY2ynblvA5ZIxdL77+AGPs+jTZ+YD4t9/OGHuZMVYi23eHaOc+xtj5su2q+oVwdsr2/QdjjDPGKsTnGTuf4Jzn9D8AWgCHAEwEYACwDcD0DNpTC2C++NgKYD+A6QB+CeB2cfvtAH4hPr4IwJsAGIClADak2d4fAXgGwGvi8+cBXCU+fhjAreLjbwF4WHx8FYC/p9nOvwH4hvjYAKAk284pgDoARwAUyM7lDdlyTgGcDmA+gJ2ybQmdQwBlAA6L/y0VH5emwc7zAOjEx7+Q2Tld/M0bATSJvkCbDr8Qzk5xez2A1RAaOysyfj7T8eVX+Yt7KoDVsud3ALgj03bJ7PkngHMB7ANQK26rBbBPfPwIgKtlrw+8Lg22jQewBsDZAF4Tv4Ddsh9T4NyKX9pTxcc68XUsTXYWi86ThWzPqnMKwcm3ij9YnXhOz8+mcwqgMcR5JnQOAVwN4BHZ9qDXqWVnyL7LATwtPg76vUvnNF1+IZydAF4AMAdAC0acfMbOZz6ka6QflkSbuC3jiLff8wBsAFDNOW8Xd50EUC0+zqT9/wvgNgB+8Xk5gH7OuTeMLQE7xf0D4uvTQROALgB/FVNLf2GMWZBl55RzfhzAgwCOAWiHcI42IzvPqUSi5zAbfm83QoiKEcWejNjJGLsUwHHO+baQXRmzMx+cfFbCGCsE8CKAH3DOB+X7uHDJzmjtKmPsYgCdnPPNmbQjTnQQbov/xDmfB8AOIbUQIEvOaSmASyFclMYBsAC4IJM2JUI2nMNYMMZWAfACeDrTtoTCGDMD+AmA/860LXLywckfh5ADkxgvbssYjDE9BAf/NOf8JXFzB2OsVtxfC6BT3J4p+08DcAljrAXAcxBSNr8FUMIY04WxJWCnuL8YQE8a7ASE6KaNc75BfP4CBKefbed0JYAjnPMuzrkHwEsQznM2nlOJRM9hxn5vjLEbAFwM4BrxgoQo9mTCzkkQLvDbxN/VeACfM8ZqMmlnPjj5zwA0ixUMBggLWK9myhjGGAPwGIA9nPNfy3a9CkBaOb8eQq5e2n6duPq+0h/iTwAAAWhJREFUFMCA7PZZNTjnd3DOx3POGyGcs7Wc82sAvAfgyxHslOz/svj6tER9nPOTAFoZY1PFTecA2I0sO6cQ0jRLGWNm8Xsg2Zl151RGoudwNYDzGGOl4p3LeeI2VWGMXQAhtXgJ59wRYv9VYqVSE4BmABuRAb/AOd/BOa/inDeKv6s2CEUYJ5HJ86n0QkQm/kFYud4PYTV9VYZtWQ7hlnc7gK3iv4sg5FrXADgA4F0AZeLrGYA/iLbvALAwAzafiZHqmokQfiQHAfwDgFHcbhKfHxT3T0yzjXMBbBLP6ysQKhGy7pwCuAvAXgA7ATwJoeojK84pgGchrBV4IDigm5I5hxBy4gfFf19Pk50HIeSupd/Uw7LXrxLt3AfgQtl2Vf1CODtD9rdgZOE1Y+eTZA0IgiDymHxI1xAEQRARICdPEASRx5CTJwiCyGPIyRMEQeQx5OQJgiDyGHLyBEEQeQw5eYIgiDzm/wMyLwpVv9kknQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dufvB464Woe1"
      },
      "source": [
        "## 3-1) 데이터 전처리\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EbQWUXojWWMz"
      },
      "source": [
        "# column마다 범위가 다르므로, 이를 정규화한다.\r\n",
        "# 정규화를 위해 표준편차와 평균을 구한다.\r\n",
        "# 처음 20만개 타입스텝을 훈련 데이터로 사용한다.\r\n",
        "mean = float_data[:200000].mean(axis=0)\r\n",
        "float_data -= mean\r\n",
        "std = float_data[:200000].std(axis=0)\r\n",
        "float_data /= std"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6jvywJNoXo1W"
      },
      "source": [
        "# samples는 입력 데이터로 사용할 배치\r\n",
        "# targets는 samples에 대응하는 타깃 온도의 배열\r\n",
        "# 한시간에 하나의 데이터를 추출하기 위해 step=6\r\n",
        "def generator(data, lookback, delay, min_index, max_index,\r\n",
        "              shuffle=False, batch_size=128, step=6):\r\n",
        "    if max_index is None:\r\n",
        "        max_index = len(data) - delay - 1\r\n",
        "    i = min_index + lookback\r\n",
        "    while 1:\r\n",
        "        if shuffle:\r\n",
        "            rows = np.random.randint(\r\n",
        "                min_index + lookback, max_index, size=batch_size)\r\n",
        "        else:\r\n",
        "            if i + batch_size >= max_index:\r\n",
        "                i = min_index + lookback\r\n",
        "            rows = np.arange(i, min(i + batch_size, max_index))\r\n",
        "            i += len(rows)\r\n",
        "\r\n",
        "        samples = np.zeros((len(rows),  # 128(batch_size)\r\n",
        "                           lookback // step,  #1440//6 = 240\r\n",
        "                           data.shape[-1]))  #14\r\n",
        "        targets = np.zeros((len(rows),)) # 128개의 영행렬\r\n",
        "        for j, row in enumerate(rows):\r\n",
        "            indices = range(rows[j] - lookback, rows[j], step)\r\n",
        "            samples[j] = data[indices]  # 독립변수\r\n",
        "            targets[j] = data[rows[j] + delay][1]  # 종속변수\r\n",
        "        yield samples, targets"
      ],
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "keTqmpMtcK1M"
      },
      "source": [
        "lookback = 1440\r\n",
        "step = 6\r\n",
        "delay = 144\r\n",
        "batch_size = 128\r\n",
        "\r\n",
        "train_gen = generator(float_data,  # 처음 20만개를 train data로 사용\r\n",
        "                      lookback=lookback,\r\n",
        "                      delay=delay,\r\n",
        "                      min_index=0,\r\n",
        "                      max_index=200000,\r\n",
        "                      shuffle=True,\r\n",
        "                      step=step, \r\n",
        "                      batch_size=batch_size)\r\n",
        "val_gen = generator(float_data,\r\n",
        "                    lookback=lookback,\r\n",
        "                    delay=delay,\r\n",
        "                    min_index=200001,\r\n",
        "                    max_index=300000,\r\n",
        "                    step=step,\r\n",
        "                    batch_size=batch_size)\r\n",
        "test_gen = generator(float_data,\r\n",
        "                     lookback=lookback,\r\n",
        "                     delay=delay,\r\n",
        "                     min_index=300001,\r\n",
        "                     max_index=None,\r\n",
        "                     step=step,\r\n",
        "                     batch_size=batch_size)\r\n",
        "\r\n",
        "# 전체 검증 세트를 순회하기 위해 val_gen에서 추출할 횟수\r\n",
        "val_steps = (300000 - 200001 - lookback) // batch_size\r\n",
        "\r\n",
        "# 전체 테스트 세트를 순회하기 위해 test_gen에서 추출할 횟수\r\n",
        "test_steps = (len(float_data) - 300001 - lookback) // batch_size"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DDZGzBhWiAbt"
      },
      "source": [
        "## 3-2) 상식 수준의 기준점"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v8Ie8g37gXSB"
      },
      "source": [
        "현재 온도와 24시간 후의 온도가 동일하다고 예측하고, 그 둘의 절대오차의 평균을 구해보자."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RCEvdFoZfWq2",
        "outputId": "361f57ae-8aaf-4ff9-ce8a-b53e4ec03ec3"
      },
      "source": [
        "def evaluate_naive_method():\r\n",
        "    batch_maes = []\r\n",
        "    for step in range(val_steps):\r\n",
        "        samples, targets = next(val_gen)\r\n",
        "        preds = samples[:, -1, 1]  # 24시간 후의 온도 예측값\r\n",
        "        mae = np.mean(np.abs(preds - targets))\r\n",
        "        batch_maes.append(mae)\r\n",
        "    print(np.mean(batch_maes))\r\n",
        "    \r\n",
        "evaluate_naive_method()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.2897359729905486\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zfDg-3hThjgp"
      },
      "source": [
        "평균 절댓값 오차(MAE)는 약 0.29이다.  \r\n",
        "딥러닝 모델이 더 나은 성능을 가지고 있는 지 확인해보자."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LTH7R9Ikim5p"
      },
      "source": [
        "## 3-3) 기본적인 머신 러닝"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "07wx89Ofg5TA",
        "outputId": "a728df6e-5c0d-41fe-d748-ad71e7167273"
      },
      "source": [
        "from keras.models import Sequential\r\n",
        "from keras import layers\r\n",
        "from keras.optimizers import RMSprop\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "\r\n",
        "# 1440//6x14 개의 데이터를 flat하게 만듬\r\n",
        "model.add(layers.Flatten(input_shape=(lookback // step, float_data.shape[-1])))\r\n",
        "model.add(layers.Dense(32, activation='relu'))  #32개의 출력을 내는 노드\r\n",
        "model.add(layers.Dense(1))\r\n",
        "\r\n",
        "model.compile(optimizer=RMSprop(), loss='mae')\r\n",
        "history = model.fit_generator(train_gen,\r\n",
        "                              steps_per_epoch=500,\r\n",
        "                              epochs=20,\r\n",
        "                              validation_data=val_gen,\r\n",
        "                              validation_steps=val_steps)"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.\n",
            "  warnings.warn('`Model.fit_generator` is deprecated and '\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "500/500 [==============================] - 16s 31ms/step - loss: 1.7436 - val_loss: 0.6419\n",
            "Epoch 2/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.5789 - val_loss: 0.3929\n",
            "Epoch 3/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.3403 - val_loss: 0.3353\n",
            "Epoch 4/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2772 - val_loss: 0.3157\n",
            "Epoch 5/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2597 - val_loss: 0.3078\n",
            "Epoch 6/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2485 - val_loss: 0.3063\n",
            "Epoch 7/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2393 - val_loss: 0.3177\n",
            "Epoch 8/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2359 - val_loss: 0.3125\n",
            "Epoch 9/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2256 - val_loss: 0.3150\n",
            "Epoch 10/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2222 - val_loss: 0.3231\n",
            "Epoch 11/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2228 - val_loss: 0.3533\n",
            "Epoch 12/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2165 - val_loss: 0.3258\n",
            "Epoch 13/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2149 - val_loss: 0.3193\n",
            "Epoch 14/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2124 - val_loss: 0.3251\n",
            "Epoch 15/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2099 - val_loss: 0.3344\n",
            "Epoch 16/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2067 - val_loss: 0.3497\n",
            "Epoch 17/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2068 - val_loss: 0.3360\n",
            "Epoch 18/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2035 - val_loss: 0.3326\n",
            "Epoch 19/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2030 - val_loss: 0.3561\n",
            "Epoch 20/20\n",
            "500/500 [==============================] - 15s 30ms/step - loss: 0.2016 - val_loss: 0.3633\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        },
        "id": "jUYJCOfshPOk",
        "outputId": "9a25edc9-5696-413b-9617-6361bf3a7475"
      },
      "source": [
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "loss = history.history['loss']\r\n",
        "val_loss = history.history['val_loss']\r\n",
        "\r\n",
        "epochs = range(1, len(loss) + 1)\r\n",
        "\r\n",
        "plt.figure()\r\n",
        "\r\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\r\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\r\n",
        "plt.title('Training and validation loss')\r\n",
        "plt.legend()"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f24ed1216a0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhU5Zn38e/NJrIoAg2ygwquLA2NG4K4vBNAB1zASIhImIgaE7ckhoREuczw5s3EmctxomaIiRrTiksmjIkajAsBNSY2iCiKERWkERCbsBhEu+F+/3hOdVc3Vd3VdHVV9+nf57rqqqqz3nW6+neees6pU+buiIhI89cq3wWIiEh2KNBFRGJCgS4iEhMKdBGRmFCgi4jEhAJdRCQmFOiSkpk9ZWaXZ3vafDKz9WZ2biMs183smOjxz8zsB5lMexDrmWFmTx9snbUsd7yZlWZ7uZJ7bfJdgGSPmX2S9LQD8BmwL3p+pbsXZ7osd5/YGNPGnbtflY3lmNlA4H2grbtXRMsuBjL+G0rLo0CPEXfvlHhsZuuBr7r7MzWnM7M2iZAQkfhQl0sLkPhIbWbfMbMtwL1mdoSZ/d7MtpnZ36PHfZPmWWpmX40ezzKzF8zstmja981s4kFOO8jMlpnZbjN7xszuNLNfp6k7kxp/aGYvRst72sy6J42/zMw2mFmZmc2rZfucYmZbzKx10rALzWx19PhkM/uzme0ws81m9lMza5dmWfeZ2b8mPf92NM+HZja7xrTnmdmrZrbLzDaa2fyk0cui+x1m9omZnZbYtknzn25mr5jZzuj+9Ey3TW3M7Pho/h1mtsbMJieNm2Rmb0bL3GRm34qGd4/+PjvMbLuZLTcz5UuOaYO3HEcCXYEBwBzC3/7e6Hl/4FPgp7XMfwrwNtAd+DfgF2ZmBzHtg8BfgW7AfOCyWtaZSY1fAr4C9ADaAYmAOQG4O1p+72h9fUnB3f8C/AM4u8ZyH4we7wNuiF7PacA5wNdqqZuohglRPf8HGAzU7L//BzAT6AKcB1xtZhdE48ZF913cvZO7/7nGsrsCTwB3RK/tP4AnzKxbjddwwLapo+a2wO+Ap6P5vgEUm9mx0SS/IHTfdQZOAp6Lhn8TKAUKgJ7A9wBdVyTHFOgtx37gFnf/zN0/dfcyd/+Nu+9x993AAuDMWubf4O4/d/d9wP1AL8I/bsbTmll/YDRws7t/7u4vAI+nW2GGNd7r7n9z90+BR4AR0fCpwO/dfZm7fwb8INoG6TwETAcws87ApGgY7r7C3V929wp3Xw/8d4o6Urkkqu8Nd/8HYQeW/PqWuvvr7r7f3VdH68tkuRB2AO+4+wNRXQ8Ba4F/Tpom3bapzalAJ+D/RX+j54DfE20boBw4wcwOc/e/u/vKpOG9gAHuXu7uy10Xiso5BXrLsc3d9yaemFkHM/vvqEtiF+EjfpfkbocatiQeuPue6GGnek7bG9ieNAxgY7qCM6xxS9LjPUk19U5edhSoZenWRWiNX2RmhwAXASvdfUNUx5CoO2FLVMf/JbTW61KtBmBDjdd3ipk9H3Up7QSuynC5iWVvqDFsA9An6Xm6bVNnze6evPNLXu7FhJ3dBjP7k5mdFg3/CbAOeNrM3jOzuZm9DMkmBXrLUbO19E3gWOAUdz+Mqo/46bpRsmEz0NXMOiQN61fL9A2pcXPysqN1dks3sbu/SQiuiVTvboHQdbMWGBzV8b2DqYHQbZTsQcInlH7ufjjws6Tl1tW6/ZDQFZWsP7Apg7rqWm6/Gv3flct191fcfQqhO2YxoeWPu+9292+6+1HAZOBGMzungbVIPSnQW67OhD7pHVF/7C2NvcKoxVsCzDezdlHr7p9rmaUhNT4GnG9mZ0QHMG+l7vf7g8B1hB3HozXq2AV8YmbHAVdnWMMjwCwzOyHaodSsvzPhE8teMzuZsCNJ2EboIjoqzbKfBIaY2ZfMrI2ZfRE4gdA90hB/IbTmbzKztmY2nvA3WhT9zWaY2eHuXk7YJvsBzOx8MzsmOlayk3DcobYuLmkECvSW63bgUOBj4GXgDzla7wzCgcUy4F+Bhwnny6dy0DW6+xrgGkJIbwb+TjhoV5tEH/Zz7v5x0vBvEcJ2N/DzqOZMangqeg3PEbojnqsxydeAW81sN3AzUWs3mncP4ZjBi9GZI6fWWHYZcD7hU0wZcBNwfo26683dPycE+ETCdr8LmOnua6NJLgPWR11PVxH+nhAO+j4DfAL8GbjL3Z9vSC1Sf6bjFpJPZvYwsNbdG/0TgkjcqYUuOWVmo83saDNrFZ3WN4XQFysiDaRvikquHQn8D+EAZSlwtbu/mt+SROJBXS4iIjGhLhcRkZjIW5dL9+7dfeDAgflavYhIs7RixYqP3b0g1bi8BfrAgQMpKSnJ1+pFRJolM6v5DeFK6nIREYkJBbqISEwo0EVEYkLnoYu0IOXl5ZSWlrJ37966J5a8at++PX379qVt27YZz6NAF2lBSktL6dy5MwMHDiT975NIvrk7ZWVllJaWMmjQoIzna1ZdLsXFMHAgtGoV7ov1c7ki9bJ37166deumMG/izIxu3brV+5NUs2mhFxfDnDmwJ/pphA0bwnOAGTPSzyci1SnMm4eD+Ts1mxb6vHlVYZ6wZ08YLiIizSjQP/igfsNFpOkpKytjxIgRjBgxgiOPPJI+ffpUPv/8889rnbekpIRrr722znWcfvrpWal16dKlnH/++VlZVq7UGehm9ksz+8jM3kgzfoaZrTaz183sJTMbnv0yoX/NH++qY7iINFy2j1t169aNVatWsWrVKq666ipuuOGGyuft2rWjoqIi7bxFRUXccccdda7jpZdealiRzVgmLfT7gAm1jH8fONPdhwI/BBZmoa4DLFgAHTpUH9ahQxguItmXOG61YQO4Vx23yvbJCLNmzeKqq67ilFNO4aabbuKvf/0rp512GoWFhZx++um8/fbbQPUW8/z585k9ezbjx4/nqKOOqhb0nTp1qpx+/PjxTJ06leOOO44ZM2aQuLrsk08+yXHHHceoUaO49tpr62yJb9++nQsuuIBhw4Zx6qmnsnr1agD+9Kc/VX7CKCwsZPfu3WzevJlx48YxYsQITjrpJJYvX57dDVaLOg+KuvsyMxtYy/jk3eHLQN+Gl3WgxIHPefNCN0v//iHMdUBUpHHUdtwq2/93paWlvPTSS7Ru3Zpdu3axfPly2rRpwzPPPMP3vvc9fvOb3xwwz9q1a3n++efZvXs3xx57LFdfffUB52y/+uqrrFmzht69ezNmzBhefPFFioqKuPLKK1m2bBmDBg1i+vTpddZ3yy23UFhYyOLFi3nuueeYOXMmq1at4rbbbuPOO+9kzJgxfPLJJ7Rv356FCxfyhS98gXnz5rFv3z721NyIjSjbZ7n8C/BUlpdZacYMBbhIruTyuNW0adNo3bo1ADt37uTyyy/nnXfewcwoLy9POc95553HIYccwiGHHEKPHj3YunUrfftWb0+efPLJlcNGjBjB+vXr6dSpE0cddVTl+d3Tp09n4cLaOxZeeOGFyp3K2WefTVlZGbt27WLMmDHceOONzJgxg4suuoi+ffsyevRoZs+eTXl5ORdccAEjRoxo0Lapj6wdFDWzswiB/p1appljZiVmVrJt27ZsrVpEGkEuj1t17Nix8vEPfvADzjrrLN544w1+97vfpT0X+5BDDql83Lp165T975lM0xBz587lnnvu4dNPP2XMmDGsXbuWcePGsWzZMvr06cOsWbP41a9+ldV11iYrgW5mw4B7gCnRr5Gn5O4L3b3I3YsKClJezldEmoh8HbfauXMnffr0AeC+++7L+vKPPfZY3nvvPdavXw/Aww8/XOc8Y8eOpTg6eLB06VK6d+/OYYcdxrvvvsvQoUP5zne+w+jRo1m7di0bNmygZ8+eXHHFFXz1q19l5cqVWX8N6TQ40M2sP+E3Ii9z9781vCQRaQpmzICFC2HAADAL9wsXNn6350033cR3v/tdCgsLs96iBjj00EO56667mDBhAqNGjaJz584cfvjhtc4zf/58VqxYwbBhw5g7dy73338/ALfffjsnnXQSw4YNo23btkycOJGlS5cyfPhwCgsLefjhh7nuuuuy/hrSqfM3Rc3sIWA80B3YCtwCtAVw95+Z2T3AxUDiousV7l5U14qLiopcP3AhkltvvfUWxx9/fL7LyLtPPvmETp064e5cc801DB48mBtuuCHfZR0g1d/LzFaky9hMznKp9RCwu38V+Gp9ihQRyaef//zn3H///Xz++ecUFhZy5ZVX5rukrGg213IREcmWG264oUm2yBuq2Xz1X0REaqdAFxGJCQW6iEhMKNBFRGJCgS4iOXPWWWexZMmSasNuv/12rr766rTzjB8/nsQpzpMmTWLHjh0HTDN//nxuu+22Wte9ePFi3nzzzcrnN998M88880x9yk+pKV1mV4EuIjkzffp0Fi1aVG3YokWLMrpAFoSrJHbp0uWg1l0z0G+99VbOPffcg1pWU6VAF5GcmTp1Kk888UTlj1msX7+eDz/8kLFjx3L11VdTVFTEiSeeyC233JJy/oEDB/Lxxx8DsGDBAoYMGcIZZ5xReYldCOeYjx49muHDh3PxxRezZ88eXnrpJR5//HG+/e1vM2LECN59911mzZrFY489BsCzzz5LYWEhQ4cOZfbs2Xz22WeV67vlllsYOXIkQ4cOZe3atbW+vnxfZlfnoYu0UNdfD6tWZXeZI0bA7benH9+1a1dOPvlknnrqKaZMmcKiRYu45JJLMDMWLFhA165d2bdvH+eccw6rV69m2LBhKZezYsUKFi1axKpVq6ioqGDkyJGMGjUKgIsuuogrrrgCgO9///v84he/4Bvf+AaTJ0/m/PPPZ+rUqdWWtXfvXmbNmsWzzz7LkCFDmDlzJnfffTfXX389AN27d2flypXcdddd3Hbbbdxzzz1pX1++L7OrFrqI5FRyt0tyd8sjjzzCyJEjKSwsZM2aNdW6R2pavnw5F154IR06dOCwww5j8uTJlePeeOMNxo4dy9ChQykuLmbNmjW11vP2228zaNAghgwZAsDll1/OsmXLKsdfdNFFAIwaNarygl7pvPDCC1x22WVA6svs3nHHHezYsYM2bdowevRo7r33XubPn8/rr79O586da112JtRCF2mhamtJN6YpU6Zwww03sHLlSvbs2cOoUaN4//33ue2223jllVc44ogjmDVrVtrL5tZl1qxZLF68mOHDh3PfffexdOnSBtWbuARvQy6/O3fuXM477zyefPJJxowZw5IlSyovs/vEE08wa9YsbrzxRmbOnNmgWtVCF5Gc6tSpE2eddRazZ8+ubJ3v2rWLjh07cvjhh7N161aeeqr238kZN24cixcv5tNPP2X37t387ne/qxy3e/duevXqRXl5eeUlbwE6d+7M7t27D1jWsccey/r161m3bh0ADzzwAGeeeeZBvbZ8X2ZXLXQRybnp06dz4YUXVna9JC43e9xxx9GvXz/GjBlT6/wjR47ki1/8IsOHD6dHjx6MHj26ctwPf/hDTjnlFAoKCjjllFMqQ/zSSy/liiuu4I477qg8GArQvn177r33XqZNm0ZFRQWjR4/mqquuOqjXlfit02HDhtGhQ4dql9l9/vnnadWqFSeeeCITJ05k0aJF/OQnP6Ft27Z06tQpKz+EUeflcxuLLp8rknu6fG7zUt/L56rLRUQkJhToIiIxoUAXaWHy1c0q9XMwfycFukgL0r59e8rKyhTqTZy7U1ZWRvv27es1n85yEWlB+vbtS2lpKdu2bct3KVKH9u3b07dv33rNo0AXaUHatm3LoEGD8l2GNBJ1uYiIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCbqDHQz+6WZfWRmb6QZb2Z2h5mtM7PVZjYy+2WKiEhdMmmh3wdMqGX8RGBwdJsD3N3wskREpL7qDHR3XwZsr2WSKcCvPHgZ6GJmvbJVoIiIZCYbfeh9gI1Jz0ujYQcwszlmVmJmJbp8p4hIduX0oKi7L3T3IncvKigoyOWqRURiLxuBvgnol/S8bzRMRERyKBuB/jgwMzrb5VRgp7tvzsJyRUSkHur8xSIzewgYD3Q3s1LgFqAtgLv/DHgSmASsA/YAX2msYkVEJL06A93dp9cx3oFrslaRiIgcFH1TVEQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGJCgS4iEhMKdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISExkFuplNMLO3zWydmc1NMb6/mT1vZq+a2Wozm5T9UkVEpDZ1BrqZtQbuBCYCJwDTzeyEGpN9H3jE3QuBS4G7sl2oiIjULpMW+snAOnd/z90/BxYBU2pM48Bh0ePDgQ+zV6KIiGQik0DvA2xMel4aDUs2H/iymZUCTwLfSLUgM5tjZiVmVrJt27aDKFdERNLJ1kHR6cB97t4XmAQ8YGYHLNvdF7p7kbsXFRQUZGnVIiICmQX6JqBf0vO+0bBk/wI8AuDufwbaA92zUaCIiGQmk0B/BRhsZoPMrB3hoOfjNab5ADgHwMyOJwS6+lRERHKozkB39wrg68AS4C3C2SxrzOxWM5scTfZN4Aozew14CJjl7t5YRYuIyIHaZDKRuz9JONiZPOzmpMdvAmOyW5qIiNSHvikqIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYkKBLiISEwp0EZGYUKCLiMSEAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGJCgS4iEhMKdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITDS7QP/sM/jDH8A935WIiDQtzS7QH3wQJk6EFSvyXYmISNPS7AJ9yhRo0wYefTTflYiINC3NLtC7doVzzgmBrm4XEZEqzS7QAaZNg/ffh5Ur812JiEjT0SwD/YILoHVrdbuIiCTLKNDNbIKZvW1m68xsbpppLjGzN81sjZk9mN0yq+vWDc49Fx55RN0uIiIJdQa6mbUG7gQmAicA083shBrTDAa+C4xx9xOB6xuh1mrU7SIiUl0mLfSTgXXu/p67fw4sAqbUmOYK4E53/zuAu3+U3TIPpG4XEZHqMgn0PsDGpOel0bBkQ4AhZvaimb1sZhOyVWA63brpbBcRkWTZOijaBhgMjAemAz83sy41JzKzOWZWYmYl27Zta/BKp02D996DV19t8KJERJq9TAJ9E9Av6XnfaFiyUuBxdy939/eBvxECvhp3X+juRe5eVFBQcLA1V7rwQnW7iIgkZBLorwCDzWyQmbUDLgUerzHNYkLrHDPrTuiCeS+LdaaU6HbR2S4iIhkEurtXAF8HlgBvAY+4+xozu9XMJkeTLQHKzOxN4Hng2+5e1lhFJ1O3i4hIYJ6npm1RUZGXlJQ0eDkffwxHHgnf/jb86EdZKExEpAkzsxXuXpRqXLP8pmiy7t3h7LN1touISLMPdAjdLu++C6tW5bsSEZH8iUWg62wXEZGYBHqi20Vnu4hISxaLQAd1u4iIxCbQ1e0iIi1dbAK9e3c46yyd7SIiLVdsAh3gkktg3Tp47bV8VyIiknuxCnR1u4hISxarQE90u+hsFxFpiWIV6BDOdlG3i4i0RLELdHW7iEhLFbtALyiA8eNTn+1SXAwDB0KrVuG+uDgPBYqINJLYBTqEs13eeQdWr64aVlwMc+bAhg0h6DdsCM8V6iISF7EM9FTdLvPmwZ491afbsycMFxGJg1gGeqLbJflslw8+SD1tuuEiIs1NLAMdwtkuyd0u/funni7dcBGR5ia2gX7hheHgZ6LbZcEC6NCh+jQdOoThIiJxENtA79Gj+rVdZsyAhQthwAAwC/cLF4bhIiJxENtAh9Dt8re/weuvh+czZsD69bB/f7hXmItInMQ60BPdLo88ku9KREQaX6wDvUeP9F8yEhGJm1gHOhzY7SIiElexD/SLLqp+touISFzFPtDV7SIiLUXsAx1Ct8vbb8Mbb+S7EhGRxtMiAj3R7aKzXUQkzlpEoPfoAWeeqW4XEYm3FhHooG4XEYm/FhPoOttFROIuo0A3swlm9raZrTOzubVMd7GZuZkVZa/E7OjZU90uIhJvdQa6mbUG7gQmAicA083shBTTdQauA/6S7SKzZdo0WLsW1qzJdyUiItmXSQv9ZGCdu7/n7p8Di4ApKab7IfBjYG8W68sqne0iInGWSaD3ATYmPS+NhlUys5FAP3d/orYFmdkcMysxs5Jt27bVu9iG6tkTxo1Tt4uIxFODD4qaWSvgP4Bv1jWtuy909yJ3LyooKGjoqg/KJZeEbpcf/ADKy/NSgohIo8gk0DcB/ZKe942GJXQGTgKWmtl64FTg8aZ4YBTgK1+BmTPDLxWddloIdxGROMgk0F8BBpvZIDNrB1wKPJ4Y6e473b27uw9094HAy8Bkdy9plIobqH17uP9+eOyx8CMXhYXwX/8VfvRCRKQ5qzPQ3b0C+DqwBHgLeMTd15jZrWY2ubELbCwXXxwuqXv22XDttTBhAmzaVPd8IiJNlXmejg4WFRV5SUn+G/Hu4bdFb7wRDjkE7r4bvvjFfFclIpKama1w95Rd2i3mm6LpmMGVV8KqVTBkCFx6KXzpS/D3v+e7MhGJg4oK2LIFXnsN/vhHKC6GxmrLtmmcxTY/gwfDCy/Aj34Et94Ky5bBfffBuefmuzIRaUrc4ZNP4KOPwm3r1qrHNZ9v3QplZQcu41vfgqJGOG2kxXe5pFJSApddFs6Aue66EPKHHprvqqQp2LIFnn4aRoyAoUPDJzyJvy1bYMkSeOqp0Mrevj31dF26hKu79uwZ7tM97tULOnc+uFpq63JRCz2FoiJYsQLmzoX//M/wD/zAAzBqVL4rk3zYswcWLw7vgaefrjojasiQcDmJadNg2DCFezoVFfDOO6Fb87XX4M03oVs3OOGEcDvxROjfP3yLu6moqIA//xn+8IcQ4q++Gob37Annnw8nnXRgSBcUhONw+aQWeh3++EeYNSt8fJo/H77zHWij3WDs7d8PS5eGEH/ssfARu39/+PKX4YILwg7/0UfDNPv3hy67RLgPH95yw33XLli9uiq8V60Kl6zeG10QpG3bsCMsKwut3oQOHeD446sCPhH2AwdC69a5qX3Tpuqt8J07w7pPPz2cBTdxYvjb5nvHU1sLXYGege3b4ZprYNGi8GWkX/0Kjjkm31VJY1izJoR4cTGUloaPxdOmhS64ceMO/Gf+6CP47W9DuD//fAj3Y46pCvcRI+IZ7u6wYUNVaCfu33+/appu3UIAjhhRdX/ccdCuXRi/fTu89VbY5m++WXVLPn340EPDPImAT9wKCsJOoF27g9++5eXw4otVrfDVq8Pw3r1DeE+YEI6hdelycMtvLAr0LHnoIfja1+DTT+GMM8Jt7Fg49VTo2DHf1cnB2ro1/G0feABWrgytsi98IYT45MkhODKxbVv1cN+3L4T71KnhkhPNKdz37w87q9JS2Lix+v0HH4RW986dYVqz8AmlZnj37n1wr3fHjhD0iYBPBP7GjQdO27p1+Pt07Bhuice1DWvXDv7yF3jmGdi9O3ziPuOMqhBv6sdGFOhZUlwculw2bQofHSsqQkuldWsYOTKE+9ixMGZMaEFI0/Xpp/C//xtCfMmSEL4jR4bLQlx6aegXbYht20K/+6OPwnPPheUffXQI96lTQ6uzY8f8BEdyWKcK7I0bw3u85rWO2rWDvn2hX7/QSh4+PNyGDs1Ng2bXrnCiwltvhdD/xz/Cbc+e6vfpHu/ZA59/HpbVr18I8IkTw5cLDzus8evPFgV6FhQXw5w54U2RcOih8PWvh3Bfvhz++lf47LMw7rjjqlrwY8eGvsCmvNePo4qKEFwffgibN1fd1q0LYb57dwioL385tMZPOOAq/9nx8cdV4f7ssyHcIRxA69YNundPfUs1LvnTwr59Idi2bw/fm9i+PfNbooaERFgnAjvVfUFB838Pl5eH/vxOnZrva1GgZ8HAgaHPsKYBA8I1YSCEeUlJCPcXXgj9czt2hHG9e1eF+2mnhfm6dm0eb6ry8vAx+913w23jxrAz69Il3A4/vOpx4tapU+McPHIP/5Cpgrrm7aOPUl8muUcPmDQptMbPPDO3B7nKysIngk2bQtAnbmVlVY+3b09/eedDD4UjjggNi8R7K53DDgvvscTtiCOqHvfuXT2wu3fP/8E+yYwCPQtatUr9T2aW/sJe+/eHvsYXXgghv3x59QM+bdvCkUdW3Xr1Sv34yCPDRcUa0z/+URXYNW8bNlRv0bVufWALr6ZWrUKg1Az6Ll3CgcaKirAD3Lu37vuaw9Ktr2fPsN1q3nr3rnrcs2fVQbmmat++0OJOFfaJwO/YMX1Yd+0atnPbtvl+JdIYFOhZkEkLvS6JMwNeeaWqdbllS9Vt8+bQ95rqT9KlS1W49+gRAr5Nm/BPm3yrOSzVcwh1JId28ilkEALi6KOrbsccU/W4V68QOjt3hlZi4lbzebphiQNR7duHbod09+nGtW9f9eWMRGAXFOTu9DaRfNIXi7JgwYID+9A7dAjDM2UWdgwDB6afpqIihHpy2NcM/lWrQku1vDzcKiqqHpeXZ34p4D59QkBPmlQ9vI8+OgR6bVq1qurXFZGmQYGeoRkzwv28eaE/uX//EOaJ4dnSpk1Vy/Ng7d9/YMgnB797CHNdzkAkXhTo9TBjRvYDvDG0ahX6iZt6X7GIZJeOa4uIxIQCPYeKi0P/eatW4b64ON8ViUicqMslR2p+MWnDhvAcmkc3jog0fWqh58i8edXPkIHwfN68/NQjIvGjQM+RDz6o3/BU1GUjIrVRoOdI//71G15Tostmw4aqLyjNmaNQF5EqCvQcWbDgwMuw1ueLSdnqslErXyS+FOg5MmMGLFwYLhVgFu4XLsz8gGi2umwa2srXDumAPHsAAAbySURBVEGk6VKg59CMGeG6L/v3h/v6nN3S0C4baHgrXzsEkaZNgd5MNLTLBhreym8KOwQRSU+B3kw0tMsGGt7Kz/cOAdTCF6mNAr0ZaUiXDTS8lZ/vHUJT6PLRDkWaNHfPy23UqFEuuffrX7sPGOBuFu5//ev6zduhg3uI03Dr0CHzZQwYUH3exG3AgNzM39D6Gzp/YhkHu/2zMb80f0CJp8lVBbrUSz53CGapA90ss/m1Q9EOJQ4U6NJkNCQQGhqoDd0haIfS/HcocdghNTjQgQnA28A6YG6K8TcCbwKrgWeBAXUtU4Eu9dXcu3y0Q8nvDiUOOyT3BgY60Bp4FzgKaAe8BpxQY5qzgA7R46uBh+targJdDkY+u3y0Q2neO5TmvkNKaGignwYsSXr+XeC7tUxfCLxY13IV6JIP+Wxh5TsQ8h2I+d6hNPcdUkJDA30qcE/S88uAn9Yy/U+B76cZNwcoAUr69+9fv1chEgPaoTTfFnq+dygJOQt04MvAy8AhdS1XLXSR3GvJO5TmvkNKyEmXC3Au8BbQo65lugJdpEXK90HF5rxDSmhooLcB3gMGJR0UPbHGNIXRgdPBdS0vcVOgi0hzk+8dknvtgW5hfO3MbBJwe3TGyy/dfYGZ3Rot+HEzewYYCmyOZvnA3SfXtsyioiIvKSmpc90iIlLFzFa4e1GqcRn9SLS7Pwk8WWPYzUmPz21QhSIi0mC6OJeISEwo0EVEYkKBLiISEwp0EZGYyOgsl0ZZsdk2YENeVl637sDH+S6iFk29Pmj6Naq+hlF9DdOQ+ga4e0GqEXkL9KbMzErSnRbUFDT1+qDp16j6Gkb1NUxj1acuFxGRmFCgi4jEhAI9tYX5LqAOTb0+aPo1qr6GUX0N0yj1qQ9dRCQm1EIXEYkJBbqISEy02EA3s35m9ryZvWlma8zsuhTTjDeznWa2KrrdnGpZjVjjejN7PVr3AZemtOAOM1tnZqvNbGQOazs2abusMrNdZnZ9jWlyvv3M7Jdm9pGZvZE0rKuZ/dHM3onuj0gz7+XRNO+Y2eU5rO8nZrY2+hv+1sy6pJm31vdDI9Y338w2Jf0dJ6WZd4KZvR29H+fmsL6Hk2pbb2ar0szbqNsvXabk9P2X7rq6cb8BvYCR0ePOwN848MevxwO/z2ON64HutYyfBDwFGHAq8Jc81dka2EL4wkNetx8wDhgJvJE07N+AudHjucCPU8zXlXDd/67AEdHjI3JU3z8BbaLHP05VXybvh0asbz7wrQzeA7X+mHxj1Vdj/L8DN+dj+6XLlFy+/1psC93dN7v7yujxbsKvLfXJb1X1NgX4lQcvA13MrFce6jgHeNfd8/7NX3dfBmyvMXgKcH/0+H7gghSzfgH4o7tvd/e/A38EJuSiPnd/2t0roqcvA32zvd5Mpdl+mTgZWOfu77n758AiwnbPqtrqMzMDLgEeyvZ6M1FLpuTs/ddiAz2ZmQ0k/OrSX1KMPs3MXjOzp8zsxJwWBg48bWYrzGxOivF9gI1Jz0vJz07pUtL/E+Vz+yX0dPfEj69sAXqmmKapbMvZhE9dqdT1fmhMX4+6hH6ZpsugKWy/scBWd38nzficbb8amZKz91+LD3Qz6wT8Brje3XfVGL2S0I0wHPgvYHGOyzvD3UcCE4FrzGxcjtdfJzNrB0wGHk0xOt/b7wAePt82yXN1zWweUAEUp5kkX++Hu4GjgRGEXyX79xytt76mU3vrPCfbr7ZMaez3X4sOdDNrS9jwxe7+PzXHu/sud/8kevwk0NbMuueqPnffFN1/BPyW8LE22SagX9LzvtGwXJoIrHT3rTVH5Hv7Jdma6IqK7j9KMU1et6WZzQLOB2ZE//QHyOD90Cjcfau773P3/cDP06w339uvDXAR8HC6aXKx/dJkSs7efy020KP+tl8Ab7n7f6SZ5shoOszsZML2KstRfR3NrHPiMeHA2Rs1JnscmBmd7XIqsDPpo12upG0V5XP71fA4kDhr4HLgf1NMswT4JzM7IupS+KdoWKMzswnATcBkd9+TZppM3g+NVV/ycZkL06z3FWCwmQ2KPrVdStjuuXIusNbdS1ONzMX2qyVTcvf+a6wjvk39BpxB+OizGlgV3SYBVwFXRdN8HVhDOGL/MnB6Dus7Klrva1EN86LhyfUZcCfh7ILXgaIcb8OOhIA+PGlYXrcfYeeyGSgn9EP+C9ANeBZ4B3gG6BpNWwTckzTvbGBddPtKDutbR+g/TbwPfxZN2xt4srb3Q47qeyB6f60mhFOvmvVFzycRzux4N5f1RcPvS7zvkqbN6farJVNy9v7TV/9FRGKixXa5iIjEjQJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhIT/x+l8RCeRcoQtgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xlw_lUzzjj9a"
      },
      "source": [
        "우리가 적용한 \"상식\" 에는 머신러닝 모델이 찾지 못한 핵심 정보가 많이 들어있으므로, 기준 모델의 성능이 더 좋은 것을 알 수 있다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AT8LR-M2khtR"
      },
      "source": [
        "## 3-4) 첫 번째 순환 신경망"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9QNsCgT9k4E9"
      },
      "source": [
        "첫번째 완전 연결 네트워크는 시계열 데이터를 펼쳐서 사용했기 때문에, 입력 데이터에서 시간 정보가 손실됐다. 이 모델 대신, 순서가 의미가 있는 시퀀스 데이터 처리 모델을 사용해 보자."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ryXwJjtwk3R4",
        "outputId": "caa639ba-4379-464b-d7a8-f85677c38432"
      },
      "source": [
        "# LSTM층 대신 GRU층 사용\r\n",
        "from keras.models import Sequential\r\n",
        "from keras import layers\r\n",
        "from keras.optimizers import RMSprop\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "model.add(layers.GRU(32, input_shape=(None, float_data.shape[-1])))\r\n",
        "model.add(layers.Dense(1))\r\n",
        "\r\n",
        "model.compile(optimizer=RMSprop(), loss='mae')\r\n",
        "history = model.fit_generator(train_gen,\r\n",
        "                              steps_per_epoch=500,\r\n",
        "                              epochs=20,\r\n",
        "                              validation_data=val_gen,\r\n",
        "                              validation_steps=val_steps)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.\n",
            "  warnings.warn('`Model.fit_generator` is deprecated and '\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "500/500 [==============================] - 83s 163ms/step - loss: 0.3565 - val_loss: 0.2692\n",
            "Epoch 2/20\n",
            "500/500 [==============================] - 81s 162ms/step - loss: 0.2849 - val_loss: 0.2636\n",
            "Epoch 3/20\n",
            "500/500 [==============================] - 81s 163ms/step - loss: 0.2808 - val_loss: 0.2639\n",
            "Epoch 4/20\n",
            "500/500 [==============================] - 83s 166ms/step - loss: 0.2748 - val_loss: 0.2605\n",
            "Epoch 5/20\n",
            "500/500 [==============================] - 82s 163ms/step - loss: 0.2690 - val_loss: 0.2642\n",
            "Epoch 6/20\n",
            "500/500 [==============================] - 82s 165ms/step - loss: 0.2635 - val_loss: 0.2649\n",
            "Epoch 7/20\n",
            "500/500 [==============================] - 84s 167ms/step - loss: 0.2569 - val_loss: 0.2611\n",
            "Epoch 8/20\n",
            "500/500 [==============================] - 85s 170ms/step - loss: 0.2539 - val_loss: 0.2660\n",
            "Epoch 9/20\n",
            "500/500 [==============================] - 83s 167ms/step - loss: 0.2492 - val_loss: 0.2651\n",
            "Epoch 10/20\n",
            "500/500 [==============================] - 83s 166ms/step - loss: 0.2433 - val_loss: 0.2709\n",
            "Epoch 11/20\n",
            "500/500 [==============================] - 85s 169ms/step - loss: 0.2398 - val_loss: 0.2652\n",
            "Epoch 12/20\n",
            "500/500 [==============================] - 84s 169ms/step - loss: 0.2383 - val_loss: 0.2741\n",
            "Epoch 13/20\n",
            "500/500 [==============================] - 86s 171ms/step - loss: 0.2315 - val_loss: 0.2721\n",
            "Epoch 14/20\n",
            "500/500 [==============================] - 84s 167ms/step - loss: 0.2256 - val_loss: 0.2774\n",
            "Epoch 15/20\n",
            "500/500 [==============================] - 85s 171ms/step - loss: 0.2250 - val_loss: 0.2803\n",
            "Epoch 16/20\n",
            "500/500 [==============================] - 84s 168ms/step - loss: 0.2196 - val_loss: 0.2835\n",
            "Epoch 17/20\n",
            "500/500 [==============================] - 84s 168ms/step - loss: 0.2171 - val_loss: 0.2846\n",
            "Epoch 18/20\n",
            "500/500 [==============================] - 83s 167ms/step - loss: 0.2113 - val_loss: 0.2941\n",
            "Epoch 19/20\n",
            "500/500 [==============================] - 85s 171ms/step - loss: 0.2091 - val_loss: 0.2898\n",
            "Epoch 20/20\n",
            "500/500 [==============================] - 84s 169ms/step - loss: 0.2058 - val_loss: 0.2919\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        },
        "id": "Im8d9QVwldE4",
        "outputId": "9dc8b8dd-e36a-46a9-9ec5-0c59a0664b1e"
      },
      "source": [
        "loss = history.history['loss']\r\n",
        "val_loss = history.history['val_loss']\r\n",
        "\r\n",
        "epochs = range(1, len(loss) + 1)\r\n",
        "\r\n",
        "plt.figure()\r\n",
        "\r\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\r\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\r\n",
        "plt.title('Training and validation loss')\r\n",
        "plt.legend()"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f24eb365e48>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXyU1dn/8c/FJiKIsmiVyOaCouwBFBS1pQpqQXEpNBYoblitC1aL8oioxfZRn/4sdUUt2haLaxEVxLogrpVFKqJQgYIEFRVlE8EA1++PcweGmEwmyWyZfN+v17xm7v2aO5Nrzpz7nHObuyMiIrmrVqYDEBGR1FKiFxHJcUr0IiI5ToleRCTHKdGLiOQ4JXoRkRynRC8VYmYzzGxYstfNJDNbYWZ9U7BfN7NDotf3mtn1iaxbieMUmNkLlY0zzn5PMLPCZO9X0q9OpgOQ1DOzTTGTDYCtwPZo+iJ3n5zovty9fyrWzXXuPjIZ+zGz1sB/gbruvi3a92Qg4b+h1DxK9DWAuzcsfm1mK4Dz3f3FkuuZWZ3i5CEiuUNVNzVY8U9zM/uNmX0GTDKzfc3sWTP7wsy+jl7nxWwzy8zOj14PN7PXzez2aN3/mln/Sq7bxsxmm9lGM3vRzO4ys7+VEXciMd5sZm9E+3vBzJrFLP+5ma00s7VmNibO+elpZp+ZWe2YeWeY2XvR6x5m9paZrTOzT83sTjOrV8a+HjKz38ZMXx1t84mZjSix7qlm9q6ZbTCzVWY2Lmbx7Oh5nZltMrNjis9tzPa9zGyOma2Pnnslem7iMbMjou3XmdkiMxsQs+wUM/sg2udqM/t1NL9Z9PdZZ2ZfmdlrZqa8k2Y64fIDoAnQCriQ8JmYFE23BL4F7oyzfU9gCdAMuBV40MysEus+ArwDNAXGAT+Pc8xEYvwZ8AtgP6AeUJx42gP3RPs/MDpeHqVw938B3wA/LLHfR6LX24Ero/dzDPAj4Jdx4iaKoV8Uz4+BQ4GS1we+AYYC+wCnAheb2enRsj7R8z7u3tDd3yqx7ybAc8CE6L39AXjOzJqWeA/fOzflxFwXeAZ4IdruV8BkM2sXrfIgoRqwEXAU8HI0/yqgEGgO7A9cB2jclTRTopcdwA3uvtXdv3X3te7+pLtvdveNwHjg+Djbr3T3+919O/AwcADhHzrhdc2sJdAdGOvu37n768C0sg6YYIyT3P0/7v4t8BjQOZp/FvCsu892963A9dE5KMvfgSEAZtYIOCWah7vPc/e33X2bu68A7isljtKcE8X3vrt/Q/hii31/s9x9obvvcPf3ouMlsl8IXwwfuftfo7j+DiwGfhKzTlnnJp6jgYbA76O/0cvAs0TnBigC2pvZ3u7+tbvPj5l/ANDK3Yvc/TXXAFtpp0QvX7j7luIJM2tgZvdFVRsbCFUF+8RWX5TwWfELd98cvWxYwXUPBL6KmQewqqyAE4zxs5jXm2NiOjB231GiXVvWsQil90FmtgcwCJjv7iujOA6LqiU+i+K4hVC6L89uMQArS7y/nmb2SlQ1tR4YmeB+i/e9ssS8lUCLmOmyzk25Mbt77Jdi7H7PJHwJrjSzV83smGj+bcBS4AUzW25moxN7G5JMSvRSsnR1FdAO6Onue7OrqqCs6phk+BRoYmYNYuYdFGf9qsT4aey+o2M2LWtld/+AkND6s3u1DYQqoMXAoVEc11UmBkL1U6xHCL9oDnL3xsC9MfstrzT8CaFKK1ZLYHUCcZW334NK1K/v3K+7z3H3gYRqnamEXwq4+0Z3v8rd2wIDgFFm9qMqxiIVpEQvJTUi1Hmvi+p7b0j1AaMS8lxgnJnVi0qDP4mzSVVifAI4zcyOjS6c3kT5/wePAJcTvlAeLxHHBmCTmR0OXJxgDI8Bw82sffRFUzL+RoRfOFvMrAfhC6bYF4SqprZl7Hs6cJiZ/czM6pjZT4H2hGqWqvgXofR/jZnVNbMTCH+jKdHfrMDMGrt7EeGc7AAws9PM7JDoWsx6wnWNeFVlkgJK9FLSHcCewJfA28DzaTpuAeGC5lrgt8CjhPb+pal0jO6+CLiEkLw/Bb4mXCyMp7iO/GV3/zJm/q8JSXgjcH8UcyIxzIjew8uEao2XS6zyS+AmM9sIjCUqHUfbbiZck3gjaslydIl9rwVOI/zqWQtcA5xWIu4Kc/fvCIm9P+G83w0MdffF0So/B1ZEVVgjCX9PCBebXwQ2AW8Bd7v7K1WJRSrOdF1EspGZPQosdveU/6IQyXUq0UtWMLPuZnawmdWKmh8OJNT1ikgVqWesZIsfAE8RLowWAhe7+7uZDUkkN6jqRkQkx6nqRkQkx2Vd1U2zZs28devWmQ5DRKRamTdv3pfu3ry0ZVmX6Fu3bs3cuXMzHYaISLViZiV7RO+kqhsRkRynRC8ikuOU6EVEclzW1dGLSPoVFRVRWFjIli1byl9ZMqp+/frk5eVRt27dhLdRohcRCgsLadSoEa1bt6bs+8ZIprk7a9eupbCwkDZt2iS8Xc5U3UyeDK1bQ61a4XmybpUskrAtW7bQtGlTJfksZ2Y0bdq0wr+8cqJEP3kyXHghbI5uW7FyZZgGKCgoezsR2UVJvnqozN8pJ0r0Y8bsSvLFNm8O80VEarqcSPQff1yx+SKSXdauXUvnzp3p3LkzP/jBD2jRosXO6e+++y7utnPnzuWyyy4r9xi9evVKSqyzZs3itNNOS8q+0iUnEn3LkjdiK2e+iFRNsq+JNW3alAULFrBgwQJGjhzJlVdeuXO6Xr16bNu2rcxt8/PzmTBhQrnHePPNN6sWZDWWE4l+/Hho0GD3eQ0ahPkiklzF18RWrgT3XdfEkt0AYvjw4YwcOZKePXtyzTXX8M4773DMMcfQpUsXevXqxZIlS4DdS9jjxo1jxIgRnHDCCbRt23a3L4CGDRvuXP+EE07grLPO4vDDD6egoIDiUXynT5/O4YcfTrdu3bjsssvKLbl/9dVXnH766XTs2JGjjz6a9957D4BXX3115y+SLl26sHHjRj799FP69OlD586dOeqoo3jttdeSe8LiyImLscUXXMeMCdU1LVuGJK8LsSLJF++aWLL/5woLC3nzzTepXbs2GzZs4LXXXqNOnTq8+OKLXHfddTz55JPf22bx4sW88sorbNy4kXbt2nHxxRd/r835u+++y6JFizjwwAPp3bs3b7zxBvn5+Vx00UXMnj2bNm3aMGTIkHLju+GGG+jSpQtTp07l5ZdfZujQoSxYsIDbb7+du+66i969e7Np0ybq16/PxIkTOfnkkxkzZgzbt29nc8mTmEI5keghfMCU2EVSL53XxM4++2xq164NwPr16xk2bBgfffQRZkZRUVGp25x66qnsscce7LHHHuy3336sWbOGvLy83dbp0aPHznmdO3dmxYoVNGzYkLZt2+5snz5kyBAmTpwYN77XX39955fND3/4Q9auXcuGDRvo3bs3o0aNoqCggEGDBpGXl0f37t0ZMWIERUVFnH766XTu3LlK56YicqLqRkTSJ53XxPbaa6+dr6+//npOPPFE3n//fZ555pky25LvscceO1/Xrl271Pr9RNapitGjR/PAAw/w7bff0rt3bxYvXkyfPn2YPXs2LVq0YPjw4fzlL39J6jHjUaIXkQrJ1DWx9evX06JFCwAeeuihpO+/Xbt2LF++nBUrVgDw6KOPlrvNcccdx+To4sSsWbNo1qwZe++9N8uWLaNDhw785je/oXv37ixevJiVK1ey//77c8EFF3D++eczf/78pL+HsijRi0iFFBTAxInQqhWYheeJE1NfdXrNNddw7bXX0qVLl6SXwAH23HNP7r77bvr160e3bt1o1KgRjRs3jrvNuHHjmDdvHh07dmT06NE8/PDDANxxxx0cddRRdOzYkbp169K/f39mzZpFp06d6NKlC48++iiXX3550t9DWbLunrH5+fmuG4+IpNeHH37IEUcckekwMm7Tpk00bNgQd+eSSy7h0EMP5corr8x0WN9T2t/LzOa5e35p66tELyISuf/+++ncuTNHHnkk69ev56KLLsp0SEmRM61uRESq6sorr8zKEnxVqUQvIpLjlOhFRHKcEr2ISI5TohcRyXFK9CKScSeeeCIzZ87cbd4dd9zBxRdfXOY2J5xwAsVNsU855RTWrVv3vXXGjRvH7bffHvfYU6dO5YMPPtg5PXbsWF588cWKhF+qbBrOOKFEb2b9zGyJmS01s9GlLB9pZgvNbIGZvW5m7WOWXRttt8TMTk5m8CKSG4YMGcKUKVN2mzdlypSEBhaDMOrkPvvsU6ljl0z0N910E3379q3UvrJVuYnezGoDdwH9gfbAkNhEHnnE3Tu4e2fgVuAP0bbtgcHAkUA/4O5ofyIiO5111lk899xzO28ysmLFCj755BOOO+44Lr74YvLz8znyyCO54YYbSt2+devWfPnllwCMHz+eww47jGOPPXbnUMYQ2sh3796dTp06ceaZZ7J582befPNNpk2bxtVXX03nzp1ZtmwZw4cP54knngDgpZdeokuXLnTo0IERI0awdevWnce74YYb6Nq1Kx06dGDx4sVx31+mhzNOpB19D2Cpuy8HMLMpwEBg51egu2+IWX8voLi77UBgirtvBf5rZkuj/b1V5chFJCWuuAIWLEjuPjt3hjvuKHt5kyZN6NGjBzNmzGDgwIFMmTKFc845BzNj/PjxNGnShO3bt/OjH/2I9957j44dO5a6n3nz5jFlyhQWLFjAtm3b6Nq1K926dQNg0KBBXHDBBQD8z//8Dw8++CC/+tWvGDBgAKeddhpnnXXWbvvasmULw4cP56WXXuKwww5j6NCh3HPPPVxxxRUANGvWjPnz53P33Xdz++2388ADD5T5/jI9nHEiVTctgFUx04XRvN2Y2SVmtoxQor+sgtteaGZzzWzuF198kWjsIpJDYqtvYqttHnvsMbp27UqXLl1YtGjRbtUsJb322mucccYZNGjQgL333psBAwbsXPb+++9z3HHH0aFDByZPnsyiRYvixrNkyRLatGnDYYcdBsCwYcOYPXv2zuWDBg0CoFu3bjsHQivL66+/zs9//nOg9OGMJ0yYwLp166hTpw7du3dn0qRJjBs3joULF9KoUaO4+05E0nrGuvtdwF1m9jPgf4BhFdh2IjARwlg3yYpJRCouXsk7lQYOHMiVV17J/Pnz2bx5M926deO///0vt99+O3PmzGHfffdl+PDhZQ5PXJ7hw4czdepUOnXqxEMPPcSsWbOqFG/xUMdVGeZ49OjRnHrqqUyfPp3evXszc+bMncMZP/fccwwfPpxRo0YxdOjQKsWaSIl+NXBQzHReNK8sU4DTK7mtiNRQDRs25MQTT2TEiBE7S/MbNmxgr732onHjxqxZs4YZM2bE3UefPn2YOnUq3377LRs3buSZZ57ZuWzjxo0ccMABFBUV7RxaGKBRo0Zs3Ljxe/tq164dK1asYOnSpQD89a9/5fjjj6/Ue8v0cMaJlOjnAIeaWRtCkh4M/Cx2BTM71N0/iiZPBYpfTwMeMbM/AAcChwLvVDlqEclJQ4YM4YwzzthZhVM8rO/hhx/OQQcdRO/eveNu37VrV37605/SqVMn9ttvP7p3775z2c0330zPnj1p3rw5PXv23JncBw8ezAUXXMCECRN2XoQFqF+/PpMmTeLss89m27ZtdO/enZEjR1bqfRXfy7Zjx440aNBgt+GMX3nlFWrVqsWRRx5J//79mTJlCrfddht169alYcOGSblBSULDFJvZKcAdQG3gz+4+3sxuAua6+zQz+yPQFygCvgYudfdF0bZjgBHANuAKd4/7laxhikXST8MUVy8VHaY4oTp6d58OTC8xb2zM6zJH0Hf38UCK7z0jIiJlUc9YEZEcp0QvIgBk293mpHSV+Tsp0YsI9evXZ+3atUr2Wc7dWbt2LfXr16/QdrrDlIiQl5dHYWEh6rCY/erXr09eXl6FtlGiFxHq1q1LmzZtMh1GxrjDTTfB55/DNddAq1aZjii5VHUjIjXe734H48bBPffAoYfCJZfA6hzq2qlELyI12kMPwZgxUFAAK1bAiBEwcSIcfHAY4O2zzzIdYdUp0YtIjfX883D++dC3L/z5z9CyJdx7L3z0EZx7Ltx5J7RtC1dfDdX58oUSvYjUSHPmwFlnQYcO8OSTUK/ermWtW8MDD8DixXD22fCHP0CbNnDddbB2bXLjKCqCt96C3/8eyrkZVqUp0YtIjbNsGZx6KjRvDjNmwN57l77eIYfAww/DokUwYEBIxm3awNixUMqdCxPy3Xfwxhtwyy1w0kmw777Qqxdcey3885+Vf0/xJDTWTTpprBsRSaXPPw+Jdd26kHDbtUt82/ffhxtvhCeegMaN4aqr4PLLy/6iANi6Fd55B2bNgldfhTffhG+/Dcs6dIATToDjj4c+fcIXT2XFG+tGiV5EaoxNm+DEE0MJ/eWX4eijK7eff/8bbrgBnn4amjQJdfiXXgoNG8KWLfCvf+1K7G+9FeaZQceOuxL7ccdBs2bJe29K9CJS4xUVwcCBMHMm/OMfoSqmqubNC9U406eH0vgRR4Qkv3VrSOydO++e2Js0qfoxy1Ll0StFRKozd7joolAff999yUnyAN26wXPPwdtvw803h2qhSy4Jyf3YY0P9ezZQoheRnDd2LEyaFJ4vvDD5+z/66JDws5Va3YhITrv3Xvjtb+G880Lv15pIiV5EctbUqaEq5dRTQ8I3y3REmaFELyI56c03YcgQyM+HRx+FOjW4olqJXkRyzuLF8JOfwEEHwbPPwl57ZTqizFKiF5Gc8skncPLJULduGMumKp2QcoUSfWTy5DC+Ra1a4Xny5ExHJCIVtX499O8PX30V2ra3bZvpiLJDDa612mXy5NDkavPmML1y5a4mWAUFmYtLRBK3dSsMGgQffBCaOnbtmumIsocSPWEs6uIkX2zz5l1jVItI9nAPHZOWLYPly8PzsmXw7rthLJq//CUMFia7KNEDH39csfkiklpFReGXdXESj03oy5fDN9/sWtcM8vJCNc0DD8DPf565uLOVEj3hZgMrV5Y+X0RS78MP4a9/DaM8LlsWClk7duxaXr9+SORt28IPfxju/tS2bXhu3Tosl7Ip0QPjx+9eRw/QoEGYLyKpsXYtTJkSxnufMwdq1w716sccE+7uFJvMDzggNJSQylGiZ1c9/JgxoSTRsmVI8qqfF0muoqLQ5PHhh2HatDDdqVO4g9PPfgb775/pCHOTEn2koECJXSRV/v3vcBPuRx4JF1KbNw9DEwwbFobyldRSoheRlFizJiT2hx8Oib5u3dBbdfhw6NcvTEt6JFTrZWb9zGyJmS01s9GlLB9lZh+Y2Xtm9pKZtYpZdquZLTKzD81sgllNHVZIJPdt3RputP2Tn0CLFjBqVLjp9p13wqef7lqmJJ9e5SZ6M6sN3AX0B9oDQ8ysfYnV3gXy3b0j8ARwa7RtL6A30BE4CugOHJ+06LOIetZKTbVjR7j36i9/GS6annUWzJ8Pv/51uGXfO++EapqmTTMdac2VSNVND2Cpuy8HMLMpwEDgg+IV3P2VmPXfBs4tXgTUB+oBBtQF1lQ97OyinrVSEy1cGKpm/v738Jnfc89wq77hw6Fv39CKRrJDIom+BbAqZroQ6Bln/fOAGQDu/paZvQJ8Skj0d7r7hyU3MLMLgQsBWlbDxuvqWSs1xcqVIbE/8khI9LVrh16ov/1tSPKNGmU6QilNUi/Gmtm5QD5R9YyZHQIcAeRFq/zTzI5z99dit3P3icBECDcHT2ZM6aCetZLLvvwSHn88/HJ9440wr1evUO9+9tmw336ZjU/Kl0iiXw0cFDOdF83bjZn1BcYAx7v71mj2GcDb7r4pWmcGcAzwWsntqzP1rJVcs2kTPP10KLm/8AJs2wZHHgm33AKDB0ObNpmOUCoikVY3c4BDzayNmdUDBgPTYlcwsy7AfcAAd/88ZtHHwPFmVsfM6hJK+t+ruqnuxo8PPWljqWetVDdFReEmHcUdl849N1TPXHVVaB65cCFce62SfHVUbone3beZ2aXATKA28Gd3X2RmNwFz3X0acBvQEHg8aj35sbsPILTA+SGwkHBh9nl3fyY1byVz1LNWqrtnnw0tYz7+GJo0gaFDw+e3Vy8NPZALzD27qsTz8/N97ty5mQ5DpEZYvRouvzy0by+umunXL7R9l+rFzOa5e35py9QzVqQG2r4d7r47/AotKgoJ/qqrlOBzlRK9SA3z7ruhn8fcuaFp5N13hxEiJXep9k2khti0KZTa8/Nh1arQHv7555XkawKV6EVqgGeeCRdbV62Ciy6C3/0O9t0301FJuqhEL5IFvvoq3DnpiCNg5MjQfn3193qrVFxhYbhh9oAB0Lhx6PB0771K8jWNEn2W0KBoNdfGjaGlyxtvQKtWoUqloCDcB/WQQ2DEiDDU74oVie9z+3aYMCF8cTz/PPz+92GgsV69UvY2JIup6iYLaFC0mmvz5jBs7/z58NRToeS9fXvooPTqq+Hx9NMwaVJYv2VL6NMHjj8+PA45JNwcO9b8+aF6Zu5cOPnkcLG1bdv0vzfJHmpHnwVaty59CIVWrSpWipPq5bvv4PTTQ4l78mQYMqT09XbsCMP9Fif+2bPDXZogDAtcnPiPPjrcYPuPfwx3cPrjH+Gcc77/RSC5KV47eiX6LFCrFpT2ZzAL/+SSe7ZtC2PGPPkk3H8/nH9+4tu6w5IluxL/q6/CJ5/sWj5yZLjYus8+yY9bspc6TGU5DYpWs+zYERL7k0/C//t/FUvyEAoAhx8eHhddFBL/8uWhjr99+9B8UiSWLsZmAQ2KlnmffRZuVH3LLaGnaKq4w2WXhYurN94IV1xR9X2ahbbwQ4cqyUvplOizQEEBTJwY6uTNwvPEiboQmy6PPw5HHRWaNI4ZE+q6Fy5MzbGuuw7uuivcZu/661NzDJGSlOizREFBuPC6Y0d4VpJPva+/Duf5nHNCq5SFC0N1yqpV0K1buGtSMkv3t9wSmjmOHAm33qqLpJI+OZPot2+HCy6ABQsyHYlUBzNnhlL8Y4/BTTfBm2+GOu9Bg+CDD8Lz9deH0v3771f9eH/6065bS951l5K8pFfOJPrly0M37+7dYdy40HStJlGHq8Rs2gQXXxw6KO2zD7z9dkjodWKaJTRrBlOmhCqdVauga9dwvWTbtsodc9KkUC9/+unw0EMa310ywN2z6tGtWzevrLVr3c891x3cO3Z0nz+/0ruqVv72N/cGDcL7Ln40aBDmyy6vv+5+8MHuZu5XXeX+7bflb/P55+7nnBPOabdu7gsXVuyYjz3mXquW+0knuW/ZUrm4RRJBuBFUqXk144m95KMqib7Y00+7/+AH7rVru19/vfvWrVXeZVZr1Wr3JF/8aNUq05Flhy1b3K+5JiT4Nm3cX3214vt4/HH3Zs3c69VzHz/evaio/G2efda9Th33Y49137Sp4scUqYgal+jdQ+l+6NDwDjt0cJ87Nym7zUpmpSd6s0xHlnnvvut+1FHhfFxwgfuGDZXf1+efu599dthXfr77+++Xve7LL7vvsUf4FbBuXeWPKZKoeIk+Z2sLmzQJbZWffRbWroWePcPFsK1bMx1Z8pXVsaomd7jati3Uq/foAV9+Cc89F5qsNmpU+X02bx4u3j76aGgZ1bVr6IFasu7+X/8KY9YcfHAY3qBx4yq9FZGqK+sbIFOPZJXoY339tfsvfhFKYkce6T5nTtIPkVG5UEe/dav7pEnuEya4P/mk+9tvu69alVgVSUlLlrj37BnOw09/6v7ll0kP19escT/zzHCM7t3dFy0K8xcscN9nn3At4JNPkn9ckbJQE6tuSjN9unuLFqHufvToxC7GVRd/+1uokzcLz9Upyc+Y4d6uXenVT7VquR94YEimp5/ufskl7rfc4v7ww+4vvuj+4Ye7qmO2b3f/05/c99zTfd993f/+99TGvWOH+5Qp7k2bhrr7665z328/97w89//+N7XHFikpXqKvcYOarV8fbqf24INhrO5Jk0K1Tk2zdWsYAXHNml3Pa9dC795wzDHpieE//4FRo0K1yiGHhHFfevYMN8tYvbr0R2EhrFv3/X3tvXd4FBaGppMPPggHHpie97FmDfzyl2GY4f32C6NLtmuXnmOLFNPolaWYOTMMJvXJJ6E7+o03Qv36KT9sSn37bUiGa9bsnsCLn2Nfr19f9n569QrnZMAAqF07+XGuXw833xxujFG/PowdG9qZ16uX2PabN5f+JfDpp2H89REj0t8hyR1eeCF8YekerJIJSvRl2LABrr46XKQ7/PBQuj/66NDL9ptvyn9s3rz79JYtYVzwM8+EPfZIy1sAwljlf/wj/O1vIdmX1KQJ7L9/KG3uv//ur4uf33oL/vCHUCKuUydcYDzkkFDiHjbs+4OuVcb27aHD0HXXwRdfhIQ8fnw4vohUTbxEn/E6+ZKPVNbRl+WFF9xbtgz12/XqlV5XHO9Rp45748ahXhhCe+urr3ZfujR1MW/fHtpp//jH4Zj167ufd16ou37++dBZbPVq9+++K39fpV3MrVcvXFCEUAc9dmy4AFlZr73m3rVr2F/v3rnd3FUkE1Adffk2bAjjkWzcGEqve+2V+KNu3bCPHTvg5ZfhnnvC7d+2b4eTTgqDWP3kJ7t3s6+sTZtCqXjCBPjoo1APfckl4daDzZpVbp9l3eGqZcvwK+H222HatPArZdiwUMpPtA561Sr4zW/CfVDz8sJgXoMHa6wXkWRTiT4DVq92v+mm0AIDQsuRG24ITQYrY/ly91Gjwi8HcO/Rw/2RRxIrsZcnkQ5Xixe7X3hh6AQE7gMGuM+eHVqelGbzZvcbbwwtYOrXD78I1DtUJHVQ88rMKSoKQzL07x8SZ61a7gMHhuqV7dvjb7tjR+iuP2hQ2K52bffBg93feiu5MVZkCIU1a0LSbto0rNOzZxgeYNu2XTE/9lioCoPQk1RNDUVSr8qJHugHLAGWAqNLWT4K+AB4D3gJaBWzrCXwAvBhtE7reMfKtUQfa9my0H6/efNw5tu2df/970PX+lhbtrg/9JB7ly5hvSZN3K+9tvK/BspTmQ5X33zjfvfd7occEkuJhsYAAA2RSURBVNZv08b91lvd+/QJ0506uc+alZp4ReT7qpTogdrAMqAtUA/4N9C+xDonAg2i1xcDj8YsmwX8OHrdsHi9sh65nOiLbd0aOtocf3z4C9St6z5kSCjljxsXOt2Ae/v27hMnhqSaapXtcLVtm/tTT7n36uU7L9zee++uEr6IpEe8RF/uxVgzOwYY5+4nR9PXRnX7vytj/S7Ane7e28zaAxPd/di4B4mRqYuxmfLhh3DffeECa3Hb9lNPhcsvh759q9dFy8WL4YADNLaLSCbEuxibyKBmLYBVMdOF0byynAfMiF4fBqwzs6fM7F0zu83MUtAFp/o64gi4447Qceupp2DJkjAQ249/XL2SPIS+CEryItknCQ3+djGzc4F84PiY/R8HdAE+Bh4FhgMPltjuQuBCgJY1dMjFBg3gjDMyHYWI5KJESvSrgYNipvOiebsxs77AGGCAuxcPBlwILHD35e6+DZgKdC25rbtPdPd8d89v3rx5Rd+DiIjEkUiinwMcamZtzKweMBiYFrtCVC9/HyHJf15i233MrDh7/5DQ8kZyjO5ZK5K9yq26cfdtZnYpMJPQAufP7r7IzG4iXOWdBtxGaFHzuIWK5Y/dfYC7bzezXwMvWVgwD7g/VW9GMmPy5NAzd/PmML1yZZgGKCjIXFwiEmgIBKmysoZQaNUq3IlJRFKvqq1uROL6+OOKzReR9FKilyrTPWtFspsSvVTZ+PHfH6++QYMwX0QyT4leqqygINy8pVWr0MmrVaswrQuxItkhqR2mpOYqKFBiF8lWKtFLVlA7fJHUUYleMk7t8EVSSyV6ybgxY3Yl+WKbN4f5IlJ1SvSScWqHL5JaSvSScWqHL5JaSvSScWqHL5JaSvSScWqHL5JaanUjWUHt8EVSRyV6EZEcp0QvIpLjlOglJ6hnrUjZVEcv1Z561orEpxK9VHvqWSsSnxK9VHvqWSsSnxK9VHvqWSsSnxK9VHvqWSsSnxK9VHvqWSsSn1rdSE5Qz1qRsqlELyKS45ToRVCHK8ltqrqRGk8driTXqUQvNZ46XEmuU6KXGk8driTXKdFLjacOV5LrEkr0ZtbPzJaY2VIzG13K8lFm9oGZvWdmL5lZqxLL9zazQjO7M1mBiySLOlxJris30ZtZbeAuoD/QHhhiZu1LrPYukO/uHYEngFtLLL8ZmF31cEWSTx2uJNclUqLvASx19+Xu/h0wBRgYu4K7v+LuxZez3gbyipeZWTdgf+CF5IQsknwFBbBiBezYEZ4rmuTVPFOyWSKJvgWwKma6MJpXlvOAGQBmVgv4P+DX8Q5gZhea2Vwzm/vFF18kEJJI9ihunrlyJbjvap6pZC/ZIqkXY83sXCAfuC2a9UtgursXxtvO3Se6e7675zdv3jyZIYmknJpnSrZLpMPUauCgmOm8aN5uzKwvMAY43t23RrOPAY4zs18CDYF6ZrbJ3b93QVekulLzTMl2iST6OcChZtaGkOAHAz+LXcHMugD3Af3c/fPi+e5eELPOcMIFWyV5ySktW4bqmtLmi2SDcqtu3H0bcCkwE/gQeMzdF5nZTWY2IFrtNkKJ/XEzW2Bm01IWsUiWUfNMyXYJ1dG7+3R3P8zdD3b38dG8se4+LXrd1933d/fO0WNAKft4yN0vTW74IpmXjOaZarUjqaRBzUSSoCrj4WtQNUk1DYEgkmFqtSOppkQvkmFqtSOppkQvkmEaVE1STYleJMPUakdSTYleJMM0qJqkmlrdiGSBqrTaESmPSvQiIjlOiV5EJMcp0YvkAPWslXhURy9SzalnrZRHJXqRak49a6U8SvQi1Zx61kp5lOhFqjn1rJXyKNGLVHPqWSvlUaIXqebUs1bKo1Y3IjlAPWslHpXoRURynBK9iEiOU6IXEfWszXGqoxep4dSzNvepRC9Sw6lnbe5Tohep4dSzNvcp0YvUcMnoWas6/uymRC9Sw1W1Z21xHf/KleC+q45fyT57KNGL1HBV7VmrOv7sZ+6e6Rh2k5+f73Pnzs10GCKSoFq1Qkm+JDPYsSP98dRUZjbP3fNLW6YSvYhUiUbPzH5K9CJSJRo9M/sllOjNrJ+ZLTGzpWY2upTlo8zsAzN7z8xeMrNW0fzOZvaWmS2Klv002W9ARDJLo2dmv3ITvZnVBu4C+gPtgSFm1r7Eau8C+e7eEXgCuDWavxkY6u5HAv2AO8xsn2QFLyLZoaAAVqwIdfIrVlQ8yat5ZmolUqLvASx19+Xu/h0wBRgYu4K7v+Luxdfd3wbyovn/cfePotefAJ8DzZMVvIhUf2qemXqJJPoWwKqY6cJoXlnOA2aUnGlmPYB6wLJSll1oZnPNbO4XX3yRQEgikivUPDP1knox1szOBfKB20rMPwD4K/ALd/9egyt3n+ju+e6e37y5CvwiNYmGYEi9RBL9auCgmOm8aN5uzKwvMAYY4O5bY+bvDTwHjHH3t6sWrojkGjXPTL1EEv0c4FAza2Nm9YDBwLTYFcysC3AfIcl/HjO/HvAP4C/u/kTywhaRXKHmmalXbqJ3923ApcBM4EPgMXdfZGY3mdmAaLXbgIbA42a2wMyKvwjOAfoAw6P5C8ysc/LfhohUV8lonqlWO/FpCAQRqdZK3jgFwi+CmtaWX0MgiEjOUqud8inRi0i1plY75VOiF5FqTa12yqdELyLVmlrtlE+JXkSqNQ2qVr46mQ5ARKSqCgqU2ONRiV5EJMcp0YuI5DglehGRHKdELyI1Xq4PoaCLsSJSo5UcQqH4xieQOxd4VaIXkRqtJgyhoEQvIjVaTRhCQYleRGq0ZAyhkO11/Er0IlKjVXUIhepwc3MlehGp0ao6hEJ1qOPXjUdERKqgVq1Qki/JDHbsSF8cuvGIiEiKVIdhkpXoRUSqoDoMk6xELyJSBdXh5ubqGSsiUkVVGSY5HT1zVaIXEcmgdLTaUaIXEcmgdPTMVaIXEcmgdLTaUaIXEcmgdLTaUaIXEcmgdNzcXK1uREQyLNU3N1eJXkQkxynRi4jkOCV6EZEcp0QvIpLjlOhFRHJc1o1Hb2ZfACszHUcczYAvMx1EHIqvahRf1Si+qqlKfK3cvXlpC7Iu0Wc7M5tb1uD+2UDxVY3iqxrFVzWpik9VNyIiOU6JXkQkxynRV9zETAdQDsVXNYqvahRf1aQkPtXRi4jkOJXoRURynBK9iEiOU6IvwcwOMrNXzOwDM1tkZpeXss4JZrbezBZEj7EZiHOFmS2Mjj+3lOVmZhPMbKmZvWdmXdMYW7uYc7PAzDaY2RUl1knrOTSzP5vZ52b2fsy8Jmb2TzP7KHret4xth0XrfGRmw9IY321mtjj6+/3DzPYpY9u4n4UUxjfOzFbH/A1PKWPbfma2JPosjk5jfI/GxLbCzBaUsW06zl+peSVtn0F31yPmARwAdI1eNwL+A7Qvsc4JwLMZjnMF0CzO8lOAGYABRwP/ylCctYHPCJ05MnYOgT5AV+D9mHm3AqOj16OB/y1luybA8uh53+j1vmmK7ySgTvT6f0uLL5HPQgrjGwf8OoG//zKgLVAP+HfJ/6dUxVdi+f8BYzN4/krNK+n6DKpEX4K7f+ru86PXG4EPgRaZjapSBgJ/8eBtYB8zOyADcfwIWObuGe3t7O6zga9KzB4IPBy9fhg4vZRNTwb+6e5fufvXwD+BfumIz91fcPdt0eTbQF6yj5uoMs5fInoAS919ubt/B0whnPekihefmRlwDvD3ZB83UXHySlo+g0r0cZhZa6AL8K9SFh9jZv82sxlmdmRaAwsceMHM5pnZhaUsbwGsipkuJDNfWIMp+x8s0+dwf3f/NHr9GbB/Ketky3kcQfiFVpryPgupdGlUtfTnMqodsuH8HQescfePylie1vNXIq+k5TOoRF8GM2sIPAlc4e4bSiyeT6iK6AT8CZia7viAY929K9AfuMTM+mQghrjMrB4wAHi8lMXZcA538vAbOSvbGpvZGGAbMLmMVTL1WbgHOBjoDHxKqB7JRkOIX5pP2/mLl1dS+RlUoi+FmdUl/DEmu/tTJZe7+wZ33xS9ng7UNbNm6YzR3VdHz58D/yD8RI61GjgoZjovmpdO/YH57r6m5IJsOIfAmuLqrOj581LWyeh5NLPhwGlAQZQIvieBz0JKuPsad9/u7juA+8s4bqbPXx1gEPBoWeuk6/yVkVfS8hlUoi8hqs97EPjQ3f9Qxjo/iNbDzHoQzuPaNMa4l5k1Kn5NuGj3fonVpgFDo9Y3RwPrY34ipkuZJalMn8PINKC4BcMw4OlS1pkJnGRm+0ZVEydF81LOzPoB1wAD3H1zGesk8llIVXyx13zOKOO4c4BDzaxN9AtvMOG8p0tfYLG7F5a2MF3nL05eSc9nMJVXmqvjAziW8PPpPWBB9DgFGAmMjNa5FFhEaEHwNtArzTG2jY797yiOMdH82BgNuIvQ4mEhkJ/mGPciJO7GMfMydg4JXzifAkWEOs7zgKbAS8BHwItAk2jdfOCBmG1HAEujxy/SGN9SQt1s8efw3mjdA4Hp8T4LaYrvr9Fn6z1CwjqgZHzR9CmEVibL0hlfNP+h4s9czLqZOH9l5ZW0fAY1BIKISI5T1Y2ISI5TohcRyXFK9CIiOU6JXkQkxynRi4jkOCV6EZEcp0QvIpLj/j8NetWAvcQu0wAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bxJiUfJjlpEv"
      },
      "source": [
        "상식 수준의 모델보다 훨씬 좋은 성능을 보인다. 따라서 시퀀스를 펼쳐서 처리할 때에는 완전 연결 net에 비해 순환 net이 더 뛰어나다고 할 수 있다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jFvNumysnaZ0"
      },
      "source": [
        "## 3-5 ) 과대적합 해소"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W-xOhJuCniuS"
      },
      "source": [
        "### 3-5-1) Dropout 사용"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lA7RP_krnyA1",
        "outputId": "c28979df-2fa4-4e9c-cf61-2869efc35d2b"
      },
      "source": [
        "from keras.models import Sequential\r\n",
        "from keras import layers\r\n",
        "from keras.optimizers import RMSprop\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "model.add(layers.GRU(32,\r\n",
        "                     dropout=0.2,\r\n",
        "                     recurrent_dropout=0.2,\r\n",
        "                     input_shape=(None, float_data.shape[-1])))\r\n",
        "model.add(layers.Dense(1))\r\n",
        "\r\n",
        "model.compile(optimizer=RMSprop(), loss='mae')\r\n",
        "history = model.fit_generator(train_gen,\r\n",
        "                              steps_per_epoch=500,\r\n",
        "                              epochs=40,\r\n",
        "                              validation_data=val_gen,\r\n",
        "                              validation_steps=val_steps)"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.\n",
            "  warnings.warn('`Model.fit_generator` is deprecated and '\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/40\n",
            "500/500 [==============================] - 139s 274ms/step - loss: 0.3338 - val_loss: 0.2758\n",
            "Epoch 2/40\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.3067 - val_loss: 0.2772\n",
            "Epoch 3/40\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.3015 - val_loss: 0.2758\n",
            "Epoch 4/40\n",
            "500/500 [==============================] - 137s 273ms/step - loss: 0.2946 - val_loss: 0.2674\n",
            "Epoch 5/40\n",
            "500/500 [==============================] - 137s 274ms/step - loss: 0.2891 - val_loss: 0.2702\n",
            "Epoch 6/40\n",
            "500/500 [==============================] - 136s 273ms/step - loss: 0.2855 - val_loss: 0.2665\n",
            "Epoch 7/40\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.2806 - val_loss: 0.2681\n",
            "Epoch 8/40\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.2800 - val_loss: 0.2683\n",
            "Epoch 9/40\n",
            "500/500 [==============================] - 136s 273ms/step - loss: 0.2753 - val_loss: 0.2738\n",
            "Epoch 10/40\n",
            "500/500 [==============================] - 136s 271ms/step - loss: 0.2722 - val_loss: 0.2696\n",
            "Epoch 11/40\n",
            "500/500 [==============================] - 136s 271ms/step - loss: 0.2698 - val_loss: 0.2684\n",
            "Epoch 12/40\n",
            "500/500 [==============================] - 136s 271ms/step - loss: 0.2661 - val_loss: 0.2684\n",
            "Epoch 13/40\n",
            "500/500 [==============================] - 135s 270ms/step - loss: 0.2628 - val_loss: 0.2767\n",
            "Epoch 14/40\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.2613 - val_loss: 0.2716\n",
            "Epoch 15/40\n",
            "500/500 [==============================] - 135s 270ms/step - loss: 0.2595 - val_loss: 0.2770\n",
            "Epoch 16/40\n",
            "500/500 [==============================] - 135s 271ms/step - loss: 0.2577 - val_loss: 0.2755\n",
            "Epoch 17/40\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.2540 - val_loss: 0.2784\n",
            "Epoch 18/40\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.2542 - val_loss: 0.2826\n",
            "Epoch 19/40\n",
            "500/500 [==============================] - 135s 271ms/step - loss: 0.2495 - val_loss: 0.2820\n",
            "Epoch 20/40\n",
            "500/500 [==============================] - 135s 270ms/step - loss: 0.2489 - val_loss: 0.2828\n",
            "Epoch 21/40\n",
            "500/500 [==============================] - 137s 273ms/step - loss: 0.2490 - val_loss: 0.2788\n",
            "Epoch 22/40\n",
            "500/500 [==============================] - 136s 271ms/step - loss: 0.2471 - val_loss: 0.2872\n",
            "Epoch 23/40\n",
            "500/500 [==============================] - 137s 275ms/step - loss: 0.2452 - val_loss: 0.2896\n",
            "Epoch 24/40\n",
            "500/500 [==============================] - 137s 274ms/step - loss: 0.2426 - val_loss: 0.2884\n",
            "Epoch 25/40\n",
            "500/500 [==============================] - 137s 274ms/step - loss: 0.2425 - val_loss: 0.2913\n",
            "Epoch 26/40\n",
            "500/500 [==============================] - 137s 275ms/step - loss: 0.2405 - val_loss: 0.2954\n",
            "Epoch 27/40\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.2385 - val_loss: 0.2923\n",
            "Epoch 28/40\n",
            "500/500 [==============================] - 139s 278ms/step - loss: 0.2368 - val_loss: 0.3002\n",
            "Epoch 29/40\n",
            "500/500 [==============================] - 137s 274ms/step - loss: 0.2364 - val_loss: 0.2969\n",
            "Epoch 30/40\n",
            "500/500 [==============================] - 138s 275ms/step - loss: 0.2334 - val_loss: 0.2981\n",
            "Epoch 31/40\n",
            "500/500 [==============================] - 137s 273ms/step - loss: 0.2337 - val_loss: 0.2968\n",
            "Epoch 32/40\n",
            "500/500 [==============================] - 137s 274ms/step - loss: 0.2335 - val_loss: 0.2947\n",
            "Epoch 33/40\n",
            "500/500 [==============================] - 138s 275ms/step - loss: 0.2310 - val_loss: 0.2985\n",
            "Epoch 34/40\n",
            "500/500 [==============================] - 138s 276ms/step - loss: 0.2316 - val_loss: 0.3062\n",
            "Epoch 35/40\n",
            "500/500 [==============================] - 140s 281ms/step - loss: 0.2293 - val_loss: 0.3019\n",
            "Epoch 36/40\n",
            "500/500 [==============================] - 138s 275ms/step - loss: 0.2274 - val_loss: 0.3040\n",
            "Epoch 37/40\n",
            "500/500 [==============================] - 138s 276ms/step - loss: 0.2264 - val_loss: 0.3061\n",
            "Epoch 38/40\n",
            "500/500 [==============================] - 137s 274ms/step - loss: 0.2255 - val_loss: 0.3035\n",
            "Epoch 39/40\n",
            "500/500 [==============================] - 137s 274ms/step - loss: 0.2255 - val_loss: 0.3061\n",
            "Epoch 40/40\n",
            "500/500 [==============================] - 138s 276ms/step - loss: 0.2232 - val_loss: 0.3101\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        },
        "id": "wlKNUO15nyD6",
        "outputId": "d7ed8a2d-97b6-432e-caf1-b8478595f15c"
      },
      "source": [
        "loss = history.history['loss']\r\n",
        "val_loss = history.history['val_loss']\r\n",
        "\r\n",
        "epochs = range(1, len(loss) + 1)\r\n",
        "\r\n",
        "plt.figure()\r\n",
        "\r\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\r\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\r\n",
        "plt.title('Training and validation loss')\r\n",
        "plt.legend()\r\n"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f24e9dc8f28>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3yUVfb48c+hV0UpLhJMQFFExCABCwIWVBAWLPhVjAqiIqysbVFRFBFlf66goiuWWFCKCygugmVVOnaKAQRhV6pRFhDEwAJC4Pz+uM/AZJLMTJKZTMl5v155MfO0OfOEnLlzn/ucK6qKMcaY5FUh1gEYY4yJLkv0xhiT5CzRG2NMkrNEb4wxSc4SvTHGJDlL9MYYk+Qs0ZtiEZGPRKRPpLeNJRHZICKdo3BcFZGTvMcvicjD4WxbgtfJFJFPShpnkOOeLyI5kT6uKXuVYh2AiT4R2e33tAbwO3DQe36bqk4K91iq2jUa2yY7VR0QieOISBqwHqisqnnesScBYf8OTfljib4cUNVavscisgG4RVVnBW4nIpV8ycMYkzys66Yc8301F5H7ReS/wDgROUZE3heRbSLyq/c4xW+feSJyi/e4r4h8JiKjvW3Xi0jXEm7bREQWiMguEZklImNFZGIRcYcT42Mi8rl3vE9EpJ7f+htEZKOIbBeRoUHOz1ki8l8Rqei37AoRWe49biciX4rIThHZLCLPi0iVIo71hog87vf8Xm+fn0WkX8C23UTkWxHJFZEfRWS43+oF3r87RWS3iJzjO7d++58rIotE5Dfv33PDPTfBiMip3v47RWSliPTwW3eZiKzyjvmTiAz2ltfzfj87RWSHiCwUEcs7ZcxOuPkDcCyQCvTH/Z8Y5z0/AdgLPB9k/7OANUA94EngNRGREmz7FvANUBcYDtwQ5DXDifE64CagAVAF8CWeFsCL3vGP914vhUKo6tfA/4ALA477lvf4IHC3937OAS4C/hQkbrwYunjxXAw0AwKvD/wPuBGoA3QDBorI5d66jt6/dVS1lqp+GXDsY4EPgOe89/Y08IGI1A14DwXOTYiYKwMzgU+8/f4MTBKRU7xNXsN1A9YGWgJzvOV/AXKA+sBxwIOA1V0pY5bozSHgEVX9XVX3qup2VZ2mqntUdRcwEugUZP+NqvqKqh4E3gQa4v6gw95WRE4A2gLDVHW/qn4GzCjqBcOMcZyq/ltV9wJTgXRveS/gfVVdoKq/Aw9756Ao/wB6A4hIbeAybxmqukRVv1LVPFXdALxcSByF+T8vvu9U9X+4Dzb/9zdPVVeo6iFVXe69XjjHBffB8B9VneDF9Q9gNfBHv22KOjfBnA3UAp7wfkdzgPfxzg1wAGghIkep6q+qutRveUMgVVUPqOpCtQJbZc4Svdmmqvt8T0Skhoi87HVt5OK6Cur4d18E+K/vgaru8R7WKua2xwM7/JYB/FhUwGHG+F+/x3v8Yjre/9heot1e1GvhWu9XikhV4Epgqapu9OI42euW+K8Xx19xrftQ8sUAbAx4f2eJyFyva+o3YECYx/Ude2PAso1AI7/nRZ2bkDGrqv+Hov9xr8J9CG4Ukfkico63fBTwA/CJiKwTkSHhvQ0TSZboTWDr6i/AKcBZqnoUR7oKiuqOiYTNwLEiUsNvWeMg25cmxs3+x/Zes25RG6vqKlxC60r+bhtwXUCrgWZeHA+WJAZc95O/t3DfaBqr6tHAS37HDdUa/hnXpeXvBOCnMOIKddzGAf3rh4+rqotUtSeuW2c67psCqrpLVf+iqk2BHsA9InJRKWMxxWSJ3gSqjevz3un19z4S7Rf0WsiLgeEiUsVrDf4xyC6lifEdoLuInOddOB1B6L+Dt4A7cR8obwfEkQvsFpHmwMAwY5gK9BWRFt4HTWD8tXHfcPaJSDvcB4zPNlxXU9Mijv0hcLKIXCcilUTkGqAFrpulNL7Gtf7vE5HKInI+7nc02fudZYrI0ap6AHdODgGISHcROcm7FvMb7rpGsK4yEwWW6E2gMUB14BfgK+BfZfS6mbgLmtuBx4EpuPH+hSlxjKq6Ergdl7w3A7/iLhYG4+sjn6Oqv/gtH4xLwruAV7yYw4nhI+89zMF1a8wJ2ORPwAgR2QUMw2sde/vuwV2T+NwbyXJ2wLG3A91x33q2A/cB3QPiLjZV3Y9L7F1x5/0F4EZVXe1tcgOwwevCGoD7fYK72DwL2A18CbygqnNLE4spPrHrIiYeicgUYLWqRv0bhTHJzlr0Ji6ISFsROVFEKnjDD3vi+nqNMaVkd8aaePEH4F3chdEcYKCqfhvbkIxJDtZ1Y4wxSc66bowxJsnFXddNvXr1NC0tLdZhGGNMQlmyZMkvqlq/sHVxl+jT0tJYvHhxrMMwxpiEIiKBd0QfZl03xhiT5CzRG2NMkrNEb4wxSc4SvTHGJDlL9MYYk+TCSvQi0kVE1ojID4XVkxaRASKyQkSyxU0X18JbfrGILPHWLRGRCwsePTImTYK0NKhQwf07yaZKNsYYIIzhld5kDmNx057lAItEZIZXp9vnLVV9ydu+B276si64Knd/VNWfRaQl8DH5J0CIiEmToH9/2ONNW7Fxo3sOkJlZ9H7GGFMehNOibwf8oKrrvFKlk3EFpw5T1Vy/pzXxJkdQ1W9V9Wdv+UqgujdTT0QNHXokyfvs2eOWG2NMeRfODVONyD/tWQ5ukud8ROR24B7cZMOFddFchZuGrUCNcRHpj5uYmhNOCJxsJ7RNm4q33BhjypOIXYxV1bGqeiJwP/CQ/zoROQ34G3BbEftmqWqGqmbUr1/oHbxBFfXZUILPDGOMSTrhJPqfyD+/ZQrB55+cDFzueyIiKcA/cbPRrC1JkKGMHAk1auRfVqOGW26MMeVdOIl+EdBMRJp4c2xei5u4+DARaeb3tBvwH295HeADYIiqfh6ZkAvKzISsLEhNBRH3b1aWXYg1xhgIo49eVfNEZBBuxExF4HVVXSkiI4DFqjoDGCQinYEDuDk4+3i7DwJOAoaJyDBv2SWqujXSbyQz0xK7McYUJu4mHsnIyFCrXmmMMcUjIktUNaOwdXZnrDHGJDlL9MYYk+Qs0RtjTJKzRG+MMUnOEr0xxiQ5S/TGGJPkLNEbY0ySs0RvjDFJzhK9McYkOUv0xhiT5CzRG2NMkrNEb4wxSc4SvTHGxAFV9xMNluiNMSbGVOG++2DAADh0KPLHt0RvjDExNnIkjB4NlSq5yZMizRK9McbE0HPPwcMPww03wN//boneGGOSyhtvwJ13wuWXw+uvQ4UoZWRL9MYYEwPvvAM33wydO8Pkya7bJlos0RtjTBn717/guuvg7LNh+nSoWjW6r2eJ3hhjytDChXDllXDaafDBB1CzZvRf0xK9McaUkSVLoHt3OOEE+PhjqFOnbF7XEr0xJmlkZ8NTT0VnLPqPP8Ijj8DKlSXbPzsbLr0UjjkGZs2CBg0iG18wluiNMUnh0CHo0wcGD4b774/ccX/7DR54AE4+GUaMgDPPhFGj4ODB8PbPy4O//hXatYMqVVyST0mJXHzhKBeJftIkSEtzQ5fS0txzY0xymT4dli+HjAx389Fzz5XuePv3u2OceCI88QRcfTUsXgzdurm7WM8/H9atC36MVavgnHNg6FC44gpYtgxOOql0cZWIqsbVT5s2bTSSJk5UrVHDV0XC/dSo4ZYbY5LDwYOqrVqpnnyy6u+/q15+uaqI6jvvFP9Yhw6pTpmi2rSpyxcXXaS6dGn+9ePHqx59tGrNmqovveSW+TtwQPWJJ1SrVFGtV0916tTSvb9wAIu1iLwa88Qe+BPpRJ+amj/J+35SUyP6MsaYGJo2zf1dT5jgnu/Zo3rOOapVq6ouXBj+cRYsUG3Xzh3r9NNVP/qoYBL32bRJtXNnt22XLqo5OW7599+rnnWWW37VVapbtpTuvYWrXCd6kcITvUhEX8YYEyP+rfkDB44s/+UXt+yYY1RXrQp+jPXrVXv1crmhUSPV119XzcsL77Wff161enX3OoMGuQ+XY49V/cc/iv6QiIZgiT7p++hPOKF4y40xieW991zf/MMP57+7tG5dd2NSlSrQtSts3lxw39274aGHoHlzN6b90Ufh3/+Gm26CihVDv3aFCnD77a7vvXlzeP556NLFjcy59tro1K0piaRP9CNHQo0a+ZfVqOGWG2MS26FDLjk3a+YSa6AmTVwC/+UXuOwyyM09st/48W4kzciR0KuXS/DDhhXMF+Fo1szdCLViBfzzn/CHP5TufUVa0if6zEzIyoLUVPfpmprqnmdmxjoyY0xpvfeea00Htub9tWkDb7/tknCvXrBggRsJ06cPNG4MX34JEyeWfshjxYrQsmX8tOL9ievaiR8ZGRm6ePHiWIdhjIlzhw65Me179rhhjKGKgo0bB/36uccNG7ohk9dfH72KkWVNRJaoakZh66JYL80YY6JnxgzXmh8/PrzKjzfdBAcOwJYtcPfdUKtW9GOMF2G16EWkC/AsUBF4VVWfCFg/ALgdOAjsBvqr6ipv3QPAzd66O1T142CvZS16Y0woqq41/7//hdeaLw+CtehDfmkRkYrAWKAr0ALoLSItAjZ7S1VPV9V04EngaW/fFsC1wGlAF+AF73hxxe6cNeaIvDz4/PPoTVQdCe+952rHPPSQJflwhNM71Q74QVXXqep+YDLQ038DVc31e1oT8P0X6QlMVtXfVXU98IN3vLgxaRL07w8bN7r/2Bs3uueW7E159fjjcN55rj97//5YR1OQqhtpc9JJrqa7CS2cRN8I+NHveY63LB8RuV1E1uJa9HcUc9/+IrJYRBZv27Yt3NgjYuhQdzHH3549brkx5U1uLjz7rLvP5I033JjwX3+NdVT5+VrzwUbamPwidr1ZVceq6onA/cBDxdw3S1UzVDWjfv36kQopLJs2FW+5McnsxRdh506YNg0mTIDPPoNzzw1dvKusWGu+ZMJJ9D8Bjf2ep3jLijIZuLyE+5Y5u3PWGGfPHlfL/dJLXQXI6693JXW3bnVT3n35ZXjHyc11d5xGw/PPW2u+JMJJ9IuAZiLSRESq4C6uzvDfQESa+T3tBvzHezwDuFZEqopIE6AZ8E3pw44cu3PWGOeVV2DbNneB06djR5fgjz4aLrgApk4tfN+tW92NiF26uNID9eq5m5OmTYN9+yITX1YW3HGHKxNsrfliKqoIjv8PcBnwb2AtMNRbNgLo4T1+FlgJZANzgdP89h3q7bcG6BrqtSJd1CwcEye6apYi7l8rYWzKm337VI8/XrVTp8LXb9um2r69K/r1//6fK9a1aZPqs8+qduyoWqGCW3fiiar33qt6552qxx3nlh11lGrfvqoff5y/6FhxvPqqO9Zll7lYTUGU5+qVxpjQXn7ZZYNPPy16m717VXv3dtuddNKRSrAtW6oOG6a6bFn+ao0HDrjj3XSTS/ag2qCB6p//7Er5hmvcONcI69LFxWAKFyzRWwkEY8q5Awdcca8GDeCrr4LXalF10+J9+qnrprnySrdvKPv2wYcfwj/+ATNnurH6/fvD8OHB504dPx769oXOnd2dsNWqFffdlR/BbpiyRG9MOTd+vCvwNWMG/PGP0X+9bdvcyJmXXnLXw4YMgbvuKnitbNIkuOEGuPBC9+FQvXr0Y0tkpboz1hiTWA4dghdecPObhnLwoGuhn3EGdO8e/dgA6td3o2dWrnRJfOhQ963gzTdd7OBa/jfe6OZlnTHDknxpWaI3Jomowl/+4ibDaN/e3fQUzLvvwpo1LtmWdXndU05xE3rPnw/HH++6aNq0gccec0M7O3RwLfmS1Ic3+VmiNyaJPPoojBkDAwa4RHnTTa5bJC+v4LaqrtxB8+aurz1WOnZ01wbeesvdhTtsmPuQev99qFkzdnElE0v0YbCiZyYRPP20S/Q33QRjx7pp9O6805U06NoVduzIv/3777sp+B54ILxp86KpQgXo3RtWr3Zj7z/8sHyVEY42uxgbgq/omX89nBo1bJYqE19efRVuvRWuvtr1b/sn7nHjXAu/cWNXJ+a001xr/uyz3YXRNWugcuXYxW4iwy7GloIVPTPxbsoU1xjp2tVNiRfYOr/pJpg3z5UlOPtsl+xnz4ZvvnEjXizJJz9r0YdQoULhdblFjowQMCZWPvgALr/czYH6r38Fv3CZkwNXXOFG4xx/vPs/vHYtVK1advGa6LEWfSlY0TMTr+bOhauugvR0198eanRKSoqbGDszE37+Ge6/35J8eWGJPgQrembi0ddfQ48ecOKJriV/1FHh7Ve9uis//O23MGhQdGM08cMSfQiZme7Ca2qq+6qbmlrwQqyNyjFlacIEV0myfn1XiqBu3eLtL+K+BZT1uHkTO1bROQyZmUWPsAkcleObitC3nzGRsm+fGxP/8svQqZMbXdOwYayjMonAWvSlZKNyTFlYv97N4/ryy26kzKxZluRN+KxFX0o2FaEpLlVXeuCf/3TJu0cPNwqmKDNnurovqm5oZI8eZRerSQ7Woi8lG5VjimPDBlchslcvV6xr4EBo1AjOOssVF1u16shw3rw8d9dqjx7QtCksXWpJ3pSMJfpSslE5JhwHDsCoUe6u1Llz3dysO3bAd9+5/yuqrrvvtNNcJcfBg+Hii+GJJ9w1n88/d8nemJKwG6YiYNIk90e6aZNryY8caRdizRFffQW33ebqyvzxj65Eb2Hf+H76yXXTTJ8Oc+a4ya9fesl12xgTik08YkwM7NwJDz7okvXxx8Pf/+7uYg1nWGNurrvzuk6d6MdpkkOwRG8XY42Jgp9+gnbt4L//hTvucDXWa9cOf/9wb4AyJhyW6I2JgmeegS1b4Isv3IVWY2LJLsYaE2G//ebunr7mGkvyJj5Yoi8DViKhfMnKgl273JR+xsQD67qJMiuRUL7s3+9mdLrwQjjzzFhHY4xjLfoosxIJ5cvkye5C7L33xjoSY46wRB9lViKh/FCF0aOhZUu49NJYR2PMEZboo8xKJJQfn34KK1a4u1qtBLCJJ5boo8xKJJQfo0a5G6N69451JMbkZxdjo8x3wdVKJJS9zZvdxNknnOC6U048seDE2f5U4ccf3exN33zj5lbt2BGGDw/dQs/OdqWD//Y3qFIlom/DmFKzRF8Ggk1cYqJj+3Y38mX16iPLqlaFU091Sd/3U6WKS+rffOMS/JYtbtsqVdwHw4gR8L//udZ6sGQ/ejTUqnVkRJUx8cQSvUk6e/a44mHr1rn5VOvVc1UifT/z5sHEifn3OflkuOQSd4NTu3ZwxhlQubIrX/DUU1CzJjz6aOGv9+OPbrTNHXdYbRoTn8JK9CLSBXgWqAi8qqpPBKy/B7gFyAO2Af1UdaO37kmgG+56wKfAnRpvldRM0sjLc3ekfvUVTJ16ZPRLmzb5t9u5E1auhL173bpjjin8eM8+6z44Roxw11buv7/wbcBN82dMXFLVoD+45L4WaApUAZYBLQK2uQCo4T0eCEzxHp8LfO4doyLwJXB+sNdr06aNlicTJ6qmpqqKuH8nTox1RInr0CHVm29WBdWxYyN33Lw81d693XGfey7/up07VWvXVr3uusi9njElASzWIvJqOC36dsAPqroOQEQmAz2BVX4fFnP9tv8KuN63CqjmfUAIUBnYUqxPoiRmd81G1rBh8Npr8NBD8Kc/Re64FSvCm2+61v8dd7iW/c03u3W+cgeDB0fu9YyJtHCGVzYCfvR7nuMtK8rNwEcAqvolMBfY7P18rKrflyzU5GN3zUbOCy/A44+7BDxiROSPX7my64fv0gVuvRXeeutIuYOLLoLWrSP/msZESkTH0YvI9UAGMMp7fhJwKpCC+3C4UEQ6FLJffxFZLCKLt23bFsmQ4lo4d81aQbTQpk2DQYPcBdiXXorezUpVq7rX6tTJzfp0882u3IG15k28CyfR/wQ09nue4i3LR0Q6A0OBHqr6u7f4CuArVd2tqrtxLf1zAvdV1SxVzVDVjPr16xf3PSSsUHfN+rp2Nm50Y7x9XTuW7I+YPx+uuw7OPtu1uCtFeRxZjRpuUu927dzIHSt3YBJBOIl+EdBMRJqISBXgWmCG/wYi0hp4GZfkt/qt2gR0EpFKIlIZ6ARY140n1F2z1rUT3KxZ0LOnG+8+c2bBcxkttWvDhx+6D5gxY6zcgYl/IRO9quYBg4CPcUl6qqquFJERItLD22wUUAt4W0SyRcT3QfAObsTOCtxonWWqOjPSbyJRZWa6i3mpqS5ZpKa6574LsVYQrXCHDrl++EsugUaN3Fj5unXLNoY6ddw3q4suKtvXNaYkys3k4GvWuK/5nTtD06YRP3xUpKW57ppAqamwYUNZRxMftm2D66+HTz6BG26AF190NzMZU94Fmxw8qYua7d4N48bBeedB8+Zw223ua/6FF7rW2N69sY4wuEQoiHboEKxfX/J9e/d2fdyTJ8O+fcG3/+ILN7pl/nz3zefNNy3JGxOOpEv0qvDll3DLLdCwIfTrB7/84opNZWe7IXgbN7pWYcOGbrz1kiVuv3gTqmsnHjz6qPuG9NFHxd939GiX4LOzXcJv2NCNnlm6NP92qvD00260S7Vq7vd7663WN25M2Iq6kypWPyW9M3bnTtVRo1RPPdXdwVizpmq/fqqffebumPR38KDqnDmqmZmq1aq57c84Q/WNN0r00uXWrl2qdeq483fssaqbNoW/76JFqpUqqV51lft9fPqpu/u0alV3vPR0dxfq2rWqV1zhll1xhfs9G2MKIsidsTFP7IE/JU30W7eqVq6seu65qq++qpqbG95+v/7qbpdv1cqdjSVLSvTyMRPLEgrPPOPO2YQJrgzA2Wer/v576P127VJt1kw1JUV1+/b863bscL+PNm3cscF9IDz1VMEPbGPMEeUi0auq/vRTiXc9XLOkd++SH6OsTZyoWqPGkYQI7nlZJPv9+1UbN1bt2NE9nzrVvf5dd4Xet18/98E0b17w7bKzVUeMUP3yy9LHa0yyKzeJvrT+8hfVihVVN2yIWQjFkpqaP8n7flJTo//aEye613r//SPL/vxnt2zatKL3830gPPhg9GM0pjwJlujLzfDKcPz4o7uwOGgQPPNMTEIolgoVCr+ILOJGtESLKqSnu5LAK1a4OAB+/x06dHBDWZcsgZNOyr/fpk2uzvvJJ8Nnn7n6McaYyCi3wyuLq3FjV8v8lVfg119jHU1osZp4/NNPYflyuPfeI0keXC2Yt9921R6vvjr/8NWDB92497w8N7TVkrwxZccSfYDBg93UcS+/HOtIQhs5smBtl2rVoj/O/skn3STY111XcF1qKkyY4IZM+k/E8be/wYIF8PzzBVv6xpjoskQfID3d3T373HOuKyKeNWjgWsi1a7vnFStC9equ+yRali6F2bNdEi9qEuxu3WDIEDfmf+JENxfrsGFw7bWu6qMxpmxZoi/EvffC5s2u5ni82rED+vZ1k11v2eL6zZcudX3zl1zibhKDyJc5HjXKfbCEmgT7scegY0d3N/I110BKiitXYDc5GVP2LNEX4uKLoVUrd+dmnF2rBlxMAwfC1q2uxVy9ulveqpWr4rhxo2tVv/ZaZMscr1/v5mEdMACOPjr4tpUqubtea9VyF7knTrSJs42JFUv0hRBxffWrVrnKiPHmrbdcwn30UTjzzPzrOnSAKVNg8WI3eiiSZY6fecZ1D915Z3jbN2wIc+e6kr7nnVey1zTGlJ4NryzC/v1uqOXJJ8OcOaG337fPjSSpWDG6cW3a5Frup53mLm4W9Xqvvebq/RSmJMMvf/nFjea55hpXKM4YE19seGUJVKniLjjOnevGhAfzr3+5oZmdOkFubvRiOnTI9csfPOhGtgT7ULn55qK7SvyHX4bbh//CC264pE2bZ0zisUQfxK23uguPo0cXvv7AATe6pGtXOOYYN7rk4oth587oxDNmjPvgefbZ8Grq//3vBYdfVq9+ZPhluFMV7t3rjtWtm/smYYxJMEXdMhurn1iWQCjM4MGuLML69fmXb9rkCqiB6q23qu7Zozp9uius1qZNwWJdpbVihWqVKqo9exavuNf48a6Gj688QtWqrmLk+PGuqFg4JRReeMEtnz8/om/JGBNBWAmEkvOVRbj9dteiBnj/fejTx/XjZ2W5Wuo+778PV10FLVq4OU1LM8XdoUPu5q3cXNea3rzZlRxo0KD4x/r9d5g3D957z/38/HPw7WfMcH35Iu7ia716rg68DY80Jj4F66OPeQs+8CfeWvSqqtdf7+rbb9niCp/56tevWVP49h995FrOp5/uyicHk5ur+vzzqp06ue2bNFGtX79gVUpQnTkzMu/n4EHVr79WPeqowlv0hf28+25kXtsYEx1Yi750li1zd8zWrQvbt7sx7E8/7coNFGXWLOjRw30bmD0bjjsu//o1a2DsWHjjDdi1y42kadrUjTsv7Kd5c3exN5J8ffT+QzCrVXPDL7t2dSn+0CG3rFWryL62MSayrEUfAZdd5vq6p0wJf5+5c13LvHlz1Z9/Vs3LU50xQ/WSS1wruXJl923hq6+iFnZIsZy4xBgTOViLvvT27HF98sW9u3PhQtc69vWrr1/vCoINHOhG9QS29OPNpEmuhb9pkxuWOXJkfM1Za4xxgrXoKxW20BRUo4b7Ka4OHeCTT6B7d2jZEp54Aq64IjHK9AZ27fiGX4Ile2MSibXoy8ihQ/lrtyeCtDSX3AOlpsKGDWUdjTEmGLszNg4kWpIH111TnOXGmPiUgOnHlJVYzWBljIksS/SmSCNHFrwuUaNG9GewMsZEliV6U6TMTHfnb2qquyM2NdU9twuxxiQWG3VjgsrMtMRuTKKzFr0psUhPU2iMiQ5L9KZEwilxbB8ExsQHS/SmRIYODT5NYbi17o0x0RdWoheRLiKyRkR+EJEhhay/R0RWichyEZktIql+604QkU9E5Htvm7TIhW9iJdQY+1AfBMaYshMy0YtIRWAs0BVoAfQWkRYBm30LZKhqK+Ad4Em/deOBUap6KtAO2BqJwE1shRpjbzdbGRM/wmnRtwN+UNV1qrofmAz09N9AVeeqqq/99hWQAuB9IFRS1U+97Xb7bWcSWKgx9nazlTHxI5xE3wj40e95jresKG7xeTcAABQlSURBVDcDH3mPTwZ2isi7IvKtiIzyviGYBBdqjH04N1vZxVpjykZEx9GLyPVABuCbIqMS0AFoDWwCpgB9gdcC9usP9Ac4wZp8CSPYGHvf8qJKHFtlTGPKTjgt+p+Axn7PU7xl+YhIZ2Ao0ENVf/cW5wDZXrdPHjAdODNwX1XNUtUMVc2oX79+cd+DiVOZma7K5aFD7l//BB7OxVpr8RsTGeEk+kVAMxFpIiJVgGuBGf4biEhr4GVckt8asG8dEfFl7wuBVaUP2yS6UBdrbXimMZETMtF7LfFBwMfA98BUVV0pIiNEpIe32SigFvC2iGSLyAxv34PAYGC2iKwABHglCu/DJJhQF2tteKYxkWMTj5iYKGxi8ho1jlzQrVDBteQDibiuIGNMfjbxiIk7oUbt2PBMYyLHEr2JmWAXa60WvjGRY4nexCWrhW9M5Fg9ehO3rBa+MZFhLXqTsGycvTHhsRa9SUh2Z60x4bMWvUlINs7emPBZojcJKZw7a61bxxjHEr1JSMHG2Vv5BGPys0RvElKwcfbWrWNMfpboTUIKNs7eZrcyJj8bdWMSVlHj7E84wXXXFLbcmPLIWvQm6djsVsbkZ4neJJ1Q5RPsYq0pb6xMsSl30tIK79pJTXXF1YxJRFam2Bg/kbhYa10/JpFYojflTmlr3VvXj0k0luhNuVPai7U2Tt8kGkv0ptwp7cVaG6dvEo0lelMuBZvdKlSLPZyuH+vDN/HEEr0xAUK12EN1/Vgfvok3luiNCRCqxR6q68f68E28sURvTIBwLtYG6/qxEsom3liiNyZAaScmtxLKJt7YnbHGRFjgNIfgvhFkZbnuG7sr10SD3RlrTBmyEsom3liZYmOiwEoom3hiLXpjylA4F3qNiTRL9MaUoXAu9NqoHBNpluiNKWPBhmaGMyrHPghMcVmiNyaOhLrZyoZnmpKwRG9MHAk1KsfuujUlEVaiF5EuIrJGRH4QkSGFrL9HRFaJyHIRmS0iqQHrjxKRHBF5PlKBG5OMQpVfCGd4pnXtmEAhE72IVATGAl2BFkBvEWkRsNm3QIaqtgLeAZ4MWP8YsKD04RqT3EKNygn1QWBdO6Yw4bTo2wE/qOo6Vd0PTAZ6+m+gqnNV1feF8isgxbdORNoAxwGfRCZkY5JXqFE5oT4IrGvHFCacRN8I+NHveY63rCg3Ax8BiEgF4ClgcLAXEJH+IrJYRBZv27YtjJCMSV7BRuWE+iCwrh1TmIjeGSsi1wMZQCdv0Z+AD1U1R0SK3E9Vs4AscLVuIhmTMcmmqLtuIfSdt4F1eHxdO77jmuQUTov+J6Cx3/MUb1k+ItIZGAr0UNXfvcXnAINEZAMwGrhRRJ4oVcTGmCJZ144pTDgt+kVAMxFpgkvw1wLX+W8gIq2Bl4EuqrrVt1xVM/226Yu7YFtg1I4xJjL8Jz/ZtMm15EeOLF7Xjkk+IVv0qpoHDAI+Br4HpqrqShEZISI9vM1GAbWAt0UkW0RmRC1iY0xQwfr4wxm1E6z/3vr3E5PVozemHAlWKx+KXpeZGXxf69+PvWD16C3RG1POTJpUeNdOWlrwSVFCrTexlfCJ/sCBA+Tk5LBv374YRWWKo1q1aqSkpFC5cuVYh2KKoUIFd5NVIBHXDRRqvYmtYIk+ISYeycnJoXbt2qSlpRFsmKaJPVVl+/bt5OTk0KRJk1iHY4oh1NBMmzQlcSVEUbN9+/ZRt25dS/IJQESoW7eufftKQKGGZtqkKYkrIRI9YEk+gdjvKjGFuus2nElTTHxKiK4bY0zZCHbXbTjrTXxKmBZ9cUR6rO/27dtJT08nPT2dP/zhDzRq1Ojw8/379wfdd/Hixdxxxx0hX+Pcc88tXZCeefPm0b1794gcy5jisnH48SnpWvTRqOVRt25dsrOzARg+fDi1atVi8OAjddry8vKoVKnwU5mRkUFGRqEXwvP54osvShacMXEi1N+e1dmJnaRr0ZdVLY++ffsyYMAAzjrrLO677z6++eYbzjnnHFq3bs25557LmjVrgPwt7OHDh9OvXz/OP/98mjZtynPPPXf4eLVq1Tq8/fnnn0+vXr1o3rw5mZmZ+IbAfvjhhzRv3pw2bdpwxx13hGy579ixg8svv5xWrVpx9tlns3z5cgDmz59/+BtJ69at2bVrF5s3b6Zjx46kp6fTsmVLFi5cGNkTZpJeqL+9cP42rcUfHUnXoi/LWh45OTl88cUXVKxYkdzcXBYuXEilSpWYNWsWDz74INOmTSuwz+rVq5k7dy67du3ilFNOYeDAgQXGm3/77besXLmS448/nvbt2/P555+TkZHBbbfdxoIFC2jSpAm9e/cOGd8jjzxC69atmT59OnPmzOHGG28kOzub0aNHM3bsWNq3b8/u3bupVq0aWVlZXHrppQwdOpSDBw+yJ/Av0pgQQv3thVpvLf7oSboWfahaHpF09dVXU7FiRQB+++03rr76alq2bMndd9/NypUrC92nW7duVK1alXr16tGgQQO2bNlSYJt27dqRkpJChQoVSE9PZ8OGDaxevZqmTZseHpseTqL/7LPPuOGGGwC48MIL2b59O7m5ubRv35577rmH5557jp07d1KpUiXatm3LuHHjGD58OCtWrKB27dolPS2mnAr1txdqvVXWjJ6kS/RlOda3Zs2ahx8//PDDXHDBBXz33XfMnDmzyHHkVatWPfy4YsWK5OXllWib0hgyZAivvvoqe/fupX379qxevZqOHTuyYMECGjVqRN++fRk/fnxEX9Mkv9KOww+nxW8Xeksm6RJ9rMb6/vbbbzRq5CbeeuONNyJ+/FNOOYV169axwSsqMmXKlJD7dOjQgUne//Z58+ZRr149jjrqKNauXcvpp5/O/fffT9u2bVm9ejUbN27kuOOO49Zbb+WWW25h6dKlEX8PJrmVdhx+sBZ/qLlwba7c4JIu0UPwMq3Rct999/HAAw/QunXriLfAAapXr84LL7xAly5daNOmDbVr1+boo48Ous/w4cNZsmQJrVq1YsiQIbz55psAjBkzhpYtW9KqVSsqV65M165dmTdvHmeccQatW7dmypQp3HnnnRF/Dyb5hfrbC7Y+WIs/Ehd6y7OEKGr2/fffc+qpp8Yoovixe/duatWqhapy++2306xZM+6+++5Yh1Uo+52ZkiiqsqYVXAstWFGzpGzRJ6tXXnmF9PR0TjvtNH777Tduu+22WIdkTEQV1eIv7YVeKN99+JboE8jdd99NdnY2q1atYtKkSdQI/J5rTJIq7YXe8t6Hb4neGBP3Snuht7z34VuiN8YkhNJc6A3nRspk7tqxRG+MSXrhTIoeqmsnkT8ILNEbY5JeqD78UF07id7Hb4k+DBdccAEff/xxvmVjxoxh4MCBRe5z/vnn4xsmetlll7Fz584C2wwfPpzRo0cHfe3p06ezatWqw8+HDRvGrFmzihN+oaycsSlPQvXhh+raSfQ+fkv0YejduzeTJ0/Ot2zy5Mlh1ZsBV3WyTp06JXrtwEQ/YsQIOnfuXKJjGVOeBevDD9W1U9piibHu9km46pV33QVeafiISU+HMWOKXt+rVy8eeugh9u/fT5UqVdiwYQM///wzHTp0YODAgSxatIi9e/fSq1cvHn300QL7p6WlsXjxYurVq8fIkSN58803adCgAY0bN6ZNmzaAGyOflZXF/v37Oemkk5gwYQLZ2dnMmDGD+fPn8/jjjzNt2jQee+wxunfvTq9evZg9ezaDBw8mLy+Ptm3b8uKLL1K1alXS0tLo06cPM2fO5MCBA7z99ts0b968yPe3Y8cO+vXrx7p166hRowZZWVm0atWK+fPnH75DVkRYsGABu3fv5pprriE3N5e8vDxefPFFOnToULpfgDExNnJk/sqZkL9rpzQTo8dDVU5r0Yfh2GOPpV27dnz00UeAa83/3//9HyLCyJEjWbx4McuXL2f+/PmHa74XZsmSJUyePJns7Gw+/PBDFi1adHjdlVdeyaJFi1i2bBmnnnoqr732Gueeey49evRg1KhRZGdnc+KJJx7eft++ffTt25cpU6awYsWKw0nXp169eixdupSBAweG7B7ylTNevnw5f/3rX7nxxhsBDpczzs7OZuHChVSvXp233nqLSy+9lOzsbJYtW0Z6enqJzqkx8SRU10444/SLarHHQ7dPwrXog7W8o8nXfdOzZ08mT57Ma6+9BsDUqVPJysoiLy+PzZs3s2rVKlq1alXoMRYuXMgVV1xx+EanHj16HF733Xff8dBDD7Fz5052797NpZdeGjSeNWvW0KRJE04++WQA+vTpw9ixY7nrrrsA98EB0KZNG959992gx/rss88O184vrJxxZmYmV155JSkpKbRt25Z+/fpx4MABLr/8ckv0JmkEmw/Xfzx+YHmGUC32spwjoyjWog9Tz549mT17NkuXLmXPnj20adOG9evXM3r0aGbPns3y5cvp1q1bkeWJQ+nbty/PP/88K1as4JFHHinxcXx8pY5LU+bYyhkbc0RRffyhWuzxUJ7BEn2YatWqxQUXXEC/fv0OX4TNzc2lZs2aHH300WzZsuVw105ROnbsyPTp09m7dy+7du1i5syZh9ft2rWLhg0bcuDAgcOlhQFq167Nrl27ChzrlFNOYcOGDfzwww8ATJgwgU6dOpXovVk5Y2NKLlSLPR7KM1iiL4bevXuzbNmyw4neV9a3efPmXHfddbRv3z7o/meeeSbXXHMNZ5xxBl27dqVt27aH1z322GOcddZZtG/fPt+F02uvvZZRo0bRunVr1q5de3h5tWrVGDduHFdffTWnn346FSpUYMCAASV6X1bO2JiSC9Vij4fyDFam2ESF/c5MeRHYRw+uxR7uhEeRKrFsZYqNMSZKSjurXVnMc22J3hhjSqk0s9qVxTzXYSV6EekiImtE5AcRGVLI+ntEZJWILBeR2SKS6i1PF5EvRWSlt+6akgYab11Mpmj2uzImfGUxz3XIcfQiUhEYC1wM5ACLRGSGqq7y2+xbIENV94jIQOBJ4BpgD3Cjqv5HRI4HlojIx6pasPBLENWqVWP79u3UrVsXESnOrqaMqSrbt2+nWrVqsQ7FmIQRbAx/JIRzw1Q74AdVXQcgIpOBnsDhRK+qc/22/wq43lv+b79tfhaRrUB9oFiJPiUlhZycHLZt21ac3UyMVKtWjZSUlFiHYYzxhJPoGwE/+j3PAc4Ksv3NQIEB5SLSDqgCrC1kXX+gP8AJhVyBqFy5Mk2aNAkjVGOMMYEiejFWRK4HMoBRAcsbAhOAm1S1wIAhVc1S1QxVzahfv34kQzLGmHIvnBb9T0Bjv+cp3rJ8RKQzMBTopKq/+y0/CvgAGKqqX5UuXGOMMcUVTot+EdBMRJqISBXgWmCG/wYi0hp4Geihqlv9llcB/gmMV9V3Ihe2McaYcIV1Z6yIXAaMASoCr6vqSBEZASxW1RkiMgs4Hdjs7bJJVXt4XTnjgJV+h+urqkVWlBeRbUAhlZ8Pqwf8EjLo2LDYSsZiKxmLrWSSNbZUVS207zvuSiCEIiKLi7rNN9YstpKx2ErGYiuZ8hib3RlrjDFJzhK9McYkuURM9FmxDiAIi61kLLaSsdhKptzFlnB99MYYY4onEVv0xhhjisESvTHGJLmESfShSiXHmohsEJEVIpItIotD7xHVWF4Xka0i8p3fsmNF5FMR+Y/37zFxFNtwEfnJO3fZ3n0bZR1XYxGZ65XbXikid3rLY37egsQWD+etmoh8IyLLvNge9ZY3EZGvvb/XKd7Nk/ES2xsist7vvKWXdWx+MVYUkW9F5H3veXTOm6rG/Q/uRq21QFNcYbRlQItYxxUQ4wagXqzj8GLpCJwJfOe37ElgiPd4CPC3OIptODA4xuesIXCm97g28G+gRTyctyCxxcN5E6CW97gy8DVwNjAVuNZb/hIwMI5iewPoFcvz5hfjPcBbwPve86ict0Rp0R8ulayq+wFfqWRTCFVdAOwIWNwTeNN7/CZweZkG5SkitphT1c2qutR7vAv4Hle5NebnLUhsMafObu9pZe9HgQsBX9mTWJ23omKLCyKSAnQDXvWeC1E6b4mS6AsrlRwX/9H9KPCJiCzxyi7Hm+NU1Vei4r/AcbEMphCDvFnIXo9Vt5KPiKQBrXEtwLg6bwGxQRycN6/7IRvYCnyK+/a9U1XzvE1i9vcaGJuq+s7bSO+8PSMiVWMRG66szH2Ar6JvXaJ03hIl0SeC81T1TKArcLuIdIx1QEVR970wblo2wIvAiUA6rl7SU7EKRERqAdOAu1Q1139drM9bIbHFxXlT1YOqmo6rbNsOaB6LOAoTGJuItAQewMXYFjgWuL+s4xKR7sBWVV1SFq+XKIk+rFLJsaSqP3n/bsVV7GwX24gK2OLNC+CbH2BriO3LjKpu8f4gDwGvEKNzJyKVcYl0kqq+6y2Oi/NWWGzxct581E0ROhc4B6gjIr4y6DH/e/WLrYvXFabqyqmPIzbnrT3QQ0Q24LqiLwSeJUrnLVESfchSybEkIjVFpLbvMXAJ8F3wvcrcDKCP97gP8F4MY8nHl0g9VxCDc+f1j74GfK+qT/utivl5Kyq2ODlv9UWkjve4Om5u6e9xSbWXt1mszlthsa32++AWXB94mZ83VX1AVVNUNQ2Xz+aoaibROm+xvupcjKvTl+FGG6zFTWIS85j8YmuKGwm0DFeSOabxAf/AfZU/gOvnuxnX/zcb+A8wCzg2jmKbAKwAluMSa8MYxHUerltmOZDt/VwWD+ctSGzxcN5aAd96MXwHDPOWNwW+AX4A3gaqxlFsc7zz9h0wEW9kTqx+gPM5MuomKufNSiAYY0ySS5SuG2OMMSVkid4YY5KcJXpjjElyluiNMSbJWaI3xpgkZ4neGGOSnCV6Y4xJcv8fPl0u82vHKugAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CMsooMAwn34w"
      },
      "source": [
        "과대적합이 거의 일어나지 않았다. 성능에는 큰 차이가 없다. 이를 해결하기 위해 네트워크 용량을 늘려보자."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dAR3RdXcnmzO"
      },
      "source": [
        "### 3-5-2) 스태킹 순환층"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QFOKstgooHIb",
        "outputId": "765c7fe4-db49-4f6a-af52-816324785d0b"
      },
      "source": [
        "from keras.models import Sequential\r\n",
        "from keras import layers\r\n",
        "from keras.optimizers import RMSprop\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "model.add(layers.GRU(32,\r\n",
        "                     dropout=0.1,\r\n",
        "                     recurrent_dropout=0.5,\r\n",
        "                     return_sequences=True,  # 마지막 타임스텝만 출력하는 것이 아니라 전체 시퀀스(3D 텐서)를 출력\r\n",
        "                     input_shape=(None, float_data.shape[-1])))\r\n",
        "model.add(layers.GRU(64, activation='relu',\r\n",
        "                     dropout=0.1, \r\n",
        "                     recurrent_dropout=0.5))\r\n",
        "model.add(layers.Dense(1))\r\n",
        "\r\n",
        "model.compile(optimizer=RMSprop(), loss='mae')\r\n",
        "history = model.fit_generator(train_gen,\r\n",
        "                              steps_per_epoch=500,\r\n",
        "                              epochs=10,\r\n",
        "                              validation_data=val_gen,\r\n",
        "                              validation_steps=val_steps)"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.\n",
            "  warnings.warn('`Model.fit_generator` is deprecated and '\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "500/500 [==============================] - 381s 753ms/step - loss: 0.3421 - val_loss: 0.2840\n",
            "Epoch 2/10\n",
            "500/500 [==============================] - 376s 752ms/step - loss: 0.3039 - val_loss: 0.2701\n",
            "Epoch 3/10\n",
            "500/500 [==============================] - 383s 766ms/step - loss: 0.2947 - val_loss: 0.2796\n",
            "Epoch 4/10\n",
            "500/500 [==============================] - 386s 771ms/step - loss: 0.2876 - val_loss: 0.2675\n",
            "Epoch 5/10\n",
            "500/500 [==============================] - 387s 774ms/step - loss: 0.2803 - val_loss: 0.2689\n",
            "Epoch 6/10\n",
            "500/500 [==============================] - 387s 775ms/step - loss: 0.2755 - val_loss: 0.2665\n",
            "Epoch 7/10\n",
            "500/500 [==============================] - 388s 776ms/step - loss: 0.2717 - val_loss: 0.2658\n",
            "Epoch 8/10\n",
            "500/500 [==============================] - 386s 773ms/step - loss: 0.2654 - val_loss: 0.2719\n",
            "Epoch 9/10\n",
            "500/500 [==============================] - 386s 773ms/step - loss: 0.2634 - val_loss: 0.2886\n",
            "Epoch 10/10\n",
            "500/500 [==============================] - 387s 774ms/step - loss: 0.2597 - val_loss: 0.2797\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        },
        "id": "cgVmpWycoHLD",
        "outputId": "b5b424c9-5f61-4afe-8ca7-15dd31796cf6"
      },
      "source": [
        "loss = history.history['loss']\r\n",
        "val_loss = history.history['val_loss']\r\n",
        "\r\n",
        "epochs = range(1, len(loss) + 1)\r\n",
        "\r\n",
        "plt.figure()\r\n",
        "\r\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\r\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\r\n",
        "plt.title('Training and validation loss')\r\n",
        "plt.legend()"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f24ecce90f0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUVfr48c9DjSGAUkQlkMAuqPQSugURVxAWEBsYFdaCuGIBG4oufFFcFdZFfsquqCu6BhFRWVRYVAQBFaXI0hVEkGBDpBpayPP749zAJKZMkpm5k8nzfr3mNTPntmcm8My55557jqgqxhhjYlc5vwMwxhgTXpbojTEmxlmiN8aYGGeJ3hhjYpwlemOMiXGW6I0xJsZZojdFIiJzRWRQqNf1k4hsFZHuYdivisjvvdf/FJGHglm3GMdJFZH3ihtnAfvtKiLpod6vibwKfgdgwk9EDgS8jQcOA8e89zeralqw+1LVnuFYN9ap6tBQ7EdEkoFvgIqqmuntOw0I+m9oyh5L9GWAqiZkvxaRrcCNqvpB7vVEpEJ28jDGxA5ruinDsk/NReQ+EfkBeFFEThGRd0Rkp4js9l4nBmyzUERu9F4PFpElIjLBW/cbEelZzHUbiMgiEdkvIh+IyDMi8ko+cQcT48Mi8rG3v/dEpFbA8mtFZJuI7BKRUQV8Px1E5AcRKR9QdqmIrPZetxeRT0Vkj4h8LyJPi0ilfPY1VUQeCXh/j7fNdyJyfa51e4nIFyKyT0S2i8iYgMWLvOc9InJARDplf7cB23cWkWUistd77hzsd1MQETnb236PiKwTkT4Byy4RkfXePneIyN1eeS3v77NHRH4RkcUiYnknwuwLN6cBNYAkYAju38SL3vv6wEHg6QK27wB8CdQCngBeEBEpxrrTgM+BmsAY4NoCjhlMjFcDfwJOBSoB2YmnCfAPb/9neMdLJA+q+hnwK9At136nea+PAcO9z9MJuBD4cwFx48XQw4vnIqARkPv6wK/AdcDJQC/gFhHp5y07z3s+WVUTVPXTXPuuAbwLTPI+25PAuyJSM9dn+M13U0jMFYG3gfe87W4D0kTkTG+VF3DNgFWBZsCHXvldQDpQG6gDPADYuCsRZoneZAGjVfWwqh5U1V2q+oaqZqjqfmAccH4B229T1edU9RjwEnA67j900OuKSH2gHfAXVT2iqkuA2fkdMMgYX1TVr1T1IDADaOWVXw68o6qLVPUw8JD3HeTnVWAggIhUBS7xylDVFaq6VFUzVXUr8GweceTlSi++tar6K+6HLfDzLVTVNaqapaqrveMFs19wPwybVPXfXlyvAhuBPwask993U5COQALwmPc3+hB4B++7AY4CTUSkmqruVtWVAeWnA0mqelRVF6sNsBVxlujNTlU9lP1GROJF5FmvaWMfrqng5MDmi1x+yH6hqhney4QirnsG8EtAGcD2/AIOMsYfAl5nBMR0RuC+vUS7K79j4Wrv/UWkMtAfWKmq27w4GnvNEj94cTyKq90XJkcMwLZcn6+DiCzwmqb2AkOD3G/2vrflKtsG1A14n993U2jMqhr4oxi438twP4LbROQjEenklY8HNgPvicgWERkZ3McwoWSJ3uSuXd0FnAl0UNVqnGgqyK85JhS+B2qISHxAWb0C1i9JjN8H7ts7Zs38VlbV9biE1pOczTbgmoA2Ao28OB4oTgy45qdA03BnNPVUtTrwz4D9FlYb/g7XpBWoPrAjiLgK22+9XO3rx/erqstUtS+uWWcW7kwBVd2vqnepakOgDzBCRC4sYSymiCzRm9yq4tq893jtvaPDfUCvhrwcGCMilbza4B8L2KQkMc4EeovIOd6F07EU/v9gGnAH7gfl9Vxx7AMOiMhZwC1BxjADGCwiTbwfmtzxV8Wd4RwSkfa4H5hsO3FNTQ3z2fccoLGIXC0iFUTkKqAJrpmlJD7D1f7vFZGKItIV9zea7v3NUkWkuqoexX0nWQAi0ltEfu9di9mLu65RUFOZCQNL9Ca3icBJwM/AUuC/ETpuKu6C5i7gEeA1XH//vBQ7RlVdB9yKS97fA7txFwsLkt1G/qGq/hxQfjcuCe8HnvNiDiaGud5n+BDXrPFhrlX+DIwVkf3AX/Bqx962GbhrEh97PVk65tr3LqA37qxnF3Av0DtX3EWmqkdwib0n7nufDFynqhu9Va4FtnpNWENxf09wF5s/AA4AnwKTVXVBSWIxRSd2XcREIxF5DdioqmE/ozAm1lmN3kQFEWknIr8TkXJe98O+uLZeY0wJ2Z2xJlqcBryJuzCaDtyiql/4G5IxscGabowxJsZZ040xxsS4qGu6qVWrliYnJ/sdhjHGlCorVqz4WVVr57Us6hJ9cnIyy5cv9zsMY4wpVUQk9x3Rx1nTjTHGxDhL9MYYE+Ms0RtjTIwLqo3eu4HlKaA88LyqPpZr+VDcbeXHcLc6D1HV9SJyEfAYbszrI8A93vCmxpgocvToUdLT0zl06FDhKxtfxcXFkZiYSMWKFYPeptBE7w39+gxukoR0YJmIzPZG9cs2TVX/6a3fBzfZQQ/cmBh/VNXvRKQZMI+cw6UaY6JAeno6VatWJTk5mfznjTF+U1V27dpFeno6DRo0CHq7YJpu2gObVXWLN7DRdNzt6YEH3xfwtgreUKqq+oWqfueVrwNO8sb1Drm0NEhOhnLl3HOaTZVsTNAOHTpEzZo1LclHORGhZs2aRT7zCqbppi45J0lIx00JlzuAW4ERuGaabrmX4yYmWOnN6hNSaWkwZAhkeNNWbNvm3gOkpua/nTHmBEvypUNx/k4huxirqs+o6u+A+4AHA5eJSFPgceDmvLYVkSEislxElu/cubPIxx416kSSz5aR4cqNMaasCybR7yDnbDiJFDxbzXQgeyJjRCQReAs3dvXXeW2gqlNUNUVVU2rXzvPGrgJ9+23Ryo0x0WXXrl20atWKVq1acdppp1G3bt3j748cOVLgtsuXL+f2228v9BidO3cOSawLFy6kd+/eIdlXpAST6JcBjUSkgTcjzwByTdwsIo0C3vYCNnnlJ+NmpB+pqh+HJuTfqp97IrZCyo0xJRPqa2I1a9Zk1apVrFq1iqFDhzJ8+PDj7ytVqkRmZma+26akpDBp0qRCj/HJJ5+ULMhSrNBEr6qZwDBcj5kNwAxVXSciY70eNgDDRGSdiKzCtdMPyi4Hfg/8RURWeY9TQ/0hxo2D+PicZfHxrtwYE1rZ18S2bQPVE9fEQt0BYvDgwQwdOpQOHTpw77338vnnn9OpUydat25N586d+fLLL4GcNewxY8Zw/fXX07VrVxo2bJjjByAhIeH4+l27duXyyy/nrLPOIjU1lexRfOfMmcNZZ51F27Ztuf322wutuf/yyy/069ePFi1a0LFjR1avXg3ARx99dPyMpHXr1uzfv5/vv/+e8847j1atWtGsWTMWL14c2i+sIKoaVY+2bdtqcbzyimpSkqqIe37llWLtxpgyaf369UGvm5Sk6lJ8zkdSUmhiGT16tI4fP14HDRqkvXr10szMTFVV3bt3rx49elRVVd9//33t37+/qqouWLBAe/XqdXzbTp066aFDh3Tnzp1ao0YNPXLkiKqqVqlS5fj61apV0+3bt+uxY8e0Y8eOunjxYj148KAmJibqli1bVFV1wIABx/cbKPB4w4YN0zFjxqiq6vz587Vly5aqqtq7d29dsmSJqqru379fjx49qhMmTNBHHnlEVVUzMzN13759xf6O8vp7Acs1n7wadYOaFVdqqvWwMSYSInlN7IorrqB8+fIA7N27l0GDBrFp0yZEhKNHj+a5Ta9evahcuTKVK1fm1FNP5ccffyQxMTHHOu3btz9e1qpVK7Zu3UpCQgINGzY83j994MCBTJkypcD4lixZwhtvvAFAt27d2LVrF/v27aNLly6MGDGC1NRU+vfvT2JiIu3ateP666/n6NGj9OvXj1atWpXouykKGwLBGFMkkbwmVqVKleOvH3roIS644ALWrl3L22+/nW9f8sqVT9yqU758+Tzb94NZpyRGjhzJ888/z8GDB+nSpQsbN27kvPPOY9GiRdStW5fBgwfz8ssvh/SYBbFEb4wpEr+uie3du5e6dd2N9VOnTg35/s8880y2bNnC1q1bAXjttdcK3ebcc88lzbs4sXDhQmrVqkW1atX4+uuvad68Offddx/t2rVj48aNbNu2jTp16nDTTTdx4403snLlypB/hvxYojfGFElqKkyZAklJIOKep0wJf9Ppvffey/3330/r1q1DXgMHOOmkk5g8eTI9evSgbdu2VK1alerVqxe4zZgxY1ixYgUtWrRg5MiRvPTSSwBMnDiRZs2a0aJFCypWrEjPnj1ZuHAhLVu2pHXr1rz22mvccccdIf8M+Ym6OWNTUlLUJh4xJrI2bNjA2Wef7XcYvjtw4AAJCQmoKrfeeiuNGjVi+PDhfof1G3n9vURkhaqm5LW+1eiNMcbz3HPP0apVK5o2bcrevXu5+eY8b+YvdWKm140xxpTU8OHDo7IGX1JWozfGmBhnid4YY2KcJXpjjIlxluiNMSbGWaI3xvjuggsuYN68eTnKJk6cyC233JLvNl27diW7K/Yll1zCnj17frPOmDFjmDBhQoHHnjVrFuvXn5gZ9S9/+QsffPBBUcLPUzQNZ2yJ3hjju4EDBzJ9+vQcZdOnT2fgwIFBbT9nzhxOPvnkYh07d6IfO3Ys3bt3L9a+opUlemOM7y6//HLefffd45OMbN26le+++45zzz2XW265hZSUFJo2bcro0aPz3D45OZmff/4ZgHHjxtG4cWPOOeec40MZg+sj365dO1q2bMlll11GRkYGn3zyCbNnz+aee+6hVatWfP311wwePJiZM2cCMH/+fFq3bk3z5s25/vrrOXz48PHjjR49mjZt2tC8eXM2btxY4Ofzezhj60dvjMnhzjth1arQ7rNVK5g4Mf/lNWrUoH379sydO5e+ffsyffp0rrzySkSEcePGUaNGDY4dO8aFF17I6tWradGiRZ77WbFiBdOnT2fVqlVkZmbSpk0b2rZtC0D//v256aabAHjwwQd54YUXuO222+jTpw+9e/fm8ssvz7GvQ4cOMXjwYObPn0/jxo257rrr+Mc//sGdd94JQK1atVi5ciWTJ09mwoQJPP/88/l+vtGjR9O6dWtmzZrFhx9+yHXXXceqVauYMGECzzzzDF26dOHAgQPExcUxZcoULr74YkaNGsWxY8fIyD1PajFYjd4YExUCm28Cm21mzJhBmzZtaN26NevWrcvRzJLb4sWLufTSS4mPj6datWr06dPn+LK1a9dy7rnn0rx5c9LS0li3bl2B8Xz55Zc0aNCAxo0bAzBo0CAWLVp0fHn//v0BaNu27fGB0PKzZMkSrr32WiDv4YwnTZrEnj17qFChAu3atePFF19kzJgxrFmzhqpVqxa472BYjd4Yk0NBNe9w6tu3L8OHD2flypVkZGTQtm1bvvnmGyZMmMCyZcs45ZRTGDx4cL7DExdm8ODBzJo1i5YtWzJ16lQWLlxYonizhzouyTDHI0eOpFevXsyZM4cuXbowb96848MZv/vuuwwePJgRI0Zw3XXXlShWq9EbY6JCQkICF1xwAddff/3x2vy+ffuoUqUK1atX58cff2Tu3LkF7uO8885j1qxZHDx4kP379/P2228fX7Z//35OP/10jh49enxoYYCqVauyf//+3+zrzDPPZOvWrWzevBmAf//735x//vnF+mx+D2dsNXpjTNQYOHAgl1566fEmnOxhfc866yzq1atHly5dCty+TZs2XHXVVbRs2ZJTTz2Vdu3aHV/28MMP06FDB2rXrk2HDh2OJ/cBAwZw0003MWnSpOMXYQHi4uJ48cUXueKKK8jMzKRdu3YMHTq0WJ8rey7bFi1aEB8fn2M44wULFlCuXDmaNm1Kz549mT59OuPHj6dixYokJCSEZIISG6bYGGPDFJcyNkyxMcaYHCzRG2NMjLNEb4wBINqacU3eivN3CirRi0gPEflSRDaLyMg8lg8VkTUiskpElohIE6+8pogsEJEDIvJ0kaMzxkREXFwcu3btsmQf5VSVXbt2ERcXV6TtCu11IyLlgWeAi4B0YJmIzFbVwLsWpqnqP731+wBPAj2AQ8BDQDPvYYyJQomJiaSnp7Nz506/QzGFiIuLIzExsUjbBNO9sj2wWVW3AIjIdKAvcDzRq+q+gPWrAOqV/wosEZHfFykqY0xEVaxYkQYNGvgdhgmTYBJ9XWB7wPt0oEPulUTkVmAEUAnoVpQgRGQIMASgfv36RdnUGGNMIUJ2MVZVn1HV3wH3AQ8Wcdspqpqiqim1a9cOVUjGGGMILtHvAOoFvE/0yvIzHehXkqCMMcaETjCJfhnQSEQaiEglYAAwO3AFEWkU8LYXsCl0IRpjjCmJQtvoVTVTRIYB84DywL9UdZ2IjAWWq+psYJiIdAeOAruBQdnbi8hWoBpQSUT6AX/I1WPHGGNMGNlYN8YYEwNsrBtjjCnDLNEbY0yMs0RvjDExzhK9McbEOEv0xhgT4yzRG2NMjLNEb4wxMc4SvTHGxDhL9MYYE+Ms0RtjTIyzRG+MMTHOEr0xxsQ4S/TGGBPjLNEbY0yMs0RvjDExzhK9McbEOEv0xhgT4yzRh1BaGiQnQ7ly7jktze+IjDEmiDljTXDS0mDIEMjIcO+3bXPvAVJT/YvLGGOsRh8io0adSPLZMjJcuTHG+MkSfYh8+23Ryo0xJlIs0YdI/fpFKzfGmEixRB8i48ZBfHzOsvh4V26MMX4KKtGLSA8R+VJENovIyDyWDxWRNSKySkSWiEiTgGX3e9t9KSIXhzL4aJKaClOmQFISiLjnKVPsQqwxxn+iqgWvIFIe+Aq4CEgHlgEDVXV9wDrVVHWf97oP8GdV7eEl/FeB9sAZwAdAY1U9lt/xUlJSdPny5SX7VMYYU8aIyApVTclrWTA1+vbAZlXdoqpHgOlA38AVspO8pwqQ/evRF5iuqodV9Rtgs7c/Y4wxERJMP/q6wPaA9+lAh9wricitwAigEtAtYNulubatm8e2Q4AhAPXt6qUxxoRUyC7Gquozqvo74D7gwSJuO0VVU1Q1pXbt2qEKyRhjDMEl+h1AvYD3iV5ZfqYD/Yq5rTHGmBALJtEvAxqJSAMRqQQMAGYHriAijQLe9gI2ea9nAwNEpLKINAAaAZ+XPGxjjDHBKrSNXlUzRWQYMA8oD/xLVdeJyFhguarOBoaJSHfgKLAbGORtu05EZgDrgUzg1oJ63BhjjAm9QrtXRpp1rzTGmKIrafdKY4wxpZglemOMiXGW6I0xJsZZojfGmBhnid4YY2KcJXpjjIlxluiNMSbGWaI3xpgYZ4neGGNinCV6Y4yJcZbojTEmxlmiN8aYGGeJPgalpUFyMpQr557T0vyOyBjjp2CmEjSlSFoaDBkCGRnu/bZt7j1Aaqp/cRlj/GM1+hgzatSJJJ8tI8OVG2PKJkv0Mebbb4tWboyJfZboY0z9+kUrN8bEPkv0MWbcOIiPz1kWH+/KjTFlkyX6GJOaClOmQFISiLjnKVPsQqwxZZn1uolBqamW2I0xJ1iN3hhjYpwlemOMiXGW6I0xJsYFlehFpIeIfCkim0VkZB7LR4jIehFZLSLzRSQpYNnjIrLWe1wVyuCNMcYUrtBELyLlgWeAnkATYKCINMm12hdAiqq2AGYCT3jb9gLaAK2ADsDdIlItdOEbY4wpTDA1+vbAZlXdoqpHgOlA38AVVHWBqmbfeL8USPReNwEWqWqmqv4KrAZ6hCZ0Y4wxwQgm0dcFtge8T/fK8nMDMNd7/T+gh4jEi0gt4AKgXu4NRGSIiCwXkeU7d+4MLnJjjDFBCWk/ehG5BkgBzgdQ1fdEpB3wCbAT+BQ4lns7VZ0CTAFISUnRUMZkjDFlXTA1+h3krIUnemU5iEh3YBTQR1UPZ5er6jhVbaWqFwECfFWykI0xxhRFMIl+GdBIRBqISCVgADA7cAURaQ08i0vyPwWUlxeRmt7rFkAL4L1QBW+im02AYkqD3bth6VK/owivQhO9qmYCw4B5wAZghqquE5GxItLHW208kAC8LiKrRCT7h6AisFhE1uOaZq7x9mdiXPYEKNu2geqJCVAs2ZtokpUFl14KnTq5ORuysvyOKDxENbqaxFNSUnT58uV+h2FKKDnZJffckpJg69ZIR2NM3p5/Hm66CTp0gM8+g8sug5degipV/I6s6ERkhaqm5LXM7ow1YWEToJho9/33cPfdcP758Mkn8OST8OabcN55sOM3VyFLN0v0JixsAhQT7W6/HQ4dcsN4lysHw4fD22/DV19Bu3YQSw0LluhNWNgEKCaa/ec/MHMm/OUv0LjxifJevVztvlIlV7OfOdO/GEPJEr0JC5sAxUSrvXvhz3+G5s3hnnt+u7x5c/j8c2jdGq64Ah55xHUoKM1s4hETNjYBiolG99/v2uffegsqVsx7nVNPhfnz3YXahx6CDRvghRcgLi6ysYaK1eiNMWXGxx/DP/4Bd9wB7dsXvG5cHLz8smtunDYNunaFH36ISJghZ4neGFMmHD4MN97omhEffji4bUTggQfgjTdgzRr347B6dXjjDAdL9MaYMuGvf4WNG12NPiGhaNv27w+LF7sbqjp3htmzC98mmliiN8bEvPXr4dFH4eqroWfP4u2jTRt3kfbss6FfPxg/vvRcpLVEb4yJaVlZ7qJqtWowcWLJ9nXGGfDRR3D55XDvvXDDDXDkSGjiDCfrdWOMiWn//KfrG//SS1C7dsn3Fx8P06e7mv3YsfD1164Nv1atku87XKxGb4yJWenpMHIkXHQRXHtt6PZbrhz83/+53jiffebGylm/PnT7D7WYSfS7drkBiVat8jsSY0w0UHU3RmVmulq9SOiPMXAgLFwIv/7qRsCcNy/0xwiFmEn0WVnul/Xyy2HPHr+jMcb4beZMN3bNww9Dw4bhO07Hju4ibYMGcMkl8PTT4TtWccVMoq9dG157zQ2N+6c/lZ6r4caY0Nu9G267Ddq2dTdHhVv9+rBkCfTu7Y57661w9Gj4jxusmEn0AF26wBNPwKxZbshRY0zZdM898PPP8NxzUCFCXU4SEtwwx/fcA5Mnu9r97t2ROXZhYirRA9x5p7u54b773C+sMaZsWbDAjUtz111uYLJIKl/eVTb/9S/XDbNTJ9i0KbIx5CUmZ5jauxdSUtwFki++gDp1QhScMSaqHTwILVq4a3Zr1vx2qOxIWrTIVTqzslxNv2vX8B6vzM0wVb26uxCze7e7E+7YMb8jMsZEwsMPw+bNbkhsP5M8uPHsP/8cTjvNde98/nn/YonJRA/QsqVrJ/vwQxgzxu9ojDHh9r//uWaTwYPhwgv9jsZp2BA+/dTFc9NNMGKEPxXPmE304HrfXH+9mzhg7ly/ozHGhMuxY25kypo1YcIEv6PJqXp1eOcd1xvn73+Hvn1h377IxhDTiR5cn9YWLeCaa1zXS2NM7Jk0yc3x+tRTLtlHmwoVXIyTJ8N//+t6CG7dGrnjx3yiP+kk116fmQlXXunGpDbGxI6tW+HBB918r1dd5Xc0BbvlFpfo09Pd2PYffxyZ4waV6EWkh4h8KSKbRWRkHstHiMh6EVktIvNFJClg2RMisk5ENojIJJFw3IhcsEaN4MUX3YWRu++O9NGNMeGiCkOHurFnJk8OzzAHoda9Oyxd6pp0unWDf/87/McsNNGLSHngGaAn0AQYKCJNcq32BZCiqi2AmcAT3radgS5AC6AZ0A44P2TRF0H//q5f7dNPu5HnjDGl37RpbnyZRx91d6eWFmee6YZs6dIFrrvOzWKVlRW+4wVTo28PbFbVLap6BJgO9A1cQVUXqGqG93YpkJi9CIgDKgGVgYrAj6EIvDj++lf3xd54o5vs1xhTev38s7tBskMHN3hZaVOjhvuRuukml5uuuMLd+xMOwST6usD2gPfpXll+bgDmAqjqp8AC4HvvMU9Vf5NiRWSIiCwXkeU7d+4MNvYiq1jRjYcTH+8GPwvXlxoN1q93V/gzM/2OxJjwGDHCDWD4/PPujtTSqGJFePZZ93911izXrBOO7pchvRgrItcAKcB47/3vgbNxNfy6QDcROTf3dqo6RVVTVDWldihmBihA3brw6quuRn/zzbE5+NnChe7W6xEjXJeuWPyMRZGWBsnJrh03Odm9N6XbvHmubXvkSGjWzO9oSkbEnZnMnu3OTMLxoxVMot8B1At4n+iV5SAi3YFRQB9Vze7bcimwVFUPqOoBXE2/U8lCLrkLL3Qzw6SluTvoYslrr8HFF7sftKFD3Tjcjz3md1T+SUuDIUNc11pV9zxkiCX70uzXX92/7TPPhFGj/I4mdHr1Cu3kKDmoaoEP3HSDW4AGuLb2/wFNc63TGvgaaJSr/CrgA28fFYH5wB8LOl7btm01Eo4dU+3ZU7VSJdXlyyNyyLD7299UQfWcc1R37XKfMTXVlb38st/R+SMpyX3+3I+kJL8jM8V1113ub7hokd+RRBdgueaXx/NboDkT9iXAV14yH+WVjcXV3vGS+Y/AKu8x2ysvDzwLbADWA08WdqxIJXpV1Z9/Vq1XTzU5WfWXXyJ22JA7dkx1+HD317zsMtWDB08sO3xYtVs31QoVVN97z78Y/SKSd6IX8TsyUxzLlqmWK6d6881+RxJ9Ckr0MTl6ZVF89hmce65r7vjPf1w7bmly+LDrnjVjxolbrHO38e3d6z7j1q1uRL1WrXwJ1RfJyXnfEZ2UFNk7E03JHT3qbjL68Ud3ja16db8jii5lbvTKoujQwU1S8s47MH6839EUzZ497gdqxgw3mNNTT+V9Iad6dZgzxz1fcgl8+23kY/XLuHG/HcUwPt6Vm9LlySfdnNBPP21Jvsjyq+r79Yhk0022rCzVq65yp4QLFkT88MWyfbtqs2aqFSuqpqUFt82aNarVq6uefXbpbqoqqldecW3yIu75lVf8jsgU1aZNqnFxqpde6nck0Qtruinc/v3Qrp2rJX/xBZx+esRDCNratdCzp2uSeeutog3JunChOwvo2NF1UYuLC1uYxoSEqutfvny5a7I54wy/I4pO1nQThBXVSu8AABBySURBVKpV4Y03XMIfMCB6bzT66CM45xx3U8XixUUfd7trV3jpJddWP2hQeG+7NiYUpk5180o88YQl+eKyRB+gaVPX73zRIjcaXrSZMQP+8Af3j/3TT93kKsUxYIC7HjFjhpvI2Jho9eOPboyqc891QwWY4rFEn8u117o7Zh9/3N2pFi3+/nc3BGu7dm7S86SkwrcpyF13we23uwtcEyeGJkZjQu2OO9wNUlOmlL4ecdHEvro8TJwIbdq4po0tW/yNJSvLJeURI9wInO+/7wZDKikRl+T793f7fv31ku/TmFB65x13p/eDD8JZZ/kdTelmiT4PcXFushJwI8odOuRPHIcPQ2qqS8jDhrmmlpNOCt3+y5eHV16Bzp3dmczixaHbtzElsX+/m6SjaVO47z6/oyn9LNHno0EDd9Fy5Uo34FCk7dkDPXq4sfMff9xNQxaOwY5OOsndKJacDH36uFEvjfHbAw/Ajh1uZMpKlfyOpvSzRF+APn1cbeLZZyMzC0y29HR38enjj12N+957wztzTs2abnqzuDjXbfO778J3LGMK8+mn8Mwz7iy2Y0e/o4kNlugL8cgjcN55brS8tWvDf7x169wQw9u2ubtZU1PDf0xwNfp334VffnF3z0Z6lvqywIZLLtyRI653TWKi3b0cSpboC1Ghgms+qVrVTVayf3/4jrVokesjn5npXnfvHr5j5aVNG3dtYu1a91mPHIns8WOZDZccnMcfd5WdyZPd/zkTGpbog3D66S7Zb9rkpiEMx83Er78OF10Ep53mTl39Gnjs4otdu+j777uaVZTdOF1qjRoFGRk5yzIyYms89ZLasMGdQV91FfTu7Xc0scUSfZC6dnWnkjNmuPbDUHrqqRN95D/+2J3W+2nwYDcxy8svw0MP+RtLrMhvILmyNMBcQbKy3BlOlSru/4MJLUv0RXDvva6mMWKEG964pLKy3J2pd94J/fqFro98KDz4oKvRjxvnLkabkqlfv2jlZc2UKe5GwL/9DerU8Tua2GOJvgjKlXNdLuvWdf3rd+0q/r6y+8hPmAC33uqabkLZR76kRFw7aa9ebh7Lt9/2O6LSzYZLzt+OHa53W7du7mzShJ4l+iKqUcMl5R9/hGuuKd6gYHv3um6M06e7+Vz/3/+LzlnsK1Rwdya2aeOalkJxFlNWpaa6WmtSkvsRTUpy7yPVqyqaDRvmLvw/+2x4uxGXZZboiyElxbUj/ve/8OijRdt2xw7XR37xYtcGft990f2Pu0oV1+3y9NNds9XmzX5HVHqlprpZrbKy3LMleXjzTZg1C8aMgd//3u9oYpeNR19Mqm7YgGnT4L33gusKuX69u9t19273D/yii8IfZ6hs2uT69598MnzyCZx6qt8RmdLup59c77I6deDzz6FiRb8jKt1sPPowEHFDGp99Nlx9taupF2TxYujSxc17uWhR6UryAI0auUGmvvvO1ex//dXviExplZnpmivPOgt+/hmee86SfLhZoi+BhAR3g1FGhmvDPno07/VmznSJvU4d10e+devIxhkqHTu66worVkT35Cwmes2f72rxt98Obdu62dxS8qyDmlCyRF9CZ5/tbjD6+GO4//7fLp80Ca680v2jjoY+8iXVp4+bnPmdd1xvoShr+TNB8GMohm++cUNid+/uKkZvveWaPJs2Df+xDcFNDg70AL4ENgMj81g+AlgPrAbmA0le+QXAqoDHIaBfQcfyY3LwUBg2TBVU33jDvT92TPWee1xZv36qGRn+xhdq99/vPtsjj/gdiSmKV15RjY93f7vsR3x8+CZMP3BAddQo1cqV3XEeeUT14MHwHKuso4DJwYNJ8uWBr4GGQCXgf0CTXOtcAMR7r28BXstjPzWAX7LXy+9RWhP9oUOq7durVqumum6d6tVXu2/3z39Wzcz0O7rQy8pSveYa9xmnTvU7GhOspKScST77kZQU2uNkZalOm6Zat67b/9VXq27fHtpjmJwKSvTBNN20Bzar6hZVPQJMB/rmOitYoKrZI3ksBRLz2M/lwNyA9WJK5cpueIQKFVwb5LRp8Ne/umaOaOwjX1Ii8MILbnLyG290p+Em+kViKIaVK10X4quvdtellixxzUOJeWUFExHBJPq6wPaA9+leWX5uAObmUT4AeDX40EqfpCT3D7pGDXcH7ciR0d1HvqQqVYI33oAmTeCyy2DVKr8jMoUJ51AMP/3khs1ISYGvvnLXrj7/3PU2M/4K6cVYEbkGSAHG5yo/HWgOzMtnuyEislxElu/cuTOUIUVcjx7w/fdw3XV+RxIZ1avD3Llwyinubt9t2/yOyBQkHEMxHD3qJq9v3BimTnVjN331FdxwQ2yezZZGwST6HUC9gPeJXlkOItIdGAX0UdXDuRZfCbylqnl2QFTVKaqaoqoptWvXDi7yKBbLtfi8nHGGS/aHDrlk/8svfkdk8hPqoRjmzYMWLdxAfx07wurVbo7jk08ObdymZAq9M1ZEKgBfARfiEvwy4GpVXRewTmtgJtBDVTflsY+lwP2quqCwgErLnbHmtz76CP7wB+jQwbXZx8X5E8eePbBli+vSl/38zTfu7KNzZ9eU0LKlu55iimfzZpfc334bfvc7V6Pv3bvsVXKiSUF3xhb6T11VM0VkGK7ZpTzwL1VdJyJjcVd5Z+OaahKA18X9pb9V1T7ewZNxZwQfheCzmCh2/vlubt2rrnLDQ7z2muurHWqHD7smoryS+ZYtLtEHOuUUN9n7unUuJnBj+HTo4JJ+ly6uNlq9euhjjTX797vxnZ580l2jeewx11RTubLfkZmC2Fg3JuSefBLuusslgL//vejbZ2W56xyBSTwwme/YkfNGrcqV3Y0/DRu6hJ77OTCBb9/ublzLfvzvf+54ItC8+YnE36XLieYN476jtDQ3CF/2NajHHnOD3ZnoUFCN3hK9CYvhw2HiRJf0hw//7fI9e3LWwgOft251tfZsIm4OgLySeMOGbvrF4p457N/vhl/OTvyffgoHDrhlZ5yRM/G3alU2m3uWLXNDFixd6mZBmzTJnQGZ6GKJ3kRcVpZrwpk5Ex54wPXMCEzmu3fnXD+7eSWvZJ6UFLmmgWPHYM2aE4l/yRJ3FgCud0pgc0+nTrHd3PPDD25Yj6lTXX/4xx5zNflwNMeZkrNEb3xx6JDrbvrRR649NzuR55XMo7mXRkHNPc2a5az1JyeX/uaeI0dcrX3sWPc3vPNON7VktWp+R2YKYone+CYry83GVadO7NQEczf3LF3qysA192T37Mlu7ilNQ/DOmeOa2r76yk0j+eSTrn+8iX4l6nVjTEmUKxd7F+yqVnWjMGZPNpO7uefjj12TFbjmnvbtXdI/55zobe756iuX4OfMcYl9zhx3T4SJDVajNyYM0tN/29xz7Jhr1jnllOI9qlUL/VnRvn3w8MNuasy4OBg9Gm67zTW1mdLFmm6M8dmBA665Z+lS1z1x9253B/Hu3TkfBU3mUq6cOxso7AehRo3CfySystxF1vvvh5074U9/cv3j69QJ+1dhwsSabozxWUKCG+nzwgvzX0fVTdGYO/kX9Ni+/cTr/GY4A3cmcfLJJxL/r7/Cxo2uKendd22Wp1hnid6YKCHifhASEqBevcLXD6TqZm4K9gfi4EEYNcqNcVPaewmZwlmiNyYGiLhhHapUsXHfzW/FSIc3Y0xp4se8tWWZ1eiNMRGVlgZDhrimJnAD1A0Z4l4Xd7hkUzCr0RtjImrUqBNJPltGhis34WGJ3hgTUZGYt9bkZIneGBNR4Zy3tqjKyrUCS/TGmIgKx7y1xZF9rWDbNtc9NftaQSwme0v0xpiICvW8tcVVlq4V2BAIxpgyqVy5nDOVZRNxQ0SUNgUNgWA1emNMmRRN1wrCzRK9MaZMipZrBZFgid4YUyZFy7WCSLA7Y40xZVZqamwm9tysRm+MMTEuqEQvIj1E5EsR2SwiI/NYPkJE1ovIahGZLyJJAcvqi8h7IrLBWyc5dOEbY4wpTKGJXkTKA88APYEmwEARaZJrtS+AFFVtAcwEnghY9jIwXlXPBtoDP4UicGOMMcEJpkbfHtisqltU9QgwHegbuIKqLlDV7FsPlgKJAN4PQgVVfd9b70DAesYYYyIgmERfF9ge8D7dK8vPDcBc73VjYI+IvCkiX4jIeO8MIQcRGSIiy0Vk+c6dO4ON3RhjTBBCejFWRK4BUoDxXlEF4FzgbqAd0BAYnHs7VZ2iqimqmlK7du1QhmSMMWVeMIl+BxA4g2WiV5aDiHQHRgF9VPWwV5wOrPKafTKBWUCbkoVsjDGmKIJJ9MuARiLSQEQqAQOA2YEriEhr4Flckv8p17Yni0h2Nb0bsL7kYRtjjAlWoYneq4kPA+YBG4AZqrpORMaKSB9vtfFAAvC6iKwSkdnetsdwzTbzRWQNIMBzYfgcxhhj8mGjVxpjTAyw0SuNMSaKhXumKxvrxhhjfJQ901X2JCjZM11B6MbhsRq9Mcb4KBIzXVmiN8YYH337bdHKi8MSvTHG+CgSM11ZojfGGB9FYqYrS/TGGOOjSMx0Zb1ujDHGZ+Ge6cpq9MYYE+Ms0RtjTIyzRG+MMTHOEr0xxsQ4S/TGGBPjom70ShHZCWzzO44SqgX87HcQUcS+j5zs+zjBvoucSvJ9JKlqnlP0RV2ijwUisjy/4ULLIvs+crLv4wT7LnIK1/dhTTfGGBPjLNEbY0yMs0QfHlP8DiDK2PeRk30fJ9h3kVNYvg9rozfGmBhnNXpjjIlxluiNMSbGWaIPIRGpJyILRGS9iKwTkTv8jslvIlJeRL4QkXf8jsVvInKyiMwUkY0iskFEOvkdk59EZLj3/2StiLwqInF+xxRJIvIvEflJRNYGlNUQkfdFZJP3fEoojmWJPrQygbtUtQnQEbhVRJr4HJPf7gA2+B1ElHgK+K+qngW0pAx/LyJSF7gdSFHVZkB5YIC/UUXcVKBHrrKRwHxVbQTM996XmCX6EFLV71V1pfd6P+4/cl1/o/KPiCQCvYDn/Y7FbyJSHTgPeAFAVY+o6h5/o/JdBeAkEakAxAPf+RxPRKnqIuCXXMV9gZe81y8B/UJxLEv0YSIiyUBr4DN/I/HVROBeIMvvQKJAA2An8KLXlPW8iFTxOyi/qOoOYALwLfA9sFdV3/M3qqhQR1W/917/ANQJxU4t0YeBiCQAbwB3quo+v+Pxg4j0Bn5S1RV+xxIlKgBtgH+oamvgV0J0Wl4aeW3PfXE/gGcAVUTkGn+jii7q+r6HpP+7JfoQE5GKuCSfpqpv+h2Pj7oAfURkKzAd6CYir/gbkq/SgXRVzT7Dm4lL/GVVd+AbVd2pqkeBN4HOPscUDX4UkdMBvOefQrFTS/QhJCKCa4PdoKpP+h2Pn1T1flVNVNVk3EW2D1W1zNbYVPUHYLuInOkVXQis9zEkv30LdBSReO//zYWU4YvTAWYDg7zXg4D/hGKnluhDqwtwLa72usp7XOJ3UCZq3AakichqoBXwqM/x+MY7s5kJrATW4HJRmRoOQUReBT4FzhSRdBG5AXgMuEhENuHOeh4LybFsCARjjIltVqM3xpgYZ4neGGNinCV6Y4yJcZbojTEmxlmiN8aYGGeJ3hhjYpwlemOMiXH/H4Q+oUTQDKicAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hP0puyepp8Ga"
      },
      "source": [
        "성능이 미미하게 향상됐다. 충분히 과대적합되도록 층의 크기를 늘릴 수도 있다. 하지만 이만큼 층을 추가했는데 성능의 큰 향상은 없는 걸로 보아, 더이상 네트워크의 용량을 늘리는 것은 도움이 안된다고 볼 수 있다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RvLCwIUXnmvY"
      },
      "source": [
        "### 3-5-3) 양방향 RNN 사용"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WbggbqKSuDC6"
      },
      "source": [
        "# 최근 데이터가 먼저 나오도록 입력 시퀀스를 처리\r\n",
        "\r\n",
        "def reverse_order_generator(data, lookback, delay, min_index, max_index,\r\n",
        "                            shuffle=False, batch_size=128, step=6):\r\n",
        "    if max_index is None:\r\n",
        "        max_index = len(data) - delay - 1\r\n",
        "    i = min_index + lookback\r\n",
        "    while 1:\r\n",
        "        if shuffle:\r\n",
        "            rows = np.random.randint(\r\n",
        "                min_index + lookback, max_index, size=batch_size)\r\n",
        "        else:\r\n",
        "            if i + batch_size >= max_index:\r\n",
        "                i = min_index + lookback\r\n",
        "            rows = np.arange(i, min(i + batch_size, max_index))\r\n",
        "            i += len(rows)\r\n",
        "\r\n",
        "        samples = np.zeros((len(rows),\r\n",
        "                           lookback // step,\r\n",
        "                           data.shape[-1]))\r\n",
        "        targets = np.zeros((len(rows),))\r\n",
        "        for j, row in enumerate(rows):\r\n",
        "            indices = range(rows[j] - lookback, rows[j], step)\r\n",
        "            samples[j] = data[indices]\r\n",
        "            targets[j] = data[rows[j] + delay][1]\r\n",
        "        yield samples[:, ::-1, :], targets\r\n",
        "        \r\n",
        "train_gen_reverse = reverse_order_generator(\r\n",
        "    float_data,\r\n",
        "    lookback=lookback,\r\n",
        "    delay=delay,\r\n",
        "    min_index=0,\r\n",
        "    max_index=200000,\r\n",
        "    shuffle=True,\r\n",
        "    step=step, \r\n",
        "    batch_size=batch_size)\r\n",
        "val_gen_reverse = reverse_order_generator(\r\n",
        "    float_data,\r\n",
        "    lookback=lookback,\r\n",
        "    delay=delay,\r\n",
        "    min_index=200001,\r\n",
        "    max_index=300000,\r\n",
        "    step=step,\r\n",
        "    batch_size=batch_size)"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i43Cyw53uMtu",
        "outputId": "36ba3749-127c-4065-e206-ad3fa79181e3"
      },
      "source": [
        "model = Sequential()\r\n",
        "model.add(layers.GRU(32, input_shape=(None, float_data.shape[-1])))\r\n",
        "model.add(layers.Dense(1))\r\n",
        "\r\n",
        "model.compile(optimizer=RMSprop(), loss='mae')\r\n",
        "history = model.fit_generator(train_gen_reverse,\r\n",
        "                              steps_per_epoch=500,\r\n",
        "                              epochs=20,\r\n",
        "                              validation_data=val_gen_reverse,\r\n",
        "                              validation_steps=val_steps)"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.\n",
            "  warnings.warn('`Model.fit_generator` is deprecated and '\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "500/500 [==============================] - 84s 164ms/step - loss: 0.5002 - val_loss: 0.4778\n",
            "Epoch 2/20\n",
            "500/500 [==============================] - 83s 167ms/step - loss: 0.4501 - val_loss: 0.4660\n",
            "Epoch 3/20\n",
            "500/500 [==============================] - 82s 165ms/step - loss: 0.4117 - val_loss: 0.4639\n",
            "Epoch 4/20\n",
            "500/500 [==============================] - 82s 164ms/step - loss: 0.3694 - val_loss: 0.4450\n",
            "Epoch 5/20\n",
            "500/500 [==============================] - 82s 163ms/step - loss: 0.3440 - val_loss: 0.4450\n",
            "Epoch 6/20\n",
            "500/500 [==============================] - 84s 169ms/step - loss: 0.3239 - val_loss: 0.4443\n",
            "Epoch 7/20\n",
            "500/500 [==============================] - 83s 166ms/step - loss: 0.3117 - val_loss: 0.4394\n",
            "Epoch 8/20\n",
            "500/500 [==============================] - 84s 169ms/step - loss: 0.2970 - val_loss: 0.4357\n",
            "Epoch 9/20\n",
            "500/500 [==============================] - 82s 164ms/step - loss: 0.2833 - val_loss: 0.4281\n",
            "Epoch 10/20\n",
            "500/500 [==============================] - 82s 164ms/step - loss: 0.2757 - val_loss: 0.4275\n",
            "Epoch 11/20\n",
            "500/500 [==============================] - 84s 168ms/step - loss: 0.2678 - val_loss: 0.4385\n",
            "Epoch 12/20\n",
            "500/500 [==============================] - 84s 168ms/step - loss: 0.2606 - val_loss: 0.4381\n",
            "Epoch 13/20\n",
            "500/500 [==============================] - 83s 166ms/step - loss: 0.2577 - val_loss: 0.4339\n",
            "Epoch 14/20\n",
            "500/500 [==============================] - 83s 167ms/step - loss: 0.2496 - val_loss: 0.4365\n",
            "Epoch 15/20\n",
            "500/500 [==============================] - 84s 168ms/step - loss: 0.2480 - val_loss: 0.4402\n",
            "Epoch 16/20\n",
            "500/500 [==============================] - 84s 168ms/step - loss: 0.2420 - val_loss: 0.4579\n",
            "Epoch 17/20\n",
            "500/500 [==============================] - 83s 166ms/step - loss: 0.2388 - val_loss: 0.4454\n",
            "Epoch 18/20\n",
            "500/500 [==============================] - 85s 171ms/step - loss: 0.2352 - val_loss: 0.4425\n",
            "Epoch 19/20\n",
            "500/500 [==============================] - 83s 167ms/step - loss: 0.2305 - val_loss: 0.4453\n",
            "Epoch 20/20\n",
            "500/500 [==============================] - 83s 165ms/step - loss: 0.2275 - val_loss: 0.4477\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        },
        "id": "SZ4FjfghuMw1",
        "outputId": "f3113b25-d08b-45f0-a191-620bdabac975"
      },
      "source": [
        "loss = history.history['loss']\r\n",
        "val_loss = history.history['val_loss']\r\n",
        "\r\n",
        "epochs = range(1, len(loss) + 1)\r\n",
        "\r\n",
        "plt.figure()\r\n",
        "\r\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\r\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\r\n",
        "plt.title('Training and validation loss')"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Training and validation loss')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de7hcdX3v8fcngUADkQCJAiE3IIBRrmeDoKBYUQNiQGktEgGtbQ5FWq1ajcYLDxgreo76aFGJFgt2Y8BLbY6KXCo3FTA73EKAwE5IQmLACAECCeT2PX/81rgnw8zs2ZnrXvvzep55ZmZdZr57ZfKZNb/1W7+liMDMzPJrWLsLMDOz5nLQm5nlnIPezCznHPRmZjnnoDczyzkHvZlZzjnobUAkXSfpvEYv206Slks6uQmvG5IOyh5/R9Jna1l2B95nhqQbdrTOKq97kqRVjX5da72d2l2ANZ+k54uejgReArZmz/93RHTX+loRcUozls27iDi/Ea8jaRLwGLBzRGzJXrsbqPnf0IYeB/0QEBG7Fx5LWg78XUTcVLqcpJ0K4WFm+eGmmyGs8NNc0iclPQF8X9Kekn4uaa2kddnj/YvWuUXS32WP3y/pN5L+T7bsY5JO2cFlJ0u6TdJ6STdJukzSf1aou5YaL5H02+z1bpA0pmj+OZJWSHpK0uwq2+d1kp6QNLxo2rsk3Z89PlbSHZKekbRG0r9JGlHhtf5D0heKnv9Lts4fJP1tybLvkHSPpOckPS7poqLZt2X3z0h6XtLxhW1btP7rJS2Q9Gx2//pat001kl6drf+MpMWSphfNO1XSg9lrrpb08Wz6mOzf5xlJT0u6XZJzp8W8wW0fYC9gIjCT9Jn4fvZ8ArAR+Lcq678OWAKMAb4M/Lsk7cCyVwO/B/YGLgLOqfKetdR4NvAB4JXACKAQPFOBb2evv1/2fvtTRkTcBbwA/GXJ616dPd4K/HP29xwPvAW4oErdZDVMy+p5KzAFKD0+8AJwLjAaeAfwD5LOyOa9MbsfHRG7R8QdJa+9F/AL4BvZ3/ZV4BeS9i75G162bfqpeWfg/wE3ZOv9I9At6ZBskX8nNQOOAl4L/Dqb/jFgFTAWeBXwacDjrrSYg962AZ+PiJciYmNEPBURP4mIDRGxHpgDvKnK+isi4rsRsRW4EtiX9B+65mUlTQCOAT4XEZsi4jfA/EpvWGON34+IRyJiI3AtcGQ2/a+An0fEbRHxEvDZbBtU8kPgvQCSRgGnZtOIiIURcWdEbImI5cDlZeoo5z1ZfQ9ExAukL7biv++WiFgUEdsi4v7s/Wp5XUhfDI9GxA+yun4IPAy8s2iZStummuOA3YEvZf9GvwZ+TrZtgM3AVEmviIh1EXF30fR9gYkRsTkibg8PsNVyDnpbGxEvFp5IGinp8qxp4zlSU8Ho4uaLEk8UHkTEhuzh7gNcdj/g6aJpAI9XKrjGGp8oeryhqKb9il87C9qnKr0Xae/93ZJ2Ad4N3B0RK7I6Ds6aJZ7I6vgiae++P9vVAKwo+fteJ+nmrGnqWeD8Gl+38NorSqatAMYVPa+0bfqtOSKKvxSLX/dM0pfgCkm3Sjo+m/4VoBe4QdIySbNq+zOskRz0Vrp39THgEOB1EfEK+poKKjXHNMIaYC9JI4umja+yfD01ril+7ew99660cEQ8SAq0U9i+2QZSE9DDwJSsjk/vSA2k5qdiV5N+0YyPiD2A7xS9bn97w38gNWkVmwCsrqGu/l53fEn7+p9fNyIWRMTppGadn5F+KRAR6yPiYxFxADAd+Kikt9RZiw2Qg95KjSK1eT+Ttfd+vtlvmO0h9wAXSRqR7Q2+s8oq9dT4Y+A0SSdkB04vpv//B1cDHyZ9ofyopI7ngOclHQr8Q401XAu8X9LU7IumtP5RpF84L0o6lvQFU7CW1NR0QIXX/iVwsKSzJe0k6W+AqaRmlnrcRdr7/4SknSWdRPo3mpf9m82QtEdEbCZtk20Akk6TdFB2LOZZ0nGNak1l1gQOeiv1deAvgD8BdwK/atH7ziAd0HwK+AJwDam/fzk7XGNELAY+RArvNcA60sHCagpt5L+OiD8VTf84KYTXA9/Naq6lhuuyv+HXpGaNX5cscgFwsaT1wOfI9o6zdTeQjkn8NuvJclzJaz8FnEb61fMU8AngtJK6BywiNpGC/RTSdv8WcG5EPJwtcg6wPGvCOp/07wnpYPNNwPPAHcC3IuLmemqxgZOPi1gnknQN8HBENP0XhVneeY/eOoKkYyQdKGlY1v3wdFJbr5nVyWfGWqfYB/gp6cDoKuAfIuKe9pZklg9uujEzyzk33ZiZ5VzHNd2MGTMmJk2a1O4yzMwGlYULF/4pIsaWm9dxQT9p0iR6enraXYaZ2aAiqfSM6D9z042ZWc456M3Mcs5Bb2aWcw56M7Occ9CbmeVcboK+uxsmTYJhw9J9ty+VbGYGdGD3yh3R3Q0zZ8KG7LIVK1ak5wAzZlRez8xsKMjFHv3s2X0hX7BhQ5puZjbU5SLoV64c2HQzs6EkF0E/ofRCbJl9921tHWZmnSgXQT9nDowc+fLpTz4JH/sYPPNM62syM+sUuQj6GTNg7lyYOBGkdH/ZZfCBD8DXvgZTpsDll8PWre2u1Mys9TpuPPqurq5o5KBm99wDH/4w3H47HHEEfP3rcNJJDXt5M7OOIGlhRHSVm5eLPfpqjjoKbr0Vrr02NeG8+c3wV38Fjz3W7srMzFoj90EPqTnnr/8aHnoILrkErrsOXv3q1P3y+efbXZ2ZWXMNiaAv+Iu/gM98Bh55JAX/F78IBx8MV14J27a1uzozs+YYUkFfMG4c/OAHcMcdMH48vP/9cNxx6bmZWd7kYgiEHVUI9+5umDULXv96GDMG9tsv9cHfb7/tHxfu99kHRoxod/VmZrUZ0kEPaRC0c86Bd70Lvvc9WLIE/vCHdHvgAXjiifLdMst9IYwbB+95D+y9d+v/DjOzSnLfvbJeW7fCn/6Ugn/Nmu3vix8XvhAOOQRuuKHy2bpmZs1QrXvlkN+j78/w4fCqV6XbUUdVXm7r1tRX/4wz4A1vgBtvhEMPbV2dZmaVDMmDsc0wfHg6EevWW2HzZjjxROigHyZmNoQ56BvsiCPgN7+B3XdPJ2fdfHO7KzKzoc5B3wQHHZTCfuJEOOUU+O//bndFZp1t27Z0Xsu//iu89FK7q8kfB32TjBsHt90GRx4JZ56ZTsoys5fbuDH1Vps9Gz79aTjsMLjppnZXlS8O+ibaa6/0gX3zm9NJWV//ersrMussa9fCW94CP/1pGmn2hhsgAt76Vjj77NSbzernoG+y3XeHn/887dX/8z/DZz+bPshmQ90jj8Dxx6cRZn/8Y/jIR1LAL1oEn/88/OQnqbvyZZd5iPF61RT0kqZJWiKpV9KsKsudKSkkdWXPJ0naKOne7PadRhU+mOyyC1xzDXzwg/CFL8CFF3psHRvafvvbFPLPPps6LLz73X3zdt0VLrooBf6xx6b/L8cdBwsXtq3cQa/foJc0HLgMOAWYCrxX0tQyy40CPgzcVTJraUQcmd3Ob0DNg9Lw4fDd78InPgHf+la6WMqmTe2uyqz1rrkmNdeMGQN33plCvJyDD05NOT/8IaxalUL/H/8xfTnYwNSyR38s0BsRyyJiEzAPOL3McpcAlwIvNrC+XJHg0kvhS1+CefPSyVUbNrS7KrPWiEif/7POgmOOgd/9Dg48sPo6Ulr+4YfhggtSM86hh6YvCzeB1q6WoB8HPF70fFU27c8kHQ2Mj4hflFl/sqR7JN0q6cRybyBppqQeST1r166ttfZB65OfTJc+/NWv4G1v8zVtLf+2bElBPWtWCu4bbxzYmFB77AHf/Cb8/vepR9tZZ8Hb3w6PPtq8mvOk7iEQJA0Dvgq8v8zsNcCEiHhK0v8CfibpNRHxXPFCETEXmAtprJt6axoM/v7vYfTo1IRz0kkp9PfZp91VmTXe+vXwN3+TLvgzaxbMmZMGE9wRXV1w113w7W+n7piHHQaf+lTaedp118bW3UwbN8Ljj8OKFbByZd9tn33SuQSNVkvQrwbGFz3fP5tWMAp4LXCLJIB9gPmSpkdED/ASQEQslLQUOBjw4ACki5/ssUcaOfOEE9JezuTJ7a7KrHH+8Ac47TS4/364/HKYObP+1xw+PB2gPfNM+OhH04Hb7u507Ovkk+t//Xpt25a6ja5c+fIgL9xKGy6kNAJus65n3e/olZJ2Ah4B3kIK+AXA2RGxuMLytwAfj4geSWOBpyNiq6QDgNuBwyLi6Urv12mjV7bCHXfAO96RroB19tmVl6ulTXLEiPQ6O3LbZZf0gTNrhEWL4NRTU9Pktdems8Sb4cYbU7NQb29q+z/kEJgyJZ2hXrjttVdj3/OFF2DZsvSeS5emW28vLF+e9tRLz+7dbbd0pvyECX33xbdx42Dnneurqa7RKyNii6QLgeuB4cAVEbFY0sVAT0TMr7L6G4GLJW0GtgHnVwv5duruTj8FV65MG37OnNSs0grHH5/Ooj3zzLRXUk21II5IPXm2bNmxOoYNSwe6Tjyx7+bhlm1H3HRT+jzvvnsa1fXII5v3XoW+91/7Wgr9W2+F//zP7ZfZa6++0C/+EpgyJc0r9/9q3bq+IC+9X7Om/OsffXT6hV4a5qNHt3cnyuPRk0J+5szte8CMHJkOmLYq7Btpy5bUBjjQ2/r16eSV3/0OnsuOokyY0Bf6J5yQLqq+o+2rNjR8//vp/9OrXw2/+EW6XGerbdwIjz2WDtb29vbdHn007cwVx97o0X3BD32Bvm7d9q+5336pl9BBB21/f+CBsOeerfvbKqm2R++gByZNSm1ppSZOTD/FhpqtW1Ob6u23992efDLN23vvNN5+IfyPPrr+n5yWDxHpjNZLLkl72T/6UToG1WleeqnylwC8PMgPOggOOCDt/HUyB30/hg0r3/4t+QxWSNtm6dLtg7+3N80bOTKd8FLY4+/qSntIln+bNqVLby5alHYMfve79Nn4wAfSgVfvALSWg74f3qMfuDVr0lDMt9+e7u+7r+9LceLENC5/4Xb44WnPKM9NPtu2wfz5qVlg5537bjvttP3z/qaNG5cO3HWSCFi9OoX5okV9wf7ww+kiO5BqP/RQOO+81BPGB/Vbz0Hfj7y10bfDs8+m09nvvjuFwH33pb29Qvjvtlvq81wI/sL9qFHtrbteEXD99akv97331v96w4bBa16Tfhkdc0y6HX546k3VCuvXwwMP9IV54b74pL7x41NNhx2WbocfnoYraFWNVp6Dvgbt7HWTVxs3wuLFKfQL4X/ffduHxgEH9AX/EUfAm97U+K5wzXLXXekEoFtuSec/XHJJOltzy5a0p7t58/aPi2/lpm/alNqJFyxIt6eeSu8zYkTaRoXg7+qCqVNTf/KBikhD/y5bltqpC/eFx48XnQM/alRfkBdC/bDD3DTXqRz01jEiUpgUB/9996WAi0ih9s53piaAadM6s5334YfTTsFPfwqvfGUaenrmzMbu0Uak5sRC6Pf0pNv69Wn+yJHpQHgh+I85Jh00lNKvq+LwLn68fDm8WDIa1bhx6Ytq8uTUB70Q7BMnuglmMHHQW8fbsCE1fVx7LVx9dTpzcOzYdALZuefCUUe1P3RWr05nYV5xRQraf/mXdI2BVjU/bduWxnAvBP+CBak7bCG4R49OTT9Pl5ypssce6ZfT5Ml994XHEycOrqEDrDIHvQ0qmzendu8rr0wHODdtgte+NgX+jBmpP3MrrVuXRhz9xjdS19MLLkh79GPHtraOcjZvhgcf7At/6eVh3gl9vK35HPQ2aK1bl4akveqqNFTEsGGpj/a556ZhnpvZt3nDhjRi4pe+lJpD3vc+uPji1EvLrNNUC/ocd3izPNhzTzj//NRHe8mSdPHohx5Ke/b77JOu2nXbbY0932HLlnSRmClT0sHWN7whNStddZVD3gYn79HboLNtWwr3q65KZ18+/3wK4HPOSWOq7LprGqCtcF/8uPh+xIjt2/0j0nVKZ8/uu57ppZemk8HMOp2bbiy3XngBfvaz1J5/000Dv+pQ8RcBwB//mLoufvGLMH16+w8Am9WqrtErzTrZbrulZpwZM1JIr1mTxjJ58cUdu3/Tm9Ivgx3po27WqRz0lhuvfGW6mdn2fDDWzCznHPRmZjnnoDczyzkHvZlZzjnozcxyzkFvZpZzDnozs5xz0JuZ5ZyD3sws5xz0ZmY556A3M8s5B72ZWc456M3Mcs5Bb2aWcw56M7Occ9CbmeVcTUEvaZqkJZJ6Jc2qstyZkkJSV9G0T2XrLZH09kYUbWZmtes36CUNBy4DTgGmAu+VNLXMcqOADwN3FU2bCpwFvAaYBnwre73c6e5OF6geNizdd3e3uyIzs6SWPfpjgd6IWBYRm4B5wOlllrsEuBR4sWja6cC8iHgpIh4DerPXy5Xubpg5E1asSBenXrEiPXfYm1knqCXoxwGPFz1flU37M0lHA+Mj4hcDXTdbf6akHkk9a9euranwTjJ7NmzYsP20DRvSdDOzdqv7YKykYcBXgY/t6GtExNyI6IqIrrFjx9ZbUsutXDmw6WZmrVRL0K8Gxhc93z+bVjAKeC1wi6TlwHHA/OyAbH/r5sKECQObbmbWSrUE/QJgiqTJkkaQDq7OL8yMiGcjYkxETIqIScCdwPSI6MmWO0vSLpImA1OA3zf8r2izOXNg5Mjtp40cmaabmbVbv0EfEVuAC4HrgYeAayNisaSLJU3vZ93FwLXAg8CvgA9FxNb6y+4sM2bA3LkwcSJI6X7u3DTdzKzdFBHtrmE7XV1d0dPT0+4yzMwGFUkLI6Kr3DyfGWtmlnMOejOznHPQm5nlnIPezCznHPRmZjnnoDczyzkHvZlZzjnozcxyzkFvZpZzDnozs5xz0JuZ5ZyD3sws5xz0ZmY556A3M8s5B72ZWc456M3Mcs5Bb2aWcw56M7Occ9CbmeWcg97MLOcc9GZmOeegNzPLOQd9h+juhkmTYNiwdN/d3e6KzCwvdmp3AZZCfeZM2LAhPV+xIj0HmDGjfXWZWT54j74DzJ7dF/IFGzak6WZm9XLQd4CVKwc23cxsIBz0HWDChIFNNzMbCAd9B5gzB0aO3H7ayJFpuplZvRz0HWDGDJg7FyZOBCndz53rA7Fm1hjuddMhZsxwsJtZc9S0Ry9pmqQlknolzSoz/3xJiyTdK+k3kqZm0ydJ2phNv1fSdxr9B5iZWXX97tFLGg5cBrwVWAUskDQ/Ih4sWuzqiPhOtvx04KvAtGze0og4srFlm5lZrWrZoz8W6I2IZRGxCZgHnF68QEQ8V/R0NyAaV6KZmdWjlqAfBzxe9HxVNm07kj4kaSnwZeCfimZNlnSPpFslnVhXtWZmNmAN63UTEZdFxIHAJ4HPZJPXABMi4ijgo8DVkl5Ruq6kmZJ6JPWsXbu2USWZmRm1Bf1qYHzR8/2zaZXMA84AiIiXIuKp7PFCYClwcOkKETE3Iroiomvs2LG11m5mZjWoJegXAFMkTZY0AjgLmF+8gKQpRU/fATyaTR+bHcxF0gHAFGBZIwo3M7Pa9NvrJiK2SLoQuB4YDlwREYslXQz0RMR84EJJJwObgXXAednqbwQulrQZ2AacHxFPN+MPMTOz8hTRWR1kurq6oqenp91lDDrd3Wm0y5Ur0xg5c+b4BCyzoUTSwojoKjfPZ8bmgMezN7NqPNZNDng8ezOrxkGfAx7P3syqcdDngMezN7NqHPQ54PHszawaB30OeDx7M6vGvW5ywuPZm1kl3qM3M8s5B72ZWc456M3Mcs5Bb2aWcw56M7Occ9CbmeWcg97MLOcc9GZmOeegNyANdTxpEgwblu67u9tdkZk1is+MNY9nb5Zz3qM3j2dvlnMOevN49mY556A3j2dvlnMOevN49mY556A3j2dvlnPudWOAx7M3yzPv0ZuZ5ZyD3sws5xz01hA+s9asc7mN3urmM2vNOpv36K1uPrPWrLM56K1uPrPWrLM56K1uPrPWrLM56K1uPrPWrLPVFPSSpklaIqlX0qwy88+XtEjSvZJ+I2lq0bxPZestkfT2RhZvncFn1pp1NkVE9QWk4cAjwFuBVcAC4L0R8WDRMq+IiOeyx9OBCyJiWhb4PwSOBfYDbgIOjoitld6vq6srenp66vurzMyGGEkLI6Kr3Lxa9uiPBXojYllEbALmAacXL1AI+cxuQOHb43RgXkS8FBGPAb3Z65mZWYvUEvTjgMeLnq/Kpm1H0ockLQW+DPzTANedKalHUs/atWtrrd1yxCdcmTVPww7GRsRlEXEg8EngMwNcd25EdEVE19ixYxtVkg0ShROuVqyAiL4Trhz2Zo1RS9CvBsYXPd8/m1bJPOCMHVzXhiCfcGXWXLUE/QJgiqTJkkYAZwHzixeQNKXo6TuAR7PH84GzJO0iaTIwBfh9/WVbnviEK7Pm6nesm4jYIulC4HpgOHBFRCyWdDHQExHzgQslnQxsBtYB52XrLpZ0LfAgsAX4ULUeNzY0TZiQmmvKTTez+vXbvbLV3L1y6CkdFA3SCVfui29Wu3q7V5o1lU+4MmsuD1NsHcGXMjRrHu/Rm5nlnIPezCznHPRmZjnnoLdc8BAKZpX5YKwNer5mrVl13qO3Qc9DKJhV56C3Qc9DKJhV56C3Qc/XrDWrzkFvg56vWWtWnYPeBj0PoWBWnYPecmHGDFi+HLZtS/cDDXl3z7Q8c/dKG/LcPdPyznv0NuS5e6blnYPehjx3z7S8c9DbkOfumZZ3Dnob8hrRPdMHc62TOehtyKu3e2bhYO6KFRDRdzDXYW+dwteMNavTpEnlL24+cWLq6mnWCr5mrFkT+WCudToHvVmdfDDXOp2D3qxOPphrnc5Bb1YnH8y1TueDsWZt5oO51gg+GGvWwXww15rNQW/WZo04mOs2fqvGQW/WZvUezHUbv/XHQW/WZvUezPXom9YfH4w1G+SGDUt78qWkdCEWGxrqPhgraZqkJZJ6Jc0qM/+jkh6UdL+k/5E0sWjeVkn3Zrf5O/5nmFk5PmHL+tNv0EsaDlwGnAJMBd4raWrJYvcAXRFxOPBj4MtF8zZGxJHZbXqD6jazjE/Ysv7Uskd/LNAbEcsiYhMwDzi9eIGIuDkiCq2EdwL7N7ZMM6vEJ2xZf2oJ+nHA40XPV2XTKvkgcF3R810l9Ui6U9IZ5VaQNDNbpmft2rU1lGRmxeq5OHojDub6F0Fna+jFwSW9D+gC3lQ0eWJErJZ0APBrSYsiYmnxehExF5gL6WBsI2sys+rqPWHLF1fvfLXs0a8Gxhc93z+bth1JJwOzgekR8VJhekSszu6XAbcAR9VRr5k1WL0Hc929s/PVEvQLgCmSJksaAZwFbNd7RtJRwOWkkP9j0fQ9Je2SPR4DvAF4sFHFm1n96j2Y24ghHNz001z9Bn1EbAEuBK4HHgKujYjFki6WVOhF8xVgd+BHJd0oXw30SLoPuBn4UkQ46M06SL0Hc+v9ReCDwc3nE6bMrC6lbfSQfhHU+mXh0Tsbw6NXmlnT1PuLwE0/zdfQXjdmNjTNmLHjPWwmTCi/Rz/Qph/3+qnMe/Rm1lb1Hgx2r5/+OejNrK3c9NN8broxs7Zz009zeY/ezAY1N/30z0FvZoNaJzT9dDoHvZkNevUM6jYUrtnroDezIW0oXLPXQW9mQ9pQuGavg97Mhrx6mn4GQ/dOB72ZWR0Gw6BuDnozszoMhu6dDnozszoMhu6dPjPWzKxO7Tyztxbeozcza6N6m35q4aA3M2ujept+auGmGzOzNqun6acW3qM3M8s5B72ZWc456M3Mcs5Bb2aWcw56M7OcU0S0u4btSFoLlDl9oGOMAf7U7iKqcH31cX31cX31qae+iRExttyMjgv6TiepJyK62l1HJa6vPq6vPq6vPs2qz003ZmY556A3M8s5B/3AzW13Af1wffVxffVxffVpSn1uozczyznv0ZuZ5ZyD3sws5xz0JSSNl3SzpAclLZb04TLLnCTpWUn3ZrfPtaHO5ZIWZe/fU2a+JH1DUq+k+yUd3cLaDinaNvdKek7SR0qWaek2lHSFpD9KeqBo2l6SbpT0aHa/Z4V1z8uWeVTSeS2s7yuSHs7+/f5L0ugK61b9LDSxvoskrS76Nzy1wrrTJC3JPouzWljfNUW1LZd0b4V1W7H9yuZKyz6DEeFb0Q3YFzg6ezwKeASYWrLMScDP21zncmBMlfmnAtcBAo4D7mpTncOBJ0gnc7RtGwJvBI4GHiia9mVgVvZ4FnBpmfX2ApZl93tmj/dsUX1vA3bKHl9arr5aPgtNrO8i4OM1/PsvBQ4ARgD3lf5/alZ9JfP/L/C5Nm6/srnSqs+g9+hLRMSaiLg7e7weeAgY196qdsjpwFWR3AmMlrRvG+p4C7A0Itp6tnNE3AY8XTL5dODK7PGVwBllVn07cGNEPB0R64AbgWmtqC8iboiILdnTO4H9G/2+taqw/WpxLNAbEcsiYhMwj7TdG6pafZIEvAf4YaPft1ZVcqUln0EHfRWSJgFHAXeVmX28pPskXSfpNS0tLAngBkkLJc0sM38c8HjR81W05wvrLCr/B2v3NnxVRKzJHj8BvKrMMp2yHf+W9AutnP4+C810Yda0dEWFZodO2H4nAk9GxKMV5rd0+5XkSks+gw76CiTtDvwE+EhEPFcy+25SU8QRwDeBn7W6PuCEiDgaOAX4kKQ3tqGGqiSNAKYDPyozuxO24Z9F+o3ckX2NJc0GtgDdFRZp12fh28CBwJHAGlLzSCd6L9X35lu2/arlSjM/gw76MiTtTPrH6I6In5bOj4jnIuL57PEvgZ0ljWlljRGxOrv/I/BfpJ/IxVYD44ue759Na6VTgLsj4snSGZ2wDYEnC81Z2f0fyyzT1u0o6f3AacCMLAhepobPQlNExJMRsTUitgHfrfC+7d5+OwHvBq6ptEyrtl+FXGnJZ9BBXyJrz/t34KGI+GqFZfbJlkPSsaTt+FQLa9xN0qjCY9JBuwdKFpsPnJv1vjkOeLboJ2KrVNyTavc2zMwHCj0YzgP+u8wy1wNvk7Rn1jTxtmxa00maBnwCmB4RGyosU8tnoVn1FR/zeVeF95s+EvYAAADzSURBVF0ATJE0OfuFdxZpu7fKycDDEbGq3MxWbb8qudKaz2AzjzQPxhtwAunn0/3AvdntVOB84PxsmQuBxaQeBHcCr29xjQdk731fVsfsbHpxjQIuI/V4WAR0tbjG3UjBvUfRtLZtQ9IXzhpgM6mN84PA3sD/AI8CNwF7Zct2Ad8rWvdvgd7s9oEW1tdLapstfA6/ky27H/DLap+FFtX3g+yzdT8psPYtrS97fiqpl8nSVtaXTf+PwmeuaNl2bL9KudKSz6CHQDAzyzk33ZiZ5ZyD3sws5xz0ZmY556A3M8s5B72ZWc456M3Mcs5Bb2aWc/8f4Mg8cDamWiAAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g026ea6_uQpw"
      },
      "source": [
        "상식수준의 기준점보다 성능이 낮은 것을 확인할 수 있다. 과거보다 최근 내용을 더 잘 기억하기 때문에, 날씨 예측 예제에서는 시간 순서대로 처리하는 것이 매우 중요하다. 하지만 자연어 처리를 포함한 많은 문제에서는 시간이 크게 중요하지 않다. 문장 전체의 의미에, 단어가 어디에 위치해있느냐는 크게 영향을 주지 않기 때문이다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OgGZc4avuyMU",
        "outputId": "969f97e8-d848-48e0-bd2c-be09b8f9d770"
      },
      "source": [
        "#자연어 처리 예제에 적용 (LSTM 활용)\r\n",
        "\r\n",
        "from keras.datasets import imdb\r\n",
        "from keras.preprocessing import sequence\r\n",
        "from keras import layers\r\n",
        "from keras.models import Sequential\r\n",
        "\r\n",
        "max_features = 10000  # 특성으로 사용할 단어의 수\r\n",
        "maxlen = 500  # 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용)\r\n",
        "\r\n",
        "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\r\n",
        "\r\n",
        "# 시퀀스를 뒤집는다\r\n",
        "x_train = [x[::-1] for x in x_train]\r\n",
        "x_test = [x[::-1] for x in x_test]\r\n",
        "\r\n",
        "# 시퀀스에 패딩을 추가\r\n",
        "x_train = sequence.pad_sequences(x_train, maxlen=maxlen)\r\n",
        "x_test = sequence.pad_sequences(x_test, maxlen=maxlen)\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "model.add(layers.Embedding(max_features, 128))\r\n",
        "model.add(layers.LSTM(32))\r\n",
        "model.add(layers.Dense(1, activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop',\r\n",
        "              loss='binary_crossentropy',\r\n",
        "              metrics=['acc'])\r\n",
        "history = model.fit(x_train, y_train,\r\n",
        "                    epochs=10,\r\n",
        "                    batch_size=128,\r\n",
        "                    validation_split=0.2)"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz\n",
            "17465344/17464789 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "<string>:6: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/datasets/imdb.py:159: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_train, y_train = np.array(xs[:idx]), np.array(labels[:idx])\n",
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/datasets/imdb.py:160: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
            "  x_test, y_test = np.array(xs[idx:]), np.array(labels[idx:])\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "157/157 [==============================] - 70s 432ms/step - loss: 0.6112 - acc: 0.6519 - val_loss: 0.4145 - val_acc: 0.8452\n",
            "Epoch 2/10\n",
            "157/157 [==============================] - 73s 463ms/step - loss: 0.3426 - acc: 0.8691 - val_loss: 0.3215 - val_acc: 0.8792\n",
            "Epoch 3/10\n",
            "157/157 [==============================] - 70s 446ms/step - loss: 0.2508 - acc: 0.9085 - val_loss: 0.4502 - val_acc: 0.8256\n",
            "Epoch 4/10\n",
            "157/157 [==============================] - 69s 442ms/step - loss: 0.2027 - acc: 0.9282 - val_loss: 0.3386 - val_acc: 0.8800\n",
            "Epoch 5/10\n",
            "157/157 [==============================] - 71s 450ms/step - loss: 0.1877 - acc: 0.9340 - val_loss: 0.4211 - val_acc: 0.8698\n",
            "Epoch 6/10\n",
            "157/157 [==============================] - 71s 452ms/step - loss: 0.1581 - acc: 0.9453 - val_loss: 0.4025 - val_acc: 0.8634\n",
            "Epoch 7/10\n",
            "157/157 [==============================] - 72s 460ms/step - loss: 0.1524 - acc: 0.9495 - val_loss: 0.4615 - val_acc: 0.8590\n",
            "Epoch 8/10\n",
            "157/157 [==============================] - 71s 455ms/step - loss: 0.1429 - acc: 0.9515 - val_loss: 0.4575 - val_acc: 0.8486\n",
            "Epoch 9/10\n",
            "157/157 [==============================] - 71s 452ms/step - loss: 0.1196 - acc: 0.9604 - val_loss: 0.4080 - val_acc: 0.8662\n",
            "Epoch 10/10\n",
            "157/157 [==============================] - 72s 456ms/step - loss: 0.1058 - acc: 0.9649 - val_loss: 0.4302 - val_acc: 0.8670\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_wG55FIWvVZU"
      },
      "source": [
        "시퀀스를 뒤집지 않고, 시간 순서로 훈련한 LSTM과 성능이 거의 동일하다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ySrWHN4vwTP"
      },
      "source": [
        "## 3-6) keras의 Bidirectional 층을 사용한 양방향 RNN"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gqO_rz3qwMMz"
      },
      "source": [
        "### 3-6-1) IMDB 감성 분석 문제에 적용"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ezdftYSDwBI6"
      },
      "source": [
        "from keras import backend as K\r\n",
        "K.clear_session()"
      ],
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-W5BIDALwQi2",
        "outputId": "7640ab9b-29ca-4221-cbdd-5260e7036b59"
      },
      "source": [
        "model = Sequential()\r\n",
        "model.add(layers.Embedding(max_features, 32))\r\n",
        "model.add(layers.Bidirectional(layers.LSTM(32)))\r\n",
        "model.add(layers.Dense(1, activation='sigmoid'))\r\n",
        "\r\n",
        "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\r\n",
        "history = model.fit(x_train, y_train, epochs=10, batch_size=128, validation_split=0.2)"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "157/157 [==============================] - 97s 591ms/step - loss: 0.6485 - acc: 0.6080 - val_loss: 0.4289 - val_acc: 0.8376\n",
            "Epoch 2/10\n",
            "157/157 [==============================] - 90s 573ms/step - loss: 0.4288 - acc: 0.8314 - val_loss: 0.4880 - val_acc: 0.7692\n",
            "Epoch 3/10\n",
            "157/157 [==============================] - 90s 574ms/step - loss: 0.3172 - acc: 0.8774 - val_loss: 0.4900 - val_acc: 0.8276\n",
            "Epoch 4/10\n",
            "157/157 [==============================] - 92s 585ms/step - loss: 0.2453 - acc: 0.9104 - val_loss: 0.3104 - val_acc: 0.8848\n",
            "Epoch 5/10\n",
            "157/157 [==============================] - 91s 581ms/step - loss: 0.2156 - acc: 0.9268 - val_loss: 0.3169 - val_acc: 0.8788\n",
            "Epoch 6/10\n",
            "157/157 [==============================] - 90s 576ms/step - loss: 0.1848 - acc: 0.9364 - val_loss: 0.3125 - val_acc: 0.8848\n",
            "Epoch 7/10\n",
            "157/157 [==============================] - 90s 576ms/step - loss: 0.1754 - acc: 0.9401 - val_loss: 0.5389 - val_acc: 0.8378\n",
            "Epoch 8/10\n",
            "157/157 [==============================] - 90s 572ms/step - loss: 0.1569 - acc: 0.9491 - val_loss: 0.3555 - val_acc: 0.8640\n",
            "Epoch 9/10\n",
            "157/157 [==============================] - 89s 570ms/step - loss: 0.1382 - acc: 0.9541 - val_loss: 0.3812 - val_acc: 0.8738\n",
            "Epoch 10/10\n",
            "157/157 [==============================] - 89s 567ms/step - loss: 0.1244 - acc: 0.9571 - val_loss: 0.5283 - val_acc: 0.8670\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tLbq9TzmwXm6"
      },
      "source": [
        "val_acc가 87% 정도로, 양방향을 적용하지 않은 LSTM보다 조금 성능이 향상됐다. 양방향인만큼, 단방향보다 모델 파라미터가 두 배 많기 때문에 과대적합이 일찍 시작된다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0XdsbvErwpgF"
      },
      "source": [
        "### 3-6-2 ) 온도 예측 문제에 적용"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LOu6IbB_ws0X",
        "outputId": "2b8af2ed-56d0-49af-de59-825b183d7787"
      },
      "source": [
        "from keras.models import Sequential\r\n",
        "from keras import layers\r\n",
        "from keras.optimizers import RMSprop\r\n",
        "\r\n",
        "model = Sequential()\r\n",
        "model.add(layers.Bidirectional(\r\n",
        "    layers.GRU(32), input_shape=(None, float_data.shape[-1])))\r\n",
        "model.add(layers.Dense(1))\r\n",
        "\r\n",
        "model.compile(optimizer=RMSprop(), loss='mae')\r\n",
        "history = model.fit_generator(train_gen,\r\n",
        "                              steps_per_epoch=500,\r\n",
        "                              epochs=10,\r\n",
        "                              validation_data=val_gen,\r\n",
        "                              validation_steps=val_steps)"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.\n",
            "  warnings.warn('`Model.fit_generator` is deprecated and '\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "500/500 [==============================] - 149s 293ms/step - loss: 0.3190 - val_loss: 0.2734\n",
            "Epoch 2/10\n",
            "500/500 [==============================] - 146s 292ms/step - loss: 0.2778 - val_loss: 0.2686\n",
            "Epoch 3/10\n",
            "500/500 [==============================] - 145s 291ms/step - loss: 0.2701 - val_loss: 0.2646\n",
            "Epoch 4/10\n",
            "500/500 [==============================] - 145s 291ms/step - loss: 0.2635 - val_loss: 0.2641\n",
            "Epoch 5/10\n",
            "500/500 [==============================] - 144s 288ms/step - loss: 0.2578 - val_loss: 0.2662\n",
            "Epoch 6/10\n",
            "500/500 [==============================] - 146s 293ms/step - loss: 0.2541 - val_loss: 0.2651\n",
            "Epoch 7/10\n",
            "500/500 [==============================] - 144s 288ms/step - loss: 0.2498 - val_loss: 0.2709\n",
            "Epoch 8/10\n",
            "500/500 [==============================] - 145s 290ms/step - loss: 0.2418 - val_loss: 0.2718\n",
            "Epoch 9/10\n",
            "500/500 [==============================] - 146s 292ms/step - loss: 0.2377 - val_loss: 0.2818\n",
            "Epoch 10/10\n",
            "500/500 [==============================] - 145s 289ms/step - loss: 0.2320 - val_loss: 0.2812\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YJyiNGwEwxVe"
      },
      "source": [
        "3-4 에서 수행한 일반 GRU 층과 성능이 비슷하다."
      ]
    }
  ]
}