6_gan_anomaly_detection/6_4_EfficientGAN_GoogleClab.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.7"
    },
    "colab": {
      "name": "6-4_EfficientGAN_GoogleClab.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7ZaLsEIeL8J7"
      },
      "source": [
        "# 6.4 Efficient GANの作成（Google Colab版）\n",
        "\n",
        "- 本ファイルでは、Efficient GANのネットワークを実装し、学習をします。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ey6RtLORL8KI"
      },
      "source": [
        "# 6.4 学習目標\n",
        "\n",
        "1.\tEfficient GANを実装し、手書き数字画像で異常検知が生成できる"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 環境確認\n",
        "\n",
        "使用している環境を確認します"
      ],
      "metadata": {
        "id": "BscEOauqOEmD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import sys\n",
        "print(\"Pythonのバージョン：\",sys.version)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XTgNNvDSOPFd",
        "outputId": "93a55010-033c-4da2-97ae-210a643d623a"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Pythonのバージョン： 3.7.12 (default, Sep 10 2021, 00:21:48) \n",
            "[GCC 7.5.0]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "print(\"PyTorchのバージョン：\", torch.__version__)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xA1MAPjqOCW_",
        "outputId": "21bba7db-2536-4747-d300-bfd06683512a"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "PyTorchのバージョン： 1.10.0+cu111\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"使用しているGPUの確認\")\n",
        "!nvidia-smi"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yEZL7NS6OqAG",
        "outputId": "51206dad-6503-4a4b-945f-413a8f658b30"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "使用しているGPUの確認\n",
            "Fri Jan 21 02:06:07 2022       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 495.46       Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   35C    P0    27W / 250W |      0MiB / 16280MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LNnV0rM1L8KK"
      },
      "source": [
        "# 事前準備\n",
        "書籍の指示に従い、本章で使用するデータを用意します"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_lGX1rk4L8KL"
      },
      "source": [
        "# パッケージのimport\n",
        "import random\n",
        "import math\n",
        "import time\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from PIL import Image\n",
        "\n",
        "import torch\n",
        "import torch.utils.data as data\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import torch.optim as optim\n",
        "\n",
        "from torchvision import transforms"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EKc25TAjL8KP"
      },
      "source": [
        "# Setup seeds\n",
        "torch.manual_seed(1234)\n",
        "torch.cuda.manual_seed(1234)\n",
        "np.random.seed(1234)\n",
        "random.seed(1234)\n"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Google Colab上に使用する訓練データ等を一時的に用意する\n",
        "書籍の指示に従い、本章で使用するデータを用意します"
      ],
      "metadata": {
        "id": "YFhIV7ciO7UF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os \n",
        "\n",
        "\n",
        "# フォルダ「data」が存在しない場合は作成する\n",
        "data_dir = \"./data/\"\n",
        "if not os.path.exists(data_dir):\n",
        "    os.mkdir(data_dir)"
      ],
      "metadata": {
        "id": "C2BPk_grO6VO"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import sklearn\n",
        "print(sklearn.__version__)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gZMW7zn0O4bt",
        "outputId": "fc194381-7271-4f01-eeec-bd3c1efbada7"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "1.0.2\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# MNISTの手書き数字画像をダウンロードし読み込みます\n",
        "from sklearn.datasets import fetch_openml\n",
        "\n",
        "# mnist = fetch_openml('mnist_784', version=1, data_home=\"./data/\")  # data_homeは保存先を指定します\n",
        "mnist = fetch_openml('mnist_784', version=1, data_home=\"./data/\", as_frame=False)  \n",
        "# Issue #153 2020年12月にリリースされたsklearn 0.24.0以降の仕様変更に合わせる場合"
      ],
      "metadata": {
        "id": "8mx842GePcac"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# データの取り出し\n",
        "X = mnist.data\n",
        "y = mnist.target"
      ],
      "metadata": {
        "id": "SYyZegr4Pcrr"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "from PIL import Image\n",
        "\n",
        "# MNISTのデータの1つ目を可視化する\n",
        "plt.imshow(X[0].reshape(28, 28), cmap='gray')\n",
        "print(\"この画像データのラベルは{}です\".format(y[0]))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "id": "UQzxtebGPflP",
        "outputId": "cccd0e19-5afa-499a-91a8-00d8f9d96b31"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "この画像データのラベルは5です\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAN9klEQVR4nO3df4xV9ZnH8c+zWP6QojBrOhKKSyEGg8ZON4gbl6w1hvojGhw1TSexoZE4/YNJaLIhNewf1WwwZBU2SzTNTKMWNl1qEzUgaQouoOzGhDgiKo5LdQ2mTEaowZEf/mCHefaPezBTnfu9w7nn3nOZ5/1Kbu6957nnnicnfDi/7pmvubsATH5/VXYDAJqDsANBEHYgCMIOBEHYgSAuaubCzIxT/0CDubuNN72uLbuZ3Wpmh8zsPTN7sJ7vAtBYlvc6u5lNkfRHSUslHZH0qqQudx9IzMOWHWiwRmzZF0t6z93fd/czkn4raVkd3weggeoJ+2xJfxrz/kg27S+YWbeZ9ZtZfx3LAlCnhp+gc/c+SX0Su/FAmerZsg9KmjPm/bezaQBaUD1hf1XSlWb2HTObKulHkrYV0xaAouXejXf3ETPrkbRD0hRJT7n724V1BqBQuS+95VoYx+xAwzXkRzUALhyEHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBJF7yGZcGKZMmZKsX3rppQ1dfk9PT9XaxRdfnJx3wYIFyfrKlSuT9ccee6xqraurKznv559/nqyvW7cuWX/44YeT9TLUFXYzOyzppKSzkkbcfVERTQEoXhFb9pvc/aMCvgdAA3HMDgRRb9hd0k4ze83Musf7gJl1m1m/mfXXuSwAdah3N36Juw+a2bckvWhm/+Pue8d+wN37JPVJkpl5ncsDkFNdW3Z3H8yej0l6XtLiIpoCULzcYTezaWY2/dxrST+QdLCoxgAUq57d+HZJz5vZue/5D3f/QyFdTTJXXHFFsj516tRk/YYbbkjWlyxZUrU2Y8aM5Lz33HNPsl6mI0eOJOsbN25M1js7O6vWTp48mZz3jTfeSNZffvnlZL0V5Q67u78v6bsF9gKggbj0BgRB2IEgCDsQBGEHgiDsQBDm3rwftU3WX9B1dHQk67t3707WG32baasaHR1N1u+///5k/dSpU7mXPTQ0lKx//PHHyfqhQ4dyL7vR3N3Gm86WHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeC4Dp7Adra2pL1ffv2Jevz5s0rsp1C1ep9eHg4Wb/pppuq1s6cOZOcN+rvD+rFdXYgOMIOBEHYgSAIOxAEYQeCIOxAEIQdCIIhmwtw/PjxZH316tXJ+h133JGsv/7668l6rT+pnHLgwIFkfenSpcn66dOnk/Wrr766am3VqlXJeVEstuxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EAT3s7eASy65JFmvNbxwb29v1dqKFSuS8953333J+pYtW5J1tJ7c97Ob2VNmdszMDo6Z1mZmL5rZu9nzzCKbBVC8iezG/1rSrV+Z9qCkXe5+paRd2XsALaxm2N19r6Sv/h50maRN2etNku4quC8ABcv72/h2dz83WNaHktqrfdDMuiV151wOgILUfSOMu3vqxJu790nqkzhBB5Qp76W3o2Y2S5Ky52PFtQSgEfKGfZuk5dnr5ZK2FtMOgEapuRtvZlskfV/SZWZ2RNIvJK2T9DszWyHpA0k/bGSTk92JEyfqmv+TTz7JPe8DDzyQrD/zzDPJeq0x1tE6aobd3buqlG4uuBcADcTPZYEgCDsQBGEHgiDsQBCEHQiCW1wngWnTplWtvfDCC8l5b7zxxmT9tttuS9Z37tyZrKP5GLIZCI6wA0EQdiAIwg4EQdiBIAg7EARhB4LgOvskN3/+/GR9//79yfrw8HCyvmfPnmS9v7+/au2JJ55IztvMf5uTCdfZgeAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIrrMH19nZmaw//fTTyfr06dNzL3vNmjXJ+ubNm5P1oaGhZD0qrrMDwRF2IAjCDgRB2IEgCDsQBGEHgiDsQBBcZ0fSNddck6xv2LAhWb/55vyD/fb29ibra9euTdYHBwdzL/tClvs6u5k9ZWbHzOzgmGkPmdmgmR3IHrcX2SyA4k1kN/7Xkm4dZ/q/untH9vh9sW0BKFrNsLv7XknHm9ALgAaq5wRdj5m9me3mz6z2ITPrNrN+M6v+x8gANFzesP9S0nxJHZKGJK2v9kF373P3Re6+KOeyABQgV9jd/ai7n3X3UUm/krS42LYAFC1X2M1s1pi3nZIOVvssgNZQ8zq7mW2R9H1Jl0k6KukX2fsOSS7psKSfunvNm4u5zj75zJgxI1m/8847q9Zq3StvNu7l4i/t3r07WV+6dGmyPllVu85+0QRm7Bpn8pN1dwSgqfi5LBAEYQeCIOxAEIQdCIKwA0FwiytK88UXXyTrF12Uvlg0MjKSrN9yyy1Vay+99FJy3gsZf0oaCI6wA0EQdiAIwg4EQdiBIAg7EARhB4KoedcbYrv22muT9XvvvTdZv+6666rWal1Hr2VgYCBZ37t3b13fP9mwZQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBILjOPsktWLAgWe/p6UnW77777mT98ssvP++eJurs2bPJ+tBQ+q+Xj46OFtnOBY8tOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EwXX2C0Cta9ldXeMNtFtR6zr63Llz87RUiP7+/mR97dq1yfq2bduKbGfSq7llN7M5ZrbHzAbM7G0zW5VNbzOzF83s3ex5ZuPbBZDXRHbjRyT9o7svlPR3klaa2UJJD0ra5e5XStqVvQfQomqG3d2H3H1/9vqkpHckzZa0TNKm7GObJN3VqCYB1O+8jtnNbK6k70naJ6nd3c/9OPlDSe1V5umW1J2/RQBFmPDZeDP7pqRnJf3M3U+MrXlldMhxB2109z53X+Tui+rqFEBdJhR2M/uGKkH/jbs/l00+amazsvosScca0yKAItTcjTczk/SkpHfcfcOY0jZJyyWty563NqTDSaC9fdwjnC8tXLgwWX/88ceT9auuuuq8eyrKvn37kvVHH320am3r1vQ/GW5RLdZEjtn/XtKPJb1lZgeyaWtUCfnvzGyFpA8k/bAxLQIoQs2wu/t/Sxp3cHdJNxfbDoBG4eeyQBCEHQiCsANBEHYgCMIOBMEtrhPU1tZWtdbb25uct6OjI1mfN29erp6K8MorryTr69evT9Z37NiRrH/22Wfn3RMagy07EARhB4Ig7EAQhB0IgrADQRB2IAjCDgQR5jr79ddfn6yvXr06WV+8eHHV2uzZs3P1VJRPP/20am3jxo3JeR955JFk/fTp07l6Quthyw4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQYS5zt7Z2VlXvR4DAwPJ+vbt25P1kZGRZD11z/nw8HByXsTBlh0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgjB3T3/AbI6kzZLaJbmkPnf/NzN7SNIDkv6cfXSNu/++xnelFwagbu4+7qjLEwn7LEmz3H2/mU2X9Jqku1QZj/2Uuz820SYIO9B41cI+kfHZhyQNZa9Pmtk7ksr90ywAztt5HbOb2VxJ35O0L5vUY2ZvmtlTZjazyjzdZtZvZv11dQqgLjV347/8oNk3Jb0saa27P2dm7ZI+UuU4/p9V2dW/v8Z3sBsPNFjuY3ZJMrNvSNouaYe7bxinPlfSdne/psb3EHagwaqFveZuvJmZpCclvTM26NmJu3M6JR2st0kAjTORs/FLJP2XpLckjWaT10jqktShym78YUk/zU7mpb6LLTvQYHXtxheFsAONl3s3HsDkQNiBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQii2UM2fyTpgzHvL8umtaJW7a1V+5LoLa8ie/ubaoWm3s/+tYWb9bv7otIaSGjV3lq1L4ne8mpWb+zGA0EQdiCIssPeV/LyU1q1t1btS6K3vJrSW6nH7ACap+wtO4AmIexAEKWE3cxuNbNDZvaemT1YRg/VmNlhM3vLzA6UPT5dNobeMTM7OGZam5m9aGbvZs/jjrFXUm8Pmdlgtu4OmNntJfU2x8z2mNmAmb1tZquy6aWuu0RfTVlvTT9mN7Mpkv4oaamkI5JeldTl7gNNbaQKMzssaZG7l/4DDDP7B0mnJG0+N7SWmf2LpOPuvi77j3Kmu/+8RXp7SOc5jHeDeqs2zPhPVOK6K3L48zzK2LIvlvSeu7/v7mck/VbSshL6aHnuvlfS8a9MXiZpU/Z6kyr/WJquSm8twd2H3H1/9vqkpHPDjJe67hJ9NUUZYZ8t6U9j3h9Ra4337pJ2mtlrZtZddjPjaB8zzNaHktrLbGYcNYfxbqavDDPeMusuz/Dn9eIE3dctcfe/lXSbpJXZ7mpL8soxWCtdO/2lpPmqjAE4JGl9mc1kw4w/K+ln7n5ibK3MdTdOX01Zb2WEfVDSnDHvv51NawnuPpg9H5P0vCqHHa3k6LkRdLPnYyX38yV3P+ruZ919VNKvVOK6y4YZf1bSb9z9uWxy6etuvL6atd7KCPurkq40s++Y2VRJP5K0rYQ+vsbMpmUnTmRm0yT9QK03FPU2Scuz18slbS2xl7/QKsN4VxtmXCWvu9KHP3f3pj8k3a7KGfn/lfRPZfRQpa95kt7IHm+X3ZukLars1v2fKuc2Vkj6a0m7JL0r6T8ltbVQb/+uytDeb6oSrFkl9bZElV30NyUdyB63l73uEn01Zb3xc1kgCE7QAUEQdiAIwg4EQdiBIAg7EARhB4Ig7EAQ/w8ie3GmjcGk5QAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# フォルダ「data」の下にフォルダ「img_78」を作成する\n",
        "data_dir_path = \"./data/img_78/\"\n",
        "if not os.path.exists(data_dir_path):\n",
        "    os.mkdir(data_dir_path)"
      ],
      "metadata": {
        "id": "qshbUen8PtB4"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# MNISTから数字7、8の画像だけフォルダ「img_78」に画像として保存していく\n",
        "count7=0\n",
        "count8=0\n",
        "max_num=200  # 画像は200枚ずつ作成する\n",
        "\n",
        "for i in range(len(X)):\n",
        "    \n",
        "    # 画像7の作成\n",
        "    if (y[i] is \"7\") and (count7<max_num):\n",
        "        file_path=\"./data/img_78/img_7_\"+str(count7)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28×28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f = pil_img_f.resize((64, 64), Image.BICUBIC)  # 64×64に拡大\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count7+=1 \n",
        "    \n",
        "    # 画像8の作成\n",
        "    if (y[i] is \"8\") and (count8<max_num):\n",
        "        file_path=\"./data/img_78/img_8_\"+str(count8)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28*28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f = pil_img_f.resize((64, 64), Image.BICUBIC)  # 64×64に拡大\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count8+=1\n",
        "        \n",
        "    # 7と8を200枚ずつ作成したらbreak\n",
        "    if (count7>=max_num) and (count8>=max_num):\n",
        "        break"
      ],
      "metadata": {
        "id": "ZBE3rGybPyvu"
      },
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# フォルダ「data」の下にフォルダ「test」を作成する\n",
        "data_dir_path = \"./data/test/\"\n",
        "if not os.path.exists(data_dir_path):\n",
        "    os.mkdir(data_dir_path)"
      ],
      "metadata": {
        "id": "jvdbs-oxPzZ_"
      },
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 上記で7,8の画像を作成するのに使用したindexの最終値\n",
        "i_start = i+1\n",
        "print(i_start)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B8_PINgvP3Am",
        "outputId": "45b9b272-fd84-4a85-cbf6-33750b2185a7"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2244\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# MNISTから数字7、8の画像だけフォルダ「img_78」に画像として保存していく\n",
        "count2=0\n",
        "count7=0\n",
        "count8=0\n",
        "max_num=5  # 画像は5枚ずつ作成する\n",
        "\n",
        "for i in range(i_start,len(X)):  # i_startから始める\n",
        "    \n",
        "    # 画像2の作成\n",
        "    if (y[i] is \"2\") and (count2<max_num):\n",
        "        file_path=\"./data/test/img_2_\"+str(count2)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28×28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f = pil_img_f.resize((64, 64), Image.BICUBIC)  # 64×64に拡大\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count2+=1\n",
        "    \n",
        "    # 画像7の作成\n",
        "    if (y[i] is \"7\") and (count7<max_num):\n",
        "        file_path=\"./data/test/img_7_\"+str(count7)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28×28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f = pil_img_f.resize((64, 64), Image.BICUBIC)  # 64×64に拡大\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count7+=1 \n",
        "    \n",
        "    # 画像8の作成\n",
        "    if (y[i] is \"8\") and (count8<max_num):\n",
        "        file_path=\"./data/test/img_8_\"+str(count8)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28*28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f = pil_img_f.resize((64, 64), Image.BICUBIC)  # 64×64に拡大\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count8+=1 "
      ],
      "metadata": {
        "id": "DbgeNSKVP41U"
      },
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# フォルダ「data」の下にフォルダ「img_78_28size」を作成する\n",
        "data_dir_path = \"./data/img_78_28size/\"\n",
        "if not os.path.exists(data_dir_path):\n",
        "    os.mkdir(data_dir_path)"
      ],
      "metadata": {
        "id": "PK0QTeImP7os"
      },
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# MNISTから数字7、8の画像だけフォルダ「img_78_28size」に画像として保存していく\n",
        "count7=0\n",
        "count8=0\n",
        "max_num=200  # 画像は200枚ずつ作成する\n",
        "\n",
        "for i in range(len(X)):\n",
        "    \n",
        "    # 画像7の作成\n",
        "    if (y[i] is \"7\") and (count7<max_num):\n",
        "        file_path=\"./data/img_78_28size/img_7_\"+str(count7)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28×28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count7+=1 \n",
        "    \n",
        "    # 画像8の作成\n",
        "    if (y[i] is \"8\") and (count8<max_num):\n",
        "        file_path=\"./data/img_78_28size/img_8_\"+str(count8)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28*28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count8+=1\n",
        "    \n",
        "    if (count7>=max_num) and (count8>=max_num):\n",
        "        break"
      ],
      "metadata": {
        "id": "9_D7YAJCP-Bo"
      },
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# フォルダ「data」の下にフォルダ「test」を作成する\n",
        "data_dir_path = \"./data/test_28size/\"\n",
        "if not os.path.exists(data_dir_path):\n",
        "    os.mkdir(data_dir_path)\n",
        "\n",
        "# 上記で7,8の画像を作成するのに使用したindexの最終値\n",
        "i_start = i+1\n",
        "print(i_start)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iwZ0X8AdQAXH",
        "outputId": "50aef33a-027d-4f93-cd44-5471f1498158"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2244\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# MNISTから数字7、8の画像だけフォルダ「img_78」に画像として保存していく\n",
        "count2=0\n",
        "count7=0\n",
        "count8=0\n",
        "max_num=5  # 画像は5枚ずつ作成する\n",
        "\n",
        "for i in range(i_start,len(X)):  # i_startから始める\n",
        "    \n",
        "    # 画像2の作成\n",
        "    if (y[i] is \"2\") and (count2<max_num):\n",
        "        file_path=\"./data/test_28size/img_2_\"+str(count2)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28×28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count2+=1 \n",
        "    \n",
        "    # 画像7の作成\n",
        "    if (y[i] is \"7\") and (count7<max_num):\n",
        "        file_path=\"./data/test_28size/img_7_\"+str(count7)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28×28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count7+=1 \n",
        "    \n",
        "    # 画像8の作成\n",
        "    if (y[i] is \"8\") and (count8<max_num):\n",
        "        file_path=\"./data/test_28size/img_8_\"+str(count8)+\".jpg\"\n",
        "        im_f=(X[i].reshape(28, 28))  # 画像を28*28の形に変形\n",
        "        pil_img_f = Image.fromarray(im_f.astype(np.uint8))  # 画像をPILに\n",
        "        pil_img_f.save(file_path)  # 保存\n",
        "        count8+=1 "
      ],
      "metadata": {
        "id": "T8Oof9PmQCjH"
      },
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vlfVYAe2L8KR"
      },
      "source": [
        "# Generatorの実装\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "68sd3hevL8KS"
      },
      "source": [
        "class Generator(nn.Module):\n",
        "\n",
        "    def __init__(self, z_dim=20):\n",
        "        super(Generator, self).__init__()\n",
        "\n",
        "        self.layer1 = nn.Sequential(\n",
        "            nn.Linear(z_dim, 1024),\n",
        "            nn.BatchNorm1d(1024),\n",
        "            nn.ReLU(inplace=True))\n",
        "\n",
        "        self.layer2 = nn.Sequential(\n",
        "            nn.Linear(1024, 7*7*128),\n",
        "            nn.BatchNorm1d(7*7*128),\n",
        "            nn.ReLU(inplace=True))\n",
        "\n",
        "        self.layer3 = nn.Sequential(\n",
        "            nn.ConvTranspose2d(in_channels=128, out_channels=64,\n",
        "                               kernel_size=4, stride=2, padding=1),\n",
        "            nn.BatchNorm2d(64),\n",
        "            nn.ReLU(inplace=True))\n",
        "\n",
        "        self.last = nn.Sequential(\n",
        "            nn.ConvTranspose2d(in_channels=64, out_channels=1,\n",
        "                               kernel_size=4, stride=2, padding=1),\n",
        "            nn.Tanh())\n",
        "        # 注意：白黒画像なので出力チャネルは1つだけ\n",
        "\n",
        "    def forward(self, z):\n",
        "        out = self.layer1(z)\n",
        "        out = self.layer2(out)\n",
        "\n",
        "        # 転置畳み込み層に入れるためにテンソルの形を整形\n",
        "        out = out.view(z.shape[0], 128, 7, 7)\n",
        "        out = self.layer3(out)\n",
        "        out = self.last(out)\n",
        "\n",
        "        return out\n"
      ],
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zWygUNjSL8KU",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "outputId": "04b3c0fc-6798-469d-85cc-4ec61d8eee77"
      },
      "source": [
        "# 動作確認\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "G = Generator(z_dim=20)\n",
        "G.train()\n",
        "\n",
        "# 入力する乱数\n",
        "# バッチノーマライゼーションがあるのでミニバッチ数は2以上\n",
        "input_z = torch.randn(2, 20)\n",
        "\n",
        "# 偽画像を出力\n",
        "fake_images = G(input_z)  # torch.Size([2, 1, 28, 28])\n",
        "img_transformed = fake_images[0][0].detach().numpy()\n",
        "plt.imshow(img_transformed, 'gray')\n",
        "plt.show()\n"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAZmUlEQVR4nO2de3DU1fnGn1cE5R4IEFNAQbwjBWxqcYwIUqgiCiogtFW0KGqBikIV0EErtkUHQVpERWEA609RkYEOVEFEaZURIo0QrgEqN8NFuQXlInB+f2TpoM15Ds1lN9PzfGaYhP3wZg9LHnaz53ve15xzEEL873NaqhcghEgOCrsQkaCwCxEJCrsQkaCwCxEJpyfzzqpVq+bS0tK8/qyzzqL1hw4d8rr8/Hxa26BBA+ozMzOp3759u9cdP36c1h45coT6xo0bU//ll19Sz3ZUQmsLPS7Hjh2jfu/evdQfPnzY60J/79WrV1N/4YUXUr9z506vS09Pp7VbtmyhvmbNmtTv2rWL+vr163td6N+b5WTXrl0oLCy04lypwm5m1wIYB6ASgJedc6PYn09LS8Ndd93l9UOHDqX3t3btWq/r1KkTre3fvz/1Dz/8MPVjxozxuv3799Pabdu2Uf/ss89SP3nyZOrZfyaFhYW0duDAgdTv27eP+lmzZlG/adMmrxs7diyt/fGPf0z9woULqR8/frzX3XbbbbR20KBB1Ldv3576iRMnUn/33Xd73ZQpU2jt4MGDvW7EiBFeV+KX8WZWCcBzAK4DcAmA3mZ2SUm/nhCifCnNz+yXA1jvnNvonDsC4HUAXctmWUKIsqY0YW8I4OQfbLYmbvsOZtbPzHLMLOebb74pxd0JIUpDub8b75yb6JzLcs5lVatWrbzvTgjhoTRh3wbg5LdTGyVuE0JUQEoT9qUAzjezpmZWBUAvALPLZllCiLKmxFtvzrmjZjYAwLso2nqb7JxbyWpq1KiB7Oxsrz948CC9T7bvumfPHlob2i+uXLky9XPmzPG6P//5z7R29OjR1GdkZFDfoUMH6q+55hqvu/TSS2ntO++8Q/3IkSOpP/PMM6n/zW9+43UbN26ktaF99quuuor6Nm3aeF3ouoqtW7dSH9oWrFu3LvWjRvl3qUN7+Gxrju3Rl2qf3Tk3F8Dc0nwNIURy0OWyQkSCwi5EJCjsQkSCwi5EJCjsQkSCwi5EJCT1PHvVqlXRsmVLrw/tTbJjhW+//Tatffrpp6kP7auy8+xVqlShtaHz7EuXLqU+dO77zjvv9DrWPwAI7xe///771F922WXUs73yX/ziF6W679B59nvvvdfrnnrqKVobOm6dlZVFfefOnalfvnx5iWsXLFjgdew6Fj2zCxEJCrsQkaCwCxEJCrsQkaCwCxEJCrsQkZDUrbdjx45h9+7dXj9t2jRazzrAhlpePfTQQ9SzrrcAMHz4cK9jXW8BYPHixdQvWrSI+jfffJP6nj17et3s2bzFQKVKlagPHTPt0aMH9fPnz/e63NxcWrtixQrq2RYUANSuXdvr1q1bR2snTZpEPdvuBMKPe9u2bb2uSZMmtJYdYz169KjX6ZldiEhQ2IWIBIVdiEhQ2IWIBIVdiEhQ2IWIBIVdiEgwNu63rElLS3NXX32118+cOZPWs3bQjRo1KvG6AGDz5s3UsxG89erVo7WhCbHjxo2j/uWXX6b+vPPO87rQUc6//e1v1O/YsYP66dOnU8+uIXjrrbdo7ZIlS6gPHc9l1z9cdNFFtJbtVwPAmjVrqK9Vqxb17NhyqDV506ZNve7BBx9Efn5+sSOb9cwuRCQo7EJEgsIuRCQo7EJEgsIuRCQo7EJEgsIuRCQk9Tx79erV8aMf/cjr8/PzaX23bt28ju2DA+F2zKHRxmzfdd68ebR21qxZ1LOWxwAwaNAg6keMGOF1oX320Hjg0Hn2kJ8xY4bXVa1aldaG9qqrVatG/YQJE7yuXbt2tDbUmnzLli3U9+rVi3o2rnrlSjr5HGPHjvW6L774wutKFXYz+xxAIYBjAI4653gzbSFEyiiLZ/b2zjl/6wwhRIVAP7MLEQmlDbsDMM/MPjWzfsX9ATPrZ2Y5Zpbz9ddfl/LuhBAlpbQv47Odc9vMrAGA+Wa2xjn3nZMPzrmJACYCQMOGDZN36kYI8R1K9czunNuW+LgTwEwAl5fFooQQZU+Jw25m1c2s5onPAXQCkFdWCxNClC2leRmfAWCmmZ34Ov/nnHuHFdSvXx/9+/f3+vT0dHqHbLRxjRo1aG316tWpZ724Ab4nHNrjnzhxIvV79+6lfv/+/dQ/8MAD1DOGDBlCfcOGDakPjbpmFBQUUL9+/XrqQ9dOLFu2zOt++9vf0trQKOpQz/vQDIQ9e/Z43fjx42kty8GvfvUrrytx2J1zGwH4h60LISoU2noTIhIUdiEiQWEXIhIUdiEiQWEXIhKS2kq6QYMGrnv37l7PxiIDwJVXXul1mzZtorWnn843Hi6++GLq2bHD0FHM0Lbet99+S/0tt9xCfatWrbzuq6++orXLly8vlWdHKgGgbt26Ja4NtQefM2cO9WzL88iRI7S2b9++1Ie2W5s3b049GwnNvs8B4Pzzz/e6u+66C2vWrFEraSFiRmEXIhIUdiEiQWEXIhIUdiEiQWEXIhIUdiEiIamtpNPS0tC1a1evD+2rsiOLAwcOpLVsvxcI7/E3a9bM69g+NwD89Kc/pf6DDz6gfu7cudS3bt3a60LjnkNHOUOEjueeccYZXpeRkUFrQ0d3S/O4/vrXv6a1Bw4coD70uIXGSW/fvt3rVq1aRWsTx8qLhV1Pomd2ISJBYRciEhR2ISJBYRciEhR2ISJBYRciEhR2ISIhqfvs+/fvx4IFC7yendMF+HjhCy64gNbWqVOH+uuvv576q6++2utC++AfffQR9evWraO+RYsW1LM941DtwYMHqW/fvj317777LvWsxffu3btp7XPPPUf9Cy+8QP0999zjdWvXrqW17HoQINyjIDs7m3q2tp49e9Jadl2H9tmFEAq7ELGgsAsRCQq7EJGgsAsRCQq7EJGgsAsRCUndZ8/MzMQjjzzi9aGxymwPccCAAbT2k08+ob5atWrUT58+3evY+WIAOHbsGPXszDcAvPLKK9RXqlTJ6zZs2EBrmzZtSn3oGoFDhw5Rf++993pdaJR1aFT1e++9R33v3r29rmPHjrQ2dA3AmjVrqL/22mupZ33pQ98v7O/9zTffeF3wmd3MJpvZTjPLO+m2umY238zyEx/5FStCiJRzKi/jpwD4/n9TQwEscM6dD2BB4vdCiApMMOzOuUUAvv+apiuAqYnPpwLoVsbrEkKUMSV9gy7DOVeQ+Hw7AG8zMTPrZ2Y5ZpYTmjsmhCg/Sv1uvCuaDOmdDumcm+icy3LOZaWnp5f27oQQJaSkYd9hZpkAkPi4s+yWJIQoD0oa9tkA+iQ+7wNgVtksRwhRXgT32c3sNQDtANQzs60AHgMwCsAbZtYXwCYA/ABugnXr1tFe3/Pnz6f17Lx7qE/3yJEjqb/pppuoZ/uyt956K60N9SAP7YWHzupfeumlXlelShVay/oLAECXLl2o37p1K/XDhg3zuj179tDaJk2aUM9mnAO8p32o13+DBg2of+ihh6hv06YN9azXf6i2T58+XseuuQiG3TnnuzKhQ6hWCFFx0OWyQkSCwi5EJCjsQkSCwi5EJCjsQkRCUo+4Nm7cGGPGjPH62rVr03o25jZ0KW5oC2rmzJnUHz9+3Ot27uTXFIXaEv/1r3+lvnLlytTPmzfP69avX09rzz33XOpDR4OnTp1K/ejRo71uy5YttPbMM8+kvmXLliWuD22tjR8/nvrQv0mPHj2oZ9+vEyZMoLXs2PCmTZu8Ts/sQkSCwi5EJCjsQkSCwi5EJCjsQkSCwi5EJCjsQkSCFTWaSQ516tRxHTr4D8uxNrgA8Mwzz3jdFVdcQWtDX3voUN4zc8SIEV531VVX0dobb7yR+gceeID6zp07U89aLv/zn/+ktR988AH1nTp1oj43N5f6IUOGeN3PfvazEtcC4X9Ttpe+bds2Wnv22WdTz46SAkDbtm2pHzRokNexI8sAP1bcvXt35OXlFdvbXM/sQkSCwi5EJCjsQkSCwi5EJCjsQkSCwi5EJCjsQkRCUvfZmzdv7l577TX/YgKjjy+55BKvO3jwIK09fPgw9aHzybVq1fK6p59+mta++eab1C9dupT6b7/9lvqjR496XWi/9+9//zv17du3p76goIB69riG2nv36tWL+lCfgLPOOsvrWDtmINxqOvS9GuojwPor7Nu3j9ayaz4WL16Mffv2aZ9diJhR2IWIBIVdiEhQ2IWIBIVdiEhQ2IWIBIVdiEhIat/4zZs3o3///l4/Z84cWp+enu51oT3Zt99+m/qVK1dS36JFC69jfycA+Pjjj6m/7777qA+tnfWNv+eee2hts2bNqA+NVZ41axb1rHf7T37yE1rbsGFD6levXk393Xff7XWPPvoorc3Ly6OeXS8CAJdddhn1gwcP9rrQHALW/2Djxo1eF3xmN7PJZrbTzPJOuu1xM9tmZrmJX7y7ghAi5ZzKy/gpAK4t5vaxzrlWiV9zy3ZZQoiyJhh259wiALuTsBYhRDlSmjfoBpjZ8sTL/Dq+P2Rm/cwsx8xyQtd4CyHKj5KG/XkAzQC0AlAAwNsJ0jk30TmX5ZzLCh02EUKUHyUKu3Nuh3PumHPuOICXAFxetssSQpQ1JQq7mWWe9NubAPB9CiFEygnus5vZawDaAahnZlsBPAagnZm1AuAAfA6Ab+YmqFmzJq655hr/Yk7ny2GzvkPvB4waNYr6tLQ06gsLC70udDb6d7/7HfXPP/889X/4wx+oz8jI8LrmzZvT2tmzZ1N/4YUXUh/qcU73fU/jzzWsPzoA3H777dRfcMEFXvf666/T2lD/A3YeHQD9PgeAhQsXet3u3fz98F27dpVoXcGwO+d6F3PzpFCdEKJioctlhYgEhV2ISFDYhYgEhV2ISFDYhYiEpB5x3b17N93yYOOcASAnJ8frXnzxRVrLRuQCwJYtW6h//PHHve6GG26gtaHtqezsbOrZqGqAj5sOjWxevHgx9W3atKG+b9++1G/fvt3rNm/eTGsHDhxI/dy5/PwVa6Mdatc8fPhw6seNG0f9hx9+SD173EJHmteuXet1bJS0ntmFiASFXYhIUNiFiASFXYhIUNiFiASFXYhIUNiFiISk7rNfdNFFWLBggddXrVqV1rN909Bo4hUrVlAfOuJ6xx13eB0boQsAnTvz5rtPPvkk9aGxyKzd85IlS2htly5dqA8dHR4/fjz1bC89tIcfGmUdGtO9bt06r3vuuedoLRv3DAB16ng7sQEItxcfMmSI14XaVLM9+k2bNnmdntmFiASFXYhIUNiFiASFXYhIUNiFiASFXYhIUNiFiARzziXtzurXr++6devGPK2fPHmy17300ku0dtq0adQ3atSIetYauGPHjrT2Bz/4AfX5+fnUs7bDAHD8+HGvu/POO2kt25cFwmOTjx07Rn3r1q29bvr06bQ21MZ62LBh1D/77LNed+jQIVobui6jXr161IdGNh84cMDrQpOTFi1a5HU333wz8vLyrDinZ3YhIkFhFyISFHYhIkFhFyISFHYhIkFhFyISFHYhIiGp59kzMzPp2e/Qnu6AAQO8rnr16rT2lltuof7IkSPUs571eXl8PH2oR3l6ejr1WVlZ1O/du9frXnnlFVob2i++8cYbqQ+dSWdns88991xay3rOA/xMOADUqlXL6z755BNa27RpU+pDY5XZiO9QfWhEd+jaBh/BZ3Yza2xmC81slZmtNLP7E7fXNbP5Zpaf+MhP8wshUsqpvIw/CmCwc+4SAG0A9DezSwAMBbDAOXc+gAWJ3wshKijBsDvnCpxzyxKfFwJYDaAhgK4Apib+2FQA/utghRAp5796g87MmgBoDeATABnOuRPN0bYDyPDU9DOzHDPLCf2cI4QoP0457GZWA8AMAIOcc/tPdq7oNE2xJ2qccxOdc1nOuay6deuWarFCiJJzSmE3s8ooCvqrzrkTIyZ3mFlmwmcC2Fk+SxRClAXBrTczMwCTAKx2zo05Sc0G0AfAqMTHWaGvdeDAAXz00UdeHxpt3KNHD68L/YhQpUoV6t9//33qX3jhBa8Ltf4N3Tc7PgsAX3zxBfVsdPFbb71Fa8855xzqX331Ver/8pe/UM9aSf/pT3+itWyrFQAmTJhAfbVq1byud+/etJaNPgaAli1bUh86nrthwwavCx3HHjx4sNedccYZXncq++xXArgNwAozy03cNhxFIX/DzPoC2ASg5yl8LSFEigiG3Tn3DwDFHoYH0KFslyOEKC90uawQkaCwCxEJCrsQkaCwCxEJCrsQkZDUVtJZWVmOHS087TT+f8/+/fu9rl+/frSWjTUGgHfffZf6ossNiueGG26gtaHxvqHRw6NHj6aeHYlk7ZQB4IknnqA+NG764osvpp5d/3D99dfT2lWrVlH/8MMPUz9z5kyv+/DDD2lt6Mhz6BqAzz77jHrW4vuHP/whrWXXfHz66acoLCxUK2khYkZhFyISFHYhIkFhFyISFHYhIkFhFyISFHYhIiGp++zNmjVzo0aN8vq1a9fS+hkzZnhdaF8zdO66bdu21LN2zqE915EjR1LP2lQD4bPTP//5z70u1BI5I6PYbmL/JjQ+OHQNQJMmTbxu+fLltDbU2Sg0pps9Luy6CQD46quvqA+dOWejyQHg448/9rqhQ3nv1lmz/K0jOnTogNzcXO2zCxEzCrsQkaCwCxEJCrsQkaCwCxEJCrsQkaCwCxEJSR3ZbGa0H/eDDz5I64cNG+Z1X3/9Na0N7Qffeuut1BcUFHhd6L5ffPFF6lu0aEH9o48+Sv2aNWu8LrRHf/PNN1MfGpsc+jdjY5l37NhBaw8dOkT9L3/5S+rZNQah8+xTpkyhPjs7m/rMzEzqc3NzvY71hQeA1atXex17zPTMLkQkKOxCRILCLkQkKOxCRILCLkQkKOxCRILCLkQknMp89sYApgHIAOAATHTOjTOzxwHcDWBX4o8Od875B4UDqFq1Klq3bu31kydPpmth5907duxIaxs2bEh9qH7x4sVet2TJElob6r3+5ZdfUv/73/+eejZbvnv37rS2UaNG1N93333UL1u2jPouXbp4Xag3e9WqVak/fPgw9WxWeejahTFjxlAfOucf+ruxvg6hXvzs2gm2rlO5qOYogMHOuWVmVhPAp2Y2P+HGOuf41SpCiArBqcxnLwBQkPi80MxWA+BPk0KICsd/9TO7mTUB0BrAiRlOA8xsuZlNNrNiZxyZWT8zyzGzHDYKSAhRvpxy2M2sBoAZAAY55/YDeB5AMwCtUPTM/0xxdc65ic65LOdcVqinmBCi/DilsJtZZRQF/VXn3NsA4Jzb4Zw75pw7DuAlAJeX3zKFEKUlGHYrasM5CcBq59yYk24/+VjPTQDyyn55Qoiy4lTejb8SwG0AVpjZiXN5wwH0NrNWKNqO+xzAPaVdzNKlS6lnxw7PPvtsWhvazmjXrh31bMRuaCTz+PHjqQ9t47BtP4AfiXzyySdp7RtvvEF9aFtxwoQJ1LOjw6GvHdrWu+OOO6jfvHmz13Xq1InWVq9enfrQmO6nnnqKerZtuGHDBlqbl+d/XmXjv0/l3fh/ACiuDzXdUxdCVCx0BZ0QkaCwCxEJCrsQkaCwCxEJCrsQkaCwCxEJSW0lfeTIEfzrX//y+lAL3T59+njd8ePHaS0buQwAPXr0oP6KK67wuubNm9Pa8847j3rWIhsAbr/9dupZ2+LatWvT2uuuu4760NHfUCvpP/7xj16XlpZGa9kxUAC4//77qX/ssce87oknnqC1w4cPpz40knnSpEnUn3aa/3l25syZtJblhI1g1zO7EJGgsAsRCQq7EJGgsAsRCQq7EJGgsAsRCQq7EJFgbF+uzO/MbBeATSfdVA8A76OcOirq2irqugCtraSU5drOcc7VL04kNez/cedmOc45frVLiqioa6uo6wK0tpKSrLXpZbwQkaCwCxEJqQ77xBTfP6Oirq2irgvQ2kpKUtaW0p/ZhRDJI9XP7EKIJKGwCxEJKQm7mV1rZmvNbL2ZDU3FGnyY2edmtsLMcs0sJ8VrmWxmO80s76Tb6prZfDPLT3zkTeuTu7bHzWxb4rHLNTM+q7r81tbYzBaa2SozW2lm9yduT+ljR9aVlMct6T+zm1klAOsAdASwFcBSAL2dc6uSuhAPZvY5gCznXMovwDCztgAOAJjmnLs0cdvTAHY750Yl/qOs45x7uIKs7XEAB1I9xjsxrSjz5DHjALoBuAMpfOzIunoiCY9bKp7ZLwew3jm30Tl3BMDrALqmYB0VHufcIgDfH33bFcDUxOdTUfTNknQ8a6sQOOcKnHPLEp8XAjgxZjyljx1ZV1JIRdgbAthy0u+3omLNe3cA5pnZp2bWL9WLKYYM51xB4vPtADJSuZhiCI7xTibfGzNeYR67kow/Ly16g+4/yXbOXQbgOgD9Ey9XKySu6GewirR3ekpjvJNFMWPG/00qH7uSjj8vLakI+zYAjU/6faPEbRUC59y2xMedAGai4o2i3nFigm7i484Ur+ffVKQx3sWNGUcFeOxSOf48FWFfCuB8M2tqZlUA9AIwOwXr+A/MrHrijROYWXUAnVDxRlHPBnCizW4fALNSuJbvUFHGePvGjCPFj13Kx58755L+C0BnFL0jvwHAI6lYg2dd5wL4LPFrZarXBuA1FL2s+xZF7230BZAOYAGAfADvAahbgdb2CoAVAJajKFiZKVpbNopeoi8HkJv41TnVjx1ZV1IeN10uK0Qk6A06ISJBYRciEhR2ISJBYRciEhR2ISJBYRciEhR2ISLh/wFNt3maRCfMeAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YIyWxLJPL8KW"
      },
      "source": [
        "# Discriminatorの実装"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ug0Iof6uL8KX"
      },
      "source": [
        "class Discriminator(nn.Module):\n",
        "\n",
        "    def __init__(self, z_dim=20):\n",
        "        super(Discriminator, self).__init__()\n",
        "\n",
        "        # 画像側の入力処理\n",
        "        self.x_layer1 = nn.Sequential(\n",
        "            nn.Conv2d(1, 64, kernel_size=4,\n",
        "                      stride=2, padding=1),\n",
        "            nn.LeakyReLU(0.1, inplace=True))\n",
        "        # 注意：白黒画像なので入力チャネルは1つだけ\n",
        "\n",
        "        self.x_layer2 = nn.Sequential(\n",
        "            nn.Conv2d(64, 64, kernel_size=4,\n",
        "                      stride=2, padding=1),\n",
        "            nn.BatchNorm2d(64),\n",
        "            nn.LeakyReLU(0.1, inplace=True))\n",
        "\n",
        "        # 乱数側の入力処理\n",
        "        self.z_layer1 = nn.Linear(z_dim, 512)\n",
        "\n",
        "        # 最後の判定\n",
        "        self.last1 = nn.Sequential(\n",
        "            nn.Linear(3648, 1024),\n",
        "            nn.LeakyReLU(0.1, inplace=True))\n",
        "\n",
        "        self.last2 = nn.Linear(1024, 1)\n",
        "\n",
        "    def forward(self, x, z):\n",
        "\n",
        "        # 画像側の入力処理\n",
        "        x_out = self.x_layer1(x)\n",
        "        x_out = self.x_layer2(x_out)\n",
        "\n",
        "        # 乱数側の入力処理\n",
        "        z = z.view(z.shape[0], -1)\n",
        "        z_out = self.z_layer1(z)\n",
        "\n",
        "        # x_outとz_outを結合し、全結合層で判定\n",
        "        x_out = x_out.view(-1, 64 * 7 * 7)\n",
        "        out = torch.cat([x_out, z_out], dim=1)\n",
        "        out = self.last1(out)\n",
        "\n",
        "        feature = out  # 最後にチャネルを1つに集約する手前の情報\n",
        "        feature = feature.view(feature.size()[0], -1)  # 2次元に変換\n",
        "\n",
        "        out = self.last2(out)\n",
        "\n",
        "        return out, feature\n"
      ],
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fsKNLk4BL8Ka",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3c6acbea-46a0-4b0e-b554-99a0e9e5da0d"
      },
      "source": [
        "# 動作確認\n",
        "D = Discriminator(z_dim=20)\n",
        "\n",
        "# 偽画像を生成\n",
        "input_z = torch.randn(2, 20)\n",
        "fake_images = G(input_z)\n",
        "\n",
        "# 偽画像をDに入力\n",
        "d_out, _ = D(fake_images, input_z)\n",
        "\n",
        "# 出力d_outにSigmoidをかけて0から1に変換\n",
        "print(nn.Sigmoid()(d_out))\n"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[0.4322],\n",
            "        [0.4629]], grad_fn=<SigmoidBackward0>)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YlRmvebQL8Kd"
      },
      "source": [
        "# Encoderの実装\n",
        "\n",
        "画像をzに変換する"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ih6gWabsL8Kd"
      },
      "source": [
        "class Encoder(nn.Module):\n",
        "\n",
        "    def __init__(self, z_dim=20):\n",
        "        super(Encoder, self).__init__()\n",
        "\n",
        "        self.layer1 = nn.Sequential(\n",
        "            nn.Conv2d(1, 32, kernel_size=3,\n",
        "                      stride=1),\n",
        "            nn.LeakyReLU(0.1, inplace=True))\n",
        "        # 注意：白黒画像なので入力チャネルは1つだけ\n",
        "\n",
        "        self.layer2 = nn.Sequential(\n",
        "            nn.Conv2d(32, 64, kernel_size=3,\n",
        "                      stride=2, padding=1),\n",
        "            nn.BatchNorm2d(64),\n",
        "            nn.LeakyReLU(0.1, inplace=True))\n",
        "\n",
        "        self.layer3 = nn.Sequential(\n",
        "            nn.Conv2d(64, 128, kernel_size=3,\n",
        "                      stride=2, padding=1),\n",
        "            nn.BatchNorm2d(128),\n",
        "            nn.LeakyReLU(0.1, inplace=True))\n",
        "\n",
        "        # ここまでで画像のサイズは7×7になっている\n",
        "        self.last = nn.Linear(128 * 7 * 7, z_dim)\n",
        "\n",
        "    def forward(self, x):\n",
        "        out = self.layer1(x)\n",
        "        out = self.layer2(out)\n",
        "        out = self.layer3(out)\n",
        "\n",
        "        # FCに入れるためにテンソルの形を整形\n",
        "        out = out.view(-1, 128 * 7 * 7)\n",
        "        out = self.last(out)\n",
        "\n",
        "        return out\n"
      ],
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vun_KvShL8Kh",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7b6f97ee-22a8-4324-873b-79f2609950bf"
      },
      "source": [
        "# 動作確認\n",
        "E = Encoder(z_dim=20)\n",
        "\n",
        "# 入力する画像データ\n",
        "x = fake_images  # fake_imagesは上のGで作成したもの\n",
        "\n",
        "# 画像からzをEncode\n",
        "z = E(x)\n",
        "\n",
        "print(z.shape)\n",
        "print(z)\n"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([2, 20])\n",
            "tensor([[-0.0669, -0.2844, -0.3601,  0.1404, -0.2922, -0.4834, -0.0025,  0.0258,\n",
            "         -0.1298, -0.2613, -0.4292, -0.2584, -0.6141, -0.1440,  0.1142,  0.2175,\n",
            "          0.0730,  0.0323, -0.7176, -0.7410],\n",
            "        [ 0.4822,  0.3681, -0.0647, -0.0773,  0.0989,  0.3012,  0.8054,  0.4399,\n",
            "          1.0848, -0.2809,  0.0619, -0.3179, -0.2963, -0.1192,  0.6562, -0.1707,\n",
            "          0.1287, -0.6156,  0.3127, -0.4819]], grad_fn=<AddmmBackward0>)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BnwXk0a-L8Kl"
      },
      "source": [
        "# DataLoaderの作成"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8lXhlnQKL8Kl"
      },
      "source": [
        "def make_datapath_list():\n",
        "    \"\"\"学習、検証の画像データとアノテーションデータへのファイルパスリストを作成する。 \"\"\"\n",
        "\n",
        "    train_img_list = list()  # 画像ファイルパスを格納\n",
        "\n",
        "    for img_idx in range(200):\n",
        "        img_path = \"./data/img_78_28size/img_7_\" + str(img_idx)+'.jpg'\n",
        "        train_img_list.append(img_path)\n",
        "\n",
        "        img_path = \"./data/img_78_28size/img_8_\" + str(img_idx)+'.jpg'\n",
        "        train_img_list.append(img_path)\n",
        "\n",
        "    return train_img_list\n"
      ],
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ffLPO9nJL8Kn"
      },
      "source": [
        "class ImageTransform():\n",
        "    \"\"\"画像の前処理クラス\"\"\"\n",
        "\n",
        "    def __init__(self, mean, std):\n",
        "        self.data_transform = transforms.Compose([\n",
        "            transforms.ToTensor(),\n",
        "            transforms.Normalize(mean, std)\n",
        "        ])\n",
        "\n",
        "    def __call__(self, img):\n",
        "        return self.data_transform(img)\n"
      ],
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pc4oS_UML8Kq"
      },
      "source": [
        "class GAN_Img_Dataset(data.Dataset):\n",
        "    \"\"\"画像のDatasetクラス。PyTorchのDatasetクラスを継承\"\"\"\n",
        "\n",
        "    def __init__(self, file_list, transform):\n",
        "        self.file_list = file_list\n",
        "        self.transform = transform\n",
        "\n",
        "    def __len__(self):\n",
        "        '''画像の枚数を返す'''\n",
        "        return len(self.file_list)\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        '''前処理をした画像のTensor形式のデータを取得'''\n",
        "\n",
        "        img_path = self.file_list[index]\n",
        "        img = Image.open(img_path)  # [高さ][幅]白黒\n",
        "\n",
        "        # 画像の前処理\n",
        "        img_transformed = self.transform(img)\n",
        "\n",
        "        return img_transformed\n"
      ],
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IfBEgPZUL8Ks",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "28834ce3-88bb-4b7f-a4a7-2de29f307f83"
      },
      "source": [
        "# DataLoaderの作成と動作確認\n",
        "\n",
        "# ファイルリストを作成\n",
        "train_img_list=make_datapath_list()\n",
        "\n",
        "# Datasetを作成\n",
        "mean = (0.5,)\n",
        "std = (0.5,)\n",
        "train_dataset = GAN_Img_Dataset(\n",
        "    file_list=train_img_list, transform=ImageTransform(mean, std))\n",
        "\n",
        "# DataLoaderを作成\n",
        "batch_size = 64\n",
        "\n",
        "train_dataloader = torch.utils.data.DataLoader(\n",
        "    train_dataset, batch_size=batch_size, shuffle=True)\n",
        "\n",
        "# 動作の確認\n",
        "batch_iterator = iter(train_dataloader)  # イテレータに変換\n",
        "imges = next(batch_iterator)  # 1番目の要素を取り出す\n",
        "print(imges.size())  # torch.Size([64, 1, 64, 64])\n"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([64, 1, 28, 28])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j9vLW5T6L8Ku"
      },
      "source": [
        "# 学習させる"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SDF9_QozL8Kv"
      },
      "source": [
        "# モデルを学習させる関数を作成\n",
        "\n",
        "\n",
        "def train_model(G, D, E, dataloader, num_epochs):\n",
        "\n",
        "    # GPUが使えるかを確認\n",
        "    device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "    print(\"使用デバイス：\", device)\n",
        "\n",
        "    # 最適化手法の設定\n",
        "    lr_ge = 0.0001\n",
        "    lr_d = 0.0001/4\n",
        "    beta1, beta2 = 0.5, 0.999\n",
        "    g_optimizer = torch.optim.Adam(G.parameters(), lr_ge, [beta1, beta2])\n",
        "    e_optimizer = torch.optim.Adam(E.parameters(), lr_ge, [beta1, beta2])\n",
        "    d_optimizer = torch.optim.Adam(D.parameters(), lr_d, [beta1, beta2])\n",
        "\n",
        "    # 誤差関数を定義\n",
        "    # BCEWithLogitsLossは入力にシグモイド（logit）をかけてから、\n",
        "    # バイナリークロスエントロピーを計算\n",
        "    criterion = nn.BCEWithLogitsLoss(reduction='mean')\n",
        "\n",
        "    # パラメータをハードコーディング\n",
        "    z_dim = 20\n",
        "    mini_batch_size = 64\n",
        "\n",
        "    # ネットワークをGPUへ\n",
        "    G.to(device)\n",
        "    E.to(device)\n",
        "    D.to(device)\n",
        "\n",
        "    G.train()  # モデルを訓練モードに\n",
        "    E.train()  # モデルを訓練モードに\n",
        "    D.train()  # モデルを訓練モードに\n",
        "\n",
        "    # ネットワークがある程度固定であれば、高速化させる\n",
        "    torch.backends.cudnn.benchmark = True\n",
        "\n",
        "    # 画像の枚数\n",
        "    num_train_imgs = len(dataloader.dataset)\n",
        "    batch_size = dataloader.batch_size\n",
        "\n",
        "    # イテレーションカウンタをセット\n",
        "    iteration = 1\n",
        "    logs = []\n",
        "\n",
        "    # epochのループ\n",
        "    for epoch in range(num_epochs):\n",
        "\n",
        "        # 開始時刻を保存\n",
        "        t_epoch_start = time.time()\n",
        "        epoch_g_loss = 0.0  # epochの損失和\n",
        "        epoch_e_loss = 0.0  # epochの損失和\n",
        "        epoch_d_loss = 0.0  # epochの損失和\n",
        "\n",
        "\n",
        "\n",
        "        # データローダーからminibatchずつ取り出すループ\n",
        "        for imges in dataloader:\n",
        "\n",
        "            # ミニバッチがサイズが1だと、バッチノーマライゼーションでエラーになるのでさける\n",
        "            if imges.size()[0] == 1:\n",
        "                continue\n",
        "\n",
        "            # ミニバッチサイズの1もしくは0のラベル役のテンソルを作成\n",
        "            # 正解ラベルと偽ラベルを作成\n",
        "            # epochの最後のイテレーションはミニバッチの数が少なくなる\n",
        "            mini_batch_size = imges.size()[0]\n",
        "            label_real = torch.full((mini_batch_size,), 1).to(device)\n",
        "            label_fake = torch.full((mini_batch_size,), 0).to(device)\n",
        "\n",
        "            # GPUが使えるならGPUにデータを送る\n",
        "            imges = imges.to(device)\n",
        "\n",
        "            # --------------------\n",
        "            # 1. Discriminatorの学習\n",
        "            # --------------------\n",
        "            # 真の画像を判定　\n",
        "            z_out_real = E(imges)\n",
        "            d_out_real, _ = D(imges, z_out_real)\n",
        "\n",
        "            # 偽の画像を生成して判定\n",
        "            input_z = torch.randn(mini_batch_size, z_dim).to(device)\n",
        "            fake_images = G(input_z)\n",
        "            d_out_fake, _ = D(fake_images, input_z)\n",
        "\n",
        "            # 誤差を計算\n",
        "            # d_loss_real = criterion(d_out_real.view(-1), label_real)\n",
        "            # d_loss_fake = criterion(d_out_fake.view(-1), label_fake)\n",
        "            # d_loss = d_loss_real + d_loss_fake\n",
        "\n",
        "            # Issue 144に従い修正\n",
        "            # https://github.com/YutaroOgawa/pytorch_advanced/issues/144\n",
        "            label_real = label_real.type_as(d_out_real.view(-1))\n",
        "            d_loss_real = criterion(d_out_real.view(-1), label_real)\n",
        "            \n",
        "            label_fake = label_fake.type_as(d_out_fake.view(-1))\n",
        "            d_loss_fake = criterion(d_out_fake.view(-1), label_fake)\n",
        "            \n",
        "            d_loss = d_loss_real + d_loss_fake\n",
        "\n",
        "\n",
        "            # バックプロパゲーション\n",
        "            d_optimizer.zero_grad()\n",
        "            d_loss.backward()\n",
        "            d_optimizer.step()\n",
        "\n",
        "            # --------------------\n",
        "            # 2. Generatorの学習\n",
        "            # --------------------\n",
        "            # 偽の画像を生成して判定\n",
        "            input_z = torch.randn(mini_batch_size, z_dim).to(device)\n",
        "            fake_images = G(input_z)\n",
        "            d_out_fake, _ = D(fake_images, input_z)\n",
        "\n",
        "            # 誤差を計算\n",
        "            g_loss = criterion(d_out_fake.view(-1), label_real)\n",
        "\n",
        "            # バックプロパゲーション\n",
        "            g_optimizer.zero_grad()\n",
        "            g_loss.backward()\n",
        "            g_optimizer.step()\n",
        "\n",
        "            # --------------------\n",
        "            # 3. Encoderの学習\n",
        "            # --------------------\n",
        "            # 真の画像のzを推定\n",
        "            z_out_real = E(imges)\n",
        "            d_out_real, _ = D(imges, z_out_real)\n",
        "\n",
        "            # 誤差を計算\n",
        "            e_loss = criterion(d_out_real.view(-1), label_fake)\n",
        "\n",
        "            # バックプロパゲーション\n",
        "            e_optimizer.zero_grad()\n",
        "            e_loss.backward()\n",
        "            e_optimizer.step()\n",
        "\n",
        "            # --------------------\n",
        "            # 4. 記録\n",
        "            # --------------------\n",
        "            epoch_d_loss += d_loss.item()\n",
        "            epoch_g_loss += g_loss.item()\n",
        "            epoch_e_loss += e_loss.item()\n",
        "            iteration += 1\n",
        "\n",
        "        # epochのphaseごとのlossと正解率\n",
        "        if epoch % 10==0:\n",
        "            print('-------------')\n",
        "            print('Epoch {}/{}'.format(epoch, num_epochs))\n",
        "            print('-------------')\n",
        "            print('（train）')\n",
        "      \n",
        "            t_epoch_finish = time.time()\n",
        "            print('-------------')\n",
        "            print('epoch {} || Epoch_D_Loss:{:.4f} ||Epoch_G_Loss:{:.4f} ||Epoch_E_Loss:{:.4f}'.format(\n",
        "                epoch, epoch_d_loss/batch_size, epoch_g_loss/batch_size, epoch_e_loss/batch_size))\n",
        "            print('timer:  {:.4f} sec.'.format(t_epoch_finish - t_epoch_start))\n",
        "            t_epoch_start = time.time()\n",
        "\n",
        "    print(\"総イテレーション回数:\", iteration)\n",
        "\n",
        "    return G, D, E\n"
      ],
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3o7-C9d7L8Kz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "cfe568f4-d466-44c4-8aee-ee1ee0a5559c"
      },
      "source": [
        "# ネットワークの初期化\n",
        "def weights_init(m):\n",
        "    classname = m.__class__.__name__\n",
        "    if classname.find('Conv') != -1:\n",
        "        # conv2dとConvTranspose2dの初期化\n",
        "        nn.init.normal_(m.weight.data, 0.0, 0.02)\n",
        "        nn.init.constant_(m.bias.data, 0)\n",
        "    elif classname.find('BatchNorm') != -1:\n",
        "        # BatchNorm2dの初期化\n",
        "        nn.init.normal_(m.weight.data, 0.0, 0.02)\n",
        "        nn.init.constant_(m.bias.data, 0)\n",
        "    elif classname.find('Linear') != -1:\n",
        "        # 全結合層Linearの初期化\n",
        "        m.bias.data.fill_(0)\n",
        "\n",
        "\n",
        "# 初期化の実施\n",
        "G.apply(weights_init)\n",
        "E.apply(weights_init)\n",
        "D.apply(weights_init)\n",
        "\n",
        "print(\"ネットワークの初期化完了\")\n"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ネットワークの初期化完了\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0-eQCLPJL8K1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0a29b940-b8d6-437e-bb2a-2cc5a0c9f11b"
      },
      "source": [
        "# 学習・検証を実行する\n",
        "# 15分ほどかかる\n",
        "num_epochs = 1500  # 1500\n",
        "G_update, D_update, E_update = train_model(\n",
        "    G, D, E, dataloader=train_dataloader, num_epochs=num_epochs)\n"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "使用デバイス： cuda:0\n",
            "-------------\n",
            "Epoch 0/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 0 || Epoch_D_Loss:0.1485 ||Epoch_G_Loss:0.0789 ||Epoch_E_Loss:0.0767\n",
            "timer:  0.6093 sec.\n",
            "-------------\n",
            "Epoch 10/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 10 || Epoch_D_Loss:0.1145 ||Epoch_G_Loss:0.0969 ||Epoch_E_Loss:0.1030\n",
            "timer:  0.2160 sec.\n",
            "-------------\n",
            "Epoch 20/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 20 || Epoch_D_Loss:0.0587 ||Epoch_G_Loss:0.1577 ||Epoch_E_Loss:0.1716\n",
            "timer:  0.2040 sec.\n",
            "-------------\n",
            "Epoch 30/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 30 || Epoch_D_Loss:0.0257 ||Epoch_G_Loss:0.2454 ||Epoch_E_Loss:0.2611\n",
            "timer:  0.2144 sec.\n",
            "-------------\n",
            "Epoch 40/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 40 || Epoch_D_Loss:0.0137 ||Epoch_G_Loss:0.3166 ||Epoch_E_Loss:0.3339\n",
            "timer:  0.2156 sec.\n",
            "-------------\n",
            "Epoch 50/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 50 || Epoch_D_Loss:0.0133 ||Epoch_G_Loss:0.3641 ||Epoch_E_Loss:0.3350\n",
            "timer:  0.2106 sec.\n",
            "-------------\n",
            "Epoch 60/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 60 || Epoch_D_Loss:0.0054 ||Epoch_G_Loss:0.4244 ||Epoch_E_Loss:0.4504\n",
            "timer:  0.2111 sec.\n",
            "-------------\n",
            "Epoch 70/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 70 || Epoch_D_Loss:0.0073 ||Epoch_G_Loss:0.4246 ||Epoch_E_Loss:0.4078\n",
            "timer:  0.2090 sec.\n",
            "-------------\n",
            "Epoch 80/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 80 || Epoch_D_Loss:0.0076 ||Epoch_G_Loss:0.3962 ||Epoch_E_Loss:0.4145\n",
            "timer:  0.2109 sec.\n",
            "-------------\n",
            "Epoch 90/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 90 || Epoch_D_Loss:0.0244 ||Epoch_G_Loss:0.3257 ||Epoch_E_Loss:0.3283\n",
            "timer:  0.2063 sec.\n",
            "-------------\n",
            "Epoch 100/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 100 || Epoch_D_Loss:0.0457 ||Epoch_G_Loss:0.2664 ||Epoch_E_Loss:0.3059\n",
            "timer:  0.2183 sec.\n",
            "-------------\n",
            "Epoch 110/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 110 || Epoch_D_Loss:0.0375 ||Epoch_G_Loss:0.2733 ||Epoch_E_Loss:0.2879\n",
            "timer:  0.2131 sec.\n",
            "-------------\n",
            "Epoch 120/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 120 || Epoch_D_Loss:0.0820 ||Epoch_G_Loss:0.2029 ||Epoch_E_Loss:0.2193\n",
            "timer:  0.2167 sec.\n",
            "-------------\n",
            "Epoch 130/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 130 || Epoch_D_Loss:0.0619 ||Epoch_G_Loss:0.1889 ||Epoch_E_Loss:0.2305\n",
            "timer:  0.2142 sec.\n",
            "-------------\n",
            "Epoch 140/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 140 || Epoch_D_Loss:0.0460 ||Epoch_G_Loss:0.2550 ||Epoch_E_Loss:0.2654\n",
            "timer:  0.2112 sec.\n",
            "-------------\n",
            "Epoch 150/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 150 || Epoch_D_Loss:0.0705 ||Epoch_G_Loss:0.2248 ||Epoch_E_Loss:0.2200\n",
            "timer:  0.2077 sec.\n",
            "-------------\n",
            "Epoch 160/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 160 || Epoch_D_Loss:0.0824 ||Epoch_G_Loss:0.1875 ||Epoch_E_Loss:0.2213\n",
            "timer:  0.2112 sec.\n",
            "-------------\n",
            "Epoch 170/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 170 || Epoch_D_Loss:0.0964 ||Epoch_G_Loss:0.1872 ||Epoch_E_Loss:0.1817\n",
            "timer:  0.2076 sec.\n",
            "-------------\n",
            "Epoch 180/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 180 || Epoch_D_Loss:0.0974 ||Epoch_G_Loss:0.1638 ||Epoch_E_Loss:0.1819\n",
            "timer:  0.2081 sec.\n",
            "-------------\n",
            "Epoch 190/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 190 || Epoch_D_Loss:0.1426 ||Epoch_G_Loss:0.1071 ||Epoch_E_Loss:0.1417\n",
            "timer:  0.2180 sec.\n",
            "-------------\n",
            "Epoch 200/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 200 || Epoch_D_Loss:0.1094 ||Epoch_G_Loss:0.1449 ||Epoch_E_Loss:0.1480\n",
            "timer:  0.2319 sec.\n",
            "-------------\n",
            "Epoch 210/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 210 || Epoch_D_Loss:0.1055 ||Epoch_G_Loss:0.1602 ||Epoch_E_Loss:0.1548\n",
            "timer:  0.2385 sec.\n",
            "-------------\n",
            "Epoch 220/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 220 || Epoch_D_Loss:0.0964 ||Epoch_G_Loss:0.1707 ||Epoch_E_Loss:0.1555\n",
            "timer:  0.2091 sec.\n",
            "-------------\n",
            "Epoch 230/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 230 || Epoch_D_Loss:0.1247 ||Epoch_G_Loss:0.1374 ||Epoch_E_Loss:0.1312\n",
            "timer:  0.2119 sec.\n",
            "-------------\n",
            "Epoch 240/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 240 || Epoch_D_Loss:0.1281 ||Epoch_G_Loss:0.1428 ||Epoch_E_Loss:0.1324\n",
            "timer:  0.2117 sec.\n",
            "-------------\n",
            "Epoch 250/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 250 || Epoch_D_Loss:0.1408 ||Epoch_G_Loss:0.1199 ||Epoch_E_Loss:0.1312\n",
            "timer:  0.2060 sec.\n",
            "-------------\n",
            "Epoch 260/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 260 || Epoch_D_Loss:0.1956 ||Epoch_G_Loss:0.0732 ||Epoch_E_Loss:0.0945\n",
            "timer:  0.2037 sec.\n",
            "-------------\n",
            "Epoch 270/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 270 || Epoch_D_Loss:0.1744 ||Epoch_G_Loss:0.0867 ||Epoch_E_Loss:0.1044\n",
            "timer:  0.2091 sec.\n",
            "-------------\n",
            "Epoch 280/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 280 || Epoch_D_Loss:0.1361 ||Epoch_G_Loss:0.1179 ||Epoch_E_Loss:0.1174\n",
            "timer:  0.2186 sec.\n",
            "-------------\n",
            "Epoch 290/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 290 || Epoch_D_Loss:0.1310 ||Epoch_G_Loss:0.1064 ||Epoch_E_Loss:0.1166\n",
            "timer:  0.2209 sec.\n",
            "-------------\n",
            "Epoch 300/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 300 || Epoch_D_Loss:0.1458 ||Epoch_G_Loss:0.0977 ||Epoch_E_Loss:0.1104\n",
            "timer:  0.2166 sec.\n",
            "-------------\n",
            "Epoch 310/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 310 || Epoch_D_Loss:0.1504 ||Epoch_G_Loss:0.0965 ||Epoch_E_Loss:0.1146\n",
            "timer:  0.2156 sec.\n",
            "-------------\n",
            "Epoch 320/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 320 || Epoch_D_Loss:0.1350 ||Epoch_G_Loss:0.1198 ||Epoch_E_Loss:0.1259\n",
            "timer:  0.2079 sec.\n",
            "-------------\n",
            "Epoch 330/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 330 || Epoch_D_Loss:0.1294 ||Epoch_G_Loss:0.1158 ||Epoch_E_Loss:0.1305\n",
            "timer:  0.2081 sec.\n",
            "-------------\n",
            "Epoch 340/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 340 || Epoch_D_Loss:0.1369 ||Epoch_G_Loss:0.1279 ||Epoch_E_Loss:0.1229\n",
            "timer:  0.2080 sec.\n",
            "-------------\n",
            "Epoch 350/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 350 || Epoch_D_Loss:0.1265 ||Epoch_G_Loss:0.1195 ||Epoch_E_Loss:0.1219\n",
            "timer:  0.2111 sec.\n",
            "-------------\n",
            "Epoch 360/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 360 || Epoch_D_Loss:0.1645 ||Epoch_G_Loss:0.0948 ||Epoch_E_Loss:0.0970\n",
            "timer:  0.2107 sec.\n",
            "-------------\n",
            "Epoch 370/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 370 || Epoch_D_Loss:0.1694 ||Epoch_G_Loss:0.0908 ||Epoch_E_Loss:0.0953\n",
            "timer:  0.2130 sec.\n",
            "-------------\n",
            "Epoch 380/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 380 || Epoch_D_Loss:0.1849 ||Epoch_G_Loss:0.0764 ||Epoch_E_Loss:0.0995\n",
            "timer:  0.2110 sec.\n",
            "-------------\n",
            "Epoch 390/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 390 || Epoch_D_Loss:0.1535 ||Epoch_G_Loss:0.1026 ||Epoch_E_Loss:0.1113\n",
            "timer:  0.2157 sec.\n",
            "-------------\n",
            "Epoch 400/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 400 || Epoch_D_Loss:0.1826 ||Epoch_G_Loss:0.0756 ||Epoch_E_Loss:0.0972\n",
            "timer:  0.2136 sec.\n",
            "-------------\n",
            "Epoch 410/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 410 || Epoch_D_Loss:0.1313 ||Epoch_G_Loss:0.0997 ||Epoch_E_Loss:0.1185\n",
            "timer:  0.2200 sec.\n",
            "-------------\n",
            "Epoch 420/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 420 || Epoch_D_Loss:0.1333 ||Epoch_G_Loss:0.1038 ||Epoch_E_Loss:0.1163\n",
            "timer:  0.2120 sec.\n",
            "-------------\n",
            "Epoch 430/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 430 || Epoch_D_Loss:0.1430 ||Epoch_G_Loss:0.0991 ||Epoch_E_Loss:0.1109\n",
            "timer:  0.2120 sec.\n",
            "-------------\n",
            "Epoch 440/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 440 || Epoch_D_Loss:0.1587 ||Epoch_G_Loss:0.0934 ||Epoch_E_Loss:0.1109\n",
            "timer:  0.2105 sec.\n",
            "-------------\n",
            "Epoch 450/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 450 || Epoch_D_Loss:0.1489 ||Epoch_G_Loss:0.1067 ||Epoch_E_Loss:0.1255\n",
            "timer:  0.2127 sec.\n",
            "-------------\n",
            "Epoch 460/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 460 || Epoch_D_Loss:0.1701 ||Epoch_G_Loss:0.0876 ||Epoch_E_Loss:0.1034\n",
            "timer:  0.2090 sec.\n",
            "-------------\n",
            "Epoch 470/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 470 || Epoch_D_Loss:0.1570 ||Epoch_G_Loss:0.0950 ||Epoch_E_Loss:0.1054\n",
            "timer:  0.2111 sec.\n",
            "-------------\n",
            "Epoch 480/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 480 || Epoch_D_Loss:0.1537 ||Epoch_G_Loss:0.0881 ||Epoch_E_Loss:0.0998\n",
            "timer:  0.2101 sec.\n",
            "-------------\n",
            "Epoch 490/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 490 || Epoch_D_Loss:0.1437 ||Epoch_G_Loss:0.0925 ||Epoch_E_Loss:0.1049\n",
            "timer:  0.2052 sec.\n",
            "-------------\n",
            "Epoch 500/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 500 || Epoch_D_Loss:0.1916 ||Epoch_G_Loss:0.0695 ||Epoch_E_Loss:0.0797\n",
            "timer:  0.2043 sec.\n",
            "-------------\n",
            "Epoch 510/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 510 || Epoch_D_Loss:0.1711 ||Epoch_G_Loss:0.0830 ||Epoch_E_Loss:0.0939\n",
            "timer:  0.2226 sec.\n",
            "-------------\n",
            "Epoch 520/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 520 || Epoch_D_Loss:0.1326 ||Epoch_G_Loss:0.0987 ||Epoch_E_Loss:0.1242\n",
            "timer:  0.2177 sec.\n",
            "-------------\n",
            "Epoch 530/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 530 || Epoch_D_Loss:0.1671 ||Epoch_G_Loss:0.0884 ||Epoch_E_Loss:0.0964\n",
            "timer:  0.2144 sec.\n",
            "-------------\n",
            "Epoch 540/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 540 || Epoch_D_Loss:0.1577 ||Epoch_G_Loss:0.0901 ||Epoch_E_Loss:0.1077\n",
            "timer:  0.2224 sec.\n",
            "-------------\n",
            "Epoch 550/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 550 || Epoch_D_Loss:0.1271 ||Epoch_G_Loss:0.1068 ||Epoch_E_Loss:0.1198\n",
            "timer:  0.2082 sec.\n",
            "-------------\n",
            "Epoch 560/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 560 || Epoch_D_Loss:0.1273 ||Epoch_G_Loss:0.1055 ||Epoch_E_Loss:0.1219\n",
            "timer:  0.2048 sec.\n",
            "-------------\n",
            "Epoch 570/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 570 || Epoch_D_Loss:0.1364 ||Epoch_G_Loss:0.1026 ||Epoch_E_Loss:0.1109\n",
            "timer:  0.2072 sec.\n",
            "-------------\n",
            "Epoch 580/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 580 || Epoch_D_Loss:0.1257 ||Epoch_G_Loss:0.1078 ||Epoch_E_Loss:0.1103\n",
            "timer:  0.2085 sec.\n",
            "-------------\n",
            "Epoch 590/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 590 || Epoch_D_Loss:0.1030 ||Epoch_G_Loss:0.1261 ||Epoch_E_Loss:0.1333\n",
            "timer:  0.2063 sec.\n",
            "-------------\n",
            "Epoch 600/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 600 || Epoch_D_Loss:0.1553 ||Epoch_G_Loss:0.0892 ||Epoch_E_Loss:0.0983\n",
            "timer:  0.2200 sec.\n",
            "-------------\n",
            "Epoch 610/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 610 || Epoch_D_Loss:0.1047 ||Epoch_G_Loss:0.1273 ||Epoch_E_Loss:0.1320\n",
            "timer:  0.2097 sec.\n",
            "-------------\n",
            "Epoch 620/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 620 || Epoch_D_Loss:0.1241 ||Epoch_G_Loss:0.1004 ||Epoch_E_Loss:0.1309\n",
            "timer:  0.2088 sec.\n",
            "-------------\n",
            "Epoch 630/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 630 || Epoch_D_Loss:0.1395 ||Epoch_G_Loss:0.1062 ||Epoch_E_Loss:0.1082\n",
            "timer:  0.2198 sec.\n",
            "-------------\n",
            "Epoch 640/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 640 || Epoch_D_Loss:0.1594 ||Epoch_G_Loss:0.0913 ||Epoch_E_Loss:0.1009\n",
            "timer:  0.2167 sec.\n",
            "-------------\n",
            "Epoch 650/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 650 || Epoch_D_Loss:0.1436 ||Epoch_G_Loss:0.0944 ||Epoch_E_Loss:0.1158\n",
            "timer:  0.2064 sec.\n",
            "-------------\n",
            "Epoch 660/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 660 || Epoch_D_Loss:0.1439 ||Epoch_G_Loss:0.0830 ||Epoch_E_Loss:0.1067\n",
            "timer:  0.2128 sec.\n",
            "-------------\n",
            "Epoch 670/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 670 || Epoch_D_Loss:0.1244 ||Epoch_G_Loss:0.1050 ||Epoch_E_Loss:0.1200\n",
            "timer:  0.2101 sec.\n",
            "-------------\n",
            "Epoch 680/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 680 || Epoch_D_Loss:0.1137 ||Epoch_G_Loss:0.1179 ||Epoch_E_Loss:0.1261\n",
            "timer:  0.2112 sec.\n",
            "-------------\n",
            "Epoch 690/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 690 || Epoch_D_Loss:0.1167 ||Epoch_G_Loss:0.1158 ||Epoch_E_Loss:0.1220\n",
            "timer:  0.2216 sec.\n",
            "-------------\n",
            "Epoch 700/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 700 || Epoch_D_Loss:0.1095 ||Epoch_G_Loss:0.1225 ||Epoch_E_Loss:0.1278\n",
            "timer:  0.2100 sec.\n",
            "-------------\n",
            "Epoch 710/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 710 || Epoch_D_Loss:0.1161 ||Epoch_G_Loss:0.1074 ||Epoch_E_Loss:0.1306\n",
            "timer:  0.2094 sec.\n",
            "-------------\n",
            "Epoch 720/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 720 || Epoch_D_Loss:0.1269 ||Epoch_G_Loss:0.1038 ||Epoch_E_Loss:0.1204\n",
            "timer:  0.2143 sec.\n",
            "-------------\n",
            "Epoch 730/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 730 || Epoch_D_Loss:0.1528 ||Epoch_G_Loss:0.0863 ||Epoch_E_Loss:0.1093\n",
            "timer:  0.2130 sec.\n",
            "-------------\n",
            "Epoch 740/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 740 || Epoch_D_Loss:0.1060 ||Epoch_G_Loss:0.1189 ||Epoch_E_Loss:0.1315\n",
            "timer:  0.2135 sec.\n",
            "-------------\n",
            "Epoch 750/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 750 || Epoch_D_Loss:0.1166 ||Epoch_G_Loss:0.1057 ||Epoch_E_Loss:0.1180\n",
            "timer:  0.2074 sec.\n",
            "-------------\n",
            "Epoch 760/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 760 || Epoch_D_Loss:0.1171 ||Epoch_G_Loss:0.1087 ||Epoch_E_Loss:0.1179\n",
            "timer:  0.2098 sec.\n",
            "-------------\n",
            "Epoch 770/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 770 || Epoch_D_Loss:0.1301 ||Epoch_G_Loss:0.1045 ||Epoch_E_Loss:0.1069\n",
            "timer:  0.2142 sec.\n",
            "-------------\n",
            "Epoch 780/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 780 || Epoch_D_Loss:0.1428 ||Epoch_G_Loss:0.1012 ||Epoch_E_Loss:0.1113\n",
            "timer:  0.2075 sec.\n",
            "-------------\n",
            "Epoch 790/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 790 || Epoch_D_Loss:0.1396 ||Epoch_G_Loss:0.0929 ||Epoch_E_Loss:0.1008\n",
            "timer:  0.2086 sec.\n",
            "-------------\n",
            "Epoch 800/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 800 || Epoch_D_Loss:0.1315 ||Epoch_G_Loss:0.1021 ||Epoch_E_Loss:0.1082\n",
            "timer:  0.2057 sec.\n",
            "-------------\n",
            "Epoch 810/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 810 || Epoch_D_Loss:0.1384 ||Epoch_G_Loss:0.0972 ||Epoch_E_Loss:0.1108\n",
            "timer:  0.2044 sec.\n",
            "-------------\n",
            "Epoch 820/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 820 || Epoch_D_Loss:0.1205 ||Epoch_G_Loss:0.1065 ||Epoch_E_Loss:0.1207\n",
            "timer:  0.2089 sec.\n",
            "-------------\n",
            "Epoch 830/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 830 || Epoch_D_Loss:0.1437 ||Epoch_G_Loss:0.0912 ||Epoch_E_Loss:0.1057\n",
            "timer:  0.2092 sec.\n",
            "-------------\n",
            "Epoch 840/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 840 || Epoch_D_Loss:0.1264 ||Epoch_G_Loss:0.0981 ||Epoch_E_Loss:0.1264\n",
            "timer:  0.2129 sec.\n",
            "-------------\n",
            "Epoch 850/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 850 || Epoch_D_Loss:0.1065 ||Epoch_G_Loss:0.1203 ||Epoch_E_Loss:0.1360\n",
            "timer:  0.2182 sec.\n",
            "-------------\n",
            "Epoch 860/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 860 || Epoch_D_Loss:0.1361 ||Epoch_G_Loss:0.0999 ||Epoch_E_Loss:0.1253\n",
            "timer:  0.2180 sec.\n",
            "-------------\n",
            "Epoch 870/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 870 || Epoch_D_Loss:0.1064 ||Epoch_G_Loss:0.1169 ||Epoch_E_Loss:0.1360\n",
            "timer:  0.2063 sec.\n",
            "-------------\n",
            "Epoch 880/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 880 || Epoch_D_Loss:0.1311 ||Epoch_G_Loss:0.0988 ||Epoch_E_Loss:0.1255\n",
            "timer:  0.2234 sec.\n",
            "-------------\n",
            "Epoch 890/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 890 || Epoch_D_Loss:0.1151 ||Epoch_G_Loss:0.1135 ||Epoch_E_Loss:0.1350\n",
            "timer:  0.2090 sec.\n",
            "-------------\n",
            "Epoch 900/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 900 || Epoch_D_Loss:0.1444 ||Epoch_G_Loss:0.0973 ||Epoch_E_Loss:0.1120\n",
            "timer:  0.2235 sec.\n",
            "-------------\n",
            "Epoch 910/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 910 || Epoch_D_Loss:0.1222 ||Epoch_G_Loss:0.1115 ||Epoch_E_Loss:0.1198\n",
            "timer:  0.2342 sec.\n",
            "-------------\n",
            "Epoch 920/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 920 || Epoch_D_Loss:0.1109 ||Epoch_G_Loss:0.1124 ||Epoch_E_Loss:0.1271\n",
            "timer:  0.2112 sec.\n",
            "-------------\n",
            "Epoch 930/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 930 || Epoch_D_Loss:0.1381 ||Epoch_G_Loss:0.1027 ||Epoch_E_Loss:0.1088\n",
            "timer:  0.2110 sec.\n",
            "-------------\n",
            "Epoch 940/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 940 || Epoch_D_Loss:0.1429 ||Epoch_G_Loss:0.1005 ||Epoch_E_Loss:0.1063\n",
            "timer:  0.2173 sec.\n",
            "-------------\n",
            "Epoch 950/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 950 || Epoch_D_Loss:0.1253 ||Epoch_G_Loss:0.1067 ||Epoch_E_Loss:0.1279\n",
            "timer:  0.2121 sec.\n",
            "-------------\n",
            "Epoch 960/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 960 || Epoch_D_Loss:0.1245 ||Epoch_G_Loss:0.1045 ||Epoch_E_Loss:0.1435\n",
            "timer:  0.2113 sec.\n",
            "-------------\n",
            "Epoch 970/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 970 || Epoch_D_Loss:0.1427 ||Epoch_G_Loss:0.0957 ||Epoch_E_Loss:0.1347\n",
            "timer:  0.2070 sec.\n",
            "-------------\n",
            "Epoch 980/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 980 || Epoch_D_Loss:0.1500 ||Epoch_G_Loss:0.0995 ||Epoch_E_Loss:0.1171\n",
            "timer:  0.2061 sec.\n",
            "-------------\n",
            "Epoch 990/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 990 || Epoch_D_Loss:0.1376 ||Epoch_G_Loss:0.1078 ||Epoch_E_Loss:0.1172\n",
            "timer:  0.2171 sec.\n",
            "-------------\n",
            "Epoch 1000/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1000 || Epoch_D_Loss:0.1403 ||Epoch_G_Loss:0.1005 ||Epoch_E_Loss:0.1200\n",
            "timer:  0.2164 sec.\n",
            "-------------\n",
            "Epoch 1010/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1010 || Epoch_D_Loss:0.1265 ||Epoch_G_Loss:0.1146 ||Epoch_E_Loss:0.1161\n",
            "timer:  0.2212 sec.\n",
            "-------------\n",
            "Epoch 1020/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1020 || Epoch_D_Loss:0.1289 ||Epoch_G_Loss:0.0997 ||Epoch_E_Loss:0.1297\n",
            "timer:  0.2103 sec.\n",
            "-------------\n",
            "Epoch 1030/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1030 || Epoch_D_Loss:0.1151 ||Epoch_G_Loss:0.1116 ||Epoch_E_Loss:0.1391\n",
            "timer:  0.2175 sec.\n",
            "-------------\n",
            "Epoch 1040/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1040 || Epoch_D_Loss:0.1112 ||Epoch_G_Loss:0.1173 ||Epoch_E_Loss:0.1354\n",
            "timer:  0.2109 sec.\n",
            "-------------\n",
            "Epoch 1050/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1050 || Epoch_D_Loss:0.1505 ||Epoch_G_Loss:0.0929 ||Epoch_E_Loss:0.1112\n",
            "timer:  0.2186 sec.\n",
            "-------------\n",
            "Epoch 1060/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1060 || Epoch_D_Loss:0.1065 ||Epoch_G_Loss:0.1287 ||Epoch_E_Loss:0.1296\n",
            "timer:  0.2072 sec.\n",
            "-------------\n",
            "Epoch 1070/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1070 || Epoch_D_Loss:0.1402 ||Epoch_G_Loss:0.1034 ||Epoch_E_Loss:0.1146\n",
            "timer:  0.2093 sec.\n",
            "-------------\n",
            "Epoch 1080/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1080 || Epoch_D_Loss:0.1406 ||Epoch_G_Loss:0.1029 ||Epoch_E_Loss:0.1206\n",
            "timer:  0.2072 sec.\n",
            "-------------\n",
            "Epoch 1090/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1090 || Epoch_D_Loss:0.1174 ||Epoch_G_Loss:0.1058 ||Epoch_E_Loss:0.1325\n",
            "timer:  0.2122 sec.\n",
            "-------------\n",
            "Epoch 1100/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1100 || Epoch_D_Loss:0.1367 ||Epoch_G_Loss:0.1019 ||Epoch_E_Loss:0.1076\n",
            "timer:  0.2096 sec.\n",
            "-------------\n",
            "Epoch 1110/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1110 || Epoch_D_Loss:0.1106 ||Epoch_G_Loss:0.1050 ||Epoch_E_Loss:0.1249\n",
            "timer:  0.2064 sec.\n",
            "-------------\n",
            "Epoch 1120/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1120 || Epoch_D_Loss:0.1109 ||Epoch_G_Loss:0.1191 ||Epoch_E_Loss:0.1271\n",
            "timer:  0.2159 sec.\n",
            "-------------\n",
            "Epoch 1130/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1130 || Epoch_D_Loss:0.1105 ||Epoch_G_Loss:0.1152 ||Epoch_E_Loss:0.1321\n",
            "timer:  0.2193 sec.\n",
            "-------------\n",
            "Epoch 1140/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1140 || Epoch_D_Loss:0.1235 ||Epoch_G_Loss:0.1171 ||Epoch_E_Loss:0.1126\n",
            "timer:  0.2176 sec.\n",
            "-------------\n",
            "Epoch 1150/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1150 || Epoch_D_Loss:0.1057 ||Epoch_G_Loss:0.1232 ||Epoch_E_Loss:0.1276\n",
            "timer:  0.2147 sec.\n",
            "-------------\n",
            "Epoch 1160/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1160 || Epoch_D_Loss:0.1105 ||Epoch_G_Loss:0.1174 ||Epoch_E_Loss:0.1281\n",
            "timer:  0.2085 sec.\n",
            "-------------\n",
            "Epoch 1170/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1170 || Epoch_D_Loss:0.1063 ||Epoch_G_Loss:0.1300 ||Epoch_E_Loss:0.1370\n",
            "timer:  0.2054 sec.\n",
            "-------------\n",
            "Epoch 1180/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1180 || Epoch_D_Loss:0.1224 ||Epoch_G_Loss:0.1217 ||Epoch_E_Loss:0.1266\n",
            "timer:  0.2143 sec.\n",
            "-------------\n",
            "Epoch 1190/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1190 || Epoch_D_Loss:0.1090 ||Epoch_G_Loss:0.1245 ||Epoch_E_Loss:0.1370\n",
            "timer:  0.2135 sec.\n",
            "-------------\n",
            "Epoch 1200/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1200 || Epoch_D_Loss:0.1017 ||Epoch_G_Loss:0.1361 ||Epoch_E_Loss:0.1474\n",
            "timer:  0.2083 sec.\n",
            "-------------\n",
            "Epoch 1210/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1210 || Epoch_D_Loss:0.1057 ||Epoch_G_Loss:0.1171 ||Epoch_E_Loss:0.1451\n",
            "timer:  0.2200 sec.\n",
            "-------------\n",
            "Epoch 1220/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1220 || Epoch_D_Loss:0.1233 ||Epoch_G_Loss:0.1103 ||Epoch_E_Loss:0.1217\n",
            "timer:  0.2188 sec.\n",
            "-------------\n",
            "Epoch 1230/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1230 || Epoch_D_Loss:0.1123 ||Epoch_G_Loss:0.1302 ||Epoch_E_Loss:0.1303\n",
            "timer:  0.2121 sec.\n",
            "-------------\n",
            "Epoch 1240/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1240 || Epoch_D_Loss:0.1084 ||Epoch_G_Loss:0.1393 ||Epoch_E_Loss:0.1294\n",
            "timer:  0.2095 sec.\n",
            "-------------\n",
            "Epoch 1250/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1250 || Epoch_D_Loss:0.1114 ||Epoch_G_Loss:0.1248 ||Epoch_E_Loss:0.1363\n",
            "timer:  0.2078 sec.\n",
            "-------------\n",
            "Epoch 1260/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1260 || Epoch_D_Loss:0.1043 ||Epoch_G_Loss:0.1265 ||Epoch_E_Loss:0.1319\n",
            "timer:  0.2096 sec.\n",
            "-------------\n",
            "Epoch 1270/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1270 || Epoch_D_Loss:0.1184 ||Epoch_G_Loss:0.1091 ||Epoch_E_Loss:0.1302\n",
            "timer:  0.2123 sec.\n",
            "-------------\n",
            "Epoch 1280/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1280 || Epoch_D_Loss:0.1156 ||Epoch_G_Loss:0.1242 ||Epoch_E_Loss:0.1338\n",
            "timer:  0.2156 sec.\n",
            "-------------\n",
            "Epoch 1290/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1290 || Epoch_D_Loss:0.1043 ||Epoch_G_Loss:0.1261 ||Epoch_E_Loss:0.1497\n",
            "timer:  0.2095 sec.\n",
            "-------------\n",
            "Epoch 1300/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1300 || Epoch_D_Loss:0.1124 ||Epoch_G_Loss:0.1323 ||Epoch_E_Loss:0.1429\n",
            "timer:  0.2164 sec.\n",
            "-------------\n",
            "Epoch 1310/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1310 || Epoch_D_Loss:0.1049 ||Epoch_G_Loss:0.1253 ||Epoch_E_Loss:0.1379\n",
            "timer:  0.2068 sec.\n",
            "-------------\n",
            "Epoch 1320/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1320 || Epoch_D_Loss:0.1123 ||Epoch_G_Loss:0.1252 ||Epoch_E_Loss:0.1313\n",
            "timer:  0.2192 sec.\n",
            "-------------\n",
            "Epoch 1330/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1330 || Epoch_D_Loss:0.1194 ||Epoch_G_Loss:0.1255 ||Epoch_E_Loss:0.1281\n",
            "timer:  0.2295 sec.\n",
            "-------------\n",
            "Epoch 1340/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1340 || Epoch_D_Loss:0.1124 ||Epoch_G_Loss:0.1412 ||Epoch_E_Loss:0.1377\n",
            "timer:  0.2181 sec.\n",
            "-------------\n",
            "Epoch 1350/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1350 || Epoch_D_Loss:0.1042 ||Epoch_G_Loss:0.1100 ||Epoch_E_Loss:0.1526\n",
            "timer:  0.2228 sec.\n",
            "-------------\n",
            "Epoch 1360/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1360 || Epoch_D_Loss:0.0871 ||Epoch_G_Loss:0.1519 ||Epoch_E_Loss:0.1600\n",
            "timer:  0.2147 sec.\n",
            "-------------\n",
            "Epoch 1370/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1370 || Epoch_D_Loss:0.1262 ||Epoch_G_Loss:0.1184 ||Epoch_E_Loss:0.1188\n",
            "timer:  0.2081 sec.\n",
            "-------------\n",
            "Epoch 1380/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1380 || Epoch_D_Loss:0.1170 ||Epoch_G_Loss:0.1188 ||Epoch_E_Loss:0.1336\n",
            "timer:  0.2125 sec.\n",
            "-------------\n",
            "Epoch 1390/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1390 || Epoch_D_Loss:0.1180 ||Epoch_G_Loss:0.1143 ||Epoch_E_Loss:0.1410\n",
            "timer:  0.2170 sec.\n",
            "-------------\n",
            "Epoch 1400/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1400 || Epoch_D_Loss:0.0926 ||Epoch_G_Loss:0.1298 ||Epoch_E_Loss:0.1521\n",
            "timer:  0.2201 sec.\n",
            "-------------\n",
            "Epoch 1410/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1410 || Epoch_D_Loss:0.0932 ||Epoch_G_Loss:0.1635 ||Epoch_E_Loss:0.1566\n",
            "timer:  0.2161 sec.\n",
            "-------------\n",
            "Epoch 1420/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1420 || Epoch_D_Loss:0.0857 ||Epoch_G_Loss:0.1336 ||Epoch_E_Loss:0.1676\n",
            "timer:  0.2201 sec.\n",
            "-------------\n",
            "Epoch 1430/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1430 || Epoch_D_Loss:0.1127 ||Epoch_G_Loss:0.1197 ||Epoch_E_Loss:0.1454\n",
            "timer:  0.2091 sec.\n",
            "-------------\n",
            "Epoch 1440/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1440 || Epoch_D_Loss:0.1118 ||Epoch_G_Loss:0.1385 ||Epoch_E_Loss:0.1313\n",
            "timer:  0.2096 sec.\n",
            "-------------\n",
            "Epoch 1450/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1450 || Epoch_D_Loss:0.1009 ||Epoch_G_Loss:0.1332 ||Epoch_E_Loss:0.1387\n",
            "timer:  0.2118 sec.\n",
            "-------------\n",
            "Epoch 1460/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1460 || Epoch_D_Loss:0.1248 ||Epoch_G_Loss:0.1208 ||Epoch_E_Loss:0.1222\n",
            "timer:  0.2123 sec.\n",
            "-------------\n",
            "Epoch 1470/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1470 || Epoch_D_Loss:0.1008 ||Epoch_G_Loss:0.1444 ||Epoch_E_Loss:0.1367\n",
            "timer:  0.2129 sec.\n",
            "-------------\n",
            "Epoch 1480/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1480 || Epoch_D_Loss:0.0915 ||Epoch_G_Loss:0.1484 ||Epoch_E_Loss:0.1563\n",
            "timer:  0.2159 sec.\n",
            "-------------\n",
            "Epoch 1490/1500\n",
            "-------------\n",
            "（train）\n",
            "-------------\n",
            "epoch 1490 || Epoch_D_Loss:0.1131 ||Epoch_G_Loss:0.1335 ||Epoch_E_Loss:0.1408\n",
            "timer:  0.2119 sec.\n",
            "総イテレーション回数: 10501\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IL_pxhfEL8K5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 372
        },
        "outputId": "3982146e-5497-42ad-c688-39a73fe1e21d"
      },
      "source": [
        "# 生成画像と訓練データを可視化する\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "# 入力の乱数生成\n",
        "batch_size = 8\n",
        "z_dim = 20\n",
        "fixed_z = torch.randn(batch_size, z_dim)\n",
        "G_update.eval()\n",
        "fake_images = G_update(fixed_z.to(device))\n",
        "\n",
        "# 訓練データ\n",
        "batch_iterator = iter(train_dataloader)  # イテレータに変換\n",
        "imges = next(batch_iterator)  # 1番目の要素を取り出す\n",
        "\n",
        "\n",
        "# 出力\n",
        "fig = plt.figure(figsize=(15, 6))\n",
        "for i in range(0, 5):\n",
        "    # 上段に訓練データを\n",
        "    plt.subplot(2, 5, i+1)\n",
        "    plt.imshow(imges[i][0].cpu().detach().numpy(), 'gray')\n",
        "\n",
        "    # 下段に生成データを表示する\n",
        "    plt.subplot(2, 5, 5+i+1)\n",
        "    plt.imshow(fake_images[i][0].cpu().detach().numpy(), 'gray')\n"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAFjCAYAAAC5RXNaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd9RcVb3/8e9OeNJDKiQBIwEBMURpAVQiUgRpEtQFS64FEAhyQUVwCYq/K0oRuIoigl5KKArYgBtEQUJRqkIIUgLBhAiSAimkV5Ls3x+M64bn+zl59jxnyjmT92stFsmHc2afmfmexsz+TogxGgAAAACgubo0ewMAAAAAANycAQAAAEAhcHMGAAAAAAXAzRkAAAAAFAA3ZwAAAABQANycAQAAAEAB5Lo5CyEcEkJ4KYQwPYRwTq02CqgXahZlRN2ibKhZlA01i6IInf2dsxBCVzP7h5kdZGYzzexJMzs2xvhC7TYPqB1qFmVE3aJsqFmUDTWLItksx7p7mdn0GOMMM7MQwq/MbKyZZRZyCKG0v3jdtWtXl61fv95lqTe7Xbr4Dy3V45mZhRA6PU49VLPttRZj9C9GupasWVUfZuk1krV+Zx+vGnlru5nbnipnzZpVWbdlqNmyqMexN89jNvBcMD/GuEWO9VuuZvMeZ5t53sw7dmrdNfM5WoNrtrJM4es2SyOOQ3n3mVSq7sx07RXtejrr+iDPzdnWZvbaBn+faWZ753i8QuvTp4/LVq1a5bLVq1cnPV7Pnj1dtnz5crlst27dXLZmzRqX5TlJZK2vsh49erhMvRYNPCinasmabWtrk7mqkdT11fv+1ltvuSyrllJP0mpsNU5WLW22mT+EqYOveky1XAFr1qxF61bJe9JPWbeaE7k61q1cuTJp7KxtVDWr6rPW61bp1Zzrt1zNqtfeTL/+all1Hl+xYkX+DUvQq1cvly1btsxlWc8x9Zjau3dvly1dujRlE2uBmq1C9+7dXabe03Xr1iWtq64B1XJZy+ah6s5M156q8bVr17qsmTdsZvluzpKEEMaZ2bh6jwPUCjWLsqFmUTbULMqIukUj5Lk5m2Vmwzf4+7sq2TvEGK82s6vNyv0RMFoCNYsy6rBuqVkUDDWLsuH6AIWR5+bsSTPbIYSwrb1dwJ8xs/+oyVY1UdZXURYvXpy0vvr6gPoIV32dIevrNqlflVTz4tRH0llf3VIfQaux1banfs2oyTapmlU233xzl6mvMqivbqlxssZWXxNQ9Zla21nyfKWr2V9bqELL1W3WV3HV8UpJ/bpi3vdY7Qfq62nq2Jn19cfUr9iq/aVOX2Gsh5ar2azXfquttnLZnDlzXKbOm6lTFrKo6RZqO9VXGNXUiqz9pdZfQyuolqvZjVHn3tRpLFlff20vq5bV8TK1r4OqxWr2GXWOUc+7b9++LmvgV3Q7f3MWY1wbQjjdzP5kZl3NbHyMcUrNtgyoMWoWZUTdomyoWZQNNYsiyTXnLMb4RzP7Y422Bag7ahZlRN2ibKhZlA01i6LI9SPUAAAAAIDa4OYMAAAAAAogNHJSfBk626gJtmZ6Mm/qbyKl/uZTVnMFNWlYTeZUzR7yTp5MbRLSKDX4Qd+qlKFm60H9boiaBJzVKEftR6n7kNpfVKMdMz35PfV3h9T+Vo/apmbzUY0LVLMNVbPqtyOzJrOrJjZ5VNOop4DNaZ6KMY5u1GCtVrOKqgd1bk/9jcmsZVObjKT+3pNZ+m9iqWN36m+k1UBDa9as3HWrzqm1/t09VYtm6deg9WjKkXo+aZSs6wM+OQMAAACAAuDmDAAAAAAKgJszAAAAACgAbs4AAAAAoAC4OQMAAACAAsj1I9StaNmyZTLP0+GlX79+LsvqcqeoDnIDBw502Ztvvpn0eOq5mOkOSqnd67bcckuXzZ07N2ld5NPW1ibzdevWuaxHjx4uUx2aVJe7aqhuTF27dnWZ6taosqwuUqkdU6vpUIr6q6ZjouoAp6Qej6vpyqg6mqV2CC1gB0bkMHjwYJnPnz/fZaldjlWW2sExa1l1rNtiiy1ctmjRIvmYitoHVSe9aq5r0FzqmKWuD9TxUtWjut7IOs6rGlXXMamdGdV2m+nrg9TzRGr333rhkzMAAAAAKABuzgAAAACgALg5AwAAAIAC4OYMAAAAAAqAhiDtZE1gVJMIUyf9Lly4MGnsrEYdauzUibdqwrCa5G5mtmDBApepSZpqIinNP5pHvR9ZQgidHmfQoEEuy2rUkTrpVlE1m9X0RE1WVq9HauMQNEZqk49qqPdYTRTPGlvVWOpxVj1mNTWn9ku1H6h6VxPxUXuq8YeZfu/VdYB6j7fZZhuXbbvtti478sgj5dhnnHGGy5YsWeKyzTff3GXXXXedy26++WY5zoMPPugydYxXr4Xar1IbjSE/1bjFTDfbUOdOVbeqntSxsprmS6nXMapRR9Z1iGpcoq7x+/Tp47JqGubUA5+cAQAAAEABcHMGAAAAAAXAzRkAAAAAFECuOWchhFfMbKmZrTOztTHG0bXYKKCeqFuUDTWLsqFmUTbULIoiZP3yfNLKbxfy6Bijninrl+/8YA2S1TAh9XX6wAc+4LKHH37YZWpC5euvvy4fc+jQoUnboxp6qEnMEyZMkON897vfdZmaPLlq1SqXqcnr9fg19Rhj5ztaVFRTt2Wo2SyqwUxqow5Vx2ry+sEHHyzXP/3005PGmTZtmsseeughlz399NNy/alTp7ps8uTJLlNNeVTNrlmzRo6TBzWbj5rQriazK6r5UdbkcWXgwIEuUxPfVVMO1YzETJ9j8jTQqZOn8l6YtlrNZjUlUo0MBgwY4LJLL73UZccff7zLspqSKaqWsxp+5Vn3iiuucNnXvvY1l6n9QNV7nuvOjWhozVaWL3zdViPPsXbIkCEu+/znPy+XVde06nrx4x//uMv23ntvl/3pT3+S41xyySUuU81tminr+oCvNQIAAABAAeS9OYtmdm8I4akQwrhabBDQANQtyoaaRdlQsygbahaFkPd3zsbEGGeFELY0s4khhKkxxnd8J6lS4BQ5imSjdUvNooCoWZQNNYuy4ZoWhZDrk7MY46zKv+ea2R1mtpdY5uoY42gmVqIoOqpbahZFQ82ibKhZlA3XtCiKTjcECSH0NrMuMcallT9PNLPvxRjv2cg6hZ88qX7h3sxs/fr1LvvFL37hssMPP9xl6tfHu3btmjz2smXLkh5zyZIlSctljXPDDTe47Oabb3bZfffd5zL1fNTk4LzyNleotm7LULPqtTfTr79a9rzzznPZt7/97VzbpCYR9+7d22VqMr3SvXt3matJ7f/1X//lMjWhvR7NPxRq1qvmOJunccuWW27psrlz58plVQMP1fxIbbvar1JrO4t63mo/SJ2wX6VczRVasWarcf7557ss9ZiqzveLFi2Sy77wwgsumzNnTlI2YsQIl330ox+V46hGIUcccYTLHnnkEZel7lc10NCaraxT+LpV14Bmus6Ufffd12X777+/y770pS+5TDX+MNPHb3W8U428+vfv77KsRn7q+uBzn/ucy+644w6XNapus64P8nytcYiZ3VF5UTYzs1s2VsRAQVC3KBtqFmVDzaJsqFkURqdvzmKMM8xslxpuC1B31C3KhppF2VCzKBtqFkVCK30AAAAAKABuzgAAAACgAPK20t9kjBw50mUHHXSQywYMGJD0eKpZQ9YkdzWhc/Xq1S7bfPPNXbZ27VqXZU3GP+6441x2wAEHuOyYY45x2dNPP+2yejQEgZf1OqtJ3N/85jddpiaqL1682GVqgmxWQ6H99tvPZWoC8rBhw1ymJiB/+ctfluNsscUWLttsM39Ya1TzD6SppiGIeu9UAw5Vn5/97GddlnWM3msv15jN5s2b5zI1oVxNFD/11FPlOKrxkno9VAMdNUEejZFVs+p9+tSnPuUyVduqocekSZNcdvLJJ8ux1fldUcdEdd7IOp4/+uijLttzzz1d9sQTT7isTs0/kCir8YdqwKGu9/7whz+4LGtfSB1bXdOq/UMdq1XNZ53f1TXQqFGjXDZhwgSXNbtu+eQMAAAAAAqAmzMAAAAAKABuzgAAAACgALg5AwAAAIACCFkTQOsyWAl+TT3LrFmzXLbVVlu5bOnSpS5TE4bVhMqs90JN3FUTfNUERjXpM4uaaKkmaaqx29raksfJI+vX1OslhBDb//q8ev5vvfWWXF81Lkh9P9X7oR4v6z1Wk2lfffXVpLFfe+01l82YMcNlZ599thz7ueeec9mKFStcpibsquY7Dz30kBxHvZYf+tCHXPbPf/7TZcuXL5ePWWvNqNn2xxdVN1nHG1V3qccRNTH7+OOPd9k3vvENub5q8KJqJLXBi2qSlDqZ3Uy/FqkNZ6ZOnSofc5dd/E8pqfdHZQ1sbPNUjHF0owYrw7WBet/NdI289NJLLtt+++1d1q9fP5ep41L7c9C/qfOzovYhdTzOos4l2223ncu6d+/usp49e7ps7NixLvvKV76SvD1qv16yZElDa9as9ep24sSJLvvYxz5W8236zne+47Lvfve7Lps/f77LBg8e7LKLLrpIjnPuuee67JBDDnHZgw8+6DLVdK8esq4P+OQMAAAAAAqAmzMAAAAAKABuzgAAAACgALg5AwAAAIAC4OYMAAAAAApAt3HZhGV1tsnT2U11B7vllltcpjo8mZndcMMNLlOdGVXHwJNOOsllp556qhxnm222cZna9sWLF7tMdcPbdttt5Thl075TlnqdVZcqM91xTXXJylq/PdVh6XOf+5xc9uSTT3aZqpsvfOELLvvHP/7hsrlz57ps3rx5cmxFdQzr06ePy6666iqXqY5fZmYf+chHXKb2I/WeDRw40GVvvvmmHKdsUmpW1WYW1SVQHRv69+/vslNOOcVlI0aMkONkvc/tqf0ltXNdVmfVlStXumzOnDkumz17tss++tGPukx18zUz23nnnV02ZcoUl6kOmep5N6qr2KYuq2OiovaNSy+91GXqXKDGyaptNY5aVo2T0fFQjqO6NX7gAx9w2Zlnnuky1a162rRpLhs6dKgc+/XXX3dZozrttgJ1zWCmr3X333//pMdUx8pJkya5bN9995Xr9+jRw2V77rmny4444giXLVu2zGUHH3ywHOfb3/62y+655x65bHtqP2xkd3s+OQMAAACAAuDmDAAAAAAKgJszAAAAACiADm/OQgjjQwhzQwjPb5ANDCFMDCFMq/zbfyEZaCLqFmVDzaJsqFmUDTWLMggdTXALIexrZsvM7KYY46hKdqmZvRljvDiEcI6ZDYgxnt3hYCE0bjZdJ+20004yf+yxx1ymJgyqBgdqIvB3v/tdl2VNVE+dhKgaLqhJ6RdeeKFc/5hjjnGZmlysmgOoCZ7VTKBOFWNMetBa1W1qzVbzXPNMKr3oootcNm7cOLnsoEGDXDZmzBiXqYm8qU0GVL2b6Qnb6nlffvnlLvvKV77iskWLFslx1CRyte2quYKq43poRs22bxSQ2izDzKxv374uU40H9t57b5ddcsklLlONhvr16yfHnjp1qsseeeQRlz3xxBMue/nll12m6qaa46xqoDN//nyX/elPf3LZqFGj5Dhf/vKXXaaaPjXZUzHG0R0t1OjjbDNlHeNV3Xzve99zmWr68utf/9plqlFDVlOHtrY2l6n6VueCBQsWuGy77baT49x9990u23HHHV2m9peHH37YZT/96U9dduedd8qxq9DQmq2sV/i6Vec+M90QSjXbUPWt1lXXn1nnWNX849FHH3WZOu+ofSGroZo676U2wlHXNur1ySvr+qDDT85ijA+ZWfsWZmPN7MbKn280s6NybR1QY9QtyoaaRdlQsygbahZl0Nk5Z0NijP/uMfy6mQ2p0fYA9UTdomyoWZQNNYuyoWZRKLl/5yzGGDf20W4IYZyZ6e9dAU2ysbqlZlFE1CzKhppF2XBNiyLo7Cdnb4QQhpmZVf7tf522IsZ4dYxxdMp3gYE6S6pbahYFQs2ibKhZlA3XtCiUzn5ydqeZHWdmF1f+PaFmW9RkhxxyiMwHDPDNe9TkQDWZ95ZbbnGZmijZu3dvObZqrpD66+5XXHGFy7bddls5jtomNeFY/Tr8F7/4RfmYBVOTuk2dhJ1FTZxdsWKFy7bcckuXffOb33SZmoRtpifdPvPMMy5T296/f3+XqeYK1UyQVdt+0kknuUw1dnjf+94nHzP1dVcNa9Rk46zXsok6VbPtJ0JXU7Opk8JVzapmGapJyIsvvijHfv311132xhtvyGVrTdWDmlCujr1qQnpWE4eFCxe6TNWnGls1pUht3tNALXt9oKjmCOqcf/3117vsvvvuc5lq1JFF1ayizs+qAdjo0fp+Qx0r7r33XpfddtttLrvxxhtdVk3NqgZFav2cDZ5atmazXpeBAwe67A9/+IPLPvGJTySNs8MOO7js2WeflctedtllLlO1rN5ndT7JanilrttTa68ezT+qkdJK/1Yze9zM3htCmBlCONHeLuCDQgjTzOxjlb8DhUHdomyoWZQNNYuyoWZRBh1+chZjPDbjPx1Y420Baoa6RdlQsygbahZlQ82iDDo75wwAAAAAUEPcnAEAAABAAeRupd9qZs+enbys+gVx9Uvjl156qcu+9KUvuey1116T4xx88MEu23nnnV32jW98w2Xq19CzJoiqX5KfOXOmy6ZOneqy22+/XT5mK2g/UVVNjlavnZl+rVUjBVVL22+/fdLjTZkyRY49adKk5O1sL29jjA9+8IMuO+oo/7ueqjnKT37yE5dlNa/o169f0rJqv2xl7RtHqNdETbQ3M1u6dGnSGJMnT3aZOlapZiJ5qbpRjRlU86KsRh1qQrpqwHHEEUe4TDVZUsdOM7O77rrLZevWrZPLojiy6njo0KEu++c//+myrbbaymVjxoxx2QMPPOCyrCYGquHBr3/9a5epRh9qH8q6/lHXMKrZmGrMoJrdqP1KNS0yq0vzD5jZm2+2/x1usyuvvNJlhx56qMtUU6TrrrvOZf/v//0/ObZqVqaoRkvf+c53kpYz082XUqU2bqsXPjkDAAAAgALg5gwAAAAACoCbMwAAAAAoAG7OAAAAAKAAaAjSzllnnSVzNSlVTUJUEyUPO+wwl91yyy0uUxPSzcz22msvl6mGDaqphJI1qVE1i5g7d67LDjrooKRxWkX7Sc5qMnPeCf3q1+hVg5inn37aZXvvvbd8zHvuucdlhxxyiMvUhO3UhiBDhgyR+de//nWXvf/973fZ9ddf77KrrrrKZarxh5nZ4sWLXdazZ0+XqfdM7atZjUfKpv2xRNVn1qR61VhDHVuWL1/uMtU0Qb3O6j3K2ib1nqhGH3kbj6j11b4xduxYl/Xv399l06ZNk+Oo90K9vuqYgOKZNWuWy9Tx5vHHH3fZCSec4DLV4EkdO83MbrjhhqSxVXMD1djrE5/4hBznlVdecZlq/qEakKU2Y8o69tajodCmZNCgQTJfsGCBy/70pz+57OKL/e9xn3nmmS4bMWKEy+6+++6ELcymrpOradSh9gX1esyfPz/5MRuFT84AAAAAoAC4OQMAAACAAuDmDAAAAAAKgJszAAAAACgAGoK0c8ABB8j8qaeectl73/tel6lfuVeT18eMGZO8TarpgWqQoMZRE+/VNpqZffGLX3TZxIkTUzbRevfu7TLVMKAVqMn7S5cuTV4/9bVSDUHU+7HzzjvLcT760Y+67JprrnHZ5Zdf7rLnn39ePmZ7H/7wh2V+4IEHuuy+++5zmao5NWFXTV42001s1H6gtErzD6V90wnVbEg1OTLTxwx1DFJUg5g33njDZdXsL4pqMqImf6tmAlmNl1R+4oknuuzggw9O2US76667kpYzo/lHGahmF2a6ccDatWtdduyxx7rssccec9lzzz3nMtWYxsxs0aJFLlPNaT73uc+57JlnnnHZ9OnT5Tip1D6Yqm/fvjJXx3P1euQ9prSqrHOner3Va3jZZZe57IgjjnDZrrvu6rKsc4xqpqeohiBqf6um6Ylq/qEMHjy40+vWAp+cAQAAAEABcHMGAAAAAAXAzRkAAAAAFAA3ZwAAAABQAB3enIUQxocQ5oYQnt8gOy+EMCuE8PfKP4fVdzOBdNQsyoi6RdlQsygbahZlEFQ3q3csEMK+ZrbMzG6KMY6qZOeZ2bIY4w+qGiyEjQ9WANtvv73MVVcl1c0ltWOY6s6nugBmUY+5atUql/3ud79z2bXXXisf869//avL1qxZ4zLV7bFRne9ijB22g6p3zaqubuvXr5frp75WqnuR6nTUq1cvl40dO1aO/fOf/9xlquPYpEmTXPa1r33NZaoT33XXXSfHVh0p99lnn6Sx1WupuvNl5Wo/SO0i1tHxsDNSatasdnWralbVjep6ZZZei127dnVZ+y6R1Y6t9i313mXtb7Vc18zs6aefdpnqSqa6LX7yk5+Uj/nkk0+6THXDrOZ1q4OnYoyjO1qonjVbFmo/UDWmji2qe+5nPvMZl6kOqmb6/H7zzTe77Pe//71cv72s46Ta9tSuw6qjrrquyDrGq86XGefVhtZsZb3S1q2ijv3qvbr99ttddtRRRyWPo95T1ZVz6623dpk6BqrzThbV6VPVU6O6f2ZdH3T4yVmM8SEze7PmWwTUCTWLMqJuUTbULMqGmkUZ5JlzdnoI4dnKR8QDshYKIYwLIUwKIfj/RQ40FjWLMuqwbqlZFAw1i7Lh+gCF0dmbs5+Z2XvMbFczm2NmP8xaMMZ4dYxxdMrHzUAdUbMoo6S6pWZRINQsyobrAxRKp27OYoxvxBjXxRjXm9k1ZrZXbTcLqC1qFmVE3aJsqFmUDTWLotEzMDsQQhgWY5xT+esnzez5jS2/ofYTvtXEWTWBNIuarKioCbVqQutvf/tbuf4WW2zhMjVJVk0MV5Ms1SRiNUnSTE+UVRN3P/3pT7vsL3/5i8uyJk+mNkPI2s4iy1Oz7VXTUCC1UYpquKCoybATJkyQy86dO9dlX/nKV1y2ZMmSpMdUdagm15qZHXnkkS574okn5LIp1P5ippt/KP369XPZokWLOr09jVKruq2mkURqLaZOwlZjq8YfZvq4psZJbUagjmlZNauaQQ0aNEgu296Pf/xjl913331y2dTzW+rrVs3xqBE6U7NdunRx70s1NateU3XsVfWQ9zVV9alqTDWI2XvvvV2mnkvW/qJeI9X8I7UxVdY1gNov1dipzT+Uaq4rat2ALM9xtv15UT0Pdbwy06+3el0HDPDfslTNMqZPn+6yrHOkeq/OPPNMl6naUc0/1PPOavKiqNdI7R/quKqusc107anaST3nNVKHr1wI4VYz28/MBocQZprZd8xsvxDCrmYWzewVMzuljtsIVIWaRRlRtygbahZlQ82iDDq8OYsxHiti3T8bKABqFmVE3aJsqFmUDTWLMsjTrREAAAAAUCPcnAEAAABAAXSqIUgeKU0nsiaaq0m6aqKsmhyofn38/PPPd9moUaPk2KrRR58+fZIyRU2yzHreaqLljBkzXKaaHlQzqTqVen3Vttd60i60rPf48ccfd9n999/vMlXzBx54oMtUfWU16jjuuONcNnnyZJctWLDAZWo/z5qw26tXr6T1VfOPvn37umzp0qVyHNSWajBglt6UKE8TjKwJ8t/61rdcNnz4cJepSfe/+93vXJY1GT61QcKmVJ/r169POldlNcZIbU6jjlfqfVLHG3WsMTMbOHCgy1Rzrh/84AdJYz/99NMuW7hwoRz7gAMOcNm73vUul82cOVOu317W8Vztb2pfVdcq6nqhiA0YOivlWJT1fFOPY6ppl6qd1AZZWdukzpP9+/d3Wer18Pz58+XYv/zlL102cuRIl02cONFl6tyR9TqqGlWNUNRrkXouqhc+OQMAAACAAuDmDAAAAAAKgJszAAAAACgAbs4AAAAAoAAa2hAkhOAmMVbTNEJN0FMTWFXzD+Xzn/+8y/JO4lYTIGfPnu2ynXfe2WVqMq2ZngC53XbbuWy//fZz2YQJE1yWNWlUvb5q7CJOntyUZU3iTt23Bg0a5LJhw4Ylrfvkk0/K/Mgjj3TZ+9//fpephiBXX321y9QEYjNdn6q+1WvUqs0Vyix1grw6xvfs2dNlah+46qqr5GMedthhSWOPHz/eZc8884zLevToIdfPOs63l9U0ZVOhzsVZDUFSz8+qcUhqM5GDDz5Y5tdee63L1DFVXQfcc889Lvv+97/vsqyGIFOnTnXZoYce6rKbb77ZZaoBS9b1j9qPqmnM0MpS9tNqGlYoqkazaqK9MWPGyFy9/3mafzz22GMuO/fcc+XYf/7zn1123333yWVTZF3rpD4f9f6oa4bUY0Ut8MkZAAAAABQAN2cAAAAAUADcnAEAAABAAXBzBgAAAAAF0NCGIDFGN/FOTaasprlE6uTqUaNGuWzgwIHJj6eWVds5btw4l82bN89lv/3tb102dOhQObaa7Kgm7s6dO9dlagJjNa+vmhSZ+pqjMbImqaq8V69eLhs7dqzLVM39z//8j8uyJvx+5CMfcdldd90ll01Z9+STT5bLnnXWWS6bNm2ayxYvXpw0NhqjmonVqgmEmsCtGhWpxjRZtaSohkqqoYg6TmY1Xkq1ZMkSl7W1tbmsmqZaRdb+fVbnqdTGH9VQr+kee+zhsjvuuCP5MWfMmOGyk046yWUPPvigy7bffnuXZTWxGTx4sMt22mknl3Xv3t1lqiFEVi2p/U014FGNetR+2UpSrmmzGoKoY5u6tlPXXCpTxyF1PjXT1wKpzT8mTZrksn322cdlqu7M9PP+0Y9+5LIDDzxQrp9KnfdTr3979+7tMnVMrhc+OQMAAACAAuDmDAAAAAAKgJszAAAAACiADm/OQgjDQwgPhhBeCCFMCSF8tZIPDCFMDCFMq/x7QP03F+gYNYuyoWZRRtQtyoaaRRmkfHK21szOijGONLMPmtlpIYSRZnaOmd0fY9zBzO6v/B0oAmoWZUPNooyoW5QNNYvC67BbY4xxjpnNqfx5aQjhRTPb2szGmtl+lcVuNLM/m9nZHT1e+y5d1XS2UV1sUrsRXnrppR1tWuYYZmYLFy50mery9LWvfc1lqqOS6qJShSgAACAASURBVMz4+uuvy7HVsqqrzj333OOyarpbqfdCZYrqElRNR7ZaqnXNllXfvn1dpjrIHXbYYS574YUXXHbJJZe4LKt70f333++yHXbYwWXXX3+9y3bZZReX7bvvvnKcxx9/3GU9evSQy6Ysl7fDXmdRs9lUxy91jFfv3Ve/+lWXzZ8/X44zaNAgl1155ZUuUx3AVEczdYw2053KVJbaja2Z3RprVbchBPfcqnle6jylzs/qfLj77ru77Mc//rHLso4N6jG/8Y1vuOyZZ55x2a9//WuXqe56Wcc01Qn6iSeecJm6flGyrr2U1GuDoqn1sbaa7tftqddbXYPmGUMdm8z09ZmqM1Vje+65Z9K6WfvM2Wf7l/WAAw5w2aJFi1x28cUXu+yCCy6Q46hjsHp91TmmkZ0ZlarmnIUQRpjZbmb2NzMbUilyM7PXzWxITbcMqAFqFmVDzaKMqFuUDTWLokr+nbMQQh8zu83MzogxLtnw/5rEGGMIQd7ahxDGmZn/8S+gzqhZlA01izLqTN1Ss2gmjrUosqRPzkIIbfZ2Ed8cY7y9Er8RQhhW+e/DzMz/ArKZxRivjjGOjjGOrsUGAymoWZQNNYsy6mzdblizZf2KHMqJYy2KLqVbYzCz68zsxRjjZRv8pzvN7LjKn48zswm13zygetQsyoaaRRlRtygbahZlkPK1xn3M7PNm9lwI4e+V7FtmdrGZ/SaEcKKZvWpmx9RnEzdOTZTs2bOnyw499FCXqQmRWQ00BgxI66qqmh6ohh7K5ptvnrScmdnf//53l02ZMsVlaqJj1kRrNSk9dSJqs5p/ZCh0zTaKmoz77ne/22Xbbbedy1RzGTU5t5qx5871/yPyl7/8pcvGjBnjsqzmClOnTnVZt27dXKZqe/ny5fIxm2STqlnV2MJMT5BXtaSOS9tvv73LVHOFlStXyrGvuOIKl/35z392mZr4rupT1aGZPseo5hWqIUizGtZsRE3qNsaY1ABE7cf/Xr+91IYiAwcOdNnee++dtK6ZrtmLLrrIZTvuuKPLVHMa1UAsy+233+6yP/7xj0nrqmuDrE8wVd2pTK2v3rMmXy/U9VirXoNqGnrkeW3UulnHO1W36ji0xRZbuGyPPfZw2VFHHeWyrGvar3zlKzJPcc456U00e/Xq5TL1vNW1QO/evZOWq5eUbo2PmFnWdw4OrO3mAPlRsygbahZlRN2ibKhZlEFV3RoBAAAAAPXBzRkAAAAAFAA3ZwAAAABQAMm/c1azAdv9Orea9Jw1WTx1oqRabsGCBS5TkxXVZMFqpDb/UJNp1eRFMz3ZXE04Xrx4cdLYWVJ/rV5Ry6n3FrXXp08fmau6GTRokMvUe7f//vu77Pvf/77LbrzxRjm2av7x2c9+1mXf/va35frtZTUjOfXUU12WOilaTVTPu/8jTdYE+dTmAWrSvaoldS7JGvvCCy90mWoqoRqC5G3UobZpUz+mqvc469pAvU+pDa5eeukllz399NMu22mnneTY6ryd2kBMNf/4xz/+4bIrr7xSrq8agixdujRp7NTmHVnUsXJTPKa2fx3zvobqeJf6mOrxVOOXrGVXrFjhMlXfkyZN6vS6ZropkmqgpJrbqKZm//rXv+Q4apuUIjYM45MzAAAAACgAbs4AAAAAoAC4OQMAAACAAuDmDAAAAAAKIFTzy+W5Bwshtp88qcbv2bOnXH/16tUuU5Ma1aThW2+91WVHH320y+6++2459gEHHOAyte1qsnjqhNisyd6q+cf3vvc9l6kJvoqaeGmmJ7WrZdVkzkaJMaY9yRoJITRuB6kxtR+pfePhhx922a677uoyNdE86/ihmu0oqmnJjBkzXHbaaafJ9R955BGXpU5KV69FatOhalCz6dTkdXXcV42X5syZ4zL1fn7nO9+RY6uGIKk1ohpALFy4UI6T2qRE1aw6xtfpHP5UjHF0PR5YUTXb1tbmlss6l6r3RB2DVGMg1UzkhBNOcNm1114rx05t7vXUU0+57JprrnHZr371K5dV0+xLnbPVc6ymblL3A/WeqbHrpKE1a5Z+rM26Nkt9D9Trr17rahoTqbHVe6XeZ3WdmzqGmX49PvKRj7jsySefdJk6H1TTFE1pZiObrOsDPjkDAAAAgALg5gwAAAAACoCbMwAAAAAoAG7OAAAAAKAAGtoQpEuXLrH9hO9qJjDmmWyqJmxvt912Lnv22Wfl+pMnT3bZqFGjXDZt2jSXfeELX3DZXnvt5bJ7771Xjj116lSZb4poruBlNXhJbdyi6viUU05x2fHHH++yrIm4ar985ZVXXHbVVVe5bPz48S5bsmSJHEdN5FVoCFIsffv2lblqOqOaK3zta19z2X/913+57De/+Y3LzjzzTDn2vHnzZL6JanpDEKBKhW0IUkTnn3++y8444wyXqXO8arShzu/qXG5mdvnll7usUQ04ioaGIAAAAABQYNycAQAAAEABcHMGAAAAAAXQ4c1ZCGF4COHBEMILIYQpIYSvVvLzQgizQgh/r/xzWP03F+gYNYuyoWZRNtQsyoi6RRl02BAkhDDMzIbFGCeHEPqa2VNmdpSZHWNmy2KMP0geLOfkyc0228xlahJh6sTCfv36uWzx4sVy2d69e7tsxYoVLkttsKLGXr58uVx27dq1SY+5KUhprlCkmm0E1djCTO8vqklIo5oCqcYOah9Ssp6jUo+mHnlQs+m23nprl+23334u++EPf+iyIUOGuGzMmDEue/TRRzu3cZuWDpsrULMomKSGINTt29T5WJ1nVZMmJQR/mqumWVkjmxMWSdb1gb968yvOMbM5lT8vDSG8aGb+DAoUBDWLsqFmUTbULMqIukUZVDXnLIQwwsx2M7O/VaLTQwjPhhDGhxB8r3qgyahZlA01i7KhZlFG1C2KKvnmLITQx8xuM7MzYoxLzOxnZvYeM9vV3v6/EP57Jm+vNy6EMCmEMKkG2wsko2ZRNtQsyoaaRRlRtyiypJuzEEKbvV3EN8cYbzczizG+EWNcF2Ncb2bXmJn/VeW3l7s6xji60T8OiE0bNYuyoWZRNtQsyoi6RdGldGsMZnadmb0YY7xsg3zYBot90syer/3mAdWjZlE21CzKhppFGVG3KIOUbo1jzOxhM3vOzP7dBvFbZnasvf3xbzSzV8zslMpEy409Vq52LKqTTJcu/v5SdTdM7QTT1tYm87feestlqjuNGkd1zaumA6PqeKO63WwKEjvfFaZmi6Z79+4uU90NVab2tawuimrfWL16dcomysfM24FR7YNqX61Hp0dq1uvZs6fMV65c6TLVhXHChAku23333V2mOpJlHXv79u3rstROZS0opVvjJlWzKLzUbo3UbRXUuVNdC1RzTVqPc3xZ5enW+IiZqZX/mHejgHqgZlE21CzKhppFGVG3KIOqujUCAAAAAOqDmzMAAAAAKABuzgAAAACgADpsCFLTwUKI7ScCqvHXr1/vso08pstUUw/1mNU05ejRo4fL1ARG1ThESW0mgndKaa5QS2WY8Nu7d2+Zr1ixwmWpNVaP+lSTgFWm9susY4KamKyaTaj9slFNdajZdKlNZ9T7mdp4KWt/Wb58ecombiqSmivUSplrFoXR0Jo1K0fdqmtXM30MTW3KoY7T6pqhGjQEeSc+OQMAAACAAuDmDAAAAAAKgJszAAAAACgAbs4AAAAAoAAa3RBknpm9WvnrYDOb37DB66uVnotZcZ/PNjHGLRo5IDVbGkV9PtRs7bTSczEr9vNpaN22cM2atdbzKfJzaeaxtsivS2e00vMp8nPJrNmG3py9Y+AQJjW6s069tNJzMWu951MrrfS6tNJzMWu951MrrfS6tNJzMWu951Mrrfa6tNLzaaXnUkut9rq00vMp63Pha40AAAAAUADcnAEAAABAATTz5uzqJo5da630XMxa7/nUSiu9Lq30XMxa7/nUSiu9Lq30XMxa7/nUSqu9Lq30fFrpudRSq70urfR8SvlcmjbnDAAAAADwf/haIwAAAAAUQMNvzkIIh4QQXgohTA8hnNPo8fMKIYwPIcwNITy/QTYwhDAxhDCt8u8BzdzGVCGE4SGEB0MIL4QQpoQQvlrJS/l86oWaLQ5qNg01WxzUbLoy120r1awZdZuqzDVr1lp120o129CbsxBCVzO70swONbORZnZsCGFkI7ehBm4ws0PaZeeY2f0xxh3M7P7K38tgrZmdFWMcaWYfNLPTKu9HWZ9PzVGzhUPNdoCaLRxqNkEL1O0N1jo1a0bddqgFatasteq2ZWq20Z+c7WVm02OMM2KMa8zsV2Y2tsHbkEuM8SEze7NdPNbMbqz8+UYzO6qhG9VJMcY5McbJlT8vNbMXzWxrK+nzqRNqtkCo2STUbIFQs8lKXbetVLNm1G2iUtesWWvVbSvVbKNvzrY2s9c2+PvMSlZ2Q2KMcyp/ft3MhjRzYzojhDDCzHYzs79ZCzyfGqJmC4qazUTNFhQ1u1GtWLct8R5Tt5lasWbNWuA9LnvN0hCkxuLb7S9L1QIzhNDHzG4zszNijEs2/G9lfD6oThnfY2p201bG95ia3bSV9T2mbjdtZXyPW6FmG31zNsvMhm/w93dVsrJ7I4QwzMys8u+5Td6eZCGENnu7iG+OMd5eiUv7fOqAmi0YarZD1GzBULNJWrFuS/0eU7cdasWaNSvxe9wqNdvom7MnzWyHEMK2IYRuZvYZM7uzwdtQD3ea2XGVPx9nZhOauC3JQgjBzK4zsxdjjJdt8J9K+XzqhJotEGo2CTVbINRsslas29K+x9RtklasWbOSvsctVbMxxob+Y2aHmdk/zOxlMzu30ePXYPtvNbM5ZvaWvf394hPNbJC93QFmmpndZ2YDm72dic9ljL398e6zZvb3yj+HlfX51PF1omYL8g81m/w6UbMF+Yeareq1Km3dtlLNVp4PdZv2OpW2Zivb3zJ120o1GypPCAAAAADQRDQEAQAAAIAC4OYMAAAAAAqAmzMAAAAAKABuzgAAAACgALg5AwAAAIAC4OYMAAAAAAqAmzMAAAAAKABuzgAAAACgALg5AwAAAIAC4OYMAAAAAAqAmzMAAAAAKABuzgAAAACgALg5AwAAAIAC4OYMAAAAAAqAmzMAAAAAKABuzgAAAACgAHLdnIUQDgkhvBRCmB5COKdWGwXUCzWLMqJuUTbULMqGmkVRhBhj51YMoauZ/cPMDjKzmWb2pJkdG2N8YSPrdG4woCLGGDq7LjWrbbbZZi5bu3aty7p08f8vZ/369XXZplaSp2bNqq/bTaFmG6V79+4uW716dRO2pOHmxxi36OzK1GzzdOvWzWVr1qxpwpY0XENrtrIOdVvRtWtXl61bty55/U31+iLr+sBflaXby8ymxxhnmJmFEH5lZmPNLLOQUTsh+Pezszfam5CWrFlVC2bp9TBw4ECXzZs3z2U9e/Z02YoVK5LHVgffrG1vL+sgrx5TZepms0Rasm4bIe8Fw/Dhw102ffr0XNuUR97nU4VXc65PzbaTevGp/mdZ1nusjrNDhw512b/+9a+UTayLBl6rULN1kPr+9enTx2WLFy9OHqdXr14uW7ZsWfL6qVKfT7OvsfN8rXFrM3ttg7/PrGRAUVGzKCPqFmVDzaJsqFkURp5PzpKEEMaZ2bh6jwPUCjWLsqFmUTbULMqIukUj5Lk5m2VmG37n412V7B1ijFeb2dVmfD8XTUfNoow6rFtqFgVDzaJsuD5AYeS5OXvSzHYIIWxrbxfwZ8zsP2qyVU2kvu9tlv5dU7Vc6qRG9Z30rPUb9d3X1DkOJZnM2ZI1myX1vVPf61b1pRohtLW1JW+PmveltidrH1TUsqrumv398Zw2mbpV9fTWW291+vHyzsd6+eWXc62fKs9xtk5zzvIqTc0OGjTIZQsXLnRZ3vPZgAEDXLZgwQKXVXNMVMfUZs4vK7nS1GwjqaZIq1atcpm6jqhmPnw95pcpZZlz1umbsxjj2hDC6Wb2JzPrambjY4xTarZlQI1Rsygj6hZlQ82ibKhZFEmuOWcxxj+a2R9rtC1A3VGzKCPqFmVDzaJsqFkURa4foQYAAAAA1AY3ZwAAAABQAHVvpV82WT9Wmzo5MLVBQVbzj0bIGls9RzU5udkTJfFOWa99aqMA9UPStW7MYKbrRk2SX7RoUfJjqn1L7cOpP3aN5ipac4s8E8WzjrPqHLFmzZqk5QrYZKn0VFOOesi6tmhP1VLeY29eeRrWcG1Qbqr5h1KPY3fqsVbVp5lZt27dXLZy5cpOj9NIfHIGAAAAAAXAzRkAAAAAFAA3ZwAAAABQANycAQAAAEABcHMGAAAAAAVAt8YcVCcY1XVLSe0ClpWrTl6pHZ3ydvxS26O6NKV2p0LxqFpSHRyzOhr16dPHZapL0pIlSzqxdf8ntcbU/qLWbXaHpk1FM7vV5pVaI1nH2dRzBMfP8lL1vXjx4iZsSW2kduKjm2h59OrVy2WrV692WZ4ujFnXtKnH0NTlsrZRXXPkGaeRynuGBAAAAIAWws0ZAAAAABQAN2cAAAAAUADcnAEAAABAAdAQJJGaMKgmdqsJkKmZaqJgZrZs2TKXde/ePWkb804qV9upJv2qyaVqu9E4qU0wFDWhPbXhjJnZokWLkpdNGTtrwq5qUqLqk+YfxVLExgHqWEeNoDNUfeepr6zGCkrRalYdz4u4/29qVqxYUfcxGlW3eRuPFBGfnAEAAABAAXBzBgAAAAAFwM0ZAAAAABRArjlnIYRXzGypma0zs7UxxtG12CignqhblA01i7KhZlE21CyKohYNQfaPMc6vweOUTuoE39RJiYsXL04ee/ny5Z0eJ6/UJiEFl1y37Z9v0SaZZk2GVU0w1OTsU0891WX/8R//4bL3ve99Luvdu7cce/58/9JecMEFLhs/frzLevbs6bKsBiPVNClpr4QNIDbZY20j1Pq9V/ta1jgFr7s8NsmaVc2YRowY4bJXXnnFZeq4rRqAmaWfi9VxslHn7Fa+NsDGNeq9Hzx4sMyXLFnistSmaOqY3Mha5muNAAAAAFAAeW/OopndG0J4KoQwrhYbBDQAdYuyoWZRNtQsyoaaRSHk/VrjmBjjrBDClmY2MYQwNcb40IYLVAqcIkeRbLRuqVkUEDWLsqFmUTZc06IQcn1yFmOcVfn3XDO7w8z2EstcHWMczcRKFEVHdUvNomioWZQNNYuy4ZoWRRE6OxE5hNDbzLrEGJdW/jzRzL4XY7xnI+u07KznRkv95XX1/ub9NXU12bmtrc1lK1euTHq8asQY039yXqi2bstcs127dnXZnnvu6bKbb77ZZdttt13SGFkTZFUtrVu3Lmn9v/3tby476aST5DizZs1y2erVq5O2p1FNGKhZlNBTeS4+N6WaVc06/vu//9tl3bp1c9k99/iXQx3/5syZk7w9zWx0pK4NUhsw1EBDa7ayTmnrFsWQdX2Q52uNQ8zsjsqBYDMzu2VjRQwUBHWLsqFmUTbULMqGmkVhdPrmLMY4w8x2qeG2AHVH3aJsqFmUDTWLsqFmUSS00gcAAACAAuDmDAAAAAAKoNMNQTo1WBMnT6pJsv3793fZAQccINefN2+eyx599FGXqaYHasKwykaOHCnHPvbYY1126KGHumz+fP+j9nfffbfLVMMEM7Of/OQnLluzZo3LVPOPt956y2X1+DX1vM0VqlXmCb+HH364y1TzD9U4ZNGiRS7r06ePyy6//HI59gUXXOCyoUOHuuzFF190We/evV2m6jBr/d12200u216XLv7/TVGz2FBq4yW13Oabby6XVfuWouqzgY1tcjVXqFbemlXNNtTrpxpWrFq1ymXqGDRq1Cg59i233OKyIUOGJG2jul5QjbQmTJggx7799ttdppokzZ0712WvvfZa0vaUSENr1oxjbUfU/mZmNnq0f5vU9cqJJ57osoEDB7rsmmuukeOcffbZLlP7uzqGquvcrOuQPLKuD/jkDAAAAAAKgJszAAAAACgAbs4AAAAAoAC4OQMAAACAAthkGoKopgfjxo1zWVaDAzXhW004njFjhstUswzVjGTAgAFybLW+mrCsqO3Oes9POukkl916660uUxMqG6VszRVU3dV60nVWLagmNsuWLXPZc8895zK1H9x7770uy5ogm6exxrBhw1w2c+bM5HFUQxA1IV5l9VC2ms14TJc16tyhjrOqucLatWtdpva1rO1W++q2227rsjvvvNNl2223ncsefvhhOY5aXzXqefPNN13WwPeh4Q1B2r/PPXv2dMstX75crq/e59SGP3/5y19ctv/++2dua3vq9VfvkzpvptZ2FnVtsGLFCpepJmDnnHOOy37961/LcdT6BWweUrqGIKox3Oc+9zmXXXfddanb47JevXrJZffYYw+XqWOgat6hztFqnE9/+tNy7MGDB8u8s9Sx0szsgQcecNkvf/lLl/3+9793Wb9+/Vy2cOHCTmzdxtEQBAAAAAAKjJszAAAAACgAbs4AAAAAoAC4OQMAAACAAuDmDAAAAAAKYJPp1jhw4ECXfehDH3LZL37xC7l+aheczTbbLGld9bpnvReqo1NqhyiVqa5mZmZf/vKXXXbTTTclja2yenR1bIXOd6ojkspUJ0RVXzfeeKMc57DDDnOZeo9vu+02l6lOXKqz1NKlS+XYeaiaXbJkiVxWdapUr2Ujj3Ni7FLVrKqxj33sYy5TXe7e8573uGzQoEEu23HHHeXYV1xxhctULf7ud79z2T777OOyao6zquOiOlaq10dt48qVK+U4qfWturY9/vjjLlPHiR49esixqzgmN7xbY8pyH/7wh2V+6qmnuuyzn/2sy1LPm0o152fV1W3nnXd22RZbbOGyyZMnJ41hpt97tWxbW5tcv72sDqP333+/y9R5R23PnDlzXKZey2222UaOPWvWLJdlXMOUrluj6sz58ssvu0x1Ad99991dps7RWXWbWhOqw2lWPbanuoma6XP04sWLXTZ9+nSXvetd73JZVqfz1Od4+umnu+zaa691mTrOb7XVVvIx1b4wf/58l9GtEQAAAAAKjJszAAAAACgAbs4AAAAAoAA6vDkLIYwPIcwNITy/QTYwhDAxhDCt8m/9hU+gSahblA01i7KhZlE21CzKoMOGICGEfc1smZndFGMcVckuNbM3Y4wXhxDOMbMBMcazOxysiQ1BFDVpuk+fPnLZRYsWJS27//77u0xNfuzfv7/LTjzxRDn2tttu67Itt9zSZamNRx544AE5zhFHHOEyNfF2yJAhLnv99dddpiZP5pXaXKFWdVu0mt1jjz1cpiZrm5mddtppLvvtb3/rMjVxtWhOOOEEmV922WUuU40d1AT9RilCzaY2PTDTk9Tvvfdel6lj2KhRo1y2fPlyl6mJ62a62YaakJ46SV0de6t5LVKpY50au5r1Tz75ZJepJgzqtaiBpOYKja7ZappyqOOAavh19dVXu0w1rBk3bpwce+7cuS77+te/7rK7777bZeq9U9clqnmZmdmll17qMtWURzXV2GmnnVyWVbOpDZVUYxvV5Oe8885zWQ3quKE1W1kvqW7VMdXMrG/fvi5T75Vq5KPWVddrs2fPlmOrhnYrVqxw2YMPPuiyoUOHukztM1OnTpVjq6Yn6lz+6quvuuyUU05x2U9/+lM5TuoxWD3vkSNHuuxf//qXy/I2G+t0Q5AY40Nm9ma7eKyZ/fsscaOZHZVr64Aao25RNtQsyoaaRdlQsyiDzs45GxJj/Hd/1NfNzH+UAhQPdYuyoWZRNtQsyoaaRaH475BUKcYYN/bVrxDCODPT3wcAmmRjdUvNooioWZQNNYuy4ZoWRdDZT87eCCEMMzOr/Nt/6boixnh1jHF0o38cEBCS6paaRYFQsygbahZlwzUtCqWzn5zdaWbHmdnFlX9PqNkWNZCaZKmyLKpJyB133OGy1Ano48ePl/nhhx/uMtXYQU2yV7/Q/sMf/lCOs3r1apepCfpqwmrqL8Y3WanqVk0i/tKXvuSytrY2ub6qm1tvvTX/htWQ2je22morl2XV7Oabb+6ys8/287jPOeecTmxdIVRds126dHFNBdS+nVU3qkHM6NH+OiS12YaauJ5l2bJlLlMTrufMmeMy9XxUM6VqqAn26vh3+eWXu0w15DEze/e73+0y1QxBTZqvU/OPWqvJcVa97wMG6CZ6quGPap704Q9/uNPj3HDDDXJs1SRg5cqVctkU6hpENWUwM9tzzz2THlPtl0ceeaTLss4PqkmJekzVJOiiiy5ymTp2NLm2O12z7Z+Lau6jGmiY6YYX6ji2ePFilz3xxBMuU7Wc1RTjmGOOcZmqW3UdopY7//zzXabeezPdOCb1/b/++uuTljMzu+CCC1w2aNAgl6nmKAcffLDLfv/737tMNcOrhZRW+rea2eNm9t4QwswQwon2dgEfFEKYZmYfq/wdKAzqFmVDzaJsqFmUDTWLMujwk7MY47EZ/+nAGm8LUDPULcqGmkXZULMoG2oWZVCK76IBAAAAQKvj5gwAAAAACiDk/XXrqgbbSHvSTY2aZKkm4puZbbPNNi57/vnnXdanTx+XLV261GXDhw+X46hJp6kTd9VyajJ9Xlm/pl4vzaxZ1WRFvcePPPKIXF9NfN19991d9sYbb3Ri6zZOTRZXz2fIEP9zMqeeeqrLshp6LFiwwGVHHeV/P/Svf/1r0jbW43jY6JrdbLPNYvsmHMuXL3fLqWZBZvp1Ua+zmvyvJsNX0yxITa5W793f/vY3l91zzz0uU/WuJq6b6Qn2X/ziF12mXrdRo0a57Ac/+IEc5yMf+YjLpk6d6rL999/fZWpfrdM5/KlGdqPr2rVrbN904qtf/apb7vvf/37W+i5T78n06dNdphowqGZfZaauN9S+Onv2bLn+lltu2emxr7jiCpepxmlZTU+q0NCaNXu7bnv27PmOLG89Pfnkk0nrz53rG0qeeOKJLqumRqXSIwAAGKxJREFUyV0e6toki3qNVFMfRe3rWWPn2Y/VMXns2LEu+8c//tHpMcyyrw/45AwAAAAACoCbMwAAAAAoAG7OAAAAAKAAuDkDAAAAgAKgIUiTqEn3atKumdno0X6O6wMPPJC0vpp4q34Z3kw38NhsM/9TeKnNP9SE47w2pYYgqkbUaz9jxgy5/rve9a6kcU444QSX/eUvf3HZK6+8kvR4WbbaaiuXPfrooy4bMWKEy5YsWSIfUzV8UNuparF90wwz3UAnr0bXbFtbW2zfrEM19MiizgkDBw502auvvuqyF154wWWnnXaay3r37i3H3mWXXVz20EMPuUy9x6qhkaKOaWZmRx55pMvU8VPtl6pB00477ZS0PWa6GcIll1zisj//+c8uW716dfI4VWhocwXVxEa9n426XmlUs6B6UPWtzs+pDcTyUg101DYedNBBcn1V8xnXFoVoCKKaL+XVvlmOmdk+++zjMnUcUQ3c6qH962CW3QxK1V5qYzLVFCirYVgqVaNqfz/44INdpq6VqkFDEAAAAAAoMG7OAAAAAKAAuDkDAAAAgALg5gwAAAAACkDPjK6j9hNtUyevmpVnQm5nbbvttjI/+uijXZbVPKQ99QvpWRPi1euemrWy9jWrJovXY9KtapqwcuVKlx177LFy/fHjx7tsxx13dNkNN9yQtD2//e1vXXb88cfLZQ899NCk9RVVs9/97nflsvPnz3fZtdde67IrrrjCZZMnT07anrJZt26dm9ivmp9kNVlR3nzzTZcNGzbMZao5hZpsneXhhx92Wa33razj12233eYyNRH/Yx/7mMv69evnsqztVk1TTj75ZJepRj+teg5ct26dLVu27B3Z3nvv7Zb761//2pDtKdrrrM45Zno7U8/PqvlONVSTob322stlTzzxhMsGDBjgsuuvv16Os80223Ri6xpj/fr17pjXq1cvt9yKFStyjbNq1SqXqUYpjWr+oahtzKKaprS1tblMXduMGTOmug1LsP/++7vssccec1kjjwt8cgYAAAAABcDNGQAAAAAUADdnAAAAAFAAHd6chRDGhxDmhhCe3yA7L4QwK4Tw98o/h9V3M4F01CzKiLpF2VCzKBtqFmWQ8snZDWZ2iMh/FGPctfLPH2u7WUAuNxg1i/K5wahblMsNRs2iXG4wahYF12G3xhjjQyGEEbUasGvXru/4u+rkpbrdmOXveJMiqyuSylXnltRuLgMHDnTZTjvtJJc95ZRTkh5zzZo1Lnv22WeTljMz69LF36s3s/tPZ9WyZrt06eLqsX1XsXpJHWfSpEkyP+aYY1w2atQol/3gBz9wWZ8+fVw2duxYly1cuFCOrfaXmTNnuuzCCy902Z133umyww7T/yPz5Zdfdtn06dNdNnv2bLl+kdSqbmOMtm7dundk1XRmVNT7mVqfat0tt9xSLqvqKet41Vntz0H/prrcqf1FdUFVx/PXX39djrPvvvu6LGs/KrpaHmvbn2sa1ZkxlTo/mjXmHFmPLnGPPPJIrvFVF2nVTXjQoEEuU69Z1vVPrZ97LWs2hOCOb+o1qKbbZqr2x3gz3Yk7a2x1vMuzPdV0sVbLqi6411xzjcvU8Ttru2+66SaX/fOf/3SZ6ijabHnmnJ0eQni28hGx74sKFA81izKiblE21CzKhppFYXT25uxnZvYeM9vVzOaY2Q+zFgwhjAshTAoh6P+9DzRGp2q2aL93g01OUt1ynEWBULMoG64PUCidujmLMb4RY1wXY1xvZteYmf/lwf9b9uoY4+gY4+jObiSQV2drNusrAUAjpNYtx1kUBTWLsuH6AEXTqZuzEMKwDf76STN7PmtZoAioWZQRdYuyoWZRNtQsiqbDhiAhhFvNbD8zGxxCmGlm3zGz/UIIu5pZNLNXzCytY4XpSYjtNaLxRxY1odJMNy7Jo1+/fi4744wz5LI9evRwmfo4XTX/+PnPf+6yrP/boyaYllEta3b9+vUNawDSWatXr5b5Sy+95LJnnnnGZao5w8UXX+yytrY2l2XV0sSJE112//33u0w1/3jjjTdcdvPNN8tx/v73v7vs6aefdlkZGtvUsm5T9uVq/q9vnq/vqHXffPNNuWytj7PdunVzWc+ePZOXvfXWW13Wv39/l82YMcNlWU1sytr8Q6llzRb9K2KDBw+W+fLly5OyVKoxxty5c+WyWftRran3Ztddd3XZBz/4QZepRirNfK9rXbPtr2mb+dzU9bW6fjSr/bFWGTJkiMw/85nPuOySSy5xmXo+U6dOddmnP/1pOY5qGJbaNKXZx6OUbo3Hivi6OmwLUBPULMqIukXZULMoG2oWZZCnWyMAAAAAoEa4OQMAAACAAuDmDAAAAAAKoMM5Z7XW/te9i9aEIqVhyb+lTqhXExDVRN499tgjeWy1/vnnn+8yNRlXTXw30xNE1etRxMmT8Nrva2Zm22yzjctUIxrV/EPVwjHHHCPHvueee1y2Zs0al6U26li1apXMJ0+e7LJNvRa7du1qffv2fUe2ZMkSt1w9mqR0797dZanHFTN9vFKZqs9Bgwa5bObMmXIc5ayzznLZtttu6zJVx9/4xjdctnjx4uSxFXWcVmO3ghCCa1ywcuXKJm2NltWUQ50PU987tZxqeJBX6rVKVrMa1Wzstddec9m//vUvl6l9SDXQKaMuXbpYr1693pGpum3mdW7WuVNRx9oRI0a4TDUhmzNnjst22GEHOc4FF1zgsnnz5rnspptuctn3v/99l2Wd81WuzkeqaUqzr3P55AwAAAAACoCbMwAAAAAoAG7OAAAAAKAAuDkDAAAAgAJoeEOQojUAaS9r4qyaCJhncuCxx/rfQWw/sXRj/vCHP7gstQlDFvXc+/fv7zI1GVRl9Wg4AE81YTAz+9SnPuWyq666ymVZDWLau+46/zud//u//5u0bhY1AblRddPMsetp3bp1tmjRoqaMvf3227tsypQpLlNNkszMBgwY4DLVWENNulfNP9R7fPTRR8uxzz77bJep89W5557rsrvvvttl6piYRR17hwwZ4rJZs2a5rBVqNsZYuAYgeaSed+vR4KVnz54u++c//5m07u233y7z8847z2WzZ892mdp/FyxYkDR2Ga1fv96WLVvWlLFVwy91LMg6v6tcPRfVvEWNPXz4cJddfvnlcmzV0Onkk0922YMPPuiyao6rqVSDqmY3FuOTMwAAAAAoAG7OAAAAAKAAuDkDAAAAgALg5gwAAAAACqDhDUGKLqu5gpqsrCa1q4mFqqnGl770JZdlNSNRkzRvu+22pLHVhPhqJjquWLHCZWoSc9a2o/5UfZmZfetb33JZ6iTicePGuezaa6/txNZtXN5mBqqWVS2q5VqhkULRTJ06NWk5VYdmZvPmzXNZasMa5cQTT3TZj3/8Y7msqhHV/ONnP/uZy/JOUlc1+9prr+V6TDRGMxsH9OvXz2UTJ050mWouo2r2sssuk+Ooawt1/Gzl5h9Fk9pcL6vpjHr/U6/jhg4d6rJnn33WZX369JHrf/azn3XZAw884LJ6NBBU5x5V383GJ2cAAAAAUADcnAEAAABAAXBzBgAAAAAF0OHNWQhheAjhwRDCCyGEKSGEr1bygSGEiSGEaZV/+18fBJqAmkXZULMoI+oWZUPNogxSPjlba2ZnxRhHmtkHzey0EMJIMzvHzO6PMe5gZvdX/g4UATWLsqFmUUbULcqGmkXhdditMcY4x8zmVP68NITwopltbWZjzWy/ymI3mtmfzezsumxlA7311lvJy6puRao74qmnnuqyUaNGJY9z0003ueyuu+5K2p68sjr9tJfaIa8RWrlmVX3tvvvuctkRI0a4THUq+tSnPuUyVV9F1NbW5rLUzkvUbO2ldteqprth6nuiOuD+6Ec/cllW98df/OIXLlMdSletWpW0PdVQdZy3A2QjtErd1trw4cNdlqf7ZlZ309GjR7tst912S3rM6667zmUvvfSSXLYeXfOapVVqNrUTdzXnNNX988wzz3TZWWed5TLV6TyrM+5vfvMbl6nr16y6by+ry2SZuzRXNecshDDCzHYzs7+Z2ZBKkZuZvW5mvlcr0GTULMqGmkUZUbcoG2oWRZX8O2chhD5mdpuZnRFjXLLhnWqMMYYQ5O15CGGcmfkfTQLqjJpF2VCzKKPO1C01i2biWIsiS/rkLITQZm8X8c0xxtsr8RshhGGV/z7MzOaqdWOMV8cYR8cY/efvQJ1QsygbahZl1Nm6pWbRLBxrUXQp3RqDmV1nZi/GGDf8+fg7zey4yp+PM7MJtd88oHrULMqGmkUZUbcoG2oWZRA6miwYQhhjZg+b2XNm9u+ZdN+yt7+j+xsze7eZvWpmx8QY3+zgsZoz274KalKjmW6MoV67XXbZxWWPP/64y3r27Omy+fPny7F32GEHly1atMhl9ZggWjQxRj3zcwP1rtlGNZLo06ePyz7/+c+77MILL5Trq8m9qunB8ccfX/3G1ZF6fTfbTH8DO7WBTzObfxShZosm6/1U1LHyve99r8vuv/9+l6l9SC1nZvaf//mfLps+fbrLUmupmknqqiGIWj+1QVMNPJXyyUCt6rYMNVsN1cggT1ONKVOmyHzo0KEuGzDAd4B/3/ve57Jp06a5rJpmCWofVuecBQsWJD9mTg2t2cpjlbZuVZ0cccQRLhs/frzLXn75ZZddcsklLlPN7MzS94XUY62qOzOzxYsXJ42jajm1sVheWdcHKd0aHzGzrIuLA/NsFFAP1CzKhppFGVG3KBtqFmVQVbdGAAAAAEB9cHMGAAAAAAXAzRkAAAAAFED6rOxNRNZExdRJ3AcccIDL1ORg9Xi33XabHFs1/1DUOGpSY9Zk/EZNgGxFqhmLWfoEa9W44Prrr3fZxz/+cZetWrVKPuZjjz3mshNOOCFpe5qpHk1sUvcNNEa3bt1krhpejB071mWf+MQnXNa7d2+XzZgxw2WXXnqpHFs1/1DU5PPUY3SW1MY2qD1VN8uXL3dZNTWbavDgwS5TDcS23357ub46hqkGHCqrpvlHnrFbWQjBXU81al9W5zT1ng4bNkyuf95557lMNRxT7/OHP/xhl6lr5zxNcKqxdOnSXOsX8VqAT84AAAAAoAC4OQMAAACAAuDmDAAAAAAKgJszAAAAACgAGoK0kzUxMPWXyg8//HCXqQYcCxcudFlWQ5DUsdXkSzXZOWvCqmpqoSY7q4nReSZFt4K8k6tVg4OjjjoqaZwPfOAD8jFfeeWVXNtUJGofMDPr3r27y1avXu2yIk743ZStWLFC5qr5x5VXXukydUxVx0TV/EM1XDBLP86uXLnSZaqhz7Jly3KNo47HeY8z8LJqsb2sc5xqzLDbbru57Omnn3bZnXfe6bLtttsuaXvMdDOo/fbbz2WqZpFfjLFpzXzUtZ06z33zm9+U6x977LEuU8fVgw8+2GWqYY467+aljovq+Jl1XEw91vbq1ctlqceFeuGTMwAAAAAoAG7OAAAAAKAAuDkDAAAAgALg5gwAAAAACoCGIDm85z3vcZlqzpA6gXHevHlyHDWBUVGPmTUpPY9NvflH6vuRpWfPni678cYbXaYm9w4YMMBlalJ4q2nWpGvUx8CBA2WumuCoidlqH3z44Ydddvfdd7tMNU4y0w041LJq4ns1k+HzHM9Re6nvR1tbm8yHDx/uMtWI5sUXX3TZe9/7XpepOpw9e7Yc++Mf/7jLpkyZIpdFa+nXr5/LttlmG5edcsopcn11XTpkyBCXLVq0yGV5j01qX0o9x1dz/ZW6bLObfyh8cgYAAAAABcDNGQAAAAAUADdnAAAAAFAAHd6chRCGhxAeDCG8EEKYEkL4aiU/L4QwK4Tw98o/h9V/c4GOUbMoG2oWZUPNooyoW5RBSkOQtWZ2Voxxcgihr5k9FUKYWPlvP4ox/qB+m1cc6tfYR44c6TI1UVJNdHzooYdcltUQRP1qu2oW0UxqEnMTJ7QXumZVk4Hf/e53LvviF7/osk2h+ccmqtA1W2vve9/7ZN6jRw+X9e/f32VqAve5557rslmzZrmse/fucmw1Qb5ounbt6rKsBicNsEnVbNY5d8aMGS5TDQ9GjRrlMtXgSTUxGDRokBx7+vTpMu+srH0gTxOsAl6/tETdHnHEES678sorXaYaepiZjRkzJmnZPNdx6r03K0eDL7Uv5G0GV40Ob85ijHPMbE7lz0tDCC+a2db13jCgs6hZlA01i7KhZlFG1C3KoKo5ZyGEEWa2m5n9rRKdHkJ4NoQwPoTg/xcQ0GTULMqGmkXZULMoI+oWRZV8cxZC6GNmt5nZGTHGJWb2MzN7j5ntam//X4gfZqw3LoQwKYQwqQbbCySjZlE21CzKhppFGVG3KLKkm7MQQpu9XcQ3xxhvNzOLMb4RY1wXY1xvZteY2V5q3Rjj1THG0THG0bXaaKAj1CzKhppF2VCzKCPqFkWX0q0xmNl1ZvZijPGyDfJhGyz2STN7vvabB1SPmkXZULMoG2oWZUTdogxCR91HQghjzOxhM3vOzP7dtuVbZnasvf3xbzSzV8zslMpEy409VuNanXRSt27dZL7FFlu4bM2aNUnZzjvv7LLJkye7rE+fPnLsBQsWuCy1a0yzO87UWoyxw5Zqm1rNFlGr1V0e1Kynug5m5bNnz3aZ6p6rsmo6jRWs42yzPdXRJwObWs1WY6eddnLZM8884zJ1vaGOkz/72c/kOKeffnrS+nmV5HjeYc2atU7d/uQnP3HZl7/8ZZcdffTRcv0JEya4rFFdFHv27OmylStXNmTsosm6Pkjp1viImamV/5h3o4B6oGZRNtQsyoaaRRlRtyiDqro1AgAAAADqg5szAAAAACgAbs4AAAAAoAA6bAhS08FabNJvnkmyauJ71roqV1lJJu3mktJcoZZarWYbZVOoxVTUbD5Dhgxx2bx581yWt3kHNfsOSc0VaqXValbV0h/+8AeXzZw502W33Xaby+699145ziZcn0pDa9aseHXb1tbmskY1+ahG3759XbZ06dImbEnzZV0f8MkZAAAAABQAN2cAAAAAUADcnAEAAABAAXBzBgAAAAAF0OiGIPPM7NXKXweb2fyGDV5frfRczIr7fLaJMW7RyAGp2dIo6vOhZmunlZ6LWbGfT0PrtoVr1qy1nk+Rn0szj7VFfl06o5WeT5GfS2bNNvTm7B0DhzCp0Z116qWVnotZ6z2fWmml16WVnotZ6z2fWmml16WVnotZ6z2fWmm116WVnk8rPZdaarXXpZWeT1mfC19rBAAAAIAC4OYMAAAAAAqgmTdnVzdx7Fprpedi1nrPp1Za6XVppedi1nrPp1Za6XVppedi1nrPp1Za7XVppefTSs+lllrtdWml51PK59K0OWcAAAAAgP/D1xoBAAAAoAAafnMWQjgkhPBSCGF6COGcRo+fVwhhfAhhbgjh+Q2ygSGEiSGEaZV/D2jmNqYKIQwPITwYQnghhDAlhPDVSl7K51Mv1GxxULNpqNnioGbTlbluW6lmzajbVGWuWbPWqttWqtmG3pyFELqa2ZVmdqiZjTSzY0MIIxu5DTVwg5kd0i47x8zujzHuYGb3V/5eBmvN7KwY40gz+6CZnVZ5P8r6fGqOmi0carYD1GzhULMJWqBub7DWqVkz6rZDLVCzZq1Vty1Ts43+5GwvM5seY5wRY1xjZr8ys7EN3oZcYowPmdmb7eKxZnZj5c83mtlRDd2oTooxzokxTq78eamZvWhmW1tJn0+dULMFQs0moWYLhJpNVuq6/f/t3CFuVEEcgPHvL0BVgWgIJQHBAfA9QU+Aq+AC+N6BGyAJChKqewFS09Q0KUEBKSBrEYN4I9aUTsnue/Of/X7Jy+6+FZ1585nJNjNSs2C3jVI3C2N1O1Kzc2/OHgPfVj5/r/ey2y2lXNX3P4HdJQfzPyLiKfAC+MwA81kjm+2Uzd7IZjtls/80YrdDrLHd3mjEZmGANc7erAeCrFmZjr9MdQRmROwAH4DXpZTr1e8yzkd3k3GNbXa7ZVxjm91uWdfYbrdbxjUeodm5N2c/gCcrn/fqvex+RcQjgPr6e+HxNIuIe0wRvyulfKy3085nA2y2MzZ7K5vtjM02GbHb1Gtst7casVlIvMajNDv35uwUeB4RzyLiPvASOJ55DJtwDBzW94fApwXH0iwiAngLXJRS3qx8lXI+G2KzHbHZJjbbEZttNmK3adfYbpuM2CwkXeOhmi2lzHoBB8Al8BU4mvvvr2H874Er4A/T/xe/Ah4ynQDzBTgBHiw9zsa57DP9vHsOnNXrIOt8NvicbLaTy2abn5PNdnLZ7J2eVdpuR2q2zsdu255T2mbr+IfpdqRmo05IkiRJkrQgDwSRJEmSpA64OZMkSZKkDrg5kyRJkqQOuDmTJEmSpA64OZMkSZKkDrg5kyRJkqQOuDmTJEmSpA64OZMkSZKkDvwFyt0B24flFf0AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 1080x432 with 10 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZKa9G-CZL8K7"
      },
      "source": [
        "# テスト画像で異常検知する"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GcB7w-4xL8K8"
      },
      "source": [
        "# テスト用のDataLoaderの作成\n",
        "\n",
        "\n",
        "def make_test_datapath_list():\n",
        "    \"\"\"学習、検証の画像データとアノテーションデータへのファイルパスリストを作成する。 \"\"\"\n",
        "\n",
        "    train_img_list = list()  # 画像ファイルパスを格納\n",
        "\n",
        "    for img_idx in range(5):\n",
        "        img_path = \"./data/test_28size/img_7_\" + str(img_idx)+'.jpg'\n",
        "        train_img_list.append(img_path)\n",
        "\n",
        "        img_path = \"./data/test_28size/img_8_\" + str(img_idx)+'.jpg'\n",
        "        train_img_list.append(img_path)\n",
        "\n",
        "        img_path = \"./data/test_28size/img_2_\" + str(img_idx)+'.jpg'\n",
        "        train_img_list.append(img_path)\n",
        "\n",
        "    return train_img_list\n",
        "\n",
        "\n",
        "# ファイルリストを作成\n",
        "test_img_list = make_test_datapath_list()\n",
        "\n",
        "# Datasetを作成\n",
        "mean = (0.5,)\n",
        "std = (0.5,)\n",
        "test_dataset = GAN_Img_Dataset(\n",
        "    file_list=test_img_list, transform=ImageTransform(mean, std))\n",
        "\n",
        "# DataLoaderを作成\n",
        "batch_size = 5\n",
        "\n",
        "test_dataloader = torch.utils.data.DataLoader(\n",
        "    test_dataset, batch_size=batch_size, shuffle=False)\n"
      ],
      "execution_count": 34,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-ua6uY7_L8K-",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 194
        },
        "outputId": "c470204c-31c1-4d7c-8746-3fef96dabc32"
      },
      "source": [
        "# テストデータの確認\n",
        "batch_iterator = iter(test_dataloader)  # イテレータに変換\n",
        "imges = next(batch_iterator)  # 1番目の要素を取り出す\n",
        "\n",
        "fig = plt.figure(figsize=(15, 6))\n",
        "for i in range(0, 5):\n",
        "    # 上段に訓練データを\n",
        "    plt.subplot(2, 5, i+1)\n",
        "    plt.imshow(imges[i][0].cpu().detach().numpy(), 'gray')\n"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAACxCAYAAABAxMXKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dedRddXn28etHJJh5AAIIRGhAAshkY6SFVpChvICS0kqJ1tLCMmqljAuKfduCtmh1KVqptVDFgOWV0oUsgk1fFRYSUilKkCEMMsRAyYABkvAkJIy/9w+erjc+97XJ7+QMz94n389arCQ355y99zn3nvLkvk7KOQsAAAAAMLy2Ge4VAAAAAABwcwYAAAAAtcDNGQAAAADUADdnAAAAAFAD3JwBAAAAQA1wcwYAAAAANdDWzVlK6biU0s9TSo+nlC7q1EoB3ULPoonoWzQNPYumoWdRF2lLv+cspTRC0qOSjpH0tKSfSpqdc37oTZ7Dl6qhLTnntKXP7deeTcm/JU34DkO37u2u9zbbxL9zev3113uybKednpVa79sm9GyV0s/Efcbucd34PEv7q+GezTnvuKVP7nXPdnpfbvf1qo7J7bxmO8suXU6TzyXqcc8OPqf2b0y7n+lw9rLTjfN2r64FnKrrg7e08ZozJT2ec14iSSml6ySdJKmykYFh1pc9u+2229r6yy+/vMWv6Q5W7qL0tdde2+JlSNJb3/rWUHvppZeK1qdq+WPGjAm1gYGBUHvLW+Lh75VXXrHLGWZ917dVn2fpZzJ69OhQc72wYcOG4mWX3gS6nn3xxRfta3ZaD28Mn2zz+V3p2REjRhTXS49/rufc+1z1eq6ftttuu1Bzn1M7x+gqI0eODDW3b7z66quh1sq5pIZ/SVHLnh1u7V4fuF52/dTuudPtw245bnuqrkNKr0/ca3Zj32xFO/+scVdJ/73Jn58erAF1Rc+iiehbNA09i6ahZ1Eb7fzkrEhKaY6kOd1eDtAp9Cyahp5F09CzaCL6Fr3Qzs3ZMkm7b/Ln3QZrvyLnfKWkK6Vm/Ptc9DV6Fk202b6lZ1Ez9CyahusD1EY7N2c/lbR3SmlPvdHAp0r6UEfWCuiOvuzZVv5ttJsTKB2GdfME7vWq6m62w80EtcL9W3E3X+bUdL7M6bu+rRq2dp+J65t169YVLWfixImh9sILLxQ9V/I97+bL3Axc1RzapEmTitbJzUsM57xbi7rSs1Xzgu4YWPqZuNkrx733krRx48aiWqnSeTXJ7y+tzOyWLscZNWrUFi+7psfevjvOStXXB27Gy80rtnOOrtpn3OfvzgluH3bbUzVb5nrPzaTXsR+3+OYs5/xqSulMSd+XNELSVTnnBzu2ZkCH0bNoIvoWTUPPomnoWdRJWzNnOef5kuZ3aF2ArqNn0UT0LZqGnkXT0LOoi7a+hBoAAAAA0BncnAEAAABADXQ9Sh9Ad7mhWckPuXZ68LUq2MENlrtB3NLBeTckL/kBdGfcuHGh5gadS8MB0B4XiiFJq1evDjUX+jJ58uRQc4Pia9asKV4ntxy3v7hB+laCOtw2ln5xcE3DP9qWUgrvv3vvW9k/S8Mt3HvvAo2qjlWl4Ueub9w6uu1u98udXX+6Y3fVl3y79339+vVtrRN6w513Jf/5uXOiC9Vwx0rX81XXG6VfLt3u8c6F1pSGSU2YMCHU1q5d29b6tIKfnAEAAABADXBzBgAAAAA1wM0ZAAAAANQAN2cAAAAAUAPcnAEAAABADaSqtLWuLCyl3i0MfSnnHKODuqjfetalcbk0Jpem5I4V7rlSeXKTS1NyqWitpIi5derlcc4sm54t5FI5SxM5Sx111FG27pZz+OGHh9qll14aanPmzAm1G2+80S7HJX65/nQprG7f6FLC6KKc84xuvLBT2rNVqa2lqYdNVXWcHTt2bKh1I+XTpT265bj33J0L+qFnpWYfa0u5NNLS9NCqvnXnfdejLmnS9ZNLmZR8gq+7lnC93CtV1wf85AwAAAAAaoCbMwAAAACoAW7OAAAAAKAGuDkDAAAAgBqIk3UAGsUN10o+PKDTg69VQRtuMNwNB1cN8g5VNUC+/fbbh9pzzz1X9Jo777xzqK1cubLouegOF8oxZcqUUNt7771D7YQTTgi1o48+OtTe/e5322W7/cUFITiXX355qLmwBkn6+te/HmpuyH3dunWhtu222xatTxMNPT64Y4sb8Jd8eEppIIj7nEqDhiTfN27d3WdcGpxUddweGBiw9U5z21jKBTB0KRAERtUxzPWUO764QBB3nHYhIVV968I/JkyYEGouPKldrh9d0FC7ITrt4idnAAAAAFAD3JwBAAAAQA1wcwYAAAAANdDWzFlKaamkAUmvSXq1118ACGwJ+hZNQ8+iaehZNA09i7pIVQP9RU9+o5Fn5JyfLXx833+bOrqr6tvUW9FK3za5Z91wrxuGdUP2I0eODLXp06eH2rHHHmuX/Z73vCfU9tprr6La8uXLQ22fffaxy3FDyG7I3g01V4ULdBo9W84Nr8+fPz/UXKhHVQDHUM8+69/CHXbYIdRcj7gh96pQHmfBggWh9t73vrf4+T2yqN0L01Z6dptttslDh/LbCaEYfM1Qc8dEF27QDS5QpJ3rL8n3nQtZcst2x/iq98IFrjhuf+lh+EdPe3bw8bU/1rpzvlQeDjZr1qxQ+8AHPhBqv/7rvx5q73jHO+xruuP8vHnzQu3LX/5yqC1cuDDUqnrM9a0L+hjOY0XV9QH/rBEAAAAAaqDdm7Ms6QcppUUppTmdWCGgB+hbNA09i6ahZ9E09Cxqod3vOTs857wspTRF0g9TSo/knH/l32wMNjhNjjp5076lZ1FD9Cyahp5F03BNi1po6ydnOedlg7/+UtKNkmaax1yZc57BYCXqYnN9S8+ibuhZNE0rPVv1Bc9AL3FNi7rY4p+cpZTGSNom5zww+PtjJX2mY2sGdEE/9q0b7JbKh7OPO+64UJszJ/7FoHucG+yV/NCtC+pwQ/8uJOTDH/6wXY4bIh4zZkyouZARtz49HF4v1o89WzWk7vph/Pjxxc8vcd1119n6kiVLQu2mm24KtWXLloXaySefHGrf/va37XIOOeSQUJs5M1wD6vHHHw+1559/3r5m3bTasznn8Nm7z7gqxMAdA88///xQc5/dj3/841B7+umnQ63q2OBCiVytNPzDhRgceOCB9rEHH3xwqLngJvc495o/+clP7HKuuuqqULv++uvtY4dy++8LL7xQ9Nxe6sfjrFS9z7jQrjPPPDPU/uAP/iDUXFiG6/lXXnnFLtvtSy5k5Oijjw61uXPnhtonP/lJuxx3HeK4fW7dunVFz+2Wdv5Z406Sbhz8G6+3SPo/Oef/25G1ArqHvkXT0LNoGnoWTUPPoja2+OYs57xE0kEdXBeg6+hbNA09i6ahZ9E09CzqhCh9AAAAAKgBbs4AAAAAoAbajdIHMMxc8IckzZgRw6T+5E/+JNRmz54dapMmTQq1VatWhZobFJekW265JdTcgO2sWbNC7Ywzzgi1j3zkI3Y5EyZMCLXPfCbOcH/qU58KtSuuuMK+JrqvakjdcX0zatSoUHvkkUdC7dOf/nSoVQWClHJBMi4c4Q//8A/t848//vhQ+9CHPhRq55xzTqi5Qfyqofumc2FD69evt491/fS3f/u3obbNNvHvo5cuXRpqLj2yKlHSLduFmbjPzj3XhSWMHTvWLnvy5Mm2PtTAwECouWPnsccea5/vwh7uv//+UHP74HAHK8A79dRTQ63qmDWU66dx48aF2nbbbWef70J43L7wW7/1W0U1FwIm+b7dsGFDqLlrqOEODOMnZwAAAABQA9ycAQAAAEANcHMGAAAAADXAzRkAAAAA1EAtA0HcN8pL5d8q7wZvHTdI7QZ5JT+s6IaL3dBwzjnU3KDkSy+9ZJftnu9qjhuqdkOSUnWwxFBuOHlrH/qtGkitGmAfauTIkaHmeskNpFYN8X71q18NNdd3rubCMm677bZQc0EIku9PF+LggkPuvPPOUNtrr73sci6//PJQc724tfdnU7hj6tFHHx1qbj9w+4s7F7hB76rXbMczzzxj6y+++GKovfOd7yx6zX4N/5Diubf02Cn587M7Brlz39SpU4teryoQxNXd5+TOxRs3bgy1VgJB3PPdcqqua4aqugb4nd/5neJ1Kll21TUIOu/QQw+1dRe85dxzzz2hduKJJ4baihUritfJHedPOeWUUHPhHwcccEDR+kjSv/7rvxYt2+1zw92j/OQMAAAAAGqAmzMAAAAAqAFuzgAAAACgBrg5AwAAAIAa4OYMAAAAAGqgp2mNI0aM0Lhx436ltmbNmvC4qlTG3/zN3ww1l3LnkpJKk7yq0rBcipF7zSVLloTaqlWrQm3Dhg2h5lKWJGnXXXcNNZcwNXPmzFCbPn16qH3ve9+zy7nhhhtCzSXsueQ79zmUpj/WWUoppBm6hKxWksXc5+x6yfWI81d/9Ve27tK0XFLRySefHGo33nhj0bKruPTK0vfIJSedd9559rEuVW3evHmhdu2114ZaK4mp6I3SBFzHpfO1cgxyqXKlKb1uOVVpdq7vli9fXrKKdtnDnSrWKUPfa5eqWdUL7rx9+umnh9o555wTaocccshm16VqfaqU9qw7b7rP062PJK1cuTLU9thjj6Lnt5JI6Y7dbt3d9ULpeQzdsc8++9j60GtxSVq9enWonX322aH27LPPFi276prWXUP94Ac/CLWHHnoo1Pbbb7+i15P88bI0/byV+4Nu4CdnAAAAAFAD3JwBAAAAQA1wcwYAAAAANbDZm7OU0lUppV+mlBZvUpucUvphSumxwV8ndXc1gdbQt2gaehZNQ8+iaehZNEHJhOtcSf8g6ZpNahdJujXn/HcppYsG//znm3uh1157LQSAuIFBN8QnSf/yL/8SanvuuaddzlBu0NUtx4VdSH5Y3A2/DgwMhNqLL74Yam7Y0D2uatk777xzqLmhXTeUXjV87oIUSod5aziUPlcd6tt2ts0Nkbv+dIOmrkeOPPLIUJs2bZpdthvudcEF3//+90PN7S+u391QuOT3g+233z7U9t1331D7x3/8x6L1kaRrrrkm1P70T/801Nw+VMPwj7nqUM/WXdWguOvP0v3PfcalwQxVj3X7qtsv3XoffPDBdjnusb/4xS9KVtGGiQxz4MJcdahnh56PXTBQFfeZuGPDo48+GmqHHXZYqLle+tznPmeX7XrErbs7fq5duzbUHnjggVC7/fbb7bLnz58far/9278dar/3e78Xai5ArCp45Oabbw61H//4x/axDTBXW8lxVpJmzJhR/Nhbbrkl1Nzn7I7JEydODDUX+FfFrefb3va2UHPHz6oQQXf95fZNFwgy3Db7k7Oc8wJJzw8pnyTp6sHfXy1pVofXC2gLfYumoWfRNPQsmoaeRRNs6czZTjnnFYO/Xylppw6tD9BN9C2ahp5F09CzaBp6FrXS9vec5ZxzSqnyZ4IppTmS5rS7HKCT3qxv6VnUET2LpqFn0TRc06IOtvQnZ8+klHaRpMFff1n1wJzzlTnnGTnn8n/4CnRHUd9u2rOtzKwAXdByz/Z07YCInkXTcE2LWtnSn5zNk3SapL8b/PWmLV0BFyhQNdx33XXXhZobYHXBGqWDgVUDjG5AePTo0UXr4waB3ePuv/9+u+xly5aFmlv3008/PdTOOeecUPuv//ovu5zSbz93YSQrV64seu4wa7lvc85hANUFyVQFY7j3tHTQffz48aF20EEHhVpViM2kSTFw6vjjjw81F4zhBmTdflUV7OD6e4cddgi1yy67LNTcei9atMgux/W829fdZ1Y1/F4zHTvW1snGjRs7/pruL1JaCZVwXN+44KVZs+KIyq/92q/Z13R9t2DBgqL1KT1GD7Mt6tl2gpfcc91xwJ37XM2FF33hC1+wy3Y94mrumNqNY5DbHhe28K53vSvUqs5jLnDFXf+457cSCjGM+vI4K0n33Xefrbvee9/73hdqO+64Y6i5sDH3OVf95bYL8jv//PNDzYWMfPvb3w61e+65xy7HhYe4XnbHD/e4Xh5/S6L0vyPpTkn7pJSeTimdoTca+JiU0mOSjh78M1Ab9C2ahp5F09CzaBp6Fk2w2Z+c5ZxnV/yvozq8LkDH0LdoGnoWTUPPomnoWTTBls6cAQAAAAA6iJszAAAAAKiB1Mtvxn6zeNISLnygdLC8atB1KDdAWPV8NzDohsVdmIh7brtD8jfffHOoveMd7wi13/iN37DPf/75od/L6LX7TfDtyDn3ND4xpZSHDrW6IddWhtndILUL5XADu26I+84777TLcT27ePHiUDvggANCzYVybNiwIdSqevZtb3tbqH3lK18JtQ9+8IOhdtttt4XaWWedZZfzxBNPFK2n4/bLbgzoD0fP9nJ5neSCFFywg9sH3T7UitJkVnfOvOuuu0LN7auS9Nxzz4XaLrvsEmrufFfa2x2wqJdpdK5n3XGyigsrcgFELsigdJ93x4tWnu+4nnPXBlW96cLGpk+fHmru2sC9P1X70LXXXhtqZ5xxhn3sMOppz0rNONaOHTvW1pcsWRJq7truF7/4RagdccQRobZ27dpQc/ulJM2bNy/U3v/+94eaC3Q69thjQ81dM1Rx10Uu6KNXIWJV1wf85AwAAAAAaoCbMwAAAACoAW7OAAAAAKAGuDkDAAAAgBrY7PecddKIESPCcKIbInRDiZK0bt26UCsdFq8K+ijlnl86HOge10oQi3s/dtttt1A75phjQu1b3/pWqFUFf0yePLnosS78ww1PthKSUWdDt63d7XJDrtttt12ouWHae+65J9T+/u//3i7nzDPPDDU3LO568Xvf+16o/du//Vuo/fu//7tddmn4xwMPPBBqf/3Xfx1qLshE8qEBbqDeDfy65w4MDNjloLPc8ULy+1bpsdsd912oRtX+WxooMmNGzBw46KCDQq0qQOLqq68ONbcPtnvOapqhn587TrZy7H322WdDrTScyx0v2g0DcMtx1y+thL64nnUhYKVhN+48JPlrA3f8nDJlSqjtueeeodZKgAPaU/XZu+PYHXfcEWqun+67775QmzVrVqh94hOfsMt24R/OTTfdFGo//elPQ62qb0uPIe74243wj1bwkzMAAAAAqAFuzgAAAACgBrg5AwAAAIAa4OYMAAAAAGqgp4Egr7/+ujZu3LjZx7mQEKk8RMMN2bpBYPd6VYPqjhs2dNw3tLtB3qphZxcMsd9++4WaG4pcvXp1ySpK8uEf48ePDzX3XraynKYpGQx13zov+aH+dsJl3Ht/6aWX2sfuscceoXbiiSeGmuv5448/vui5jz/+uF32XnvtFWqPPPJIqJ177rmhtnDhwlCrCldw+4b7LFwwBOEfw6fqWOc+O9ef7jzijufuGO3CHqq445/rWXc8X7VqlX3NK664ItRGjRoVai4Ywg33txIuVWdDt62VQJQJEyaEmruOcMdZd2xxfVi1Pq4/J02aFGovvPBCqJWGf+y///62/vGPfzzUXDDD9ttvH2puH3KhJZI0c+bMUHNBVEcccUSoufdn2rRpdjnovKrznKufdNJJoXb99deHmguI+c///M/idXLHYBdCdvHFF4eaCwas4nrPHQNa2d97hZ+cAQAAAEANcHMGAAAAADXAzRkAAAAA1AA3ZwAAAABQA5u9OUspXZVS+mVKafEmtUtSSstSSvcO/hfTA4BhQs+iiehbNA09i6ahZ9EEJWmNcyX9g6RrhtS/nHP+YisLyznbVKuhRo8ebesumc0lVblEK8eltlSliJUaM2ZMqB100EGhNnXq1FCbMmWKfc3DDz881A455JCi9Zk9e3aovfvd77aPdUlNzz33XKg98cQToXbRRReFWrvvZRvmqkM9W6oq5bM0Xa00TdT1bFW6qfvsd99991Dbd999Q+2SSy4JtXe+852h5lIZJb+eO+ywQ6hVJdoNVdVLLpnVpSy5/bKG5qrHfTtcXHqmJL3nPe8JtT333DPUnnzyyVBbtGhRqLmEvFbWye0vLsnUJeW61FFJeuqpp0LNpUq643ErSZM9Mlcd6tmSpOSq1FZ3DHQpyS7prTSVrWrZrm+mT58eagcccECo7bPPPqE2bty4UKu6Nnj/+98faqXb49a76rnLly8PtZ///Oehtnjx4lBzqY7DbK62kuPsm3E9de+994aaS/988MEHQ62V6+nSY5tbTivJiqVJtqXXZKWp2p2w2aNhznmBpJixDtQUPYsmom/RNPQsmoaeRRO0M3N2Zkrp/sEfEccv9RiUUpqTUro7pXR3G8sCOoGeRRNttm/pWdQMPYum4foAtbGlN2dflzRN0sGSVkj6UtUDc85X5pxn5JxnbOGygE6gZ9FERX1Lz6JG6Fk0DdcHqJUtujnLOT+Tc34t5/y6pH+WFL8+HqgRehZNRN+iaehZNA09i7opCQQJUkq75JxXDP7xdyXFKdA2rF+/vq3nd2NoujTYwQ0rnnzyyaE2Z86cUHMDzFXLcYOW7nFuIL5qOW7wfvz48aE2MDAQahdeeKF9zbrods9u3LixreeXDnE7VYP0bnh15cqVobZ06dJQc6EHM2fG89WGDRvsst1+4AJB/uM//iPUjjzyyFB79NFH7XLc0K4LV6hhkEKRbvZtVcCBC1lxQU6lx8Q/+qM/CrW//Mu/tMueNm1aqLk+duFQ9913X6h961vfCjU39F5V/9KX4l+gT5w4MdTuuOOOUHPHeKk8KMntLytWrDCPrJct7dmh+60LWSkJFPsfpcfUHXfcMdRuvfXWUNt7773t812whjsuuse5fcitd9W+6nqpdL90j3N9LEkf+chHQq0JvViq29cHdeTC3kaNGhVq733ve4tez5133T4s+fPxKaecEmou+M497qGHHrLLcftSK4Eiw2mzN2cppe9IOkLSDimlpyVdLOmIlNLBkrKkpZI+1sV1BFpCz6KJ6Fs0DT2LpqFn0QSbvTnLOcc8bumbXVgXoCPoWTQRfYumoWfRNPQsmqCdtEYAAAAAQIdwcwYAAAAANZBKv0G7IwtLqXcL20JuGF6SJkyYEGpuoLJUK0OJbp2+8IUvhNrHP/7xUDv22GNDbeHChSWrKKl+w5M55zjF3EVN6NkqLszFBcS4UI73ve99oeaCR6688kq77K9+9auhdskll4TarFmzQm3evHmh9rGP+REAtz2lgQu9Mhw9O/SzcvtxuyE2O++8c6h95jOfCTUXJlAV+vTggw+GmguD2X///UPNDY+7EIbly5fbZbt1evvb3x5qrr9OPPHEUPvJT35il+Pe99LAmtKwhw5Y1Muo8G222SYPPc+1EuLjjk2u5kIL9tprr1D74Q9/GGouoEXyAVvuHOmCbVwAg1vHqusSdw2y/fbbh5rr2dWrV4fawQcfbJfz9NNPh1rptcHo0aNDzQX6dEBPe1ZqxvVBVe+4PjvwwAND7Wc/+1mouV52jzvhhBPssk8//fRQu+CCC0JtzJgxoeaO0y50SvLXNnVTdX3AT84AAAAAoAa4OQMAAACAGuDmDAAAAABqgJszAAAAAKiBzX7P2dbGDUlK7YV/tDsQ6wY63eCt+zZ2N8jbCrfubtmlg5voPPe5S9KOO+4Yai5wYbfddgs19xm7oVsX3iH5z/7uu+8OtVNPPTXUTjnllFA77bTT7HIQpZTCMaPd8A8XiORCWj760Y8Wvd6nPvUpW//a174Wai4YYtKkSaHmtvHP/uzPQu3zn/98ySpWcmFMDz/8cKgNDAwUv2bpcbbq/NR0OeeWAkCGGjFiRKi50ALnmWeeCbVrrrkm1KqOQS4QxIW0uPCPNWvWhJoLVqh6b1zglwtJcsE4Ljik6nrBnWPce+56tkvhHyhUdcxw15XTpk0LtdKwnaVLl4bas88+a5ftAsPmzp0bau5a4PLLLw+1f/qnf7LLOe+880LthhtusI8dyvX8Sy+9VPTcTuAnZwAAAABQA9ycAQAAAEANcHMGAAAAADXAzRkAAAAA1ACBIEO4wWzJDwKWDhynFL8A3NXcELHkB93dsKIbvJ06dWqoVYWbuAF2N1zsBpsJ/xg+rpckacGCBaG20047hZrrm7vuuivUvvOd7xSvkwuIKR2mdcPvVYEWbtDdPd/tq26o+fXXXy9ZxVrLOdvB/KGqjnVugHzt2rWhNmvWrKL1uf3220PtK1/5StFzJWncuHGhtnr16qLnrlq1KtSq3pt169aFmuvjs88+O9R+9KMfhVrVfrls2bJQ27BhQ6i5gf1Wzhv9aOTIkbZe0u+StO2224aaO+9deumloVYVfuQCldxxZMqUKaHmgmTuvffeUKvavkceeSTU9txzz6L1cdcBVe+vO36647l7vntu6bUT2ld1HHI94QJm3PHFnWP/5m/+JtQmT55sl/3888+HmjvHf+Mb3wg1dz64+OKL7XJcaJULBHHH2l6Gfzj85AwAAAAAaoCbMwAAAACoAW7OAAAAAKAGNntzllLaPaV0W0rpoZTSgymlswfrk1NKP0wpPTb4a/xWUGAY0LNoGnoWTUTfomnoWTRByU/OXpV0fs55P0mHSvpkSmk/SRdJujXnvLekWwf/DNQBPYumoWfRRPQtmoaeRe1tNq0x57xC0orB3w+klB6WtKukkyQdMfiwqyX9SNKfd2Ute8gl10k+zaU07c29pkvQccuQfAKSW87YsWOLlu3Sqaq45bhkMbeOpQlanba19ewee+xh6+4zGTFiRKjdeeedofb7v//7oTZhwoRQq0o0cumdO+ywg33sUPfff3+ouXRSySdJlSaBufdiuNIaO9mzKaWQSuf2xar3yaU1Oi4x0aW73nLLLaHmUhAl3zfueOUSYw899NBQu+qqq0KtKvnTHStdj7j9YP78+aFWlW7q0h7dfuTey7rpVN+mlMLxyiXCtasqjXAot78sWrTIPnbJkiWhVvrZuX3Q9dz5559vnz9t2rSi5bhrlc9//vOh1u452yXArlmzpq3X7LSt7fqgKpXXHWvHjx8fau58cNNNN4Xa4sWLQ82lo0r++O/2TXeO+dznPhdqn/3sZ+1yjjnmmFCbPXt2qLlzlEv67aWWZs5SSntIOkTSXTaIDoQAAA2dSURBVJJ2GmxySVopKWZ0A8OMnkXT0LNoIvoWTUPPoq6Kv+cspTRW0g2Szsk5v7DpT35yzjmlZL9sJaU0R9KcdlcUaBU9i6ahZ9FEW9K39CyGE8da1FnRT85SStvqjSa+Nuf83cHyMymlXQb//y6Sfumem3O+Muc8I+c8oxMrDJSgZ9E0nerZqi8dBbphS/uWnsVw4foAdVeS1pgkfVPSwznnyzb5X/MknTb4+9MkxX+ECgwDehZNQ8+iiehbNA09iyYo+WeNh0n6iKQHUkr3Dtb+QtLfSbo+pXSGpCclndKdVayH0sFdFyjgQgtcrSqMwA1Gu+H1559/PtQeeugh+5rOjjvuGGpuQHjt2rWhVhoi0CNbVc9WDbm7AA7XSxdeeGGolQ5xVwV1uEH1U04pe7vvuOOOUHP7mlQd4FPy/G4EDrShYz2bcy7aH6vCXBwXwOECL4488shQ++AHPxhqX/ziF4uXPWXKlFA74YQTQu2KK64INRf+4QKNJOm4444LtQcffDDULrvsslA799xzQ80NnkvSzJkzQ+3DH/5wqD311FOh9swzz4TacIXYDOpI3+acw/7otqtqW13gheNCENwxzIVzVe1TLrSgHe5aw4V3SP4Ytm7dulCbNCmmwt98883F6+SCIlzoSRNCbLSVXR+4npf8PrNw4cJQ+9CHPhRq7tzhXq/qHOvqVcfloaZPn170OElasWJFqC1fvjzUXPiHCzPp5TVDSVrjQklV/+bgqM6uDtA+ehZNQ8+iiehbNA09iyZoKa0RAAAAANAd3JwBAAAAQA1wcwYAAAAANVD8PWdbOxfgURqC4QYl3/rWt4aaGwSW/PDlhAkTQs0N7buQj6qh3Xa+EX2Yh9K3ag8//LCt33rrraF21FHxn9SfeuqpoeZ6wQWPHH744XbZH/vYx0Jt3333DbWlS5eG2g033BBqpcEfkl/PmoV/dN3Q/dG9Jy70QPLvtRvWvu6660LtrLPOCrUDDzww1Fzoi+SPi+PGjQu1iRMnhpoLfXFD3fvvv79dthseHz16dKhdcMEFoXb77beH2qc//Wm7nIMOOijUFixYEGp//Md/HGouhKVflJxDqiL33WfvzrvuXFoajFMVOuLW2/XdmDFjQs3tV60E9bjwD3dt8MQTT4TasmXLipczMDBQ9Dj3XrjrktLwB3SP25d+9rOfhZr7rE488cRQmzp1aqi587vkQ3jc9fSuu+4aaldeeWWoVZ3f3bWuO1Y7bh17eR3BT84AAAAAoAa4OQMAAACAGuDmDAAAAABqgJszAAAAAKgBAkGGqBqSd8OKbqDSDSa757qh+6phZ8c93w3eumH6J598sng5ztixY0PNDSajN6qCaebMmRNqP/jBD0LNBRyce+65oeb2DReiIEm77LJLqK1ZsybUrrnmmlBbuHChfU3HDd6//PLLRc91A/rr168vXnZdpZRCAIgLGSh9nyR/HHEhAe9617tCrd0hahcSsnHjxlArDR5wPSOVr6cbFJ8/f36o3XTTTfb5pdvj9HO4wtDjiwvgqDpHuv6uOpcPNX78+FBzQQJVgSVuPd0x2R3/nOnTpxc/d9KkSaHm3otvfOMboeb2X3f9IvmwMvdZuOC0funPpnLnOcmf6+66665Qu/DCC0Pt8ssvD7XHH3881K6//nq7bLfPfOADHwg1d7xz563nnnvOLuf8888vek23bw/3NS0/OQMAAACAGuDmDAAAAABqgJszAAAAAKgBbs4AAAAAoAYIBBmiKlzBccOvrTy/5PWq3HvvvaF22223hZoLDhk9erR9TfdYNyDsBo4xfFxAgSQtWbIk1A477LBQO++880Lt9NNPDzUXZOCGgCXp5ptvDrXPfvazoVY1yNsOF/jgwh76IfzDyTl3fB914QFOu+EfTmlYRmnwQLvr2O57W7o9Tj+HK7Rz7mzn9Vz4RyvcedudY93xxoWWfPSjHw21iRMn2mW7cAR3Pnjsscfs84eqCstxgSBuu11IiKtVhaug89x1XRX3mV511VWhtnr16lA788wzQ2327NnFy3bcNcx3v/vdULv66qvt8xcvXly0HHed62puP+gWfnIGAAAAADXAzRkAAAAA1AA3ZwAAAABQA5u9OUsp7Z5Sui2l9FBK6cGU0tmD9UtSSstSSvcO/nd891cX2Dx6Fk1Dz6Jp6Fk0EX2LJkibC6FIKe0iaZec8z0ppXGSFkmaJekUSetyzl8sXlhK5YkXaNnUqVND7amnnhqGNemenHOcLh5ia+vZbbbxf8fSztC1Gwx3w7BVy3Dr5AJFXMBBK8E4bjnu+a28ZqfRs2igRTnnGW/2gK2tZ6uCl9oJiBkzZkyoXXDBBaF21lln2edPmjQp1Nwx9ROf+ESoVYUoOO44O3LkyFBzx9keBohttmelra9v3eck+c/UfVal504X/DKc590qbj92695OcFMrqq4PNpvWmHNeIWnF4O8HUkoPS9q1s6sHdA49i6ahZ9E09CyaiL5FE7Q0c5ZS2kPSIZLuGiydmVK6P6V0VUop/hUOMMzoWTQNPYumoWfRRPQt6qr45iylNFbSDZLOyTm/IOnrkqZJOlhv/C3ElyqeNyeldHdK6e4OrC9QjJ5F09CzaBp6Fk1E36LOim7OUkrb6o0mvjbn/F1Jyjk/k3N+Lef8uqR/ljTTPTfnfGXOeUbJvwUGOoWeRdPQs2gaehZNRN+i7krSGpOkb0p6OOd82Sb1XTZ52O9KKvsqbqDL6Fk0DT2LpqFn0UT0LZqgJK3xcEl3SHpA0v9Es/2FpNl648e/WdJSSR8bHLR8s9eqX3RLQ7kEnpdffjnUXDKNS92TpBEjRoRaD5OWihQm39GzkkaPHh1qLoXxxRdfDLVXXnmlK+u0paoSKdtJZnSv2U7CZRV6Fg1UktZIz1Zw5+e3vCXmr7lEOHcedsdtyR+7XSpuafKcW7ZUfc1Q8nx3nO3S+aU0rZG+reBSC911xKuvvhpqrVwruutS1yeu5nrRXftK3Tmfd1o7aY0LJbknz293pYBuoGfRNPQsmoaeRRPRt2iCltIaAQAAAADdwc0ZAAAAANQAN2cAAAAAUAObnTlDPbmhXzcUOX78+FBbtWqVfU035Om4AVE3mIzh1enPxH3uVdwAeulwrhucd4PKUnnPOqXBIQDgzrmSP9a5c3FVaMFQ7jhZFaAxatSoUNuwYUOouUARV6sK/nDBDO7Y655fGiaC7hg3bpytDwwMhJo7J65fv77j61TaO73iri9cuE071xut4idnAAAAAFAD3JwBAAAAQA1wcwYAAAAANcDNGQAAAADUQOrlUHxKaZWkJwf/uIOkZ3u28O7qp22R6rs9b88579jLBdKzjVHX7aFnO6eftkWq9/b0tG/7uGel/tqeOm/LcB5r6/y+bIl+2p46b0tlz/b05uxXFpzS3TnnGcOy8A7rp22R+m97OqWf3pd+2hap/7anU/rpfemnbZH6b3s6pd/el37ann7alk7qt/eln7anqdvCP2sEAAAAgBrg5gwAAAAAamA4b86uHMZld1o/bYvUf9vTKf30vvTTtkj9tz2d0k/vSz9ti9R/29Mp/fa+9NP29NO2dFK/vS/9tD2N3JZhmzkDAAAAAPx//LNGAAAAAKiBnt+cpZSOSyn9PKX0eErpol4vv10ppatSSr9MKS3epDY5pfTDlNJjg79OGs51LJVS2j2ldFtK6aGU0oMppbMH643cnm6hZ+uDni1Dz9YHPVuuyX3bTz0r0belmtyzUn/1bT/1bE9vzlJKIyR9TdL/krSfpNkppf16uQ4dMFfScUNqF0m6Nee8t6RbB//cBK9KOj/nvJ+kQyV9cvDzaOr2dBw9Wzv07GbQs7VDzxbog76dq/7pWYm+3aw+6Fmpv/q2b3q21z85mynp8Zzzkpzzy5Kuk3RSj9ehLTnnBZKeH1I+SdLVg7+/WtKsnq7UFso5r8g53zP4+wFJD0vaVQ3dni6hZ2uEni1Cz9YIPVus0X3bTz0r0beFGt2zUn/1bT/1bK9vznaV9N+b/PnpwVrT7ZRzXjH4+5WSdhrOldkSKaU9JB0i6S71wfZ0ED1bU/RsJXq2pujZN9WPfdsXnzF9W6kfe1bqg8+46T1LIEiH5TfiLxsVgZlSGivpBknn5Jxf2PT/NXF70Jomfsb07NatiZ8xPbt1a+pnTN9u3Zr4GfdDz/b65myZpN03+fNug7WmeyaltIskDf76y2Fen2IppW31RhNfm3P+7mC5sdvTBfRszdCzm0XP1gw9W6Qf+7bRnzF9u1n92LNSgz/jfunZXt+c/VTS3imlPVNKIyWdKmlej9ehG+ZJOm3w96dJumkY16VYSilJ+qakh3POl23yvxq5PV1Cz9YIPVuEnq0RerZYP/ZtYz9j+rZIP/as1NDPuK96Nufc0/8kHS/pUUlPSPrfvV5+B9b/O5JWSHpFb/z74jMkba83EmAek3SLpMnDvZ6F23K43vjx7v2S7h387/imbk8X3yd6tib/0bPF7xM9W5P/6NmW3qvG9m0/9ezg9tC3Ze9TY3t2cP37pm/7qWfT4AYBAAAAAIYRgSAAAAAAUAPcnAEAAABADXBzBgAAAAA1wM0ZAAAAANQAN2cAAAAAUAPcnAEAAABADXBzBgAAAAA1wM0ZAAAAANTA/wNIiv+EqycrxAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 1080x432 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OB-bso3WL8LB"
      },
      "source": [
        "def Anomaly_score(x, fake_img, z_out_real, D, Lambda=0.1):\n",
        "\n",
        "    # テスト画像xと生成画像fake_imgのピクセルレベルの差の絶対値を求めて、ミニバッチごとに和を求める\n",
        "    residual_loss = torch.abs(x-fake_img)\n",
        "    residual_loss = residual_loss.view(residual_loss.size()[0], -1)\n",
        "    residual_loss = torch.sum(residual_loss, dim=1)\n",
        "\n",
        "    # テスト画像xと生成画像fake_imgを識別器Dに入力し、特徴量マップを取り出す\n",
        "\n",
        "    _, x_feature = D(x, z_out_real)\n",
        "    _, G_feature = D(fake_img, z_out_real)\n",
        "\n",
        "    # テスト画像xと生成画像fake_imgの特徴量の差の絶対値を求めて、ミニバッチごとに和を求める\n",
        "    discrimination_loss = torch.abs(x_feature-G_feature)\n",
        "    discrimination_loss = discrimination_loss.view(\n",
        "        discrimination_loss.size()[0], -1)\n",
        "    discrimination_loss = torch.sum(discrimination_loss, dim=1)\n",
        "\n",
        "    # ミニバッチごとに2種類の損失を足し算する\n",
        "    loss_each = (1-Lambda)*residual_loss + Lambda*discrimination_loss\n",
        "\n",
        "    # ミニバッチ全部の損失を求める\n",
        "    total_loss = torch.sum(loss_each)\n",
        "\n",
        "    return total_loss, loss_each, residual_loss\n"
      ],
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wl7OER2yL8LD",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 390
        },
        "outputId": "eaa9826b-c932-4fc4-b06f-a96280454dbf"
      },
      "source": [
        "# 異常検知したい画像\n",
        "x = imges[0:5]\n",
        "x = x.to(device)\n",
        "\n",
        "# 教師データの画像をエンコードしてzにしてから、Gで生成\n",
        "E_update.eval()\n",
        "G_update.eval()\n",
        "z_out_real = E_update(imges.to(device))\n",
        "imges_reconstract = G_update(z_out_real)\n",
        "\n",
        "# 損失を求める\n",
        "loss, loss_each, residual_loss_each = Anomaly_score(\n",
        "    x, imges_reconstract, z_out_real, D_update, Lambda=0.1)\n",
        "\n",
        "# 損失の計算。トータルの損失\n",
        "loss_each = loss_each.cpu().detach().numpy()\n",
        "print(\"total loss：\", np.round(loss_each, 0))\n",
        "\n",
        "# 画像を可視化\n",
        "fig = plt.figure(figsize=(15, 6))\n",
        "for i in range(0, 5):\n",
        "    # 上段に訓練データを\n",
        "    plt.subplot(2, 5, i+1)\n",
        "    plt.imshow(imges[i][0].cpu().detach().numpy(), 'gray')\n",
        "\n",
        "    # 下段に生成データを表示する\n",
        "    plt.subplot(2, 5, 5+i+1)\n",
        "    plt.imshow(imges_reconstract[i][0].cpu().detach().numpy(), 'gray')\n"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "total loss： [172. 226. 284. 302. 246.]\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAFjCAYAAAC5RXNaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dedhdZX3v/++dkEDmiSTEBAiGIQwytDFQ4VQQoRxAQXqkRPTQSglSEVAukHLagihOrVClFo0SA9ZK7QUcwHKOAhdCUIqMYsKYBIgZycyTOYH790d2fyc+38/iufez9nCvnffrunIl+WStvdba67um7Of+7hBjNAAAAABAe/Vp9woAAAAAAHg4AwAAAIAs8HAGAAAAABng4QwAAAAAMsDDGQAAAABkgIczAAAAAMhAqYezEMIpIYSXQgjzQghXNWqlgGahZlFF1C2qhppF1VCzyEXo7fechRD6mtnLZnaSmS0ysyfMbFqM8fnGrR7QONQsqoi6RdVQs6gaahY52a3EvFPNbF6McYGZWQjhdjM7w8wKCzmEwDdeo5QYYygxe0fWbAj6LanCF8yrdS+73n36+B8IePvtt1uybKVkzZrVWbdVqNkiqftE7WM1XTP2Z2p9VdzKGOPoEvO3tGYbfSyXfb2ic3KZ1yyz7NTlVPlaYi2u2do02b8xZfdpO2tZacZ1u1X3AkrR/UGZh7PxZva7nf6+yMyOLvF6QLN1ZM3269dP5lu3bu31a6qTlbopfeutt3q9DDOzPfbYw2VbtmxJWp+i5Q8aNMhlXV1dLtttN3/627Ztm1xOm3Vc3Rbtz9R9MnDgQJepWti0aVPyslMfAlXNbty4Ub5mo7XwwfD1kvM3pWb79u2bnKee/1TNqfe56PVUPe2+++4uU/upzDm6SP/+/V2mjo3t27e7rJ5rSYb/SZFlzbZb2fsDVcuqnspeO9UxrJajtqfoPiT1/kS9ZjOOzXqUeThLEkKYbmbTm70coFGoWVQNNYuqoWZRRdQtWqHMw9liM9t7p79PqGW/J8Y4w8xmmFXjI2B0NGoWVdRj3VKzyAw1i6rh/gDZKPNw9oSZHRBC2M92FPA5ZvaxhqwV0BwdWbP1fPyufhQl9eet1Y+sqNcrytWPD6kfO6uH+nEE9SOMSqY/wqh0XN0W/Ty/2ieqbtavX5+0nOHDh7vszTffTJrXTNe8+hFG9WOWRT/qOGLEiKR1Uj+S084fqaxTU2q26EdS1TkwdZ+oH+9T1HtvZrZ58+akLFXqj0Sa6eOlnh8LT12OMmDAgF4vO9Nzb8edZ82K7w/UjxGqH4ktc40uOmbU/lfXBHUMq+0p+vFFVXtq2EOO9djrh7MY4/YQwsVm9jMz62tmM2OMcxu2ZkCDUbOoIuoWVUPNomqoWeSk1JizGON9ZnZfg9YFaDpqFlVE3aJqqFlUDTWLXJT6EmoAAAAAQGPwcAYAAAAAGWh6K30AzaUGzZrpQa6NHvha1NhBDSxXA3FTB86rQfJmegC6MmTIEJepgc6pzQFQjmqKYWa2Zs0al6mmLyNHjnSZGii+du3a5HVSy1HHixpIX0+jDrWNqd9NlWnzj9JCCO79V+99PcdnanML9d6rhkZF56rU5keqbtQ6qu0u+/1hqj7Vubvoe+TU+75hw4ZS64TWUNddM73/Ur8XUp0r6/ne0NTvLyt7vlNNa1KbSQ0bNsxl69atK7U+9eCTMwAAAADIAA9nAAAAAJABHs4AAAAAIAM8nAEAAABABng4AwAAAIAMhKJua01ZWAitWxg6UozRtw5qok6rWdWNS3VjUt2U1LlCzWuW3rlJdVNSXdHq6SKm1qmV5zmxbGo2kerKmdqRM9WJJ54oc7Wc4447zmXXX3+9y6ZPn+6yu+66Sy5HdfxS9am6sKpjo0kdRp+KMU5pxgsrqTVb1LU1tethVRWdZwcPHuyyZnT5VN0e1XLUe66uBZ1Qs2bVPtemUt1IU7uHFtWtuu6rGlWdJlU9qS6TZrqDr7qXULXcKkX3B3xyBgAAAAAZ4OEMAAAAADLAwxkAAAAAZICHMwAAAADIgB9ZB6BS1OBaM908oNEDX4sabaiB4WpwcNFA3u6KBpCPGjXKZatWrUp6zb322stly5YtS5oXzaGacowZM8ZlBxxwgMtOO+00l33wgx902Xvf+165bHW8qEYIyk033eQy1azBzOzmm292mRrkvn79epf169cvaX2qqPv5QZ1b1AB/M908JbUhiNpPqY2GzHTdqHVX+zi1cVLReburq0vmjaa2MZVqwNCkhiAQis5hqqbU+UU1BFHnadUkpKhuVfOPYcOGuUw1TypL1aNqNFS2iU5ZfHIGAAAAABng4QwAAAAAMsDDGQAAAABkoNSYsxDCa2bWZWZvmdn2Vn8BINAb1C2qhppF1VCzqBpqFrkIRQP6k2beUchTYowrE6fv+G9TR3MVfZt6Peqp2yrXrBrcqwbDqkH2/fv3d9nkyZNddvLJJ8tlH3300S7bf//9k7IlS5a47KCDDpLLUYOQ1SB7Nai5qLlAo1Gz6dTg9fvuu89lqqlHUQOO7lau1G/hnnvu6TJVI2qQe1FTHuWRRx5x2fvf//7k+VvkqbI3pvXUbJ8+fWL3QfllmlDUXtNl6pyomhs0g2ooUub+y0zXnWqypJatzvFF74VquKKo46WFzT9aWrO16bM/16prvll6c7AzzzzTZR/+8Idd9od/+IcuO/DAA+VrqvP8Pffc47Ibb7zRZY8++qjLimpM1a1q9NHOc0XR/QE/1ggAAAAAGSj7cBbN7OchhKdCCNMbsUJAC1C3qBpqFlVDzaJqqFlkoez3nB0XY1wcQhhjZveHEF6MMf7ez2zUCpwiR07esW6pWWSImkXVULOoGu5pkYVSn5zFGBfXfn/DzO4ys6limhkxxikMrEQueqpbaha5oWZRNfXUbNEXPAOtxD0tctHrT85CCIPMrE+Msav255PN7LqGrRnQBJ1Yt2pgt1n64OxTTjnFZdOn+/8YVNOpgb1metCtatShBv2rJiHnnnuuXI4aRDxo0CCXqSYjan1aOHg9WSfWbNEgdVUPQ4cOTZ4/xe233y7zBQsWuOzuu+922eLFi1121llnueyHP/yhXM5RRx3lsqlT3T2gzZs3z2WrV6+Wr5mbems2xuj2vdrHRU0M1Dnw8ssvd5nad7/61a9ctmjRIpcVnRtUUyKVpTb/UE0MDj/8cDntkUce6TLVuElNp17z17/+tVzOzJkzXfaTn/xETtudOn7ffPPNpHlbqRPPs2bFx4xq2nXxxRe77M/+7M9cppplqJrftm2bXLY6llSTkQ9+8IMumzVrlss+/elPy+Wo+xBFHXPr169PmrdZyvxY41gzu6v2P167mdm/xhj/b0PWCmge6hZVQ82iaqhZVA01i2z0+uEsxrjAzI5o4LoATUfdomqoWVQNNYuqoWaRE1rpAwAAAEAGeDgDAAAAgAyUbaUPoM1U4w8zsylTfDOpv/iLv3DZtGnTXDZixAiXrVixwmVqoLiZ2QMPPOAyNcD2zDPPdNn555/vsk984hNyOcOGDXPZddf5Mdx//dd/7bLvfve78jXRfEWD1BVVNwMGDHDZiy++6LIvfOELLitqCJJKNZJRzRE+/vGPy/lPPfVUl33sYx9z2WWXXeYyNRC/aNB91almQxs2bJDTqnr60pe+5LI+ffz/R7/22msuU90jizpKqmWrZiZq36l5VbOEwYMHy2WPHDlS5t11dXW5TJ07Tz75ZDm/avbw3HPPuUwdg+1urADtnHPOcVnROas7VU9Dhgxx2e677y7nV0141LHw3/7bf0vKVBMwM123mzZtcpm6h2p3wzA+OQMAAACADPBwBgAAAAAZ4OEMAAAAADLAwxkAAAAAZCDLhiDqG+XN0r9VXg28VdRAajWQ10wPVlSDi9Wg4Rijy9RAyS1btshlq/lVpqhB1WqQpFlxY4nu1ODkXX3Qb9GA1KIB7N3179/fZaqW1IDUokG83/rWt1ym6k5lqlnGQw895DLVCMFM16dq4qAahzz22GMu23///eVybrrpJpepWtzV67Mq1Dn1gx/8oMvUcaCOF3UtUAO9i16zjOXLl8t848aNLjvssMOSXrNTm3+Y+Wtv6rnTTF+f1TlIXfv22WefpNcragiicrWf1LV48+bNLqunIYiaXy2n6L6mu6J7gD/5kz9JXqeUZRfdg6DxjjnmGJmrxlvK008/7bLTTz/dZUuXLk1eJ3WeP/vss12mmn+85z3vSVofM7N/+7d/S1q2OubaXaN8cgYAAAAAGeDhDAAAAAAywMMZAAAAAGSAhzMAAAAAyAAPZwAAAACQgZZ2a+zbt68NGTLk97K1a9e66Yq6Mr7vfe9zmepypzolpXbyKuqGpboYqddcsGCBy1asWOGyTZs2uUx1WTIzGz9+vMtUh6mpU6e6bPLkyS776U9/Kpdzxx13uEx12FOd79R+SO3+mLMQgutmqDpk1dNZTO1nVUuqRpS//du/lbnqpqU6FZ111lkuu+uuu5KWXUR1r0x9j1TnpM997nNyWtVV7Z577nHZj370I5fV0zEVrZHaAVdR3fnqOQeprnKpXXrVcoq62am6W7JkScoqymW3u6tYo3R/r1VXzaJaUNftT37yky677LLLXHbUUUf1uC5F61MktWbVdVPtT7U+ZmbLli1z2cSJE5Pmr6cjpTp3q3VX9wup1zE0x0EHHSTz7vfiZmZr1qxx2aWXXuqylStXJi276J5W3UP9/Oc/d9nzzz/vskMOOSTp9cz0+TK1+3k9zwfNwCdnAAAAAJABHs4AAAAAIAM8nAEAAABABnp8OAshzAwhvBFCmLNTNjKEcH8I4ZXa7yOau5pAfahbVA01i6qhZlE11CyqIGWE6ywz+yczu22n7CozezDG+NUQwlW1v3++pxd66623XAMQNWBQDeIzM/uXf/kXl+23335yOd2pga5qOarZhZkeLK4Gv3Z1dbls48aNLlODDdV0Rcvea6+9XKYG7apB6UWDz1UjhdTBvBkOSp9lDarbMtumBpGr+lQDTVWNnHDCCS6bNGmSXLYa3KsaF/zsZz9zmTpeVL2rQeFm+jgYNWqUyw4++GCX/fM//3PS+piZ3XbbbS77q7/6K5epYyjD5h+zrEE1m7uigeKqPlOPP7WPUxszFE2rjlV1XKr1PvLII+Vy1LSvvvpqyirKZiJtbrgwyxpUs92vx6oxUBG1T9S54eWXX3bZscce6zJVS1/5ylfkslWNqHVX589169a57Le//a3LHn74Ybns++67z2V//Md/7LI//dM/dZlqIFbUeOTee+912a9+9Ss5bQXMsl3kPGtmNmXKlORpH3jgAZep/azOycOHD3eZavhXRK3nu971Lpep82dRE0F1/6WOTdUQpN16/OQsxviIma3uFp9hZrfW/nyrmZ3Z4PUCSqFuUTXULKqGmkXVULOogt6OORsbY1xa+/MyMxvboPUBmom6RdVQs6gaahZVQ80iK6W/5yzGGEMIhZ8JhhCmm9n0sssBGumd6paaRY6oWVQNNYuq4Z4WOejtJ2fLQwjjzMxqv79RNGGMcUaMcUqMMf0HX4HmSKrbnWu2njErQBPUXbMtXTvAo2ZRNdzTIiu9/eTsHjM7z8y+Wvv97t6ugGooUDS47/bbb3eZGsCqGmukDgwsGsCoBggPHDgwaX3UQGA13XPPPSeXvXjxYpepdf/kJz/psssuu8xl//mf/ymXk/rt56oZybJly5LmbbO66zbG6AagqkYyRY0x1HuaOtB96NChLjviiCNcVtTEZsQI33Dq1FNPdZlqjKEGyKrjqqixg6rvPffc02U33HCDy9R6P/XUU3I5qubVsa72WdHg98w07Fybk82bNzf8NdV/pNTTVEJRdaMaL515ph+i8u53v1u+pqq7Rx55JGl9Us/Rbdarmi3TeEnNq84D6tqnMtW86Otf/7pctqoRlalzajPOQWp7VLOFP/iDP3BZ0XVMNVxR9z9q/nqaQrRRR55nzcx+85vfyFzV3gc+8AGXjR492mWq2Zjaz0X/ua0a+V1++eUuU01GfvjDH7rs6aeflstRzUNULavzh5qulefflFb6Pzazx8zsoBDCohDC+bajgE8KIbxiZh+s/R3IBnWLqqFmUTXULKqGmkUV9PjJWYxxWsE/ndjgdQEahrpF1VCzqBpqFlVDzaIKejvmDAAAAADQQDycAQAAAEAGQiu/Gfud2pOmUM0HUgeWFw107U4NICyaXw0YVIPFVTMRNW/ZQfL33nuvyw488ECX/dEf/ZGcf/Xq7t/LqJX9JvgyYowtbZ8YQojdB7WqQa71DGZXA6lVUw41YFcN4n7sscfkclTNzpkzx2Xvec97XKaacmzatMllRTX7rne9y2X/+I//6LKPfvSjLnvooYdcdskll8jlzJ8/P2k9FXVcNmOAfjtqtpXLayTVSEE1dlDHoDqG6pHamVVdMx9//HGXqWPVzGzVqlUuGzdunMvU9S61thvgqVZ2o1M1q86TRVSzItWASDUySD3m1fminvkVVXPq3qCoNlWzscmTJ7tM3Ruo96foGPrRj37ksvPPP19O20YtrVmzapxrBw8eLPMFCxa4TN3bvfrqqy47/vjjXbZu3TqXqePSzOyee+5x2Yc+9CGXqYZOJ598ssvUPUMRdV+kGn20qolY0f0Bn5wBAAAAQAZ4OAMAAACADPBwBgAAAAAZ4OEMAAAAADLQ4/ecNVLfvn3d4EQ1iFANSjQzW79+vctSB4sXNfpIpeZPHRyopqunEYt6PyZMmOCyk046yWU/+MEPXFbU+GPkyJFJ06rmH2rwZD1NMnLWfdvKbpca5Lr77ru7TA2mffrpp132zW9+Uy7n4osvdpkaLK5q8ac//anL/v3f/91l//Ef/yGXndr847e//a3L/u7v/s5lqpGJmW4aoAbUqwG/at6uri65HDSWOl+Y6WMr9dytzvuqqUbR8ZvaUGTKFN9z4IgjjnBZUQOJW2+91WXqGCx7zaqa7vtPnSfrOfeuXLnSZanNudT5omwzALUcdf9ST9MXVbOqCVhqsxt1HTLT9wbq/DlmzBiX7bfffi6rp4EDyina9+o8Nnv2bJepevrNb37jsjPPPNNlF110kVy2av6h3H333S574oknXFZUt6nnEHX+bUbzj3rwyRkAAAAAZICHMwAAAADIAA9nAAAAAJABHs4AAAAAIAMtbQjy9ttv2+bNm3ucTjUJMUtvoqEG2aqBwOr1igaqK2qwoaK+oV0N5C0a7KwaQxxyyCEuU4Mi16xZk7KKZqabfwwdOtRl6r2sZzlVkzIwVH3rvJke1F+muYx676+//no57cSJE112+umnu0zV/Kmnnpo077x58+Sy999/f5e9+OKLLvvsZz/rskcffdRlRc0V1LGh9oVqDEHzj/YpOtepfafqU11H1PlcnaNVs4ci6vynaladz1esWCFf87vf/a7LBgwY4DLVGEIN7q+nuVTOum9bPQ1Rhg0b5jJ1H6HOs+rcouqwaH1UfY4YMcJlb775pstSm38ceuihMv/Upz7lMtWYYdSoUS5Tx5BqWmJmNnXqVJepRlTHH3+8y9T7M2nSJLkcNF7RdU7lZ5xxhst+8pOfuEw1iPnlL3+ZvE7qHKyakF1zzTUuU40Bi6jaU+eAeo73VuGTMwAAAADIAA9nAAAAAJABHs4AAAAAIAM8nAEAAABABnp8OAshzAwhvBFCmLNTdm0IYXEI4dnaL989AGgTahZVRN2iaqhZVA01iypI6dY4y8z+ycxu65bfGGP8h3oWFmOUXa26GzhwoMxVZzbVqUp1tFJU15aiLmKpBg0a5LIjjjjCZfvss4/LxowZI1/zuOOOc9lRRx2VtD7Tpk1z2Xvf+145rerUtGrVKpfNnz/fZVdddZXLyr6XJcyyBtVsqqIun6nd1VK7iaqaLepuqvb93nvv7bKDDz7YZddee63LDjvsMJeproxmej333HNPlxV1tOuuqJZUZ1bVZUkdlxmaZS2u23ZR3TPNzI4++miX7bfffi57/fXXXfbUU0+5THXIq2ed1PGiOpmqTrmq66iZ2cKFC12mukqq83E9nSZbZJY1qGZTOiUXdW1V50DVJVl1ekvtyla0bFU3kydPdtl73vMelx100EEuGzJkiMuK7g0+9KEPuSx1e9R6F827ZMkSl7300ksumzNnjstUV8c2m2W7yHn2naiaevbZZ12mun/OnTvXZfXcT6ee29Ry6umsmNrJNvWeLLWrdiP0eDaMMT5iZr7HOpApahZVRN2iaqhZVA01iyooM+bs4hDCc7WPiP2XetSEEKaHEJ4MITxZYllAI1CzqKIe65aaRWaoWVQN9wfIRm8fzm42s0lmdqSZLTWzbxRNGGOcEWOcEmOc0stlAY1AzaKKkuqWmkVGqFlUDfcHyEqvHs5ijMtjjG/FGN82s++Zmf/6eCAj1CyqiLpF1VCzqBpqFrlJaQjihBDGxRiX1v76ETPzo0BL2LBhQ6n5mzFoOrWxgxqseNZZZ7ls+vTpLlMDmIuWowZaqunUgPii5aiB90OHDnVZV1eXy6688kr5mrlods1u3ry51Pypg7iVooH0avDqsmXLXPbaa6+5TDU9mDrVX682bdokl62OA9UQ5P/8n//jshNOOMFlL7/8slyOGrSrmitk2EghSTPrtqjBgWqyoho5pZ4T/+f//J8u+5u/+Ru57EmTJrlM1bFqDvWb3/zGZT/4wQ9cpga9F+Xf+Ib/D/Thw4e7bPbs2S5T53iz9EZJ6nhZunSpmDIvva3Z7setarKS0lDsv6SeU0ePHu2yBx980GUHHHCAnF811lDnRTWdOobUehcdq6qWUo9LNZ2qYzOzT3ziEy6rQi2mavb9QY5Us7cBAwa47P3vf3/S66nrrjqGzfT1+Oyzz3aZanynpnv++eflctSxVE9DkXbq8eEshPBjMzvezPYMISwys2vM7PgQwpFmFs3sNTO7sInrCNSFmkUVUbeoGmoWVUPNogp6fDiLMfp+3Ga3NGFdgIagZlFF1C2qhppF1VCzqIIy3RoBAAAAAA3CwxkAAAAAZCCkfoN2QxYWQusW1ktqMLyZ2bBhw1ymBlSmqmdQolqnr3/96y771Kc+5bKTTz7ZZY8++mjKKppZfoMnY4x+FHMTVaFmi6hmLqpBjGrK8YEPfMBlqvHIjBkz5LK/9a1vuezaa6912Zlnnumye+65x2UXXqiHAKjtSW240CrtqNnu+0odx2Wb2Oy1114uu+6661ymmgkUNX2aO3euy1QzmEMPPdRlavC4asKwZMkSuWy1Tvvuu6/LVH2dfvrpLvv1r38tl6Pe99SGNanNHhrgqVa2Cu/Tp0/sfp2rp4mPOjepTDUt2H///V12//33u0w1aDHTDbbUNVI1tlENGNQ6Ft2XqHuQUaNGuUzV7Jo1a1x25JFHyuUsWrTIZan3BgMHDnSZaujTAC2tWbNq3B8U1Y6qs8MPP9xlzzzzjMtULavpTjvtNLnsT37yky674oorXDZo0CCXqfO0ajplpu9tclN0f8AnZwAAAACQAR7OAAAAACADPJwBAAAAQAZ4OAMAAACADPT4PWe7GjVI0qxc84+yA2LVgE418FZ9G7sayFsPte5q2akDN9F4ar+bmY0ePdplquHChAkTXKb2sRp0q5p3mOl9/+STT7rsnHPOcdnZZ5/tsvPOO08uB14IwZ0zyjb/UA2RVJOWCy64IOn1/vqv/1rm3/72t12mGkOMGDHCZWobP/OZz7jsa1/7WsoqFlLNmF544QWXdXV1Jb9m6nm26PpUdTHGuhqAdNe3b1+XqaYFyvLly1122223uazoHKQagqgmLar5x9q1a12mGisUvTeq4ZdqkqQa46jGIUX3C+oao95zVbNNav6BREXnDHVfOWnSJJelNtt57bXXXLZy5Uq5bNUwbNasWS5T9wI33XSTy77zne/I5Xzuc59z2R133CGn7U7V/JYtW5LmbQQ+OQMAAACADPBwBgAAAAAZ4OEMAAAAADLAwxkAAAAAZICGIN2ogdlmeiBg6oDjEPwXgKtMDSI20wPd1WBFNfB2n332cVlRcxM1gF0NLlYDm2n+0T6qlszMHnnkEZeNHTvWZapuHn/8cZf9+Mc/Tl4n1SAmdTCtGvxe1NBCDXRX86tjVQ1qfvvtt1NWMWsxRjkwv7uic50aQL5u3TqXnXnmmUnr8/DDD7vsH//xH5PmNTMbMmSIy9asWZM074oVK1xW9N6sX7/eZaqOL730Upf94he/cFnRcbl48WKXbdq0yWVqwH49141O1L9/f5mn1LuZWb9+/VymrnvXX3+9y4qaH6mGSuo8MmbMGJepRjLPPvusy4q278UXX3TZfvvtl7Q+6j6g6P1V5091Plfzq3lT751QXtF5SNWEajCjzi/qGvvFL37RZSNHjpTLXr16tcvUNf773/++y9T14JprrpHLUU2rVEMQda5tZfMPhU/OAAAAACADPJwBAAAAQAZ4OAMAAACADPT4cBZC2DuE8FAI4fkQwtwQwqW1fGQI4f4Qwiu13/23ggJtQM2iaqhZVBF1i6qhZlEFKZ+cbTezy2OMh5jZMWb26RDCIWZ2lZk9GGM8wMwerP0dyAE1i6qhZlFF1C2qhppF9nrs1hhjXGpmS2t/7gohvGBm483sDDM7vjbZrWb2CzP7fFPWsoVU5zoz3c0ltdubek3VQUctw0x3QFLLGTx4cNKyVXeqImo5qrOYWsfUDlqNtqvV7MSJE2Wu9knfvn1d9thjj7nsf/yP/+GyYcOGuayoo5Hq3rnnnnvKabt77rnnXKa6k5rpTlKpncDUe9Gubo2NrNkQgutKp47FovdJdWtUVMdE1d31gQcecJnqgmim60adr1TH2GOOOcZlM2fOdFlR5091rlQ1oo6D++67z2VF3U1Vt0d1HKn3MjeNqtsQgjtfqY5wZRV1I+xOHS9PPfWUnHbBggUuS9136hhUNXf55ZfL+SdNmpS0HHWv8rWvfc1lZa/ZqgPs2rVrS71mo+1q9wdFXXnVuXbo0KEuU9eDu+++22Vz5sxxmeqOaqbP/+rYVNeYr3zlKy778pe/LJdz0kknuWzatGkuU7wPo84AACAASURBVNco1em3leoacxZCmGhmR5nZ42Y2tlbkZmbLzMz36AbajJpF1VCzqCLqFlVDzSJXyd9zFkIYbGZ3mNllMcY3d/7kJ8YYQwjyy1ZCCNPNbHrZFQXqRc2iaqhZVFFv6paaRTtxrkXOkj45CyH0sx1F/KMY4521eHkIYVzt38eZ2Rtq3hjjjBjjlBjjlEasMJCCmkXVNKpmi750FGiG3tYtNYt24f4AuUvp1hjM7BYzeyHGeMNO/3SPmZ1X+/N5ZuZ/CBVoA2oWVUPNooqoW1QNNYsqSPmxxmPN7BNm9tsQwrO17Goz+6qZ/SSEcL6ZvW5mZzdnFfOQOnBXNRRQTQtUVtSMQA2MVoPXV69e7bLnn39evqYyevRol6kBwuvWrXNZahOBFtmlarZokLtqwKFq6corr3RZ6iDuokYdaqD62Wenvd2zZ892mTrWzIob+KTM34yGAyU0rGZjjEnHY1EzF0U14FANL0444QSXffSjH3XZP/zDPyQve8yYMS477bTTXPbd737XZar5h2poZGZ2yimnuGzu3Lkuu+GGG1z22c9+1mVq4LmZ2dSpU1127rnnumzhwoUuW758ucva1cSmpiF1G2N0x6ParqJtVQ0vFNUEQZ3DVHOuomNKNS0oQ91rqOYdZvoctn79epeNGOG7wt97773J66QaRaimJ1VoYmO72P2Bqnkzfcw8+uijLvvYxz7mMnXtUK9XdI1VedF5ubvJkycnTWdmtnTpUpctWbLEZar5h2pm0sp7hpRujY+aWdHPHJzY2NUByqNmUTXULKqIukXVULOogrq6NQIAAAAAmoOHMwAAAADIAA9nAAAAAJCB5O8529WpBh6pTTDUQMk99tjDZWogsJkefDls2DCXqUH7qslH0aDdMt+I3uZB6bu0F154QeYPPvigy0480f9I/TnnnOMyVQuq8chxxx0nl33hhRe67OCDD3bZa6+95rI77rjDZamNP8z0embW/KPpuh+P6j1RTQ/M9HutBmvffvvtLrvkkktcdvjhh7tMNX0x0+fFIUOGuGz48OEuU01f1KDuQw89VC5bDR4fOHCgy6644gqXPfzwwy77whe+IJdzxBFHuOyRRx5x2Z//+Z+7TDVh6RQp15Cilvtq36vrrrqWpjbGKWo6otZb1d2gQYNcpo6rehr1qOYf6t5g/vz5Llu8eHHycrq6upKmU++Fui9Jbf6A5lHH0jPPPOMyta9OP/10l+2zzz4uU9d3M92ER91Pjx8/3mUzZsxwWdH1Xd3rqnO1otaxlfcRfHIGAAAAABng4QwAAAAAMsDDGQAAAABkgIczAAAAAMgADUG6KRokrwYrqgGVamCymlcNui8a7Kyo+dXAWzWY/vXXX09ejjJ48GCXqYHJaI2ixjTTp0932c9//nOXqQYHn/3sZ12mjg3VRMHMbNy4cS5bu3aty2677TaXPfroo/I1FTXwfuvWrUnzqgH6GzZsSF52rkIIrgGIajKQ+j6Z6fOIahLwB3/wBy4rO4haNQnZvHmzy1IbD6iaMUtfTzVQ/L777nPZ3XffLedP3R6lk5srdD+/qAYcRddIVd9F1/Luhg4d6jLVSKCoYYlaT3VOVuc/ZfLkycnzjhgxwmXqvfj+97/vMnX8qvsXM92sTO0L1TitU+qzqtR1zkxf6x5//HGXXXnllS676aabXDZv3jyX/eQnP5HLVsfMhz/8YZep8526bq1atUou5/LLL096TXVst/uelk/OAAAAACADPJwBAAAAQAZ4OAMAAACADPBwBgAAAAAZoCFIN0XNFRQ1+LWe+VNer8izzz7rsoceeshlqnHIwIED5WuqadUAYTXgGO2jGhSYmS1YsMBlxx57rMs+97nPueyTn/yky1QjAzUI2Mzs3nvvddmXv/xllxUN5C1DNXxQzR46ofmHEmNs+DGqmgcoZZt/KKnNMlIbD5Rdx7Lvber2KJ3cXKHMtbPM66nmH/VQ1211jVXnG9W05IILLnDZ8OHD5bJVcwR1PXjllVfk/N0VNctRDUHUdqsmISoraq6CxlP3dUXUPp05c6bL1qxZ47KLL77YZdOmTUtetqLuYe68806X3XrrrXL+OXPmJC1H3eeqTB0HzcInZwAAAACQAR7OAAAAACADPJwBAAAAQAZ6fDgLIewdQngohPB8CGFuCOHSWn5tCGFxCOHZ2q9Tm7+6QM+oWVQNNYuqoWZRRdQtqiD01IQihDDOzMbFGJ8OIQwxs6fM7EwzO9vM1scY/yF5YSGkd7xA3fbZZx+XLVy4sA1r0jwxRj+6uJtdrWb79NH/x1Jm0LUaGK4GwxYtQ62TaiiiGhzU0xhHLUfNX89rNho1iwp6KsY45Z0m2NVqtqjxUpkGMYMGDXLZFVdc4bJLLrlEzj9ixAiXqXPqRRdd5LKiJgqKOs/279/fZeo828IGYj3WrNmuV7dqP5npfar2Veq1UzV+aed1t4g6jtW6l2ncVI+i+4MeuzXGGJea2dLan7tCCC+Y2fjGrh7QONQsqoaaRdVQs6gi6hZVUNeYsxDCRDM7yswer0UXhxCeCyHMDCH4/8IB2oyaRdVQs6gaahZVRN0iV8kPZyGEwWZ2h5ldFmN808xuNrNJZnak7fhfiG8UzDc9hPBkCOHJBqwvkIyaRdVQs6gaahZVRN0iZ0kPZyGEfrajiH8UY7zTzCzGuDzG+FaM8W0z+56ZTVXzxhhnxBinpPwsMNAo1CyqhppF1VCzqCLqFrlL6dYYzOwWM3shxnjDTvm4nSb7iJmlfRU30GTULKqGmkXVULOoIuoWVZDSrfE4M5ttZr81s/9qzXa1mU2zHR//RjN7zcwurA20fKfXyq91S0WpDjxbt251mepMo7rumZn17dvXZS3stJQksfMdNWtmAwcOdJnqwrhx40aXbdu2rSnr1FtFHSnLdGZUr1mmw2URahYVlNKtkZotoK7Pu+3m+6+pjnDqOqzO22b63K264qZ2nlPLNiu+Z0iZX51nm3R9Se3WSN0WUF0L1X3E9u3bXVbPvaK6L1V1ojJVi+re16w51/NGK9Ot8VEzUzPfV3algGagZlE11CyqhppFFVG3qIK6ujUCAAAAAJqDhzMAAAAAyAAPZwAAAACQgR7HnCFPatCvGhQ5dOhQl61YsUK+phrkqagBompgMtqr0ftE7fciagB66uBcNXBeDVQ2S69ZJbVxCACoa66ZPtepa3FR04Lu1HmyqIHGgAEDXLZp0yaXqYYiKitq/KEaM6hzr5o/tZkImmPIkCEy7+rqcpm6Jm7YsKHh65RaO62i7i9Uc5sy9xv14pMzAAAAAMgAD2cAAAAAkAEezgAAAAAgAzycAQAAAEAGQisHxYcQVpjZ67W/7mlmK1u28ObqpG0xy3d79o0xjm7lAqnZysh1e6jZxumkbTHLe3taWrcdXLNmnbU9OW9LO8+1Ob8vvdFJ25PzthTWbEsfzn5vwSE8GWOc0paFN1gnbYtZ521Po3TS+9JJ22LWedvTKJ30vnTStph13vY0Sqe9L520PZ20LY3Uae9LJ21PVbeFH2sEAAAAgAzwcAYAAAAAGWjnw9mMNi670TppW8w6b3sapZPel07aFrPO255G6aT3pZO2xazztqdROu196aTt6aRtaaROe186aXsquS1tG3MGAAAAAPh/+LFGAAAAAMhAyx/OQginhBBeCiHMCyFc1erllxVCmBlCeCOEMGenbGQI4f4Qwiu130e0cx1ThRD2DiE8FEJ4PoQwN4RwaS2v5PY0CzWbD2o2DTWbD2o2XZXrtpNq1oy6TVXlmjXrrLrtpJpt6cNZCKGvmX3bzP67mR1iZtNCCIe0ch0aYJaZndItu8rMHowxHmBmD9b+XgXbzezyGOMhZnaMmX26tj+quj0NR81mh5rtATWbHWo2QQfU7SzrnJo1o2571AE1a9ZZddsxNdvqT86mmtm8GOOCGONWM7vdzM5o8TqUEmN8xMxWd4vPMLNba3++1czObOlK9VKMcWmM8enan7vM7AUzG28V3Z4moWYzQs0moWYzQs0mq3TddlLNmlG3iSpds2adVbedVLOtfjgbb2a/2+nvi2pZ1Y2NMS6t/XmZmY1t58r0RghhopkdZWaPWwdsTwNRs5miZgtRs5miZt9RJ9ZtR+xj6rZQJ9asWQfs46rXLA1BGizuaH9ZqRaYIYTBZnaHmV0WY3xz53+r4vagPlXcx9Tsrq2K+5ia3bVVdR9Tt7u2Ku7jTqjZVj+cLTazvXf6+4RaVnXLQwjjzMxqv7/R5vVJFkLoZzuK+EcxxjtrcWW3pwmo2cxQsz2iZjNDzSbpxLqt9D6mbnvUiTVrVuF93Ck12+qHsyfM7IAQwn4hhP5mdo6Z3dPidWiGe8zsvNqfzzOzu9u4LslCCMHMbjGzF2KMN+z0T5XcniahZjNCzSahZjNCzSbrxLqt7D6mbpN0Ys2aVXQfd1TNxhhb+svMTjWzl81svpn9r1YvvwHr/2MzW2pm22zHzxefb2ajbEcHmFfM7AEzG9nu9UzcluNsx8e7z5nZs7Vfp1Z1e5r4PlGzmfyiZpPfJ2o2k1/UbF3vVWXrtpNqtrY91G3a+1TZmq2tf8fUbSfVbKhtEAAAAACgjWgIAgAAAAAZ4OEMAAAAADLAwxkAAAAAZICHMwAAAADIAA9nAAAAAJABHs4AAAAAIAM8nAEAAABABng4AwAAAIAM8HAGAAAAABng4QwAAAAAMsDDGQAAAABkgIczAAAAAMgAD2cAAAAAkAEezgAAAAAgAzycAQAAAEAGeDgDAAAAgAyUejgLIZwSQngphDAvhHBVo1YKaBZqFlVE3aJqqFlUDTWLXIQYY+9mDKGvmb1sZieZ2SIze8LMpsUYn3+HeXq3MKAmxhh6Oy81a9anj///GHUOUFkI/q3v7fmjUXbbbbek6bZv397kNSlWpmbN6q/bTqvZMnWXY80q6rh8++2327Am/7+VMcbRvZ15V6rZqtRYo2W43S2t2do8HbWj1T5VqlLfVb4/SFtzbaqZzYsxLjAzCyHcbmZnmFlhIVeZ2smqQN966y2XVbng+/bt6zJ105Djugu7VM0qgwYNctmWLVtcpvaxquNt27Y1ZsV6ac8993SZOtGuXr3aZW2++a1H2+u2VTdiajn9+/d3mdrHal710LN169Zerl1jqHPqHnvs4bLNmze7TF1fmuT1kvO3vWZbRd0bqP2Uer4pe6y16kFfHZfqWtJC1GxJ6jykzrWqvlXdFZ2vWnW/qO4P1DqtWrXKZe2+PyjzY43jzex3O/19US0DckXNooqoW1QNNYuqoWaRjTKfnCUJIUw3s+nNXg7QKNQsqoaaRdVQs6gi6hatUObhbLGZ7b3T3yfUst8TY5xhZjPMOu/nc1E51CyqqMe6pWaRGWoWVcP9AbJR5uHsCTM7IISwn+0o4HPM7GMNWasMpY4vS51XKRqbpuZvxlgw9ZoVGUuWapeqWaWrqytpOlWLqj7KUq9Zz7ia4cOHu2z+/Pkua/fPj5fU9rpt53kgdRxLarObes6zihprU884NjXuc8OGDS6jZqth2LBhLlu5cmWvX6/ssdaMulHHlsoqbpep2SKbNm3q9byp51+z9HGVZe8PBg8e7LLXX/dDE3M81/b64SzGuD2EcLGZ/czM+prZzBjj3IatGdBg1CyqiLpF1VCzqBpqFjkpNeYsxnifmd3XoHUBmo6aRRVRt6gaahZVQ80iFx33uTQAAAAAVBEPZwAAAACQgaa30u8UjR4wWPaLJtWgyNTBk0WNHdTycxwoid4r8wWl6ssoy5o4caLLFi1a5DLVRMHM7NVXX3VZu78YGz2r5xyUel5s1blKrY86rlTjEDM96F6dp4salyAva9asafcq9Ci1gVg9TT7KNI9A69RzHlHTqjpR09Vzf5B6Tj/ggANctnDhQpepxh9muvlHVe4P+OQMAAAAADLAwxkAAAAAZICHMwAAAADIAA9nAAAAAJABHs4AAAAAIAN0a+ymqLONylM72yj1dGZMpTp+lZmuSGr3n2ZsI8ppZ30qquOX6qbU1dUl56ejXTVVuQtsaleyLVu2yLxfv35J83P+zEvRfqtC97dW3RugOsp0xm3VuUldJ7Zu3eqydevWyfmrfA7lkzMAAAAAyAAPZwAAAACQAR7OAAAAACADPJwBAAAAQAZ2mYYgqnGAGizYp49+Xk0dKJu6nCpT26MGS1dhoDTaa+nSpS6r57gcN26cy4YMGeKyF1980WW77eZPf0VNHNBYRY1cqtAopOz5XDUUGTBggMv69u3rMvX+bNy4sdT6IA3XM1RRle8/FyxY4DJ1L150fzBq1CiXjR492mUvvfSSy/r37++yDRs2yOU0A5+cAQAAAEAGeDgDAAAAgAzwcAYAAAAAGSg15iyE8JqZdZnZW2a2PcY4pRErBTQTdYuqoWZRNdQsqoaaRS4a0RDkhBjjyga8TlOlDopUTQKK5leDENVgbzVdFQa+F1HbUzTAP2OVqNsURe+9qmU1rRroPnDgQJeddNJJLrvxxhvlsp9++mmXqaYxxxxzjMuGDh3qsqJtvPnmm112zTXXuEwdvxVs/tExNatqwUwP9lbnSpWpelevV88AeVV3apD5hAkTkqYzM3vkkUdctnnzZpep82xqY6qMdEzNoncqeP/TUTVbdO1UDYfUtOqedtiwYS6bNm2ay770pS/JZc+bN89lqiYOP/xwl6lrR9E23nbbbS674oorkpbdyuYfCj/WCAAAAAAZKPtwFs3s5yGEp0II0xuxQkALULeoGmoWVUPNomqoWWSh7I81HhdjXBxCGGNm94cQXowx/t7PbNQKnCJHTt6xbqlZZIiaRdVQs6ga7mmRhVKfnMUYF9d+f8PM7jKzqWKaGTHGKQysRC56qltqFrmhZlE11Cyqhnta5KLXn5yFEAaZWZ8YY1ftzyeb2XUNW7M2qadJQOqg1swHvzZEVb6FvlPrVlGNPtTA2eHDh7tMNdU477zzXFa031XzkCFDhrhs+fLlya+pXHTRRS5Tg5Wvvvpqly1dujR5Oe3UiTWrBpmbpTe8UHV8wgknJC1n/vz58jWPPvpol6mGM6+++qrLjjzySJcVbcuHP/xhl82ePdtl6vitik6sWfTOgAEDXNbuZgvKrlaz6tyozqsjRoxw2fXXX+8y1RBENR0xMzvssMNcppqQdXV1uUw1BCk6V3784x932ZgxY1z2mc98xmWvvfaafM1WKfNjjWPN7K7aztzNzP41xvh/G7JWQPNQt6gaahZVQ82iaqhZZKPXD2cxxgVmdkQD1wVoOuoWVUPNomqoWVQNNYuc0EofAAAAADLAwxkAAAAAZKBsK300UNHgyYkTJ7rspZdectkbb7zhsrPOOstlzzzzjFxOajMU1eBkV2h6kis14NrMbOPGjS5TzTbWrVvnsiVLlrhMDSBetGiRXPbQoUOT1uemm25y2R133OGyD33oQ3I5X/va11x2/PHHu2zs2LEuq0pDEHiqjtX+fOKJJ1ymBr2bme22m78cqnOyajjzv//3/3bZySefLJfz1a9+1WXvfe97XVaVJkvAO8mx+ceuRDXQMDPbunWry9Q5R+0/dQ5U94BFTTUWLFjgsn333ddl113n+7G8/vrrLrvgggvkclTDMNWMZMKECS5rd0MQPjkDAAAAgAzwcAYAAAAAGeDhDAAAAAAywMMZAAAAAGSAhiAlqIHlQ4YMcZkaeDlr1iyXHXvssXI5qQPV1Teff+9733PZaaedJpezcOFCmaOxVN2UGfy/adOmMqsjB/I+99xzLtu8ebPL/vIv/1K+5vPPP580f+p2Dxo0SOYrVqxw2e233+6yVatWJS0HrVG2gZA6hg499FCX7bHHHi5T52MzswceeMBlv/71r112yy23uEzV19y5c+VyfvnLX7rsrbfektMCjdDoaw6ap9H7quh8V2b+2bNnu0zdv5577rnyNdX9QZltLGo6o87L3/rWt1y2bNmyXi+7WfjkDAAAAAAywMMZAAAAAGSAhzMAAAAAyAAPZwAAAACQAR7OAAAAACADu3S3xj59/LNpUceYcePGuexXv/qVy/baay+X9evXL2nZ69evl8tevHixy1QHR7WcK664wmVvvPGGXE4qOj+V0+j3Su2PssvZtm2by8aOHeuyww8/XM7/9NNPJ62POg5Uh9Girk/33HOPy77xjW+4jG6NeSl7DOy+++4u++Y3v5k07zPPPCPzD33oQy5T63n66ae77N/+7d9cVtSBMXU9UV3q+pzaoVRdx4cPHy6nfemll1z2xBNPuOyMM85wmerCV3RcqlpW153UewPuF4rldn+gpjvggANcpu6RVQdxs+JOtt3179/fZU899ZTLDjnkEDn/3//937tsxowZLtu4cWPS+rQSn5wBAAAAQAZ4OAMAAACADPBwBgAAAAAZ6PHhLIQwM4TwRghhzk7ZyBDC/SGEV2q/j2juagL1oW5RNdQsqoaaRdVQs6iC0NOgwBDCH5vZejO7LcZ4WC37upmtjjF+NYRwlZmNiDF+vseFhZD9KNB3v/vdMr///vuTp02xfft2l911111yWtXgQDUjUYN2hw0b5rJNmzalrGIhtd0LFy50mdrGsmKMenRrN42q2yrUrGqqYVZuILYalL5582aXFQ2kPfDAA1325ptvumzatGku+853vuOyouYKP/3pT132V3/1Vy5bvny5nL8VOqFmW9UEaMCAAS5T5ytVnwsWLHDZ+PHjXXbCCSfIZc+ePdtlH//4x112yy23uKxv374u+9KXviSX88UvftFlqQ0Xio6DJngqxjilp4kaWbPdtze3phFFjRXU+XfgwIEu+/nPf+4ydX2eMGGCy4YMGZKyinW57LLLXLZs2TI5rWrCMH36dJfNnDnTZfPmzXNZM+4NrMU1W5svryIViupW5alNa0aNGuUydY1du3atnF81qNmyZYvLrrrqKpd95CMfcVnRfYhq3vTYY4+5TDXHaZWi+4MePzmLMT5iZqu7xWeY2a21P99qZmeWWjugwahbVA01i6qhZlE11CyqoLdjzsbGGJfW/rzMzHyPbSA/1C2qhppF1VCzqBpqFlkp/T1nMcb4Th/thhCmm5n//Btoo3eqW2oWOaJmUTXULKqGe1rkoLefnC0PIYwzM6v9XvitxjHGGTHGKSk/Cww0WVLdUrPICDWLqqFmUTXc0yIrvf3k7B4zO8/Mvlr7/e6GrVGbFQ1gfPbZZ132zDPPuGzbtm0ue9/73ucyNfD98ssvl8u+9tprXaYGbr700ksuK9v8Qw0affXVV0u9Zht1ZN2OGzdO5osXL+71a6r6VAPf1XRmZr/+9a9dpo6NiRMnukwNFj/66KPlclTNpzZXUHJrQmCZ1Gyj3xfVMMGseGB3d2p9xowZ4zK131XDmSKpTZ9uvPFGl33hC1+Q06bWp9pGdQymDuJvoYbUbKua0CjqfVaNCMz0vlcNa1QTBbWNu+1W+geaHFUjV199tcuuv/56Of/SpUtd9v3vf99l8+fP78XaZSGL82wzFN0fLFmypNevuccee7hMHTOq4Y2Z2X/8x3+4TN0fqGOmq6vLZeo+wsxs3bp1LlPrqY5DNV0LGzIltdL/sZk9ZmYHhRAWhRDOtx0FfFII4RUz+2Dt70A2qFtUDTWLqqFmUTXULKqgx/+iiTH6Xtc7nNjgdQEahrpF1VCzqBpqFlVDzaIKejvmDAAAAADQQDycAQAAAEAGGj/ytOJWr+7+3YQ7/Omf/mlDl6MGG+6+++5y2tGjRyfN/8ADD7is7KBqNW1qcwW0xsqVKxv+mps3b06aTtWhWfEg5O7UIOBzzjnHZUVNaNR6ZtjUA92UbVSkBmarZgQHH3ywyw488ED5mqnnyscff9xl3/zmN5PWsUhqzXZybee0bSNHjnTZD37wAzntoEGDXLZlyxaXqeZFquGBmreo6ctrr73mMtXE5qGHHnLZRz/6UZepbTEz27Bhg8tefvllOW13ffv2TZoOzbFq1aqGv2bRfXJ3Rft+8ODBLlM1rpbz/ve/32WqSUjRa6Y2UGpl8w+FT84AAAAAIAM8nAEAAABABng4AwAAAIAM8HAGAAAAABmgIUibqEGJ++67r5z2qKOOctnWrVtdppojlG0Iktq4pOwAf/ReMwaupr5mUXMYNRB4yZIlLrvxxhtd9tOf/tRl27dvl8sZMGCAy9Sxobanf//+LlOD8dF4zWj+8OMf/9hl1113ncuKmtgoDz/8sMsuvvhily1cuDD5NdXy1XGUeu5u98D1TrR+/XqXXXrppXLaww47zGUvvviiy+69916XqYYH6vxV5MQT/VdznXXWWS47//zzXabOf//yL/+SvGxFnfepz/ZqxrlWXSfraRSn6mTx4sUu+/M//3OXqWOrqMaGDh3qso0bNybN369fP5fVc2yWxSdnAAAAAJABHs4AAAAAIAM8nAEAAABABng4AwAAAIAMZNkQpGhgYTMGNraLGmy43377yWnHjBnjMtVQRA2IV4q+tV0NilTLoflH+6gGGEWDYQcPHuwyte/UoNlDDz00aX1UfZjp5jSHH364y9SAeHWcq+OliJp/t938qY7mH+1T9hy/xx57uOzYY48ttU6qYY1q/vHCCy8kvZ5qnGSmj1eVqcYhndxcoXtNtOp6r95ndf6aNWuWnL+d9yX/+Z//6TJ1H5Da/ON3v/tdqfXp5PqsAnXN37Ztm5x24MCBLlPXxGHDhrnsiCOOSFqfovsDtZxjjjnGZcuWLXOZOt6KzrVq+an3B0XvW6vwyRkAAAAAZICHMwAAAADIAA9nAAAAAJCBHh/OQggzQwhvhBDm7JRdG0JYHEJ4tvbr1OauJpCOmkUVUbeoGmoWVUPNogpSPjmbZWaniPzGGOORtV/3NXa1gFJmGTWL6pll1C2qZZZRs6iWWUbNInM9dmuMMT4SQpjYqAWmdGTqpK6MHv9pKAAAFOpJREFURVR3mG9961tyWtWpbvbs2S5bs2aNy1RXtKKOSqqLo5pWvWZO+6zRNdu9m1dRB6JWUN0Wizrfqf353ve+12X//u//7rLx48cnrU/Rft++fbvLtm7dmjx/d0Xv+caNG5Pmr0IXsUbWbbs636UqWh/VVW7vvfd22b333uuyyZMnJy27qJaefvppl6kOjqnHvzoGiuav6nWwkTXb/XxV9P41Wur+bOf+UPcLZmbXXnuty0aMGOEy1fXuwgsvLL1eVdTJ9wfr1693WdH9QWrH2x/+8IcumzBhQtL6FB0zqp7L3FcWnStSOzLneH9QZszZxSGE52ofEfuzAZAfahZVRN2iaqhZVA01i2z09uHsZjObZGZHmtlSM/tG0YQhhOkhhCdDCE/2cllAI1CzqKKkuqVmkRFqFlXD/QGy0quHsxjj8hjjWzHGt83se2Y29R2mnRFjnBJjnNLblQTKomZRRal1S80iF9Qsqob7A+SmVw9nIYRxO/31I2Y2p2haIAfULKqIukXVULOoGmoWuemxIUgI4cdmdryZ7RlCWGRm15jZ8SGEI80smtlrZpY8sjT3Qc5FgyeVMtty7rnnuuzd7363nFYNMP385z/vskWLFrlMraMaCGpmtnnzZpmnvGZOGl2z7Rzgm6JofwwfPtxlp512msv22msvly1fvtxlL7/8ssuOO+64lFU0M7Nt27YlT7sramTd5n6MFtlnn31c9pWvfMVlBx10kMtSm2oUneNVc5m1a9fKaVOoRk5mnXOeNWtszeY4KH9nRXXTvSGEWbltUa932GGHyWk/9alPuUw1gvq7v/s7l6nmUrsC7g92GDp0qMtOOcU3sRw3bpzL5s6d67I777zTZX/zN38jl63es5EjR7pMNWRKfb2qS+nWOE3EtzRhXYCGoGZRRdQtqoaaRdVQs6iCMt0aAQAAAAANwsMZAAAAAGSAhzMAAAAAyECPY84arfug2twGPavBuGbpA3wHDhyY9JoXXpg83tTuuusul82Z0/tmQmrAsJnZ2LFjXfbGG28kvWZu+7FqVI2kDnItaiQzY8YMl33gAx9w2YYNG1x26aWXuuzZZ5912QsvvCCXrZrObN++XU6boqiJjVp3RQ3mp2bLKfOeHn300TL/4he/6LITTzzRZaq5zDPPPOOyQw891GUDBgyQy7755ptdVqZGis6zKk+9vnRqHYcQbLfdfv92pFUNhLov10yfq4ruDfr37+8ytZ9UIxh1XlONZD772c/KZQ8ePNhlr776qstmz57tMrXeRbWkrkWpddeJzRparcxx/653vUvm3/nOd1x2+umnu0xdYz/zmc+47IknnnDZ1VdfLZetam/VqlVy2hS77767zFObL+V4XuWTMwAAAADIAA9nAAAAAJABHs4AAAAAIAM8nAEAAABABlreEKTdg+x6kjowu8jWrVtd9pd/+ZcuO/zww11W9N5cdNFFLlu/fn0v1m4HtY5mZmvWrHGZGiytBrSnDryEHlh+4IEHuuzFF190mWpmcM0118jlnHDCCS578803Xbb//vu7TB0HamBx0SB5tRw1+D21jjdu3Jg0XZHczzu5UwOmR4wY4bLVq1e7TJ0vVM2Z6eYfqkbe//73u0yd1372s5+5TDVRMCvXZEkpamihalEdRyor01QnZzHGljUA6S71Pa2nscWmTZtcpva7Os9OnjzZZdOmqe9N1stRTZ9ef/31pPWphzqfq8YM69atK7WcXY265zrssMNcphp0qWYyl1xyiVzOaaed5jLV/OPII4902fLly12mjiO1Pmb63Dhu3DiXLV26VM7fXdn7zxzvD/jkDAAAAAAywMMZAAAAAGSAhzMAAAAAyAAPZwAAAACQgZY3BOl0auD85z//+aR5Z8yYIfOVK1f2en3UAN2ipidqQKcalK4Gc+b4Deu5Uu+Lav6h3tP3ve99LvvEJz4hl7NlyxaXHXfccS5Tg4D3228/l5177rlyOYqqEdUYQhk4cKDLiprYlGmQoGq7nkH/uzrV/EPV7MiRI132ne98R76m2idXX321y1TDGTXAfejQoXI5ijpeUs9rqraLBsMX1XJ3qhap2fyophypVC1dddVVyfM/99xzLkutG9XQQ10LiqhzvDqGUEydX9T+U80/lEmTJrnsiiuukNOq5Zx00kkuU0051D2kahxSj9TaUQ2dihqClLk/UI1ZWtmQiU/OAAAAACADPJwBAAAAQAZ4OAMAAACADPT4cBZC2DuE8FAI4fkQwtwQwqW1fGQI4f4Qwiu13/03kgJtQM2iaqhZVBF1i6qhZlEFKZ+cbTezy2OMh5jZMWb26RDCIWZ2lZk9GGM8wMwerP0dyAE1i6qhZlFF1C2qhppF9nrs1hhjXGpmS2t/7gohvGBm483sDDM7vjbZrWb2CzNLa0vYQLl1CRw1apTLhgwZ4jLV9eWGG25o+PqU7Z5U1Nmxu5w6M+ZUs/3793dZarc21e1NdWYs6tb29a9/3WXPP/980nIGDBjgstNPP91lRfv93nvvdZnqsKeOX3VsNKNLUk5d7nKq2TLnVNUVTp0TVX2Z6fPVrbfe6jJVD2PGjHGZ6vxZ1F1PTdvV1SWn7U7VUj2d71LlVLNmedVtszXjGjd58mSXffjDH3ZZ0X6/8MILXbZw4UKXqWO6GfWZ031AkZxqVnXTLuo82J3qEHvCCSe4rKh2HnroIZf95je/cZnqyqnO6ddcc41cjrJ48WKXzZs3z2XqepJ6z1BWKzszKnWNOQshTDSzo8zscTMbWytyM7NlZja2oWsGNAA1i6qhZlFF1C2qhppFrpK/5yyEMNjM7jCzy2KMb+789BpjjCEE+V8mIYTpZja97IoC9aJmUTXULKqoN3VLzaKdONciZ0mfnIUQ+tmOIv5RjPHOWrw8hDCu9u/jzOwNNW+McUaMcUqMcUojVhhIQc2iaqhZVFFv65aaRbtwrkXuUro1BjO7xcxeiDHuPCjqHjM7r/bn88zs7savHlA/ahZVQ82iiqhbVA01iypI+bHGY83sE2b22xDCs7XsajP7qpn9JIRwvpm9bmZnN2cV31luA1AvuOAClw0fPtxlK1ascFnqQNAiavCkWvbq1atLLacCWl6zanCuWXrzD0UNxD333HNdtnbtWjn/TTfd5DI1kFcNfP2nf/onlx166KEuW7BggVy2akaiqGY5qglNmfexIrI+z6ZS++60005zWdEg9dtvvz1p2pEjR7ps2rRpLlPH5S233CKXrQaaK6qZiWr8s27duqTXq7iOqNsU6vpqlt6kZdCgQS677rrrXKaaRLz44ovyNefOnZu0bHUcqGO1nvup1GZhGWp5ze62m77VLtOwTb3/qp6KGlv867/+q8tGjx7tsiVLlrjskksucdmf/MmfuEw1ATMz+4u/+AuXqfudPfbYw2Vl75OrIqVb46NmVnTVOrGxqwOUR82iaqhZVBF1i6qhZlEFdXVrBAAAAAA0Bw9nAAAAAJABHs4AAAAAIAPJ33MGTw0Mf8973uMyNdBcDdxUAy/roQb9qsHKRQPfyzRXUa+ZW7OWZioaHN3o92X+/PkuO+CAA+S0jzzyiMs+97nPuezP/uzPXHb44Ye7TDXlmDp1qlx26qDdd73rXS6bN29e0rxmZv369XOZqnn1nqcO5N/VqMHrarB2qr322stlRQ10urq6XDZixAiXffzjH3fZhAkTXKYGpH/xi1+Uy05tlKSaLG3cuDFpXrNy54Rd/TzbTmXf5w0bNrjs5JNPTlrOlVdeKV+zqNlDymuidYr2U2oTolQPP/ywyz7ykY/IaVXDsMsuu8xlQ4cOTZpObcsf/dEfyWUvXrxY5t2p5jibNm1KmrdofnUsqPu3dje84ZMzAAAAAMgAD2cAAAAAkAEezgAAAAAgAzycAQAAAEAGQisHioYQKjsqVTUeGDdunMuefPJJl6kB5AMHDnRZ6uDeevTp45+/i/a52kbVBCL1NZtRWzHGxo6g7YGqWdXMoFWDR3/1q1+57Oijj5bTpg42Vs0eVHMENbi3aGCvqpHUWqrnvVTNK1Sjni1btiStTzPs6jV70UUXuezb3/62nFatk2rcoupGNeU4+OCDXbZs2TK5bHW8qPUp25RD7Qs1f+r6NMlTMcYprVpYle8NUqn9vnbt2qR5x48fL3PV8Kad1HHZwsZLLa1ZM1236prUjHs75Yc//KHLzjnnHDmtWk91HlLrrq6nF1xwgcvuvfdeuezU1yx73VLzpzZ0qqfxSBlF9wd8cgYAAAAAGeDhDAAAAAAywMMZAAAAAGSAhzMAAAAAyIAfEQhp8ODBLps5c6bLRo4c6TI1SHbOnDkuu/POO+Wy//Zv/9ZlqYMi6xmMm9ogoYUDfLPUyiY63Z100kkuu+SSS+S0Kh87dqzLHn74YZc99NBDLlu3bp3LigY6p75HZRscqOV3dXWVes1O1MJGEs6MGTNcdsYZZ8hpP/CBD7isf//+Lps9e7bLrrnmGpetWLHCZWXPX2WP/3buC7SPagKmamnQoEEu+9KXviRfs+jc3y67+r2BWXvvD8477zyX/fKXv5TTXnnllS7bd999XTZv3jyXPffcc0mZarRh1rr7AzX/qlWrSr1mq/DJGQAAAABkgIczAAAAAMgAD2cAAAAAkIEeH85CCHuHEB4KITwfQpgbQri0ll8bQlgcQni29uvU5q8u0DNqFlVDzaJqqFlUEXWLKgg9DcwLIYwzs3ExxqdDCEPM7CkzO9PMzjaz9THGf0hemPg29apQTT3Gjx/vsscff9xlK1euTJquaNDv4sWLXdaqb5zPTdG3qe8s95pVtVRmIHUI+i3p16+fy1TTFzV/Owc1d5pOqNlGK6pZlatjg5ptuqdijFPeaYJdrWbLGjBggMtUY5s//MM/dNn+++8vX3P+/PnlV6zJWnis9liztfXJpm7Ve6PuD5rRRKhv375Jy+Fc21xF9wc9dmuMMS41s6W1P3eFEF4wM/9UAmSCmkXVULOoGmoWVUTdogrqGnMWQphoZkeZ2X997HNxCOG5EMLMEMKIBq8bUBo1i6qhZlE11CyqiLpFrpIfzkIIg83sDjO7LMb4ppndbGaTzOxI2/G/EN8omG96COHJEMKTDVhfIBk1i6qhZlE11CyqiLpFzpIezkII/WxHEf8oxninmVmMcXmM8a0Y49tm9j0zm6rmjTHOiDFOSflZYKBRqFlUDTWLqqFmUUXULXKX0q0xmNktZvZCjPGGnfJxO032ETOb0/jVA+pHzaJqqFlUDTWLKqJuUQUp3RqPM7PZZvZbM/uvtllXm9k02/HxbzSz18zswtpAy3d6rV2yxUs7u93svvvuLtuyZUtLlt0MiZ3vsqnZ3XbzPXfKdF5SdXPYYYfJaefM4dqSg6rVrOripTomqulSu8iOGjVK5qtWrUqaH02X0q0xm5qtAtU9d/Xq1S67//77XfbRj35UvmaZa0nqcW5Wme58qd0as6nb1H1Q5v2fNGmSzKvQ6XNXUKZb46Nmpma+r+xKAc1AzaJqqFlUDTWLKqJuUQV1dWsEAAAAADQHD2cAAAAAkAEezgAAAAAgAz2OOctdnz7++bJoUGu7tGowrWo8UuXmH51gyJAhLnvzzTdd1r9/f5dt2rQpaRkLFy6sf8UaRNWcWWUGkENQtagafahmN6kNQdasWVP/ijUINYtmUzV28sknu+wXv/iFy1Sjhmbc05RpJoLGUOfarVu3ukw1DlHTKStWrKh/xRpE3Z+b5XePniM+OQMAAACADPBwBgAAAAAZ4OEMAAAAADLAwxkAAAAAZCC0chB0CGGFmb1e++ueZrayZQtvrk7aFrN8t2ffGOPoVi6Qmq2MXLeHmm2cTtoWs7y3p6V128E1a9ZZ25PztrTzXJvz+9IbnbQ9OW9LYc229OHs9xYcwpMxxiltWXiDddK2mHXe9jRKJ70vnbQtZp23PY3SSe9LJ22LWedtT6N02vvSSdvTSdvSSJ32vnTS9lR1W/ixRgAAAADIAA9nAAAAAJCBdj6czWjjshutk7bFrPO2p1E66X3ppG0x67ztaZROel86aVvMOm97GqXT3pdO2p5O2pZG6rT3pZO2p5Lb0rYxZwAAAACA/4cfawQAAACADLT84SyEcEoI4aUQwrwQwlWtXn5ZIYSZIYQ3QghzdspGhhDuDyG8Uvt9RDvXMVUIYe8QwkMhhOdDCHNDCJfW8kpuT7NQs/mgZtNQs/mgZtNVuW47qWbNqNtUVa5Zs86q206q2ZY+nIUQ+prZt83sv5vZIWY2LYRwSCvXoQFmmdkp3bKrzOzBGOMBZvZg7e9VsN3MLo8xHmJmx5jZp2v7o6rb03DUbHao2R5Qs9mhZhN0QN3Oss6pWTPqtkcdULNmnVW3HVOzrf7kbKqZzYsxLogxbjWz283sjBavQykxxkfMbHW3+Awzu7X251vN7MyWrlQvxRiXxhifrv25y8xeMLPxVtHtaRJqNiPUbBJqNiPUbLJK120n1awZdZuo0jVr1ll120k12+qHs/Fm9rud/r6ollXd2Bjj0tqfl5nZ2HauTG+EECaa2VFm9rh1wPY0EDWbKWq2EDWbKWr2HXVi3XbEPqZuC3VizZp1wD6ues3SEKTB4o72l5VqgRlCGGxmd5jZZTHGN3f+typuD+pTxX1Mze7aqriPqdldW1X3MXW7a6viPu6Emm31w9liM9t7p79PqGVVtzyEMM7MrPb7G21en2QhhH62o4h/FGO8sxZXdnuagJrNDDXbI2o2M9Rskk6s20rvY+q2R51Ys2YV3sedUrOtfjh7wswOCCHsF0Lob2bnmNk9LV6HZrjHzM6r/fk8M7u7jeuSLIQQzOwWM3shxnjDTv9Uye1pEmo2I9RsEmo2I9Rssk6s28ruY+o2SSfWrFlF93FH1WyMsaW/zOxUM3vZzOab2f9q9fIbsP4/NrOlZrbNdvx88flmNsp2dIB5xcweMLOR7V7PxG05znZ8vPucmT1b+3VqVbenie8TNZvJL2o2+X2iZjP5Rc3W9V5Vtm47qWZr20Pdpr1Pla3Z2vp3TN12Us2G2gYBAAAAANqIhiAAAAAAkAEezgAAAAAgAzycAQAAAEAGeDgDAAAAgAzwcAYAAAAAGeDhDAAAAAAywMMZAAAAAGSAhzMAAAAAyMD/B+BwYQYGq3yVAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 1080x432 with 10 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OPOfqd1VL8LE"
      },
      "source": [
        "以上"
      ]
    }
  ]
}