MNIST_GAN_Exercise.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generative Adversarial Network\n",
    "\n",
    "In this notebook, we'll be building a generative adversarial network (GAN) trained on the MNIST dataset. From this, we'll be able to generate new handwritten digits!\n",
    "\n",
    "GANs were [first reported on](https://arxiv.org/abs/1406.2661) in 2014 from Ian Goodfellow and others in Yoshua Bengio's lab. Since then, GANs have exploded in popularity. Here are a few examples to check out:\n",
    "\n",
    "* [Pix2Pix](https://affinelayer.com/pixsrv/) \n",
    "* [CycleGAN & Pix2Pix in PyTorch, Jun-Yan Zhu](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)\n",
    "* [A list of generative models](https://github.com/wiseodd/generative-models)\n",
    "\n",
    "The idea behind GANs is that you have two networks, a generator $G$ and a discriminator $D$, competing against each other. The generator makes \"fake\" data to pass to the discriminator. The discriminator also sees real training data and predicts if the data it's received is real or fake. \n",
    "> * The generator is trained to fool the discriminator, it wants to output data that looks _as close as possible_ to real, training data. \n",
    "* The discriminator is a classifier that is trained to figure out which data is real and which is fake. \n",
    "\n",
    "What ends up happening is that the generator learns to make data that is indistinguishable from real data to the discriminator.\n",
    "\n",
    "<img src='assets/gan_pipeline.png' width=70% />\n",
    "\n",
    "The general structure of a GAN is shown in the diagram above, using MNIST images as data. The latent sample is a random vector that the generator uses to construct its fake images. This is often called a **latent vector** and that vector space is called **latent space**. As the generator trains, it figures out how to map latent vectors to recognizable images that can fool the discriminator.\n",
    "\n",
    "If you're interested in generating only new images, you can throw out the discriminator after training. In this notebook, I'll show you how to define and train these adversarial networks in PyTorch and generate new images!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import numpy as np\n",
    "import torch\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchvision import datasets\n",
    "import torchvision.transforms as transforms\n",
    "\n",
    "# number of subprocesses to use for data loading\n",
    "num_workers = 0\n",
    "# how many samples per batch to load\n",
    "batch_size = 64\n",
    "\n",
    "# convert data to torch.FloatTensor\n",
    "transform = transforms.ToTensor()\n",
    "\n",
    "# get the training datasets\n",
    "train_data = datasets.MNIST(root='data', train=True,\n",
    "                                   download=True, transform=transform)\n",
    "\n",
    "# prepare data loader\n",
    "train_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size,\n",
    "                                           num_workers=num_workers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f6009b043d0>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMUAAADDCAYAAAAyYdXtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAALqUlEQVR4nO3dbYxU9RXH8e8RywspihvDQhCKEIMisdtEwSipEkOFBoPrU9zEhgQCvmATbAwp4Y3aBkMq2pZIGmmKQmIRE7WsxBQMILSx2YiIT1gqMVZXNqABZMEHAnv6Yu/a7X//w96duXPngd8nMTNz9u6dM8Ff7p3/zJ5r7o6I/M8FlW5ApNooFCIBhUIkoFCIBBQKkYBCIRIoKRRmNsvMDpjZQTNbllVTIpVkxX5OYWZDgH8DM4EO4E2gxd33n+N39KGIVA13t1i9lCPFVOCgu3/s7qeB54G5JexPpCqUEooxwGd9HnckNZGadmEJvxs79PQ7PTKzRcCiEp5HJFelhKIDGNvn8eXAoXAjd18LrAW9p5DaUMrp05vAlWZ2hZkNBe4D2rJpS6Ryij5SuPsZM2sFtgJDgHXu/kFmnYlUSNFLskU9mU6fpIqUY0lWpC4pFCIBhUIkoFCIBBQKkYBCIRJQKEQCCoVIQKEQCSgUIgGFQiSgUIgEFAqRgEIhElAoRAIKhUhAoRAJKBQiAYVCJKBQiARKmfuEmX0CdAFngTPufl0WTZ1vhgwZEq1fcsklmey/tbU1Wr/ooov61SZNmhTddvHixdH6qlWrovWWlpZo/dtvv43WV65cGa0/+uij0Xo5lRSKxAx3/zKD/YhUBZ0+iQRKDYUD28zsrWRmrEjNK/X06SZ3P2RmI4HXzOxf7r677wYasCy1pqQjhbsfSm6PAC/Tc82KcJu17n6d3oRLrSj6SGFmw4AL3L0ruf8z4NeZdVZlxo0bF60PHTo0Wr/xxhv71aZPnx7ddsSIEdH6XXfdlbK77HR0dETrq1evjtabm5uj9a6urmj9nXfeidZ37dqVort8lHL61Ai8bGa9+/mLu/8tk65EKqiUqeMfAz/OsBeRqqAlWZGAQiESUChEArpoS6CpqSla37FjR7Se1feTKqG7u7tfbf78+dFtT548Oah9d3Z2RuvHjh2L1g8cODCo/WdBF20RSUmhEAkoFCIBhUIkoFCIBLT6FGhoaIjW29vbo/UJEyaUs52oQr0cP348Wp8xY0a0fvr06X61Wl5NGyytPomkpFCIBBQKkYBCIRJQKEQCWYy4qStHjx6N1pcuXRqtz5kzJ1p/++23+9UK/fVaIfv27YvWZ86cGa2fOnUqWr/mmmui9SVLlgyqn/OFjhQiAYVCJKBQiAQUCpHAgKEws3VmdsTM3u9TazCz18zso+T20vK2KZKfAb/7ZGY/BU4CG9x9SlL7LXDU3Vea2TLgUnf/1YBPVgPffRqsiy++OFqPzT16+umno9suWLAgWr///vuj9Y0bN6bsTs6l6O8+JWMww3XKucD65P564I6SuhOpIsW+p2h0906A5HZkdi2JVFbZP7zTgGWpNcUeKQ6b2WiA5PZIoQ01YFlqTbFHijZgHrAyud2cWUc15sSJE6m3/eqrrwa174ULF0brmzZtitZjI2tk8NIsyW4E/glMMrMOM1tATxhmmtlHwMzksUhdGPBI4e7xK/rBrRn3IlIV9Im2SEChEAkoFCIBjbjJ0bBhw6L1V155JVq/+eabo/XZs2dH69u2bSuusfOURtyIpKRQiAQUCpGAQiESUChEAlp9qgITJ06M1vfu3RutFxqkvHPnzmh9z5490fqaNWv61fL8/6HStPokkpJCIRJQKEQCCoVIQKEQCWj1qYo1NzdH688880y0Pnz48EHtf/ny5f1qGzZsiG5b6GLxtUyrTyIpKRQiAYVCJKBQiAQUCpFAmgHL64A5wJE+A5YfARYCXySbLXf3Vwd8Mq0+ZWLKlCnR+pNPPhmt33pr+sErhYZAr1ixIlr//PPPU++72pSy+vQsMCtS/527NyX/DRgIkVpR7NRxkbpVynuKVjN7N7moS8GLtpjZIjPbY2bx7y+LVJliQ/FHYCLQBHQCTxTaUAOWpdYUFQp3P+zuZ929G/gTMDXbtkQqJ9V3n8xsPLClz+rT6N6LtpjZL4Fp7n5fiv1o9amMRowYEa3ffvvt0XrsO1Rm0QUZduzYEa0XutB9LSi0+jTggOVk6vgtwGVm1gE8DNxiZk2AA58AD2TWqUiFFTt1/M9l6EWkKugTbZGAQiESUChEAvrLu/PYd99916924YXxt5lnzpyJ1m+77bZo/fXXXy+6r7zoL+9EUlIoRAIKhUhAoRAJFHtxeamga6+9Nlq/++67o/Xrr78+Wi/0pjpm//790fru3btT76NW6EghElAoRAIKhUhAoRAJKBQiAa0+VYFJkyZF662trdH6nXfeGa2PGjWq5F7Onj0brRcasNzd3V3yc1YbHSlEAgqFSEChEAkoFCIBhUIkkGaax1hgAzAK6AbWuvsfzKwB2ASMp2eix73ufqx8rdaW2EpQS0tsBkThVabx48dn2VI/sYvOFxqk3NbWVtZeqkmaI8UZ4CF3vxq4AVhsZpOBZcB2d78S2J48Fql5aQYsd7r73uR+F/AhMAaYC6xPNlsP3FGuJkXyNKgP75JJgT8B2oHG3imB7t5pZiML/M4iYFFpbYrkJ3UozOyHwIvAg+5+otB4xZC7rwXWJvvQ4AKpeqlWn8zsB/QE4jl3fykpHzaz0cnPRwNHytOiSL7SrD4ZPWMyP3T3vtePagPmASuT281l6bBKNDY2RuuTJ0+O1p966ql+tauuuirTnkLt7e3R+uOPPx6tb97c/5+sHr/LNFhpTp9uAn4BvGdm+5LacnrC8IKZLQA+Be4pT4si+UozYPkfQKE3EOmvMChSI/SJtkhAoRAJKBQigfP2L+8aGhqi9UIXV29qaorWJ0yYkFlPoTfeeCNaf+KJ+HU3t27dGq1/8803mfV0PtCRQiSgUIgEFAqRgEIhElAoRAJ1s/o0bdq0aH3p0qXR+tSpU6P1MWPGZNZT6Ouvv47WV69eHa0/9thj0fqpU6cy60n605FCJKBQiAQUCpGAQiESUChEAnWz+tTc3Dyo+mAVuubbli1bovXYxdgLfWfp+PHjxTcmmdORQiSgUIgEFAqRgEIhEjD3c88nO8eA5UeAhcAXyabL3f3VAfalYWhSNdw9OpAjTShGA6Pdfa+ZDQfeomdu7L3ASXdflbYJhUKqSaFQpBlx0wn0zoztMrPeAcsidWlQ7ymCAcsArWb2rpmtM7NLC/zOIjPbY2b9L4YgUoUGPH36fsOeAcu7gBXu/pKZNQJfAg78hp5TrPkD7EOnT1I1in5PAd8PWN4CbA3myfb+fDywxd2nDLAfhUKqRqFQDHj6VGjAcu/E8UQz8H6pTYpUgzSrT9OBvwPv0bMkCz0DlluAJnpOnz4BHui9iMs59qUjhVSNkk6fsqJQSDUp+vRJ5HyjUIgEFAqRgEIhElAoRAIKhUhAoRAJKBQiAYVCJJD3iJsvgf8k9y9LHtc7vc7q9KNCP8j1ax7/98Rme9z9uoo8eY70OmuPTp9EAgqFSKCSoVhbwefOk15njanYewqRaqXTJ5FA7qEws1lmdsDMDprZsryfv5ySqSZHzOz9PrUGM3vNzD5KbqNTT2qJmY01s51m9qGZfWBmS5J6XbzWXENhZkOANcBsYDLQYmaT8+yhzJ4FZgW1ZcB2d78S2J48rnVngIfc/WrgBmBx8u9YF6817yPFVOCgu3/s7qeB54G5OfdQNu6+GzgalOcC65P76+mZrljT3L3T3fcm97uA3gF5dfFa8w7FGOCzPo87qP9pg429Ax2S25EV7idTwYC8uniteYci9ofiWv6qUcmAvBeBB939RKX7yUreoegAxvZ5fDlwKOce8na4d0ZWcnukwv1kIhmQ9yLwnLu/lJTr4rXmHYo3gSvN7AozGwrcB7Tl3EPe2oB5yf15wOYK9pKJQgPyqJPXmvuHd2b2c+D3wBBgnbuvyLWBMjKzjcAt9Hxj9DDwMPBX4AVgHPApcI+7h2/Ga8o5BuS1UwevVZ9oiwT0ibZIQKEQCSgUIgGFQiSgUIgEFAqRgEIhElAoRAL/BZ2LiVIchAuEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# obtain one batch of training images\n",
    "dataiter = iter(train_loader)\n",
    "images, labels = dataiter.next()\n",
    "images = images.numpy()\n",
    "\n",
    "# get one image from the batch\n",
    "img = np.squeeze(images[0])\n",
    "\n",
    "fig = plt.figure(figsize = (3,3)) \n",
    "ax = fig.add_subplot(111)\n",
    "ax.imshow(img, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Define the Model\n",
    "\n",
    "A GAN is comprised of two adversarial networks, a discriminator and a generator."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discriminator\n",
    "\n",
    "The discriminator network is going to be a pretty typical linear classifier. To make this network a universal function approximator, we'll need at least one hidden layer, and these hidden layers should have one key attribute:\n",
    "> All hidden layers will have a [Leaky ReLu](https://pytorch.org/docs/stable/nn.html#torch.nn.LeakyReLU) activation function applied to their outputs.\n",
    "\n",
    "<img src='assets/gan_network.png' width=70% />\n",
    "\n",
    "#### Leaky ReLu\n",
    "\n",
    "We should use a leaky ReLU to allow gradients to flow backwards through the layer unimpeded. A leaky ReLU is like a normal ReLU, except that there is a small non-zero output for negative input values.\n",
    "\n",
    "<img src='assets/leaky_relu.png' width=40% />\n",
    "\n",
    "#### Sigmoid Output\n",
    "\n",
    "We'll also take the approach of using a more numerically stable loss function on the outputs. Recall that we want the discriminator to output a value 0-1 indicating whether an image is _real or fake_. \n",
    "> We will ultimately use [BCEWithLogitsLoss](https://pytorch.org/docs/stable/nn.html#bcewithlogitsloss), which combines a `sigmoid` activation function **and** and binary cross entropy loss in one function. \n",
    "\n",
    "So, our final output layer should not have any activation function applied to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Discriminator(nn.Module):\n",
    "\n",
    "    def __init__(self, input_size, hidden_dim, output_size):\n",
    "        super(Discriminator, self).__init__()\n",
    "        self.fc1 = nn.Linear(input_size, hidden_dim*4)\n",
    "        self.lrelu = nn.LeakyReLU(0.2)\n",
    "        self.fc2 = nn.Linear(hidden_dim*4,hidden_dim*2)\n",
    "        self.fc3 = nn.Linear(hidden_dim*2,hidden_dim)\n",
    "        self.fc4 = nn.Linear(hidden_dim, output_size)\n",
    "        self.dropout = nn.Dropout(0.3)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        # flatten image\n",
    "        x = x.view(-1,28*28)\n",
    "        # pass x through all layers\n",
    "        x = self.lrelu(self.fc1(x))\n",
    "        x = self.dropout(x)\n",
    "        x = self.lrelu(self.fc2(x))\n",
    "        x = self.dropout(x)\n",
    "        x = self.lrelu(self.fc3(x))\n",
    "        x = self.dropout(x)\n",
    "        x = self.lrelu(self.fc4(x))\n",
    "        # apply leaky relu activation to all hidden layers\n",
    "\n",
    "        return x\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generator\n",
    "\n",
    "The generator network will be almost exactly the same as the discriminator network, except that we're applying a [tanh activation function](https://pytorch.org/docs/stable/nn.html#tanh) to our output layer.\n",
    "\n",
    "#### tanh Output\n",
    "The generator has been found to perform the best with $tanh$ for the generator output, which scales the output to be between -1 and 1, instead of 0 and 1. \n",
    "\n",
    "<img src='assets/tanh_fn.png' width=40% />\n",
    "\n",
    "Recall that we also want these outputs to be comparable to the *real* input pixel values, which are read in as normalized values between 0 and 1. \n",
    "> So, we'll also have to **scale our real input images to have pixel values between -1 and 1** when we train the discriminator. \n",
    "\n",
    "I'll do this in the training loop, later on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Generator(nn.Module):\n",
    "\n",
    "    def __init__(self, input_size, hidden_dim, output_size):\n",
    "        super(Generator, self).__init__()        \n",
    "        # define all layers\n",
    "        self.fc1 = nn.Linear(input_size, hidden_dim)\n",
    "        self.lrelu = nn.LeakyReLU(0.2)\n",
    "        self.fc2 = nn.Linear(hidden_dim,hidden_dim*2)\n",
    "        self.fc3 = nn.Linear(hidden_dim*2,hidden_dim*4)\n",
    "        self.fc4 = nn.Linear(hidden_dim*4, output_size)\n",
    "        self.tanh = nn.Tanh()\n",
    "        self.dropout = nn.Dropout(0.3)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        # flatten image\n",
    "        #x = x.view(-1,28*28)\n",
    "        # pass x through all layers\n",
    "        x = self.lrelu(self.fc1(x))\n",
    "        x = self.dropout(x)\n",
    "        x = self.lrelu(self.fc2(x))\n",
    "        x = self.dropout(x)\n",
    "        x = self.lrelu(self.fc3(x))\n",
    "        x = self.dropout(x)\n",
    "        x = self.tanh(self.fc4(x))\n",
    "        # apply leaky relu activation to all hidden layers\n",
    "        \n",
    "        return x\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Discriminator hyperparams\n",
    "\n",
    "# Size of input image to discriminator (28*28)\n",
    "input_size = 28*28\n",
    "# Size of discriminator output (real or fake)\n",
    "d_output_size = 1\n",
    "# Size of *last* hidden layer in the discriminator\n",
    "d_hidden_size = 32\n",
    "\n",
    "# Generator hyperparams\n",
    "\n",
    "# Size of latent vector to give to generator\n",
    "z_size = 100\n",
    "# Size of discriminator output (generated image)\n",
    "g_output_size = 28*28\n",
    "# Size of *first* hidden layer in the generator\n",
    "g_hidden_size = 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build complete network\n",
    "\n",
    "Now we're instantiating the discriminator and generator from the classes defined above. Make sure you've passed in the correct input arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discriminator(\n",
      "  (fc1): Linear(in_features=784, out_features=128, bias=True)\n",
      "  (lrelu): LeakyReLU(negative_slope=0.2)\n",
      "  (fc2): Linear(in_features=128, out_features=64, bias=True)\n",
      "  (fc3): Linear(in_features=64, out_features=32, bias=True)\n",
      "  (fc4): Linear(in_features=32, out_features=1, bias=True)\n",
      "  (dropout): Dropout(p=0.3, inplace=False)\n",
      ")\n",
      "\n",
      "Generator(\n",
      "  (fc1): Linear(in_features=100, out_features=32, bias=True)\n",
      "  (lrelu): LeakyReLU(negative_slope=0.2)\n",
      "  (fc2): Linear(in_features=32, out_features=64, bias=True)\n",
      "  (fc3): Linear(in_features=64, out_features=128, bias=True)\n",
      "  (fc4): Linear(in_features=128, out_features=784, bias=True)\n",
      "  (tanh): Tanh()\n",
      "  (dropout): Dropout(p=0.3, inplace=False)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "# instantiate discriminator and generator\n",
    "D = Discriminator(input_size, d_hidden_size, d_output_size)\n",
    "G = Generator(z_size, g_hidden_size, g_output_size)\n",
    "\n",
    "# check that they are as you expect\n",
    "print(D)\n",
    "print()\n",
    "print(G)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Discriminator and Generator Losses\n",
    "\n",
    "Now we need to calculate the losses. \n",
    "\n",
    "### Discriminator Losses\n",
    "\n",
    "> * For the discriminator, the total loss is the sum of the losses for real and fake images, `d_loss = d_real_loss + d_fake_loss`. \n",
    "* Remember that we want the discriminator to output 1 for real images and 0 for fake images, so we need to set up the losses to reflect that.\n",
    "\n",
    "<img src='assets/gan_pipeline.png' width=70% />\n",
    "\n",
    "The losses will by binary cross entropy loss with logits, which we can get with [BCEWithLogitsLoss](https://pytorch.org/docs/stable/nn.html#bcewithlogitsloss). This combines a `sigmoid` activation function **and** and binary cross entropy loss in one function.\n",
    "\n",
    "For the real images, we want `D(real_images) = 1`. That is, we want the discriminator to classify the the real images with a label = 1, indicating that these are real. To help the discriminator generalize better, the labels are **reduced a bit from 1.0 to 0.9**. For this, we'll use the parameter `smooth`; if True, then we should smooth our labels. In PyTorch, this looks like `labels = torch.ones(size) * 0.9`\n",
    "\n",
    "The discriminator loss for the fake data is similar. We want `D(fake_images) = 0`, where the fake images are the _generator output_, `fake_images = G(z)`. \n",
    "\n",
    "### Generator Loss\n",
    "\n",
    "The generator loss will look similar only with flipped labels. The generator's goal is to get `D(fake_images) = 1`. In this case, the labels are **flipped** to represent that the generator is trying to fool the discriminator into thinking that the images it generates (fakes) are real!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate losses\n",
    "def real_loss(D_out, smooth=False):\n",
    "    # compare logits to real labels\n",
    "    # smooth labels if smooth=True\n",
    "    batch_size = D_out.size(0)\n",
    "    if smooth:\n",
    "        labels = torch.ones(batch_size) * 0.9\n",
    "    else:\n",
    "        labels = torch.ones(batch_size)\n",
    "    criterion = nn.BCEWithLogitsLoss()\n",
    "    loss = criterion(D_out.squeeze(), labels)\n",
    "    return loss\n",
    "\n",
    "def fake_loss(D_out):\n",
    "    batch_size = D_out.size(0)\n",
    "    # compare logits to fake labels\n",
    "    labels = torch.zeros(batch_size)\n",
    "\n",
    "    criterion = nn.BCEWithLogitsLoss()\n",
    "    loss = criterion(D_out.squeeze(), labels)\n",
    "\n",
    "    return loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimizers\n",
    "\n",
    "We want to update the generator and discriminator variables separately. So, we'll define two separate Adam optimizers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "# learning rate for optimizers\n",
    "lr = 0.002\n",
    "\n",
    "# Create optimizers for the discriminator and generator\n",
    "d_optimizer = optim.Adam(D.parameters(),lr=lr)\n",
    "g_optimizer = optim.Adam(G.parameters(),lr=lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Training\n",
    "\n",
    "Training will involve alternating between training the discriminator and the generator. We'll use our functions `real_loss` and `fake_loss` to help us calculate the discriminator losses in all of the following cases.\n",
    "\n",
    "### Discriminator training\n",
    "1. Compute the discriminator loss on real, training images        \n",
    "2. Generate fake images\n",
    "3. Compute the discriminator loss on fake, generated images     \n",
    "4. Add up real and fake loss\n",
    "5. Perform backpropagation + an optimization step to update the discriminator's weights\n",
    "\n",
    "### Generator training\n",
    "1. Generate fake images\n",
    "2. Compute the discriminator loss on fake images, using **flipped** labels!\n",
    "3. Perform backpropagation + an optimization step to update the generator's weights\n",
    "\n",
    "#### Saving Samples\n",
    "\n",
    "As we train, we'll also print out some loss statistics and save some generated \"fake\" samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [    1/   40] | d_loss: 1.4177 | g_loss: 0.6208\n",
      "Epoch [    1/   40] | d_loss: 1.2134 | g_loss: 1.1262\n",
      "Epoch [    1/   40] | d_loss: 1.1728 | g_loss: 0.8554\n",
      "Epoch [    2/   40] | d_loss: 1.3672 | g_loss: 0.8820\n",
      "Epoch [    2/   40] | d_loss: 1.5700 | g_loss: 1.0195\n",
      "Epoch [    2/   40] | d_loss: 1.1356 | g_loss: 0.8883\n",
      "Epoch [    3/   40] | d_loss: 1.5188 | g_loss: 0.8095\n",
      "Epoch [    3/   40] | d_loss: 1.3889 | g_loss: 0.8236\n",
      "Epoch [    3/   40] | d_loss: 1.0637 | g_loss: 1.5688\n",
      "Epoch [    4/   40] | d_loss: 1.3024 | g_loss: 1.0751\n",
      "Epoch [    4/   40] | d_loss: 1.3041 | g_loss: 0.7517\n",
      "Epoch [    4/   40] | d_loss: 1.1731 | g_loss: 1.0619\n",
      "Epoch [    5/   40] | d_loss: 1.3061 | g_loss: 1.0596\n",
      "Epoch [    5/   40] | d_loss: 1.3582 | g_loss: 0.8503\n",
      "Epoch [    5/   40] | d_loss: 1.2000 | g_loss: 2.4447\n",
      "Epoch [    6/   40] | d_loss: 1.0827 | g_loss: 1.8777\n",
      "Epoch [    6/   40] | d_loss: 0.3655 | g_loss: 10.8160\n",
      "Epoch [    6/   40] | d_loss: 0.3447 | g_loss: 10.1371\n",
      "Epoch [    7/   40] | d_loss: 0.3478 | g_loss: 10.5393\n",
      "Epoch [    7/   40] | d_loss: 0.3426 | g_loss: 12.3823\n",
      "Epoch [    7/   40] | d_loss: 0.3423 | g_loss: 10.3868\n",
      "Epoch [    8/   40] | d_loss: 0.3452 | g_loss: 13.1678\n",
      "Epoch [    8/   40] | d_loss: 0.3339 | g_loss: 14.8250\n",
      "Epoch [    8/   40] | d_loss: 0.3356 | g_loss: 11.4055\n",
      "Epoch [    9/   40] | d_loss: 0.3892 | g_loss: 4.5386\n",
      "Epoch [    9/   40] | d_loss: 0.3388 | g_loss: 11.6818\n",
      "Epoch [    9/   40] | d_loss: 0.3349 | g_loss: 10.8031\n",
      "Epoch [   10/   40] | d_loss: 0.3381 | g_loss: 9.5781\n",
      "Epoch [   10/   40] | d_loss: 0.3351 | g_loss: 8.8895\n",
      "Epoch [   10/   40] | d_loss: 0.3285 | g_loss: 11.8467\n",
      "Epoch [   11/   40] | d_loss: 0.3287 | g_loss: 11.5133\n",
      "Epoch [   11/   40] | d_loss: 0.3274 | g_loss: 12.1009\n",
      "Epoch [   11/   40] | d_loss: 0.3262 | g_loss: 13.0590\n",
      "Epoch [   12/   40] | d_loss: 0.3261 | g_loss: 12.9624\n",
      "Epoch [   12/   40] | d_loss: 0.3256 | g_loss: 12.6384\n",
      "Epoch [   12/   40] | d_loss: 0.3256 | g_loss: 12.9145\n",
      "Epoch [   13/   40] | d_loss: 0.3255 | g_loss: 13.1748\n",
      "Epoch [   13/   40] | d_loss: 0.3254 | g_loss: 12.4929\n",
      "Epoch [   13/   40] | d_loss: 0.3254 | g_loss: 12.5082\n",
      "Epoch [   14/   40] | d_loss: 0.3263 | g_loss: 10.4895\n",
      "Epoch [   14/   40] | d_loss: 0.3279 | g_loss: 11.7560\n",
      "Epoch [   14/   40] | d_loss: 0.3283 | g_loss: 11.8993\n",
      "Epoch [   15/   40] | d_loss: 0.3515 | g_loss: 11.7189\n",
      "Epoch [   15/   40] | d_loss: 0.3372 | g_loss: 9.0605\n",
      "Epoch [   15/   40] | d_loss: 0.3305 | g_loss: 13.0528\n",
      "Epoch [   16/   40] | d_loss: 0.3356 | g_loss: 6.7755\n",
      "Epoch [   16/   40] | d_loss: 0.3427 | g_loss: 13.2326\n",
      "Epoch [   16/   40] | d_loss: 0.3331 | g_loss: 12.7218\n",
      "Epoch [   17/   40] | d_loss: 0.3315 | g_loss: 8.0784\n",
      "Epoch [   17/   40] | d_loss: 0.3372 | g_loss: 12.0186\n",
      "Epoch [   17/   40] | d_loss: 0.3470 | g_loss: 7.8386\n",
      "Epoch [   18/   40] | d_loss: 0.3685 | g_loss: 8.8819\n",
      "Epoch [   18/   40] | d_loss: 0.3412 | g_loss: 14.1162\n",
      "Epoch [   18/   40] | d_loss: 0.3496 | g_loss: 8.5376\n",
      "Epoch [   19/   40] | d_loss: 0.3537 | g_loss: 13.0780\n",
      "Epoch [   19/   40] | d_loss: 0.3401 | g_loss: 6.7060\n",
      "Epoch [   19/   40] | d_loss: 0.3540 | g_loss: 10.0841\n",
      "Epoch [   20/   40] | d_loss: 0.3445 | g_loss: 11.5623\n",
      "Epoch [   20/   40] | d_loss: 0.3375 | g_loss: 11.1646\n",
      "Epoch [   20/   40] | d_loss: 0.3679 | g_loss: 12.8437\n",
      "Epoch [   21/   40] | d_loss: 0.3493 | g_loss: 14.2906\n",
      "Epoch [   21/   40] | d_loss: 0.3393 | g_loss: 9.3332\n",
      "Epoch [   21/   40] | d_loss: 0.3366 | g_loss: 9.3351\n",
      "Epoch [   22/   40] | d_loss: 0.3400 | g_loss: 9.9845\n",
      "Epoch [   22/   40] | d_loss: 0.3440 | g_loss: 8.9415\n",
      "Epoch [   22/   40] | d_loss: 0.3462 | g_loss: 10.2510\n",
      "Epoch [   23/   40] | d_loss: 0.3490 | g_loss: 10.6273\n",
      "Epoch [   23/   40] | d_loss: 0.3386 | g_loss: 10.4623\n",
      "Epoch [   23/   40] | d_loss: 0.3503 | g_loss: 10.6094\n",
      "Epoch [   24/   40] | d_loss: 0.3806 | g_loss: 10.3101\n",
      "Epoch [   24/   40] | d_loss: 0.3352 | g_loss: 8.2735\n",
      "Epoch [   24/   40] | d_loss: 0.3391 | g_loss: 11.5119\n",
      "Epoch [   25/   40] | d_loss: 0.3756 | g_loss: 12.3183\n",
      "Epoch [   25/   40] | d_loss: 0.3338 | g_loss: 10.8130\n",
      "Epoch [   25/   40] | d_loss: 0.3429 | g_loss: 11.3526\n",
      "Epoch [   26/   40] | d_loss: 0.3369 | g_loss: 10.8954\n",
      "Epoch [   26/   40] | d_loss: 0.3488 | g_loss: 10.5299\n",
      "Epoch [   26/   40] | d_loss: 0.3483 | g_loss: 9.6234\n",
      "Epoch [   27/   40] | d_loss: 0.3459 | g_loss: 9.3860\n",
      "Epoch [   27/   40] | d_loss: 0.3489 | g_loss: 8.4379\n",
      "Epoch [   27/   40] | d_loss: 0.3521 | g_loss: 10.5000\n",
      "Epoch [   28/   40] | d_loss: 0.3337 | g_loss: 9.5927\n",
      "Epoch [   28/   40] | d_loss: 0.3385 | g_loss: 9.9235\n",
      "Epoch [   28/   40] | d_loss: 0.3432 | g_loss: 10.0609\n",
      "Epoch [   29/   40] | d_loss: 0.3499 | g_loss: 12.1998\n",
      "Epoch [   29/   40] | d_loss: 0.3810 | g_loss: 8.5781\n",
      "Epoch [   29/   40] | d_loss: 0.3454 | g_loss: 9.0896\n",
      "Epoch [   30/   40] | d_loss: 0.3766 | g_loss: 10.3046\n",
      "Epoch [   30/   40] | d_loss: 0.3579 | g_loss: 7.9535\n",
      "Epoch [   30/   40] | d_loss: 0.3448 | g_loss: 8.4727\n",
      "Epoch [   31/   40] | d_loss: 0.3402 | g_loss: 8.6602\n",
      "Epoch [   31/   40] | d_loss: 0.3549 | g_loss: 9.7380\n",
      "Epoch [   31/   40] | d_loss: 0.3409 | g_loss: 9.0786\n",
      "Epoch [   32/   40] | d_loss: 0.3621 | g_loss: 6.6276\n",
      "Epoch [   32/   40] | d_loss: 0.3797 | g_loss: 7.5283\n",
      "Epoch [   32/   40] | d_loss: 0.3932 | g_loss: 6.6292\n",
      "Epoch [   33/   40] | d_loss: 0.3473 | g_loss: 6.0341\n",
      "Epoch [   33/   40] | d_loss: 0.3699 | g_loss: 6.1555\n",
      "Epoch [   33/   40] | d_loss: 0.3578 | g_loss: 9.1363\n",
      "Epoch [   34/   40] | d_loss: 0.3536 | g_loss: 6.3319\n",
      "Epoch [   34/   40] | d_loss: 0.3466 | g_loss: 7.6963\n",
      "Epoch [   34/   40] | d_loss: 0.3476 | g_loss: 7.8618\n",
      "Epoch [   35/   40] | d_loss: 0.3486 | g_loss: 6.6763\n",
      "Epoch [   35/   40] | d_loss: 0.3426 | g_loss: 8.3675\n",
      "Epoch [   35/   40] | d_loss: 0.3547 | g_loss: 7.6171\n",
      "Epoch [   36/   40] | d_loss: 0.3376 | g_loss: 7.4986\n",
      "Epoch [   36/   40] | d_loss: 0.3366 | g_loss: 10.5351\n",
      "Epoch [   36/   40] | d_loss: 0.3997 | g_loss: 8.3238\n",
      "Epoch [   37/   40] | d_loss: 0.3300 | g_loss: 11.0098\n",
      "Epoch [   37/   40] | d_loss: 0.3432 | g_loss: 10.0950\n",
      "Epoch [   37/   40] | d_loss: 0.3458 | g_loss: 7.7393\n",
      "Epoch [   38/   40] | d_loss: 0.3489 | g_loss: 8.4156\n",
      "Epoch [   38/   40] | d_loss: 0.3396 | g_loss: 9.8488\n",
      "Epoch [   38/   40] | d_loss: 0.3420 | g_loss: 8.2658\n",
      "Epoch [   39/   40] | d_loss: 0.3608 | g_loss: 7.3411\n",
      "Epoch [   39/   40] | d_loss: 0.3643 | g_loss: 9.2192\n",
      "Epoch [   39/   40] | d_loss: 0.3399 | g_loss: 9.5675\n",
      "Epoch [   40/   40] | d_loss: 0.3499 | g_loss: 6.3215\n",
      "Epoch [   40/   40] | d_loss: 0.3405 | g_loss: 7.7598\n",
      "Epoch [   40/   40] | d_loss: 0.3386 | g_loss: 7.1121\n"
     ]
    }
   ],
   "source": [
    "import pickle as pkl\n",
    "\n",
    "# training hyperparams\n",
    "num_epochs = 40\n",
    "\n",
    "# keep track of loss and generated, \"fake\" samples\n",
    "samples = []\n",
    "losses = []\n",
    "\n",
    "print_every = 400\n",
    "\n",
    "# Get some fixed data for sampling. These are images that are held\n",
    "# constant throughout training, and allow us to inspect the model's performance\n",
    "sample_size=16\n",
    "fixed_z = np.random.uniform(-1, 1, size=(sample_size, z_size))\n",
    "fixed_z = torch.from_numpy(fixed_z).float()\n",
    "\n",
    "# train the network\n",
    "D.train()\n",
    "G.train()\n",
    "for epoch in range(num_epochs):\n",
    "    \n",
    "    for batch_i, (real_images, _) in enumerate(train_loader):\n",
    "                \n",
    "        batch_size = real_images.size(0)\n",
    "        \n",
    "        ## Important rescaling step ## \n",
    "        real_images = real_images*2 - 1  # rescale input images from [0,1) to [-1, 1)\n",
    "        \n",
    "        # ============================================\n",
    "        #            TRAIN THE DISCRIMINATOR\n",
    "        # ============================================\n",
    "        D.train()\n",
    "                \n",
    "        d_optimizer.zero_grad()\n",
    "        # 1. Train with real images\n",
    "\n",
    "        # Compute the discriminator losses on real images\n",
    "        # use smoothed labels\n",
    "        D_real = D(real_images)\n",
    "        real_l = real_loss(D_real,smooth=True)\n",
    "        # 2. Train with fake images\n",
    "        \n",
    "        # Generate fake images\n",
    "        z = np.random.uniform(-1, 1, size=(batch_size, z_size))\n",
    "        z = torch.from_numpy(z).float()\n",
    "        fake_images = G(z)\n",
    "        \n",
    "        # Compute the discriminator losses on fake images \n",
    "        D_fake = D(fake_images)\n",
    "        fake_l = fake_loss(D_fake)        \n",
    "        \n",
    "        # add up real and fake losses and perform backprop\n",
    "        d_loss = real_l+fake_l\n",
    "        d_loss.backward()\n",
    "        \n",
    "        d_optimizer.step()\n",
    "        \n",
    "        \n",
    "        # =========================================\n",
    "        #            TRAIN THE GENERATOR\n",
    "        # =========================================\n",
    "        g_optimizer.zero_grad()\n",
    "        D.eval()\n",
    "        \n",
    "        # 1. Train with fake images and flipped labels\n",
    "        \n",
    "        # Generate fake images\n",
    "        z = np.random.uniform(-1,1, size=(batch_size, z_size))\n",
    "        z = torch.from_numpy(z).float()\n",
    "        \n",
    "        fake_images = G(z)\n",
    "        \n",
    "        # Compute the discriminator losses on fake images \n",
    "        # using flipped labels!\n",
    "        \n",
    "        # perform backprop\n",
    "        D_fake = D(fake_images)\n",
    "        g_loss = real_loss(D_fake)\n",
    "        g_loss.backward()\n",
    "        \n",
    "        g_optimizer.step()\n",
    "\n",
    "        # Print some loss stats\n",
    "        if batch_i % print_every == 0:\n",
    "            # print discriminator and generator loss\n",
    "            print('Epoch [{:5d}/{:5d}] | d_loss: {:6.4f} | g_loss: {:6.4f}'.format(\n",
    "                    epoch+1, num_epochs, d_loss.item(), g_loss.item()))\n",
    "\n",
    "    \n",
    "    ## AFTER EACH EPOCH##\n",
    "    # append discriminator loss and generator loss\n",
    "    losses.append((d_loss.item(), g_loss.item()))\n",
    "    \n",
    "    # generate and save sample, fake images\n",
    "    G.eval() # eval mode for generating samples\n",
    "    samples_z = G(fixed_z)\n",
    "    samples.append(samples_z)\n",
    "    G.train() # back to train mode\n",
    "\n",
    "\n",
    "# Save training generator samples\n",
    "with open('train_samples.pkl', 'wb') as f:\n",
    "    pkl.dump(samples, f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training loss\n",
    "\n",
    "Here we'll plot the training losses for the generator and discriminator, recorded after each epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f6009a33410>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2deXxU9dX/32eys4Yl7IRNFIGwBgFBUVTcl7ZardZqa2t96lLFWn3aPq3a5ddaq9VqVepa9alaFLdqn4qCO2gAZRXZ9yUkEhIgyWTm+/vje28yhCyz3NnCeb9e87ozd+5ycjPzmXPP93zPEWMMiqIoSvrhS7YBiqIoSnSogCuKoqQpKuCKoihpigq4oihKmqICriiKkqaogCuKoqQpKuBKyiIiGSJSJSKFXm6rKG0F0TxwxStEpCrkZTugBgg4r39ojHk28VbFjoj8BuhnjLky2bYoSiiZyTZAaTsYYzq4z0VkI/B9Y8zc5rYXkUxjTF0ibFOUtoiGUJSEISK/EZHnReQfIlIJfFtEJovIAhHZKyI7ROR+Eclyts8UESMiA53XzzjvvykilSLysYgMinRb5/0zReRLEakQkb+IyIcicmUUf9MIEXnXsX+ZiJwd8t45IrLKOf9WEbnJWd9DRN5w9ikXkfdC9uknInNEpFRENojItSHvTRKRxSKyT0R2icgfI7VXaVuogCuJ5mvA/wKdgeeBOuDHQHdgCnAG8MMW9r8U+B+gK7AZ+HWk24pID+AF4BbnvBuA4yL9Q0QkG3gd+BdQANwEPC8iRzmbPAFcZYzpCIwC3nXW3wKsd/bp5diIiGQ4x/sU6AucBtwiIqc4+/0F+KMxphNwFDA7UpuVtoUKuJJoPjDGvGaMCRpjDhpjPjXGLDTG1Blj1gOzgGkt7D/bGFNijPEDzwJjotj2HOAzY8wrznv3Anui+FumANlYUfU74aI3gUuc9/3AcBHpaIwpN8YsDlnfByg0xtQaY1xhnwR0Msb8zlm/Fnis0fGGikg3Y0ylMWZhFDYrbQgVcCXRbAl9ISLDRORfIrJTRPYBd2K94ubYGfL8ANChuQ1b2LZPqB3GjuRvDcP2xvQBNptDMwE2Yb1nsHcb5wGbRWS+iEx01v/e2e5tEVknIrc46wcAhU5oZa+I7AV+ivXSAb4LDAdWi8gnInJWFDYrbQgVcCXRNE57egRYDhzlhAZ+CUicbdgB9HNfiIjQILqRsB3o7+zvUghsA3DuLM4DemBDI8856/cZY24yxgwELgBuFZFp2B+VNcaY/JBHR2PMuc5+q40xlzjH+xPwoojkRmG30kZQAVeSTUegAtgvIsfScvzbK14HxonIuSKSiY3BF7SyT4aI5IY8coCPsDH8m0UkS0SmA2cBL4hInohcKiKdnDBNJU5KpXPeIY7wVzjrA8DHQK2I3OycI0NEikRkvLPf5SLS3RgTdPYzQNDja6OkESrgSrK5GbgCK3CPYAc244oxZhdwMXAPUAYMAZZg89ab49vAwZDHamNMDXAucD42hn4/cKkx5ktnnyuATU5o6Crgcmf9McA7QBXwIXCfMeYDJ6XyLOyA6kbnmI8AnZz9zgJWORk8dwMXG2Nqo78SSrqjE3mUIx4n+2M7cKEx5v1k26Mo4aIeuHJEIiJniEhnJxTyP9hQyCdJNktRIkIFXDlSmYrNxd6DzT2/wAmJKEraoCEURVGUNEU9cEVRlDQlocWsunfvbgYOHJjIUyqKoqQ9ixYt2mOMOSzVNaECPnDgQEpKShJ5SkVRlLRHRDY1tV5DKIqiKGmKCriiKEqa0qqAi8jjIrJbRJY38d5PnBrMLRUfUhRFUeJAODHwJ4EHgL+HrhSR/th6xZu9N0tRlHjh9/vZunUr1dXVyTZFaURubi79+vUjKysrrO1bFXBjzHtul5NG3IstdflKJAYqipJctm7dSseOHRk4cCCHFlJUkokxhrKyMrZu3cqgQYNa34EoY+Aich6wzRjzeRjbXi0iJSJSUlpaGs3pFEXxkOrqarp166binWKICN26dYvozihiAReRdsDPsXWbW8UYM8sYU2yMKS4oaK1ip6IoiUDFOzWJ9P8SjQc+BBgEfC6283g/YLGI9GpxL0VJFWoPwKInIRhItiWKEhMRC7gxZpkxpocxZqDTUWQrMM4Ys7OVXRUlNVj8d3jtx7Dpw2RbcsSSkZHBmDFjGDFiBKNHj+aee+4hGLS9KUpKSrjhhhtiPsfDDz/M3//+99Y3DOH444+P+nxPPvkk27dvj3r/aGh1EFNE/gGcBHQXka3Ar4wxj8XbMEWJGytftsudy2DQicm15QglLy+Pzz77DIDdu3dz6aWXUlFRwR133EFxcTHFxcUxHb+uro5rrrkm4v0++uijqM/55JNPMnLkSPr06RP2PoFAgIyMjKjP2aoHboz5ljGmtzEmyxjTr7F4O554NB29FSXx7NsBmxfY5zsPm9qgJIEePXowa9YsHnjgAYwxzJ8/n3POOQeAd999lzFjxjBmzBjGjh1LZWUlAHfddRdFRUWMHj2a2267DYCTTjqJn/3sZ0ybNo377ruP22+/nbvvvrv+vZtuuokTTzyRY489lk8//ZSvf/3rDB06lF/84hf1tnToYPtez58/n5NOOokLL7yQYcOGcdlll+FWbr3zzjuZMGECI0eO5Oqrr8YYw+zZsykpKeGyyy5jzJgxHDx4kLfffpuxY8dSVFTE9773PWpqbLXigQMHcueddzJ16lT++c9/xnTtEloLRVGSzqrXAANdBlkP/AjnjtdWsHL7Pk+PObxPJ3517oiI9hk8eDDBYJDdu3cfsv7uu+/mwQcfZMqUKVRVVZGbm8ubb77Jyy+/zMKFC2nXrh3l5eX12+/du5d3330XgNtvv/2QY2VnZ/Pee+9x3333cf7557No0SK6du3KkCFDuOmmm+jWrdsh2y9ZsoQVK1bQp08fpkyZwocffsjUqVO57rrr+OUvbQ7H5Zdfzuuvv86FF17IAw88wN13301xcTHV1dVceeWVvP322xx99NF85zvf4aGHHuLGG28EbL73Bx98ENE1agqdSq8cWax8GQqOheHnQ+kXUKctJVOFpnoTTJkyhZkzZ3L//fezd+9eMjMzmTt3Lt/97ndp164dAF27dq3f/uKLL272+Oeddx4ARUVFjBgxgt69e5OTk8PgwYPZsmXLYdsfd9xx9OvXD5/Px5gxY9i4cSMA8+bNY+LEiRQVFfHOO++wYsWKw/ZdvXo1gwYN4uijjwbgiiuu4L333gvLzkhQD1w5cqjcBZs+gmm3QvehEPTDntXQqyjZliWNSD3leLF+/XoyMjLo0aMHq1atql9/2223cfbZZ/PGG28wadIk5s6dizGm2XS79u3bN3uOnJwcAHw+X/1z93VdXV2z24MddK2rq6O6upof/ehHlJSU0L9/f26//fYm87Zba5TTkp2RoB64cuSw6lXAwIgLGkRb4+BJp7S0lGuuuYbrrrvuMGFet24dRUVF3HrrrRQXF/PFF18wY8YMHn/8cQ4cOABwSAgl3rhi3b17d6qqqpg9e3b9ex07dqyP0Q8bNoyNGzeydu1aAJ5++mmmTZvmuT3qgStHDitfge5HQ8EwMEHIzHPi4N9KtmVHHAcPHmTMmDH4/X4yMzO5/PLLmTlz5mHb/fnPf2bevHlkZGQwfPhwzjzzTHJycvjss88oLi4mOzubs846i9/97ncJsTs/P58f/OAHFBUVMXDgQCZMmFD/3pVXXsk111xDXl4eH3/8MU888QQXXXQRdXV1TJgwIaqsmNZIaE/M4uJic8Q1dNi3HRDo1DvZlhzZVO2GPx0DJ/wEpv/crpt1MuR0gCteS65tCWbVqlUce+yxyTZDaYam/j8issgYc1hupYZQ4s3sq+DV65NtRdvj7+fDu3eFv/2q16zXPeKChnW9iqwHro29lTRFBTyeBOpg+2Ko1EmqnuI/COvnw3t/hPIN4e2z8hXodhT0GN6wrlcRHPzKuUtSlPRDBTyelK2Fumqo3puc81fuhOUvQcXW5Jw/XpSvt8tALbx9Z+vb798DG9+H4RdA6CBZ/UCm5oMr6YkOYsYTVxgOJknA//MLWObM9OoyCAadAANPtMuOaVx7rMyO7DP0dFjxEky+Fvq1MPW6qfAJQE8nhW7XMjjmjPjYqihxRAU8nux0yqXXVtpwSkYCL3ftAfjiDTj2PCicbD3QFa/YQk4A3YbaOiBjL4O+4xNnlxeUrbPL8+6Hh0+wP1TfffNQ7zqUla9A18HQc+Sh63M6OjMyNZVQSU9UwOPJjqUNz2v2QbuuzW/rNV++Cf79cNzV1uOe/CNbPnXnUtjwvhX0z/8Bmz+GH32cOLu8oGwddOhl7yJO/hm8fiN88Toce+7h2+4vgw3vwZQfNy3wvUZqCEVJWzQGHi+MscKQ5cy4SnQcfNmL0LE3DAgpj+nLgD5jYcoNcNk/YepM2L3SDuSlE+XroNsQ+3zs5dD9GHjrl01Pi//idTABO3W+KXoW2Zh6TVX87G3MnjXglE49ktm1axeXXnopgwcPZvz48UyePJk5c+YkxZb58+fHVIkwWaiAx4t92+BgORROsq8TGQc/uBfWvgUjvm5FuzkKJ9rllk8TY5dXlK1tEPCMTJjxayvCi544fNuVr0CXgdB7dNPH6lUEGPtDFm+Mgff/BA8Uw6ePxv98KYwxhgsuuIATTzyR9evXs2jRIp577jm2bo3fgHtT0+VdohHwlo6XKFTA44V7W+7Wm06kB77qNZuhMfIbLW/Xdzz4Mm0YJV2oroD9pdB1SMO6oTPsdZ7/e/u+y4Fy2PDu4dknofRy4uLxDqMYA3N/5WTNCKz5T3zPl+K88847ZGdnHzI7ccCAAVx//fUEAgFuueUWJkyYwKhRo3jkkUeAlku8Llq0iGnTpjF+/HhOP/10duzYARxeYva1115j4sSJjB07llNPPZVdu3axceNGHn74Ye69917GjBnD+++/z6ZNmzjllFMYNWoUp5xyCps3bwbsbMuZM2dy8sknc+uttyb4qh2OxsDjxY6lgMDAqfZ1qLDEm+Uv2sG5vuNa3i67PfQaBVsWJsYuL3AHMLsd1bBOBGb8Bh6ZBu/fA6fdYdevfgOCdc2HTwA694fczvEV8GAA/nWzvUMovgow8PnzNuSTmR2/84bDm7d5/7f3KoIzf9/iJitWrGDcuKY/n4899hidO3fm008/paamhilTpjBjxgyg6RKvEydO5Prrr+eVV16hoKCA559/np///Oc8/vjjwKElZr/66isWLFiAiPDoo49y11138ac//YlrrrmGDh068JOf/ASAc889l+985ztcccUVPP7449xwww28/LJtBPLll18yd+7cmBoxeIUKeLzYudTe5ndyunMkKoRStdt6nVNnNu91hlI4CUoeTw0xCYemBBxsiGT0JbDgIZhwFeQXwoqX7bLP2OaPJ2Lj4LvilIkS8MOca2D5bPs/OeWX9g6p5HHYtggGTI7PedOMa6+9lg8++IDs7GwGDBjA0qVL6wtFVVRUsGbNGrKzs+tLvAL1JV7z8/NZvnw5p512GmC73PTu3VC6IrR069atW7n44ovZsWMHtbW1DBo0qEl7Pv74Y1566SXA1vz+6U9/Wv/eRRddlBLiDSrg8WPnUuhbbL07SFwIZcXLNue56MLwti+cBAv+Cjs+h/4TWt8+2ZSvA8TGtRsz/RewYg68/Ws46y47W3PSNa3/kPUqgsVPWU+5pTGDSPEfhBeugDX/B6feDlNvsusHTrV/w4Z3ky/grXjK8WLEiBG8+OKL9a8ffPBB9uzZQ3FxMYWFhfzlL3/h9NNPP2Sf+fPnN1ni1RjDiBEj+PjjpkOBoaVbr7/+embOnMl5553H/PnzD2v60ByhVRK9KgXrBRoDjwcH98LezVYYstqBLytxIZTls6HHCOgRZrGi/s4g65YF8bPJS8rWQn5/yMo9/L3O/eyknmUvwLzf2Xrfw7/W+jF7jQT/gfCn5YdD9T545kIb6z7n3gbxBptO2ns0rH/Xu/OlGdOnT6e6upqHHnqofp1bHvb000/noYcewu/3AzZksX///maPdcwxx1BaWlov4H6/v8kmC2C9+b59+wLw1FNP1a8PLQULtrnxc889B8Czzz7L1KlTo/kz444KeDxwY4q9R1nvL7dzYkIoezfbeHZRK4OXoXTsab3ZzWkk4KEDmI2ZciO06w6fzLLx7dbGASBkSv3SlrcLlwPl8Pfz7ODwNx6F4u8dvs3gabD1U6htXpjaMiLCyy+/zLvvvsugQYM47rjjuOKKK/jDH/7A97//fYYPH864ceMYOXIkP/zhD1vM+MjOzmb27NnceuutjB49mjFjxjSbUXL77bdz0UUXccIJJ9C9e/f69eeeey5z5sypH8S8//77eeKJJxg1ahRPP/009913n+fXwBOMMS0+gMeB3cDykHV/BL4AlgJzgPzWjmOMYfz48eaI4KMHjflVJ2Mqd9nX948z5oUr43/e9++x5y3fENl+L/3QmD8MNiYYjItZnhEMGvO7/sa8fnPL2y2cZa/Dv38W3nH91cbc0dWYuXfEbqMxxvztVGPuLDDmizea32bNW9bGNW95c84IWLlyZcLPqYRPU/8foMQ0oanheOBPAo0LRbwFjDTGjAK+BP7bix+TNsPOpXamYIce9nVufmJi4MtftHH3puLDLdF/IhzY01AkKlXZvwdqKg4fwGzM+Cvh5F/ApB+Fd9zMHDsZyItsjNLVsPUTmwlzzJnNb1c42YbWjuAwihI7rQq4MeY9oLzRuv8YY9x7mgVAvzjYlr7sXHZon8VEhFBKv7TnDXfwMpRCZyAt1cMobhGrbi2EUAAysmDaLdC5b/jH7jXSm5ooK1+1y+EXtLxddnvof5wdyFSUKPEiBv494M3m3hSRq0WkRERKSktLPThdilNXY7ud9x7VsC4vP/6DmMtng/hgRBiDdo3pfrS9S0j1CT3lbgphKwIeDb2KoHK7rZ0SC6tesXc04XRgGjTNzhc4kLieji5Gm1ikJJH+X2IScBH5OVAHPNuCQbOMMcXGmOKCgoJYTpce7F5pJ4/0ChHweIdQjIFls216WjRlYn0+KzqpPqGnbK2dOdq50Ptju5UKd8UQRinfYO+Cjj0vvO0HTwOMLSyWQHJzcykrK1MRTzGMMZSVlZGb20SGVTNEnQcuIlcA5wCnGP0kNODGURuHUKorrNCGM7kmUnZ8Zr3TKT+O/hiFk2y+8v4yaN/NO9u8pGydnWEaj7K8oV3qB58U3TFWOeGTpqoiNkXf8ZDdwcbBW5ot6iXPXEi/wqls7X8+R8QdcZqRm5tbP1EpHKL6JojIGcCtwDRjzIFojtFm2bEUsp060y55+dYrr91vm+h6zbLZdkBseJieX1O4Rbe2LIRhZ3ljl9eUrWt9ADNa2ne31RtjGchc+Sr0HgNdBoS3fUaWrRaZqDh46Zew9i2y9qxm0AnNlNdV0opWQygi8g/gY+AYEdkqIlcBDwAdgbdE5DMReTjOdqYPO5fZATFfyKWN52zMYNDOPjzqVMjrEv1x+oy1PwKpOqEnGDy0jGw86Dky+in1FdtgW0nkP6KDptnQUMW26M4bCSttLQ/2brYlbZW0J5wslG8ZY3obY7KMMf2MMY8ZY44yxvQ3xoxxHte0dpwjgmDQCkBo/BtsDBziM5C5ZYEtXRtN9kkoWXlWxFM1E6Vyu+0vGk8B71VkB6DraiLfd9VrdnlshKGQwdPsMhFe+Io5DZOg1r4V//MpcUdnYnrJVxugturQ+DfYEArEJ5Vw2WzIzIOjPejpWDgRti8Bf3Xsx/Ka+hTCOIVQwN45BetsLnekrHrVdrzvHqF9PUZAu27xzwcv/dIOsB93NRQMgzUq4G0BFXAv2eH0wOzd2AOPUwgl4Le3xcec6U1svf8kW0d8x2exH8tr3CqELU2jjxX3zinSOHjlLtj0UfjZJ6H4fLaW+YZ37SB3vHDDJ8PPs+G2TR8mtguREhdUwL1k5zKb5lYw7ND18QqhrHsHDpTFHj5xcQcyw8kHN8ZWPkyUCJSts4XBOoaRXx0tXQfbu5lI4+BfvA6Y6AeRB02Dyh3xjUuveNn+QHfqA0NPsz/UCU5fVLxHBdxLdi6FgmPt1OxQXA/c6xDKh/dDxz5w1GneHK99dxui2BxGPvhnz8I/r7DLRFC21gqsL44fWV8G9BwRuQe+6lV7Z9BjeHTnjXccfM8a2L0CRjizQwsn216tGkZJe1TAvWTnssPDJxASQvHQA9/yCWz6AI6/zttGDIWTbCphS013D5TDf/7HPt+2yLtzt0S8M1Bc3C714YYzDpTDhvet9x1tWl6XQXZyUrwEfIUTPnFDPJk5Ntd97VvxDdsocUcF3Csqd0HVrsMHMMF6djmdvY2Bv3+PTRscd4V3xwR7m32wHMpauJ1/65dQs8+m3W0t8fb8TRHww1cb4zuA6dKryP6f9oWZ1rf6Ddv1Ppr4t4uIEwd/3zaV8JqVL9uZtqG1YYae6qQTfun9+ZSEoQLuFW4t6cYphC5eFrTatRK+fBMmXuP9xKD6OHgz6YSbPoYlT9vGCSO/YT3jeNfy2LvZZofEcwDTpac7IzPMMMrKV6333FLbtnAYPM3+cHhVk9xlz1ob029cXMsNu2kYJa1RAfeKegEf2fT7eZ29C6F8cK+NYR53tTfHC6XbUTatram6KAE//GumbZQw7VY7FRxs6mE8aa4PZjzoORyQ8CoTVu+D9fPs1PlYZzUOOtEuvU4nXDnHLhtP1c/vb8drNB88rVEB94odS20dbjfe3RivClp9tdHW/S7+rm3N5TUiNozSVCbKgr/aXOIz77LlUPuMAQS2LfbejlDCLSPrBTkdoeug8DzhL//PZnPEUsLApWMvm73kdRx8xSuHh09chp5q0x81nTBtUQH3isY1wBvjVQjlw/tt2djJ18Z+rOYonGibO1Ttbli3dzPM/z0cc3ZDrZTczrYUbbwHMsvX2XO1S1CRrV5FVsBbG+Bb9Ypt3NHvOG/OO2iaDVFFMxO0KcrW2eqKzdUmP8pJJ9zwnjfnUxKOCrgX1FRakek1uvltvKgJXrkLljwDY75l83njhdvgITSM8uatdnnmHw7dtu94K+DxzGYoW2vDJ4kqvjRgqr3Tee5S2wWoKWr3w5q5cOw53qU2Dp4GdQdtr0wvWNFM+MSlcLKthqhhlLRFBdwLdjkdsFv0wD0IoSz4q+20PuXG2I7TGr1HQ0ZOw0DmF/+y2RYn3WZjp6H0HQf7d0PF1vjZU7YuMQOYLhO+D6f/P1j7Nvx1ctMDfWvnWrGNJfukMQOm2LurpuLgxtgp/h8/aDOQAv7Wj7fyZXt30Fxnosxsm064Zq6mE6YpKuBesMOJlzaVA+6Smw/+A1BXG905Du6FTx+z3lS8Y8GZOdaz3rzAxkff+KmdpNJUj0l3IDNeYRT/QfvjkIgBTBefDyb/CK6eZyc3PXshvHGLtcVl5auQ19WKrlfk5dtsFjcOXl1hz/PqDXDvSHjwOPi/n8Hbd8CL34dA853aKVtnw3ojmgmfuBx1KlRoOmG6ogLuBTuXQrvuLU/zzotxOv2nj0JtJUy9Kbr9I6Vwoq3tMvd22LcVzrnX1q9uTM+RkJEdPwEv3wCYxAxgNqbnCPjBPJh0LXwyCx5xWqDV1dgBzGFne99cYtA0m1v/+Jnwh0HwwuWw/CU7YHzOn+HGZTDjN9a7fvW65idctRY+cRnqphP+x7u/IRXZtx0WzmpzdxpxaG1yBLJzqQ2ftBSjDS1o1SHC1nK1B2DBQ9Zb6t1CnN1L+k+C4L3w6d9g7OUN+eGNycy2ue/xykRJZAZKU2Tlwhm/sxkbc/4L/jbdpg3WVsani86x58BH94N/P0y9EYacYpsfh/54Hn+9vRuY91vIzLU/ro0/eytfhn4ToHMr3V0697PphGvessdtq3z6GLx/t/0/dh2cbGs8QwU8VgJ+2L0KJv1Xy9vFUtBqyTNwYA9MnRn5vtHS38msyOsKp93Z8rZ9x1sbgwE769RLyhNQhTAchkyHH30Mr90AK16yM2sHTfP+PH3Hwy9KWx8YPfEWG5L74F5by/303zWIuBs+mfHb8M459DRY+LANl8WjY1QqsN1xMHYub1MCriGUWCldbVOxmpuB6RJtQauA33pk/Sfa9luJol1XKxJfe6T1fPO+463HWPqF93aUrYX2PSC3k/fHjpR2XeGbT8M3HoPzH/C2Bk0o4WS1iMApv4KJ/2UHt9/5TcN79aVjw7xDGNrG0wmNaZhsFm3HpRRFPfBYaW0KvUt9DDxCAV82Gyq2wFl3J76H4fRfhLdd6EBmzxHe2lC2PrEDmK0h4l353lgRgTP+n82Gef9uG+458RZbvKpv8eEZQ83Rf1JDOmGq9kONhb2b4OBX9nk4M2zTCBXwWNm90sYhW4vR5kYh4MGgvUXuMQKOPj16G+NN18H2DmPbIhj3HW+PXbYWjp7h7THbEiJw9r22i9I7v4H9ZdapCDd8AiHphE51wrbW7Nj1vrsOthOb2hAaQomVA+U2A6W12G80IZQ9q+1j4tWp/aXy+aDPOO8zUar32RzzVPLAUxGfD85/0IZMFj5k10U6wDr0NHunF007uVRn+xKbKVV0kZ1RHI/etElCBTxWqiuar38SSlau9dQj+fDs226X3Y+OzrZE0ne8rZJYe8C7Y5YnsIhVupORCV9/1IrU8AvCD5+4uNUJ2+KszG2Lbbprn3H29a6VybXHQ1oVcBF5XER2i8jykHVdReQtEVnjLLvE18wUJlwBh8hnY1btsssOPSO3K9H0HW/rYntZDjURfTDbEpnZ8I1H4ZtPRb5v5752slZbKy8bDNr5DH3HNVQKbUMDmeF44E8CjVue3wa8bYwZCrztvD4yqa4IP0Mi0oJWlTvssmOvyO1KNPGYkVm2DhBbHVCJP0e51Qkrk22Jd5Svt81H+oyFTn2tExVpy7wUplUBN8a8BzSu2H8+4P7MPwW0Ml+3DROJBx5pQavKXZDTyZZuTXU69rR1wj0V8LX2mFl53h1TaZ6hM2ytnbaUTugOYPYZa8eRehUdcR54U/Q0xuwAcJY9mttQRK4WkRIRKYXjdPYAACAASURBVCktLY3ydClMzb4IQigRtlWr2pke3rdLX48HMsvWQre2M+ki5Sl00wnfTrYl3rF9MWTmQfdj7OueI20MPB6t65JA3AcxjTGzjDHFxpjigoIIp5CnOsZYjzon3BBKpB74zvSIf7v0HW/LsO4vi/1YxjiNjHUAM2FkZNnOQOveSbYl3rF9iS0y59as6TXS5s2Xb0iuXR4RrYDvEpHeAM5ydyvbt01qq8AEIwuhRBQD39lygaxUI5I4eDDQMLmiKQ6U2R87HcBMLEOmw1cbbOw43QkG7ABmaL9Sd6JZG8kHj1bAXwXcduhXAK94Y06aUb3PLiMKoVQ0X0EuFGMcAU8jD7z3GFvPOhwBf/NWuGswPHcZrJ9/eJW4+iJW6oEnlCHT7bIteOF7vrT1Ytz0QbCFuySjzczIDCeN8B/Ax8AxIrJVRK4Cfg+cJiJrgNOc10cebjgkkjRCjK1k1+qx90KgJr088JwOtq9jawK+5VNbHrf/RNt78+/n21rXC2c1/CjWNzJWDzyhdB0M+YWwbl6yLYmd0AFMl6xc6D60zQxktjqV3hjzrWbeOsVjW9KPegEPMwbu1kM5uLd10a/caZfpFAMHO5D5xRvNT8kO1MHrN9mWcJf9E3xZtvjSJ7PgzVtss4LR37LX1pcJ+QMS/zccyYhYL3z5S7aQWlM14NOFbYvtoGzju7ieIxu6TaU5OhMzFiL2wENqgreGK+Dp5IGDjYMfLLeDmU3xySM2/njmH2wH+KxcGH0J/OAd+P47ttb24qdg2QtWvL1umKC0zpBTbHZVvJtVx5vtS2xYr3F1x14jbZOSlsZg0gQV8FiocWPg+eFtH0lN8HoBT6M0Qmh5ILNiK7zzWxh6Ogw75/D3+42Hrz0MM1fZGuSn3h5PS5XmGHSiHctI53TCgN9O2Okz5vD3ejq9a91etmmMCngsROuBh5OJUpWmAt5juK350lSHnn/fZrN2zrqr5eJc7bvDlB/DcA8bBivhk5dvy9Gm80Dm7lV2DKnvuMPfc6fUJ2ogc38ZfPQX+GqT54dWAY8FNxQSbh54JH0xK3emzyzMUDKybNu3xh746n/Dqtdg2k+hy8CkmKZEwJDpdhLMgcaTsNOEpgYwXTr0hHbdEpdKuGs5/OcXNj3TY1TAY6F6n53lFW5nlkhqgqfbJJ5Q+o63+bcBv31de8B2dS8YBpOvS65tSngMmW7vltJ1Wv32xfaOt0sTdXRE7EBmojzwOKbEqoDHQiR1UMCOiIsvvBBKZZpNow+l73g72233Kvv6vbugYjOcfU/82pAp3tJ3vO37ma5hlO1LGuqfNEWvIifMUhd/W8rXW0evYx/PD60CHguRVCIEOxruTuZpjXSrgxKKG3fctsh+ST76C4z5Ngyckly7lPDJyITBzrT6xpOsUh1/ta130lT4xKXnSBsjd2vOx5OytXY+Qzi9TiNEBTwWIvXAIbyCVvWzMNNUwLsMst3st5bYnO+cjq13tldSjyHTbZceNwSQLuxeYasqtiTg9QOZCYiDuwIeB1TAYyGSSoQu4RS0qq6AumrokKYCLmJvwZc+b2danvZraN8t2VYpkZKu0+rrBzCbyEBx6X6MnSgW7xmZAb+dExGnkhAq4LEQjQceTkGrdM0BD6XveOsF9Z8EYy5LtjVKNHQZaKfWp5uAb1ti+9R27tf8NpnZVsTjPZC5dzME61TAU5JISsm6hBNCSdcc8FCGzrCDNufcG5fYn5IghkyHDe9DXW2yLQmf1gYwXXqNjL8H7oaf4lRVU79ZsVAdpxBKuk6jD6XfeLh5FfQcnmxLlFgYcgr498OWhcm2JDxqD0Dpqpbj3y49R9q2hV7Ur2+OOFfVVAGPFn+1HcWOZwglXfPAlbbDwKk2VpwuYZSdy2z+ejgCnogmx2XrrNPWrmtcDq8CHi2RViJ0ye1shd9/sPltKndCdkdbnlVRkkluJ+h3XPoIeEszMBtTXxMlngK+1nrfrYVzokQFPFrqBTzMQlYu4RS0qkqzRg5K22bIdDuzdv+eZFvSOtsX29BjpzDCjx0K7F1uPAcyy+LbFlAFPFpqIuzG4xJOQat0a6WmtG2GTAeM7ZyU6rgDmOHSc2T8aqLUHrBla1XAUxA3kySaGDi07IGncx0Upe3RZwzkdUn9MEr1PtizJkIBHwGlqxvq9niJ21c0jl2lVMCjxRXgiNMIuzj7N+OBp/ssTKXt4cuAwSel/rT6nUsBE5mA9yqCQK0Vfq9JQF9XFfBoibQWuEtrIZSafbYQlAq4kkoMmW5T7kq/SLYlzRPJAKZLzzhmotTngA/2/tgOKuDREmlHepfWQihtIQdcaXsMPtkuU7lLz7bF0LnQNgQJl+5DISM7PjVRytfb73Ecs8liEnARuUlEVojIchH5h4jkemVYyuM23c3Ki2y/1vpiag64kork94fuR6d2HHz7kqZbqLVERpatUx8vDzyO4ROIQcBFpC9wA1BsjBkJZACXeGVYyuPWQYk0vzMjC7LaqweupB9DpsOmD+0ktlTj4Fe2400k4ROXXkXx6Y8ZxyqELrGGUDKBPBHJBNoB22M3KU2IphKhS0uzMevroKgHrqQYQ06xVTJTsUvP+/fYZf/jIt+350io2gVVpc1vs7/MpgWGy4FyOFCWuh64MWYbcDewGdgBVBhj/tN4OxG5WkRKRKSktLSFC5RuRFOJ0KWlglaVO23nnpyO0dumKPFg8DT72V32z2RbcigLZ8FH90PxVTAgiqYhPUfYZeN8cGNsIa/nL4e7h8JrN4R/zPoUwhQVcBHpApwPDAL6AO1F5NuNtzPGzDLGFBtjigsKCqK3NNWIphKhS0sFrTQHXElVMnNg+AXwxb+gdn+yrbGseg3e/Ckccxac9cfopqz3cqbUuzMya6rg08fgr5PhqXNg4/vQdZAdwA03jTIBKYQQWwjlVGCDMabUGOMHXgKO98asNCAWD7ylEIrOwlRSmVHftNUJV7+ZbEtg80J48fu29vw3HrP56tHQrqstfbx+Hrx5G9xzLPxrph2vOv9BmLkKptwIB8vtpJ9wKFsLkgH5A6KzKUwyY9h3MzBJRNoBB4FTgBJPrEoHoikl65LbGaqbSVuq2hndQIyiJILC46FTX1j6AhRdmDw79qyBf1wMnfrApc9DdrvYjtdrJKz5jw2ZjLgAjrsa+k1o8OgHOL7p5o+gx7DWj1e2DvIL497EO2oBN8YsFJHZwGKgDlgCzPLKsJQnphh4MyGU+lmY6oErKYrPByO/AQv+agf2ktEqr3IXPPN16+F++8XI8r6bY9ptMOhEKPpm0wkEXQfb0Oamj6D4e60fLwEphBBjFoox5lfGmGHGmJHGmMuNMTVeGZbSBPz2NjIWD7xmHwQDh66vqQT/AY2BK6nNqG/aNmEr5yT+3DVV8L/ftJURL3vBu1mO/cbD8dc3n/0lAoWTYdPHrR/LmLhXIXTRmZjRUFNpl7HEwOFwL1xzwJV0oOdIO/llaYKzUQJ++OeVtubJhU/Y2HciGTDFVhfcu7nl7Sp3WgcvzjngoAIeHdFWInSprwneaCBTc8CVdEAEii6CLQvgq02JO++/ZsLat+Dse+CYMxJ3XpcBk+1y00ctb5egDBRQAY+OaCsRujRX0Eo9cCVdcAcwl89OzPnWzYPFf7fZIMXfTcw5G9NjOOR0VgFPe6KtROjSWghFY+BKqtNlIPSfaMMo4eZGr58PS56J/FzBIMy93RaqOvlnke/vFb4MKJwUnoBn5tpsnXibFPcztEWirUTo0lxBq8qdtk6KzsJU0oGii2wH+HDqiHy1EZ67DF65FrZ8Etl5Vs6BHZ9Z8c7MicpUzxgwGcrWtDztvnw9dB1iM3bijAp4NETb0NjFjYE3DqG4vTDj1ABVUTxlxNdtRc5lL7S8XTAIL18LiA0Pvn4TBOrCO0fAD2//GnqMsNkvycadqr+5BS+8bC10i18N8FBUwKMhniEUjX8r6UL7brZC4bIXrUg3x8KHYNMHcObv4cy7bOnWhQ+Hd45FT9oqg6f+KvqZll7Sewxk5jWfThiog/INCYl/gwp4dNTsAwSyowx1ZLWznktTIRSNfyvpRNE3bWrd5mYErXQ1zL0Djj4TxlwGx54LQ2fAvN9BxdaWj11TBe/eZb3eoTO8tz0aMrOhX7Etq9sUFZsh6FcBT2mqK2z4JNoYl0jTszHVA1fSjWFnWYekqTBKwA9zfgjZ7eHc++znXsQWnTIB+PdtLR97wV9h/2449Y7UCisOON7eRbhjYaGUrbNLFfAUprrCphPFQuOCVjWVNvlfc8CVdCK7PQw7G1a8DHW1h773/j22S8459x76ue4yEE68xVYS/PKwCtSW/Xvgw/th2DnQf0LczI+KAceDCTY9GJvAFEJQAY+OWOqguDSuCa454Eq6UvRN+1leO7dh3fYl8N5dNlNlxAWH73P8DbZF2xs/abpRwnt3W4fmlF/Fz+5o6TfBhkCbCqOUrbXf7XaJqRGjAh4NsVQidGkcQtEccCVdGXKyFSw3jOKvhjnXQPsCGy5pisxsO6Ny7yZ4/+5D3/tqI3z6KIz9NhQcHVfToyK7vR3MbCru79ZASVDIRwU8GrzywA+qB660ATKyYMTXbI3wmkqY9xso/QLO+wvkdWl+v0EnwKhLbKgktM72vN/ZjJOT/jv+tkfLgMmwbdHh/UHL1tkc8AShAh4N7iBmLOQ18sC1DoqSzhR90/bL/M8v4KMHYPyVMPS01veb8Rtby/tfN9sZnTuX2VrjE6+xtb5TlQFTIFBrRdzFfxAqtiQs/g0q4NHhiQeeb+OG7jTkyp12ND/a+iqKkkz6H2cbGCx60i5n/Ca8/ToUwKm327ZlS5+3KYe5nWHqjXE01gP6T7TL0Gn15RsAk5AqhC4q4JESDMbWkd4lt7Otqez2FnRzwFMpXUpRwkXEhkMQ+NrDkZWDGHcl9C22Xvjat+CEmS2HXlKBdl1tcavQGZkJzkABFfDIqa0ETOyecuPZmJoDrqQ7J9wMP1rQ0H4sXHw+m2roP2ALQB13dXzs85oBx9tUQrcsQL2AqweeusQ6jd6lcUErtw6KoqQrWbnh9Ytsit6j4OJn4JJnISvPW7viReFkqK2yDSbADmB26JXQYnQq4JESayVCl8YFrdQDV450hp2dXg293TsNNw5enpg2aqGogEeKVx54aAilptL+kmsOuKKkD5362Fmlbj542dqEhk8gRgEXkXwRmS0iX4jIKhGZ7JVhKUuspWRdQkMolbvsc/XAFSW9GDDFeuAHv4L9pekl4MB9wL+NMcOA0cCq2E1KcTyLgYd44JoDrijpSeFkOFhuJzFBwkMomdHuKCKdgBOBKwGMMbVAbUv7tAlq3Bh4fmzHCe2LqbMwFSU9cePgi5+2yzSKgQ8GSoEnRGSJiDwqIu09sit1qW9oHONIsy/DpiJW79U6KIqSrnQdbL+3mz8C8dmYeAKJRcAzgXHAQ8aYscB+4LACvyJytYiUiEhJaWkLfeTSheoK27cyIyv2Y7kFrSp32C4fsYZlFEVJLCINXnh+YcJ7dsYi4FuBrcaYhc7r2VhBPwRjzCxjTLExprigoCCG06UIXkyjd3ELWlXt0l6YipKuFDoCnuDwCcQg4MaYncAWETnGWXUKsNITq1IZLwU8L78hhKLxb0VJTwYkT8CjHsR0uB54VkSygfXAd2M3KcXxohKhS25nKF9vW0/1HOHNMRVFSSw9httqjMeel/BTxyTgxpjPgGKPbEkPqiugQw9vjpXrtFWrqYSjTvXmmIqiJBafD77xt+ScOilnTWe8qETokpdvm7bWVmoOuKIoEaMCHinVFd7V7HZLyoLGwBVFiRgV8EgwxuMslJDJQJoDrihKhKiAR4L/oPWYvQyhuKgHrihKhKiAR4JXdVBcQo+jMXBFUSJEBTwSvKpE6OKGUDJzY6+toijKEYcKeCR47YG7IRTthakoShSogEeCV5UIXdwfAo1/K4oSBSrgkVBfidDjEIrGvxVFiQIV8EhwGxB7FULJyoXsjtC5vzfHUxTliCLWWihHFl7HwAG+PdvWFFYURYkQFfBIqN4HGTnWc/aKwkneHUtRlCMKDaFEgpeVCBVFUWJEBTwSvJxGryiKEiMq4JHgZSVCRVGUGFEBjwQvKxEqiqLEiAp4JGgIRVGUFEIFPBJUwBVFSSFUwCOhWmPgiqKkDirg4VJXA3UHNY1QUZSUQQU8XKo9LmSlKIoSIzELuIhkiMgSEXndC4NSlvpKhBpCURQlNfDCA/8xsMqD46Q2biErTSNUFCVFiEnARaQfcDbwqDfmpDDxKGSlKIoSA7F64H8GfgoEm9tARK4WkRIRKSktLY3xdElEBVxRlBQjagEXkXOA3caYRS1tZ4yZZYwpNsYUFxQURHu65FOtMXBFUVKLWDzwKcB5IrIReA6YLiLPeGJVKuJ1Q2NFUZQYiVrAjTH/bYzpZ4wZCFwCvGOM+bZnlqUa1RUgPsjukGxLFEVRAM0DDx+3EqF2j1cUJUXwpCOPMWY+MN+LY6UsWgdFUZQUQz3wcNFSsoqipBgq4OGiHriiKCmGCni4aCVCRVFSDBXwcFEPXFGUFEMFPFxUwBVFSTFUwMMhGIDaShVwRVFSChXwcNBSsoqipCAq4OHgTqPXNEJFUVIIFfBw0EqEiqKkICrg4aCVCBVFSUFUwMNBKxEqipKCqICHg4ZQFEVJQVTAw0EFXFGUFEQFPBzcNELNQlEUJYVQAQ+H6grI7gi+jGRboiiKUo8KeDjoNHpFUVIQFfBwUAFXFCUFUQEPh+oKTSFUFCXlUAEPB/XAFUVJQVTAw0EFXFGUFCRqAReR/iIyT0RWicgKEfmxl4alFDXajUdRlNQjlq70dcDNxpjFItIRWCQibxljVnpkW2pgjDY0VhQlJYnaAzfG7DDGLHaeVwKrgL5eGZYy1FaBCaoHrihKyuFJDFxEBgJjgYVNvHe1iJSISElpaakXp0ssWolQUZQUJWYBF5EOwIvAjcaYfY3fN8bMMsYUG2OKCwoKYj1d4tFKhIqipCgxCbiIZGHF+1ljzEvemJRiaCErRVFSlFiyUAR4DFhljLnHO5NSDBVwRVFSlFg88CnA5cB0EfnMeZzlkV2pw7ZFdpnXJbl2KIqiNCLqNEJjzAeAeGhLsyzfVsG+aj/FA7qSnZnAuUcr5sB7d8HwC6DLoMSdV1EUJQxiyQNPGI9/uIGXFm+jQ04mU4/qzknHFHDSMT3o1Tk3fifdvABe+iH0nwRfewQkIb9ViqIoYZMWAn7n+SM5Y0Qv5q0u5d3Vu/n3ip0ADOvVkZOH9eCkowsYN6ALWRkeeed71sI/vgWd+8G3/gFZcfyhUBRFiRIxxiTsZMXFxaakpCSmYxhj+HJXFfNX72be6t2UbPyKuqAhN8vHqL75jB2Qz7jCLowr7EJBx5zIT1BVCo+dCjVV8P23oOvgmOxVFEWJFRFZZIwpPmx9ugl4Yyqr/Xy4dg+fbPiKxZu/YsX2CvwB+zf165LHuMIujC3Mp2NuFjV1AWr8QWoDQWr8Qfu6LkjQGE4b3pPJ/fOQp86DXSvgyteh32HXS1EUJeG0WQFvTLU/wIrt+1iy2Qr64k172bmvusltszKEnMwM6oJBav11PN3xAY73L8R/4d/JHnleXO1UFEUJl+YEPC1i4JGQm5XB+AFdGD+gIe1v175qavxBcrJ85GT6yMnMIDvTR4bPDkxW+wNsfvYGjt64gNv93+G1OXlctuNLvj2pkB4dNf6tKEpq0uYEvCl6dmokwsEgBKqhphrqashd+jxHb3wGM+lHnHbUzWz5YAP3v72Gh+av5dzRfbhq6iBG9NGJPIqipBZtW8D91bD0eSh5DPbtgLpq+wjUHr7tseciM37LFJ+PKUd1Z31pFU99tJF/LtrKy0u28cxVEzn+qO6J/xsURVGaoc3FwAGbSVLyGHzyNziwB3oWQf8JkJkLmTmHL3M7wzFnN5kuuPdALV9/6CMO1AR488cn0KV9dvztVxRFCSG9Y+AfPQBr50LPEdBjOPQcDgXDICvv0O1KV8PHD8Lnz0GgBoaeDsdfBwNPiHoiTn67bO6/ZCxf++uH/GzOMv562ThEJ/UoipICpIeAZ2TBwXL49FEbAgEQn83R7jHcPrYtgrVvWa96zLdg0rVQcLQnpx/ZtzM3zziG37/5BS+UbOHiCYWeHFdRFCUW0iuEEgxA+Xqbp717pV3uWgFfbYT23WHCD2DCVfa5xwSDhm8/tpAlm/fyrxumMrigg+fnUBRFaYq2nQdee8B66RlZ3h87hJ0V1Zz+5/cY0K0ds685PrGFtRRFOWJpTsDbhgJlt4u7eAP06pzLH75RxNKtFfx57pdxP5+iKEpLtA0BTyBnjOzNJRP689C76/h4XVmyzVEU5QhGBTwK/uec4Qzs1p6ZL3zG3gNN5JQriqIkgPTIQkkx2udkct8lY/j6Xz/iZ3OW8eClmlqoKLFSWe3nQG0AAUQEEfCJOK/B5xM6ZGfi88Xvu+YPBCmtrGHXvmp2V9bYx75qdu+roXO7LEb3y2d0/870zc9Lie+8CniUjOqXz8wZR3PXv1fzt/fXc/yQ7mRn+sjO8JHlLLMzfGRn+qj2B9hTVUNpVQ17qmrZU1nDnir7KN9fS25WBr065dKrcy49O9lHr0659OiUQ25WRrL/1MMIBA21dUFq64IgkOETMn2CT4QMn+ATIvpwB4OGg/6AfdQ2LIPGkJedQW5mBrlZGeRm+cjNyiAn09fs8Y0xGANBY8jwSUp8yUIJBg3+YJCWcgeCxuAPGPwBe439gaDz3FAbsNUz87Iy7CPbeWRlRF0PP/SaBeuXIc+Dhz6vDQSbsM++DgQNWRk+sjOFrAyf89z5XmT42F9bx5byA2z56iBbyw+w5asDbC4/wJbyg1Qc9Ldqa1aG0KNjLj075dCrcy49OtrvTa9OufTomENWpq/+c5ghgs8HmT4fGT6oCxrKqmoprayxj6oa9jhLd13Z/sPvqH0CXdvnsO+gn9pAEIBu7bMZ3T+/XtBH9cunY24mgWDDtQsE7fUKONetU16W59/ntpGFkiQCQcNljy5gwfryiPfN8And2mfTrUMOB2vr2Lmvmmp/8LDt8ttlkZXhq/dCBHGWDSLp8x263uc8qfdkWrAj9L/f+LNggLqAoaYuQG1dkBpHtOuCrX9mQr9AghV1n+tVOWIfNIaDtbakbySIQI7zRbWicqjohOIT9wtsf2QyMhp+bAw4Qmr3M8bUr3NFzdBwbLvevna9wwxfyCPkddAY6hxRqwva53XB4GH2eUmmT6yQZ/rqxdb9e4IhIm1M89crkWRn+ujXJY/+XdrRv6tddsjNrL/uppHNgaCh/EAtuyqq2bnPPnbvq6Gqpi7q8xd0yKF7xxwKOuRQ0DGHnp1y6n8g3GXX9tlkZviorQvyxc59fL61gs+37OXzLXtZW1rV4o9xKE997zimHV0Qla1tO40wiVT7AyzcUE6NP4A/YKgNWLGrDTR4qdmZPrp3yK7/sHTvkEN+XtYht4LGGPZV17FrXzU7K9wPp72Ns/XNDxWQeqHBftoPXWdf477fCodIfCO1z/JJffXGnEwfOVk+sjMyyMmyHpVxhCAQhEAwaJfG1D83IeIXrP8yWuEQocGLzMqgXbb1tPOy7XN7fYNU+wNU+4Mc9Aeo9geocbx1Yxp+DHzOnYCIFVIRHLsMdUFnGbB2ua/t7597x3D4j6Mr0u4dRegtvcF60wHHwwo4z13h9omQmWG90EyfkFm/tOsa3xiE/g9EsJ6rs212pq/em83J9IFAjT/AgZC7leqQ1/5A0LkGh9rs8zUsfdLwwyoiIX+nfT/DWRf6Q+Vum32ITVJ/x5mV4cMn1Hvjroce6rHnZPro37UdhV3bUdAhx5NwSFVNHTsrqimtrKEuGKz/X7j/D/cz6RPo7gh19w45dMrNjPkOraqmjmVbK1i+rYJqf8BeO1/DZ9D9QfeJcPKwHvTNz2v9oE2gAq4oipKmxCUPXETOEJHVIrJWRG6L5ViKoihKZEQt4CKSATwInAkMB74lIsO9MkxRFEVpmVg88OOAtcaY9caYWuA54HxvzFIURVFaIxYB7wtsCXm91Vl3CCJytYiUiEhJaWlpDKdTFEVRQolFwJsavj1sRNQYM8sYU2yMKS4oiC6FRlEURTmcWAR8K9A/5HU/YHts5iiKoijhEouAfwoMFZFBIpINXAK86o1ZiqIoSmtEPZXeGFMnItcB/wdkAI8bY1Z4ZpmiKIrSIgmdyCMipcCmKHfvDuzx0BwvUduiQ22LDrUtOtLZtgHGmMMGERMq4LEgIiVNzURKBdS26FDbokNti462aJvWA1cURUlTVMAVRVHSlHQS8FnJNqAF1LboUNuiQ22LjjZnW9rEwBVFUZRDSScPXFEURQlBBVxRFCVNSQsBT+W64yKyUUSWichnIpLUbhUi8riI7BaR5SHruorIWyKyxll2SSHbbheRbc61+0xEzkqSbf1FZJ6IrBKRFSLyY2d90q9dC7Yl/dqJSK6IfCIinzu23eGsHyQiC53r9rwzUztVbHtSRDaEXLcxibYtxMYMEVkiIq87ryO/bsZpcZWqD+wsz3XAYCAb+BwYnmy7QuzbCHRPth2OLScC44DlIevuAm5znt8G/CGFbLsd+EkKXLfewDjneUfgS2yN+6RfuxZsS/q1wxa06+A8zwIWApOAF4BLnPUPA/+VQrY9CVyY7M+cY9dM4H+B153XEV+3dPDAte54mBhj3gMad1g+H3jKef4UcEFCjXJoxraUwBizwxiz2HleCazClkZO+rVrwbakYyxVzsss52GA6cBsZ32yrltztqUEItIPOBt41HktRHHd0kHAw6o7nkQM8B8RWSQiVyfbmCboaYzZAVYMgB5Jtqcx14nIUifEkpTwTigiMhAYi/XYUuraNbINUuDaOWGAz4Dd9d624AAAAi1JREFUwFvYu+W9xhi3VXzSvq+NbTPGuNftt851u1dEcpJhG/Bn4KdA0HndjSiuWzoIeFh1x5PIFGPMOGxruWtF5MRkG5RGPAQMAcYAO4A/JdMYEekAvAjcaIzZl0xbGtOEbSlx7YwxAWPMGGw56eOAY5vaLLFWOSdtZJuIjAT+GxgGTAC6Arcm2i4ROQfYbYxZFLq6iU1bvW7pIOApXXfcGLPdWe4G5mA/xKnELhHpDeAsdyfZnnqMMbucL1kQ+BtJvHYikoUVyGeNMS85q1Pi2jVlWypdO8eevcB8bJw5X0TcSqdJ/76G2HaGE5Iyxpga4AmSc92mAOeJyEZsSHg61iOP+Lqlg4CnbN1xEWkvIh3d58AMYHnLeyWcV4ErnOdXAK8k0ZZDcMXR4Wsk6do58cfHgFXGmHtC3kr6tWvOtlS4diJSICL5zvM84FRsjH4ecKGzWbKuW1O2fRHygyzYGHPCr5sx5r+NMf2MMQOxevaOMeYyorluyR6JDXO09izs6Ps64OfJtifErsHYrJjPgRXJtg34B/Z22o+9c7kKG1t7G1jjLLumkG1PA8uApVix7J0k26Zib1eXAp85j7NS4dq1YFvSrx0wClji2LAc+KWzfjDwCbAW+CeQk0K2veNct+XAMziZKsl6ACfRkIUS8XXTqfSKoihpSjqEUBRFUZQmUAFXFEVJU1TAFUVR0hQVcEVRlDRFBVxRFCVNUQFXFEVJU1TAFUVR0pT/D6NvO2mUW/gEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "losses = np.array(losses)\n",
    "plt.plot(losses.T[0], label='Discriminator')\n",
    "plt.plot(losses.T[1], label='Generator')\n",
    "plt.title(\"Training Losses\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generator samples from training\n",
    "\n",
    "Here we can view samples of images from the generator. First we'll look at the images we saved during training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# helper function for viewing a list of passed in sample images\n",
    "def view_samples(epoch, samples):\n",
    "    fig, axes = plt.subplots(figsize=(7,7), nrows=4, ncols=4, sharey=True, sharex=True)\n",
    "    for ax, img in zip(axes.flatten(), samples[epoch]):\n",
    "        img = img.detach()\n",
    "        ax.xaxis.set_visible(False)\n",
    "        ax.yaxis.set_visible(False)\n",
    "        im = ax.imshow(img.reshape((28,28)), cmap='Greys_r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load samples from generator, taken while training\n",
    "with open('train_samples.pkl', 'rb') as f:\n",
    "    samples = pkl.load(f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are samples from the final training epoch. You can see the generator is able to reproduce numbers like 1, 7, 3, 2. Since this is just a sample, it isn't representative of the full range of images this generator can make."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZYAAAGOCAYAAAC0fSVgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAANE0lEQVR4nO3dQXLbxhYFUML1t+Cx978sz7UG4w9UqsAROwTIS+B19zlDV6Ko8ARdv5sGuKzregOAlB9XfwMAjEWwABAlWACIEiwARAkWAKIECwBR/zvyDy/L4mxyyLquS+prmUtOai5mEvWxruvPxBcyl6jmXGwsQHW/r/4GuKs5F8ECQJRgASBKsAAQJVgAiBIsAEQJFgCiBAsAUYIFgCjBAkCUYAEg6tC7wka0/WjmZYm9voudWtffXGCfiveKjQWAKMECQNTQVdh2RaSO1lzMC/7W6z1hYwEgSrAAEDV0FdbSOn0Eo6p4cohx2VgAiBIsAERNWYVtqQXy1C71nDUHsz9m1OtlYwEgSrAAEDVlFTbq+lnFvWv6yuk783rdWdfQfI5Jn1Ctcq/YWACIEiwARE1ZhfFe6YdO1Sv/bc/1TlzD1n/HfJ436r1iYwEgSrAAEDVlFVZlXRyVd7GNyX2TN+q9YmMBIEqwABA1TBU20ho5q1FrgZTWw2+tiso15Co2FgCiBAsAUcNUYYzFCaTvWlXhnj9/VpV3T9FWcS42FgCiBAsAUaow4pxGer+zTtBVrFlGMuq9YmMBIKrrjeVI2vubFzPwc04FNhYAogQLAFFdV2HU8ez/hDxa3Xiu4rt3XgfXOy/xP+z3zOLK2dlYAIgSLABEdV2FPTrLb3U/j+cqzuXz5/s1w71iYwEgSrAAENVdFTbqKxB6Zy7neuepILPM2PMG6iN6qjltLABECRYAorqrwpwEu5YH5vrx7Hxa95jZX6un629jASBKsAAQ1V0V9uhERU/rYo9mO90yo/RpplnNfK/YWACIEiwARHVRhfmkSLiee4u9bCwARAkWAKK6qMKcTLnWWZ94x7n2nKB0yvIYv58+2VgAiBIsAER1UYVZL6+liuTfVGT3zfxQ5JaNBYAowQJAVBdVmCrmWq1rbi71JCqqPf/eCHXNO7gPPtlYAIgSLABEdVGFbVnBz3e0GnFi6DpHrreZ5e25dnuq5d7ZWACIEiwARHVXhVHfSCv9LMzsPDNcaxsLAFGCBYAowQKTWdfVg3y8lWABIEqwABDlVBhMZoZTSVzLxgJAlGABIOpoFfZxu91+v+Mbmcyv8Nczl4zkXMwkx1xqas5lcewQgCRVGABRggWAKMECQJRgASBKsAAQJVgAiBIsAEQJFgCiBAsAUYIFgCjBAkCUYAEgSrAAECVYAIg69Hksy7J4x37Iuq6xz4c1l5zUXMwk6mNd15+JL2QuUc252FiA6nwwV03NuQgWAKIECwBRggWAKMECQJRgASBKsAAQJVgAiBIsAEQJFgCiBAsAUYfeFQZnWdd/Xum0LLHXqsFwKt4rNhYAogQLAFGqME63Xd23qqzxs6tYrcyqda9sVZyRjQWAKMECQFTXVZiVfVzmCX/fB61arOLvQRsLAFGCBYCorquwKmsfwDs4FQYAN8ECQFjXVVjF0xAAZ6r4e9DGAkCUYAEgqmwVtme9S699FVfKmbj+MAYbCwBRggWAqLJV2NZZr1lXv1zL9b/OngfxqKnifWNjASBKsAAQVbYKq7jecdyjisWc6zGTaxypI6vPyMYCQJRgASCqVBV21gNyHsQDJ8Gq+fpdtGcu1X+H2VgAiBIsAESVqsK2K907H4qsuDoCc3MqDAAaSm0sLdXTGXpy72/G7rFrjLSlbNlYAIgSLABEla3CEmtf9bPeo/J8RH/cK/X1NCMbCwBRggWAqLJVWEJrXexppRyRa16PmVxjz7N79/7Z6mwsAEQJFgCiuqvC1Fg1jbTGj+LRvWIm15jh1KSNBYAowQJAVHdVWPrtxqq1jCOnWziHn/OaHt0rI8zHxgJAlGABIKq7KuxZrSpghLWzgkf1lyoGPs1wr9hYAIgSLABETVOFkTfqp9+NyPW/1mz3io0FgCjBAkDUNFWYh8XyPBQJ3GNjASBKsAAQNU0VRp76C7jHxgJAlGABIGrKKsxJsAynwmCf2e4VGwsAUYIFgKihqzAPQp7nyPU1l2t9XX8PDV9jhutuYwEgSrAAEDV0FTbSajkSc7nWvetvJtcY9brbWACIEiwARAkWAKIECwBRggWAKMECQJRgASBKsAAQdfQByY/b7fb7Hd/IZH6Fv565ZCTnYiY55lJTcy7LDJ8NAMB5VGEARAkWAKIECwBRggWAKMECQJRgASBKsAAQJVgAiBIsAEQJFgCiBAsAUYIFgCjBAkCUYAEg6tDnsSzL4h37Ieu6LqmvZS45qbmYSdTHuq4/E1/IXKKac7GxANX5YK6amnMRLABECRYAogQLAFGCBYAowQJAlGABIEqwABAlWACIEiwARAkWAKIOvStsROv6z6uDliX2+i5eZC7QLxsLAFGCBYCoKauwVs2ifjnH9jpTj/uAV9lYAIgSLABEdV2F3atUWqt7q37Z/vmPH3L2Sq1aEkZ1pHbsqaL0mxSAKMECQFTXVdhWYjX88+dP9OvxPNefGRz5Oe/pnrCxABAlWACI6q4Ke3S6K3XKq6cTGACV2FgAiBIsAESVqsISD8WlHnJUf51P/QhjsLEAECVYAIgqVYW98q4o1Ulte+ZphjV8fHxc/S0Mp/XzP+rPvI0FgCjBAkDU5VXYnpNAPuWxX15/3wf31fOO/g6bgY0FgCjBAkDU5VXY0dXxnSvlbOsqc1NTZqjqv7OxABAlWACIurwK2+Os9dIaC7xCLfbJxgJAVBcbSyvtH/2NwGthrnHkurvmNZgDSTYWAKIECwBRXVRh76QCYDaeXznH0d8tX3MZ4XeSjQWAKMECQFSpKmzPue/WP5NY72c+d34m1xa+G+m+sLEAECVYAIgqVYUlPvP+lUpspFX0bE4a1fOo2vXzXotTYQDQIFgAiCpVhW2NsA7O5FGNaZ7n86bd+kadi40FgCjBAkBU2SrsWXtOlo20clbx6FTYqCs/vOLIvdDTPWRjASBKsAAQNVwV5kG98/ikyH64/v3raYY2FgCiBAsAUWWrsKOv0Od9ejqNAlzPxgJAlGABIKpsFXYFlc99rgWcZ4TfQzYWAKIECwBRZauwo+/8Sryqvde18yqvfOInMC4bCwBRggWAqLJV2Farchnh9MQoWtffXOr5um98wmRNI8zFxgJAlGABIKqLKmyrp3UQp8UquncPua9q6nUuNhYAogQLAFGCBYAowQJAlGABIEqwABAlWACIEiwARB19QPLjdrv9fsc3Mplf4a9nLhnJuZhJjrnU1JzL4sloAJJUYQBECRYAogQLAFGCBYAowQJAlGABIEqwABAlWACIEiwARAkWAKIECwBRggWAKMECQJRgASBKsAAQdeiDvpZl8eEtIeu6LqmvZS45qbmYSdTHuq4/E1/IXKKac7GxANX5xMeamnMRLABECRYAogQLAFGCBYAowQJAlGABIEqwABAlWACIEiwARAkWgALWdb2t6xhvnBEsAEQJFgCiDr3duDettXJZYi8W5gnmAt99/fxv74+j98Qr/26SjQWAKMECQNTQVdh2FRzltAU8o0pFwt/uzeWV+VSZrY0FgCjBAkDU0FWY+gs+ValI+NtZczm7CrWxABAlWACIGroKAz45FVbTvbp+z3yOzvPsmdtYAIgSLABEDV2FeUCSGeypRdJViGrtsT2/c579HbXn3/OuMACGIVgAiBq6ClN/MZuzahH1V96z17Ri5W9jASBKsAAQNUwV9rUCnnUqhn0erebmsl/rJFbrGlapRUZ39FReei4V52xjASBKsAAQNUwV9rVqVlwLIaFVp7yz/vUg5GN7aq70daz4UOSWjQWAKMECQNQwVZgKjJmc9a6oKtVKL856WHHPSUDvCgNgGMNsLPf42xaz8TNf0zvfLn30v3PGVmNjASBKsAAQNXQVxjUcpDiX+osjzvh5sbEAECVYAIhShRH36Cy/6uZ1Zz2vUOW5iF68swbe87WrzMvGAkCUYAEgargqzLp+DR/oda6z6i+OadXAZ1VUVe4zGwsAUYIFgKiuqzArex1nvdWV96v+5txevPMDvaqzsQAQJVgAiOq6CqOORyu7GqUfrVmZWx3Vq2cbCwBRggWAqGGqMGv6+Y6s4OYD+7SqyFc+NfJsNhYAogQLAFFdV2EVV8CZVD+Zwn/r6TXsPUpcu9aMqtdiNhYAogQLAFFdV2FbX6thlVVwBuqvcbRqTffT845eu2fvp4qzs7EAECVYAIjqugqrsvbNyqmwMbmX8vb8rvr681dOfFWZnY0FgCjBAkBU11XYvSqm4gmJGez51MF7f2Yu13GvnOfINR2hYraxABAlWACI6roK27q3alrpr2cufTCTmnqdi40FgCjBAkCUYAEoYF3Xbk+B/ZtgASBKsAAQNcypMICe9XoC7B4bCwBRggWAqKNV2Mftdvv9jm9kMr/CX89cMpJzMZMcc6mpOZdllONtANSgCgMgSrAAECVYAIgSLABECRYAogQLAFGCBYAowQJAlGABIOr/QUr8nn+ITIAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 504x504 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# -1 indicates final epoch's samples (the last in the list)\n",
    "view_samples(-1, samples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below I'm showing the generated images as the network was training, every 10 epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAKeCAYAAABpgVZYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2dZ7gkR3WGv9l7V5sksQiFFRKSQIBIIkkgcs5gksmY7EAwOdjG2AY/tklGGMNDzmDQQ5TIweQoI5FEBkUklFZiFVZpw/iHeG/NnHt7u3umZqan+3v/3OfO9HRXfV3hnKpTVb1+vy9jjDEmJ6tmnQBjjDHtw52LMcaY7LhzMcYYkx13LsYYY7LjzsUYY0x23LkYY4zJzmKdi1etWtVfWFhY+n/nzp2SJMKZV69ePfT5jh07JEm9Xu/qhy1e/bjt27cP/b9t2zbuH5+34n34y3Pj/0Xh1aSLdHLdzp07tXPnTvX7/V6pCClt/cH0cm8gb2Va8Dm6ok3Ugt9xP/6P1NWCdAxeP4oWCwsLS88mT9xzt912G3omeeR63gflAC2iZgPPG7pf/J7nxvvE8gKxXAyW6z+VjdpaxHvz7Kh31XIRteH3Ta8j42gR319ZHbEWiaL2In7O7yahRa3ORbpahN13312S9Mc//lGStO+++0qStm7dKknasGGDJOnKK6+UJF1xxRVDCaKxQdCNGzdKSoJdeumlQxni+jVr1kiS1q1bJ0m65JJLhq7jeQiI8Hy/fv16SdLatWuHvu/3+9qyZUtdKbR69eqltPD7a13rWkN55lmkjUYCyDMvGW3JM5qSB65HC/KEZmjM87nvVVddNfRc3hH3gZ07d+qiiy6qlP9BFhcXl97jeeedJ0m69rWvLUm6+OKLh/KGBnxOOYgNLNejIXm8/PLLJaWCfo1rXGMoT5s3bx5KGxoCGvH+99xzT0lJi8EKeMEFF1QX4U+sXr16KS2Ui7333nso7ZQbygV/Y8fI36I6Eg07tOJvUR2hXPAuyurIYF7qsLi4uEyLffbZR5J02WWXDT2zqha8b8pJrCNRC7SmvBXVEfJKA77HHnsM3WfQSBm1vSCvdbXg2ZRR0kzZRQvuE8sF7UksF1xH+8B9eS6gIc8f7ByLtPCwmDHGmOzU8lwWFhZ0jWtcY8mCoDfE+okuG70mRNeM3hGLAUuBXpTP+ct98Ziw3rCM6JW5D5bvz3/+c0nJUqGXH+xxSXtVVq1apQ0bNizdK1p63A8N+B/LoEgLLAkshKhFTC9akLdoeWJxoNHJJ5889DzeIVr0er3aWiwuLmqfffZZ5sJTLoo81qhJtKKwjqLnEodTyTvfY5VFT3evvfaSJF3zmteUJP3gBz8Yug/X43n1er1lupexsLCgPffccymN5Jn78CzKZvREY/kgb2V1BK24Px5SHE1AC57P31/84heSkuWbo1wUaVHUXkQPNg5fRiseqx0t+DxqjRbRO4haoNGvfvWroedRh6lro2ox2F7wvkljbOMoB5T9Ii2oM+SJ+8Tywf14n5SLODoQtfjlL3859Fy0qNJ22nMxxhiTnV6dvcUWFhb6a9euXbIY4qRQtDgGe/o//X7F6xlPxko78MADJSVLJ86l0OufccYZkqT9999fknTOOedISr15tDywYPbbbz9J0plnninp6l568+bN2rZtW60JurVr1y5ZDAMaxeskpZ6e//nL9eQNbdGCtGJlYZ1HLwBrm/HoCy+8UNJy7w8tsGy4nt9v2LBBF154YS0tFhYW+uvWrVt6j3GuDOuMZ/OsqFWctGTOhjxTLnj/jKFzX6x6LM+DDz5YUioXaIzVxrvjPgcddJAk6bTTTpN09bs455xzdOWVV9bSYqU6wrNj2Y/j1dHbj/MMlIvrXOc6Q9fzeRw1+P3vfy9J2rRpkyTp3HPPHUoX5YDfUWeYR81RR9AiTqhHTUZtLw444IChPFPPo5dw+umnD2lx/vnnS1peR/gd919JiwsuuKB2HVm7du1SGS3SIpaLIi3inAvvLWpBHmLb+Yc//EFSmvOhTkYPOJYL5g75/fr16wvbC3suxhhjslM7WqzX6y1ZBjECKY4HY20DFinjeliWWFf3ute9JKVe8vjjj5eUetu73e1ukpb34meddZakFKmFpUo6sXyxTKKVt2XLltpjqMA9STM9P1owzhvH1Pkd1jNWM2knr/B///d/kpJ2t771rYfu9+tf/1pS8uawMM4++2xJ6Z2gSRyP5j4XX3zxyFrgAcQwS7TgWeQ5ziswznvYYYdJSh7IPe95z6HrKBdYmJSbOJeDFrFc8E54Z/z/m9/8RlLyuC644ILacy7kK0ZKkle04Ps4ps7zYh2hjJNX3uf3v/99SUnTO9/5zpKWR1jx+1guSF+MGkO7Qc+7brno9Xrq9XpLzyhqLygXUQvSRLmgjtBe3OMe9xi6LrYXd73rXSWlcoEWeCCUC7QgHXHuJkaVXXTRRSPXEe4Vo7FiJCVt52DI7+B1eK54EHe/+92Hrovl4g53uMNQHtCC3xe1nXEOOZbnXZULey7GGGOyU2vOhTFUeq04Xsf/WAqM717/+teXlHprLBGue+ITnyhJOuKIIyRJH/7whyUlC+Poo4+WJL3qVa+SJD3+8Y+XJD396U+XJN3vfve7OjN/SheWzQ9/+ENJaYw9WiaMV5577rnatm1b7cVya9asWXpmnOsgMgkrivHdQw45RFKyUPgdPOEJT5Ak3fKWt5QkfepTn5KULM9XvvKVkqTXve51kqRHP/rRkqTnP//5kpLligWDVXbiiSdKSpYH1npc37B582Zt3769lhYLCwv99evXL71PrCDKBf9jqZ566qmSpBvd6EZDaWEsnWiv5z73uZKkI488UpL09re/XVLSEg1e85rXSJIe/vCHS0rl4qEPfaikZBFjpX3ta1+TlCzp6NkSTbZ582ZdccUV2rFjR+15BsoFngJpjnUEjQ499FBJqY7E9U5/8Rd/IUm6xS1uIUn62Mc+JinVkde+9rVDWlBHnvnMZ0qS7nvf+w6lk/mHn/zkJ5JSHeFdxLUeo9aRQS0oB1EL5gdoL653vesNaUEasLapI5SLY445RlJ6v2jx6le/WpL0uMc9bkUtaPvQkPYCDWId4Z2cf/75Y7cXsVygDe+bcoEW1FvSgGfzmMc8RlJqLz75yU9KSp7n61//eknL2060YFQAj4ZycdJJJ0lKoxG0nVGL8847r7C9sOdijDEmO7U8l8XFxf7uu+++ZFliUdALY/3wPyu96ZWJZGCs9AY3uIEk6eMf/7gk6SY3uYmkZMHiydz4xjeWlCIoWLfy05/+VFKy6rnf7373O0laWl39ve99b+j3WKh4UNLVY41XXXVVLWt9w4YNS3nGkogrseO4MloQCYWnwXjyV7/6VUnS4YcfLil5FLe61a0kSTe72c0kJYuCuRasLiweIqt++9vfDn3O2g7eTVz52+v1tHnz5pG0wNLDI8QyjdFejOsShUP0zp/92Z8N/f/FL35RUoqYw+rDQ+F9syIfK5yxdyxS5il+/OMfD+X9O9/5ztB1WNBosbi4qLPOOqtWtFhRHSGvfE454T3GOsIYOpbrcccdJynVEa6nnNz0pjeVlMo461Z+9rOfSUpWPZ4z651iHUGbON/R6/VGipwbrCPci3LB5+QFbfifKFDaC0ZAjj322CEtKG+3ve1tJSWPOGrB+6e9YG6PuTa0YL4itheDc0LnnXderTpCuYi7VVAuqCN4zbGc0F4wF0u5+MxnPiMptZFoS3mgfPBeKQ/8pS5S7k455RRJqU6hBemJc0K70sKeizHGmOzUihbr9Xpas2bNkgWCNU4PjyXJ2CkWJ9YbvSBj4cytMO9ARAN/iRbCOmeMHMtmcM5ESlY5Fm60HrFMsEyxtBcXFws3bNuVFqtXr16yFBgT5Zl4IszBRC2woh70oAdJSh4LFs3tb397SSkqjLUXjInicaAd94/zTUVasKaAMV607fV6yzbhLGNhYUG77777ktWFlYTeWJKMLxMBhSX4jW98Q5L02Mc+VpL0nve8ZyitzDegCePBzJ1QDrHWsMrRCksXKxBPhfRiyZJvysXq1atra0Ed4T1yL/RmbgUtsFwpy0QFPvjBD5YkfeITn5C0vI4cddRRkpKXTp6xLMkbf3nej370o6H0FtUR3s3gmpG6WqxatUpr1qxZFjlJe4Englcd6wj1+WEPe5gk6aMf/eiQFne84x2HtMAD4S9zNrQX/I72Ag93MApMWr5HHXWEOr569era7YV0tYbkkXuhxXWve92hZ1Iu0II5U7x7PBY8CMrF7W53O0lpxCLuyIEWzAnjuTAKxDuOK/hJJ+VicLeTIi3suRhjjMlOrTmX1atX9691rWstjRsyJopVxvxA3COICAXGFbHS73//+0tKY6dY+1hx/MWS+OxnPytJeuQjHykpWfNf+cpXJCUvgedimcToDyzawd1xL7vsslpRQYuLi/2NGzcuGzeO1k/cV4n5AsZx8eYYY8fKRwvui8WA9fX5z39eUrLqyNM3v/lNSckCxfLEIo5rf7Di4aqrrqodIbV69er+3nvvvZR3xocpF4Pj9lLyup797GdLSvMBvN8HPOABkpI1hxaUNzwQNMHTIWIOa+yDH/zgUDqw1r773e9K0rIdj4liG9zO/uKLL9b27dtra0EdYf4o1hHmd7Caifrh/Xz961+XlOoIXhnzVliU1BHu+4UvfEFSKmfk6Vvf+pak5Ts3UC7i/mzULdi2bVvtOrJ69er+XnvtteQdYbVjBcf2Ai2I7iItlOn73Oc+ktL8Ah4wWjBXS3kj0hJty9oL5p1oJ3gXRF6N015ELfBMqDNoEXdYiO0F75F5ROZUKBdxh3D+fulLX5KURo3waLhf3KPwhBNOkJQ8Ht4F7QyaXXXVVbr88stX1MKeizHGmOzU8lx22223/j777LNkeWCVxRXPWOl8z3g+Y+p8T6+J5fmKV7xCkvSyl71MUuptP/3pT0uSXvziF0tKlmmM+mIsnl437obKDp/xUKGNGzfqlFNO0eWXX17bEsFCxRKJu9iSV6w2xjSxILA0sKbf/OY3S0prOF7wghcMaYBFi5XOGG08v4HIKcatSScWChYIDO6ecOaZZ9aKCtptt936mzZtWkoLq4ej/lg7fE9anvGMZ0hKHgxpZ64ED+cNb3iDpOT5vu1tb5OUtGKOjvLJcygPzHMBXie/iwcw7bPPPvrFL36hrVu31tJi0IujDqBF3O8Ob4xnsm4JSxarnTU+rHN6yUteIinVETxZygVzLJQLvDwiLaNXj2XKWD2MU0fQIrYXcYcPtIjtBVpQh4juo714+ctfLim1F//7v/8rKXkstBfMscQ1RERMUUfivClzN3FXjY0bN+rUU0+t3V4MjvrQHsQdM9CCuVz0RwvS/u1vf1uS9Na3vlVSWsdCnhkFQBPaEea34u7rRNTF6DDqCHN7QDu2ceNGnXbaaStqYc/FGGNMdmp5LmvXru0ffPDBS2PV9F5Y31gc9LqMcTIvQC/I+DHzDEQ8YKXRSz75yU+WlMZCsSCwkO9973tLShYvlkiMMiOPWOvx/I6tW7dq+/bttY4tXbNmTf/a1772sl1tsTjRJmrxkIc8RFKySJhjQQuiQxgXZmyUiCm0wFvDKkMLosWw0vHWuA/P5bp4psafxk9rabFu3br+9a9//SWrCAsPb45ywZwH1jPlgvfKGDoa8X6YR8C6YnUxFiflh4gXyk0sB1yPFnFHh+j9XXrppbXLBXWE8fy41xxlD21iHcGDuPnNby5Justd7sJ9JSXrnTrC7hYx8ok6SrminOAV4qGgDeWAOhI971HqyNq1a/sHHXRQoRY8g3lFtMCrx9NgTja2F4x88J6f8pSnSErWN3M5sb0gOpCRDuoInkrcXy3uKP6n+ZbaWhx44IFL3jplLe4pFtuLRzziEZKSJ8E6N9a70N4wosH7YxcDykvcKYS1Q7SdaEG5QNPYXkQP6/LLLy8sF/ZcjDHGZKfWOpdt27bp7LPPXhobhXjyH1Y1q8pveMMbSkpRPlgcjDsSp859sCy//OUvD/2OCAl6YywQrDz2GOL3WDyMT8bVyVg269atW3bOehnbt2/Xeeedt5Q2LH/GVLEo0IS9f1jnQJQP1jaWLGs5uA8WDFpwHZYLFitWFp7QRz7ykaHPWY2M54P2WC6D54gwDl+Vq666SqeddtrSHAdaYDGiBRbrne50J0lp11osVDxXLE/2gMIjwspmXQyeELscoBnjzKwJeOMb3ygplS/WAmDtEZWG1pSLPfbYY+meVYl1BO+IORisbjShXLDimjkYyizvm9Xn5AELNdYR9ooiCgxLlbrAnmRoSXlhzo90YMFSLkapI2hBHtAC7yy2F+zIEdsL6jv34f2hKRFxsY6gBe0NdYH2gnV2rEpHI55He8H3eHGSRtLi3HPPXUoboAWeaSwXREpSt0gb3hZaUGdJK94+nhCRdtR/ygWeELukUC5oLygXRKNR7tCi3+8Xthf2XIwxxmSn9gp9rDopWXr0uozz0vtipbMzL1Y53zP+iyXB96y8JoY7ntaIpcv4JatXibxgbBaLhbh4xtzj+eXxnIkq9Ho97bbbbkv3YEwSawyrih4eC4Kxb1YXY+GiBdYTWmBpYNXFiDz+Z1yYVczMBXEfPBzeCR5TjK8fVYvVq1cvvZ94LgoeDOUE65g0sG4JjZg7wzp84AMfKClFRvF7rD7Sjua8gze96U1D3zPOzHwWVhvlhHKBtc79xtGCOkKamH/Ag2FXCyxJvDq0QiPmKW9zm9sMpZm6M2hVS6lcUA5i1CCeL5YwliqRdXiZzOHE80eqELXAc4hakNe4wwerzuOuFrG9wOPl83hyJWWa90m0GRY3HnTUgnaFcpFDC6C9IO+0ZfyPR0nd4X3xnikXtJGUC7Tg83imFnMmjFiwkwd1ibYTjwUtuC62F7vSwp6LMcaY7NTyXHbs2KGLLrpoqZeLZ5fHM9DpXbFAGQOP6xDimDsRMvSifE+viYWCBRpXcjMGiwXEuhgskLgq9sorr6y9V9COHTu0ZcuWpbyhBc8Y9PCkFCEVT7eLce/AmDkaMK8Qd6tFC+YpGBsl4ooxVOYteDdYSKR/cI+xUbWgXKAr80pYZeQd6xiPlhXZXIeWWG2sW6D8sIKf90j5Y44Pi5OzT/BcGY+mPHD/uP8W0Y+XX3557f20ohaUA7Qgb2hMHaGM85dywbg/n+PJYtVjdWOdoz2eDqMA3IfygRbUEd4F5SrWkW3bto1ULlZqL9ACzyFqQZlkfzXKBTv3UneYe6NuUFeiFsxf0F7g+cY52NheoAXvgjqTo45QLoraC0YyyAtaoQ1tXjyRlBGReKovWjAyQluLprQ/UQvmtaMWeJ+70sKeizHGmOzUWufCiYNYHkSuxBP8GDtnHJBeGYsBy5J5BOZq6C1f9KIXSUpjrIxD03uy3oE5FnpXImziefE8j/RijZKunTt36qKLLqq1hxRaYHlw77hbMTH8UQvGVIkvx7omzVhPrLjFkonzVlh7QJQPng/pimfhYEFFLbZv365LLrmklhaLi4v9PfbYYylNcdUxGlA+yAOeLvNORMBgTZEmxupZhfy5z31OUtpRmuuYq8HCZO6H3Q1IH1Yg48iM/ZMevLidO3fqwgsv1LZt22qXC94/ll6sI1iKnKZIHpgTI218j5eFB/qc5zxHUqoLeDLkES15F4yxM1eD9c/fojqCJv1+X1u2bBmpjuBZxhM/2QuMqDAi4ngmuybj6bJDOGWZOsJuBbG9oBywZoO80l4QLUg5xStg5+hYRwa1GKW9WLduXWHbiRaUC9oL5o2IoCOPeOmUXdpOTm+lXUALyiPtDW0kozx4xGiEFjxvV+WiSAt7LsYYY7JTa85Furqnwhqi94qrNrHCuI7xWyxGxgWxQFmFisWK9c/vWLmNpYE1x7ghFipWG70sz2c9BF4anhLRH5deemntsfV+v68dO3YsPYN7oQXWL5ow9knasCCw1libwdoOxsbRgN+z4zCRLcwrMOaKFkC+ohcRtYCtW7fW1mLnzp268sorlzQgz3EnBMoL1lY894WYfdZiEEXG2g9WpTNfgMeDl4YnTGQc3h5WG+nBOosnXGIJ8+62bNmyVBbrMBj7jyaANoM7IkjJSuY944kQIcccGu8/rhlh/pJ5CLTAU8GjxdPGo0ILPCrKBZ7S4NlHdcuFdHXZ4BmxvUDn2F7Ec9tpL1i7wW4GtBcxMg4tYnuBFnh7tBfMtXEf2gugfKL5KHVEGm47uRfEXbLxHChHeDjUEU5pRYtYLsgLbSdtJR4P5YRoxjinEssF+Y1nYu1KC3suxhhjslNrzmXNmjX9/ffff2kMnBhq/tK7MreChUI0B70plihWFJEqzC/E817odbHq2QGW+zL+TC8erTXSxbgyXgGW8G677VZ4JkER7PiKtRtX2PJMrCt6eiwHensi3PgdlsHznvc8Scn6Iu/8zz5JWDBYdy984QslSfe73/0kJaue+QjShecUT2dcXFzUFVdcoZ07d9baW+yQQw5ZijSJWlAOiHDBOiLSCSspWl94eex+yxoe9lnjlEaizd797ndLSh4P5wg961nPkpS8PcaXsZjRiOgh5iv22GOP2vNPa9as6W/atGlpHUNckY/+zK3gGfH+Yh1hjJ55At5vPNeDdQms5WG1OpYuu+LiAfF76gDpYq6HdzFOHSnSIrYX6E45IVo0eutY18w7oQXtBeuhqCOsSsfjob1gl23OSol7lNEm0l7wLpifGLW92LRp01I9o46gBeWAOgLMT8ZywTxWnJuN7QN543+0wDPmd7QXzFtTl3lHjALg7aPF6tWrC89/sudijDEmO7WjxTZs2LBsNTCRB3gEjCtiQRLpQK/HeB8eDtYW1hlzMXwfLV6sNCxcxpexTOnNGb/EAiFePq7EPuecc7R9+/Za1jqRMIOrbge1KDq/BYsVrfDKiBZjDoYxbywRPCDi1Jm3wDJBc8aXsUDifBceSzwZkfHwP/7xj9q2bVstLRYXF/t77rnnUrnA2iHvjPuS5qc+9amS0opsImWwqrDWiHQhwo31LuzPxAr7uKcY7x0t8ETQAq2ICsJbZHyZfJx11lkjabH77rsvW7cQ6wiWKNYz0YPkjbKMZ4s1T1QQngnliXkGvD3mnbCI0YL7Ej1GHYqRc3EXi/POO2+kOrJhw4Zla3soH7G9YO6V9oI5NeoA0WOsZ8EjwnunnYjtBXkGtKAdob0A2gvmcKIW559//thaAN4aHgnlhHJBnqn/5IU6hcdLe8G6JsoN16EFXjrPZSSF+1K+gDoST5WtooU9F2OMMdmp7bnEvWqk4R0ypTQ/gNXE54yN45lgscZzXugl+f3TnvY0SWmuhfkLvufUNcYNYy/M/RgnjOsxLrnkktrnxqNFjJQgbVhn7AkUzwnHq8PaxmomrVgUWBxYNFEL8oIWrOngudE6435EXPE+B8/KvvLKK0fy4mI0UNQC7w1rmbQQm8+cCl4WVh4eCu+LOTrmYtj1GKuKcoUVh9VPucDi5d3h2Qzu2CBd7enUHVsvqiNoQV1gPoD3RrngxEHeL54MHgVaxHNBnvSkJ0lK0WVxvotyQd1ECyxevDfKBVoORnCNWkdiG4MW6M98ANfF9gKPNq4RYf8stOB+1BHaBeY3KHd4wGiBlx/rCPMOaDG4q0bdeckiLUgbaWcOjetIM14dXhojI3hXjGjwP/ejXLA7Ac8jL5QzdpDG40ULykWcZ49tpz0XY4wxU6H2rsiDOwHTo2NJYC0xRo6lQUQSY6dYpliyWE/0yqzdYIwcSxSrjd+zcpfxyve9732SksVLuuJJiFhncU+0ulosLi4u/Tbu1cXaCqxk/sdDwXLE0mC/LNYtYHUzF4P1j6bsKRb3nEJr1opgwURvkLwz9j44rhxj8MtYtWqV1q9fv/QsrG30597scxZP6fzkJz8pKUXrEP2HNkSFMRYfz6EnSgiNsEyx/ogig7iTNOWH56NxXKNSBXa/xZunXPBe0Z06QnlAM+bgsAyJ9sJzITKKuoBlSR2hvMSV98xvfPCDH5SUyjzvKK7xoI6gxah1ZPXq1cvaC7SgXKA7/8f2gjSytxxlmfaCukA7ghZ4/zyPnaGZy3n/+98/lLei3RQ4w2SwvahbR9Bi8AypwbRxb9bgsB4tzsHxXikXaEF7gQcTywXzm9R/5l6Yy+H8J6DtjTsIsC6mSnthz8UYY0x2as25rFq1qr+4uLhs3JBejc+xfojKIGIBD4JemkgXenF6S+ZOiK2m1yZ6iN4XS5i5FMYPuT+WLXHx8TRArAb+1jkTu0iLaPXisaAFURzA92gBWO3sDEs0D5YNew/hBRDVEU9rxJss0mLwXHDpaou53+/X0qLX6/UHo6PQBKsLq5j/2aUWa5p5AbRDCzRkvJex9M985jNDWrBeBm+B/ZCIPsSTwSKmfFBu8BIoF1iwl156ae2z0letWtVfvXr1snIRVzrHOkKEE2mhTrEGhPfIe2O+gPVLjJUTbcjzmU8ir1j7eBHxnHnKHVrgNWzdurV2uajbXhD1GdsL6ggaMWdD2mgveO/sVsFaDjTHQ8IrQ4uy9iJ6n5dddtnIWkQo81j/sb1Ai1hH8OLRAk+F/fbYW4zRHdoLIGoULeJcHfelXOAhrVQupJXbTnsuxhhjslNrzoWxdawrrDGsXiwQLA6sKixVemFW3GIxcB9W1BPDjwURI6LiSYL0osS/EyWChUTvyzgh45ZYIvvtt9+yPbbK6PV6WrNmzTJvjHszZorFwUpnxoHRgLU8WARYLmjBOhbmbrDOsSQYR2ZMFWuQtSJoRfrYRwnLFa+Cd7jvvvsuje9XZWFhQde4xjWWLH7yRhqZW+F7rGneFx4FmmGd8Ze0vve975WUxoHJE9ZTnI9gPonn4PFQ3igXPIf78PeAAw5YtgaijFWrVmndunXL6gj3JK+UC7wv5g2Yb2RegDyiDZ4p8xFxhwisdN47Y/BxV4RYR6hr1JG459kodYT2grIV2wvyRBlkHiqeH8+8AGnjd+yjR17JQ9wBAC+e+SjeBVow4kE5oFzQ7sSdgkdtL9auXbuUV3SPbWcclZkx7u4AACAASURBVCEiDm+eukVa0RTt0AKvPe7zRx0hMpc8EVFJZB7tUFG54He7ai/suRhjjMlO7TmXtWvXLllf8RRDxsixojh/gc+JyqFXxEqjd8WzoVfGOmPe4e1vf7ukFOVBxARWO2djM16I5crzuS89Lb25pNrrGdACC4QePa5nwYqKeY37qzHGSuQK48vcn1h89lsjAuqYY46RlKx1vITjjjtu6HmsDWFsF4uYMXasvH6/P9J6hvXr1y/zULD8YtRVPFUTK53rGCtnzBurntj+D33oQ5KkZzzjGZKkN7zhDZJStCCrltH06KOPlpSsw7jeiXeEFnij7G48SrlAi1hHgGfjyfLeo5VOuaFOYc3zPeuYmHd45zvfKSlFhTEHQ91CO8pFXKlP3aSOoEWv1xtZC3SnjsQdgIvaizjCgVZ8Tx0hzVjlRKG+4x3vkJTaC7Tgd3zO+8faj/OludqLNWvWLNMilot4wiztCVrQHuCRcj1zc7FcsJ7qXe96l6RULpiDYWeHD3/4w5JS20l7AbtqO723mDHGmKlRy3PhxEHGBeOe/1isWAb0cuwdhnXEalDG/4l0YtyPsXp+j2VBtBGx+0RC8DkrarECSV/cLwmwytauXastW7bUPnFwcCU2Y6lYO4ylYm1jqWC1k0a0IY3MR+CVkQfGPrFksGDQgvktrDzi0eP5DrwjLJ6Y/nXr1tU+cXBxcbG/cePGpfF+xunxSBifZsybdQOsTwF2FSCChYgW5qWwvhmP5n3yHOYviC6M54KjHXlFc7TAokbzNWvW6IILLqhVLsrqCPOD5AXvjHkhYNU62lFHKC+UB/KAxYllTOQUv2OeAm3jeTLcBwuVdmHwDPa6p3KyzxrlIp6vRFnEG4ta4EGQ59heUJcob/EMGvKMFrQXeJV4sDyHdxX3IIRx2gvKBSMEsb3Aw8WTiHNkgCdCuSBvaBZ3JcErZLSJ9oJIXMplbDtjHaFdoVxELXwSpTHGmKlQO1ps3bp1Sz0944VYx1gkjF3zOZYkEUzEmzP+i8eChfrxj39cUorVZk4lnqZGr8r4IOPVWGekL65GxtLFOlgpFr+KFoOWTdztFIskrgZnPQJj6UR5HHvssZLSWChWN/uw4dEQNUJEHXMtWLKMG2OpxCizeNId7wxtFhYWRtJi/fr1S54BWsSVzozvY1kSmcI8Eh4N80nMIzzqUY+SlMaNWcfAuDT/U46w6jg7g3JFdBEeJ+UTi5l1N1jAq1atGkmLdevWLd076h6tdu5PHWFeAC+MOTXmoSi7RAlSR8gjc3NEGaEFdYS5PupIPBmTdFH+xqkjvV5vqL2IdSSWPWC+4KijjpKU2gvmBVjvRp7Z4QEt2EmYSEvqWjxvHi3x8uNcEOliZGRcLdasWbPkScRyET1bYOcN5g8ZoWBFPW0qaaR9ofygEfPXcTcD6iTvO2oR2068Pv5fae0O2HMxxhiTndonUW7atGnJAqAXxkJlvJA1HXHfLTwLxmCZN+BzxvGYr2BVOmPieET0nvS63A+LNPa68UyTeHb3XnvtpXPOOUdXXnll7ZMoYw+Onoz3Yj1joeDtxLh2LBfmq+K+aFgsg2dXDz4Hy5Tnk0e0iGeux+g2rt999921efPmWuPJa9as6R944IGF5QLrifHheCIhXlY8K4I5Gt5f3COKcWC0QosY9ROj1tCE+8botsEx+zPOOENXXHFF7RMHKavci+ga6ghj3KSJvONZxHJBtBD34f0V1REsWbSgjhRZzpOqI5s2bVoa/4+7mZOnOA9EuYha0F6gIRowOsBcTNzNILYX1BG0II/Re0OLWEeuec1r6pxzztFVV11VS4t99913KU1xx3C0iHOl5IX3GSPZ+F3UAi8u7n6Ch0K5QFu0KGo7Y3vBdRs3btR55523ohb2XIwxxmRnpL3FiCiJ+xDRC9O7xj2AsCCwQIjO4D5YuFw/eJaElHpNns/9GUMdXKsxCHM2zEvEMdUdO3Zo69attePWFxcXlyxG0ogWg+sDpGSB8EwsUqxvLAiscywSrKcY7YUW8VRNrESeD1iuWH1YJGiF1jt37hwphn9xcXHZuoC4ChlrKO5aTV7JC2knL5zSyP+UG8aNsSjxXHhOXMMB/H/YYYdJSl4f9xk8yfTSSy+tFTlXVkfIG2mkXPBe8eIoH7wX3heeLZ4r2sa94ihXWKTUETyfeHYKe1VxHemjPG7fvn3kOkJa4t5+UQvSSrlAC7TkerQi4o40xvYELfg975eos8G1GoNaMGcT2ws03759+0hrflbSgjyTlui9kVfKNnUMLbgPWsRywe9juYh7h8X2Ai1i5C9w/11pYc/FGGNMdkY6iTLGhWMNY/FhcWBRYJVHz4Z5B3pbeuvBc7sH4Xvux+/ojbFIiZCJY7Yxhntw7PfUU0/V5ZdfXnv1MXnhnjwzni2BBRrPlydNWO3cJ86ZkKc4v4TFw++4D+8VK46xViwU7sv/pGvffffVmWeeWWtsnRX6QFqwAIlYY9yYckHayVOMOoyr28kz9yOPaEweyDOfA+8ET4nn8XzSzf+HHnqofvnLX2rr1q211z/hWWLhUUfwDLBEi+pIXM8U6wjlDis8jhbEOoL1DmV1JJaLAw44QKecckqtOrKwsNBfu3btsnVnROUVaRHrCHmLnnCcM0ELiCMq/I46wvvHKid6LL6DGPF54IEH1m4visoF8820dbG9iHUELYu0QCveb4xa5D78jrYTohY8n/Yi1pH99ttPp59++orzkvZcjDHGZKeW59Lr9c6XdPrkkjNTDu73+/tUvdhaJKxFwlokrEWii1rU6lyMMcaYKnhYzBhjTHbcuRhjjMmOOxdjjDHZcedijDEmO+5cjDHGZMedizHGmOy4czHGGJMddy7GGGOy487FGGNMdmodc9zr9Vq9nL/f71feiM5aJKxFwlokrEWii1rU6lzMMIPnhLCNDp+V/V/2u6JnVf2+7PqV8hDTZEanqv65fl90/aifR1wmTF3cuYxBv98v7SxW+s1K3xd1CmyhzRbXu0rLSr+PFD236PsmMW6DPU2qdupVG/0Ih0bFQ9+K7hP/r1pOKX9V0mRmT9X2Yhp4zsUYY0x26m653wjThQNzOOAoF3XHUKsMKY1qoXKoEHmMB1tFC4XrOdCo6P4cGhSPNV0hH3M3nhw1yMUo5aLMkyzzION1fB/LfpHnweFRHNAVy0+khoc8d+WCQ+ri4YPjMg9axPJTtZ2oy0pa2HMxxhiTnUZ5LlhZWF3TZhwLtYwiy5DPsSDiMbNV2bBhg6Tlx5DG51W1VGZhlRWl7e1vf7sk6a//+q+z3K8uk9Qieh78T7mIHsqDH/xgSdKxxx674u9iXilP/I1HCJf9PgatNMlaL5s3mjTT1KKoXMALX/hCSdLrXve6Wvcr82irYs/FGGPMVJhqtFjRmHi0QOiVsbaqWihvetObJEl/+7d/u+L3cRx6XAatvFGjvzZu3ChJ2rJliyRp27Ztu3xm0XzTta51LUnSRz7yEUnShRdeKEl60IMeJElat26dJOnyyy9flvZZE8sFeUQL/v/yl78sSbr3ve8tKWnI9xAjoh7+8IdLkj7xiU9IKp/HGJdBj5Z7R8uwLGqL69esWSMpRYXx/xFHHCFJOvHEEyUlj5U5taLnPPOZz5QkvfnNb5aU6gTlItaRWZaTupGVRf8XeWVl80yxHcpJXa+atHL96tWrJaX3zvcf//jHJUl//ud/Lqk8D+T97/7u7yRJr371q4fuVzSnWynNla80xhhjKjLTOZdoLcXeHOvpwAMPlCRdcMEFkqRnP/vZkqQ3vvGNlZ5D747VV8Qk5lzIE5YG12MRRGuZ63fffXdJ0qWXXipJetazniVJeuADHygpjb1j3RMRg8eChcFzi6JDitI/jfHkaI3x99Of/rQk6T73uY+kVD7Qijxf73rXkySdddZZQ9cxX1U2jvyWt7xFkvSMZzxjl9dNQgvyjkVI2eR9o8Xd7353SdJHP/pRSdJ+++0nSbrkkkskSV/60pckSc973vMkSaeddpok6WlPe5ok6X3ve5+k5RZntEy/9a1vSZLucpe7DF0Xy8cs51xGXQhad26m6vWT0CI+M3rVpO3oo4+WlOYh99xzT0mpraNO4MHg4dJObN68WVL53Nu73vUuSak8FWnuORdjjDFTYaKey2Me8xhJ0jHHHCNJevKTnyxJeu973zt0XdF8RFFEw6jRIWVrPHKuc4nRX1iO//Ef/yFJeulLXyopeRasuMai2LRpkyTpjDPOkJTmTIjVf+pTnypJOu6444aef8opp0iSjjzySEnJmv/DH/4gKUWVbd26dSidk7RQo054HCeffLKk5IkyZxY912te85qSknVN3vbYYw9JyVtDy7J0xLzyu6L5rhxaxGgf0sCc22WXXSYpeTL8D49//OMlSf/zP/8jKZUn7ve9731PknSLW9xCkrR+/fqh35e99yIPd4W5ool5LqNumRTXfhVdV5cKuxhk0yKWC+r/2WefLUm6wx3uICm9Z67j+xvd6EaSpHPOOUdSai9ucpObSJJ+9KMfSSofvSlqg4ui1MCeizHGmKkwUc+l6vg+kTBYcYDleuqpp0qSDjvsMEnSxRdfLClZwFiucMghh0hK489VGcUSqbqXFxYp1nGMYHrEIx4hKXl5T3jCEyRJH/7whyVJv/71ryVJRx11lKSkWV1vrmwdBXmZpIUaPUj+Rq2Yk4tWOFbYDW5wA0nLo8we9ahHSZK+8IUvDP3u6U9/uqQ011KVUbQoi0TieyxJ8vqVr3xFknTnO99ZUvIo8GTwQIC8M1eCZ4Ple+1rX1uS9Mc//nHod0URcrvaAWDa61yK6hTeF/OSZdc/7nGPkyR94AMfkJTKV/xdXQ9nEu0FxDpC+aDc8D2eyv777z/0e+ZcaDduetObSkp1ibaUEQ24znWuI0n6/e9/H9O/y3TbczHGGDMVJrrOpchqw/LAgsBjoVdkzQdW241vfGNJ0s1udjNJ0kknnSRJuv3tby9J+sxnPiNpeYTUpFm1alXpHlJYGngs//Zv/yZJ+qd/+idJ0m9/+1tJyQon7X/1V381dJ/rX//6kpI2WJjMO0BcVxHH9vk9luk0dlGN2jCPROQTz0YLNDj00EOHfvf1r3996Pe3utWtJEn/+Z//KUl67nOfK0l66EMfKimVC+axeN6k2dXeYjFq8G1ve5uk5KniYXzjG9+QlKICiQLidwcffPDQ/b/73e9Kku51r3tJSlri/TMX99WvflWS9Jd/+Zcrpr1oPc40KHoWVvfPf/5zScljidfznvfZZx9JKfLuQx/60C6fM80V/kV5jHMuvK8DDjhAUmo/mHP57Gc/K0l6zWteM/S7t771rZKSh3uPe9xDkvSQhzxEkvSxj31MknTb295WkvTBD35QUhoNwGOumu5dYc/FGGNMdrJ6LkSq/PSnPx36HMvgc5/7nCTpAQ94gKTUG2JRYmmwipj5hmgFlu27NC3+NAa94neMeX//+9+XlLwu5of4HR5LXEnLflrAOgcsS8bgn//850uS3vGOd0iS7nrXu0pKFupTnvIUSdJ73vOeoftN82wU5sYYx8WyJA1ode6550pKebzhDW8oSfrBD34gSTr++OMlSf/wD/8wlHbKA58zDo2WL3/5y4fSM428F92b9UqMiRMFRh7+67/+S1LywmLZZ90Cv2fsHc1Y94AHc+aZZ0pK1n9ZlOA0wfs6/fTThz6PafvFL34x9P0rXvGKFe9Huaq7sr8JEAFJBCV1gnKBVswjR88XTb75zW9KSuWMckE5oa3FU2Hk4573vKek0XdxXwl7LsYYY7IzlRX6cc8wVpOy4h4LkwgoVpHmPq+l7FyHnNEfMeZ+1DkN5h34PePOwDqW+DzAIjrhhBMkVT/PYRpRQbz3X/7yl5JSrD7jzeyXRiw/1jdpr7AOYej/uG8WmjE+zar3Fe4zshZxTqtoVwLyQHTP7W53O0npfTHnQvQXa8ZYL8VaIOZs2GMOTycS6wJzQaQjRvANpHfi5YL9sYioxNrGs43ENTpFVD3LCKa5zqUI3j/eO/PM7AF20EEHSUojJETEUQ5ilBltb9EJplEj6iifc7+Io8WMMcZMhYl6LkQmsfo88s///M+SpH/913/l/ru8HxYKvSq9OtY8lu+ojOO5DHwuKfX0WJxEvGGlVx3//c1vfiMpeTADaR36PZqwMjem50lPepIk6f3vf/+Kzxu8X+71DDGPpIH1B/FMm0c/+tGS0pxb0Y6ueByMGxNFhofywx/+UFKKKiuirA7kXKFPXcByxCJlvpIoMaJ8WAfDbhdYpsD8JJYq6x041+P888+XlOa9mNcEyiVrySivs1ihT0RUXIeyi/tXuq7qyv+65NQiri8iios1W9QR/rIeBU83rvmJnsqd7nQnSWlHD8oJ++rFeaxY58pGkey5GGOMmQoTXedCrwnEaLNPDhZE7CUZa2dOhvFExqHLznHAw4mnMeayWAYpsnoZ1ydvMeIlRsrg6bDS+vOf/7ykZMXxHMaV+RyPhb3GIuyqG/dzi0wy1j9qxBkzRL6w9of/iXQj+gcrDOuddQuPfOQjJaW8oWF8z1hdROaxKpl3hBdR5GHXZaXdsuMaLtL2ta99TVLaVeL1r3+9pBTtxe/22msvScliRUO8vFve8paS0k4PsU7gseAZsa4GzxiN8ICYw5lmZFX0WPDC49kyZWU17ig+6rkvk2gvioieAZ4GUYHMsbz73e+WlOY+aCvR6m/+5m8kpfUunIHEzh5FZ2RxP0ZIiGpFe9LBupsq2HMxxhiTnaxzLvSOrEeIez4V7X5cFOlEr8q4czxBknUQWKJE+7DOht8znj2NsfVI2Y67A8+WlLTgpMEvfvGLkpLlCkV7Te0ivZKqn4yYUws8AvaEe+c73ykprcHB+4pRW3FPMTj88MMlJSuqaA8xViOz7oX/iUbib1kk3zhaRGsYC5M8//d//7ekNP/EHAhpptxQjphfQiM8X842wvsjr9HqjudyMJoQPeui/QBzlouq8454rLyvOqchDoK3x3lBMR3cN57mWUROLSj7tHHsnv6iF71IkvSTn/xEUmoXqP+Up6ghO4CwawHr5vCQWEfD6BBzc5xQynzlscceK6lcc8+5GGOMmQoTjRaL+1kxbsh5zXG9QtwTjBhuooBiWjldjfUPMVIKqp6FnTMqKFqMWMd3u9vdJCULk12Rb33rW0tKeWIXA2L773vf+0qSPvWpT0lKY62suGaMPO4QXQTvhvmGQSa9+y3zC1jd5JG1F4yVowW7DgB5x3rHiuM98z/vO8798W6Yb2AupohJRs7F3ZGJdMOj4L2zboWxeDwazrjB8yHqi90QKFeMqcc1PpRLooYYq18pvZMuF7QTlE3qM2lHszve8Y6S0n5qwFou1nbF9uJhD3uYJOmTn/wk6VsxHVXnWiapBR4qdYF5RTxPRoeYI2OODtCASFrqChrjvdFusB9fLJ88l3VVddbF2XMxxhiTnYlGi9FL0jsyhl517x+iyrA8+RxrDQum7ES6svmOnMQ8xPF8dvYFLI4418KY/GMf+1hJabw5rsiP81pFsM6BFdl4PEXpnyR4bYyBRwuVaB88kTjvQDQZ+yoRZfSP//iPktL6KcpfjIAij3xedPpeTi2KIpJ4Jh4F7535RLw75jNj+cITxtJFi29/+9uS0vomds9l/zXWwTC/wDwW5Yt3EU9SnSRxd/RIWXtB3Sk642hXa7tmTSwXeBrMtVIO8GTiKE1s+xj1IeKSckGdYZdk3jd7jbEmjPQQnRh39qiym7o9F2OMMdmZ6JwLvRq97nWve92hz+kd2cGTiIYiKymmNc6lMG5dtHdQ2dxLzjHUuOp81H3SiHzC4oj/xwirF7zgBZKkN7zhDZKW7wlUtr/SJFboR9g7jB1ciRpjroTygQdCHtnJAZgrYe6E+SPmnShvWG9EzkTwhFgVH6mrRTy9cSUoHzybXWkpJ6Qdj4N5SuZU4v2x+sk7Xht5p04xehDXjgDvhPVW0zyJsu5OvMwDvO9976t0vxXSVyk9s9hbjPVIL37xiyWlnRzY4QMv/iUveYmkVAc46wriXCxRiG9605skpTVmlB8iOqN2nHDKLt5V9pyz52KMMSY7E/Fcila6AtZZ3M00zpVwn7J5hbi/FteVzUNEckaLAZFNcWX2zW9+c0kpIgrrHOud65hvwOOIngjRHFhvxKdjxb/yla+UlMbai9I7yb3FIK6cjqdh/uxnP5OUosOYW7vJTW4ydB+uxyPGOov7K5G3OLdSdyX+JKLF4plEWIyUA3ZWYHcK1iHg1WFpUsaxSFmn8C//8i+kfeh5zPERFcYJp2Wr2GEaOwGXzbmM+n3ZdXXnXiZRLnj/1BH2eqPeM6/4lre8RVKaW2NfxbgHGSvr8VTxiGO5YC6YnR4oX1Wx52KMMWYqTCRaLFoE7MDJ+PJ3vvMdSdLvfvc7SeUeCX9Zic+ZFvEs7RjRMGl6vV6pl1Y0zo/Hwu+J3oqRTWhDZBXnwpNXznd41ateJSl5LIyt47FgoTC2z55Dg3lZKf3jULS/FZAWdklmxf2Pf/xjSWn/NfKKF8f+WUS6sE4Gqw8rjnIQI6CmQdFu2XgecQ4Obyp686xToTzE6BzKx1e+8hVJy6OI+J65ODxdVmhD1Hil/EwjklCqHk0adxIuo6p3VnT9JIhRV7GO7L333pLSyaTAjgyccAuklehQPBfWTT372c+WlMoTK/Spe7Q/MI4G9lyMMcZkJ6vnEj2HOK7H2Hn0SOL4M58To8/96F1ZsR97+bhj8KTp9/uF1hAWKNY0HkXRnl7APkq//e1vJaWIqLh/FusaDjvsMEnJC2QuBwsU6w7LGAs3UiXCqS5Fu1QzjoxHQUw+acD6ZtU5Z1vwPePCRJ2xWzbj0eyTRd7RAs2mSfRoyTNlNe6jFqP4zjzzTEnL15vEdVGsXyKqjJ0fsER5/6997WslpbmWonUss1j7gUaswWFfLYjaxNNUma+E+HnROpeivQ2n4alRJ2K5wMvmc6K6yBOjNnjplHm+v8997iMpzcU95znPkSR96UtfkpTm8KibaMQcMdhzMcYY0yiyRosVeQ5VVnMOwu/xZMp+F3vXuCdZVSssx+63cWyd/4kTZ96JnYFZYc1cChYJK3OJT+dzVrezjgVLhL2G+B2eUvQGo7W20vvPHS3G2TTsl0YasUSLzq6P4AVi1cWx9qg5Vjke7m1ucxtJKaIOq7DMOssZRch8IWPheKZ45XEHh7j+gDxicbInFHuPAeUJzVkDhnb8noi7OL8Vz0KCSe6KPCpFIya7SNfQdXF1+zTai8jLXvYySdK///u/S0pzYnj5dfcMxFuPozvM7ca97LiOuonHg8db1oY7WswYY8xUyOq5cN5ztKIi7NTJbsakgc+JgCjaI4jxx5e+9KWS0twMViGrW7HmqzLOSuyBewz9jyXA2DfzCYxxYrWzfxpjnkSZYWEeffTRktKuuKxqJ7IKLTgHAtgzCm2hbB+tnFYZuyDjdRXBrgOcpkeamCdijQZWFavPyQNWPxEwjEMzN8PZOPy+KjlX6HO2ECupi94D0YF4fViieDhnn322pOVrwPAG455RzFOxCwHaMFdTFO0YyVkuqu5WvkIa4nNW/LzseXiD1NG6u2jk1CKeTFsE7QDRf+T5iU98oiTpAx/4gKTkGe+33348X1IqB6yH4nfM1bIDAHO445QLey7GGGOyM9G9xapSNJ4Xx2TjatW4JgCw1lkPwWrlMkaxRKruTxThLAp2fo4r9OPvHv7wh0tKUUBRqxjxwtko3Jc9qeJcS5yD4b7TWIkNcVVxXJ/C6mLeN9Y5Y+1oRrnAWsMq5x3h6QDflzFOuShbsxEhaixGPhV9Hu+Pp4KHFE+BRTM8XPZvQ3Os90nOP9Ul5p0yHXf2jloQRcgaD9oDdhgH5jMZRZjFnEsZ0bvjPcVdL2L0X/TS0JJ1LszNUrcYZcD7Z8TFnosxxphGMNWTKCFGZxSdwwJxhTcWCGs/6p5tUsQoY+tFq33j6mFW2jKXwvfsIYUnc9vb3lbS8qgh7sP8A3Hskbjr8a6iwgY/r7LLaRF1y0W0tgDrivknrCbmoSgnhx56qKQ0LszaIGL32X2A+S3OASdyqi45ogiLovRYSX3yyScP3Qeviqih6K0B84ycPMmODdyPiEm8NtbBvPzlLx9KR1UmWS6qjmBU2K24LF2V7lPGJLXA+2b3CSDaD++N9865K2hHe0M5Ynftd7zjHZKkX/3qV5LS2i/msePaoqrYczHGGDMVZjrnUrainrSxWp3T+QArHUu3aB+vquSYc6k7xh4p2h+N9QxEwsU9gO5///tLStFFZfso7cqintSuyGXgoRCxwrxRnHshzVhfN77xjSUt3/UYK45x5VHJ6dFC1bVfvE/eb4RygjePlwZYtESXFUVEVd1faxblgrx99KMfHUoTkW+sHSONeHms5YjkWl8zCS3KIt+YG2Fkg9N4V5gzlSSdcMIJktIaL+7PWkC8/bgbew6P1p6LMcaY7EzFc2F9A5EIVaE3JcIhgkXzsY99bJRkLSNHVFBViixXxlTj7rYRoj2Yz6q602sZkzzPJcI53scdd1yt33GGBWt+4vxW0Xj1qOSMFit6L9QN1jnE64h4fMUrXjH0Od48JxHiqTAWX1ROKuRj6P9pei6jehZV52Kq3neUVelFjKoF61J+8pOf8MxKaSQakPVv0ftnlICosHGx52KMMWYqNGKdS9VxxqKzJnJR1xJZXFws3E02prnM4uB6VgszFsrKWU5p5Hl4Nqz9KYsGK1rfUsQsxtZjOYgaxu+xxso0GJdx5lwGPh/6C9HijDuF8z0ruBkFiOsUgHlINBnIw4rpWGFnhqJ8je3R1t1jcFRyzalUeM7U6kjR3oVxzU+csy3bKy4X9lyMMcZMhUZ4Lk0hpyWSKwa/8Ah1kwAAIABJREFU6PMiC7To+6oWyzTnXOaFaUQFVZ0nKCs/RR5q3Tmgla53uRjGWiTsuRhjjJkKdY9s3Cxp19t2zi8H17x+l1qMOv5fNncS/y+7foR4dSmzFnPORLSoOtdR9XvYRWTTWM9zuViGtUisqEWtYTFjjDGmCh4WM8YYkx13LsYYY7LjzsUYY0x23LkYY4zJjjsXY4wx2XHnYowxJjvuXIwxxmTHnYsxxpjsuHMxxhiTnVrbv3Rx87UirEXCWiSsRcJaJLqohT0XY4wx2XHnYowxJjvuXIwxxmTHnYsxxpjsuHMxxhiTHXcuxhhjsuPOxRhjTHbcuRhjjMmOOxdjjDHZcedijDEmO+5cjDHGZMedizHGmOw0qnPp9/vq9/uF/3cJa5GwFglrkbAWiSZq0ajOxRhjTDuoteX+pOn1eiv+H3vgeF0bsRYJa5GwFglrkWiiFvZcjDHGZMedizHGmOy4czHGGJOdRs25FNGFMdOqWIuEtUhYi4S1SMxSC3suxhhjstPozqWJsduzwlokrEXCWiSsRaIJWjS6czHGGDOfTHTOhZ6y7rhf0e/meSzVWiSsRcJaJKxFog1a2HMxxhiTnYl6LuP2umX/zxPWImEtEtYiYS0SbdDCnosxxpjsNGqdS1Gv2sWID2uRsBYJa5GwFokmamHPxRhjTHYaFS1WtZedx7FUa5GwFglrkbAWiTZoYc/FGGNMdhoVLVZ0BsG4920C1iJhLRK5tOhihJS1WH59k+qIPRdjjDHZaVS0WKQNFkgurEXCWiSsRcJaJJqghT0XY4wx2WlUtFgRbbBArEXCWiSsRcJaJNqghT0XY4wx2WlUtFgT9sOZFNYiYS0S1iJhLRJt0MKeizHGmOw0Klqs6AyCJvTCuSnLU5e0KMNaJKxFoktazGN7Yc/FGGNMdhrluRTRJgsERs1TG7UYFWuRsBaJNmoxj+2FPRdjjDHZmYvOpd/vd/KMhpWwFglrkbAWCWuRmKUWc9G5GGOMmS9qdS5HHHHEVHtBet1er9e4cVRrkbAWCWuRsBaJLmphz8UYY0x2enV6016vN5Gutylx6f1+v3ICrEXCWiSsRcJaJLqohT0XY4wx2WnEOpdZ97pNwlokrEXCWiSsRaLJWthzMcYYkx13LsYYY7LjzsUYY0x23LkYY4zJjjsXY4wx2akbLbZZ0umTSEgDOLjm9dYiYS0S1iJhLRKd06LWIkpjjDGmCh4WM8YYkx13LsYYY7LjzsUYY0x23LkYY4zJjjsXY4wx2XHnYowxJjvuXIwxxmTHnYsxxpjsuHMxxhiTnVrbv+y99979Qw45pPL1J554oiTpiCOOqJWoWXDaaadp8+bNlU/esRYJa5GwFglrkeiiFrU6l0MOOUQnnHBC6XWcjjbu1jLxlLV4v1zPkaQjjzyy1vXWImEtEtYiYS0SXdTCw2LGGGOyU3dX5CGKer+i3rDovOey3rPo+yZtumktEtYiYS0S1iLRBS3suRhjjMnOWJ5L3d6v6vXjjgfmHE+sirVIWItELi34nDx0WYuItUg0SQt7LsYYY7JTy3M58cQT1ev1Jt7Dj3v/aVgg1iJhLRKT0qIs+qcu86xFxFokmqSFPRdjjDHZqdW5HHHEEY2IuOj1eoXRE6Pej0VLVbEWCWuRsBYJa5Hoohb2XIwxxmRnpGixGFEwagx21fvE+1VdbVo18qHf79decVv0DGthLVZ6hrWwFis9o81a2HMxxhiTnZE8l6qrSqGoF9yxY8eK9ynrNatGzozaG9chlxbbt29f8T6T0mIS5NKi6P8ulgtrUX6fNmtR975N0sKeizHGmOxMZG+xSNH3q1atWvE+ReOIZb1+WXqq3ncUxtViYWFhxftMSot4XVn66jCuFkX36WK5KLqPteiGFkUUeSpN0sKeizHGmOyM5blA3YiDss8h9pZVf1fGJOcfrEVi1lrUpc1auFyMTpu1mGS5sOdijDEmO1nWuRRR1kuOalmU9baTiPIY91nWImEtEtYiYS0SbdDCnosxxpjsZFnnUkbZeF/V8cWq/4+brjpYi4S1SFiLhLVIdEkLey7GGGOykyVarIy6EQpFvWJRL1nWi9a93yQZVwt2NbAWxfcpu37U+00Sa1H8TGuRmCct7LkYY4zJzlQ8FyjqdYvGAatS9vtZWBxljKoFK/mL6JIWZViL6r+3Fsu/bxLzqIU9F2OMMdnJsrcYVO0FyyIX4v1zXTdJ5kWLaWhjLZanAaxFwlokmqrFONhzMcYYk52xPJei3q1qL1v2f9Xxv7q97CQsk2lrceWVV9ZKR9H3bdCibjqKvrcWCWuRaIMWs2g77bkYY4zJzlSjxUbtJYt+X2Rh5E7HJLAWo6fBWiSsRcJaJJqghT0XY4wx2anluZx44onq9Xojj22WjQs+8YlPXPH3/K5qZENR+nKOnU5ai7JoEWvRTS2Kft9FLVwuyn8/Sy3suRhjjMlOr05PdOSRR/ZPOOGEsXv0st/njsmu8bxqA4+yFoNYi4S1SFiLRBe1sOdijDEmO1nOcxm1N67be5alo4gq45NHHnlkpXuV3XvSWlRNx6jXWYvh761F+t5apO/nRYtZtp32XIwxxmQnyzqXur1uHPeDuhEMTWTSWoz6nFlgLRLWImEtEtNqO88888xRkzgy9lyMMcZkZ6or9CFXjPUk9vyZNrPS4vDDD5cknXTSSdUTO2FcLhLWImEtEvOkhT0XY4wx2ZmJ55IrxnvUGO0mWTCz0gKPxVqUfz4LrEXCWiTmSQt7LsYYY7KT1XPJtXq06POimO2q45DTtECsRfVnWYvy761F/c+tRfr8MY95TK375tDCnosxxpjsZPVcqqzmHOfzcePZi3rjSWAtip9V9fuq1lMXtSj6vItauFyUf37MMceM9Nyi66poYc/FGGNMdmYSLZabsrHWKr37qHsFNY02a1F37LvNWtSlzVq4XIzOJLWw52KMMSY7Ez2JclpUHWvNibVIWIuEtUhYi0QXtbDnYowxJju1OpcjjjiicT3vrLAWCWuRsBYJa5Hoohb2XIwxxmSnV6c37fV650s6fXLJmSkH9/v9fapebC0S1iJhLRLWItFFLWp1LsYYY0wVPCxmjDEmO+5cjDHGZMedizHGmOy4czHGGJMddy7GGGOy487FGGNMdty5GGOMyY47F2OMMdlx52KMMSY7tbbc7/V6rV7O3+/3K59jai0S1iJhLRLWItFFLey5GGOMyY47F2OMMdlx52KMMSY77lyMMcZkx52LMcaY7LhzMcYYkx13LsYYY7LjzsXMDf1+Xz451URiuSj730wHdy7GGGOyU2uFvjHTACuz1xte9Bv/L7q+6PdmvoneB++3qJy4XMwWey7GGGOyY8/FNI66HkrR922mS3mFmNciT6bL5aJJ2HMxxhiTnbnwXIoslC7SJS2ixVlkmUKbx9aLtIjfQ5vyHqnqsZR93+byArMsF/ZcjDHGZGcuPBdoo2VRlS5ZppGqaxTaPNZe5qkUXRevb4MWkbLyUfZ9FzSZRR7tuRhjjMnOXHgubbQs6tIlDcYdOy/6vA0aFuVlnqzzUd/HpMpF/LwNNCEv9lyMMcZkp9GeS5ssznHpohZ1LcwueDBFHss85alqWuvuBzZuuWgTTSgX9lyMMcZkp9GeCzShF24KbdLijDPOkCQddNBBK35fN6/zNO+QizbmqWwnBhi1XLR5rqVJ2HMxxhiTnV6d8cZerzfVwclpWxj9fr/yA6xFIpcWZZFQZXke18LdxX0bWy6mzSzLxS6eM9Lv21wupj3CsZIW9lyMMcZkpxFzLnUt0zaPkZadZcL3bdRi1PUHXTi3o4t1ZNzori6XiyZEFdpzMcYYk51GeC7j7pvUJrqsRdUoobLfF/0/T5odfvjhkqSTTjpJ0ujzCvNI1byU7aNW9f+y+zWZJrcX9lyMMcZkp1HRYrNep9DE6I9ZWSJNigqa9fjxPJSLNmsxbrmYVHlxuRh6rqPFjDHGTJ5GzLnAuGPt8zhmavKvwG5TdFDV6MGy382jFqOeODrq+S7zpM2oTLMc2HMxxhiTnUZ5LqPSBYujzRRZnkUeSd3dj+fRaoey+YOqa4PmOe91o726UC4iVT1Yr3Mxxhgz1zTSc2mjZTEq1qLYej/rrLMkSQcccMDQ522w2ouYdeTcLKjqnRV5Jl2YY2liubDnYowxJjuNWucCs+p1Hbc+9NzG7oqc64TCGulqTLkoy/vOnTslSQsLC5N6/tS0qOqpjEqbygU0aV2cPRdjjDHZafScS5dpwpjppMmVx7JooDbSpXJRdeeOulFibaKJ7YU9F2OMMdlppOdiuhElVjeqq+75Hm3UDOZ5x+cyxi0XRTAf1SatYNSzkCaJPRdjjDHZsefScNpoZUXKvLMyL67NY+mRLpQHqLt3WJk2XdCuSXm052KMMSY79lzMzOnSufCmOrnft8vPdLHnYowxJjsz9VxyWaRtsGzbkIdJUXXfJGvXTqru7Os61CzsuRhjjMnOTD2XXBZGGyyVNuRhWnRhDZBJVH2/ReXC5WM22HMxxhiTHUeLmbnFFqlZCZeLZmDPxRhjTHbcuRhjjMmOOxdjjDHZqTvnslnS6ZNISAM4uOb11iJhLRLWImEtEp3TotYxx8YYY0wVPCxmjDEmO+5cjDHGZMedizHGmOy4czHGGJMddy7GGGOy487FGGNMdty5GGOMyY47F2OMMdlx52KMMSY7tbZ/6fV6rV7O3+/3K+/VbS0S1iJhLRLWItFFLey5GGOMyY47F2OMMdlx52KMMSY77lyMMcZkx52LMcaY7LhzMcYYk526J1E2Eg486/UqRwaaFhAPuvP7N1JxuXA7MV3suRhjjMnOXHguZRZHlyyRqEUXrbEiDezJJLpULmJey8pFl5hle2HPxRhjTHbmwnMxxdZXFyxTiBp0yTqPlJWHNmlS9T2XzbWUXdcmmtBe2HMxxhiTHXsuprFUtUSLro+/axNlWrSJ+P6q5rnsujaWiyZhz8UYY0x2Gum5FI2xdnGMvQuWaVWqWqJd0KwLeRyXqh5Pm9qVJpULey7GGGOy00jPpcgCbYNlUZWi+PQuUDWvZZ5tG9e/dKFcjJvHuh5Lm2hSubDnYowxJjuN9FxMt8llfXV5F4N5xu+pHdhzMcYYkx17Lg2hbH+kLjHqrrZF17XJEi4qF9u2bZtFciZK1bmXovdbtq9WGz3aJrUX9lyMMcZkZ648lzZaGlDXGu8So+a9zZrFcrHbbrvNMjlZKdsLrCplHk38vw3lpEkjHvZcjDHGZGeuPJc2WBaRNllN41J1TL0L5/t0uVzktr7LtGyClV+VuuViluXHnosxxpjsNNJzKbMkumTVtVGLqmmuG/01j1qMistFdQ2K/p/HaMK6Z9oUfe+TKI0xxswljfRciphHa6yMulZaG/bJqjtXUnWsvE2RdV0sF5FR517qngPUZmZZLuy5GGOMyc5ceS4m0QbLNBdRiy7Pycxz3kY9cbKMLuyWDU3aUd6eizHGmOw02nNpgyUxLk2wQKZN3XUJVdfHtEm7LpSLMk+mrqfTBc2alEd7LsYYY7LTaM+lC2PkZXQx0mXU990lrdqc19z1vk1RhPOEPRdjjDHZmYnnYgsiUVeLNms3bt7apE2Xy0XVuZaqXluRNvOoWd0zjWaJPRdjjDHZmYnnMk+WwqQZd6+gNjGqFm1c1+JyUbw+ZefOnVnvO0/MU7mw52KMMSY7jYwWa9K44awp0mKera8iRh1bL9KizVFCXagjZavNq+b9W9/61tD1LhfTwZ6LMcaY7PTq9HC9Xm+q3eG0LYt+v1/5QdPSYlYrbpukRdUyOiltmqQFdLFcjGqNu1xM5bnLHmTPxRhjTHYaOediimnjOHFdmjSuPGvaPH8AZetUwOViObMsF/ZcjDHGZMeeS8NpkyVa1YqK19U9obILtGnH57qrzl0uimlS3u25GGOMyU6jPZcm9cJmfKq+T7/3buFy0U7suRhjjMmOOxdjzFzT7/cdIdZA3LkYY4zJTqPnXIwxpgzPxTQTey7GGGOyU9dz2Szp9EkkpAEcXPN6a5GwFglrkbAWic5pUWvjSmOMMaYKHhYzxhiTHXcuxhhjsuPOxRhjTHbcuRhjjMmOOxdjjDHZcedijDEmO+5cjDHGZMedizHGmOy4czHGGJOdWtu/9Hq9Vi/n7/f7lXfAsxYJa5GwFglrkeiiFvZcjDHGZMedizHGmOy4czHGGJMddy7GGGOy06jOxWdhG2NMO2hU52KMMaYd1D2JcqLEs7DxYnxGtjEr4zpSjjWaDfZcjDHGZKdRngtESyPOw3TRAinToEvWmbVYrkEX8jwqsR3pgkZNqCP2XIwxxmSnkZ4LdDFyrMyiKLLC2midWYv6dCHPRSMbZda5R0IS06gj9lyMMcZkp5GeS5GF0QXK8l401h5/3wasRTFtrCNlHkjR9WWfF13XRg+3TJNp5tmeizHGmOzMxHOp2msW9bKz6IWbQhcjo4posxbj1pF5ZNy0l0WZlj13njUs887iddPIoz0XY4wx2ZmK51IUtVH1d5H4+3m0NKBqRFQR85z3iLVItCkvo1J3Dqbq90X3nwfNizSpmvZp5tGeizHGmOxMxXMZt7fsQnz6qGOhbdTEWlSnC7tZ1J07qfq7eZxrKUpj3bU/ZffLgT0XY4wx2WnkOpdIG60xyB3NMU9WWMRaVKfLa3wiuUZG5pmqEXKeczHGGDPXNNJzKbLC2rQaGapGrIwbSTWPWIvqtDHPZfW+7hqgos/bpF2T2kp7LsYYY7LTSM8l0kYLA8rmGepa6W3SalwtukSb3nsRda3yqvtszTOj7kowDey5GGOMyU4jPZeiXrhNFkdVupTXMkZd99JGDZtoqU6Kunl9ylOeMsnkNIqilfpoNcs6YM/FGGNMdnp1LJ9erzcVM2nU6JAMz638gGlpAVVP24Pjjz9eknTUUUeN+rzGalHEpKy0JmrhOlLOqCv7a9y/sVpM22NZSQt7LsYYY7LTKM9l1qtLm2yJRCZtmViLofs3Tosuei513/OkNWpiuZgV9lyMMcZMhUZFi7V5Z9dx6dIq4zKsxXLanOeqUaOzHvloEk1oQ+25GGOMyU6jPJcuxOyPS9kJdF2y3ruohetI/d0qukxZ2V9cvLoL2L59e/Zn23MxxhiTnUZ5LkW00QKtSt1zO9qskbXo1u4VVfPU5f33InXryCQ8FrDnYowxJjtT8VxGtRTaaFnkwtok2qBFXSsd2rzHWN3z3ye9In+ecbSYMcaYVjAVz6XumGkTdvScNWVWWJe06YIWY+xvlTklzaHsvdY912Wey0cZTawj9lyMMcZkpxHRYj5ZsD7WJtElLbq0i0VZXquuyG+zd1cVz7kYY4xpBY3wXCJttMLKKNvBtUvWl7VIFM1HdkmLqutUZrVT9Cwo0qBJ5cKeizHGmOw00nPpInVj+tuMtUiU7TrQJS2q7krQBU3moVzYczHGGJMdey4NIVphTbA8ZoW1MLvC0aXzsXbHnosxxpjs2HNpCE22QKaNtTBm18xDHbHnYowxJjv2XIwxc8E8WOsmYc/FGGNMdup6LpslnT6JhDSAg2teby0S1iJhLRLWItE5LXpN2CbAGGNMu/CwmDHGmOy4czHGGJMddy7GGGOy487FGGNMdty5GGOMyY47F2OMMdlx52KMMSY77lyMMcZkx52LMcaY7NTa/qXX67V6OX+/36+8M561SFiLhLVIWItEF7Ww52KMMSY77lyMMcZkx52LMcaY7LhzMcYYkx13LsYYY7LjzmXO6Pf78hk8xpim487FGGNMduoeczxTsNh7vcrh5XNHWR7bnHdjTHuw52KMMSY7c+G5tMljqeqZ1M1zvL5Nmhlj5g97LsYYY7LTaM8lRkW1wRovSnvM27h5nGeNymhDOchFrCNt1uQb3/iGJOmud73rSL/vUrlpQrmw52KMMSY7vTprJma1s+e0euEm7nJa9H4mbYk0UYvItCzRedJiIB2Tek7jtKi77iuXNk3UIjLLOmLPxRhjTHYaPefSBoosyiKLoqoVVnZdm8aVvSNBNxi3Toz7nHmmiXXEnosxxpjsNNpzeepTnzrrJIxNmXU07nhxEy2WcalqWVaNvGszbXr/Vdd+Vb1Pm7QpoyjPs6wD9lyMMcZkp9HRYtOOlJpF9Me4eZyURvMQCTMtmqxFm+vIuPOKZdqM6+HOU7lwtJgxxphW0Og5ly6Twaoa6/ezIFea5zHvuWhTnqt6JnXnarpUPjznYowxplU00nOZ1ar0STCqlVT1+iZGiYxK3TQXaTuPea9LlyKhiqi6ZqyL0WMwSy/NnosxxpjsNNJzaRNV12L4PJbEuOtcTDuoWifKPJYu0gRvzZ6LMcaY7NhzmRLr16+XJF122WWS8lnlbRxH7rLF2WX+/u//XpL0qle9StLo5aAJVvusaYL3Zs/FGGNMdhq5Qr/NZ5hUjWzJdd9RafLq42nTRC3aXEd28WzuW+u6SVnxTSwXMG3PxSv0jTHGTAXPuUyZstXEVSmyXJsw1jot2rC2JxddyPu4a7+6RBOiTu25GGOMyY49lzmj7smWXaCLeTbF2KuXjj/++FknwZ6LMcaY/DQqWmzW58I3OfoDpjXP0GQt2nyGSYW0lD1/ko9vhBZlHojriM9zMcYY01LmYs6lC2Oko+KIqW7mucvU3QncdWQ22HMxxhiTnbnwXIzZFV2KAiqii3mPHsys56OazCzqiD0XY4wx2Wm059IlSyNXBEwbrPi6eZjnvI5Ll/JedrJkm05lzYV3RTbGGNMqGu25tMEKr0pVjyV+3sbz4+tq0WW6VEfAcyvV8d5ixhhjWkUjPBdbpMXYCkt0WYtZnd/SJLzbcTlNmm+y52KMMSY7jfBcumR9QRfHys3odLGctHlecVI0SRt7LsYYY7LTCM+lizTJwjCmiZTVkUsuuUSStMcee0wjOaYm9lyMMcZkx56LMWYuscfSbOy5GGOMyY47F2OMMdlx52KMMSY7dedcNks6fRIJaQAH17zeWiSsRcJaJKxFonNa9LyVgjHGmNx4WMwYY0x23LkYY4zJjjsXY4wx2XHnYowxJjvuXIwxxmTHnYsxxpjsuHMxxhiTHXcuxhhjsuPOxRhjTHZqbf/S6/VavZy/3+9XPsHLWiSsRcJaJKxFoota2HMxxhiTHXcuxhhjsuPOxRhjTHbcuRhjjMmOOxdjjDHZcedijDEmO3PVufT7fflwM2NMFdxezFaDuepcjDHGzAe1FlHOitjz8n+vV3kNkzGmo3S5vZhlnu25GGOMyU6jPZeyscIuWyTGDBLrShfrhNuLZmHPxRhjTHYa4bkUWRT83+WIj7rWVputszbnzVQnloMutw9Nxp6LMcaY7DTCcymyRMsskjZZsONaX22y6ou0KCsnbch7EV3IYxnWYDlN1sSeizHGmOw0wnMxxVS1SJpouYxKHEsvy1ub8l5EVe++zVoU5a3q3EsbtWlynuy5GGOMyc5cei5N7q2rEq3yutZ6myiLFjTV6WL5gTIPpkvaNCGv9lyMMcZkZy49lzZQZGWNamnM89h77rQ2wWozk6fuepculodZ1gV7LsYYY7Jjz2XG5LIovFo50SULtctzdd7Ro5gmlAN7LsYYY7LTSM+li5bHuJZnmyzXNuVl0nSxrhRRd2eHeaasjjShDtlzMcYYk51Gei65I6nmAXssCWthctLGcjHqrhXT1MKeizHGmOw0ynPp8klynnMZny5FTnV5rqVu3ttcDuoyTS3suRhjjMlOozyXMtpsgYxrdbdRG5/CWZ0u5dnrWeYDey7GGGOyM1eeSxewlZ6w97acLkVQ1sUeTbOw52KMMSY7c+G52DorpgseTBld0sDWeTH26ppVF+y5GGOMyc5ceC5N6o2NMbOlyGvbf//9JUlnn332NJPTKJrURtpzMcYYk51enbHbXq83kYHeppwg1+/3Kz9oUlqUMS0vzloMPWfmWjTFe2+SFhWeP4nHD6Zj5lqUMcs6Ys/FGGNMdmY659IUj2UeiFo1xZKdBV2KlCp6z21+/2V5LqKNWoyLz3MxxhjTKmbqudS1TLpMkVZttmCLKCs3bdKii3Wk6P2VrfHpYl2oi89zMcYYM9c0Yp1LF6yx3Ng6S3RJiy7ltS7Wphyf52KMMWauaYTn0sVx5bp4PNl0nVgH3E40G3suxhhjstMIzyVi63w51sR0nVgHXCeajT0XY4wx2XHnYowxJjvuXIwxxmTHnYsxxpjsuHMxxhiTnbrRYpslnT6JhDSAg2teby0S1iJhLRLWItE5LWodFmaMMcZUwcNixhhjsuPOxRhjTHbcuRhjjMmOOxdjjDHZcedijDEmO+5cjDHGZMedizHGmOy4czHGGJMddy7GGGOyU2v7l16v1+rl/P1+v/LpQ9YiYS0S1iJhLRJd1MKeizHGmOy4czHGGJMddy7GGGOy487FGGNMdty5GGOMyY47F2OMMdlx52KM6QT9fl8+HHF6uHMxxhiTnVqLKGcF1kavV3nNkjGdxHWlmC5qMsvyYM/FGGNMdubCc+kytkRNFbpcTlatutpG3rlz54rfd1mbWWLPxRhjTHYa7bnEyI4uWyBleT/00EMlSSeffPLU0jRtYnnoYjmIFNURaJNGsQ6URX7FvBdd3yaNoAlRcfZcjDHGZKdXp4eb9rbR07Y0JrmF9rx5XfOsxcLCgiRpx44dWe7X5K3Vu1BHJm2Fj6rVPJWLSbc73nLfGGPMVGj0nEubKLIcclvxZdZfEzynSWuBx9KkPJtyxvVYqs7FlD2vTeVllnmx52KMMSY7jfRc2hjVURTpUvR/bpqk3ay1aANNiAbKTdX3XlZOqnow8fdtLG9eoW+MMaZVNNJzqTt2Og9MMHpnKs/JSVXLtC5dsEShqI7Mc56L0h7zWndOpU0azRP2XIwxxmSnkZ4LtNnCyDXfUOasXYhIAAACoElEQVTlzcP8Ra65libncVrMw/uuy6QjKtuoWRPyYs/FGGNMdhrtuXRhrLQo0mVS928yk17r0wXalOdx10OVRZ22uZw0oe2052KMMSY7jfRcmtDrzgveOTrRpby3KZKyiEmth2pzOWlS22nPxRhjTHYa6bl0idxW1DxbY7m06JJl2mZG9Vi6pFGkSWXenosxxpjs2HMxZo7owqrzsp0Wxs1rm7Qqogneuz0XY4wx2Wmk59LGvcWKcKRLwlqU4zqRnzZ7gbPEnosxxpjsNMpz6YJVlhtbWQlr0S7qeqajeiAuN5PBnosxxpjsNMpz6dJcy7jzBV2cbyiiC1p0oU4UUdUjKTv3pc3lo4nYczHGGJOdRnkuXbLOcsXq2zrrhhYf+tCHJEmPe9zjhj5vUx4jo0YPmmZgz8UYY0x2enV6+16vN1HToOz8hUnT7/crP2jSWkDRvkorpCf3cxuvxbRokhaz9s6apAXM6tyWJmvRhHJhz8UYY0x2GjXnEhn3JLo2UBYZ0yW6mGeo67G0oY5UPc/FJJpULuy5GGOMyU6jPJeqFsk8W2Pj0uW8d5m61nobyknRbsh1z3dpgxa5mKYW9lyMMcZkp1GeC0w6ymMeKdKiixp1WYuYt912202SdNVVV80iOY2grL3oQrmINCHP9lyMMcZkp5GeC9jSSHRpPNlaFBO16ZLHMupcbBfKRaQJebbnYowxJjuN9ly6RBMsjaZgLYrpsjZdzvs8Ys/FGGNMdty5GGOMyY47F2OMMdmpO+eyWdLpk0hIAzi45vXWImEtEtYiYS0SndOi1pb7xhhjTBU8LGaMMSY77lyMMcZkx52LMcaY7LhzMcYYkx13LsYYY7LjzsUYY0x23LkYY4zJjjsXY4wx2XHnYowxJjv/D5lVGvtaEbV8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x864 with 60 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "rows = 10 # split epochs into 10, so 100/10 = every 10 epochs\n",
    "cols = 6\n",
    "fig, axes = plt.subplots(figsize=(7,12), nrows=rows, ncols=cols, sharex=True, sharey=True)\n",
    "\n",
    "for sample, ax_row in zip(samples[::int(len(samples)/rows)], axes):\n",
    "    for img, ax in zip(sample[::int(len(sample)/cols)], ax_row):\n",
    "        img = img.detach()\n",
    "        ax.imshow(img.reshape((28,28)), cmap='Greys_r')\n",
    "        ax.xaxis.set_visible(False)\n",
    "        ax.yaxis.set_visible(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It starts out as all noise. Then it learns to make only the center white and the rest black. You can start to see some number like structures appear out of the noise like 1s and 9s."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sampling from the generator\n",
    "\n",
    "We can also get completely new images from the generator by using the checkpoint we saved after training. **We just need to pass in a new latent vector $z$ and we'll get new samples**!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZYAAAGOCAYAAAC0fSVgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAOSklEQVR4nO3dMZbbxhIFUMKyE6c+irWJvzYvwnv7i1A+Sh3ZgiNp6GNCRIMPQHXj3miO7OHwdBHzVKVuYJrn+QYAKT+d/QYAGItgASBKsAAQJVgAiBIsAEQJFgCifm75n6dpsjc5ZJ7nKfVa6pKTqouaRL3N8/wx8ULqErVYFx0LUN3ns98ADy3WRbAAECVYAIgSLABECRYAogQLAFGCBYAowQJAlGABIEqwABAlWACIarpX2IjuH808TbHbdwFclo4FgCjBAkDU0KOw+zEX/TKurEdN+BEdCwBRggWAqKFHYZwjPSYxanndo7HwK+uqJhmjjhR1LABECRYAoi4/Chup/aziqDUddYywt61rtWa91aTNqGukYwEgSrAAEHX5URh5R41DRh0jpCwdEP7252vGWWte+/511KTN1mul+shRxwJAlGABIKrrUdijdtD9wfa1Zn0TrfnSz6nY9p+h9XN+v27PvveVERnv9rxW1tTzzGtFxwJAlGABIKrrUViijaSmltHNFb2yPs8+/9a7pvu6fP369fvXFX+f6VgAiBIsAER1PQq7p33Paj0AZ/33t+ZQnBHi8Y66ViqOvJboWACIEiwARA0zCiOrdaSSPhTp/lP/tVSTNX++53u5utb13/NaqULHAkCUYAEgyiiMp47aaVSxpa/q7N1f1UcxZznqHl7V11zHAkBU1x1Ly9/Uqid8j6xpPem/DVe8c26PrrZeOhYAogQLAFFdj8IYi7HLNns+WI3tjqpLxetDxwJAlGABIMoojKeMSepJnCNZc1uYe2fcOqY36XVpPa9UpS46FgCiBAsAUd2NwhyKPF7rA4ta1t2YbZvE3adfuS2Ma+uxNeuy57iqSl10LABECRYAorobhT1r36u0gldwxq4X/uuoz3yVHUe9S69dxbroWACIEiwARHU3Cns2LqnYFvJvrQfwOI+a1FT9d5uOBYAowQJAVBejMIcix+IJhfWoCUk6FgCiBAsAUaVGYXZ0jaunW373Zuu6rfk+NTneCHXRsQAQJVgAiCo1CnMYi0eqt/1nO2NN1KSmKnXRsQAQJVgAiCo1CrvX+tRCjre17W5t0Y1a1kuMQtZ8n5q0uVpddCwARAkWAKLKjsKWLO0cq9ICXsnW8Ze67adlnVvXXt22u9q1omMBIEqwABDV3SjsXk+tIe/U7RjpdVa34/W65joWAKIECwBRgoVdzfPsUGsxasLeBAsAUYIFgKiud4VRX6+7WkamJuxNxwJAlGABIKp1FPZ2u90+7/FGLuZT+PXUJSNZFzXJUZeaFusy2XYIQJJRGABRggWAKMECQJRgASBKsAAQJVgAiBIsAEQJFgCiBAsAUYIFgCjBAkCUYAEgSrAAECVYAIhqeh7LNE3usR8yz3Ps+bDqkpOqi5pEvc3z/DHxQuoStVgXHQtQnQdz1bRYF8ECQJRgASBKsAAQJVgAiBIsAEQJFgCiBAsAUYIFgCjBAkCUYAEgquleYSOa5/dbB01T7PZdAJss/U5q/fMz6VgAiBIsAEQNPQq7bxGpqWIbf0VL14qaHG9pzNUTHQsAUYIFgKihR2EAo6s4rtSxABAlWACI6noUZkcRQD06FgCiBAsAUV2Pwu4lDhIZrR2j10NfwDo6FgCiBAsAUcOMwhKMv44xwr2QrsJ4mC10LABECRYAoroehT1qzY1W6lOjfhh/HW+E60PHAkCUYAEgqutR2L2W9tGupDqMWo737DOvJucYqS46FgCiBAsAUcOMwlo49HU8I0e4Dh0LAFGCBYCorkdhxisA9ehYAIjqumPZyj/YH+dRV2n9YZ1erxUdCwBRggWAqO5GYf7BHtq5bmoatS46FgCiBAsAUd2NwtyZGNo9u2563X3Uu1F/n+lYAIgSLABEdTcK29ouavX39eyO0db/eCM9OGpUo44ldSwARAkWAKK6G4Vt5eFe+xp1d0vP1ISz6FgAiBIsAEQNPQoz8uLKno2/jIdrGqEuOhYAogQLAFFdjMJadrT02jqOSj2O5VqpqWW8NUJddCwARAkWAKJKjcIc4hrDCK18rxyKrGmpLqPWS8cCQJRgASCq1Chs1LYQjuK66cuo9dKxABAlWACIKjUKYwwj3OuoV8bJVKBjASBKsAAQVXYUtjRCWWrvjVzqeDSOWXNAjKyWa0hNjtOyvr3WRccCQJRgASCq7ChsSU/tII/rpYbnUpN+9FoXHQsAUYIFgCjBAkCUYAEgSrAAECVYAIgSLABECRYAoloPSL7dbrfPe7yRi/kUfj11yUjWRU1y1KWmxbpMntkAQJJRGABRggWAKMECQJRgASBKsAAQJVgAiBIsAEQJFgCiBAsAUYIFgCjBAkCUYAEgSrAAECVYAIhqeh7LNE3usR8yz/OUei11yUnVRU2i3uZ5/ph4IXWJWqyLjgWozoO5alqsi2ABIEqwABAlWACIEiwARAkWAKIECwBRggWAKMECQJRgASBKsAAQJVgAiBIsAEQJFgCimm6bP7p5fr+j9jTF7mrPD1jzetSEV+lYAIgSLABEdTEKW2rN0y27th+oqqcRpY4FgCjBAkBUF6OwpbbvqHawpxYU4Gw6FgCiBAsAUV2MwpYcNaIy/mJ099cSNfX0e0jHAkCUYAEgquwobE1rnmgNl35OT21nz6xzDUsHj2ELHQsAUYIFgKiyo7CjGMWA8deRrnDgWscCQJRgASDq9FHYUlu41CImWvajbsPPY9a5NjXZV8v69nqt6FgAiBIsAESdPgprPZjlnmB9suuoHjVhLzoWAKIECwBRp4/C1uzQSjPyAqrqdSfYPR0LAFGndyxrErlSav/xxx/fv/79999PfCd9eNR5VqrnFalJbSPUQscCQJRgASBqavlH8mmaDtv4vuc/YFV4uNc8z7EfdmRdWvU2dknVRU2i/j/P8/8SL9RDXYrX4t5iXXQsAEQJFgCiTt8Vdm/NWC49InN3Y4AsHQsAUYIFgKhSo7Cl8dPff//98msvjdmMv45hbY/37LOtJue4wu8cHQsAUYIFgKhSo7AlHz58ePk1jrpzMta3Cp/5mq6wE1XHAkCUYAEgqotRWMIZDxQD+JGRxl/3dCwARAkWAKIuMwpbMmoreiZrWpv6sDcdCwBRggWAqOFGYWsOHI16KAmW+JxzJB0LAFGCBYCo4UZh9/7888+z3wLA5ehYAIgSLABEdTEK27qL69dff93j7QDwAzoWAKIECwBRXYzCWsZfV3g6W0++1UBd6lCTfvRaFx0LAFGCBYCoLkZhW/XUOo7qUQ3U5Vxq0o9e66JjASBKsAAQNUywzPP8rx0UAJxjmGABoAbBAkDUMLvCet09ATAaHQsAUYIFgKjWUdjb7Xb7vMcbuZhP4ddTl4xkXdQkR11qWqzLZIsuAElGYQBECRYAogQLAFGCBYAowQJAlGABIEqwABAlWACIEiwARAkWAKIECwBRggWAKMECQJRgASBKsAAQ1fSgr2maPLwlZJ7nKfVa6pKTqouaRL3N8/wx8ULqErVYFx0LUJ0nPta0WBfBAkCUYAEgSrAAECVYAIgSLABECRYAogQLAFGCBYAowQJAVNMtXYBxfPny5fvXv/322/ev5/n9rifTND39c85VsS46FgCiBAsAUZcchWn1+6Iu+1gaf5GX+AyvqVGVa0XHAkCUYAEgqutR2KPWsLX9q9I69mjNSPHRf2+lLjWoQ5vW3y3f/v9n19JePz9JxwJAlGABIKrrUdi9Z61ery1lZUtrcdQaqQuVrdlxuudn2K4wAIYhWACIKjsKa9lxdP/nr7R/S69tzNJm666w1rGAurzGoci81kOMrd+bfi97XUM6FgCiBAsAUaVGYXu2iK2HjoxZHmtdu5barfk+dcn5+vXr969/+un975jWOGNpHfccQVa5hnQsAEQJFgCiSo3C9hxX2QFzjq0HV1P3S2LZhw8fvn9tjTNadzPuuRNs6ecccbBYxwJAlGABIOr0UdiatsxY5HhLdUnvdFnTrpNlbY/Resj7r7/+ut1ut9vPP+d/LbttPgBdEywARJ0+Cmsdf+3Z0jkY9u7sW34vvRfoRes19Msvv/zw9Xo62K1jASBKsAAQdfoobMkZO1c8kfBd644W4DWPHv3R0/jrno4FgKiyHcu9rYnc+rfrisl/Fg/auha1PM6z30utv7fWTFqOnsboWACIEiwARJ0yCmu9jQt1JOpiA8DxrPl+UmOmb9+bGuGfuRlJxwJAlGABIOqUUdierVvr+YszblHSm/S6qMvxnEXaT+rhXom6VLlWdCwARAkWAKJKHZBM3MrglZbfmOWx9C0m1KUG67qvPcdS1WunYwEgSrAAEFVqFHbvqPuDVdlF0Yuj1khdsuwEO8YrY96W761+fehYAIgSLABElR2FbfXK4bvq7eXoHOI7ls97xppD2elDkal7LO71GdCxABAlWACIGm4UlrpXGPtprQvrGW8d4+zfFelb9afpWACIEiwARJUdhaVvob/mtY0OnttalzXfZ4zzujWPG/j259Z4uzWf4TN+fhU6FgCiBAsAUWVHYa8c9NnKKOa5M9ZFXbZpeXyBda1vTY2q7LLUsQAQJVgAiCo7Cru3tb1rbf+MA9okRilrvk9dXnf2gT5et+Z6q3Kt6FgAiBIsAER1MQrb895SdsZst+Yw3pb/d+3/w3p2iGW1HrJeM4rcc92Prq+OBYAowQJAVBejsHvpNk7bn9GyjnaC1WO92/S249QBSQC6JlgAiOpuFLbErcBrUhdYNupuPB0LAFGCBYCoYUZhI7WRI1EXWDbq9aFjASBKsAAQ1ToKe7vdbp/3eCMX8yn8euqSkayLmuSoS02LdZk8pwGAJKMwAKIECwBRggWAKMECQJRgASBKsAAQJVgAiBIsAEQJFgCi/gGO9ghou8RGsQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x504 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# randomly generated, new latent vectors\n",
    "sample_size=16\n",
    "rand_z = np.random.uniform(-1, 1, size=(sample_size, z_size))\n",
    "rand_z = torch.from_numpy(rand_z).float()\n",
    "\n",
    "G.eval() # eval mode\n",
    "# generated samples\n",
    "rand_images = G(rand_z)\n",
    "\n",
    "# 0 indicates the first set of samples in the passed in list\n",
    "# and we only have one batch of samples, here\n",
    "view_samples(0, [rand_images])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}