pytorch_computer_vision.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "69a244a0-17cc-40c1-bc48-57a28aa36890",
   "metadata": {},
   "source": [
    "### PyTorch Computer Vision \n",
    "\n",
    "* See reference notebook - https://github.com/mrdbourke/pytorch-deep-learning/blob/main/03_pytorch_computer_vision.ipynb\n",
    "* See reference online book - https://www.learnpytorch.io/03_pytorch_computer_vision/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "559ec56e-dfe3-4a3d-9720-3ce6449d825c",
   "metadata": {},
   "source": [
    "## 0. Computer vision libaries in pytorch\n",
    "\n",
    "* [`torchvision`](https://www.learnpytorch.io/03_pytorch_computer_vision/) - base domain library for PyTorch computer vision\n",
    "* `torchvision.datasets` - get datasets and data loading functions for computer vision here\n",
    "* `torchvision.models` - get pretrained computer vision models that you can leverage for your own problems\n",
    "* `torchvision.transforms` - functions for manipulating your vision data (images) to be suitable for use with an ML model.\n",
    "* `torch.utils.data.Dataset` - Base dataset class for PyTorch.\n",
    "* `torch.utils.data.DataLoader` - Creates a Python iterable over a dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1aaaa5f3-3332-4912-abbf-e4764e81a7e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0.1+cu117\n",
      "0.15.2+cu117\n"
     ]
    }
   ],
   "source": [
    "# Import Pytorch\n",
    "import torch\n",
    "from torch import nn\n",
    "\n",
    "# Import torchvision\n",
    "import torchvision\n",
    "from torchvision import datasets\n",
    "from torchvision import transforms\n",
    "from torchvision.transforms import ToTensor\n",
    "\n",
    "# Import matplotlib for visualization\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Check versions\n",
    "print(torch.__version__)\n",
    "print(torchvision.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce5324a4-bca1-454f-93e8-e3bc8e15a771",
   "metadata": {},
   "source": [
    "## Getting a dataset\n",
    "The dataset we will be using is FashionMNIST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cdfb9bac-59a1-4bf2-86f4-8a4b35c192bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchvision import datasets\n",
    "train_data = datasets.FashionMNIST(\n",
    "    root=\"data\",\n",
    "    train=True,\n",
    "    download=True,\n",
    "    transform=torchvision.transforms.ToTensor(),\n",
    "    target_transform=None # how do we want to transform the labels / targets ?\n",
    ")\n",
    "\n",
    "test_data = datasets.FashionMNIST(\n",
    "    root=\"data\",\n",
    "    train=False,\n",
    "    download=True,\n",
    "    transform=ToTensor(),\n",
    "    target_transform=None\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "30bb9ca0-5b5f-4486-a57b-d518718ef0fe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 10000)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train_data) , len(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fcb059a7-7f73-4130-ad85-29436a02bce0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0039, 0.0000, 0.0000, 0.0510,\n",
       "           0.2863, 0.0000, 0.0000, 0.0039, 0.0157, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0039, 0.0039, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0118, 0.0000, 0.1412, 0.5333,\n",
       "           0.4980, 0.2431, 0.2118, 0.0000, 0.0000, 0.0000, 0.0039, 0.0118,\n",
       "           0.0157, 0.0000, 0.0000, 0.0118],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0235, 0.0000, 0.4000, 0.8000,\n",
       "           0.6902, 0.5255, 0.5647, 0.4824, 0.0902, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0471, 0.0392, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.6078, 0.9255,\n",
       "           0.8118, 0.6980, 0.4196, 0.6118, 0.6314, 0.4275, 0.2510, 0.0902,\n",
       "           0.3020, 0.5098, 0.2824, 0.0588],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0039, 0.0000, 0.2706, 0.8118, 0.8745,\n",
       "           0.8549, 0.8471, 0.8471, 0.6392, 0.4980, 0.4745, 0.4784, 0.5725,\n",
       "           0.5529, 0.3451, 0.6745, 0.2588],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0039, 0.0039, 0.0039, 0.0000, 0.7843, 0.9098, 0.9098,\n",
       "           0.9137, 0.8980, 0.8745, 0.8745, 0.8431, 0.8353, 0.6431, 0.4980,\n",
       "           0.4824, 0.7686, 0.8980, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.7176, 0.8824, 0.8471,\n",
       "           0.8745, 0.8941, 0.9216, 0.8902, 0.8784, 0.8706, 0.8784, 0.8667,\n",
       "           0.8745, 0.9608, 0.6784, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.7569, 0.8941, 0.8549,\n",
       "           0.8353, 0.7765, 0.7059, 0.8314, 0.8235, 0.8275, 0.8353, 0.8745,\n",
       "           0.8627, 0.9529, 0.7922, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0039, 0.0118, 0.0000, 0.0471, 0.8588, 0.8627, 0.8314,\n",
       "           0.8549, 0.7529, 0.6627, 0.8902, 0.8157, 0.8549, 0.8784, 0.8314,\n",
       "           0.8863, 0.7725, 0.8196, 0.2039],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0235, 0.0000, 0.3882, 0.9569, 0.8706, 0.8627,\n",
       "           0.8549, 0.7961, 0.7765, 0.8667, 0.8431, 0.8353, 0.8706, 0.8627,\n",
       "           0.9608, 0.4667, 0.6549, 0.2196],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0157, 0.0000, 0.0000, 0.2157, 0.9255, 0.8941, 0.9020,\n",
       "           0.8941, 0.9412, 0.9098, 0.8353, 0.8549, 0.8745, 0.9176, 0.8510,\n",
       "           0.8510, 0.8196, 0.3608, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0039, 0.0157, 0.0235, 0.0275, 0.0078, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.9294, 0.8863, 0.8510, 0.8745,\n",
       "           0.8706, 0.8588, 0.8706, 0.8667, 0.8471, 0.8745, 0.8980, 0.8431,\n",
       "           0.8549, 1.0000, 0.3020, 0.0000],\n",
       "          [0.0000, 0.0118, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.2431, 0.5686, 0.8000, 0.8941, 0.8118, 0.8353, 0.8667,\n",
       "           0.8549, 0.8157, 0.8275, 0.8549, 0.8784, 0.8745, 0.8588, 0.8431,\n",
       "           0.8784, 0.9569, 0.6235, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0706, 0.1725, 0.3216, 0.4196,\n",
       "           0.7412, 0.8941, 0.8627, 0.8706, 0.8510, 0.8863, 0.7843, 0.8039,\n",
       "           0.8275, 0.9020, 0.8784, 0.9176, 0.6902, 0.7373, 0.9804, 0.9725,\n",
       "           0.9137, 0.9333, 0.8431, 0.0000],\n",
       "          [0.0000, 0.2235, 0.7333, 0.8157, 0.8784, 0.8667, 0.8784, 0.8157,\n",
       "           0.8000, 0.8392, 0.8157, 0.8196, 0.7843, 0.6235, 0.9608, 0.7569,\n",
       "           0.8078, 0.8745, 1.0000, 1.0000, 0.8667, 0.9176, 0.8667, 0.8275,\n",
       "           0.8627, 0.9098, 0.9647, 0.0000],\n",
       "          [0.0118, 0.7922, 0.8941, 0.8784, 0.8667, 0.8275, 0.8275, 0.8392,\n",
       "           0.8039, 0.8039, 0.8039, 0.8627, 0.9412, 0.3137, 0.5882, 1.0000,\n",
       "           0.8980, 0.8667, 0.7373, 0.6039, 0.7490, 0.8235, 0.8000, 0.8196,\n",
       "           0.8706, 0.8941, 0.8824, 0.0000],\n",
       "          [0.3843, 0.9137, 0.7765, 0.8235, 0.8706, 0.8980, 0.8980, 0.9176,\n",
       "           0.9765, 0.8627, 0.7608, 0.8431, 0.8510, 0.9451, 0.2549, 0.2863,\n",
       "           0.4157, 0.4588, 0.6588, 0.8588, 0.8667, 0.8431, 0.8510, 0.8745,\n",
       "           0.8745, 0.8784, 0.8980, 0.1137],\n",
       "          [0.2941, 0.8000, 0.8314, 0.8000, 0.7569, 0.8039, 0.8275, 0.8824,\n",
       "           0.8471, 0.7255, 0.7725, 0.8078, 0.7765, 0.8353, 0.9412, 0.7647,\n",
       "           0.8902, 0.9608, 0.9373, 0.8745, 0.8549, 0.8314, 0.8196, 0.8706,\n",
       "           0.8627, 0.8667, 0.9020, 0.2627],\n",
       "          [0.1882, 0.7961, 0.7176, 0.7608, 0.8353, 0.7725, 0.7255, 0.7451,\n",
       "           0.7608, 0.7529, 0.7922, 0.8392, 0.8588, 0.8667, 0.8627, 0.9255,\n",
       "           0.8824, 0.8471, 0.7804, 0.8078, 0.7294, 0.7098, 0.6941, 0.6745,\n",
       "           0.7098, 0.8039, 0.8078, 0.4510],\n",
       "          [0.0000, 0.4784, 0.8588, 0.7569, 0.7020, 0.6706, 0.7176, 0.7686,\n",
       "           0.8000, 0.8235, 0.8353, 0.8118, 0.8275, 0.8235, 0.7843, 0.7686,\n",
       "           0.7608, 0.7490, 0.7647, 0.7490, 0.7765, 0.7529, 0.6902, 0.6118,\n",
       "           0.6549, 0.6941, 0.8235, 0.3608],\n",
       "          [0.0000, 0.0000, 0.2902, 0.7412, 0.8314, 0.7490, 0.6863, 0.6745,\n",
       "           0.6863, 0.7098, 0.7255, 0.7373, 0.7412, 0.7373, 0.7569, 0.7765,\n",
       "           0.8000, 0.8196, 0.8235, 0.8235, 0.8275, 0.7373, 0.7373, 0.7608,\n",
       "           0.7529, 0.8471, 0.6667, 0.0000],\n",
       "          [0.0078, 0.0000, 0.0000, 0.0000, 0.2588, 0.7843, 0.8706, 0.9294,\n",
       "           0.9373, 0.9490, 0.9647, 0.9529, 0.9569, 0.8667, 0.8627, 0.7569,\n",
       "           0.7490, 0.7020, 0.7137, 0.7137, 0.7098, 0.6902, 0.6510, 0.6588,\n",
       "           0.3882, 0.2275, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1569,\n",
       "           0.2392, 0.1725, 0.2824, 0.1608, 0.1373, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000],\n",
       "          [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n",
       "           0.0000, 0.0000, 0.0000, 0.0000]]]),\n",
       " 9)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# see the first training example\n",
    "image, label = train_data[0]\n",
    "image, label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c1176497-f1ab-4d00-af41-125482463a1d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['T-shirt/top',\n",
       " 'Trouser',\n",
       " 'Pullover',\n",
       " 'Dress',\n",
       " 'Coat',\n",
       " 'Sandal',\n",
       " 'Shirt',\n",
       " 'Sneaker',\n",
       " 'Bag',\n",
       " 'Ankle boot']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class_names = train_data.classes\n",
    "class_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "baa3f503-2db6-420e-8e4b-fc8cad0aecd8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'T-shirt/top': 0,\n",
       " 'Trouser': 1,\n",
       " 'Pullover': 2,\n",
       " 'Dress': 3,\n",
       " 'Coat': 4,\n",
       " 'Sandal': 5,\n",
       " 'Shirt': 6,\n",
       " 'Sneaker': 7,\n",
       " 'Bag': 8,\n",
       " 'Ankle boot': 9}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class_to_idx = train_data.class_to_idx\n",
    "class_to_idx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "57f9698e-5faa-4da2-a223-de12aa77384c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([9, 0, 0,  ..., 3, 0, 5])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.targets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6bb2a4ed-2be8-42b9-8182-1330b12d61fe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Image shape: torch.Size([1, 28, 28]) -> [color_channels, height, width]\n",
      "Image label: Ankle boot\n"
     ]
    }
   ],
   "source": [
    "# check the shape of our image\n",
    "print(f\"Image shape: {image.shape} -> [color_channels, height, width]\")  ## only have one color channel, since the image is black and white\n",
    "print(f\"Image label: {class_names[label]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d3407bd2-98c9-4805-9412-322b4687551a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Image shape: torch.Size([1, 28, 28])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(-0.5, 27.5, 27.5, -0.5)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGbCAYAAAAr/4yjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAWu0lEQVR4nO3da2yedf0/8M/d9bBuHTB2YFT2owibTEQgAzmOHRCUw8QgsvjAMIGIJgQhGJ/4gBiNykEkCIahxpCxZKAZJ+UgKEFlmDEMQgxEBhsKw43BNrd2bbf2+j8wfOIc0n6vvy1TX69kIffd7/v+Xr3u++6719Z+aFRVVQUARETTe30AAOw9lAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQC/3EWL14cHR0dQ66bN29ezJs379+277x58+JDH/rQv+3xYG+kFBgV3//+96PRaMTxxx//Xh/Kf6RvfvObcc8997zXh8H/AKXAqFi2bFl0dXXFqlWrYs2aNe/14fzHUQqMFqXAiFu7dm2sXLkybrjhhpgyZUosW7bsvT4k4F9QCoy4ZcuWxcSJE+Pss8+O888//x1LYd26ddFoNOL666+P2267LQ499NBoa2uL4447Lp566qkh93jmmWdiypQpMW/evNi+ffu/XNfX1xdXX311HHbYYdHW1hbTp0+Pr3zlK9HX1zfsz+fpp5+Ok046Kdrb2+OQQw6JW2+9dY81GzdujIsvvjgOOOCAGDt2bBx11FFx++2377Guu7s7rrrqqpg+fXq0tbXFBz7wgbj++uvjH4cXNxqN6O7ujttvvz0ajUY0Go1YvHjxsI8XilQwwg4//PDq4osvrqqqqn79619XEVGtWrVqtzVr166tIqI65phjqsMOO6y65pprqmuvvbaaPHlyddBBB1X9/f259sILL6zGjx+ft1etWlVNnDixOv3006uenp68f+7cudXcuXPz9sDAQHXGGWdU48aNq6644opqyZIl1WWXXVY1NzdX55577pCfx9y5c6vOzs5q6tSp1WWXXVbddNNN1SmnnFJFRPWjH/0o1/X09FSzZs2qWlpaqiuvvLK66aabqjlz5lQRUd144425bnBwsFqwYEHVaDSqSy65pLr55purhQsXVhFRXXHFFblu6dKlVVtbWzVnzpxq6dKl1dKlS6uVK1cOfeKhBqXAiFq9enUVEdUjjzxSVdXfvxAedNBB1Ze+9KXd1r1dCpMmTareeuutvP/ee++tIqK6//77875/LIXf/va31T777FOdffbZVW9v726P+c+lsHTp0qqpqan6zW9+s9u6W2+9tYqI6oknnnjXz2Xu3LlVRFTf+c538r6+vr7q6KOPrqZOnZrFdeONN1YRUd1xxx25rr+/vzrxxBOrjo6O6m9/+1tVVVV1zz33VBFRfeMb39htn/PPP79qNBrVmjVr8r7x48dXF1544bseH/w7+OsjRtSyZcvigAMOiPnz50fE3/8qZNGiRbF8+fIYGBjYY/2iRYti4sSJeXvOnDkREfHyyy/vsfaxxx6Lj33sY3HaaafFihUroq2t7V2P5Sc/+UnMmjUrDj/88Ni0aVP+WbBgQT7eUJqbm+PSSy/N262trXHppZfGxo0b4+mnn46IiAceeCCmTZsWn/nMZ3JdS0tLXH755bF9+/Z4/PHHc92YMWPi8ssv322Pq666KqqqigcffHDI44F/N6XAiBkYGIjly5fH/PnzY+3atbFmzZpYs2ZNHH/88bFhw4b45S9/uUfm//7v/3a7/XZBbN68ebf7e3t74+yzz45jjjkm7rrrrmhtbR3yeF588cX44x//GFOmTNntz8yZMyPi7/8OMJTOzs4YP378bve9nV+3bl1ERLzyyisxY8aMaGra/e01a9as/Pjb/+3s7IwJEya86zoYTc3v9QHw3+tXv/pVvP7667F8+fJYvnz5Hh9ftmxZnHHGGbvdN2bMmHd8rOqf/q+xbW1tcdZZZ8W9994bDz30UJxzzjlDHs/g4GAceeSRccMNN7zjx6dPnz7kY8B/O6XAiFm2bFlMnTo1brnllj0+tmLFirj77rvj1ltvjfb29uLHbjQasWzZsjj33HPj05/+dDz44IND/vbyoYceGn/4wx/itNNOi0ajUbxnRMT69euju7t7t6uFP/3pTxER0dXVFRERBx98cDz77LMxODi429XCCy+8kB9/+7+PPvpobNu2bberhX9e9/bnC6PBXx8xInbs2BErVqyIc845J84///w9/lx22WWxbdu2uO+++2rv0draGitWrIjjjjsuFi5cGKtWrXrX9RdccEG89tpr8YMf/OAdj7e7u3vIPXft2hVLlizJ2/39/bFkyZKYMmVKzJ49OyIizjrrrPjrX/8ad9555265733ve9HR0RFz587NdQMDA3HzzTfvtsd3v/vdaDQaceaZZ+Z948ePjy1btgx5fPD/y5UCI+K+++6Lbdu2xSc+8Yl3/PgJJ5yQv8i2aNGi2vu0t7fHz372s1iwYEGceeaZ8fjjj//L+USf/exn46677oovfOEL8dhjj8XJJ58cAwMD8cILL8Rdd90VDz/8cBx77LHvul9nZ2dcc801sW7dupg5c2bceeed8cwzz8Rtt90WLS0tERHx+c9/PpYsWRKLFy+Op59+Orq6uuKnP/1pPPHEE3HjjTfmVcHChQtj/vz58dWvfjXWrVsXRx11VPziF7+Ie++9N6644oo49NBDc9/Zs2fHo48+GjfccEN0dnbGIYccYmQII+O9/vEn/jstXLiwGjt2bNXd3f0v1yxevLhqaWmpNm3alD+Set111+2xLiKqq6++Om//8+8pVFVVbdq0qfrgBz9YTZs2rXrxxRerqtrzR1Kr6u8/GnrNNddURxxxRNXW1lZNnDixmj17dvW1r32t2rp167t+TnPnzq2OOOKIavXq1dWJJ55YjR07tjr44IOrm2++eY+1GzZsqD73uc9VkydPrlpbW6sjjzyy+vGPf7zHum3btlVXXnll1dnZWbW0tFQzZsyorrvuumpwcHC3dS+88EJ16qmnVu3t7VVE+PFURkyjqv7pX/AA+J/l3xQASEoBgKQUAEhKAYCkFABISgGANOxfXvNr9gD/2YbzGwiuFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASM3v9QHAUBqNRnGmqqoROJI9TZgwoThzyimn1NrrwQcfrJUrVed8jxkzpjiza9eu4szers65q2ukXuOuFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYBkIB57vaam8u9dBgYGijOHHXZYceaSSy4pzuzYsaM4ExHR3d1dnOnt7S3OrFq1qjgzmsPt6gydq/MaqrPPaJ6HOkMIh8OVAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJAMxGOvV2fwV52BeAsWLCjOfPSjHy3OvPrqq8WZiIi2trbizLhx44ozp59+enHmhz/8YXFmw4YNxZmIiKqqijN1Xg91dHR01MoNDg4WZ3p6emrtNRRXCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEAyEI+9Xn9//6jsc9xxxxVnurq6ijN1BvxFRDQ1lX8P9/DDDxdnjjnmmOLMtddeW5xZvXp1cSYi4rnnnivOPP/888WZj3zkI8WZOq+hiIiVK1cWZ5588slaew3FlQIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQDMRj1DQajVq5qqqKM6effnpx5thjjy3ObNu2rTgzfvz44kxExMyZM0cl89RTTxVn1qxZU5zp6OgozkREnHjiicWZ8847rzizc+fO4kydcxcRcckllxRn+vr6au01FFcKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKAKRGNcwRlHUnXLL329uf2zpTUn/3u98VZ7q6uoozddQ937t27SrO9Pf319qrVG9vb3FmcHCw1l6///3vizN1prjWOd8f//jHizMREe9///uLM+973/uKM8N5L7lSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAICkFAFLze30AvPfqDJzb223evLk4c+CBBxZnduzYUZxpa2srzkRENDeXv107OjqKM3WG27W3txdn6g7EmzNnTnHmpJNOKs40NZV/zzx16tTiTETEQw89VCs3ElwpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAMlAPP4rjRs3rjhTZwBanUxPT09xJiJi69atxZk333yzONPV1VWcqTNUsdFoFGci6p3zOq+HgYGB4kzdIX/Tp0+vlRsJrhQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGAZCAetQaT1RlKVmfAWERER0dHcaazs7M409fXNyqZtra24kxERH9/f3GmzvC9/fbbrzhTZ/BenSF1ERGtra3FmW3bthVn9t133+LMs88+W5yJqPcaP/bYY2vtNRRXCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkU1KJqqqKM2PGjCnO1J2SumjRouLMtGnTijNvvPFGcaa9vb04Mzg4WJyJiBg/fnxxZvr06cWZOtNY60x+3blzZ3EmIqK5ufzLVp3nadKkScWZW265pTgTEXH00UcXZ+qch+FwpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgCkRjXMaWiNRmOkj4X3SJ3BWrt27RqBI3lnxx9/fHHm5z//eXFmx44dxZnRHAw4YcKE4kxvb29x5s033yzOtLS0jEomot5gwM2bN9faq1Sd8x0Rcd111xVn7rjjjuLMcL7cu1IAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAUvkktBFWd/BencFkTU3lnVjn+Hbu3FmcGRwcLM7UNZrD7ep44IEHijPd3d3FmToD8VpbW4szw5xBuYc33nijOFPnfTF27NjiTJ3XeF2j9X6qc+4+/OEPF2ciIrZu3VorNxJcKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBpRAfi1RkoNTAwUGuvvX2o297s1FNPLc586lOfKs6cfPLJxZmIiJ6enuLMm2++WZypM9yuubn8LVT3NV7nPNR5D7a1tRVn6gzRqzsYsM55qKPO62H79u219jrvvPOKM/fff3+tvYbiSgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABIjWqYU6kajcZIH8uo23///YsznZ2dxZkZM2aMyj4R9QZrzZw5szjT19dXnGlqqvc9yM6dO4sz7e3txZn169cXZ1paWoozdQatRURMmjSpONPf31+cGTduXHFm5cqVxZmOjo7iTES9AY6Dg4PFma1btxZn6rweIiI2bNhQnJk1a1ZxZjhf7l0pAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJBGdErqCSecUJz5+te/XpyJiJgyZUpxZr/99ivODAwMFGfGjBlTnNmyZUtxJiJi165dxZk6UzHrTN+sO2l3x44dxZnnn3++OHPBBRcUZ1avXl2cmTBhQnEmImLixInFma6urlp7lXr55ZeLM3XPw7Zt24ozPT09xZk6k3brTn7dZ599ijN13rempAJQRCkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQhj0Qr7m5ufjBn3zyyeLMgQceWJyJqDeork6mzmCtOuoM0YuoNzxutOy77761cpMnTy7OLF68uDhzxhlnFGe++MUvFmfWr19fnImI6O3tLc6sXbu2OFNnuN2MGTOKM5MmTSrORNQbxtjS0lKcqTOwr84+ERGDg4PFmYMPPrg4YyAeAEWUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAGnYA/Euuuii4gf/9re/XZx56aWXijMRER0dHaOSaWtrK87UUXewVp2hc3/5y1+KM3WGuk2ZMqU4ExHR1FT+vcu0adOKM5/85CeLM2PHji3OdHV1FWci6r1eZ8+ePSqZOs9RncF2dfdqbW2ttVepRqNRK1fn/X7CCScUZ/785z8PucaVAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCah7tw48aNxQ9eZ9DahAkTijMREX19fcWZOsdXZyhZnWFc++yzT3EmIuKtt94qzrzyyivFmTrnYceOHcWZiIje3t7izK5du4ozd999d3HmueeeK87UHYi3//77F2fqDJ3bsmVLcWbnzp3FmTrPUUTE4OBgcabOwLk6+9QdiFfna8TMmTNr7TUUVwoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAGvZAvNdee634wauqKs68+uqrxZmIiPHjxxdnJk+eXJypMyxs06ZNxZk33nijOBMR0dw87Kc0tbW1FWfqDBgbO3ZscSai3pDEpqby73fqPE+zZs0qznR3dxdnIuoNcNy8eXNxps7roc65qzNEL6LeIL06e7W3txdnpk2bVpyJiNi6dWtx5uijj66111BcKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQhj1S85lnnil+8BUrVhRnLrroouJMRMT69euLMy+//HJxpre3tzjT0dFRnKkzhTSi3mTH1tbW4syYMWOKM319fcWZiIiBgYHiTJ0JvT09PcWZ119/vThT59gi6p2HOlNzR+s13t/fX5yJqDepuE6mzmTVOhNcIyIOOeSQ4syGDRtq7TUUVwoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAalTDnM7VaDRG+lgiIuLMM8+slfvyl79cnJk6dWpxZtOmTcWZOsO46gw/i6g3qK7OQLw6g9bqHFtEvddenaFzdYYQ1snUOd919xqt922dfUZqoNs7qXPOBwcHizPTpk0rzkREPPvss8WZCy64oDgznPeFKwUAklIAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgDXsgXp1hZnUGSo2m+fPnF2e+9a1vFWfqDN7bd999izMREU1N5T1f57mtMxCv7pC/OjZu3FicqTNE77XXXivO1H1fbN++vThTdwhhqTrnbufOnbX26unpKc7UeV888sgjxZnnn3++OBMRsXLlylq5UgbiAVBEKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCGPRCv0WiM9LHwDw4//PBaucmTJxdntmzZUpw56KCDijPr1q0rzkTUG5z20ksv1doL/psZiAdAEaUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJFNSAf5HmJIKQBGlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKAKTm4S6sqmokjwOAvYArBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUA0v8DLIGL+5XJ9CsAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "image, label = train_data[0]\n",
    "print(f\"Image shape: {image.shape}\")\n",
    "plt.imshow(image.squeeze(), cmap=\"gray\")\n",
    "plt.title(class_names[label])\n",
    "plt.axis(False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "26efe8de-15b7-4bbc-87fc-4c269a98c46c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAALfCAYAAAB1k5QvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAACmVUlEQVR4nOzdd3xVVb7//08MJISEhBYICZBA6EVQQLAgRRAVRB1QYdQBbIyKZcYZv5Y7V51Rx4qoWOfnKCIOlgErqKioI+hgAwWl9xpK6E1h//7wQa5hvddmHxJIez0fj3ncy4e1zt5nn7XXWR72Z33igiAIDAAAAIB0TEmfAAAAAFCasWAGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEJU+AXz0KFDLSUl5ZDtunfvbt27dy+243bv3t3atGlTbK8HFFVcXJyNGDHikO2ef/55i4uLs6VLlx75kwKAco51SNlQJhfMTzzxhMXFxVnnzp1L+lTKpHvuucdef/31kj4NHEXff/+9DRw40LKzs61KlSqWlZVlvXv3tscee+yIH5vxhqPhwH/I/fp/derUsR49etjkyZNL+vRQzrAOKZqy+L1QJhfM48aNs5ycHJsxY4YtXLiwpE+nzCmLAxWHb/r06daxY0ebNWuWXXHFFTZ69Gi7/PLL7ZhjjrFHHnkk5te75JJLbNeuXZadnR2pPeMNR9Nf//pXGzt2rL3wwgt200032fr16+2ss86yt99+u6RPDeUI65CiKYvfC5VK+gRitWTJEps+fbpNmDDBhg8fbuPGjbPbb7+9pE8LKLXuvvtuS0tLsy+//NKqV69e6O/y8vJifr34+HiLj48PbRMEge3evduSkpJifn2gKM4880zr2LFjwZ8vu+wyq1u3rv3rX/+yfv36leCZobxgHVIxlblfmMeNG2c1atSwvn372sCBA23cuHFOm6VLl1pcXJw9+OCD9swzz1hubq4lJiZap06d7MsvvzzkMWbOnGnp6enWvXt32759u7fdnj177Pbbb7cmTZpYYmKiNWjQwG666Sbbs2dP5Pfz9ddf20knnWRJSUnWqFEje+qpp5w2eXl5BZN+lSpVrF27djZmzBin3Y4dO+zGG2+0Bg0aWGJiojVv3twefPBBC4KgoE1cXJzt2LHDxowZU/DPlkOHDo18vih7Fi1aZK1bt3YWy2ZmderUcWKvv/66tWnTxhITE61169b27rvvFvp79QxzTk6O9evXz9577z3r2LGjJSUl2dNPP814Q4mrXr26JSUlWaVK//f70IMPPmgnnXSS1apVy5KSkqxDhw722muvOX137dpl1113ndWuXduqVatm/fv3t1WrVllcXJzdcccdR/FdoDRhHVJB1yFBGdOiRYvgsssuC4IgCD799NPAzIIZM2YUarNkyZLAzILjjjsuaNKkSXDfffcF999/f1C7du2gfv36wd69ewvaDhkyJEhOTi7484wZM4IaNWoEvXv3Dnbu3FkQ79atW9CtW7eCP+/bty84/fTTg6pVqwY33HBD8PTTTwcjRowIKlWqFJxzzjmHfB/dunULMjMzgzp16gQjRowIHn300eCUU04JzCx49tlnC9rt3LkzaNmyZVC5cuXgD3/4Q/Doo48GXbt2DcwsGDVqVEG7/fv3Bz179gzi4uKCyy+/PBg9enRw9tlnB2YW3HDDDQXtxo4dGyQmJgZdu3YNxo4dG4wdOzaYPn36oS88yqzTTz89qFatWvD999+HtjOzoF27dkG9evWCv/3tb8GoUaOCxo0bB1WrVg02bNhQ0O65554LzCxYsmRJQSw7Ozto0qRJUKNGjeDmm28OnnrqqWDq1KmMNxw1B8blBx98EKxfvz7Iy8sLZs+eHQwfPjw45phjgvfff7+gbf369YOrr746GD16dDBy5MjghBNOCMwsePvttwu95gUXXBCYWXDJJZcEjz/+eHDBBRcE7dq1C8wsuP3224/yO0RpwTqkYq5DytSC+auvvgrMLJgyZUoQBL98OPXr1w+uv/76Qu0ODNRatWoFmzZtKoi/8cYbgZkFb731VkHs1wP1s88+C1JTU4O+ffsGu3fvLvSaBw/UsWPHBsccc0zwn//8p1C7p556KjCzYNq0aaHvpVu3boGZBQ899FBBbM+ePUH79u2DOnXqFNxMo0aNCswsePHFFwva7d27NzjxxBODlJSUYOvWrUEQBMHrr78emFlw1113FTrOwIEDg7i4uGDhwoUFseTk5GDIkCGh54fy4/333w/i4+OD+Pj44MQTTwxuuumm4L333is0YQfBLwvmhISEQmNl1qxZgZkFjz32WEHMt2A2s+Ddd991js94w9FwYFwe/L/ExMTg+eefL9T214uQIPhlTm3Tpk3Qs2fPgtjXX3/tfNEHQRAMHTqUBXMFxjrkFxVxHVKmHskYN26c1a1b13r06GFmv/ysf+GFF9r48eNt3759TvsLL7zQatSoUfDnrl27mpnZ4sWLnbZTp061Pn362GmnnWYTJkywxMTE0HN59dVXrWXLltaiRQvbsGFDwf969uxZ8HqHUqlSJRs+fHjBnxMSEmz48OGWl5dnX3/9tZmZTZo0yTIyMmzw4MEF7SpXrmzXXXedbd++3T755JOCdvHx8XbdddcVOsaNN95oQRCQJV6B9e7d2z7//HPr37+/zZo1y+6//37r06ePZWVl2Ztvvlmoba9evSw3N7fgz8cee6ylpqbKe+ZgjRo1sj59+hT7+QOxePzxx23KlCk2ZcoUe/HFF61Hjx52+eWX24QJEwra/PrZ+vz8fNuyZYt17drVvvnmm4L4gUeRrr766kKvf+211x7hd4DSjHXILyriOqTMLJj37dtn48ePtx49etiSJUts4cKFtnDhQuvcubOtW7fOPvzwQ6dPw4YNC/35wKDNz88vFN+9e7f17dvXjjvuOHvllVcsISHhkOezYMECmzNnjqWnpxf6X7NmzcwsWjJVZmamJScnF4od6H/g+dBly5ZZ06ZN7ZhjCn9ULVu2LPj7A/83MzPTqlWrFtoOFVOnTp1swoQJlp+fbzNmzLBbbrnFtm3bZgMHDrQffvihoN3B94zZL/fNwfeM0qhRo2I9Z+BwnHDCCdarVy/r1auXXXTRRfbOO+9Yq1atbMSIEbZ3714zM3v77betS5cuVqVKFatZs6alp6fbk08+aVu2bCl4nWXLltkxxxzjjOsmTZoc1feD0oN1SMVeh5SZXTI++ugjW7NmjY0fP97Gjx/v/P24cePs9NNPLxTzZfIHv3r43MwsMTHRzjrrLHvjjTfs3XffjZRJvX//fmvbtq2NHDlS/n2DBg0O+RrA0ZaQkGCdOnWyTp06WbNmzWzYsGH26quvFmR4R71nFHbEQGl0zDHHWI8ePeyRRx6xBQsW2KZNm6x///526qmn2hNPPGH16tWzypUr23PPPWcvvfRSSZ8uSjHWIRVbmVkwjxs3zurUqWOPP/6483cTJkywiRMn2lNPPXVYX9pxcXE2btw4O+ecc+z888+3yZMnH7KaTm5urs2aNctOO+00i4uLi/mYZmarV6+2HTt2FPqvu/nz55vZL7sOmJllZ2fbd999Z/v37y/0X3dz584t+PsD//eDDz6wbdu2Ffqvu4PbHXi/wIGtt9asWXNEj8N4Q0n7+eefzcxs+/bt9u9//9uqVKli7733XqF/8n7uuecK9cnOzrb9+/fbkiVLrGnTpgVx9tytuFiHVOx1SJl4JGPXrl02YcIE69evnw0cOND534gRI2zbtm3O85ixSEhIsAkTJlinTp3s7LPPthkzZoS2v+CCC2zVqlX2j3/8Q57vjh07DnnMn3/+2Z5++umCP+/du9eefvppS09Ptw4dOpiZ2VlnnWVr1661l19+uVC/xx57zFJSUqxbt24F7fbt22ejR48udIyHH37Y4uLi7MwzzyyIJScn2+bNmw95figfpk6dKn8hnjRpkpmZNW/e/Igen/GGkvTTTz/Z+++/bwkJCdayZUuLj4+3uLi4Qs+bLl261CmicOB5/CeeeKJQ/GhUx0TpwzqEdUiZ+IX5zTfftG3btln//v3l33fp0sXS09Nt3LhxduGFFx72cZKSkuztt9+2nj172plnnmmffPKJt876JZdcYq+88or9/ve/t6lTp9rJJ59s+/bts7lz59orr7xSsB9tmMzMTLvvvvts6dKl1qxZM3v55Zdt5syZ9swzz1jlypXNzOzKK6+0p59+2oYOHWpff/215eTk2GuvvWbTpk2zUaNGFfxX3Nlnn209evSw2267zZYuXWrt2rWz999/39544w274YYbCiVydejQwT744AMbOXKkZWZmWqNGjSjvWY5de+21tnPnTjvvvPOsRYsWtnfvXps+fbq9/PLLlpOTY8OGDTuix2e84WiaPHlywS9aeXl59tJLL9mCBQvs5ptvttTUVOvbt6+NHDnSzjjjDPvtb39reXl59vjjj1uTJk3su+++K3idDh062IABA2zUqFG2ceNG69Kli33yyScFv76VxV/IcPhYh7AOKRPbyp199tlBlSpVgh07dnjbDB06NKhcuXKwYcOGgu1cHnjgAaedHbQd0MH7HwZBEGzYsCFo1apVkJGRESxYsCAIAnc7lyD4ZVuV++67L2jdunWQmJgY1KhRI+jQoUNw5513Blu2bAl9T926dQtat24dfPXVV8GJJ54YVKlSJcjOzg5Gjx7ttF23bl0wbNiwoHbt2kFCQkLQtm3b4LnnnnPabdu2LfjDH/4QZGZmBpUrVw6aNm0aPPDAA8H+/fsLtZs7d25w6qmnBklJSYGZlbmtXRCbyZMnB5deemnQokWLICUlJUhISAiaNGkSXHvttcG6desK2plZcM011zj9s7OzC40R37Zyffv2lcdnvOFoUNvKValSJWjfvn3w5JNPFpoHn3322aBp06ZBYmJi0KJFi+C5554Lbr/99uDgr8QdO3YE11xzTVCzZs0gJSUlOPfcc4N58+YFZhbce++9R/stogSxDmEdEhcEEbJ5AACAzZw504477jh78cUX7aKLLirp0wFwlJSJZ5gBADjadu3a5cRGjRplxxxzjJ166qklcEYASkqZeIYZAICj7f7777evv/7aevToYZUqVbLJkyfb5MmT7corr2TLLqCC4ZEMAACEKVOm2J133mk//PCDbd++3Ro2bGiXXHKJ3XbbbVapEr83ARUJC2YAAAAgBM8wAwAAACFYMAMAAAAhWDADAAAAISJnLVDVCEdKST5GXx7GtXoP6pomJyfL/oMGDXJi27dvd2L5+fmyf0ZGhhPbtm2bbDtx4kQZL48Y1yiPGNcoj6KMa35hBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEJQqggohaIm8oXFD9a3b18Zr1GjhhOrXLmyE1PJfWZmbdu2dWItW7aUbY9m0l8s1xAAgDD8wgwAAACEYMEMAAAAhGDBDAAAAIRgwQwAAACEYMEMAAAAhIgLIqaNU5JSa968uRNLT0+XbXft2uXE1G4EZmZ79+6N1Hbfvn2y//79+yPFfMc65hj3v6VUzEyPjWrVqsm23377rRNTZZiPlvIwrlNTU53YgAEDnFjHjh1l/+nTpzux//f//p8TU7thmJmtXr3aif3tb3+TbVV57mXLljmxKVOmyP5btmyR8dKIEsIojxjX5Y/vupbGXYUyMzNlXO3i5FvzzJw504lRGhsAAAAoIhbMAAAAQAgWzAAAAEAIFswAAABACJL+BF9ym3qA/O6773Zi9erVk/337NnjxHwlhFWCX9WqVZ2YStgz0+WOfXbv3u3EKlVyq6avXLlS9ldDyPew/aOPPurEJk2adKhTPGJK67g+9thjndhxxx0n26rP+ueff3ZiDRs2lP3VWFPXJTc3V/Z///33ndiCBQtk2yZNmkQ6vi9pdP369U7MlyC4cOFCGT9aSjJhRs1hsZxPLPdFaUwMwpFD0l/RqPdQ1HtTxXzfwer7olGjRrLtvHnznNiOHTsOdYqhsrOzZbxu3bpOTCWJ+6SlpTkxlRBvZvbaa685sSjvi1+YAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBAsmAEAAIAQ7lYIiCljVe18oXadMNOlsefPny/bVqlSxYnFx8c7sU2bNsn+tWrVcmK+3T8SEhIiHct3XX766ScnlpiYKNsuWrRIxiuqCy64QMZbtWrlxJYsWSLb5ufnOzE1LtTnZKazi1WG9Ycffij7qx05ateuLduqMujqvFS5bTOz6tWrO7EhQ4bItv/+97+dmCqJWtGpe33fvn2R+6uxqsaEj2+nH3UOakcV37ym3pfa/ac4dl5Qc6OK+c41luul7s2kpKTIr6nazp07V7ZV9ytKVlF3KWnevLkTS09Pl2137tzpxHxjRRkwYEDktmoXMDWG1XeAmR6ram1j5l+jHQq/MAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAhSPorIpWw4kuYUQ+g+x5KV4koqm1WVpbsrxJDVMKLmU6OUckpvlKbqq1KzjE7/Ifty4OMjAwn5iuNPmfOHCfmS05Sn59K7PFde5U0qJJGt27dKvurhBFfwpG6N9R5+Upjq7aLFy+Wbdu3b+/EKkrSXyyJQbEk+J1++ulOrG/fvk5s6dKlsv+GDRucWNu2bWVblcSjEqd9c6hKyFb3kC8RL5ZkwKiv67vWURP5zHRpY3W91bUy0+Xpp02bJtu+/vrrMo7DdyRKi6vX9I1fNTf7kvRTUlKcWJ06dZzYZZddJvurublmzZqyrVqfqHtbJSKa6fvFd78d7mfAL8wAAABACBbMAAAAQAgWzAAAAEAIFswAAABACJL+ikglIakkLDOdyOR7AF4lzakH1X1VsmJJOFHnq17Xdyx1rpmZmbKt74H9iqBFixZObPPmzbJtLNXztmzZ4sRiGSvq81Pn5auwpMaVL8FUJTepqpa+RKxt27Y5MV+CoDpflQhzJJJwyrp77rlHxlUinUrOU1UWzfTn75sTTjzxRCemqkLGMt+qseq7L9S5+hKflajzqplOblLX2sxs3rx5Tkxdb1/i7dVXX+3EunXrJtt+9NFHMo7SzzcHq6TRHTt2yLbqHlJJfytXrpT9VRVZ37HUvaE2NfD1V/e2b244XPzCDAAAAIRgwQwAAACEYMEMAAAAhGDBDAAAAIRgwQwAAACEYJeMIkpOTnZivjKTKotTZf2b6UxWleHvK3+qsq59GaMqm1q19WVtq10WfJmsqjRuReErF67EkmFftWpVJ6bGha80thqvajcCXwlitXOAb6wo6rr4xrW6Lr5dFlSGttppZP369Yc6xTJHXVP1mZqZffDBB07s8ccfl23VjiYDBgxwYldddZXsX69ePSe2evVq2VaNoTPPPNOJqTLyZnpcq91XfDtfqPm6qKV2ValhM72jga+8eHZ2thPr2rVr5HNS78F3bzdt2lTGUXKi7vSjdpgwM2vUqJET882B6jsjPT3diW3atEn2z8jIcGK+tYGamzdu3OjEfGNV3du+7zzfDiKHwi/MAAAAQAgWzAAAAEAIFswAAABACBbMAAAAQAiS/opIlY70JZGoB/NVWVkzncikHor3JRimpqY6MV+pVBVXD8v7ks5UMpHvYfuKTCX8rFmzRrZViRnLli2TbVUinEqA8CU6RE3Q8yWG1KpVK9LxfXGVSOZLhlVtVeKtmR6vKmGqPCb9qXuye/fusm3dunWd2MSJE2Xbl19+2YmpcZWbmyv7q88qJydHth05cqQTU/PaySefLPurEtJqbvYl8qm5VSVcmelxrd5rfn6+7P/jjz86MV+SZqtWrZyYSoSKZb5XicNh54DSz5dgqpKhfXOgugd8CdmKWh/5khGjzsO+ZNZYEtXVpgpR8AszAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEIIFMwAAABCCXTKEqGVOzXSGta8Etcrw92VoK+p1fVnMKq4yVs105vnmzZudmG83ArUjhu9YFZnaTWLRokWybbt27ZyYL4tY7SihSiP7MvwVNdZUdrWvrW/nDZWNr3ZJ+M9//iP7H3vssZGPpcawKs1cUQwbNkzGr7322siv0bBhQye2du3ayP3VWK1evbpse9lllzmxv/3tb07MtyNPixYtnJiag33UHOq7h5YvX+7E1q1b58S2bt0q+6vdM3y7h6gxvGDBAifm20FJfb81a9ZMtvWVnUfJiVoa27dDhLpffHO7mkPVmqVx48ayvyqN7btfa9as6cTU+POV1lbU96DZ4e/+wi/MAAAAQAgWzAAAAEAIFswAAABACBbMAAAAQIgKn/QX9QF6H5WA4Uv627JlixPzlZtWiXifffaZE/OVO1bH8iWRqHhSUpIT27Bhg+yfmZnpxL799lvZtiJTCW++JCBVGluVzzXzl2KPSvVXYziW0tq+tirhY+bMmU7MV9Y1Ly/PifnuN5X04ktcLW9U6df+/fvLtkOGDIn8umpeUPOl7/NXn9XSpUtl2w4dOjixwYMHO7FZs2bJ/h999JETO+GEE5zY4sWLZX81t2/cuFG2Pfvss53YtGnTnJhvDlaJVOq9mpl9+OGHTkyNa1+CorrfVXKXmX5fKBt8yXUrV650YtnZ2bKtSuhV8+r27dtlf5X8r+YQM7O5c+c6MZW46tsoQcVjaRsFvzADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIUj6K2LSn0ra8lXYUQl+vmOppKdWrVpFPq9Vq1Y5sZ9//lm2VdW3VIKaSjo0M+vXr58T++abbw51iuWaqpx0zDHuf5/6kqNUYoQvUUG9bixV/VR/dSxfEolvvCuq8lIsyRrqHkpPT5dtVSKTL+GkvPnd737nxCZPnhy5v2/8+KrHRaXmOzX+zHQVzNNOO82J+SoNqsqaqlLh119/Lfu/8MILTsyXIKmSUdVYXbZsmex/+eWXOzGVDGsWvVqhb25R1dN891ubNm0iHQtHT9T1yYoVK2RcJfj5kj5V8rlKzvMlXk+fPt2J+ZLU1fpEzde+xG1Vvc/33XS43wP8wgwAAACEYMEMAAAAhGDBDAAAAIRgwQwAAACEYMEMAAAAhCiVu2QUdeeKI0Wd165du5yYKgdppsvV+jI+VcaoKpftywKtW7du5GMpKmu2c+fOkfsvWLAgctvySJXVVWNFZfaa6c/aR2Udq/Hj2yUl6i4bvjLederUiXROvmOpbH7fuarX9e3coMq1Vq9eXbYtb1R2+5gxYyL39823KkNe7bDgy5pXfLs+RC3hfMUVV8j+U6ZMcWLz5893Yr75+tZbb3VivjlUjbU+ffo4MVWa28zs888/d2Kq5LxZ9F1pfPeg2qlG7Zxhpnf6QNFEXd/4dqqJ2t93D6oduHzHUt9Par5VaxszPdZ88vPznVjNmjWdWFZWluy/cOFCJ7Zjxw7Z1rez0qHwCzMAAAAQggUzAAAAEIIFMwAAABCCBTMAAAAQolQm/R3NBL9YSgir5BT1AL0qK20WvSywmU6OUsfylStWSRy+B+CVBg0aRDq+jy+JpKJQiRHqs/Ylt6lkC5XIaaZLCKvEIF+pXJVcpO6LzMxM2V8lPPnuAZWIpMa6L2FJJe1lZGTItt99950TU5+LLzHFl5BZFqhEzHnz5kXu70sYimW+VGJJblJjQCWhrVy5UvY/5ZRTnJgqietLZv3xxx+dmK9UtLreGzZscGIff/yx7K/uTd9nEPV+iSVpzHes0pBsX95Evaa+dlHvwcaNG8u4mu/T0tJkW5Worr6zfOcUS0KwSsRT78FXXn7jxo2RXtPM/717KPzCDAAAAIRgwQwAAACEYMEMAAAAhGDBDAAAAIQolUl/R1MsSQ1NmzZ1YuoBdl9ynXow3le1SSV2FLXqUixJBMuXL3divuQodQ1UpbuKRFUEUwkQvmu6Zs0aJ6YSlsyij2FfIp36/NT4i6XymO9Yijp/X4KiSsTzJb6qaxtLwotK2ior1PVTCTw+vgRPNVZimZdiSfpTbdXnr8aqmVleXp4TizrWzfR870sQVIlI6r7w3e+xJFOqhCX1HmKZ7333K5X+il9RKxmrz0qNq/r168v+27Ztc2K+5DhVaU9V8fUlSKtz9a0NVPL2Dz/84MR8Sb4q8dZX2XXJkiUyfij8wgwAAACEYMEMAAAAhGDBDAAAAIRgwQwAAACEYMEMAAAAhKhQu2TEUoJX6d27txNTGafVqlWT/VWGdizZySoT1VeCWJWxVmWBfa+hsux9uxGoXUHat28v27711lsyXt5E3TnC95moTHhfGXU1rmIpf6vOS8V8n7/aEcRXelTdg1FjZnqXC1+GuTovdV1UyfuyTn1WsezE4NuRRVHXvzh2aFBzkBpXvnGtxLIbgdpVJJadgtRY9e2KpOZ73+eljqXmhlh2H/FdF3bJiCaWnS+i3oex9D/hhBOcmG9XI7U28H23zJ49O9Lr+nYUat26tRPzfed9++23Mn6wJk2ayLia8zZv3izbHu645hdmAAAAIAQLZgAAACAEC2YAAAAgBAtmAAAAIMQRSfqLJbkkliSMqHxldX0Pth/svPPOk3FVZlG9pi+xQyWM+M4patKeL2FJxX1JACqRRj1Av3PnTtlfJWj5SlJWFOoeiCWRT7X1JZOqtipB1JewFDUZ1pcIpvrHktykkph8iVzqevmuoboH1LXyJaGUZSoZ2Ve+VollrMRS8r2oyWlRy537XleNCV/J8Kiluc30/RJ1DjDT18WXDBlLkqyiziGWxEu4ilraWvF9ps2aNXNiar7duHGj7J+Tk+PEfMlxah5u1KiRE/OV4Vb9Fy1aFLmtel/5+fmyv5obfHN7LInCv8YvzAAAAEAIFswAAABACBbMAAAAQAgWzAAAAEAIFswAAABAiMgpsLGUflRxX8Zn1Oxe37FUxqkvE1np0aOHEzv22GNl25UrVzoxVQK6Ro0asr/aZcK384HK5lYZ2r7sVpVJ6ruGqjS2yjj1nasaGzVr1pRtK4qon58vW1fdF75xrY4Vtdy1L66y433nqsa1LztZjRV1XXzZ+aq/b5eFzMxMJ7Z9+3YndrgZ06WZmquuvPJK2fbvf/+7E/ONtagZ/r6dS9S49n1+UXeOiGWHCPX5xzIH+6xbt86JxbJzgmrruy7qGqhr5buH1P0Wy3dmSYplHRK1v8+R2MHLzCw1NdWJqblK7WZhps9rx44dTqx58+ayv/puV7vqmOldjdRY882h6nXVPWimPxt1LLWDmJnZpk2bnNiaNWtk26jz2MH4hRkAAAAIwYIZAAAACMGCGQAAAAjBghkAAAAIcUSS/pTDfcj6cPhK+Pbr18+JNWnSxImph8fNzGrXru3E1MP2sVyXrVu3yrgqa6muYSyJl77EDvVgf4MGDZxYLMk5KomhIomaiOdLzIllDKnyoaq0ue81VdKUStpbu3at7K+SSHylsVU8lnLJaqz67iE1BlWC4tGcm46W7777zok99dRTsq1K+vNd0+zsbCc2e/ZsJ+ZLAopaGj0sfjDfuFbJbXXq1HFivtLYeXl5Tkydv5lZRkaGE1MJ2b7kqliS0aLyJV6qUuK+a+grO16axFJavaiJfL4E0+TkZCfWuHFj2bZ69epOTI0LX8Ka6q++b9TaxCy2z1/NjXXr1nVivu8GtWZS94pZ9CTbxYsXy7h6v6+88opsm5ubG+lYB+MXZgAAACAEC2YAAAAgBAtmAAAAIAQLZgAAACBE5KQ/9fB3rVq1ZNuWLVu6B4qh6pB6iF8lFpnpB9DVg+a+11APq6vEEDOdNDd37lwnphL2zMzq1asX6TXNdOKiivmSEGKp5qReQyU++j4Dlciljl+RqGut7gFVZdHMbMWKFU5MJayZRa8qGEvCi+rvq4imkot8yUK+CoBRqbHqSxaJmgjj+wzKsnfeeceJff/997LtSSed5MSmT58u26oxrOYF31hTc7vv8ytq9TnVX31nXXTRRbL/zJkzIx/rf/7nf5xYnz59nJhKJDTT19CXtOebh6NSyZQqSdjMn/xZUopa1U8lzJnpyrTp6elOzDd/RU2cNtPzeP369Z2YLxl1/fr1TiwtLc2JbdmyRfZXiavqNc3MOnXq5MTU94BvzaW+B3xrAzUG1ToklqrRvnvlcOcWfmEGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEJE3iVDadq0qYyr8qm+TPSi7gaxYcMGJ+bLLt2+fbsTU5m0qp1Z9AxxVRLVTGeH+jJp1e4fsZyryoRVu3SYmaWmpjqx1atXOzHfZ6gyYYuayV3WqWxudZ1SUlJkf7WjgW9XGpUhrcagb6caNa7Urji+zzRqCWMzfQ+oezuW0ti+MtzqPahdMirKWL3ppptk/A9/+IMT8+2S8fbbbzuxY4891omp7HYzvSOGL2O9qOWio+5coHakMdP3i+9c1W4S6li+nS/UdYnlflX3oO9c1dzk+379z3/+I+OlSY0aNWRcza2+uUp9LsuXL3divtLoiu+aqnNQ362+8a/mO7Xm8e1wotYBXbp0kW3VWiwrK8uJqTWEmdmSJUucmG+3JzVfq2vlK62tvjM///xz2da3dj0UfmEGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQkRO+lPlort27Srbrlu3zon5HmBXSTjbtm1zYsnJybK/KnXpS6SLWi7Yl4ilEgPUw+6+Uq8qYcCXRKAejPcl+CnqHGJJjlCJBSoBwHdeKjHAzF9Cs7xR40olNfg+k08++cSJdejQQbZVyYBREyh8bdW9otr5+BKWVCKMOi/ffKHaquQcM7MmTZo4MfUeilquuzRS12/27NmyrUoQffPNN2VbNQeq6+e7z1Uimi85Sn1W6vixlJBetmyZEzv99NNl/x9//NGJ+RLp1PfT4sWLnZhvrKn36ruHFHW/+pJZVdKUSp43M/vss88in0NJadWqlYxnZmY6MV9ZZpU0p74Dfd/Xah3ja6vGihrDvu9bNS7U973vvarvC9+5qpLd6rzmzJkj+6vy3Kq0tplOXlffo7731bBhQyf2+OOPy7a+pOZD4RdmAAAAIAQLZgAAACAEC2YAAAAgBAtmAAAAIAQLZgAAACBE5DTczp07O7F+/frJtj/88IMT85VpVNmZapeNVatWyf4qi9O3m4TKrlQ7RKjMTDOdNauym9UuH2Y6O1adv5nOGI0lu1XxlRBWmd9q9wbfLgnqdX2Z62q3lfJIXROVyezbDWLjxo1OzLfLhdplwJdhraj7Qp2r7zP1ZS0rapcA1V+VETfT49KXoX3iiSc6MTWGd+/eLfuXZSqT3vc5/fe//3ViJ5xwgmy7dOlSJ6Y+K98uGerz981halyrmG9eU/eWmptHjBgh+6tsfrWbgpneJUHFfCWEfTuFKGoeUP19JYg//fRTJ3bnnXdGPn5p45sX1ZrDNy+qeSGWXanq1q3rxHzfdfn5+U5MvQfffdGxY0cnpkprr169WvZX96DaUcRMz/nz5893Yr4dXdT19r0vdW+q1/XdK+pc1c4ZvvOKgl+YAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBCRk/7eeustJ+ZLtjj33HOdWLNmzWRblXCmknDWrl0r+6sHxX0P66tEKlU+1FeGW8VVaW6VAGCmS5L6kiHVNRg7dqwTu+CCC2R/lczoK+vqK+V9MF8ypEpYUEkIZvp6lUdqrKl7wJcAofr7kh3UGFYJEL77Vb2uek1fyfg1a9Y4MV/5U0WNH18ij0pY8ZV8VonCKpl2wYIFhzrFMsd3rypqrKmyzj5qvvV9/moM+dqqcak+f3X+ZnoMqfti2rRpsr+aL30J3b7xejA1/sx0MqEvSVPdb6qM9/fffy/7+75zFF9ScklR81JWVpZsq75r1q9fL9tmZ2c7MZVg7PucYykBrb4bo34H+6jS5jVr1pRt1Vjzra9UMmTLli2dmG9MqfsllpLfah7zfWeqser7vHwJ7IfCL8wAAABACBbMAAAAQAgWzAAAAEAIFswAAABAiMhJf8q///3vyPEWLVrItoMHD3ZijRs3dmJNmzaV/dUD9L4HvVUShXpQ3JfooOKbN292Yr7KY3/5y1+cmC/hJKqHH35YxtV5+ZIhoybH+Cr9qevqSzBs0KCBjJc3USvK+ZKIFF/Sn0rmiyWRTlFJKL6kQfX5+5LOot5vvveq4r5KfSqJRB1LzTdmumJpWeG7/5QPP/zQiX300UeyrUqOUuPCV9FOJU77EoTVeFWfXyzvdcWKFU7MlzRaXsVSVdA355eU5s2bOzFf9cXly5c7Md93u/q+VGPFNwcqvnnJt6nAwVSCo5lOsFPvy1ctVX2mvvtV3VvqO9yXyBdLJWJ1v6trGMt3SyxrwSj4hRkAAAAIwYIZAAAACMGCGQAAAAjBghkAAAAIwYIZAAAACFGkXTJ8WZwqA3Hu3Lmy7e233x7pWL7sVpU1W69ePdm2du3aTkxl0vvKRK5evdqJzZs3T7Y9Wn7/+9/L+Lp165yYKp9pprNmVYatLxtZZcL6SmXm5eU5sfHjx8u2ZZnaFUaVcPWV9VVmzpwp4+3bt3dialz7MpbV56+yi3391Y4avuxk9Roqa79WrVqyv8pm91HnkJmZWaTXrCh8mehLly49uieCYlfadr6Ihfr+GDBggGyr5kDfzhFqN4hYdopSc5ivrXpdtfuGb/cYtSOFaqvKyPv4zlXN1zt37nRivl0nYrmGO3bscGLqPRRHuWvf/HYo/MIMAAAAhGDBDAAAAIRgwQwAAACEYMEMAAAAhIgLIj797Eu6A4rqcB/ALw5HalyrJI7q1as7MV8ChC9BU+ndu7cTO+2005zYqlWrIh8rIyPDifnOdeXKlU4sJSVFtlXJMdWqVXNiKrHEzOyFF15wYrGUX1Wf95Eaf+VxXAOlbVyrpGMzneBbo0YN2VaVhlaJbL4S0CruS0JTmyWoY/lKvqv5btu2bU7MN1+ra+jbwEElTvraFpW6Xir5O5brkp+fL9vOmDHDiUUZ1/zCDAAAAIRgwQwAAACEYMEMAAAAhGDBDAAAAIRgwQwAAACEYJcMlLjSlnVdHqjs4jZt2si2NWvWdGIqa9y3G4Xa0cKXSa2ywVV5+blz58r+ZQnjGuUR4xrlEbtkAAAAAEXEghkAAAAIwYIZAAAACMGCGQAAAAgROekPAAAAqIj4hRkAAAAIwYIZAAAACMGCGQAAAAjBghkAAAAIwYIZAAAACMGCGQAAAAjBghkAAAAIwYIZQLF5/vnnLS4uzpYuXRpz36FDh1pOTk6xnxMAwMV8HZtyvWCOi4uL9L+PP/64pE8VOGzff/+9DRw40LKzs61KlSqWlZVlvXv3tscee6ykTw04KhYtWmTDhw+3xo0bW5UqVSw1NdVOPvlke+SRR2zXrl1H5JgvvfSSjRo16oi8Nsov5uuyq1JJn8CRNHbs2EJ/fuGFF2zKlClOvGXLlkfztIBiM336dOvRo4c1bNjQrrjiCsvIyLAVK1bYF198YY888ohde+21JX2KwBH1zjvv2Pnnn2+JiYn2u9/9ztq0aWN79+61zz77zP785z/bnDlz7Jlnnin247700ks2e/Zsu+GGG4r9tVE+MV+XbeV6wXzxxRcX+vMXX3xhU6ZMceIH27lzp1WtWvVIntoRsWPHDktOTi7p08BRdPfdd1taWpp9+eWXVr169UJ/l5eXVzInBRwlS5YssUGDBll2drZ99NFHVq9evYK/u+aaa2zhwoX2zjvvlOAZAv+H+bpsK9ePZETRvXt3a9OmjX399dd26qmnWtWqVe3WW281s18G8GWXXWZ169a1KlWqWLt27WzMmDGF+n/88cfysY6lS5daXFycPf/88wWxtWvX2rBhw6x+/fqWmJho9erVs3POOcd5fmjy5MnWtWtXS05OtmrVqlnfvn1tzpw5hdoMHTrUUlJSbNGiRXbWWWdZtWrV7KKLLiq264KyYdGiRda6dWtn8jUzq1OnTsH//9xzz1nPnj2tTp06lpiYaK1atbInn3zS6ZOTk2P9+vWzzz77zE444QSrUqWKNW7c2F544QWn7Zw5c6xnz56WlJRk9evXt7vuusv279/vtHvjjTesb9++lpmZaYmJiZabm2t/+9vfbN++fUV786jw7r//ftu+fbs9++yzhRbLBzRp0sSuv/56MzP7+eef7W9/+5vl5uZaYmKi5eTk2K233mp79uwp1CfKeO3evbu98847tmzZsoJH+yra85yIHfN12Vauf2GOauPGjXbmmWfaoEGD7OKLL7a6devarl27rHv37rZw4UIbMWKENWrUyF599VUbOnSobd68uWASjsWAAQNszpw5du2111pOTo7l5eXZlClTbPny5QWT7dixY23IkCHWp08fu++++2znzp325JNP2imnnGLffvttoUn5559/tj59+tgpp5xiDz74YJn8VRxFk52dbZ9//rnNnj3b2rRp42335JNPWuvWra1///5WqVIle+utt+zqq6+2/fv32zXXXFOo7cKFC23gwIF22WWX2ZAhQ+yf//ynDR061Dp06GCtW7c2s1/+469Hjx72888/280332zJycn2zDPPWFJSknPs559/3lJSUuyPf/yjpaSk2EcffWT/+7//a1u3brUHHnigeC8IKpS33nrLGjdubCeddNIh215++eU2ZswYGzhwoN1444323//+1/7+97/bjz/+aBMnTixoF2W83nbbbbZlyxZbuXKlPfzww2ZmlpKScmTeJMoN5usyLqhArrnmmuDgt9ytW7fAzIKnnnqqUHzUqFGBmQUvvvhiQWzv3r3BiSeeGKSkpARbt24NgiAIpk6dGphZMHXq1EL9lyxZEphZ8NxzzwVBEAT5+fmBmQUPPPCA9/y2bdsWVK9ePbjiiisKxdeuXRukpaUVig8ZMiQws+Dmm2+O/P5R/rz//vtBfHx8EB8fH5x44onBTTfdFLz33nvB3r17C7XbuXOn07dPnz5B48aNC8Wys7MDMws+/fTTglheXl6QmJgY3HjjjQWxG264ITCz4L///W+hdmlpaYGZBUuWLAk99vDhw4OqVasGu3fvLogNGTIkyM7OjvzeUbFt2bIlMLPgnHPOOWTbmTNnBmYWXH755YXif/rTnwIzCz766KOCWNTx2rdvX8YrYsJ8XbZV+EcyzMwSExNt2LBhhWKTJk2yjIwMGzx4cEGscuXKdt1119n27dvtk08+iekYSUlJlpCQYB9//LHl5+fLNlOmTLHNmzfb4MGDbcOGDQX/i4+Pt86dO9vUqVOdPldddVVM54HypXfv3vb5559b//79bdasWXb//fdbnz59LCsry958882Cdr/+JWHLli22YcMG69atmy1evNi2bNlS6DVbtWplXbt2Lfhzenq6NW/e3BYvXlwQmzRpknXp0sVOOOGEQu3UY0G/Pva2bdtsw4YN1rVrV9u5c6fNnTu3aBcAFdbWrVvNzKxatWqHbDtp0iQzM/vjH/9YKH7jjTeamRV6zpnxiiOF+bpsY8FsZllZWZaQkFAotmzZMmvatKkdc0zhS3RgR41ly5bFdIzExES77777bPLkyVa3bl079dRT7f7777e1a9cWtFmwYIGZmfXs2dPS09ML/e/99993kgIqVapk9evXj+k8UP506tTJJkyYYPn5+TZjxgy75ZZbbNu2bTZw4ED74YcfzMxs2rRp1qtXL0tOTrbq1atbenp6wbP6B0/ADRs2dI5Ro0aNQv+hd+D+OFjz5s2d2Jw5c+y8886ztLQ0S01NtfT09ILE24OPDUSVmppqZr98qR/KsmXL7JhjjrEmTZoUimdkZFj16tULzeeMVxxJzNdlF88wm8nneKKKi4uTcfWA/A033GBnn322vf766/bee+/ZX/7yF/v73/9uH330kR133HEFD+CPHTvWMjIynP6VKhX+uBITE50FPSquhIQE69Spk3Xq1MmaNWtmw4YNs1dffdUuvvhiO+2006xFixY2cuRIa9CggSUkJNikSZPs4YcfdhI/4uPj5esHQRDzOW3evNm6detmqamp9te//tVyc3OtSpUq9s0339j/+3//TyadAFGkpqZaZmamzZ49O3If33x9AOMVRwvzddnDgtkjOzvbvvvuO9u/f3+hRemBf5LIzs42s1/+S87sl4H2a75foHNzc+3GG2+0G2+80RYsWGDt27e3hx56yF588UXLzc01s1+yZXv16lXcbwkVSMeOHc3MbM2aNfbWW2/Znj177M033yz0a4R6xCeq7Ozsgn8R+bV58+YV+vPHH39sGzdutAkTJtipp55aEF+yZMlhHxs4oF+/fvbMM8/Y559/bieeeKK3XXZ2tu3fv98WLFhQaN/9devW2ebNmwvm81jG66EW30BUzNdlAz9Pepx11lm2du1ae/nllwtiP//8sz322GOWkpJi3bp1M7NfBmJ8fLx9+umnhfo/8cQThf68c+dO2717d6FYbm6uVatWrWBboz59+lhqaqrdc8899tNPPznntH79+mJ5byg/pk6dKn9JOPDMZvPmzQt+gfh1uy1btthzzz132Mc966yz7IsvvrAZM2YUxNavX2/jxo0r1E4de+/evc79ARyOm266yZKTk+3yyy+3devWOX+/aNEie+SRR+yss84yM3Mq840cOdLMzPr27WtmsY3X5OTkCv9P1IgN83XZxi/MHldeeaU9/fTTNnToUPv6668tJyfHXnvtNZs2bZqNGjWqINEkLS3Nzj//fHvssccsLi7OcnNz7e2333aeN54/f76ddtppdsEFF1irVq2sUqVKNnHiRFu3bp0NGjTIzH75J8Ynn3zSLrnkEjv++ONt0KBBlp6ebsuXL7d33nnHTj75ZBs9evRRvxYova699lrbuXOnnXfeedaiRQvbu3evTZ8+3V5++WXLycmxYcOG2bp16ywhIcHOPvtsGz58uG3fvt3+8Y9/WJ06dWzNmjWHddybbrrJxo4da2eccYZdf/31BdsUHfiXmQNOOukkq1Gjhg0ZMsSuu+46i4uLs7Fjxx7WPxcCB8vNzbWXXnrJLrzwQmvZsmWhSn/Tp08v2Ar0+uuvtyFDhtgzzzxT8M/OM2bMsDFjxti5555rPXr0MLPYxmuHDh3s5Zdftj/+8Y/WqVMnS0lJsbPPPvtoXwKUIczXZVzJbM5RMnzbyrVu3Vq2X7duXTBs2LCgdu3aQUJCQtC2bduCbeJ+bf369cGAAQOCqlWrBjVq1AiGDx8ezJ49u9C2chs2bAiuueaaoEWLFkFycnKQlpYWdO7cOXjllVec15s6dWrQp0+fIC0tLahSpUqQm5sbDB06NPjqq68K2gwZMiRITk4+/IuBcmHy5MnBpZdeGrRo0SJISUkJEhISgiZNmgTXXnttsG7duoJ2b775ZnDssccGVapUCXJycoL77rsv+Oc//+lsKZSdnR307dvXOU63bt2Cbt26FYp99913Qbdu3YIqVaoEWVlZwd/+9rfg2WefdV5z2rRpQZcuXYKkpKQgMzOzYCslO2g7xoq4TRGKx/z584MrrrgiyMnJCRISEoJq1aoFJ598cvDYY48VbIX1008/BXfeeWfQqFGjoHLlykGDBg2CW265pdBWWUEQfbxu3749+O1vfxtUr149MDPGLg6J+bpsiwsC/tMBAAAA8OEZZgAAACAEC2YAAAAgBAtmAAAAIAQLZgAAACAEC2YAAAAgBAtmAAAAIAQLZgAAACBE5Ep/cXFxR/I8SszGjRud2IYNG2Tb/fv3O7GUlBQnNn/+fNm/Ro0aTqxy5cqy7fbt251YzZo1ndjMmTNl/wsvvFDGS6OS3Aq8vI5rlDzG9dFx6qmnynjPnj2dWNWqVZ1YlSpVZH9V9nr58uWy7bPPPuvE1PdFecC4RnkUZVzzCzMAAAAQggUzAAAAEIIFMwAAABAiLoj4QFJpfXZInZfvLTVv3tyJzZ0714mtXLlS9o+Pj3diiYmJTsz37NqaNWsi9ffFt23b5sT27t0r+3fo0EHGSyOeiUN5xLh2xTJfK6tWrXJial420/PwMce4vxElJyfL/iq/xXes+vXrO7FTTjnFiU2bNk32L0sY1yiPeIYZAAAAKCIWzAAAAEAIFswAAABACBbMAAAAQAgWzAAAAECIyJX+SqtYMnb/+c9/OrHVq1c7sRUrVsj+KkNXVfpLSEiQ/Xfu3OnEfFnXavcL9V59xwKA4qYqk/7000+R+6v5as+ePbLt0KFDnZjaPUjtPmSmd79Qx1q2bJnsr+Z2X1XAJUuWOLGPP/7YifkquypqRw+z8ltBECjt+IUZAAAACMGCGQAAAAjBghkAAAAIwYIZAAAACFHmk/5icdJJJzmxhQsXOrGaNWtGfk1fYoaiEl58SSA///xzpJgqyQoAR4JK8Iul3LUvwU/Jzs52Ylu2bHFi1atXl/2rVavmxNLS0pyY71x37drlxNQc7It///33sm1UJPcBpQu/MAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIcrlLhkdOnSQ8Y0bNzoxld2ssr7NdBlrlaG9b98+2d8Xj9q2UiX34/JliKuysDt27Ih8fACIwrfLhKLmpccee0y2Pfvss53YihUrnFhmZqbsn5SU5MReeuklJ6Z23jAzO//8852YbwelxYsXOzFVxvuTTz6R/W+99VYnNm3aNNlWiWWnEgCHh1+YAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBDlMunvhBNOkHFVhlqVaq1Ro4bsr0pbq+Q8X7ns1NRUGVfUufrKsioq4YSkPwBFoRKf1RzoS45TiWzp6emy7Zo1a5yYmsPy8vJkf/W6c+fOdWLfffed7D948GAnlp+fL9vu3r3biak5PCsrS/Z/8803ndiwYcMit1XH2rt3r+wP4PDwCzMAAAAQggUzAAAAEIIFMwAAABCCBTMAAAAQolwm/Z1zzjkyHrWq39atW2V/VTmqatWqkc9LVerbv3+/bKuqNPmSCRXfewCAwxW1Wulll10m41WqVHFi69ati3x8lcysEu7MdILfGWec4cS6d+8u+6s5eOnSpbKtSrpTCZK+RLxNmzY5sSuuuEK2VUl/JPgBRx6/MAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIcrlLhkNGjSQ8Z9++smJxbLzhMqEVmW0VSa3mdnGjRudmK/ctdpRQ+3o4cswj6WMNo6OWMaaytBXsaPp+OOPl3G1U8xnn30W+XXVuPZR10DdK2bR74Fq1arJ+LZt2yKfFwpTZaXNzHbt2uXEfDtvqM9PxRISEmR/Nd8nJyc7saZNm8r+am71jVX1PaDelyrt7WubkZEh20blm298OzMBCMcvzAAAAEAIFswAAABACBbMAAAAQAgWzAAAAECIcpn0l5OTI+NbtmxxYiphSSWLmOmyrqok6ahRo2T/m2++2YmtWLFCtlXJJepcv/rqK9kfpc/RTLZR48eXNKgSoS699FIn5ktCWr58uRNr27atbPvss886sVjK+qoEP19yX1ZWlhN79NFHndjmzZtl/wULFjix1157TbZduHChjFcEsYw1VS7al/TnS5A7mC/Jevv27ZHaLlu2TPZX7yE9PV22VeeqkkZ9CYpKWlqajKtS3h9//HHk1wWKmy8ZtqiJ6h9++KETGzNmjGz7wgsvFOlYUfALMwAAABCCBTMAAAAQggUzAAAAEIIFMwAAABCCBTMAAAAQoszvkhF1Nwkzs7y8vEiv6cvsrFOnjhO7+uqrndjTTz8t+6tdMmIp66syzOfMmSP7o2RF3TngSGUXx9J/586dTkztCKNKw5uZbdq0yYnVqlVLtn3kkUec2F133eXEVq1aJfur+6JFixaRj1W3bl0nNn78eNm/Zs2aTuzkk0+WbSvyLhnNmzd3YklJSbKtGpe+nSPUa6g50Lf7jNr9RR1LjXUzsz179jgx344uW7dudWLqvfrKsKsdPXz32ymnnOLE2CUDR0ssOxUpp512moxPnDjRiW3YsMGJqR2czMwmTJjgxNR9ZabnkSj4hRkAAAAIwYIZAAAACMGCGQAAAAjBghkAAAAIUeaT/jp06BC5rSp5rRJGGjVqJPurxIonn3wy8vFjETVB7Pvvvz8ix0fRRE26K2pyX3Ho2bOnE+vfv78TU0l0ZmYXXHCBE/v0009lW5UwcvfddzsxXxLTzJkzndh1110n26qS3epYzZo1k/1VaW1fgmFFpsqg+8pVq0Q6X+KzouZw3z2k5stdu3Y5MV9ikOJLFjrmGPe3JxVT52+mz9WXzNitWzcnphJnff2BolAJfr7E3f/5n/9xYpdffrlsO23aNCe2ZcsWJ9a7d2/Z/7777nNi11xzjWyr7s0o+IUZAAAACMGCGQAAAAjBghkAAAAIwYIZAAAACFHmk/46duwYua16WH3fvn1OzPcAe58+fSIdx1dpUPE9fK6SQHbv3u3EPv/888jHQjS+6nuxtFWJTKpKmKqSZmZWvXp1J+ZLRlVJTy+//LJsq6jEDHX+Q4YMkf1VsoVKmDPTSVPr1693YieccILs37lzZyc2adIk2VZVcDv33HOdmO9+VdfA1zaWMVPedOrUyYn5Es7UfOebb1WlOxXzJdKpBD/1mfqOr+4r9X1hFr0ypm++jzpfmPmTVFG8oiZy+vjugZJOxoxahdZHJXmPGTNGtp09e7YTW7ZsmWyrKnuq78Fnn31W9r/ppptkXImlMuGv8QszAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEIIFMwAAABCizO+SkZOT48R8WagqwzklJcWJ/ec//5H9fVnLB9u5c2ekdmb+7HoVr127thObO3du5GPBpa6z7zNRmcS+DHu1o4kaqyeeeKLsv23btkivaWbWunVrJ9amTRsn1rhxY9lfva97773XiV199dWy/y233OLEfDt6tGzZ0omprOdFixbJ/hkZGU7s9NNPl21V1rXa5SI/P1/2V7sv+HbJqFatmoxXBPXq1XNivix0NTfXqlVLtlVltNVY9R1L7ciidjnw7Xyh+HZJUOel7tc6derI/ps3b3Zivu8xtatMRRbLDjW+3SDUWFHj4mjucBHLWFO7rPh2j4llR4w333zTibVt29aJ+dYhO3bscGK+70xV8n3kyJFOLJbdMIobvzADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIcp80p8qy7px40bZVj2wr0qaPvPMM0U/MUElscSSsLB9+/biPB14xJIU4UvEU1Sywpw5c2TbL7/80ompxBQznQh3/vnnOzFfEskDDzzgxNLT053Y4sWLZf++ffs6sbffflu2veGGG5yYSkZUJbB95+BLZlTvVyVTJiYmyv4qkU+VW/Ydq6LIzMx0Yr4EaXWd1qxZI9uuW7fOialkUvWZmulEKJUgqEpYm+l5wDdfq+TxvLw8J7Zq1SrZX91vvvelxmXdunWdmLp+5VEs87VP1MRPNdeZmQ0YMMCJqTFhZvbQQw85sf/+979OLJYEQ1+Cn/KHP/zBiankOjNd2lqN67S0NNlfra981+U3v/mNE5s4caJsW1SHO2Yq7iwPAAAARMCCGQAAAAjBghkAAAAIwYIZAAAACFHmk/4aNmzoxFR1GTOd3KMSpo7Ug+ZbtmyJ3FYlrKxdu7Y4Twemk3h8yRYq2caXmHPeeec5saysLCfmGxN///vfnViNGjVk248//tiJqcSS/v37y/7qPagkpD/+8Y+y/1/+8hcn1r17d9lWJdesXr3aifk+A1XVUN0rvtdo0qSJE/MlnY0ZM8aJvfHGG5GPVVGoOdiXeN20aVMn5ptvVQXG4447zomtWLFC9lf3tko6jCXx2pccppKbVEW+r776Sva//fbbndh3330n26pKaaraYnlM+osluVa19VWFbNCggRN74oknnJhK3DfTc5gvIfx///d/ndi8efOcmBoTZmbVq1d3YgMHDnRi1113neyvvnMuueQS2VYlCGZnZzsx3/3eokULJ+arbjtjxgwZL034hRkAAAAIwYIZAAAACMGCGQAAAAjBghkAAAAIwYIZAAAACFHmd8lQ5aJr1qwp26pdMlR2686dO4t+YoLKmlUZz2Y6w/fHH38s9nOq6GLZ3cC3I4aidnNQWe++0thqDPp21Pjkk08itfXtBtC2bVsnpkq13nrrrbJ/ly5dnJjvukbN3Fe7DpjpMsa+zHV1v99zzz1OzLfzheK7hhW5NLbavUXtEGGmy+Ju2rRJtlX3myoP77v2vt1TolLlc33l6aP2//TTT2VbNa58Ozqoc1DzzcyZMw9xhmWPuqa+MsexzO1qp58pU6Y4sUcffVT2P+WUU5yYKpdtZpaTk+PE1O49w4cPl/3VvbVkyRIn9swzz8j+K1eudGK+OXTq1KlOTO3IctJJJ8n+CxcudGKLFi2Sbdu0aePEqlat6sROO+002b9+/fpOTO1+YmY2bNgwGT+UijvLAwAAABGwYAYAAABCsGAGAAAAQrBgBgAAAEKU+aQ/9QC570HvXbt2OTFfYsWRoMpH+s5VJYEsX7682M+polOJBqp8r5nZhx9+6MS2bt0q286aNcuJqcSO+fPny/7jx4+XcSUtLc2JdejQwYmp8qtmOjklOTnZifmSDt98800nphLuzHQZZVVqVd2rZjrJ15f08+WXXzqxWBL8VDKZL5HIdw4VQVJSUuS26ppu2LBBts3IyHBiqjS1LxEzatl7XyJfLJ//Tz/95MRUcpRq5xNLyff27ds7sXHjxkU+Vlmh7rP09HTZVs1hS5culW0/++wzJ3b55Zc7sd69e8v+HTt2dGJr166VbV977TUnphL5VIKzmU6crVatmhNT321mZmeccUbkY3377beRYr5EPkUllJvp70z1ndW5c2fZX52DSoY0M2vevHnYKXrxCzMAAAAQggUzAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEKLM75Lx+eefO7GuXbvKtirD1pdhfSSobF7fLh2qNLDKjvVR76siZ/L79OzZ04mpjGczs0GDBjkxX4a/+vzULhN//vOfZf8777zTibVs2VK2Vdn0qtRpVlaW7L948WInFrUkqplZ//79nZjKUDfT56p2GlG7YZiZ7dixQ8aVunXrOjGVJT937lzZf9WqVU6sRYsWsu1f//rXyOdVlmVmZjoxNdZ98+ru3budmG+sqB1VNm/e7MR8u1wcifnOV4ZblfdWu3z4svPVDkqxvK9GjRrJtuVNbm6uExsxYoRsO2PGDCdWs2ZN2VZ9Lvn5+U7Mt8vJ5MmTnZhvhwa104aar9X8ZabvLbVLhm/nCvW+fLu/qF1p1Gfg2xVJ3RfqWpnptZDagec///mP7K/OtVmzZrKtb7eUQ+EXZgAAACAEC2YAAAAgBAtmAAAAIAQLZgAAACBEmU/6U+WiYymV6kusOBJUud/U1NTI/ffu3VucpwMze/TRRyO3Pf30052YKklrZnbeeec5MZXw5Ev6vPvuu52YL4mkdu3aTkwl+PlKWzdu3NiJXX/99U5MJaaYmVWtWtWJJSQkyLbfffedE1OJXL4kJl8yoKJeV5UL/uqrr2T/vLw8J+ZLGoqlNGxZVr9+fSemkm18yXGqBPDvfvc72VaNV5UgeqSS/tT7UgmOZjppSiWo+hLUVDKZ7/zVnOFL6C1v0tLSnJgak2b6mvo2BFD3r5qrfPO9+h5XpbXNzH788Ucn9vTTT0c+lkpGVslxOTk5sr9KRlWJgGb6Gqr70vfdoPjWZ1HL1qvvUTOz4447zonNmTNHtl29enXYKXrxCzMAAAAQggUzAAAAEIIFMwAAABCCBTMAAAAQoswn/X322WdOzPcAukqiUEkoR0osCX4qaUZVzUHRtGvXzompqklmZp988okTe//992Xb+++/P9LxfclR1atXd2K+qkWqAp+qvKTeayzn5buvVBKK71iqotn3338fqZ2Z2ezZs53YsmXLZNui3ttUy3SpaqXqmvgSc1QinS9pSyVtqsQgX5UxdV6+imaKugd8CUuVK1d2YipBVSWtmekKmL5jqWuoqiKWR998840Tu/LKK2VblaTdoUMH2faUU05xYirhzZf0O3/+fCf2xhtvyLYqGa9Tp05OzPfdcM4550Rq65uvVdKeL0lbJZjWqlXLiakxaabvN9/7Uuer7u2mTZvK/ioh+K677pJtDxe/MAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIcr8Lhlr1qxxYr6sa5VNn5ycXOzn5KPKqvqyyVV2qS8TFYdP7dDQo0cP2VaVtfXtXKLKmH/99ddObN68ebK/et0vvvhCto1q/PjxRepf0ahdFnw7F1QUqgx7LLuJqGx8X8l2NQeq1/XNiyob31dGW4ml5LfK8FfvVe06YKbfayw7F9StW1e2rQh8ZZ1ffvnlSDGfOnXqOLHMzEzZtlGjRk6sTZs2sq3avUftiqR2TjEze/PNN51Y1F1azPQ9VLVqVdlWUeeqvu/M9I4avh2Q1LVV99vrr78u+z/11FMyrhzuPM4vzAAAAEAIFswAAABACBbMAAAAQAgWzAAAAECIMp/0pyxcuFDG1UP06mF3XwLFunXrinResZTVLWpZV0SjrumHH34o26q4L4lIlYZu2bKlE7vqqqtkf5UItW3bNtlWJY6qseobfxs2bHBiWVlZkY5jZpaUlOTEfEkVKpFJtfWVEFaJs76kMXVe6j34El5U/xUrVsi2sSQTlWXqWqkSwr77Qo3BWBIEFV8ini8eVSylsdX7Vf19SX8q7iv5rRKs1PFTU1Nlf1VCGK68vLxIMTOzmTNnOrGJEycW9ymhmMSyFvs1fmEGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKUy10yfJnIKutaxXwZ+kXdJUNlMvuyNVU2ttohACXLV5b3m2++iRQjkxplSUpKihOLZfeeWOYwtauRmtt9O1dELaPt240iFuocYinDXbNmTSfm280i6i4X7du3l/FPP/008nkB+D/8wgwAAACEYMEMAAAAhGDBDAAAAIRgwQwAAACEKJdJf/Xr15fxzZs3OzGVrFG5cuXiPiUz00ksvoSZWMqqAsDRoOYwNVdVq1ZN9lfznS8RUB1L8SXXFTURT/ElaavXVWXYs7OzZf///ve/Tiw3N1e2VYnqKiG9Tp06sj+Aw8MvzAAAAEAIFswAAABACBbMAAAAQAgWzAAAAECIcpn0p5L7zHRyytGsqLdgwQInpio8menz2rt3b7GfEwBEVaNGDSe2atUqJ+arlvrOO+84MZUcZ2Y2YsQIJzZz5kwn5ksOjJq87Uvki6WCoaogqBIBU1NTZf9evXo5senTp8u2GRkZTkx9t9WqVUv2B3B4+IUZAAAACMGCGQAAAAjBghkAAAAIwYIZAAAACMGCGQAAAAhRLnfJyM/Pl3GV4a3KTderV6/Yz8lM73wRC5UJHcuxfNngABBF06ZNnZial5KSkmR/tSPGtddeK9uqXTIaNGjgxHbt2iX7q12F1Hzvm1fVLhe+0tpVq1Z1YtWrV3dizz//vOyvzuv777+XbXNycmQ8yjkBOHz8wgwAAACEYMEMAAAAhGDBDAAAAIRgwQwAAACEKJdJf77kOlWGOiEhwYm1bdtW9n/77beLdF4qYcRX1lXFY0n6A4DippLuVFnon376Sfb/5ptvIh9LJa2NHj3aiZ166qmyv0qOW7p0qROLZV5V79XMbO3atU7sxhtvdGLjx4+PfKzHHntMxs844wwnppIsW7VqFflYAA6NFRgAAAAQggUzAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEKJc7pLx0ksvyfhxxx3nxDZs2ODEpkyZUuznZGa2ZcsWJ+bL0N62bZsTmz17duRjUQYbQHHr2LGjE1O7EiUmJsr+qjS2jyp5fdlll0XuH1XlypVlvFq1ak5MzeFm/t0zimLmzJkyrsqTp6WlObE1a9YU9ykBFRq/MAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAh4gKywwAAAAAvfmEGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKwYI4oLi7O7rjjjoI/P//88xYXF2dLly4tsXMCSpOlS5daXFycPfjggyV9KqjAmKtREcTFxdmIESMO2Y7xX3zK7YL5wCA58L8qVapYs2bNbMSIEbZu3bqSPj3gsHz//fc2cOBAy87OtipVqlhWVpb17t3bHnvssZI+NeCwMFcDhZXkPH/PPffY66+/fsSPUxZVKukTONL++te/WqNGjWz37t322Wef2ZNPPmmTJk2y2bNnW9WqVUv69IDIpk+fbj169LCGDRvaFVdcYRkZGbZixQr74osv7JFHHrFrr722pE8ROGzM1UDxz/OXXHKJDRo0yBITEyO1v+eee2zgwIF27rnnHsbZl2/lfsF85plnWseOHc3M7PLLL7datWrZyJEj7Y033rDBgweX8NkdOTt27LDk5OSSPg0Uo7vvvtvS0tLsyy+/tOrVqxf6u7y8vJI5qaNs586dLJ7KKeZqoPjn+fj4eIuPjw9tEwSB7d6925KSkmJ+/Yqk3D6S4dOzZ08zM1uyZIl1797dunfv7rQZOnSo5eTkHNbrP/HEE9a6dWtLTEy0zMxMu+aaa2zz5s0Ffz9ixAhLSUmxnTt3On0HDx5sGRkZtm/fvoLY5MmTrWvXrpacnGzVqlWzvn372pw5c5zzTUlJsUWLFtlZZ51l1apVs4suuuiwzh+l16JFi6x169bOJGpmVqdOnYL//8Czba+//rq1adPGEhMTrXXr1vbuu+86/VatWmWXXnqp1a1bt6DdP//5z0Jt9u7da//7v/9rHTp0sLS0NEtOTrauXbva1KlTD3nOQRDYlVdeaQkJCTZhwoSC+IsvvmgdOnSwpKQkq1mzpg0aNMhWrFhRqG/37t2tTZs29vXXX9upp55qVatWtVtvvfWQx0T5wFyNiijqPH/AoeZ59QxzTk6O9evXz9577z3r2LGjJSUl2dNPP21xcXG2Y8cOGzNmTMEjUkOHDi3md1h2VbgF86JFi8zMrFatWsX+2nfccYddc801lpmZaQ899JANGDDAnn76aTv99NPtp59+MjOzCy+80Hbs2GHvvPNOob47d+60t956ywYOHFjwX4Njx461vn37WkpKit133332l7/8xX744Qc75ZRTnAf4f/75Z+vTp4/VqVPHHnzwQRswYECxvz+UrOzsbPv6669t9uzZh2z72Wef2dVXX22DBg2y+++/33bv3m0DBgywjRs3FrRZt26ddenSxT744AMbMWKEPfLII9akSRO77LLLbNSoUQXttm7dav/f//f/Wffu3e2+++6zO+64w9avX299+vSxmTNnes9h3759NnToUHvhhRds4sSJ9pvf/MbMfvkF5Xe/+501bdrURo4caTfccIN9+OGHduqppxZasJiZbdy40c4880xr3769jRo1ynr06BHTNUPZxVyNiqi453mfefPm2eDBg6137972yCOPWPv27W3s2LGWmJhoXbt2tbFjx9rYsWNt+PDhxfG2yoegnHruuecCMws++OCDYP369cGKFSuC8ePHB7Vq1QqSkpKClStXBt26dQu6devm9B0yZEiQnZ1dKGZmwe233+68/pIlS4IgCIK8vLwgISEhOP3004N9+/YVtBs9enRgZsE///nPIAiCYP/+/UFWVlYwYMCAQq//yiuvBGYWfPrpp0EQBMG2bduC6tWrB1dccUWhdmvXrg3S0tIKxYcMGRKYWXDzzTfHeplQhrz//vtBfHx8EB8fH5x44onBTTfdFLz33nvB3r17C7UzsyAhISFYuHBhQWzWrFmBmQWPPfZYQeyyyy4L6tWrF2zYsKFQ/0GDBgVpaWnBzp07gyAIgp9//jnYs2dPoTb5+flB3bp1g0svvbQgtmTJksDMggceeCD46aefggsvvDBISkoK3nvvvYI2S5cuDeLj44O777670Ot9//33QaVKlQrFu3XrFphZ8NRTT8V6qVCGMFcD/6e45/mDx38QBEF2dnZgZsG7777rHD85OTkYMmRIsb+v8qDc/8Lcq1cvS09PtwYNGtigQYMsJSXFJk6caFlZWcV6nA8++MD27t1rN9xwgx1zzP9d1iuuuMJSU1MLfqWIi4uz888/3yZNmmTbt28vaPfyyy9bVlaWnXLKKWZmNmXKFNu8ebMNHjzYNmzYUPC/+Ph469y5s/zn8KuuuqpY3xNKl969e9vnn39u/fv3t1mzZtn9999vffr0saysLHvzzTcLte3Vq5fl5uYW/PnYY4+11NRUW7x4sZn98qjEv//9bzv77LMtCIJCY6xPnz62ZcsW++abb8zsl2fgEhISzMxs//79tmnTJvv555+tY8eOBW1+be/evXb++efb22+/bZMmTbLTTz+94O8mTJhg+/fvtwsuuKDQMTMyMqxp06bOuE5MTLRhw4YVzwVEqcZcDRTvPB+mUaNG1qdPn2I///Ks3Cf9Pf7449asWTOrVKmS1a1b15o3b15okiwuy5YtMzOz5s2bF4onJCRY48aNC/7e7Jd/6hs1apS9+eab9tvf/ta2b99ukyZNsuHDh1tcXJyZmS1YsMDM/u85voOlpqYW+nOlSpWsfv36xfZ+UDp16tTJJkyYYHv37rVZs2bZxIkT7eGHH7aBAwfazJkzrVWrVmZm1rBhQ6dvjRo1LD8/38zM1q9fb5s3b7ZnnnnGnnnmGXmsXyeYjBkzxh566CGbO3duwT9Zm/0y6R7s73//u23fvt0mT57sPHe6YMECC4LAmjZtKo9ZuXLlQn/OysoqWKyjfGOuBn5RXPN8GDV3I1y5XzCfcMIJBZnXB4uLi7MgCJz4rxM5joQuXbpYTk6OvfLKK/bb3/7W3nrrLdu1a5ddeOGFBW32799vZr88G5eRkeG8RqVKhT+6xMTEI/LlgtIpISHBOnXqZJ06dbJmzZrZsGHD7NVXX7Xbb7/dzMybFX1gvB8YXxdffLENGTJEtj322GPN7JcEvaFDh9q5555rf/7zn61OnToWHx9vf//73wueM/21Pn362Lvvvmv333+/de/e3apUqVLwd/v377e4uDibPHmyPMeUlJRCfyZru+JgrgYKK+o8H4a5NXblfsEcpkaNGvKfLn79C0NU2dnZZvbLg/SNGzcuiO/du9eWLFlivXr1KtT+ggsusEceecS2bt1qL7/8suXk5FiXLl0K/v7AP7PUqVPH6Qv82oFFxpo1ayL3SU9Pt2rVqtm+ffsOOb5ee+01a9y4sU2YMKHgVzUzK5i0D9alSxf7/e9/b/369bPzzz/fJk6cWLBoyM3NtSAIrFGjRtasWbPI54uKjbkaFd3hzPOH49dzPAqr0P+Zm5uba3PnzrX169cXxGbNmmXTpk2L+bV69eplCQkJ9uijjxb6r7tnn33WtmzZYn379i3U/sILL7Q9e/bYmDFj7N1337ULLrig0N/36dPHUlNT7Z577in0T+AH/PqcUTFMnTpV/nIwadIkM3P/iTlMfHy8DRgwwP7973/LbOxfj68Dv2L8+tj//e9/7fPPP/e+fq9evWz8+PH27rvv2iWXXFLwK9xvfvMbi4+PtzvvvNN5L0EQRMruRsXDXI2Kojjn+cORnJzs7FaEX1ToX5gvvfRSGzlypPXp08cuu+wyy8vLs6eeespat25tW7dujem10tPT7ZZbbrE777zTzjjjDOvfv7/NmzfPnnjiCevUqZNdfPHFhdoff/zx1qRJE7vttttsz549hf6Jz+yX596efPJJu+SSS+z444+3QYMGWXp6ui1fvtzeeecdO/nkk2306NFFvgYoO6699lrbuXOnnXfeedaiRQvbu3evTZ8+veBXr1iT4+69916bOnWqde7c2a644gpr1aqVbdq0yb755hv74IMPbNOmTWZm1q9fP5swYYKdd9551rdvX1uyZIk99dRT1qpVq0LJUAc799xz7bnnnrPf/e53lpqaak8//bTl5ubaXXfdZbfccostXbrUzj33XKtWrZotWbLEJk6caFdeeaX96U9/KtJ1QvnDXI2Korjn+Vh16NDBPvjgAxs5cqRlZmZao0aNrHPnzkf0mGVGCezMcVQc2Erlyy+/DG334osvBo0bNw4SEhKC9u3bB++9995hbVV0wOjRo4MWLVoElStXDurWrRtcddVVQX5+vjz2bbfdFphZ0KRJE+/5TZ06NejTp0+QlpYWVKlSJcjNzQ2GDh0afPXVVwVthgwZEiQnJ4e+T5R9kydPDi699NKgRYsWQUpKSpCQkBA0adIkuPbaa4N169YVtDOz4JprrnH6Z2dnO9sFrVu3LrjmmmuCBg0aBJUrVw4yMjKC0047LXjmmWcK2uzfvz+45557guzs7CAxMTE47rjjgrffftu5T369rdyvPfHEE4GZBX/6058KYv/+97+DU045JUhOTg6Sk5ODFi1aBNdcc00wb968gjbdunULWrdufbiXC2UEczXwf4p7nvdtK9e3b195/Llz5wannnpqkJSUFJgZW8z9SlwQRHg6HAAAAKigKvQzzAAAAMChsGAGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKwYAYAAABCRK70V5bqi6enp8v4Oeec48S2bNnixFasWBH5WCtXrnRilSrpy5qQkODEUlJSZNtu3bo5sU8++cSJffPNN4c6xVKvJLcCL0vjGmUL47r4NWzY0ImtWrVKtt23b1+xH3/AgAEy/u9//7vYj1XUz/BIjT/Gdck6/fTTnViDBg2cmCrTbmbWtm1bJ/aPf/xDtp0/f74TU59BeSjnEeU98AszAAAAEIIFMwAAABCCBTMAAAAQIi6I+PBJST871KZNGxnv27evE/M9Q6yeF1ax+Ph42T8/P9+J7dmzx4nt3LlT9k9LS3NivnNVtm/f7sQqV64s286bN8+J/etf/4p8rKOJZ+JQHpXHcV3U5xfbt2/vxHbt2iXbZmZmOrGXX37ZiflyVh544AEntn79eieWm5sr+1900UVO7Jhj9G9MEyZMcGIvvfSSExs+fLjsf+6558q4or6f1Gewf//+yK8Zi/I4rksjlcdkZjZ69Ggntnr1aifmWxv06NHDic2cOVO2Pe6440LO8NDU/XKkxmVR8QwzAAAAUEQsmAEAAIAQLJgBAACAECyYAQAAgBAsmAEAAIAQZWaXjFtuuUXGVYb1smXLZFuVdZ2dne3E1G4YZjoTtVmzZpHameldLho1aiTbqt035syZ48SSk5Nl/7p16zoxtXOGmdnkyZOd2NHMbiXrGuVReRzXRZ0XNmzY4MQWLFgQ+Viqv2+XCxWP5fzV98h3330n29asWdOJVa1aNdLxzfTcrHbp8Dma1dfK47gujR577DEZ7969uxNTu1yoe8XMbNCgQU7so48+km3ff/99JzZmzBjZtqxjlwwAAACgiFgwAwAAACFYMAMAAAAhWDADAAAAIUpl0l/9+vWd2A033CDbrly50on99NNPsq1KulPHqlGjhuw/d+7cSK/pk5GR4cQaNmwo286aNcuJxVKGu3Hjxk4sNTVVtv3rX/8q40cLSSQojyryuPYlLKlyv2oONzOrVKlSpGOpctdmOpmvSpUqTsw3hyq+MtyqDLFKukpISJD9W7du7cSee+452fa+++5zYupa/fzzz7J/UVXkcR2LYcOGyfjxxx/vxFSSfkpKiuy/b98+J1a7du3I/ZW1a9fKeGJiohPbvHmzE9u0aZPsf9NNNzmxvLw82baky2iT9AcAAAAUEQtmAAAAIAQLZgAAACAEC2YAAAAgBAtmAAAAIESp3CWjXbt2TuzGG2+UbRcuXOjEfKWpt2zZ4sTi4+OdWL169WT/tLQ0J7ZkyRIn5stOVWW4f/zxR9k26u4b6vzNdMluH3bJAIpfRR7XX331lYyr+Wrbtm2yrdq9Ipb3pXaJ2LFjhxOrVq2a7K/O1Ze1r143KSnJiandNMzMkpOTnZjve6hRo0YyfjDftSrquKzI49rn5JNPdmJ33HGHbLt3714npnbAUqXVzfTOFWqXDLUjjJnZ/PnznZhv9xe1DlGfgW/NM2PGDCd2zTXXyLYljV0yAAAAgCJiwQwAAACEYMEMAAAAhGDBDAAAAISIVnv0KFNJGL4kOJUA4Su9qB6WV2Umly5dKvur8pUtWrRwYr5z/e677yId30yfq0osadKkieyvHuJftmyZbAsAh0vNQTVr1pRtVeK1bw5UyUUqMceXiKf6q3nxp59+kv1V0qAvaU8lXa1bt86JNW3aVPZX5+BL+opaQvhIJf3BNXDgQCe2atUq2VYl46nPyleyfePGjU5MJc76+qt7U5VxN4s+1nzl6XNycpzYKaecItt+9tlnTizqHHC08AszAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEKJUJv2pCkdr166VbVWFnZNOOkm2/de//uXE1AP46kF3M/2w/ebNm2VbRSV2+BJWKlVyPxr1EL+v6pM6VwAobqqCqS/hTCV079q1S7ZVydMq5qsypqrn7d6924n55nuV4Ld161bZVlWBjSUhXCU+rly5UrZVc/6iRYucGMl9xc+XZK82BFAJrj5qHeK7L2rUqOHE1qxZ48RURUEzs8zMTCfmSxBUGw2otYnvHlKJsxdddJFsq5L+StsY5hdmAAAAIAQLZgAAACAEC2YAAAAgBAtmAAAAIAQLZgAAACBEqdwlQ2WB+rI4586d68S6d+8u2z7zzDNOLD4+3on5SrWqrGnVX5W1NjNLSkpyYr5M2CVLljgxlWHuy9r98ccfnZjK5DaLXv4SAA52/PHHR26rdgqqU6eObKt2lFAZ+r45VM3Nag5Wmfy+uG9XItVWfY+o45vp3TcSEhJk29zcXCemdsmgNHbx6927t4yrXS586wi1s1Ys6wi1FqpevboT27Nnj+yfn5/vxHzl4dU6QI1L344c6hqkpqbKtmUBvzADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIUpl0l/VqlWdmO8B+Ly8PCd27LHHyrbnnnuuE1MlpH3lS9XD8ioR0Ee19SXtZWRkODH1sL0qS2umE2F8yTUk/aEoVAninJwc2bZt27ZObPz48ZGPVdSxqhKhSIIqmlatWjkx3zVVn59KeDIzq127thNT830syczqe0TN6762vu+GWrVqObH169c7MV/S4IYNG5yY77qoktvvv/++E2NcF79u3brJuPq+9SW3qXLTKhFQJfmb6VLwKunUV65aJfj5El/V3Kreqy/BUCW5qk0dzPS4Vps6lCR+YQYAAABCsGAGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQpTKXTIUXya8is+ePVu2VdmZKuvZdyyVCap2vvBlp6qsZ18mrXpdlR2rMrHN9HvwZV2rDO9169bJtqi4br75Zhm/6KKLnNiKFStk29atWzsxNdamTp0q+8eyI0YsZe8VtctAr169ZNsPP/ww8uuWN5mZmU7Mt0ODypr3tVXlftWOGKtXr5b91a5CaocAX1lfVe5Y7V5kpne5UO9V7fxhZrZ8+fLI59W+fXsZPxi7ZBQ/3/e1+h5Wu32Z+ctQH0ztpmGmx3XUnTfMzJo2berEtm3bJtv6dic7mG/No+ZrX9uOHTs6MXbJAAAAAMoQFswAAABACBbMAAAAQAgWzAAAAECIUpn0p5LQ1q5dK9uqB+CnTJki26qEDV/SnKIe1o+aCGhmVqmSe7kXLVok26rX2LlzpxP77LPPZH+V4OhLeIqlvDdKjirrbFb05J709HQnNm3aNCemkkXMzP7nf/7HiTVs2FC2VUl/H3zwgRN78803Zf8//OEPTmzp0qWybdQEP998oZJmOnXqJNtW5KS/3NxcJ+YrtavGmq+srkocVUlzjRs3lv1VGW01B2dnZ8v+qjSxek0zfQ+qpFOVSGgWPUHRTJcQRvFTCW++OUUlsvmS29T3bSxzuDqH5ORkJ+b7vlD91X3hE0vitTov3zVUSX8vvvhi5GMdDfzCDAAAAIRgwQwAAACEYMEMAAAAhGDBDAAAAIQolUl/6kFxX2JI586dndi9994r21511VVOTD2A7quIpyr3qES8WB72V5UGzczq1KkT6bxWrVol+6uElY0bN0Y+1sqVK2VbRKMSLlRiRyyJfLEkhqiKVHfeeadsO2zYMCf20EMPObErr7xS9v/9738f+bzU/aLGmq+i3pIlS5zYRx99JNuOHz/eiQ0ePNiJ1atXT/ZX59WzZ0/Z1jfnVAQqwdhXKVRVJNu0aZNsq+ZLlaStjm+m52tf9TxFJfj5Ep7UnK+O5buHVaK7b75WSZYofuo6+z4/NS58Y0V9j6v7wrcOUecQtSKfmU7I9a1ZVFuVIOj7Hou6eYGZP3m3NOEXZgAAACAEC2YAAAAgBAtmAAAAIAQLZgAAACAEC2YAAAAgRKncJUOVD/WVCVW7XKiStmY661TFfKWifdmdB/Pt6KGytn2ZsCprddu2bU5s9uzZsv8ll1zixH788UfZtn79+k7sm2++kW0rMvWZ+DKho+5oEcvOFzk5OTL+3HPPObHu3bs7MbVLjJlZ27ZtndiaNWucmNpNw0zvMrFs2TLZVu3eoq6rb/cXdQ/5dq5QcbUrje9Yareedu3aybYVxfHHH+/E1Bj2zaELFy50Yr7rr8qQ79q1y4n5dtlQn7U6V19ZYBX33a9Ry7D7zlWNa19b9Z2hynv77kFEU7NmTSfmK/keyzy+e/duJ6Z2mfDtXKHKqOfn50eKmZk1a9bMialdOsz0WFPrIN9ONeoe8h2rQYMGMl6a8AszAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEKLEk/5UYo1K8PMl3KlSqc2bN5dtU1JSIh1LPVQfi1jKr/qohBVV7tj3YP+cOXOcmK/UqkoYqSiilrA28yf4HQnXX3+9E/Ml7an3sGDBAif28ccfy/533HGHE7v00kud2IwZM2T/zMxMJ1a3bl3ZVo1XVdbVV+pVJcx8++23sq1KLlFJh0lJSbK/uo8bNWok2/rmnPJGJeaohLfatWvL/u+8844T8yUBnXrqqU5M3YO+sry+hOqo1OfvO5b6zlDfLb75Ws3BvmRIlZAbS+ItolFzRSzlrmP5vlDf7WoNYKbHpUquU+dvpjcl8N0rqq263333hboGvu9XNd7Vddm6davsfzTwCzMAAAAQggUzAAAAEIIFMwAAABCCBTMAAAAQosST/lRVP/Wwuy8JSFWk8yUcqcRBlZjhq+ajRK0e6OOrMqUebFcJN77Egry8PCemEqbM/Ne2IlAJCE2aNJFtzz//fCemqi+amR133HGRju+rkNS0aVMn9pe//EW2Pffcc53Y4MGDnZiv0qO630aNGuXE/vCHP8j+Y8eOdWIXX3yxbLt27Vonpj6DWCoo+qrKqYRixZekG0v1rqImmJUVqvqZ+qx885qar1X1PjNd6cxX8VVR87j6vvF9zrGMNXUNVPU+XyKeaqtiZjrBqmPHjk7siy++kP0RjRorviq+sSTCqSRj1dY3B6p7QCX4qfM30+PaNweqc1CxLVu2yP6+c1DUHFqnTh0nRtIfAAAAUEqxYAYAAABCsGAGAAAAQrBgBgAAAEKwYAYAAABClPguGbVq1YrUzpedumHDBifWrl072VbtBpCWlubEVMarmc4OVZncPiq7VZXrNjNbvXp1pPNSpSPN9Ln6MlbT09NlvKKaOHGijKtdSh5//HHZdvbs2U7spJNOcmKqrLSZ2cqVK51Yv379ZNv27ds7sXXr1jkxXwlitStIixYtnJgvw1/dA76yrtWrV3diKjvat8tCUXdOUDvF+HZJUPebb6cZdb3LIzVfquvv2zVE7SqzefNm2TZq2XrffK36+3YuiNrft3OBiqvvtmnTpsn+mzZtcmInnniibKvuLd/OPjh86rvZN9eoMai+w830d7a6L2Ipw63mdt+aSe1045sD1bHUOsK3A5PafUN9B5jpuT0jI8OJLVy4UPY/GviFGQAAAAjBghkAAAAIwYIZAAAACMGCGQAAAAhR4kl/qkykerDelwSkHipXr+l7XfWwvu9he1VCWvX3ldZWyRq+c1UP26v+vmNt3LjRidWvX1+29b3fikAl7eXm5sq2M2fOdGKDBg2SbVXCiBorvvLNvlKliio3rZJAfO9r0aJFTkwlLKmELzNd2thXvlSNV1/SlxLLWFXHUqWNfQlqKhEmltK45ZGvNPTBfMl1qoSvKgNvpj9rdXxfwpKar1Vb37mq7xzf56/OVX3f+K5fXl6eE6tdu7Zsq8oQ+5K/cfhUcpvv+1Z9j6t52Uwneqvva9+GAiq+c+dOJ6YSSc30uIol6U+NtaVLl0bu36VLF9lW3UNqvihJ/MIMAAAAhGDBDAAAAIRgwQwAAACEYMEMAAAAhGDBDAAAAIQo8V0yopa19WWy5+fnOzFfJrLaZUJl+Pt25IiaIe7LmFc7H6jjm+nykevXr498Tr4yyIrKsFWZuOVxN40JEyY4sf79+8u2UUtQm+lsejXWfVnXCQkJTsyXyayyntVYmzt3ruyvxvuaNWuc2Lx582R/NS5856pEva/M9I4GsZSnj6U0srqPq1atKtu2a9cu8uuWZVE/a99nsnz5cifWsWNH2VbNjWpc+46lvjNiKZet4r6xqsaKmoN9JazVvRnLfBvLPYRo1Lzo+25Xn5XazcRMl8FWY0XttGSmv0fUd4BvByb13RTLLmR16tSJdE5meqcQtXuNmb4GvjLaJYVfmAEAAIAQLJgBAACAECyYAQAAgBAsmAEAAIAQJZ70p8pPqgfIfUl/KmnKV25alZ+MWprbTCcNqiQgX2JQLOVzVWlilfRXs2ZN2d+XjKb4yr1WBB9++KETa9CggWx70003ObGLL75Ytm3btm3RTkyIJQlIjTVfcpRK7FBJICQW+Z1xxhklfQpHhRqDalz5kj7VfNuoUSPZVs23sYxr9Z0RS2lsxZccpa6Lmld9pX7VfOErbazuw/KYkF3SVOK1L0lbfa7ffvtt5LaxlDZXn7Waw33rDTV+Ylmb+JL2lAULFjixunXrRm6bnp4e+VhHA78wAwAAACFYMAMAAAAhWDADAAAAIVgwAwAAACFKPOlPJbepZAn1ULtZbNW4Vq9e7cTUA/RpaWmyf15enhNTCSe+5CjV1lfNR10D1d+XLPDuu+86MV81MnUN1MP2sSQSlkf3339/pJiPqpDUqlUr2VZVP6tXr55sG7Uaku8eUolMURNLzHRFNl8iqYrv3r3bifkSsdR5+RKe1P2i3oMvuUpVqfIda+rUqU7s5ptvlm3LMvX+VXKdb6ycffbZTsyX2KPGRdSxGst5+caaShD0HUvN+Srmuy5ZWVkyrkQd1ygaldzm23xAfdZbt26N3DZq0qiZ3ihBbQigKviamR177LFObMOGDbKtou6XjIwM2VZV9ozlflOJlyWJX5gBAACAECyYAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBAlvktG1DLYmzdvlv1VW1/W9cKFCyOdU35+voyrc1CZ3Dt27JD91bn6sm5Vhq6K+bK2VYatb/cO9RnEUv4S0ahdVlTMzOzjjz8+wmcDxE7NKyrr3Teue/fu7cTmz58v20YtTRxLaeuoZeDN9G4UvmOp66LmUF8J4o0bNzoxtauOmZ7za9asKdvi8KndKHzf14pvXKnXUOPK932t+teoUcOJ+caEKjnvKy+v4mp907x5c9l/+vTpTsy3+4faJcN3XiWldJ0NAAAAUMqwYAYAAABCsGAGAAAAQrBgBgAAAEKUeNKferBdJUuo5Dqf77//XsbVw+6qrHBmZqbs36BBAyemztX3oLoqIawS7sz8iYcHS0pKknFVclu9f19bX3lxABWXSsxRMZWcZ6YT+erXry/bqqQlNS+q45vppC11Xr752ve6ikrmS01NjXws9T3gSxBUSX+xJD4iGnWdfYl4iq+ss/q+VcmosSTeq/HjO1c1rnzJjCqukv5q1ap1qFM8JHVvkPQHAAAAlCEsmAEAAIAQLJgBAACAECyYAQAAgBAsmAEAAIAQJb5LhspaVjs/qB0mzHQW5aBBg2TblStXOrFVq1Y5MV+56Z07dzoxVS7bl9mpslZ9ZbybNGnixNSOHr7yqQ8//HDk81LZ3L5rAKDiUrsVqd0kfFn3qizutGnTZNvk5ORI/X07RPh2KTiYbwemWMpoRy1tvH79etn/5JNPdmINGzaUbdVuR2oHJxTNli1bnJja4cLMbNOmTU6sbdu2kY+l1kG+XVqi7iKmyq2bmTVr1syJqZ0vfNQuG75dtRo3buzE8vLyZFs1Z6idbkoSvzADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIUo86U8lUagEP19y2xdffOHELrvsMtlWJb1lZGREPpZ64D+WMpPr1q1zYiqxxEwnEagkhHnz5sn+iq/U5tatW52YL7kBQMWlkttUwpJvrnn22Wed2L333lv0Eyvj1HfWfffdJ9uq7xGVEI6i2bBhgxNTSadmOkn+lFNOkW3V97ham/hKo6vNB6pVq+bEfIl4viRXJer6Rp2TmdlZZ53lxFQZbzOd5Fva8AszAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEKLEk/5UlTn1oLlq5/PVV18V6ZzKK1+1RFVtMDMz04l98803xX5OAMoOlVyUn5/vxHxJaGpe8VGJUL7qZ0XhqxQYy7HUa6jzVwmSZmY5OTmRjx+1qiCKRlXx9V1nVZ34mWeekW1/+9vfOrFatWo5MV9lXpVgmJaW5sR81ftU9TzfWFMJfuoa+BIJJ02a5MS6desm26qEyv/+97+ybUnhF2YAAAAgBAtmAAAAIAQLZgAAACAEC2YAAAAgBAtmAAAAIESJ75KhdmhQfNnFsVBluIvjdY8WlTWrMmZj6R/rawCouKKW8PXNKbGUvz1a81Jx7LxR1NdYv369E/OVRlalhVesWOHE1M4JZro0M1zLli1zYrF8zm+//XbkePv27Z3YscceK/vXqFHDidWrV8+JqfWOmdnevXudmK+MthqXH374oRP74osvZH+lS5cuMq5271DHL0n8wgwAAACEYMEMAAAAhGDBDAAAAIRgwQwAAACEKPGkP0U9/J2YmFjk1y1LCX5KUZNgfNdQJQeoUp8AKrbOnTs7MZUIqErqmvkTmcojX8ltRSVt+RKxVOKkKlfcq1cv2f/f//535POqyHJzc51Yw4YNZdvly5c7MZWcZ6ZLyc+cOTNSrDzwlRdX90DNmjWP9OnEhF+YAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBAsmAEAAIAQJb5Lxrp165yYyqJUWaiIzfz582W8UaNGTmzz5s1H+GwAlDXTpk1zYmrXhq1bt8r+33zzTbGfU2kVyy4ZTz31lBPzlRFXuxotWrTIib3xxhuRjw/Xe++958SaN28u265du9aJqd0wfNROM0erNHwYNYZVLJZznTp1qowvWLDAif3nP/+J/LpHA78wAwAAACFYMAMAAAAhWDADAAAAIVgwAwAAACHigiAISvokAAAAgNKKX5gBAACAECyYAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBAsmAEAAIAQLJgBAACAECyYI4qLi7M77rij4M/PP/+8xcXF2dKlS0vsnACfoUOHWkpKyiHbde/e3bp3715sx+3evbu1adOm2F4P+DXGNfCLuLg4GzFixCHbsVYpPuV2wXxgkBz4X5UqVaxZs2Y2YsQIW7duXUmfHuB44oknLC4uzjp37lzSp1Im3XPPPfb666+X9GngIIzromFcVzzff/+9DRw40LKzs61KlSqWlZVlvXv3tscee+yIH5vx5lduF8wH/PWvf7WxY8fa6NGj7aSTTrInn3zSTjzxRNu5c2dJnxpQyLhx4ywnJ8dmzJhhCxcuLOnTKXOY6EsnxnXRMK4rlunTp1vHjh1t1qxZdsUVV9jo0aPt8ssvt2OOOcYeeeSRmF/vkksusV27dll2dnak9ow3v0olfQJH2plnnmkdO3Y0M7PLL7/catWqZSNHjrQ33njDBg8eXMJnd+Ts2LHDkpOTS/o0ENGSJUts+vTpNmHCBBs+fLiNGzfObr/99pI+LaBIGNdAbO6++25LS0uzL7/80qpXr17o7/Ly8mJ+vfj4eIuPjw9tEwSB7d6925KSkmJ+/Yqk3P/CfLCePXua2S8Tue85t6FDh1pOTs5hvf4TTzxhrVu3tsTERMvMzLRrrrnGNm/eXPD3I0aMsJSUFPkL9+DBgy0jI8P27dtXEJs8ebJ17drVkpOTrVq1ata3b1+bM2eOc74pKSm2aNEiO+uss6xatWp20UUXHdb5o2SMGzfOatSoYX379rWBAwfauHHjnDZLly61uLg4e/DBB+2ZZ56x3NxcS0xMtE6dOtmXX355yGPMnDnT0tPTrXv37rZ9+3Zvuz179tjtt99uTZo0scTERGvQoIHddNNNtmfPnsjv5+uvv7aTTjrJkpKSrFGjRvbUU085bfLy8uyyyy6zunXrWpUqVaxdu3Y2ZswYp92OHTvsxhtvtAYNGlhiYqI1b97cHnzwQQuCoKBNXFyc7dixw8aMGVPwGNbQoUMjny+ODMY14xqxWbRokbVu3dpZLJuZ1alTx4m9/vrr1qZNG0tMTLTWrVvbu+++W+jv1TPMOTk51q9fP3vvvfesY8eOlpSUZE8//TTj7RAq3IJ50aJFZmZWq1atYn/tO+64w6655hrLzMy0hx56yAYMGGBPP/20nX766fbTTz+ZmdmFF15oO3bssHfeeadQ3507d9pbb71lAwcOLPivwbFjx1rfvn0tJSXF7rvvPvvLX/5iP/zwg51yyinOA/w///yz9enTx+rUqWMPPvigDRgwoNjfH46ccePG2W9+8xtLSEiwwYMH24IFC7yLhZdeeskeeOABGz58uN111122dOlS+81vflMwxpQvv/zSevbsaccdd5xNnjzZmzi1f/9+69+/vz344IN29tln22OPPWbnnnuuPfzww3bhhRdGei/5+fl21llnWYcOHez++++3+vXr21VXXWX//Oc/C9rs2rXLunfvbmPHjrWLLrrIHnjgAUtLS7OhQ4cW+mfHIAisf//+9vDDD9sZZ5xhI0eOtObNm9uf//xn++Mf/1jQbuzYsZaYmGhdu3a1sWPH2tixY2348OGRzhdHDuOacY3YZGdn29dff22zZ88+ZNvPPvvMrr76ahs0aJDdf//9tnv3bhswYIBt3LjxkH3nzZtngwcPtt69e9sjjzxi7du3Z7wdSlBOPffcc4GZBR988EGwfv36YMWKFcH48eODWrVqBUlJScHKlSuDbt26Bd26dXP6DhkyJMjOzi4UM7Pg9ttvd15/yZIlQRAEQV5eXpCQkBCcfvrpwb59+wrajR49OjCz4J///GcQBEGwf//+ICsrKxgwYECh13/llVcCMws+/fTTIAiCYNu2bUH16tWDK664olC7tWvXBmlpaYXiQ4YMCcwsuPnmm2O9TCgFvvrqq8DMgilTpgRB8MsYqV+/fnD99dcXardkyZLAzIJatWoFmzZtKoi/8cYbgZkFb731VkFsyJAhQXJychAEQfDZZ58FqampQd++fYPdu3cXes2D74GxY8cGxxxzTPCf//ynULunnnoqMLNg2rRpoe+lW7dugZkFDz30UEFsz549Qfv27YM6deoEe/fuDYIgCEaNGhWYWfDiiy8WtNu7d29w4oknBikpKcHWrVuDIAiC119/PTCz4K677ip0nIEDBwZxcXHBwoULC2LJycnBkCFDQs8PRw/j+heMa8Ti/fffD+Lj44P4+PjgxBNPDG666abgvffeKxhjB5hZkJCQUGiszJo1KzCz4LHHHiuIHbxWCYIgyM7ODswsePfdd53jM978yv0vzL169bL09HRr0KCBDRo0yFJSUmzixImWlZVVrMf54IMPbO/evXbDDTfYMcf832W94oorLDU1teAX5bi4ODv//PNt0qRJhf758OWXX7asrCw75ZRTzMxsypQptnnzZhs8eLBt2LCh4H/x8fHWuXNnmzp1qnMOV111VbG+Jxwd48aNs7p161qPHj3M7JcxcuGFF9r48eMLPZ5zwIUXXmg1atQo+HPXrl3NzGzx4sVO26lTp1qfPn3stNNOswkTJlhiYmLoubz66qvWsmVLa9GiRaFxd+BRJjXuDlapUqVCv0okJCTY8OHDLS8vz77++mszM5s0aZJlZGQUyiOoXLmyXXfddbZ9+3b75JNPCtrFx8fbddddV+gYN954owVBYJMnTz7k+aBkMK5/wbhGLHr37m2ff/659e/f32bNmmX333+/9enTx7KysuzNN98s1LZXr16Wm5tb8Odjjz3WUlNT5T1zsEaNGlmfPn2K/fzLs3K/YH788cdtypQpNnXqVPvhhx9s8eLFR2SQLFu2zMzMmjdvXiiekJBgjRs3Lvh7s1++GHbt2lUw+Ldv326TJk2y888/3+Li4szMbMGCBWb2yzPX6enphf73/vvvOw//V6pUyerXr1/s7wtH1r59+2z8+PHWo0cPW7JkiS1cuNAWLlxonTt3tnXr1tmHH37o9GnYsGGhPx9YZOTn5xeK79692/r27WvHHXecvfLKK5aQkHDI81mwYIHNmTPHGXPNmjUzs2hJJ5mZmU7C6YH+Bx4lWrZsmTVt2rTQf1yambVs2bLg7w/838zMTKtWrVpoO5QujGvGNQ5fp06dbMKECZafn28zZsywW265xbZt22YDBw60H374oaDdwfeM2S/3zcH3jNKoUaNiPeeKoNzvknHCCScU7JJxsLi4uEIJFgeoXz+KU5cuXSwnJ8deeeUV++1vf2tvvfWW7dq1q9CzdPv37zezX55hy8jIcF6jUqXCH11iYqIzSaP0++ijj2zNmjU2fvx4Gz9+vPP348aNs9NPP71QzJfxfPBYTkxMtLPOOsveeOMNe/fdd61fv36HPJ/9+/db27ZtbeTIkfLvGzRocMjXABjXQNElJCRYp06drFOnTtasWTMbNmyYvfrqqwU7zUS9ZxR2xIhduV8wh6lRo4b8p4vD+a/7A3sczps3zxo3blwQ37t3ry1ZssR69epVqP0FF1xgjzzyiG3dutVefvlly8nJsS5duhT8/YF/ZqlTp47TF+XHuHHjrE6dOvb44487fzdhwgSbOHGiPfXUU4c1ucXFxdm4cePsnHPOsfPPP98mT558yOpnubm5NmvWLDvttNMK/rUjVqtXr3a2NZw/f76ZWcHuM9nZ2fbdd9/Z/v37C/2H3ty5cwv+/sD//eCDD2zbtm2Ffo07uN2B94vSgXHNuEbxOvDD35o1a47ocRhvfhX6J8nc3FybO3eurV+/viA2a9YsmzZtWsyv1atXL0tISLBHH3200H/dPfvss7Zlyxbr27dvofYXXnih7dmzx8aMGWPvvvuuXXDBBYX+vk+fPpaammr33HOPzBL/9TmjbNq1a5dNmDDB+vXrZwMHDnT+N2LECNu2bZvz3FosEhISbMKECdapUyc7++yzbcaMGaHtL7jgAlu1apX94x//kOe7Y8eOQx7z559/tqeffrrgz3v37rWnn37a0tPTrUOHDmZmdtZZZ9natWvt5ZdfLtTvscces5SUFOvWrVtBu3379tno0aMLHePhhx+2uLg4O/PMMwtiycnJhbZwRMlgXDOucfimTp0qfyGeNGmSmbmPfRY3xptfhf6F+dJLL7WRI0danz597LLLLrO8vDx76qmnrHXr1rZ169aYXis9Pd1uueUWu/POO+2MM86w/v3727x58+yJJ56wTp062cUXX1yo/fHHH29NmjSx2267zfbs2eNsbZSammpPPvmkXXLJJXb88cfboEGDLD093ZYvX27vvPOOnXzyyc5ki7LlzTfftG3btln//v3l33fp0sXS09Nt3Lhxkbe+UpKSkuztt9+2nj172plnnmmffPKJtWnTRra95JJL7JVXXrHf//73NnXqVDv55JNt3759NnfuXHvllVcK9u0Mk5mZaffdd58tXbrUmjVrZi+//LLNnDnTnnnmGatcubKZmV155ZX29NNP29ChQ+3rr7+2nJwce+2112zatGk2atSogl/dzj77bOvRo4fddttttnTpUmvXrp29//779sYbb9gNN9xQKOGlQ4cO9sEHH9jIkSMtMzPTGjVqRDnmEsC4Zlzj8F177bW2c+dOO++886xFixa2d+9emz59esG/RA8bNuyIHp/xFqLkNug4sg5spfLll1+GtnvxxReDxo0bBwkJCUH79u2D995777C2lTtg9OjRQYsWLYLKlSsHdevWDa666qogPz9fHvu2224LzCxo0qSJ9/ymTp0a9OnTJ0hLSwuqVKkS5ObmBkOHDg2++uqrgja/3moJZcfZZ58dVKlSJdixY4e3zdChQ4PKlSsHGzZsKNh+64EHHnDaHTw+1ZjYsGFD0KpVqyAjIyNYsGBBEATu9ltB8Ms2WPfdd1/QunXrIDExMahRo0bQoUOH4M477wy2bNkS+p66desWtG7dOvjqq6+CE088MahSpUqQnZ0djB492mm7bt26YNiwYUHt2rWDhISEoG3btsFzzz3ntNu2bVvwhz/8IcjMzAwqV64cNG3aNHjggQeC/fv3F2o3d+7c4NRTTw2SkpICM2NrpBLCuGZc4/BNnjw5uPTSS4MWLVoEKSkpQUJCQtCkSZPg2muvDdatW1fQzsyCa665xumfnZ1daIz4tpXr27evPD7jzS8uCCI8HQ4AAABUUBX6GWYAAADgUFgwAwAAACFYMAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAISJX+qO+OI6UktwKvDyM6/j4eCe2b9++Ir1mpUru1NCsWTPZtkGDBk6sfv36sq0q61qvXj0nlpycLPurths2bJBtP/nkEyf2xBNPOLGdO3fK/kXFuEZ5xLgufmpeHDx4sGz7ww8/OLGTTjrJic2bN0/2X758uRPr1KmTbPvBBx84sc8++0y2LeuijGt+YQYAAABCsGAGAAAAQrBgBgAAAEKwYAYAAABCxAURn+AvrQ/bx3JeUZMVVBKVmdmrr77qxNQD9JUrV5b9d+3a5cR69eol215wwQVObP78+bKtcswx7n8L+d5/SSZxlPTxS+u4VtRnama2f/9+J1alShUndvPNN8v+7dq1c2Lt27d3YjVr1pT9U1NTZbwo1qxZI+Pq3tyyZYtsq+IrV650Yuedd57sr8ZGLGOVcY3yiHHt6tixoxPLzs6Wbbt06eLE1Hztu86LFy92YikpKU7s+++/l/2rVq0q40pWVpYTS0tLc2Kffvqp7P/ll186sc2bN0c+/tFE0h8AAABQRCyYAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBBlZpeMWHYIiIXK/Fflc83MEhISnJi6LtWqVZP9f/75Zye2e/du2Xbbtm1O7M4773RiCxculP3LErKuo0lMTJTxPXv2OLFBgwY5sbFjx8r+agypcenbjULdFzVq1JBt1T2gdtnw3UPqvli7dq1sm5GR4cQ2btzoxI4//njZv6gY1zgSVNl6dV8dKWVlXBd1l5uLLrrIiak5xcwsOTnZifnmpUWLFjkxtUvGvn37Ih9LzcG+MaGuizq+md6ZS72uKu1tpudx3/fIpk2bnNh7770n2x4J7JIBAAAAFBELZgAAACAEC2YAAAAgBAtmAAAAIISbPVBKxZLcp8pUmpmdf/75TiwzM9OJqYfqzfRD+Bs2bHBiKinDzCw/Pz9yW5WMeN999zkxX7nscePGObHZs2fLtigbfvrpp8htVRLH9u3bZVuVSKfG5axZs2R/lXDiK6Ndq1atSMf3JWCoeUCV9jbT10C9L19p761bt8o4yi6VPO4ba0VNbjvzzDOdmC/B9IwzznBiqiyxmf7OueWWW5zYjz/+KPuvXr1axsubWD6/Sy65xImdfvrpTuy1116T/ZcuXerEkpKSIh9fJcL51jyqtLRax/iS/tS85ttUQV1D9b4WLFgg+6v3oMp4m5l1797diakk7a+++kr2Pxr4hRkAAAAIwYIZAAAACMGCGQAAAAjBghkAAAAIwYIZAAAACFFmSmP7qHLRzZo1k2337t3rxHbs2OHEfO9V7Qagyjk2bdpU9l+xYoUT82XSqjLIKsPfVy45Pj7eif3www+yrcqwPprKSqnVkhZLefhHHnnEiQ0cOFD2nzNnjhOrX7++E/v+++9l/9q1azsx3+4v9erVc2KrVq1yYlWrVpX969at68R8ZbTVbjfqvvjLX/4i+997770yHhXjuvRR91AsOzCdd955Mv7oo486MXUP+XYTUOPSd15qXFeuXNmJqfvSTH8PdO7cWbZVO+uU5XGtdukxMzvrrLOcWJ06dZzYvHnzZH+1jlA7PJj5y1AfrKjlzn2ltXfv3h35nNRnrebmXbt2yf5qZye1jjLT3xnqvBYvXiz7F3X3F0pjAwAAAEXEghkAAAAIwYIZAAAACMGCGQAAAAhRZpL+Lr74YhlXSRjr1q07IuegHlZXSXe+krq+RChFva5KAlDJImY6uUUlXJmZzZw504nddNNNhzjD4lOWk0iOJt+5qus3fvx4J+YrGa+SKFq2bOnEYkn685U/VeeqSv36klBatWrlxFRpbTOzGjVqRH5dpahjg3Fd+qgka1/C0pVXXunEVJK5mVl+fr4TU3OwL+FJJe2pEshmupS7GmsqEc1MJ76lpaXJtup6leVxfdJJJ8l4bm6uE1Ofn2+sLF++3In5vu/VZ62S43xJfyquzlVtcuDjmxfVsdT7Usmhvra+Y6lk1mXLljkxlYxpZjZ9+nQZj4qkPwAAAKCIWDADAAAAIVgwAwAAACFYMAMAAAAhomehlTBfJSJfwk9UKonA9/C3qry0Z88eJ+ar3qce7PclBqhjqZivv3oPvspDxx9/vIyjdIkl2UZVefIlrPjG68F8yRaq8pNKbDEz27lzpxNTiVC+Sn+qf/Xq1WXbW2+91Yk9/PDDTuyLL76Q/YcOHerEnn/+edkWpY+aG9U90KVLF9n/f/7nf5yYLxFP3UOqAqUav2Y6edtXxVXdWyrBKysrS/ZfunSpE1PfY2ZmI0aMkPGyqkGDBjKuEtFUFV+VcGmmE59VRT1f3DdfRqWS84qayOdrq/j6q3PwfY+oSn1qzRRLMmNx4xdmAAAAIAQLZgAAACAEC2YAAAAgBAtmAAAAIAQLZgAAACBEmdklw5edqjJOfRmbUUtd+jJWo+5SEEsmta/Up4qrmMoiNdPv1ZfxWtQMXRS/WHZvURo2bOjEfFn3vvjBYil37csQV2NQjT+1I4yZzpD27fKxYMECGT/YWWedJePvvPOOE2OXjPJHlbU20ztH+O4V9Z2jdsTwZfirssC+cR3LzkyK2pEjPT098rHKsnr16sn4li1bnJgqIe77/FUZct+uROqz9o1BRY0htQ7wfa/HssuEaqvGuq+0utqtScXM9K4y6viqnZkew+vXr5dtDxe/MAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAhykzSn6/Mo3rY3fdQ+IYNG5xYLIlUKpFOJWf5HmpXbX3JTeq81PF9yVXqAXjfe1WJWDVq1HBisSQmoGjUZ+VLEFVtVcLaRRddJPurhCF1r/iSaVVyiRqrZvpcY0nWiFqq1cysT58+Tuztt992YqoErpnZCy+8EPlYKH188/DB5s2bJ+MqOS6WcsNqrPvOSc2tsYx1dV6+5C6VuOY7r3/84x9O7Jlnnol8XiWpbt26Tsw3h6lrpa6Tr7S2Wods3LhRtlVx9Vn7Pn/1HqImbpvFto5QbVVMlVs3M2vUqJETU0mPZvq6qPe6bds22b9Vq1ZO7JNPPpFtDxe/MAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIcrMLhm+Mo9q54CMjAzZNi8vL1J/384VKmtaZRf7ynirY8Wym4A6vi8TWu1ysXXrVtlWZaJmZ2c7MXbJKDvuvffeSDEzvUtA9erVnZhv5wpVAthHjWE1/qZOnSr79+rVy4n5xuXQoUOd2LXXXnuIM/w/Tz75ZOS2KH1i2QFJUTvFqJLzZrq0sm9uV9R3jm83gai2b98u42pHBfXdWNadeOKJTqxbt26y7UsvveTEGjdu7MT69u0r+z/44INOzLdzhPpcYylXHXVc+dqpHVV8ZbzVDkLqvkhISJD91U4fPXr0kG2//vprJ/buu+86sQ4dOsj+6juLXTIAAACAo4gFMwAAABCCBTMAAAAQggUzAAAAEKJUJv2lpKQ4MZUsZKYTO3xJd+p1fYlwijoHlazhK2EcSxKIot6rL0GxTp06TmzHjh2yrTpf1R8lq6hJTD6qNHZaWlqkmJkePyoxxEyP1xkzZjgxlbRqphM7fEl/OTk5Tqxfv35OTJXLNoue5IvyaeHChU7suOOOk23XrFnjxKpWrerEfOWOVdyXTKvuodq1azsxlYhoZlarVi0n9sMPP8i2Zdnrr7/uxHyJeEOGDHFiN9xwgxP78ssvZX/13Zqeni7bRi337Ev69H2PH8y3NlDrI19pbDWGfa+rqDGcm5sr2/7ud79zYrfeeqsT++9//yv7v/POO5HP63DxCzMAAAAQggUzAAAAEIIFMwAAABCCBTMAAAAQolQm/akkpFgq4vmoh+1VsoWvao2qxqMqEPrOSSUM+ZKI1PtVMV8ypEoCWb58uWz7008/OTH1sD9KJ5V0p8aFL+FIVflSCbLq/vG13bx5s2yrqgWqsdqkSRPZX90DqnKZmU5keeGFF5xYzZo1ZX8S/Cq2b775xomdf/75sq0aK1GTs8x09TXf/bZx40Ynpr6z9uzZI/urZDRV7bM8mjlzZuS4+r5ctGiR7P/b3/7WiY0ZM0a29c1XB/PN12rNoZLrfJsfqO8G35pHUXOwGr9mOvFaJfKZmV144YVO7JFHHol8XkcDvzADAAAAIVgwAwAAACFYMAMAAAAhWDADAAAAIVgwAwAAACFK5S4ZqiSkb5cMlXHqy9BfsWKFE6tbt64T82U3q+xStSOGL7s1an8fdQ18JS3Vzge+UpuK2vkApVMsY1BZvXq1E2vUqJET27Rpk+yvyurOmTNHtlXltVUZdlW+10xnY/vuV5U5rl63V69esv8HH3wg4yi71BzqK/WrdqPw7ZyixrXavcZHjUvf3K52dVHz/a5duyIf/8cff4zctqyI5bNWHn744chtBw4c6MSaNWsm26r1ifqsfOeqymirnTN840f1r1evnmyrXlf1933fZGVlObFXXnlFtv3qq69k/GCx3FexrK+i4BdmAAAAIAQLZgAAACAEC2YAAAAgBAtmAAAAIESZSfrzPVSukv58D3qr8p9NmzZ1Ylu3bpX9VcKRSgJRD8qb6SSEWEpjq/KV69atk/1nz57txJo3by7bqmQuX5Ilyp+oJdt9Y1WNyzZt2si277//vhObNm2aE7vttttkf5UIo0q7m+n7VSWMqJKsZiT9lUexJH2p7yFfCeG9e/c6MTXWfOWuVVvfd14srxuV73ukLCvuhK8wvo0GorZV48dX2lyNi6SkJCfmS/pT/Tds2CDbqmREdayqVavK/r7XLQq1eYKZ//upOLEqAgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKUyqQ/VWXO90C3SuzZsmWLbLtq1SonppIGfceKmkTg668STmJ5UF0le/iSUNTD9p07d5ZtVaU334P1KH2iVrTyfaYqQU8lnPgSplSVM1Vlz8wsPT3dibVs2dKJqWpmZrqime99qUQolXDjS1hBxdatWzcn5kvuUvdL9erVnVi1atVkf5W46kv6iyUhN6r8/Pwi9a/o1PXzfTeruVHNYbVr15b9fRVXD+abr9UY9CWNqvWVSib0bRLgS1yMSq15fO/raCR58gszAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEIIFMwAAABCiVO6SoTIzVelIM7O6des6sUWLFsm2KrtT7ZLhyxhVWZgqkzmW8qexZHaqtr7+27Ztc2KqpKWZvrZq5wOUTlHL/d53330yXqdOHSemSuX6Ssarce0rV926dWsn1q5dOyemxq/vdX27XKxcudKJqaztopYVRtmmSmCbmXXv3t2JqZ2WzMxSU1OdmPoe8+2coO4t3w4DakcFdQ/GsvuL2n2mrDuapbHVzhW+71C15lBrE98cqj5/NYepMWmmx5oaq2b+nVoO5ltb+HYsK4pYytsXN35hBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKUmWwXXwKESmxQZaHNzCpXruzEYkkMUA/W+x7MV1RihjonM/1gv0oW8CV2qCQEXwnhXbt2OTFVnhxlW79+/WRcJWaoUqe+sTZz5kwntnHjRtm2RYsWTkyVG/YlrCi+hGB1vzRv3tyJ3X///ZGPhaMnapK0audrq/z1r3+V8Vjme1UGW5VA9pWwjiV53Fd2/mAqkcznpJNOkvFvvvkm8mtUZOr71vf5Rf1cfN/XUcug+46jxpqvrUoGVPeA717zrW/KKn5hBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKwYAYAAABClMpdMlSZRV8mtMrC9O2SUa1aNSemMo59pRdVdqmK+fqrrFlfJqyislN9pVbXrl3rxOrVqyfbqvcQS1lVHB2x7AZw7rnnOrGsrCzZX5WQVveVun/MzN59910nNn/+fNn2iiuucGJdunRxYr7dCNTuHb6s8fT0dCe2ZMkSJ/bKK6/I/hVZUXeeOFLU5x9Lqdxrr73Wif3xj3+Ubb/99lsnVqtWLdlWXRcV8+1woeK+kt1qvMeye8jq1aud2GmnnSbbjh49WsZxaGqsmunPSn2P++41taOF2s3Ct0uHul98bZWo66DyiF+YAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBClMukvIyPDicWSBLRw4ULZViUtqQfgVUld3zmoh/VjSULxvS/1uqoEsC85TyVd+c5LXUOVRICSFUvC1cSJE53Yd999J9uqz79+/fqRXtNMJ/0df/zxsq16XZX46it3rfiSm1RyzPLlyyO/bnnju06xJB5HTTjzUWPNd15FPVbfvn2d2PXXX+/EzjnnHNn/2WefdWL5+fmyrUraU2PYl/SnElR9n4u6hqo0ty8hfNu2bU5MlaxHdFu3bnVidevWlW0zMzMjtd28ebPsrxL01DogagltM7M2bdrIuHpfKiHcl6Ba1CThkk4yPhi/MAMAAAAhWDADAAAAIVgwAwAAACFYMAMAAAAhykzSn+/h79atWzux//znP7Lt+eefH+n4sSRbqAfwfVVvYqlSpR6sV/1r1qwp+6tqh77zUskhqtoiip+vGlQsiaObNm1yYqpK2b/+9S/Z/95773ViM2bMcGK7du2S/YcNG+bEunXrJtuq5BL1ur7KU+p6xXK/vffee7Jt1P6xfC6ljW8OPZpVuo7E9Rs8eLCM33zzzU6sXbt2TuxPf/qT7J+SkuLEFi1aJNuq7yyV9KfamenER19CuEpeV7E9e/bI/uozqFGjhmyLwnwJqhdffLETmzJlimyr5jb1ujt27JD91aYEDRs2dGIbN26U/bdv3+7EfMmsKplQjUtfgqFK/p48ebJsWxbmVn5hBgAAAEKwYAYAAABCsGAGAAAAQrBgBgAAAEKwYAYAAABClMpdMtQODb4Mb1X+1pcdmpqaGun4sZSQjqWd2n3D19aXIX0wX3bzli1bnJgqiWqmy2D7MqxRmC9rWmVCq3EVS/nSUaNGybgaA2r3lKlTp8r+agw++uijTkxlV5uZXXrppU7MV1pdXQN1X/pKWKsdXXyfgSpD/P7778u2FVnt2rWdmG/+UfPKkdKlSxcndssttzixRo0ayf4PP/ywE7v77rudmCqXbWa2fv16J9a0aVPZVu00osoF++4LdW/5dipSOxeo70ffsdR94SujXVGoOURdU9/uP8qSJUtk/Mwzz3RiK1ascGJ5eXmyf1ZWlhNT5++7V9V86/v81TpArbnUrlxmZvXr13diaucMM7OvvvpKxksTfmEGAAAAQrBgBgAAAEKwYAYAAABCsGAGAAAAQpTKpD+VLKFKRZuZrVq1KvLr1qlTx4mpZA1fIlYsCVqKSnjyJRiqhAOViKMSQMx0gt/KlStlW5Uw4LveKMyXjBo1aVMlXJnpcr/Dhw+XbVWp0b/85S9OrG3btrL/1q1bndiVV17pxNauXSv7b9682Yn5EmzT09OdmCrLqhKTzPTc4CvZre6B7777TrZVykKp1lioEuZmZr/97W+d2OLFi2VbVS5azYsNGjSQ/VUJ5+zsbNlWjSGVtKnKwJuZ3XHHHZHOyzcvKur8fXGVTOtLvFZtfXPImjVrnJjvfonKlyBYUfjm8YM1adJExtVn4vv8atWq5cTUOsa3IUBOTo4TU2XY1b1qppP2fNS9qZJhfXOwSjz0Jc6S9AcAAACUcSyYAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBClcpcMtUODL4t3/vz5kV9XlQtW2Z2qhLWZ3k1CZaf6suvV+/KVpFTHUq/r66/azp07V7atXr26E1M7JyC6s88+24mlpaU5MV/WtSrB+s0338i2rVq1cmK9evVyYuvWrZP9f/zxRyemsr59u8Scd955Tsx3v+7YscOJqbLAbdq0kf1VqVa1+42Z2SuvvCLjFdV1110n42rnCDVXmukdTTZt2uTEfGNVfa6+z0mVfFflsjt16iT7q50j1G4Evp0v1I4svvLwCxcudGKqtLFvNwZ1LDXWzfQuB2rnBd/OBaptedsR5kjJzMyUcfVZqx24zPTOSO3atXNin3/+uexfr149J6Z2X/HN92rNoMafmVnnzp2dmFpz+XYfysjIcGLq+8rMrFIldznqu4YlhV+YAQAAgBAsmAEAAIAQLJgBAACAECyYAQAAgBClMulPJQz5EvFmzZoV+XVVaWCVsOJLllClKqOW1DTTiXwqFovu3bvLuCqZrZK7zHQiDaWxo5kzZ46Mb9y40YmpJCZfIp1K7EhKSpJtfUlLB8vKypLxDRs2OLGWLVs6MV8JY3Vv+pKj1Lhq1KiRE1OJSWZmPXv2dGLHHnusbKtKIysq2cSs9CWcFNUXX3wh46oEtfr8zXQinfqsW7RoIfura3366afLtio5SZ2Xryy0mgPVd4tvDlbzpa+0tRrD6r5U95pZ9HM10wmZqjz9ihUrZH+V4HfXXXfJtmWZ+lxj+b5Wn4lvDp45c6YT831+ar7Mzc11YmpDATOzxMREJxbL5gXq8/e9L1VeW93vvvWCuofVWDXT3y+LFi2SbZWift5R8AszAAAAEIIFMwAAABCCBTMAAAAQggUzAAAAEKJUJv2pB7V9lYh8FaWUyZMnOzH1UPpPP/0k+6ukKfVgve9cY3kAXVVaU0kIL774ouyvEmEWL14s23bt2tWJUfnJ1b59eyeWk5Mj26prrfpv2bJF9leVk3wJR77knoMdf/zxMt68eXMnpqqJ+cavGiu+ZEZVVfKNN95wYioZ18zstddeixSLRXlL7vO56qqr/v/27tclsjAK4/jZJmgxiGVQg0nFYnGKCAa7yWZRNFgEsSgTJwgmk9Fm9E+wCBoFQcTgDwwqgiCY3bx7nnP2vTur6zjfTzy8d+bOzJ2ZlwvPObKufleiwOTKyoqrqdCgekwzPb0uChiq32b1WUWhTXW9qsCRCnOb6ZB4RF3D6jsQvdbLy0tXi6bKqUClCn2dnp7K41UgeW9vT65tZ+rzrxLwVVPqooYAatqo+g010+//wMBA8XOpyZyqFk32Vd+raGJwX1+fq6nvsJpeGIkC4SrorkJ/UUj3Xwf8FO4wAwAAAAk2zAAAAECCDTMAAACQYMMMAAAAJNgwAwAAAIkv2SVDdQNQaU0zs+vr6+LHbTQaf31O30E0ZlIlhKOEbyfb2tpytaibiOqIodZGyV6VcL66upJrVaeWWq3malH3F5V6VqNSo84X6nVFo60fHx9dbXFxUa5VVJq7Slea6DV8N1GSXFEddTY2NuRaVZ+fn3e19fV1efzExETxeanPr8rrKhV1ZNnZ2XG17e1tufbp6cnVlpeXXW1ubk4e39/f72pqBLaZ2eHhoaupjhpjY2Py+N3dXVnvBFU64oyPj7ta1KlIdX6IRlur6011Senu7pbHn52duZr6rkSjsVWnkOi5bm5uXE118FLjus30+xX956kuZMpndMOIcIcZAAAASLBhBgAAABJsmAEAAIAEG2YAAAAg8SVDfyqYo0bqmpnd398XP64agdruI6CrjImMQmMqXFLlfe0Ux8fHrjY7OyvXqhCF+qzUSFMzs9XV1eLzUp/16+urq0WjVlWQTl0T6jHNdJDm5eVFrp2ZmXG15+dnuVaJgjT4VZVgjLouqxx/cHBQVItMT0/LugoIRkE4RY2MPzo6crVohHCr1Ljpk5MTuVaFo1QY08ysp6fH1VTo7OHh4U+n+K21el2r60+NJTfTv0sXFxdyrRqDPTg46Gpq3LaZ2ejoqKup/VEUcFRro8Cd+m+Ympoqfq6uri5Xi77D0X/GV8IdZgAAACDBhhkAAABIsGEGAAAAEmyYAQAAgAQbZgAAACDx470wNvoRI0kja2trrjYyMiLXLi0tFT9up3fJiOzv77ua6pKxublZfmIV/M9Rl61e1yrdbGY2OTnpasPDw0U1M7Pe3l5Xi8aXRiOvfxd1mHh7e3M1lfCORqur7iF3d3dF51RVq8n3z9TO1zUQ6eTremhoSNZV55Lz83O5tlarudrCwoKrNZtNebx6/9UY79vbW3l8vV4vOicz/RrUaOxovLzqiKH+b8yqdUv6CCXXNXeYAQAAgAQbZgAAACDBhhkAAABIsGEGAAAAEsWhPwAAAKATcYcZAAAASLBhBgAAABJsmAEAAIAEG2YAAAAgwYYZAAAASLBhBgAAABJsmAEAAIAEG2YAAAAgwYYZAAAASPwE3tse9QMU9Z8AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 900x900 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot more images\n",
    "torch.manual_seed(42)\n",
    "fig = plt.figure(figsize=(9, 9))\n",
    "rows, cols = 4, 4\n",
    "for i in range(1, rows*cols+1):\n",
    "    random_idx = torch.randint(0, len(train_data), size=[1]).item()\n",
    "    img, label = train_data[random_idx]\n",
    "    fig.add_subplot(rows, cols, i)\n",
    "    plt.imshow(img.squeeze(), cmap=\"gray\")\n",
    "    plt.title(class_names[label])\n",
    "    plt.axis(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb287cde-79f8-4ac8-877d-d8cc25f54324",
   "metadata": {},
   "source": [
    "Do you think these items of clothing (images) could be modeled with pure linear lines? Or do you think we'll need non-linearities"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "105b9b50-aa6e-409c-ba22-433cde895484",
   "metadata": {},
   "source": [
    "### 2. prepare dataLoader\n",
    "Right now, our data is in the form of pyTorch datasets\n",
    "\n",
    "Dataloader turns our dataset into a python iterable\n",
    "\n",
    "More specifically, we want to turn our data into batches (or mini-batches)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "21806a18-dd25-4bc6-9bcf-fe17f9c1164c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(Dataset FashionMNIST\n",
       "     Number of datapoints: 60000\n",
       "     Root location: data\n",
       "     Split: Train\n",
       "     StandardTransform\n",
       " Transform: ToTensor(),\n",
       " Dataset FashionMNIST\n",
       "     Number of datapoints: 10000\n",
       "     Root location: data\n",
       "     Split: Test\n",
       "     StandardTransform\n",
       " Transform: ToTensor())"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data, test_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "60e158a7-5dde-4d48-a8bd-20cbbb7c0221",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<torch.utils.data.dataloader.DataLoader at 0x157c96d94d0>,\n",
       " <torch.utils.data.dataloader.DataLoader at 0x157ca3406d0>)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "\n",
    "# Setup the batch size hyperparameter\n",
    "BATCH_SIZE = 32\n",
    "\n",
    "# Turn datasets into iterables (batches)\n",
    "train_dataloader = DataLoader(dataset = train_data,\n",
    "                             batch_size = BATCH_SIZE,\n",
    "                             shuffle = True)\n",
    "\n",
    "test_dataloader = DataLoader(dataset = test_data,\n",
    "                            batch_size = BATCH_SIZE,\n",
    "                            shuffle=False)\n",
    "\n",
    "train_dataloader, test_dataloader\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "4f7e1a9a-480c-4658-8fdc-7bc4d820954a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DataLoaders: (<torch.utils.data.dataloader.DataLoader object at 0x00000157C96D94D0>, <torch.utils.data.dataloader.DataLoader object at 0x00000157CA3406D0>)\n",
      "Length of train_dataLoader: 1875, batches of 32...\n",
      "Length of test_dataloader: 313, batches of 32...\n"
     ]
    }
   ],
   "source": [
    "# let's check out what we've create\n",
    "\n",
    "print(f\"DataLoaders: {train_dataloader, test_dataloader}\")\n",
    "print(f\"Length of train_dataLoader: {len(train_dataloader)}, batches of {BATCH_SIZE}...\")\n",
    "print(f\"Length of test_dataloader: {len(test_dataloader)}, batches of {BATCH_SIZE}...\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ff7ce1bb-f9fd-485b-9701-1c7525340c3d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([32, 1, 28, 28]), torch.Size([32]))"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check out what's inside the  trainiing dataloader\n",
    "train_features_batch, train_labels_batch = next(iter(train_dataloader))\n",
    "train_features_batch.shape, train_labels_batch.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "8c988a71-29c6-44b2-a18f-ab2dd8137697",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Image size: torch.Size([1, 28, 28])\n",
      "Label:8, label size: torch.Size([])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGbCAYAAAAr/4yjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAATTUlEQVR4nO3dW4idd7kG8HfNTOaQ49SmTSK0JhoPkVaorRJEhdRi1ZRceMCaNhCtKIiHijeCUq0IRilaqUmuSoNU03pAwWLUXhQPlxaKJ8SGGAnBNibm0MnMZA5Z++7de1R25//urpXs6e8HhWY6z3zfWutb8+TLtE873W63GwAQEQOX+gQAuHwoBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklJgSTlw4EB0Op0Ff1199dWxbdu2OHTo0KU+PbjsDV3qE4Be+NKXvhSbNm2Kbrcbzz77bBw4cCDe9a53xU9+8pO47bbbLvXpwWVLKbAkvfOd74ybbropf33XXXfFunXr4uDBg0oB/hf++IgXhfHx8RgbG4uhof/+fdB9990Xb3rTm+LKK6+MsbGxuPHGG+MHP/jBv2Wnpqbik5/8ZKxduzZWrVoVO3bsiOPHj0en04kvfvGLfXwU0HvuFFiSzp49GydPnoxutxsnTpyIBx54ICYmJuLOO+/Mz/nmN78ZO3bsiDvuuCNmZmbikUceife9733x2GOPxfbt2/Pzdu/eHd/73vdi165dsXXr1vjlL3+54J/DktKFJeShhx7qRsS//TUyMtI9cODAgs+dnJxc8OuZmZnudddd17355pvzY08++WQ3Irp33333gs/dvXt3NyK6X/jCF3r2WOBScKfAkrR379541ateFRERzz77bDz88MPx4Q9/OFatWhXvfve7IyJibGwsP//06dMxPz8fb3nLW+LgwYP58Z/97GcREfGxj31swdf/xCc+EQcOHOjxo4D+UwosSW984xsX/KD5Ax/4QNxwww3x8Y9/PG677bYYHh6Oxx57LL785S/HU089FRcuXMjP7XQ6+fd/+9vfYmBgIDZt2rTg62/evLn3DwIuAT9o5kVhYGAgtm3bFn//+9/j6aefjl//+texY8eOGB0djX379sVPf/rTePzxx2Pnzp3R9X+o5UXMnQIvGnNzcxERMTExET/84Q9jdHQ0fv7zn8fIyEh+zkMPPbQg87KXvSwuXrwYf/3rX+OVr3xlfvzw4cP9OWnoM3cKvCjMzs7GL37xixgeHo4tW7bE4OBgdDqdmJ+fz885evRo/PjHP16Qu/XWWyMiYt++fQs+/sADD/T8nOFScKfAknTo0KH485//HBERJ06ciO9+97vx9NNPx2c/+9lYvXp1bN++Pb7+9a/HO97xjti5c2ecOHEi9u7dG5s3b47f/e53+XVuvPHGeM973hP3339/nDp1Kv+V1L/85S8RsfDnD7AUKAWWpHvuuSf/fnR0NF7zmtfE/v3746Mf/WhERNx8883x4IMPxp49e+Luu++OTZs2xVe/+tU4evToglKIiPj2t78d69evj4MHD8aPfvSjuOWWW+LRRx+NV7/61TE6OtrXxwW91un6qRo0e+qpp+KGG26Ihx9+OO64445LfTrwgvEzBXgeU1NT//ax+++/PwYGBuKtb33rJTgj6B1/fATP42tf+1o8+eSTsW3bthgaGopDhw7FoUOH4iMf+Uhcc801l/r04AXlj4/geTz++ONx7733xp/+9KeYmJiIa6+9Nnbt2hWf+9znFgzswVKgFABIfqYAQFIKAKRF/4Go/0iH/6uBgdrvQS5evNicWbt2bXNm165dzZnKzxTuu+++5kxElDaZKs955fnm/4fFXEPuFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYC06P+fgkE8/qdly5Y1Z2ZnZ0vHuvXWW5sz99xzT3PmW9/6VnNmfHy8OfPmN7+5ORMR8ZWvfKU584c//KE5Uxn5m5uba87QfwbxAGiiFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEgG8ejbuN3b3va25kxExO7du5szu3btKh2rHzZu3FjKfepTn2rOfPrTny4dq1Xl+8Miv/XwAjKIB0ATpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgCkoUt9ArywKmuVlcXTVatWNWd27tzZnIno3+Lp6Ohoc2ZwcLA5c/To0eZMRMTx48ebM7fffntz5pFHHmnODA21fyupXHf0njsFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAIBnEW2IGBtp7fn5+vjnz+c9/vjnz29/+tjlTVRm3m56ebs4sX768OVP1zDPPNGfe8IY3NGcqg3iVcbvKeGNERLfbLeVYHHcKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQDKId5kaHBws5SrjdkND7ZfB61//+ubMvn37mjNVleehYm5uri/HiYh44oknmjPbt29vzlx11VXNmX/84x/Nmcp4Y0TExYsXmzNG9BbPnQIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlswgXqfT6ctxKsNaleGv6lhYZQhu48aNzZnR0dHmzMmTJ5szVf26HirjbFWV0bmrr766OXPnnXc2Z77xjW80Z6ojddX3RqvK+VWvu8qxenXtuVMAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUA0pIZxOvXUF2/hqv6ObS2Zs2avhynn4/pch5Nq5qZmWnODA21v8Wvv/765kxFP68HFs+dAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBpyaykVvRrpXF4eLg5s2zZstKxzp8/35y57rrrmjMrVqxozkxNTTVnqqanp/tynPn5+b4cp2pycrI5s27duh6cyYvD6OhoKdev63Ux3CkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIA6bIbxBsfHy/lvvOd7zRnfvOb3zRnXv7ylzdnzp0715xZtWpVcyYiYmCgvefHxsaaM5Vxu/e///3NmYiI48ePN2cqz8PMzExzpjIet2HDhuZMRMSpU6dKuVZXXHFFc+Yzn/lMc6YyFBlRe29UMpXz++c//9mciYg4fPhwc+bBBx8sHev5uFMAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAUqfb7XYX9YmdTq/PJSIi1q9fX8rt37+/OXPs2LHmTGXw6sKFC82Z5cuXN2ciIq688srmzLJly5ozK1asaM5cc801zZmIiEVeov/nTOW1nZuba85UXqOI2ut05syZ5szs7GxzpvL9YXp6ujkTUXs/VcYEK49p5cqVzZmIiNWrVzdnbr/99ubMYt4X7hQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGANHSpT+Bfbdy4sZSrjFetXbu2OTM01P6Uzc/PN2cq42cREWvWrGnOnDhxojnz+9//vjlTGamLqD0XY2NjzZnKa1s5TmXQLaI2IPfMM880Z44cOdKcuf7665szlfdFRMTFixebMxs2bCgdq9VLX/rSUm7Pnj0v8JnUuVMAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAIHW6i5yurKyQvv3tb2/OvPe9723ORNQWT1esWNGcOXnyZHOmsvI5MFDr68HBweZMZR20onJuERHnzp17gc/kP6tc45XXqbLyGRExNzfXnBkZGWnOVK7XymOqruZWnD9/vjkzPj7enFm9enVzJiLi8OHDzZm77rqrObOY59ydAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJB6uoR2yy23NGf27t1bOta9997bnKkMoK1bt645Uxn+qoySRUQMDw83Z/o1TFYd+asMk83MzDRnKuNx1cdUUbleL1y40JypDO9VRhVnZ2ebMxG198b8/HxfjlMZ2YyIeMUrXlHK9YI7BQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACD1dBBv5cqVzZkjR46UjrV69ermTGXEqzK0Nj093Zy5ePFicyaiNoC2fPny5kzleejX8F5ExNTUVHPm/PnzzZnKiF5V5fmrXHuVobrR0dHmTGWkLiJizZo1zZnKUGRlEO8lL3lJcyYi4vvf/34p1wvuFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYDU00G8yuDVc88914Mz+c86nU5zZnBwsDlTGTKbnJxszkTURv4qg3hjY2PNmeogXmV8r3J+lRHCygBhdQiuonKsSqbyGlWdPn26OVO59iqDeBMTE82ZiIiBgcvn9+eXz5kAcMkpBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAICkFAFJPB/H6qTJuVxkzu+qqq5oza9eubc5UhwErQ3qVTGVEb25urjkTURuqq2Qq51cZIKxcqxG1obrK+VWeu8rgXOXcIiLGx8ebM8PDw82ZyvUwOjranLncuFMAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAIPV0JbWfi4GVVdHK0ufs7GxzpnJuAwO1vq4sSFYeU0VlfTMiYmJiojlTeUyV86ssl1Zf22XLljVn1qxZ05xZtWpVc6aiupo7MzPTnDl//nxzpvJ8V5+7o0ePlnK94E4BgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASEtmEO/s2bPNmcp4XGUArTLo1u12mzMREZOTk82ZyhBc5bWtjoUNDw83ZyoDaJXnoXJu1eehMuBYeUyVAcfK+6/T6TRnqrmpqanmzIYNG5oz09PTzZmIiF/96lelXC+4UwAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQBSTwfxxsbGevnlF1i2bFlzpjI6NzMz05xZuXJlc6Y6iDcw0N7zldG0ubm55szp06ebMxERIyMjzZmhofZLuzJuV3m+T5061ZyJiDh+/HhzpvK+qAzOVY4zODjYnImoPeeV67Uy+lgZv4yovba94k4BgKQUAEhKAYCkFABISgGApBQASEoBgKQUAEhKAYCkFABISgGApBQASD0dxOunFStWNGeWL1/enJmammrOVMa4KqNfEbVxu0qmcn6VobWI2vlVBgXPnz/fnJmenm7OVFWGASsjf5XX6XK/HiqDfWfOnGnObNmypTkTEbFt27bmzBNPPFE61vNxpwBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgCkng7iPffcc7388gvMzs42ZyojWZXHND8/35wZGqq9NJUhuMHBweZMPwfQKo+pXyrnVhl0i6i9ThcuXGjOVF6nfmWqucrYYeX7w9mzZ5szERG7d+9uzhjEA6DnlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQerqSeubMmV5++QUqK4j9WgetZPq5klpZnayeX8Xc3FxfjlN5TJUlzcpyaURtXXVkZKQvx+nnWmxF5bWtvG8nJyebMxERmzdvLuV6wZ0CAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkHq6anb48OHmzE033VQ61h//+MfmzNatW5szlRG90dHR5kx1NK0y0FZ5TJUBtPn5+eZM9VgVs7OzzZnKqFs/n4d+DcH1KxNRe28MDw83ZyrnV71Wjx8/Xsr1gjsFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAIHW6i1xw6nQ6vT6XiIjYv39/Kbdly5bmTGXMrDI4t2LFiuZM1blz55ozlYG26phZRWVkrJKpXOP9OreI/r0HK/p5bpVrr3J+lff62NhYcyYi4tixY82ZHTt2NGcWc+25UwAgKQUAklIAICkFAJJSACApBQCSUgAgKQUAklIAICkFAJJSACApBQDS0KU+gX81MTFRyq1fv745c+LEiebM+Ph4c2ZoqP1pnp2dbc5ERFx77bXNmU2bNjVnjhw50pyZmZlpzlRz1dG5VpUxwcoQYzVXGYKrZPo1UldVeQ9Wzq9yPUT073pdDHcKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKAKTLbiX10UcfLeU+9KEPNWeOHTvWnLlw4UJzprJu+brXva45ExHxwQ9+sDlz6NCh5kxllbby3FVNTk42Z6orrq2qi5iVXCVTWRQdHh7uy3Eiauulc3NzfclUr6GRkZFSrhfcKQCQlAIASSkAkJQCAEkpAJCUAgBJKQCQlAIASSkAkJQCAEkpAJCUAgCp013kYlZlhKqftm7d2py54oormjOvfe1rmzNjY2PNmcoYV0TEnj17Sjlg6VvMt3t3CgAkpQBAUgoAJKUAQFIKACSlAEBSCgAkpQBAUgoAJKUAQFIKACSlAEBa9CAeAEufOwUAklIAICkFAJJSACApBQCSUgAgKQUAklIAICkFANJ/Aa33cJCzUi79AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## show a sample\n",
    "# torch.manual_seed(42)\n",
    "random_idx = torch.randint(0, len(train_features_batch), size=[1]).item()\n",
    "image, label = train_features_batch[random_idx], train_labels_batch[random_idx]\n",
    "plt.imshow(image.squeeze(), cmap=\"gray\")\n",
    "plt.title(class_names[label])\n",
    "plt.axis(False)\n",
    "print(f\"Image size: {img.shape}\")\n",
    "print(f\"Label:{label}, label size: {label.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "cdfdb1e4-5979-4c6b-bcb9-44747d399580",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape before flattening: torch.Size([1, 28, 28])\n",
      "Shape after flattening: torch.Size([1, 784])\n"
     ]
    }
   ],
   "source": [
    "# Create a flatten layer\n",
    "flatten_model = nn.Flatten()\n",
    "\n",
    "# Get a signle sample\n",
    "x = train_features_batch[0]\n",
    "\n",
    "# Flatten the sample\n",
    "output = flatten_model(x) # perform forward pass\n",
    "\n",
    "# print out what happened\n",
    "print(f\"Shape before flattening: {x.shape}\")\n",
    "print(f\"Shape after flattening: {output.shape}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "48f1d06b-ad77-4678-a101-fff43ae2f1a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# start to code the model\n",
    "from torch import nn\n",
    "class FashionMNISTModelV0(nn.Module):\n",
    "    def __init__(self,\n",
    "                input_shape: int,\n",
    "                hidden_units: int,\n",
    "                output_shape: int):\n",
    "        super().__init__()\n",
    "        self.layer_stack = nn.Sequential(\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(in_features=input_shape,\n",
    "                     out_features=hidden_units),\n",
    "            nn.Linear(in_features=hidden_units,\n",
    "                     out_features=output_shape)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.layer_stack(x)\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "52c4db7c-fc63-4c3d-88d4-4b76e0cffc99",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FashionMNISTModelV0(\n",
       "  (layer_stack): Sequential(\n",
       "    (0): Flatten(start_dim=1, end_dim=-1)\n",
       "    (1): Linear(in_features=784, out_features=10, bias=True)\n",
       "    (2): Linear(in_features=10, out_features=10, bias=True)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.manual_seed(42)\n",
    "\n",
    "#setup model with iunput parameters\n",
    "model_0 = FashionMNISTModelV0(\n",
    "    input_shape=784,\n",
    "    hidden_units=10,  # =how many units in the hidden layer\n",
    "    output_shape=len(class_names)\n",
    ")\n",
    "\n",
    "model_0.to(\"cpu\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "fe8a96c8-3334-49bc-86e7-f21620810a73",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.0315,  0.3171,  0.0531, -0.2525,  0.5959,  0.2112,  0.3233,  0.2694,\n",
       "         -0.1004,  0.0157]], grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dummy_x = torch.rand([1, 1, 28, 28])\n",
    "model_0(dummy_x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38f9db84-935f-4071-a27e-0adeca6af893",
   "metadata": {},
   "source": [
    "### 3.1 Setp up loss , optimizer and evaluation metrics\n",
    "* Loss function - since we are working with mult-class data, our loss function will be 'nn.CrossEntropyLoss()'\n",
    "* Optimizer - our optimizer `torch.optim.SGD()` (stochastic gradient descent)\n",
    "* Evaluation metric - since we are working on a classification problem, lets use accruacy as our evaluation metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "066c33b6-3db0-456f-ab6c-2ccaffecf9b2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "help_funtion already exists\n"
     ]
    }
   ],
   "source": [
    "import requests \n",
    "from pathlib import Path\n",
    "#download helper fuinction from learn pytorch repo \n",
    "if Path(\"helper_functions.py\").is_file():\n",
    "    print(\"help_funtion already exists\")\n",
    "else:\n",
    "    print(\"Download helper_functions\")\n",
    "    request = requests.get(\"https://raw.githubsercontent.com/mrdbourke/pytorch-deep-learning/main/helper_functions.py\")\n",
    "    with open(\"helper_functions.py\", \"wb\") as f:\n",
    "        f.write(request.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "789501a1-b3f5-44c3-8a3e-1af81e7e4b29",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import the accuracy metric\n",
    "from helper_functions import accuracy_fn\n",
    "\n",
    "## set up loss functiopn and optimizer\n",
    "loss_fn = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.SGD(params=model_0.parameters(), lr=0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1428f4e2-f5b8-4958-b45e-4235cb2ade5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from timeit import default_timer as timer\n",
    "\n",
    "def print_train_time(start: float,\n",
    "                    end: float,\n",
    "                    device: torch.device = None):\n",
    "    \"\"\"Prints difference between start and end time\"\"\"\n",
    "    total_time = end - start\n",
    "    print(f\"Train time on {device}: {total_time:.3f} second\")\n",
    "    return total_time\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "5387a40a-46ba-421a-8278-ff3b73a1006c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train time on cpu: 0.000 second\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.550000160932541e-05"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "start_time = timer()\n",
    "## some code goes here\n",
    "\n",
    "end_time = timer()\n",
    "print_train_time(start=start_time, end=end_time, device=\"cpu\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "42e96863-477d-414c-9b82-897300a20b30",
   "metadata": {},
   "outputs": [],
   "source": [
    "## highlight that the optimizer will update a model parameters once per batch\n",
    "## 3.3 Create a traing loop and training a modl onbatches of data\n",
    "# 1. loop through epochs.\n",
    "# 2. loop through training batches, perform training steps, calculate the train loss per batch\n",
    "# 3. loop through testing batches, perform testing steps, calculate the test loss per batch,\n",
    "# 4. Print out what's happening.\n",
    "# 5. Time it all (for fun)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "fd98535e-2846-4eac-ba62-acfb38019724",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Sheng\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "  0%|                                                                                            | 0/3 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0 \n",
      "--------\n",
      "Looked at 0/60000\n",
      "Looked at 12800/60000\n",
      "Looked at 25600/60000\n",
      "Looked at 38400/60000\n",
      "Looked at 51200/60000\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 33%|████████████████████████████                                                        | 1/3 [00:06<00:13,  6.53s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Loss: 0.5904 | Test loss: 0.5095, Test_acc: 82.0387\n",
      "Epoch: 1 \n",
      "--------\n",
      "Looked at 0/60000\n",
      "Looked at 12800/60000\n",
      "Looked at 25600/60000\n",
      "Looked at 38400/60000\n",
      "Looked at 51200/60000\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 67%|████████████████████████████████████████████████████████                            | 2/3 [00:13<00:06,  6.70s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Loss: 0.4763 | Test loss: 0.4799, Test_acc: 83.1969\n",
      "Epoch: 2 \n",
      "--------\n",
      "Looked at 0/60000\n",
      "Looked at 12800/60000\n",
      "Looked at 25600/60000\n",
      "Looked at 38400/60000\n",
      "Looked at 51200/60000\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:20<00:00,  6.76s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Loss: 0.4550 | Test loss: 0.4766, Test_acc: 83.4265\n",
      "Train time on cpu: 20.273 second\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# import tqdm for progress bar\n",
    "from tqdm.auto import tqdm\n",
    "\n",
    "## set the seed and start the timer\n",
    "torch.manual_seed(42)\n",
    "train_time_start_on_cpu = timer()\n",
    "## set the number of epochs (we will keep this small for faster training time)\n",
    "epochs = 3\n",
    "\n",
    "# Create training and test loop\n",
    "for epoch in tqdm(range(epochs)):\n",
    "    print(f\"Epoch: {epoch} \\n--------\")\n",
    "    ### Training\n",
    "    train_loss=0\n",
    "    # Add a loop to loop through the training batches\n",
    "    for batch, (X, y) in enumerate(train_dataloader):\n",
    "        model_0.train()\n",
    "        # 1. Forward pass\n",
    "        y_pred = model_0(X)\n",
    "        # 2. Calculate loss(per batch)\n",
    "        loss = loss_fn(y_pred, y)\n",
    "        train_loss += loss ## sum up the loss then devided the len of batch\n",
    "\n",
    "        # 3. optimizer zero grad\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # 4. loss backward\n",
    "        loss.backward()\n",
    "\n",
    "        # 5. Optimizer step\n",
    "        optimizer.step()\n",
    "\n",
    "        # print out what's happening \n",
    "        if batch % 400 == 0:\n",
    "            print(f\"Looked at {batch * len(X)}/{len(train_dataloader.dataset)}\")\n",
    "\n",
    "    # Divide total train loss by len of train dataloader\n",
    "    train_loss /= len(train_dataloader)\n",
    "    ### Testing\n",
    "    test_loss, test_acc = 0, 0\n",
    "    model_0.eval()\n",
    "    with torch.inference_mode():\n",
    "        for X_test, y_test in test_dataloader:\n",
    "            #1. Forward pass\n",
    "            test_pred = model_0(X_test)\n",
    "            \n",
    "            #2. Calculate loss(accumulatively)\n",
    "            test_loss += loss_fn(test_pred, y_test)\n",
    "            #3. accuracy\n",
    "            test_acc += accuracy_fn(y_true=y_test, y_pred=test_pred.argmax(dim=1))\n",
    "\n",
    "        test_loss /= len(test_dataloader)\n",
    "        test_acc /= len(test_dataloader)\n",
    "    #print out what happening \n",
    "    print(f\"\\nTrain Loss: {train_loss:.4f} | Test loss: {test_loss:.4f}, Test_acc: {test_acc:.4f}\")\n",
    "\n",
    "train_time_end_on_cpu = timer()\n",
    "\n",
    "total_train_time_model_0 = print_train_time(start = train_time_start_on_cpu,\n",
    "                                           end = train_time_end_on_cpu,\n",
    "                                           device=str(next(model_0.parameters()).device))\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "e273753b-d243-4a79-9953-b083f563e2ec",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|                                                                                          | 0/313 [00:00<?, ?it/s]\n"
     ]
    },
    {
     "ename": "NameError",
     "evalue": "name 'device' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[26], line 30\u001b[0m\n\u001b[0;32m     25\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel_name\u001b[39m\u001b[38;5;124m\"\u001b[39m : model\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__class__\u001b[39m\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__name__\u001b[39m,\n\u001b[0;32m     26\u001b[0m            \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel_loss\u001b[39m\u001b[38;5;124m\"\u001b[39m: loss\u001b[38;5;241m.\u001b[39mitem(),\n\u001b[0;32m     27\u001b[0m            \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel_acc\u001b[39m\u001b[38;5;124m\"\u001b[39m: acc}\n\u001b[0;32m     29\u001b[0m \u001b[38;5;66;03m# Calculate model 0 results on the test dataset\u001b[39;00m\n\u001b[1;32m---> 30\u001b[0m model_0_res \u001b[38;5;241m=\u001b[39m \u001b[43meval_model\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmodel_0\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m     31\u001b[0m \u001b[43m                         \u001b[49m\u001b[43mdata_loader\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtest_dataloader\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m     32\u001b[0m \u001b[43m                         \u001b[49m\u001b[43mloss_fn\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mloss_fn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m     33\u001b[0m \u001b[43m                         \u001b[49m\u001b[43maccuracy_fn\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43maccuracy_fn\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     34\u001b[0m model_0_res\n",
      "Cell \u001b[1;32mIn[26], line 13\u001b[0m, in \u001b[0;36meval_model\u001b[1;34m(model, data_loader, loss_fn, accuracy_fn)\u001b[0m\n\u001b[0;32m     10\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m torch\u001b[38;5;241m.\u001b[39minference_mode():\n\u001b[0;32m     11\u001b[0m     \u001b[38;5;28;01mfor\u001b[39;00m X, y \u001b[38;5;129;01min\u001b[39;00m tqdm(data_loader):\n\u001b[0;32m     12\u001b[0m         \u001b[38;5;66;03m# make our data device agnostic\u001b[39;00m\n\u001b[1;32m---> 13\u001b[0m         X, y \u001b[38;5;241m=\u001b[39m X\u001b[38;5;241m.\u001b[39mto(\u001b[43mdevice\u001b[49m), y\u001b[38;5;241m.\u001b[39mto(device)\n\u001b[0;32m     14\u001b[0m         \u001b[38;5;66;03m# make predictions\u001b[39;00m\n\u001b[0;32m     15\u001b[0m         y_pred \u001b[38;5;241m=\u001b[39m model(X)\n",
      "\u001b[1;31mNameError\u001b[0m: name 'device' is not defined"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(42)\n",
    "\n",
    "def eval_model(model: torch.nn.Module,\n",
    "              data_loader: torch.utils.data.DataLoader,\n",
    "              loss_fn: torch.nn.Module,\n",
    "              accuracy_fn):\n",
    "    \"\"\"Returns a dictionary containing the results of model predicting on data_loader.\"\"\"\n",
    "    loss, acc = 0, 0\n",
    "    model.eval()\n",
    "    with torch.inference_mode():\n",
    "        for X, y in tqdm(data_loader):\n",
    "            # make our data device agnostic\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # make predictions\n",
    "            y_pred = model(X)\n",
    "\n",
    "            # Accumulate the loss and acc values pre batch\n",
    "            loss += loss_fn(y_pred, y)\n",
    "            acc += accuracy_fn(y_true = y,\n",
    "                              y_pred=y_pred.argmax(dim = 1))\n",
    "        #Scale losss and acc to find the average\n",
    "        loss /= len(data_loader)\n",
    "        acc /= len(data_loader)\n",
    "\n",
    "    return {\"model_name\" : model.__class__.__name__,\n",
    "           \"model_loss\": loss.item(),\n",
    "           \"model_acc\": acc}\n",
    "\n",
    "# Calculate model 0 results on the test dataset\n",
    "model_0_res = eval_model(model=model_0,\n",
    "                         data_loader=test_dataloader,\n",
    "                         loss_fn=loss_fn,\n",
    "                         accuracy_fn=accuracy_fn)\n",
    "model_0_res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d364cc19-08c1-4e14-8dc8-c0e1ec325a58",
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7c4cff9-af79-416b-849a-bb20e306f579",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "model_0.to(\"cuda\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7e5a27b-3da0-480a-822c-fef1bc907212",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "## set the seed and start the timer\n",
    "torch.manual_seed(42)\n",
    "train_time_start_on_cpu = timer()\n",
    "## set the number of epochs (we will keep this small for faster training time)\n",
    "epochs = 3\n",
    "\n",
    "# Create training and test loop\n",
    "for epoch in tqdm(range(epochs)):\n",
    "    print(f\"Epoch: {epoch} \\n--------\")\n",
    "    ### Training\n",
    "    train_loss=0\n",
    "    # Add a loop to loop through the training batches\n",
    "    for batch, (X, y) in enumerate(train_dataloader):\n",
    "        model_0.train()\n",
    "        # 1. Forward pass\n",
    "        y_pred = model_0(X.to('cuda'))\n",
    "        # 2. Calculate loss(per batch)\n",
    "        loss = loss_fn(y_pred.to('cuda'), y.to('cuda'))\n",
    "        train_loss += loss ## sum up the loss then devided the len of batch\n",
    "\n",
    "        # 3. optimizer zero grad\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # 4. loss backward\n",
    "        loss.backward()\n",
    "\n",
    "        # 5. Optimizer step\n",
    "        optimizer.step()\n",
    "\n",
    "        # print out what's happening \n",
    "        if batch % 400 == 0:\n",
    "            print(f\"Looked at {batch * len(X)}/{len(train_dataloader.dataset)}\")\n",
    "\n",
    "    # Divide total train loss by len of train dataloader\n",
    "    train_loss /= len(train_dataloader)\n",
    "    ### Testing\n",
    "    test_loss, test_acc = 0, 0\n",
    "    model_0.eval()\n",
    "    with torch.inference_mode():\n",
    "        for X_test, y_test in test_dataloader:\n",
    "            #1. Forward pass\n",
    "            test_pred = model_0(X_test.to('cuda'))\n",
    "            \n",
    "            #2. Calculate loss(accumulatively)\n",
    "            test_loss += loss_fn(test_pred.to('cuda'), y_test.to('cuda'))\n",
    "            #3. accuracy\n",
    "            test_acc += accuracy_fn(y_true=y_test.to('cuda'), y_pred=test_pred.argmax(dim=1).to('cuda'))\n",
    "\n",
    "        test_loss /= len(test_dataloader)\n",
    "        test_acc /= len(test_dataloader)\n",
    "    #print out what happening \n",
    "    print(f\"\\nTrain Loss: {train_loss:.4f} | Test loss: {test_loss:.4f}, Test_acc: {test_acc:.4f}\")\n",
    "\n",
    "train_time_end_on_cpu = timer()\n",
    "\n",
    "total_train_time_model_0 = print_train_time(start = train_time_start_on_cpu,\n",
    "                                           end = train_time_end_on_cpu,\n",
    "                                           device=str(next(model_0.parameters()).device))\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3040c77a-eed3-48f0-9167-5aafb0137bd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "class FashionMNISTModelV1(nn.Module):\n",
    "    def __init__(self,\n",
    "              input_shape: int,\n",
    "              hidden_units: int,\n",
    "              output_shape: int):\n",
    "        super().__init__()\n",
    "        self.layer_stack = nn.Sequential(\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(in_features=input_shape,\n",
    "                     out_features=hidden_units),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(in_features=hidden_units,\n",
    "                     out_features=output_shape),\n",
    "            nn.ReLU()\n",
    "        )\n",
    "\n",
    "    def forward(self, x: torch.Tensor):\n",
    "        return self.layer_stack(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6354d291-9604-47cf-86bc-6d4f810ccfcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42)\n",
    "model_1 = FashionMNISTModelV1(input_shape=784, hidden_units=10, output_shape=len(class_names)).to(\"cuda\")\n",
    "\n",
    "next(model_1.parameters()).device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88a5403c-1c6a-4032-b8d2-39542975b29b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup device agnostic code\n",
    "import torch\n",
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04a445f3-578a-4d8e-b04d-69380006d0c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from helper_functions import accuracy_fn\n",
    "loss_fn = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.SGD(params=model_1.parameters(), \n",
    "                            lr=0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "475b9ce7-4a00-438e-aa79-9d606ceb280f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_step(model: torch.nn.Module,\n",
    "               data_loader: torch.utils.data.DataLoader,\n",
    "               loss_fn: torch.nn.Module,\n",
    "               optimizer: torch.optim.Optimizer,\n",
    "               accuracy_fn,\n",
    "               device: torch.device = device):\n",
    "    train_loss, train_acc = 0, 0\n",
    "    model.to(device)\n",
    "    for batch, (X, y) in enumerate(data_loader):\n",
    "        # Send data to GPU\n",
    "        X, y = X.to(device), y.to(device)\n",
    "\n",
    "        # 1. Forward pass\n",
    "        y_pred = model(X)\n",
    "\n",
    "        # 2. Calculate loss\n",
    "        loss = loss_fn(y_pred, y)\n",
    "        train_loss += loss\n",
    "        train_acc += accuracy_fn(y_true=y,\n",
    "                                 y_pred=y_pred.argmax(dim=1)) # Go from logits -> pred labels\n",
    "\n",
    "        # 3. Optimizer zero grad\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # 4. Loss backward\n",
    "        loss.backward()\n",
    "\n",
    "        # 5. Optimizer step\n",
    "        optimizer.step()\n",
    "\n",
    "    # Calculate loss and accuracy per epoch and print out what's happening\n",
    "    train_loss /= len(data_loader)\n",
    "    train_acc /= len(data_loader)\n",
    "    print(f\"Train loss: {train_loss:.5f} | Train accuracy: {train_acc:.2f}%\")\n",
    "\n",
    "def test_step(data_loader: torch.utils.data.DataLoader,\n",
    "              model: torch.nn.Module,\n",
    "              loss_fn: torch.nn.Module,\n",
    "              accuracy_fn,\n",
    "              device: torch.device = device):\n",
    "    test_loss, test_acc = 0, 0\n",
    "    model.to(device)\n",
    "    model.eval() # put model in eval mode\n",
    "    # Turn on inference context manager\n",
    "    with torch.inference_mode(): \n",
    "        for X, y in data_loader:\n",
    "            # Send data to GPU\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            \n",
    "            # 1. Forward pass\n",
    "            test_pred = model(X)\n",
    "            \n",
    "            # 2. Calculate loss and accuracy\n",
    "            test_loss += loss_fn(test_pred, y)\n",
    "            test_acc += accuracy_fn(y_true=y,\n",
    "                y_pred=test_pred.argmax(dim=1) # Go from logits -> pred labels\n",
    "            )\n",
    "        \n",
    "        # Adjust metrics and print out\n",
    "        test_loss /= len(data_loader)\n",
    "        test_acc /= len(data_loader)\n",
    "        print(f\"Test loss: {test_loss:.5f} | Test accuracy: {test_acc:.2f}%\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e1feb4b-2da9-4d83-a953-15ec138c552a",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42)\n",
    "\n",
    "# Measure time\n",
    "from timeit import default_timer as timer\n",
    "train_time_start_on_gpu = timer()\n",
    "\n",
    "epochs = 3\n",
    "for epoch in tqdm(range(epochs)):\n",
    "    print(f\"Epoch: {epoch}\\n---------\")\n",
    "    train_step(data_loader=train_dataloader, \n",
    "        model=model_1, \n",
    "        loss_fn=loss_fn,\n",
    "        optimizer=optimizer,\n",
    "        accuracy_fn=accuracy_fn\n",
    "    )\n",
    "    test_step(data_loader=test_dataloader,\n",
    "        model=model_1,\n",
    "        loss_fn=loss_fn,\n",
    "        accuracy_fn=accuracy_fn\n",
    "    )\n",
    "\n",
    "train_time_end_on_gpu = timer()\n",
    "total_train_time_model_1 = print_train_time(start=train_time_start_on_gpu,\n",
    "                                            end=train_time_end_on_gpu,\n",
    "                                            device=device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8206448c-7ad6-47e3-95cb-9466bad8ec92",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get model_1 result dictionary \n",
    "model_1_results = eval_model(model= model_1,\n",
    "                            data_loader=test_dataloader,\n",
    "                            loss_fn=loss_fn,\n",
    "                            accuracy_fn=accuracy_fn)\n",
    "\n",
    "model_1_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5297d75-298e-4013-bcaf-77fc0bde6439",
   "metadata": {},
   "source": [
    "### Convolutional neural newworks [卷积神经网络] -- CNN \n",
    "\n",
    "* CNN are known for their capabilities to find patterns in visual data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4455f4a-7d4e-4a68-b8ec-58de29194bcc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}