02 tensorflow.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "celltoolbar": "Slideshow",
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    },
    "colab": {
      "name": "02 tensorflow.ipynb",
      "provenance": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "81_TVtnsOh1l",
        "colab_type": "text"
      },
      "source": [
        "# Семинар 2: Введение в tensorflow\n",
        "\n",
        "На этом семинаре мы познакомимся с классическим фреймворком `tensorflow`, разберёмся с автоматическим дифференцированием, а также обучим нашу первую нейросеть."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gj-LkiyoOh10",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7iwS_xCbOh2Q",
        "colab_type": "text"
      },
      "source": [
        "# Как устроен tf изнутри?\n",
        "\n",
        "Идея tensorflow заключается в построении так называемых **графов вычислений**, где вершины представляют из себя **тензоры** (многомерные массивы, в точности как `np.array` в знакомом вам `numpy`)вместе с **операции** над этими тензорами, а рёбра -- **передачу данных из одной операции в другую**.\n",
        "\n",
        "**ВНИМАНИЕ!**\n",
        "\n",
        "Вводя каким-нибудь образом тензоры и проводя над ними операции в питоновском коде Вы эти операции **не запускаете**, а лишь **определяете их заранее**, конструируя при этом граф вычислений в памяти. Как запускать эти определённые только что операции для получения результата мы узнаем немного позже.\n",
        "\n",
        "Два основных вида тензоров:\n",
        "\n",
        "1. `tf.placeholder` -- это тензор, который необходимо инициализировать руками **на этапе запуска операции**. Удобно использовать его как мнимый вход нейронной сети, которому мы впоследствии присвоим данные для обучения и тестирования.\n",
        "2. `tf.Variable` -- это тензор-переменная, который имеет некоторую **инициализацию по умолчанию**. Удобно под ним понимать обучаемые параметры нейронных сетей, линейных моделей и т.п.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V9vYUDIJOh2W",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Когда Вы создаёте плейсхолдер, обязательно необходимо указывать его тип, в данном случае это float32.\n",
        "input_vector = tf.placeholder(\"float32\",)\n",
        "\n",
        "# Опционально Вы можете указать ещё и размерность тензора. \n",
        "# При запуске операций его размерность станет известна, лучше указывать это заранее, это очень полезно для дебага\n",
        "input_vector_with_shape = tf.placeholder(\"float32\", shape=(12,))\n",
        "\n",
        "# Вы также можете указать размерности, которые заранее неизвестны (например, размер батча)\n",
        "# при помощи ключевого слова None\n",
        "input_vector_with_unknown_dim = tf.placeholder(\"float32\", shape=(None, None, 12))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JMkFQE2AOh2n",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Для переменных необходимо подать значение по умолчанию\n",
        "# Тип переменной может быть определён по значению по умолчанию\n",
        "variable_vector = tf.Variable(0.)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TC3RmTQcOh23",
        "colab_type": "text"
      },
      "source": [
        "Ниже мы попробуем поделать какие-нибудь операции над тензорами. Они фактически точно такие же, как в `numpy`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z3f8kDf7Oh28",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.reset_default_graph()\n",
        "x = tf.placeholder(\"float32\", shape=(12,))\n",
        "y = tf.placeholder(\"float32\", shape=(12,))\n",
        "\n",
        "# Поэлементное сложение, умножение, деление, whatever векторов\n",
        "z = x + y\n",
        "z = x - y\n",
        "z = x * y\n",
        "\n",
        "# Нахождеие среднего и среднего по всем элементам\n",
        "# np.sum -> tf.reduce_sum\n",
        "# np.mean -> tf.reduce_mean\n",
        "s = tf.reduce_sum(x)\n",
        "m = tf.reduce_mean(x)\n",
        "\n",
        "# Математические операции\n",
        "sin_x = tf.sin(x)\n",
        "exp_x = tf.exp(x)\n",
        "\n",
        "# Скалярное произведение\n",
        "z = tf.tensordot(x, y, 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EcyrqXKXOh3O",
        "colab_type": "text"
      },
      "source": [
        "Список всевозможных трансформаций тензоров мы советуем искать в официальной документации к tensorflow, она очень качественно и подробно описана. В упражнениях мы попробуем давать хинты к операциям, которые необходимо использовать."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "718DWU2JOh3U",
        "colab_type": "text"
      },
      "source": [
        "Tensorflow также предоставляет возможность использования уже инициализированных тензоров, в точности как в numpy."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "034xyxKJOh3b",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# np.arange -> tf.range\n",
        "range_12 = tf.range(12)\n",
        "\n",
        "# np.zeros, np.ones -> tf.zeros, tf.ones\n",
        "zeros_12 = tf.zeros(12)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mbDOvPRYOh3v",
        "colab_type": "text"
      },
      "source": [
        "**Упражнение** Попробуйте сами поиграть с тензорами и вычислить значение выражения\n",
        "\n",
        "$$\n",
        "    z = \\cos \\langle \\cfrac{x}{x + y}, \\cfrac{y}{x + y}\\rangle\n",
        "$$\n",
        "\n",
        "Где $\\langle \\cdot, \\cdot \\rangle$ обозначает скалярное произведение векторов."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9bUC6bs3Oh34",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = tf.placeholder(\"float32\", shape=(12,))\n",
        "y = tf.placeholder(\"float32\", shape=(12,))\n",
        "\n",
        "z = tf.cos(tf.tensordot(x / (x + y), y / (x + y), 1))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XvWUwXG6Oh4M",
        "colab_type": "text"
      },
      "source": [
        "Можете не вникать в происходящее, просто запустите ячейку ниже для самопроверки. Должно вывести нечто близкое к -0.8893267."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9tb5FVwdOh4R",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "feafed07-c20b-4aee-dc75-38a78ae89d73"
      },
      "source": [
        "import numpy as np\n",
        "with tf.Session() as sess:\n",
        "    print(sess.run(z, {x: np.ones(12), y: 2*np.ones(12)}))"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "-0.8893267\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PY3zRMZCOh4f",
        "colab_type": "text"
      },
      "source": [
        "**Упражнение посложнее**. Попробуйте имплементировать квадратичную функцию потерь между двумя векторами:\n",
        "\n",
        "$$\n",
        "    MSE(x, y) = \\sum_{i=1}^{n} (x_i - y_i)^2\n",
        "$$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FbyASUa2Oh4k",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = tf.placeholder(\"float32\")\n",
        "y = tf.placeholder(\"float32\")\n",
        "\n",
        "mse = tf.reduce_mean(tf.square(x - y))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DI41NNn8Oh40",
        "colab_type": "text"
      },
      "source": [
        "Тесты для проверки."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AJAvC2vqOh46",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "3a49f6c0-8f46-4a15-90eb-0e179a4e7940"
      },
      "source": [
        "# Tests\n",
        "from sklearn.metrics import mean_squared_error\n",
        "\n",
        "sess = tf.Session()\n",
        "\n",
        "for n in [1,5,10,10**3]:\n",
        "    \n",
        "    elems = [np.arange(n),np.arange(n,0,-1), np.zeros(n),\n",
        "             np.ones(n),np.random.random(n),np.random.randint(100,size=n)]\n",
        "    \n",
        "    for el in elems:\n",
        "        for el_2 in elems:\n",
        "            true_mse = np.array(mean_squared_error(el,el_2))\n",
        "            my_mse = sess.run(mse, {x:el, y:el_2})\n",
        "            if not np.allclose(true_mse,my_mse):\n",
        "                print('Wrong result:')\n",
        "                print('mse(%s,%s)' % (el,el_2))\n",
        "                print(\"should be: %f, but your function returned %f\" % (true_mse,my_mse))\n",
        "                sess.close()\n",
        "                raise ValueError(\"Что-то не так\")\n",
        "sess.close()\n",
        "print(\"All tests passed\")"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "All tests passed\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UeMYgrvdOh5G",
        "colab_type": "text"
      },
      "source": [
        "## А как запускать операции?\n",
        "\n",
        "Для того, чтобы граф вычислений запустился, необходимо населить все тензоры значениями, то есть положить их в оперативную память (или в память на Вашей видеокарте, если у вас таковая есть =)). Всё общение тензорфлоу с аппаратным памятью происходит через **сессии**.\n",
        "\n",
        "Создавая сессию, Вы резервируете место в памяти под тензоры. Через сессию Вы также можете запускать операции.\n",
        "\n",
        "**Замечание** \n",
        "Ваша запускаемая операция может зависеть от других операций, которые в свою очередь зависят от ещё каких-нибудь, и так далее. Тензорфлоу способен сам выявить какие операции от каких зависят и последовательно их запустить."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QwyM33M2Oh5I",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "1609e3a5-a8e3-4a9d-dc45-e15859b555a5"
      },
      "source": [
        "# создаём какой-нибудь простой граф вычислений\n",
        "x = tf.ones(12)\n",
        "y = tf.ones(12)\n",
        "z = tf.reduce_sum(x + y)\n",
        "\n",
        "# создаём сессию\n",
        "sess = tf.Session()\n",
        "\n",
        "# Запускаем операцию\n",
        "print(sess.run(z))\n",
        "sess.close()"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "24.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q7TyxgMGOh5X",
        "colab_type": "text"
      },
      "source": [
        "Если ваша операция как-то зависит от placeholder'ов, то их необходимо явным образом инициализировать через так называемый **feed dict** -- это словарь, где ключом является ваш плейсхолдер, а значением -- numpy-массив с его значением. Этот словать нужно передать вторым аргументом в функции run."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5hjYDsg8Oh5d",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "673a091d-f777-46c8-e887-5631e91dfe47"
      },
      "source": [
        "# абсолютно тот же самый код, но уже с плейсхолдерами\n",
        "x = tf.placeholder(\"float32\", (12,))\n",
        "y = tf.placeholder(\"float32\", (12,))\n",
        "z = tf.reduce_sum(x + y)\n",
        "\n",
        "# создаём сессию\n",
        "sess = tf.Session()\n",
        "\n",
        "feed_dict = {x: np.ones(12), y: np.ones(12)}\n",
        "# Запускаем операцию\n",
        "print(sess.run(z, feed_dict=feed_dict))\n",
        "\n",
        "sess.close()"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "24.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JL0v_yeVOh5w",
        "colab_type": "text"
      },
      "source": [
        "**ВНИМАНИЕ** Обязательно закрывайте сессии при помощи `sess.close()`  или пользуйтесь конструкцией `with tf.Session() as sess:` после использования иначе Вы рискуете столкнуться с вопросом \"А почему мой компьютер так медленно работает?\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hMO7gnzuOh5z",
        "colab_type": "text"
      },
      "source": [
        "## Подсчёт градиентов.\n",
        "\n",
        "А теперь перейдём к самой главной функции тф'a, а именно к автоматическому подсчёту производных! Дело в том, что если Вы делаете какие-то поддерживаемые тф'ом операции над тензорами, тф умеет по этим операциям считать производные как производные функций многих переменных.\n",
        "\n",
        "Для подсчёта градиента  $\\cfrac{\\partial y}{\\partial x}$ Вы просто вызываете функцию `tf.gradients(y, x)`. \n",
        "\n",
        "Чтобы убедиться, что эта функция действительно считает градиенты, попробуем построить график произвольной гладкой функции и касательной к ней.\n",
        "\n",
        "Напомним, что уравнение касательной к функции f(x) в точке ($x_0, y_0$) выглядит как \n",
        "$$\n",
        "    y(x) = y'(x_0) \\cdot (x - x_0) + y_0\n",
        "$$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mUBY5gREOh57",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 486
        },
        "outputId": "95dd89aa-b2ea-41cc-ef62-a6dfe3e3f98e"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# определяем функцию\n",
        "x = tf.placeholder(\"float32\")\n",
        "y = tf.sin(x * 5) / (x * 5)\n",
        "\n",
        "# вычисляем наклон и сдвиг графика\n",
        "[slope] = tf.gradients(y, x)\n",
        "bias = -slope * x + y\n",
        "\n",
        "with tf.Session() as sess:\n",
        "    plt.figure(figsize=(10,8))\n",
        "    x_support = np.arange(0, 10, 0.01)\n",
        "    y_res = sess.run(y, feed_dict={x: x_support})\n",
        "    slope_res, bias_res = sess.run([slope, bias], feed_dict={x: 4.})\n",
        "    plt.plot(x_support, y_res, label=\"function\")\n",
        "    plt.plot(x_support, slope_res * x_support + bias_res, label=\"derivative\")\n",
        "    plt.legend()\n",
        "    plt.show()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl8AAAHVCAYAAADPfJ4+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd81dX9x/HXybgJudmLvbdAWGGD\nCGIBQdyCm1al1tXa2v7QtoI4q9YqVatQdymO2iriRgGZSthbIKyEmQFkr/v9/XEBEVlJvvd+b5L3\n8/Ho48G9+eZ7PolpeHPO+X6OsSwLEREREfGPIKcLEBEREalLFL5ERERE/EjhS0RERMSPFL5ERERE\n/EjhS0RERMSPFL5ERERE/EjhS0RERMSPFL5ERERE/EjhS0RERMSPQpwu4HQSExOtFi1aOF2GiIiI\nyFktX748y7KspHO5NmDDV4sWLUhLS3O6DBEREZGzMsbsPNdrtewoIiIi4kcKXyIiIiJ+pPAlIiIi\n4kcBu+dLREREqqesrIyMjAyKi4udLqXWCA8Pp0mTJoSGhlb5HgpfIiIitVRGRgZRUVG0aNECY4zT\n5dR4lmWRnZ1NRkYGLVu2rPJ9tOwoIiJSSxUXF5OQkKDgZRNjDAkJCdWeSVT4EhERqcUUvOxlx/dT\n4UtERETEjxS+RERExGemTp1Kx44duf766225344dO/j3v/99/HVaWhr33HOPLff2F224FxEREZ95\n8cUXmTNnDk2aNLHlfsfC13XXXQdAamoqqampttzbXxS+RERE6oCHPlrPhj1HbL3neY2imXRJp9N+\n/Pbbbyc9PZ2RI0eya9cu/vznP3PfffcB0LlzZ2bPng3AyJEjGThwIIsXL6Zx48Z8+OGH1KtXj61b\nt3L77bdz8OBBgoODee+995g4cSIbN26kW7du3HzzzXTv3p2nn36a2bNnk5OTwy9+8QvS09OJiIhg\n2rRppKSkMHnyZHbt2kV6ejq7du3iN7/5jaOzZVp2FBEREZ946aWXaNSoEXPnzuXee+897XVbtmzh\nzjvvZP369cTGxvL+++8DcP3113PnnXeyevVqFi9eTMOGDXniiScYNGgQq1at+sk9J02aRPfu3Vmz\nZg2PPfYYN9100/GPbdq0ic8//5zvvvuOhx56iLKyMt980edAM18iIiJ1wJlmqJzWsmVLunXrBkDP\nnj3ZsWMHeXl5ZGZmcvnllwPe5qZns3DhwuPBbejQoWRnZ3PkiHe2b9SoUYSFhREWFkZycjL79++3\nbSm0sjTzJSIiIj4XEhKCx+M5/vrEXllhYWHH/xwcHEx5ebnt4/tjjHNlS/gyxrxqjDlgjFl3mo8b\nY8xUY8xWY8waY0wPO8YVERGRmqFFixasWLECgBUrVrB9+/YzXh8VFUWTJk344IMPACgpKaGwsJCo\nqCjy8vJO+TmDBg1ixowZAMybN4/ExESio6Nt/CrsYdfM1+vAiDN8fCTQ9uj/JgD/sGlcERERqQGu\nvPJKcnJy6NSpE88//zzt2rU76+e89dZbTJ06lZSUFPr378++fftISUkhODiYrl278re//e1H10+e\nPJnly5eTkpLCxIkTeeONN3z15VSLsSzLnhsZ0wKYbVlW51N87GVgnmVZM4++3gxcYFnW3tPdLzU1\n1UpLS7OlttPxeCwycotolhDh03FEREScsHHjRjp27Oh0GbXOqb6vxpjllmWdU88Lf+35agzsPuF1\nxtH3HPXKwu0Mf/Ybvli/z+lSREREpI4IqA33xpgJxpg0Y0zawYMHfT7emG6NaNcginveXsnunEKf\njyciIiLir/CVCTQ94XWTo+/9iGVZ0yzLSrUsKzUpKcnnRdWPDuflG3piMDw7Z4vPxxMRERHxV/ia\nBdx09KnHvsDhM+338qcGMeFc1bMJH63ZQ25BqdPliIiISC1nV6uJmcASoL0xJsMYc4sx5nZjzO1H\nL/kESAe2AtOBO+wY1y7X921GabmH91dkOF2KiIiI1HK2dLi3LOvas3zcAu60Yyxf6NAgmi6NY/hk\n7V5uHdTK6XJERESkFguoDfdOGtohmZW7D5GjpUcRERGfmTx5Mk8//fQ5Xz9r1iyeeOKJKo31wQcf\nsGHDhuOvH3zwQebMmVOle9lJ4euooR2SsSyY//0Bp0sRERERoLy8nDFjxjBx4sQqff7J4WvKlCkM\nGzbMrvKqTAdrH9WlcQyJkS7mbz7I5d2dOWhTRETEZz6dCPvW2nvPBl1g5NlnpR599FHeeOMNkpOT\nadq0KT179mTbtm3ceeedHDx4kIiICKZPn06HDh0YP3484eHhrFy5kgEDBpCSkkJaWhqPPvooKSkp\nbN++naCgIAoKCujQoQPp6em8/vrrTJs2jdLSUtq0acNbb73FqlWrmDVrFvPnz+eRRx7h/fff5+GH\nH2b06NFERkbyyiuv8N577wHeo4iefvppZs+ezRdffMGkSZMoKSmhdevWvPbaa0RGRtr6bdPM11FB\nQYZeLeJZtiPX6VJERERqjeXLl/P222+zatUqPvnkE5YtWwbAhAkT+Pvf/87y5ct5+umnueOOH57F\ny8jIYPHixTzzzDPH34uJiaFbt27Mnz8fgNmzZzN8+HBCQ0O54oorWLZsGatXr6Zjx4688sor9O/f\nnzFjxvDUU0+xatUqWrduffxew4YN49tvv6WgoACAd955h3HjxpGVlcUjjzzCnDlzWLFiBampqT+q\nwS6a+TpBz+ZxfLpuH/sOF9MgJtzpckREROxzDjNUvrBgwQIuv/xyIiK8R/mNGTOG4uJiFi9ezNVX\nX338upKSkuN/vvrqqwkODv7JvcaOHcs777zDkCFDePvtt48HtnXr1vGnP/2JQ4cOkZ+fz/Dhw89Y\nU0hICCNGjOCjjz7iqquu4uOPP+bJJ59k/vz5bNiwgQEDBgBQWlpKv379qv09+Mn4tt+xBkttEQ/A\n8p25jEpp6HA1IiIitZPH4yE2NpZVq1ad8uNut/uU748ZM4YHHniAnJwcli9fztChQwEYP348H3zw\nAV27duX1119n3rx5Z61h3LhxPP/888THx5OamkpUVBSWZXHRRRcxc+bMKn9t50LLjifo1Cia8NAg\n0nbmOF2KiIhIrXD++efzwQcfUFRURF5eHh999BERERG0bNny+J4ry7JYvXr1We8VGRlJr169+PWv\nf83o0aOPz47l5eXRsGFDysrKmDFjxvHro6KiyMvLO+W9Bg8ezIoVK5g+fTrjxo0DoG/fvixatIit\nW7cCUFBQwPfff1+tr/9UFL5OEBocROdGMazNOOx0KSIiIrVCjx49GDt2LF27dmXkyJH06tULgBkz\nZvDKK6/QtWtXOnXqxIcffnhO9xs7diz/+te/GDt27PH3Hn74Yfr06cOAAQPo0KHD8ffHjRvHU089\nRffu3dm2bduP7hMcHMzo0aP59NNPGT16NABJSUm8/vrrXHvttaSkpNCvXz82bdpU3W/BTxhv/9PA\nk5qaaqWlpfl93EkfruM/yzNYO3k4QUHG7+OLiIjYZePGjXTs2NHpMmqdU31fjTHLLctKPZfP18zX\nSc5rFE1BaQU7cwqdLkVERERqIYWvk3RqFAPAhj1HHK5EREREaiOFr5O0rR9JSJBh/R7t+xIRkZov\nULcX1VR2fD8Vvk4SFhJMm+RINu079dMRIiIiNUV4eDjZ2dkKYDaxLIvs7GzCw6vXC1R9vk6hdXIk\n6zI18yUiIjVbkyZNyMjI4ODBg06XUmuEh4fTpEn1jiFU+DqF1kmRfLp2L8VlFYSH/rTDroiISE0Q\nGhpKy5YtnS5DTqJlx1NokxyJx4Kd2XriUUREROyl8HUKrZO8xxpsPZDvcCUiIiJS2yh8nUKrxEiM\ngW0HFb5ERETEXgpfp1DPFUzj2HoKXyIiImI7ha/TaJ0UqWVHERERsZ3C12m0Took/WABHo96o4iI\niIh9FL5Oo3Wym6KyCvYcLnK6FBEREalFFL5Oo2Wi94nHXWo3ISIiIjZS+DqNZvERAOzKUfgSERER\n+yh8nUbDmHqEBBmFLxEREbGVwtdpBAcZGsfVU/gSERERWyl8nUGz+Ah252rDvYiIiNhH4esMmsZH\nsFszXyIiImIjha8zaBYfQU5BKXnFZU6XIiIiIrWEwtcZNI3zPvG4O0dLjyIiImIPha8zULsJERER\nsZvC1xkcC18ZuQpfIiIiYg+FrzOIiQglOjxEM18iIiJiG4Wvs2gaH6HwJSIiIrZR+DqLRrH12Huo\n2OkyREREpJZQ+DqLRjHh7Dmspx1FRETEHgpfZ9Eoth55xeXq9SUiIiK2UPg6i4ax9QDYe1hLjyIi\nIlJ9Cl9n0SgmHIA9h7T0KCIiItWn8HUWx2a+9mjTvYiIiNhA4ess6keFEWRgrzbdi4iIiA0Uvs4i\nJDiI+tHhmvkSERERWyh8nYOGMeHa8yUiIiK2UPg6B41i62nZUURERGyh8HUOGsXWY8/hYizLcroU\nERERqeEUvs5Bw5hwSss9ZBeUOl2KiIiI1HAKX+egYczRRqvadC8iIiLVpPB1DhoebbS6/4jCl4iI\niFSPwtc5qB99NHzlKXyJiIhI9Sh8nYPESBfGwP4jJU6XIiIiIjWcwtc5CAkOIjEyjANadhQREZFq\nsiV8GWNGGGM2G2O2GmMmnuLjzYwxc40xK40xa4wxF9sxrj8lR4Vpz5eIiIhUW7XDlzEmGHgBGAmc\nB1xrjDnvpMv+BLxrWVZ3YBzwYnXH9bf60eEcyNOyo4iIiFSPHTNfvYGtlmWlW5ZVCrwNXHrSNRYQ\nffTPMcAeG8b1q/rRYdrzJSIiItUWYsM9GgO7T3idAfQ56ZrJwBfGmLsBNzDMhnH9KjkqnOyCEsoq\nPIQGa6uciIiIVI2/UsS1wOuWZTUBLgbeMsb8ZGxjzARjTJoxJu3gwYN+Ku3c1I8Ox7IgK1+zXyIi\nIlJ1doSvTKDpCa+bHH3vRLcA7wJYlrUECAcST76RZVnTLMtKtSwrNSkpyYbS7FM/OgxQuwkRERGp\nHjvC1zKgrTGmpTHGhXdD/ayTrtkFXAhgjOmIN3wF1tTWWRxvtKonHkVERKQaqh2+LMsqB+4CPgc2\n4n2qcb0xZooxZszRy34H3GaMWQ3MBMZblmVVd2x/Sj4686VeXyIiIlIddmy4x7KsT4BPTnrvwRP+\nvAEYYMdYTklwhxFkULsJERERqRY9tneOgoMMSWq0KiIiItWk8FUJ9aPDteFeREREqkXhqxKSo8I1\n8yUiIiLVovBVCfWjw7TnS0RERKpF4asS6keHk1NQSkl5hdOliIiISA2l8FUJxxqtHtTsl4iIiFSR\nwlclJB9vtKrwJSIiIlWj8FUJSZGa+RIREZHqUfiqhKQob/jS4doiIiJSVQpflRDvdgEKXyIiIlJ1\nCl+VEBocRFxEqJYdRUREpMoUviopKSpMM18iIiJSZQpflZQYGUZWfqnTZYiIiEgNpfBVSd7wpZkv\nERERqRqFr0pKjAwjS3u+REREpIoUviopKSqMgtIKCkvLnS5FREREaiCFr0pKjDzabiJP+75ERESk\n8hS+KinxaKPVg9r3JSIiIlWg8FVJx44Y0qZ7ERERqQqFr0pKVPgSERGRalD4qqSEo3u+1OVeRERE\nqkLhq5KOHTGkmS8RERGpCoWvKvD2+tLTjiIiIlJ5Cl9VoC73IiIiUlUKX1WQFBWmVhMiIiJSJQpf\nVaAjhkRERKSqFL6qIDHKRUFpBUWlFU6XIiIiIjWMwlcVqNeXiIiIVJXCVxUk6YghERERqSKFryo4\nfsSQ9n2JiIhIJSl8VUG829vlPqdAvb5ERESkchS+quBY+MpW+BIREZFKUviqgvDQYCLDQsjOV/gS\nERGRylH4qqJ4t4ucAu35EhERkcpR+KqieLdLy44iIiJSaQpfVZQY6dKyo4iIiFSawlcVeWe+tOwo\nIiIilaPwVUXx7jByCkqxLMvpUkRERKQGUfiqosRIF2UVFnkl5U6XIiIiIjWIwlcVHe/1pX1fIiIi\nUgkKX1WUcPSIIbWbEBERkcpQ+KqiBM18iYiISBUofFWRjhgSERGRqlD4qiIdri0iIiJVofBVRTrf\nUURERKpC4asa1GhVREREKkvhqxoSIl1adhQREZFKUfiqhgS3zncUERGRylH4qoYEd5iWHUVERKRS\nFL6qIf7osqPOdxQREZFzpfBVDQlu7/mOR4p1vqOIiIicG1vClzFmhDFmszFmqzFm4mmuucYYs8EY\ns94Y8287xnVaQqR6fYmIiEjlhFT3BsaYYOAF4CIgA1hmjJllWdaGE65pC9wPDLAsK9cYk1zdcQNB\nvPuH8x1bJrodrkZERERqAjtmvnoDWy3LSrcsqxR4G7j0pGtuA16wLCsXwLKsAzaM67hj5ztm6YlH\nEREROUd2hK/GwO4TXmccfe9E7YB2xphFxpilxpgRNozrOC07ioiISGVVe9mxEuO0BS4AmgDfGGO6\nWJZ16MSLjDETgAkAzZo181NpVafzHUVERKSy7Jj5ygSanvC6ydH3TpQBzLIsq8yyrO3A93jD2I9Y\nljXNsqxUy7JSk5KSbCjNt8JCvOc7ZuWr15eIiIicGzvC1zKgrTGmpTHGBYwDZp10zQd4Z70wxiTi\nXYZMt2Fsx+mIIRERkQDlqYDV78A3TztdyY9UO3xZllUO3AV8DmwE3rUsa70xZooxZszRyz4Hso0x\nG4C5wO8ty8qu7tiBIN6t8CUiIhJQLAu+/wJePh/+NwE2fwoVgdOT05Y9X5ZlfQJ8ctJ7D57wZwv4\n7dH/1SoJ7jAyDxU5XYaIiIgA7F4GcybBzkUQ1xKufAU6XQFBgdNX3l8b7muteHco6zIPO12GiIhI\n3XZwM3w1BTbNBncSXPw09LgZQlxOV/YTCl/VFOd2kVPoPd/RGON0OSIiInXL4UyY9zismgGhbhjy\nR+h7B4RFOl3ZaSl8VVN8hIvScg+FpRW4w/TtFBER8YvCHFj4N/huGlge6HM7DPoduBOdruyslBaq\n6cReXwpfIiIiPlZaCN+97A1exUcgZSwMeQDimjtd2TlTWqimY+Ert7CUpvERDlcjIiJSS1WUw6p/\nwbwnIG8vtB0OFz4IDTo7XVmlKXxVU5y63IuIiPiOZcHGj7yb6bO3QJNe3icYWwxwurIqU/iqpviI\nH2a+RERExEbbF8CcyZCZBontYewM6DAKavgDbgpf1fTDzFeZw5WIiIjUEnvXwFcPwdY5EN0YxjwP\nXa+F4NoRW2rHV+Gg6PAQgoMMuVp2FBERqZ7cHfD1o7D2XQiPhYumQO8JEFrP6cpspfBVTcYY4iK8\nvb5ERESkCvIPwjdPQdqrEBQCA++FAb+BerFOV+YTCl82iHeHkpOv8CUiIlIpJXmw+HlY8jyUFUGP\nG2HwRIhu6HRlPqXwZQPNfImIiFRCeQmkvead7SrMgo5jvG0jEts6XZlfKHzZIN7tYuuBfKfLEBER\nCWweD6z7D3z9CBzaCS0GwbCHoElPpyvzK4UvG8S5XWo1ISIicjqW5X1ycc5DsH8t1O8CN7wPrS+s\n8W0jqkLhywbxES5yC8vweCyCgureD5GIiMhpZaTBl5Ng50KIbQ5X/BM6XwlBQU5X5hiFLxvEuV1U\neCzyisuJiQh1uhwRERHnHfwevp7i7U7vToKRT0HP8RDicroyxyl82SDe7Q1cOYWlCl8iIlK3HdkD\n8x6HlTO8/bkueAD63QFhUU5XFjAUvmwQ7w4DvOc7tkx0O1yNiIiIA4pyYeGz8O1L4KmA3rfBoPsg\nMsnpygKOwpcNjp/vqC73IiJS15QVwbcvw8JnoPgIpFwDQx6AuBZOVxawFL5sEHfCsqOIiEidUFEO\nq/8Ncx+HvD3Q5iIYNgkadHG6soCn8GWDeLdmvkREpI6wLNg0G76aAlnfQ+NUuHI6tBjodGU1hsKX\nDeqFBhMWEqSZLxERqd12LIQ5kyFjGSS2g7H/gg6j62SvrupQ+LKBMYZ4t0szXyIiUjvtWwdfPQRb\nvoCoRnDJVOh2PQQrRlSFvms2iYtwkVNQ5nQZIiIi9sndAXMfgzXvQni09yigPr/0tpCQKlP4skm8\n20VOQYnTZYiIiFRfQZb30Otlr0BQMAy4BwbeC/XinK6sVlD4skmc20XmoSKnyxAREam6knxY8gIs\nngplhdD9Bhg8EWIaO11ZraLwZZP4iFBytOdLRERqovJSWP46fPMkFByEjpfA0AchqZ3TldVKCl82\niXO7OFxURnmFh5DguntYqIiI1CAeD6x7H+Y+4t3f1XwgjJsJTXs5XVmtpvBlk2O9vg4VlZEYGeZw\nNSIiImdgWbDtK2/biH1roX4XuP4/0GaY2kb4gcKXTeJOOGJI4UtERAJWxnKYMwl2LIDY5nDFdOh8\nFQRp1cZfFL5scmzmS/u+REQkIGVt8Xal3zgLIhJh5JPQ8+cQ4nK6sjpH4csmx48YUpd7EREJJEf2\nwvwnYMVb3v5cgydC/7sgLMrpyuoshS+b/DDzpUarIiISAIoOwaJnYelL4CmHXrfC+b+HyCSnK6vz\nFL5sEhsRCmjmS0REHFZWBN9NgwXPQPEh6HI1DPkjxLd0ujI5SuHLJmEhwUSGhWjPl4iIOKOiHFbP\nhHmPw5FM75OLF06ChilOVyYnUfiyUZw7VIdri4iIf1kWbPrYu5k+azM07gmXvwwtBzldmZyGwpeN\n4iNc5GjZUURE/GXHIm+vrozvIKENXPOWtzu9enUFNIUvG8W5XZr5EhER39u/HuY8BFs+h6iGcMlz\n0O0GCNZf6zWB/ivZKD7CxdYD+U6XISIitVXuTpj7GKx5B8KjYdhk6P1LcEU4XZlUgsKXjTTzJSIi\nPlGQDQuehmX/BAz0vxsG3gsR8U5XJlWg8GWjeLeLgtIKissqCA8NdrocERGp6UryYemLsGgqlBVA\nt+vhgvshprHTlUk1KHzZ6Nj5jocKy2gQo/AlIiJVVF4KK96A+U9CwQHoMBoufBCS2jtdmdhA4ctG\n8W5vo9WcglIaxIQ7XI2IiNQ4Hg+s/y98/QjkbofmA2DcDGja2+nKxEYKXzY6NvOlLvciIlIplgXb\nvva2jdi3BpI7wXXvQduL1DaiFlL4slFC5LHzHRW+RETkHGUu94au7d9ATDNvg9QuV0OQtq/UVgpf\nNtLMl4iInLOsrfD1w7DhA4hIgBFPQOovICTM6crExxS+bBRTLxRjNPMlIiJnkLcP5j0BK96EkHAY\n/H/Q7y5v3y6pExS+bBQSHERMPZ3vKCIip1B8GBY9B0teBE+Zd5Zr8B8gMtnpysTPFL5s5j3fsczp\nMkREJFCUFcOy6bDgr1CUC52vgqF/hPhWTlcmDlH4spm63IuICACeClg9E+Y+DkcyoPWFMGwSNOzq\ndGXiMIUvm8VFuNhzqMjpMkRExCmWBZs/ga+mwMFN0KgHXPYitBrsdGUSIBS+bBbvDmX9nsNOlyEi\nIk7YuQTmTILd30JCG7j6DTjvUvXqkh8JsuMmxpgRxpjNxpitxpiJZ7juSmOMZYxJtWPcQBTndpFd\nUIplWU6XIiIi/rJ/A/x7LLw2AnJ3wuhn4Y6l0OkyBS/5iWrPfBljgoEXgIuADGCZMWaWZVkbTrou\nCvg18G11xwxk8REuSss9FJZW4A7TxKKISK12aBfMfQxWvw1h0d7zF/v8ClwRTlcmAcyOdNAb2GpZ\nVjqAMeZt4FJgw0nXPQz8Bfi9DWMGrDj3D13uFb5ERGqpgmzv04vLpgMG+t8FA38LEfFOVyY1gB3p\noDGw+4TXGUCfEy8wxvQAmlqW9bEx5rThyxgzAZgA0KxZMxtK87/4E7rcN43Xv3xERGqV0gJvn67F\nU6E0H7pdBxfcDzFNnK5MahCfT80YY4KAZ4DxZ7vWsqxpwDSA1NTUGrlp6sSZLxERqSUqymDFGzD/\nScjfD+1HeZcYkzs4XZnUQHaEr0yg6Qmvmxx975gooDMwz3g3HTYAZhljxliWlWbD+AElXuFLRKT2\n8Hhgw//g60cgJx2a9Ydr3oJmfc7+uSKnYUf4Wga0Nca0xBu6xgHXHfugZVmHgcRjr40x84D7amPw\ngh/CV6663IuI1Gzb5sKcybB3FSSfB9e9C21/pqcXpdqqHb4syyo3xtwFfA4EA69alrXeGDMFSLMs\na1Z1x6hJosNDCA4y5BSUOF2KiIhUxZ6V3tCVPg9imsJlL0HKNRAU7HRlUkvYsufLsqxPgE9Oeu/B\n01x7gR1jBipjDHERLnIKNPMlIlKjZG+Drx+G9f+DevEw/HHodQuEhDldmdQy6oXgA/HuUJ3vKCJS\nU+Tth/l/8W6oD3bB+b+H/ndDeIzTlUktpfDlA/FuFzmFCl8iIgGt+DAsmgpLX4SKUug5Hs7/A0TV\nd7oyqeUUvnwg3u3i+/35TpchIiKnUlYMy/7pbZJalAOdroChf4KE1k5XJnWEwpcPxEW4tOwoIhJo\nPBWw5h3vcUCHd0OrITBsEjTq7nRlUscofPlAgttFbmEpHo9FUJAeSRYRcZRlwfefwVdT4MAGb9i6\n9HlodYHTlUkdpfDlA3FuFx4LDheVHe94LyIiDti1FL6cBLuXQnwruPp1OO8y9eoSRyl8+cDxLveF\npQpfIiJOOLDRO9O1+ROIrA+jnoEeN0FwqNOViSh8+cLxLvcFpZDkcDEiInXJod0w73FYPRNckTD0\nz9D3V+ByO12ZyHEKXz4QF+ENX9nadC8i4h+FOd6nF7+bDljQ9w4Y9DuIiHe6MpGfUPjygR/NfImI\niO+UFsDSf8Ci56A0H7peCxfcD7FNna5M5LQUvnzgxD1fIiLiAxVlsOJNb2f6/P3Q/mK48EFI7uh0\nZSJnpfDlA+GhwUS4gsnJV/gSEbGVZXnPXvz6EcjZBk37wjVvQrO+Tlcmcs4UvnwkLkJHDImI2Cp9\nHsyZDHtWQlJHuPZtaDdCbSOkxlH48pF4t7rci4jYYs8qb+hKnwsxTeGyf0DKWAgKdroykSpR+PIR\n7+HaZU6XISJSc2Vv8y4vrv8v1IuDnz0KvW6F0HCnKxOpFoUvH4l3u0jP0uHaIiKVlrcfvnkSlr8O\nwS4YdB8MuAfCY5yuTMQWCl8+4j1cWzNfIiLnrPgILJ4KS16A8hLoOR4G/wGiGjhdmYitFL58JCHS\nRX5JOSXlFYSFaF+CiMhplZcGKZXaAAAgAElEQVTAsldgwdNQmA2dLvd2pk9o7XRlIj6h8OUjx7rc\n5xaU0SBG4UtE5Cc8FbDmXZj7GBzeBa0ugGGToVF3Z+sS8TGFLx+Jd3sPb80pKKVBjDaHiogcZ1mw\n5QvvE4wHNkDDrjDmOWg91OnKRPxC4ctHjs98qdeXiMgPdn8HX06CXYshvhVc9SqcdzkEBTldmYjf\nKHz5SEKkDtcWETnuwCb4agps/hjcyTDqr9DjZggOdboyEb9T+PKRH/Z8KXyJSB12OAPmPg6r/w2h\nbhjyJ+j7KwiLdLoyEccofPlIbIQLY7x7vkRE6pzCHFj4DHw7DbCgz69g0O/AneB0ZSKOU/jykeAg\nQ2y9UO35EpG6pbQQvv0HLHwOSo5A12thyP0Q28zpykQChsKXD8W5XdrzJSJ1Q0U5rHwL5j0B+fu8\nB15f+CDU7+R0ZSIBR+HLh+IjdLi2iNRylgUbPoSvH4bsrdC0D1z9OjTv53RlIgFL4cuH4t0uduUU\nOl2GiIhvpM/39uraswKSOsC4mdB+JBjjdGUiAU3hy4fi3S5W7T7kdBkiIvbau9oburZ9DdFN4NIX\noes4CNJpHiLnQuHLh+LcLnILS7EsC6N/CYpITZeTDl8/Cuv+A/Xi4GePQK/bIFSneIhUhsKXDyW4\nXZRVWOSXlBMVrkaCIlJD5R+A+U/C8tcgKNTbMqL/PVAv1unKRGokhS8fOtZoNaegVOFLRGqe4iOw\n5HlY/DyUF0OPm2Dw/0F0Q6crE6nRFL58KN79Q/hqnuB2uBoRkXNUXgJpr8I3T0FhNpx3GQz9MyS2\ncboykVpB4cuH4tw6XFtEahBPBax9D+Y+Cod2QcvzYdhkaNzT6cpEahWFLx9KOBq+svMVvkQkgFkW\nbPkSvnoI9q+DBilww7PQeqjaRoj4gMKXD2nmS0QC3u5lMGcS7FwEcS3hyleg0xUQFOR0ZSK1lsKX\nD7ldwbiCg8gpKHO6FBGRHzu4Gb6aAptmgzsJLn4aetwMIS6nKxOp9RS+fMgYQ7xbRwyJSAA5nAnz\nHodVMyDUDUP+CH3vgLBIpysTqTMUvnxMh2uLSEAozIGFf4PvpoHlgT63e/t1uROdrkykzlH48rF4\nd6j2fImIc0oL4buXvcGr+AikjIUhD0Bcc6crE6mzFL58LN4dxvrMw06XISJ1TUU5rPoXzHsC8vZC\n2+Fw4YPQoLPTlYnUeQpfPhYfEaplRxHxH8uCjbPgq4chews06Q1XvQrN+ztdmYgcpfDlY3FuF4eL\nyiiv8BASrEe3RcSHti/wto3IXA6J7WHsDOgwSr26RAKMwpePHTti6FBRGYmRYQ5XIyK10t413gap\nW+dAdGMY8zx0vRaC9SteJBDp/5k+duL5jgpfImKrnO3eo4DWvgfhsXDRw9D7Ngit53RlInIGCl8+\nFh/xQ/gSEbFF/kHvoddpr0JQCAy8Fwb8BurFOl2ZiJwDhS8fO37EkMKXiFRXSR4sfh6WPA9lRdDj\nRhg8EaIbOl2ZiFSCwpePHTtcO0e9vkSkqspLIO0172xXYRZ0HONtG5HY1unKRKQKFL58LPbosmN2\nvsKXiFSSxwPr/gNfPwKHdkKLQTDsIWjS0+nKRKQaFL58zBUSRHR4iPZ8ici5syzvk4tzHoL9a6FB\nF7jhfWh9odpGiNQCCl9+kBgZRlZ+idNliEhNkJEGX06CnQshrgVc+Qp0ugKC1CdQpLawJXwZY0YA\nzwHBwD8ty3ripI//FrgVKAcOAr+wLGunHWPXBAmRLi07isiZHfwevp4CGz8CdxKMfAp6jocQl9OV\niYjNqh2+jDHBwAvARUAGsMwYM8uyrA0nXLYSSLUsq9AY8yvgSWBsdceuKRLcYaRn5TtdhogEoiN7\nYN7jsHKGtz/XBQ9AvzsgLMrpykTER+yY+eoNbLUsKx3AGPM2cClwPHxZljX3hOuXAjfYMG6NkRDp\nYtkOzXyJyAmKcmHhs/DtS+Cp8DZHHXQfRCY5XZmI+Jgd4asxsPuE1xlAnzNcfwvw6ak+YIyZAEwA\naNasmQ2lBYYEt4vcwlIqPBbBQdosK1KnlRXBty/Dwmeg+AikXANDHvDu7xKROsGvG+6NMTcAqcDg\nU33csqxpwDSA1NRUy4+l+VRCZBgeCw4VlpKgI4ZE6qaKclg1A+Y9AXl7oM1FMGyS90lGEalT7Ahf\nmUDTE143OfrejxhjhgF/BAZbllWnHv1LiDza66tA4UukzrEs2DQbvpoCWd9D41S4cjq0GOh0ZSLi\nEDvC1zKgrTGmJd7QNQ647sQLjDHdgZeBEZZlHbBhzBolwe0NXFn5JbSrr020InXGjoUwZzJkLIPE\ndjD2X9BhtHp1idRx1Q5flmWVG2PuAj7H22riVcuy1htjpgBplmXNAp4CIoH3jPeXzi7LssZUd+ya\nIjFSXe5F6pR9a70NUrd+CVGN4JKp0O16CFZrRRGxac+XZVmfAJ+c9N6DJ/x5mB3j1FTHlhqz1WhV\npHbL3QFzH4M170J4tPcooD6/9LaQEBE5Sv8M84PYeqEEGXTEkEhtVZDlPfR62SsQFAwDfg0DfwP1\n4pyuTEQCkMKXHwQFGeLdLrIUvkRql5J8WPICLJ4KZYXQ/QYYPBFiGjtdmYgEMIUvP0lwh2nZUaS2\nKC+F5a/DN09CwUHoeAkMfRCS2jldmYjUAApffqLzHUVqAY8H1r0Pcx/x7u9qMQiufRuapDpdmYjU\nIApffpIQGca6zMNOlyEiVWFZsO0rb9uIfWuhfhe4/n1oc6HaRohIpSl8+UmC20WWlh1Fap6M5TBn\nEuxYALHN4Yrp0PkqCApyujIRqaEUvvwkMdJFXnE5JeUVhIUEO12OiJxN1hZvV/qNsyAiEUY+CT1/\nDiEupysTkRpO4ctP4o92uc8tKKNBjMKXSMA6ssd7/uLKf3n7cw2eCP3vgjCdTiEi9lD48pNj5ztm\n5ZfQICbc4WpE5CeKDsGiZ2HpS+Aph163wvm/h8gkpysTkVpG4ctPEk84XFtEAkhZEXw3DRY8A8WH\nocvVMOQBiG/pdGUiUkspfPnJscO11etLJEBUlMPqmTDvcTiSCW2GwYWToGGK05WJSC2n8OUnCTpc\nWyQwWBZs+ti7mT5rMzTuCZe/DC0HOV2ZiNQRCl9+EhkWgiskiKwCzXyJOGbHIm+vrozvIKEtXPOW\ntzu9enWJiB8pfPmJMYZEt7rcizhi/3qY8xBs+RyiGsIlz0G3GyBYvwJFxP/0m8eP4iNd5NTwDffF\nZRUsTc8mt7CULo1jaZMc6XRJIqeXuxPmPgZr3oHwaBg2GXr/ElwRTlcmInWYwpcf1fTDtb9Nz+a3\n764m81DR8fcu7daIxy7vgjtMP0oSQAqyYcHTsOyfYIKg/90w8F6IiHe6MhERhS9/Soh0sfVAvtNl\nVMm36dnc9Op3NIwJ5583pdIi0c0HKzN5cd5WdmQXMvO2PkS49OMkDivJh6UvwqKpUFYA3a6HC+6H\nmMZOVyYicpz+tvSjxMgwsvJLsCwLU4M2+Gbll3DHjBU0iavHe7f3J97tfXLzvuHt6dw4hl/NWM59\n763mhet61KivS2qR8lJY8QbMfxIKDkCH0XDhg5DU3unKRER+QifD+lFipIuScg8FpRVOl1IpD364\njryScl66oefx4HXMiM4N+MPwDnyydh8frMp0qEKpszweWPsfeKE3fHIfJLaFW76EcTMUvEQkYCl8\n+VFSlLfR6sG8mrPvK21HDp+s3cfdQ9rQtv6pz7abcH4rejaPY8pHGzhSXObnCqVOsizY+hVMGwzv\n3wIuN1z3Hoz/GJr2dro6EZEzUvjyo8TImhe+nvp8M8lRYdwy6PRHrQQHGSZf0oncwjKmf5Pux+qk\nTspcDm+OgX9d4T2P8fJp8MsF0O5n6tclIjWCwpcf1bSZr3WZh/l2ew4Tzm911s30XZrEcEnXRvxz\nwXYOHCn2U4VSp2RthXdvhulDvX27RjwBd6dB17EQpF9lIlJz6DeWHyUdnfnKqiHtJl5fvIMIVzBX\npzY9p+vv+1k7Sis8vLJwu48rkzolbx989Bvvvq4tX8Lg/4N7VkHfX0FImNPViYhUmp529KO4CBfB\nQaZGzHxl5Zcwa9UexvZqSky90HP6nOYJbkZ2bsC/v9vF3Re2JVK9v6Q6ig7Boudg6T/AUwa9boHz\nfw+RyU5XJiJSLZr58qOgIENipKtGhK8PVmZSWuHhpn7NK/V5tw5qRV5xOe+l7fZRZVLrlRXD4r/D\n1G6w8BnoMAruWgYXP6XgJSK1gqYm/CwpKoyDNWDZcdbqPXRpHHPaJxxPp1vTWFKbx/Haoh3c3K8F\nQUHaAC3nyFMBq2fC3MfhSAa0vhCGTYKGXZ2uTETEVpr58rPEyLCAn/nanlXAmozDjOnaqEqff2O/\n5uzKKWRperbNlUmtZFmw6WP4R3/48E7v7NbNH8GN/1XwEpFaSTNffpYUGcamvXlOl3FGH63eA8Co\nlIZV+vzhnRoQHR7CO2m76d8m0c7SpLbZuQTmTILd30JCG7j6DTjvUrWMEJFaTeHLz5KivEcMeTxW\nwC7JzV6zh94t4mkUW69Knx8eGszl3Rszc9luphSWERNxbhv2pQ7ZvwG+egi+/wwiG8DoZ6H7DRCs\nnxURqf207OhnSVFhlHssDhcFZif4XdmFfL8/n+GdG1TrPtf0akppuYcPV+vIITnBoV3wv9u9S4w7\nl8CFk+CelZD6cwUvEakzNPPlZ8cbreaXEHfSOYmB4OtN+wG4sEP1nirr1CiG9vWjmLVqDzf1a2FD\nZVKjFWTDgr/CsumAgf53wcDfQkS805WJiPidwpefJZ1wxFC7Sj5J6A9fbTpAqyQ3LRLd1b7XmG6N\neOrzzWQeKqJxFZcwA1FxWQUHjpQQFR4SkAE6oJQWwJIXYfFUKM2HbtfBBfdDTBOnKxMRcYzCl58F\n8hFDBSXlfJuew839K9fb63QuSfGGr49W7+H2wa1tuaeTlu/M4e9fb2XBliwqPBYAbZIjubFvc67t\n3QxXiFbxj6sogxVvwPwnIX8/tB8FFz4IyR2crkxExHEKX36WGMDha9HWLEorPAyp5pLjMc0SIujW\nNJZZq2p2+Cqv8PCXzzYxfcF2kqLCuGVgS9okR5JbUMpn6/cxadZ63lm2mxev72HLjGGN5vHAhv/B\n149ATjo06w/XvAXN+jhdmYhIwFD48rOosBDCQoICstHqoq1ZRLiCSW1u3z6cMV0bMWX2BrYeyKdN\ncqRt9/WXCo/Fve+u5qPVe7ipX3Mmjuzwo0PGJ5zfii827Gfi+2u46qXFvDa+N12axDhYsYO2zfW2\njdi7GpLPg+vehbY/U9sIEZGTaJ3Ez4wx3i73ATjztXhbNqkt4m1dPhud0hBjvO0rahrLsrj/v2v4\naPUeJo7swJRLO/8oeIH3v+fwTg34z6/6Ex4azPX/XMrWA4Hdx812e1bCm5fCW5dBYS5c9hLcvhDa\nDVfwEhE5BYUvBxzr9RVIDuaVsOVAPv1bJ9h63+TocFKbx/H5+v223tcfZny7i3fTMrh7aJuzLpu2\nTopk5m19cYUEM/61ZQEZrm2XvQ3eGw/TLoC9a2D443B3GnS7FoKCna5ORCRgKXw5ICkAjxhacvQo\noH6t7A1f4O14v3HvEXZmF9h+b1/ZsOcIUz7awOB2Sdw7rN05fU7T+AheHZ9KVn4Jv313FZ6jm/Jr\nnbx9MPteeKE3fP85nP8H+PVq6HcHhIQ5XZ2ISMBT+HJAIC47LtmWTVRYCJ0aRdt+7+GdvA1bP1+/\nz/Z7+0J5hYf/e38N0fVC+dvYbpU6iSClSSwPju7Egi1ZTFuQ7sMq/c9TeIiM9x+g7G9dqUh7g1XJ\nl/HZhZ+S2+f3EG7/z42ISG2lDfcOSIwMI6ewlLIKD6HBgZF/l2zLok+reEJ8UE/T+Ag6NYrms3X7\nmHB+4D/1+PriHazNPMzz13Unvgp9vK7t3ZSFWw/y1y82M6xjMm2SA6+fW2WUlxSy+r9/pfXml2lC\nHrMq+vGiGcfmnUlYOzIJnrWHS7s14p6hbfW0p4jIOVD4ckD96HAsC7LyS2gY43zz0T2HitiRXcgN\nfe3p73UqIzo14K9ffs+BI8UkR4f7bJzqysov4W9ffs+Q9kmM6lK1g8WNMUy5tDOLtmZz/3/X8s6E\nfgF7jucZeSrIXvwWFV8/Qk/PQVa5enB4wAOc32swYyJclFd4WJt5mFmr9zDzu118vGYvfxjRgZ/3\nb1Ezv14RET8JjGmXOqZ+tHdfzL7DxQ5X4rX02H4vmzfbn2jE0bMiP98Q2Bvvn/96K8XlHv40+jxM\nNZ7US4wM44+jOrJsRy5vL9ttY4V+YFmw+VMKp/YlYc6vOeCJZsnAV+l6/9cMHnwRsRHe2cCQ4CC6\nN4tj0iWd+Ob3QxjUNpGHZ2/gzn+voKi0wuEvwneKyyrIyi8hv6Tc6VJEpIbSzJcD6h+d+dl/JDD2\nfS3bkUtUeAgdG/hu306b5EhaJbn5fN0+bvThDFt17MwuYMa3O7kmtSmtk6rfk+zqnk3474oMnvh0\nIyM6N6jSEqbf7VoKX06C3UvZbzXgXxF/YPytv6FzwpmXE5Ojw5l+UyqvLNzOo59sZP8/l/LGL3oT\nFV7zD8u2LIu0nbm8vzyDpenZ7MguPP6xeLeLPi3jGdG5ASM7N9QpByJyThS+HPBD+AqMma8VO3Pp\n0SzOp0tFx/phTfsmnUOFpcdnTwLJX7/4npCgIO4d1taW+xljeGhMZ0Y+9w1Tv9rC5DGdbLmvTxzY\nCF9Ngc2fUFYviYcrbmVlwmjeuG3AOYdGYwy3DmpFo9h63DNzJb94fRmv/7w37rCa+2tm+c5cHv9k\nI2k7c4lwBTOwTSJX9GhCbEQohaUVbDuQz8KtWXy6bh+PRm3k18PaMq5XM4K17CoiZ1BzfyvWYAlu\nFyFBJiDC1+GiMr4/kMeolKrtb6qMEZ0a8I9525iz8QBX9Qysg5W3HsjnozXeY5Ds3JPWvkEU1/Zu\nxltLd3JD3+aB1+X/0G6Y9zisngmuSA71ncio7zoTHBXJ/27tX6XZuou7NMSy4O6ZK5jwVhqvje9d\n42aESsoreObL75n2TTpJkWE8fFlnruje+JRB0uOx+GbLQV6Yu5U//m8d7yzbzbNju9HKhtnTQFBW\n4WHh1izmbjrA2szD7M4ppKi0gnquEBrHhtO1aSyD2iYxuF1SjfvvLOIUhS8HBAUZkqPC2BcA4WvV\n7kNYFvRsHufzsVKaxNAwJpzP1+8LuPD10vxthIUEcevAlrbf+96L2jFr1R4e/2Qjr4zvZfv9q6Qw\nBxb8Fb6b7n3d9w4K+/yaq17dSAEl/PfnvUiIrHrPrlEpDSku68rv3lvNpFnreOzyLtXaQ+dP+48U\nc+sbaazNPMy1vZvxp1Edzzh7FxRkuKB9MoPbJTF7zV7+/OE6Rv99IQ9f2pkrA+znvDIKS8t5bdEO\n3li8gwN5JdQLDaZr0xguOq8BEa5gCkrK2ZVTyH+WZ/Dmkp3ERYRyc/8W/HxAS2Lq1fzl5hOVVXjY\nuPcI27MKyDxURFm5RYQrmIax4aQ0jqVpfL0a8/MtgUHhyyH1Y8I5EAB7vpbvzCXIQLemsT4f69jS\n48zvdlFYWv6To3qcknmoiA9WZnJD3+bVChynkxgZxp1D2/DEp5tYuCWLgW0TbR/jnJUWwNIXYdFU\nKM2HrtfCBfdDbFMmvbeabQfzmXFLH1tmba7s2YT0rHxemLuNNslR3OKDYGu3TfuO8PPXlnGkqIxp\nN/bkZ0d71J0LYwyXdG1Eaos47n1nFb87+v2872fta9TTn5Zl8b+Vmfzls03sP1LC+e2SePTy5gxq\nm0h46E9PLiir8LBwSxYzvt3Fs3O28MqC7dx7UTtu7t+iRi+/ejwW874/wPvLM/lmy0Hyik//gEVS\nVBgXd27AZd0b061pbK0LYsVlFWzZn096Vj55xeWUlHuIqRdKYqSL8xpGB/QT7IEqMP72q4PqR4Wz\n9WC+02WwfGcOHRtG+21fzvBODXh98Q7mbz7IyCq2crDb9G+8zVAnnN/KZ2OM79+Ct5bs5PFPN/JR\n64H+/8u4ogxWvAnz/wL5+6H9xXDhg5DcEYAPVmby3vIM7hnahv5t7AuHv7uoPVsP5PPoxxtoneTm\ngvbJtt3bbusyD3Pd9KWEhwbz7u396NSoagekN4ypx1u39OHBD9fz4rxt7M4t4plrugZMT78zycov\nYeL7a5mzcT9dm8bywnU9SG0Rf8bPCQ0OYkiHZIZ0SGbDniP85bNNTJm9gf+uzOCZa7rRrn7N6nNX\n4bF4f0UGL8zdys7sQhIjw7i4c0MGtUukff0omsRF4AoJIr+knN05hazafYgl27J5e9lu3liykx7N\nYrlzSBuGdkiu0SFs7+EiPlq9hzkbDrB8Vy4VZzixo0F0OEM6JDOicwMGtkms0aHbX4xlBeYRKKmp\nqVZaWprTZfjM5FnreX9FBmsnD3eshvIKD10f+oIrezZhyqWd/TZmr0fncH67JJ4b190vY57JocJS\n+j7+FZekNOKpq7v6dKwPVmbym3dW8ezYblzWvbFPxzrOsmD9/+DrRyBnGzTrB8MmQ7O+xy/ZkVXA\nqKkLOK9RNDNv62t7o93C0nKueHEx+48U8/E9g2gU63xvu5Nt2HOE6/65FLcrhLcn9KVpfES172lZ\nFi/NT+cvn23iovPq8/x13QkLCdwzL9dlHubnry/jcFEZfxjenl8MaFmlfyRYlsXHa/cyedZ68kvK\nmXxJJ8b2alojgsjS9Gwmz1rPpn15pDSJ4bZBrRjRucE5BecjxWV8uDKTl+ank3moiL6t4pk8phMd\nfPgUuS+s3JXLy/PT+XLjfio8Fh0bRjOkfRJdGsfQJjmSmHqhuEKCOFJUzt7DRazfc4S0nTnM23yQ\nwtIKGsfW48Z+zbm+T7Na8bRzZRhjlluWlXou12rmyyHJ0WHkFZc7uvy2eX8eBaUVftnvdUxIcBAX\nnVefT9fuo7Tc4/gG3XeW7aa4zMMtg3y/JDamayOmL0jnqc83M7JLA9//RZw+D+ZMhj0rIakjXPs2\ntBsBJ/wl6PFY/P4/qwkOMjw3rrtPTjiIcIXwwvU9GPP3hdw9cyVvT+gbULNAW/bnccMr31IvNJiZ\nt9kTvMC7DPmrC1rjDgvmwQ/XM+HN5bx8Y89TLt05bcGWg9z+1nJi6oUy664B1QoMxhhGpzSid8t4\n7n1nFRP/u5Yl6dn85cqUgPzaAfJLyvnLp5t4a+lOmsbX44XrenBxlwaVCozR4aHc2K8F43o34+1l\nu/nrF5sZNXUhN/drwe+Ht6eeKzC/9mM27j3CX7/YzJyNB4iLCOXWQS25rnczmp+mzUxshItmCRH0\naZXALwa2pLisgq82HuCtpTt44tNNvDR/GxPOb8X4/i0CZotJIAmc34B1TIOja+RONlpdsTMXgB7N\n/Be+wLv0mFdSzuJtWX4d92TlFR7eXLKTvq3i/fKv06Agw/0jO5J5qIi3luz03UB7VsGbl8Gbl0JB\nFlz2D/jVImg/8kfBC2DGtztZtiOXP48+z6czUq2TInnsii4s35nL059v9tk4lXXgSDHjX1tGcJDh\n37f1pVmCPcHrRDf1a8ETV3Thmy0Hue3NNIrLAqsB7QcrM/n5a8toGh/Bf++oXvA6UXJUOG/+og+/\nu6gds1bv4ZqXlwTEE94nW7X7ECOe/YZ/fbuTWwa25IvfDGZUSsMqz9SFBgdxY9/mzP3dBYzr1ZRX\nF23n4qkLWH70922gKSqt4LFPNjJq6gK+257D74e3Z+H/DeX+kR1PG7xOJTw0mFEpDXl7Qj9m3z2Q\nHs3iePKzzVzw1Dw+XJVJoK6yOcWW8GWMGWGM2WyM2WqMmXiKj4cZY945+vFvjTEt7Bi3JguERqur\ndh8mMdJFkzj/LgMNaJOI2xXM5+ud7XY/Z+MBMg8VMb5/C7+NObBtIoPaJvL83K0cLiqz9+bZ2+C9\nn8O0wbB3FQx/DO5Kg27XQdBP/9WdeaiIJz7dxKC2iX55+vTSbo25oW8zXv4mnTkBcNJBQUk5v3hj\nGbmFpbw2vhctfXgu5bjezXjyyhQWbs0KqAA2/Zt0fvPOKlJbxPHOL/vRIMbejdPBQYa7L2zLtBtT\n2XognzHPL2RNxiFbx6gqy7J4fdF2rn5pMZYF7/2yH38efZ5tM1RxbhePXt6Ff9/Wh9JyD1e/tJi/\nfLaJ0nKPLfe3w+JtWYx47humfZPO2F7NWPCHodw5pE219wB3bhzDq+N78f6vvD9Tv357FWOnLWXT\nviM2VV7zVTt8GWOCgReAkcB5wLXGmPNOuuwWINeyrDbA34C/VHfcmi4QGq2uzTxEShP/P5kTHhrM\nBR2S+XLD/jNu4vS1NxbvoHFsPYZ1rO/XcSeO7MDhojL+MW+bPTfM2w8f/w5e6A3ffwaD7oNfr4Z+\nd0Loqf8ytSyLP/5vLRb4tQ3En0adR+fG0fz23VXszik8+yf4SHmFh7tnrmTDniO8cF0POjeu2ub6\nyrg6tSn/3959B0ZVpX0c/5703gghJKRQQu8EpFcLIoKgoKIIuxYUELvL6uuKWLDsuooIKisoIjYQ\nBAsqVXonQEKHBNIDIb0n5/1jgqICJmRm7g15Pv8wCTP3PnAh85tzz3nOa5UB7IFPdhkawCoqNC9+\nG8fL3x/kpvYN+fjv3WzaHuK61g1Y8lBPnBwcGPXeFlbEJNvsXFWRU1TKpEW7mbYijn7N6/PdlN5/\nubDgSvVsGsjKR/swqksYc9Yd55Z3N3E0Ldcm56qq7MJSpi7Zx5i521DAZ/d3Z8bIdvh6WPffQJeI\nAJZO7MWMke04mpbLTTM38sKKWHKLrPzBsxayxshXN+CY1vqE1roE+BwY/ofnDAc+rny8GBikasPs\nSxs6v7+jUeErv7iMY+l5tLPDm87FDG4TzJm8YnafMmYo/lBqDltOnGVsjwibzHO6nDYhvozoGMq8\nTSdJziq88gMV5Vgm0r97I9AAACAASURBVM/sCLs+gs7jYMoeGPQcuF3+un6zN5l1hzN46oYWVpvj\nVBVuzo7MHtMFDUxetJviMvsHEK0101bEsuZQOtOHt2VAS/utwBwdHcZrI9uz4WiGYQGsuKycR77Y\ny4cbT/K3XpG8c4d9FgK0aujD8sm9aN/Il4c/28ObPx+hwoAPX3HJOQx7ZyM/xqbxzJCWzL0n2uY7\nbni7OfPabe2Ze080qTlFDH1nIx9tOmnIrbgfY1O57s31fLnzNBP6NWHlo31tuq+vo4Pizm7hrKm8\nDfvR5ngG/md9nb8VaY13nVDgwp2DEyu/d9HnaK3LgGzgT1dbKfWAUmqnUmpnRkaGFUozL283Zzxd\nHA1rtBqbnEOFhg5hxoSv/i3q4+LowI8HUg05/8ebE3B1cuD26DBDzv/49c1Bw5s/H6n+i8uKYcts\nS+j65Q1ofgNM2g5D3wTvv+5LdSavmBdWxNI53I97ekRW//w1FF7Pgzdu60BMYjavfHfQ7uefu+EE\nC7eeYkK/JtxtwD6jo7v+FsAm2DmA5RSVMn7eDlbEJPPPG1vyr6Gt7dr2pJ6XKwvvu4ZRXRoxc/VR\nJn9m303Yv9xxmhGzN1FYWs7nD3Tngb5N7Tryf13rBqx8tA89m9Zj2oo4xs3fYbcP4Bm5xUz6dDcT\nPtlFgKcLyyb14p83trLbIojzt2GXTexFsI/lVuSYuds4lm77UUCtNWfzjO+reSFTTbjXWn+gtY7W\nWkfXr1/f6HJsroGPcY1Wz8+7aBdq++aqF+Pt5kzvqEBWxqba/dNPVkEJS/ckMqJTKP4GbXbdyN+D\ncT0jWLI7kYMpVZwHUVEOez+Dd6Lhx39CcDt4YB2M+gjqNa3yuV9YEUd+cTmv3dresH48g9sGc1/v\nxny8JcGut6C+35/CK98f4qZ2DfnHDS3tdt4/Oh/A1h/J4MGF9glgaTlFjH5vCzviM3lzdAcm9LNv\n8DjP1cmR129rz7NDWvHDgVRGvb+ZlOwajABXQWFJOU99FcPTS/bRNTKA76b0oauNbjP+lSBvN+aN\n78pLt7Rl+8mz3PDWL6w8kGKz81VUaD7ffopr31zPz3FpPHl9c1Y83Jv2jYz52d8hzI9lk3rx4i1t\niU3OZvBbG5jxw0Hyiy/dxLYmdsRnMnLOZsbP32HISOulWCN8JQEXDh80qvzeRZ+jlHICfIGzVjh3\nrdbAx82wka+YxGxCfN2o7239ju5VdUObBiSeKySuquHDSs63lxhnx4n2FzNpQDO8XZ14beWhyz9R\nazjyI7zXG5Y9CB4BMHYZ3PMNhFSvV9rPcWmsiEnm4YHNiDK4+eU/bmxJ53A/pi7Zxwk7NBzelXCO\nx77YS5cIf/4zuoPhXedHdw3jtVvbse6w7QPYsfQ8Rs7ezKnMAuaN78rIzsZue6SU4v6+TfhwXDTx\nZwoYPmsTuxIybXKuExl5jJi9icW7E5kyKIqP/96NQBvsZFEdSinu7h7Bd1P6EB7gwYMLd/PElzFW\nH505kpbL7R9sYerX+2nRwJvvH+nN5IFRhrd6cXRQjO0ewZon+3NLp1DeX3+Cgf9Zx4It8VabirAr\n4RxjP9zGqPe2kHSukLEGjHJfjjWuwA4gSinVWCnlAtwBLP/Dc5YD4yof3was0XX5Zm+lBj6uhs35\n2p+YZdgnn/OubdUAB4Vdbz2WV2gWbEngmsYBtGpobPNDPw8XJg1oxrrDGWw+dom2G6e3w/whsGg0\nlBXBbfPg/rXQdEC1z5ddWMqzS/fTMtibCf2qPlJmK86ODswa0xkXJwcmfrrbZp98ARLO5nP/gp0E\n+7ox955o0/Sbur1rOK+OtASwhxbusskcuM3HzzBy9iaKy8r54oEe9G1unrsKA1s24OuJPXFzdmT0\n+1uZufqo1RbhVFRYVjMOmbmB1Jwi5o/vyuPXNTdV9/Wm9b1Y8lBPHh7YjG/2JtH/3+uYt/EkpeU1\nWxGZllPEs0v3M+TtDRxLz+P129rzxYTuNAsy124DgV6u/HtUBxY/2IMwfw/+9U0s/d9Yx/vrj3Pm\nCoJoaXkFP+xPYczcrdw6ZzNxyTk8O6QV658awOiuYYZ/4LqQVTrcK6WGAG8BjsA8rfXLSqnpwE6t\n9XKllBvwCdAJyATu0FqfuNwxr/YO9wAzfjjI/I3xHH5psF2H/7MLSukw/SeeuqEFkwY0s9t5L+b2\n97eQVVDKj4/1tcv5foxNZcInu5hzV2dTbG9UVFrOoP+sJ8DThW8m9frth0P6IVg9HQ5/B55B0P8f\nlgn1jle+GunpxTEs2Z3Esom9aNfImLl+F/PLkQzGz9/OgBZBfHBPtNXfHM/mFTPqvS1kFpTw9UM9\nrbJvpbV9vv0UU7/eT/8W9Xl3TGerbff15c7TPPP1fiIDPZk/vqtdF1dUR05RKc8tO8A3e5PpGunP\nyyPa1WhbomPpeTy37ABbTpylf4v6vDqyvdXbaFjbsfQ8pn8bxy9HMmjk786Evk24rUtYtVpfxJ/J\nZ8GWBBZtT6CsXHNnt3Aeu645AQZNr6gOrTWbjp1l5pqjbD+ZibOjYmDLIAa1bECvqEBCfN0u+j6Z\nVVDCnlNZrD6Uxo+xaWTkFhPi68Y9PSMZ2z3CblvnQfU63Mv2Qgaav+kkL6yIY+f/XWvXYfANRzMY\n++F2Ft57jbGbPPPb38Gqx/va5VPZmLlbiT+Tzy9PD7D7KsdLWbIrkSe+imHmnZ0YFlkBa2dAzCJw\n8YJeU6D7RHCpWQ+q89d8Yv+mPD3YuLlOl/LJ1gSeW3aA8T0jmTasjdWOm11Qyh1zt3IiI4+F911j\n2Dyfqvh8+ymeWbqf5g28+WBsdI0avhaVljP92zgWbTtF72aBvHtXZ5u2krCWpXsSmbY8jvziMsb3\njGRCv6bVmhqRnlPEnPXH+WRLAu7Ojjx7U6tas7URWALIuiMZvLP6KLtPZeHl6sTgtsFc26oB0ZH+\nf3qfKCuv4OSZfDYcPcNPcalsPZGJk4NiWIcQHr22uU2aBtvD0bRcPt9xmhUxyaTnWkbAvN2caBzo\nibebE86ODmQVlJKeU0RyZaNyDxdH+kQFMjo6jP4tggwZ4ZTthWqJ8x3FU7KK7Bq+9iVmAxjWZuJC\nN7VryIvfxrF8bzKPX9/Cpuc6lJrD5uNneXpwC9MEL4BbOoXy2foY8ldMReuVKLQlcPV+HDxrvgQ8\nr7iMqUv207S+J1MGRVmhYusb2z2C+DP5fLjxJKF+7txvhU3O84rLGDd/O8fT85g7LtrUwQssjVhD\n/d2ZvGgPN8/ayLRhrbmlY2i1g8P+xGyeWhzDodRcJvRtwpM3tDB8jk9VjejUiH7Ng3h95SE+3HSS\nT7YmcGuXRgzvEEJ0ZMBF31BLyirYdvIsy/YkszwmifIKzR3dwnniuubUM3huV3UppRjQIoj+zeuz\nI/4ci3ed5vv9qSzelQhAgKcL9b1ccXRQFJaWk5RV+GvT1qggLx69Noox3cIJ8jH3KN9fiWrgzXND\nW/N/N7XiUGouO+IzOZqWR0JmAQXFZeQUluHn4UzjwHq0CPamXagvXSL8TTOdoCokfBkotDJ8JWUV\n2PU20L7ELCLreVi9od6VCPJxo2fTQL6JSeax65rb9BPqR5vicXN24M6u4TY7R7WVFOC4bQ6fF/4X\nh9I8YoOG0PauV8HPejW+8v1BkrMLWfxgD1P/cHpmSCtSsgt5+fuDKAX39bnyAJaZX8K9H+9gf1I2\nc+7qTD8TzXO6nD5R9Vk+uRePfxnDY1/EsCImhWeGtKzSqPCZvGJmrTnGgi3xBHi6Mm98NANb2reB\nsDUEeLrw6q3teaBvE+ZuOMGSXYks2nYKfw9n2ob60sjfHTdnRwqKy4k/m09ccg65xWV4uDgypls4\nf+/duFrb4piRUopujQPo1jiAF29py4GkHHYlZHIqs4D0nGIqtMbdxYnrWjegZbA3ncP9ibThDg1G\nUUrRqqGP4fNzbUHCl4F+C1/2nXS/LzHbVKMAwzqG8PTifcQkZtMxzDaLADLzS1i6J4mRnRsZ1l7i\nd8pLYc8nsO41yEvFqfmNvFF2O+8fcmV5gR+trfTXsPJAKou2nWJC3yZ0iTDPNb+Y85t7a72Hl747\nSHFZBRP7V78dwunMAsbN307iuULeHdOZ69v8de8zM4mo58mXE3owf9NJ3vz5CNf/9xeubx3MrV0a\n0b1JAN5uv31oKiotZ/epc6yISWbpniSKyyq4+5oInryhRa24zXg5Tep7MWNke/7vptasPZzOL0cy\niEvJ4VBqLoUl5Xi6OtLQ151hHUPo3yKIPlGBpv5wcaVcnRzpEuFPlwj77sErbEvCl4H8PJxxd3Yk\n6Zxte9xcKD23iJTsItqbaML14LbB/N+yA3yzN8lm4euz7acoLqvg770ibXL8KtMa4r6BNS/C2WMQ\ndo2lT1dED+4vKOGLN9fzjyX7+HpizxrfKkrOKuQfS/bRvpEvT9j4lq61ODs6MPPOTjzxZQxv/HiY\nY+l5vHRL2ypPml0Vl8aTi2OoqNB8avI5Xpfj6KC4r08TRnQKZe6Gk3y18zQrY1NxUNDQ1x0fd2cK\nSspIOldIWYXGw8WRYR1CmNCvKU1NuKCgJjxdnRjaPoSh7UOMLkUIq5HwZSClFKH+7jXbYqaa9lfO\n9zK6zcSFfNycGdgiiBUxKTw7pJXV52OVllewYEs8faICje1tdWI9rJoGybuhfku44zNocSNUjuz4\nebgwfXhbJn66mzd/PsI/ajAxvqi0nEmLdlNaXsHbd3TCxal2zPkBSwB7+46ONAvy4r+rjrAjPpNp\nN7dhUKugS46CJWUV8toPh1gek0ybEB9mjels042y7aWelytTb2zJE9c3Z/vJTHbEW2495RSWVoaS\nhnRo5EefqPpW2xBaCGF7Er4MFuLnTrKNuztfKCYxGwcFbUPNdQ/9lk6hrIxNZf2RDAZZeaPr7/en\nkJZTzIyR7ax63CpLibGEruNrwKcRDJ8NHe4Ahz+/WQ5p15A7u1k24O3epN4VzVXSWvPM1/vZcyqL\nOXfVzhCilGLKoCh6Nq3H04v3cd+CnbQJ8WFYhxCiI/2p7+VGfkkZcck5vy4xd3RQTBnYjIkDml11\nt5+cHR3o1SyQXs2MXZ0shLAOCV8GC/VzJzYp227n25+YRVSQNx4u5rr0g1oFEejlymfbT1s9fM3f\nFE/jQE/6N7ffBsoAZJ6ANS/DgcXg7g/XvwRd7wfny69E+tfQNuw5lcXkRbtZ8lDPavc7mrXmGF/v\nSeLx65qbopdZTURHBvDjY31ZujuJj7fEM+OHP+8G4O/hzN97RTKuZySN/Gvn0nohRN1irnfgOijU\nz42z+SUUlZbb/NO61pp9idkMaGnnEFIFzo4OjIpuxPvrj5OaXWS1hojbTpxl7+kspg9vY7/uxnnp\nsP512DUfHJyhzxPQcwq4V+1Wr7uLIx+O78qIdzcxft52Pn+gR5X79by3/jj/+fkIIzqF8vBAYxvo\nWouzowOju4YxumsYyVmFHE7LJSO3GG9XJ8LredAq2MdUnauFEOKv1J6JIFepkF9XPNr+1mNydhFn\n80voYKLJ9he6o2sYFdrSldtaZq09RqCXC6Ojw/76yTVVlANrX4G3O8LOedBpLEzZA4P+VeXgdV6o\nnzvzxneloLScW9/bTGzy5UdHS8srmLY8lld/OMTNHUJ447b2taaxZHWE+LkzoEUQo6PDuLFdQ9qE\n+ErwEkLUOhK+DHa+3YQ9Jt3vO50FQDsTTba/UEQ9T3o3C+SLHaetsr9bzOksNhw9w729m9h2VLGs\nGLbOgZkdYf1rEHUdTNoON78FPld+269tqC9fTeiBo1KMeHczs9cdu+jmy7tPnePWOZv5aHM89/Zu\nzH9HdzBVE1khhBC/J7cdDRZix/AVk5iNs6OiVUNzba56oTHXhDPx0938HJfK4LY1m6/07tpj+Lg5\ncXd3GzVVrSiH/V/B2pch6xQ07gvXToPQLlY7RVQDb76d0pvnlh3g9ZWHmbfxJANbBhFRz5OcolK2\nnchk7+ksgrxdmTWmkyzHF0KIWkDCl8GCfd1wUNil19f+pCxaBvvg6mTelWA3tAkmPMCDOetPcEOb\n4Cu+dXYgKZuf4tKYMijqd00prUJrOPozrH4B0g5AcHu4+y1oOvDXthHWFOjlyuy7OrPl+FkWbkvg\np7g0sgpKcXJQtA7x4ZkhLRlzTQRedtxAVgghxJWTn9YGc3Z0oIGPm8273FdUWCbb39zB3CMjjg6K\n+/s24bllB9h2MpPuTa5sb8PXVh7Cz8OZe3s3tm6Bp3fAquchYRP4N4ZbP4Q2I8HBtrf5lFL0bBZI\nz8pWA0Wl5bg4Osh8JyGEqIVkYogJhPi5k5RVYNNzxJ/NJ7eozLST7S80qksjAr1ceHvVUbSu/tyv\nDUcz2HD0DJMHNLPeFisZh+Hzu+DDa+HMURjyb8u8rna32Tx4XYybs6MELyGEqKUkfJlAqJ87yTYe\n+dqfZL7O9pfi5uzI5AHN2HLiLGsPp1frtcVl5bywIo6wAHfG9oioeTHZSfDNZJjd3dKhfsCzlhWM\n3e4HJxPsESmEEKLWkfBlAiF+7qRkF1JhhRV+lxJzOhs3ZweigmrHvm9jromgcaAnM74/RFl5RZVf\n9966ExxLz2P68LY1m9tWkAk/PQfvdIZ9X8A1D8Ije6Hf0+BaO/4OhRBCmJOELxMI9XentFyTnlts\ns3PsS8yiTYhvrWlB4OLkwNQbW3I0PY85645X6TUHkrJ5d+0xbu4QwoAWV9hItqQANv7X0jZi8zvQ\nZgRM3gmDZ4CnbO0ihBCi5mrHO/FVLiLA0r084Wy+TY5fVl7BgeRs2teC+V4XuqFNMMM6hPD26qPs\nS8y67HOzC0p56NNdBHi6MO3m1tU/WXkZ7PrIMtK1ahqEdYcHN8KI98DfCrcvhRBCiEoSvkwgonLr\nmIRM20y6P5KWR1FpBR3DzD/f649eHN6W+t6uPPjJLlIusQF5YUk5Dy7cRWp2EbPv7kw9L9eqn0Br\niPvGMqdrxSPgGwZ/+wHu+hKC21rpTyGEEEL8RsKXCYT4uePooGw28nV+1Kg2TLb/I18PZ+beE01O\nURmj3tvCwZSc3/1+clYhd/1vK1tPnuX129rTOdy/6gc/uQH+Nwi+vAeUA9yxCO79CSJ6WvlPIYQQ\nQvxG+nyZgLOjA4383Uk4a5uRr5jEbHzcnIis4ubMZtM21JeF913D/Qt2MvSdjdzYNpjWIT6czMhn\neUwyTg6Kd8d0Zki7KnbET9lnaZB6bBX4hMKwWdDhTnCU/w5CCCFsT95tTCI8wMNm4WtfYhbtG/nV\n6o2WO4b5sfKRPryz5hjf7kvm230p+Lg5MbR9CI8MiiK8KsEy86RlK6D9X4GbH1z3oqVlhLO77f8A\nQgghRCUJXyYRWc+Tb04nWf24RaXlHE7N5YG+Tax+bHur5+XKtGFteP7m1hSXVVS9w3teBvzyBuyc\nBw5O0Ptx6PUIuNe+27BCCCFqPwlfJhFRz4OcojKyCkrw87Be8864lBzKKnStnO91KUop3Jyr0MOr\nOBc2z4Its6C0EDqPhX5TwadmG3YLIYQQNSHhyyTCK9tNxJ8toKMVw9e+05bJ9rVxpeMVKyuGnfMt\no10FZ6D1cBj4HARGGV2ZEEIIIeHLLCIDPQFLry9rBqWYxGyCvF0J9nWz2jFNq6ICDiyGNS9C1imI\n7APXvgCNuhhdmRBCCPErCV8mEf5ro1XrTrqPqZxsf1XT2rJycdULkLYfgtvB3Uug6SCoxYsMhBBC\nXJ0kfJmEm7MjwT5uxJ+xXq+vnKJSTmTkM6JjqNWOaTqJO+Hn5yFhI/hHwq0fQpuR4CAt7IQQQpiT\nhC8TaRrkyfGMPKsd70BiNgDtr8b5XhlHYM10OLgCPOvDjW9Al/HgZL35ckIIIYQtSPgykaggb77a\neRqttVV6csWcD1+htWtPx8vKSYZ1M2DPQnD2gP7PQI9J4OpldGVCCCFElUj4MpGmQV7kl5STkl1E\niF/NG3/uOXWOiHoe+HteBaNBhedg41uw7T2oKIduD0CfJ8GrvtGVCSGEENUi4ctEooIsozdH0/Nq\nHL601uxKOEe/FrU8nJQWwrb3YeObUJQD7UfDgGcs87uEEEKIWkjCl4k0qwxfx9Lz6Ne8ZqEp4WwB\nZ/NLiI4IsEZp9ldeBns/hXWvQm4yRF0Pg56H4LZGVyaEEELUiIQvE6nn6YK/hzPH0ms+6X5nwjkA\nukT41/hYdqU1HPoWVk+HM0cgNBpunQuRvY2uTAghhLAKCV8mopSiWZAXx9Jza3ysXQnn8HZz+vVW\nZq0QvxFWTYPEHRDYHG5fCC2HSq8uIYQQVxUJXybTLMibHw6k1HjF466ETDqH+1dt42mjpe63NEg9\n9jN4h8Cwd6DDGHCUf55CCCGuPvLuZjLNgrzIKijlTF4J9b1dr+gY2YWlHEnL4+b2IVauzsrOxcPa\nV2Dfl+DmA9dNt6xidK75Sk8hhBDCrCR8mUyrYG8ADqbkUN/7yibd7z5l8vle+Wcsm17v+BAcHKHX\nI9D7UXA3ab1CCCGEFUn4Mpk2IZaGqAeSs+l7hSsedyecw9FB0cFsne2Lc2HLu7D5HUsLiU53Q/+p\n4GPyETohhBDCiiR8mYyvhzPhAR7EJuVc8TG2ncikTYgPnq4mubxlJbDrI1j/GhScgVY3w8B/Qf3m\nRlcmhBBC2J1J3p3FhdqG+nAgOfuKXltQUsae0+e4t3cTK1d1BSoq4MASWPuSZX5XZB+4dho0ija4\nMCGEEMI4Er5MqE2IL9/vTyW7sBRfd+dqvXZn/DlKyzU9m9azUXVVoDUcWw2rp1lWMjZoB3ctgWaD\npG2EEEKIOk/Clwm1rdwIOzY5m55NA6v12s3Hz+LsqIiONGjyeuIuWPU8xG8AvwgYORfa3gYODsbU\nI4QQQpiMhC8TahviA0BsUk61w9eW42foFOaPh4udL+2Zo5au9AeXg0cg3Pg6dPkbOF0Fm3oLIYQQ\nViThy4TqebkS4uvGvqTqzfvKLixlf1I2Dw+MslFlF5GTbNl/cc9CS3+u/v+EHpPA1dt+NQghhBC1\niIQvk+oc4c+Ok5nV6nT/y5EMKjT0iareaNkVKcyCTW/B1vegogy63gd9nwKvmm0ILoQQQlztJHyZ\nVLfGAXy7L4XEc4WEBXhU6TVrDqUT4OlCp3AbzvcqLYTtH8CGN6EoG9qNggHPQEBj251TCCGEuIpI\n+DKpbo0DANh+MrNK4ausvIK1h9MZ2DIIR1vs51heBjGfwboZkJMEza6FQc9Dw/bWP5cQQghxFZPw\nZVLNg7zxdXdm+8lMbu3S6C+fv+d0FlkFpQxq2cC6hWgNh76zTKY/cxhCu8CI96FxH+ueRwghhKgj\nJHyZlIODokeTevxyNKNK875WHUzDyUHRt7kV53vFb4JV0yBxO9SLgtGfWLrTS68uIYQQ4opJ+DKx\ngS2DWBmbyqHUXFo19Lnk87TWfLcvhZ7NAvF2q15T1otKi4VVL8DRH8G7Idz8NnS8Gxzln4sQQghR\nUzXqfKmUClBK/ayUOlr5659meiulOiqltiilYpVS+5RSt9fknHVJ/xaWlYNrDqVf9nm7Es6ReK6Q\nWzrWcIPqcwnw9QSY0wtOb7VsBfTwbugyXoKXEEIIYSU1bTs+FVittY4CVld+/UcFwD1a6zbAYOAt\npZRfDc9bJwT5uNEu1JeVB1Iv+7yle5Jwc3bg+jbBV3ai/DPww1SYFQ1xy6DXFHgkBno/Bi5VW2kp\nhBBCiKqp6XDGcKB/5eOPgXXAPy58gtb6yAWPk5VS6UB9IKuG564ThncM4aXvDnI0LZeoBn9uXJpT\nVMrSPUnc1C4EL9dqXs7iPNg6GzbNhNJ86HiXpUmqb6iVqhdCCCHEH9V05KuB1jql8nEqcNmldkqp\nboALcPwSv/+AUmqnUmpnRkZGDUu7OgzvGIqjg2Lx7sSL/v5XOxMpKClnfM/Iqh+0rAS2z4WZnWDt\ny9CkH0zcCsNnSfASQgghbOwvh0qUUquAi93PevbCL7TWWimlL3OchsAnwDitdcXFnqO1/gD4ACA6\nOvqSx6pL6nu7cl2rBny27RQT+zfD1/23CfUFJWW8v/443SIDaNfI968PVlEBsV/DmhfhXDxE9II7\nFkFYV9v9AYQQQgjxO38ZvrTW117q95RSaUqphlrrlMpwddGZ4UopH+A74Fmt9dYrrraOmjywGStj\nU/lwwwkev77Fr99/b/0J0nOLmXN358sfQGs4vsbSNiJ1HzRoC2O+gqjrpG2EEEIIYWc1ve24HBhX\n+Xgc8M0fn6CUcgGWAgu01otreL46qW2oL0PbN2T2uuPsjM8EYOPRM8xac5ThHUPoEhFw6Rcn7YIF\nw2DhSCjKghEfwIQN0Px6CV5CCCGEAZTWV353TylVD/gSCAcSgNFa60ylVDTwoNb6PqXU3cB8IPaC\nl47XWu+93LGjo6P1zp07r7i2q012YSnDZm0kI7eYXs0CWX84g8hAD76e2OviE+3PHLPcXoxbBh71\noO/TEP03cHK1f/FCCCHEVU4ptUtrHV2l59YkfNmShK8/S80u4sXv4ohLzqFrpD9Tb2xFgKfL75+U\nmwrrXoXdC8DJDXpOhh6Twe3STVqFEEIIUTPVCV/SObMWCfZ1490xl5jfVZgFm96GrXOgogy63gt9\nnwKvIPsWKYQQQojLkvBV25UWwY65sOE/UHgO2o2CAc9CQGOjKxNCCCHERUj4qq0qyiHmM1g7A3IS\noekguPZ5aNjB6MqEEEIIcRkSvmobreHw97B6OmQcgpDOMGIONO5rdGVCCCGEqAIJX7VJwhZY9Tyc\n3gb1msHoBdBqmLSMEEIIIWoRCV+1QVocrH4BjqwEr2AY+hZ0GguOcvmEEEKI2kbevc0s6xSsfQVi\nPgdXHxj0PFzzILh4GF2ZEEIIIa6QhC8zyj9rWb24Yy6gLL26ej8OHpfpZC+EEEKIWkHCl5mU5MOW\n2bB5JpTkQccxbOVv+AAABulJREFU0P+f4NvI6MqEEEIIYSUSvsygvBR2fwzrX4e8NGg5FAY+B0Et\nja5MCCGEEFYm4ctIFRUQtxTWvASZJyC8J4z+BMKvMboyIYQQQtiIhC+jHF9raRuREgNBbWDMlxB1\nvbSNEEIIIa5yEr7sLXkPrJoGJ9aBbziMeN+yJZCDo9GVCSGEEMIOJHzZy9njsOZFiF0K7gFwwwzL\n5tdOrkZXJoQQQgg7kvBla7mpsP412L0AHF2h79PQ82Fw8zG6MiGEEEIYQMKXrRRlw6aZsHU2lJdA\nl79Bv6fBK8joyoQQQghhIAlf1lZaBDv+Z2mSWpgJbW+FAc9CvaZGVyaEEEIIE5DwZS0V5ZZtgNbN\ngOzT0HSgZTugkI5GVyaEEEIIE5HwVVNaWza8XvUCZByEkE4wfBY06W90ZUIIIYQwIQlfNXFqK/z8\nPJzeCgFNYdRH0PoW6dUlhBBCiEuS8HUl0uJg9XQ48gN4BcPQ/0KnseDobHRlQgghhDA5CV/VkXXa\nMqdr7yJw9bbsv9j9IXDxNLoyIYQQQtQSEr6qoiDTsnpx+1zL1z0mQZ8nwCPA2LqEEEIIUetI+Lqc\nknxLn65NM6EkDzqMgf5TwS/M6MqEEEIIUUtJ+LqY8lJLR/r1r0FeGrQYAoP+BUGtjK5MCCGEELWc\nhK8LaW3Ze3HNS5B5HMJ7wOgFEN7d6MqEEEIIcZWQ8HXeiXWwahok74H6reDOz6H5YGkbIYQQQgir\nkvCVvNcSuk6sBd8wuGUOtL8dHByNrkwIIYQQV6G6G76Kc2HFI3BgCbgHwA2vQPS94OxmdGVCCCGE\nuIrV3fDl4gV56dDnSeg1Bdx8ja5ICCGEEHVA3Q1fSsG4FTKnSwghhBB25WB0AYaS4CWEEEIIO6vb\n4UsIIYQQws4kfAkhhBBC2JGELyGEEEIIO5LwJYQQQghhRxK+hBBCCCHsSMKXEEIIIYQdSfgSQggh\nhLAjCV9CCCGEEHYk4UsIIYQQwo4kfAkhhBBC2JGELyGEEEIIO5LwJYQQQghhRxK+hBBCCCHsSMKX\nEEIIIYQdSfgSQgghhLAjCV9CCCGEEHYk4UsIIYQQwo6U1troGi5KKZUBJNjhVIHAGTucR1SdXBNz\nkutiPnJNzEmuiznZ+rpEaK3rV+WJpg1f9qKU2qm1jja6DvEbuSbmJNfFfOSamJNcF3My03WR245C\nCCGEEHYk4UsIIYQQwo4kfMEHRhcg/kSuiTnJdTEfuSbmJNfFnExzXer8nC8hhBBCCHuSkS8hhBBC\nCDuS8CWEEEIIYUd1NnwppQYrpQ4rpY4ppaYaXY8ApVSYUmqtUipOKRWrlHrE6JqEhVLKUSm1Ryn1\nrdG1CAullJ9SarFS6pBS6qBSqofRNQlQSj1W+fPrgFLqM6WUm9E11TVKqXlKqXSl1IELvheglPpZ\nKXW08ld/I2usk+FLKeUIvAvcCLQG7lRKtTa2KgGUAU9orVsD3YFJcl1M4xHgoNFFiN95G1iptW4J\ndECuj+GUUqHAFCBaa90WcATuMLaqOukjYPAfvjcVWK21jgJWV35tmDoZvoBuwDGt9QmtdQnwOTDc\n4JrqPK11itZ6d+XjXCxvJqHGViWUUo2Am4D/GV2LsFBK+QJ9gQ8BtNYlWussY6sSlZwAd6WUE+AB\nJBtcT52jtf4FyPzDt4cDH1c+/hi4xa5F/UFdDV+hwOkLvk5E3uRNRSkVCXQCthlbiQDeAp4GKowu\nRPyqMZABzK+8Hfw/pZSn0UXVdVrrJODfwCkgBcjWWv9kbFWiUgOtdUrl41SggZHF1NXwJUxMKeUF\nLAEe1VrnGF1PXaaUGgqka613GV2L+B0noDMwR2vdCcjH4NsoAirnEQ3HEo5DAE+l1N3GViX+SFt6\nbBnaZ6uuhq8kIOyCrxtVfk8YTCnljCV4faq1/troegS9gGFKqXgst+cHKqUWGluSwDJan6i1Pj8y\nvBhLGBPGuhY4qbXO0FqXAl8DPQ2uSVikKaUaAlT+mm5kMXU1fO0AopRSjZVSLlgmRC43uKY6Tyml\nsMxhOai1ftPoegRorf+ptW6ktY7E8v9kjdZaPskbTGudCpxWSrWo/NYgIM7AkoTFKaC7Usqj8ufZ\nIGQhhFksB8ZVPh4HfGNgLTgZeXKjaK3LlFKTgR+xrEaZp7WONbgsYRllGQvsV0rtrfzeM1rr7w2s\nSQizehj4tPID5AngbwbXU+dprbcppRYDu7Gs3t6Diba0qSuUUp8B/YFApVQi8DzwKvClUupeIAEY\nbVyFsr2QEEIIIYRd1dXbjkIIIYQQhpDwJYQQQghhRxK+hBBCCCHsSMKXEEIIIYQdSfgSQgghhLAj\nCV9CCCGEEHYk4UsIIYQQwo7+HyDqHiCUYuWCAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 720x576 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m7OqmGUeOh6K",
        "colab_type": "text"
      },
      "source": [
        "Должна получиться касательная к графику функции $\\sin(5x) / (5x)$ в точке x=4."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dce-_J_9Oh6S",
        "colab_type": "text"
      },
      "source": [
        "В случае, когда у нас функция y = f(x) имеет многомерный аргумент (да и возвращает она многомерное значение), Вы получите на выходе из tf.gradients также мнгомерный тензор, в котором подсчитана производная каждого элемента $y$ по каждому элементу $x$. Например, если у вас $y$ и $x$ являются одномерными массивами, то на tf.gradients вернёт двумерную матрицу $A$ производных, $A_{ij} = \\cfrac{\\partial y_i}{\\partial x_j}$ (то есть так называемый **Якобиан**). \n",
        "\n",
        "По-правде можно и голову сломать, пытаясь понять, какой размерности будут тензоры после подсчёта градиентов, однако на практике сверх рассмотренных нами примеров практически ничего не бывает. Ну или бывает, но такие случаи мы рассматривать в курсе не будем. =)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4pa57HfOOh6Y",
        "colab_type": "text"
      },
      "source": [
        "## Градиентный спуск.\n",
        "\n",
        "Подсчёт градиентов необходим нам для прямого применения методов оптимизации. Чтобы нам не приходилось постоянно программировать формулы для вычисления направлений оптимизации в разных методах, создатели tensorflow уже запрогали многое за нас. Нас будут интересовать так называемые **optimizer**'ы. Эти объекты умеют производить шаги градиентного спуска и оптимизировать произвольные функции. Попробуем с ними поиграть."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sjidnu_IOh6c",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 93
        },
        "outputId": "9f9177ae-cce1-46c2-c60e-ea3f8d90d140"
      },
      "source": [
        "y_guess = tf.Variable(np.zeros(2,dtype='float32'))\n",
        "y_true = tf.range(1,3,dtype='float32')\n",
        "\n",
        "loss = tf.reduce_mean((y_guess - y_true + tf.random_normal([2]))**2) \n",
        "\n",
        "optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss,var_list=y_guess)\n",
        "\n",
        "#same, but more detailed:\n",
        "#updates = [[tf.gradients(loss,y_guess)[0], y_guess]]\n",
        "#optimizer = tf.train.MomentumOptimizer(0.01,0.9).apply_gradients(updates)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:1205: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hqbnk2EAOh6o",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        },
        "outputId": "12a028d8-aadc-40d1-caea-5c10f990ece5"
      },
      "source": [
        "from IPython.display import clear_output\n",
        "\n",
        "sess = tf.Session()\n",
        "sess.run(tf.global_variables_initializer())\n",
        "\n",
        "guesses = [sess.run(y_guess)]\n",
        "\n",
        "for _ in range(250):\n",
        "    sess.run(optimizer)\n",
        "    guesses.append(sess.run(y_guess))\n",
        "    \n",
        "    clear_output(True)\n",
        "    plt.plot(*zip(*guesses),marker='.')\n",
        "    plt.scatter(*sess.run(y_true),c='red')\n",
        "    plt.show()\n",
        "sess.close()"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt81NWd//HXZyYJyD0SBAQCImgR\ntApRsHZVfl6KrpW2akW0tV2VXuhuL7vb7VVd3N1HW3e7225tFS2ttqDVapXWC94XqwTIgMpNEAMJ\nidwZ7pckM5/fH9/JMAkJGWBymcz7+XjkwXzP9zuT8xV85+Sc8z3H3B0REckdofaugIiItC0Fv4hI\njlHwi4jkGAW/iEiOUfCLiOQYBb+ISI5R8IuI5BgFv4hIjlHwi4jkmLz2rkBTioqKfNiwYe1dDRGR\nrBGJRLa5e790ru2QwT9s2DDKysrauxoiIlnDzCrSvVZdPSIiOUbBLyKSYxT8IiI5RsEvIpJjFPwi\nIjlGwS8ikmMU/CIiOabF4DezIWb2mpmtNLMVZvb1Jq4xM/u5ma01s3fNbGzKuVvN7P3E162ZvgER\nETk26bT464B/dPezgAnAdDM7q9E1VwEjE1/TgF8BmNnJwF3AeOAC4C4zK8xQ3UVEstfs2TBsGIRC\nwZ+zZ7fZt24x+N19o7svSbzeA6wCBjW6bDLwiAdKgT5mNhD4BPCSu+9w9yjwEjApo3cgIpJtZs+G\nadOgogLcgz+nTWuz8D+mPn4zGwacByxsdGoQsCHluCpR1ly5iEju+v73Yf/+hmX79wflbSDt4Dez\nHsCTwDfcfXemK2Jm08yszMzKtm7dmumPFxHpOCorj608w9IKfjPLJwj92e7+VBOXVANDUo4HJ8qa\nKz+Cu8909xJ3L+nXL60F5kREslNx8bGVZ1g6s3oM+DWwyt1/2sxlc4HPJ2b3TAB2uftGYB5wpZkV\nJgZ1r0yUiYjkrn//d+jWrWFZt25BeRtIZ1nmi4DPAcvM7O1E2feAYgB3vx94DrgaWAvsB76YOLfD\nzO4BFifeN8Pdd2Su+iIiWejmm4M/v//9oHunuDgI/fryVmbu3ibf6FiUlJS41uMXEUmfmUXcvSSd\na/XkrohIjlHwi4jkGAW/iEiOUfCLiOQYBb+ISI5R8IuI5BgFv4hIjlHwi4jkGAW/iEiOUfCLiOQY\nBb+ISI5R8IuI5BgFv4hIjlHwi4jkGAW/iEiOUfCLiOSYFnfgMrNZwDXAFncf08T5fwbqt43JA0YB\n/RK7b60H9gAxoC7dTQJERKT1pNPi/y0wqbmT7n6vu5/r7ucC3wX+r9H2ihMT5xX6IiIdQIvB7+7z\ngXT3yb0JePSEaiQiIq0qY338ZtaN4DeDJ1OKHXjRzCJmNq2F908zszIzK9u6dWumqiUiIo1kcnD3\nk8Cbjbp5Pu7uY4GrgOlmdnFzb3b3me5e4u4l/fr1y2C1RESaF6mIct9ra4lURNu7Km2mxcHdYzCF\nRt087l6d+HOLmf0JuACYn8HvKSJy3CIVUW5+qJRDtXHywsb/3HguJ3cvYNab69my+yA3nl/M1PHF\nDa4vLd/GhOFFjBta2I41PzEZCX4z6w1cAtySUtYdCLn7nsTrK4EZmfh+IiKZUFq+nZq6OA7Uxpzp\nc5Y2OP9O1TIAzhzQk9mlFTy1tBqAgvBaHp02IWvDP53pnI8ClwJFZlYF3AXkA7j7/YnLPg286O77\nUt7aH/iTmdV/nznu/kLmqi4icmImDO9LQV6Imro4AHE/8prv/WnZEWU1sTjf+sPbXDVmAN26hLlo\nRL/kD4Hgt4LtTBjet8P+YDD3Ju60nZWUlHhZWVl7V0NEOqnUcIag5V/YrYA7n1lOXVPp34Ku+SFm\n3z4BgJsfKqWmLk5B3uGytvhBYGaRdKfNZ7KPX0Skw4qs38GsN9ezZtMe3t+6FwO6JAJ7+sQR3Pfa\n+8cV+gAHa+N8dXaEfQfrOFgbT5bd/vAidh2oAyAvHOL6cYO5buzgdv9NQC1+Eem06lv2uw/U8sD8\n8iPOG3DO4N7069mFhet2sOdgXavXKWxw2aj+ABT17JKxHwRq8YtIzlu8fgc3PrCgyX77eg68U7WL\nU3t3ZdLoAdTF4sx9dyPuTsiMk7vns2VPTUbrFXN4ceXm5PHjZRv4w7QL2/S3AAW/iHRKf31/21FD\nv17Y4OYJQ5k+cQQAt1w4rEGf/G0PL+KVVa33UGldzHlqSZWCX0TkRF18Rj8emP8BtXVx8vNC3HnN\naFZ8uAsHxpzamxl/WZE8Vz/ICzBuaGGDEP7qpSP5vzXbqIu1Xrd4W3e4K/hFpFMaN7SQ2bdPaHZG\nzZkDerY42yZSEeXJJVVMPOMUNu0+wPLq3ccd0kbTAR8OwXVjBx/npx4fBb+IdFqNW+/pngOYs7CS\nHzy9LK3uopaEQ8Y9k4NV7Z9fvpG+3QtYt20f/Xt15UuXnN7ms3wU/CIiKSIVUX7z5jqeW7YxI6EP\ncOP5Q5JLP6QuAdFeFPwiIgmRiihTZi6g9gT68xt36RTkhdq8K6clCn4RkYTS8u3EjqGZf8GwQj51\n3mDunruc2piTHzbuvnYMKz7cxZY9hzilZxc+0wEe2GpMwS8iOampNXVS1+45Wv5fcVZ/vpzSN5/O\nQHFHouAXkZxRH/b5YePeeauJxT25pk79YO8/X3km//HcezQ3yfLLFw/nO1ePalDW0kBxR6PgF5Gc\n0Fz//cHaOD9+YRVDT+7OWx9so3rnwSbfP6BXF/7hsjM6xODsiVLwi0hO+OXra5sdtF20LsqidQ13\n4KofpA1ZMEB7383jsqpVfzQKfhHp1LbsPsh/vriaV1ZtSfs9IeCikUVcNWYg0f01WdN3n650NmKZ\nBVwDbHH3MU2cvxR4BliXKHrK3Wckzk0CfgaEgYfc/UcZqreISLPmlFbwh7INuMOqTbtbnJ4ZNrj1\nY8P4/cJKYrFgGYdvXH5Gpwr7VOm0+H8L/AJ45CjXvOHu16QWmFkYuA+4AqgCFpvZXHdfeZx1FRFp\n0ZyFlXzv6eXJ424FYWpjsaO+58YLirnzk6P523NOzarZOcerxeB39/lmNuw4PvsCYK27lwOY2WPA\nZEDBLyKt5vnlGxsc7685euinPmCVbbNzjlcoQ59zoZm9Y2bPm9noRNkgYEPKNVWJMhGRYxKpiHLf\na2uJVAQDsJH1Oxocp7pqzMC0P/ejg3vz6B3Zu2n68crE4O4SYKi77zWzq4GngZHH+iFmNg2YBlBc\nnP3TpUTk+EQqojy1pArn8KqVNz0Y7GNrwEkF4WQrviAc4u5rRycHYEef2ouNuw40+bkGfOni4fx2\nwfrDSzV/cnTOhT5kIPjdfXfK6+fM7JdmVgRUA0NSLh2cKGvuc2YCMyHYevFE6yUi2efht9bxr39e\nmXxq9o9lG7ihZAg1dcE+tg706prHgZoYDtTE4nz/T8twgmCvv6YxA/7902czdXwxV4wekBP9+Edz\nwsFvZgOAze7uZnYBQffRdmAnMNLMTiMI/CnA1BP9fiLSOc1ZWMldcxsOAdbEnDkLKxuUTRjelxdW\nbKK2Lo5D8oeEA0MKT6IqeiD5g6B/ry6cM7hPg6WPc6Uf/2jSmc75KHApUGRmVcBdQD6Au98PXA98\nxczqgAPAFA92cK8zs68B8wimc85y9xWtchciktUiFVF++MzyJs91yQ9xsDaePN6+rya5wUpht4IG\nO2l95dIRDY4700NXmZTOrJ6bWjj/C4Lpnk2dew547viqJiK5IFIR5T/nvdfsqpipoQ/B4G1qq73x\nAmnZtmBae9CTuyLSbl5etZk7Hi5rcTvDK87qz8HaGFeNGXjEWjmNu27UldMyBb+ItJsV1btaDH0z\nGiyBLCcuU/P4RUSO2cdH9iNsR7+mRC34jFPwi0ibq38gC+CSM/sd9dpPn9exti3sDNTVIyJtKlIR\nZeqDpRxKPJAVOkqLPwRE99e0VdVyhoJfRNpUafl2DqU8kBUHppw/hD7d8ik+uTsrPtzFE2UbiMWd\n/LwQE4b3bdf6dkYKfhFpUxOG9yVsRsyDYV13qN55gBtKhiT78j8zdrCmZLYic+94qyOUlJR4WVlZ\ne1dDRFrJnIWV/PDp5cnwB+iaf3jvWzl2ZhZx95J0rtXgroi0uanji7nnU2NI7d6vqYtTWr693eqU\nS9TVIyLtovGgbdyDB7oMiLtz4elFav23EgW/iLSJSEWU0vLt9Dkpn827D9I1P0Q4ZNSlLNWwtHIn\nSyt3YkCX/LXq+mklCn4RaVWRiihPLqni0YWVLT6lW9SjgO17a3CgNtH1o+DPPPXxi0iriVREuWnm\nAuakEfp5IeNbV5xJl/wQYUNTOVuRWvwiklFBl842uhfk8ZN5q6mJtTxzMGQwY/IYpo4v1uqabUDB\nLyIZE6mIMmXmAmqbCfuPDu7NhcP78uAb5dRfEg4Z9yRCH7S6ZltIZyOWWcA1wBZ3H9PE+ZuBfyHY\n8GYP8BV3fydxbn2iLAbUpTvHVESyU2n59mZDP2wk97i9YvQAnlxShRE8rKWgb1vptPh/S7DRyiPN\nnF8HXOLuUTO7imDf3PEp5ye6+7YTqqWIdFgvr9xMpDLKxDP7NeiTzwsbITPqYnFCZsyYPEbbH3YQ\n6ezANd/Mhh3l/Fsph6UEm6qLSCe14INtPLmkmoM1MZZuiFK98yAAv3r9A7oXhJPXhYC7Pzma6P4a\n9dd3MJnu478NeD7l2IEXzcyBB9x9Zoa/n4i0oTfe38rnfr0oeTywd1cMkpub9+yax76aGACxuBPd\nX8P0iSPapa7SvIxN5zSziQTB/y8pxR9397HAVcB0M7v4KO+fZmZlZla2devWTFVLRDLonQ07abyK\ncn44RMiCQdpPnTsoeDBL0zE7tIy0+M3sHOAh4Cp3Ty624e7ViT+3mNmfgAuA+U19RuK3gZkQLNKW\niXqJSGZdeHoRXfLXUlsXx8zYe6iOmliwxHLcnQf/uo47Pn4aPU/KV/dOB3bCLX4zKwaeAj7n7mtS\nyrubWc/618CVwPIT/X4i0n7GDS1k9u0T+NaVZ/KHL13Iou9dzohTeiTPx+LOg2+UK/Q7uHSmcz4K\nXAoUmVkVcBeQD+Du9wN3An2BX5oZHJ622R/4U6IsD5jj7i+0wj2ISBtKnZHz3y+tYe2WvQ3Oxxwt\ntdDBpTOr56YWzt8O3N5EeTnw0eOvmoh0ZH9aUs3PXnn/iPKCsKlvv4PTk7sickwiFVH+/M6HPLqo\nskG5AVec1Z8vXXK6WvsdnIJfRNIWqYhy04Ol1CT2zM0PG7GYEwpZcq0d6fgU/CLSrPo19Au7FbB+\n+z5eXLEpGfohgxtKhjCoz0kazM0yCn4RaVLj1n29sAWPbOXnhbhO6+xkJQW/iDSptHz7EaFvwJhB\nvRgzqLcWV8ti2ohFRJpU2K3giDIH3qnaxZNLqtq+QpIxCn4RaVLjzdBTHaoNtkWU7KTgF5EmTRje\nl675oSPW5oGg5d/UbwSSHdTHLyJNql+eobR8O3sO1PLA/PIG++au2byn3eomJ0bBLyINzFlYyfPL\nN3LVmIFMHV+cHMAt7tudO59ZTl08iP9HFqxn656DjDilBxefcYoGerOIgl8kx9XP1R9SeBJz3/mQ\nl1dtAeCN97fx+uotXDbqFPp270JdPM7Y4j7s2F/LwdoYVdEDPLtsEwY8ML+c2bdPUPhnCQW/SA4r\nW7+Dmx4sbXaf3BdXbubFlZuPKD9rYE/GDS1kSUUUB2rr4lqYLYtocFckB0Uqotz59HK+/PtIMvQN\nuHhkUYPr7v7kWbzx7Yl8bsJQQolR3rDB355zKt+7ehRdtOlKVlKLXySHRCqizPprOc8v30Siq568\nkOEePIn79cvPYNKYgQ36+AE+dd4gnohsoLYungz51MFfLdmQXcy94212VVJS4mVlZe1dDZGsVN9n\nP2F4X0YN7Mny6t28s2Enr63ewoIPtjeYmRM2uPGC4rTW20n9XIV8x2NmkcReKC1Si18kCzUVwtF9\nNTy7bCP/+ucV1MY8Of++Puh7dc1rEPoGx7TeTuoGLJLd0gp+M5sFXANscfcxTZw34GfA1cB+4Avu\nviRx7lbgB4lL/83dH85ExUVyTRD228gLhfivF9dQG4sTChkfGdCDzbsPsW1vwydtHbjw9L7c8Ten\ncc7gPlRs38/ND5VSWxcnHDJuKBmi9XZyVLot/t8CvwAeaeb8VcDIxNd44FfAeDM7mWCrxhKCf4cR\nM5vr7tETqbRIrikt38bNDy4i1qhrNhZ3ovtquewj/RnZvwcO/Oe81dTFgr74f7ryzGSwF/Xooj55\nAdIMfnefb2bDjnLJZOARDwYMSs2sj5kNJNir9yV33wFgZi8Bk4BHT6TSIrkiUhHljfe38tiiygah\nn7o08v9OHdsgxMcWFzYb7uquEchcH/8gYEPKcVWirLnyI5jZNGAaQHGxdvERmbOwkh8+s5xYYvpN\n6uybO68ZTXR/jcJdjkuHGdx195nATAhm9bRzdUTaTOpA7dmDerN60x7+/O6HPJiyNo4Bnz1fu11J\nZmQq+KuBISnHgxNl1QTdPanlr2foe4pkvUhFlKkPlnKoLo4BZiTn16cKh0y7XUnGZOrJ3bnA5y0w\nAdjl7huBecCVZlZoZoXAlYkyEaHhLldOEPr5YeOfrjiDrvkhQgRdPDMmj1HoS8akO53zUYKWe5GZ\nVRHM1MkHcPf7gecIpnKuJZjO+cXEuR1mdg+wOPFRM+oHekVyWaQiyoIPtrG/po5wyJIrXgLE446F\nTDNwpNWkO6vnphbOOzC9mXOzgFnHXjWRzilSEeXGBxY0CPuiHgVE99Xg0GBJBAW+tIYOM7grkiue\nWlKVDP2QwafHDmJ4UQ8KuxU0O1NHJJMU/CKtIHWmzrlD+rBu217erdrFK6s289yyTcnr4g5PRqoB\n6Jof0pr20iYU/CIZFqmIMmXmAupijhnkh0McSgzg5oWM5uYqa017aSsKfpEMm79ma3KNe3cYfWov\npo4fytmDerPrQA2fn7WIQ7XxJhdM05r20hYU/CIZUN+1c87g3pRv29vg3PXjhnD9uMHJ49m3T+DJ\nJVX8sayKWDxOOBzi+nGDNU9f2oyCXyRNTS2FvPdQHc+/u5HvPb2sye0LQ0B0f8NVM+tn61w3drCm\na0q7UPCLpKF+3Zx43AmFjFEDerJ17yE27z50xLWTRg/g9TVbGuxW1RRN15T2ouAXaUGkIsoPnl6W\nXEohFne276vhohFFnN6vB4bzs1fWJpdCvuPi4dxx8XC15qXDUvCLtODxssoG6+fkhYxfNFoKefzw\noiOCXoEvHZWCXzq9xn3zx7J37Jtrt/HHSFXyONzMujnqtpFsouCXTm3eik18dfYSYnEnbMb40wpZ\ntD5KLO7khY3pE0cw+tTenJQf5qSCMGs27+G197ZQF49THT3Ims17GiyNfOP5Q5g6XvtFSHZT8Eun\n9PjiSma9uZ7Vmw4Hd8ydheuiyZ2samPO/7z8frOfcV5xH24oGczTS6uJxT25MblItlPwS6fz8srN\nfPvJZUCwFk5eyIjHD+9cNeMvK6iti5MXDnHv9edwWlEPDtTGeLyskicj1TgQNrh8VH+mTxzBjecX\na6BWOhUFv3Q6y6t3JV8bcENJw52rzhzQs8kgD4eMv7y78YhpmOq/l85GwS+dzim9uiZfh8OhI56I\nbS7Ixw0t1Br4khPS3YhlEvAzIAw85O4/anT+v4GJicNuwCnu3idxLgYsS5yrdPdrM1FxkaZEKqLc\n+cyy5HEsHufJJcGsnHRm9Kh1L7mgxeA3szBwH3AFUAUsNrO57r6y/hp3/2bK9X8PnJfyEQfc/dzM\nVVmkeaXl20kshAlALB48dfv44g2MP+1kStftwN0pyNMSyJK70tlz9wJgrbuXu3sN8Bgw+SjX3wQ8\nmonKiRyrCcP7UhC2I8rr4s6C8u3E4k7c4VBtnKeWVDXxCSKdXzrBPwjYkHJclSg7gpkNBU4DXk0p\n7mpmZWZWamafOu6aiqRh3NBCHp12IVPHF3PlWf0pCBthCzY5+bdPnU1+4oeCA48urGTOwsr2rbBI\nO8j04O4U4I/uHkspG+ru1WY2HHjVzJa5+weN32hm04BpAMXFekBGjl9qP33jPv3X39vCi6s2AxAH\nfvj0Ms4c0FNdPpJT0mnxVwNDUo4HJ8qaMoVG3TzuXp34sxx4nYb9/6nXzXT3Encv6devXxrVEmnZ\nuKGFTJ84go8M6MnshRUsXL+9wfmYB+MCIrkknRb/YmCkmZ1GEPhTgKmNLzKzjwCFwIKUskJgv7sf\nMrMi4CLgJ5mouEg6nn33Q2a9uZ5VG3ezvybGsL7d2Huwjvql8wvCpl2vJOe0GPzuXmdmXwPmEUzn\nnOXuK8xsBlDm7nMTl04BHnP31N0oRgEPmFmc4LeLH6XOBhJpLYfqYvzg6eU8URYM4IYM/uPTY7jp\ngmKWVO7kqSVVOGjXK8lJafXxu/tzwHONyu5sdHx3E+97Czj7BOonkrb6/vxeXfP4zVvrKd+6L3nO\ngOj+WsxMc/Ul5+nJXekUIhVRpj5YyqHEJP5Tehbww2tGce+81S3uhCWSaxT80uE19bTtnIWVPL98\nI1eNHsDpp/Tgh88sT4a+ATePH8ptHx/OuUMKtQSDSCMKfunQIhVRPnv/guRSygAn5Yc4UBuE/Bvv\nbwOgR5cw4ZAln8r9+MhgZpi6dUSOpOCXDq20fHuD0AeSoV/PgJmfK6FLflite5E0KPilQ5swvC9d\n80PUJLpxCsIhDtY1DH4Hlm7YyfSJIxT4ImlI5wEukXYztrgPX7xoGA7EHQ7Wxbn4jCJCKcvxFGjg\nVuSYqMUvHUr9QO6QwpNYu2Uvz7zzIRXb9yfPhw3Gn9aXr192Bk8uqcKAz2guvsgxUfBLm2tqls6O\nfTXMfbuaf3t2FXXxoE/fgI+N6Ms1Zw/kob+uoy52eFqmBm1Fjp+CX9pUpCLKzQ+Vcqg2TihknD2o\nNxt3HWDz7kMNrjPgK5eezrcnfQSA/zeqvwZuRTJEwS9tqrR8OzV1cRyIxZ3qnQf4mxFFjBrYi3AI\nfvzC6mTL/rJR/ZPvUwtfJHMU/NKmJgzvS344xKG6OGbwzcvPYOr4w8twf1QPXIm0Os3qkTY1bmgh\nd31yNAa4w4y/rCBSEW1wXtMyRVqXgl/aXHR/TfJ1TV1c6+GLtDEFv7S5CcP7UpB3+J9eYbeCdqyN\nSO5R8EubO2dwb26ZEPTrx5vo7hGR1pVW8JvZJDNbbWZrzew7TZz/gpltNbO3E1+3p5y71czeT3zd\nmsnKS/bYvvcQT0aqmD5nCWPveYlf/3V98lytuntE2lSLs3rMLAzcB1wBVAGLzWxuEztp/cHdv9bo\nvScDdwElBEuqRBLvVfOuk4us38Ez73xIbV2c9zbv4e0NO3GHfj27cNWYAQzr242fv7KW2pjWyhdp\na+lM57wAWJvYLB0zewyYDKSzheIngJfcfUfivS8Bk2i0Ibtkt/oncc8d0od9h+p4omwDL63akjw/\nol93vn7ZSC77SH9Gn9qLUGKhnfHDizR1U6QdpBP8g4ANKcdVwPgmrrvOzC4G1gDfdPcNzbx30HHW\nVTqgSEWUKTMXUBs7vHRyQfhwD2LY4NNjBzN94ogj3quHskTaR6YGd/8MDHP3c4CXgIeP9QPMbJqZ\nlZlZ2datWzNULWltcxZVJEPfgBtLBvO72y6ga36IsKFuHJEOKJ0WfzUwJOV4cKIsyd1TR+YeAn6S\n8t5LG7339aa+ibvPBGYClJSUeFPXSMcSqYjy9NLD/xTywsZnzy9m3NBCZt8+Qd04Ih1UOsG/GBhp\nZqcRBPkUYGrqBWY20N03Jg6vBVYlXs8D/sPM6v/PvxL47gnXWtpdpCLK/7y8hljKnijx+OGf1+rG\nEem4Wgx+d68zs68RhHgYmOXuK8xsBlDm7nOBfzCza4E6YAfwhcR7d5jZPQQ/PABm1A/0SvaKVESZ\n+mBpcnPzeu7BImwKfJGOLa1F2tz9OeC5RmV3prz+Ls205N19FjDrBOooHUDqGvql5dupTWnqW+Kr\nIF/9+SLZQKtzSosi63cw5cFS6mJOKGScNaAnqfufh0Nw4/nF2glLJEtoyQZp0UN/XUdtzJNr6Fft\nOkC/nofX13GHU/ucpNAXyRJq8ctRLV6/gxdXbEoeF+SFeOjz5wNw80Ol1NbpyVuRbKPglyNEKqL8\n3+ot7DxQy9NLq6l/NsuA68cd7s7RlE2R7KTglwYaz9gZ1rcbB2pjxONOfl6I68YOTl6rKZsi2UnB\nLw00nrFz0YgiPjN2sFr2Ip2IBncFCAZt363ayaZdBxvM2HmiLFhqSdshinQeavHnqMj6HTy7bCMh\nMyp37Ke0fDu7D9YB0PukPHYdCF7H4q6HskQ6GQV/Dnpt9Rb+7jeLqW/Yn9KzgKvGDORjI/py4el9\n2bDjgGbsiHRiCv4c9PBb65OhHzK49WOnNVg2+ZSeXTVjR6QTU/DnmEhFlDfWHF72Oi/cdIteM3ZE\nOi8N7uaYx8sqm52XLyK5QcGfQyIVUZ4oq0oeN56XLyK5QcGfQ3728hriau2L5Dz18ecAd+dfnnyX\n+e9vI2RB6Ku1L5K7FPw54GevvM/jiS6evJBxQ8kQLaEsksPS6uoxs0lmttrM1prZd5o4/y0zW2lm\n75rZK2Y2NOVczMzeTnzNzWTlpWWvvreZX7y6Nnkci7uWUBbJcS22+M0sDNwHXAFUAYvNbK67r0y5\nbClQ4u77zewrBJut35g4d8Ddz81wvaUF+w7V8dMXVzPrzYZz9vVAloik09VzAbDW3csBzOwxYDKQ\nDH53fy3l+lLglkxWUtL3wvKN/Pqv61jx4W7218SS5SGCBde+cfkZau2L5Lh0unoGARtSjqsSZc25\nDXg+5birmZWZWamZfeo46ihpqNy+n7/7zSK+/PslLF4f5WBtjC9fMpyu+SHCFuyHq9AXEcjw4K6Z\n3QKUAJekFA9192ozGw68ambL3P2DJt47DZgGUFxcnMlqdWrzVmziF6+uZeWHu8AMA5xg5k7Prvla\nekFEjpBOi78aGJJyPDhR1oCZXQ58H7jW3Q/Vl7t7deLPcuB14Lymvom7z3T3Encv6devX9o3kMt+\n/UY5X/pdhGXVQeh/+xNn0iU8dHX2AAAJAUlEQVTRwq/vyx83tFBLKotIA+m0+BcDI83sNILAnwJM\nTb3AzM4DHgAmufuWlPJCYL+7HzKzIuAigoFfSdPLKzfz/PKNDOjVle5d89i2p4Ztew+xbttellXv\nTl7n7tTFXS18EWlRi8Hv7nVm9jVgHhAGZrn7CjObAZS5+1zgXqAH8ISZAVS6+7XAKOABM4sT/Hbx\no0azgaQZtbE4M/68kt+VVjQo714Qpqhnlwa7ZAGEzJJhr8AXkaNJq4/f3Z8DnmtUdmfK68ubed9b\nwNknUsFctLQyynefWsZ7m/Yky0IGX79sJF+//AwgWHfn5odKqamNEwoZMyaPUeCLSFr05G4ri1RE\nk10vQLPdMO7Oa6u38L+vrmVp5U4G9OrKtyedyc9feT+5IcrHRx4e+xg3tFDdOiJyXBT8Jyg12Eef\n2ovt+2rYsbeG7fsOEamI8qvXPyAW92CNHDNicSccMq4+ewBgbNp9kC27D/LhzoPUJLpvwiHj3hvO\n4W9G9mP8aX2bDXd164jI8VDwn4DX3tvMHY9EqIt7i9fGHOp3Ma+LO88u28TgwpPo36srZw/uQ1GP\nA0QqosFTtu68W7WLvxnZT+EuIhmn4D9GkYoozy/bSMWO/bz63hZiidA34OMji/jbswdycvcC+vYo\nYOOug/zT4+9QGwv64SFYKyc/HGLO7eMZN+zkBp+rfW5FpC0o+FtQ35Uz+tReLF6/g1+9/kFyTfvz\nhxbybvUu6mJBWDf1ZOzA3iel1cevPnsRaSvm3nI3RVsrKSnxsrKy9q4GL6/czJd/33RXTtjgW1ee\nyYThzffBi4i0FTOLuHtJOteqxZ9Q37I/rW93qnbu54Xlm1hSuTN53oBPjB7A62u2NOiOUR+8iGQb\nBT/w5tpt3DprUYOW/ehTe3HT+UN4aml1sivnjouHc8fFw9XCF5GslvPB/+p7m/nGY28nQ9+AOy4e\nzveuHgXA9SVDjgh6Bb6IZLOcDf6XV23m3nmrWb1pD4MLT+JgbZxYPGjZf2L0gOR16soRkc4m54I/\nHnd+8sJ73D+/HAj2oP2vGz5KXjikLhwRyQk5E/yRiijPLvuQBWu3syplDRx3p6wiqqWLRSRn5ETw\nRyqi3PjAgmQ//nVjB/Hsso16WEpEclJOBP9jiyqToR82GN6vhx6WEpGc1emDP1IR5aklVcnjcFjz\n70Ukt6Wz9SJmNsnMVpvZWjP7ThPnu5jZHxLnF5rZsJRz302UrzazT2Su6ul5aklVsEAawVTN68cN\nVuCLSE5rMfjNLAzcB1wFnAXcZGZnNbrsNiDq7iOA/wZ+nHjvWQRbNY4GJgG/THxem3hh+UYeW1SZ\nPM7PC3Hd2MFt9e1FRDqkdLp6LgDWJjZLx8weAyYDqVsoTgbuTrz+I/ALC/ZgnAw8lth8fZ2ZrU18\n3oLMVL+hSEWUF1ds4mBtjHeqdvH2hoZLLqi1LyKSXvAPAjakHFcB45u7JrFH7y6gb6K8tNF7Bx13\nbY/irbXbuPnXC+uXvGd4UTduGT+UJyIbkksuqLUvItKBBnfNbBowDaC4uPiY3790w85k6IcMrhs3\nhOkTR/DpsYM0e0dEJEU6wV8NDEk5Hpwoa+qaKjPLA3oD29N8LwDuPhOYCcGyzOlUPtWE4X3pmh86\nYm6+Zu+IiDSUTvAvBkaa2WkEoT0FmNromrnArQR999cDr7q7m9lcYI6Z/RQ4FRgJLMpU5VNpIxMR\nkfS0GPyJPvuvAfOAMDDL3VeY2QygzN3nAr8GfpcYvN1B8MOBxHWPEwwE1wHT3T3WSvei1r2ISBq0\nA5eISCdwLDtwpfUAl4iIdB4KfhGRHKPgFxHJMQp+EZEco+AXEckxHXJWj5ltBSqO8+1FwLYMVqej\n0/12frl2z7rf4zPU3fulc2GHDP4TYWZl6U5p6gx0v51frt2z7rf1qatHRCTHKPhFRHJMZwz+me1d\ngTam++38cu2edb+trNP18YuIyNF1xha/iIgcRdYG/4lsAJ+N0rjfb5nZSjN718xeMbOh7VHPTGnp\nflOuu87M3MyyehZIOvdrZp9N/B2vMLM5bV3HTEvj33Sxmb1mZksT/66vbo96ZoKZzTKzLWa2vJnz\nZmY/T/y3eNfMxrZqhdw9674Ilof+ABgOFADvAGc1uuarwP2J11OAP7R3vVv5ficC3RKvv9LZ7zdx\nXU9gPsH2niXtXe9W/vsdCSwFChPHp7R3vdvgnmcCX0m8PgtY3971PoH7vRgYCyxv5vzVwPME24NP\nABa2Zn2ytcWf3ADe3WuA+g3gU00GHk68/iNwWWID+GzU4v26+2vuvj9xWEqw21m2SufvF+Ae4MfA\nwbasXCtI537vAO5z9yiAu29p4zpmWjr37ECvxOvewIdtWL+Mcvf5BHuVNGcy8IgHSoE+ZjawteqT\nrcHf1AbwjTdxb7ABPFC/AXw2Sud+U91G0HrIVi3eb+JX4SHu/mxbVqyVpPP3ewZwhpm9aWalZjap\nzWrXOtK557uBW8ysCngO+Pu2qVq7ONb/x09Ih9lsXTLDzG4BSoBL2rsurcXMQsBPgS+0c1XaUh5B\nd8+lBL/NzTezs919Z7vWqnXdBPzW3f/LzC4k2OVvjLvH27ti2S5bW/zHsgE8jTaAz0ZpbVpvZpcD\n3weudfdDbVS31tDS/fYExgCvm9l6gj7RuVk8wJvO328VMNfda919HbCG4AdBtkrnnm8DHgdw9wVA\nV4J1bTqjtP4fz5RsDf7kBvBmVkAweDu30TX1G8BDygbwbVjHTGrxfs3sPOABgtDP9v7fo96vu+9y\n9yJ3H+buwwjGNK5192zdrzOdf89PE7T2MbMigq6f8rasZIalc8+VwGUAZjaKIPi3tmkt285c4POJ\n2T0TgF3uvrG1vllWdvX4CWwAn43SvN97gR7AE4kx7Ep3v7bdKn0C0rzfTiPN+50HXGlmK4EY8M/u\nnq2/waZ7z/8IPGhm3yQY6P1CtjbezOxRgh/cRYkxi7uAfAB3v59gDONqYC2wH/hiq9YnS/87iojI\nccrWrh4RETlOCn4RkRyj4BcRyTEKfhGRHKPgFxHJMQp+EZEco+AXEckxCn4RkRzz/wG4J1zSes/x\negAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r9jYXPeLOh64",
        "colab_type": "text"
      },
      "source": [
        "## Собираем всё воедино: моя первая нейросеть.\n",
        "\n",
        "Нейронные сети обучаются при помощи алгоритма обратного распространения ошибки (англ. backpropagation или просто backprop). Хорошая визуализация работы этого алгоритма приведена [тут](https://google-developers.appspot.com/machine-learning/crash-course/backprop-scroll/). На следующем семинаре мы в больших подробностях рассмотрим принцип его работы и даже запрограммируем его вручную при помощи библиотеки numpy, а пока что всё, что нам нужно знать -- это то, что `tensorflow` умеет **эффективно вычислять градиенты** при помощи бэкпропа.\n",
        "\n",
        "Ниже уже идёт более приближенный к реальности код на тензорфлоу. Попробуйте разобраться с ним сами. =)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pVO3R5J-Oh69",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 240
        },
        "outputId": "a72d3604-e9da-4e4b-a670-7542dd36ecbc"
      },
      "source": [
        "from sklearn.datasets import load_digits\n",
        "from sklearn.model_selection import train_test_split\n",
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "tf.reset_default_graph()\n",
        "\n",
        "mnist = load_digits()\n",
        "\n",
        "X,y = mnist.data, mnist.target\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8)\n",
        "print(\"y [shape - %s]:\" % (str(y.shape)), y[:10])\n",
        "print(\"X [shape - %s]:\" % (str(X.shape)))\n",
        "n_labels = len(np.unique(y))\n",
        "\n",
        "batch_size = 32\n",
        "n_batch_train = len(X_train) // batch_size\n",
        "n_batch_test = len(X_test) // batch_size\n",
        "\n",
        "# Входные данные\n",
        "X_input = tf.placeholder(\"float32\", (None, 64))# YOUR CODE\n",
        "y_input = tf.placeholder(\"int32\", (None))# YOUR CODE\n",
        "\n",
        "# Архитектура\n",
        "z2 = tf.layers.dense(X_input, 128, activation=tf.nn.relu)\n",
        "#z2 = tf.layers.dense(z1, 64, activation=tf.nn.relu)\n",
        "logits = tf.layers.dense(z2, n_labels)\n",
        "predictions = tf.nn.softmax(logits)\n",
        "\n",
        "# Лосс\n",
        "loss = tf.nn.softmax_cross_entropy_with_logits_v2(logits=logits, labels=tf.one_hot(y_input, depth=n_labels))\n",
        "\n",
        "# Запускаемая операция одного шаша градиентного спуска.\n",
        "train_op = tf.train.AdamOptimizer().minimize(loss)\n",
        "\n",
        "with tf.Session() as sess:\n",
        "    sess.run(tf.global_variables_initializer())\n",
        "    for epoch in range(10):\n",
        "        y_pred_array = []\n",
        "        for b in range(n_batch_train):\n",
        "            _, loss_ = sess.run([train_op, loss], feed_dict={X_input: X_train[b*batch_size:(b+1)*batch_size],\n",
        "                                                             y_input: y_train[b*batch_size:(b+1)*batch_size]\n",
        "                                                             })\n",
        "        for b in range(n_batch_test + 1):\n",
        "            loss_, y_pred = sess.run([loss, predictions], feed_dict={X_input: X_test[b*batch_size:(b+1)*batch_size],\n",
        "                                          y_input: y_test[b*batch_size:(b+1)*batch_size]\n",
        "                                         })\n",
        "            y_pred_array += np.argmax(y_pred, axis=-1).tolist()\n",
        "        print(accuracy_score(y_pred_array, y_test))"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "y [shape - (1797,)]: [0 1 2 3 4 5 6 7 8 9]\n",
            "X [shape - (1797, 64)]:\n",
            "0.7916666666666666\n",
            "0.9027777777777778\n",
            "0.9305555555555556\n",
            "0.9416666666666667\n",
            "0.95\n",
            "0.9611111111111111\n",
            "0.9694444444444444\n",
            "0.9722222222222222\n",
            "0.975\n",
            "0.9777777777777777\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}