DL_book_Nikolenko/DL_chapter_2_tf.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DL_chapter_2_tf.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "jLtnqhBWWIVc",
        "OBkyCTGqI_rX",
        "Hkm0AJEWKAQX"
      ],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/LenaVolzhina/playing-with-neural-networks/blob/dl-book-chapter-2/DL_chapter_2_tf.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H-yBy5GpFvIz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5f5j-t3bF-wp",
        "colab_type": "text"
      },
      "source": [
        "# 2.6. И о практике: введение в TensorFlow и Keras\n",
        "\n",
        "\n",
        "Среди библиотек общего назначения, которые способны строить граф вычислений и проводить автоматическое дифференцирование, долгое время бесспорным лидером была Theano. Однако в ноябре 2015 года Google выпустила (с открытым исходным кодом) библиотеку TensorFlow, предназначенную для того же самого\n",
        "\n",
        "На конец 2016 Torch считается более эффективной. Пример разделения фронтенда и бекенда: TinyFlow — это библиотека, позволяющая конструировать\n",
        "граф вычислений на TensorFlow, а обучать его на том же самом Torch."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HztpV0OXHR3q",
        "colab_type": "text"
      },
      "source": [
        "Тензор -- многомерный массив чисел\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jLtnqhBWWIVc",
        "colab_type": "text"
      },
      "source": [
        "## Базовое объявление переменных"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LG0QzFVmGByU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "89f3481c-bc8a-4515-dd7d-259294e1b7c3"
      },
      "source": [
        "import tensorflow as tf\n",
        "print(tf.__version__)"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<p style=\"color: red;\">\n",
              "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.<br>\n",
              "We recommend you <a href=\"https://www.tensorflow.org/guide/migrate\" target=\"_blank\">upgrade</a> now \n",
              "or ensure your notebook will continue to use TensorFlow 1.x via the <code>%tensorflow_version 1.x</code> magic:\n",
              "<a href=\"https://colab.research.google.com/notebooks/tensorflow_version.ipynb\" target=\"_blank\">more info</a>.</p>\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "1.15.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zoC2yniqGB1I",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "004f2f5d-45c1-4d8b-e340-a0cf6cad1165"
      },
      "source": [
        "w = tf.Variable(\n",
        "    tf.random_normal([3, 2], mean=0.0, stddev=0.4), \n",
        "    name='weights'\n",
        ")\n",
        "w"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'weights:0' shape=(3, 2) dtype=float32_ref>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8U-zNbdtGB5X",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "956f64ea-ce63-42b0-af10-30ecf475fa5c"
      },
      "source": [
        "b = tf.Variable(tf.zeros([2]), name='biases')\n",
        "b"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'biases_1:0' shape=(2,) dtype=float32_ref>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5mQtPe1GGB9-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "be64598e-7791-420b-c5f0-d732552a36db"
      },
      "source": [
        "w.device, b.device"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('', '')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X_J-81xiGCA7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "with tf.device('/gpu:0'):\n",
        "  w = tf.Variable(\n",
        "      tf.random_normal([3, 2], mean=0.0, stddev=0.4), \n",
        "      name='weights'\n",
        "  )\n",
        "  b = tf.Variable(tf.zeros([2]), name='biases')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UU2rQaqHIF9O",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "44c410ee-26bf-4f8e-b685-6889d6de9200"
      },
      "source": [
        "w.device, b.device"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('/device:GPU:0', '/device:GPU:0')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IlSeGsQ4IGF7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# init values\n",
        "init = tf.global_variables_initializer()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-5ZuToXOIGOW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# how to save a session\n",
        "# saved = saver.save()\n",
        "# saver.restore('model.ckpt')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OBkyCTGqI_rX",
        "colab_type": "text"
      },
      "source": [
        "## 2 * 3 = 6\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6AGmopUKIGQ1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = tf.placeholder(tf.float32)\n",
        "y = tf.placeholder(tf.float32)\n",
        "output = tf.multiply(x, y)\n",
        "with tf.Session() as sess:\n",
        "  result = sess.run(output, feed_dict={x:2, y:3})   # interesting syntax!"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PhOs93jjIF6r",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "6296f1f1-63eb-4789-b575-b432319ec702"
      },
      "source": [
        "result"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "6"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hkm0AJEWKAQX",
        "colab_type": "text"
      },
      "source": [
        "## Суммирование"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KWYg7HhzJiDt",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "5b4e38aa-088c-4cba-d709-105225835372"
      },
      "source": [
        "m = tf.Variable(\n",
        "    tf.random_normal([10, 100], mean=0.0, stddev=0.4), \n",
        "    name='matrix'\n",
        ")\n",
        "v = tf.Variable(\n",
        "    tf.random_normal([1, 100], mean=0.0, stddev=0.4), \n",
        "    name='vector'\n",
        ")\n",
        "sum_1 = m + v\n",
        "sum_1"
      ],
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor 'add_11:0' shape=(10, 100) dtype=float32>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gsbEr-A7L9Ie",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TxqgSM-2Ln6l",
        "colab_type": "text"
      },
      "source": [
        "Хочу изобразить результат"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vZYD2jXqMvxS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "init = tf.global_variables_initializer()\n",
        "\n",
        "sess = tf.Session()\n",
        "sess.run(init)\n",
        "\n",
        "array_m = m.eval(sess)\n",
        "array_v = v.eval(sess)\n",
        "\n",
        "result_sum_1 = sum_1.eval(session=sess)   # :O"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wgpwnGqePF0Y",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "8c51d7bf-53f0-42f5-f69f-22a8b684d6de"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "_, axes = plt.subplots(nrows=3, figsize=(10, 7))\n",
        "axes[0].imshow(array_m)\n",
        "axes[1].imshow(array_v)\n",
        "axes[2].imshow(result_sum_1)"
      ],
      "execution_count": 62,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7fb14d938710>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 62
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlsAAAFlCAYAAADcXS0xAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZydZX3///c158y+JJNJMllmspEE\nSAIECEsAkb2ICIqooFbrUvpttS7VqtRWq9+vbbX+qrbaBQGl/SlqWRRZZAfZSUIIIRsJ2ZOZSTL7\nvl6/P+bQ5uHnc7N86T0zJ7/X8/HwYeYzF/e5z31d130+c871OVeIMQoAAADpKBjvEwAAADiSkWwB\nAACkiGQLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIrGNNkKIVwcQtgSQtgWQvjSWD42AADAeAhj9T1b\nIYSMpJckXShpr6RVkq6OMW702ldUF8aa2SUm3tQ62TYuGfEfs8fmkiPZhPMrssfItvm56OCkhGvm\nnEbI+G3jkD12yPrPI9uScB4VTjApfS6w5xEGgt/WOeVM+ZDbdGjAv6AF/U5s0H+4oXIbC8N+W6+v\nC7N+48GhTMJBnMcLfj8VHrAXdKDSv26xOGEc9jl9nTSEnMvpjU1JKuiyxx0uSzhw0jR3xkV1aa/b\ntK3ddlQo8a99gXPcof6E/hj2r2e2zI65kXZ/vGUmOYPrUKHbdnByQj/12+uZ9S+FovNUhov9tpm+\n1982adwXtdlr0TfVvxbe1cyW+ZNvsD/phujEnD6V/L6uLu5x27Y2V5pYLPf7Izr9IfnjYihhrme6\n7BMJ/sNpqNQJJtxPCzv9uHdPPq76oNt2y86pJjaSTbgnJ93XnecyVJnQT/3OtajwB1zstNdzxJ9O\nifcyb8wljbeiNhsbqPaPO6vSNt7f7jfOFvuvWyOd9jySxmFBh5NHVPlt+7fvPxRjnOaei/tfpONU\nSdtijNslKYTwM0mXS3KTrZrZJfriLSeb+HdvuczERhb5E7t4rX1x6JnhX6Si+m4Tq77VyQIkNV4y\n4MbjgO2Uokq/7eBBO7OLa/3nMfXmMjfesNIZBGUJNy7nBlW0t8htm3GSsPLTDrltm3f5g7x8h52s\nFfv9c2s6zcaKEhJdHWvvcjOrO9ymjW1V/jEc2YSErfYfbcK/91z/lXJ4of/KnN1s+6/Avweof4qT\nTM61Y1OSih+zL1ztJzpZriSN+DfxgiL7vK9cttZte/s9K+25He1f+8pSex5Nu6e4bbNt/gtlzfH2\nRar3vun+413caGLxBveep6Z3+nOycKudk1M2+ONioNKOz46j3Kaa9JKNdSzw+6PIv5yq+3WTiW39\nqH8tCpxTnnay/e8lad8O+4I/ehD7Cpqt9BO2klJ7Pa9YsM5te/tNbzWxwZV+5jK03ftrUqo5zo6L\nA4f8uT7pGTt/C7v87KD5BBuPxX7bmY+4YTWcbWPPvvvf3LbnfvQPTax3qj8Xhov88ZLpt+fXdLY/\nZst22myp6PQWt+3II3audtX79++Mk8RJ0vQT7Zjb5ySYkjTvdnvsnVf4x/36ObeZ2FfueY/btmZR\nsxvve9DeGwZO98dhxb12HHb9Xpfbdut7vrrL/YXG9mPE2ZL2HPbz3lwMAADgiDWhFsiHEK4JIawO\nIazuak34zAkAACCPjGWytU9S/WE/1+Vi/yXGeF2McUWMcUVFdcIHxAAAAHlkLJOtVZIWhRDmhxCK\nJF0l6Y4xfHwAAIAxN2bViJIUQrhE0nclZSTdGGP8RlLb0oWz4vxvX2Pik8rsIuSWJ2e87nPor0la\nQG4XFRY4FRmSlJ3tL2T3FllfPG+T2/bBH59uYh2LEypDSv14xWa7wL3qArtQWJKamieZ2Atv9Rdu\nnvPlT5vYQJW/WLHzNH9ReOl6u9h4YLm/0HvKXbbtG1nkmVTBdcq71rvx1Q31JlZa5H9s3bHaLqQc\nSqj4+6O33efG77z2fBMr+kyD23b3k3UmNtlZYC1JbYttbMom/9waz/dX5BcetNczzvH7tHCTXei/\n5CL/5DY22TlZWuwvTO/stouYJWl2TbuJ7dxa67Z978pnTey2++yCfkladL2/WHzzn9q+rtrq3wM6\nT7XXqGydV84mdS21xQJXHJ9QhLD+RDdeWmlLGqdV+vOp+f5ZJjb/0u1u2ySbnplvYp96+91u2zsb\njzOxbS/YcSxJZQ3273uvWlOSuhLuLcvq9pvY+t32OUtSwX47toaq/bkw9SlbL1bxfvtYkrRn3Uw3\nPlxhX1+mPeWPoUOn2rbV6/z3P/pq/PuvV1kZk95CcQ6R9V/K3Grbqp3+jbZtYcLrpHPsjoX+62/5\nPnvSvScklAI7Kp/w596KD/uFGg+sXWpiRc2vv3q9ImEZ/PP/9rk1McYV3u/GshpRMca7JfkzFgAA\n4Ag0oRbIAwAAHGlItgAAAFJEsgUAAJCiMV0g/0ZMPXZqfMdN7zDxxx+wizGTFgQO1dmFqZ886RG3\nbfuwXWD3i9vttx1L0mWXP+nGH/kHuyB35H3+N9gO3mu/Sbf9BH8Bccb59ndJqnzcLlhuW+4v9C46\naJfnFQz6iy7L99kxUXalv/C+pdv/dvsyZzH04F3+N3q3nWjPeeWSbW7bp9YtMrGqrf7Sw+IL/G0y\nmrfWmFi2M2kBqhN3vsVekpbM8K/R5gN2UXfvIf+6eYs0a17w5+jBkxK29nCEhG+hH9ljd0koTLgW\nQxX2PI5asdttu2WL/b7id5ziLwp/qcP/JvQtW+2i57Jd/lfCeIusMwlfpj/9Cv+cm39qCyf+7S+/\n57b9xKb3m1hHj7/Qf1K5XejbtC3hm9ur/Pk7/X5bDNN0zusvehhK2F6kdJ+/KLj3aHtBCw74OycU\ndtvx0l+dsPXJVNspwwnbOIVuPz5rkZ3XLU8kFEmd4Hwl/zr/2+Z759hrX7XRH29F57/+HTUyzrZa\nkjQ02fZfQcJznr3UL+poWmWf96Kzdrpte/7GzsmuWf7z673cFqd0Nfnf6P+nZz3gxq+/+WITGy7x\n72XeNmPTVrtNNVJox1vrEr/t4GR/UX8otdf+3KP9Yp9HttnXnGyhf9xt7/1K4gJ53tkCAABIEckW\nAABAiki2AAAAUkSyBQAAkCKSLQAAgBSN6TfIvxF9TaXa/A/LTLy01lYiLHjPVvcYW++0VQS3Tl/u\ntj2r1m5nMVTqV04sLd3rxn9xpq1QWFntV8Q9tcxun5Nt9itDSjfaSiTJrzzMtvhdOu/0PSZ2xlR/\nC48fP3umifUlVB2Gp+zzkKSRVnvtRmzhmySpfo6t7Hlqve07STpx2Q4Tqz7J39Zh2zf8EpUFn9ln\nYtt2+tvAZFqc7YF2+lU515z0Wzf+lzd91MQGzvP3KBnqs3//xD/wx1Bht62g7dvvX+Tqh/141rl0\n/X6XqtLZoqL7KX9blsJT7fPY3uVX4B3o8q9n+XZ77XuP8/s6NNhKwMrFrW7bl3b7fV00z8b+9Euf\nctuWftxut9Qhvxqx9Vn7eDd+8F/dtp/71h/55/b7ttK1YIP/PKafaCvX4g1+JXBhl1/9uKvO3kfK\nFrW5bXuc7ZauWrLGbXvr3fbeMul4v2K7dn6XG+/4ga0aDf7tQj1O1W+pXyipsil2f5nOhf7YrLnN\nVjRLUnDm9VDGv68XV9u2I83+4+19ya/YVZ2t+u4a8KtG9Xl7n40/87cdKrvN3gR63+bPvR/faKsO\nJanveKccOPivqZcs2WBid5Wd4LYtbLcVm0tW+q9l67bZsSJJsdeO74P9/rV/+bwfmdiKr/6x2/bV\n8M4WAABAiki2AAAAUkSyBQAAkCKSLQAAgBRN2O16iuvr46zPfcbESw/Y/DCc5i/cHNhoF/klLXrP\nOltOTDvF3yIhaWuImU/YBYF7LvAXt5c028cr7PTPreVEf2uAgn5nMXVBQn86u0DEYv+4ythjZBPa\nfnjZ02789h+ca2J9U/xtYGpX2es2MMlf6F/+J3Zx+0s7ErbqcJ6HJBXvsX2SXeZs6yEpPG3HUH+1\nf9yyBv/5Xfnxh0zs6Zb5bts9bZNNrG+DjUlShbOwuHODv3B3sMbf2uVH591gYh+96w/dtrMW24X6\nRd+225NI0u6L7aLgEX+dsMr2+3/zFZ7RYmLte/zV+8HZeqp6o98fLSckbF2z35skblP119hjFLUm\nbMuyzG6VVD/NX7zf9IBfcDB8st0iqq8zYSG08/QKD/kXv3Kpvzh96EFbzDCc8HCZlfa5dO30+6ls\nrp1nFSX+vkoHN/iL+ksW2GvR3e4XJ2Sb7Fw/4Uy/oGrjvYtNrG+Gf98raUjY5ugou2C9qNzfhu03\np/+ziX294W1u20fWH+PG6+6xY27feW5Td2umir2v//W/q86fT/Fk/95Z/LDdFin7Nn+bo5ZWW8Az\ne7r/un7uDLutzsONtu8kac8OfwyFAftcYnnC6+GwbVuQsIXezg98me16AAAAxgPJFgAAQIpItgAA\nAFJEsgUAAJCiMf0G+RDCTkmdkoYlDSUtJAMAADhSjGk1Yi7ZWhFj9EsSDlMyuz7W/8lnTbzK+Vb+\nloStTwr22wqV2SfabTYkadd2ux3CtHq/Yqj3Yb/CoWe2U6HU7r952DfDVjOcs3yT23ZOqa3KkqQ7\nf3C2iU27arfbduvzdtuCWONXyYRmW8ETixLGSYVflVHgVD8lVZ151X2D1X7F2OQN9hjVW/znUfDl\nA268+RZb8VV3ld0GSJJ237LAxHrP8rcRGd7tb4kzXG23RCnd6Vepzn7MbonRvMSvtBq40FYBFT7g\nV4Gt/Utb+SRJ8+/5uInNutf/GyyM2H5qOcavyhoucfq0yu/TmgX+PGtucbbPSBiGRaX2Gg/v8Lff\nqF7m334uq19vYjc++la37ekn2YqoRRX+eHvw62eZWOkBf8zO+7Y9riQ98OKxJjZpnT+GuuvsRQpz\nbEWkJP3Jcf4WU5u67TYuz950ov949fbxkqocW1ttnxTu8cscB+v8KsXJ1fa5dL8wxW3ryfT7VXXD\ny+y8nj/Nfx5Nt8114xX7bUVbw7v9vj76i3a8bPyav33Ownl+ZfzuQ7YaeKDVv19cccpqE7vtBb9P\ny7fYPulemHCfTahUH+l2tjpzKvskadkyuxfY5qf8iu1h57XI+5YCSeo73m7BJEmx0V6jkcn+a1nR\nfvs8huf7OceO91ONCAAAMC7GOtmKku4LIawJIVwzxo8NAAAw5sZ0zZaks2KM+0II0yXdH0LYHGP8\nr/excwnYNZKUneR/WSIAAEA+GdN3tmKM+3L/f0DS7ZJO/Z3fXxdjXBFjXJEp99e/AAAA5JMxS7ZC\nCOUhhMpX/i3pIkkvjtXjAwAAjIcxq0YMISzQ6LtZ0ujHlz+NMX4jqX3x3Lo449pPm3j9UXZ/troK\nfw+lZ5+0+0kNJ1QclOyxFQf9tX6VRabar5LJZG37/pZSt21Bn81zR0oTHq/Lr/iqO67RxIZG/Pz5\n4lkbTexH61a6bYs323O+6Ipn3bb33X6qGx9aaiuGplfbPc0k6dAzdm/DrF9E4uqd4Ve5jVT5fZ0t\ntfGpCedWVWT7enuT3TdOkirK/QqVtqZKE6ue4e8nlvmlU1V1hV8R1d5lK2qKnvcr8OoustU+krTr\nEVtVNXysX7k2699t9dtff9/urShJf77pShNrbfffra542p8jNZfvNbH2m2e7bduciuSkW1tFhd9P\n3n5+2S6/eqpgyMbL9vsP2GULgfXuS59w2/7BlKfc+Ec/92cmdvXX73bb/j8PXGJi85f4Vdg7Nsxy\n45k++/xCwtZx2V7btuJ0e5+WpEyBnau9d9e6bQt/z68aLf2h3Su08TT/Hrnw/7Vzp+e7flXdgUft\nteida6tcJWn6Y/4KnAPn2WMXlvj3oS8uv9fEvvPjK9y2x1zqV6kOjNjzqCvzXw/Xfme5iTUfn7Df\noROb9YQ/ADrr/Gv/5c/+xMS+9q8fdNv2TbWPWLDQr/oeOFBmYsW1CVWHm/z74XlvW2tiaw/595b2\nx+34LDi53W276V1fS6xGHLM1WzHG7ZJOGKvHAwAAmAj46gcAAIAUkWwBAACkiGQLAAAgRWO6Xc8b\nUVZbHxdebReFnv/hp03sV5v9pWCZnXYB8WC9v7i9pMwubFw+c5/b9qn1i9x45Qy7yLprT5XbNjvV\nLtLNbvAXEJfv8/uo42K7kHlkp3+MoWn+Qk9PaZU9t+EtdpG3JA3U+sctdLbrKd+TsBjTSfmH/TXT\n6p1mr8Vwub9w89hj7QJrSer5e7sQcs/V/iLWiufsiZRd5G+d0birxo17q01LGvzlkhlneFbu8QsA\nMh+y2300P2mLDSQps9xf0DmyxlkUfoq/fc7AC3Zh8kDCtkrBWUDuxSSpdlXCwvKZdmB0Lvb7unSv\nv0jXU56wkL15uY3XL7VFKJJ06EG7mHr2Rf5WWUPOAN//W7tllCRVv+Rfz6a324FRsjGh+Ma5RNkz\n/S2/+tb629yMZO21KEwoFvC22xqa7i9C/9NTHjax+w7YrYgkafszc9z4cKl9vJEyf1yEQXvtY9a/\nxm8/6QUTu2uN/9oy6yH/fYqGS+z9MCaM+2yLvUdm/NoNlfn1DW77Q2f497KqDfbxuuYlzN9pdry9\n69jn3bZPfcMvktIf2iKJzy54wG36+Yeusucw6F837z5y2qlb3LZPv+xv+VO6yeYGmdMStufrsVsX\nDbf4W0zt+sSfs10PAADAeCDZAgAASBHJFgAAQIpItgAAAFJEsgUAAJCiMfsG+TdquEjqdColNrTP\nNLEldX6pRs8PbcVQ21F+FcG0Z+zX/T/3Hr9KZuZGv4LjwApbrZWt97c+GWyz1RCVB/wqqfbFblgZ\nZyuCEr+gQiOH7FYrxa3+401d02tiL33Ef84fO/VxN/7L751rYs0r/CqZyS/aYdhX45/byHRbJVPQ\nbJ+bJG1Z51czjVxmn0v1k7Y/JKn1RFtdVPzz6W7bug7/GvV/zFaCxXq/0sbbgudArd2eQpIq77aV\nh8Nn+FtcnD7Lr5R74d5lJta7ptptO+M5ey26/9ivcjy0186FTJVfoTZpha2qlKQDDfY6V5b51cQd\nGVstW/Gyf3v76l/9yI1/8qHfN7GDj/jb2dR980kT2116ht/2YTufKuclVEQuTahc223HxcBx/hYl\nxSW2n4ajf9yBKn/MViyw/drZ5o9Dtdsqt7n/6f8d/89FbzWxo2b4W/skbQ9UMGCfSwx+NWosts/v\n2MV+lfmj/3myiU1p9/tpuMiPV0x2+uQxfz4Nnm6r1yff7leTtx/lX8+hpXa+nzNvh9t29UvHmVjS\nNY7OsLjtkdPctv/4zR+78U89dbWJ/c3gxW7bwkl2Xg81+/fksnl2q7ODff62PMXb/WP0TbNPsHi1\n309Vp9lto7qdrfleC+9sAQAApIhkCwAAIEUkWwAAACki2QIAAEjRhN2up2R2fZz7R3a7nn5ne5BP\nnn+fe4zfNtttddZtq3fbFh60izw/8LZH3bY/XuMvhC3fYhdqz3zCX8S67QO27VfP/aXbdkOP3V5G\nkp7785NMrOkUvwAgnmIXvE76hb8FT+MldiFz9RR/4fWvTrjRjZ/9yKdM7Jh6f+uT7Y/NNbGYsPvK\n4CTb/9mpdgGyJP3o1B+78Q8+co2JTX3M9r8kZQbt/EharFpi11FKkrpn22NUb/bbDl5pF9NX3Gi3\n1JGkxqvsXh0lzuJoSar8qX+M5ittAUdVub9nSHmRHRctd/ljs3O+7aeKuf5i+oIH/YWpQ85a4aod\n/oLu4WK7aLrnXf7jDbzoX4uyJm+LIf/+OO/920xs4yML3baDC+z1XFrvF/W8+IKdC5JUUGMXEM//\nZ7epmr9g58MHF6xy2/7T4xe48arNTtGKs1WW5G+fUrnLb1uz1i5u3nmtP59W1O1x48/81hYu1Z28\n321bEOx57Gz0t9Uq2GsXUyfN067ZfsHB0HH2Pjmy1y8sGHG2HSo+4N/4Bpz7niSFWjsuhvv8Y5y8\neKeJrdnsb2dT86zT/2+zfSdJPQ3+4vRsu+3X2lX+81j8xQ0mtrTC79Nf/N1FJtZ0VsL16ffHVlm9\nLU4o/bW/td5ghe3rjpP9Qp1dH76W7XoAAADGA8kWAABAiki2AAAAUkSyBQAAkCKSLQAAgBRN2GrE\nEMJBSbtyP06VlFDrhTxA/+Uv+i6/0X/5jf7LL3NjjNO8X0zYZOtwIYTVSeWUmPjov/xF3+U3+i+/\n0X9HDj5GBAAASBHJFgAAQIryJdm6brxPAG8K/Ze/6Lv8Rv/lN/rvCJEXa7YAAADyVb68swUAAJCX\nJnyyFUK4OISwJYSwLYTwpfE+HyQLIdSHEB4OIWwMIWwIIXw6F58SQrg/hLA19//+zsOYEEIImRDC\n2hDCnbmf54cQnsnNwZ+HEOwu6hh3IYTJIYRbQgibQwibQggrmXv5I4Tw2dx988UQws0hhBLm3pFj\nQidbIYSMpB9IepukJZKuDiEsGd+zwqsYkvS5GOMSSadL+kSuv74k6cEY4yJJD+Z+xsT1aUmbDvv5\nm5K+E2NcKKlV0sfG5azwWr4n6TcxxmMknaDRPmTu5YEQwmxJn5K0Isa4TFJG0lVi7h0xJnSyJelU\nSdtijNtjjAOSfibp8nE+JySIMTbEGJ/L/btTozf72Rrts5tyzW6S9M7xOUO8lhBCnaS3S7o+93OQ\ndJ6kW3JN6L8JKIQwSdLZkm6QpBjjQIyxTcy9fJKVVBpCyEoqk9Qg5t4RY6InW7Ml7Tns5725GCa4\nEMI8SSdKekZSbYyxIferRkm143RaeG3flfQFSSO5n2sktcUYh3I/MwcnpvmSDkr6Ue4j4OtDCOVi\n7uWFGOM+Sd+WtFujSVa7pDVi7h0xJnqyhTwUQqiQdKukz8QYOw7/XRwtf6UEdgIKIVwq6UCMcc14\nnwvesKykkyT9S4zxREnd+p2PDJl7E1duLd3lGk2aZ0kql3TxuJ4U/kdN9GRrn6T6w36uy8UwQYUQ\nCjWaaP0kxnhbLtwUQpiZ+/1MSQfG6/zwqs6UdFkIYadGP7I/T6PrgCbnPtqQmIMT1V5Je2OMz+R+\nvkWjyRdzLz9cIGlHjPFgjHFQ0m0anY/MvSPERE+2VklalKvIKNLogsE7xvmckCC3vucGSZtijP9w\n2K/ukPTh3L8/LOlXY31ueG0xxmtjjHUxxnkanWsPxRg/IOlhSVfmmtF/E1CMsVHSnhDC0bnQ+ZI2\nirmXL3ZLOj2EUJa7j77Sf8y9I8SE/1LTEMIlGl1HkpF0Y4zxG+N8SkgQQjhL0mOS1uu/1/z8hUbX\nbf1C0hxJuyS9N8bYMi4nidclhHCOpM/HGC8NISzQ6DtdUyStlfTBGGP/eJ4frBDCco0WNhRJ2i7p\nIxr9g5q5lwdCCF+T9D6NVnWvlfRxja7RYu4dASZ8sgUAAJDPJvrHiAAAAHmNZAsAACBFJFsAAAAp\nItkCAABIEckWAABAiki2AAAAUkSyBQAAkCKSLQAAgBSRbAEAAKSIZAsAACBFJFsAAAApItkCAABI\nEckWAABAiki2AAAAUkSyBQAAkCKSLQAAgBSRbAEAAKSIZAsAACBFJFsAAAApItkCAABIEckWAABA\niki2AAAAUkSyBQAAkCKSLQAAgBSRbAEAAKSIZAsAACBFJFsAAAApItkCAABIEckWAABAiki2AAAA\nUkSyBQAAkCKSLQAAgBSRbAEAAKSIZAsAACBFJFsAAAApItkCAABIEckWAABAiki2AAAAUkSyBQAA\nkCKSLQAAgBSRbAEAAKSIZAsAACBFJFsAAAApItkCAABIEckWAABAiki2AAAAUkSyBQAAkCKSLQAA\ngBSRbAEAAKSIZAsAACBFJFsAAAApItkCAABIEckWAABAiki2AAAAUkSyBQAAkCKSLQAAgBSRbAEA\nAKSIZAsAACBFJFsAAAApItkCAABIEckWAABAiki2AAAAUkSyBQAAkCKSLQAAgBSRbAEAAKSIZAsA\nACBFJFsAAAApItkCAABIEckWAABAiki2AAAAUkSyBQAAkCKSLQAAgBSRbAEAAKSIZAsAACBFJFsA\nAAApItkCAABIEckWAABAiki2AAAAUkSyBQAAkCKSLQAAgBSRbAEAAKSIZAsAACBFJFsAAAApItkC\nAABI0ZtKtkIIU0II94cQtub+vzqh3XAI4fnc/+54M48JAACQT0KM8f/+Pw7hW5JaYox/F0L4kqTq\nGOMXnXZdMcaKN3GeAAAAeenNJltbJJ0TY2wIIcyU9EiM8WinHckWAAD4/6U3u2arNpdoXSzpEUkL\nc+9w/a6SEEJLCKEvhPBSCGHem3xcAACAvPCa72yFEB6QNMP51Zcl3SSpRtJLki6UtEbSbklXxxg3\nHnaMayXNlfQtSc9IeibGeKnzWNdIukaSQknhySV1NfZ8WrImNlzsn3ssGTGxgh4/v8xUDZrYUHeh\n27agbMiNLylrNbH1zdPctsGemgr8w6poSr8bLywYNrH+PSVu25FC+7wzvQkPONee3OCwf90qCgfc\neP92e+36Z2fctkWNNjYy2z43SRpptcctrulz2w4e9K9FrLHPuzRj+1+SultKTSwkTJmRIj8+b8oB\nE2sbLnfbdjba+Ig/DJVxLn3VzE63bWt/mRsfGbL9Wl7qj7eRGEysv92ffIWN3SY2sMBeS0nKtPlj\na8g55aIO/+IPTnMmVI8/3mbVtLjxpoYp9hzKEzrbuRbZXr/piL1l6ahpTW7bbc21brzAn2auohrb\nf8UJN5f2Nn8cemM82+Nfi4HJNlZc4s+ngW47SQrLE9r2Jdx/neleUOU/v2Gnn6Iz5kd/YUNh2P73\nkhQSbp1ymhcd8u9PlUfbTm3p9/tD7f5YPnqmHUebDvpjqGySPY++Q/49MjNgL0bJbH+A9zT595Zh\nZ7qHEv++XtDqPz9P/Wx7P60Ifj9t6nWXkWu4y07Kmmr/3tnWWGliI9X+8+h7ueFQjNF90f+f+Bjx\nc5L+WtJkSXMkPSTp0Rjj3x7Wbr2kOkm7cm1KJZXFV3nwskUz4+LvfMzEi35hL17HXH/yDC7tMbHS\nNf7AqLrQvuIfXOUP2vIT/Jv1cyt+bmKL/uOP3baFHXZwlLT4l2POVdvdeG1ph4lt/zPzKa4kqXeG\nfVGs2uA/j4J/6TKx/R1VblOkVO0AACAASURBVNszZu5049vfP9vEdvyNfyOZ+y0b6/rf9sVaknpv\ns31y1Idectvu/f4iNz7yoUMmduwU/8XvuZuPM7GkpLhnlt9/N139fRO7vf1kt+1vv3W6iXXV+eN7\n0g474c//q8fdtrfvON6NdzbZT/dXLtvmtu0asmPo5d8scNvW/e2TJrbj5hPctlPu9JOwQ8vtHJlz\nv//C3HiN84K21h+zX/nQzW78O9+4ysQOrPRvqmHA9knNOv+G3zfVxm/9xN+7bd/x759345U7bSwm\nvD7N/sAOE1tUYV+gJOme2+x4kyTvb4+a9f6133WZfX6LF+932+54pt7E6k/d57fdPNONV261T7zo\nQjunJam7zyZ3va3+eJOTWBW2+Be5pNnva69P6m/Y7LY999FdJvbTl09x2xbc7ScNj/7Vd0zslB/+\nmdv2hIvseWz7sf96UbXL9vWi/7PBbfvCd/x53Xy8cz0X29csSaq83SY0wZ96+vY3/tnEzizx75Er\n173bjbc/bl9HPvi+B922d3zzXBPrvMJPzDZf8bU1McYV3u/e7MeId0i6WtJiSbdJ+kdJSyQte6VB\nrkJxiqRfSrpAUrOkgxp9RwwAAOCI9maTrb+TdJpG36k6SdLfSHpa0nEhhOtzbY6VNF3S5ZIezv03\n7p9IIYRrQgirQwirh9rtu1IAAAD55k0lWzHGZkk3SmqKMV4QY3zlc6nuGOPHc22elLRZ0ohGPxW/\nRFK1Rt/h+t3jXRdjXBFjXJGd5H/cBwAAkE/e1JotSQohvFfS9ZJOkLRP0jZJj8cY339Ymy9IWhhj\nvCaE8ENJl8UYzYemhy+Ql3S0pC25f0+V5H8oj3xA/+Uv+i6/0X/5jf7LL3NTWSAvSSGElZK+L6lS\nUkajydYjGv1ocXWM8Y4QQomk/5B0oqQWScfEGP3Vq/5jrE5adIaJj/7LX/RdfqP/8hv9d+RwipLf\nsFUa/VjwfI2+s7VK0h0xxsNLF6pjjO+RpBDCuySZb5kHAAA4Er3pZCvGOBRC+KSkezX6ztaNMcYN\nIYSvK/fOlqRPhRAukzSk0Xe2/uDNPi4AAEA++J94Z0sxxrsl3f07sa8c9u9rJV37Jh7iujfx32L8\n0X/5i77Lb/RffqP/jhBves0WAAAAkr3Z79kCAADAq5jwyVYI4eIQwpYQwraETa4xQYQQ6kMID4cQ\nNoYQNoQQPp2LTwkh3B9C2Jr7f3/vCUwIIYRMCGFtCOHO3M/zQwjP5Obgz0MICbtAYjyFECaHEG4J\nIWwOIWwKIaxk7uWPEMJnc/fNF0MIN4cQSph7R44JnWyFEDKSfiDpbRrdBujqEMKS8T0rvIohSZ+L\nMS6RdLqkT+T660uSHowxLpL0YO5nTFyflrTpsJ+/Kek7McaFklol2U1LMRF8T9JvYozHaPR7DzeJ\nuZcXQgizJX1K0ooY4zKNFptdJebeEWNCJ1uSTpW0Lca4PcY4IOlnGt32BxNQjLEhxvhc7t+dGr3Z\nz9Zon92Ua3aTpHeOzxnitYQQ6iS9XaNfVKwQQpB0nqRbck3ovwkohDBJ0tmSbpCkGONAjLFNzL18\nkpVUGkLISiqT1CDm3hFjoidbsyXtOeznvbkYJrgQwjyNfontM5JqY4wNuV81SrJbrmOi+K6kL2h0\ney1pdMP4thjjUO5n5uDENF/SQUk/yn0EfH0IoVzMvbwQY9wn6duSdms0yWqXtEbMvSPGRE+2kIdC\nCBWSbpX0mRhjx+G/i6Plr5TATkAhhEslHYgxrhnvc8EblpV0kqR/iTGeKKlbv/ORIXNv4sqtpbtc\no0nzLEnlki4e15PC/6iJnmztk1R/2M91uRgmqBBCoUYTrZ/EGG/LhZtCCDNzv58p6cB4nR9e1ZmS\nLgsh7NToR/bnaXQd0OTcRxsSc3Ci2itpb4zxmdzPt2g0+WLu5YcLJO2IMR6MMQ5Kuk2j85G5d4SY\n6MnWKkmLchUZRRpdMHjHOJ8TEuTW99wgaVOM8R8O+9Udkj6c+/eHJf1qrM8Nry3GeG2MsS7GOE+j\nc+2hGOMHJD0s6cpcM/pvAooxNkraE0I4Ohc6X9JGMffyxW5Jp4cQynL30Vf6j7l3hJjwX2oaQrhE\no+tIXtkK6BvjfEpIEEI4S9Jjktbrv9f8/IVG1239QtIcSbskvTfG2DIuJ4nXJYRwjqTPxxgvDSEs\n0Og7XVMkrZX0wRhj/3ieH6wQwnKNFjYUSdou6SMa/YOauZcHQghfk/Q+jVZ1r5X0cY2u0WLuHQEm\nfLIFAACQzyb6x4gAAAB5jWQLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpItgAAAFJEsgUAAJAiki0A\nAIAUkWwBAACkiGQLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpItgAAAFJEsgUAAJAiki0AAIAUkWwB\nAACkiGQLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpItgAAAFJEsgUAAJAiki0AAIAUkWwBAACkiGQL\nAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpItgAAAFJEsgUAAJAiki0AAIAUkWwBAACkiGQLAAAgRSRb\nAAAAKSLZAgAASBHJFgAAQIpItgAAAFJEsgUAAJAiki0AAIAUkWwBAACkiGQLAAAgRSRbAAAAKSLZ\nAgAASBHJFgAAQIpItgAAAFJEsgUAAJAiki0AAIAUkWwBAACkiGQLAAAgRSRbAAAAKSLZAgAASBHJ\nFgAAQIpItgAAAFJEsgUAAJAiki0AAIAUkWwBAACkiGQLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpI\ntgAAAFJEsgUAAJAiki0AAIAUkWwBAACkiGQLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpItgAAAFJE\nsgUAAJAiki0AAIAUkWwBAACkiGQLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpItgAAAFJEsgUAAJAi\nki0AAIAUkWwBAACkiGQLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpItgAAAFJEsgUAAJCiMU22QggX\nhxC2hBC2hRC+NJaPDQAAMB5CjHFsHiiEjKSXJF0oaa+kVZKujjFu9NqXVhfHSTPLTbyr0caGyhIe\nc9iJjfhtY6VtnD3o56IDNf4xqkp6Tayjyz+5gkHnHIJ/3ALneUjScJntu8J2/yAjWRvL9vgXo7/a\nPu+CYv8kYsJJF++zxx6sdE5CUmbQPo+BSv+4Xp8WVw24bQfaitx4LLfnls34zy+zx57HULn/PIZK\n3bBCkX28OOQ/v2yXjQ8nHLeo3R53eIbfp0ODGf8gTv9Nqehym7Z0Vdhgwtgs7LGxwUq/bdZ/OA1V\n2nFRfMi/Xw3OcObCAX/+hlnO5JM0fMAfL56kueopGLLnNjTN76eCNr+fMi32gg5P8e8tQ/YWqbLS\nPrdtX3uJG/d4c0/y70OTy5wBIKnrgD25kckJ95Ye/1pEL1yYcD17nHvZkNtUw8Xeg/ltizr9xxss\nt483UuofpKDPDqKka5ztTbhXT3GOMegPTu+6Jb0eeveW/un+cQvb/PhglfO8E65nSaOdk4OT/Pm4\ncEajie0eqHbbDgz7Yyi02Ht4Zqr/OqJ9tm3/dL/pwI79h2KM07zf+a8a6ThV0rYY43ZJCiH8TNLl\nktxka9LMcn3gpxeY+JN/e6qJHTzJv6kWdthBUJhwYx88p93Epl3v38x2X+3PiAuP2WRiDzx5gtu2\ntMGeszvZJRW3+fGOk/pNbNadfpf2TLOPN/V5/4b48nvsq3vFAnt9JKm3r9CNL/xKt4kdeKs/Qssb\n7fXcd7Y/SYqcZHLBBTvctrvumu/Gh07tNLFpVf7AqPiUPY/m09y5pObj3LCyc+2x+5v9DKr2CdtP\nrcf6N7M5v7EvoG1ftNddkg41Vbnx0Guf3/vPetJt+9MnzjCxTKc/96Y/Z++qDWf7d9raJ/3n13iu\nHReLf+gnDY1/YV9Bp33Pv8YlX21w460/mGNiIeHFYaj49WdbpYfsuTX/sd9PJbdPduNTfvacibVc\ndpLb9tAK+0K5Yvk2t+2Gu452497zLmr3L0brSfb5XXHSGrftb79/mon1XebfW4bX+NfCexEfmemP\ni/Ln7RgoSUjY2xfaWGbA7+e6h/x7Z+Np9jWjc5n/Il6x2SYTSX94TN1g/5CXpG1X2/tvqZMcSFJ/\ntR0Xhc4fd5I09247Prf+iX/cGXf5SVHD+c7r5Ij/eEv+1s7Jxovr3La3f/nvTewzO9/ltt3V7idh\nRTdPMbGqj+112+rLtu3Ln/Lvezuu/std/kHG9mPE2ZL2HPbz3lwMAADgiDWhFsiHEK4JIawOIazu\nabPv2gAAAOSbsUy29kmqP+znulzsv8QYr4sxrogxriibnPCZGgAAQB4Zy2RrlaRFIYT5IYQiSVdJ\numMMHx8AAGDMjdkC+RjjUAjhk5LulZSRdGOMcUNS+7b+Ut358jJ7nONsfli13T9G90znPBKKsno7\nbVVO83H+4u9sob/gcUHpIROrmOcv/qx4cpKJeYvYJan7LH8xbcF+u/izY45/DK8C5+VP+G1L19v4\n0EF/oWHlGfY5S5Ky9kK3nOV/NFz7pf0mNvJufzlf9bN2yG6pmue2HZnpl9qUZmy8NOtXqB1aOcPE\nhhOK1qYs8a9F27qpJnbeeevdtlseXGpidQ/559Y31Z5I16qE0sV6f8yWNNl+unv3Erdt6HcqM6f6\n55Z1qsAKp/mLmLtmO1WOkuT0ScdCp9RO0qIauwB823GL/cP+ZJ7/eHZKJt4v2o62i6zrHvLHW9Mp\ntp9+ePx1btuPbviEG+/685NNbNCp1pSkbJdTDFPs30N6Fvj9N/l5e+9rPd4vDCqstPP6ttX2fCWp\nxrmeQ8/7C+H7pydUKWbt8y5IqFforbVtu2f7161mnT3Iwbf48+bgcr94asR5yTj6j15w2+77hV2R\nP/y4fy16p/k3nfLpdkV9ySq/GKbfrvPWjKf90swY7LUoX+9XrhZ1+GNo+mP2Xn1whX/tu5fa++xQ\nqd+pX9hzmYmt3+u/Xpw4Z48b3zDP3pOb9vkFXDPm2mtf/uwbf59qLKsRFWO8W9LdY/mYAAAA42lC\nLZAHAAA40pBsAQAApIhkCwAAIEVjumbrjSgpHNSi6QdN/NAtdoFs71Q/Z+yrdxbuTfYX6V4y7yUT\ne2KVv8jzxHn+N5b/6zNvtcEh/9y8JcGJ3yBf4i9A7O+3izTLG/xFupkBuzCx9h/9xZF7LrSxijNs\nX0hSeZG/gPTAGbPsMZwFqJLUevY8Ezt+fsK3wtceZWJh2F90WZzw7ebdJXYMveRsIyJJ05xD98z0\nn8dAi7/Q+4ILnjexJ/YucNtOclb6Fgz4fTpcZNsOLPK/aTpT4F+jkaxd/Nm2y1+kG8uc80gY3yWH\n7DybP63Zbbu9xr/2RfuchakNfpHFprvtYvjJDf4C63O/7H9D/rN/Yr+Rfck/+TU8v1q33MQOnuAv\nYu6vsdftr3dc7rZN2gao/gG7ELpxpT/eOhbZef3QjkVu26lP+i8BXfU2Vnd/wvxdZM9jcKF/z2p3\nahbibH/MFu7wF2QPO9vfFLT5beM8e+zSdf7i9pbjbT9Ne9wvkmpZ6s+n0ibnGp3gF2oUZe29s3WO\nP9d7p/vzbKqzDVPjGX6RzJRn7XMpTFjc3nSqnZODp9idNyRp1zH+C1f1ahvLdvtjqGeaHYezHmhx\n2z47317Po5btc1pKq7b4u4gUO6f88eVPuG1/vvZ8E+te4Y/ZV8M7WwAAACki2QIAAEgRyRYAAECK\nSLYAAABSRLIFAACQoglbjTjQXKx9P7GVBJMP2gqOhov8CofyrbY6qHe6v//G4FybdyZVB84pbXXj\n5yzbYmJP7PSrIdoW2gqe0ia/wqVjk7OPiKTsYlsdUvyYX4nScIazdcJyv5pp+KgeE+vs8at9ml+q\nceNH//plE9vzIbs9hSR1z7TXft3LTjmUpLDIVutcuHKd23bv+/ztFzr/1T7eoU6/Im6otNLEyvf7\n/XTUuY1u/P5Vx5tY7QJ/a5/BUrvVxvYr/IFYts/ZEqfIrzAtesY+D0kqPWCfS+ux/t9gRe12ns18\nwo4VScq02vjBHv8aJ1UolRx0zm2Rfy28ir/Gaf5xnzrkz8neejt3Vn1zhds2XOpU4SZUElZvsL+4\n/32/dtue+W//y43veKe9dpXb/XG4YLEdh40P1rltp2yyVY6S1DfV3hs65vr3zp56W/V5+Ulr3bYP\n/8epJla81D+HtoV+NelR37WxnW/3x9Zgm30NKPEvm4Yr7eN11fvViPUP+FXYjafbxxuq9KtUp5Xb\n15GOfv9+WrHLH1zHn2u3Omvc5R9j5tU7Taz/r+w2OZI0+64GEzt4oX8tZl3nP7+9tohPg7UJ26JN\nsmNrpChhi7i5bSb253N/47b9X3t/341nnC8laB70x9CMS3ebWOMdc9y2r4Z3tgAAAFJEsgUAAJAi\nki0AAIAUkWwBAACkaMIukB8ulVqX2UWvFfvtQrpMq58zDlbYlZCVO/y2D46cYM9hsb/Y+D/vOsuN\njzjrB0sOJCzed7bVKWvyFw+2npOwXUuvfcDhUv/5VW13zq3NP26Tswh9cNBfHPunF/kLE+//1jH2\n3Pw19pr193b7lMz/Wum27Vhk+/Th7f5WJOWX+AUAXeucrYuOb3LbRmcLnoEKv08P3jDPjf/0a983\nsX9vPtNt++jsWhObvMltqj5nHexgvz+l+4/xx9bZV71gYndvWuq2nXua3bIpc59d0C9JsdCeR+sW\nfxugujX+ue26zFmQ/4g/vqMzPGc/6o/vjsX+QCztt+3bFvjXs2SrPcaUTf6C7oPL7Tn/oM0vAOmv\n8sdWttPGB/3hrd0HppjY8NH+Nkfbav2ta2Y86d2f/GN0LLMX/1fPnei2DQvsNarM+Ncts9F/gjsv\ntfN3sN4/t2yTLagoOMMvcKp4wi7ITiqS6pnuLxYPzktG3xS/7a8X/aeJ/X7Bu922e/b7RR33P22L\nb8oa/Hv1pp65JnbU8Ovfdqb5ZTuuJKn3ZH9OTtlgx1DX0Qnbia23RWBd9X4lw3XH32xiewb9ooCY\nsJ1YgdNPv23wC7iG77LH7jzNH2+vhne2AAAAUkSyBQAAkCKSLQAAgBSRbAEAAKRoTBfIhxB2SuqU\nNCxpKMbofz0zAADAEWI8qhHPjTH6e5UcbkTK9Ng33koOdZtYNmEbkBFnF4GOhX6FUqz0ykj8N/4y\nfX7F0LFn2S1qdt56lNu2c449dtNb/CqSq5etcuO33mkr2loX++dWscc+75GE3h/sdEpwhvzj/mir\nXzU47Wi79UmBX3Sm4XNOMrHWk/1K0Pn/aStUOub4/d85z69mKW6xz2VeVYvbdnf/VBPrOsa/cFln\nCwhJ+j+7LzWxDZv9arRjr7NbPnWd6Y+hgcn2PEKjX2l307v/xY1/5Fd2e5hyZxsgSTqwxp5z0XL/\nGk97zG4ZExO2s2k6xa/WynTaYw/4Xa0w3V78jjn+1lVn1NrtNyTpnnfYysrChC20vK2y9s33r/3C\nG23l0veH3uG2HT7f3t8kqfQZ+8SrX0qo4lxsb3yz5/m33M7JfrldQ7W9dov/yX+80h227aQz/ere\n5nV2C639W/xttUK1f6/O9NuBVHDAfx7BKXQcfsrfBqb7KOeeU+xXSk590b9XT3rY9nXfNP/cLvnC\nn5lY44X+fS9T51+Lq97ylIn98la/Wv7Kc582sXv2neG2rX3W3gNK6+yYl6TMdn87ubZF9hgD+/0K\n08xMO8+mrvXn3hc2X2livQP+PSRb6l/PMGznSAj+45U223hbQttXw8eIAAAAKRrrZCtKui+EsCaE\ncM0YPzYAAMCYG+uPEc+KMe4LIUyXdH8IYXOM8bev/DKXgF0jSdnJ/lu9AAAA+WRM39mKMe7L/f8B\nSbdLOvV3fn9djHFFjHFFQXnC4gwAAIA8MmbJVgihPIRQ+cq/JV0k6cWxenwAAIDxMJYfI9ZKuj2E\n8Mrj/jTG6G+sJ0mZqKHJthJk56X2Ha/BugH/EE224qCswc8v+/tsNUNhh18+NbC0x42fOHmPiW2e\nusBt6x27arPfHTdnTnPjJ5+z1cRe+uVit23TW2w1y8yHE6ot222lzYqVL7ltn30+YV/CGU7F0On+\nnmThCVvFVdToX4vBcvs8ms/wq6Qmr3bKUSV1v8VWfLX0+3vE7T3PqYi1WwSOHneOX7k0NGKPcdyx\nfkXcwPxZJrb/LX4/le+1seFy/xyufekKN17aaI9deHaz2zbcZfdGq3j/frdtR+cMExuZ5FcGzbrT\nr+zZ8SEbm35jh9u25S22krB3mn/c/mF/bE19wt4Demr9e0DmKft4R63xy1Fbj7XVejf9wffctt9v\nPN+NH/xmnYlt+4I/vtVnn9+CSX414toHl7nxSU5RZO9M/7plnKddkFCtNejc00v2+8ftm+/vP7fo\n+/YBt11d6batv9/eG/Zc498vKlbbSrnBKr/qsLDDf83Ze559farZ4M/Jr/3vG0zs0//xh27b/qn+\nMU6tsJve3rLU35fy7ptt5WHpoYQ54uznmH3Mr+6dvsZ/Pcx83d4o99xv92eUpKqdzj6Kdf59r9rZ\nS/OyBevdtjfee54b7znZ7tF4fLV/Y9/db5c0ZYv9e9mrGbNkK8a4XZLd7RkAAOAIxlc/AAAApIhk\nCwAAIEUkWwAAACkaj+16XpeCgaCyXfb0eubYhWlFexMWijpG/G/1V/Eiu/C2t8c/7kinf5BKZ6Vo\nSNjmpr/G2Z5gnb9YcajUf7x9dXabhCmb/MWf2R57LQuG/C0gqha3mdiaJ45225Yt9BcsV+yx59z1\nW/+70wqbvcW7/tYnfdX274OCIn+xane9fz0n32sXw+99p7/lxOxH7TEaT0vYBuigv5h2ONpzXr/Z\nLoSXpLl2dyDNu8t/fvve4lyjQv/cDrX722QUOUOgrMgfQ/uX2sYD985229ZttcUQ2VMmu22Lm/yt\nkmoetQvyW5fX+MfYbq9x5W7/Wjxa6i8Kr3aa9x/nL/6tfMKOoV2X+NuyDJfZ6/bRtX/gtp36Q79Q\no/lCO59qb/UXTR84xV6LQ/MT+r8jYdsRJ1zY6S8KHiqz95YDLbaAQJJq59i+nrTILyzY++AcN960\n0l7n4Wp/juy4wrnv7fPvp+XnHTCx9mf9rYQy/f61l3O7L+z2237i2Q/Y/7zU7495v/aP8dmyq0ys\nZLf/utXvDPAie6uXJHXX2nvZez76kNv2yXv8+bRpm73HfeB9j7lt7/5Xu8VQibNNjiQNDNtze2vF\nJrftjbP87Yim3G/vnV0f8edv20I7hoYO+cUCr4Z3tgAAAFJEsgUAAJAiki0AAIAUkWwBAACkiGQL\nAAAgRRO2GnGkUOqdaSswss5WMgMz/Oqp+l87lWsDfoVDfN5us9Ax1788BYP+MR5YfIw9t2q/4q/k\noLMNzCG/oqbhbL/KrWenrcw6Zl+n2zbjbEdTsnGf2zZm5plY6zkJFTVHP+HG73vJbuPT9X5/6yIN\n22s0OMm/bn019lqM9PvXp2az/3AHz3IqWl/0K+XaFtjyorn3+NVTI0X+3y6nvHOXiVUe6x9j61Z7\n3TIDfvVU3cN2y4mhr/iVfeWF/thqbLXjfv8WvwJr8mZ7LdoX+/00XGYrooam+fN02+/bqkNJGi6y\n86xit3+N+2bYPq3a4Y+L+9/zbTd+xY4vmFjZGr86cMa/2+1B2i9Z6radvN5WZm76pD/esk7lkySV\nN9jr3LrYf36Tltnq3pHoV0UPlvvx3hn22ocRv1prpNi2nfSwX63VvNDGJ62w25yNPp4b9u+//f64\niBnbtnhel9u291477qub/JNImuuDFfbxSjc2uG0XzXQqnR/zt7Ppq/bHxXtPetrEbi3xt+sJ+2wF\nXtI19qoqb3zer+zLfs2vUs3utOd8+8vHu20LKp3t3Vr819kV0+x4OTjsV7+GvQlV7VPs463b4le/\nljq3gIKE6tdXwztbAAAAKSLZAgAASBHJFgAAQIpItgAAAFI0YRfIhxEp221zwUlbbduBY7rdY/Rd\nYxfudT8+zW9ba1cKVsyzC1slqa3VXzQ72G63fFn4c3+7j73n2u0zGv7IX3R3xYINbvyhG043MW9h\nsiQdOMkubq172l9Mf+jd9pwri/3FzZdXvOjGb3nr75nY1Hn+4u3BGZUmlrTNUdbp6upp/vPoeoe/\nsDx022uU6ffb1q62C9m76hK2h0rY+eTOXXbh9MA6f+uiAmcN8qzH/QW9jafbMVSftBD+J/PcePMp\ndo4sXNjots0cZ+dIyb/Vu20L+uxxsyV+n9bf51+41qPtdZ555263bcOl9jyaT/CPe9ETn3Tjczb1\nm1j7fL+vt18/38Sm/NJtqh3vtXswffW8W9y2f7/nSjc+VG7vhfP+wxZeSFLbW+0gumrWKv/xiv0F\n2dNX22uX7UkoWplqX0b6/F2VtOjfm01s9yJ/q6xwWsJeMvfY9jPm2+NK0sCQLSJo2eMXJ5Q4p1G5\n1x9DnXV+sUCBc5scmuUXgDS02znS508nheGEbd9G7LUv3OoXJ5zye/ZevW6fv9XO7Afs9ax4j50f\nktT185lufGCSPeeSTf5C9kMn2wKsrmP8oqx7HrcFAB2n+/0xVOUfo6fAjovCZj8dmvW4fQ14eZ5f\nnPJqeGcLAAAgRSRbAAAAKSLZAgAASBHJFgAAQIpItgAAAFIUYkwooRpnIYSDkl4pt5kqye5BgXxB\n/+Uv+i6/0X/5jf7LL3NjjO5XHkzYZOtwIYTVMcYV430e+L9D/+Uv+i6/0X/5jf47cvAxIgAAQIpI\ntgAAAFKUL8nWdeN9AnhT6L/8Rd/lN/ovv9F/R4i8WLMFAACQr/LlnS0AAIC8NOGTrRDCxSGELSGE\nbSGEL433+SBZCKE+hPBwCGFjCGFDCOHTufiUEML9IYStuf/3d2HGhBBCyIQQ1oYQ7sz9PD+E8Exu\nDv48hJCwEzfGUwhhu7ySOgAAAy1JREFUcgjhlhDC5hDCphDCSuZe/gghfDZ333wxhHBzCKGEuXfk\nmNDJVgghI+kHkt4maYmkq0MIS8b3rPAqhiR9Lsa4RNLpkj6R668vSXowxrhI0oO5nzFxfVrSpsN+\n/qak78QYF0pqlfSxcTkrvJbvSfpNjPEYSSdotA+Ze3kghDBb0qckrYgxLpOUkXSVmHtHjAmdbEk6\nVdK2GOP2GOOApJ9JunyczwkJYowNMcbncv/u1OjNfrZG++ymXLObJL1zfM4QryWEUCfp7ZKuz/0c\nJJ0n6ZZcE/pvAgohTJJ0tqQbJCnGOBBjbBNzL59kJZWGELKSyiQ1iLl3xJjoydZsSXsO+3lvLoYJ\nLoQwT9KJkp6RVBtjbMj9qlFS7TidFl7bdyV9QdJI7ucaSW0xxqHcz8zBiWm+pIOSfpT7CPj6EEK5\nmHt5Ica4T9K3Je3WaJLVLmmNmHtHjImebCEPhRAqJN0q6TMxxo7DfxdHy18pgZ2AQgiXSjoQY1wz\n3ueCNywr6SRJ/xJjPFFSt37nI0Pm3sSVW0t3uUaT5lmSyiVdPK4nhf9REz3Z2iep/rCf63IxTFAh\nhEKNJlo/iTHelgs3hRBm5n4/U9KB8To/vKozJV0WQtip0Y/sz9PoOqDJuY82JObgRLVX0t4Y4zO5\nn2/RaPLF3MsPF0jaEWM8GGMclHSbRucjc+8IMdGTrVWSFuUqMoo0umDwjnE+JyTIre+5QdKmGOM/\nHParOyR9OPfvD0v61VifG15bjPHaGGNdjHGeRufaQzHGD0h6WNKVuWb03wQUY2yUtCeEcHQudL6k\njWLu5Yvdkk4PIZTl7qOv9B9z7wgx4b/UNIRwiUbXkWQk3Rhj/MY4nxIShBDOkvSYpPX67zU/f6HR\ndVu/kDRH0i5J740xtozLSeJ1CSGcI+nzMcZLQwgLNPpO1xRJayV9MMbYP57nByuEsFyjhQ1FkrZL\n+ohG/6Bm7uWBEMLXJL1Po1XdayV9XKNrtJh7R4AJn2wBAADks4n+MSIAAEBeI9kCAABIEckWAABA\niki2AAAAUkSyBQAAkCKSLQAAgBSRbAEAAKSIZAsAACBF/x8jPgtRA9SvZwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x504 with 3 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YekKO0dMTL-5",
        "colab_type": "text"
      },
      "source": [
        "## Пространства переменных"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IsUwgMenKe7N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def linear_transform(vec, shape):\n",
        "  w = tf.Variable(\n",
        "      tf.random_normal(shape, mean=0.0, stddev=1.0),\n",
        "      name='matrix'\n",
        "  )\n",
        "\n",
        "  # matrix multiplication\n",
        "  return tf.matmul(vec, w)\n",
        "\n",
        "\n",
        "# two runs will create two transform matrices\n",
        "# result1 = linear_transform(vec1, shape)\n",
        "# result2 = linear_transform(vec2, shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0I5sKex-KkC9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def linear_transform_reproducible(vec, shape):\n",
        "  with tf.variable_scope('transform'):\n",
        "    w = tf.get_variable(\n",
        "        'matrix', shape,\n",
        "        initializer=tf.random_normal_initializer()\n",
        "    )\n",
        "    return tf.matmul(vec, w)\n",
        "\n",
        "vec1 = tf.Variable(tf.random_normal([1, 5]), name='vec1')\n",
        "vec2 = tf.Variable(tf.random_normal([1, 5]), name='vec2')\n",
        "shape = (5, 3)\n",
        "\n",
        "with tf.variable_scope('transform') as scope:    # ?!\n",
        "  result1 = linear_transform(vec1, shape)\n",
        "  scope.reuse_variables()\n",
        "  result2 = linear_transform(vec2, shape)\n",
        "\n",
        "init = tf.global_variables_initializer()\n",
        "sess = tf.Session()\n",
        "sess.run(init)\n",
        "\n",
        "array_1 = result1.eval(session=sess)\n",
        "array_2 = result2.eval(session=sess)\n",
        "array_3 = result2.eval(session=sess)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5T19UXwIUrRg",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "6fe60515-b83e-49bb-cde7-b7029d003a83"
      },
      "source": [
        "array_1"
      ],
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.7866912 , -0.34893978, -1.4939145 ]], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 81
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MxemP2lsU2WZ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ca7593cb-6f6e-437f-fb27-329942989a1f"
      },
      "source": [
        "array_2"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-1.0306478, -2.7686694,  1.8277066]], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 82
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SmtxVU4iXDHV",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "49069468-27d3-45af-cbae-696a8704c456"
      },
      "source": [
        "array_3"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-1.0306478, -2.7686694,  1.8277066]], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GSyMDggxV3mU",
        "colab_type": "text"
      },
      "source": [
        "Эм, обещали ошибку, а её нет о_О\n",
        "\n",
        "И результат вроде бы воспроизводится"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uO_18ePXV7hS",
        "colab_type": "text"
      },
      "source": [
        "# Линейная регрессия\n",
        "\n",
        "Приближаем функцию $f(x) = kx + b$ для $k = 2$ и $b = 1$. \n",
        "\n",
        "$k$ и $b$ -- искомые параметры, которые мы хотим обучить"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fis3R6LQVCsS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "\n",
        "n_samples, batch_size, num_steps = 1000, 100, 20000\n",
        "\n",
        "X_data = np.random.uniform(1, 10, (n_samples, 1))\n",
        "y_data = 2 * X_data + 1 + np.random.normal(0, 2, (n_samples, 1))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "12JlyGDxWxLj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X = tf.placeholder(tf.float32, shape=(batch_size, 1))\n",
        "y = tf.placeholder(tf.float32, shape=(batch_size, 1))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZTM2spn0W4E3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "with tf.variable_scope('linear-regression'):\n",
        "  k = tf.Variable(tf.random_normal((1, 1)), name='slope')\n",
        "  b = tf.Variable(tf.zeros((1,)), name='bias')\n",
        "\n",
        "y_pred = tf.matmul(X, k) + b\n",
        "loss = tf.reduce_sum((y - y_pred) ** 2)\n",
        "optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.0001).minimize(loss)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-oXxAPe-Xd-s",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "665f7a38-8b45-4594-f759-b26dfe4db8d0"
      },
      "source": [
        "display_step = 100\n",
        "with tf.Session() as sess:\n",
        "  sess.run(tf.global_variables_initializer())\n",
        "  for i in range(num_steps):\n",
        "    # form mini-batch\n",
        "    indices = np.random.choice(n_samples, batch_size)\n",
        "    X_batch, y_batch = X_data[indices], y_data[indices]\n",
        "\n",
        "    # run\n",
        "    _, loss_val, k_val, b_val = sess.run(\n",
        "        [optimizer, loss, k, b],\n",
        "        feed_dict={X: X_batch, y: y_batch}\n",
        "    )\n",
        "\n",
        "    # print results\n",
        "    if (i + 1) % display_step == 0:\n",
        "      print('Epoch {}: loss = {:.8f}, k = {:.4f}, b = {:.4f}'.format(\n",
        "          i + 1, loss_val, k_val[0][0], b_val[0]\n",
        "      ))"
      ],
      "execution_count": 95,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 100: loss = 421.36270142, k = 2.0937, b = 0.5002\n",
            "Epoch 200: loss = 334.00466919, k = 2.0332, b = 0.6174\n",
            "Epoch 300: loss = 335.55249023, k = 2.0309, b = 0.7152\n",
            "Epoch 400: loss = 407.35946655, k = 2.0200, b = 0.7785\n",
            "Epoch 500: loss = 381.70211792, k = 2.0170, b = 0.8198\n",
            "Epoch 600: loss = 324.88937378, k = 2.0377, b = 0.8543\n",
            "Epoch 700: loss = 300.89495850, k = 1.9952, b = 0.8962\n",
            "Epoch 800: loss = 431.33248901, k = 2.0169, b = 0.9530\n",
            "Epoch 900: loss = 353.72406006, k = 2.0017, b = 0.9468\n",
            "Epoch 1000: loss = 328.98483276, k = 2.0278, b = 0.9553\n",
            "Epoch 1100: loss = 283.53033447, k = 2.0240, b = 0.9385\n",
            "Epoch 1200: loss = 321.68170166, k = 1.9751, b = 0.9298\n",
            "Epoch 1300: loss = 413.78790283, k = 1.9903, b = 0.9397\n",
            "Epoch 1400: loss = 372.58465576, k = 1.9940, b = 0.9344\n",
            "Epoch 1500: loss = 403.03509521, k = 1.9707, b = 0.9243\n",
            "Epoch 1600: loss = 454.09646606, k = 2.0246, b = 0.9217\n",
            "Epoch 1700: loss = 438.88409424, k = 1.9672, b = 0.9154\n",
            "Epoch 1800: loss = 310.84017944, k = 2.0416, b = 0.8970\n",
            "Epoch 1900: loss = 448.66156006, k = 2.0203, b = 0.9063\n",
            "Epoch 2000: loss = 353.74139404, k = 2.0167, b = 0.9041\n",
            "Epoch 2100: loss = 385.81903076, k = 2.0101, b = 0.9003\n",
            "Epoch 2200: loss = 442.23800659, k = 2.0017, b = 0.9105\n",
            "Epoch 2300: loss = 361.73297119, k = 2.0010, b = 0.9117\n",
            "Epoch 2400: loss = 356.70068359, k = 1.9966, b = 0.9100\n",
            "Epoch 2500: loss = 426.63449097, k = 2.0089, b = 0.9131\n",
            "Epoch 2600: loss = 348.88980103, k = 1.9640, b = 0.9041\n",
            "Epoch 2700: loss = 498.09469604, k = 1.9775, b = 0.9143\n",
            "Epoch 2800: loss = 305.79165649, k = 1.9980, b = 0.9131\n",
            "Epoch 2900: loss = 322.94732666, k = 2.0354, b = 0.8838\n",
            "Epoch 3000: loss = 416.35479736, k = 2.0238, b = 0.8975\n",
            "Epoch 3100: loss = 471.52197266, k = 1.9743, b = 0.9022\n",
            "Epoch 3200: loss = 436.25213623, k = 1.9912, b = 0.8980\n",
            "Epoch 3300: loss = 443.87435913, k = 1.9669, b = 0.9065\n",
            "Epoch 3400: loss = 369.05862427, k = 2.0124, b = 0.9259\n",
            "Epoch 3500: loss = 390.39602661, k = 1.9784, b = 0.9443\n",
            "Epoch 3600: loss = 358.46820068, k = 1.9643, b = 0.9490\n",
            "Epoch 3700: loss = 282.59628296, k = 1.9852, b = 0.9422\n",
            "Epoch 3800: loss = 351.91638184, k = 1.9764, b = 0.9315\n",
            "Epoch 3900: loss = 294.86303711, k = 2.0266, b = 0.9373\n",
            "Epoch 4000: loss = 403.30914307, k = 1.9901, b = 0.9268\n",
            "Epoch 4100: loss = 350.11492920, k = 2.0327, b = 0.9439\n",
            "Epoch 4200: loss = 385.39151001, k = 2.0071, b = 0.9189\n",
            "Epoch 4300: loss = 458.52407837, k = 1.9928, b = 0.9424\n",
            "Epoch 4400: loss = 344.14367676, k = 1.9767, b = 0.9350\n",
            "Epoch 4500: loss = 316.76123047, k = 2.0143, b = 0.9405\n",
            "Epoch 4600: loss = 389.98742676, k = 2.0123, b = 0.9333\n",
            "Epoch 4700: loss = 276.00778198, k = 2.0312, b = 0.9222\n",
            "Epoch 4800: loss = 291.90158081, k = 2.0161, b = 0.9046\n",
            "Epoch 4900: loss = 339.73510742, k = 1.9591, b = 0.8876\n",
            "Epoch 5000: loss = 403.86401367, k = 1.9953, b = 0.9107\n",
            "Epoch 5100: loss = 435.04757690, k = 2.0155, b = 0.9171\n",
            "Epoch 5200: loss = 369.13616943, k = 2.0205, b = 0.9146\n",
            "Epoch 5300: loss = 349.19808960, k = 2.0267, b = 0.9170\n",
            "Epoch 5400: loss = 415.51025391, k = 1.9845, b = 0.9130\n",
            "Epoch 5500: loss = 313.87976074, k = 1.9528, b = 0.9217\n",
            "Epoch 5600: loss = 453.85427856, k = 2.0226, b = 0.9367\n",
            "Epoch 5700: loss = 466.51333618, k = 1.9973, b = 0.9422\n",
            "Epoch 5800: loss = 341.27957153, k = 2.0076, b = 0.9331\n",
            "Epoch 5900: loss = 410.19961548, k = 2.0474, b = 0.9383\n",
            "Epoch 6000: loss = 385.19769287, k = 2.0107, b = 0.9295\n",
            "Epoch 6100: loss = 398.58908081, k = 2.0178, b = 0.9364\n",
            "Epoch 6200: loss = 389.62881470, k = 1.9973, b = 0.9393\n",
            "Epoch 6300: loss = 355.47680664, k = 2.0080, b = 0.9250\n",
            "Epoch 6400: loss = 423.92660522, k = 2.0171, b = 0.9430\n",
            "Epoch 6500: loss = 399.15081787, k = 1.9761, b = 0.9519\n",
            "Epoch 6600: loss = 366.48220825, k = 1.9700, b = 0.9440\n",
            "Epoch 6700: loss = 368.58377075, k = 2.0118, b = 0.9620\n",
            "Epoch 6800: loss = 340.10485840, k = 1.9936, b = 0.9462\n",
            "Epoch 6900: loss = 333.63803101, k = 2.0032, b = 0.9156\n",
            "Epoch 7000: loss = 304.10409546, k = 1.9920, b = 0.9144\n",
            "Epoch 7100: loss = 382.20071411, k = 1.9891, b = 0.9108\n",
            "Epoch 7200: loss = 423.35714722, k = 1.9919, b = 0.9060\n",
            "Epoch 7300: loss = 337.89718628, k = 2.0078, b = 0.9244\n",
            "Epoch 7400: loss = 381.78942871, k = 2.0103, b = 0.9466\n",
            "Epoch 7500: loss = 424.22360229, k = 2.0200, b = 0.9449\n",
            "Epoch 7600: loss = 320.51309204, k = 1.9859, b = 0.9277\n",
            "Epoch 7700: loss = 363.30139160, k = 2.0157, b = 0.9360\n",
            "Epoch 7800: loss = 368.18371582, k = 1.9938, b = 0.9185\n",
            "Epoch 7900: loss = 452.16162109, k = 1.9609, b = 0.9213\n",
            "Epoch 8000: loss = 414.89688110, k = 1.9534, b = 0.9384\n",
            "Epoch 8100: loss = 497.41583252, k = 2.0279, b = 0.9433\n",
            "Epoch 8200: loss = 354.87701416, k = 1.9528, b = 0.9306\n",
            "Epoch 8300: loss = 500.98516846, k = 1.9957, b = 0.9465\n",
            "Epoch 8400: loss = 351.56494141, k = 2.0023, b = 0.9249\n",
            "Epoch 8500: loss = 341.91247559, k = 1.9936, b = 0.9571\n",
            "Epoch 8600: loss = 430.64813232, k = 1.9816, b = 0.9311\n",
            "Epoch 8700: loss = 314.55493164, k = 1.9890, b = 0.9347\n",
            "Epoch 8800: loss = 385.11264038, k = 2.0257, b = 0.9432\n",
            "Epoch 8900: loss = 388.58312988, k = 1.9851, b = 0.9515\n",
            "Epoch 9000: loss = 362.00094604, k = 1.9744, b = 0.9446\n",
            "Epoch 9100: loss = 329.99288940, k = 2.0218, b = 0.9620\n",
            "Epoch 9200: loss = 331.68664551, k = 1.9679, b = 0.9504\n",
            "Epoch 9300: loss = 363.29174805, k = 2.0174, b = 0.9645\n",
            "Epoch 9400: loss = 368.81237793, k = 1.9644, b = 0.9625\n",
            "Epoch 9500: loss = 417.41796875, k = 1.9940, b = 0.9390\n",
            "Epoch 9600: loss = 401.17266846, k = 1.9767, b = 0.9167\n",
            "Epoch 9700: loss = 456.07943726, k = 1.9983, b = 0.9101\n",
            "Epoch 9800: loss = 363.69039917, k = 2.0611, b = 0.9437\n",
            "Epoch 9900: loss = 363.25689697, k = 1.9715, b = 0.9306\n",
            "Epoch 10000: loss = 324.13598633, k = 1.9887, b = 0.9450\n",
            "Epoch 10100: loss = 370.51287842, k = 1.9834, b = 0.9468\n",
            "Epoch 10200: loss = 458.76864624, k = 2.0399, b = 0.9475\n",
            "Epoch 10300: loss = 442.19235229, k = 2.0052, b = 0.9424\n",
            "Epoch 10400: loss = 407.61059570, k = 1.9811, b = 0.9353\n",
            "Epoch 10500: loss = 392.55990601, k = 1.9724, b = 0.9341\n",
            "Epoch 10600: loss = 400.31607056, k = 1.9953, b = 0.9337\n",
            "Epoch 10700: loss = 404.16271973, k = 1.9907, b = 0.9421\n",
            "Epoch 10800: loss = 520.96057129, k = 1.9315, b = 0.9504\n",
            "Epoch 10900: loss = 358.03344727, k = 2.0230, b = 0.9745\n",
            "Epoch 11000: loss = 361.35296631, k = 1.9805, b = 0.9592\n",
            "Epoch 11100: loss = 348.89450073, k = 1.9985, b = 0.9574\n",
            "Epoch 11200: loss = 444.12649536, k = 2.0487, b = 0.9809\n",
            "Epoch 11300: loss = 356.13711548, k = 1.9786, b = 0.9708\n",
            "Epoch 11400: loss = 349.43008423, k = 1.9869, b = 0.9580\n",
            "Epoch 11500: loss = 359.46087646, k = 1.9885, b = 0.9377\n",
            "Epoch 11600: loss = 398.10177612, k = 1.9751, b = 0.9446\n",
            "Epoch 11700: loss = 358.26416016, k = 2.0127, b = 0.9520\n",
            "Epoch 11800: loss = 367.85000610, k = 1.9812, b = 0.8988\n",
            "Epoch 11900: loss = 286.82952881, k = 2.0040, b = 0.9091\n",
            "Epoch 12000: loss = 430.07168579, k = 1.9967, b = 0.9306\n",
            "Epoch 12100: loss = 315.86386108, k = 2.0309, b = 0.9368\n",
            "Epoch 12200: loss = 391.69464111, k = 2.0185, b = 0.9396\n",
            "Epoch 12300: loss = 451.43283081, k = 1.9496, b = 0.9491\n",
            "Epoch 12400: loss = 269.05911255, k = 1.9928, b = 0.9461\n",
            "Epoch 12500: loss = 371.08004761, k = 2.0256, b = 0.9460\n",
            "Epoch 12600: loss = 283.30966187, k = 1.9832, b = 0.9483\n",
            "Epoch 12700: loss = 366.03610229, k = 1.9949, b = 0.9492\n",
            "Epoch 12800: loss = 436.68115234, k = 2.0341, b = 0.9527\n",
            "Epoch 12900: loss = 427.29803467, k = 1.9852, b = 0.9263\n",
            "Epoch 13000: loss = 397.94454956, k = 2.0113, b = 0.9410\n",
            "Epoch 13100: loss = 450.95156860, k = 1.9850, b = 0.9415\n",
            "Epoch 13200: loss = 417.13317871, k = 1.9715, b = 0.9223\n",
            "Epoch 13300: loss = 329.34060669, k = 1.9581, b = 0.9265\n",
            "Epoch 13400: loss = 419.36154175, k = 1.9996, b = 0.9406\n",
            "Epoch 13500: loss = 454.90512085, k = 2.0578, b = 0.9488\n",
            "Epoch 13600: loss = 349.14160156, k = 1.9724, b = 0.9184\n",
            "Epoch 13700: loss = 456.03839111, k = 2.0029, b = 0.9536\n",
            "Epoch 13800: loss = 499.70858765, k = 1.9551, b = 0.9651\n",
            "Epoch 13900: loss = 395.48065186, k = 2.0444, b = 0.9588\n",
            "Epoch 14000: loss = 384.18768311, k = 2.0230, b = 0.9360\n",
            "Epoch 14100: loss = 421.98263550, k = 2.0430, b = 0.9385\n",
            "Epoch 14200: loss = 295.94207764, k = 2.0153, b = 0.9372\n",
            "Epoch 14300: loss = 442.54055786, k = 1.9753, b = 0.9328\n",
            "Epoch 14400: loss = 438.83511353, k = 2.0087, b = 0.9685\n",
            "Epoch 14500: loss = 431.67196655, k = 2.0453, b = 0.9679\n",
            "Epoch 14600: loss = 382.90472412, k = 2.0005, b = 0.9526\n",
            "Epoch 14700: loss = 314.94073486, k = 1.9819, b = 0.9206\n",
            "Epoch 14800: loss = 317.07446289, k = 1.9898, b = 0.9548\n",
            "Epoch 14900: loss = 429.39837646, k = 1.9830, b = 0.9592\n",
            "Epoch 15000: loss = 335.48046875, k = 2.0203, b = 0.9561\n",
            "Epoch 15100: loss = 387.72653198, k = 1.9878, b = 0.9522\n",
            "Epoch 15200: loss = 359.02737427, k = 2.0541, b = 0.9428\n",
            "Epoch 15300: loss = 283.37597656, k = 1.9711, b = 0.9238\n",
            "Epoch 15400: loss = 328.44741821, k = 2.0048, b = 0.9261\n",
            "Epoch 15500: loss = 414.26184082, k = 1.9503, b = 0.9369\n",
            "Epoch 15600: loss = 469.59390259, k = 1.9878, b = 0.9528\n",
            "Epoch 15700: loss = 318.19223022, k = 2.0140, b = 0.9417\n",
            "Epoch 15800: loss = 336.42718506, k = 2.0099, b = 0.9555\n",
            "Epoch 15900: loss = 324.14437866, k = 1.9545, b = 0.9614\n",
            "Epoch 16000: loss = 333.20071411, k = 2.0233, b = 0.9553\n",
            "Epoch 16100: loss = 429.01837158, k = 1.9818, b = 0.9333\n",
            "Epoch 16200: loss = 344.83312988, k = 2.0241, b = 0.9580\n",
            "Epoch 16300: loss = 357.05923462, k = 1.9746, b = 0.9374\n",
            "Epoch 16400: loss = 409.61508179, k = 2.0063, b = 0.9270\n",
            "Epoch 16500: loss = 368.33590698, k = 2.0044, b = 0.9384\n",
            "Epoch 16600: loss = 449.15942383, k = 1.9657, b = 0.9512\n",
            "Epoch 16700: loss = 341.67086792, k = 2.0247, b = 0.9411\n",
            "Epoch 16800: loss = 361.66534424, k = 2.0171, b = 0.9204\n",
            "Epoch 16900: loss = 491.99108887, k = 2.0165, b = 0.9235\n",
            "Epoch 17000: loss = 411.20321655, k = 1.9952, b = 0.9172\n",
            "Epoch 17100: loss = 395.84765625, k = 1.9995, b = 0.9260\n",
            "Epoch 17200: loss = 337.56933594, k = 2.0088, b = 0.9048\n",
            "Epoch 17300: loss = 451.79605103, k = 2.0088, b = 0.8755\n",
            "Epoch 17400: loss = 329.09552002, k = 1.9658, b = 0.8827\n",
            "Epoch 17500: loss = 389.01025391, k = 2.0017, b = 0.9041\n",
            "Epoch 17600: loss = 462.96960449, k = 1.9751, b = 0.9198\n",
            "Epoch 17700: loss = 405.66131592, k = 1.9687, b = 0.9031\n",
            "Epoch 17800: loss = 425.47918701, k = 1.9979, b = 0.9194\n",
            "Epoch 17900: loss = 440.95532227, k = 2.0199, b = 0.9249\n",
            "Epoch 18000: loss = 497.28643799, k = 1.9658, b = 0.9001\n",
            "Epoch 18100: loss = 404.41061401, k = 2.0065, b = 0.9132\n",
            "Epoch 18200: loss = 375.25418091, k = 1.9738, b = 0.9016\n",
            "Epoch 18300: loss = 412.72805786, k = 2.0044, b = 0.9173\n",
            "Epoch 18400: loss = 387.01571655, k = 1.9702, b = 0.9067\n",
            "Epoch 18500: loss = 348.42770386, k = 1.9807, b = 0.9191\n",
            "Epoch 18600: loss = 252.64794922, k = 2.0449, b = 0.9072\n",
            "Epoch 18700: loss = 469.36807251, k = 1.9894, b = 0.9076\n",
            "Epoch 18800: loss = 370.53817749, k = 1.9845, b = 0.9076\n",
            "Epoch 18900: loss = 348.46798706, k = 2.0223, b = 0.9203\n",
            "Epoch 19000: loss = 328.32055664, k = 1.9904, b = 0.9246\n",
            "Epoch 19100: loss = 341.37240601, k = 2.0091, b = 0.9218\n",
            "Epoch 19200: loss = 430.66656494, k = 1.9552, b = 0.9180\n",
            "Epoch 19300: loss = 410.99053955, k = 2.0113, b = 0.9393\n",
            "Epoch 19400: loss = 398.64019775, k = 1.9586, b = 0.9439\n",
            "Epoch 19500: loss = 421.62261963, k = 1.9654, b = 0.9353\n",
            "Epoch 19600: loss = 404.71560669, k = 1.9911, b = 0.9374\n",
            "Epoch 19700: loss = 370.93991089, k = 1.9817, b = 0.9279\n",
            "Epoch 19800: loss = 276.73043823, k = 2.0342, b = 0.9619\n",
            "Epoch 19900: loss = 318.15243530, k = 1.9969, b = 0.9566\n",
            "Epoch 20000: loss = 327.64382935, k = 2.0316, b = 0.9527\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "REL21HJkbH6q",
        "colab_type": "text"
      },
      "source": [
        "# WOW! Работает!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VdAXsn7nYnQ2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}