S4TF_Tutorial_2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "S4TF Tutorial 2",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "language_info": {
      "file_extension": ".swift",
      "mimetype": "text/x-swift",
      "name": "swift",
      "version": ""
    },
    "kernelspec": {
      "display_name": "Swift",
      "language": "swift",
      "name": "swift"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "QyCcF45zBQ3E"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors. [Licensed under the Apache License, Version 2.0](#scrollTo=y_UVSRtBBsJk)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "CPII1rGR2rF9",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\"); { display-mode: \"form\" }\n",
        "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "// you may not use this file except in compliance with the License.\n",
        "// You may obtain a copy of the License at\n",
        "//\n",
        "// https://www.apache.org/licenses/LICENSE-2.0\n",
        "//\n",
        "// Unless required by applicable law or agreed to in writing, software\n",
        "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "// See the License for the specific language governing permissions and\n",
        "// limitations under the License."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "YHI3vyhv5p85"
      },
      "source": [
        "## **The Basics: Training Your First Model**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "OVi775ZJ2bsy"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/drive/1Gp5ypYsDZ3QPzgNBCUbW3h6fMj3asz9y\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"Link to be updated\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />GitHub link to be updated accordingly</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "F8YVA_634OFk"
      },
      "source": [
        "Welcome to this Colab where you will train your first machine learning model!\n",
        "\n",
        "We'll try to keep things simple here, and only introduce basic concepts. Later Colabs will cover more advanced problems.\n",
        "\n",
        "The problem we will solve is to convert from Celsius to Fahrenheit, where the approximate formula is:\n",
        "\n",
        "$$ f = c \\times 1.8 + 32 $$\n",
        "\n",
        "\n",
        "Of course, it would be simple enough to create a conventional Swift function that directly performs this calculation, but that wouldn't be machine learning.\n",
        "\n",
        "\n",
        "Instead, we will give TensorFlow some sample Celsius values (0, 8, 15, 22, 38) and their corresponding Fahrenheit values (32, 46, 59, 72, 100).\n",
        "Then, we will train a model that figures out the above formula through the training process."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KaGgOzPQbGj3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import TensorFlow\n",
        "\n",
        "import Python\n",
        "%include \"EnableIPythonDisplay.swift\"\n",
        "IPythonDisplay.shell.enable_matplotlib(\"inline\")\n",
        "let plt = Python.import(\"matplotlib.pyplot\")\n",
        "\n",
        "let np = Python.import(\"numpy\")  // Make numpy available using np."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "wwJGmDrQ0EoB"
      },
      "source": [
        "## Set up Training Data\n",
        "\n",
        "As we saw before, supervised machine learning is all about figuring out an algorithm given a set of inputs and outputs. Since the task in this Codelab is to create a model that can give the temperature in Fahrenheit when given the degrees in Celsius, we create two lists `celsiusTemperatures` and `fahrenheitTemperatures` that we can use to train our model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q_XrcNGNbejX",
        "colab_type": "code",
        "outputId": "13bab562-3e88-4b50-e86a-a05979764ecd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "let celsiusTemperatures: [Float] = [-40, -10,  0,  8, 15, 22,  38]\n",
        "let input = Tensor<Float>(shape: [7, 1], scalars: celsiusTemperatures)\n",
        "let fahrenheitTemperatures: [Float] = [-40, 14, 32, 46, 59, 72, 100]\n",
        "let output = Tensor<Float>(shape: [7, 1], scalars: fahrenheitTemperatures)\n",
        "\n",
        "for (index, celsiusTemperature) in celsiusTemperatures.enumerated() {\n",
        "    print(\"\\(celsiusTemperature) degrees Celsius = \\(fahrenheitTemperatures[index]) degrees Fahrenheit\")\n",
        "}"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "-40.0 degrees Celsius = -40.0 degrees Fahrenheit\r\n",
            "-10.0 degrees Celsius = 14.0 degrees Fahrenheit\r\n",
            "0.0 degrees Celsius = 32.0 degrees Fahrenheit\r\n",
            "8.0 degrees Celsius = 46.0 degrees Fahrenheit\r\n",
            "15.0 degrees Celsius = 59.0 degrees Fahrenheit\r\n",
            "22.0 degrees Celsius = 72.0 degrees Fahrenheit\r\n",
            "38.0 degrees Celsius = 100.0 degrees Fahrenheit\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I1XJ5nk3tZHJ",
        "colab_type": "text"
      },
      "source": [
        "### Some machine learning terminology\n",
        "\n",
        " - **Feature** — The input(s) to our model. In this case, a single value — the degrees in Celsius.\n",
        "\n",
        " - **Labels** — The output our model predicts. In this case, a single value — the degrees in Fahrenheit.\n",
        " \n",
        " - **Example** — A pair of inputs/outputs used during training. In our case a pair of values from `celsiusTemperatures` and `fahrenheitTemperatures` at a specific index, such as `(22, 72)`.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3BEsYpvYtkwy",
        "colab_type": "text"
      },
      "source": [
        "### Why model?\n",
        "\n",
        "A *[model](https://developers.google.com/machine-learning/crash-course/glossary#model)* is a relationship between features and the label. For the Celsius to Fahrenheit conversion problem, the model defines the relationship between the Celsius temperatures and Fahrenheit temperatures. Some simple models can be described with a few lines of algebra, but complex machine learning models have a large number of parameters that are difficult to summarize.\n",
        "\n",
        "Could you determine the relationship between the two *without* using machine learning?  That is, could you use traditional programming techniques (for example, a lot of conditional statements) to create a model?  Perhaps—if you analyzed the dataset long enough to determine the relationships between temperatures. And this becomes difficult—maybe impossible—on more complicated datasets. A good machine learning approach *determines the model for you*. If you feed enough representative examples into the right machine learning model type, the program will figure out the relationships for you.\n",
        "\n",
        "### Select the model\n",
        "\n",
        "We need to select the kind of model to train. We will use simplest possible model we can, a Dense network. Since the problem is straightforward, this network will require only a single layer, with a single neuron.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EycLbiRvt7wE",
        "colab_type": "text"
      },
      "source": [
        "### Create a model using the Swift for TensorFlow Deep Learning Library\n",
        "\n",
        "The [Swift for TensorFlow Deep Learning Library](https://github.com/tensorflow/swift-apis) defines primitive layers and conventions for wiring them together, which makes it easy to build models and experiment.\n",
        "\n",
        "A model is a `struct` that conforms to [`Layer`](https://www.tensorflow.org/swift/api_docs/Protocols/Layer), which means that it defines a [`call(_:)`](https://www.tensorflow.org/swift/api_docs/Protocols/Layer#call_:) method that maps input `Tensor`s to output `Tensor`s. The `call(_:)` method often simply sequences the input through sublayers. Let's define an `CelsiusToFahrenheit` struct that sequences the input through a single [`Dense`](https://www.tensorflow.org/swift/api_docs/Structs/Dense) layer.\n",
        "\n",
        "*   `inputSize: 1` — This specifies that the input to this layer is a single value. Since this is the first (and only) layer, that input shape is the input shape of the entire model. The single value is a tensor of floating point number, representing degrees Celsius.\n",
        "\n",
        "* `outputSize: 1` — This specifies that the output of this layer is a single value. The single value is a tensor of floating point number, representing degrees Fahrenheit."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VNF9rry4biBh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "struct CelsiusToFahrenheit: Layer {\n",
        "    var neuron = Dense<Float>(inputSize: 1, outputSize: 1)\n",
        "  \n",
        "    @differentiable\n",
        "    func callAsFunction(_ input: Tensor<Float>) -> Tensor<Float> {\n",
        "        return neuron(input)\n",
        "    }\n",
        "}\n",
        "\n",
        "var model = CelsiusToFahrenheit()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zqqNLzNcwr0m",
        "colab_type": "text"
      },
      "source": [
        "### Create an optimizer\n",
        "\n",
        "An *[optimizer](https://developers.google.com/machine-learning/crash-course/glossary#optimizer)* applies the computed gradients to the model's variables to minimize the `loss` function. You can think of the loss function as a curved surface (see Figure 3) and we want to find its lowest point by walking around. The gradients point in the direction of steepest ascent—so we'll travel the opposite way and move down the hill. By iteratively calculating the loss and gradient for each batch, we'll adjust the model during training. Gradually, the model will find the best combination of weights and bias to minimize loss. And the lower the loss, the better the model's predictions.\n",
        "\n",
        "<table>\n",
        "  <tr><td>\n",
        "    <img src=\"https://cs231n.github.io/assets/nn3/opt1.gif\" width=\"70%\"\n",
        "         alt=\"Optimization algorithms visualized over time in 3D space.\">\n",
        "  </td></tr>\n",
        "  <tr><td align=\"center\">\n",
        "    <b>Figure 3.</b> Optimization algorithms visualized over time in 3D space.<br/>(Source: <a href=\"http://cs231n.github.io/neural-networks-3/\">Stanford class CS231n</a>, MIT License, Image credit: <a href=\"https://twitter.com/alecrad\">Alec Radford</a>)\n",
        "  </td></tr>\n",
        "</table>\n",
        "\n",
        "Swift for TensorFlow has many [optimization algorithms](https://github.com/tensorflow/swift-apis/tree/master/Sources/TensorFlow/Optimizers) available for training. This model uses the Adam optimizer that implements the *[Adam Algorithm](http://arxiv.org/abs/1412.6980)*. \n",
        "\n",
        "During training, the optimizer function is used to calculate adjustments to the model's internal variables. The goal is to adjust the internal variables until the model (which is really a math function) mirrors the actual equation for converting Celsius to Fahrenheit.\n",
        "\n",
        "TensorFlow uses numerical analysis to perform this tuning, and all this complexity is hidden from you so we will not go into the details here. What is useful to know about these parameters are:\n",
        "\n",
        "The loss function ([mean squared error](https://en.wikipedia.org/wiki/Mean_squared_error)) and the optimizer ([Adam](https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/)) used here are standard for simple models like this one, but many others are available. It is not important to know how these specific functions work at this point.\n",
        "\n",
        "One part of the optimizer you may need to think about when building your own models is the learning rate (`0.1` in the code above). This is the step size taken when adjusting values in the model. If the value is too small, it will take too many iterations to train the model. Too large, and accuracy may decrease or training could diverge entirely. Finding a good value often involves some trial and error, but the range is usually within 0.001 (default), and 0.1"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wcQ0ZisSvpQL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "let optimizer = Adam(for: model, learningRate: 0.1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-LDM-Xh5vqbz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "Context.local.learningPhase = .training"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "01cGnbrRvqlM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "let epochCount = 500\n",
        "var trainLossResults: [Float] = []"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8jR2xg5oxbGm",
        "colab_type": "text"
      },
      "source": [
        "### Training loop\n",
        "\n",
        "With all the pieces in place, the model is ready for training! A training loop feeds the dataset examples into the model to help it make better predictions. The following code block sets up these training steps:\n",
        "\n",
        "1. Iterate each *epoch*. An epoch is one pass through the dataset.\n",
        "2. Within an epoch, iterate over each example in the training `Dataset` grabbing its *features* (`x`) and *label* (`y`).\n",
        "3. Using the example's features, make a prediction and compare it with the label. Measure the inaccuracy of the prediction and use that to calculate the model's loss and gradients.\n",
        "4. Use Adam Algorithm to update the model's variables.\n",
        "5. Keep track of some stats for visualization.\n",
        "6. Repeat for each epoch.\n",
        "\n",
        "The `epochCount` variable is the number of times to loop over the dataset collection. Counter-intuitively, training a model longer does not guarantee a better model. `epochCount` is a *[hyperparameter](https://developers.google.com/machine-learning/glossary/#hyperparameter)* that you can tune. Choosing the right number usually requires both experience and experimentation."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iyhGF0Q_v8LU",
        "colab_type": "text"
      },
      "source": [
        "### Choose a loss function\n",
        "\n",
        "Both training and evaluation stages need to calculate the model's *[loss](https://developers.google.com/machine-learning/crash-course/glossary#loss)*. This measures how off a model's predictions are from the desired label, in other words, how bad the model is performing. We want to minimize, or optimize, this value.\n",
        "\n",
        "Our model will calculate its loss using the [`meanSquaredError(predicted:expected:)`](https://www.tensorflow.org/swift/api_docs/Functions#meansquarederrorpredicted:expected:) function which takes the model's class ouput predictions and the desired label, and returns the mean squared error across the examples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "C32YpsAevqtR",
        "colab_type": "code",
        "outputId": "082ad0d2-c44d-4cf7-ddf5-d2dd84b3bb8a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 442
        }
      },
      "source": [
        "for epoch in 1...epochCount {\n",
        "    var epochLoss: Float = 0\n",
        "    let (loss, grad) = model.valueWithGradient { model -> Tensor<Float> in\n",
        "        let pred = model(input)\n",
        "        return meanSquaredError(predicted: pred, expected: output)\n",
        "    }\n",
        "    optimizer.update(&model.allDifferentiableVariables, along: grad)\n",
        "    epochLoss = loss.scalarized()\n",
        "    trainLossResults.append(epochLoss)\n",
        "    if epoch.isMultiple(of: 20) {\n",
        "        print(\"Epoch \\(epoch): Loss: \\(loss)\")\n",
        "    }\n",
        "}"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 20: Loss: 874.9848\r\n",
            "Epoch 40: Loss: 791.3388\r\n",
            "Epoch 60: Loss: 676.0081\r\n",
            "Epoch 80: Loss: 586.5841\r\n",
            "Epoch 100: Loss: 507.94962\r\n",
            "Epoch 120: Loss: 437.5428\r\n",
            "Epoch 140: Loss: 374.84808\r\n",
            "Epoch 160: Loss: 319.3676\n",
            "Epoch 180: Loss: 270.5711\n",
            "Epoch 200: Loss: 227.91536\n",
            "Epoch 220: Loss: 190.85803\n",
            "Epoch 240: Loss: 158.86754\n",
            "Epoch 260: Loss: 131.42899\n",
            "Epoch 280: Loss: 108.049866\n",
            "Epoch 300: Loss: 88.26414\n",
            "Epoch 320: Loss: 71.63471\n",
            "Epoch 340: Loss: 57.756245\n",
            "Epoch 360: Loss: 46.256603\n",
            "Epoch 380: Loss: 36.79729\n",
            "Epoch 400: Loss: 29.07372\n",
            "Epoch 420: Loss: 22.814518\n",
            "Epoch 440: Loss: 17.780321\n",
            "Epoch 460: Loss: 13.762193\n",
            "Epoch 480: Loss: 10.579621\n",
            "Epoch 500: Loss: 8.078375\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q187ZniLxwc6",
        "colab_type": "text"
      },
      "source": [
        "### Visualize the loss function over time\n",
        "\n",
        "While it's helpful to print out the model's training progress, it's often more helpful to see this progress. We can create basic charts using Python's `matplotlib` module.\n",
        "\n",
        "Interpreting these charts takes some experience, but you really want to *see* the *loss* go down and the *accuracy* go up."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LNUOZb-qxptI",
        "colab_type": "code",
        "outputId": "17558474-66a7-410b-f388-4d9e9ecd7c50",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 540
        }
      },
      "source": [
        "plt.figure(figsize: [12, 8])\n",
        "\n",
        "plt.plot(trainLossResults)\n",
        "plt.xlabel(\"Epoch\")\n",
        "plt.ylabel(\"Loss\")\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAt0AAAHjCAYAAAD/r6OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmUXGd95//Pt/auruq9W2q19sWL\n8CLZwjbGdoTBxhAHO5CAGRbDQExYAplJJiE5J0N+meF3yEwCgeAAdjBLMIsJBgRxDMbYGNvYlrzI\nkuVF+y61Wt3qfanlmT/qdqslS6It1a3bXff9OqdO3fvcW6Vvn3vc/ujR9z7XnHMCAAAA4J9I0AUA\nAAAA1Y7QDQAAAPiM0A0AAAD4jNANAAAA+IzQDQAAAPiM0A0AAAD4jNANAAAA+IzQDQAAAPiM0A0A\nAAD4LBZ0AX5oaWlxCxcuDLoMAAAAVLknn3yyyznX+tvOq8rQvXDhQq1bty7oMgAAAFDlzGznVM6j\nvQQAAADwGaEbAAAA8BmhGwAAAPAZoRsAAADwGaEbAAAA8BmhGwAAAPAZoRsAAADwGaEbAAAA8Bmh\nGwAAAPAZoRsAAADwGaEbAAAA8BmhGwAAAPAZoRsAAADwGaEbAAAA8BmhGwAAAPAZoRsAAADwGaG7\njAZH8xoYzQddBgAAAKYZQncZXfu5h/Q/f7wx6DIAAAAwzRC6yyidiGp4rBB0GQAAAJhmCN1llE5E\nNUToBgAAwHEI3WVUw0w3AAAAToDQXUbpRExDOW6kBAAAwLEI3WVUQ3sJAAAAToDQXUa1tJcAAADg\nBAjdZZROxJjpBgAAwMsQusuIGykBAABwIoTuMkrHoxorFJUrFIMuBQAAANMIobuMahJRSaLFBAAA\nAMcgdJdROhGTJFpMAAAAcAxCdxmlJ2a6WasbAAAARxG6y4j2EgAAAJwIobuMxme6h3OEbgAAABxF\n6C6j8Z5uZroBAAAwGaG7jCZmuunpBgAAwCSE7jJK09MNAACAEyB0lxE3UgIAAOBECN1ldLSnm/YS\nAAAAHEXoLqOaODPdAAAAeDlCdxlFI6ZkLMITKQEAAHAMQneZpRNRZroBAABwDEJ3maUTMUI3AAAA\njkHoLrOaRFTDOW6kBAAAwFGE7jKrpb0EAAAAxyF0l1kNoRsAAADHIXSXWToRY/USAAAAHIPQXWY1\niagGeTgOAAAAJiF0l1k6HmWmGwAAAMcgdJcZ63QDAADgeITuMquhpxsAAADHIXSXWToR1VihqHyh\nGHQpAAAAmCYI3WWWTkQlSUM5ZrsBAABQQugus3QiJkm0mAAAAGCCb6HbzOaZ2QNmtsnMnjOzT3jj\nf2tme83sGe/15kmf+Ssz22JmL5rZGyeNX+eNbTGzT/pVczlMzHQTugEAAOCJ+fjdeUl/5px7ysyy\nkp40s/u8Y59zzv3D5JPNbLmkmyS9StIcSb8ws7O8w7dKukbSHklrzWyNc26Tj7WftpqJ0M1a3QAA\nACjxLXQ75/ZL2u9t95vZ85I6TvGRGyR91zk3Kmm7mW2RdIl3bItzbpskmdl3vXOnZegen+mmvQQA\nAADjKtLTbWYLJa2U9Lg39DEze9bM7jCzRm+sQ9LuSR/b442dbHxaGg/dg4RuAAAAeHwP3WaWkfQD\nSX/qnOuT9CVJSyStUGkm/B/L9OfcYmbrzGzdoUOHyvGVp6UmPn4jJe0lAAAAKPE1dJtZXKXAfadz\n7m5Jcs4ddM4VnHNFSbfraAvJXknzJn18rjd2svFjOOduc86tcs6tam1tLf8PM0XcSAkAAIDj+bl6\niUn6qqTnnXOfnTTePum035e00dteI+kmM0ua2SJJyyQ9IWmtpGVmtsjMEirdbLnGr7rPFKEbAAAA\nx/Nz9ZLXSnqPpA1m9ow39teS3mlmKyQ5STskfUiSnHPPmdldKt0gmZf0UedcQZLM7GOSfiYpKukO\n59xzPtZ9Rmq4kRIAAADH8XP1kocl2QkO3XOKz3xa0qdPMH7PqT43nYw/HIeZbgAAAIzjiZRlFo2Y\nkrGIhnLcSAkAAIASQrcP0oko7SUAAACYQOj2QToRo70EAAAAEwjdPqhJRHkMPAAAACYQun2QTkSZ\n6QYAAMAEQrcPauKEbgAAABxF6PYBN1ICAABgMkK3D0o3UtLTDQAAgBJCtw9qmOkGAADAJIRuH9Qm\nohrKEboBAABQQuj2QQ3rdAMAAGASQrcP0omoxvJF5QvFoEsBAADANEDo9kE6EZUkWkwAAAAgidDt\ni3QiJkncTAkAAABJhG5f1CZLM90DoywbCAAAAEK3L2q9me5BQjcAAABE6PZFbXI8dNNeAgAAAEK3\nL8bbS5jpBgAAgETo9sXETDePggcAAIAI3b7I0F4CAACASQjdPhhfp5v2EgAAAEiEbl+Mr17CkoEA\nAACQCN2+iERM6URUQ/R0AwAAQIRu36QTMQ3Q0w0AAAARun2TSUbp6QYAAIAkQrdv0okY7SUAAACQ\nROj2TSYZ40ZKAAAASCJ0+6Y2GWWdbgAAAEgidPsmnYzxREoAAABIInT7JpOIcSMlAAAAJBG6fVOb\njGmI9hIAAACI0O2b2mRUg2N5OeeCLgUAAAABI3T7pDYZU9FJwzlmuwEAAMKO0O2T2kRUkljBBAAA\nAIRuv9QmY5LEzZQAAAAgdPtlPHTzgBwAAAAQun1SmyiF7qEx2ksAAADCjtDtk9rkeE83M90AAABh\nR+j2SYb2EgAAAHgI3T5JJ8fbSwjdAAAAYUfo9kkmMT7TTU83AABA2BG6fZKmpxsAAAAeQrdP4tGI\nErGIBmkvAQAACD1Ct48yyRgz3QAAACB0+ymdiPIYeAAAABC6/cRMNwAAACRCt69qkzF6ugEAAEDo\n9lM6EWXJQAAAABC6/ZRJxjREewkAAEDoEbp9VEtPNwAAAETo9lVtIqrBMdpLAAAAwo7Q7aPxmW7n\nXNClAAAAIECEbh/VJmPKF51G88WgSwEAAECACN0+qk1EJUlDtJgAAACEGqHbR7XJmCRxMyUAAEDI\nEbp9NB66BwjdAAAAoUbo9tF46B7iqZQAAAChRuj2USZZ6unmqZQAAADhRuj2UTpBTzcAAAAI3b7K\ncCMlAAAAROj2VdpbMpDQDQAAEG6Ebh9NLBnIOt0AAAChRuj2UTIWUSxiLBkIAAAQcr6FbjObZ2YP\nmNkmM3vOzD7hjTeZ2X1mttl7b/TGzcy+YGZbzOxZM7to0nfd7J2/2cxu9qvmcjMzZVMxDYwQugEA\nAMLMz5nuvKQ/c84tl3SZpI+a2XJJn5R0v3NumaT7vX1JepOkZd7rFklfkkohXdKnJF0q6RJJnxoP\n6jNBJhVT/0gu6DIAAAAQIN9Ct3Nuv3PuKW+7X9Lzkjok3SDpG95p35B0o7d9g6RvupLHJDWYWbuk\nN0q6zznX7ZzrkXSfpOv8qrvcssm4+pnpBgAACLWK9HSb2UJJKyU9LmmWc26/d+iApFnedoek3ZM+\ntscbO9n48X/GLWa2zszWHTp0qKz1n4lsKkboBgAACDnfQ7eZZST9QNKfOuf6Jh9zzjlJrhx/jnPu\nNufcKufcqtbW1nJ8ZVlkUzH1cyMlAABAqPkaus0srlLgvtM5d7c3fNBrG5H33umN75U0b9LH53pj\nJxufEbKpOD3dAAAAIefn6iUm6auSnnfOfXbSoTWSxlcguVnSjyeNv9dbxeQySb1eG8rPJF1rZo3e\nDZTXemMzAu0lAAAAiPn43a+V9B5JG8zsGW/sryV9RtJdZvYBSTslvd07do+kN0vaImlI0vslyTnX\nbWb/S9Ja77y/c851+1h3WWVTMQ2M5uWcU+nvIQAAAAgb30K3c+5hSSdLma8/wflO0kdP8l13SLqj\nfNVVTiYZV6HoNJwrKJ3w8+84AAAAmK54IqXPsqlS0KbFBAAAILwI3T4jdAMAAIDQ7bO6VFySWMEE\nAAAgxAjdPssw0w0AABB6hG6f0V4CAAAAQrfPsl57ycAo7SUAAABhRej2WSbJTDcAAEDYEbp9Nh66\n+wjdAAAAoUXo9lk0YsokYxogdAMAAIQWobsCsqkYSwYCAACEGKG7AjLJGD3dAAAAIUboroBsKqaB\nUUI3AABAWBG6KyCbitNeAgAAEGKE7grIpGgvAQAACDNCdwXUpWIsGQgAABBihO4KyKbiPJESAAAg\nxAjdFZBNxjSSKypXKAZdCgAAAAJA6K6ATIpHwQMAAIQZobsCsqm4JPFUSgAAgJAidFdA1pvp7mPZ\nQAAAgFAidFdAlvYSAACAUCN0V0A2WWov4QE5AAAA4UToroDxmW4eBQ8AABBOhO4KoL0EAAAg3Ajd\nFXB0yUDaSwAAAMKI0F0ByVhUiVhE/bSXAAAAhBKhu0LqUjHaSwAAAEKK0F0h2VSc0A0AABBShO4K\nySRjGqCnGwAAIJQI3RWSpb0EAAAgtAjdFULoBgAACC9Cd4VkknGWDAQAAAgpQneFMNMNAAAQXoTu\nCqmriat/NK9C0QVdCgAAACqM0F0h9TVxSdIAs90AAAChQ+iukPHQ3TtMXzcAAEDYELorhNANAAAQ\nXoTuCqlLxSQRugEAAMKI0F0h9WlmugEAAMKK0F0htJcAAACEF6G7QsZDdx8PyAEAAAgdQneF1MSj\nikWMmW4AAIAQInRXiJmpviZO6AYAAAghQncFEboBAADCidBdQXU1cfURugEAAEKH0F1Bdcx0AwAA\nhBKhu4LqmekGAAAIJUJ3BdXXxJjpBgAACCFCdwXV18TVN5KXcy7oUgAAAFBBhO4KqkvFVSg6DYzm\ngy4FAAAAFUToriAeBQ8AABBOhO4KInQDAACEE6G7ghrSCUlS7xChGwAAIEwI3RXUkC7NdPcQugEA\nAEKF0F1Bjd5M95HhsYArAQAAQCURuitofKb7CDPdAAAAoULorqBUPKpUPKIjQ8x0AwAAhAmhu8Ia\nahLMdAMAAIQMobvCGtJxbqQEAAAIGUJ3hTWk4+rlRkoAAIBQIXRXWGM6wUw3AABAyBC6K6whTU83\nAABA2BC6K6whHdeRoTE554IuBQAAABXiW+g2szvMrNPMNk4a+1sz22tmz3ivN0869ldmtsXMXjSz\nN04av84b22Jmn/Sr3kppTMeVLzoNjhWCLgUAAAAV4udM99clXXeC8c8551Z4r3skycyWS7pJ0qu8\nz/yLmUXNLCrpVklvkrRc0ju9c2eshprSUyl7BrmZEgAAICx8C93OuYckdU/x9Bskfdc5N+qc2y5p\ni6RLvNcW59w259yYpO96585Y40+l7B2mrxsAACAsgujp/piZPeu1nzR6Yx2Sdk86Z483drLxlzGz\nW8xsnZmtO3TokB91l0VD2pvp5qmUAAAAoVHp0P0lSUskrZC0X9I/luuLnXO3OedWOedWtba2lutr\ny67Rm+lmBRMAAIDwiFXyD3POHRzfNrPbJf3U290rad6kU+d6YzrF+IzETDcAAED4VHSm28zaJ+3+\nvqTxlU3WSLrJzJJmtkjSMklPSForaZmZLTKzhEo3W66pZM3lNt7T3c2NlAAAAKHh20y3mX1H0mpJ\nLWa2R9KnJK02sxWSnKQdkj4kSc6558zsLkmbJOUlfdQ5V/C+52OSfiYpKukO59xzftVcCfFoRPU1\ncUI3AABAiPgWup1z7zzB8FdPcf6nJX36BOP3SLqnjKUFrqk2QegGAAAIEZ5IGQBCNwAAQLgQugPQ\nmCZ0AwAAhMmUQreZLTGzpLe92sw+bmYN/pZWvZqZ6QYAAAiVqc50/0BSwcyWSrpNpWX8vu1bVVWu\nsTahnqExOeeCLgUAAAAVMNXQXXTO5VVa5u+fnXP/Q1L7b/kMTqK5NqFcwal/NB90KQAAAKiAqYbu\nnJm9U9LNOvpAm7g/JVW/xtrSA3K6B2gxAQAACIOphu73S3qNpE8757Z7D7D5N//Kqm7N46Gbp1IC\nAACEwpTW6XbObZL0cUkys0ZJWefc3/tZWDVrYqYbAAAgVKa6esmDZlZnZk2SnpJ0u5l91t/SqlcT\nM90AAAChMtX2knrnXJ+kt0r6pnPuUklv8K+s6jYRulk2EAAAIBSmGrpjZtYu6e06eiMlTlM6EVUi\nFlEPoRsAACAUphq6/07SzyRtdc6tNbPFkjb7V1Z1MzM11ybURU83AABAKEz1RsrvS/r+pP1tkt7m\nV1Fh0JJJ6vDgaNBlAAAAoAKmeiPlXDP7oZl1eq8fmNlcv4urZi2ZhA71E7oBAADCYKrtJV+TtEbS\nHO/1E28Mp6klk1TXAKEbAAAgDKYauludc19zzuW919cltfpYV9VrySZ1eGBMxaILuhQAAAD4bKqh\n+7CZvdvMot7r3ZIO+1lYtWvJJJUvOvUO54IuBQAAAD6bauj+ryotF3hA0n5JfyDpfT7VFAotmdJa\n3bSYAAAAVL8phW7n3E7n3Fucc63OuTbn3I1i9ZIz0ppNSpIOEboBAACq3lRnuk/kv5etihBqzZRC\nN2t1AwAAVL8zCd1WtipCqGU8dLNsIAAAQNU7k9DNshtnoL4mrljE6OkGAAAIgVM+kdLM+nXicG2S\nanypKCQiEVMzD8gBAAAIhVOGbudctlKFhBEPyAEAAAiHM2kvwRlqzSa5kRIAACAECN0Bas0kaS8B\nAAAIAUJ3gGbXp3RoYFQFHgUPAABQ1QjdAWqrS6lQdDpMXzcAAEBVI3QHaJb3VMqDfYRuAACAakbo\nDtCsupQk6WDfSMCVAAAAwE+E7gCNh+4DhG4AAICqRugOUEsmoYhJnYRuAACAqkboDlAsGlFLJklP\nNwAAQJUjdAdsVl1KB/uZ6QYAAKhmhO6AzapL6kAvoRsAAKCaEboD1laXUidPpQQAAKhqhO6Aza5L\nqXtwTKP5QtClAAAAwCeE7oDNrvfW6u5lthsAAKBaEboD1tFQI0nae2Q44EoAAADgF0J3wOYQugEA\nAKoeoTtg7V57yT5CNwAAQNUidAcsFY+qJZMkdAMAAFQxQvc00NGQor0EAACgihG6p4E5DTXMdAMA\nAFQxQvc00NFQo71HhuWcC7oUAAAA+IDQPQ3MaajRSK6onqFc0KUAAADAB4TuaWB82UBaTAAAAKoT\noXsamNtYCt27u4cCrgQAAAB+IHRPA/Ob05KknYRuAACAqkTongbqUnE11Sa08zChGwAAoBoRuqeJ\n+U1p7eoeDLoMAAAA+IDQPU0saE5rRxcz3QAAANWI0D1NLGhKa3/vsMbyxaBLAQAAQJkRuqeJBc21\nKjppTw+z3QAAANWG0D1NLGAFEwAAgKpF6J4mJpYN7OJmSgAAgGpD6J4mWjNJZZIxbSd0AwAAVB1C\n9zRhZlraltHmzoGgSwEAAECZEbqnEUI3AABAdSJ0TyPL2jI61D+q3qFc0KUAAACgjAjd08iyWRlJ\n0pZD/QFXAgAAgHIidE8jy9qykqTNB2kxAQAAqCaE7mmko6FGqXiEvm4AAIAq41voNrM7zKzTzDZO\nGmsys/vMbLP33uiNm5l9wcy2mNmzZnbRpM/c7J2/2cxu9qve6SASKa1g8tJB2ksAAACqiZ8z3V+X\ndN1xY5+UdL9zbpmk+719SXqTpGXe6xZJX5JKIV3SpyRdKukSSZ8aD+rV6lXt9dq4t1fOuaBLAQAA\nQJn4Frqdcw9J6j5u+AZJ3/C2vyHpxknj33Qlj0lqMLN2SW+UdJ9zrts51yPpPr08yFeV8zrq1DOU\n077ekaBLAQAAQJlUuqd7lnNuv7d9QNIsb7tD0u5J5+3xxk42/jJmdouZrTOzdYcOHSpv1RV0Xke9\nJGnDnt6AKwEAAEC5BHYjpSv1T5Sth8I5d5tzbpVzblVra2u5vrbizm2vUzRi2riX0A0AAFAtKh26\nD3ptI/LeO73xvZLmTTpvrjd2svGqlYpHtawto437CN0AAADVotKhe42k8RVIbpb040nj7/VWMblM\nUq/XhvIzSdeaWaN3A+W13lhVO6+jXhv2cDMlAABAtfBzycDvSPqNpLPNbI+ZfUDSZyRdY2abJb3B\n25ekeyRtk7RF0u2SPiJJzrluSf9L0lrv9XfeWFW7eEGjDg+OaVvXYNClAAAAoAxifn2xc+6dJzn0\n+hOc6yR99CTfc4ekO8pY2rR36aImSdLj27q1pDUTcDUAAAA4UzyRchpa1FKr1mxSj28/HHQpAAAA\nKANC9zRkZrp0UZMe39ZNXzcAAEAVIHRPU5cubtaBvhHtODwUdCkAAAA4Q4TuaWr1WaW1xn+x6WDA\nlQAAAOBMEbqnqXlNaZ3bXqefbzoQdCkAAAA4Q4Tuaeza5bO0bmePugZGgy4FAAAAZ4DQPY1d+6pZ\nck76z43MdgMAAMxkhO5pbHl7nc5tr9Odj+1kFRMAAIAZjNA9jZmZ3nPZAr1woF9P7eoJuhwAAACc\nJkL3NHfDijnKJmP68q+2BV0KAAAAThOhe5qrTcb0x6uX6L5NB/Xo1q6gywEAAMBpIHTPAB+4YpE6\nGmr0Nz/aqP6RXNDlAAAA4BUidM8AqXhU//cPL9DOw0P66Lef1uBoPuiSAAAA8ArEgi4AU3P5khZ9\n+vfP0yfv3qDf++LD+ujqpbp0cZOSsaj6RnI6MjSmzr5RdfaPaixfVG0ypoXNaa2c36iaRDTo8gEA\nAEKN0D2DvOPV8zW/qVZ//cMN+rPvr5/SZxLRiN543mx96KrFOq+j3ucKAQAAcCJWjes/r1q1yq1b\nty7oMnzjnNNTu47oxQP9KhSLqquJq64mrrZsUm3ZlFLxiPpH8nrpYL8efPGQ7n5qj/pG8nrrRR36\nn9cvV0M6EfSPAAAAUBXM7Enn3Krfeh6hu/r1jeT0pQe36vaHtqk1m9Tt713FrDcAAEAZTDV0cyNl\nCNSl4vrL687RDz/yWpmkP/zyb/SfG/YHXRYAAEBoELpD5Py59frRx16rc9qz+vCdT+nrj2wPuiQA\nAIBQIHSHTFs2pe/80WW6dvks/e1PNulff82TLgEAAPxG6A6hVDyqW991kd503mz97/94Xrc9tDXo\nkgAAAKoaoTuk4tGI/vmdK3X9Be36/+95QXc8TKsJAACAX1inO8Ri0Yg+944VKhSd/u6nmxSPRfSe\nyxYEXRYAAEDVYaY75OLRiD5/00q94dw2/c2PNup7a3cFXRIAAEDVIXRDiVhEt77rIv3OWa365N0b\ndPdTe4IuCQAAoKoQuiFJSsai+sp7LtblS5r1599fr5+s3xd0SQAAAFWD0I0JqXhUt793lVYtbNKf\nfu8ZHqADAABQJoRuHCOdiOmO971aK+Y16GPfeVp3rd0ddEkAAAAzHqEbL5NJxvTN/3qJLl/SrL/4\nwbP64i83yzkXdFkAAAAzFqEbJ1SbjOmrN79aN66Yo3/4+Uv6s++v10iuEHRZAAAAMxLrdOOkErGI\nPvv2FVrQXKvP379ZLx7o15fffbHmNaWDLg0AAGBGYaYbpxSJmP7bNWfpqzev0q7uIb3587/WXet2\n024CAADwChC6MSWvP3eW7vn4lVo+p05/8e/P6oPfWKfd3UNBlwUAADAjELoxZfOa0vrOH12mv7l+\nuR7Z2qXXf/ZX+oefvajB0XzQpQEAAExrhG68IpGI6QNXLNIDf75abz5vtr74wBa97h8e1F3rditf\nKAZdHgAAwLRE6MZpaa+v0T/dtFJ3f+RytTfU6C/+/Vld87mHdPdTe1Qo0u8NAAAwGaEbZ+Si+Y36\n0Ucu11fec7FS8aj++13rdc1nf6UfP7OX8A0AAOAhdOOMmZne+KrZ+o8/uUJffvdFSsQi+sR3n9Eb\n/+khrVm/j/ANAABCj9CNsolETNed1657Pn6lbv0vFyli0se/87Su+6eHdO/GAywzCAAAQovQjbKL\nREy/e0G77v3EVfrnd65UwTn98bee1I3/8qge3doVdHkAAAAVR+iGbyIR0+9dOEc//9Or9PdvO1+d\nfSP6L7c/rvd89XFt2NMbdHkAAAAVY9X4T/6rVq1y69atC7oMHGckV9C//Wanbn1wi44M5fS7F7Tr\nz689W4taaoMuDQAA4LSY2ZPOuVW/9TxCNyqtbySn2x/apq8+vF1j+aLeecl8ffz1y9SaTQZdGgAA\nwCtC6CZ0T3ud/SP6/C8267trdysVi+iPrlqsP7pysWqTsaBLAwAAmBJCN6F7xth6aED/994Xde9z\nB9SSSeoTb1imm149T/EotxwAAIDpbaqhm1SDwC1pzejL77lYP/jw5VrUktbf/Gij3vi5h3Tvxv0s\nMwgAAKoCoRvTxsULGnXXh16j29+7SpGI6Y+/9ZTe+qVH9cT27qBLAwAAOCOEbkwrZqZrls/SvZ+4\nUp956/nad2RYb//Kb/TBb6zV5oP9QZcHAABwWujpxrQ2PFbQHY9s15ce3KqhsbxuXNmhP7l6GcsM\nAgCAaYEbKQndVaV7cEy3PrBF33psp3KFom5c0aGPXb1Ui1szQZcGAABCjNBN6K5Knf0juu1X2/St\nx3dqLF/UDV74XkL4BgAAASB0E7qr2qH+Ud320Fb922Ol8P17F87Rn1y9VEvbskGXBgAAQoTQTegO\nha6BUd3+0DZ98zc7NZIv6I3LZ+sjr1uiC+Y2BF0aAAAIAUI3oTtUDg+M6muP7NA3frND/SN5Xbms\nRR9evUSvWdwsMwu6PAAAUKUI3YTuUOofyenOx3fpX3+9XV0Do1o5v0EfWb1Urz+nTZEI4RsAAJQX\noZvQHWojuYK+v263vvLQNu3pGdbZs7L68Ooluv6CdsV4vDwAACgTQjehG5JyhaJ++uw+/csDW7W5\nc0Dzmmr0oauW6A8unqtUPBp0eQAAYIYjdBO6MUmx6PSL5w/q1ge3av3uI2rNJvXBKxbpXZctUCYZ\nC7o8AAAwQxG6Cd04AeecfrP1sP7lwa16eEuX6lIxvfuyBXrf5QvVVpcKujwAADDDELoJ3fgt1u8+\noi//aqt+9twBRSOmG1d06INXLtbZs1nrGwAATA2hm9CNKdp5eFB3PLxdd63bo+FcQb9zVqtuuWqx\nLl/CcoMAAODUCN2EbrxCPYNjuvPxnfr6ozvVNTCq5e11+qOrFun6C+YozoonAADgBKZ16DazHZL6\nJRUk5Z1zq8ysSdL3JC2UtEPS251zPVaaavy8pDdLGpL0PufcU6f6fkI3zsRIrqAfP7NXt/96u7Z0\nDqi9PqX3v3ahbrpkvupS8aDRtHd2AAAZlklEQVTLAwAA08hMCN2rnHNdk8b+j6Ru59xnzOyTkhqd\nc39pZm+W9Ccqhe5LJX3eOXfpqb6f0I1yKBadHnypU7c/tF2/2XZYmWRMf3DxXN18+UItaqkNujwA\nADANzMTQ/aKk1c65/WbWLulB59zZZvYVb/s7x593su8ndKPcNuzp1b8+vE33bNivXMHpdWe36n2v\nXaQrl7bwpEsAAEJsuofu7ZJ6JDlJX3HO3WZmR5xzDd5xk9TjnGsws59K+oxz7mHv2P2S/tI5t+64\n77xF0i2SNH/+/It37txZwZ8IYdHZN6I7H9+lOx/fpa6BUS1uqdXNly/U2y6ey3rfAACE0HQP3R3O\nub1m1ibpPpXaR9aMh27vnB7nXONUQ/dkzHTDb6P5gv5zwwF97ZHtWr+nV5lkTH+4aq5ufs1CLaT1\nBACA0Jhq6A5kas45t9d77zSzH0q6RNJBM2uf1F7S6Z2+V9K8SR+f640BgUnGorpxZYduXNmhp3f1\n6OuP7tC3Htuprz+6Q6vPovUEAAAcq+LroJlZrZllx7clXStpo6Q1km72TrtZ0o+97TWS3msll0nq\nPVU/N1BpK+c36vM3rdQjf3m1Pn71Mm3Y26eb73hCr/vHB/WVX21V9+BY0CUCAICAVby9xMwWS/qh\ntxuT9G3n3KfNrFnSXZLmS9qp0pKB3V5/9xclXafSkoHvP1VriUR7CYI1mi/o3o0H9K3Hdmrtjh4l\nohG9+fzZevdlC3TxgkYeuAMAQBWZ1j3dfiN0Y7p48UC/vv34Tt391F71j+Z1zuys3nXpfN24skNZ\n1vwGAGDGI3QTujGNDI7m9ZP1+/Stx3dq494+pRNR3bCiQ+++bL5eNac+6PIAAMBpInQTujENOef0\n7J5e3fn4Tq1Zv08juaJWzGvQTa+ep+svnMOygwAAzDCEbkI3prneoZzufnqPvv34Lm3uHFA6EdX1\nF7TrHa+ep4vm0/sNAMBMQOgmdGOGcM7p6d1H9L0nduunz+7T4FhBS1pr9Y5Xz9NbL5qrlkwy6BIB\nAMBJELoJ3ZiBBkfz+o9n9+t763bryZ09ikVMrz+3Te949TxdtaxVsWjFV/kEAACnQOgmdGOG29LZ\nr++t3a27n9qrw4Njml2X0tsu7tDbLpqrxa2ZoMsDAAAidBO6UTXG8kX98oWD+u7a3XropUMqOmnl\n/Aa99aK5+r0L2tWQTgRdIgAAoUXoJnSjCh3sG9GPn9mrHzy5Vy8e7FciGtHV57TprRd1aPXZbUrE\naD8BAKCSCN2EblQx55w27e/T3U/t1Y+f2auugTE1puN6y4Vz9LaL5+r8jnpWPwEAoAII3YRuhESu\nUNTDm7v0g6f26OebDmosX9TStox+f2WH3nLhHM1rSgddIgAAVYvQTehGCPUO5/SfG/br7qf26okd\n3ZJK/d9vuXCOfveCdrVlUwFXCABAdSF0E7oRcru7h/TTZ/drzfp9en5/nyImvWZJs95y4Rxd96p2\n1afjQZcIAMCMR+gmdAMTNh/s10/W79Oa9fu04/CQ4lHT75zVpt+7sF3XLJ+ldILHzwMAcDoI3YRu\n4GWcc9qwt1c/Wb9PP1m/Xwf6RlQTj+oNy2fpLRfO0ZXLWpSKR4MuEwCAGYPQTegGTqlYdFq7o1tr\n1u/TPRv2q2cop0wypqvPadObz5+t3zmrTTUJAjgAAKdC6CZ0A1OWKxT1yJYu3bvxgH6+6aC6B8dU\nE4/qdee06rrz2nX1OW3KJGlBAQDgeIRuQjdwWvKFop7Y3q3/3HhA9z53QIf6R5WIRXTVsla96bzZ\nesPyWaqv4SZMAAAkQjehGyiDQtHpqV09umfDft278YD2944oHjVdvqRlIoC3ZJJBlwkAQGAI3YRu\noKyKRaf1e47o3o0HdM/G/drdPSwz6eL5jXrD8lm6ZvksLWnNBF0mAAAVRegmdAO+cc7puX19+sXz\nB3XfpoN6bl+fJGlxS+1EAL9ofqOiER5FDwCoboRuQjdQMfuODE8E8Me2HVau4NRUm9Drzm7TNctn\n6cplLarlRkwAQBUidBO6gUD0j+T0q5cO6RebDuqXL3SqbySvRCyi1y5p1uvPnaXVZ7dqbmM66DIB\nACiLqYZupp4AlFU2Fdf1F8zR9RfMUa5Q1Nod3frFpk7d9/wBPfDiIUnSsraMVp/dqted3aZVC5uU\niEUCrhoAAH8x0w2gIpxz2npoUA++2KkHXzykJ7Z3a6xQVG0iqtcubdHqs9u0+uxWzWmoCbpUAACm\njJluANOKmWlpW0ZL2zL64JWLNTia16NbD0+E8J9vOihJOntWVqvPadXqs9q0amGj4lFmwQEAMx8z\n3QAC55zTls4BPeAF8LU7upUrOGWSMV22uEmvXdqiK5e1aElrRmasiAIAmD6Y6QYwY5iZls3Katms\nrG65aokGRvN6ZEuXHnrpkB7Z0qVfPN8pSZpdl5oI4JcvbVZbNhVw5QAATA0z3QCmvd3dQ3p4S5ce\n3tKlR7d0qWcoJ0k6Z3ZWVyxt0RXLWnTJoialE8wjAAAqiyUDCd1AVSoWnTbt79OvN3fp4S2HtHZH\nj8byRSWiEa2c36DXLGnWaxY3a8X8BiVj0aDLBQBUOUI3oRsIhZFcQWt3dOvhzV16ZGuXntvXJ+ek\nZCyii+Y36rLFzbpscRMhHADgC0I3oRsIpd7hnNZu79Zj2w7rse2HjwnhFy8YD+HNunBePSEcAHDG\nCN2EbgCSeodyemKHF8K3Hdam/ceG8FcvbNKqhY1aOb9RGR5VDwB4hQjdhG4AJ9A7lNPj2w/rsW2l\nIP7CgT4VnRQxafmcOq1aUArhr17YpFl1rI4CADg1QjehG8AU9I/k9PSuI1q3s0frdnTr6V1HNJwr\nSJLmNdUcE8KXtmYUibBOOADgKNbpBoApyKbiuuqsVl11VqskKVco6vn9fVq7oxTCH97SpR8+vVeS\nVF8T18ULGnXxgkatmNegC+bWK5uKB1k+AGCGYKYbAE7BOadd3UNat6NH63Z2a+2OHm3pHJAkmUlL\nWzNaMa9BK+Y3aMW8Bp09K6sYj64HgNCgvYTQDcAnvUM5rd9zRM/sPvrqHhyTJNXEozp/br1WzmuY\nCOPt9TUBVwwA8AvtJQDgk/r0sS0pzjnt7h7W07t79PSuUgj/2iM7NFYoSpJm1SW1Yl6Dzu+o13kd\n9Tq/o17NmWSQPwIAoMII3QBwhsxM85vTmt+c1g0rOiRJo/mCnt/fr6d39eiZ3Ue0fvcR/ey5gxOf\nmVOfmgjg582t13lz6tWaJYgDQLUidAOAD5KxaKm9ZF7DxFjvcE6b9vVp495ebdjbq417e/XzTUeD\n+Oy6o0H8/Ll1Oq+jXm1Zli0EgGpA6AaACqmvies1S5r1miXNE2P9Izk9d1wQv/+Fgxq/3aYlk9S5\n7Vktb6/TOe1ZndtepyWtGcW5WRMAZhRCNwAEKJuKTzyaftzAaF6b9vVpw95ePb+/T8/v7zumRzwe\nNS1ty06E8XPb63TO7Cx94gAwjRG6AWCaySRjumRRky5Z1DQxlisUte3QoF440KdN+/v0/P5+Pby5\nS3c/tXfinLZsshTAvTC+rC2rxa21SsWjQfwYAIBJCN0AMAPEoxGdPTurs2dnJ27WlKSugVG9sL+/\nNCN+oBTGH93apVyh1J8SMWlBc62WtWW0bFZGZ83KamlbRktaM4RxAKggQjcAzGAtmaSuWJbUFcta\nJsbG8kVt6xrQ5oMD2nywX5s7B/TSwX7d/0KnCsVjw/jStozOmpXRsrasls0ijAOAXwjdAFBlErGI\nzpldp3Nm1x0zPpYvanvXoDZ39uulSYH8gRc6lZ8Uxuc1pbW4pVaLWjJa3Fqrxa21WtKaUVs2KTML\n4kcCgBmP0A0AIZGIHW1RmWwsX9SOw4N66WApjG89NKBthwb1m22HNZIrTpxXm4hqUWutFk+E8YwX\nzmtVm+R/JwBwKvyWBICQS8QiOmtWVmfNOjaMF4tOB/pGtO3QoLZ1DXjvg3pqV49+8uy+iWUNpdIa\n44tba7WguVYLmtNa0FR6WNCC5lplCOQAQOgGAJxYJGKa01CjOQ01x/SMS9JIrqAdhwe13Qvi47Pj\nP3vugLoHx445t7k2UQrgTWnNb67Vgqa0FnhP8GzN0LICIBwI3QCAVywVj56wb1yS+kZy2nV4SLu6\nh7Tz8JB2dQ9q5+Ehrd3RozXr96k4aYY8nYhqflNa870gPrcxrbmNNeporFFHQ42yqXgFfyoA8A+h\nGwBQVnWpuM7rqNd5HfUvOzaWL2pPz5B2dg9pZ9egdnYPadfhIW3rGtSDLx3SWL54zPn1NXF1NNRM\nBPG5jemJ/bmNNaqviTNTDmBGIHQDAComEYuUbsBszUhnH3usWHTqGhzV3p5h7ekZ1t4jw9rTM6S9\nPcPa3jWoh7d0aWiscMxnahPRUhD3ZsbbG1Jqr09pdl1N6b0+xRKIAKYFQjcAYFqIRExt2ZTasimt\nnN/4suPOOR0ZynmBfEh7vHA+HtDX7ehW30j+ZZ9rTMc1u/5oCJ9Tnzpmv70+pXSC/x0C8Be/ZQAA\nM4KZqbE2ocbahM6f+/LWFUkaHM3rQN+IDvSOaH/viA70Dnvvpf1ndh952Y2eklSXiqm9vkaz61Nq\nyybVmk2qLZtUW13q6HY2pZoEs+YATg+hGwBQNWqTMS1pLT1Z82RGcgUd7Bs5JoxPhPO+Eb14oF+H\nBkYnnt45WSYZmwjlrV4Qb6tLqjWTVFtdab81m1Rjml5zAMcidAMAQiUVj3rridee9Jxi0al7aEyH\n+kfV2T+qzr4RHRoYVWffqA71l14b9/aqs7/zZX3mkhSPmpprk2qqTag5k1BzbULNmeTR7dqkmjIJ\ntdSWxtKJKCEdqHKEbgAAjhOJmFoySbVkkjq3/dTnDo7mXxbMO/tHdXhgVIcHx3R4cEzbuwbVPTh2\nwoAuSclYRC2ZySH9aEBvrE2oMZ1QYzquhnRcDemEGmriikUjPvzkAPxC6AYA4AzUJmNalIxpUcvJ\nZ87HDY8VdHhwVIcHxia9j6l7cExdA97+wJheOtCvrsGxly2hOFk2GVNDbVyN6YTqa+ITwbzee29M\nJ1SfPjreUJNQNhVTJMKMOhAEQjcAABVSk4hqbqL0EKDfxjmnwbGCugfGdGR4TEeGcuoZKr0f3R7T\nkeGceoZy2tU9pCNDOfUO5076nRGT6mriyqZiqkvFS68ab7vmRPuxo+fXxJVJENqB00XoBgBgGjIz\nZZIxZZIxzddvD+njCkWn3uGcjgyNqWcop97hMfUMHg3s/SM59Y3k1TecU99ITju6htQ3klPfcE6D\nJ2l/OVpTaYZ9ckDPpuLKJmOqTcaUScUmaq5NTt6OKps6OlZLeEcIEboBAKgi0YipqTahptrEK/5s\nvlBU/0jeC+H5iTB+/P7kc3YdHtLAaF6DY3kNjOSVP8GqLydSm4geDeapUhA/PrSnE1GlE1HVjL/H\njx+LHd2OR+lzx7RG6AYAAJKkWDQysRb66XDOaTRf1OBoXoOjBfWP5jQ4WtDAaE4DowUNjpaC+cBo\n6TU4euz2np5hDUx8Jn/KnvYTSUQjRwO6956Ox14+loipJl7aT8UiSsajSsUjSsWiSsWjSsYjpfdY\n6T0VL503vh1llh6ngdANAADKwswmgmnzyZdKn7J8oajhXEHDYwUNea/hXF5DYwUNjh7dPub4WGls\naOJzeR0ZGtO+I+OfL42N5F5ZoJ8sFhn/OSNKxrzAfnxI90J80jsnGYsoEYsoEfXeYxHFve3kceOJ\naERx733ic8eNJ6IRWnRmmBkTus3sOkmflxSV9K/Ouc8EXBIAAPBRLBpRNhpRNhUv+3cXi6VZ+ZFc\nQSP5gkZype2JsVxpbDR//HZxYn/EOzZ63Gf7RnLHnDeaK2isUNRYoSg3te6bKYlH7dggPimcJ2MR\nxaIRxaOmeDSiWMQm9mORiGJRUzwSUTxW2o9HvePeeRPHoyf4nPd98VhE8YmxExz3vicaKR2LRsa3\nS+9Rs1D9xWFGhG4zi0q6VdI1kvZIWmtma5xzm4KtDAAAzESRiKnGazmpFOec8kWnsXyx9Coc936y\nsUn7uUJRo8ef642P5YsanTSWLxaVyzsN5PPKF5xyhaLyRad8oahcwZWOj49P2q+kiGlSGI9MhPLI\npHA+EdIjpmgkcsz45PM+dNUSXbGspaL1vxIzInRLukTSFufcNkkys+9KukESoRsAAMwIZjYx81yb\nDLqaE3POqVD0/nIwHsYLReWOC+v5wtSOF73vKky8F1UoSoVi8ZjxY88rqnDMZ7z3gneuO/pd+ULp\nLzEF55Qrnn7LUCXMlNDdIWn3pP09ki6dfIKZ3SLpFkmaP39+5SoDAACoEmamWNQUi0qpeOX+FSAM\nqmZtHefcbc65Vc65Va2trUGXAwAAAEyYKaF7r6R5k/bnemMAAADAtDdTQvdaScvMbJGZJSTdJGlN\nwDUBAAAAUzIjerqdc3kz+5ikn6m0ZOAdzrnnAi4LAAAAmJIZEbolyTl3j6R7gq4DAAAAeKVmSnsJ\nAAAAMGMRugEAAACfEboBAAAAnxG6AQAAAJ8RugEAAACfEboBAAAAnxG6AQAAAJ8RugEAAACfEboB\nAAAAnxG6AQAAAJ8RugEAAACfEboBAAAAnxG6AQAAAJ+Zcy7oGsrOzA5J2hnQH98iqSugPxuVw3UO\nB65zOHCdw4HrHA5BXOcFzrnW33ZSVYbuIJnZOufcqqDrgL+4zuHAdQ4HrnM4cJ3DYTpfZ9pLAAAA\nAJ8RugEAAACfEbrL77agC0BFcJ3DgescDlzncOA6h8O0vc70dAMAAAA+Y6YbAAAA8BmhGwAAAPAZ\nobtMzOw6M3vRzLaY2SeDrgdnxszuMLNOM9s4aazJzO4zs83ee6M3bmb2Be/aP2tmFwVXOabKzOaZ\n2QNmtsnMnjOzT3jjXOcqYmYpM3vCzNZ71/n/88YXmdnj3vX8npklvPGkt7/FO74wyPrxyphZ1Mye\nNrOfevtc5ypjZjvMbIOZPWNm67yxGfF7m9BdBmYWlXSrpDdJWi7pnWa2PNiqcIa+Lum648Y+Kel+\n59wySfd7+1Lpui/zXrdI+lKFasSZyUv6M+fcckmXSfqo998t17m6jEq62jl3oaQVkq4zs8sk/b2k\nzznnlkrqkfQB7/wPSOrxxj/nnYeZ4xOSnp+0z3WuTq9zzq2YtB73jPi9Teguj0skbXHObXPOjUn6\nrqQbAq4JZ8A595Ck7uOGb5D0DW/7G5JunDT+TVfymKQGM2uvTKU4Xc65/c65p7ztfpX+R90hrnNV\n8a7XgLcb915O0tWS/t0bP/46j1//f5f0ejOzCpWLM2BmcyX9rqR/9fZNXOewmBG/twnd5dEhafek\n/T3eGKrLLOfcfm/7gKRZ3jbXf4bz/ml5paTHxXWuOl7LwTOSOiXdJ2mrpCPOubx3yuRrOXGdveO9\nkporWzFO0z9J+gtJRW+/WVznauQk/dzMnjSzW7yxGfF7OxbUHwzMZM45Z2ast1kFzCwj6QeS/tQ5\n1zd5sovrXB2ccwVJK8ysQdIPJZ0TcEkoMzO7XlKnc+5JM1sddD3w1RXOub1m1ibpPjN7YfLB6fx7\nm5nu8tgrad6k/bneGKrLwfF/lvLeO71xrv8MZWZxlQL3nc65u71hrnOVcs4dkfSApNeo9M/M4xNP\nk6/lxHX2jtdLOlzhUvHKvVbSW8xsh0otnldL+ry4zlXHObfXe+9U6S/Rl2iG/N4mdJfHWknLvLuk\nE5JukrQm4JpQfmsk3ext3yzpx5PG3+vdJX2ZpN5J/8yFacrr3/yqpOedc5+ddIjrXEXMrNWb4ZaZ\n1Ui6RqX+/Qck/YF32vHXefz6/4GkXzqeIjftOef+yjk31zm3UKX/B//SOfcucZ2ripnVmll2fFvS\ntZI2aob83uaJlGViZm9WqZ8sKukO59ynAy4JZ8DMviNptaQWSQclfUrSjyTdJWm+pJ2S3u6c6/bC\n2xdVWu1kSNL7nXPrgqgbU2dmV0j6taQNOtoD+tcq9XVznauEmV2g0o1VUZUmmu5yzv2dmS1WaUa0\nSdLTkt7tnBs1s5Skf1Opx79b0k3OuW3BVI/T4bWX/Llz7nquc3XxrucPvd2YpG875z5tZs2aAb+3\nCd0AAACAz2gvAQAAAHxG6AYAAAB8RugGAAAAfEboBgAAAHxG6AYAAAB8RugGgCpjZgUze2bS65Nl\n/O6FZraxXN8HAGHBY+ABoPoMO+dWBF0EAOAoZroBICTMbIeZ/R8z22BmT5jZUm98oZn90syeNbP7\nzWy+Nz7LzH5oZuu91+XeV0XN7HYze87Mfu496REAcAqEbgCoPjXHtZe8Y9KxXufc+So9pe2fvLF/\nlvQN59wFku6U9AVv/AuSfuWcu1DSRZKe88aXSbrVOfcqSUckvc3nnwcAZjyeSAkAVcbMBpxzmROM\n75B0tXNum5nFJR1wzjWbWZekdudczhvf75xrMbNDkuY650YnfcdCSfc555Z5+38pKe6c+9/+/2QA\nMHMx0w0A4eJOsv1KjE7aLoj7gwDgtyJ0A0C4vGPS+2+87Ucl3eRtv0vSr73t+yV9WJLMLGpm9ZUq\nEgCqDbMTAFB9aszsmUn79zrnxpcNbDSzZ1WarX6nN/Ynkr5mZv9D0iFJ7/fGPyHpNjP7gEoz2h+W\ntN/36gGgCtHTDQAh4fV0r3LOdQVdCwCEDe0lAAAAgM+Y6QYAAAB8xkw3AAAA4DNCNwAAAOAzQjcA\nAADgM0I3AAAA4DNCNwAAAOCz/weKhIJjMqfEbgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 864x576 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "None\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y9a_TxBFx4mI",
        "colab_type": "text"
      },
      "source": [
        "## Use the trained model to make predictions\n",
        "\n",
        "We've trained a model and demonstrated that it's good—but not perfect—at converting temperatures. Now let's use the trained model to make some predictions on [unlabeled examples](https://developers.google.com/machine-learning/glossary/#unlabeled_example); that is, on examples that contain features but not a label.\n",
        "\n",
        "In real-life, the unlabeled examples could come from lots of different sources including apps, CSV files, and data feeds. For now, we're going to manually provide degree Celsius temperatures to predict their respective degree Fahrenheit temperatures."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hg_hGeJaxqZX",
        "colab_type": "code",
        "outputId": "13a2e64c-0bf8-4268-f637-efcd908ba960",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "let unlabeledDataset: [Float] = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]\n",
        "let unlabeledDatasetPredictions = Tensor<Float>(shape: [10, 1], scalars: unlabeledDataset)\n",
        "\n",
        "let pred = model(unlabeledDatasetPredictions)\n",
        "\n",
        "print(\"prediction:\")\n",
        "print(pred)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "prediction:\r\n",
            "[[47.307415],\r\n",
            " [ 65.53156],\r\n",
            " [ 83.75571],\r\n",
            " [101.97985],\r\n",
            " [  120.204],\r\n",
            " [138.42815],\r\n",
            " [ 156.6523],\r\n",
            " [174.87643],\r\n",
            " [193.10059],\r\n",
            " [211.32474]]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UU2hM02EymdW",
        "colab_type": "text"
      },
      "source": [
        "**The** correct answers are \n",
        "\n",
        "$10 \\times 1.8 + 32 = 50$\n",
        "\n",
        "$20 \\times 1.8 + 32 = 68$\n",
        "\n",
        "$30 \\times 1.8 + 32 = 86$\n",
        "\n",
        "$40 \\times 1.8 + 32 = 104$\n",
        "\n",
        "$50 \\times 1.8 + 32 = 122$\n",
        "\n",
        "$60 \\times 1.8 + 32 = 140$\n",
        "\n",
        "$70 \\times 1.8 + 32 = 158$\n",
        "\n",
        "$80 \\times 1.8 + 32 = 176$\n",
        "\n",
        "$90 \\times 1.8 + 32 = 194$\n",
        "\n",
        "$100 \\times 1.8 + 32 = 212$\n",
        "\n",
        ", so our model is doing really well.\n",
        "\n",
        "### To review\n",
        "\n",
        "\n",
        "*   We created a model with a Dense layer.\n",
        "*   We trained it with 3500 examples (7 pairs, over 500 epochs).\n",
        "\n",
        "Our model tuned the variables (weights) in the Dense layer until it was able to return the correct Fahrenheit value for any Celsius value. (Remember, none of these degree Celsius temperatures were part of our training data.)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ndGNLPYjzflz",
        "colab_type": "text"
      },
      "source": [
        "## Looking at the layer weights\n",
        "\n",
        "Finally, let's print the internal variables of the Dense layer. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e3CYb7sqxr9i",
        "colab_type": "code",
        "outputId": "033e6242-09fe-47b7-cd81-6158042f9afd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "print(model.neuron.weight)\n",
        "print(model.neuron.bias)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[1.8224146]]\r\n",
            "[29.08327]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YwPV0Orhzh8h",
        "colab_type": "text"
      },
      "source": [
        "The first variable is close to ~1.8 and the second to ~32. These values (1.8 and 32) are the actual variables in the real conversion formula.\n",
        "\n",
        "This is really close to the values in the conversion formula. For a single neuron with a single input and a single output, the internal math looks the same as [the equation for a line](https://en.wikipedia.org/wiki/Linear_equation#Slope%E2%80%93intercept_form), $y = mx + b$, which has the same form as the conversion equation, $f = 1.8c + 32$.\n",
        "\n",
        "Since the form is the same, the variables should converge on the standard values of 1.8 and 32, which is exactly what happened.\n",
        "\n",
        "With additional neurons, additional inputs, and additional outputs, the formula becomes much more complex, but the idea is the same. \n",
        "\n",
        "### A little experiment\n",
        "\n",
        "Just for fun, what if we created more Dense layers with different units, which therefore also has more variables?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Yz7ATykXxtSA",
        "colab_type": "code",
        "outputId": "576c4e1d-93d8-447e-b5c8-8021bf48a982",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "let hiddenSize: Int = 4\n",
        "struct CelsiusToFahrenheit2: Layer {\n",
        "    var layer1 = Dense<Float>(inputSize: 1, outputSize: hiddenSize)\n",
        "    var layer2 = Dense<Float>(inputSize: hiddenSize, outputSize: hiddenSize)\n",
        "    var layer3 = Dense<Float>(inputSize: hiddenSize, outputSize: 1)\n",
        "  \n",
        "    @differentiable\n",
        "    func callAsFunction(_ input: Tensor<Float>) -> Tensor<Float> {\n",
        "        return input.sequenced(through: layer1, layer2, layer3)\n",
        "    }\n",
        "}\n",
        "\n",
        "var model = CelsiusToFahrenheit2()\n",
        "\n",
        "let optimizer = Adam(for: model, learningRate: 0.1)\n",
        "let epochCount = 500\n",
        "var trainLossResults: [Float] = []\n",
        "\n",
        "for epoch in 1...epochCount {\n",
        "    var epochLoss: Float = 0\n",
        "    let (loss, grad) = model.valueWithGradient { model -> Tensor<Float> in\n",
        "        let pred = model(input)\n",
        "        return meanSquaredError(predicted: pred, expected: output)\n",
        "    }\n",
        "    \n",
        "    optimizer.update(&model.allDifferentiableVariables, along: grad)\n",
        "    epochLoss = loss.scalarized()\n",
        "    trainLossResults.append(epochLoss)\n",
        "    if epoch.isMultiple(of: 20) {\n",
        "        print(\"Epoch \\(epoch): Loss: \\(loss)\")\n",
        "    }\n",
        "}\n",
        "\n",
        "plt.figure(figsize: [10, 6])\n",
        "\n",
        "plt.plot(trainLossResults)\n",
        "plt.xlabel(\"Epoch\")\n",
        "plt.ylabel(\"Loss\")\n",
        "\n",
        "plt.show()\n",
        "\n",
        "let unlabeledDataset: [Float] = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]\n",
        "let unlabeledDatasetPredictions = Tensor<Float>(shape: [10, 1], scalars: unlabeledDataset)\n",
        "let pred = model(unlabeledDatasetPredictions)\n",
        "\n",
        "print(\"Experimental prediction:\")\n",
        "print(pred)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 20: Loss: 381.63824\n",
            "Epoch 40: Loss: 23.773376\n",
            "Epoch 60: Loss: 5.9554267\n",
            "Epoch 80: Loss: 0.94320196\n",
            "Epoch 100: Loss: 0.16744521\n",
            "Epoch 120: Loss: 0.067692265\n",
            "Epoch 140: Loss: 0.064259484\n",
            "Epoch 160: Loss: 0.06321911\n",
            "Epoch 180: Loss: 0.063052796\n",
            "Epoch 200: Loss: 0.06304897\n",
            "Epoch 220: Loss: 0.06304616\n",
            "Epoch 240: Loss: 0.06304521\n",
            "Epoch 260: Loss: 0.063045904\n",
            "Epoch 280: Loss: 0.06304621\n",
            "Epoch 300: Loss: 0.06304634\n",
            "Epoch 320: Loss: 0.06304652\n",
            "Epoch 340: Loss: 0.06304531\n",
            "Epoch 360: Loss: 0.06304527\n",
            "Epoch 380: Loss: 0.06304527\n",
            "Epoch 400: Loss: 0.06304527\n",
            "Epoch 420: Loss: 0.06304527\n",
            "Epoch 440: Loss: 0.06304527\n",
            "Epoch 460: Loss: 0.06304527\n",
            "Epoch 480: Loss: 0.06304527\n",
            "Epoch 500: Loss: 0.06304527\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm4AAAF3CAYAAAAcmcfdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XuwnXV97/H3d+1LrkBumxgSYiJE\nBVGRkwJWzxQvQKQd0dZjYVqhlpn0WDzqtLWiZ069tJzWc1q1tpYZPKLgqBRvNfVQMSIesZVLgoAE\nxEQgkhhIyD3kvvf3/LGevVnEBHfW2ut51sp+v2bW7Of5rWc9+7v3M+x8+F2eJzITSZIkdb5a1QVI\nkiRpdAxukiRJXcLgJkmS1CUMbpIkSV3C4CZJktQlDG6SJEldwuAmSZLUJdoW3CJiYkTcFRH3RcSq\niPhw0f65iHg0Iu4tXmcW7RERn4yINRFxf0Sc1XCuyyNidfG6vF01S5IkdbLeNp57H/DazNwVEX3A\nDyLi34r33puZXznk+DcAi4rXOcA1wDkRMQP4ILAYSGBlRCzLzK1trF2SJKnjtK3HLet2Fbt9xeu5\nHtNwMXBD8bk7gGkRMQe4EFiemVuKsLYcWNKuuiVJkjpVO3vciIgeYCVwKvCpzLwzIt4BXB0RfwHc\nClyVmfuAucDjDR9fV7Qdqf2IZs2alQsWLBizn0OSJKldVq5c+VRmDozm2LYGt8wcBM6MiGnA1yPi\nDOD9wBNAP3At8D7gI61+r4hYCiwFmD9/PitWrGj1lJIkSW0XEWtHe2wpq0ozcxtwG7AkMzcUw6H7\ngM8CZxeHrQdObvjYvKLtSO2Hfo9rM3NxZi4eGBhVaJUkSeoq7VxVOlD0tBERk4DzgZ8U89aIiADe\nBDxQfGQZcFmxuvRcYHtmbgBuAS6IiOkRMR24oGiTJEkaV9o5VDoHuL6Y51YDbsrMb0bEdyNiAAjg\nXuC/FsffDFwErAF2A28HyMwtEfGXwN3FcR/JzC1trFuSJKkjReZzLfTsTosXL07nuEmSpG4QESsz\nc/FojvXJCZIkSV3C4CZJktQlDG6SJEldwuAmSZLUJQxukiRJXcLgJkmS1CUMbpIkSV3C4NakH6x+\nitVP7qy6DEmSNI4Y3Jr0ji+s5It3/bzqMiRJ0jhicGtSLYJj8KETkiSpgxncmlQLGDK5SZKkEhnc\nmlSLMLhJkqRSGdyaFBEMmdskSVKJDG5NqgWkPW6SJKlEBrcm1SIYGqq6CkmSNJ4Y3Jrk4gRJklQ2\ng1uTnOMmSZLKZnBrUq3mHDdJklQug1uTvB2IJEkqm8GtSQEOlUqSpFIZ3Jpkj5skSSqbwa1JEWBs\nkyRJZTK4Nan+kHmjmyRJKo/BrUnegFeSJJXN4Nak8Aa8kiSpZAa3JtW8Aa8kSSqZwa1J3oBXkiSV\nzeDWJG8HIkmSymZwa5LPKpUkSWUzuDWp5uIESZJUMoNbk+r3cau6CkmSNJ4Y3Jpkj5skSSqbwa1J\n4eIESZJUMoNbk+o9blVXIUmSxhODW5N8VqkkSSqbwa1JPjlBkiSVrW3BLSImRsRdEXFfRKyKiA8X\n7Qsj4s6IWBMR/xwR/UX7hGJ/TfH+goZzvb9ofzgiLmxXzUfDZ5VKkqSytbPHbR/w2sx8OXAmsCQi\nzgU+Cnw8M08FtgJXFMdfAWwt2j9eHEdEnA5cArwEWAL8U0T0tLHuUbHHTZIkla1twS3rdhW7fcUr\ngdcCXynarwfeVGxfXOxTvP+6iIii/cbM3JeZjwJrgLPbVfdo1cJnlUqSpHK1dY5bRPRExL3ARmA5\n8DNgW2YeLA5ZB8wttucCjwMU728HZja2H+Yzjd9raUSsiIgVmzZtaseP8yw+q1SSJJWtrcEtMwcz\n80xgHvVeshe38Xtdm5mLM3PxwMBAu77NiIhgaKjt30aSJGlEKatKM3MbcBvwSmBaRPQWb80D1hfb\n64GTAYr3TwA2N7Yf5jOV8ckJkiSpbO1cVToQEdOK7UnA+cBD1APcW4rDLge+UWwvK/Yp3v9u1ieR\nLQMuKVadLgQWAXe1q+7RisBnlUqSpFL1/upDmjYHuL5YAVoDbsrMb0bEg8CNEfFXwI+AzxTHfwb4\nfESsAbZQX0lKZq6KiJuAB4GDwJWZOdjGukelFkFicpMkSeVpW3DLzPuBVxym/REOsyo0M/cC/+UI\n57oauHqsa2yFtwORJEll88kJTfIGvJIkqWwGtybVn1VadRWSJGk8Mbg1yVWlkiSpbAa3JnkDXkmS\nVDaDW5O8Aa8kSSqbwa1JPqtUkiSVzeDWJG8HIkmSymZwa1Kt5uIESZJULoNbk8IeN0mSVDKDW5Oc\n4yZJkspmcGuStwORJEllM7g1ycUJkiSpbAa3JvmsUkmSVDaDW5N8VqkkSSqbwa1JPqtUkiSVzeDW\nJBcnSJKkshncmuR93CRJUtkMbk3yPm6SJKlsBrcmeTsQSZJUNoNbk7wdiCRJKpvBrUnh7UAkSVLJ\nDG5NqkX9q/PcJElSWQxuTapFPbk5z02SJJXF4Nak4R4357lJkqSyGNyaFCM9bgY3SZJUDoNbk4aH\nSs1tkiSpLAa3JjlUKkmSymZwa5KLEyRJUtkMbk0Ke9wkSVLJDG5NGpnjNlRxIZIkadwwuDXJOW6S\nJKlsBrcm1WreDkSSJJXL4NakcHGCJEkqmcGtST6rVJIklc3g1iRvByJJksrWtuAWESdHxG0R8WBE\nrIqIdxftH4qI9RFxb/G6qOEz74+INRHxcERc2NC+pGhbExFXtavmo+HiBEmSVLbeNp77IPCnmXlP\nRBwHrIyI5cV7H8/Mv208OCJOBy4BXgKcBHwnIl5YvP0p4HxgHXB3RCzLzAfbWPuv5LNKJUlS2doW\n3DJzA7Ch2N4ZEQ8Bc5/jIxcDN2bmPuDRiFgDnF28tyYzHwGIiBuLYysNbj6rVJIkla2UOW4RsQB4\nBXBn0fTOiLg/Iq6LiOlF21zg8YaPrSvajtReKYdKJUlS2doe3CJiKvBV4D2ZuQO4BjgFOJN6j9zf\njdH3WRoRKyJixaZNm8bilM/JxQmSJKlsbQ1uEdFHPbR9ITO/BpCZT2bmYGYOAZ/mmeHQ9cDJDR+f\nV7Qdqf1ZMvPazFycmYsHBgbG/oc5hM8qlSRJZWvnqtIAPgM8lJkfa2if03DYm4EHiu1lwCURMSEi\nFgKLgLuAu4FFEbEwIvqpL2BY1q66RytG5rgZ3CRJUjnauar0VcDbgB9HxL1F2weASyPiTCCBx4A/\nAsjMVRFxE/VFBweBKzNzECAi3gncAvQA12XmqjbWPSrP3IC32jokSdL40c5VpT8A4jBv3fwcn7ka\nuPow7Tc/1+eq4Bw3SZJUNp+c0CRXlUqSpLIZ3JrkDXglSVLZDG5N8ga8kiSpbAa3JjlUKkmSymZw\na5KLEyRJUtkMbk3yBrySJKlsBrcm1bwBryRJKpnBrUkOlUqSpLIZ3Jo0sjjB5CZJkkpicGtS2OMm\nSZJKZnBr0jPPKjW5SZKkchjcmlSr2eMmSZLKZXBrkjfglSRJZTO4NclnlUqSpLIZ3Jrks0olSVLZ\nDG5NcqhUkiSVzeDWJG/AK0mSymZwa5LPKpUkSWUzuDXJZ5VKkqSyGdya9EyPW7V1SJKk8cPg1iRX\nlUqSpLIZ3JrkqlJJklQ2g1uTvAGvJEkqm8GtSQ6VSpKkshncmuRQqSRJKpvBrUnegFeSJJXN4NYk\nb8ArSZLKZnBrkjfglSRJZTO4NcmhUkmSVDaDW5NcnCBJkspmcGtS2OMmSZJKZnBr0nCPm3PcJElS\nWQxuTRqZ42aXmyRJKonBrUkuTpAkSWUzuDUpit+cixMkSVJZ2hbcIuLkiLgtIh6MiFUR8e6ifUZE\nLI+I1cXX6UV7RMQnI2JNRNwfEWc1nOvy4vjVEXF5u2o+Gj6rVJIkla2dPW4HgT/NzNOBc4ErI+J0\n4Crg1sxcBNxa7AO8AVhUvJYC10A96AEfBM4BzgY+OBz2quTtQCRJUtnaFtwyc0Nm3lNs7wQeAuYC\nFwPXF4ddD7yp2L4YuCHr7gCmRcQc4EJgeWZuycytwHJgSbvqHi3nuEmSpLKVMsctIhYArwDuBGZn\n5obirSeA2cX2XODxho+tK9qO1F4pn1UqSZLK1vbgFhFTga8C78nMHY3vZf0maGOSfCJiaUSsiIgV\nmzZtGotTPiefVSpJksrW1uAWEX3UQ9sXMvNrRfOTxRAoxdeNRft64OSGj88r2o7U/iyZeW1mLs7M\nxQMDA2P7gxyGQ6WSJKls7VxVGsBngIcy82MNby0DhleGXg58o6H9smJ16bnA9mJI9RbggoiYXixK\nuKBoq1QxUupQqSRJKk1vG8/9KuBtwI8j4t6i7QPA3wA3RcQVwFrgrcV7NwMXAWuA3cDbATJzS0T8\nJXB3cdxHMnNLG+selRh55FW1dUiSpPGjbcEtM3/AMx1Th3rdYY5P4MojnOs64Lqxq651EUGEc9wk\nSVJ5fHJCC2oRznGTJEmlMbi1oBbOcZMkSeUxuLUg7HGTJEklMri1oOYcN0mSVCKDWwvqc9wMbpIk\nqRwGtxa4OEGSJJXJ4NaCcHGCJEkqkcGtBbUIb8ArSZJKY3BrQS1g0LFSSZJUEoNbC3pqLk6QJEnl\nMbi1oBZhj5skSSqNwa0FvTWDmyRJKo/BrQU1g5skSSqRwa0FPbVg0DlukiSpJAa3FvTY4yZJkko0\nquAWEadExIRi+7yIeFdETGtvaZ2vx8UJkiSpRKPtcfsqMBgRpwLXAicDX2xbVV3CHjdJklSm0Qa3\nocw8CLwZ+IfMfC8wp31ldQeDmyRJKtNog9uBiLgUuBz4ZtHW156SuoeLEyRJUplGG9zeDrwSuDoz\nH42IhcDn21dWd7DHTZIklal3NAdl5oPAuwAiYjpwXGZ+tJ2FdQMXJ0iSpDKNdlXp9yLi+IiYAdwD\nfDoiPtbe0jqfPW6SJKlMox0qPSEzdwC/DdyQmecAr29fWd3B4CZJkso02uDWGxFzgLfyzOKEcc/F\nCZIkqUyjDW4fAW4BfpaZd0fEC4DV7SurO/TUgiF73CRJUklGuzjhy8CXG/YfAX6nXUV1i54IDhrc\nJElSSUa7OGFeRHw9IjYWr69GxLx2F9fpnOMmSZLKNNqh0s8Cy4CTite/Fm3jmsFNkiSVabTBbSAz\nP5uZB4vX54CBNtbVFWouTpAkSSUabXDbHBG/HxE9xev3gc3tLKwb9NrjJkmSSjTa4PaH1G8F8gSw\nAXgL8Adtqqlr+OQESZJUplEFt8xcm5lvzMyBzDwxM9+Eq0q9HYgkSSrVaHvcDudPxqyKLtVT83Yg\nkiSpPK0EtxizKrpUTy0YcnGCJEkqSSvBbdwnFnvcJElSmZ4zuEXEzojYcZjXTur3c3uuz15X3Kz3\ngYa2D0XE+oi4t3hd1PDe+yNiTUQ8HBEXNrQvKdrWRMRVLfysY67m4gRJklSi53zkVWYe18K5Pwf8\nI3DDIe0fz8y/bWyIiNOBS4CXUA+E34mIFxZvfwo4H1gH3B0RyzLzwRbqGjO9Lk6QJEklGtWzSpuR\nmd+PiAWjPPxi4MbM3Ac8GhFrgLOL99YUz0YlIm4sju2I4OZQqSRJKlMrc9ya9c6IuL8YSp1etM0F\nHm84Zl3RdqT2juDiBEmSVKayg9s1wCnAmdRv5Pt3Y3XiiFgaESsiYsWmTZvG6rTPyR43SZJUplKD\nW2Y+mZmDmTkEfJpnhkPXAyc3HDqvaDtS++HOfW1mLs7MxQMD5TxGtRZBJqS9bpIkqQSlBreImNOw\n+2ZgeMXpMuCSiJgQEQuBRcBdwN3AoohYGBH91BcwLCuz5ufSW6vfys6VpZIkqQxtW5wQEV8CzgNm\nRcQ64IPAeRFxJvV7wD0G/BFAZq6KiJuoLzo4CFyZmYPFed4J3AL0ANdl5qp21Xy0akVwOziU9PZU\nXIwkSTrmtXNV6aWHaf7Mcxx/NXD1YdpvBm4ew9LGzHCPmwsUJElSGapYVXrM6GnocZMkSWo3g1sL\nalH0uBncJElSCQxuLejtcXGCJEkqj8GtBcM9boNDybu+9COu+8GjFVckSZKOZQa3FozcDiST//jZ\nZu75+daKK5IkSccyg1sLRm4HMpjs2X+QPfsHK65IkiQdywxuLeiJZ24HsufAIHsOGNwkSVL7GNxa\nMLw4Yff+QYYSg5skSWorg1sLhhcn7Np3EMChUkmS1FYGtxYML07YufcAYI+bJElqL4NbC2ojwc0e\nN0mS1H4Gtxb0HDpUao+bJElqI4NbC3qKxQm77HGTJEklMLi14NAet4NDyYHBoSpLkiRJxzCDWwt6\nD5njBg6XSpKk9jG4teDQxQkAex0ulSRJbWJwa8Fwj9uufQdG2nYb3CRJUpsY3FpQqz17jhs4VCpJ\nktrH4NaCkcUJznGTJEklMLi1oGd4jts+57hJkqT2M7i1YDi4Nfa4OcdNkiS1i8GtBd4ORJIklcng\n1oLhxQl7Dgwyobc2si1JktQOBrcWDC9OAJg5pR+AvQY3SZLUJga3FgzPcQOYMbUe3JzjJkmS2sXg\n1oLG4DZ9cj24+aB5SZLULga3FvQ2BLcp/b1M7Ks5VCpJktrG4NaCWkNwm9zfw6S+HodKJUlS2xjc\nWtC4OGHqxF4m9fW4qlSSJLWNwa0FPT0NixOm9DOp3+AmSZLax+DWgsYet+Hg5iOvJElSuxjcWnDo\nqlLnuEmSpHYyuLXgWfdxm9LPROe4SZKkNjK4taBxqHT65H4m9/d4OxBJktQ2BrcW1A7pcXNVqSRJ\naieD2xiZNrmPSf3OcZMkSe3TtuAWEddFxMaIeKChbUZELI+I1cXX6UV7RMQnI2JNRNwfEWc1fOby\n4vjVEXF5u+pt1cS+Hib19bqqVJIktU07e9w+Byw5pO0q4NbMXATcWuwDvAFYVLyWAtdAPegBHwTO\nAc4GPjgc9jrRpP6aQ6WSJKlt2hbcMvP7wJZDmi8Gri+2rwfe1NB+Q9bdAUyLiDnAhcDyzNySmVuB\n5fxyGOwYk/p6ODiUHBgcqroUSZJ0DCp7jtvszNxQbD8BzC625wKPNxy3rmg7UvsviYilEbEiIlZs\n2rRpbKsepYl9PQDOc5MkSW1R2eKEzEwgx/B812bm4sxcPDAwMFanPSqT+3sBRm4J8rV71vGtBzY8\n10ckSZJGrbfk7/dkRMzJzA3FUOjGon09cHLDcfOKtvXAeYe0f6+EOkftHeedwsvmngDU57gB7Cl6\n3P72loc5flIfS86YU1l9kiTp2FF2j9syYHhl6OXANxraLytWl54LbC+GVG8BLoiI6cWihAuKto7x\nviUv5g0vrQezSQ1Dpdt27+cX2/fy0yd3snv/wSpLlCRJx4h23g7kS8APgRdFxLqIuAL4G+D8iFgN\nvL7YB7gZeARYA3wa+GOAzNwC/CVwd/H6SNHWkYbnuO05MMhDG3YCMJTwwPodVZYlSZKOEW0bKs3M\nS4/w1usOc2wCVx7hPNcB141haW3TOMft4Sd2jrTf9/g2zl44o6qyJEnSMaLsOW7HtOGh0j37B3lw\nww5mTZ3AhN4a963bVnFlkiTpWGBwG0PDixN2Hxjkp0/u5LQ5x5EJ67buqbgySZJ0LPBZpWNoeI7b\n3v2DbNi+l3nTJzFjSj9bnt5fcWWSJOlYYI/bGBqe47Zz30Ge2rWPE4+byPbeAwY3SZI0JgxuY2h4\njtvjW3aTCbOPn0hvLdi17yD7Dg4yoben4golSVI3c6h0DE3orf86H9v8NACzj5/AjKn9APa6SZKk\nlhncxlCtFkzsq7F2826g3uM2c0o9uG3eZXCTJEmtcah0jE3u7x3pcTvx+AkjD5y3x02SJLXKHrcx\nNqmvh0zoqQUzp0xgxhSHSiVJ0tgwuI2x4yf1ATAwdUIR3oqhUoObJElqkcFtjL118TwAntixF4AT\nJvXRUwu2PL2vyrIkSdIxwOA2xi75tfkAvHD2VKC+YGH65D6HSiVJUstcnDDGJvX3cPufv4YJfc9k\n4hlT+l1VKkmSWmZwa4OTZ0x+1r6PvZIkSWPBodISzJwyweAmSZJaZnArwYwp/a4qlSRJLTO4lWDG\nlH627znAgcGhqkuRJEldzOBWgpnF80q37rbXTZIkNc/gVgKfniBJksaCwa0EI8HNW4JIkqQWGNxK\nMHPKBMDHXkmSpNYY3ErgUKkkSRoLBrcSTJ9cf/C8PW6SJKkVBrcS9PbUmDa5zwfNS5KklhjcSuJj\nryRJUqsMbiWZNWUCT+00uEmSpOYZ3Eoyb8Ykfr5ld9VlSJKkLmZwK8kLZk3hiR172b3/IB/91k+4\n5Nofsn3PgarLkiRJXcTgVpKFs6YC8Mimp7nmez/jjke28Gdfvq/iqiRJUjcxuJVk4awpANyy6gkA\nJvTWuOvRLVWWJEmSuozBrSQLZk0G4PN3rAXgbec+n+17DrDNB89LkqRRMriVZHJ/LxGwbfcBFp04\nlXNfMBOAxza7YEGSJI2Owa1EL583DYC/+Z2XjvTArd38dJUlSZKkLtJbdQHjyT/93lkcHEzmz5zM\n3gODRMBjT9njJkmSRsfgVqKTpk0a2Z7Y18NJJ0ziMXvcJEnSKFUyVBoRj0XEjyPi3ohYUbTNiIjl\nEbG6+Dq9aI+I+GRErImI+yPirCpqbofnz5zMo08Z3CRJ0uhUOcftNZl5ZmYuLvavAm7NzEXArcU+\nwBuARcVrKXBN6ZW2ycnTJ7Nu656qy5AkSV2ikxYnXAxcX2xfD7ypof2GrLsDmBYRc6oocKzNmz6J\np3btY++BwapLkSRJXaCq4JbAtyNiZUQsLdpmZ+aGYvsJYHaxPRd4vOGz64q2rjd3en3O2/pt9rpJ\nkqRfrarFCa/OzPURcSKwPCJ+0vhmZmZE5NGcsAiASwHmz58/dpW20dxiscL6rXs4ZWBqxdVIkqRO\nV0mPW2auL75uBL4OnA08OTwEWnzdWBy+Hji54ePzirZDz3ltZi7OzMUDAwPtLH/M2OMmSZKORunB\nLSKmRMRxw9vABcADwDLg8uKwy4FvFNvLgMuK1aXnAtsbhlS72vOOn0hPLVjvAgVJkjQKVQyVzga+\nHhHD3/+LmfmtiLgbuCkirgDWAm8tjr8ZuAhYA+wG3l5+ye3R21PjecdPtMdNkiSNSunBLTMfAV5+\nmPbNwOsO057AlSWUVom50yexbqtPT5AkSb9aJ90OZFyaP6N+E956PpUkSToyg1vFXjbvBJ7atZ8N\n2/dWXYokSepwBreKvXzeNADue3xbxZVIkqROZ3Cr2IvnHEdfT3DvOoObJEl6bga3ik3o7eH0Ocez\n8rGtZKZz3SRJ0hEZ3DrA60+bzYq1W3nlX3+X3/jf32PDdm8PIkmSfpnBrQNc+ZpTufI1pzB/5mS2\nPL2fpTesrLokSZLUgap6Vqka1GrBey98MQCf/fdH+fC/PsjazU/z/JlTKq5MkiR1EnvcOsxvvLD+\nnNXbVz9VcSWSJKnTGNw6zMJZU5g7bRK3r95UdSmSJKnDGNw6TETw6lNn8cOfbXaFqSRJehaDWwc6\nc/40duw9yONbXF0qSZKeYXDrQGecdAIAD/xie8WVSJKkTmJw60AvfN5UemvBKoObJElqYHDrQBN6\nezj1xKk8sH5H1aVIkqQOYnDrUGfMPYFVv9juAgVJkjTC4NahXnLS8Ty1az8bd+6ruhRJktQhDG4d\n6oy59QUKznOTJEnDDG4d6rQ5xxOB89wkSdIIg1uHmjqhl4Uzp9jjJkmSRhjcOtjpJx1vj5skSRph\ncOtgv7ZgBuu37eHhJ3ZWXYokSeoABrcO9psvm0NvLfjaPeuqLkWSJHUAg1sHmzV1Aue9aICvrFxn\nr5skSTK4dbr3vP6FRARv/qd/Z+XaLVWXI0mSKmRw63BnzD2B//uuV3PicRP4w8+t4Ol9B6suSZIk\nVcTg1gVmHz+R//nbL2X7ngN87+FNVZcjSZIqYnDrEucsnMnMKf3c/MCGqkuRJEkVMbh1iZ5acOEZ\nz+O2n2xk74HBqsuRJEkVMLh1kde86ER27x/kvse3VV2KJEmqgMGti5y9YAYRcOejri6VJGk8Mrh1\nkRMm93Ha847njkc2V12KJEmqgMGty5zzghmsXLvVeW6SJI1DBrcuc/5ps9l3cIh/c3WpJEnjjsGt\ny5z7gpk8f+ZkPv/DtXz/p5v49PcfYf/BoSMev233fm744WNs272/vCIlSVJbdE1wi4glEfFwRKyJ\niKuqrqcqtVpw2SsXcM/Pt3HZdXdx9c0P8Wdfvo/MBGDXvoN8ZeU6du49wJqNO3nD39/OX3xjFRf9\n/e3c8chmNu7Yy/bdByr+KSRJUjNi+B/8ThYRPcBPgfOBdcDdwKWZ+eDhjl+8eHGuWLGixArLlZms\nXLuVrbsPsOoX2/nEd1Zz7dv+E/950QCXXXcndz+2lemT++ipBRD89998MZ/4zmrWbt49co4/+PUF\nfOCi0+jv7ZrsLknSMSkiVmbm4tEc29vuYsbI2cCazHwEICJuBC4GDhvcjnURweIFMwB4zYsG+Jcf\nrecT31nNbQ9vZMXarbxvyYt5cMMONmzbw4fe+BLOmHsCrz9tNl9ZuY7enho/2bCDz/3HY9y/bhtv\nPmse+w4M8vyZU/j1U2YyZUIvQ0PJ41t3s/fAELWACNi2+wCbdu6jt6fGtMl9TJvUx8S+Hvp7a5xQ\nbEuSpPbqluA2F3i8YX8dcE5FtXSU3p4af3LBi3jXl37Egxt2cMWrF/KO8075peOOm9jH21+1cGT/\nlafM5P1f+zH/418eGGmb2Ffj5OmTWb9tD7v3H92q1eMn9jJ1Qi8RQQTUGr9SD39RbAMcbT9v/OpD\niFEcFKM402jOM1oxipON2c9W4s8/6l9RiT+/JLXDvOmT+YdLX1F1GSO6Jbj9ShGxFFgKMH/+/Iqr\nKdcbX34Sc06YyO2rn+KPDxPaDue3XnYSS17yPJ7atZ/+3hoPP7GTbz/4BOu37uFVp87itDnHcdzE\nPoYyGcp6MBs4bgJDQ7Btz3627j7A3gOD7D84xPY9B3hyx1527x8ksz6Um8BQJpnFV4r2fOYf4dGE\nCIAcRcwbzYj/qI4Zo+9VP9cojhnVucbo5x/NdxrFiUYbususSZLaZUp/Z40odcsct1cCH8rMC4v9\n9wNk5l8f7vhjfY6bJEk6dhxR3IdDAAAGc0lEQVTNHLdumZl+N7AoIhZGRD9wCbCs4pokSZJK1RVD\npZl5MCLeCdwC9ADXZeaqisuSJEkqVVcEN4DMvBm4ueo6JEmSqtItQ6WSJEnjnsFNkiSpSxjcJEmS\nuoTBTZIkqUsY3CRJkrqEwU2SJKlLGNwkSZK6hMFNkiSpSxjcJEmSuoTBTZIkqUtEZlZdw5iLiE3A\n2hK+1SzgqRK+j0bPa9KZvC6dyevSebwmnand1+X5mTkwmgOPyeBWlohYkZmLq65Dz/CadCavS2fy\nunQer0ln6qTr4lCpJElSlzC4SZIkdQmDW2uurboA/RKvSWfyunQmr0vn8Zp0po65Ls5xkyRJ6hL2\nuEmSJHUJg1sTImJJRDwcEWsi4qqq6xlPIuK6iNgYEQ80tM2IiOURsbr4Or1oj4j4ZHGd7o+Is6qr\n/NgVESdHxG0R8WBErIqIdxftXpcKRcTEiLgrIu4rrsuHi/aFEXFn8fv/54joL9onFPtrivcXVFn/\nsS4ieiLiRxHxzWLf61KhiHgsIn4cEfdGxIqirSP/hhncjlJE9ACfAt4AnA5cGhGnV1vVuPI5YMkh\nbVcBt2bmIuDWYh/q12hR8VoKXFNSjePNQeBPM/N04FzgyuK/Ca9LtfYBr83MlwNnAksi4lzgo8DH\nM/NUYCtwRXH8FcDWov3jxXFqn3cDDzXse12q95rMPLPhth8d+TfM4Hb0zgbWZOYjmbkfuBG4uOKa\nxo3M/D6w5ZDmi4Hri+3rgTc1tN+QdXcA0yJiTjmVjh+ZuSEz7ym2d1L/x2guXpdKFb/fXcVuX/FK\n4LXAV4r2Q6/L8PX6CvC6iIiSyh1XImIe8JvA/yn2A69LJ+rIv2EGt6M3F3i8YX9d0abqzM7MDcX2\nE8DsYttrVbJiGOcVwJ14XSpXDMfdC2wElgM/A7Zl5sHikMbf/ch1Kd7fDswst+Jx4xPAnwNDxf5M\nvC5VS+DbEbEyIpYWbR35N6y3rG8klSEzMyJcKl2BiJgKfBV4T2buaOwU8LpUIzMHgTMjYhrwdeDF\nFZc07kXEbwEbM3NlRJxXdT0a8erMXB8RJwLLI+InjW920t8we9yO3nrg5Ib9eUWbqvPkcDd18XVj\n0e61KklE9FEPbV/IzK8VzV6XDpGZ24DbgFdSH9YZ/p/2xt/9yHUp3j8B2FxyqePBq4A3RsRj1Kfa\nvBb4e7wulcrM9cXXjdT/J+dsOvRvmMHt6N0NLCpWAPUDlwDLKq5pvFsGXF5sXw58o6H9smIF0LnA\n9oZub42RYr7NZ4CHMvNjDW95XSoUEQNFTxsRMQk4n/r8w9uAtxSHHXpdhq/XW4Dvpjf6HHOZ+f7M\nnJeZC6j/+/HdzPw9vC6ViYgpEXHc8DZwAfAAHfo3zBvwNiEiLqI+R6EHuC4zr664pHEjIr4EnAfM\nAp4EPgj8C3ATMB9YC7w1M7cUgeIfqa9C3Q28PTNXVFH3sSwiXg3cDvyYZ+bsfID6PDevS0Ui4mXU\nJ1T3UP+f9Jsy8yMR8QLqPT0zgB8Bv5+Z+yJiIvB56nMUtwCXZOYj1VQ/PhRDpX+Wmb/ldalO8bv/\nerHbC3wxM6+OiJl04N8wg5skSVKXcKhUkiSpSxjcJEmSuoTBTZIkqUsY3CRJkrqEwU2SJKlLGNwk\njTsRMRgR9za8rvrVnxr1uRdExANjdT5JauQjrySNR3sy88yqi5Cko2WPmyQVIuKxiPhfEfHjiLgr\nIk4t2hdExHcj4v6IuDUi5hftsyPi6xFxX/H69eJUPRHx6YhYFRHfLp5cIEktM7hJGo8mHTJU+rsN\n723PzJdSvzP6J4q2fwCuz8yXAV8APlm0fxL4f5n5cuAsYFXRvgj4VGa+BNgG/E6bfx5J44RPTpA0\n7kTErsycepj2x4DXZuYjEdEHPJGZMyPiKWBOZh4o2jdk5qyI2ATMy8x9DedYACzPzEXF/vuAvsz8\nq/b/ZJKOdfa4SdKz5RG2j8a+hu1BnE8saYwY3CTp2X634esPi+3/AC4ptn8PuL3YvhV4B0BE9ETE\nCWUVKWl88v8CJY1HkyLi3ob9b2Xm8C1BpkfE/dR7zS4t2v4b8NmIeC+wCXh70f5u4NqIuIJ6z9o7\ngA1tr17SuOUcN0kqFHPcFmfmU1XXIkmH41CpJElSl7DHTZIkqUvY4yZJktQlDG6SJEldwuAmSZLU\nJQxukiRJXcLgJkmS1CUMbpIkSV3i/wObTks7DolkwgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Experimental prediction:\r\n",
            "[[  49.93202],\r\n",
            " [  67.91151],\r\n",
            " [    85.891],\r\n",
            " [ 103.87049],\r\n",
            " [121.849976],\r\n",
            " [ 139.82948],\r\n",
            " [ 157.80896],\r\n",
            " [ 175.78847],\r\n",
            " [ 193.76794],\r\n",
            " [ 211.74745]]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xg3mZNTl03_0",
        "colab_type": "text"
      },
      "source": [
        "We can clearly see that adding more layers to the model made it more accurate as compared to single layered model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KSbh5HUO_31j",
        "colab_type": "code",
        "outputId": "62109fc7-e262-4658-cbc2-c95567c0ea09",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "print(model.layer1.weight)\n",
        "print(model.layer1.bias)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ -0.2057025,  -0.3703212,    0.660952, -0.18842027]]\r\n",
            "[2.3675368, -3.537131, 3.5914605, 1.3384908]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pGy0iZkIAByb",
        "colab_type": "code",
        "outputId": "06271593-c6a9-43f4-ad03-0074bf39a40f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "print(model.layer2.weight)\n",
        "print(model.layer2.bias)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ -0.7672395, -0.11166922, -0.36101082,  0.28020477],\r\n",
            " [  0.8099734, 0.016118046,   0.4657686,   1.1790013],\r\n",
            " [ -0.5593068,  -0.1577901,  -0.1571194,  -1.0818883],\r\n",
            " [-0.19889235, -0.10707955, -0.24257602, -0.25237373]]\r\n",
            "[-3.4427536, -1.5308913, -3.3530464, -3.5225964]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N4WKNrwgADmX",
        "colab_type": "code",
        "outputId": "dc3aeb44-3e4e-4fe2-e389-8b21503228d9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "print(model.layer3.weight)\n",
        "print(model.layer3.bias)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ -1.3547337],\r\n",
            " [ -0.2261485],\r\n",
            " [-0.55318516],\r\n",
            " [ -0.9069645]]\r\n",
            "[3.3482618]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l5tfkz9BAeXf",
        "colab_type": "text"
      },
      "source": [
        "As you can see, this model is also able to predict the corresponding Fahrenheit value really well. But when you look at the variables (weights) in the `layer1`, `layer2` and `layer3` layers, they are nothing even close to ~1.8 and ~32. The added complexity hides the \"simple\" form of the conversion equation."
      ]
    }
  ]
}