NLP_misc.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gxj61b5yT-t8"
      },
      "source": [
        "## 1. NLP CountVectorizer\n",
        "\n",
        "CountVectorizer is a fundamental tool used in Natural Language Processing (NLP) for converting a collection of text documents into a matrix of token counts. This process is part of the text preprocessing workflow before applying machine learning algorithms to the text data.\n",
        "\n",
        "Here's a breakdown of how CountVectorizer works and its key features:\n",
        "\n",
        "- Tokenization:\n",
        "  - CountVectorizer tokenizes the text by extracting words (tokens) based on specific rules, typically separating words by whitespace or using a predefined tokenizer. It can also handle n-grams (sequences of words) as tokens.\n",
        "\n",
        "- Counting Frequencies:\n",
        "  - After tokenization, CountVectorizer builds a vocabulary of all unique words (or n-grams) in the documents and counts the occurrence of each word in every document.\n",
        "\n",
        "- Vectorization:\n",
        "  - The result is a sparse matrix where each row represents a document and each column represents a word in the vocabulary. The values in the matrix are the word counts (or frequencies) in each document.\n",
        "\n",
        "- Features:\n",
        "  - CountVectorizer supports several parameters, including:\n",
        "    - max_features: Limits the vocabulary size by selecting the most frequent words.\n",
        "    - stop_words: Filters out common words (e.g., \"and\", \"the\") which may not be informative.\n",
        "    - ngram_range: Specifies the range of n-grams to consider (e.g., unigrams, bigrams).\n",
        "    - binary: If true, it uses binary occurrence instead of frequency.\n",
        "\n",
        "\n",
        "Below is and illustration of how CountVectorizer works.\n",
        "\n",
        "<div align=\"center\" style=\" font-size: 80%; text-align: center; margin: 0 auto\">\n",
        "<img src=\"https://www.researchgate.net/publication/354354484/figure/fig2/AS:1080214163595268@1634554534648/Illustration-of-count-vectorization.jpg\"  style=\"display: block; margin-left: auto; margin-right: auto;\";/>\n",
        "</div>\n",
        "\n",
        "\n",
        "    Code Example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "id": "kbTWFr72QaSS",
        "outputId": "2db86f35-4b49-4606-ee0a-7da1aad85fdf"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"df\",\n  \"rows\": 3,\n  \"fields\": [\n    {\n      \"column\": \"another\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1,\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"document\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 1,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"example\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"is\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sample\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"this\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "df"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-4670fca4-80ec-44c4-873b-669805e97ade\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>another</th>\n",
              "      <th>document</th>\n",
              "      <th>example</th>\n",
              "      <th>is</th>\n",
              "      <th>sample</th>\n",
              "      <th>short</th>\n",
              "      <th>this</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4670fca4-80ec-44c4-873b-669805e97ade')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-4670fca4-80ec-44c4-873b-669805e97ade button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-4670fca4-80ec-44c4-873b-669805e97ade');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-6096f1a9-15da-442e-9dd9-e1da65c9365b\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-6096f1a9-15da-442e-9dd9-e1da65c9365b')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-6096f1a9-15da-442e-9dd9-e1da65c9365b button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "   another  document  example  is  sample  short  this\n",
              "0        0         1        0   1       1      0     1\n",
              "1        1         1        1   0       0      0     0\n",
              "2        0         1        0   1       0      1     1"
            ]
          },
          "execution_count": 1,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "\n",
        "# Sample documents\n",
        "documents = [\n",
        "    \"This is a sample document.\",\n",
        "    \"Another example document.\",\n",
        "    \"This document is short.\"\n",
        "]\n",
        "\n",
        "# Create CountVectorizer object\n",
        "vectorizer = CountVectorizer()\n",
        "\n",
        "# Learn vocabulary and transform documents into a document-term matrix\n",
        "X = vectorizer.fit_transform(documents)\n",
        "\n",
        "# Get feature names (vocabulary)\n",
        "feature_names = vectorizer.get_feature_names_out()\n",
        "\n",
        "# Convert matrix to array and display as DataFrame\n",
        "import pandas as pd\n",
        "df = pd.DataFrame(X.toarray(), columns=feature_names)\n",
        "df\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PN8WxZ-Ccsz1",
        "outputId": "3ed5e1e1-a1fc-4ee6-93d7-157d8f989066"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Matrix X (token counts):\n",
            "[[0 1 0 1 1 0 1]\n",
            " [1 1 1 0 0 0 0]\n",
            " [0 1 0 1 0 1 1]]\n"
          ]
        }
      ],
      "source": [
        "# Print the matrix X (token counts)\n",
        "print(\"Matrix X (token counts):\")\n",
        "print(X.toarray())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MU8VxR6EeOkl"
      },
      "source": [
        "## 2. How to Compute and Print Accuracy\n",
        "\n",
        "In machine learning, accuracy is a metric used to evaluate the performance of a classification model. It measures the proportion of correctly predicted instances (or observations) out of the total number of instances.\n",
        "\n",
        "the accuracy of a classification model can be calculated as:\n",
        "\n",
        "<div align=\"center\", style=\" font-size: 80%\">\n",
        "Accuracy = $\\frac{Number of Correct Predictions}{Total Number of Predictions} = \\frac{TP+TN}{TP+TN+FP+FN}$\n",
        "</div>\n",
        "\n",
        "In simpler terms:\n",
        "\n",
        "- Accuracy measures the proportion of correctly predicted instances (both positive and negative) out of the total number of instances.\n",
        "- A higher accuracy score indicates that the model is making more correct predictions.\n",
        "\n",
        "*However, accuracy might not always be the most suitable metric, especially when dealing with imbalanced datasets (where one class is much more frequent than the other). In such cases, accuracy alone can be misleading. For example, if 90% of the instances belong to one class, a model that predicts that class for all instances would achieve 90% accuracy, but it would not be useful in practice.*\n",
        "\n",
        "    Code Example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R8tPG-Z6dtC8",
        "outputId": "2312621b-e275-4619-c560-cf6c5226fddf"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Accuracy: 0.82\n"
          ]
        }
      ],
      "source": [
        "from sklearn.datasets import make_classification\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "# Generate synthetic dummy data for binary classification\n",
        "X, y = make_classification(n_samples=1000, n_features=20, n_informative=10, n_classes=2, random_state=42)\n",
        "\n",
        "# Split the data into training and testing sets (80% train, 20% test)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Initialize and train a logistic regression model\n",
        "model = LogisticRegression()\n",
        "model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing set\n",
        "y_pred = model.predict(X_test)\n",
        "\n",
        "# Compute accuracy\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "\n",
        "# Print the accuracy score\n",
        "print(\"Accuracy:\", accuracy)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4M_enTmgr9wj"
      },
      "source": [
        "## 3. Interpreting Confusion Matrix\n",
        "\n",
        "\n",
        "Interpreting a confusion matrix is essential for understanding the performance of a classification model. A confusion matrix is a table that summarizes the performance of a classification model on a set of test data for which the true values are known. It provides a breakdown of predicted versus actual class labels.\n",
        "\n",
        "Let's break down a confusion matrix and understand how to interpret its components using a typical binary classification scenario:\n",
        "\n",
        "**Components of a Confusion Matrix**\n",
        "\n",
        "Consider a binary classification problem where we have two classes: \"Positive\" (P) and \"Negative\" (N).\n",
        "\n",
        "- True Positive (TP): Instances that belong to the positive class (P) and are correctly predicted as positive by the model.\n",
        "\n",
        "- False Positive (FP): Instances that belong to the negative class (N) but are incorrectly predicted as positive by the model (Type I error).\n",
        "\n",
        "- True Negative (TN): Instances that belong to the negative class (N) and are correctly predicted as negative by the model.\n",
        "\n",
        "- False Negative (FN): Instances that belong to the positive class (P) but are incorrectly predicted as negative by the model (Type II error).\n",
        "\n",
        "<div align=\"center\">\n",
        "\n",
        "\n",
        "                  Predicted Class\n",
        "\n",
        "|          | Positive (1) | Negative (0)|\n",
        "|:--------:|:--------:|:--------:|\n",
        "|  **Positive (1)**   |  TP   |  FN   |\n",
        "|  **Negative (0)**   |  FP   |  TN   |\n",
        "\n",
        "</div>\n",
        "\n",
        "\n",
        "Note: The **above table** is **NOT** the way Scikit-Learn interpretes Confusion Matrix\n",
        "\n",
        "**Scikit-Learn** Interpretes Confusion Matrix in the table **below**\n",
        "\n",
        "<div align=\"center\">\n",
        "\n",
        "                    Predicted Class\n",
        "\n",
        "|          | Negative (0) | Positive (1)|\n",
        "|:--------:|:--------:|:--------:|\n",
        "|  **Negative (0)**   |  TN   |  FP   |\n",
        "|  **Positive (1)**   |  FN   |  TP   |\n",
        "\n",
        "</div>\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "- TP (True Positive): The number of instances correctly predicted as positive.\n",
        "\n",
        "- FN (False Negative): The number of instances incorrectly predicted as negative when they are actually positive.\n",
        "\n",
        "- FP (False Positive): The number of instances incorrectly predicted as positive when they are actually negative.\n",
        "\n",
        "- TN (True Negative): The number of instances correctly predicted as negative.\n",
        "\n",
        "\n",
        "    Code Example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TSsGaALDsVpx",
        "outputId": "15bc0aa9-7b4c-48ef-83f8-a80c0e96b3bb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Confusion Matrix:\n",
            "[[81 23]\n",
            " [13 83]]\n",
            "\n",
            "True Positive (TP): 83\n",
            "False Positive (FP): 23\n",
            "True Negative (TN): 81\n",
            "False Negative (FN): 13\n"
          ]
        }
      ],
      "source": [
        "from sklearn.datasets import make_classification\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import confusion_matrix\n",
        "\n",
        "# Generate synthetic dummy data for binary classification\n",
        "X, y = make_classification(n_samples=1000, n_features=20, n_informative=10, n_classes=2, random_state=42)\n",
        "\n",
        "# Split the data into training and testing sets (80% train, 20% test)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Initialize and train a logistic regression model\n",
        "model = LogisticRegression()\n",
        "model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing set\n",
        "y_pred = model.predict(X_test)\n",
        "\n",
        "# Compute confusion matrix\n",
        "cm = confusion_matrix(y_test, y_pred)\n",
        "\n",
        "# Extract TP, FP, TN, FN from confusion matrix\n",
        "TN = cm[0][0]\n",
        "FP = cm[0][1]\n",
        "FN = cm[1][0]\n",
        "TP = cm[1][1]\n",
        "\n",
        "# Print confusion matrix\n",
        "print(\"Confusion Matrix:\")\n",
        "print(cm)\n",
        "\n",
        "print()\n",
        "# Print TP, FP, TN, FN\n",
        "print(\"True Positive (TP):\", TP)\n",
        "print(\"False Positive (FP):\", FP)\n",
        "print(\"True Negative (TN):\", TN)\n",
        "print(\"False Negative (FN):\", FN)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xsA7wcYpyySH"
      },
      "source": [
        "## 4. Hyperparameter Tuning\n",
        "\n",
        "Hyperparameter tuning is the process of finding the best set of hyperparameters for a machine learning algorithm to optimize its performance on a given dataset.\n",
        "\n",
        "Hyperparameter tuning can be approached using several techniques, each with its own advantages and trade-offs. Here are some common types of hyperparameter tuning methods:\n",
        "\n",
        "- **Grid Search:**\n",
        " - Method: Involves defining a grid of hyperparameter values to search **exhaustively**.\n",
        " - Process: Iterates through all combinations of hyperparameters specified in the grid and evaluates each combination using cross-validation.\n",
        " - Pros:\n",
        "    - Guarantees finding the best combination of hyperparameters within the specified grid.\n",
        "    - Simple and straightforward to implement.\n",
        " - Cons:\n",
        "    - Can be computationally expensive, especially with a large number of hyperparameters and values.\n",
        "    - May not be efficient for continuous or large search spaces.\n",
        "\n",
        "- **Randomized Search:**\n",
        "  - Method: Involves sampling random combinations of hyperparameter values from specified distributions.\n",
        "  - Process: Randomly selects a **fixed number** of combinations and evaluates them using cross-validation.\n",
        "  - Pros:\n",
        "    - More efficient for large search spaces compared to grid search.\n",
        "    - Can handle both discrete and continuous hyperparameters.\n",
        "  - Cons:\n",
        "    - Does not guarantee finding the optimal solution as it relies on random sampling.\n",
        "    - The number of iterations may need tuning to balance exploration and exploitation.\n",
        "\n",
        "Other types of hyperparameter tuning includes;\n",
        "- Bayesian Optimization:\n",
        "- Evolutionary Algorithms:\n",
        "- Gradient-Based Optimization:\n",
        "\n",
        "Each of these methods has its strengths and weaknesses, and the choice of approach depends on factors such as the complexity of the hyperparameter space, the computational budget, and the characteristics of the objective function. Experimentation with different techniques and careful tuning of parameters are often required to achieve optimal performance in hyperparameter tuning.\n",
        "\n",
        "**Image illustrating Grid Search and Random Search;**\n",
        "\n",
        "<div align=\"center\" style=\" font-size: 80%; text-align: center; margin: 0 auto\">\n",
        "<img src=\"https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1bf0f3af-22ce-46b1-a0a5-d34ee4544514_772x441.png\"  style=\"display: block; margin-left: auto; margin-right: auto;\";/>\n",
        "</div>\n",
        "\n",
        "The random search algorithm randomly samples hyperparameter values from the defined distributions and then tests them by generating a model. Like grid search, random search uses a pre-defined metric to determine the best set of hyperparameter values.  Random search has been found to be more efficient for hyperparameter optimization - both in theory and in practice. Random search effectively searches a larger configuration space than grid search.\n",
        "\n",
        "\n",
        "\n",
        "Part of the reason random search typically outperforms grid search is that typically, only a few hyperparameters really matter for a given dataset, and finding the optimal values for these dominant hyperparameters will have more impact than getting an optimal combination of all hyperparameters. The hyperparameters that are important are different across datasets, so there is not a way of knowing which hyperparameters matter most for your specific dataset (i.e., there is no free lunch). Random search is more likely than grid search to find the optimal value for the important hyperparameter values because it searches a larger area for hyperparameter values (given the same computational budget).\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "<div align=\"center\" style=\" font-size: 80%; text-align: center; margin: 0 auto\">\n",
        "<img src=\"https://community.alteryx.com/t5/image/serverpage/image-id/74545i97245FDAA10376E9/image-size/large?v=v2&px=999\"  style=\"display: block; margin-left: auto; margin-right: auto;\";/>\n",
        "</div>\n",
        "\n",
        "    Code Example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rc26jVA_xkJa",
        "outputId": "13cad7ae-76fa-4737-c338-6a5f681feb76"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Best Parameters: {'C': 10, 'gamma': 0.01, 'kernel': 'rbf'}\n",
            "Best Accuracy: 0.94375\n"
          ]
        }
      ],
      "source": [
        "from sklearn.datasets import make_classification\n",
        "from sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV\n",
        "from sklearn.svm import SVC\n",
        "\n",
        "# Generate synthetic dummy data for binary classification\n",
        "X, y = make_classification(n_samples=1000, n_features=20, n_informative=10, n_classes=2, random_state=42)\n",
        "\n",
        "# Split the data into training and testing sets\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Define the parameter grid to search\n",
        "param_grid = {\n",
        "    'C': [0.1, 1, 10, 100],\n",
        "    'kernel': ['linear', 'rbf'],\n",
        "    'gamma': [0.1, 0.01, 0.001]\n",
        "}\n",
        "\n",
        "# Initialize the Support Vector Classifier (SVC)\n",
        "svc = SVC()\n",
        "\n",
        "# Initialize GridSearchCV with the SVC classifier and parameter grid\n",
        "grid_search = GridSearchCV(estimator=svc, param_grid=param_grid, cv=5, scoring='accuracy')\n",
        "\n",
        "# Perform grid search to find the best hyperparameters\n",
        "grid_search.fit(X_train, y_train)\n",
        "\n",
        "# Get the best hyperparameters and corresponding score\n",
        "best_params = grid_search.best_params_\n",
        "best_score = grid_search.best_score_\n",
        "\n",
        "print(\"Best Parameters:\", best_params)\n",
        "print(\"Best Accuracy:\", best_score)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "loozkJNP7hHW"
      },
      "source": [
        "**Total number of iteration for GridSearchCV**\n",
        "\n",
        "\n",
        "    'C': 4 values to iterate,\n",
        "    'kernel': 2 values to iterate,\n",
        "    'gamma': 3 values to iterate\n",
        "\n",
        "Total number of iterateion = 4 x 2 x 3 = 24 models\n",
        "\n",
        "Each model will be trained on 5 folds of the data\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 228
        },
        "id": "10w5YRz35yrn",
        "outputId": "edcb122f-6a0e-4288-9cb9-50914f82720a"
      },
      "outputs": [
        {
          "ename": "NameError",
          "evalue": "name 'svc' is not defined",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-7-513e96db9fb2>\u001b[0m in \u001b[0;36m<cell line: 6>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;31m# Initialize RandomizedSearchCV with the SVC classifier and parameter distribution\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0mrandom_search\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mRandomizedSearchCV\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mestimator\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msvc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparam_distributions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mparam_grid\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn_iter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcv\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mscoring\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'accuracy'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrandom_state\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m42\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;31m# Perform random search to find the best hyperparameters\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'svc' is not defined"
          ]
        }
      ],
      "source": [
        "# Implementing RandomSearch CV\n",
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "\n",
        "\n",
        "# Initialize RandomizedSearchCV with the SVC classifier and parameter distribution\n",
        "random_search = RandomizedSearchCV(estimator=svc, param_distributions=param_grid, n_iter=10, cv=5, scoring='accuracy', random_state=42)\n",
        "\n",
        "# Perform random search to find the best hyperparameters\n",
        "random_search.fit(X_train, y_train)\n",
        "\n",
        "# Get the best hyperparameters and corresponding score\n",
        "best_params = random_search.best_params_\n",
        "best_score = random_search.best_score_\n",
        "\n",
        "print(\"Best Parameters:\", best_params)\n",
        "print(\"Best Accuracy:\", best_score)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZmP5ZLuNfYLu"
      },
      "source": [
        "## 5. Modify the param_grid for a GreadSearchCV\n",
        "\n",
        "To modify the param_grid of an existing GridSearchCV object;\n",
        "- Add the new hyperparameter to the param_grid dictionary\n",
        "- fit the existing GridSearchCV object to the training data.\n",
        "\n",
        "Note: You do **NOT** need to re-instantiate the GridSearchCV\n",
        "\n",
        "\n",
        "    Code Example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-7Ge9dlNfUwp",
        "outputId": "4fa16300-57bd-4ab9-d841-d333e6845ca2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Best Parameters: {'C': 1, 'gamma': 0.01, 'kernel': 'rbf'}\n",
            "Best Cross-Validation Score: 0.9412499999999999\n"
          ]
        }
      ],
      "source": [
        "from sklearn.datasets import make_classification\n",
        "from sklearn.model_selection import train_test_split, GridSearchCV\n",
        "from sklearn.svm import SVC\n",
        "\n",
        "# Generate synthetic dummy data for binary classification\n",
        "X, y = make_classification(n_samples=1000, n_features=20, n_informative=10, n_classes=2, random_state=42)\n",
        "\n",
        "# Split the data into training and testing sets\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Define the initial parameter grid for GridSearchCV\n",
        "param_grid = {\n",
        "    'C': [0.1, 1, ],\n",
        "    'kernel': ['linear', 'rbf'],\n",
        "}\n",
        "\n",
        "# Initialize the Support Vector Classifier (SVC)\n",
        "svc = SVC(random_state=42)\n",
        "\n",
        "# Initialize GridSearchCV with the SVC classifier and initial param_grid\n",
        "grid_search = GridSearchCV(estimator=svc, param_grid=param_grid, cv=5, scoring='accuracy', )\n",
        "\n",
        "# Update the para_grid with the new hyperparameter\n",
        "param_grid['gamma'] = [0.1, 0.01]\n",
        "\n",
        "\n",
        "# Fit the GridSearchCV again with the modified param_grid\n",
        "grid_search.fit(X_train, y_train)\n",
        "\n",
        "# Get the best hyperparameters and corresponding cross-validation score\n",
        "best_params = grid_search.best_params_\n",
        "best_score = grid_search.best_score_\n",
        "\n",
        "print(\"Best Parameters:\", best_params)\n",
        "print(\"Best Cross-Validation Score:\", best_score)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SlX58ATprrvM"
      },
      "source": [
        "## 6. Adjusting a Hyperparameter value of an SVC Model\n",
        "\n",
        "    Code Example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SW3XfjC6rqMn",
        "outputId": "d5ba69e0-cf2a-4893-8b29-f0123b37194a"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'C': 0.5,\n",
              " 'break_ties': False,\n",
              " 'cache_size': 200,\n",
              " 'class_weight': None,\n",
              " 'coef0': 0.0,\n",
              " 'decision_function_shape': 'ovr',\n",
              " 'degree': 3,\n",
              " 'gamma': 'scale',\n",
              " 'kernel': 'rbf',\n",
              " 'max_iter': -1,\n",
              " 'probability': False,\n",
              " 'random_state': None,\n",
              " 'shrinking': True,\n",
              " 'tol': 0.001,\n",
              " 'verbose': False}"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from sklearn.datasets import load_iris\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "# Load the Iris dataset\n",
        "iris = load_iris()\n",
        "X = iris.data\n",
        "y = iris.target\n",
        "\n",
        "# Split the data into training and test sets\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Initialize an SVC model with default hyperparameters\n",
        "svc_model = SVC(C=1.0, kernel='rbf', gamma='scale')\n",
        "\n",
        "\n",
        "\n",
        "# Update the value of C (adjusting the regularization parameter)\n",
        "svc_model.set_params(C=0.5)\n",
        "\n",
        "\n",
        "# train the model with updated hyperparameters on the entire training set\n",
        "svc_model.fit(X_train, y_train)\n",
        "\n",
        "# View the model Parameters\n",
        "svc_model.get_params()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6NdouxCU8z7L"
      },
      "source": [
        "## 7. Clasification Report\n",
        "\n",
        "A classification report is a method of evaluating the performance of a classification model by providing a summary of key metrics for each class in the dataset. It is particularly useful when dealing with classification tasks where the goal is to assign an instance to one of multiple predefined classes or categories.\n",
        "\n",
        "The classification report typically includes the following metrics for each class:\n",
        "\n",
        "- Precision:\n",
        "  - Precision measures the proportion of true positive predictions (correctly predicted instances) **among all instances predicted as belonging to a particular class**. It is calculated as:\n",
        "  - *Precision* = $\\frac{True Positives}{True Positives + False Positives}$\n",
        "  - Precision indicates the accuracy of positive predictions for a specific class. A high precision value suggests that **when the model predicts a positive class, it is likely to be correct**.\n",
        "\n",
        "- Recall (Sensitivity):\n",
        "  - Recall measures the proportion of true positive predictions (correctly predicted instances) **among all actual instances that belong to a particular class**. It is calculated as:\n",
        "  - *Recall* = $\\frac{True Positives}{True Positives+False Negatives}$\n",
        "  - Recall indicates the ability of the model to correctly identify all instances of a specific class. A high recall value suggests that **the model is sensitive to the presence of the class**.\n",
        "\n",
        "- F1-Score:\n",
        "  - The F1-score is the harmonic mean of precision and recall. It provides a balance between precision and recall and is particularly useful when classes are imbalanced. It is calculated as:\n",
        "  - *F1-Score* = 2 × $\\frac{Precision× Recall}{Precision+Recall}$\n",
        "  - The F1-score reaches its best value at 1 (perfect precision and recall) and its worst value at 0.\n",
        "\n",
        "- Support:\n",
        "  - Support represents the number of actual occurrences of each class in the dataset. It is the number of true instances for each class.\n",
        "\n",
        "\n",
        "    Code Example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LxMs2sBy6aKK",
        "outputId": "c6dcc969-e1b5-4f6d-fe67-7a4222644998"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "NOT Efficient Coding Practice because it is assigned to a variable report and saved to memory\n",
            "Classification Report:\n",
            "               precision    recall  f1-score   support\n",
            "\n",
            "           0       0.98      0.94      0.96       104\n",
            "           1       0.94      0.98      0.96        96\n",
            "\n",
            "    accuracy                           0.96       200\n",
            "   macro avg       0.96      0.96      0.96       200\n",
            "weighted avg       0.96      0.96      0.96       200\n",
            "\n",
            "\n",
            "Efficient Coding Practice because it just prints and not using memory\n",
            "Classification Report:\n",
            "               precision    recall  f1-score   support\n",
            "\n",
            "           0       0.98      0.94      0.96       104\n",
            "           1       0.94      0.98      0.96        96\n",
            "\n",
            "    accuracy                           0.96       200\n",
            "   macro avg       0.96      0.96      0.96       200\n",
            "weighted avg       0.96      0.96      0.96       200\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from sklearn.datasets import make_classification\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.metrics import classification_report\n",
        "\n",
        "# Generate synthetic dummy data for binary classification\n",
        "X, y = make_classification(n_samples=1000, n_features=20, n_informative=10, n_classes=2, random_state=42)\n",
        "\n",
        "# Split the data into training and testing sets\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Initialize the Support Vector Classifier (SVC)\n",
        "svc = SVC(C=1.0, kernel='rbf', gamma='scale', random_state=42)\n",
        "\n",
        "# Train the SVC classifier on the training data\n",
        "svc.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the test data\n",
        "y_pred = svc.predict(X_test)\n",
        "\n",
        "# Print the classification report (NOT Efficient Coding Practice)\n",
        "print(\"NOT Efficient Coding Practice because it is assigned to a variable report and saved to memory\")\n",
        "report = classification_report(y_test, y_pred)\n",
        "print(\"Classification Report:\\n\", report)\n",
        "\n",
        "print()\n",
        "# Print the classification report (Efficient Coding Practice)\n",
        "print(\"Efficient Coding Practice because it just prints and not using memory\")\n",
        "print(\"Classification Report:\\n\", classification_report(y_test, y_pred))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X_wt6p9HHYhq"
      },
      "source": [
        "## 8. Misclassified Samples\n",
        "\n",
        "The number of misclassified samples of a model refers to the count of instances in the test set that were incorrectly predicted by the model. These are the instances where the predicted label (output from the model) does not match the true label (actual ground truth label).\n",
        "\n",
        "To calculate the number of misclassified samples, you can compare the predicted labels (y_pred) with the true labels (y_true) and count the instances where they differ.\n",
        "\n",
        "\n",
        "    Code Example"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xs2xbmUM9IpS",
        "outputId": "a99e3c2a-1a07-4142-88c1-7b1c459ab812"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Number of misclassified samples: 36\n"
          ]
        }
      ],
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "def train_predict_and_report(X_train, y_train, X_test, y_test):\n",
        "    # Step i: Train the model\n",
        "    model = LogisticRegression()  # Example: Using Logistic Regression\n",
        "    model.fit(X_train, y_train)\n",
        "\n",
        "    # Step ii: Predict the test set labels\n",
        "    y_pred = model.predict(X_test)\n",
        "\n",
        "    # Step iii: Count the number of misclassified samples\n",
        "    misclassified_samples = (y_pred != y_test).sum()\n",
        "\n",
        "    # Step iv: Print the result\n",
        "    print(f\"Number of misclassified samples: {misclassified_samples}\")\n",
        "\n",
        "# Example usage:\n",
        "# Assuming X_train, X_test, y_train, y_test are already defined\n",
        "\n",
        "# Split data into training and test sets\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Call the function to train, predict, and print results\n",
        "train_predict_and_report(X_train, y_train, X_test, y_test)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ov9NyQmueKdA"
      },
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NMclpjWTvBU_"
      },
      "source": [
        "## 9. Precision Recall Trade-offs\n",
        "\n",
        "Understanding the trade-offs between precision and recall is crucial in evaluating and optimizing classification models, especially in scenarios where the class distribution is imbalanced or the cost of **false positives** and **false negatives** varies.\n",
        "**Definitions**\n",
        "\n",
        "- Precision: **The proportion of predicted positive instances that are actually positive**. It measures the accuracy of positive predictions.\n",
        "<div align=\"center\">\n",
        "Precision = $\\frac{TP}{TP+FP}$\n",
        "</div>\n",
        "\n",
        "- Recall (Sensitivity): **The proportion of actual positive instances that are correctly predicted as positive**. It measures the ability of the model to identify all positive instances.\n",
        "<div align=\"center\">\n",
        "Recall= $\\frac{TP}{TP+FN}$\n",
        "</div>\n",
        "\n",
        "**Precision-Recall Trade-off**\n",
        "\n",
        "- Precision-Recall Curve: A graphical representation that shows the trade-off between precision and recall at different thresholds.\n",
        "  - Each point on the curve represents a different threshold for predicting positive class labels.\n",
        "  - Moving the threshold for classification affects both precision and recall.\n",
        "​\n",
        "\n",
        "**Trade-offs and Use Cases**\n",
        "\n",
        "- High Precision:\n",
        "  - Suitable when the cost of **false positives is high** (e.g., in medical diagnostics where a false positive could lead to unnecessary treatments).\n",
        "  - Focuses on **minimizing false positive** at the expense of potentially missing some true positives.\n",
        "\n",
        "- High Recall:\n",
        "  - Important when the cost of **false negatives is high** (e.g., in fraud detection where missing a fraudulent transaction is costly).\n",
        "  - Focuses on **minimizing false negative** at the expense of potentially missing some true negative.\n",
        "  - Emphasizes capturing as many positive instances as possible, even at the risk of higher false positives.\n",
        "\n",
        "**Adjusting the Trade-off**\n",
        "\n",
        "In the context of classification models, the threshold refers to a value used to make a decision about the predicted class labels based on the model's output probabilities or scores. When a classifier predicts the probability of a sample belonging to a certain class (e.g., positive class), a threshold is applied to convert these probabilities into discrete class predictions.\n",
        "\n",
        "- Threshold Adjustment:\n",
        "  - By changing the classification threshold, you can adjust the trade-off between precision and recall.\n",
        "  - Lowering the threshold increases recall but may decrease precision, and vice versa.\n",
        "\n",
        "- Model Selection:\n",
        "  - Choose a model based on the specific requirements and priorities of the application.\n",
        "  - Use precision, recall, F1-score, and other evaluation metrics to compare and select the best model for the given use case.\n",
        "\n",
        "\n",
        "<div align=\"center\" style=\" font-size: 80%; text-align: center; margin: 0 auto\">\n",
        "<img src=\"https://miro.medium.com/v2/resize:fit:1248/1*TqzfzabXrej1FTdZuNNYIQ.png\"  style=\"display: block; margin-left: auto; margin-right: auto;\";/>\n",
        "</div>\n",
        "\n",
        "\n",
        "**Practical Considerations**\n",
        "\n",
        "- Imbalanced Classes:\n",
        "  - In scenarios with imbalanced classes, optimizing solely for accuracy may not be sufficient.\n",
        "  - Consider using precision, recall, or F1-score to assess model performance more comprehensively.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "    Code Example\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 564
        },
        "id": "_VdmY2LEHlyX",
        "outputId": "c5efb61f-1cbc-489c-c32c-3c433626cf9b"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAArMAAAIjCAYAAAAQgZNYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABUT0lEQVR4nO3deXxTVf7/8Xe6t7SlLC1lqZQdQQQF4YfIIrJWURhURhCBUUSBUamCoEJBR5AREXVAcGEZv4wguKGUpVZRQWZUBEZl3/cCKrRQ2qbN+f2RaSC0ha5JL7yej8d9JLk5N/dzcxp49/TkXpsxxggAAACwIB9vFwAAAAAUF2EWAAAAlkWYBQAAgGURZgEAAGBZhFkAAABYFmEWAAAAlkWYBQAAgGURZgEAAGBZhFkAAABYFmEWwFVj8ODBio2NLdI2a9askc1m05o1a8qkJqvr1KmTOnXq5Hq8b98+2Ww2zZ8/32s1Abi6EGYBlJn58+fLZrO5lqCgIDVs2FAjR45USkqKt8sr93KDYe7i4+OjypUrq2fPnlq/fr23yysVKSkpeuqpp9S4cWOFhISoQoUKatmypf72t7/p1KlT3i4PgAX4ebsAAFe+559/XnXq1FFGRobWrl2rN998U4mJifrll18UEhLisTrefvttORyOIm3ToUMHnTt3TgEBAWVU1eXdd999iouLU05Ojnbs2KFZs2bp1ltv1Q8//KBmzZp5ra6S+uGHHxQXF6czZ87o/vvvV8uWLSVJP/74o1566SV98803Wr16tZerBFDeEWYBlLmePXuqVatWkqSHHnpIVapU0fTp0/Xpp5/qvvvuy3ebs2fPqkKFCqVah7+/f5G38fHxUVBQUKnWUVQ33nij7r//ftfj9u3bq2fPnnrzzTc1a9YsL1ZWfKdOnVKfPn3k6+urjRs3qnHjxm7Pv/jii3r77bdLZV9l8bMEoPxgmgEAj+vcubMkae/evZKcc1lDQ0O1e/duxcXFKSwsTAMGDJAkORwOzZgxQ02bNlVQUJCqVaumYcOG6Y8//sjzuitWrFDHjh0VFham8PBw3XTTTfrXv/7lej6/ObOLFi1Sy5YtXds0a9ZMr732muv5gubMLlmyRC1btlRwcLCqVq2q+++/X4cPH3Zrk3tchw8fVu/evRUaGqrIyEg99dRTysnJKfb71759e0nS7t273dafOnVKTzzxhGJiYhQYGKj69etr6tSpeUajHQ6HXnvtNTVr1kxBQUGKjIxUjx499OOPP7razJs3T507d1ZUVJQCAwPVpEkTvfnmm8Wu+WJz5szR4cOHNX369DxBVpKqVaum5557zvXYZrNp4sSJedrFxsZq8ODBrse5U1u+/vprDR8+XFFRUapVq5aWLl3qWp9fLTabTb/88otr3bZt23T33XercuXKCgoKUqtWrbRs2bKSHTSAMsHILACPyw1hVapUca3Lzs5W9+7ddcstt2jatGmu6QfDhg3T/PnzNWTIED322GPau3ev/vGPf2jjxo1at26da7R1/vz5+stf/qKmTZtq3LhxioiI0MaNG7Vy5Ur1798/3zqSkpJ033336bbbbtPUqVMlSVu3btW6dev0+OOPF1h/bj033XSTpkyZopSUFL322mtat26dNm7cqIiICFfbnJwcde/eXW3atNG0adP0xRdf6JVXXlG9evX06KOPFuv927dvnySpUqVKrnXp6enq2LGjDh8+rGHDhumaa67Rd999p3Hjxuno0aOaMWOGq+2DDz6o+fPnq2fPnnrooYeUnZ2tb7/9Vv/+979dI+hvvvmmmjZtqjvvvFN+fn767LPPNHz4cDkcDo0YMaJYdV9o2bJlCg4O1t13313i18rP8OHDFRkZqQkTJujs2bO6/fbbFRoaqg8++EAdO3Z0a7t48WI1bdpU1113nSTp119/Vbt27VSzZk2NHTtWFSpU0AcffKDevXvrww8/VJ8+fcqkZgDFZACgjMybN89IMl988YU5ceKEOXjwoFm0aJGpUqWKCQ4ONocOHTLGGDNo0CAjyYwdO9Zt+2+//dZIMgsXLnRbv3LlSrf1p06dMmFhYaZNmzbm3Llzbm0dDofr/qBBg0zt2rVdjx9//HETHh5usrOzCzyGr776ykgyX331lTHGmKysLBMVFWWuu+46t319/vnnRpKZMGGC2/4kmeeff97tNW+44QbTsmXLAveZa+/evUaSmTRpkjlx4oQ5duyY+fbbb81NN91kJJklS5a42r7wwgumQoUKZseOHW6vMXbsWOPr62sOHDhgjDHmyy+/NJLMY489lmd/F75X6enpeZ7v3r27qVu3rtu6jh07mo4dO+aped68eZc8tkqVKpnmzZtfss2FJJmEhIQ862vXrm0GDRrkepz7M3fLLbfk6df77rvPREVFua0/evSo8fHxceuj2267zTRr1sxkZGS41jkcDnPzzTebBg0aFLpmAJ7BNAMAZa5Lly6KjIxUTEyM/vznPys0NFQff/yxatas6dbu4pHKJUuWqGLFiuratatOnjzpWlq2bKnQ0FB99dVXkpwjrGlpaRo7dmye+a02m63AuiIiInT27FklJSUV+lh+/PFHHT9+XMOHD3fb1+23367GjRtr+fLlebZ55JFH3B63b99ee/bsKfQ+ExISFBkZqejoaLVv315bt27VK6+84jaquWTJErVv316VKlVye6+6dOminJwcffPNN5KkDz/8UDabTQkJCXn2c+F7FRwc7Lp/+vRpnTx5Uh07dtSePXt0+vTpQtdekNTUVIWFhZX4dQoydOhQ+fr6uq3r16+fjh8/7jZlZOnSpXI4HOrXr58k6ffff9eXX36pe++9V2lpaa738bffflP37t21c+fOPNNJAHgX0wwAlLmZM2eqYcOG8vPzU7Vq1dSoUSP5+Lj/Lu3n56datWq5rdu5c6dOnz6tqKiofF/3+PHjks5PW8j9M3FhDR8+XB988IF69uypmjVrqlu3brr33nvVo0ePArfZv3+/JKlRo0Z5nmvcuLHWrl3rti53TuqFKlWq5Dbn98SJE25zaENDQxUaGup6/PDDD+uee+5RRkaGvvzyS73++ut55tzu3LlT//3vf/PsK9eF71WNGjVUuXLlAo9RktatW6eEhAStX79e6enpbs+dPn1aFStWvOT2lxMeHq60tLQSvcal1KlTJ8+6Hj16qGLFilq8eLFuu+02Sc4pBi1atFDDhg0lSbt27ZIxRuPHj9f48ePzfe3jx4/n+UUMgPcQZgGUudatW7vmYhYkMDAwT8B1OByKiorSwoUL892moOBWWFFRUdq0aZNWrVqlFStWaMWKFZo3b54eeOABLViwoESvnevi0cH83HTTTa6QLDlHYi/8slODBg3UpUsXSdIdd9whX19fjR07VrfeeqvrfXU4HOratavGjBmT7z5yw1ph7N69W7fddpsaN26s6dOnKyYmRgEBAUpMTNSrr75a5NOb5adx48batGmTsrKySnTas4K+SHfhyHKuwMBA9e7dWx9//LFmzZqllJQUrVu3TpMnT3a1yT22p556St27d8/3tevXr1/segGUPsIsgHKrXr16+uKLL9SuXbt8w8mF7STpl19+KXLQCAgIUK9evdSrVy85HA4NHz5cc+bM0fjx4/N9rdq1a0uStm/f7jorQ67t27e7ni+KhQsX6ty5c67HdevWvWT7Z599Vm+//baee+45rVy5UpLzPThz5owr9BakXr16WrVqlX7//fcCR2c/++wzZWZmatmyZbrmmmtc63OndZSGXr16af369frwww8LPD3bhSpVqpTnIgpZWVk6evRokfbbr18/LViwQMnJydq6dauMMa4pBtL5997f3/+y7yWA8oE5swDKrXvvvVc5OTl64YUX8jyXnZ3tCjfdunVTWFiYpkyZooyMDLd2xpgCX/+3335ze+zj46Prr79ekpSZmZnvNq1atVJUVJRmz57t1mbFihXaunWrbr/99kId24XatWunLl26uJbLhdmIiAgNGzZMq1at0qZNmyQ536v169dr1apVedqfOnVK2dnZkqS+ffvKGKNJkyblaZf7XuWOJl/43p0+fVrz5s0r8rEV5JFHHlH16tX15JNPaseOHXmeP378uP72t7+5HterV8817zfXW2+9VeRTnHXp0kWVK1fW4sWLtXjxYrVu3dptSkJUVJQ6deqkOXPm5BuUT5w4UaT9ASh7jMwCKLc6duyoYcOGacqUKdq0aZO6desmf39/7dy5U0uWLNFrr72mu+++W+Hh4Xr11Vf10EMP6aabblL//v1VqVIlbd68Wenp6QVOGXjooYf0+++/q3PnzqpVq5b279+vN954Qy1atNC1116b7zb+/v6aOnWqhgwZoo4dO+q+++5znZorNjZWo0aNKsu3xOXxxx/XjBkz9NJLL2nRokUaPXq0li1bpjvuuEODBw9Wy5YtdfbsWf38889aunSp9u3bp6pVq+rWW2/VwIED9frrr2vnzp3q0aOHHA6Hvv32W916660aOXKkunXr5hqxHjZsmM6cOaO3335bUVFRRR4JLUilSpX08ccfKy4uTi1atHC7AthPP/2k999/X23btnW1f+ihh/TII4+ob9++6tq1qzZv3qxVq1apatWqRdqvv7+//vSnP2nRokU6e/aspk2blqfNzJkzdcstt6hZs2YaOnSo6tatq5SUFK1fv16HDh3S5s2bS3bwAEqXN0+lAODKlnuapB9++OGS7QYNGmQqVKhQ4PNvvfWWadmypQkODjZhYWGmWbNmZsyYMebIkSNu7ZYtW2ZuvvlmExwcbMLDw03r1q3N+++/77afC0/NtXTpUtOtWzcTFRVlAgICzDXXXGOGDRtmjh496mpz8am5ci1evNjccMMNJjAw0FSuXNkMGDDAdaqxyx1XQkKCKcw/v7mnuXr55ZfzfX7w4MHG19fX7Nq1yxhjTFpamhk3bpypX7++CQgIMFWrVjU333yzmTZtmsnKynJtl52dbV5++WXTuHFjExAQYCIjI03Pnj3Nhg0b3N7L66+/3gQFBZnY2FgzdepUM3fuXCPJ7N2719WuuKfmynXkyBEzatQo07BhQxMUFGRCQkJMy5YtzYsvvmhOnz7tapeTk2OefvppU7VqVRMSEmK6d+9udu3aVeCpuS71M5eUlGQkGZvNZg4ePJhvm927d5sHHnjAREdHG39/f1OzZk1zxx13mKVLlxbquAB4js2YS/wNDgAAACjHmDMLAAAAyyLMAgAAwLIIswAAALAswiwAAAAsizALAAAAyyLMAgAAwLKuuosmOBwOHTlyRGFhYbLZbN4uBwAAABcxxigtLU01atSQj8+lx16vujB75MgRxcTEeLsMAAAAXMbBgwdVq1atS7a56sJsWFiYJOebEx4eXub7s9vtWr16tesynLAe+tD66EProw+tjf6zPk/3YWpqqmJiYly57VKuujCbO7UgPDzcY2E2JCRE4eHhfIAtij60PvrQ+uhDa6P/rM9bfViYKaF8AQwAAACWRZgFAACAZRFmAQAAYFlX3ZxZAAA8IScnR3a73dtllAt2u11+fn7KyMhQTk6Ot8tBMZRFH/r7+8vX17fEr0OYBQCglJ05c0aHDh2SMcbbpZQLxhhFR0fr4MGDnOPdosqiD202m2rVqqXQ0NASvQ5hFgCAUpSTk6NDhw4pJCREkZGRhDc5L1h05swZhYaGXvYE+CifSrsPjTE6ceKEDh06pAYNGpRohJYwCwBAKbLb7TLGKDIyUsHBwd4up1xwOBzKyspSUFAQYdaiyqIPIyMjtW/fPtnt9hKFWX6iAAAoA4zIApdWWp8RwiwAAAAsizALAAAAyyLMAgAAr7HZbPrkk09Kva3VrVmzRjabTadOnZIkzZ8/XxEREV6tqbwizAIAAA0ePFg2m002m00BAQGqX7++nn/+eWVnZ5fpfo8ePaqePXuWetuSiI2Ndb0XISEhatasmd55550y329p+OqrrxQXF6cqVaooJCRETZo00ZNPPqnDhw97u7QyQ5gFAACSpB49eujo0aPauXOnnnzySU2cOFEvv/xyvm2zsrJKZZ/R0dEKDAws9bYl9fzzz+vo0aP65ZdfdP/992vo0KFasWKFR/ZdXHPmzFGXLl0UHR2tDz/8UFu2bNHs2bN1+vRpvfLKK8V+3dLq67JCmAUAoAwZI509652lqNdsCAwMVHR0tGrXrq1HH31UXbp00bJlyyQ5R2579+6tF198UTVq1FCjRo0kSQcPHtS9996riIgIVa5cWXfddZf27dvn9rpz585V27ZtFRwcrOrVq2vkyJGu5y6cOpCVlaWRI0eqevXqCgoKUu3atTVlypR820rSzz//rM6dOys4OFhVqlTRww8/rDNnzriez6152rRpql69uqpUqaIRI0YU6spsYWFhio6OVt26dfX000+rcuXKSkpKcj1/6tQpPfTQQ4qMjFR4eLg6d+6szZs3u73GZ599pptuuklBQUGqWrWq+vTp43ruvffeU6tWrVz76d+/v44fP37Zugpy6NAhPfbYY3rsscc0d+5cderUSbGxserQoYPeeecdTZgwQZI0ceJEtWjRwm3bGTNmKDY21vU4v75+9tln1aVLlzz7bd68uZ5//nnX43feeUfXXnutgoKC1LhxY82aNavYx1RYXg2z33zzjXr16qUaNWoUeh7MmjVrdOONNyowMFD169fX/Pnzy7xOAACKKz1dCg31zpKeXrLag4OD3UblkpOTtX37diUlJenzzz+X3W5X9+7dFRYWpm+//Vbr1q1TaGioevTo4druzTff1F//+lcNGjRImzdv1rJly1S/fv189/f6669r2bJl+uCDD7R9+3YtXLjQLWRd6OzZs+revbsqVaqkH374QUuWLNEXX3zhFpQl55/dd+/era+++koLFizQ/Pnzi5QdHA6HPvzwQ/3xxx8KCAhwrb/nnnt0/PhxrVixQhs2bNCNN96o2267Tb///rskafny5erTp4/i4uK0ceNGJScnq3Xr1q7t7Xa7XnjhBW3evFmffPKJ9u3bp8GDBxe6rostWbJEWVlZGjNmTL7PF3W+7cV93b9/f23YsEG7d+92tfn111/13//+V/3795ckLVy4UBMmTNCLL76orVu3avLkyRo/frwWLFhQ7OMqFONFiYmJ5tlnnzUfffSRkWQ+/vjjS7bfs2ePCQkJMfHx8WbLli3mjTfeML6+vmblypWF3ufp06eNJHP69OkSVl84WVlZ5pNPPjFZWVke2R9KH31offSh9VmpD8+dO2e2bNlizp07Z4wx5swZY5xjpJ5fzpwpfN2DBg0yd911lzHGGIfDYZKSkkxgYKB56qmnXM9Xq1bNZGZmurZ57733TKNGjYzD4XCty8zMNMHBwWbVqlXGGGNq1KhhnnnmGfPHH3+YnJycPPu98P//v/71r6Zz585ur1dQ27feestUqlTJnLngIJcvX258fHzMsWPHXDXXrl3bZGdnu9rcc889pl+/fpd8L2rXrm0CAgJMhQoVjJ+fn5FkKleubHbu3GmMMebbb7814eHhJiMjw227evXqmTlz5hhjjGnbtq0ZMGDAJfdzoR9++MFIMmlpacYYY7766isjyfzxxx/GGGPmzZtnKlasWOD2jz76qAkPD7/sfhISEkzz5s3d1r366qumdu3arsf59XVOTo657rrrzKRJk1zrxo0bZ9q0aeN6XK9ePfOvf/3L7bVfeOEF07Zt23xrufizcqGi5DWvXgGsZ8+eRZrIPXv2bNWpU8c17+Paa6/V2rVr9eqrr6p79+5lVWaJbNokrV9fXZmZNvlxvTVLys626aef6EMrK40+9PGROnWS+DIxiiokRLrgL98e33dRfP755woNDZXdbpfD4VD//v01ceJE1/PNmjVzG53cvHmzdu3apbCwMLfXycjI0O7du3X8+HEdOXJEnTt3LtT+Bw8erK5du6pRo0bq0aOH7rjjDnXr1i3ftlu3blXz5s1VoUIF17p27drJ4XBo+/btqlatmiSpadOmbleXql69un7++WdJ0uTJkzV58mTXc1u2bNE111wjSRo9erQGDx6so0ePavTo0Ro+fLhrRHnz5s06c+aMqlSp4lbTuXPnXCOXmzZt0tChQws81g0bNmjixInavHmz/vjjDzkcDknSgQMH1KRJk0K9XxcyxpTqhTou7mvJORr9/vvva8KECTLG6P3331d8fLwk50j57t279eCDD7odd3Z2tipWrFhqdeXHUv81r1+/Ps98je7du+uJJ54ocJvMzExlZma6HqempkpyDu8XZs5MSb3zjvTWW60v3xDlmJ8k+tDaSqcPe/Z06NNPc0peDoos999rT/y7XVK5l7N1OByugOKtq9rmjtEWrq1Rp06dNGvWLAUEBKhGjRry+99vfw6HQ8YYhYSEuI5JktLS0tSyZUu99957eV4vMjLSddlT878ict+Xi+W+Vy1atNDu3bu1YsUKJScn695779Vtt92mJUuW5Gmb+5oXvl7u/Qvb+Pn55dln7vMPP/yw7r77btf66OhoV9sqVaqobt26qlu3rhYvXqzmzZvrxhtvVJMmTZSWlqbq1avryy+/zHMsERERcjgcCg4OdvsZuFDuFIlu3brpvffeU2RkpA4cOKCePXsqIyPDbbvc+xc+zk+DBg10+vRpHT58WNWrV8+3jeScd3xxP+ROCcldl19fG2PUt29fTZw4UT/++KPOnTungwcP6p577pHD4XDlqzlz5qhNmzZu+/T19S2w340x+V7OtiifdUuF2WPHjrl+08pVrVo1paam6ty5c/leA3vKlCmaNGlSnvWrV69WSFF/ZS2GrKx6uvbagn+oAJR/Z8746+DBcG3dmqbExDXeLueqduEXcMorPz8/RUdH68yZM+X+W+AXstvtCgwMVFRUlCQp/aIJt3a7XdnZ2a7QIjn/Qrp48WIFBQUpPDw8z2saY3TNNddo5cqVatWqldLS0vLd97lz59xeN/cvtz179tTdd9+t/fv3q1KlSm5tY2NjNX/+fB09etQ1OpuUlCQfHx/VqFFDqamp+daclZXlWufn5+c63guP2eFwKCMjw7VdxYoV1bt3b40ZM0b/+te/1KhRIx07dkwZGRmukdwLpaamqkmTJlq1apX69u2b5/lNmzbpt99+0zPPPKNatWpJkr799ltJzqCbmprqqiUtLU0+Pj7KyMiQMcbtWC7UrVs3BQQE6MUXX3Qbbc51+vRpVaxYUaGhoTp69KhOnz7tGsn94Ycf3AJpfu+bJNWsWVPt2rXT/Pnzde7cOXXq1ElBQUFKTU11fblv27Zt6tWrV77vycWysrJ07tw5ffPNN3lOAXfxz9+lWCrMFse4ceNcQ+CS882MiYlRt27d8v3glbauXe1KSkpS165d5e/vX+b7Q+mz2+lDqytpH37xhU1xcVJ4eLji4uLKoEJcjpU+hxkZGTp48KBCQ0MVFBTk7XIKzd/fX35+fgX+35jf8w8++KBmzpypQYMGaeLEiapVq5b279+vjz/+WKNHj1atWrU0ceJEDR8+XJGRkbrrrrt05swZfffdd25f1AoODlZ4eLheffVVRUdH64YbbpCPj48SExMVHR2tmJgY1yhvbtsHH3xQU6dO1WOPPaaEhASdOHFC48aN0/333++aDpBfzQEBAZc8Tkny8fHJE9CfeuopXX/99dqxY4fuvPNOtW3bVg888IBeeuklNWzYUEeOHFFiYqJ69+6tVq1aadKkSeratasaN26sfv36KTs7WytWrNCYMWN07bXXKiAgQAsWLNCwYcP0yy+/aPr06ZKkChUqKDw83DXgFhYWpvDwcAUFBclmsxVYd5MmTTR9+nT99a9/VUZGhgYOHKjY2FgdOnRI7733nkJDQzVt2jT16NFDo0eP1pw5c9S3b1+tWrVKycnJCg8Pd712fu+bMUZpaWkaOHCgJk2apKysLL3yyitubSZOnKgnnnhCUVFR6t69uzIzM/Xjjz/q1KlTGjVqVJ6aMzIyFBwcrA4dOuT5rBQU2vNjqTAbHR2tlJQUt3UpKSkKDw/Pd1RWcp5mJL9z0vn7+3v0H0RP7w+ljz60vuL2Ye48W5vNxs+Al1nhc5iTkyObzSYfHx9XALOC3IsEFFRzfs+Hhobqm2++0dNPP627775baWlpqlmzpm677TZFRETIx8dHQ4YMUUZGhl599VWNHz9eVatW1d133+32OrnvVXh4uKZNm6adO3fK19dXN910kxITE13THS5sGxoaqlWrVunxxx9XmzZtFBISor59+2r69Omu186v5tzRyMv1zcXbXXfdderWrZsmTpyoxMREJSYm6tlnn9WDDz6oEydOKDo6Wh06dFD16tXl4+Ojzp07a8mSJXrhhRc0depUhYeHq0OHDvLx8VG1atU0f/58PfPMM3rjjTd04403atq0abrzzjtdx5e77/weF2TEiBFq1KiRpk2bpr59++rcuXOKjY3VHXfcofj4ePn4+Khp06aaNWuWJk+erL/97W/q27evnnrqKb311luXfN9ypwncfffdeuyxx+Tr66s//elPbm0efvhhhYaG6uWXX9aYMWNUoUIFNWvWTE888US+dfv4+Lj+Xb34c12Uz7nNmMLOpilbNptNH3/8sXr37l1gm6efflqJiYmuiduS1L9/f/3+++9auXJlofaTmpqqihUr6vTp0x4ZmbXb7UpMTFRcXFy5/wcY+aMPra+kfZiUJHXrJjVv7vxSJzzPSp/DjIwM7d27V3Xq1LHUyGxZyv0Tdnh4uKUCPs4riz681GelKHnNqz9RZ86c0aZNm7Tpf/877N27V5s2bdKBAwckOacIPPDAA672jzzyiPbs2aMxY8Zo27ZtmjVrlj744IN8h64BAABw5fNqmP3xxx91ww036IYbbpAkxcfH64YbbnBdpeLo0aOuYCtJderU0fLly5WUlKTmzZvrlVde0TvvvFNuT8sFAACAsuXVObOdOnXSpWY55HeFjk6dOmnjxo1lWBUAAACsgokrAAAAsCzCLAAAZaCcfL8aKLdK6zNCmAUAoBTlXsnIShdMALwh9zNy8dW/ispS55kFAKC88/PzU0hIiE6cOCF/f39ORSXnaZ2ysrKUkZHB+2FRpd2HDodDJ06cUEhIiNt5hIuDMAsAQCmy2WyqXr269u7dq/3793u7nHLBGOO67HzuRQtgLWXRhz4+PrrmmmtK/HqEWQAASllAQIAaNGjAVIP/sdvt+uabb9ShQ4dyf9EL5K8s+jAgIKBURnkJswAAlAEfHx+uAPY/vr6+ys7OVlBQEGHWospzHzJxBQAAAJbFyCwAlEN2u3TqVN7ljz/O38/MlIYOlRo39mKhAOBlhFkAKCPp6dLvv0u//ea8vTCIXhxML3589mzh9nHsmLRwYdnUDwBWQJgFgEI6d05au/Z8QM1dLnx84f2MjJLvMyxMqlRJiog4v1SqJO3fL61ZU/jQCwBXKsIsABTSjh1S+/ZF28bPT6pSRapc2RlCLw6mueE0v8fh4c7t8/PWW84wCwBXO8IsAFxGy5ZSgwbSiRPOYJq7VK5c8OPc+2FhEqfVBICyQ5gFgMuoXNk5KgsAKH84NRcAAAAsizALAAAAyyLMAgAAwLIIswAAALAswiwAAAAsizALAAAAyyLMAgAAwLIIswAAALAswiwAAAAsiyuAAcAVJCtLOnJEOnTIfYmOlp5+mkvrArjyEGYBwMI2bpT69DkfWlNSJGPyb9ujh9SihUfLA4AyR5gFAAsKCXHeHjjgXC4UECDVqnV++ewzKS1NOnfO83UCQFkjzAKABfXuLY0fL+XkuAfXWrWkqlXdpxPUq+cMswBwJSLMAoAFhYZKzz/v7SoAwPs4mwEAAAAsizALAAAAyyLMAgAAwLIIswAAALAswiwAAAAsizALAAAAy+LUXACAEktLk/bscd62aSP5+3u7IgBXC8IsAOCyjJGOHXMG1t278y4nTpxv++qr0hNPeK1UAFcZwiwAQJKUnS3t25d/WN2zR0pPv/T2fn7O19i/3yPlAoAkwiwAXFWMkY4ckXbsOL/s3Om83b3bGUYL4uMjxcQ4L4+bu9Ste/7+1KnSlCmeOxYAkAizAHDVGDBAOn5cOnu24DZBQe5h9cKldm0pIMBz9QJAYRBmAeAKV7myc5rA3r3Ox76+Up06UsOGeZeaNZ0jsABgFYRZALjCLVggJSefD7B16nC2AQBXDsIsAFzhmjRxLnByOKTDh8+fmWHPHueyb58UFyc995y3KwRQFIRZAMAV5+xZ57SKCwNr7u3evVJWVv7bbdpEmAWshjALALCkM2ecZ2LIPRvDzp3Srl3OwHrs2KW39fNzfqEt92wMFSs6z8bgcHimdgClhzALACi3MjKcI6pbt9r02Wf1tWyZr3bvdobXo0cvvW1ExPnTh+WG1tz7MTHOQJvrwAFnmAVgPYRZAIBXORzSwYPS1q3S9u3u5749cMB5blznf1dN82xbtarzS20NGpxfckNrpUqePhIA3kCYBQB4hN3unAawdav7sm3bpa8uFh4uNWjgUEjIEXXsWF2NG/u6giuBFQBhFgBQqtLTpR9/dIbUC0Prrl0FX2HM398ZThs3lho1Oj/K2rChFBkpZWfnKDFxg+Li4uTv7+vZAwJQrhFmAQCl6q23nEt+QkOdgfXaa92XunU59y2A4iHMAgBKRePG5+9HRTlD6sXBtVYtyWbzXo0ArjyEWQBAqRg4UGrf3jnHtUoVb1dT9tLSnF9Uy/3S2sGD0qBBzvcAgOcQZgEApcJmc14q90pitzuvDJYbWC+8ze/UYHv3Oi8dfLVJT3cee+6Se3GKypWlOXO8XR2udIRZAAD+JztbGj36fGjdvbvgL61JzukUDRtKvr7S1187z4t7JcrOdo48XxxWc5eUlIK3/ctfpDZtPFcrrj6EWQDAVc/3fydIyMmRpk1zfy442BlYGzZ0nmnhwtuICGebTz5xhlmrMkY6fjz/0dW9e53n+83JufRrVKzoHJmvW9d5+89/SidOXPqXAaA0EGYBAFe9GjWkUaOkLVvcA2ujRlLNmpKPj7crLDljnJf53bXr/KV/c2937XJeHvhSAgKk2Fj3wHrh/YvP+btihTPMAmWNMAsAuOrZbNL06d6uouSMcc7lzQ2oF4fWs2cL3tZmcwb3i0Nq7v3q1a+MUI8rD2EWAAALyQ2sFwfV3NtLXU3Nx0eqXVuqX995UYr69c/fr1NHCgz03HEApYUwCwBAOXTunDOgbtvm/ELatm3OZceOS08JyA2suWH1wtvYWAIrrjyEWQAAvCR3HmtuWL0wtO7f73w+Pz4+zmB6cVitX985whoQ4NHDALyKMAsAQBnLyXGeFWDLFueydev50JqaWvB2ERHOq6jlLo0aOW/r1iWwArkIswAAlJKcHOc0gF9/PR9ct2xxhtaCzkHr4+McTc0NqheG1shILv8LXA5hFgCAUvKf/ziDaH6CgpwBtUkT6dprnUujRs6pAUFBnq0TuJIQZgEAKKEGDc7fDw52BtWmTZ3BNXepU+f8xRkAlB7CLAAAJdS0qbRvn+RwOM8kwPlYAc8hzAIAUApq1/Z2BcDViTALAAAsxW53joTv2OFcfv9dGjZMqlXL25XBGwizAACg3DFGOnLkfGC9cNmzR8rOdm+fkSG9/LJ3aoV3EWYBAIDXnDqVN6xu3+68+tnZswVvFxwsNWzovHzvzp2XvioarmyEWQAAUKbsdunw4VB9+qlNu3a5B9cTJwreztfXeYGIhg3zLjVqOL9oN2mSNHGixw4F5RBhFgAAlJkBA6Tjx/2UnX1bgW1q1Mg/sNatK/n7e7BYWBJhFgAAlLoqVZy3R45Ikk1BQdlq0sRHjRv7qFEj5wUjGjZ0XjQiLMyblcLqCLMAAKDUzZ0rffml82IR9erZ9d//Jur22+Pk789JeFG6CLMAAKDU1a/vXCTnnNmff/bs/o2RbDbP7hPewa9HAADA8n75RRo3TurTx3k54eBgadQob1cFT2BkFgAAWFbu6Ovatc7lQp9/Lr36qudrgmcRZgEAgGX16SMtXy6FhkqNGzuXzExp9GhvVwZPIcwCAADLatZM+s9/3Nd99513aoF3MGcWAAAAlkWYBQAAgGV5PczOnDlTsbGxCgoKUps2bfT9998X2NZut+v5559XvXr1FBQUpObNm2vlypUerBYAAADliVfD7OLFixUfH6+EhAT99NNPat68ubp3767jx4/n2/65557TnDlz9MYbb2jLli165JFH1KdPH23cuNHDlQMAAKA88GqYnT59uoYOHaohQ4aoSZMmmj17tkJCQjR37tx827/33nt65plnFBcXp7p16+rRRx9VXFycXnnlFQ9XDgAAgPLAa2czyMrK0oYNGzRu3DjXOh8fH3Xp0kXr16/Pd5vMzEwFBQW5rQsODtbai08sd9E2mZmZrsepqamSnFMW7HZ7SQ6hUHL34Yl9oWzQh9ZHH1offWhtnu6/7GybnBHHyG7P9sg+r3Se7sOi7MdrYfbkyZPKyclRtWrV3NZXq1ZN27Zty3eb7t27a/r06erQoYPq1aun5ORkffTRR8rJySlwP1OmTNGkSZPyrF+9erVCQkJKdhBFkJSU5LF9oWzQh9ZHH1offWhtnuq/bdsqSeqgs2fPKjEx2SP7vFp4qg/T09ML3dZS55l97bXXNHToUDVu3Fg2m0316tXTkCFDCpyWIEnjxo1TfHy863FqaqpiYmLUrVs3hYeHl3nNdrtdSUlJ6tq1q/z9/ct8fyh99KH10YfWRx9am6f7r1Il52XBKlSooLi4uDLf39XA032Y+5f0wvBamK1atap8fX2VkpLitj4lJUXR0dH5bhMZGalPPvlEGRkZ+u2331SjRg2NHTtWdevWLXA/gYGBCgwMzLPe39/fo/8genp/KH30ofXRh9ZHH1qbp/rPz5VubPy8lDJP9WFR9uG1L4AFBASoZcuWSk4+P/zvcDiUnJystm3bXnLboKAg1axZU9nZ2frwww911113lXW5AAAAKIe8Os0gPj5egwYNUqtWrdS6dWvNmDFDZ8+e1ZAhQyRJDzzwgGrWrKkpU6ZIkv7zn//o8OHDatGihQ4fPqyJEyfK4XBozJgx3jwMAABgEcZIBw5IP/8s/fe/ztvMTGnOHCky0tvVoTi8Gmb79eunEydOaMKECTp27JhatGihlStXur4UduDAAfn4nB88zsjI0HPPPac9e/YoNDRUcXFxeu+99xQREeGlIwAAAOWV3S6tW3c+tObe5jcds1cv6X9jabAYr38BbOTIkRo5cmS+z61Zs8btcceOHbVlyxYPVAUAAKxu/37pllvyrvfzk669VmrWTPr+e2nXLukSJ0ZCOef1MAsAAFCa6teXQkKk9HSpVi1naL3++vO3jRpJAQHOtnfd5QyzsC7CLAAAuKJERUkpKc5pBpUqebsalDXCLAAAuOKEhnq7AniK107NBQAAAJQUYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFlcNAEAAEBSVpa0ZYu0YYN04IA0aJBUt663q8LlEGYBAMBVb8IEacQIZ6DNdfiw9M473qsJhcM0AwAAcNWqWNF5e/SoM8hGREi1azvXpaV5rSwUAWEWAABctSZNkv7+d2nJEmn3bun336Unn/R2VSgKphkAAICrVp060ujR3q4CJcHILAAAACyLMAsAAADLIswCAADAsgizAAAAsCzCLAAAACyLMAsAAADLIswCAADAsgizAAAAsCzCLAAAACyLMAsAAADLIswCAADAsgizAAAAsCzCLAAAACyLMAsAAADLIswCAADAsgizAAAAsCzCLAAAACyLMAsAAADLIswCAADAsgizAAAAsCzCLAAAACzLz9sFAAAAWNHx49K6dVJEhHTrrd6u5upFmAUAALgMY6QdO6S1a50Bdu1aaedO53M2m7Rvn3TNNV4t8apFmAUAAMjHkSPSyy87w+u6ddLJk/m3M8b5HGHWOwizAAAA+Vi71rnkCgqSWreW2rWTbrlFattWatZMOnzYezWCMAsAAOCmTRspMFAKCzsfXG+5RbrxRikgwNvV4WKEWQAAgAu0bi2lpUl+fs75sCjfCLMAAAAX8ff3dgUoLM4zCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAUEJ2u/Tvf0tTpkjduknNm0tbtni7qquDn7cLAAAAsLr27Z2B9kKffy41aeKdeq4mjMwCAAAUU+XKzlu73Xm/Tx/nqCw8h5FZAACAYvq//3NOL2jTRmrWTPLxkYYMkTZv9nZlVw/CLAAAQDFdf71zgfcwzQAAAKAMGCPt3CnNmSP16ydde630z396u6orDyOzAAAAZWDCBGnsWPd1770nPfCAd+q5UjEyCwAAUIoqVXLeZmVJAQFSx47S7bd7t6YrGSOzAAAApejpp6X69aWGDaWbb5ZCQqSFC6Xly71d2ZWJMAsAAFCKqlWThg/3dhVXD6YZAAAAwLIIswAAALAswiwAAAAsizALAAAAyyLMAgAAwLIIswAAALAswiwAAAAsizALAAAAyyLMAgAAwLIIswAAALAswiwAAAAsizALAAAAyyLMAgAAwLK8HmZnzpyp2NhYBQUFqU2bNvr+++8v2X7GjBlq1KiRgoODFRMTo1GjRikjI8ND1QIAAKA88WqYXbx4seLj45WQkKCffvpJzZs3V/fu3XX8+PF82//rX//S2LFjlZCQoK1bt+rdd9/V4sWL9cwzz3i4cgAAAJQHXg2z06dP19ChQzVkyBA1adJEs2fPVkhIiObOnZtv+++++07t2rVT//79FRsbq27duum+++677GguAAAArkx+3tpxVlaWNmzYoHHjxrnW+fj4qEuXLlq/fn2+29x88836v//7P33//fdq3bq19uzZo8TERA0cOLDA/WRmZiozM9P1ODU1VZJkt9tlt9tL6WgKlrsPT+wLZYM+tD760ProQ2uj/6TsbJskPxnjkN2e4+1yiszTfViU/XgtzJ48eVI5OTmqVq2a2/pq1app27Zt+W7Tv39/nTx5UrfccouMMcrOztYjjzxyyWkGU6ZM0aRJk/KsX716tUJCQkp2EEWQlJTksX2hbNCH1kcfWh99aG1Xc/9t3lxLUkudPHlSiYn5D9pZgaf6MD09vdBtvRZmi2PNmjWaPHmyZs2apTZt2mjXrl16/PHH9cILL2j8+PH5bjNu3DjFx8e7HqempiomJkbdunVTeHh4mddst9uVlJSkrl27yt/fv8z3h9JHH1offWh99KG10X/SqVM2SVLVqlUVFxfn5WqKztN9mPuX9MLwWpitWrWqfH19lZKS4rY+JSVF0dHR+W4zfvx4DRw4UA899JAkqVmzZjp79qwefvhhPfvss/LxyTsFODAwUIGBgXnW+/v7e/QD5en9ofTRh9ZHH1offWhtV3P/+f0vcdlsPvL39/rJpIrNU31YlH0UK8zm5ORo/vz5Sk5O1vHjx+VwONye//LLLy/7GgEBAWrZsqWSk5PVu3dvSZLD4VBycrJGjhyZ7zbp6el5Aquvr68kyRhTjCMBAACAlRUrzD7++OOaP3++br/9dl133XWy2WzF2nl8fLwGDRqkVq1aqXXr1poxY4bOnj2rIUOGSJIeeOAB1axZU1OmTJEk9erVS9OnT9cNN9zgmmYwfvx49erVyxVqAQAAcPUoVphdtGiRPvjggxLP+ejXr59OnDihCRMm6NixY2rRooVWrlzp+lLYgQMH3EZin3vuOdlsNj333HM6fPiwIiMj1atXL7344oslqgMAAADWVKwwGxAQoPr165dKASNHjixwWsGaNWvcHvv5+SkhIUEJCQmlsm8AAABYW7FmID/55JN67bXXmKcKAAAAryrWyOzatWv11VdfacWKFWratGmeb5x99NFHpVIcAAAAcCnFCrMRERHq06dPadcCAAAAFEmxwuy8efNKuw4AAACgyEp00YQTJ05o+/btkqRGjRopMjKyVIoCAAAACqNYXwA7e/as/vKXv6h69erq0KGDOnTooBo1aujBBx8s0rV0AQAAgJIoVpiNj4/X119/rc8++0ynTp3SqVOn9Omnn+rrr7/Wk08+Wdo1AgAAAPkq1jSDDz/8UEuXLlWnTp1c6+Li4hQcHKx7771Xb775ZmnVBwAAABSoWCOz6enprqt0XSgqKoppBgAAAPCYYoXZtm3bKiEhQRkZGa51586d06RJk9S2bdtSKw4AAAC4lGJNM3jttdfUvXt31apVS82bN5ckbd68WUFBQVq1alWpFggAAHCl+u03KTxcuuj6UyiCYoXZ6667Tjt37tTChQu1bds2SdJ9992nAQMGKDg4uFQLBAAAuFIYI/30k/TZZ85lwwapeXNp0yZvV2ZdxT7PbEhIiIYOHVqatQAAAFzRkpOlli3d123e7J1arhSFDrPLli1Tz5495e/vr2XLll2y7Z133lniwgAAAK4UVauev1+hgtS1q3TLLdJTT3mvpitFocNs7969dezYMUVFRal3794FtrPZbMrJySmN2gAAAK4IXbtK//qXFBEh3XqrFBQkHT9OmC0NhQ6zDocj3/sAAAC4NB8f6b77vF3FlalYp+bKz6lTp0rrpQAAAIBCKVaYnTp1qhYvXux6fM8996hy5cqqWbOmNjOLGQAAAB5SrDA7e/ZsxcTESJKSkpL0xRdfaOXKlerZs6dGjx5dqgUCAAAABSnWqbmOHTvmCrOff/657r33XnXr1k2xsbFq06ZNqRYIAAAAFKRYI7OVKlXSwYMHJUkrV65Uly5dJEnGGM5kAAAAAI8p1sjsn/70J/Xv318NGjTQb7/9pp49e0qSNm7cqPr165dqgQAAAEBBihVmX331VcXGxurgwYP6+9//rtDQUEnS0aNHNXz48FItEAAAAChIscKsv7+/nsrnLL+jRo0qcUEAAABAYXE5WwAAAFgWl7MFAAAoJ44ckZYtk9LSpFGjJL9i/Q396sLlbAEAALzspZekTz6R/vOf8+vatpVuucVrJVkGeR8AAMDLxo07f9/XV8rJkTIyvFePlRTrPLOPPfaYXn/99Tzr//GPf+iJJ54oaU0AAABXvIgIqXp1yd9f6tFDmj3bOc2gSRNvV2YtxQqzH374odq1a5dn/c0336ylS5eWuCgAAIArXUCAtHOn9Mcf0ooV0rBhznCLoinWNIPffvtNFStWzLM+PDxcJ0+eLHFRAAAAV4MKFbxdgfUVa2S2fv36WrlyZZ71K1asUN26dUtcFAAAAFAYxRqZjY+P18iRI3XixAl17txZkpScnKxXXnlFM2bMKM36AAAAgAIVK8z+5S9/UWZmpl588UW98MILkqTY2Fi9+eabeuCBB0q1QAAAAKAgxZpmIEmPPvqoDh06pJSUFKWmpmrPnj0EWQAAgFJ07pz06afS4MHS9ddL333n7YrKn2KfZzY7O1tr1qzR7t271b9/f0nSkSNHFB4ertDQ0FIrEAAA4Gr0zDPSr79K6enn1332mXTzzd6rqTwqVpjdv3+/evTooQMHDigzM1Ndu3ZVWFiYpk6dqszMTM2ePbu06wQAALgq2GzO2x9+cN7GxEjBwdKOHZIx3qurvCrWNIPHH39crVq10h9//KHg4GDX+j59+ig5ObnUigMAALjaDBok3XSTc2T2xx+l/fulO+7wdlXlV7FGZr/99lt99913CggIcFsfGxurw4cPl0phAAAAV6P4eOeCwinWyKzD4VBOTk6e9YcOHVJYWFiJiwIAAAAKo1hhtlu3bm7nk7XZbDpz5owSEhIUFxdXWrUBAAAAl1SsaQbTpk1Tjx491KRJE2VkZKh///7auXOnqlatqvfff7+0awQAAADyVawwGxMTo82bN2vx4sXavHmzzpw5owcffFADBgxw+0IYAAAAUJaKHGbtdrsaN26szz//XAMGDNCAAQPKoi4AAADgsoo8Z9bf318ZGRllUQsAAABQJMX6AtiIESM0depUZWdnl3Y9AAAAQKEVa87sDz/8oOTkZK1evVrNmjVThQoV3J7/6KOPSqU4AAAA4FKKFWYjIiLUt2/f0q4FAAAAKJIihVmHw6GXX35ZO3bsUFZWljp37qyJEydyBgMAAAB4RZHmzL744ot65plnFBoaqpo1a+r111/XiBEjyqo2AAAA4JKKFGb/+c9/atasWVq1apU++eQTffbZZ1q4cKEcDkdZ1QcAAAAUqEhh9sCBA26Xq+3SpYtsNpuOHDlS6oUBAAAAl1OkMJudna2goCC3df7+/rLb7aVaFAAAAFAYRfoCmDFGgwcPVmBgoGtdRkaGHnnkEbfTc3FqLgAAAHhCkcLsoEGD8qy7//77S60YAAAAoCiKFGbnzZtXVnUAAAAARVasy9kCAAAA5QFhFgAAAJZFmAUAAIBlEWYBAABgWYRZAAAAWBZhFgAAAJZFmAUAAIBlEWYBAABgWYRZAAAAWBZhFgAAAJZFmAUAAIBlEWYBAABgWYRZAAAAWBZhFgAAAJZFmAUAAIBlEWYBAABgWYRZAAAAWBZhFgAAAJZFmAUAAIBlEWYBAABgWYRZAAAAWBZhFgAAAJZFmAUAAIBlEWYBAABgWeUizM6cOVOxsbEKCgpSmzZt9P333xfYtlOnTrLZbHmW22+/3YMVAwAAoDzwephdvHix4uPjlZCQoJ9++knNmzdX9+7ddfz48Xzbf/TRRzp69Khr+eWXX+Tr66t77rnHw5UDAADA27weZqdPn66hQ4dqyJAhatKkiWbPnq2QkBDNnTs33/aVK1dWdHS0a0lKSlJISAhhFgAAXNV27ZKef1667jqpc2fJGG9X5Bl+3tx5VlaWNmzYoHHjxrnW+fj4qEuXLlq/fn2hXuPdd9/Vn//8Z1WoUCHf5zMzM5WZmel6nJqaKkmy2+2y2+0lqL5wcvfhiX2hbNCH1kcfWh99aG30X8k5HD6SfJWTkyO73eFan5IiLVnio/fft+mHH86PUf76q3TihF2VKpXO/j3dh0XZj1fD7MmTJ5WTk6Nq1aq5ra9WrZq2bdt22e2///57/fLLL3r33XcLbDNlyhRNmjQpz/rVq1crJCSk6EUXU1JSksf2hbJBH1offWh99KG10X/Ft2dPU0n1tWfPHi1dukP/+U91ff11Lf33v5FyOGySJB8fo2bNTmjz5ihJzvc7NLR0w6en+jA9Pb3Qbb0aZkvq3XffVbNmzdS6desC24wbN07x8fGux6mpqYqJiVG3bt0UHh5e5jXa7XYlJSWpa9eu8vf3L/P9ofTRh9ZHH1offWht9F/JrVnjHHX9/vv6Skysr4wMm+u51q0duu8+o7vvdqhKlUrKHavr2rVrqY7MerIPc/+SXhheDbNVq1aVr6+vUlJS3NanpKQoOjr6ktuePXtWixYt0vPPP3/JdoGBgQoMDMyz3t/f36MfKE/vD6WPPrQ++tD66ENro/+Kz9fXeXvokDPENmokDRgg9e8v1auXO73AV9nZ57dxvt+lW4en+rAo+/DqF8ACAgLUsmVLJScnu9Y5HA4lJyerbdu2l9x2yZIlyszM1P3331/WZQIAAHhV377S//t/Uny8tGGDtHWrNH68VK+etyvzPq9PM4iPj9egQYPUqlUrtW7dWjNmzNDZs2c1ZMgQSdIDDzygmjVrasqUKW7bvfvuu+rdu7eqVKnijbIBAAA8pm1bqZDfjb/qeD3M9uvXTydOnNCECRN07NgxtWjRQitXrnR9KezAgQPy8XEfQN6+fbvWrl2r1atXe6NkAAAAlBNeD7OSNHLkSI0cOTLf59asWZNnXaNGjWSulpOnAQAAoEBev2gCAAAAUFyEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAgCtcTo6UmurtKsoGYRYAAOAK9fPP0ujRUkyMFBkpbd7s7YpKn5+3CwAAAEDpa9dO2rrVfd327VLz5t6pp6wwMgsAAHAF2rpV8veX/vQnqW5db1dTdgizAAAAVwg/P2nIEKlDB2nWLOnYMenDD53TDK5UTDMAAAC4gsyd6+0KPIuRWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAriLGSGvWSAMHStWrSx995O2KSoYwCwAAcJX4v/+TGjSQbr3Vef/YMWnlSm9XVTJ+3i4AAAAAnvHZZ87b0FCpZk1p+3bv1lMaGJkFAAC4wjVp4rxt106aO1c6etQ5zeBKwMgsAADAFe6NN6RJk6TISG9XUvoYmQUAALjC+fpemUFWIswCAADAwgizAAAAsCzCLAAAACyLMAsAAADLIswCAADAsgizAAAAsCzCLAAAACzL62F25syZio2NVVBQkNq0aaPvv//+ku1PnTqlESNGqHr16goMDFTDhg2VmJjooWoBAABQnnj1CmCLFy9WfHy8Zs+erTZt2mjGjBnq3r27tm/frqioqDzts7Ky1LVrV0VFRWnp0qWqWbOm9u/fr4iICM8XDwAAAK/zapidPn26hg4dqiFDhkiSZs+ereXLl2vu3LkaO3ZsnvZz587V77//ru+++07+/v6SpNjYWE+WDAAAgHLEa2E2KytLGzZs0Lhx41zrfHx81KVLF61fvz7fbZYtW6a2bdtqxIgR+vTTTxUZGan+/fvr6aeflq+vb77bZGZmKjMz0/U4NTVVkmS322W320vxiPKXuw9P7Atlgz60PvrQ+uhDa6P/yqecHB9JvnI4HLLbcy7Z1tN9WJT9eC3Mnjx5Ujk5OapWrZrb+mrVqmnbtm35brNnzx59+eWXGjBggBITE7Vr1y4NHz5cdrtdCQkJ+W4zZcoUTZo0Kc/61atXKyQkpOQHUkhJSUke2xfKBn1offSh9dGH1kb/lS87djSUdK0OHDigxMTNhdrGU32Ynp5e6LZenWZQVA6HQ1FRUXrrrbfk6+urli1b6vDhw3r55ZcLDLPjxo1TfHy863FqaqpiYmLUrVs3hYeHl3nNdrtdSUlJ6tq1q2tqBKyFPrQ++tD66ENro//Kp82bnecBuOaaaxQXV/OSbT3dh7l/SS8Mr4XZqlWrytfXVykpKW7rU1JSFB0dne821atXl7+/v9uUgmuvvVbHjh1TVlaWAgIC8mwTGBiowMDAPOv9/f09+oHy9P5Q+uhD66MPrY8+tDb6r3zJjVM+Pj7y9y/cCa481YdF2YfXTs0VEBCgli1bKjk52bXO4XAoOTlZbdu2zXebdu3aadeuXXI4HK51O3bsUPXq1fMNsgAAALiyefU8s/Hx8Xr77be1YMECbd26VY8++qjOnj3rOrvBAw884PYFsUcffVS///67Hn/8ce3YsUPLly/X5MmTNWLECG8dAgAAALzIq3Nm+/XrpxMnTmjChAk6duyYWrRooZUrV7q+FHbgwAH5+JzP2zExMVq1apVGjRql66+/XjVr1tTjjz+up59+2luHAAAAAC/y+hfARo4cqZEjR+b73Jo1a/Ksa9u2rf7973+XcVUAAACwAq9fzhYAAAAoLsIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAAMsizAIAAMCyCLMAAACwLMIsAAAALIswCwAAgALl5EgrV9r0009R3i4lX37eLgAAAADlz8GD0rvvOpdDh/wktdXDD9tVs6a3K3NHmAUAAIAkKTtbWrFCmjPHeetwuD+fluadui6FaQYAAABXuQMHpIQEKTZWuvNOaflyZ5Dt1El6/32pQgXj7RILxMgsAADAVWzpUudUgtxR2KpVpcGDpaFDpYYNnesefthr5V0WYRYAAOAqZLM5b//4w3l7663O0NqnjxQY6L26ioowCwAAcBW6807nvNj/9//cR2GthjALAABwFbruOunbb71dRcnxBTAAAABYFmEWAAAAlkWYBQAAgGURZgEAAGBZhFkAAABYFmEWAAAAlkWYBQAAgGURZgEAAGBZhFkAAABYFmEWAAAAlkWYBQAAgGURZgEAAGBZhFkAAABYFmEWAAAAluXn7QIAAABQvrVpY3T48O8KCgr3dil5EGYBAABwSYmJOUpMXKuaNeO8XUoeTDMAAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmEWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFl+3i7A04wxkqTU1FSP7M9utys9PV2pqany9/f3yD5RuuhD66MPrY8+tDb6z/o83Ye5OS03t13KVRdm09LSJEkxMTFergQAAACXkpaWpooVK16yjc0UJvJeQRwOh44cOaKwsDDZbLYy319qaqpiYmJ08OBBhYeHl/n+UProQ+ujD62PPrQ2+s/6PN2HxhilpaWpRo0a8vG59KzYq25k1sfHR7Vq1fL4fsPDw/kAWxx9aH30ofXRh9ZG/1mfJ/vwciOyufgCGAAAACyLMAsAAADLIsyWscDAQCUkJCgwMNDbpaCY6EProw+tjz60NvrP+spzH151XwADAADAlYORWQAAAFgWYRYAAACWRZgFAACAZRFmAQAAYFmE2VIwc+ZMxcbGKigoSG3atNH3339/yfZLlixR48aNFRQUpGbNmikxMdFDlaIgRenDt99+W+3bt1elSpVUqVIldenS5bJ9jrJX1M9hrkWLFslms6l3795lWyAuq6h9eOrUKY0YMULVq1dXYGCgGjZsyL+nXlTU/psxY4YaNWqk4OBgxcTEaNSoUcrIyPBQtbjYN998o169eqlGjRqy2Wz65JNPLrvNmjVrdOONNyowMFD169fX/Pnzy7zOfBmUyKJFi0xAQICZO3eu+fXXX83QoUNNRESESUlJybf9unXrjK+vr/n73/9utmzZYp577jnj7+9vfv75Zw9XjlxF7cP+/fubmTNnmo0bN5qtW7eawYMHm4oVK5pDhw55uHLkKmof5tq7d6+pWbOmad++vbnrrrs8UyzyVdQ+zMzMNK1atTJxcXFm7dq1Zu/evWbNmjVm06ZNHq4cxhS9/xYuXGgCAwPNwoULzd69e82qVatM9erVzahRozxcOXIlJiaaZ5991nz00UdGkvn4448v2X7Pnj0mJCTExMfHmy1btpg33njD+Pr6mpUrV3qm4AsQZkuodevWZsSIEa7HOTk5pkaNGmbKlCn5tr/33nvN7bff7rauTZs2ZtiwYWVaJwpW1D68WHZ2tgkLCzMLFiwoqxJxGcXpw+zsbHPzzTebd955xwwaNIgw62VF7cM333zT1K1b12RlZXmqRFxCUftvxIgRpnPnzm7r4uPjTbt27cq0ThROYcLsmDFjTNOmTd3W9evXz3Tv3r0MK8sf0wxKICsrSxs2bFCXLl1c63x8fNSlSxetX78+323Wr1/v1l6SunfvXmB7lK3i9OHF0tPTZbfbVbly5bIqE5dQ3D58/vnnFRUVpQcffNATZeISitOHy5YtU9u2bTVixAhVq1ZN1113nSZPnqycnBxPlY3/KU7/3XzzzdqwYYNrKsKePXuUmJiouLg4j9SMkitPecbP43u8gpw8eVI5OTmqVq2a2/pq1app27Zt+W5z7NixfNsfO3aszOpEwYrThxd7+umnVaNGjTwfanhGcfpw7dq1evfdd7Vp0yYPVIjLKU4f7tmzR19++aUGDBigxMRE7dq1S8OHD5fdbldCQoInysb/FKf/+vfvr5MnT+qWW26RMUbZ2dl65JFH9Mwzz3iiZJSCgvJMamqqzp07p+DgYI/VwsgsUAIvvfSSFi1apI8//lhBQUHeLgeFkJaWpoEDB+rtt99W1apVvV0OisnhcCgqKkpvvfWWWrZsqX79+unZZ5/V7NmzvV0aCmHNmjWaPHmyZs2apZ9++kkfffSRli9frhdeeMHbpcGCGJktgapVq8rX11cpKSlu61NSUhQdHZ3vNtHR0UVqj7JVnD7MNW3aNL300kv64osvdP3115dlmbiEovbh7t27tW/fPvXq1cu1zuFwSJL8/Py0fft21atXr2yLhpvifA6rV68uf39/+fr6utZde+21OnbsmLKyshQQEFCmNeO84vTf+PHjNXDgQD300EOSpGbNmuns2bN6+OGH9eyzz8rHh7G28q6gPBMeHu7RUVmJkdkSCQgIUMuWLZWcnOxa53A4lJycrLZt2+a7Tdu2bd3aS1JSUlKB7VG2itOHkvT3v/9dL7zwglauXKlWrVp5olQUoKh92LhxY/3888/atGmTa7nzzjt16623atOmTYqJifFk+VDxPoft2rXTrl27XL+ISNKOHTtUvXp1gqyHFaf/0tPT8wTW3F9MjDFlVyxKTbnKMx7/ytkVZtGiRSYwMNDMnz/fbNmyxTz88MMmIiLCHDt2zBhjzMCBA83YsWNd7detW2f8/PzMtGnTzNatW01CQgKn5vKyovbhSy+9ZAICAszSpUvN0aNHXUtaWpq3DuGqV9Q+vBhnM/C+ovbhgQMHTFhYmBk5cqTZvn27+fzzz01UVJT529/+5q1DuKoVtf8SEhJMWFiYef/9982ePXvM6tWrTb169cy9997rrUO46qWlpZmNGzeajRs3Gklm+vTpZuPGjWb//v3GGGPGjh1rBg4c6Gqfe2qu0aNHm61bt5qZM2dyai4re+ONN8w111xjAgICTOvWrc2///1v13MdO3Y0gwYNcmv/wQcfmIYNG5qAgADTtGlTs3z5cg9XjIsVpQ9r165tJOVZEhISPF84XIr6ObwQYbZ8KGoffvfdd6ZNmzYmMDDQ1K1b17z44osmOzvbw1UjV1H6z263m4kTJ5p69eqZoKAgExMTY4YPH27++OMPzxcOY4wxX331Vb7/t+X226BBg0zHjh3zbNOiRQsTEBBg6tata+bNm+fxuo0xxmYM4/kAAACwJubMAgAAwLIIswAAALAswiwAAAAsizALAAAAyyLMAgAAwLIIswAAALAswiwAAAAsizALAAAAyyLMAsBVzGaz6ZNPPpEk7du3TzabTZs2bfJqTQBQFIRZAPCSwYMHy2azyWazyd/fX3Xq1NGYMWOUkZHh7dIAwDL8vF0AAFzNevTooXnz5slut2vDhg0aNGiQbDabpk6d6u3SAMASGJkFAC8KDAxUdHS0YmJi1Lt3b3Xp0kVJSUmSJIfDoSlTpqhOnToKDg5W8+bNtXTpUrftf/31V91xxx0KDw9XWFiY2rdvr927d0uSfvjhB3Xt2lVVq1ZVxYoV1bFjR/30008eP0YAKEuEWQAoJ3755Rd99913CggIkCRNmTJF//znPzV79mz9+uuvGjVqlO6//359/fXXkqTDhw+rQ4cOCgwM1JdffqkNGzboL3/5i7KzsyVJaWlpGjRokNauXat///vfatCggeLi4pSWlua1YwSA0sY0AwDwos8//1yhoaHKzs5WZmamfHx89I9//EOZmZmaPHmyvvjiC7Vt21aSVLduXa1du1Zz5sxRx44dNXPmTFWsWFGLFi2Sv7+/JKlhw4au1+7cubPbvt566y1FRETo66+/1h133OG5gwSAMkSYBQAvuvXWW/Xmm2/q7NmzevXVV+Xn56e+ffvq119/VXp6urp27erWPisrSzfccIMkadOmTWrfvr0ryF4sJSVFzz33nNasWaPjx48rJydH6enpOnDgQJkfFwB4CmEWALyoQoUKql+/viRp7ty5at68ud59911dd911kqTly5erZs2abtsEBgZKkoKDgy/52oMGDdJvv/2m1157TbVr11ZgYKDatm2rrKysMjgSAPAOwiwAlBM+Pj565plnFB8frx07digwMFAHDhxQx44d821//fXXa8GCBbLb7fmOzq5bt06zZs1SXFycJOngwYM6efJkmR4DAHgaXwADgHLknnvuka+vr+bMmaOnnnpKo0aN0oIFC7R792799NNPeuONN7RgwQJJ0siRI5Wamqo///nP+vHHH7Vz506999572r59uySpQYMGeu+997R161b95z//0YABAy47mgsAVsPILACUI35+fho5cqT+/ve/a+/evYqMjNSUKVO0Z88eRURE6MYbb9QzzzwjSapSpYq+/PJLjR49Wh07dpSvr69atGihdu3aSZLeffddPfzww7rxxhsVExOjyZMn66mnnvLm4QFAqbMZY4y3iwAAAACKg2kGAAAAsCzCLAAAACyLMAsAAADLIswCAADAsgizAAAAsCzCLAAAACyLMAsAAADLIswCAADAsgizAAAAsCzCLAAAACyLMAsAAADL+v+pbyxSrveEuAAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 800x600 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.datasets import make_classification\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import precision_recall_curve\n",
        "\n",
        "# Generate synthetic dataset\n",
        "X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)\n",
        "\n",
        "# Split dataset into train and test sets\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Create and train a logistic regression model\n",
        "model = LogisticRegression()\n",
        "model.fit(X_train, y_train)\n",
        "\n",
        "# Predict probabilities on the test set\n",
        "y_scores = model.predict_proba(X_test)[:, 1]  # Probabilities of the positive class\n",
        "\n",
        "# Compute precision-recall values\n",
        "precision, recall, thresholds = precision_recall_curve(y_test, y_scores)\n",
        "\n",
        "# Plot precision-recall curve\n",
        "plt.figure(figsize=(8, 6))\n",
        "plt.plot(recall, precision, color='blue', label='Precision-Recall Curve')\n",
        "plt.xlabel('Recall')\n",
        "plt.ylabel('Precision')\n",
        "plt.title('Precision-Recall Curve')\n",
        "plt.legend(loc='best')\n",
        "plt.grid(True)\n",
        "plt.show()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "92UXezGvJDLb"
      },
      "source": [
        "    Code Example"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "SF7YcW0nvJki",
        "outputId": "9c126e3b-3bf9-491c-9b9c-b3b31a361a7a"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"pd\",\n  \"rows\": 201,\n  \"fields\": [\n    {\n      \"column\": \"Thresholds\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.38312224685010327,\n        \"min\": 0.002820136633281656,\n        \"max\": 0.9998268832413293,\n        \"num_unique_values\": 200,\n        \"samples\": [\n          0.43520239890159507,\n          0.019129169100920933,\n          0.05396071605901911\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Precision\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.1589835100166486,\n        \"min\": 0.535,\n        \"max\": 1.0,\n        \"num_unique_values\": 154,\n        \"samples\": [\n          0.5675675675675675,\n          0.8773584905660378,\n          0.9787234042553191\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Recall\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.3094100329476684,\n        \"min\": 0.0,\n        \"max\": 1.0,\n        \"num_unique_values\": 108,\n        \"samples\": [\n          0.2803738317757009,\n          0.9065420560747663,\n          0.9626168224299065\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-5f6d247c-6ca6-454c-a6e4-8330aff6814f\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Thresholds</th>\n",
              "      <th>Precision</th>\n",
              "      <th>Recall</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.002820</td>\n",
              "      <td>0.535000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.003337</td>\n",
              "      <td>0.537688</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.003791</td>\n",
              "      <td>0.540404</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.004425</td>\n",
              "      <td>0.543147</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.004797</td>\n",
              "      <td>0.545918</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>196</th>\n",
              "      <td>0.999008</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.037383</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>197</th>\n",
              "      <td>0.999373</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.028037</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>198</th>\n",
              "      <td>0.999478</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.018692</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>199</th>\n",
              "      <td>0.999827</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.009346</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>200</th>\n",
              "      <td>NaN</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>201 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5f6d247c-6ca6-454c-a6e4-8330aff6814f')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-5f6d247c-6ca6-454c-a6e4-8330aff6814f button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-5f6d247c-6ca6-454c-a6e4-8330aff6814f');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-de3d52b0-c842-4b3e-9cc1-7157d71ff509\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-de3d52b0-c842-4b3e-9cc1-7157d71ff509')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-de3d52b0-c842-4b3e-9cc1-7157d71ff509 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "     Thresholds  Precision    Recall\n",
              "0      0.002820   0.535000  1.000000\n",
              "1      0.003337   0.537688  1.000000\n",
              "2      0.003791   0.540404  1.000000\n",
              "3      0.004425   0.543147  1.000000\n",
              "4      0.004797   0.545918  1.000000\n",
              "..          ...        ...       ...\n",
              "196    0.999008   1.000000  0.037383\n",
              "197    0.999373   1.000000  0.028037\n",
              "198    0.999478   1.000000  0.018692\n",
              "199    0.999827   1.000000  0.009346\n",
              "200         NaN   1.000000  0.000000\n",
              "\n",
              "[201 rows x 3 columns]"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import pandas as pd\n",
        "pd.DataFrame(data=[thresholds, precision, recall]).set_index(pd.Index([\"Thresholds\", \"Precision\", \"Recall\"])).T"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NHILpUUjNaPo",
        "outputId": "6328e22f-5047-4277-a672-f89b6503b03e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array([0.00282014, 0.00333743, 0.00379067, 0.00442489, 0.00479681,\n",
              "       0.00540631, 0.00597639, 0.00710695, 0.00809071, 0.0114572 ,\n",
              "       0.01301475, 0.0138346 , 0.01482591, 0.01821643, 0.01886999,\n",
              "       0.01912917, 0.02034842, 0.02472672, 0.02499844, 0.02821846,\n",
              "       0.03214769, 0.03487677, 0.0421629 , 0.04519972, 0.04618539,\n",
              "       0.04718903, 0.04802179, 0.04810664, 0.04876595, 0.04932778,\n",
              "       0.05396072, 0.05421462, 0.05422912, 0.05433003, 0.05701041,\n",
              "       0.05752645, 0.05997328, 0.06017626, 0.06687969, 0.07110928,\n",
              "       0.07605716, 0.07665951, 0.08147488, 0.08437592, 0.08515805,\n",
              "       0.08593247, 0.08752646, 0.08957979, 0.09835634, 0.09958243,\n",
              "       0.10357857, 0.10417754, 0.10501068, 0.10690501, 0.11028911,\n",
              "       0.11055969, 0.11113497, 0.11802273, 0.12469223, 0.12545459,\n",
              "       0.12647337, 0.1301934 , 0.13680667, 0.13803035, 0.15593745,\n",
              "       0.15748029, 0.16748245, 0.17084627, 0.18354163, 0.20246974,\n",
              "       0.20365695, 0.20646656, 0.21314204, 0.2147069 , 0.21779583,\n",
              "       0.220019  , 0.22262833, 0.22293329, 0.22705006, 0.23064213,\n",
              "       0.26142058, 0.26455856, 0.28970888, 0.29009988, 0.29948459,\n",
              "       0.31352426, 0.34037724, 0.34495166, 0.36009537, 0.37033013,\n",
              "       0.38937462, 0.39184206, 0.39255542, 0.4233831 , 0.42992497,\n",
              "       0.4352024 , 0.44752399, 0.45218244, 0.45248069, 0.46886024,\n",
              "       0.47163453, 0.47400747, 0.47523701, 0.47865364, 0.48219376,\n",
              "       0.48603552, 0.50911559, 0.5094972 , 0.52481599, 0.54823393,\n",
              "       0.55126757, 0.5518797 , 0.55903565, 0.56870579, 0.59142063,\n",
              "       0.63698413, 0.64865099, 0.65578971, 0.65820034, 0.67068104,\n",
              "       0.69317133, 0.71134762, 0.71943051, 0.72050587, 0.72064765,\n",
              "       0.7229651 , 0.745385  , 0.75481007, 0.76843821, 0.77078125,\n",
              "       0.81532557, 0.82163009, 0.82735254, 0.8469117 , 0.85254162,\n",
              "       0.85791034, 0.86376034, 0.86772283, 0.88487398, 0.88519374,\n",
              "       0.88682222, 0.88878638, 0.89023237, 0.89065959, 0.89272064,\n",
              "       0.89687167, 0.89768426, 0.91361422, 0.91707177, 0.92396031,\n",
              "       0.92591504, 0.93431141, 0.93484787, 0.93729892, 0.93913877,\n",
              "       0.94310276, 0.94445939, 0.95057662, 0.95235596, 0.95267428,\n",
              "       0.95405622, 0.9550902 , 0.9554264 , 0.95666507, 0.9576473 ,\n",
              "       0.96229168, 0.9628737 , 0.96559338, 0.96684295, 0.96702209,\n",
              "       0.96822953, 0.97327003, 0.97347106, 0.97542923, 0.97645205,\n",
              "       0.98184863, 0.9828045 , 0.98419212, 0.98770978, 0.98772267,\n",
              "       0.98805708, 0.98988794, 0.99221759, 0.99253826, 0.99372878,\n",
              "       0.9945036 , 0.99482093, 0.99543541, 0.99602   , 0.99602877,\n",
              "       0.99685804, 0.99723943, 0.99732   , 0.99750538, 0.99755714,\n",
              "       0.99843775, 0.99900808, 0.9993735 , 0.99947813, 0.99982688])"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "thresholds"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r_zUigauOBnE"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}