DL_Obligatorio.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/MarianoChic09/Agentes-Inteligentes/blob/main/DL_Obligatorio.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "![AIRBNB](https://www.stevenridercpa.au/wp-content/uploads/2022/09/airbnb-tax.jpeg)"
      ],
      "metadata": {
        "id": "Qvw4Fb6NN2AZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Obligatorio Deep Learning"
      ],
      "metadata": {
        "id": "3S4N_09LN4Cy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Obligatorio de Deep Learning\n",
        "## Semestre 2 - 2023\n",
        "-------"
      ],
      "metadata": {
        "id": "Z9tUZ2tYN-oO"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Problema\n",
        "\n",
        "Se presenta un dataset que contiene información de alojamientos publicados en AirBnB con sus respectivos precios. El tamaño del dataset de train es de 1.5 Gb aproximadamente, y 0.5 Gb el de test. Este cuenta con 84 variables predictoras que se podrán utilizar como consideren adecuado.\n",
        "\n",
        "El objetivo es asignar el precio correcto a los alojamientos listados.\n",
        "\n",
        "Además del dataset se les provee esta notebook conteniendo el script de carga de datos y un modelo baseline que corresponde a una arquitectura feed forward.\n",
        "\n",
        "------"
      ],
      "metadata": {
        "id": "-sP6g5Y_OCYz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Consigna\n",
        "\n",
        "### A) <u>Participación en Competencia Kaggle</u>:\n",
        "El objetivo de este punto es participar en la competencia de Kaggle y obtener como mínimo un Mean Absolute Error inferior a 70 puntos. [->Link a la competencia<-](https://www.kaggle.com/t/69c648e3aa214d1f812bf2314c8d4ffa).\n",
        "\n",
        "### B) <u>Utilización de Grid Search (o equivalente)</u>:\n",
        "Para cumplir con la busqueda de modelos óptimos se debe realizar un grid search lo más abarcativo y metódico posible. Recomendamos enfáticamente [Weights and Biases](https://wandb.ai/site)\n",
        "\n",
        "### C) <u>Se debe a su vez investigar e implementar las siguientes técnicas</u>:\n",
        "#### 1. [Batch Normalization](https://machinelearningmastery.com/how-to-accelerate-learning-of-deep-neural-networks-with-batch-normalization/)\n",
        "#### 2. [Gradient Normalization y/o Gradient Clipping](https://machinelearningmastery.com/how-to-avoid-exploding-gradients-in-neural-networks-with-gradient-clipping/)\n",
        "\n",
        "\n",
        "Además como en todas las tareas se evaluará la prolijidad de la entrega, el preprocesamiento de datos, visualizaciones y exploración de técnicas alternativas."
      ],
      "metadata": {
        "id": "eZbqx1dJOFKr"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TeJvT8iRPC4l",
        "outputId": "a36e6cc7-05a9-49b1-9fd3-243e11547dc3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rtbvD_8uPqKQ",
        "outputId": "1021573a-cc26-4df7-90f6-39627edbcff4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: langdetect in /usr/local/lib/python3.10/dist-packages (1.0.9)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from langdetect) (1.16.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (4.66.1)\n",
            "Collecting wandb\n",
            "  Downloading wandb-0.16.0-py3-none-any.whl (2.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m21.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: Click!=8.0.0,>=7.1 in /usr/local/lib/python3.10/dist-packages (from wandb) (8.1.7)\n",
            "Collecting GitPython!=3.1.29,>=1.0.0 (from wandb)\n",
            "  Downloading GitPython-3.1.40-py3-none-any.whl (190 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.6/190.6 kB\u001b[0m \u001b[31m26.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: requests<3,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb) (2.31.0)\n",
            "Requirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb) (5.9.5)\n",
            "Collecting sentry-sdk>=1.0.0 (from wandb)\n",
            "  Downloading sentry_sdk-1.37.0-py2.py3-none-any.whl (251 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m251.7/251.7 kB\u001b[0m \u001b[31m28.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting docker-pycreds>=0.4.0 (from wandb)\n",
            "  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)\n",
            "Requirement already satisfied: PyYAML in /usr/local/lib/python3.10/dist-packages (from wandb) (6.0.1)\n",
            "Collecting setproctitle (from wandb)\n",
            "  Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from wandb) (67.7.2)\n",
            "Requirement already satisfied: appdirs>=1.4.3 in /usr/local/lib/python3.10/dist-packages (from wandb) (1.4.4)\n",
            "Requirement already satisfied: protobuf!=4.21.0,<5,>=3.19.0 in /usr/local/lib/python3.10/dist-packages (from wandb) (3.20.3)\n",
            "Requirement already satisfied: six>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from docker-pycreds>=0.4.0->wandb) (1.16.0)\n",
            "Collecting gitdb<5,>=4.0.1 (from GitPython!=3.1.29,>=1.0.0->wandb)\n",
            "  Downloading gitdb-4.0.11-py3-none-any.whl (62 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.7/62.7 kB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb) (3.4)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb) (2023.7.22)\n",
            "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb)\n",
            "  Downloading smmap-5.0.1-py3-none-any.whl (24 kB)\n",
            "Installing collected packages: smmap, setproctitle, sentry-sdk, docker-pycreds, gitdb, GitPython, wandb\n",
            "Successfully installed GitPython-3.1.40 docker-pycreds-0.4.0 gitdb-4.0.11 sentry-sdk-1.37.0 setproctitle-1.3.3 smmap-5.0.1 wandb-0.16.0\n"
          ]
        }
      ],
      "source": [
        "!pip install langdetect\n",
        "!pip install tqdm\n",
        "!pip install wandb"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jIXHpzJcT0kb"
      },
      "source": [
        "# Installing cudf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "67T0090Jk2KL",
        "outputId": "f85d88c7-1ed6-4a0b-800b-23aa45c4933a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fri Nov 24 12:06:47 2023       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   46C    P8    10W /  70W |      0MiB / 15360MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B0C8IV5TQnjN",
        "outputId": "bfc0c9dc-1dd7-4453-cecd-4137b7ee12c6"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'rapidsai-csp-utils'...\n",
            "remote: Enumerating objects: 395, done.\u001b[K\n",
            "remote: Counting objects: 100% (126/126), done.\u001b[K\n",
            "remote: Compressing objects: 100% (75/75), done.\u001b[K\n",
            "remote: Total 395 (delta 92), reused 53 (delta 51), pack-reused 269\u001b[K\n",
            "Receiving objects: 100% (395/395), 108.50 KiB | 4.17 MiB/s, done.\n",
            "Resolving deltas: 100% (194/194), done.\n",
            "Collecting pynvml\n",
            "  Downloading pynvml-11.5.0-py3-none-any.whl (53 kB)\n",
            "     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.1/53.1 kB 1.5 MB/s eta 0:00:00\n",
            "Installing collected packages: pynvml\n",
            "Successfully installed pynvml-11.5.0\n",
            "***********************************************************************\n",
            "Woo! Your instance has the right kind of GPU, a Tesla T4!\n",
            "We will now install RAPIDS via pip!  Please stand by, should be quick...\n",
            "***********************************************************************\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# This get the RAPIDS-Colab install files and test check your GPU.  Run this and the next cell only.\n",
        "# Please read the output of this cell.  If your Colab Instance is not RAPIDS compatible, it will warn you and give you remediation steps.\n",
        "!git clone https://github.com/rapidsai/rapidsai-csp-utils.git\n",
        "!python rapidsai-csp-utils/colab/env-check.py"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4sKA_9Dbg4s2",
        "outputId": "6358e7de-2087-44a3-c62d-1822f652846e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Updating your Colab environment.  This will restart your kernel.  Don't Panic!\n",
            "Found existing installation: cupy-cuda11x 11.0.0\n",
            "Uninstalling cupy-cuda11x-11.0.0:\n",
            "  Successfully uninstalled cupy-cuda11x-11.0.0\n",
            "PPA publishes dbgsym, you may need to include 'main/debug' component\n",
            "Repository: 'deb https://ppa.launchpadcontent.net/ubuntu-toolchain-r/test/ubuntu/ jammy main'\n",
            "Description:\n",
            "Toolchain test builds; see https://wiki.ubuntu.com/ToolChain\n",
            "\n",
            "More info: https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test\n",
            "Adding repository.\n",
            "Adding deb entry to /etc/apt/sources.list.d/ubuntu-toolchain-r-ubuntu-test-jammy.list\n",
            "Adding disabled deb-src entry to /etc/apt/sources.list.d/ubuntu-toolchain-r-ubuntu-test-jammy.list\n",
            "Adding key to /etc/apt/trusted.gpg.d/ubuntu-toolchain-r-ubuntu-test.gpg with fingerprint 60C317803A41BA51845E371A1E9377A2BA9EF27F\n",
            "Get:1 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease [3,626 B]\n",
            "Hit:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  InRelease\n",
            "Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease\n",
            "Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]\n",
            "Get:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]\n",
            "Get:6 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease [18.1 kB]\n",
            "Hit:7 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease\n",
            "Hit:8 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease\n",
            "Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease\n",
            "Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,242 kB]\n",
            "Get:11 https://ppa.launchpadcontent.net/ubuntu-toolchain-r/test/ubuntu jammy InRelease [23.8 kB]\n",
            "Hit:12 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease\n",
            "Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,284 kB]\n",
            "Get:14 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main Sources [2,244 kB]\n",
            "Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1,016 kB]\n",
            "Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,498 kB]\n",
            "Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,521 kB]\n",
            "Get:18 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main amd64 Packages [1,152 kB]\n",
            "Get:19 https://ppa.launchpadcontent.net/ubuntu-toolchain-r/test/ubuntu jammy/main amd64 Packages [17.4 kB]\n",
            "Fetched 10.2 MB in 4s (2,776 kB/s)\n",
            "Reading package lists... Done\n",
            "Added repo\n",
            "Hit:1 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease\n",
            "Hit:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  InRelease\n",
            "Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease\n",
            "Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease\n",
            "Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease\n",
            "Hit:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease\n",
            "Hit:7 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease\n",
            "Hit:8 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease\n",
            "Hit:9 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease\n",
            "Hit:10 https://ppa.launchpadcontent.net/ubuntu-toolchain-r/test/ubuntu jammy InRelease\n",
            "Hit:11 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease\n",
            "Reading package lists... Done\n",
            "Installing libstdc++\n",
            "Reading package lists... Done\n",
            "Building dependency tree... Done\n",
            "Reading state information... Done\n",
            "Package libstdc++6 is not available, but is referred to by another package.\n",
            "This may mean that the package is missing, has been obsoleted, or\n",
            "is only available from another source\n",
            "\n",
            "E: Version '11.1.0*' for 'libstdc++6' was not found\n",
            "restarting Colab...\n"
          ]
        }
      ],
      "source": [
        "# This will update the Colab environment and restart the kernel.  Don't run the next cell until you see the session crash.\n",
        "!bash rapidsai-csp-utils/colab/update_gcc.sh\n",
        "import os\n",
        "os._exit(00)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "d1C4pNiBhYHX",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5f13b13e-7ed4-4a94-f9a5-69398f437b7e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "⏬ Downloading https://github.com/conda-forge/miniforge/releases/download/23.1.0-1/Mambaforge-23.1.0-1-Linux-x86_64.sh...\n",
            "📦 Installing...\n",
            "📌 Adjusting configuration...\n",
            "🩹 Patching environment...\n",
            "⏲ Done in 0:00:09\n",
            "🔁 Restarting kernel...\n"
          ]
        }
      ],
      "source": [
        "# This will install CondaColab.  This will restart your kernel one last time.  Run this cell by itself and only run the next cell once you see the session crash.\n",
        "import condacolab\n",
        "condacolab.install()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-Ko_pJZbhZrM",
        "outputId": "cbfe9ebb-3a34-4015-a2ab-6178ba7af4a4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "✨🍰✨ Everything looks OK!\n"
          ]
        }
      ],
      "source": [
        "# you can now run the rest of the cells as normal\n",
        "import condacolab\n",
        "condacolab.check()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G_2dancFhf70",
        "outputId": "60860d74-26c8-45ca-a4cb-b233e0d12d7f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Found existing installation: cffi 1.15.1\n",
            "Uninstalling cffi-1.15.1:\n",
            "  Successfully uninstalled cffi-1.15.1\n",
            "WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\n",
            "Found existing installation: cryptography 40.0.1\n",
            "Uninstalling cryptography-40.0.1:\n",
            "  Successfully uninstalled cryptography-40.0.1\n",
            "WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\n",
            "Collecting cffi==1.15.0\n",
            "  Downloading cffi-1.15.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (446 kB)\n",
            "     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 446.3/446.3 kB 6.3 MB/s eta 0:00:00\n",
            "Requirement already satisfied: pycparser in /usr/local/lib/python3.10/site-packages (from cffi==1.15.0) (2.21)\n",
            "Installing collected packages: cffi\n",
            "Successfully installed cffi-1.15.0\n",
            "WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\n",
            "Installing RAPIDS Stable 23.10\n",
            "Starting the RAPIDS install on Colab.  This will take about 15 minutes.\n",
            "Collecting package metadata (current_repodata.json): ...working... done\n",
            "Solving environment: ...working... done\n",
            "\n",
            "\n",
            "==> WARNING: A newer version of conda exists. <==\n",
            "  current version: 23.1.0\n",
            "  latest version: 23.10.0\n",
            "\n",
            "Please update conda by running\n",
            "\n",
            "    $ conda update -n base -c conda-forge conda\n",
            "\n",
            "Or to minimize the number of packages updated during conda update use\n",
            "\n",
            "     conda install conda=23.10.0\n",
            "\n",
            "\n",
            "\n",
            "## Package Plan ##\n",
            "\n",
            "  environment location: /usr/local\n",
            "\n",
            "  added / updated specs:\n",
            "    - mamba\n",
            "\n",
            "\n",
            "The following packages will be downloaded:\n",
            "\n",
            "    package                    |            build\n",
            "    ---------------------------|-----------------\n",
            "    ca-certificates-2023.11.17 |       hbcca054_0         151 KB  conda-forge\n",
            "    certifi-2023.11.17         |     pyhd8ed1ab_0         155 KB  conda-forge\n",
            "    openssl-3.1.4              |       hd590300_0         2.5 MB  conda-forge\n",
            "    python_abi-3.10            |          4_cp310           6 KB  conda-forge\n",
            "    ------------------------------------------------------------\n",
            "                                           Total:         2.8 MB\n",
            "\n",
            "The following packages will be UPDATED:\n",
            "\n",
            "  ca-certificates                      2022.12.7-ha878542_0 --> 2023.11.17-hbcca054_0\n",
            "  certifi                            2022.12.7-pyhd8ed1ab_0 --> 2023.11.17-pyhd8ed1ab_0\n",
            "  openssl                                  3.1.0-h0b41bf4_0 --> 3.1.4-hd590300_0\n",
            "  python_abi                                   3.10-3_cp310 --> 3.10-4_cp310\n",
            "\n",
            "\n",
            "\n",
            "Downloading and Extracting Packages\n",
            "\n",
            "ca-certificates-2023 | 151 KB    |            |   0%\n",
            "\n",
            "python_abi-3.10      | 6 KB      |            |   0% \u001b[A\n",
            "\n",
            "\n",
            "certifi-2023.11.17   | 155 KB    |            |   0% \u001b[A\u001b[A\n",
            "\n",
            "\n",
            "\n",
            "openssl-3.1.4        | 2.5 MB    |            |   0% \u001b[A\u001b[A\u001b[A\n",
            "ca-certificates-2023 | 151 KB    | #          |  11%\n",
            "\n",
            "\n",
            "certifi-2023.11.17   | 155 KB    | #          |  10% \u001b[A\u001b[A\n",
            "\n",
            "python_abi-3.10      | 6 KB      | ########## | 100% \u001b[A\n",
            "\n",
            "python_abi-3.10      | 6 KB      | ########## | 100% \u001b[A\n",
            "\n",
            "\n",
            "\n",
            "openssl-3.1.4        | 2.5 MB    |            |   1% \u001b[A\u001b[A\u001b[A\n",
            "ca-certificates-2023 | 151 KB    | ########## | 100%\n",
            "\n",
            "\n",
            "certifi-2023.11.17   | 155 KB    | ########## | 100% \u001b[A\u001b[A\n",
            "\n",
            "\n",
            "\n",
            "openssl-3.1.4        | 2.5 MB    | ########## | 100% \u001b[A\u001b[A\u001b[A\n",
            "\n",
            "\n",
            "\n",
            "openssl-3.1.4        | 2.5 MB    | ########## | 100% \u001b[A\u001b[A\u001b[A\n",
            "\n",
            "\n",
            "                                                     \u001b[A\n",
            "\n",
            "\n",
            "                                                     \u001b[A\u001b[A\n",
            "\n",
            "\n",
            "\n",
            "                                                     \u001b[A\u001b[A\u001b[A\n",
            "Preparing transaction: ...working... done\n",
            "Verifying transaction: ...working... done\n",
            "Executing transaction: ...working... done\n",
            "Transaction\n",
            "\n",
            "  Prefix: /usr/local\n",
            "\n",
            "  Updating specs:\n",
            "\n",
            "   - python=3.10\n",
            "   - cudatoolkit=11.8\n",
            "   - rapids=23.10\n",
            "   - llvmlite\n",
            "   - gcsfs\n",
            "   - openssl\n",
            "   - dask-sql\n",
            "   - ca-certificates\n",
            "   - certifi\n",
            "   - openssl\n",
            "\n",
            "\n",
            "  Package                               Version  Build                              Channel                    Size\n",
            "─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
            "  Install:\n",
            "─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
            "\n",
            "  + aiohttp                               3.9.0  py310h2372a71_0                    conda-forge/linux-64      690kB\n",
            "  + aiosignal                             1.3.1  pyhd8ed1ab_0                       conda-forge/noarch         13kB\n",
            "  + annotated-types                       0.6.0  pyhd8ed1ab_0                       conda-forge/noarch         17kB\n",
            "  + anyio                                 3.7.1  pyhd8ed1ab_0                       conda-forge/noarch         97kB\n",
            "  + appdirs                               1.4.4  pyh9f0ad1d_0                       conda-forge/noarch         13kB\n",
            "  + argon2-cffi                          23.1.0  pyhd8ed1ab_0                       conda-forge/noarch         19kB\n",
            "  + argon2-cffi-bindings                 21.2.0  py310h2372a71_4                    conda-forge/linux-64       34kB\n",
            "  + arrow                                 1.3.0  pyhd8ed1ab_0                       conda-forge/noarch        100kB\n",
            "  + async-timeout                         4.0.3  pyhd8ed1ab_0                       conda-forge/noarch         11kB\n",
            "  + attrs                                23.1.0  pyh71513ae_1                       conda-forge/noarch         55kB\n",
            "  + aws-c-auth                            0.7.0  hf8751d9_2                         conda-forge/linux-64      102kB\n",
            "  + aws-c-cal                             0.6.0  h93469e0_0                         conda-forge/linux-64       50kB\n",
            "  + aws-c-common                         0.8.23  hd590300_0                         conda-forge/linux-64      197kB\n",
            "  + aws-c-compression                    0.2.17  h862ab75_1                         conda-forge/linux-64       19kB\n",
            "  + aws-c-event-stream                    0.3.1  h9599702_1                         conda-forge/linux-64       54kB\n",
            "  + aws-c-http                           0.7.11  hbe98c3e_0                         conda-forge/linux-64      194kB\n",
            "  + aws-c-io                            0.13.28  h3870b5a_0                         conda-forge/linux-64      155kB\n",
            "  + aws-c-mqtt                           0.8.14  h2e270ba_2                         conda-forge/linux-64      154kB\n",
            "  + aws-c-s3                             0.3.13  heb0bb06_2                         conda-forge/linux-64       87kB\n",
            "  + aws-c-sdkutils                       0.1.11  h862ab75_1                         conda-forge/linux-64       53kB\n",
            "  + aws-checksums                        0.1.16  h862ab75_1                         conda-forge/linux-64       50kB\n",
            "  + aws-crt-cpp                          0.20.3  he9c0e7f_4                         conda-forge/linux-64      321kB\n",
            "  + aws-sdk-cpp                         1.10.57  hbc2ea52_17                        conda-forge/linux-64        4MB\n",
            "  + beautifulsoup4                       4.12.2  pyha770c72_0                       conda-forge/noarch        115kB\n",
            "  + bleach                                6.1.0  pyhd8ed1ab_0                       conda-forge/noarch        131kB\n",
            "  + blinker                               1.7.0  pyhd8ed1ab_0                       conda-forge/noarch         18kB\n",
            "  + blosc                                1.21.5  h0f2a231_0                         conda-forge/linux-64       49kB\n",
            "  + bokeh                                 3.3.1  pyhd8ed1ab_0                       conda-forge/noarch          5MB\n",
            "  + branca                                0.7.0  pyhd8ed1ab_1                       conda-forge/noarch         29kB\n",
            "  + brotli                                1.0.9  h166bdaf_9                         conda-forge/linux-64       20kB\n",
            "  + brotli-bin                            1.0.9  h166bdaf_9                         conda-forge/linux-64       20kB\n",
            "  + cached-property                       1.5.2  hd8ed1ab_1                         conda-forge/noarch          4kB\n",
            "  + cached_property                       1.5.2  pyha770c72_1                       conda-forge/noarch         11kB\n",
            "  + cachetools                            5.3.2  pyhd8ed1ab_0                       conda-forge/noarch         15kB\n",
            "  + cairo                                1.16.0  hbbf8b49_1016                      conda-forge/linux-64        1MB\n",
            "  + cfitsio                               4.2.0  hd9d235c_0                         conda-forge/linux-64      848kB\n",
            "  + click                                 8.1.7  unix_pyh707e725_0                  conda-forge/noarch         84kB\n",
            "  + click-plugins                         1.1.1  py_0                               conda-forge/noarch          9kB\n",
            "  + cligj                                 0.7.2  pyhd8ed1ab_1                       conda-forge/noarch         10kB\n",
            "  + cloudpickle                           3.0.0  pyhd8ed1ab_0                       conda-forge/noarch         25kB\n",
            "  + colorcet                              3.0.1  pyhd8ed1ab_0                       conda-forge/noarch          2MB\n",
            "  + contourpy                             1.2.0  py310hd41b1e2_0                    conda-forge/linux-64      239kB\n",
            "  + cubinlinker                           0.3.0  py310hfdf336d_0                    rapidsai/linux-64           8MB\n",
            "  + cucim                              23.10.00  cuda11_py310_231011_g8dd1727_0     rapidsai/linux-64           4MB\n",
            "  + cuda-cudart                         11.8.89  0                                  nvidia/linux-64           202kB\n",
            "  + cuda-nvrtc                          11.8.89  0                                  nvidia/linux-64            20MB\n",
            "  + cuda-profiler-api                   11.8.86  0                                  nvidia/linux-64            19kB\n",
            "  + cuda-python                          11.8.3  py310hf9913ef_0                    nvidia/linux-64             6MB\n",
            "  + cuda-version                           11.8  h70ddcb2_2                         conda-forge/noarch         21kB\n",
            "  + cudatoolkit                          11.8.0  h4ba93d1_12                        conda-forge/linux-64      716MB\n",
            "  + cudf                               23.10.02  cuda11_py310_231116_gece2b2c524_0  rapidsai/linux-64           8MB\n",
            "  + cudf_kafka                         23.10.02  cuda11_py310_231116_gece2b2c524_0  rapidsai/linux-64         139kB\n",
            "  + cugraph                            23.10.00  cuda11_py310_231011_gc3dc860d_0    rapidsai/linux-64           2MB\n",
            "  + cuml                               23.10.00  cuda11_py310_231011_g623996072_0   rapidsai/linux-64          10MB\n",
            "  + cuproj                             23.10.00  cuda11_py310_231011_g29cddcdc_0    rapidsai/linux-64         875kB\n",
            "  + cupy                                 12.2.0  py310hf4db66c_4                    conda-forge/linux-64       42MB\n",
            "  + cuspatial                          23.10.00  cuda11_py310_231011_g29cddcdc_0    rapidsai/linux-64         996kB\n",
            "  + custreamz                          23.10.02  cuda11_py310_231116_gece2b2c524_0  rapidsai/linux-64          34kB\n",
            "  + cuxfilter                          23.10.00  cuda11_py310_231011_g6600aff_0     rapidsai/linux-64         112kB\n",
            "  + cycler                               0.12.1  pyhd8ed1ab_0                       conda-forge/noarch         13kB\n",
            "  + cyrus-sasl                           2.1.27  h54b06d7_7                         conda-forge/linux-64      220kB\n",
            "  + cytoolz                              0.12.2  py310h2372a71_1                    conda-forge/linux-64      368kB\n",
            "  + dask                               2023.9.2  pyhd8ed1ab_0                       conda-forge/noarch          7kB\n",
            "  + dask-core                          2023.9.2  pyhd8ed1ab_0                       conda-forge/noarch        859kB\n",
            "  + dask-cuda                          23.10.00  py310_231011_gdc811d3_0            rapidsai/linux-64         188kB\n",
            "  + dask-cudf                          23.10.02  cuda11_py310_231116_gece2b2c524_0  rapidsai/linux-64         138kB\n",
            "  + dask-sql                          2023.11.0  py310hac45122_0                    conda-forge/linux-64       12MB\n",
            "  + datashader                           0.16.0  pyhd8ed1ab_0                       conda-forge/noarch         17MB\n",
            "  + decorator                             5.1.1  pyhd8ed1ab_0                       conda-forge/noarch         12kB\n",
            "  + defusedxml                            0.7.1  pyhd8ed1ab_0                       conda-forge/noarch         24kB\n",
            "  + distributed                        2023.9.2  pyhd8ed1ab_0                       conda-forge/noarch        785kB\n",
            "  + dlpack                                  0.5  h9c3ff4c_0                         conda-forge/linux-64       13kB\n",
            "  + entrypoints                             0.4  pyhd8ed1ab_0                       conda-forge/noarch          9kB\n",
            "  + exceptiongroup                        1.2.0  pyhd8ed1ab_0                       conda-forge/noarch         20kB\n",
            "  + expat                                 2.5.0  hcb278e6_1                         conda-forge/linux-64      137kB\n",
            "  + fastapi                             0.104.1  pyhd8ed1ab_0                       conda-forge/noarch         71kB\n",
            "  + fastrlock                             0.8.2  py310hc6cd4ac_1                    conda-forge/linux-64       37kB\n",
            "  + fiona                                 1.9.3  py310ha325b7b_0                    conda-forge/linux-64      902kB\n",
            "  + folium                               0.15.0  pyhd8ed1ab_0                       conda-forge/noarch         73kB\n",
            "  + font-ttf-dejavu-sans-mono              2.37  hab24e00_0                         conda-forge/noarch        397kB\n",
            "  + font-ttf-inconsolata                  3.000  h77eed37_0                         conda-forge/noarch         97kB\n",
            "  + font-ttf-source-code-pro              2.038  h77eed37_0                         conda-forge/noarch        701kB\n",
            "  + font-ttf-ubuntu                        0.83  hab24e00_0                         conda-forge/noarch          2MB\n",
            "  + fontconfig                           2.14.2  h14ed4e7_0                         conda-forge/linux-64      272kB\n",
            "  + fonts-conda-ecosystem                     1  0                                  conda-forge/noarch          4kB\n",
            "  + fonts-conda-forge                         1  0                                  conda-forge/noarch          4kB\n",
            "  + fonttools                            4.45.1  py310h2372a71_0                    conda-forge/linux-64        2MB\n",
            "  + fqdn                                  1.5.1  pyhd8ed1ab_0                       conda-forge/noarch         14kB\n",
            "  + freetype                             2.12.1  h267a509_2                         conda-forge/linux-64      635kB\n",
            "  + freexl                                2.0.0  h743c826_0                         conda-forge/linux-64       60kB\n",
            "  + frozenlist                            1.4.0  py310h2372a71_1                    conda-forge/linux-64       54kB\n",
            "  + fsspec                            2023.10.0  pyhca7485f_0                       conda-forge/noarch        125kB\n",
            "  + gcsfs                             2023.10.0  pyhd8ed1ab_0                       conda-forge/noarch         36kB\n",
            "  + gdal                                  3.6.4  py310h5c4b078_20                   conda-forge/linux-64        1MB\n",
            "  + gdk-pixbuf                          2.42.10  h6c15284_3                         conda-forge/linux-64      574kB\n",
            "  + geopandas                            0.14.1  pyhd8ed1ab_0                       conda-forge/noarch          8kB\n",
            "  + geopandas-base                       0.14.1  pyha770c72_0                       conda-forge/noarch          1MB\n",
            "  + geos                                 3.12.0  h59595ed_0                         conda-forge/linux-64        2MB\n",
            "  + geotiff                               1.7.1  hee599c5_13                        conda-forge/linux-64      133kB\n",
            "  + gettext                              0.21.1  h27087fc_0                         conda-forge/linux-64        4MB\n",
            "  + gflags                                2.2.2  he1b5a44_1004                      conda-forge/linux-64      117kB\n",
            "  + giflib                                5.2.1  h0b41bf4_3                         conda-forge/linux-64       77kB\n",
            "  + glog                                  0.6.0  h6f12383_0                         conda-forge/linux-64      114kB\n",
            "  + gmock                                1.14.0  ha770c72_1                         conda-forge/linux-64        7kB\n",
            "  + google-api-core                      2.14.0  pyhd8ed1ab_0                       conda-forge/noarch         79kB\n",
            "  + google-auth                          2.23.4  pyhca7485f_0                       conda-forge/noarch        102kB\n",
            "  + google-auth-oauthlib                  1.1.0  pyhd8ed1ab_0                       conda-forge/noarch         21kB\n",
            "  + google-cloud-core                     2.3.3  pyhd8ed1ab_0                       conda-forge/noarch         29kB\n",
            "  + google-cloud-storage                 2.13.0  pyhca7485f_0                       conda-forge/noarch         87kB\n",
            "  + google-crc32c                         1.1.2  py310hc5c09a0_5                    conda-forge/linux-64       24kB\n",
            "  + google-resumable-media                2.6.0  pyhd8ed1ab_0                       conda-forge/noarch         46kB\n",
            "  + googleapis-common-protos             1.61.0  pyhd8ed1ab_0                       conda-forge/noarch        123kB\n",
            "  + grpcio                               1.54.2  py310heca2aa9_2                    conda-forge/linux-64      755kB\n",
            "  + gtest                                1.14.0  h00ab1b0_1                         conda-forge/linux-64      405kB\n",
            "  + h11                                  0.14.0  pyhd8ed1ab_0                       conda-forge/noarch         48kB\n",
            "  + h2                                    4.1.0  pyhd8ed1ab_0                       conda-forge/noarch         47kB\n",
            "  + hdf4                                 4.2.15  h501b40f_6                         conda-forge/linux-64      758kB\n",
            "  + hdf5                                 1.14.2  nompi_h4f84152_100                 conda-forge/linux-64        4MB\n",
            "  + holoviews                            1.18.1  pyhd8ed1ab_0                       conda-forge/noarch          3MB\n",
            "  + hpack                                 4.0.0  pyh9f0ad1d_0                       conda-forge/noarch         25kB\n",
            "  + httpcore                              1.0.2  pyhd8ed1ab_0                       conda-forge/noarch         45kB\n",
            "  + httpx                                0.25.1  pyhd8ed1ab_0                       conda-forge/noarch         64kB\n",
            "  + hyperframe                            6.0.1  pyhd8ed1ab_0                       conda-forge/noarch         15kB\n",
            "  + imagecodecs-lite                  2019.12.3  py310h1f7b6fc_7                    conda-forge/linux-64      169kB\n",
            "  + imageio                              2.31.5  pyh8c1a49c_0                       conda-forge/noarch        291kB\n",
            "  + importlib-metadata                    6.8.0  pyha770c72_0                       conda-forge/noarch         26kB\n",
            "  + importlib_metadata                    6.8.0  hd8ed1ab_0                         conda-forge/noarch          9kB\n",
            "  + importlib_resources                   6.1.1  pyhd8ed1ab_0                       conda-forge/noarch         30kB\n",
            "  + isoduration                         20.11.0  pyhd8ed1ab_0                       conda-forge/noarch         17kB\n",
            "  + jbig                                    2.1  h7f98852_2003                      conda-forge/linux-64       44kB\n",
            "  + jinja2                                3.1.2  pyhd8ed1ab_1                       conda-forge/noarch        101kB\n",
            "  + joblib                                1.3.2  pyhd8ed1ab_0                       conda-forge/noarch        221kB\n",
            "  + json-c                                 0.17  h7ab15ed_0                         conda-forge/linux-64       83kB\n",
            "  + jsonpointer                             2.4  py310hff52083_3                    conda-forge/linux-64       16kB\n",
            "  + jsonschema                           4.20.0  pyhd8ed1ab_0                       conda-forge/noarch         72kB\n",
            "  + jsonschema-specifications         2023.11.1  pyhd8ed1ab_0                       conda-forge/noarch         15kB\n",
            "  + jsonschema-with-format-nongpl        4.20.0  pyhd8ed1ab_0                       conda-forge/noarch          7kB\n",
            "  + jupyter-server-proxy                  4.1.0  pyhd8ed1ab_0                       conda-forge/noarch         34kB\n",
            "  + jupyter_client                        8.6.0  pyhd8ed1ab_0                       conda-forge/noarch        106kB\n",
            "  + jupyter_core                          5.5.0  py310hff52083_0                    conda-forge/linux-64       79kB\n",
            "  + jupyter_events                        0.9.0  pyhd8ed1ab_0                       conda-forge/noarch         21kB\n",
            "  + jupyter_server                       2.10.1  pyhd8ed1ab_0                       conda-forge/noarch        319kB\n",
            "  + jupyter_server_terminals              0.4.4  pyhd8ed1ab_1                       conda-forge/noarch         19kB\n",
            "  + jupyterlab_pygments                   0.3.0  pyhd8ed1ab_0                       conda-forge/noarch         19kB\n",
            "  + kealib                                1.5.2  hcd42e92_1                         conda-forge/linux-64      170kB\n",
            "  + kiwisolver                            1.4.5  py310hd41b1e2_1                    conda-forge/linux-64       73kB\n",
            "  + lazy_loader                             0.3  pyhd8ed1ab_0                       conda-forge/noarch         14kB\n",
            "  + lcms2                                  2.15  h7f713cb_2                         conda-forge/linux-64      241kB\n",
            "  + lerc                                  4.0.0  h27087fc_0                         conda-forge/linux-64      282kB\n",
            "  + libabseil                        20230125.3  cxx17_h59595ed_0                   conda-forge/linux-64        1MB\n",
            "  + libaec                                1.1.2  h59595ed_1                         conda-forge/linux-64       35kB\n",
            "  + libarrow                             12.0.1  hd2d78f0_7_cpu                     conda-forge/linux-64       28MB\n",
            "  + libblas                               3.9.0  20_linux64_openblas                conda-forge/linux-64       14kB\n",
            "  + libboost-headers                     1.82.0  ha770c72_6                         conda-forge/linux-64       14MB\n",
            "  + libbrotlicommon                       1.0.9  h166bdaf_9                         conda-forge/linux-64       71kB\n",
            "  + libbrotlidec                          1.0.9  h166bdaf_9                         conda-forge/linux-64       33kB\n",
            "  + libbrotlienc                          1.0.9  h166bdaf_9                         conda-forge/linux-64      265kB\n",
            "  + libcblas                              3.9.0  20_linux64_openblas                conda-forge/linux-64       14kB\n",
            "  + libcrc32c                             1.1.2  h9c3ff4c_0                         conda-forge/linux-64       20kB\n",
            "  + libcublas                         11.11.3.6  0                                  nvidia/linux-64           382MB\n",
            "  + libcublas-dev                     11.11.3.6  0                                  nvidia/linux-64           413MB\n",
            "  + libcucim                           23.10.00  cuda11_231011_g8dd1727_0           rapidsai/linux-64           5MB\n",
            "  + libcudf                            23.10.02  cuda11_231116_gece2b2c524_0        rapidsai/linux-64         292MB\n",
            "  + libcudf_kafka                      23.10.02  cuda11_231116_gece2b2c524_0        rapidsai/linux-64          39kB\n",
            "  + libcufft                          10.9.0.58  0                                  nvidia/linux-64           150MB\n",
            "  + libcufile                          1.4.0.31  0                                  nvidia/linux-64           561kB\n",
            "  + libcufile-dev                      1.4.0.31  0                                  nvidia/linux-64             2MB\n",
            "  + libcugraph                         23.10.00  cuda11_231011_gc3dc860d_0          rapidsai/linux-64         594MB\n",
            "  + libcugraph_etl                     23.10.00  cuda11_231011_gc3dc860d_0          rapidsai/linux-64         620kB\n",
            "  + libcugraphops                      23.10.00  cuda11_231011_gca4b54ac_0          nvidia/linux-64            38MB\n",
            "  + libcuml                            23.10.00  cuda11_231011_g623996072_0         rapidsai/linux-64         265MB\n",
            "  + libcumlprims                       23.10.00  cuda11_231011_ge818397_0           nvidia/linux-64             5MB\n",
            "  + libcurand                         10.3.0.86  0                                  nvidia/linux-64            56MB\n",
            "  + libcurand-dev                     10.3.0.86  0                                  nvidia/linux-64            56MB\n",
            "  + libcusolver                       11.4.1.48  0                                  nvidia/linux-64           101MB\n",
            "  + libcusolver-dev                   11.4.1.48  0                                  nvidia/linux-64            70MB\n",
            "  + libcusparse                       11.7.5.86  0                                  nvidia/linux-64           185MB\n",
            "  + libcusparse-dev                   11.7.5.86  0                                  nvidia/linux-64           377MB\n",
            "  + libcuspatial                       23.10.00  cuda11_231011_g29cddcdc_0          rapidsai/linux-64          29MB\n",
            "  + libdeflate                             1.19  hd590300_0                         conda-forge/linux-64       67kB\n",
            "  + libevent                             2.1.12  hf998b51_1                         conda-forge/linux-64      427kB\n",
            "  + libexpat                              2.5.0  hcb278e6_1                         conda-forge/linux-64       78kB\n",
            "  + libgdal                               3.6.4  h4a9e2c2_20                        conda-forge/linux-64       10MB\n",
            "  + libgfortran-ng                       13.2.0  h69a702a_0                         conda-forge/linux-64       23kB\n",
            "  + libgfortran5                         13.2.0  ha4646dd_0                         conda-forge/linux-64        1MB\n",
            "  + libglib                              2.78.1  hebfc3b9_0                         conda-forge/linux-64        3MB\n",
            "  + libgoogle-cloud                      2.12.0  hac9eb74_1                         conda-forge/linux-64       46MB\n",
            "  + libgrpc                              1.54.2  hb20ce57_2                         conda-forge/linux-64        6MB\n",
            "  + libjpeg-turbo                       2.1.5.1  hd590300_1                         conda-forge/linux-64      496kB\n",
            "  + libkml                                1.3.0  h01aab08_1018                      conda-forge/linux-64      514kB\n",
            "  + libkvikio                          23.10.00  cuda11_231011_g5ea0525_0           rapidsai/linux-64          38kB\n",
            "  + liblapack                             3.9.0  20_linux64_openblas                conda-forge/linux-64       14kB\n",
            "  + libllvm14                            14.0.6  hcd5def8_4                         conda-forge/linux-64       31MB\n",
            "  + libnetcdf                             4.9.2  nompi_h80fb2b6_112                 conda-forge/linux-64      848kB\n",
            "  + libntlm                                 1.4  h7f98852_1002                      conda-forge/linux-64       33kB\n",
            "  + libnuma                              2.0.16  h0b41bf4_1                         conda-forge/linux-64       41kB\n",
            "  + libopenblas                          0.3.25  pthreads_h413a1c8_0                conda-forge/linux-64        6MB\n",
            "  + libpng                               1.6.39  h753d276_0                         conda-forge/linux-64      283kB\n",
            "  + libpq                                  16.1  hfc447b1_0                         conda-forge/linux-64        3MB\n",
            "  + libprotobuf                         3.21.12  hfc55251_2                         conda-forge/linux-64        2MB\n",
            "  + libraft                            23.10.00  cuda11_231011_gafdddfb3_0          rapidsai/linux-64         639MB\n",
            "  + libraft-headers                    23.10.00  cuda11_231011_gafdddfb3_0          rapidsai/linux-64           6kB\n",
            "  + libraft-headers-only               23.10.00  cuda11_231011_gafdddfb3_0          rapidsai/linux-64           3MB\n",
            "  + librdkafka                            1.9.2  ha5a0de0_2                         conda-forge/linux-64       14MB\n",
            "  + librmm                             23.10.00  cuda11_231011_gf8ac6f8e_0          rapidsai/linux-64         776kB\n",
            "  + librttopo                             1.1.0  hb58d41b_14                        conda-forge/linux-64      233kB\n",
            "  + libsodium                            1.0.18  h36c2ea0_1                         conda-forge/linux-64      375kB\n",
            "  + libspatialindex                       1.9.3  h9c3ff4c_4                         conda-forge/linux-64        5MB\n",
            "  + libspatialite                         5.0.1  h090f1da_29                        conda-forge/linux-64        3MB\n",
            "  + libthrift                            0.18.1  h8fd135c_2                         conda-forge/linux-64        4MB\n",
            "  + libtiff                               4.6.0  h29866fb_1                         conda-forge/linux-64      277kB\n",
            "  + libutf8proc                           2.8.0  h166bdaf_0                         conda-forge/linux-64      101kB\n",
            "  + libuv                                1.44.2  hd590300_1                         conda-forge/linux-64      824kB\n",
            "  + libwebp                               1.3.2  hdffd6e0_0                         conda-forge/linux-64       86kB\n",
            "  + libwebp-base                          1.3.2  hd590300_0                         conda-forge/linux-64      402kB\n",
            "  + libxcb                                 1.15  h0b41bf4_0                         conda-forge/linux-64      384kB\n",
            "  + libxgboost                            1.7.6  rapidsai_h795055c_2310             rapidsai/linux-64         115MB\n",
            "  + libzip                               1.10.1  h2629f0a_3                         conda-forge/linux-64      107kB\n",
            "  + linkify-it-py                         2.0.0  pyhd8ed1ab_0                       conda-forge/noarch         22kB\n",
            "  + llvmlite                             0.40.1  py310h1b8f574_0                    conda-forge/linux-64        3MB\n",
            "  + locket                                1.0.0  pyhd8ed1ab_0                       conda-forge/noarch          8kB\n",
            "  + lz4                                   4.3.2  py310h350c4a5_1                    conda-forge/linux-64       37kB\n",
            "  + mapclassify                           2.6.1  pyhd8ed1ab_0                       conda-forge/noarch         39kB\n",
            "  + markdown                              3.5.1  pyhd8ed1ab_0                       conda-forge/noarch         77kB\n",
            "  + markdown-it-py                        3.0.0  pyhd8ed1ab_0                       conda-forge/noarch         64kB\n",
            "  + markupsafe                            2.1.3  py310h2372a71_1                    conda-forge/linux-64       24kB\n",
            "  + matplotlib-base                       3.8.0  py310h62c0568_1                    conda-forge/linux-64        7MB\n",
            "  + mdit-py-plugins                       0.4.0  pyhd8ed1ab_0                       conda-forge/noarch         41kB\n",
            "  + mdurl                                 0.1.0  pyhd8ed1ab_0                       conda-forge/noarch         14kB\n",
            "  + minizip                               4.0.3  h0ab5242_0                         conda-forge/linux-64       92kB\n",
            "  + mistune                               3.0.2  pyhd8ed1ab_0                       conda-forge/noarch         66kB\n",
            "  + msgpack-python                        1.0.6  py310hd41b1e2_0                    conda-forge/linux-64      197kB\n",
            "  + multidict                             6.0.4  py310h2372a71_1                    conda-forge/linux-64       57kB\n",
            "  + multipledispatch                      0.6.0  py_0                               conda-forge/noarch         12kB\n",
            "  + munch                                 4.0.0  pyhd8ed1ab_0                       conda-forge/noarch         15kB\n",
            "  + munkres                               1.1.4  pyh9f0ad1d_0                       conda-forge/noarch         12kB\n",
            "  + nbclient                              0.8.0  pyhd8ed1ab_0                       conda-forge/noarch         65kB\n",
            "  + nbconvert-core                       7.11.0  pyhd8ed1ab_0                       conda-forge/noarch        188kB\n",
            "  + nbformat                              5.9.2  pyhd8ed1ab_0                       conda-forge/noarch        100kB\n",
            "  + nccl                               2.19.4.1  h6103f9b_0                         conda-forge/linux-64      118MB\n",
            "  + networkx                              3.2.1  pyhd8ed1ab_0                       conda-forge/noarch          1MB\n",
            "  + nodejs                               20.5.1  hf52ce11_0                         conda-forge/linux-64       17MB\n",
            "  + nspr                                   4.35  h27087fc_0                         conda-forge/linux-64      227kB\n",
            "  + nss                                    3.94  h1d7d5a4_0                         conda-forge/linux-64        2MB\n",
            "  + numba                                0.57.1  py310h0f6aa51_0                    conda-forge/linux-64        4MB\n",
            "  + numpy                                1.24.4  py310ha4c1d20_0                    conda-forge/linux-64        7MB\n",
            "  + nvcomp                                2.6.1  h838ba91_3                         conda-forge/linux-64        8MB\n",
            "  + nvtx                                  0.2.8  py310h2372a71_1                    conda-forge/linux-64      105kB\n",
            "  + oauthlib                              3.2.2  pyhd8ed1ab_0                       conda-forge/noarch         92kB\n",
            "  + openjpeg                              2.5.0  h488ebb8_3                         conda-forge/linux-64      357kB\n",
            "  + openslide                             3.4.1  h4738aba_10                        conda-forge/linux-64      112kB\n",
            "  + orc                                   1.9.0  h2f23424_1                         conda-forge/linux-64        1MB\n",
            "  + overrides                             7.4.0  pyhd8ed1ab_0                       conda-forge/noarch         30kB\n",
            "  + packaging                              23.2  pyhd8ed1ab_0                       conda-forge/noarch         49kB\n",
            "  + pandas                                1.5.3  py310h9b08913_1                    conda-forge/linux-64       12MB\n",
            "  + pandocfilters                         1.5.0  pyhd8ed1ab_0                       conda-forge/noarch         12kB\n",
            "  + panel                                 1.3.2  pyhd8ed1ab_0                       conda-forge/noarch         15MB\n",
            "  + param                                 2.0.1  pyhca7485f_0                       conda-forge/noarch        100kB\n",
            "  + partd                                 1.4.1  pyhd8ed1ab_0                       conda-forge/noarch         21kB\n",
            "  + pcre2                                 10.40  hc3806b6_0                         conda-forge/linux-64        2MB\n",
            "  + pillow                               10.0.1  py310h29da1c1_1                    conda-forge/linux-64       46MB\n",
            "  + pixman                               0.42.2  h59595ed_0                         conda-forge/linux-64      385kB\n",
            "  + pkgutil-resolve-name                 1.3.10  pyhd8ed1ab_1                       conda-forge/noarch         11kB\n",
            "  + platformdirs                          4.0.0  pyhd8ed1ab_0                       conda-forge/noarch         20kB\n",
            "  + poppler                             23.08.0  hf2349cb_2                         conda-forge/linux-64        2MB\n",
            "  + poppler-data                         0.4.12  hd8ed1ab_0                         conda-forge/noarch          2MB\n",
            "  + postgresql                             16.1  h8972f4a_0                         conda-forge/linux-64        5MB\n",
            "  + proj                                  9.3.0  h1d62c97_2                         conda-forge/linux-64        3MB\n",
            "  + prometheus_client                    0.19.0  pyhd8ed1ab_0                       conda-forge/noarch         48kB\n",
            "  + prompt-toolkit                       3.0.41  pyha770c72_0                       conda-forge/noarch        270kB\n",
            "  + protobuf                            4.21.12  py310heca2aa9_0                    conda-forge/linux-64      324kB\n",
            "  + psutil                                5.9.5  py310h2372a71_1                    conda-forge/linux-64      361kB\n",
            "  + pthread-stubs                           0.4  h36c2ea0_1001                      conda-forge/linux-64        6kB\n",
            "  + ptxcompiler                           0.8.1  py310h70a93da_2                    conda-forge/linux-64        8MB\n",
            "  + ptyprocess                            0.7.0  pyhd3deb0d_0                       conda-forge/noarch         17kB\n",
            "  + py-xgboost                            1.7.6  rapidsai_py310h1395376_2310        rapidsai/linux-64         216kB\n",
            "  + pyarrow                              12.0.1  py310h0576679_7_cpu                conda-forge/linux-64        4MB\n",
            "  + pyasn1                                0.5.0  pyhd8ed1ab_0                       conda-forge/noarch         63kB\n",
            "  + pyasn1-modules                        0.3.0  pyhd8ed1ab_0                       conda-forge/noarch         96kB\n",
            "  + pyct                                  0.4.6  py_0                               conda-forge/noarch          3kB\n",
            "  + pyct-core                             0.4.6  py_0                               conda-forge/noarch         14kB\n",
            "  + pydantic                              2.5.2  pyhd8ed1ab_0                       conda-forge/noarch        262kB\n",
            "  + pydantic-core                        2.14.5  py310hcb5633a_0                    conda-forge/linux-64        2MB\n",
            "  + pyee                                  8.1.0  pyhd8ed1ab_0                       conda-forge/noarch         14kB\n",
            "  + pygments                             2.17.2  pyhd8ed1ab_0                       conda-forge/noarch        860kB\n",
            "  + pyjwt                                 2.8.0  pyhd8ed1ab_0                       conda-forge/noarch         25kB\n",
            "  + pylibcugraph                       23.10.00  cuda11_py310_231011_gc3dc860d_0    rapidsai/linux-64         755kB\n",
            "  + pylibraft                          23.10.00  cuda11_py310_231011_gafdddfb3_0    rapidsai/linux-64           2MB\n",
            "  + pynvml                               11.4.1  pyhd8ed1ab_0                       conda-forge/noarch         41kB\n",
            "  + pyparsing                             3.1.1  pyhd8ed1ab_0                       conda-forge/noarch         90kB\n",
            "  + pyppeteer                             1.0.2  pyhd8ed1ab_0                       conda-forge/noarch         64kB\n",
            "  + pyproj                                3.6.1  py310h32c33b7_4                    conda-forge/linux-64      527kB\n",
            "  + python-confluent-kafka                1.9.2  py310h5764c6d_2                    conda-forge/linux-64      149kB\n",
            "  + python-dateutil                       2.8.2  pyhd8ed1ab_0                       conda-forge/noarch        246kB\n",
            "  + python-fastjsonschema                2.19.0  pyhd8ed1ab_0                       conda-forge/noarch        226kB\n",
            "  + python-json-logger                    2.0.7  pyhd8ed1ab_0                       conda-forge/noarch         13kB\n",
            "  + pytz                           2023.3.post1  pyhd8ed1ab_0                       conda-forge/noarch        187kB\n",
            "  + pyu2f                                 0.1.5  pyhd8ed1ab_0                       conda-forge/noarch         32kB\n",
            "  + pyviz_comms                           3.0.0  pyhd8ed1ab_0                       conda-forge/noarch         48kB\n",
            "  + pywavelets                            1.4.1  py310h1f7b6fc_1                    conda-forge/linux-64        4MB\n",
            "  + pyyaml                                6.0.1  py310h2372a71_1                    conda-forge/linux-64      171kB\n",
            "  + pyzmq                                25.1.1  py310h795f18f_2                    conda-forge/linux-64      457kB\n",
            "  + raft-dask                          23.10.00  cuda11_py310_231011_gafdddfb3_0    rapidsai/linux-64         420kB\n",
            "  + rapids                             23.10.00  cuda11_py310_231011_gd3958fe_0     rapidsai/linux-64           6kB\n",
            "  + rapids-xgboost                     23.10.00  cuda11_py310_231011_gd3958fe_0     rapidsai/linux-64           5kB\n",
            "  + rdma-core                              28.9  h59595ed_1                         conda-forge/linux-64        4MB\n",
            "  + re2                              2023.03.02  h8c504da_0                         conda-forge/linux-64      201kB\n",
            "  + referencing                          0.31.0  pyhd8ed1ab_0                       conda-forge/noarch         38kB\n",
            "  + requests-oauthlib                     1.3.1  pyhd8ed1ab_0                       conda-forge/noarch         22kB\n",
            "  + rfc3339-validator                     0.1.4  pyhd8ed1ab_0                       conda-forge/noarch          8kB\n",
            "  + rfc3986-validator                     0.1.1  pyh9f0ad1d_0                       conda-forge/noarch          8kB\n",
            "  + rich                                 13.7.0  pyhd8ed1ab_0                       conda-forge/noarch        184kB\n",
            "  + rmm                                23.10.00  cuda11_py310_231011_gf8ac6f8e_0    rapidsai/linux-64         671kB\n",
            "  + rpds-py                              0.13.1  py310hcb5633a_0                    conda-forge/linux-64      989kB\n",
            "  + rsa                                     4.9  pyhd8ed1ab_0                       conda-forge/noarch         30kB\n",
            "  + rtree                                 1.1.0  py310hbdcdc62_0                    conda-forge/linux-64       50kB\n",
            "  + s2n                                  1.3.46  h06160fa_0                         conda-forge/linux-64      367kB\n",
            "  + scikit-image                         0.20.0  py310h9b08913_1                    conda-forge/linux-64       10MB\n",
            "  + scikit-learn                          1.3.2  py310h1fdf081_1                    conda-forge/linux-64        8MB\n",
            "  + scipy                                1.11.4  py310hb13e2d6_0                    conda-forge/linux-64       15MB\n",
            "  + send2trash                            1.8.2  pyh41d4057_0                       conda-forge/noarch         23kB\n",
            "  + shapely                               2.0.2  py310h7dcad9a_0                    conda-forge/linux-64      482kB\n",
            "  + simpervisor                           1.0.0  pyhd8ed1ab_0                       conda-forge/noarch         13kB\n",
            "  + six                                  1.16.0  pyh6c4a22f_0                       conda-forge/noarch         14kB\n",
            "  + snappy                               1.1.10  h9fff704_0                         conda-forge/linux-64       39kB\n",
            "  + sniffio                               1.3.0  pyhd8ed1ab_0                       conda-forge/noarch         14kB\n",
            "  + sortedcontainers                      2.4.0  pyhd8ed1ab_0                       conda-forge/noarch         26kB\n",
            "  + soupsieve                               2.5  pyhd8ed1ab_1                       conda-forge/noarch         37kB\n",
            "  + spdlog                               1.11.0  h9b3ece8_1                         conda-forge/linux-64      221kB\n",
            "  + sqlite                               3.39.2  h4ff8645_0                         conda-forge/linux-64        2MB\n",
            "  + starlette                            0.27.0  pyhd8ed1ab_0                       conda-forge/noarch         55kB\n",
            "  + streamz                               0.6.4  pyh6c4a22f_0                       conda-forge/noarch         65kB\n",
            "  + tabulate                              0.9.0  pyhd8ed1ab_1                       conda-forge/noarch         36kB\n",
            "  + tblib                                 2.0.0  pyhd8ed1ab_0                       conda-forge/noarch         17kB\n",
            "  + terminado                            0.18.0  pyh0d859eb_0                       conda-forge/noarch         22kB\n",
            "  + threadpoolctl                         3.2.0  pyha21a80b_0                       conda-forge/noarch         21kB\n",
            "  + tifffile                           2020.6.3  py_0                               conda-forge/noarch        111kB\n",
            "  + tiledb                               2.16.3  hf0b6e87_3                         conda-forge/linux-64        6MB\n",
            "  + tinycss2                              1.2.1  pyhd8ed1ab_0                       conda-forge/noarch         23kB\n",
            "  + tornado                               6.3.3  py310h2372a71_1                    conda-forge/linux-64      642kB\n",
            "  + traitlets                            5.13.0  pyhd8ed1ab_0                       conda-forge/noarch        110kB\n",
            "  + treelite                              3.9.1  py310h4a6579d_0                    conda-forge/linux-64      765kB\n",
            "  + types-python-dateutil             2.8.19.14  pyhd8ed1ab_0                       conda-forge/noarch         21kB\n",
            "  + typing-extensions                     4.8.0  hd8ed1ab_0                         conda-forge/noarch         10kB\n",
            "  + typing_extensions                     4.8.0  pyha770c72_0                       conda-forge/noarch         35kB\n",
            "  + typing_utils                          0.1.0  pyhd8ed1ab_0                       conda-forge/noarch         14kB\n",
            "  + tzcode                                2023c  h0b41bf4_0                         conda-forge/linux-64       69kB\n",
            "  + tzlocal                                 5.2  py310hff52083_0                    conda-forge/linux-64       36kB\n",
            "  + uc-micro-py                           1.0.1  pyhd8ed1ab_0                       conda-forge/noarch          9kB\n",
            "  + ucx                                  1.14.1  h64cca9d_5                         conda-forge/linux-64       15MB\n",
            "  + ucx-proc                              1.0.0  gpu                                rapidsai/linux-64           9kB\n",
            "  + ucx-py                              0.34.00  py310_231011_g17dceab_0            rapidsai/linux-64         474kB\n",
            "  + unicodedata2                         15.1.0  py310h2372a71_0                    conda-forge/linux-64      374kB\n",
            "  + uri-template                          1.3.0  pyhd8ed1ab_0                       conda-forge/noarch         24kB\n",
            "  + uriparser                             0.9.7  hcb278e6_1                         conda-forge/linux-64       47kB\n",
            "  + uvicorn                              0.24.0  py310hff52083_0                    conda-forge/linux-64       97kB\n",
            "  + wcwidth                              0.2.12  pyhd8ed1ab_0                       conda-forge/noarch         32kB\n",
            "  + webcolors                              1.13  pyhd8ed1ab_0                       conda-forge/noarch         18kB\n",
            "  + webencodings                          0.5.1  pyhd8ed1ab_2                       conda-forge/noarch         16kB\n",
            "  + websocket-client                      1.6.4  pyhd8ed1ab_0                       conda-forge/noarch         46kB\n",
            "  + websockets                             10.4  py310h5764c6d_1                    conda-forge/linux-64      129kB\n",
            "  + xarray                            2023.11.0  pyhd8ed1ab_0                       conda-forge/noarch        713kB\n",
            "  + xerces-c                              3.2.4  h8d71039_2                         conda-forge/linux-64        2MB\n",
            "  + xgboost                               1.7.6  rapidsai_py310h1395376_2310        rapidsai/linux-64          16kB\n",
            "  + xorg-kbproto                          1.0.7  h7f98852_1002                      conda-forge/linux-64       27kB\n",
            "  + xorg-libice                           1.1.1  hd590300_0                         conda-forge/linux-64       58kB\n",
            "  + xorg-libsm                            1.2.4  h7391055_0                         conda-forge/linux-64       27kB\n",
            "  + xorg-libx11                           1.8.7  h8ee46fc_0                         conda-forge/linux-64      829kB\n",
            "  + xorg-libxau                          1.0.11  hd590300_0                         conda-forge/linux-64       14kB\n",
            "  + xorg-libxdmcp                         1.1.3  h7f98852_0                         conda-forge/linux-64       19kB\n",
            "  + xorg-libxext                          1.3.4  h0b41bf4_2                         conda-forge/linux-64       50kB\n",
            "  + xorg-libxrender                      0.9.11  hd590300_0                         conda-forge/linux-64       38kB\n",
            "  + xorg-renderproto                     0.11.1  h7f98852_1002                      conda-forge/linux-64       10kB\n",
            "  + xorg-xextproto                        7.3.0  h0b41bf4_1003                      conda-forge/linux-64       30kB\n",
            "  + xorg-xproto                          7.0.31  h7f98852_1007                      conda-forge/linux-64       75kB\n",
            "  + xyzservices                       2023.10.1  pyhd8ed1ab_0                       conda-forge/noarch         36kB\n",
            "  + yaml                                  0.2.5  h7f98852_2                         conda-forge/linux-64       89kB\n",
            "  + yarl                                  1.9.2  py310h2372a71_1                    conda-forge/linux-64       98kB\n",
            "  + zeromq                                4.3.5  h59595ed_0                         conda-forge/linux-64      343kB\n",
            "  + zict                                  3.0.0  pyhd8ed1ab_0                       conda-forge/noarch         36kB\n",
            "  + zipp                                 3.17.0  pyhd8ed1ab_0                       conda-forge/noarch         19kB\n",
            "  + zlib                                 1.2.13  h166bdaf_4                         conda-forge/linux-64       94kB\n",
            "\n",
            "  Change:\n",
            "─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
            "\n",
            "  - libarchive                            3.6.2  h3d51595_0                         conda-forge\n",
            "  + libarchive                            3.6.2  h039dbb9_1                         conda-forge/linux-64      844kB\n",
            "  - libmamba                              1.4.1  hcea66bb_0                         conda-forge\n",
            "  + libmamba                              1.4.1  hcea66bb_0                         conda-forge/linux-64     Cached\n",
            "\n",
            "  Upgrade:\n",
            "─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
            "\n",
            "  - krb5                                 1.20.1  h81ceb04_0                         conda-forge\n",
            "  + krb5                                 1.21.2  h659d440_0                         conda-forge/linux-64        1MB\n",
            "  - libcurl                              7.88.1  hdc1c0ab_1                         conda-forge\n",
            "  + libcurl                               8.4.0  hca28451_0                         conda-forge/linux-64      386kB\n",
            "  - libsqlite                            3.40.0  h753d276_0                         conda-forge\n",
            "  + libsqlite                            3.44.1  h2797004_0                         conda-forge/linux-64      846kB\n",
            "  - libssh2                              1.10.0  hf14f497_3                         conda-forge\n",
            "  + libssh2                              1.11.0  h0841786_0                         conda-forge/linux-64      271kB\n",
            "  - libxml2                              2.10.3  hfdac1af_6                         conda-forge\n",
            "  + libxml2                              2.11.5  h0d562d8_0                         conda-forge/linux-64      705kB\n",
            "  - zstd                                  1.5.2  h3eb15da_6                         conda-forge\n",
            "  + zstd                                  1.5.5  hfc55251_0                         conda-forge/linux-64      545kB\n",
            "\n",
            "  Summary:\n",
            "\n",
            "  Install: 382 packages\n",
            "  Change: 2 packages\n",
            "  Upgrade: 6 packages\n",
            "\n",
            "  Total download: 5GB\n",
            "\n",
            "─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
            "\n",
            "\n",
            "\n",
            "Looking for: ['python=3.10', 'cudatoolkit=11.8', 'rapids=23.10', 'llvmlite', 'gcsfs', 'openssl', 'dask-sql']\n",
            "\n",
            "\n",
            "Pinned packages:\n",
            "  - python 3.10.*\n",
            "  - python_abi 3.10.* *cp310*\n",
            "  - cudatoolkit 11.8.*\n",
            "\n",
            "\n",
            "\n",
            "Downloading and Extracting Packages\n",
            "\n",
            "Preparing transaction: ...working... done\n",
            "Verifying transaction: ...working... done\n",
            "Executing transaction: ...working... By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html\n",
            "\n",
            "\n",
            "By downloading and using the CubinLinker conda packages, you accept the terms and conditions of the CubinLinker License Agreement: https://docs.rapids.ai/licenses/CubinLinker.txt\n",
            "\n",
            "done\n",
            "RAPIDS conda installation complete.  Updating Colab's libraries...\n",
            "Copying /usr/local/lib/libcudf.so to /usr/lib/libcudf.so\n",
            "Copying /usr/local/lib/libnccl.so to /usr/lib/libnccl.so\n",
            "Copying /usr/local/lib/libcuml.so to /usr/lib/libcuml.so\n",
            "Copying /usr/local/lib/libcugraph.so to /usr/lib/libcugraph.so\n",
            "Copying /usr/local/lib/libxgboost.so to /usr/lib/libxgboost.so\n",
            "Copying /usr/local/lib/libcuspatial.so to /usr/lib/libcuspatial.so\n",
            "Copying /usr/local/lib/libgeos.so to /usr/lib/libgeos.so\n",
            "Copying /usr/local/lib/libgeos_c.so to /usr/lib/libgeos_c.so\n"
          ]
        }
      ],
      "source": [
        "# Installing RAPIDS is now 'python rapidsai-csp-utils/colab/install_rapids.py <release> <packages>'\n",
        "# The <release> options are 'stable' and 'nightly'.  Leaving it blank or adding any other words will default to stable.\n",
        "!python rapidsai-csp-utils/colab/install_rapids.py stable\n",
        "import os\n",
        "os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'\n",
        "os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'\n",
        "os.environ['CONDA_PREFIX'] = '/usr/local'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wKSvbcSjO7_Q"
      },
      "source": [
        "# Obligatorio Deep Learning\n",
        "## 1. Setup\n",
        "### 1.1 Imports"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ntv1OfCFPON4",
        "outputId": "35b13fbd-8302-4e4e-d4eb-89a88e244f76"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/content/drive/MyDrive/Colab Notebooks/Datasets\n"
          ]
        }
      ],
      "source": [
        "%cd /content/drive/MyDrive/Colab Notebooks/Datasets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "SKGamGUw7P-K"
      },
      "outputs": [],
      "source": [
        "# !git clone https://github.com/rapidsai/rapidsai-csp-utils.git\n",
        "# !python rapidsai-csp-utils/colab/pip-install.py\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "k4mtmw9c7WNk"
      },
      "outputs": [],
      "source": [
        "import cudf\n",
        "\n",
        "# Path to your CSV file\n",
        "file_path = '/content/drive/MyDrive/Colab Notebooks/Datasets/obligatorio_DL/public_train_data.csv'\n",
        "\n",
        "# Read the CSV file using dask_cudf\n",
        "df = cudf.read_csv(file_path)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 439
        },
        "id": "VfIN1Gs7AWmU",
        "outputId": "ecb5e72c-dd13-44ff-d9c8-33542306b9f2"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   id Last Scraped                                         Name  \\\n",
              "0   0   2017-05-12  Grand Loft in the heart of historic Antwerp   \n",
              "1   1   2017-05-03             CHARMING, CLEAN & COZY BUNGALOW!   \n",
              "\n",
              "                                             Summary  \\\n",
              "0  Best location for visiting Antwerp!! Beautiful...   \n",
              "1  Very centrally located and less than 15 min fr...   \n",
              "\n",
              "                                               Space  \\\n",
              "0  Welcome in Antwerp!! The loft is situated on t...   \n",
              "1       Well lit, private entrance with small patio.   \n",
              "\n",
              "                                         Description Experiences Offered  \\\n",
              "0  Best location for visiting Antwerp!! Beautiful...                none   \n",
              "1  Very centrally located and less than 15 min fr...                none   \n",
              "\n",
              "                          Neighborhood Overview  \\\n",
              "0                                          <NA>   \n",
              "1  Quiet. Pretty tree lined streets, safe area.   \n",
              "\n",
              "                                        Notes  \\\n",
              "0                                        <NA>   \n",
              "1  Has dining table and high back desk chair.   \n",
              "\n",
              "                                             Transit  ...  \\\n",
              "0                                               <NA>  ...   \n",
              "1  Uber, bus line and metro link is less than 5 m...  ...   \n",
              "\n",
              "  Review Scores Location Review Scores Value License       Jurisdiction Names  \\\n",
              "0                   10.0                 9.0    <NA>                     <NA>   \n",
              "1                   <NA>                <NA>    <NA>  City of Los Angeles, CA   \n",
              "\n",
              "  Cancellation Policy Calculated host listings count  Reviews per Month  \\\n",
              "0              strict                            2.0                2.6   \n",
              "1            flexible                            1.0               <NA>   \n",
              "\n",
              "                             Geolocation  \\\n",
              "0  51.21938762207894, 4.4034442505151885   \n",
              "1  34.1892692286356, -118.41993491931177   \n",
              "\n",
              "                                 Features  Price  \n",
              "0   Host Has Profile Pic,Instant Bookable  159.0  \n",
              "1  Host Has Profile Pic,Is Location Exact   49.0  \n",
              "\n",
              "[2 rows x 85 columns]"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>Last Scraped</th>\n",
              "      <th>Name</th>\n",
              "      <th>Summary</th>\n",
              "      <th>Space</th>\n",
              "      <th>Description</th>\n",
              "      <th>Experiences Offered</th>\n",
              "      <th>Neighborhood Overview</th>\n",
              "      <th>Notes</th>\n",
              "      <th>Transit</th>\n",
              "      <th>...</th>\n",
              "      <th>Review Scores Location</th>\n",
              "      <th>Review Scores Value</th>\n",
              "      <th>License</th>\n",
              "      <th>Jurisdiction Names</th>\n",
              "      <th>Cancellation Policy</th>\n",
              "      <th>Calculated host listings count</th>\n",
              "      <th>Reviews per Month</th>\n",
              "      <th>Geolocation</th>\n",
              "      <th>Features</th>\n",
              "      <th>Price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>2017-05-12</td>\n",
              "      <td>Grand Loft in the heart of historic Antwerp</td>\n",
              "      <td>Best location for visiting Antwerp!! Beautiful...</td>\n",
              "      <td>Welcome in Antwerp!! The loft is situated on t...</td>\n",
              "      <td>Best location for visiting Antwerp!! Beautiful...</td>\n",
              "      <td>none</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>...</td>\n",
              "      <td>10.0</td>\n",
              "      <td>9.0</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>strict</td>\n",
              "      <td>2.0</td>\n",
              "      <td>2.6</td>\n",
              "      <td>51.21938762207894, 4.4034442505151885</td>\n",
              "      <td>Host Has Profile Pic,Instant Bookable</td>\n",
              "      <td>159.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>2017-05-03</td>\n",
              "      <td>CHARMING, CLEAN &amp; COZY BUNGALOW!</td>\n",
              "      <td>Very centrally located and less than 15 min fr...</td>\n",
              "      <td>Well lit, private entrance with small patio.</td>\n",
              "      <td>Very centrally located and less than 15 min fr...</td>\n",
              "      <td>none</td>\n",
              "      <td>Quiet. Pretty tree lined streets, safe area.</td>\n",
              "      <td>Has dining table and high back desk chair.</td>\n",
              "      <td>Uber, bus line and metro link is less than 5 m...</td>\n",
              "      <td>...</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>City of Los Angeles, CA</td>\n",
              "      <td>flexible</td>\n",
              "      <td>1.0</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>34.1892692286356, -118.41993491931177</td>\n",
              "      <td>Host Has Profile Pic,Is Location Exact</td>\n",
              "      <td>49.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>2 rows × 85 columns</p>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "df.head(2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2Lp-JkqZO7_V",
        "outputId": "7130c274-cf21-40ab-b388-42630ef52221"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'cudf.core.dataframe.DataFrame'>\n",
            "RangeIndex: 326287 entries, 0 to 326286\n",
            "Data columns (total 85 columns):\n",
            " #   Column                          Non-Null Count   Dtype\n",
            "---  ------                          --------------   -----\n",
            " 0   id                              326287 non-null  int64\n",
            " 1   Last Scraped                    326286 non-null  object\n",
            " 2   Name                            326018 non-null  object\n",
            " 3   Summary                         315651 non-null  object\n",
            " 4   Space                           228792 non-null  object\n",
            " 5   Description                     326188 non-null  object\n",
            " 6   Experiences Offered             326287 non-null  object\n",
            " 7   Neighborhood Overview           192513 non-null  object\n",
            " 8   Notes                           130729 non-null  object\n",
            " 9   Transit                         200649 non-null  object\n",
            " 10  Access                          177108 non-null  object\n",
            " 11  Interaction                     169193 non-null  object\n",
            " 12  House Rules                     195763 non-null  object\n",
            " 13  Thumbnail Url                   264638 non-null  object\n",
            " 14  Medium Url                      264637 non-null  object\n",
            " 15  Picture Url                     325786 non-null  object\n",
            " 16  XL Picture Url                  264638 non-null  object\n",
            " 17  Host ID                         326287 non-null  int64\n",
            " 18  Host URL                        326287 non-null  object\n",
            " 19  Host Name                       325970 non-null  object\n",
            " 20  Host Since                      325970 non-null  object\n",
            " 21  Host Location                   324801 non-null  object\n",
            " 22  Host About                      195571 non-null  object\n",
            " 23  Host Response Time              250846 non-null  object\n",
            " 24  Host Response Rate              250845 non-null  float64\n",
            " 25  Host Acceptance Rate            27630 non-null   object\n",
            " 26  Host Thumbnail Url              325971 non-null  object\n",
            " 27  Host Picture Url                325971 non-null  object\n",
            " 28  Host Neighbourhood              244207 non-null  object\n",
            " 29  Host Listings Count             325971 non-null  float64\n",
            " 30  Host Total Listings Count       325970 non-null  float64\n",
            " 31  Host Verifications              325751 non-null  object\n",
            " 32  Street                          326287 non-null  object\n",
            " 33  Neighbourhood                   227447 non-null  object\n",
            " 34  Neighbourhood Cleansed          326287 non-null  object\n",
            " 35  Neighbourhood Group Cleansed    68297 non-null   object\n",
            " 36  City                            326004 non-null  object\n",
            " 37  State                           294992 non-null  object\n",
            " 38  Zipcode                         313815 non-null  object\n",
            " 39  Market                          322672 non-null  object\n",
            " 40  Smart Location                  326286 non-null  object\n",
            " 41  Country Code                    326286 non-null  object\n",
            " 42  Country                         326286 non-null  object\n",
            " 43  Latitude                        326287 non-null  float64\n",
            " 44  Longitude                       326287 non-null  float64\n",
            " 45  Property Type                   326280 non-null  object\n",
            " 46  Room Type                       326287 non-null  object\n",
            " 47  Accommodates                    326244 non-null  float64\n",
            " 48  Bathrooms                       325300 non-null  float64\n",
            " 49  Bedrooms                        325873 non-null  float64\n",
            " 50  Beds                            325693 non-null  float64\n",
            " 51  Bed Type                        326287 non-null  object\n",
            " 52  Amenities                       323399 non-null  object\n",
            " 53  Square Feet                     7993 non-null    float64\n",
            " 54  Security Deposit                136142 non-null  float64\n",
            " 55  Cleaning Fee                    208181 non-null  float64\n",
            " 56  Guests Included                 326286 non-null  float64\n",
            " 57  Extra People                    326274 non-null  float64\n",
            " 58  Minimum Nights                  326286 non-null  float64\n",
            " 59  Maximum Nights                  326286 non-null  float64\n",
            " 60  Calendar Updated                326287 non-null  object\n",
            " 61  Has Availability                6249 non-null    object\n",
            " 62  Availability 30                 326286 non-null  float64\n",
            " 63  Availability 60                 326286 non-null  float64\n",
            " 64  Availability 90                 326286 non-null  float64\n",
            " 65  Availability 365                326286 non-null  float64\n",
            " 66  Calendar last Scraped           326286 non-null  object\n",
            " 67  Number of Reviews               326286 non-null  float64\n",
            " 68  First Review                    246983 non-null  object\n",
            " 69  Last Review                     247046 non-null  object\n",
            " 70  Review Scores Rating            243160 non-null  float64\n",
            " 71  Review Scores Accuracy          242584 non-null  float64\n",
            " 72  Review Scores Cleanliness       242732 non-null  float64\n",
            " 73  Review Scores Checkin           242378 non-null  float64\n",
            " 74  Review Scores Communication     242710 non-null  float64\n",
            " 75  Review Scores Location          242423 non-null  float64\n",
            " 76  Review Scores Value             242347 non-null  float64\n",
            " 77  License                         9804 non-null    object\n",
            " 78  Jurisdiction Names              89415 non-null   object\n",
            " 79  Cancellation Policy             326286 non-null  object\n",
            " 80  Calculated host listings count  325689 non-null  float64\n",
            " 81  Reviews per Month               246983 non-null  float64\n",
            " 82  Geolocation                     326287 non-null  object\n",
            " 83  Features                        326092 non-null  object\n",
            " 84  Price                           326287 non-null  float64\n",
            "dtypes: float64(31), int64(2), object(52)\n",
            "memory usage: 1.3+ GB\n"
          ]
        }
      ],
      "source": [
        "df.info()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fBWWhtzoO7_V"
      },
      "source": [
        "### 3.4 Estadísticas descriptivas"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 471
        },
        "id": "0qEeGOEMO7_V",
        "outputId": "21bf5fdb-5519-4561-ba9e-6d56cfe46f73"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/site-packages/cudf/core/dataframe.py:5062: FutureWarning: `datetime_is_numeric` is deprecated. Specify `datetime_is_numeric=True` to silence this warning and adopt the future behavior now.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/site-packages/cudf/core/series.py:3334: FutureWarning: `datetime_is_numeric` is deprecated and will be removed in a future release. Specify `datetime_is_numeric=True` to silence this warning and adopt the future behavior now.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                  id       Host ID  Host Response Rate  Host Listings Count  \\\n",
              "count  326287.000000  3.262870e+05       250845.000000        325971.000000   \n",
              "mean   163143.000000  3.236757e+07           93.408264             9.586000   \n",
              "std     94191.087979  3.174572e+07           17.536835            57.399711   \n",
              "min         0.000000  1.900000e+01            0.000000             0.000000   \n",
              "25%     81571.500000  6.869780e+06           98.000000             1.000000   \n",
              "50%    163143.000000  2.186737e+07          100.000000             1.000000   \n",
              "75%    244714.500000  4.799166e+07          100.000000             3.000000   \n",
              "max    326286.000000  1.350885e+08          100.000000          1114.000000   \n",
              "\n",
              "       Host Total Listings Count       Latitude      Longitude   Accommodates  \\\n",
              "count              325970.000000  326287.000000  326287.000000  326244.000000   \n",
              "mean                    9.586026      38.042816     -15.323924       3.270764   \n",
              "std                    57.399797      22.910029      70.101677       2.037446   \n",
              "min                     0.000000     -38.224427    -123.218712       1.000000   \n",
              "25%                     1.000000      38.923154     -73.968081       2.000000   \n",
              "50%                     1.000000      42.304549       0.090277       2.000000   \n",
              "75%                     3.000000      50.863658      12.342749       4.000000   \n",
              "max                  1114.000000      55.994889     153.637837      18.000000   \n",
              "\n",
              "           Bathrooms       Bedrooms  ...  Review Scores Rating  \\\n",
              "count  325300.000000  325873.000000  ...         243160.000000   \n",
              "mean        1.239482       1.358072  ...             92.880063   \n",
              "std         0.574784       0.921763  ...              8.569521   \n",
              "min         0.000000       0.000000  ...             20.000000   \n",
              "25%         1.000000       1.000000  ...             90.000000   \n",
              "50%         1.000000       1.000000  ...             95.000000   \n",
              "75%         1.000000       2.000000  ...            100.000000   \n",
              "max         8.000000      96.000000  ...            100.000000   \n",
              "\n",
              "       Review Scores Accuracy  Review Scores Cleanliness  \\\n",
              "count           242584.000000              242732.000000   \n",
              "mean                 9.524713                   9.326067   \n",
              "std                  0.855361                   1.038858   \n",
              "min                  2.000000                   2.000000   \n",
              "25%                  9.000000                   9.000000   \n",
              "50%                 10.000000                  10.000000   \n",
              "75%                 10.000000                  10.000000   \n",
              "max                 10.000000                  10.000000   \n",
              "\n",
              "       Review Scores Checkin  Review Scores Communication  \\\n",
              "count          242378.000000                242710.000000   \n",
              "mean                9.691416                     9.708253   \n",
              "std                 0.731702                     0.723143   \n",
              "min                 2.000000                     2.000000   \n",
              "25%                10.000000                    10.000000   \n",
              "50%                10.000000                    10.000000   \n",
              "75%                10.000000                    10.000000   \n",
              "max                10.000000                    10.000000   \n",
              "\n",
              "       Review Scores Location  Review Scores Value  \\\n",
              "count           242423.000000        242347.000000   \n",
              "mean                 9.468215             9.321031   \n",
              "std                  0.805116             0.906478   \n",
              "min                  2.000000             2.000000   \n",
              "25%                  9.000000             9.000000   \n",
              "50%                 10.000000            10.000000   \n",
              "75%                 10.000000            10.000000   \n",
              "max                 10.000000            10.000000   \n",
              "\n",
              "       Calculated host listings count  Reviews per Month          Price  \n",
              "count                   325689.000000      246983.000000  326287.000000  \n",
              "mean                         6.881531           1.486211     138.229041  \n",
              "std                         42.025986           1.752082     149.790527  \n",
              "min                          1.000000           0.010000       0.000000  \n",
              "25%                          1.000000           0.320000      55.000000  \n",
              "50%                          1.000000           0.890000      90.000000  \n",
              "75%                          2.000000           2.040000     150.000000  \n",
              "max                        752.000000         223.000000     999.000000  \n",
              "\n",
              "[8 rows x 33 columns]"
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>Host ID</th>\n",
              "      <th>Host Response Rate</th>\n",
              "      <th>Host Listings Count</th>\n",
              "      <th>Host Total Listings Count</th>\n",
              "      <th>Latitude</th>\n",
              "      <th>Longitude</th>\n",
              "      <th>Accommodates</th>\n",
              "      <th>Bathrooms</th>\n",
              "      <th>Bedrooms</th>\n",
              "      <th>...</th>\n",
              "      <th>Review Scores Rating</th>\n",
              "      <th>Review Scores Accuracy</th>\n",
              "      <th>Review Scores Cleanliness</th>\n",
              "      <th>Review Scores Checkin</th>\n",
              "      <th>Review Scores Communication</th>\n",
              "      <th>Review Scores Location</th>\n",
              "      <th>Review Scores Value</th>\n",
              "      <th>Calculated host listings count</th>\n",
              "      <th>Reviews per Month</th>\n",
              "      <th>Price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>326287.000000</td>\n",
              "      <td>3.262870e+05</td>\n",
              "      <td>250845.000000</td>\n",
              "      <td>325971.000000</td>\n",
              "      <td>325970.000000</td>\n",
              "      <td>326287.000000</td>\n",
              "      <td>326287.000000</td>\n",
              "      <td>326244.000000</td>\n",
              "      <td>325300.000000</td>\n",
              "      <td>325873.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>243160.000000</td>\n",
              "      <td>242584.000000</td>\n",
              "      <td>242732.000000</td>\n",
              "      <td>242378.000000</td>\n",
              "      <td>242710.000000</td>\n",
              "      <td>242423.000000</td>\n",
              "      <td>242347.000000</td>\n",
              "      <td>325689.000000</td>\n",
              "      <td>246983.000000</td>\n",
              "      <td>326287.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>163143.000000</td>\n",
              "      <td>3.236757e+07</td>\n",
              "      <td>93.408264</td>\n",
              "      <td>9.586000</td>\n",
              "      <td>9.586026</td>\n",
              "      <td>38.042816</td>\n",
              "      <td>-15.323924</td>\n",
              "      <td>3.270764</td>\n",
              "      <td>1.239482</td>\n",
              "      <td>1.358072</td>\n",
              "      <td>...</td>\n",
              "      <td>92.880063</td>\n",
              "      <td>9.524713</td>\n",
              "      <td>9.326067</td>\n",
              "      <td>9.691416</td>\n",
              "      <td>9.708253</td>\n",
              "      <td>9.468215</td>\n",
              "      <td>9.321031</td>\n",
              "      <td>6.881531</td>\n",
              "      <td>1.486211</td>\n",
              "      <td>138.229041</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>94191.087979</td>\n",
              "      <td>3.174572e+07</td>\n",
              "      <td>17.536835</td>\n",
              "      <td>57.399711</td>\n",
              "      <td>57.399797</td>\n",
              "      <td>22.910029</td>\n",
              "      <td>70.101677</td>\n",
              "      <td>2.037446</td>\n",
              "      <td>0.574784</td>\n",
              "      <td>0.921763</td>\n",
              "      <td>...</td>\n",
              "      <td>8.569521</td>\n",
              "      <td>0.855361</td>\n",
              "      <td>1.038858</td>\n",
              "      <td>0.731702</td>\n",
              "      <td>0.723143</td>\n",
              "      <td>0.805116</td>\n",
              "      <td>0.906478</td>\n",
              "      <td>42.025986</td>\n",
              "      <td>1.752082</td>\n",
              "      <td>149.790527</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.900000e+01</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-38.224427</td>\n",
              "      <td>-123.218712</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>20.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.010000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>81571.500000</td>\n",
              "      <td>6.869780e+06</td>\n",
              "      <td>98.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>38.923154</td>\n",
              "      <td>-73.968081</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>90.000000</td>\n",
              "      <td>9.000000</td>\n",
              "      <td>9.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>9.000000</td>\n",
              "      <td>9.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.320000</td>\n",
              "      <td>55.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>163143.000000</td>\n",
              "      <td>2.186737e+07</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>42.304549</td>\n",
              "      <td>0.090277</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>95.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.890000</td>\n",
              "      <td>90.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>244714.500000</td>\n",
              "      <td>4.799166e+07</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>50.863658</td>\n",
              "      <td>12.342749</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.040000</td>\n",
              "      <td>150.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>326286.000000</td>\n",
              "      <td>1.350885e+08</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>1114.000000</td>\n",
              "      <td>1114.000000</td>\n",
              "      <td>55.994889</td>\n",
              "      <td>153.637837</td>\n",
              "      <td>18.000000</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>96.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>10.000000</td>\n",
              "      <td>752.000000</td>\n",
              "      <td>223.000000</td>\n",
              "      <td>999.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>8 rows × 33 columns</p>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "df.describe()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LkP5LYdtO7_W",
        "outputId": "dd63c0e9-c095-4efe-e8ee-fd4d600306e3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Index(['id', 'Last Scraped', 'Name', 'Summary', 'Space', 'Description',\n",
              "       'Experiences Offered', 'Neighborhood Overview', 'Notes', 'Transit',\n",
              "       'Access', 'Interaction', 'House Rules', 'Thumbnail Url', 'Medium Url',\n",
              "       'Picture Url', 'XL Picture Url', 'Host ID', 'Host URL', 'Host Name',\n",
              "       'Host Since', 'Host Location', 'Host About', 'Host Response Time',\n",
              "       'Host Response Rate', 'Host Acceptance Rate', 'Host Thumbnail Url',\n",
              "       'Host Picture Url', 'Host Neighbourhood', 'Host Listings Count',\n",
              "       'Host Total Listings Count', 'Host Verifications', 'Street',\n",
              "       'Neighbourhood', 'Neighbourhood Cleansed',\n",
              "       'Neighbourhood Group Cleansed', 'City', 'State', 'Zipcode', 'Market',\n",
              "       'Smart Location', 'Country Code', 'Country', 'Latitude', 'Longitude',\n",
              "       'Property Type', 'Room Type', 'Accommodates', 'Bathrooms', 'Bedrooms',\n",
              "       'Beds', 'Bed Type', 'Amenities', 'Square Feet', 'Security Deposit',\n",
              "       'Cleaning Fee', 'Guests Included', 'Extra People', 'Minimum Nights',\n",
              "       'Maximum Nights', 'Calendar Updated', 'Has Availability',\n",
              "       'Availability 30', 'Availability 60', 'Availability 90',\n",
              "       'Availability 365', 'Calendar last Scraped', 'Number of Reviews',\n",
              "       'First Review', 'Last Review', 'Review Scores Rating',\n",
              "       'Review Scores Accuracy', 'Review Scores Cleanliness',\n",
              "       'Review Scores Checkin', 'Review Scores Communication',\n",
              "       'Review Scores Location', 'Review Scores Value', 'License',\n",
              "       'Jurisdiction Names', 'Cancellation Policy',\n",
              "       'Calculated host listings count', 'Reviews per Month', 'Geolocation',\n",
              "       'Features', 'Price'],\n",
              "      dtype='object')"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "df.columns"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wedud4uTQtiZ",
        "outputId": "5d70cb2e-6dca-497d-8d39-6a977348b0f5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(326287, 85)"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "df.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "63gdNiQiFLn7"
      },
      "outputs": [],
      "source": [
        "import cudf\n",
        "import cuml\n",
        "from cuml.preprocessing import StandardScaler, OneHotEncoder\n",
        "from sklearn.impute import SimpleImputer\n",
        "from sklearn.model_selection import train_test_split\n",
        "# from cudf.core.subword_tokenizer import SubwordTokenizer\n",
        "# import dask_cudf\n",
        "# import dask\n",
        "import numpy as np\n",
        "\n",
        "# Assuming cuDF tokenizer setup and other relevant imports are already done\n",
        "\n",
        "# Define the threshold for unique values to differentiate categorical and textual data\n",
        "unique_value_threshold = 20\n",
        "\n",
        "# Load your data into a cuDF DataFrame\n",
        "# df = cudf.read_csv('your_data.csv')  # Replace with your data source\n",
        "\n",
        "# Separate features and target\n",
        "df_train = df.drop('Price', axis=1)\n",
        "y = df['Price']\n",
        "\n",
        "# Identify numerical columns\n",
        "numerical_cols = df.select_dtypes(include=['float64', 'int64']).columns.tolist()\n",
        "\n",
        "# Initialize lists for categorical and textual columns\n",
        "categorical_cols = []\n",
        "text_cols = []\n",
        "\n",
        "# Iterate over columns and classify them as categorical or textual based on unique values\n",
        "for col in df.columns:\n",
        "    if col not in numerical_cols:\n",
        "        unique_values = df[col].nunique(dropna=False)\n",
        "        if unique_values < unique_value_threshold:\n",
        "            categorical_cols.append(col)\n",
        "        else:\n",
        "            text_cols.append(col)\n",
        "\n",
        "# Split the data\n",
        "X_train, X_test, y_train, y_test = train_test_split(df_train, y, test_size=0.2, random_state=0)\n",
        "\n",
        "# Define important text columns\n",
        "important_text_cols = ['Name', 'Summary', 'Description', 'Neighborhood Overview', 'Transit', 'Access', 'Interaction', 'House Rules']\n",
        "text_cols_important = [column for column in text_cols if column in important_text_cols]\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "1pTNU1eQGySs"
      },
      "outputs": [],
      "source": [
        "# Now create the cuDF DataFrame\n",
        "X_text_train = cudf.DataFrame(X_train[text_cols_important])\n",
        "X_text_test = cudf.DataFrame(X_test[text_cols_important])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4q0g4VSuGIpy",
        "outputId": "7e77b87b-d7c1-4d55-b111-d14b20f4555f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ]
        }
      ],
      "source": [
        "import nltk\n",
        "from nltk.corpus import stopwords\n",
        "\n",
        "nltk.download('stopwords')\n",
        "\n",
        "# List of top 10 languages (modify as needed)\n",
        "languages = ['english', 'spanish', 'chinese', 'arabic', 'portuguese', 'indonesian', 'french', 'russian', 'german']\n",
        "\n",
        "# Create a combined set of stopwords from these languages\n",
        "combined_stopwords = set()\n",
        "for lang in languages:\n",
        "    combined_stopwords.update(stopwords.words(lang))\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "5vFd6DpvFcds"
      },
      "outputs": [],
      "source": [
        "import cudf\n",
        "import re\n",
        "from bs4 import BeautifulSoup\n",
        "import numpy as np\n",
        "\n",
        "# Define the batch size\n",
        "batch_size = 1000  # Adjust this based on your GPU memory\n",
        "\n",
        "# Function to clean a batch of text\n",
        "def clean_batch(batch):\n",
        "    # Remove HTML tags\n",
        "    batch = batch.str.replace('<.*?>', '', regex=True)\n",
        "\n",
        "    # Correct encoding issues and remove special characters\n",
        "    batch = batch.str.replace(\"&amp;\", \"&\").replace(\"&lt;\", \"<\").replace(\"&gt;\", \">\")\n",
        "    batch = batch.str.replace(r'[^a-zA-Z0-9.,!?/:;\\\"\\'\\s]', '', regex=True)\n",
        "\n",
        "    # Replace multiple spaces with a single space\n",
        "    batch = batch.str.replace(r'\\s+', ' ', regex=True).str.strip()\n",
        "\n",
        "    return batch\n",
        "\n",
        "# Function to process text data in batches\n",
        "def process_in_batches(df, column_name):\n",
        "    num_rows = len(df)\n",
        "    num_batches = np.ceil(num_rows / batch_size).astype(int)\n",
        "\n",
        "    for i in range(num_batches):\n",
        "        batch_start = i * batch_size\n",
        "        batch_end = min(batch_start + batch_size, num_rows)\n",
        "        batch = df[column_name].iloc[batch_start:batch_end]\n",
        "        cleaned_batch = clean_batch(batch)\n",
        "        df[column_name].iloc[batch_start:batch_end] = cleaned_batch\n",
        "\n",
        "    return df\n",
        "\n",
        "# Apply cleaning to each important text column\n",
        "for col in important_text_cols:\n",
        "    X_text_train = process_in_batches(X_text_train, col)\n",
        "    X_text_test = process_in_batches(X_text_test, col)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 588
        },
        "id": "pgERuvHrRxof",
        "outputId": "0b9920bd-105f-44a7-b47b-e13609c7b0a8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                     Name  \\\n",
              "67291                    Modern flat in a greenquiet area   \n",
              "41204                           3 blocks from SOCO Living   \n",
              "282576                Piazza Fiume sognando la Dolce Vita   \n",
              "2822    FamilyFriendly Modern Ranch in Great SE Location!   \n",
              "257477           Lovely openspace flat 45m2 near Bastille   \n",
              "\n",
              "                                                  Summary  \\\n",
              "67291   Our two bedroom flat is part of a new building...   \n",
              "41204   Tree Studio in Travis Heights, is 3 blocks fro...   \n",
              "282576  Accogliente e confortevole appartamento recent...   \n",
              "2822    Welcome to this SE Portland familyfriendly hou...   \n",
              "257477  Welcome to a super well located modern flat, p...   \n",
              "\n",
              "                                              Description  \\\n",
              "67291   Our two bedroom flat is part of a new building...   \n",
              "41204   Tree Studio in Travis Heights, is 3 blocks fro...   \n",
              "282576  Accogliente e confortevole appartamento recent...   \n",
              "2822    Welcome to this SE Portland familyfriendly hou...   \n",
              "257477  Welcome to a super well located modern flat, p...   \n",
              "\n",
              "                                    Neighborhood Overview  \\\n",
              "67291   Quiet and friendly families and young couples ...   \n",
              "41204   Despite its proximity to South Congress and th...   \n",
              "282576  L' appartamento situato nel quartiere Salario ...   \n",
              "2822    This is a great neighborhood. It is quiet, cle...   \n",
              "257477                                               <NA>   \n",
              "\n",
              "                                                  Transit  \\\n",
              "67291   17 minutes via foot away from Hackbridge train...   \n",
              "41204   There's a major bus route 3 blocks from the st...   \n",
              "282576  La zona dove si trova l'appartamento ben colle...   \n",
              "2822    It is really easy to get around Portland from ...   \n",
              "257477                                               <NA>   \n",
              "\n",
              "                                                   Access  \\\n",
              "67291                                                <NA>   \n",
              "41204   Refrigerator, Bathroom, Internet, Record Playe...   \n",
              "282576  Gli ospiti possono usufruire dell'intero appar...   \n",
              "2822                                The washer and dryer.   \n",
              "257477                                               <NA>   \n",
              "\n",
              "                                              Interaction  \\\n",
              "67291   We try to be available via emailtelephonein pe...   \n",
              "41204   You will have the studio all to yourself but i...   \n",
              "282576  Sar a disposizione degli ospiti per qualsiasi ...   \n",
              "2822                                                 <NA>   \n",
              "257477                                               <NA>   \n",
              "\n",
              "                                              House Rules  \n",
              "67291   Please ensure you remove your shoes once you a...  \n",
              "41204   Key exchange will be done on day of arrival an...  \n",
              "282576  Facendo parte di un condominio non possibile f...  \n",
              "2822    The home office, garage and tiny house in the ...  \n",
              "257477                                               <NA>  "
            ],
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Name</th>\n",
              "      <th>Summary</th>\n",
              "      <th>Description</th>\n",
              "      <th>Neighborhood Overview</th>\n",
              "      <th>Transit</th>\n",
              "      <th>Access</th>\n",
              "      <th>Interaction</th>\n",
              "      <th>House Rules</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>67291</th>\n",
              "      <td>Modern flat in a greenquiet area</td>\n",
              "      <td>Our two bedroom flat is part of a new building...</td>\n",
              "      <td>Our two bedroom flat is part of a new building...</td>\n",
              "      <td>Quiet and friendly families and young couples ...</td>\n",
              "      <td>17 minutes via foot away from Hackbridge train...</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>We try to be available via emailtelephonein pe...</td>\n",
              "      <td>Please ensure you remove your shoes once you a...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41204</th>\n",
              "      <td>3 blocks from SOCO Living</td>\n",
              "      <td>Tree Studio in Travis Heights, is 3 blocks fro...</td>\n",
              "      <td>Tree Studio in Travis Heights, is 3 blocks fro...</td>\n",
              "      <td>Despite its proximity to South Congress and th...</td>\n",
              "      <td>There's a major bus route 3 blocks from the st...</td>\n",
              "      <td>Refrigerator, Bathroom, Internet, Record Playe...</td>\n",
              "      <td>You will have the studio all to yourself but i...</td>\n",
              "      <td>Key exchange will be done on day of arrival an...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>282576</th>\n",
              "      <td>Piazza Fiume sognando la Dolce Vita</td>\n",
              "      <td>Accogliente e confortevole appartamento recent...</td>\n",
              "      <td>Accogliente e confortevole appartamento recent...</td>\n",
              "      <td>L' appartamento situato nel quartiere Salario ...</td>\n",
              "      <td>La zona dove si trova l'appartamento ben colle...</td>\n",
              "      <td>Gli ospiti possono usufruire dell'intero appar...</td>\n",
              "      <td>Sar a disposizione degli ospiti per qualsiasi ...</td>\n",
              "      <td>Facendo parte di un condominio non possibile f...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2822</th>\n",
              "      <td>FamilyFriendly Modern Ranch in Great SE Location!</td>\n",
              "      <td>Welcome to this SE Portland familyfriendly hou...</td>\n",
              "      <td>Welcome to this SE Portland familyfriendly hou...</td>\n",
              "      <td>This is a great neighborhood. It is quiet, cle...</td>\n",
              "      <td>It is really easy to get around Portland from ...</td>\n",
              "      <td>The washer and dryer.</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>The home office, garage and tiny house in the ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>257477</th>\n",
              "      <td>Lovely openspace flat 45m2 near Bastille</td>\n",
              "      <td>Welcome to a super well located modern flat, p...</td>\n",
              "      <td>Welcome to a super well located modern flat, p...</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "X_text_train.head(5)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "jVv_Fuz5UcDu"
      },
      "outputs": [],
      "source": [
        "# Concatenate the text columns\n",
        "for df in [X_text_train, X_text_test]:\n",
        "    # Convert all columns to string type\n",
        "    for col in important_text_cols:\n",
        "        df[col] = df[col].astype('str')\n",
        "\n",
        "    # Initialize the combined_text column\n",
        "    df['combined_text'] = ''\n",
        "\n",
        "    # Concatenate each column to the combined_text\n",
        "    for col in important_text_cols:\n",
        "        df['combined_text'] = df['combined_text'] + df[col] + ' '\n",
        "\n",
        "    # Strip any trailing whitespaces\n",
        "    df['combined_text'] = df['combined_text'].str.strip()\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9_iMi_k1Uv_e",
        "outputId": "8f38a06e-85d9-4a7c-c0ce-b89fc6ab66f1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "67291                                                  <NA>\n",
              "41204     3 blocks from SOCO Living Tree Studio in Travi...\n",
              "282576    Piazza Fiume sognando la Dolce Vita Accoglient...\n",
              "2822                                                   <NA>\n",
              "257477                                                 <NA>\n",
              "                                ...                        \n",
              "122579                                                 <NA>\n",
              "304137                                                 <NA>\n",
              "152315                                                 <NA>\n",
              "117952                                                 <NA>\n",
              "305711                                                 <NA>\n",
              "Name: combined_text, Length: 261029, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "X_text_train['combined_text']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 284,
          "referenced_widgets": [
            "072c2330f61a48f4a2b365c8a0baea0d",
            "c239cbea081b46c7b1833d4cac549c56",
            "8412a735a6064f7ba9873e74bf661874",
            "aa250ad4c3ff4666bc9fdcaa7a3500f6",
            "285b1bfd052b4368ae67bda600968d53",
            "5373e45dce56410e84938e266d146c96",
            "eaee7f1e6a754915a8fe86a5c6620473",
            "ed28017e9258421383e9f33adc2b1bf6",
            "1746380716bf4494bed8ee9e7880f483",
            "1bc0673a98d4471ca1579cb29e972cb1",
            "1fef3442448f4c759924d676c76c7faf",
            "4f3145a9ecb44f20b18ec319e9f1bfc0",
            "ed96cee05a5b48ecb3bdd6e43d4e578e",
            "4552ffd333c242cfbdf187d80872040f",
            "4003bfc4c6a7432a832cd9af414c098c",
            "f3327c46b67f434aa6b21881504ac4a2",
            "a2a09e53bed64212868e1a32f7345872",
            "3b391b2916ad4b1291efc93f1944d4ee",
            "60e312d335bc457394f0e4919d97f241",
            "400863841dfd4be9a986d00236b05879",
            "43c290549cf5440b9ef7ec4b330cd9b2",
            "cbd3c9725a6342fbaca0c26a9250c9a0",
            "e1ee2475d363447782b1fd16837ffedb",
            "c2a0dd9a889b45d788bc5fd9bc0ac1a1",
            "50ba5cc20eb542549ee69ec99a2c6af8",
            "2d5ef0ee53c34a6e90a4a1a679f0caff",
            "7c2bb187bd684cc29b7aa05f157d15d3",
            "a39be9b4bb6c4383a9967dab6de30f29",
            "2e0cb0c287234a92a8260fb95d056fb5",
            "9291b9a447284dd4813229c09ad7e13e",
            "71030b081fb84a11b59a5a780352a9a7",
            "f0cddbc7d0304cae81661f17f71e3c0f",
            "ba53b5c33a7249afbabdafc4e790e4e8",
            "493ba3002f004f949e98cdb91a48cb0d",
            "1db817ae82aa43fcb70206e00653cd81",
            "0c9d12c7c9e24a6698753e2338952043",
            "fa250a63821b4c719d7ba2bd022e00ad",
            "3a74879b4a1340f2a378d611b7fdd6ed",
            "3eb9105e683f47ecbafa3dc730d5b774",
            "bec3d34a45e940589f9e03bb164ce262",
            "44349162382e43d0bdab378f650b8977",
            "98edae3fc637429cbd380d88ae254ec1",
            "4cc7dbf2eb134f309190923882b62d76",
            "264ba538038c4b0aa0ac9831964e7675",
            "4125aa36719741a1a2b4def934193962",
            "41d9e3fcd4b5461a9290cba46d9e0f76",
            "b510e3aaf6a442f681f03e13af862c0d",
            "2fcbdfe8e11549dfadcfb21d0cc40c30",
            "d4d8c7abf26c4a6fb13e65d7e0789ee9",
            "6f279bbad6e847ad92919a4537432136",
            "6040b7a42fe3481ebcf1e972d32c56e7",
            "c334d67683f740b4a418822f4d09afdd",
            "4ebe50ab4ae44fd582d827b08b34095b",
            "ac0042cd4e2148c7aabd5c5bcfc3abd8",
            "54cf3ad96d5a438fb2abf0d867302d5e"
          ]
        },
        "id": "trbqnEFySfa9",
        "outputId": "e8c7308b-4e62-446d-f779-02fa19a6e1ed"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/29.0 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "072c2330f61a48f4a2b365c8a0baea0d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "vocab.txt:   0%|          | 0.00/996k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4f3145a9ecb44f20b18ec319e9f1bfc0"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/1.96M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e1ee2475d363447782b1fd16837ffedb"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "config.json:   0%|          | 0.00/466 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "493ba3002f004f949e98cdb91a48cb0d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "model.safetensors:   0%|          | 0.00/542M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4125aa36719741a1a2b4def934193962"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFDistilBertModel: ['vocab_layer_norm.weight', 'vocab_layer_norm.bias', 'vocab_transform.weight', 'vocab_transform.bias', 'vocab_projector.bias']\n",
            "- This IS expected if you are initializing TFDistilBertModel from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing TFDistilBertModel from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "All the weights of TFDistilBertModel were initialized from the PyTorch model.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFDistilBertModel for predictions without further training.\n"
          ]
        }
      ],
      "source": [
        "# Load BERT tokenizer and model\n",
        "import tensorflow as tf\n",
        "from transformers import TFBertModel, BertTokenizer\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "# tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-uncased')\n",
        "# bert_model = TFBertModel.from_pretrained('bert-base-multilingual-uncased')\n",
        "seed = 42\n",
        "\n",
        "from transformers import DistilBertTokenizer, TFDistilBertModel\n",
        "\n",
        "tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-multilingual-cased')\n",
        "model = TFDistilBertModel.from_pretrained('distilbert-base-multilingual-cased')\n",
        "\n",
        "# Rest of your code for processing...\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "AhsRgT2sSpwn"
      },
      "outputs": [],
      "source": [
        "from transformers import BertTokenizer\n",
        "import tensorflow as tf\n",
        "\n",
        "max_length = 300  # Reduce the sequence length if needed\n",
        "\n",
        "\n",
        "def batch_encode_cudf(cudf_series, batch_size=512):\n",
        "    input_ids = []\n",
        "    attention_masks = []\n",
        "\n",
        "    # Convert cuDF Series to Pandas Series for compatibility\n",
        "    texts = cudf_series.to_pandas()\n",
        "\n",
        "    for i in range(0, len(texts), batch_size):\n",
        "        batch = texts[i:i+batch_size]\n",
        "        encoded = tokenizer.batch_encode_plus(\n",
        "            batch,\n",
        "            add_special_tokens=True,\n",
        "            padding='max_length',\n",
        "            truncation=True,\n",
        "            max_length=max_length,\n",
        "            return_attention_mask=True,\n",
        "            return_tensors='tf'\n",
        "        )\n",
        "        input_ids.append(encoded['input_ids'])\n",
        "        attention_masks.append(encoded['attention_mask'])\n",
        "\n",
        "    input_ids = tf.concat(input_ids, axis=0)\n",
        "    attention_masks = tf.concat(attention_masks, axis=0)\n",
        "\n",
        "    # Optionally, convert results back to cuDF DataFrames for further GPU processing\n",
        "    input_ids = cudf.DataFrame.from_pandas(pd.DataFrame(input_ids.numpy()))\n",
        "    attention_masks = cudf.DataFrame.from_pandas(pd.DataFrame(attention_masks.numpy()))\n",
        "\n",
        "    return input_ids, attention_masks"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "LYqGorNVXMva"
      },
      "outputs": [],
      "source": [
        "def batch_encode(texts, batch_size=512):\n",
        "    input_ids = []\n",
        "    attention_masks = []\n",
        "\n",
        "    # Convert cuDF Series to Pandas Series and handle null values\n",
        "    if isinstance(texts, cudf.Series):\n",
        "        texts = texts.to_pandas().fillna(\"\")\n",
        "\n",
        "    # Ensure all data are strings\n",
        "    texts = texts.astype(str)\n",
        "\n",
        "    for i in range(0, len(texts), batch_size):\n",
        "        batch = texts[i:i+batch_size]\n",
        "        encoded = tokenizer.batch_encode_plus(\n",
        "            batch,\n",
        "            add_special_tokens=True,\n",
        "            padding='max_length',\n",
        "            truncation=True,\n",
        "            max_length=max_length,\n",
        "            return_attention_mask=True,\n",
        "            return_tensors='tf'\n",
        "        )\n",
        "        input_ids.append(encoded['input_ids'])\n",
        "        attention_masks.append(encoded['attention_mask'])\n",
        "\n",
        "    input_ids = tf.concat(input_ids, axis=0)\n",
        "    attention_masks = tf.concat(attention_masks, axis=0)\n",
        "    return input_ids, attention_masks\n",
        "\n",
        "# Example usage\n",
        "train_input_ids, train_attention_masks = batch_encode(X_text_train['combined_text'])\n",
        "test_input_ids, test_attention_masks = batch_encode(X_text_test['combined_text'])\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "3O-_kbd_Sw11"
      },
      "outputs": [],
      "source": [
        "def create_dataset(input_ids, attention_masks, labels, batch_size=32):\n",
        "    dataset = tf.data.Dataset.from_tensor_slices((\n",
        "        {\n",
        "            'input_ids': input_ids,\n",
        "            'attention_mask': attention_masks\n",
        "        },\n",
        "        labels\n",
        "    ))\n",
        "    dataset = dataset.shuffle(len(labels)).batch(batch_size).prefetch(tf.data.AUTOTUNE)\n",
        "    return dataset\n",
        "\n",
        "# Scaling the labels\n",
        "price_scaler = StandardScaler()\n",
        "price_scaler.fit(y_train.to_numpy().reshape(-1, 1))\n",
        "train_labels = price_scaler.transform(y_train.to_numpy().reshape(-1, 1))#.reshape(-1, 1))\n",
        "test_labels = price_scaler.transform(y_test.to_numpy().reshape(-1, 1))#.reshape(-1, 1))\n",
        "\n",
        "# labels = [...]  # Your label data\n",
        "train_dataset = create_dataset(train_input_ids, train_attention_masks, train_labels)\n",
        "test_dataset = create_dataset(test_input_ids, test_attention_masks, test_labels)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Train model text only"
      ],
      "metadata": {
        "id": "FTv57269PfPS"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true,
          "base_uri": "https://localhost:8080/"
        },
        "id": "SLXFIBQmSywh",
        "outputId": "8af8d3f2-3554-417d-d188-8d0d715190cc"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFDistilBertModel: ['vocab_layer_norm.weight', 'vocab_transform.bias', 'vocab_projector.bias', 'vocab_transform.weight', 'vocab_layer_norm.bias']\n",
            "- This IS expected if you are initializing TFDistilBertModel from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing TFDistilBertModel from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "All the weights of TFDistilBertModel were initialized from the PyTorch model.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFDistilBertModel for predictions without further training.\n"
          ]
        }
      ],
      "source": [
        "# Define the model\n",
        "bert_model = TFDistilBertModel.from_pretrained('distilbert-base-multilingual-cased')\n",
        "\n",
        "def build_model():\n",
        "    input_ids = tf.keras.layers.Input(shape=(max_length,), dtype=tf.int32, name='input_ids')\n",
        "    attention_mask = tf.keras.layers.Input(shape=(max_length,), dtype=tf.int32, name='attention_mask')\n",
        "\n",
        "    bert_output = bert_model([input_ids, attention_mask])\n",
        "    cls_token_output = bert_output.last_hidden_state[:, 0, :]\n",
        "    output = tf.keras.layers.Dense(1)(cls_token_output)\n",
        "\n",
        "    model = tf.keras.Model(inputs=[input_ids, attention_mask], outputs=output)\n",
        "    model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=5e-5, epsilon=1e-8),\n",
        "                  loss=tf.keras.losses.MeanSquaredError(),\n",
        "                  metrics=[tf.keras.metrics.MeanAbsoluteError()])\n",
        "    return model\n",
        "\n",
        "model = build_model()\n",
        "\n",
        "epochs=5\n",
        "\n",
        "history = model.fit(\n",
        "    train_dataset,\n",
        "    validation_data=test_dataset,\n",
        "    epochs=epochs\n",
        ")\n",
        "model.save_weights(f'trained_bert_{epochs}.h5')\n",
        "# Loading model\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true
        },
        "id": "fe95ebQT7XUC"
      },
      "outputs": [],
      "source": [
        "model.save_weights(f'trained_bert_2ymedia.h5')\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mx26hARHh5is"
      },
      "outputs": [],
      "source": [
        "model.load_weights('/content/drive/MyDrive/Colab Notebooks/Datasets/trained_bert.h5')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cqnQKuTCLFui"
      },
      "source": [
        "### Kaggle test data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true
        },
        "id": "fuffeXqDLId3"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "# Assuming 'basic_cleaning' and 'batch_encode' functions are already defined\n",
        "\n",
        "# Load the test data\n",
        "file_path2 = './obligatorio_DL/private_data_to_predict.csv'\n",
        "kaggle_test = cudf.read_csv(file_path2)\n",
        "\n",
        "# Apply basic cleaning\n",
        "\n",
        "\n",
        "# Apply cleaning to each important text column\n",
        "for col in important_text_cols:\n",
        "    kaggle_test = process_in_batches(kaggle_test, col)\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true
        },
        "id": "cxAL2l0BihAw"
      },
      "outputs": [],
      "source": [
        "# kaggle_test = data_for_kaggle.applymap(basic_cleaning)\n",
        "kaggle_test['combined_text'] = ''\n",
        "\n",
        "for col in important_text_cols:\n",
        "    kaggle_test[col] = kaggle_test[col].astype('str')\n",
        "    kaggle_test['combined_text'] = kaggle_test['combined_text'] + kaggle_test[col] + ' '\n",
        "    kaggle_test['combined_text'] = kaggle_test['combined_text'].str.strip()\n",
        "\n",
        "kaggle_input_ids, kaggle_attention_masks = batch_encode(kaggle_test['combined_text'])\n",
        "\n",
        "# Make predictions\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CQIEQnpJillm"
      },
      "outputs": [],
      "source": [
        "predictions = model.predict(x=[kaggle_input_ids, kaggle_attention_masks])\n",
        "\n",
        "# Rescale predictions\n",
        "y_pred = price_scaler.inverse_transform(predictions)\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "my6TXIV_o73r",
        "outputId": "5665ddf3-ea77-4d48-e19e-217706902c30"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array([[207.5733  ],\n",
              "       [109.61352 ],\n",
              "       [118.88889 ],\n",
              "       ...,\n",
              "       [118.888885],\n",
              "       [118.888885],\n",
              "       [118.888885]], dtype=float32)"
            ]
          },
          "execution_count": 31,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vPp7bt27ozID"
      },
      "outputs": [],
      "source": [
        "# Prepare output\n",
        "test_ids = kaggle_test['id']\n",
        "output = np.hstack((test_ids.to_numpy().reshape(-1, 1), y_pred))\n",
        "df = pd.DataFrame(output, columns=['id', 'expected'])\n",
        "\n",
        "# Handle any NaNs\n",
        "df['expected'] = df['expected'].fillna(0)\n",
        "\n",
        "# Save to CSV\n",
        "df.to_csv(\"output_to_submit.csv\", index=False)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rCjeyrVn6tYM"
      },
      "source": [
        "## Incorporating Numerical and Categorical data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "253BZL51-Lm3"
      },
      "outputs": [],
      "source": [
        "numerical_cols.remove('Price')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "_DuG9ash9TH8"
      },
      "outputs": [],
      "source": [
        "# Scaling numerical columns\n",
        "import cupy as cp\n",
        "\n",
        "scaler = cuml.preprocessing.StandardScaler()\n",
        "X_train_numerical = scaler.fit_transform(X_train[numerical_cols])\n",
        "X_test_numerical = scaler.transform(X_test[numerical_cols])\n",
        "\n",
        "# Encoding categorical columns\n",
        "encoder = cuml.preprocessing.OneHotEncoder(handle_unknown='ignore')\n",
        "X_train_categorical = encoder.fit_transform(X_train[categorical_cols])\n",
        "X_test_categorical = encoder.transform(X_test[categorical_cols])\n",
        "\n",
        "# Handling missing values (if needed)\n",
        "# You can use cuDF's fillna() method or cuml's SimpleImputer\n",
        "# Example for numerical columns:\n",
        "X_train_numerical = X_train_numerical.fillna(X_train_numerical.mean())\n",
        "X_test_numerical = X_test_numerical.fillna(X_test_numerical.mean())\n",
        "\n",
        "# Convert to Pandas DataFrame\n",
        "X_train_numerical_pd = X_train_numerical.to_pandas()\n",
        "# X_train_categorical_pd = X_train_categorical.to_pandas()\n",
        "X_test_numerical_pd = X_test_numerical.to_pandas()\n",
        "# X_test_categorical_pd = X_test_categorical.to_pandas()\n",
        "# Converting coo_matrix to dense format and then to Pandas DataFrame\n",
        "# X_train_categorical_pd = pd.DataFrame(X_train_categorical.toarray())\n",
        "# X_test_categorical_pd = pd.DataFrame(X_test_categorical.toarray())\n",
        "# Convert the cuML OneHotEncoder output (sparse matrix) to a dense CuPy array\n",
        "# Convert the CuPy array to a NumPy array\n",
        "# Convert the cuML OneHotEncoder output (sparse matrix) to a dense CuPy array\n",
        "X_train_categorical_cupy = cp.asarray(X_train_categorical.toarray())\n",
        "X_test_categorical_cupy = cp.asarray(X_test_categorical.toarray())\n",
        "\n",
        "X_train_categorical_np = X_train_categorical_cupy.get()\n",
        "X_test_categorical_np = X_test_categorical_cupy.get()\n",
        "\n",
        "# Convert the NumPy array to a TensorFlow tensor\n",
        "X_train_categorical_tf = tf.convert_to_tensor(X_train_categorical_np)\n",
        "X_test_categorical_tf = tf.convert_to_tensor(X_test_categorical_np)\n",
        "\n",
        "\n",
        "# Convert to TensorFlow tensors\n",
        "X_train_numerical_tf = tf.convert_to_tensor(X_train_numerical_pd.values, dtype=tf.float32)\n",
        "# X_train_categorical_tf = [tf.convert_to_tensor(X_train_categorical_pd[col].values, dtype=tf.int32) for col in X_train_categorical_pd.columns]\n",
        "X_test_numerical_tf = tf.convert_to_tensor(X_test_numerical_pd.values, dtype=tf.float32)\n",
        "# X_test_categorical_tf = [tf.convert_to_tensor(X_test_categorical_pd[col].values, dtype=tf.int32) for col in X_test_categorical_pd.columns]\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HskQ2nsYDPNI",
        "outputId": "d33d1f8c-3ad6-459d-ab21-01e33b5aada5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Encoded Column Names: ['Experiences Offered_1' 'Experiences Offered_business'\n",
            " 'Experiences Offered_family' 'Experiences Offered_none'\n",
            " 'Experiences Offered_romantic' 'Experiences Offered_social'\n",
            " 'Host Response Time_-122.41000020536359'\n",
            " 'Host Response Time_a few days or more' 'Host Response Time_within a day'\n",
            " 'Host Response Time_within a few hours'\n",
            " 'Host Response Time_within an hour' 'Host Response Time_None'\n",
            " 'Room Type_9' 'Room Type_Entire home/apt' 'Room Type_Private room'\n",
            " 'Room Type_Shared room' 'Bed Type_9' 'Bed Type_Airbed' 'Bed Type_Couch'\n",
            " 'Bed Type_Futon' 'Bed Type_Pull-out Sofa' 'Bed Type_Real Bed'\n",
            " 'Has Availability_0.21' 'Has Availability_t' 'Has Availability_None'\n",
            " 'Cancellation Policy_flexible' 'Cancellation Policy_flexible_new'\n",
            " 'Cancellation Policy_moderate' 'Cancellation Policy_moderate_new'\n",
            " 'Cancellation Policy_no_refunds' 'Cancellation Policy_strict'\n",
            " 'Cancellation Policy_strict_new' 'Cancellation Policy_super_strict_30'\n",
            " 'Cancellation Policy_super_strict_30_new'\n",
            " 'Cancellation Policy_super_strict_60'\n",
            " 'Cancellation Policy_super_strict_60_new' 'Cancellation Policy_None']\n",
            "length of encoded column names: 37\n"
          ]
        }
      ],
      "source": [
        "# Retrieve new column names\n",
        "encoded_column_names = encoder.get_feature_names(categorical_cols)\n",
        "\n",
        "# Print the new column names\n",
        "print(\"Encoded Column Names:\", encoded_column_names)\n",
        "print(\"length of encoded column names:\",len(encoded_column_names))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XeI7fvh8BN_5",
        "outputId": "b3013bb5-2052-4000-8a10-eadc84a82539"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Shapes of Train and Test Data Components:\n",
            "-----------------------------------------\n",
            "X_train_numerical_tf shape: (261029, 32)\n",
            "X_test_numerical_tf shape: (65258, 32)\n",
            "X_train_categorical_tf shape: (261029, 37)\n",
            "X_test_categorical_tf shape: (65258, 37)\n",
            "train_input_ids shape: (261029, 300)\n",
            "test_input_ids shape: (65258, 300)\n",
            "train_attention_masks shape: (261029, 300)\n",
            "test_attention_masks shape: (65258, 300)\n"
          ]
        }
      ],
      "source": [
        "# Assuming the following variables are already defined:\n",
        "# X_train_numerical_tf, X_test_numerical_tf, X_train_categorical_tf, X_test_categorical_tf\n",
        "# train_input_ids, test_input_ids, train_attention_masks, test_attention_masks\n",
        "\n",
        "print(\"Shapes of Train and Test Data Components:\")\n",
        "print(\"-----------------------------------------\")\n",
        "\n",
        "# Print shapes of numerical data\n",
        "print(\"X_train_numerical_tf shape:\", X_train_numerical_tf.shape)\n",
        "print(\"X_test_numerical_tf shape:\", X_test_numerical_tf.shape)\n",
        "\n",
        "# Print shapes of categorical data\n",
        "print(\"X_train_categorical_tf shape:\", X_train_categorical_tf.shape)\n",
        "print(\"X_test_categorical_tf shape:\", X_test_categorical_tf.shape)\n",
        "\n",
        "# Print shapes of tokenized inputs\n",
        "print(\"train_input_ids shape:\", train_input_ids.shape)\n",
        "print(\"test_input_ids shape:\", test_input_ids.shape)\n",
        "print(\"train_attention_masks shape:\", train_attention_masks.shape)\n",
        "print(\"test_attention_masks shape:\", test_attention_masks.shape)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "42DhWKW9jm1u",
        "outputId": "e30c4f59-b415-4823-d456-cc1ece3d5121"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFDistilBertModel: ['vocab_layer_norm.weight', 'vocab_layer_norm.bias', 'vocab_transform.weight', 'vocab_transform.bias', 'vocab_projector.bias']\n",
            "- This IS expected if you are initializing TFDistilBertModel from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing TFDistilBertModel from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "All the weights of TFDistilBertModel were initialized from the PyTorch model.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFDistilBertModel for predictions without further training.\n"
          ]
        }
      ],
      "source": [
        "# tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-multilingual-cased')\n",
        "bert_model = TFDistilBertModel.from_pretrained('distilbert-base-multilingual-cased')"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.callbacks import ModelCheckpoint\n",
        "\n",
        "# Define the checkpoint callback\n",
        "checkpoint_filepath = 'model_weights_epoch_{epoch:02d}.h5'\n",
        "model_checkpoint_callback = ModelCheckpoint(\n",
        "    filepath=checkpoint_filepath,\n",
        "    save_weights_only=True,\n",
        "    monitor='val_loss',\n",
        "    mode='min',\n",
        "    save_best_only=False,\n",
        "    save_freq='epoch'\n",
        ")"
      ],
      "metadata": {
        "id": "wzrzy8R9SfJ_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "hZmVolS-BkuS"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow.keras.layers import Input, Dense, Dropout\n",
        "from tensorflow.keras.regularizers import l2\n",
        "from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau\n",
        "\n",
        "num_numerical_features = len(numerical_cols)\n",
        "num_categorical_features = len(encoded_column_names)\n",
        "\n",
        "input_ids = tf.keras.layers.Input(shape=(max_length,), dtype=tf.int32, name='input_ids')\n",
        "attention_mask = tf.keras.layers.Input(shape=(max_length,), dtype=tf.int32, name='attention_mask')\n",
        "# bert_output = bert_model([input_ids, attention_mask]).last_hidden_state[:, 0, :]\n",
        "bert_output = bert_model([input_ids, attention_mask]).last_hidden_state[:, 0, :]\n",
        "\n",
        "bert_dense = tf.keras.layers.Dense(64, activation='relu')(bert_output)\n",
        "\n",
        "# Numerical Input\n",
        "numerical_input = tf.keras.layers.Input(shape=(num_numerical_features,), name='numerical_data')\n",
        "numerical_dense = tf.keras.layers.Dense(32, activation='relu', kernel_regularizer=l2(0.01))(numerical_input)\n",
        "\n",
        "# Categorical Input\n",
        "categorical_input = tf.keras.layers.Input(shape=(num_categorical_features,), name='categorical_data')\n",
        "categorical_dense = tf.keras.layers.Dense(32, activation='relu', kernel_regularizer=l2(0.01))(categorical_input)\n",
        "\n",
        "\n",
        "concatenated = tf.keras.layers.concatenate([bert_dense, numerical_dense, categorical_dense])\n",
        "# Concatenate all inputs\n",
        "dropout_layer = Dropout(0.5)(concatenated)\n",
        "\n",
        "# Additional layers as needed\n",
        "output_layer = tf.keras.layers.Dense(1)(dropout_layer)\n",
        "\n",
        "# Build model\n",
        "model = tf.keras.Model(inputs=[input_ids, attention_mask, numerical_input, categorical_input], outputs=output_layer)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "y7ongVuZBqrB"
      },
      "outputs": [],
      "source": [
        "model.compile(optimizer='adam', loss='mean_squared_error', metrics=['mae'])\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Import the W&B Python Library and log into W&B\n",
        "import wandb\n",
        "\n",
        "wandb.login()\n",
        "\n",
        "#Creamos un proyecto en WandB a través de su interfaz\n",
        "project = \"obligatorio_dl\"\n",
        "entity = \"marian-ai\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 104
        },
        "id": "HAo0NPhVTAAc",
        "outputId": "c4bbd66d-472c-4928-aad7-71c380eb38b3"
      },
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ],
            "application/javascript": [
              "\n",
              "        window._wandbApiKey = new Promise((resolve, reject) => {\n",
              "            function loadScript(url) {\n",
              "            return new Promise(function(resolve, reject) {\n",
              "                let newScript = document.createElement(\"script\");\n",
              "                newScript.onerror = reject;\n",
              "                newScript.onload = resolve;\n",
              "                document.body.appendChild(newScript);\n",
              "                newScript.src = url;\n",
              "            });\n",
              "            }\n",
              "            loadScript(\"https://cdn.jsdelivr.net/npm/postmate/build/postmate.min.js\").then(() => {\n",
              "            const iframe = document.createElement('iframe')\n",
              "            iframe.style.cssText = \"width:0;height:0;border:none\"\n",
              "            document.body.appendChild(iframe)\n",
              "            const handshake = new Postmate({\n",
              "                container: iframe,\n",
              "                url: 'https://wandb.ai/authorize'\n",
              "            });\n",
              "            const timeout = setTimeout(() => reject(\"Couldn't auto authenticate\"), 5000)\n",
              "            handshake.then(function(child) {\n",
              "                child.on('authorize', data => {\n",
              "                    clearTimeout(timeout)\n",
              "                    resolve(data)\n",
              "                });\n",
              "            });\n",
              "            })\n",
              "        });\n",
              "    "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[34m\u001b[1mwandb\u001b[0m: Logging into wandb.ai. (Learn how to deploy a W&B server locally: https://wandb.me/wandb-server)\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: You can find your API key in your browser here: https://wandb.ai/authorize\n",
            "wandb: Paste an API key from your profile and hit enter, or press ctrl+c to quit:"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            " ··········\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[34m\u001b[1mwandb\u001b[0m: Appending key for api.wandb.ai to your netrc file: /root/.netrc\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint\n",
        "from tensorflow.keras.optimizers import Adam, SGD\n",
        "\n",
        "# Define the sweep configuration\n",
        "sweep_config = {\n",
        "    'method': 'bayes',  # Bayesian optimization\n",
        "    'metric': {\n",
        "        'name': 'val_loss',\n",
        "        'goal': 'minimize'\n",
        "    },\n",
        "    'parameters': {\n",
        "        'learning_rate': {\n",
        "            'min': 1e-5,\n",
        "            'max': 1e-3\n",
        "        },\n",
        "        'batch_size': {\n",
        "            'values': [64, 128, 256]\n",
        "        },\n",
        "        # Add other hyperparameters here\n",
        "    }\n",
        "}\n",
        "\n",
        "def train():\n",
        "    with wandb.init() as run:\n",
        "        config = run.config\n",
        "\n",
        "        # Callbacks\n",
        "        early_stopping = EarlyStopping(monitor='val_loss', patience=3)\n",
        "        reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=2, min_lr=1e-6)\n",
        "        model_checkpoint_callback = ModelCheckpoint(\n",
        "            filepath='model_weights_epoch_{epoch:02d}.h5',\n",
        "            save_weights_only=True,\n",
        "            save_best_only=False,\n",
        "            save_freq='epoch'\n",
        "        )\n",
        "\n",
        "        # Build and compile your model here using config parameters\n",
        "        # For example, setting the optimizer\n",
        "        optimizer = Adam(learning_rate=config.learning_rate)\n",
        "        # model.compile(optimizer=optimizer, ...)\n",
        "\n",
        "        # Fit the model\n",
        "        history = model.fit(\n",
        "            {\n",
        "                'input_ids': train_input_ids,\n",
        "                'attention_mask': train_attention_masks,\n",
        "                'numerical_data': X_train_numerical_tf,\n",
        "                'categorical_data': X_train_categorical_tf\n",
        "            },\n",
        "            train_labels,\n",
        "            validation_data=(\n",
        "                {\n",
        "                    'input_ids': test_input_ids,\n",
        "                    'attention_mask': test_attention_masks,\n",
        "                    'numerical_data': X_test_numerical_tf,\n",
        "                    'categorical_data': X_test_categorical_tf\n",
        "                },\n",
        "                test_labels\n",
        "            ),\n",
        "            epochs=10,  # You can also use config.epochs if defined in sweep_config\n",
        "            batch_size=int(config.batch_size),\n",
        "            callbacks=[early_stopping, reduce_lr, model_checkpoint_callback]\n",
        "        )\n",
        "# Initialize the sweep\n",
        "sweep_id = wandb.sweep(sweep_config, project=project)\n",
        "\n",
        "# Run the sweep agent\n",
        "wandb.agent(sweep_id, function=train, count=10, project=project)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "174NSAjjftuc",
        "outputId": "15377015-9679-4bec-dde0-e22dfe71b52f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Create sweep with ID: 24f4bwy8\n",
            "Sweep URL: https://wandb.ai/marian-ai/obligatorio_dl/sweeps/24f4bwy8\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[34m\u001b[1mwandb\u001b[0m: Agent Starting Run: bechg4rf with config:\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \tbatch_size: 256\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \tlearning_rate: 0.0005375532911570234\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Tracking run with wandb version 0.16.0"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Run data is saved locally in <code>/content/drive/MyDrive/Colab Notebooks/Datasets/wandb/run-20231124_132247-bechg4rf</code>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Syncing run <strong><a href='https://wandb.ai/marian-ai/obligatorio_dl/runs/bechg4rf' target=\"_blank\">super-sweep-1</a></strong> to <a href='https://wandb.ai/marian-ai/obligatorio_dl' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>Sweep page: <a href='https://wandb.ai/marian-ai/obligatorio_dl/sweeps/24f4bwy8' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/sweeps/24f4bwy8</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View project at <a href='https://wandb.ai/marian-ai/obligatorio_dl' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View sweep at <a href='https://wandb.ai/marian-ai/obligatorio_dl/sweeps/24f4bwy8' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/sweeps/24f4bwy8</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View run at <a href='https://wandb.ai/marian-ai/obligatorio_dl/runs/bechg4rf' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/runs/bechg4rf</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Traceback (most recent call last):\n",
            "  File \"<ipython-input-36-93fce9622464>\", line 43, in train\n",
            "    history = model.fit(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\n",
            "    raise e.with_traceback(filtered_tb) from None\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/tensorflow/python/eager/execute.py\", line 60, in quick_execute\n",
            "    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,\n",
            "tensorflow.python.framework.errors_impl.ResourceExhaustedError: Graph execution error:\n",
            "\n",
            "Detected at node model/tf_distil_bert_model_1/distilbert/transformer/layer_._0/ffn/Gelu/Erf defined at (most recent call last):\n",
            "  File \"/usr/lib/python3.10/threading.py\", line 973, in _bootstrap\n",
            "\n",
            "  File \"/usr/lib/python3.10/threading.py\", line 1016, in _bootstrap_inner\n",
            "\n",
            "  File \"/usr/lib/python3.10/threading.py\", line 953, in run\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/wandb/agents/pyagent.py\", line 298, in _run_job\n",
            "\n",
            "  File \"<ipython-input-36-93fce9622464>\", line 43, in train\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1783, in fit\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1377, in train_function\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1360, in step_function\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1349, in run_step\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1126, in train_step\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 589, in __call__\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/base_layer.py\", line 1149, in __call__\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 96, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/functional.py\", line 515, in call\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/functional.py\", line 672, in _run_internal_graph\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 589, in __call__\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/base_layer.py\", line 1149, in __call__\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 96, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/modeling_tf_utils.py\", line 538, in run_call_with_unpacked_inputs\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/distilbert/modeling_tf_distilbert.py\", line 539, in call\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/base_layer.py\", line 1149, in __call__\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 96, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/modeling_tf_utils.py\", line 538, in run_call_with_unpacked_inputs\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/distilbert/modeling_tf_distilbert.py\", line 403, in call\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/base_layer.py\", line 1149, in __call__\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 96, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/distilbert/modeling_tf_distilbert.py\", line 315, in call\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/distilbert/modeling_tf_distilbert.py\", line 319, in call\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/base_layer.py\", line 1149, in __call__\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 96, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/distilbert/modeling_tf_distilbert.py\", line 276, in call\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 65, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/base_layer.py\", line 1149, in __call__\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 96, in error_handler\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/transformers/models/distilbert/modeling_tf_distilbert.py\", line 230, in call\n",
            "\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/activations.py\", line 348, in gelu\n",
            "\n",
            "failed to allocate memory\n",
            "\t [[{{node model/tf_distil_bert_model_1/distilbert/transformer/layer_._0/ffn/Gelu/Erf}}]]\n",
            "Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.\n",
            " [Op:__inference_train_function_60843]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint\n",
        "from tensorflow.keras.optimizers import Adam, SGD\n",
        "model.load_weights('/content/drive/MyDrive/Colab Notebooks/Datasets/model_weights_epoch_05.h5')\n",
        "# Define the sweep configuration\n",
        "sweep_config = {\n",
        "    'method': 'bayes',  # Bayesian optimization\n",
        "    'metric': {\n",
        "        'name': 'val_loss',\n",
        "        'goal': 'minimize'\n",
        "    },\n",
        "    'parameters': {\n",
        "        'learning_rate': {\n",
        "            'min': 1e-5,\n",
        "            'max': 1e-3\n",
        "        },\n",
        "        'batch_size': {\n",
        "            'values': [64, 128, 256]\n",
        "        },\n",
        "        'data_combination': {\n",
        "            'values': ['numerical', 'num_cat', 'num_cat_text']\n",
        "        },\n",
        "        # Add other hyperparameters here\n",
        "    }\n",
        "}\n",
        "\n",
        "def train():\n",
        "    with wandb.init() as run:\n",
        "        config = run.config\n",
        "\n",
        "        # Callbacks\n",
        "        early_stopping = EarlyStopping(monitor='val_loss', patience=3)\n",
        "        reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=2, min_lr=1e-6)\n",
        "        model_checkpoint_callback = ModelCheckpoint(\n",
        "            filepath='model_weights_epoch_{epoch:02d}.h5',\n",
        "            save_weights_only=True,\n",
        "            save_best_only=False,\n",
        "            save_freq='epoch'\n",
        "        )\n",
        "\n",
        "        # Build and compile your model here using config parameters\n",
        "        # For example, setting the optimizer\n",
        "        optimizer = Adam(learning_rate=config.learning_rate)\n",
        "        # model.compile(optimizer=optimizer, ...)\n",
        "\n",
        "        # Select data based on the chosen combination\n",
        "        if config.data_combination == 'numerical':\n",
        "            input_data = {'numerical_data': X_train_numerical_tf}\n",
        "            validation_data = {'numerical_data': X_test_numerical_tf}\n",
        "        elif config.data_combination == 'num_cat':\n",
        "            input_data = {\n",
        "                'numerical_data': X_train_numerical_tf,\n",
        "                'categorical_data': X_train_categorical_tf\n",
        "            }\n",
        "            validation_data = {\n",
        "                'numerical_data': X_test_numerical_tf,\n",
        "                'categorical_data': X_test_categorical_tf\n",
        "            }\n",
        "        elif config.data_combination == 'num_cat_text':\n",
        "            input_data = {\n",
        "                'input_ids': train_input_ids,\n",
        "                'attention_mask': train_attention_masks,\n",
        "                'numerical_data': X_train_numerical_tf,\n",
        "                'categorical_data': X_train_categorical_tf\n",
        "            }\n",
        "            validation_data = {\n",
        "                'input_ids': test_input_ids,\n",
        "                'attention_mask': test_attention_masks,\n",
        "                'numerical_data': X_test_numerical_tf,\n",
        "                'categorical_data': X_test_categorical_tf\n",
        "            }\n",
        "\n",
        "        # Fit the model\n",
        "        history = model.fit(\n",
        "            input_data,\n",
        "            train_labels,\n",
        "            validation_data=(validation_data, test_labels),\n",
        "            epochs=10,  # Or config.epochs if defined in sweep_config\n",
        "            batch_size=int(config.batch_size),\n",
        "            callbacks=[early_stopping, reduce_lr, model_checkpoint_callback]\n",
        "        )\n",
        "\n",
        "# Initialize the sweep\n",
        "sweep_id = wandb.sweep(sweep_config, project=project)\n",
        "\n",
        "# Run the sweep agent\n",
        "wandb.agent(sweep_id, function=train, count=10, project=project)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000,
          "referenced_widgets": [
            "1c4c1ce5995f482f9ab9b62c669fb895",
            "230c94756b064aa49cda6c8b5f7fe878",
            "e78d273ec8014966aa1a9d8773dab48a",
            "53470ac1564847ce835fabfa56cc4b7b",
            "024691a7beb04d789703350930db6a60",
            "ca906c64283e4061b7319aa074c3edc3",
            "286fae4f23a54dae9c76952060efbd21",
            "83ccb822b99349d095a1a4ecc66c36bd",
            "9502d8a483174ac2bd65ea2669426d8d",
            "af8649e3ba1b4198b39464318345efb9",
            "e476dc3f65d54eeb8bab2191da01c720",
            "5c34baf4d85e411ab0defa7d75316856",
            "83bb47bb71f34ed996c5cd38a1ec2a64",
            "7f6c9332495441e7b81b73e50f7c7fba",
            "417f19710a1c4b3f8072279a11f79974",
            "378669fb8a314525860d9ad72f1a7d39"
          ]
        },
        "id": "cHtyltYuSR5w",
        "outputId": "19a4043d-026d-4d6a-bfa4-3ddbe156d8ac"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Create sweep with ID: 7jauby7v\n",
            "Sweep URL: https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[34m\u001b[1mwandb\u001b[0m: Agent Starting Run: i9t5qbf5 with config:\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \tbatch_size: 64\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \tdata_combination: numerical\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \tlearning_rate: 0.0004288002783228434\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mmariano-chicatun\u001b[0m (\u001b[33mmarian-ai\u001b[0m). Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Tracking run with wandb version 0.16.0"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Run data is saved locally in <code>/content/drive/MyDrive/Colab Notebooks/Datasets/wandb/run-20231124_132037-i9t5qbf5</code>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Syncing run <strong><a href='https://wandb.ai/marian-ai/obligatorio_dl/runs/i9t5qbf5' target=\"_blank\">devout-sweep-1</a></strong> to <a href='https://wandb.ai/marian-ai/obligatorio_dl' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>Sweep page: <a href='https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View project at <a href='https://wandb.ai/marian-ai/obligatorio_dl' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View sweep at <a href='https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View run at <a href='https://wandb.ai/marian-ai/obligatorio_dl/runs/i9t5qbf5' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/runs/i9t5qbf5</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Traceback (most recent call last):\n",
            "  File \"<ipython-input-35-30605d4c367c>\", line 73, in train\n",
            "    history = model.fit(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\n",
            "    raise e.with_traceback(filtered_tb) from None\n",
            "  File \"/tmp/__autograph_generated_filen0ckdp6o.py\", line 15, in tf__train_function\n",
            "    retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)\n",
            "ValueError: in user code:\n",
            "\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1377, in train_function  *\n",
            "        return step_function(self, iterator)\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1360, in step_function  **\n",
            "        outputs = model.distribute_strategy.run(run_step, args=(data,))\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1349, in run_step  **\n",
            "        outputs = model.train_step(data)\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1126, in train_step\n",
            "        y_pred = self(x, training=True)\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\n",
            "        raise e.with_traceback(filtered_tb) from None\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/input_spec.py\", line 197, in assert_input_compatibility\n",
            "        raise ValueError(\n",
            "\n",
            "    ValueError: Missing data for input \"input_ids\". You passed a data dictionary with keys ['numerical_data']. Expected the following keys: ['input_ids', 'attention_mask', 'numerical_data', 'categorical_data']\n",
            "\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "VBox(children=(Label(value='0.002 MB of 0.002 MB uploaded\\r'), FloatProgress(value=1.0, max=1.0)))"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "1c4c1ce5995f482f9ab9b62c669fb895"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View run <strong style=\"color:#cdcd00\">devout-sweep-1</strong> at: <a href='https://wandb.ai/marian-ai/obligatorio_dl/runs/i9t5qbf5' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/runs/i9t5qbf5</a><br/>Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Find logs at: <code>./wandb/run-20231124_132037-i9t5qbf5/logs</code>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Run i9t5qbf5 errored: ValueError('in user code:\\n\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1377, in train_function  *\\n        return step_function(self, iterator)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1360, in step_function  **\\n        outputs = model.distribute_strategy.run(run_step, args=(data,))\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1349, in run_step  **\\n        outputs = model.train_step(data)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1126, in train_step\\n        y_pred = self(x, training=True)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\\n        raise e.with_traceback(filtered_tb) from None\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/input_spec.py\", line 197, in assert_input_compatibility\\n        raise ValueError(\\n\\n    ValueError: Missing data for input \"input_ids\". You passed a data dictionary with keys [\\'numerical_data\\']. Expected the following keys: [\\'input_ids\\', \\'attention_mask\\', \\'numerical_data\\', \\'categorical_data\\']\\n')\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[32m\u001b[41mERROR\u001b[0m Run i9t5qbf5 errored: ValueError('in user code:\\n\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1377, in train_function  *\\n        return step_function(self, iterator)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1360, in step_function  **\\n        outputs = model.distribute_strategy.run(run_step, args=(data,))\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1349, in run_step  **\\n        outputs = model.train_step(data)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1126, in train_step\\n        y_pred = self(x, training=True)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\\n        raise e.with_traceback(filtered_tb) from None\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/input_spec.py\", line 197, in assert_input_compatibility\\n        raise ValueError(\\n\\n    ValueError: Missing data for input \"input_ids\". You passed a data dictionary with keys [\\'numerical_data\\']. Expected the following keys: [\\'input_ids\\', \\'attention_mask\\', \\'numerical_data\\', \\'categorical_data\\']\\n')\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: Agent Starting Run: 84s3i52u with config:\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \tbatch_size: 64\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \tdata_combination: numerical\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \tlearning_rate: 0.0002486511985375514\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Tracking run with wandb version 0.16.0"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Run data is saved locally in <code>/content/drive/MyDrive/Colab Notebooks/Datasets/wandb/run-20231124_132049-84s3i52u</code>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Syncing run <strong><a href='https://wandb.ai/marian-ai/obligatorio_dl/runs/84s3i52u' target=\"_blank\">vital-sweep-2</a></strong> to <a href='https://wandb.ai/marian-ai/obligatorio_dl' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>Sweep page: <a href='https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View project at <a href='https://wandb.ai/marian-ai/obligatorio_dl' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View sweep at <a href='https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/sweeps/7jauby7v</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View run at <a href='https://wandb.ai/marian-ai/obligatorio_dl/runs/84s3i52u' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/runs/84s3i52u</a>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Traceback (most recent call last):\n",
            "  File \"<ipython-input-35-30605d4c367c>\", line 73, in train\n",
            "    history = model.fit(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\n",
            "    raise e.with_traceback(filtered_tb) from None\n",
            "  File \"/tmp/__autograph_generated_filen0ckdp6o.py\", line 15, in tf__train_function\n",
            "    retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)\n",
            "ValueError: in user code:\n",
            "\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1377, in train_function  *\n",
            "        return step_function(self, iterator)\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1360, in step_function  **\n",
            "        outputs = model.distribute_strategy.run(run_step, args=(data,))\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1349, in run_step  **\n",
            "        outputs = model.train_step(data)\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1126, in train_step\n",
            "        y_pred = self(x, training=True)\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\n",
            "        raise e.with_traceback(filtered_tb) from None\n",
            "    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/input_spec.py\", line 197, in assert_input_compatibility\n",
            "        raise ValueError(\n",
            "\n",
            "    ValueError: Missing data for input \"input_ids\". You passed a data dictionary with keys ['numerical_data']. Expected the following keys: ['input_ids', 'attention_mask', 'numerical_data', 'categorical_data']\n",
            "\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "VBox(children=(Label(value='0.002 MB of 0.002 MB uploaded\\r'), FloatProgress(value=1.0, max=1.0)))"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9502d8a483174ac2bd65ea2669426d8d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              " View run <strong style=\"color:#cdcd00\">vital-sweep-2</strong> at: <a href='https://wandb.ai/marian-ai/obligatorio_dl/runs/84s3i52u' target=\"_blank\">https://wandb.ai/marian-ai/obligatorio_dl/runs/84s3i52u</a><br/>Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "Find logs at: <code>./wandb/run-20231124_132049-84s3i52u/logs</code>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Run 84s3i52u errored: ValueError('in user code:\\n\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1377, in train_function  *\\n        return step_function(self, iterator)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1360, in step_function  **\\n        outputs = model.distribute_strategy.run(run_step, args=(data,))\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1349, in run_step  **\\n        outputs = model.train_step(data)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1126, in train_step\\n        y_pred = self(x, training=True)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\\n        raise e.with_traceback(filtered_tb) from None\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/input_spec.py\", line 197, in assert_input_compatibility\\n        raise ValueError(\\n\\n    ValueError: Missing data for input \"input_ids\". You passed a data dictionary with keys [\\'numerical_data\\']. Expected the following keys: [\\'input_ids\\', \\'attention_mask\\', \\'numerical_data\\', \\'categorical_data\\']\\n')\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[32m\u001b[41mERROR\u001b[0m Run 84s3i52u errored: ValueError('in user code:\\n\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1377, in train_function  *\\n        return step_function(self, iterator)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1360, in step_function  **\\n        outputs = model.distribute_strategy.run(run_step, args=(data,))\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1349, in run_step  **\\n        outputs = model.train_step(data)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py\", line 1126, in train_step\\n        y_pred = self(x, training=True)\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\\n        raise e.with_traceback(filtered_tb) from None\\n    File \"/usr/local/lib/python3.10/dist-packages/keras/src/engine/input_spec.py\", line 197, in assert_input_compatibility\\n        raise ValueError(\\n\\n    ValueError: Missing data for input \"input_ids\". You passed a data dictionary with keys [\\'numerical_data\\']. Expected the following keys: [\\'input_ids\\', \\'attention_mask\\', \\'numerical_data\\', \\'categorical_data\\']\\n')\n",
            "\u001b[34m\u001b[1mwandb\u001b[0m: Ctrl + C detected. Stopping sweep.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true,
          "base_uri": "https://localhost:8080/",
          "height": 262
        },
        "id": "jz0hJmZABsDN",
        "outputId": "afc7dcdd-8e7a-4e0f-ddae-b8c7e93475cb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "2040/2040 [==============================] - 1869s 916ms/step - loss: 0.5552 - mae: 0.4238 - val_loss: 0.4969 - val_mae: 0.3915 - lr: 0.0010\n",
            "Epoch 2/10\n",
            "2040/2040 [==============================] - 1869s 916ms/step - loss: 0.5252 - mae: 0.4146 - val_loss: 0.4651 - val_mae: 0.3787 - lr: 0.0010\n",
            "Epoch 3/10\n",
            "2040/2040 [==============================] - 1869s 916ms/step - loss: 0.5720 - mae: 0.4164 - val_loss: 0.4479 - val_mae: 0.3681 - lr: 0.0010\n",
            "Epoch 4/10\n",
            "2040/2040 [==============================] - 1869s 916ms/step - loss: 0.4983 - mae: 0.4044 - val_loss: 0.4368 - val_mae: 0.3706 - lr: 0.0010\n",
            "Epoch 5/10\n",
            "2040/2040 [==============================] - 1869s 916ms/step - loss: 0.4902 - mae: 0.4011 - val_loss: 0.4318 - val_mae: 0.3729 - lr: 0.0010\n",
            "Epoch 6/10\n",
            "2040/2040 [==============================] - ETA: 0s - loss: 0.4827 - mae: 0.3981"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "ERROR:root:Internal Python error in the inspect module.\n",
            "Below is the traceback from this internal error.\n",
            "\n",
            "ERROR:root:Internal Python error in the inspect module.\n",
            "Below is the traceback from this internal error.\n",
            "\n",
            "ERROR:root:Internal Python error in the inspect module.\n",
            "Below is the traceback from this internal error.\n",
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3553, in run_code\n",
            "    exec(code_obj, self.user_global_ns, self.user_ns)\n",
            "  File \"<ipython-input-39-64faaa3a29d0>\", line 11, in <cell line: 11>\n",
            "    history = model.fit(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\n",
            "    raise e.with_traceback(filtered_tb) from None\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/h5py/_hl/files.py\", line 567, in __init__\n",
            "    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/h5py/_hl/files.py\", line 237, in make_fid\n",
            "    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)\n",
            "  File \"h5py/_objects.pyx\", line 54, in h5py._objects.with_phil.wrapper\n",
            "  File \"h5py/_objects.pyx\", line 55, in h5py._objects.with_phil.wrapper\n",
            "  File \"h5py/h5f.pyx\", line 126, in h5py.h5f.create\n",
            "OSError: [Errno 107] Unable to create file (unable to open file: name = 'model_weights_epoch_06.h5', errno = 107, error message = 'Transport endpoint is not connected', flags = 13, o_flags = 242)\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2099, in showtraceback\n",
            "    stb = value._render_traceback_()\n",
            "AttributeError: 'OSError' object has no attribute '_render_traceback_'\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1101, in get_records\n",
            "    return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 248, in wrapped\n",
            "    return f(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 281, in _fixed_getinnerframes\n",
            "    records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 1662, in getinnerframes\n",
            "    frameinfo = (tb.tb_frame,) + getframeinfo(tb, context)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 1620, in getframeinfo\n",
            "    filename = getsourcefile(frame) or getfile(frame)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 829, in getsourcefile\n",
            "    module = getmodule(object, filename)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 861, in getmodule\n",
            "    file = getabsfile(object, _filename)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 845, in getabsfile\n",
            "    return os.path.normcase(os.path.abspath(_filename))\n",
            "  File \"/usr/lib/python3.10/posixpath.py\", line 384, in abspath\n",
            "    cwd = os.getcwd()\n",
            "OSError: [Errno 107] Transport endpoint is not connected\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3553, in run_code\n",
            "    exec(code_obj, self.user_global_ns, self.user_ns)\n",
            "  File \"<ipython-input-39-64faaa3a29d0>\", line 11, in <cell line: 11>\n",
            "    history = model.fit(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\n",
            "    raise e.with_traceback(filtered_tb) from None\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/h5py/_hl/files.py\", line 567, in __init__\n",
            "    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/h5py/_hl/files.py\", line 237, in make_fid\n",
            "    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)\n",
            "  File \"h5py/_objects.pyx\", line 54, in h5py._objects.with_phil.wrapper\n",
            "  File \"h5py/_objects.pyx\", line 55, in h5py._objects.with_phil.wrapper\n",
            "  File \"h5py/h5f.pyx\", line 126, in h5py.h5f.create\n",
            "OSError: [Errno 107] Unable to create file (unable to open file: name = 'model_weights_epoch_06.h5', errno = 107, error message = 'Transport endpoint is not connected', flags = 13, o_flags = 242)\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2099, in showtraceback\n",
            "    stb = value._render_traceback_()\n",
            "AttributeError: 'OSError' object has no attribute '_render_traceback_'\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3473, in run_ast_nodes\n",
            "    if (await self.run_code(code, result,  async_=asy)):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3575, in run_code\n",
            "    self.showtraceback(running_compiled_code=True)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2101, in showtraceback\n",
            "    stb = self.InteractiveTB.structured_traceback(etype,\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1367, in structured_traceback\n",
            "    return FormattedTB.structured_traceback(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1267, in structured_traceback\n",
            "    return VerboseTB.structured_traceback(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1124, in structured_traceback\n",
            "    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1082, in format_exception_as_a_whole\n",
            "    last_unique, recursion_repeat = find_recursion(orig_etype, evalue, records)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 382, in find_recursion\n",
            "    return len(records), 0\n",
            "TypeError: object of type 'NoneType' has no len()\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2099, in showtraceback\n",
            "    stb = value._render_traceback_()\n",
            "AttributeError: 'TypeError' object has no attribute '_render_traceback_'\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1101, in get_records\n",
            "    return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 248, in wrapped\n",
            "    return f(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 281, in _fixed_getinnerframes\n",
            "    records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 1662, in getinnerframes\n",
            "    frameinfo = (tb.tb_frame,) + getframeinfo(tb, context)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 1620, in getframeinfo\n",
            "    filename = getsourcefile(frame) or getfile(frame)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 829, in getsourcefile\n",
            "    module = getmodule(object, filename)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 861, in getmodule\n",
            "    file = getabsfile(object, _filename)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 845, in getabsfile\n",
            "    return os.path.normcase(os.path.abspath(_filename))\n",
            "  File \"/usr/lib/python3.10/posixpath.py\", line 384, in abspath\n",
            "    cwd = os.getcwd()\n",
            "OSError: [Errno 107] Transport endpoint is not connected\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3553, in run_code\n",
            "    exec(code_obj, self.user_global_ns, self.user_ns)\n",
            "  File \"<ipython-input-39-64faaa3a29d0>\", line 11, in <cell line: 11>\n",
            "    history = model.fit(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\", line 70, in error_handler\n",
            "    raise e.with_traceback(filtered_tb) from None\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/h5py/_hl/files.py\", line 567, in __init__\n",
            "    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/h5py/_hl/files.py\", line 237, in make_fid\n",
            "    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)\n",
            "  File \"h5py/_objects.pyx\", line 54, in h5py._objects.with_phil.wrapper\n",
            "  File \"h5py/_objects.pyx\", line 55, in h5py._objects.with_phil.wrapper\n",
            "  File \"h5py/h5f.pyx\", line 126, in h5py.h5f.create\n",
            "OSError: [Errno 107] Unable to create file (unable to open file: name = 'model_weights_epoch_06.h5', errno = 107, error message = 'Transport endpoint is not connected', flags = 13, o_flags = 242)\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2099, in showtraceback\n",
            "    stb = value._render_traceback_()\n",
            "AttributeError: 'OSError' object has no attribute '_render_traceback_'\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3473, in run_ast_nodes\n",
            "    if (await self.run_code(code, result,  async_=asy)):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3575, in run_code\n",
            "    self.showtraceback(running_compiled_code=True)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2101, in showtraceback\n",
            "    stb = self.InteractiveTB.structured_traceback(etype,\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1367, in structured_traceback\n",
            "    return FormattedTB.structured_traceback(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1267, in structured_traceback\n",
            "    return VerboseTB.structured_traceback(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1124, in structured_traceback\n",
            "    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1082, in format_exception_as_a_whole\n",
            "    last_unique, recursion_repeat = find_recursion(orig_etype, evalue, records)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 382, in find_recursion\n",
            "    return len(records), 0\n",
            "TypeError: object of type 'NoneType' has no len()\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2099, in showtraceback\n",
            "    stb = value._render_traceback_()\n",
            "AttributeError: 'TypeError' object has no attribute '_render_traceback_'\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3030, in _run_cell\n",
            "    return runner(coro)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/async_helpers.py\", line 78, in _pseudo_sync_runner\n",
            "    coro.send(None)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3257, in run_cell_async\n",
            "    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 3492, in run_ast_nodes\n",
            "    self.showtraceback()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2101, in showtraceback\n",
            "    stb = self.InteractiveTB.structured_traceback(etype,\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1367, in structured_traceback\n",
            "    return FormattedTB.structured_traceback(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1267, in structured_traceback\n",
            "    return VerboseTB.structured_traceback(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1142, in structured_traceback\n",
            "    formatted_exceptions += self.format_exception_as_a_whole(etype, evalue, etb, lines_of_context,\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1082, in format_exception_as_a_whole\n",
            "    last_unique, recursion_repeat = find_recursion(orig_etype, evalue, records)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 382, in find_recursion\n",
            "    return len(records), 0\n",
            "TypeError: object of type 'NoneType' has no len()\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py\", line 2099, in showtraceback\n",
            "    stb = value._render_traceback_()\n",
            "AttributeError: 'TypeError' object has no attribute '_render_traceback_'\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 1101, in get_records\n",
            "    return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 248, in wrapped\n",
            "    return f(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/IPython/core/ultratb.py\", line 281, in _fixed_getinnerframes\n",
            "    records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 1662, in getinnerframes\n",
            "    frameinfo = (tb.tb_frame,) + getframeinfo(tb, context)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 1620, in getframeinfo\n",
            "    filename = getsourcefile(frame) or getfile(frame)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 829, in getsourcefile\n",
            "    module = getmodule(object, filename)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 861, in getmodule\n",
            "    file = getabsfile(object, _filename)\n",
            "  File \"/usr/lib/python3.10/inspect.py\", line 845, in getabsfile\n",
            "    return os.path.normcase(os.path.abspath(_filename))\n",
            "  File \"/usr/lib/python3.10/posixpath.py\", line 384, in abspath\n",
            "    cwd = os.getcwd()\n",
            "OSError: [Errno 107] Transport endpoint is not connected\n"
          ]
        }
      ],
      "source": [
        "batch_size = 128\n",
        "epochs = 10\n",
        "# Callbacks\n",
        "early_stopping = EarlyStopping(monitor='val_loss', patience=3)\n",
        "reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=2, min_lr=0.001)\n",
        "import datetime\n",
        "\n",
        "current_time = datetime.datetime.now()\n",
        "\n",
        "history = model.fit(\n",
        "    {\n",
        "        'input_ids': train_input_ids,\n",
        "        'attention_mask': train_attention_masks,\n",
        "        'numerical_data': X_train_numerical_tf,\n",
        "        'categorical_data': X_train_categorical_tf\n",
        "    },\n",
        "    train_labels,\n",
        "    validation_data=(\n",
        "        {\n",
        "            'input_ids': test_input_ids,\n",
        "            'attention_mask': test_attention_masks,\n",
        "            'numerical_data': X_test_numerical_tf,\n",
        "            'categorical_data': X_test_categorical_tf\n",
        "        },\n",
        "        test_labels\n",
        "    ),\n",
        "    epochs=epochs,\n",
        "    batch_size=batch_size,\n",
        "    callbacks=[early_stopping, reduce_lr,model_checkpoint_callback]\n",
        "\n",
        ")\n",
        "model.save_weights(f'num_cat_and_trained_bert_{epochs}epochs_{current_time}.h5')\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "k4m9muWUKCeO"
      },
      "outputs": [],
      "source": [
        "model.save_weights(f'num_cat_and_trained_bert_{epochs}epochs.h5')\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "vM2spTD0QWFK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wACZkGCzJ3yj"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "# Your code here...\n",
        "\n",
        "# Command to crash the Colab runtime\n",
        "os._exit(0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dFQhfKDSGLht"
      },
      "source": [
        "### Test with kaggle (Num, Categorical and Text)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "yo9zXexLQxdy"
      },
      "outputs": [],
      "source": [
        "# model.load_weights('/content/drive/MyDrive/Colab Notebooks/Datasets/num_cat_and_trained_bert_10epochs.h5')\n",
        "model.load_weights('/content/drive/MyDrive/Colab Notebooks/Datasets/model_weights_epoch_05.h5')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "Dj9GNFKbGQWQ"
      },
      "outputs": [],
      "source": [
        "import cudf\n",
        "import cuml\n",
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from cuml.preprocessing import StandardScaler, OneHotEncoder\n",
        "from sklearn.impute import SimpleImputer\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Load the Kaggle test dataset\n",
        "file_path2 = './obligatorio_DL/private_data_to_predict.csv'\n",
        "kaggle_test = cudf.read_csv(file_path2)\n",
        "\n",
        "# Assuming your important text columns are already defined\n",
        "important_text_cols = ['Name', 'Summary', 'Description', 'Neighborhood Overview', 'Transit', 'Access', 'Interaction', 'House Rules']\n",
        "kaggle_test_text = kaggle_test[important_text_cols]\n",
        "\n",
        "# Apply cleaning and encoding to text data\n",
        "for col in important_text_cols:\n",
        "    # Assuming process_in_batches is your batch processing function\n",
        "    kaggle_test_text = process_in_batches(kaggle_test_text,col) #[col].astype(str)\n",
        "\n",
        "# Combine the cleaned text columns\n",
        "kaggle_test_text['combined_text'] = ''\n",
        "for col in important_text_cols:\n",
        "    kaggle_test_text['combined_text'] = kaggle_test_text['combined_text'] + kaggle_test_text[col] + ' '\n",
        "    kaggle_test_text['combined_text'] = kaggle_test_text['combined_text'].str.strip()\n",
        "# Encode the combined text\n",
        "kaggle_input_ids, kaggle_attention_masks = batch_encode(kaggle_test_text['combined_text'])\n",
        "\n",
        "# X_kaggle_numerical = kaggle_test[numerical_cols]\n",
        "# X_kaggle_categorical = kaggle_test[categorical_cols]\n",
        "\n",
        "# # Apply transformations\n",
        "# X_kaggle_numerical_transformed = scaler.transform(X_kaggle_numerical)  # Assuming 'scaler' is your StandardScaler\n",
        "# X_kaggle_categorical_transformed = encoder.transform(X_kaggle_categorical)  # Assuming 'encoder' is your OneHotEncoder\n",
        "\n",
        "# # Convert to TensorFlow tensors\n",
        "# X_kaggle_numerical_tf = tf.convert_to_tensor(X_kaggle_numerical_transformed)\n",
        "# X_kaggle_categorical_tf = tf.convert_to_tensor(X_kaggle_categorical_transformed)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "j-7E__GkjFGy"
      },
      "outputs": [],
      "source": [
        "# Assuming kaggle_test is your DataFrame loaded with the test data for Kaggle\n",
        "\n",
        "# Preprocess Numerical Columns\n",
        "# Scale the numerical columns using the same scaler as for the training data\n",
        "kaggle_test_numerical = scaler.transform(kaggle_test[numerical_cols])\n",
        "kaggle_test_numerical = kaggle_test_numerical.fillna(kaggle_test_numerical.mean())\n",
        "kaggle_test_numerical_pd = kaggle_test_numerical.to_pandas()\n",
        "kaggle_test_numerical_tf = tf.convert_to_tensor(kaggle_test_numerical_pd.values, dtype=tf.float32)\n",
        "\n",
        "# Categorical\n",
        "kaggle_test_categorical = encoder.transform(kaggle_test[categorical_cols])\n",
        "kaggle_test_categorical_cupy = cp.asarray(kaggle_test_categorical.toarray())\n",
        "kaggle_test_categorical_np = kaggle_test_categorical_cupy.get()\n",
        "kaggle_test_categorical_tf = tf.convert_to_tensor(kaggle_test_categorical_np)\n",
        "\n",
        "# Now kaggle_test_numerical_tf and kaggle_test_categorical_tf are ready to be used in your model prediction\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f5ti1-IJjUG4",
        "outputId": "9bbfff37-19de-4257-a69e-0075ca62c370"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "5023/5023 [==============================] - 1646s 327ms/step\n"
          ]
        }
      ],
      "source": [
        "# Make predictions\n",
        "predictions = model.predict({\n",
        "    'input_ids': kaggle_input_ids,\n",
        "    'attention_mask': kaggle_attention_masks,\n",
        "    'numerical_data': kaggle_test_numerical_tf,\n",
        "    'categorical_data': kaggle_test_categorical_tf\n",
        "})\n",
        "\n",
        "# Rescale predictions\n",
        "y_pred = price_scaler.inverse_transform(predictions)\n",
        "\n",
        "# Prepare output for submission\n",
        "test_ids = kaggle_test['id']\n",
        "output = np.hstack((test_ids.to_numpy().reshape(-1, 1), y_pred))\n",
        "submission_df = pd.DataFrame(output, columns=['id', 'expected'])\n",
        "submission_df['expected'] = submission_df['expected'].fillna(0)\n",
        "submission_df.to_csv(\"output_to_submit.csv\", index=False)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RBhLVZc3SdzV"
      },
      "source": [
        "# Basura"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 280
        },
        "id": "ONTKzVb88iCH",
        "outputId": "72b957ec-1a94-47b8-fa32-cba87821dc87"
      },
      "outputs": [
        {
          "ename": "NameError",
          "evalue": "ignored",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-38-44bb2ae73395>\u001b[0m in \u001b[0;36m<cell line: 24>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     22\u001b[0m \u001b[0;31m# Assuming categorical_data_train and categorical_data_test are lists of categorical columns\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     23\u001b[0m \u001b[0;31m# For each categorical column, convert to a TensorFlow tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 24\u001b[0;31m \u001b[0mcategorical_data_train\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconvert_to_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcategorical_data_train\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mint32\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_categorical_cols\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     25\u001b[0m \u001b[0mcategorical_data_test\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconvert_to_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcategorical_data_test\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mint32\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_categorical_cols\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     26\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'num_categorical_cols' is not defined"
          ]
        }
      ],
      "source": [
        "def create_dataset(input_ids, attention_masks, numerical_data, categorical_data, labels, batch_size=32):\n",
        "    # Ensure categorical data is a list of tensors, one for each categorical column\n",
        "    if not isinstance(categorical_data, list):\n",
        "        categorical_data = [categorical_data]\n",
        "\n",
        "    # Create a dictionary for all inputs\n",
        "    input_dict = {\n",
        "        'input_ids': input_ids,\n",
        "        'attention_mask': attention_masks,\n",
        "        'numerical_data': numerical_data\n",
        "    }\n",
        "\n",
        "    # Add each categorical feature to the input dictionary\n",
        "    for i, cat_data in enumerate(categorical_data):\n",
        "        input_dict[f'cat_input_{i}'] = cat_data\n",
        "\n",
        "    # Create the TensorFlow dataset\n",
        "    dataset = tf.data.Dataset.from_tensor_slices((input_dict, labels))\n",
        "    dataset = dataset.shuffle(len(labels)).batch(batch_size).prefetch(tf.data.AUTOTUNE)\n",
        "\n",
        "    return dataset\n",
        "# Assuming categorical_data_train and categorical_data_test are lists of categorical columns\n",
        "# For each categorical column, convert to a TensorFlow tensor\n",
        "categorical_data_train = [tf.convert_to_tensor(categorical_data_train[i], dtype=tf.int32) for i in range([num_categorical_cols])]\n",
        "categorical_data_test = [tf.convert_to_tensor(categorical_data_test[i], dtype=tf.int32) for i in range(num_categorical_cols)]\n",
        "\n",
        "# Convert numerical data to TensorFlow tensors\n",
        "numerical_data_train = tf.convert_to_tensor(numerical_data_train, dtype=tf.float32)\n",
        "numerical_data_test = tf.convert_to_tensor(numerical_data_test, dtype=tf.float32)\n",
        "\n",
        "# Create the datasets\n",
        "train_dataset = create_dataset(train_input_ids, train_attention_masks, numerical_data_train, categorical_data_train, train_labels)\n",
        "test_dataset = create_dataset(test_input_ids, test_attention_masks, numerical_data_test, categorical_data_test, test_labels)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 512
        },
        "id": "ZNW_2UKa6xn7",
        "outputId": "08e21a31-8627-47c4-bdae-556e87f0a17f"
      },
      "outputs": [
        {
          "ename": "TypeError",
          "evalue": "ignored",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-37-8158594b8372>\u001b[0m in \u001b[0;36m<cell line: 8>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0minput_ids\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mInput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmax_length\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mint32\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'input_ids'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0mattention_mask\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mInput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmax_length\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mint32\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'attention_mask'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 8\u001b[0;31m \u001b[0mnumerical_input\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mInput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnumerical_cols\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'numerical_data'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      9\u001b[0m \u001b[0mcategorical_inputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mInput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34mf'cat_input_{i}'\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcategorical_cols\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py\u001b[0m in \u001b[0;36merror_handler\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m     68\u001b[0m             \u001b[0;31m# To get the full stack trace, call:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     69\u001b[0m             \u001b[0;31m# `tf.debugging.disable_traceback_filtering()`\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 70\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwith_traceback\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfiltered_tb\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     71\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     72\u001b[0m             \u001b[0;32mdel\u001b[0m \u001b[0mfiltered_tb\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/tensor_shape.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, value)\u001b[0m\n\u001b[1;32m    215\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_value\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvalue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__index__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    216\u001b[0m       \u001b[0;32mexcept\u001b[0m \u001b[0mAttributeError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 217\u001b[0;31m         raise TypeError(\n\u001b[0m\u001b[1;32m    218\u001b[0m             \u001b[0;34m\"Dimension value must be integer or None or have \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    219\u001b[0m             \"an __index__ method, got value '{0!r}' with type '{1!r}'\".format(\n",
            "\u001b[0;31mTypeError\u001b[0m: Dimension value must be integer or None or have an __index__ method, got value '['id', 'Host ID', 'Host Response Rate', 'Host Listings Count', 'Host Total Listings Count', 'Latitude', 'Longitude', 'Accommodates', 'Bathrooms', 'Bedrooms', 'Beds', 'Square Feet', 'Security Deposit', 'Cleaning Fee', 'Guests Included', 'Extra People', 'Minimum Nights', 'Maximum Nights', 'Availability 30', 'Availability 60', 'Availability 90', 'Availability 365', 'Number of Reviews', 'Review Scores Rating', 'Review Scores Accuracy', 'Review Scores Cleanliness', 'Review Scores Checkin', 'Review Scores Communication', 'Review Scores Location', 'Review Scores Value', 'Calculated host listings count', 'Reviews per Month', 'Price']' with type '<class 'list'>'"
          ]
        }
      ],
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow.keras.layers import Input, Dense, Embedding, Concatenate, Normalization\n",
        "from tensorflow.keras.models import Model\n",
        "\n",
        "# Define input layers\n",
        "input_ids = Input(shape=(max_length,), dtype=tf.int32, name='input_ids')\n",
        "attention_mask = Input(shape=(max_length,), dtype=tf.int32, name='attention_mask')\n",
        "numerical_input = Input(shape=(numerical_cols,), name='numerical_data')\n",
        "categorical_inputs = [Input(shape=(1,), name=f'cat_input_{i}') for i in range(categorical_cols)]\n",
        "\n",
        "# BERT model for text\n",
        "bert_output = bert_model([input_ids, attention_mask]).last_hidden_state[:, 0, :]\n",
        "\n",
        "# Numerical data processing\n",
        "numerical_layer = Normalization()(numerical_input)\n",
        "numerical_output = Dense(64, activation='relu')(numerical_layer)\n",
        "\n",
        "# Categorical data processing\n",
        "embedding_layers = [Embedding(input_dim=cat_col_vocab_size, output_dim=embedding_dim)(cat_input) for cat_input, embedding_dim in zip(categorical_inputs, embedding_dims)]\n",
        "flattened_embeddings = [tf.keras.layers.Flatten()(embedding) for embedding in embedding_layers]\n",
        "\n",
        "# Concatenate all outputs\n",
        "concatenated = Concatenate()([bert_output, numerical_output] + flattened_embeddings)\n",
        "dense_layer = Dense(128, activation='relu')(concatenated)\n",
        "output = Dense(1)(dense_layer)\n",
        "\n",
        "# Build and compile the model\n",
        "model = Model(inputs=[input_ids, attention_mask, numerical_input] + categorical_inputs, outputs=output)\n",
        "model.compile(optimizer='adam', loss='mean_squared_error', metrics=['mean_absolute_error'])\n",
        "\n",
        "# Train the model\n",
        "model.fit(train_dataset, validation_data=test_dataset, epochs=5)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LLfLjdWpFuNK"
      },
      "outputs": [],
      "source": [
        "from cudf.core.subword_tokenizer import SubwordTokenizer\n",
        "from cudf.utils.hash_vocab_utils import hash_vocab\n",
        "\n",
        "vocab_file = '/content/drive/MyDrive/Colab Notebooks/Datasets/obligatorio_DL/bert-multilingual-cased/multi_cased_L-12_H-768_A-12/vocab.txt'\n",
        "# Assuming you have the BERT multilingual vocabulary file\n",
        "# vocab_file = 'bert-base-multilingual-uncased-vocab.txt'\n",
        "hash_vocab(vocab_file, 'voc_hash.txt')\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5RmcunXmB0lg"
      },
      "outputs": [],
      "source": [
        "\n",
        "# Initialize tokenizer\n",
        "tokenizer = SubwordTokenizer('voc_hash.txt', do_lower_case=True)\n",
        "\n",
        "# Tokenize function\n",
        "def tokenize_text_column(df, col_name, max_length=128):\n",
        "    tokenized_output = tokenizer(\n",
        "        df[col_name],\n",
        "        max_length=max_length,\n",
        "        padding='max_length',\n",
        "        truncation=True,\n",
        "        return_tensors='pt'\n",
        "    )\n",
        "    return tokenized_output['input_ids']\n",
        "\n",
        "# Apply tokenization\n",
        "for col in text_cols:\n",
        "    df[col] = tokenize_text_column(df, col)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_sB4Qr58JiHD"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "machine_shape": "hm",
      "provenance": [],
      "authorship_tag": "ABX9TyNhxE7B7dkYV7mrMZhBQtcY",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "072c2330f61a48f4a2b365c8a0baea0d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c239cbea081b46c7b1833d4cac549c56",
              "IPY_MODEL_8412a735a6064f7ba9873e74bf661874",
              "IPY_MODEL_aa250ad4c3ff4666bc9fdcaa7a3500f6"
            ],
            "layout": "IPY_MODEL_285b1bfd052b4368ae67bda600968d53"
          }
        },
        "c239cbea081b46c7b1833d4cac549c56": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5373e45dce56410e84938e266d146c96",
            "placeholder": "​",
            "style": "IPY_MODEL_eaee7f1e6a754915a8fe86a5c6620473",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "8412a735a6064f7ba9873e74bf661874": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ed28017e9258421383e9f33adc2b1bf6",
            "max": 29,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1746380716bf4494bed8ee9e7880f483",
            "value": 29
          }
        },
        "aa250ad4c3ff4666bc9fdcaa7a3500f6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1bc0673a98d4471ca1579cb29e972cb1",
            "placeholder": "​",
            "style": "IPY_MODEL_1fef3442448f4c759924d676c76c7faf",
            "value": " 29.0/29.0 [00:00&lt;00:00, 2.42kB/s]"
          }
        },
        "285b1bfd052b4368ae67bda600968d53": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5373e45dce56410e84938e266d146c96": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eaee7f1e6a754915a8fe86a5c6620473": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ed28017e9258421383e9f33adc2b1bf6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1746380716bf4494bed8ee9e7880f483": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1bc0673a98d4471ca1579cb29e972cb1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1fef3442448f4c759924d676c76c7faf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4f3145a9ecb44f20b18ec319e9f1bfc0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ed96cee05a5b48ecb3bdd6e43d4e578e",
              "IPY_MODEL_4552ffd333c242cfbdf187d80872040f",
              "IPY_MODEL_4003bfc4c6a7432a832cd9af414c098c"
            ],
            "layout": "IPY_MODEL_f3327c46b67f434aa6b21881504ac4a2"
          }
        },
        "ed96cee05a5b48ecb3bdd6e43d4e578e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a2a09e53bed64212868e1a32f7345872",
            "placeholder": "​",
            "style": "IPY_MODEL_3b391b2916ad4b1291efc93f1944d4ee",
            "value": "vocab.txt: 100%"
          }
        },
        "4552ffd333c242cfbdf187d80872040f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_60e312d335bc457394f0e4919d97f241",
            "max": 995526,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_400863841dfd4be9a986d00236b05879",
            "value": 995526
          }
        },
        "4003bfc4c6a7432a832cd9af414c098c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_43c290549cf5440b9ef7ec4b330cd9b2",
            "placeholder": "​",
            "style": "IPY_MODEL_cbd3c9725a6342fbaca0c26a9250c9a0",
            "value": " 996k/996k [00:00&lt;00:00, 9.17MB/s]"
          }
        },
        "f3327c46b67f434aa6b21881504ac4a2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a2a09e53bed64212868e1a32f7345872": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3b391b2916ad4b1291efc93f1944d4ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "60e312d335bc457394f0e4919d97f241": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "400863841dfd4be9a986d00236b05879": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "43c290549cf5440b9ef7ec4b330cd9b2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cbd3c9725a6342fbaca0c26a9250c9a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e1ee2475d363447782b1fd16837ffedb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c2a0dd9a889b45d788bc5fd9bc0ac1a1",
              "IPY_MODEL_50ba5cc20eb542549ee69ec99a2c6af8",
              "IPY_MODEL_2d5ef0ee53c34a6e90a4a1a679f0caff"
            ],
            "layout": "IPY_MODEL_7c2bb187bd684cc29b7aa05f157d15d3"
          }
        },
        "c2a0dd9a889b45d788bc5fd9bc0ac1a1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a39be9b4bb6c4383a9967dab6de30f29",
            "placeholder": "​",
            "style": "IPY_MODEL_2e0cb0c287234a92a8260fb95d056fb5",
            "value": "tokenizer.json: 100%"
          }
        },
        "50ba5cc20eb542549ee69ec99a2c6af8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9291b9a447284dd4813229c09ad7e13e",
            "max": 1961828,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_71030b081fb84a11b59a5a780352a9a7",
            "value": 1961828
          }
        },
        "2d5ef0ee53c34a6e90a4a1a679f0caff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f0cddbc7d0304cae81661f17f71e3c0f",
            "placeholder": "​",
            "style": "IPY_MODEL_ba53b5c33a7249afbabdafc4e790e4e8",
            "value": " 1.96M/1.96M [00:00&lt;00:00, 22.8MB/s]"
          }
        },
        "7c2bb187bd684cc29b7aa05f157d15d3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a39be9b4bb6c4383a9967dab6de30f29": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2e0cb0c287234a92a8260fb95d056fb5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9291b9a447284dd4813229c09ad7e13e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "71030b081fb84a11b59a5a780352a9a7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f0cddbc7d0304cae81661f17f71e3c0f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ba53b5c33a7249afbabdafc4e790e4e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "493ba3002f004f949e98cdb91a48cb0d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1db817ae82aa43fcb70206e00653cd81",
              "IPY_MODEL_0c9d12c7c9e24a6698753e2338952043",
              "IPY_MODEL_fa250a63821b4c719d7ba2bd022e00ad"
            ],
            "layout": "IPY_MODEL_3a74879b4a1340f2a378d611b7fdd6ed"
          }
        },
        "1db817ae82aa43fcb70206e00653cd81": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3eb9105e683f47ecbafa3dc730d5b774",
            "placeholder": "​",
            "style": "IPY_MODEL_bec3d34a45e940589f9e03bb164ce262",
            "value": "config.json: 100%"
          }
        },
        "0c9d12c7c9e24a6698753e2338952043": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_44349162382e43d0bdab378f650b8977",
            "max": 466,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_98edae3fc637429cbd380d88ae254ec1",
            "value": 466
          }
        },
        "fa250a63821b4c719d7ba2bd022e00ad": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4cc7dbf2eb134f309190923882b62d76",
            "placeholder": "​",
            "style": "IPY_MODEL_264ba538038c4b0aa0ac9831964e7675",
            "value": " 466/466 [00:00&lt;00:00, 40.5kB/s]"
          }
        },
        "3a74879b4a1340f2a378d611b7fdd6ed": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3eb9105e683f47ecbafa3dc730d5b774": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bec3d34a45e940589f9e03bb164ce262": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "44349162382e43d0bdab378f650b8977": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "98edae3fc637429cbd380d88ae254ec1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "4cc7dbf2eb134f309190923882b62d76": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "264ba538038c4b0aa0ac9831964e7675": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4125aa36719741a1a2b4def934193962": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_41d9e3fcd4b5461a9290cba46d9e0f76",
              "IPY_MODEL_b510e3aaf6a442f681f03e13af862c0d",
              "IPY_MODEL_2fcbdfe8e11549dfadcfb21d0cc40c30"
            ],
            "layout": "IPY_MODEL_d4d8c7abf26c4a6fb13e65d7e0789ee9"
          }
        },
        "41d9e3fcd4b5461a9290cba46d9e0f76": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6f279bbad6e847ad92919a4537432136",
            "placeholder": "​",
            "style": "IPY_MODEL_6040b7a42fe3481ebcf1e972d32c56e7",
            "value": "model.safetensors: 100%"
          }
        },
        "b510e3aaf6a442f681f03e13af862c0d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c334d67683f740b4a418822f4d09afdd",
            "max": 541795680,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4ebe50ab4ae44fd582d827b08b34095b",
            "value": 541795680
          }
        },
        "2fcbdfe8e11549dfadcfb21d0cc40c30": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ac0042cd4e2148c7aabd5c5bcfc3abd8",
            "placeholder": "​",
            "style": "IPY_MODEL_54cf3ad96d5a438fb2abf0d867302d5e",
            "value": " 542M/542M [00:03&lt;00:00, 143MB/s]"
          }
        },
        "d4d8c7abf26c4a6fb13e65d7e0789ee9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6f279bbad6e847ad92919a4537432136": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6040b7a42fe3481ebcf1e972d32c56e7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c334d67683f740b4a418822f4d09afdd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4ebe50ab4ae44fd582d827b08b34095b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ac0042cd4e2148c7aabd5c5bcfc3abd8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "54cf3ad96d5a438fb2abf0d867302d5e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1c4c1ce5995f482f9ab9b62c669fb895": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "VBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "VBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "VBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_230c94756b064aa49cda6c8b5f7fe878",
              "IPY_MODEL_e78d273ec8014966aa1a9d8773dab48a"
            ],
            "layout": "IPY_MODEL_53470ac1564847ce835fabfa56cc4b7b"
          }
        },
        "230c94756b064aa49cda6c8b5f7fe878": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "LabelModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "LabelModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "LabelView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_024691a7beb04d789703350930db6a60",
            "placeholder": "​",
            "style": "IPY_MODEL_ca906c64283e4061b7319aa074c3edc3",
            "value": "0.013 MB of 0.013 MB uploaded\r"
          }
        },
        "e78d273ec8014966aa1a9d8773dab48a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_286fae4f23a54dae9c76952060efbd21",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_83ccb822b99349d095a1a4ecc66c36bd",
            "value": 1
          }
        },
        "53470ac1564847ce835fabfa56cc4b7b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "024691a7beb04d789703350930db6a60": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ca906c64283e4061b7319aa074c3edc3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "286fae4f23a54dae9c76952060efbd21": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "83ccb822b99349d095a1a4ecc66c36bd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9502d8a483174ac2bd65ea2669426d8d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "VBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "VBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "VBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_af8649e3ba1b4198b39464318345efb9",
              "IPY_MODEL_e476dc3f65d54eeb8bab2191da01c720"
            ],
            "layout": "IPY_MODEL_5c34baf4d85e411ab0defa7d75316856"
          }
        },
        "af8649e3ba1b4198b39464318345efb9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "LabelModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "LabelModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "LabelView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_83bb47bb71f34ed996c5cd38a1ec2a64",
            "placeholder": "​",
            "style": "IPY_MODEL_7f6c9332495441e7b81b73e50f7c7fba",
            "value": "0.013 MB of 0.013 MB uploaded\r"
          }
        },
        "e476dc3f65d54eeb8bab2191da01c720": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_417f19710a1c4b3f8072279a11f79974",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_378669fb8a314525860d9ad72f1a7d39",
            "value": 1
          }
        },
        "5c34baf4d85e411ab0defa7d75316856": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "83bb47bb71f34ed996c5cd38a1ec2a64": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7f6c9332495441e7b81b73e50f7c7fba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "417f19710a1c4b3f8072279a11f79974": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "378669fb8a314525860d9ad72f1a7d39": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}