simple-text-to-query.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X40HKSvLmwSw"
      },
      "source": [
        "The goal of this notebook is to give a simple demo on how one could turn their natural language instructions into SQL query and then query a local database. This tutorial can be easily extended to databases that utilize client-server architecture."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "axYgX1nBnewe"
      },
      "source": [
        "This tutorial uses ChatGPT API to convert natural language to queries. For users not comfortable sending their personal queries to OpenAI, we have also demonstrated the same use case by using local Gemma model weights."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YWtA5nmioFRS"
      },
      "source": [
        "Additionally, we have added a bonus section whereby the LLM would be able to understand the existing schema of database and would avoid hallucinating any information regarding any tables."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I8HZbyz9o7rx"
      },
      "source": [
        "## This notebook is divided into following sections - <br>\n",
        "\n",
        "1. **Database Creation** - Create a SQLite3 DB and insert some data in it. Test some basic queries.\n",
        "\n",
        "2. **Setup ChatGPT API connection** - Connect to OpenAI's GPT-4 model to translate natural language into SQL queries.\n",
        "\n",
        "3. **Write prompts and query DB** - Write prompts and enhance them to get good results from LLM. Execute SQL queries on local DB.\n",
        "\n",
        "4. **Prevent hallucination** - Describe the DB schema in prompt so that the LLM does not hallucinates.\n",
        "\n",
        "5. **Download LLM locally** - Download Google's Gemma model weights to local machine and use the LLM."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pHM4hunbttxs"
      },
      "source": [
        "# Database Creation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RunO7fJOtseu"
      },
      "source": [
        "SQLite3 is the library we are going to use to create and query a DB.It is a built-in library so you may not need to install it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a2bMw2OephAU"
      },
      "source": [
        "Let us first setup a simple database and insert some data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "JlTS3RM1kj_v"
      },
      "outputs": [],
      "source": [
        "import sqlite3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "I2riX1p1kyvw"
      },
      "outputs": [],
      "source": [
        "# Connect to a SQLite database. This will create a database with the given name if it does not exists.\n",
        "conn = sqlite3.connect('student.db')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TOf9zTnKpZ3d",
        "outputId": "ed562af8-fc39-4ca4-da2e-979ec517c0b9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[0m\u001b[01;34msample_data\u001b[0m/  student.db\n"
          ]
        }
      ],
      "source": [
        "ls"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "0tDuYBBmlgYP"
      },
      "outputs": [],
      "source": [
        "# Create a cursor object\n",
        "cursor = conn.cursor()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bNaBlCpunA8j",
        "outputId": "013104d6-dcd8-4ce1-c80b-da34ba70743d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<sqlite3.Cursor at 0x79b7fb1b6e40>"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create a table\n",
        "cursor.execute('''\n",
        "CREATE TABLE IF NOT EXISTS student (\n",
        "    id INTEGER PRIMARY KEY,\n",
        "    name TEXT NOT NULL,\n",
        "    age INTEGER NOT NULL,\n",
        "    gender TEXT NOT NULL,\n",
        "    grade TEXT NOT NULL\n",
        ")\n",
        "''')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "0nv8swj_nu2N"
      },
      "outputs": [],
      "source": [
        "# Insert some data rows in the table\n",
        "cursor.execute(\"INSERT INTO student (id, name, age, gender, grade) VALUES (101,'Alice', 17, 'female', 'senior')\")\n",
        "cursor.execute(\"INSERT INTO student (id, name, age, gender, grade) VALUES (102,'Bob', 14, 'male','freshman')\")\n",
        "cursor.execute(\"INSERT INTO student (id, name, age, gender, grade) VALUES (103,'Charlie', 15, 'male', 'sophomore')\")\n",
        "cursor.execute(\"INSERT INTO student (id, name, age, gender, grade) VALUES (104, 'Lily', 7, 'female', 'elementary')\")\n",
        "\n",
        "#Commit the changes\n",
        "conn.commit()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "irql9-vZot3J",
        "outputId": "c7d0b692-7f5d-4480-cee8-a550fd41afd0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "('Alice', 17)\n",
            "('Charlie', 15)\n"
          ]
        }
      ],
      "source": [
        "# Query the database\n",
        "cursor.execute(\"SELECT name, age FROM student WHERE age > 14\")\n",
        "\n",
        "# Fetch all the results\n",
        "results = cursor.fetchall()\n",
        "\n",
        "# Display the results\n",
        "for row in results:\n",
        "  print(row)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "23VcsZ2opocE"
      },
      "outputs": [],
      "source": [
        "# Close the cursor and connection\n",
        "cursor.close()\n",
        "conn.close()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S1_hYuD8niXI"
      },
      "source": [
        "Lets write a function which can be used to get query results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "sJXHeC8innsF"
      },
      "outputs": [],
      "source": [
        "def execute_sql_query(sql_query):\n",
        "    # Connect to SQLite database\n",
        "    conn = sqlite3.connect('student.db')\n",
        "    cursor = conn.cursor()\n",
        "\n",
        "    try:\n",
        "        # Execute the SQL query\n",
        "        cursor.execute(sql_query)\n",
        "        columns = [column[0] for column in cursor.description]\n",
        "        results = cursor.fetchall()\n",
        "        #print(\"Query executed successfully. Here are the results:\")\n",
        "        #for row in results:\n",
        "        #    print(row)\n",
        "        rows = [dict(zip(columns,row)) for row in results]\n",
        "        return rows\n",
        "    except sqlite3.Error as e:\n",
        "        print(f\"An error occurred: {e}\")\n",
        "    finally:\n",
        "        # Close the connection\n",
        "        if cursor:\n",
        "          cursor.close()\n",
        "        if conn:\n",
        "          conn.close()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "syaROo_cwL_G",
        "outputId": "4636b6b8-4b6f-44bc-dcdd-69745b6f28cf"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'name': 'Alice', 'age': 17}, {'name': 'Charlie', 'age': 15}]"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "execute_sql_query(\"SELECT name, age FROM student WHERE age > 14\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YEgLGIPEwYfp"
      },
      "source": [
        "Great! So far we have our database set up and an easy way to query it. Let's connect to ChatGPT."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q68zt-TywnlJ"
      },
      "source": [
        "# Setup ChatGPT API connection"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xAWDU1iMpwa-"
      },
      "source": [
        "Thus far, we have successfully created a simple DB with four rows in it. Now we will try to move our focus towards generating a SQL query using LLM model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qxftwTxhxNI1"
      },
      "source": [
        "We will connect to OpenAI's GPT-4 model and ask a simple question."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b_ZNfSxMpviw"
      },
      "outputs": [],
      "source": [
        "!pip install openai"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FmUzdIaK-zW0",
        "outputId": "6cf4c340-0d5e-4652-d89c-954687575f32"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "OpenAI API Key: ··········\n"
          ]
        }
      ],
      "source": [
        "\n",
        "from openai import OpenAI\n",
        "from getpass import getpass\n",
        "\n",
        "#Input the API key\n",
        "OPENAI_API_KEY = getpass(\"OpenAI API Key: \")\n",
        "\n",
        "#Define the model\n",
        "model = \"gpt-4\" # You can also try \"gpt-3.5-turbo\"\n",
        "\n",
        "#Connect to client\n",
        "client = OpenAI(api_key=OPENAI_API_KEY)\n",
        "\n",
        "#Generate response for your query\n",
        "response = client.chat.completions.create(\n",
        "  model=model,\n",
        "  messages=[\n",
        "    {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
        "    {\"role\": \"user\", \"content\": \"What is a LLM?\"}\n",
        "  ],\n",
        "  max_tokens=500,\n",
        "  temperature=0.7 #0 means deterministic. 1 is the default value.\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        },
        "id": "KahM7l1RApnA",
        "outputId": "c4e293ba-6970-47a7-f787-49387380f886"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"An LLM, or Master of Laws, is an advanced degree in law that is often pursued by students who want to specialize in a specific area of law. It usually requires a bachelor's degree in law (LLB) as a prerequisite and is typically a one-year full-time study program. Some of the specializations could include international law, tax law, environmental law, human rights law, and more. It's particularly popular among international students who want to familiarize themselves with the legal system of a particular country.\""
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response.choices[0].message.content"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e111oH3-HMuU"
      },
      "source": [
        "# Write prompts and query DB"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YhDf7_AbyHMy"
      },
      "source": [
        "We will write a function that converts natural text to query."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YF5Z2CMnSr5A",
        "outputId": "3dffa716-4d0b-44a7-dc52-a92c599bc1ec"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "OpenAI API Key: ··········\n"
          ]
        }
      ],
      "source": [
        "from openai import OpenAI\n",
        "from getpass import getpass\n",
        "\n",
        "OPENAI_API_KEY = getpass(\"OpenAI API Key: \")\n",
        "\n",
        "client = OpenAI(api_key=OPENAI_API_KEY)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "AwSs5hwpHdEq"
      },
      "outputs": [],
      "source": [
        "def convert_text_to_query(input_text,user_prompt,system_prompt=\"You are a helpful assistant.\",model=\"gpt-4\"):\n",
        "\n",
        "  response = client.chat.completions.create(\n",
        "  model=model,\n",
        "  messages=[\n",
        "    {\"role\": \"system\", \"content\": system_prompt },\n",
        "    {\"role\": \"user\", \"content\": user_prompt.format(natural_language_query=input_text)}\n",
        "  ],\n",
        "  max_tokens=500,\n",
        "  temperature=0.7 #0 means deterministic. 1 is the default value.\n",
        "  )\n",
        "\n",
        "  return response.choices[0].message.content\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "Y80wLDL3Hq6L"
      },
      "outputs": [],
      "source": [
        "system_prompt = \"\"\"\n",
        "You are a helpful assistant that converts natural language into SQL queries.\n",
        "\"\"\"\n",
        "\n",
        "user_prompt = \"\"\"\n",
        "Convert the following text into an SQL query:{natural_language_query}.\n",
        "\"\"\"\n",
        "\n",
        "input_text = \"How many students are of above 7 years of age?\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "cwlElsXsZv9P"
      },
      "outputs": [],
      "source": [
        "query = convert_text_to_query(input_text,user_prompt=user_prompt,system_prompt=system_prompt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "AbMwabZXzBn5",
        "outputId": "b0e39e5c-0023-4288-9808-bbe058ff835c"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'SELECT COUNT(*) \\nFROM students \\nWHERE age > 7;'"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "query"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CRpwpArHy59j",
        "outputId": "2a5eee4b-617b-40d7-c344-e0a20fbcd0d5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "An error occurred: no such table: students\n"
          ]
        }
      ],
      "source": [
        "execute_sql_query(query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3MFfD8zLzjU6"
      },
      "source": [
        "You see! The LLM has no idea about the tables we have in our database. Let's enhance or query to accomodate this and also add a few examples."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "LiLuxz4PH0Yc"
      },
      "outputs": [],
      "source": [
        "system_prompt = \"\"\"\n",
        "You are a helpful assistant that converts natural language into SQL queries.\n",
        "\"\"\"\n",
        "\n",
        "user_prompt =\"\"\"\n",
        "You are a highly skilled SQL translator. Your task is to convert natural language descriptions of data queries into correct and optimized SQL statements.\n",
        "\n",
        "Here are a few examples:\n",
        "\n",
        "1. **Input**: \"Select the names and salaries of all employees in the 'Finance' department.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT name, salary\n",
        "   FROM employees\n",
        "   WHERE department = 'Finance';\n",
        "\n",
        "2. **Input**: \"Find the total number of orders placed by customers in 2023.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT COUNT(*)\n",
        "   FROM orders\n",
        "   WHERE YEAR(order_date) = 2023;\n",
        "\n",
        "3. **Input**: \"Retrieve the email addresses of customers who have not placed any orders.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT email\n",
        "   FROM customers\n",
        "   WHERE customer_id NOT IN (SELECT customer_id FROM orders);\n",
        "\n",
        "4. **Input**: \"List the product names and their average rating for products with more than 100 reviews.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT product_name, AVG(rating) AS average_rating\n",
        "   FROM reviews\n",
        "   GROUP BY product_name\n",
        "   HAVING COUNT(*) > 100;\n",
        "\n",
        "5. **Input**: \"Get the total sales and the number of orders for each product sold in January 2024.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT product_id, SUM(sales_amount) AS total_sales, COUNT(order_id) AS order_count\n",
        "   FROM sales\n",
        "   WHERE order_date BETWEEN '2024-01-01' AND '2024-01-31'\n",
        "   GROUP BY product_id;\n",
        "\n",
        "Now, translate the following natural language query into an SQL query:\n",
        "**Input**: {natural_language_query}\n",
        "**Output**:\n",
        "\n",
        "\"\"\"\n",
        "input_text = \"\"\"\n",
        "How many students are of above 7 years of age? The details can be found in table 'student'\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "nLnDtBsTapBc"
      },
      "outputs": [],
      "source": [
        "query = convert_text_to_query(input_text,user_prompt=user_prompt,system_prompt=system_prompt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZqnkZ-Bnz-nK",
        "outputId": "10abb895-798c-47c7-f4f4-3292bd19caca"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'COUNT(*)': 3}]"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "execute_sql_query(query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y3HDn2FD0coi"
      },
      "source": [
        "# Prevent hallucination"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tbUHtsLh0d7v"
      },
      "source": [
        "Did you notice how in the previous section the LLM hallucinated the table name 'students' ? It will be a bit hectic for us if we have to mention all the table and column names everytime we want to ask a question. A good idea would be to automate the insertion of the metadata information on tables and the corresponding columns in the prompt itself."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CLZSBTye01bO"
      },
      "source": [
        "For this, we would create two tables -\n",
        "1. **table_metadata** - Stores the table description.\n",
        "2. **column_metadata** - Stores the column description for each table."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d9Rjp-Hr1Jl6"
      },
      "source": [
        "Let's create them and insert information in them."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "Ul3hYfgjgMVe"
      },
      "outputs": [],
      "source": [
        "conn = sqlite3.connect('student.db')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "NpBFRo74gTrD"
      },
      "outputs": [],
      "source": [
        "# Create a cursor object\n",
        "cursor = conn.cursor()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1vv_I4Xlmfu8"
      },
      "source": [
        "First create a table that stores the meta data about the table."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6E6ILkg0gnvi",
        "outputId": "acf3cc96-f26e-4a44-a09a-92edeb9d1a3e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<sqlite3.Cursor at 0x79b7d62a4140>"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create a table\n",
        "cursor.execute('''\n",
        "CREATE TABLE IF NOT EXISTS table_metadata (\n",
        "    table_name TEXT PRIMARY KEY,\n",
        "    description TEXT\n",
        ")\n",
        "''')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "g25oklvCiD8F"
      },
      "outputs": [],
      "source": [
        "cursor.execute(\"INSERT INTO table_metadata (table_name, description) VALUES ('student','Details of students in school.')\")\n",
        "conn.commit()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w5RTuokx1XkD",
        "outputId": "52b7cf0a-ffbb-4096-a1cb-aa48003527a7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'table_name': 'student', 'description': 'Details of students in school.'}]"
            ]
          },
          "execution_count": 27,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "execute_sql_query(\"SELECT * FROM table_metadata\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3T0GKVf2nLSM"
      },
      "source": [
        "Create a table that stores metadata about the columns in each table."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UgGPvDzUiOke",
        "outputId": "265e7b2f-fc73-4b3a-c0b4-9b8dbc908619"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<sqlite3.Cursor at 0x79b7d62a4140>"
            ]
          },
          "execution_count": 28,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "cursor.execute('''\n",
        "CREATE TABLE IF NOT EXISTS column_metadata (\n",
        "  table_name TEXT,\n",
        "  column_name TEXT,\n",
        "  data_type TEXT,\n",
        "  description TEXT,\n",
        "  PRIMARY KEY (table_name, column_name),\n",
        "  FOREIGN KEY (table_name) references table_metadata (table_name)\n",
        ")\n",
        "''')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "-lYepdaZjJV3"
      },
      "outputs": [],
      "source": [
        "cursor.execute(\"INSERT INTO column_metadata (table_name, column_name, data_type,description) VALUES ('student','id','INTEGER','Identification number of the student.')\")\n",
        "cursor.execute(\"INSERT INTO column_metadata (table_name, column_name, data_type,description) VALUES ('student','name','TEXT','Name of the student.')\")\n",
        "cursor.execute(\"INSERT INTO column_metadata (table_name, column_name, data_type,description) VALUES ('student','age','INTEGER','Age of the student.')\")\n",
        "cursor.execute(\"INSERT INTO column_metadata (table_name, column_name, data_type,description) VALUES ('student','gender','TEXT','Gender of the student.')\")\n",
        "cursor.execute(\"INSERT INTO column_metadata (table_name, column_name, data_type,description) VALUES ('student','grade','TEXT','Grade in which the student studies.')\")\n",
        "conn.commit()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jQnh0RNSnyWt",
        "outputId": "12939ba5-611e-46ab-912e-edd875d07798"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'table_name': 'student',\n",
              "  'column_name': 'id',\n",
              "  'data_type': 'INTEGER',\n",
              "  'description': 'Identification number of the student.'},\n",
              " {'table_name': 'student',\n",
              "  'column_name': 'name',\n",
              "  'data_type': 'TEXT',\n",
              "  'description': 'Name of the student.'},\n",
              " {'table_name': 'student',\n",
              "  'column_name': 'age',\n",
              "  'data_type': 'INTEGER',\n",
              "  'description': 'Age of the student.'},\n",
              " {'table_name': 'student',\n",
              "  'column_name': 'gender',\n",
              "  'data_type': 'TEXT',\n",
              "  'description': 'Gender of the student.'},\n",
              " {'table_name': 'student',\n",
              "  'column_name': 'grade',\n",
              "  'data_type': 'TEXT',\n",
              "  'description': 'Grade in which the student studies.'}]"
            ]
          },
          "execution_count": 30,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "execute_sql_query(\"SELECT * FROM column_metadata\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "vN_BeaGz1bzh"
      },
      "outputs": [],
      "source": [
        "cursor.close()\n",
        "conn.close()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8U6gDY9-1kU-"
      },
      "source": [
        "Great! So how do we insert all this schema information in the prompt? Let's write a function that does that."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "id": "ujdOAOfEoEVU"
      },
      "outputs": [],
      "source": [
        "def get_table_info(table_name):\n",
        "\n",
        "    query_table = \"SELECT * FROM table_metadata where table_name = '{table_name}';\".format(table_name=table_name)\n",
        "    table_info = execute_sql_query(query_table)[0]\n",
        "    table_name = table_info[\"table_name\"]\n",
        "    description = table_info[\"description\"]\n",
        "\n",
        "    result = \"\"\"\\n\n",
        "    The table name is '{table_name}'\n",
        "    and the description is as follows:\n",
        "    '{description}'\n",
        "    \\n\n",
        "    The column information for table {table_name} is as follows :\\n\n",
        "    \"\"\".format(table_name=table_name,description=description)\n",
        "\n",
        "\n",
        "    result += \"{:<15} {:<15} {:<15}\\n\".format(\"Column Name\", \"Data Type\", \"Description\")\n",
        "\n",
        "    query_columns = \"SELECT * FROM column_metadata where table_name = '{table_name}';\".format(table_name=table_name)\n",
        "    columns_info = execute_sql_query(query_columns)\n",
        "\n",
        "    for column in columns_info:\n",
        "      column_name = column[\"column_name\"]\n",
        "      data_type = column[\"data_type\"]\n",
        "      description = column[\"description\"]\n",
        "      result += f\"{column_name:<15} {data_type:<15} {description:<15}\\n\"\n",
        "\n",
        "\n",
        "    return result\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NvnaLjTZ1y6b"
      },
      "source": [
        "The above function will get you the table information given a table's name. But we can have multiple tables. So let's get all the table information using another function"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "id": "oL8AwNIl0fra"
      },
      "outputs": [],
      "source": [
        "def get_schema_information():\n",
        "\n",
        "  result = \"\"\"\n",
        "  The name of database is student.db and following is the information on its tables.\\n\n",
        "  \"\"\"\n",
        "\n",
        "  query_schema = \"SELECT * FROM table_metadata;\"\n",
        "\n",
        "  tables = execute_sql_query(query_schema)\n",
        "\n",
        "  for table in tables:\n",
        "    result += get_table_info(table[\"table_name\"])\n",
        "\n",
        "  return result\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ycyIZCE61gcs",
        "outputId": "e293ff3f-0ca7-4bea-caef-39597477fa1a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "  The name of database is student.db and following is the information on its tables.\n",
            "\n",
            "  \n",
            "\n",
            "    The table name is 'student'\n",
            "    and the description is as follows:\n",
            "    'Details of students in school.'\n",
            "    \n",
            "\n",
            "    The column information for table student is as follows :\n",
            "\n",
            "    Column Name     Data Type       Description    \n",
            "age             INTEGER         Age of the student.\n",
            "gender          TEXT            Gender of the student.\n",
            "grade           TEXT            Grade in which the student studies.\n",
            "id              INTEGER         Identification number of the student.\n",
            "name            TEXT            Name of the student.\n",
            "\n"
          ]
        }
      ],
      "source": [
        "result = get_schema_information()\n",
        "print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TOrqRQQk2EBm"
      },
      "source": [
        "Now, we will enhance our prompt with this information. For that, lets write another function that converts text to query but using schema information."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "cHmPfyms204k"
      },
      "outputs": [],
      "source": [
        "def convert_text_to_query_with_schema(input_text,schema_info,user_prompt,system_prompt=\"You are a helpful assistant.\",model=\"gpt-4\"):\n",
        "\n",
        "  response = client.chat.completions.create(\n",
        "  model=model,\n",
        "  messages=[\n",
        "    {\"role\": \"system\", \"content\": system_prompt },\n",
        "    {\"role\": \"user\", \"content\": user_prompt.format(natural_language_query=input_text,schema_info=schema_info)}\n",
        "  ],\n",
        "  max_tokens=500,\n",
        "  temperature=0.7 #0 means deterministic. 1 is the default value.\n",
        "  )\n",
        "\n",
        "  return response.choices[0].message.content"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jFXOCLza2Qjs"
      },
      "source": [
        "Let's adjust our prompt a bit."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "id": "uGjgfxFw3C90"
      },
      "outputs": [],
      "source": [
        "system_prompt = \"\"\"\n",
        "You are a helpful assistant that converts natural language into SQL queries.\n",
        "\"\"\"\n",
        "\n",
        "user_prompt =\"\"\"\n",
        "You are a highly skilled SQL translator. Your task is to convert natural language descriptions of data queries into correct and optimized SQL statements.\n",
        "\n",
        "Here are a few examples:\n",
        "\n",
        "1. **Input**: \"Select the names and salaries of all employees in the 'Finance' department.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT name, salary\n",
        "   FROM employees\n",
        "   WHERE department = 'Finance';\n",
        "\n",
        "2. **Input**: \"Find the total number of orders placed by customers in 2023.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT COUNT(*)\n",
        "   FROM orders\n",
        "   WHERE YEAR(order_date) = 2023;\n",
        "\n",
        "3. **Input**: \"Retrieve the email addresses of customers who have not placed any orders.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT email\n",
        "   FROM customers\n",
        "   WHERE customer_id NOT IN (SELECT customer_id FROM orders);\n",
        "\n",
        "4. **Input**: \"List the product names and their average rating for products with more than 100 reviews.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT product_name, AVG(rating) AS average_rating\n",
        "   FROM reviews\n",
        "   GROUP BY product_name\n",
        "   HAVING COUNT(*) > 100;\n",
        "\n",
        "5. **Input**: \"Get the total sales and the number of orders for each product sold in January 2024.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT product_id, SUM(sales_amount) AS total_sales, COUNT(order_id) AS order_count\n",
        "   FROM sales\n",
        "   WHERE order_date BETWEEN '2024-01-01' AND '2024-01-31'\n",
        "   GROUP BY product_id;\n",
        "\n",
        "Here is the schema information for our database :\n",
        "{schema_info}\n",
        "\n",
        "Please only answer queries which makes sense for the given schema. Else just return - \"No information found\"\n",
        "\n",
        "Now, translate the following natural language query into an SQL query:\n",
        "**Input**: {natural_language_query}\n",
        "**Output**:\n",
        "\n",
        "\"\"\"\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hiii8gPF2YuZ"
      },
      "source": [
        "It is essentially the same except that we have added the following information -<br>\n",
        ">'''Here is the schema information for our database :<br>\n",
        ">{schema_info}\n",
        "\n",
        ">Please only answer queries which makes sense for the given schema. Else just return - \"No information found\"\n",
        "'''"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "id": "9GoKI-_j4aJH"
      },
      "outputs": [],
      "source": [
        "input_text = \"\"\"\n",
        "How many students are of above 7 years of age?\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "id": "0b4MrQ2x34H2"
      },
      "outputs": [],
      "source": [
        "schema_info = get_schema_information()\n",
        "query = convert_text_to_query_with_schema(input_text,schema_info,user_prompt,system_prompt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "5CDn6Had2-Ij",
        "outputId": "535ad63a-c070-493b-8404-e248488d5ca9"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'SELECT COUNT(*)\\nFROM student\\nWHERE age > 7;'"
            ]
          },
          "execution_count": 45,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "query"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TObkp0xd3IQN",
        "outputId": "bd7335fd-4d69-4934-e7d5-d2b3f14119a6"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'COUNT(*)': 3}]"
            ]
          },
          "execution_count": 46,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "execute_sql_query(query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OR-4f_hn3CUB"
      },
      "source": [
        "Viola! It works great!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1_Kii_zY3L_I"
      },
      "source": [
        "Now, lets try to ask something data that is not present in database."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "metadata": {
        "id": "3LbTCfTG3AXe"
      },
      "outputs": [],
      "source": [
        "input_text = \"\"\"\n",
        "How many teachers teach in the school?\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {
        "id": "EUmLV5fJ3lrS"
      },
      "outputs": [],
      "source": [
        "schema_info = get_schema_information()\n",
        "query = convert_text_to_query_with_schema(input_text,schema_info,user_prompt,system_prompt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "JS3fVmJm3nnU",
        "outputId": "b672f26c-20d6-47a7-d342-5b83cbfb6999"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'No information found'"
            ]
          },
          "execution_count": 49,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "query"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AQ-ohIJD3slu"
      },
      "source": [
        "You see! The LLM returns 'No information found'."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O8zwZuEcXpg6"
      },
      "source": [
        "#  Download LLM weights locally"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UWj5vlFvXtWp"
      },
      "source": [
        "Many times we may not be comfortable exposing the database schema information with an alien LLM. Who knows where this information can be used and how? So what we will do is download the weights of a LLM and use it to produce results locally."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BXbWAeFLYFHK"
      },
      "source": [
        "We are going to use Google' Gemma model with 7 billion parameters. The information on this can be found [here](https://huggingface.co/google/gemma-7b)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FQwuykdGgIo3"
      },
      "source": [
        "Please note that there are other good models out there. The best ones are not open sourced and hence, cannot be downloaded for example GPT-4. Some have too much many parameters to download on a single GPU."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KeHjHRImYmfJ"
      },
      "source": [
        "We need to have a GPU to run this model locally. I am using Google Colab's L4 GPU."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LoVSnrfs19U6"
      },
      "source": [
        "Much of the following code is reused from Daniel Bourke's fantastic [tutorial](https://github.com/mrdbourke/simple-local-rag)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VVLL1TTGZDGj"
      },
      "source": [
        "Firts let's import some libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UZiXiTSbXstK",
        "outputId": "2e712fcf-1b82-4474-d392-501fc6ee0eef"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "  Building wheel for flash-attn (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for flash-attn: filename=flash_attn-2.6.3-cp310-cp310-linux_x86_64.whl size=187309225 sha256=237ef9c6157db394e1ddde4ba609a21ebb98382377a27041edc09318801a6f24\n",
            "  Stored in directory: /root/.cache/pip/wheels/7e/e3/c3/89c7a2f3c4adc07cd1c675f8bb7b9ad4d18f64a72bccdfe826\n",
            "Successfully built flash-attn\n",
            "Installing collected packages: flash-attn\n",
            "Successfully installed flash-attn-2.6.3\n"
          ]
        }
      ],
      "source": [
        "import os\n",
        "if \"COLAB_GPU\" in os.environ:\n",
        "    !pip install -U torch # requires torch 2.1.1+ (for efficient sdpa implementation)\n",
        "    !pip install accelerate # for quantization model loading\n",
        "    !pip install flash-attn --no-build-isolation # for faster attention mechanism = faster LLM inference"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hClCGQREa0HH"
      },
      "source": [
        "Let's see how much memory have we got"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I-RWUztAZKcq",
        "outputId": "7721d9cc-07a6-4227-ea65-01a0323b7133"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Sun Aug 18 04:52:59 2024       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  NVIDIA L4                      Off | 00000000:00:03.0 Off |                    0 |\n",
            "| N/A   42C    P8              16W /  72W |      1MiB / 23034MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "|  No running processes found                                                           |\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RN5LFdFcbbkm",
        "outputId": "1bb52374-3cd3-40e5-b98d-f072ccc6c7ff"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Available GPU memory: 22 GB\n"
          ]
        }
      ],
      "source": [
        "#Let's verify it with code.\n",
        "# Get GPU available memory\n",
        "import torch\n",
        "gpu_memory_bytes = torch.cuda.get_device_properties(0).total_memory\n",
        "gpu_memory_gb = round(gpu_memory_bytes / (2**30))\n",
        "print(f\"Available GPU memory: {gpu_memory_gb} GB\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LFmIVTk7a7cK"
      },
      "source": [
        "We got around 22 GB of memory. Our model has 7 billion parameters and would need approximately 19 GB of memory"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TG6y8QUGbN7l"
      },
      "source": [
        "The following table shows different amounts of GPU memory requirements for different verions of the Gemma LLMs with varying levels of precision."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yYMtCyLVbRw6"
      },
      "source": [
        "| Model             | Precision | Min-Memory (Bytes) | Min-Memory (MB) | Min-Memory (GB) | Recommended Memory (GB) | Hugging Face ID |\n",
        "|-------------------|-----------|----------------|-------------|-------------| ----- | ----- |\n",
        "| [Gemma 2B](https://huggingface.co/google/gemma-2b-it)          | 4-bit     | 2,106,749,952  | 2009.15     | 1.96        | ~5.0 | [`gemma-2b`](https://huggingface.co/google/gemma-2b) or [`gemma-2b-it`](https://huggingface.co/google/gemma-2b-it) for instruction tuned version |\n",
        "| Gemma 2B          | Float16   | 5,079,453,696  | 4844.14     | 4.73        | ~8.0 | Same as above |\n",
        "| [Gemma 7B](https://huggingface.co/google/gemma-7b-it)          | 4-bit     | 5,515,859,968  | 5260.33     | 5.14        | ~8.0 | [`gemma-7b`](https://huggingface.co/google/gemma-7b) or [`gemma-7b-it`](https://huggingface.co/google/gemma-7b-it) for instruction tuned version |\n",
        "| Gemma 7B          | Float16   | 17,142,470,656 | 16348.33    | 15.97       | ~19 | Same as above |\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zaG8xwuUbn8o"
      },
      "source": [
        "We are using a hugging face model. Let's login to that. If you dont have an account already, please create one. You might need a key to authenticate yourself."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lrwnq6l5a6Q2"
      },
      "outputs": [],
      "source": [
        "!huggingface-cli login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z1JsT23jb9VC"
      },
      "source": [
        "Let's check my username"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d1sS3qzvb6j-"
      },
      "outputs": [],
      "source": [
        "!huggingface-cli whoami"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 54,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000,
          "referenced_widgets": [
            "b7abe071c90443c0a57432171e159e8f",
            "d0ff1d176aea48d5a229a6e1ab3028d2",
            "e81041a4c266424496f33000be5ed775",
            "9988a9e6272a48ff92fccbe8f63ece81",
            "605ecedab51d4cc3ab6c1617d13bcda5",
            "8b1846fc7c0742aebea7500c94444115",
            "ede9f9b11dec49b59e3627d9068b7380",
            "82bfbe7d7bb5488eb4379f07bdb3cf9d",
            "cebaf637eb4e4d4fafad2a041b6933e5",
            "c37bfdcaadb5403cb44f8b94c87da256",
            "0a28527b763d43248954c32d90ab84a6",
            "f651b08d00aa4c39a8ac79283bddbda5",
            "53ae8459e79d4d72b8f348c3af532c75",
            "75ee325e981549bbaf5e3efe3d589ab8",
            "f61947902bb5407a8428848f8141d134",
            "c9addaf0418e47868702d200857e6ec2",
            "fb93a5b6e0f74956b6be2f9c5d93442c",
            "1bdce5bf4e844a40a2732513a81fe9c1",
            "41baaecfe3354ce3a10977053278f8d3",
            "d22ac654d63a4ba892d9a4e952ccfaf1",
            "1fc38a1eb20d4050b2ca38dd6674fc8b",
            "b755654e0ec7474b89af6850de90acf3",
            "dcde971a54144fcb8d1f6c00431f1c62",
            "fbdf2fdd0f5b4dd1bb78d115a4992991",
            "b098e6c5330044bdb6a0b9bb38390997",
            "54840eaf055b4fad8dca39269f2e7d02",
            "38b9a3a2886b4085800fe0def52f2a12",
            "eb9a88cea4a8402288ed14d003cecda0",
            "2e46ea37ed454fc5b8ba061b8b9fa769",
            "520f078b1b26416da3ec43460b3b6691",
            "de53da01b3184348a4686e709f283bf4",
            "092910902bbd4477881d8fe30a70ecf0",
            "2aecb6f942664dfeb4132045186344f0",
            "05e42c4bef564b47a5b00beebfe3eb32",
            "1b232523dcf94046880737a105192b14",
            "e46c28ce4af94322ab6ceb867b1923a8",
            "a91b5a54c95c44d3bc4eebdbb7dbc9c3",
            "aabad6dc5f3b4638bacd292a1149047c",
            "ba95a64f0c114dba83b56f5c770dfa44",
            "68c6d4bc9fea457f9fec6d6187bd23d6",
            "398bae342f7845e1b54ef8d073df3d27",
            "2d1e706019964550b051d66c0747cf61",
            "6c35afdbe75e49b0913e0d782ebe302b",
            "9131b59d8992452b8e555bd13b109e89",
            "0eaec31313144e878e7e0a8adff4cf0f",
            "1d54f848ae54441da2d8f2168f26b791",
            "800599524ce64b4eb9fb70e63d552661",
            "630678b899bb426e8a486cc6ac632476",
            "4bc410a19d6842f9a5335efcd0cc98fd",
            "db29eb577f8e496fb103820e77d67862",
            "fd769f30fe1e45a0bf8db6eacc16537f",
            "aeb2b8d3a1fa4548a61db757ba3f2e8c",
            "539fdd36b0b841969ffedb97e6de14b5",
            "5f47840d39bc43569f5022a8d67f83df",
            "78cdeeebec9c4726a8c35686d50bcf51",
            "c72503ef69ea4857a59946051e214ced",
            "7b6c8c6c85ea477abb3cfc03b14f0194",
            "d399b44c59e44143b3bcd84cb7104fdc",
            "0e4ce29ae35e41cfa75265cd3a0402a3",
            "c4ba886b0fa1406382e07097c3aae7e2",
            "9b94eea8aa774023ab0218c9edae35fe",
            "658e7c4eb5a64458a5a296134044b9d2",
            "e8f4eeef3c494c36adaf2814c3f5fcc6",
            "d2dfefaf9fe54e909871995b782e7da9",
            "83ffee7a05fc4132b53aa6a3b6af8d02",
            "bf378e2b7468423999535bbc70ff9c6a",
            "0196d3a288474ffcbe9ebf6810f5c9ab",
            "b875ef06b75e41ce8a59d53fc62634e1",
            "60c33f2376f64a44992bd92accadd892",
            "45685571010a47c9b15a4d755702ba85",
            "e44fcf3348ac4d08bff9a88ae3dd8e82",
            "6a70571d34894bcd9ff04718a68686cb",
            "36e6c80b9195412fa5f94b72863fcffd",
            "21a20ca8f35b4bb1ad2a0391d496e09d",
            "f487f3ed5b21448483f5fab52db690dd",
            "0243cd33cd8a4c2db44dd63ce9ca0311",
            "bdf3e8483e63432fab7f2165981c0aa6",
            "6b402269d8b94c0a9e6e026e10a356e4",
            "f01951b9dd244dd9a0e206326c468299",
            "c5dcddd2f1744c01945e85ea1f57835f",
            "af924af108f74451a06a48306c5f7d95",
            "b499e28fb1f14874a62cee3857fe5499",
            "47ea1dd9660e4eacb96486b5d8408fcd",
            "9470716b1d984e6a8a403dd443249060",
            "8618a7a4e176474d802b24d021d669a5",
            "891e330b593049e9be5effc6a562263d",
            "d98be4f3768c4bf89ccba65e6436c08c",
            "6d3d433dcbee4bed814b36fadbe5ddc2",
            "2767c0cf7bb4457bb7e0ff4713794de9",
            "46b8079935b0440e95bf32c2c45e248d",
            "d63a8d62f4af4a3b804c903ff2508963",
            "1267bb2744eb4250a73bcd30b50a3619",
            "493937a09f694160bf652f01545f3bc6",
            "db41c962b1db46e3a429f386dbd069fa",
            "6ed8dd6e7f9f434b887c8cfd06f0cc37",
            "e685530588994345bc56b4280725aa55",
            "04d54078ff024ac68f9c1b0da86cfb33",
            "3bb3cb8d5196434faf0aad12900a1d32",
            "053599d1f2764dc7b55b9e3e28efccca",
            "46069cf2dfd14aa88e412cb93abea47d",
            "c3059994d519412da47798d485311cd9",
            "bffe4e632ae448868bc637354f5ad892",
            "674d0cd8ce1f452b9f479687294f3907",
            "dc5b8fdd54d645d2a671b3852db1c850",
            "b06b3426f8d146748883d218c16004f7",
            "3a122d57300c4d27aa36d5fbc17b2886",
            "0c8c70734dbc4abaaab07c522ef0eb7a",
            "a5aca8b9fc764a1e85610d760b11b451",
            "17b2e683caab49e3a0e857a3a1dca647",
            "a672510a012348bdb0ed44c0efe5806f",
            "eb225ecadd174092b872ff1ac5ccc33d",
            "13367f9b6b7140b78a0fb4baca107dd2",
            "058854285af14cf999666324bf29ff8d",
            "b3e8ff96593246829d1f00d170b28718",
            "20cfd6b6530143f6914d8ea616ae3847",
            "c469862077124062967ea461bcfdd757",
            "f60356522cd940658848658f2035a603",
            "d4eea5cc9a6e4746a86469bf6569765d",
            "d266470c38264712be92bf7a9d023f5a",
            "4b2a8378c1c54be8b7db3e485efd29e0",
            "3618c8c43a3241b8a54478b26649141a",
            "c5f7435be86d4e72846600298d9c9e86",
            "a6a2cc4568f94404a05dd99b0a4379a0",
            "053946b3326a4c4facee90985508ec6b",
            "741d8ee671a4489d82ebec6ad928767b",
            "43dfa21cd46344d6b6145ca6eb9a4847",
            "831616cc0c7b4a1f890c73cb6075e37a",
            "c13b4febf7bb4dc69bd2fa91a9560d31",
            "367fefce415c4256ac960c1326027ff8",
            "4cf8150d909d4d38acccfbdf077bb2fa",
            "9ea4d8fd79f14a529533aa7463575ebc",
            "ad135cc1c8f04a3ab52389c0dbed9ea9",
            "ed5e32084b0c40f7a8f723e5d2d18a65",
            "08dee47a9a794977bc8828f18237d38e",
            "a333eddf1ecf438a89550ea6bc4b9a9d",
            "b69c8c1744a945338cbf47b02fb990ca",
            "bb328ff690e34d2186a39aa793c29723",
            "a2735d52b7a94c4489b21aaf9d5ee6e8",
            "995f609c294842b8a2383b8d1313c4ed",
            "e26ba603400d4ec2b1ee9b5de153ea45",
            "401a85ab9f814f7e90c2b48b78191e6f",
            "d870495ce8194784b4472d38d7e27b3d",
            "f5367a7daf1b4491b3569ec140fcac84"
          ]
        },
        "id": "fm8Y2i76cKcq",
        "outputId": "2063a67b-25bb-4b3c-88f9-73d3efc98024"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Using attention implementation: flash_attention_2\n",
            "[INFO] Using model_id: google/gemma-7b-it\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:89: UserWarning: \n",
            "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
            "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
            "You will be able to reuse this secret in all of your notebooks.\n",
            "Please note that authentication is recommended but still optional to access public models or datasets.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "b7abe071c90443c0a57432171e159e8f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/34.2k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "f651b08d00aa4c39a8ac79283bddbda5",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.model:   0%|          | 0.00/4.24M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "dcde971a54144fcb8d1f6c00431f1c62",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/17.5M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "05e42c4bef564b47a5b00beebfe3eb32",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "special_tokens_map.json:   0%|          | 0.00/636 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0eaec31313144e878e7e0a8adff4cf0f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "config.json:   0%|          | 0.00/694 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "c72503ef69ea4857a59946051e214ced",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model.safetensors.index.json:   0%|          | 0.00/20.9k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0196d3a288474ffcbe9ebf6810f5c9ab",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading shards:   0%|          | 0/4 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6b402269d8b94c0a9e6e026e10a356e4",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00001-of-00004.safetensors:   0%|          | 0.00/5.00G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "2767c0cf7bb4457bb7e0ff4713794de9",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00002-of-00004.safetensors:   0%|          | 0.00/4.98G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "46069cf2dfd14aa88e412cb93abea47d",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00003-of-00004.safetensors:   0%|          | 0.00/4.98G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "eb225ecadd174092b872ff1ac5ccc33d",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00004-of-00004.safetensors:   0%|          | 0.00/2.11G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.\n",
            "`config.hidden_act` is ignored, you should use `config.hidden_activation` instead.\n",
            "Gemma's activation function will be set to `gelu_pytorch_tanh`. Please, use\n",
            "`config.hidden_activation` if you want to override this behaviour.\n",
            "See https://github.com/huggingface/transformers/pull/29402 for more details.\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "c5f7435be86d4e72846600298d9c9e86",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ed5e32084b0c40f7a8f723e5d2d18a65",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/137 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "GemmaForCausalLM(\n",
              "  (model): GemmaModel(\n",
              "    (embed_tokens): Embedding(256000, 3072, padding_idx=0)\n",
              "    (layers): ModuleList(\n",
              "      (0-27): 28 x GemmaDecoderLayer(\n",
              "        (self_attn): GemmaFlashAttention2(\n",
              "          (q_proj): Linear(in_features=3072, out_features=4096, bias=False)\n",
              "          (k_proj): Linear(in_features=3072, out_features=4096, bias=False)\n",
              "          (v_proj): Linear(in_features=3072, out_features=4096, bias=False)\n",
              "          (o_proj): Linear(in_features=4096, out_features=3072, bias=False)\n",
              "          (rotary_emb): GemmaRotaryEmbedding()\n",
              "        )\n",
              "        (mlp): GemmaMLP(\n",
              "          (gate_proj): Linear(in_features=3072, out_features=24576, bias=False)\n",
              "          (up_proj): Linear(in_features=3072, out_features=24576, bias=False)\n",
              "          (down_proj): Linear(in_features=24576, out_features=3072, bias=False)\n",
              "          (act_fn): PytorchGELUTanh()\n",
              "        )\n",
              "        (input_layernorm): GemmaRMSNorm()\n",
              "        (post_attention_layernorm): GemmaRMSNorm()\n",
              "      )\n",
              "    )\n",
              "    (norm): GemmaRMSNorm()\n",
              "  )\n",
              "  (lm_head): Linear(in_features=3072, out_features=256000, bias=False)\n",
              ")"
            ]
          },
          "execution_count": 54,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import torch\n",
        "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
        "from transformers.utils import is_flash_attn_2_available\n",
        "\n",
        "# Setup Flash Attention 2 for faster inference, default to \"sdpa\" or \"scaled dot product attention\" if it's not available\n",
        "# Flash Attention 2 requires NVIDIA GPU compute capability of 8.0 or above, see: https://developer.nvidia.com/cuda-gpus\n",
        "# Requires !pip install flash-attn, see: https://github.com/Dao-AILab/flash-attention\n",
        "if (is_flash_attn_2_available()) and (torch.cuda.get_device_capability(0)[0] >= 8):\n",
        "  attn_implementation = \"flash_attention_2\"\n",
        "else:\n",
        "  attn_implementation = \"sdpa\"\n",
        "print(f\"[INFO] Using attention implementation: {attn_implementation}\")\n",
        "\n",
        "# 1. Pick a model we'd like to use (this will depend on how much GPU memory you have available)\n",
        "\n",
        "model_id = \"google/gemma-7b-it\"\n",
        "print(f\"[INFO] Using model_id: {model_id}\")\n",
        "\n",
        "# 2. Instantiate tokenizer (tokenizer turns text into numbers ready for the model)\n",
        "model_tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path=model_id)\n",
        "\n",
        "# 3. Instantiate the model\n",
        "llm_model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path=model_id,\n",
        "                                                 torch_dtype=torch.float16, # datatype to use, we want float16\n",
        "                                                 quantization_config=None,\n",
        "                                                 low_cpu_mem_usage=False, # use full memory\n",
        "                                                 attn_implementation=attn_implementation) # which attention version to use\n",
        "\n",
        "llm_model.to(\"cuda\")\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qM8DeLb5c5ne"
      },
      "source": [
        "Let's check our model's architecture"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 55,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NmOh9KrrcsiO",
        "outputId": "7ae6c7e8-d02b-4c10-f010-0c4ac7d53b98"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "GemmaForCausalLM(\n",
              "  (model): GemmaModel(\n",
              "    (embed_tokens): Embedding(256000, 3072, padding_idx=0)\n",
              "    (layers): ModuleList(\n",
              "      (0-27): 28 x GemmaDecoderLayer(\n",
              "        (self_attn): GemmaFlashAttention2(\n",
              "          (q_proj): Linear(in_features=3072, out_features=4096, bias=False)\n",
              "          (k_proj): Linear(in_features=3072, out_features=4096, bias=False)\n",
              "          (v_proj): Linear(in_features=3072, out_features=4096, bias=False)\n",
              "          (o_proj): Linear(in_features=4096, out_features=3072, bias=False)\n",
              "          (rotary_emb): GemmaRotaryEmbedding()\n",
              "        )\n",
              "        (mlp): GemmaMLP(\n",
              "          (gate_proj): Linear(in_features=3072, out_features=24576, bias=False)\n",
              "          (up_proj): Linear(in_features=3072, out_features=24576, bias=False)\n",
              "          (down_proj): Linear(in_features=24576, out_features=3072, bias=False)\n",
              "          (act_fn): PytorchGELUTanh()\n",
              "        )\n",
              "        (input_layernorm): GemmaRMSNorm()\n",
              "        (post_attention_layernorm): GemmaRMSNorm()\n",
              "      )\n",
              "    )\n",
              "    (norm): GemmaRMSNorm()\n",
              "  )\n",
              "  (lm_head): Linear(in_features=3072, out_features=256000, bias=False)\n",
              ")"
            ]
          },
          "execution_count": 55,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "llm_model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zd5oJnUFc4kl"
      },
      "source": [
        "How about we get number of parameters in our model?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W07E1keRcweo",
        "outputId": "d2fb4990-aad8-4708-8aa2-e57d0c3233a4"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "8537680896"
            ]
          },
          "execution_count": 56,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def get_model_num_params(model: torch.nn.Module):\n",
        "    return sum([param.numel() for param in model.parameters()])\n",
        "\n",
        "get_model_num_params(llm_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A6_NCBHUdLQq"
      },
      "source": [
        "How about we get model memory requirements?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8WMz01Ewch1M",
        "outputId": "04c38a36-2149-4390-fd49-27d517670583"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'model_mem_bytes': 17075376128,\n",
              " 'model_mem_mb': 16284.35,\n",
              " 'model_mem_gb': 15.9}"
            ]
          },
          "execution_count": 57,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def get_model_mem_size(model: torch.nn.Module):\n",
        "    \"\"\"\n",
        "    Get how much memory a PyTorch model takes up.\n",
        "\n",
        "    See: https://discuss.pytorch.org/t/gpu-memory-that-model-uses/56822\n",
        "    \"\"\"\n",
        "    # Get model parameters and buffer sizes\n",
        "    mem_params = sum([param.nelement() * param.element_size() for param in model.parameters()])\n",
        "    mem_buffers = sum([buf.nelement() * buf.element_size() for buf in model.buffers()])\n",
        "\n",
        "    # Calculate various model sizes\n",
        "    model_mem_bytes = mem_params + mem_buffers # in bytes\n",
        "    model_mem_mb = model_mem_bytes / (1024**2) # in megabytes\n",
        "    model_mem_gb = model_mem_bytes / (1024**3) # in gigabytes\n",
        "\n",
        "    return {\"model_mem_bytes\": model_mem_bytes,\n",
        "            \"model_mem_mb\": round(model_mem_mb, 2),\n",
        "            \"model_mem_gb\": round(model_mem_gb, 2)}\n",
        "\n",
        "get_model_mem_size(llm_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TWpJCG3qdRxC"
      },
      "source": [
        "Generating text with LLM"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 58,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BhdAIxHbdOmn",
        "outputId": "fa70f4ca-0193-44d0-b93f-61fdc81b3f9f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Input text:\n",
            "What is LLM?\n",
            "\n",
            "Prompt (formatted):\n",
            "<bos><start_of_turn>user\n",
            "What is LLM?<end_of_turn>\n",
            "<start_of_turn>model\n",
            "\n"
          ]
        }
      ],
      "source": [
        "input_text = \"What is LLM?\"\n",
        "print(f\"Input text:\\n{input_text}\")\n",
        "\n",
        "# Create prompt template for instruction-tuned model\n",
        "dialogue_template = [\n",
        "    {\"role\": \"user\",\n",
        "     \"content\": input_text}\n",
        "]\n",
        "\n",
        "# Apply the chat template\n",
        "prompt = model_tokenizer.apply_chat_template(conversation=dialogue_template,\n",
        "                                       tokenize=False, # keep as raw text (not tokenized)\n",
        "                                       add_generation_prompt=True)\n",
        "print(f\"\\nPrompt (formatted):\\n{prompt}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 59,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oLmqtPRudlfY",
        "outputId": "f1edf7cd-18f3-4a41-8ddc-43560443fa65"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Model input (tokenized):\n",
            "tensor([[     2,    106,   1645,    108,   1841,    603,    629,  18622, 235336,\n",
            "            107,    108,    106,   2516,    108]], device='cuda:0')\n",
            "\n",
            "Model output (tokens):\n",
            "tensor([     2,    106,   1645,    108,   1841,    603,    629,  18622, 235336,\n",
            "           107,    108,    106,   2516,    108,   1650, 235296,  12353,    604,\n",
            "         14944,  18454,  14944,   5708, 235265,   1165,    603,    476,   1916,\n",
            "           576,   2910,   5255,   2091,    591,   1650, 235296, 235275,    674,\n",
            "           603,  14076,   6869,    577,   3508,    578,  11941,   3515, 235290,\n",
            "          5023,   2793, 235265,  25599,  14816,    708,  15976,  17363,    611,\n",
            "         12380,  15992,    576,   2793,   1423, 235269,    578,    984,    708,\n",
            "          3326,    577,  18254,    575,    476,   5396,   3001,    576,  13333,\n",
            "        235269,   3359,   2793,  20823,   2465, 235269,  17183, 235269,   3409,\n",
            "         10304, 235269,    578,   2872,  39534, 235265,      1],\n",
            "       device='cuda:0')\n",
            "\n",
            "CPU times: user 6.54 s, sys: 268 ms, total: 6.81 s\n",
            "Wall time: 8.24 s\n"
          ]
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "# Tokenize the input text (turn it into numbers) and send it to GPU\n",
        "input_ids = model_tokenizer.encode(prompt, add_special_tokens=False, return_tensors=\"pt\").to(\"cuda\")\n",
        "print(f\"Model input (tokenized):\\n{input_ids}\\n\")\n",
        "\n",
        "# Generate outputs passed on the tokenized input\n",
        "# See generate docs: https://huggingface.co/docs/transformers/v4.38.2/en/main_classes/text_generation#transformers.GenerationConfig\n",
        "outputs = llm_model.generate(input_ids=input_ids.to(llm_model.device),\n",
        "                             max_new_tokens=512) # define the maximum number of new tokens to create\n",
        "print(f\"Model output (tokens):\\n{outputs[0]}\\n\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 60,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eJM9ml8Sdo3E",
        "outputId": "6e64d7af-8254-40a8-85d5-f4bac65b2a7d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Model output (decoded):\n",
            "<bos><start_of_turn>user\n",
            "What is LLM?<end_of_turn>\n",
            "<start_of_turn>model\n",
            "LLM stands for Language Large Language Model. It is a type of large language model (LLM) that is specifically designed to understand and generate human-like text. LLMs are typically trained on vast amounts of text data, and they are able to engage in a wide range of tasks, including text summarization, translation, code generation, and question answering.<eos>\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# Decode the output tokens to text\n",
        "outputs_decoded = model_tokenizer.decode(outputs[0])\n",
        "print(f\"Model output (decoded):\\n{outputs_decoded}\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9guFaXYHd1G8"
      },
      "source": [
        "Let's write a function to use this LLM and generate query. We will tweak our function 'convert_text_to_query_with_schema' a bit to suit Google's Gemma."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {
        "id": "NgNtR7BleDu9"
      },
      "outputs": [],
      "source": [
        "def convert_text_to_query_locally(input_text,schema_info,user_prompt):\n",
        "\n",
        "  # Create prompt template for instruction-tuned model\n",
        "  dialogue_template = [\n",
        "      {\"role\": \"user\", \"content\": user_prompt.format(natural_language_query=input_text,schema_info=schema_info)}\n",
        "  ]\n",
        "\n",
        "  # Apply the chat template\n",
        "  prompt = model_tokenizer.apply_chat_template(conversation=dialogue_template,\n",
        "                                        tokenize=False, # keep as raw text (not tokenized)\n",
        "                                        add_generation_prompt=True)\n",
        "\n",
        "\n",
        "\n",
        "  # Tokenize the input text (turn it into numbers) and send it to GPU\n",
        "  input_ids = model_tokenizer.encode(prompt, add_special_tokens=False, return_tensors=\"pt\").to(\"cuda\")\n",
        "\n",
        "\n",
        "  # Generate outputs passed on the tokenized input\n",
        "\n",
        "  outputs = llm_model.generate(input_ids=input_ids.to(llm_model.device),\n",
        "                              max_new_tokens=512)\n",
        "\n",
        "  llm_output = model_tokenizer.decode(outputs[0])\n",
        "\n",
        "  llm_output = llm_output.replace(prompt,\"\").replace(\"\\n\",\" \")\n",
        "\n",
        "\n",
        "\n",
        "  return llm_output\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WBnwK3WDi0Cv"
      },
      "source": [
        "Notice how the Gemma does not take \"system\" role and hence it is not part of our prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 62,
      "metadata": {
        "id": "pMqkMODFeQ8B"
      },
      "outputs": [],
      "source": [
        "\n",
        "\n",
        "user_prompt =\"\"\"\n",
        "You are a highly skilled SQL translator. Your task is to convert natural language descriptions of data queries into correct and optimized SQL statements.\n",
        "\n",
        "Here are a few examples:\n",
        "\n",
        "1. **Input**: \"Select the names and salaries of all employees in the 'Finance' department.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT name, salary\n",
        "   FROM employees\n",
        "   WHERE department = 'Finance';\n",
        "\n",
        "2. **Input**: \"Find the total number of orders placed by customers in 2023.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT COUNT(*)\n",
        "   FROM orders\n",
        "   WHERE YEAR(order_date) = 2023;\n",
        "\n",
        "3. **Input**: \"Retrieve the email addresses of customers who have not placed any orders.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT email\n",
        "   FROM customers\n",
        "   WHERE customer_id NOT IN (SELECT customer_id FROM orders);\n",
        "\n",
        "4. **Input**: \"List the product names and their average rating for products with more than 100 reviews.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT product_name, AVG(rating) AS average_rating\n",
        "   FROM reviews\n",
        "   GROUP BY product_name\n",
        "   HAVING COUNT(*) > 100;\n",
        "\n",
        "5. **Input**: \"Get the total sales and the number of orders for each product sold in January 2024.\"\n",
        "   **Output**:\n",
        "\n",
        "   SELECT product_id, SUM(sales_amount) AS total_sales, COUNT(order_id) AS order_count\n",
        "   FROM sales\n",
        "   WHERE order_date BETWEEN '2024-01-01' AND '2024-01-31'\n",
        "   GROUP BY product_id;\n",
        "\n",
        "Here is the schema information for our database :\n",
        "{schema_info}\n",
        "\n",
        "Please do not return additional text besides query.\n",
        "\n",
        "Please only answer queries which makes sense for the given schema. Else just return - \"No information found\"\n",
        "\n",
        "Now, translate the following natural language query into an SQL query:\n",
        "**Input**: {natural_language_query}\n",
        "**Output**:\n",
        "\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 63,
      "metadata": {
        "id": "SpDa2iyBf8yT"
      },
      "outputs": [],
      "source": [
        "input_text = \"\"\"\n",
        "How many students are of above 7 years of age?\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 64,
      "metadata": {
        "id": "Py-CDC6OfzNu"
      },
      "outputs": [],
      "source": [
        "schema_info = get_schema_information()\n",
        "query = convert_text_to_query_locally(input_text,schema_info,user_prompt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 65,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "GSMV31GznBNY",
        "outputId": "5d3ddcf2-384d-4c86-e903-df9fce79aeea"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Sure, here is the translated SQL query:  SELECT COUNT(*) FROM student WHERE age > 7;<eos>'"
            ]
          },
          "execution_count": 65,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "query"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jO20drWAlkP4"
      },
      "source": [
        "Let's do little cleaning"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 66,
      "metadata": {
        "id": "CGWA2F0ZljoL"
      },
      "outputs": [],
      "source": [
        "import re\n",
        "\n",
        "def clean_query(query):\n",
        "\n",
        "  if(\"no information found\" in query.lower()):\n",
        "    return \"No information found\"\n",
        "\n",
        "  query =  query.replace(\"<eos>\",\"\")\n",
        "\n",
        "  pattern = r'(select.*?;)'\n",
        "\n",
        "  match = re.search(pattern, query, re.IGNORECASE)\n",
        "\n",
        "  # Extract and print the matched substring\n",
        "  if match:\n",
        "      extracted_query = match.group(1)\n",
        "      return extracted_query\n",
        "  else:\n",
        "      print(\"No matching query found.\")\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 68,
      "metadata": {
        "id": "9na8aXr4mObd"
      },
      "outputs": [],
      "source": [
        "query = clean_query(query)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 69,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yebvKaNempiZ",
        "outputId": "3b338e6c-0b88-4165-d7cc-50370bb35691"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'COUNT(*)': 3}]"
            ]
          },
          "execution_count": 69,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "execute_sql_query(query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "StU5hrLooj8n"
      },
      "source": [
        "Let's try to trick it to hallucinate"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 70,
      "metadata": {
        "id": "1TOXQxcmojWu"
      },
      "outputs": [],
      "source": [
        "input_text = \"\"\"\n",
        "How many teachers are there in the school?\n",
        "\"\"\"\n",
        "schema_info = get_schema_information()\n",
        "query = convert_text_to_query_locally(input_text,schema_info,user_prompt)\n",
        "query = clean_query(query)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 71,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "FPESD4YAo2SV",
        "outputId": "64288f1e-48e8-4a3e-b663-6099393da2af"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"SELECT COUNT(*) FROM teachers WHERE school_id = (SELECT id FROM schools WHERE name = 'My School');\""
            ]
          },
          "execution_count": 71,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "query"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 72,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "n9eXzfVUoxc_",
        "outputId": "b3e468f1-0891-47f8-f680-e577cf9efdbb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "An error occurred: no such table: teachers\n"
          ]
        }
      ],
      "source": [
        "execute_sql_query(query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NQ7VFWtVo8-J"
      },
      "source": [
        "You see! This model can be easily be hallucinated even though we have provided it with schema information. This is the drawback of using smaller LLMs."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RWE-BGQb2zzb"
      },
      "source": [
        "In the subsequent tutorial, we will try to use some simple and quick finetuning to fix this issue."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "L4",
      "machine_shape": "hm",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "0196d3a288474ffcbe9ebf6810f5c9ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_b875ef06b75e41ce8a59d53fc62634e1",
              "IPY_MODEL_60c33f2376f64a44992bd92accadd892",
              "IPY_MODEL_45685571010a47c9b15a4d755702ba85"
            ],
            "layout": "IPY_MODEL_e44fcf3348ac4d08bff9a88ae3dd8e82"
          }
        },
        "0243cd33cd8a4c2db44dd63ce9ca0311": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "04d54078ff024ac68f9c1b0da86cfb33": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "053599d1f2764dc7b55b9e3e28efccca": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "053946b3326a4c4facee90985508ec6b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_367fefce415c4256ac960c1326027ff8",
            "max": 4,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4cf8150d909d4d38acccfbdf077bb2fa",
            "value": 4
          }
        },
        "058854285af14cf999666324bf29ff8d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d4eea5cc9a6e4746a86469bf6569765d",
            "max": 2113988336,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d266470c38264712be92bf7a9d023f5a",
            "value": 2113988336
          }
        },
        "05e42c4bef564b47a5b00beebfe3eb32": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1b232523dcf94046880737a105192b14",
              "IPY_MODEL_e46c28ce4af94322ab6ceb867b1923a8",
              "IPY_MODEL_a91b5a54c95c44d3bc4eebdbb7dbc9c3"
            ],
            "layout": "IPY_MODEL_aabad6dc5f3b4638bacd292a1149047c"
          }
        },
        "08dee47a9a794977bc8828f18237d38e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a2735d52b7a94c4489b21aaf9d5ee6e8",
            "placeholder": "​",
            "style": "IPY_MODEL_995f609c294842b8a2383b8d1313c4ed",
            "value": "generation_config.json: 100%"
          }
        },
        "092910902bbd4477881d8fe30a70ecf0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0a28527b763d43248954c32d90ab84a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0c8c70734dbc4abaaab07c522ef0eb7a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0e4ce29ae35e41cfa75265cd3a0402a3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_83ffee7a05fc4132b53aa6a3b6af8d02",
            "placeholder": "​",
            "style": "IPY_MODEL_bf378e2b7468423999535bbc70ff9c6a",
            "value": " 20.9k/20.9k [00:00&lt;00:00, 1.75MB/s]"
          }
        },
        "0eaec31313144e878e7e0a8adff4cf0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1d54f848ae54441da2d8f2168f26b791",
              "IPY_MODEL_800599524ce64b4eb9fb70e63d552661",
              "IPY_MODEL_630678b899bb426e8a486cc6ac632476"
            ],
            "layout": "IPY_MODEL_4bc410a19d6842f9a5335efcd0cc98fd"
          }
        },
        "1267bb2744eb4250a73bcd30b50a3619": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3bb3cb8d5196434faf0aad12900a1d32",
            "placeholder": "​",
            "style": "IPY_MODEL_053599d1f2764dc7b55b9e3e28efccca",
            "value": " 4.98G/4.98G [00:16&lt;00:00, 385MB/s]"
          }
        },
        "13367f9b6b7140b78a0fb4baca107dd2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c469862077124062967ea461bcfdd757",
            "placeholder": "​",
            "style": "IPY_MODEL_f60356522cd940658848658f2035a603",
            "value": "model-00004-of-00004.safetensors: 100%"
          }
        },
        "17b2e683caab49e3a0e857a3a1dca647": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1b232523dcf94046880737a105192b14": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ba95a64f0c114dba83b56f5c770dfa44",
            "placeholder": "​",
            "style": "IPY_MODEL_68c6d4bc9fea457f9fec6d6187bd23d6",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "1bdce5bf4e844a40a2732513a81fe9c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1d54f848ae54441da2d8f2168f26b791": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_db29eb577f8e496fb103820e77d67862",
            "placeholder": "​",
            "style": "IPY_MODEL_fd769f30fe1e45a0bf8db6eacc16537f",
            "value": "config.json: 100%"
          }
        },
        "1fc38a1eb20d4050b2ca38dd6674fc8b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "20cfd6b6530143f6914d8ea616ae3847": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "21a20ca8f35b4bb1ad2a0391d496e09d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2767c0cf7bb4457bb7e0ff4713794de9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_46b8079935b0440e95bf32c2c45e248d",
              "IPY_MODEL_d63a8d62f4af4a3b804c903ff2508963",
              "IPY_MODEL_1267bb2744eb4250a73bcd30b50a3619"
            ],
            "layout": "IPY_MODEL_493937a09f694160bf652f01545f3bc6"
          }
        },
        "2aecb6f942664dfeb4132045186344f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2d1e706019964550b051d66c0747cf61": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2e46ea37ed454fc5b8ba061b8b9fa769": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3618c8c43a3241b8a54478b26649141a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "367fefce415c4256ac960c1326027ff8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "36e6c80b9195412fa5f94b72863fcffd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "38b9a3a2886b4085800fe0def52f2a12": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "398bae342f7845e1b54ef8d073df3d27": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3a122d57300c4d27aa36d5fbc17b2886": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3bb3cb8d5196434faf0aad12900a1d32": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "401a85ab9f814f7e90c2b48b78191e6f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "41baaecfe3354ce3a10977053278f8d3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "43dfa21cd46344d6b6145ca6eb9a4847": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "45685571010a47c9b15a4d755702ba85": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0243cd33cd8a4c2db44dd63ce9ca0311",
            "placeholder": "​",
            "style": "IPY_MODEL_bdf3e8483e63432fab7f2165981c0aa6",
            "value": " 4/4 [01:12&lt;00:00, 16.61s/it]"
          }
        },
        "46069cf2dfd14aa88e412cb93abea47d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c3059994d519412da47798d485311cd9",
              "IPY_MODEL_bffe4e632ae448868bc637354f5ad892",
              "IPY_MODEL_674d0cd8ce1f452b9f479687294f3907"
            ],
            "layout": "IPY_MODEL_dc5b8fdd54d645d2a671b3852db1c850"
          }
        },
        "46b8079935b0440e95bf32c2c45e248d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_db41c962b1db46e3a429f386dbd069fa",
            "placeholder": "​",
            "style": "IPY_MODEL_6ed8dd6e7f9f434b887c8cfd06f0cc37",
            "value": "model-00002-of-00004.safetensors: 100%"
          }
        },
        "47ea1dd9660e4eacb96486b5d8408fcd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "493937a09f694160bf652f01545f3bc6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4b2a8378c1c54be8b7db3e485efd29e0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4bc410a19d6842f9a5335efcd0cc98fd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4cf8150d909d4d38acccfbdf077bb2fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "520f078b1b26416da3ec43460b3b6691": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "539fdd36b0b841969ffedb97e6de14b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "53ae8459e79d4d72b8f348c3af532c75": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fb93a5b6e0f74956b6be2f9c5d93442c",
            "placeholder": "​",
            "style": "IPY_MODEL_1bdce5bf4e844a40a2732513a81fe9c1",
            "value": "tokenizer.model: 100%"
          }
        },
        "54840eaf055b4fad8dca39269f2e7d02": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_092910902bbd4477881d8fe30a70ecf0",
            "placeholder": "​",
            "style": "IPY_MODEL_2aecb6f942664dfeb4132045186344f0",
            "value": " 17.5M/17.5M [00:00&lt;00:00, 240MB/s]"
          }
        },
        "5f47840d39bc43569f5022a8d67f83df": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "605ecedab51d4cc3ab6c1617d13bcda5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "60c33f2376f64a44992bd92accadd892": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_21a20ca8f35b4bb1ad2a0391d496e09d",
            "max": 4,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f487f3ed5b21448483f5fab52db690dd",
            "value": 4
          }
        },
        "630678b899bb426e8a486cc6ac632476": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5f47840d39bc43569f5022a8d67f83df",
            "placeholder": "​",
            "style": "IPY_MODEL_78cdeeebec9c4726a8c35686d50bcf51",
            "value": " 694/694 [00:00&lt;00:00, 59.0kB/s]"
          }
        },
        "658e7c4eb5a64458a5a296134044b9d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "674d0cd8ce1f452b9f479687294f3907": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_17b2e683caab49e3a0e857a3a1dca647",
            "placeholder": "​",
            "style": "IPY_MODEL_a672510a012348bdb0ed44c0efe5806f",
            "value": " 4.98G/4.98G [00:22&lt;00:00, 251MB/s]"
          }
        },
        "68c6d4bc9fea457f9fec6d6187bd23d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6a70571d34894bcd9ff04718a68686cb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6b402269d8b94c0a9e6e026e10a356e4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f01951b9dd244dd9a0e206326c468299",
              "IPY_MODEL_c5dcddd2f1744c01945e85ea1f57835f",
              "IPY_MODEL_af924af108f74451a06a48306c5f7d95"
            ],
            "layout": "IPY_MODEL_b499e28fb1f14874a62cee3857fe5499"
          }
        },
        "6c35afdbe75e49b0913e0d782ebe302b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6d3d433dcbee4bed814b36fadbe5ddc2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6ed8dd6e7f9f434b887c8cfd06f0cc37": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "741d8ee671a4489d82ebec6ad928767b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9ea4d8fd79f14a529533aa7463575ebc",
            "placeholder": "​",
            "style": "IPY_MODEL_ad135cc1c8f04a3ab52389c0dbed9ea9",
            "value": " 4/4 [00:05&lt;00:00,  1.26s/it]"
          }
        },
        "75ee325e981549bbaf5e3efe3d589ab8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_41baaecfe3354ce3a10977053278f8d3",
            "max": 4241003,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d22ac654d63a4ba892d9a4e952ccfaf1",
            "value": 4241003
          }
        },
        "78cdeeebec9c4726a8c35686d50bcf51": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7b6c8c6c85ea477abb3cfc03b14f0194": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9b94eea8aa774023ab0218c9edae35fe",
            "placeholder": "​",
            "style": "IPY_MODEL_658e7c4eb5a64458a5a296134044b9d2",
            "value": "model.safetensors.index.json: 100%"
          }
        },
        "800599524ce64b4eb9fb70e63d552661": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aeb2b8d3a1fa4548a61db757ba3f2e8c",
            "max": 694,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_539fdd36b0b841969ffedb97e6de14b5",
            "value": 694
          }
        },
        "82bfbe7d7bb5488eb4379f07bdb3cf9d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "831616cc0c7b4a1f890c73cb6075e37a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "83ffee7a05fc4132b53aa6a3b6af8d02": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8618a7a4e176474d802b24d021d669a5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "891e330b593049e9be5effc6a562263d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8b1846fc7c0742aebea7500c94444115": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9131b59d8992452b8e555bd13b109e89": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9470716b1d984e6a8a403dd443249060": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "995f609c294842b8a2383b8d1313c4ed": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9988a9e6272a48ff92fccbe8f63ece81": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c37bfdcaadb5403cb44f8b94c87da256",
            "placeholder": "​",
            "style": "IPY_MODEL_0a28527b763d43248954c32d90ab84a6",
            "value": " 34.2k/34.2k [00:00&lt;00:00, 2.81MB/s]"
          }
        },
        "9b94eea8aa774023ab0218c9edae35fe": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9ea4d8fd79f14a529533aa7463575ebc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a2735d52b7a94c4489b21aaf9d5ee6e8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a333eddf1ecf438a89550ea6bc4b9a9d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e26ba603400d4ec2b1ee9b5de153ea45",
            "max": 137,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_401a85ab9f814f7e90c2b48b78191e6f",
            "value": 137
          }
        },
        "a5aca8b9fc764a1e85610d760b11b451": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a672510a012348bdb0ed44c0efe5806f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a6a2cc4568f94404a05dd99b0a4379a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_831616cc0c7b4a1f890c73cb6075e37a",
            "placeholder": "​",
            "style": "IPY_MODEL_c13b4febf7bb4dc69bd2fa91a9560d31",
            "value": "Loading checkpoint shards: 100%"
          }
        },
        "a91b5a54c95c44d3bc4eebdbb7dbc9c3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6c35afdbe75e49b0913e0d782ebe302b",
            "placeholder": "​",
            "style": "IPY_MODEL_9131b59d8992452b8e555bd13b109e89",
            "value": " 636/636 [00:00&lt;00:00, 54.9kB/s]"
          }
        },
        "aabad6dc5f3b4638bacd292a1149047c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad135cc1c8f04a3ab52389c0dbed9ea9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "aeb2b8d3a1fa4548a61db757ba3f2e8c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "af924af108f74451a06a48306c5f7d95": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d98be4f3768c4bf89ccba65e6436c08c",
            "placeholder": "​",
            "style": "IPY_MODEL_6d3d433dcbee4bed814b36fadbe5ddc2",
            "value": " 5.00G/5.00G [00:22&lt;00:00, 257MB/s]"
          }
        },
        "b06b3426f8d146748883d218c16004f7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b098e6c5330044bdb6a0b9bb38390997": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_520f078b1b26416da3ec43460b3b6691",
            "max": 17518497,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_de53da01b3184348a4686e709f283bf4",
            "value": 17518497
          }
        },
        "b3e8ff96593246829d1f00d170b28718": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4b2a8378c1c54be8b7db3e485efd29e0",
            "placeholder": "​",
            "style": "IPY_MODEL_3618c8c43a3241b8a54478b26649141a",
            "value": " 2.11G/2.11G [00:09&lt;00:00, 77.3MB/s]"
          }
        },
        "b499e28fb1f14874a62cee3857fe5499": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b69c8c1744a945338cbf47b02fb990ca": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d870495ce8194784b4472d38d7e27b3d",
            "placeholder": "​",
            "style": "IPY_MODEL_f5367a7daf1b4491b3569ec140fcac84",
            "value": " 137/137 [00:00&lt;00:00, 10.9kB/s]"
          }
        },
        "b755654e0ec7474b89af6850de90acf3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b7abe071c90443c0a57432171e159e8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d0ff1d176aea48d5a229a6e1ab3028d2",
              "IPY_MODEL_e81041a4c266424496f33000be5ed775",
              "IPY_MODEL_9988a9e6272a48ff92fccbe8f63ece81"
            ],
            "layout": "IPY_MODEL_605ecedab51d4cc3ab6c1617d13bcda5"
          }
        },
        "b875ef06b75e41ce8a59d53fc62634e1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6a70571d34894bcd9ff04718a68686cb",
            "placeholder": "​",
            "style": "IPY_MODEL_36e6c80b9195412fa5f94b72863fcffd",
            "value": "Downloading shards: 100%"
          }
        },
        "ba95a64f0c114dba83b56f5c770dfa44": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bb328ff690e34d2186a39aa793c29723": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bdf3e8483e63432fab7f2165981c0aa6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bf378e2b7468423999535bbc70ff9c6a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bffe4e632ae448868bc637354f5ad892": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0c8c70734dbc4abaaab07c522ef0eb7a",
            "max": 4982953200,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a5aca8b9fc764a1e85610d760b11b451",
            "value": 4982953200
          }
        },
        "c13b4febf7bb4dc69bd2fa91a9560d31": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c3059994d519412da47798d485311cd9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b06b3426f8d146748883d218c16004f7",
            "placeholder": "​",
            "style": "IPY_MODEL_3a122d57300c4d27aa36d5fbc17b2886",
            "value": "model-00003-of-00004.safetensors: 100%"
          }
        },
        "c37bfdcaadb5403cb44f8b94c87da256": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c469862077124062967ea461bcfdd757": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c4ba886b0fa1406382e07097c3aae7e2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c5dcddd2f1744c01945e85ea1f57835f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8618a7a4e176474d802b24d021d669a5",
            "max": 4995496656,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_891e330b593049e9be5effc6a562263d",
            "value": 4995496656
          }
        },
        "c5f7435be86d4e72846600298d9c9e86": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a6a2cc4568f94404a05dd99b0a4379a0",
              "IPY_MODEL_053946b3326a4c4facee90985508ec6b",
              "IPY_MODEL_741d8ee671a4489d82ebec6ad928767b"
            ],
            "layout": "IPY_MODEL_43dfa21cd46344d6b6145ca6eb9a4847"
          }
        },
        "c72503ef69ea4857a59946051e214ced": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7b6c8c6c85ea477abb3cfc03b14f0194",
              "IPY_MODEL_d399b44c59e44143b3bcd84cb7104fdc",
              "IPY_MODEL_0e4ce29ae35e41cfa75265cd3a0402a3"
            ],
            "layout": "IPY_MODEL_c4ba886b0fa1406382e07097c3aae7e2"
          }
        },
        "c9addaf0418e47868702d200857e6ec2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cebaf637eb4e4d4fafad2a041b6933e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d0ff1d176aea48d5a229a6e1ab3028d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8b1846fc7c0742aebea7500c94444115",
            "placeholder": "​",
            "style": "IPY_MODEL_ede9f9b11dec49b59e3627d9068b7380",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "d22ac654d63a4ba892d9a4e952ccfaf1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d266470c38264712be92bf7a9d023f5a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d2dfefaf9fe54e909871995b782e7da9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d399b44c59e44143b3bcd84cb7104fdc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e8f4eeef3c494c36adaf2814c3f5fcc6",
            "max": 20920,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d2dfefaf9fe54e909871995b782e7da9",
            "value": 20920
          }
        },
        "d4eea5cc9a6e4746a86469bf6569765d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d63a8d62f4af4a3b804c903ff2508963": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e685530588994345bc56b4280725aa55",
            "max": 4982953168,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_04d54078ff024ac68f9c1b0da86cfb33",
            "value": 4982953168
          }
        },
        "d870495ce8194784b4472d38d7e27b3d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d98be4f3768c4bf89ccba65e6436c08c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "db29eb577f8e496fb103820e77d67862": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "db41c962b1db46e3a429f386dbd069fa": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dc5b8fdd54d645d2a671b3852db1c850": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dcde971a54144fcb8d1f6c00431f1c62": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_fbdf2fdd0f5b4dd1bb78d115a4992991",
              "IPY_MODEL_b098e6c5330044bdb6a0b9bb38390997",
              "IPY_MODEL_54840eaf055b4fad8dca39269f2e7d02"
            ],
            "layout": "IPY_MODEL_38b9a3a2886b4085800fe0def52f2a12"
          }
        },
        "de53da01b3184348a4686e709f283bf4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e26ba603400d4ec2b1ee9b5de153ea45": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e44fcf3348ac4d08bff9a88ae3dd8e82": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e46c28ce4af94322ab6ceb867b1923a8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_398bae342f7845e1b54ef8d073df3d27",
            "max": 636,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2d1e706019964550b051d66c0747cf61",
            "value": 636
          }
        },
        "e685530588994345bc56b4280725aa55": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e81041a4c266424496f33000be5ed775": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_82bfbe7d7bb5488eb4379f07bdb3cf9d",
            "max": 34173,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_cebaf637eb4e4d4fafad2a041b6933e5",
            "value": 34173
          }
        },
        "e8f4eeef3c494c36adaf2814c3f5fcc6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eb225ecadd174092b872ff1ac5ccc33d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_13367f9b6b7140b78a0fb4baca107dd2",
              "IPY_MODEL_058854285af14cf999666324bf29ff8d",
              "IPY_MODEL_b3e8ff96593246829d1f00d170b28718"
            ],
            "layout": "IPY_MODEL_20cfd6b6530143f6914d8ea616ae3847"
          }
        },
        "eb9a88cea4a8402288ed14d003cecda0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ed5e32084b0c40f7a8f723e5d2d18a65": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_08dee47a9a794977bc8828f18237d38e",
              "IPY_MODEL_a333eddf1ecf438a89550ea6bc4b9a9d",
              "IPY_MODEL_b69c8c1744a945338cbf47b02fb990ca"
            ],
            "layout": "IPY_MODEL_bb328ff690e34d2186a39aa793c29723"
          }
        },
        "ede9f9b11dec49b59e3627d9068b7380": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f01951b9dd244dd9a0e206326c468299": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_47ea1dd9660e4eacb96486b5d8408fcd",
            "placeholder": "​",
            "style": "IPY_MODEL_9470716b1d984e6a8a403dd443249060",
            "value": "model-00001-of-00004.safetensors: 100%"
          }
        },
        "f487f3ed5b21448483f5fab52db690dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f5367a7daf1b4491b3569ec140fcac84": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f60356522cd940658848658f2035a603": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f61947902bb5407a8428848f8141d134": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1fc38a1eb20d4050b2ca38dd6674fc8b",
            "placeholder": "​",
            "style": "IPY_MODEL_b755654e0ec7474b89af6850de90acf3",
            "value": " 4.24M/4.24M [00:00&lt;00:00, 115MB/s]"
          }
        },
        "f651b08d00aa4c39a8ac79283bddbda5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_53ae8459e79d4d72b8f348c3af532c75",
              "IPY_MODEL_75ee325e981549bbaf5e3efe3d589ab8",
              "IPY_MODEL_f61947902bb5407a8428848f8141d134"
            ],
            "layout": "IPY_MODEL_c9addaf0418e47868702d200857e6ec2"
          }
        },
        "fb93a5b6e0f74956b6be2f9c5d93442c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fbdf2fdd0f5b4dd1bb78d115a4992991": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eb9a88cea4a8402288ed14d003cecda0",
            "placeholder": "​",
            "style": "IPY_MODEL_2e46ea37ed454fc5b8ba061b8b9fa769",
            "value": "tokenizer.json: 100%"
          }
        },
        "fd769f30fe1e45a0bf8db6eacc16537f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}